EoD SQL 2.0 – Let the Bug Hunting begin!

Now that I’ve finally got EoD SQL 2.0-alpha out the door, it’s time to start bug hunting.

Part of the upcoming 2.0 version is a lot of new unit-tests. In the past EoD SQL was not test-covered nearly enough, and so several bugs slipped under the carpet (in particular in the 1.1 release).

Already 1 new bug has been found and fixed, and 2 older bugs (introduced in 1.1) have also been fixed. In my use of the 2.0 code-base (during development of other projects), I’ve found the 2.0 code to be an amazing performance improvement over the 1.0 code-base.

For those interested, the 2.0 code is so much faster because of two structural changes:

• Less thread contention
  • Much of the synchronization locking has been removed
  • Some of the code now requires that you rely on Java’s implicit locks to ensure thread-safety
    • For example, you should set-up your own TypeMappers in a static initializer or another thread-safe structure (ie: ServletContextListener).
• Much less branching through if and switch statements
  • There are far more classes in the code-base, and as such the API has grown in size
  • Many of the new classes use inheritance to avoid conditional branches

An example of the removal of conditional branches is the binding of a column value to a field or method. In the 1.0/1.1 code-base it was done like this:

for(int i = 1; i <= meta.getColumnCount(); i++) {
    Member member = columnMap.get(new ColumnName(meta.getColumnName(i)));

    if(member != null) {
        if(member instanceof Field) {
            Field f = (Field)member;
            TypeMapper mapper = types.get(f.getType());

            if(mapper != null) {
                Object value = mapper.get(results, i);

                if(results.wasNull()) {
                    value = null;
                }

                f.set(target, value);
            } else {
                throw new SQLException("No TypeMapper for: " + f.getType().getName());
            }
        } else if(member instanceof Method) {
            Method m = (Method)member;
            Class javaType = m.getParameterTypes()[0];
            TypeMapper mapper = types.get(javaType);

            if(mapper != null) {
                Object value = mapper.get(results, i);

                if(results.wasNull()) {
                    value = null;
                }

                m.invoke(target, value);
            } else {
                throw new SQLException("No TypeMapper for: " + javaType.getName());
            }
        }
    }
}

In the new 2.0 code-base, the code looks like this:

for(int i = 0; i < columns.length; i++) {
    Object value = mappers[i].get(row, i + 1);

    if(row.wasNull()) {
        value = null;
    }

    columns[i].set(into, value);
}

Yup! That is the same piece of code! In the 2.0 version, we look at the ResultSet the first time around and construct all of the binding information into 2 arrays, one of TypeMappers and the other of a new MutableColumn class which replaces my use of the java.lang.reflect.Member class.

This means that the DataObjectBinding’s cannot be reused as much as before (one per method, rather than one per DataObject class), but they are much much faster since all the binding work is figured out up front.

I’ll probably be posting more on this pattern in the future, since it has some massive impact on how your code works (and how HotSpot can optimize it).

EoD SQL 2.0-alpha Released

EoD SQL now has the first release of the 2.0 structure in the wild. You can download it from the EoD SQL page at dev.java.net.

EoD SQL 2.0 marks a totally new way of running the API. Almost all of the internal functionality has been exposed through a new series of “spi” and “impl” packages. However, the way you interact with the API remains unchanged!

Some of the new features of the API:

• Much faster
  • Huge threading improvements
  • More upfront loading to reduce execution time
  • Much less locking
• @Call annotation for those who work with stored procs
• @SelectPrimitive falls away
  • Thanks to the new DataBinding layers, it’s no longer needed
  • It’s replaced by a good old @Select
• The ability to add your own code without climbing into the EoD SQL code
  • Hand written DataObjectBinding’s
    • Turn a row into a data object
  • Annotation implementations
    • Produce the implementations of methods annotated by @Select, @Update, @Call, @YourAnnotationHere
  • ResultSetWrappers
    • Wrap a ResultSet in a DataSet, DataIterator, or your choice of object

There are almost certainly bugs lying in wait on such a massive set of changes, which is why the API is in a beta right now. Go download the API and give it a try.

Fixing Compilation in GWT4NB

By default GWT4NB recompiles all of your client code with the GWT compiler whether or not you’ve changed it. This is a time consuming (and sometimes irritating factor). After a recent discussion on the GWT dev mailing list (which I was away for most of), I decided to post the solution I use here and on the mailing list.

In your projects build.xml, just copy-and-paste the following:

<target name="-post-compile">
<property name="output.js" location="${build.web.dir}/${gwt.module}/${gwt.module}.nocache.js" />
</target>

<target name="debug" description="Debug project in IDE." depends="init,compile,compile-jsps,-do-compile-single-jsp" if="netbeans.home">
<property name="gwt.compile.unneeded" value="true" />
    <antcall target="dist"/>

    <nbdeploy debugmode="true" clientUrlPart="${client.urlPart}"/>
    <antcall target="connect-debugger"/>
    <antcall target="debug-connect-gwt-shell"/>
</target>

<target name="-pre-dist">
    <condition property="gwt.compile.unneeded">
        <and>
            <available file="${output.js}" />
            <uptodate>
                <srcfiles dir="${src.dir}" includes="**/client/**/*.java" />
                <mergemapper to="${output.js}" />
            </uptodate>
        </and>
    </condition>
    <antcall target="do-gwt-compile" />
</target>

<target name="do-gwt-compile" unless="gwt.compile.unneeded">
    <!-- You can override this property in the 'gwt.properties' file -->
<property name="gwt.compiler.output.style" value="OBFUSCATED"/>
<property name="gwt.compiler.logLevel" value="WARN"/>

    <java classpath="${javac.classpath}:${src.dir}" failonerror="true"
      classname="com.google.gwt.dev.GWTCompiler" fork="true" maxmemory="512m">
        <arg value="-out"/>
        <arg path="${build.web.dir}/"/>
        <arg value="-style"/>
        <arg value="${gwt.compiler.output.style}"/>
        <arg value="-logLevel"/>
        <arg value="${gwt.compiler.logLevel}"/>
        <arg value="${gwt.module}"/>
    </java>
<property name="gwt.output.dir" value="${gwt.module}"/>
    <move todir="${build.web.dir}/${gwt.output.dir}">
        <fileset dir="${build.web.dir}/${gwt.module}"/>
    </move>
</target>

What does it actually do? It checks to see if the client side code is newer than the module.nocache.js file in the web output. Since GWT overwrites this file every-time it compiles (and in GWT 1.5 actually deletes it first), it will only be older if one of your source files has changed.

Hope this helps you out, happy coding!

Edit 11/14/08: The script is now fixed to only build using normal javac when run in debugging mode.

Easy Property Binding and Aync Callbacks in GWT

This is a little technique I came up with a few days ago that makes Async Callbacks a lot easier. It also adds something a lot like Property Bindings to GWT with very little work.

public interface PropertyChangeListener {
    void propertyChanged(SourcesPropertyChangeEvents source, String propertyName, Object oldValue, Object newValue);
}

public interface SourcesPropertyChangeEvents {
    void addPropertyChangeListener(PropertyChangeListener listener);

    void addPropertyChangeListener(String propertyName, PropertyChangeListener listener);

    void removePropertyChangeListener(PropertyChangeListener listener);

    void removePropertyChangeListener(String propertyName, PropertyChangeListener listener);

}

public class PropertyChangeListenerCollection extends AbstractCollection<PropertyChangeListener> {
    private final Map<String, Set<PropertyChangeListener>> listeners = new HashMap<String, Set<PropertyChangeListener>>();

    private Set<PropertyChangeListener> getListenersForProperty(final String name) {
        Set<PropertyChangeListener> set = listeners.get(name);

        if(set == null) {
                set = new HashSet<PropertyChangeListener>();
                listeners.put(name, set);
        }

        return set;
    }

    // this is a simple utility method that avoids duplicate copies of the same
    // PropertyChangeListener
    private Set<PropertyChangeListener> getAllListeners() {
        final Set<PropertyChangeListener> all = new HashSet<PropertyChangeListener>();

        for(final Set<PropertyChangeListener> set : listeners.values()) {
            all.addAll(set);
        }

        return all;
    }

    public void add(final PropertyChangeListener listener) {
        add(null, listener);
    }

    public void add(final String property, final PropertyChangeListener listener) {
        if(listener != null) {
            getListenersForProperty(property).add(listener);
        }
    }

    public void remove(final PropertyChangeListener listener) {
        if(listener != null) {
            for(final Set<PropertyChangeListener> set : listeners.values()) {
                set.remove(listener);
            }
        }
    }

    public void remove(final String property, final PropertyChangeListener listener) {
        if(listener != null) {
            getListenersForProperty(property).remove(listener);
        }
    }

    // although unused I've provided a simple implementation of the size method
    public int size() {
        return getAllListeners().size();
    }

    public Iterator<PropertyChangeListener> iterator() {
        return getAllListeners().iterator();
    }

    public void firePropertyChangeEvent(final SourcesPropertyChangeEvents source, final String name,
            final Object oldValue, final Object newValue) {

        final Set<PropertyChangeListener> propertyListeners = new HashSet<PropertyChangeListener>();
        propertyListeners.addAll(getListenersForProperty(null));
        propertyListeners.addAll(getListenersForProperty(name));

        for(final PropertyChangeListener l : propertyListeners) {
            l.propertyChanged(source, name, oldValue, newValue);
        }
    }
}

public class Property<T> implements SourcesPropertyChangeEvents, PropertyChangeListener, AsyncCallback<T> {
    private final String name;
    private T value;
    private PropertyChangeListenerCollection listeners;

    public Property(final String name) {
        this(name, null);
    }

    public Property(final String name, final T initialValue) {
        this.name = name;
        this.value = initialValue;
    }

    public void set(final T newValue) {
        final T oldValue = value;
        value = newValue;

        if(listeners != null) {
            listeners.firePropertyChangeEvent(this, name, oldValue, newValue);
        }
    }

    public T get() {
        return value;
    }

    public void onSuccess(final T newValue) {
        set(newValue);
    }

    public void onFailure(final Throwable error) {
        if(GWT.getUncaughtExceptionHandler() != null) {
            GWT.getUncaughtExceptionHandler().onUncaughtException(error);
        }
    }

    public void propertyChanged(final SourcesPropertyChangeEvents source, final String propertyName,
        final Object oldValue, final Object newValue) {

        set(newValue);
    }

    public void addPropertyChangeListener(final PropertyChangeListener listener) {
        if(listeners != null) {
            listeners = new PropertyChangeListenerCollection();
        }

        listeners.add(listener);
    }

    public void addPropertyChangeListener(final String propertyName, final PropertyChangeListener listener) {
        if(listeners != null) {
            listeners = new PropertyChangeListenerCollection();
        }

        listeners.add(propertyName, listener);
    }

    public void removePropertyChangeListener(final PropertyChangeListener listener) {
        if(listeners != null) {
            listeners.remove(listener);
        }
    }

    public void removePropertyChangeListener(final String propertyName, final PropertyChangeListener listener) {
        if(listeners != null) {
            listeners.remove(propertyName, listener);
        }
    }

}

Instead of storing your bindable properties as normal fields: you simply wrap them in Property objects. You can then use the fact that Property objects both produce and consume propertyChangeEvents to bind them together, and even pass them into RPC methods to be set when the server hands the data back to you.

They rely on a PropertyChangeListenerCollection class that I haven’t given here, but it’s a simple enough class to write.

11-November-2008: I added in an implementation of the PropertyChangeListenerCollection class.

Give it a try, it makes life a surprising amount easier considering it’s size.

A useful GWT RPC pattern I’ve been using

GWT RPC calls can wind up all over your code, doing all sorts of weird things. If you’re not careful about how you code them you can end up with bugs and other problems cropping up. The other part of RPC calls is that some of them are actually recoverable, and may even be retried a few times before giving up.

I developed the following little bit of code recently that may be useful to anyone doing RPC in GWT:

public abstract class RetryAction<T> implements AsyncCallback<T> {
    public abstract void attempt();
    public abstract void oncapture(T value);

    public void onFailure(Throwable error) {
        try {
            throw error;
        } catch(InvocationException invocationException) {
            Window.alert("A fatal error occurred, you should login " +
                "again or contact Technical Support");
        } catch(IncompatibleRemoteServiceException remoteServiceException) {
            Window.alert("A fatal error occurred, you should login " +
                "again or contact Technical Support");
        } catch(SerializationException serializationException) {
            Window.alert("A fatal error occurred, you should login " +
                "again or contact Technical Support");
        } catch(Throwable throwable) {
            String message = throwable.getLocalizedMessage();

            if(message == null) {
                message = throwable.getMessage();
            }

            if(message == null) {
                message = throwable.getClass().getName();
            }

            if(Window.confirm("An error occured:\n" + message + "\n" +
                "You may retry this operation by clicking 'OK'.\n" +
                "However if the error persists, contact Technical Support.")) {

                attempt();
            }
        }
    }

    public void onSuccess(T value) {
        try {
            oncapture(value);
        } catch(RuntimeException error) {
            onFailure(error);
        }
    }
}

The way this structure works is that you implement your RPC call in the “attempt” method, and the callback in the “oncapture” method. If a “non-fatal” error occurred, the user is given the opportunity to “Retry” the operation (resulting in another call to “attempt”). The pattern groups the RPC invocation nicely with it’s response logic. Here’s a little example:

public class GetContactListAction extends RetryAction<Contacts[]> {
    private ContactList contacts;

    public GetContactListAction(ContactList display) {
        contacts = display;
    }

    public void attempt() {
        CONTACT_LIST_SERVICE_ASYNC.getContactList(this);
    }

    public void oncapture(Contact[] data) {
        contacts.populate(data);
    }
}

And then to use this action:

GetContactListAction action = new GetContactListAction(contactList);
action.attempt();

GWT RPC is (?:called) Aynchronous for a reason

GWT RPC is totally asynchronous. You have no option to implement a synchronous call to the server. Many people find passing a callback to every remote method in order to receive the return value frustrating, and the fact that it returns immediately decidedly strange.

For those new to GWT, here’s a description of GWT RPC in pure Java terms. Read the code carefully and things will make a lot more sense.

First the definition of the service. Think of this like an RMI service interface.

public interface MyService extends RemoteService {
 public String getText();
}

This is the interface the GWT client (Javascript) side of things will be using. The reason GWT makes you use this interface in because on the client side we need to call the method, and then receive the response some-time in the future. You can think of an AsyncCallback as an EventListener, when the server sends the response back, you get an event containing the success or failure data.

public interface MyServiceAsync {
 public void getText(AsyncCallback<String> callback);
}

Now our Servlet is the implementation of the MyServer interface. You can think of this like the implementation of an RMI service or an EJB. The reason you extend RemoveServiceServlet is two-fold: (1) You need an HTTP path that the client can send data to. (2) Rather than forcing you to decode GWT’s flavour of Serialization and invoke the methods by hand, RemoteServiceServlet does it all for you (so all you do is implement the actual methods).

An important note here. This code runs on the server, under a real Java VM. It’s not compiled by GWT, it’s not even looked at in fact. You can use any classes here (surprisingly, this is something that catches a lot of people out).

public class MyServiceImpl extends RemoveServiceServlet implements MyService {
 public String getText() {
  return "Hello World";
 }
}

Now for our implementation on the client side. This is not how you would code this method call in GWT, this is a normal Java representation of what happens.

public void onModuleLoad() {
 // This is a purely local representation of what
 // GWT.create(MyService.class) would do for you
 MyServiceAsync async = new MyServiceAsync() {
  // Our pretend implementation. In real GWT,
  // this object would be on the other side of the network
  MyServiceImpl impl = new MyServiceImpl();

  public void getText(final AsyncCallback<String> callback) {
   // When this method gets called, we spawn a
   // Thread to make the call to the server.
   // In JavaScript the call is often put in a queue,
   // by the browser and executed in a pool.
   // However, whichever way things happen the
   // method call returns immediately and does
   // not wait for the server to respond.

   Thread runner = new Thread() {
    public void run() {
     try {
      // Once we have the content, pass it
      // to the AsynCallback we were given.

      callback.onSuccess(impl.getText());
     } catch(Exception error) {
      // If an Exception occurs (unlikely in our
      // little example here), we pass it to the
      // AsyncCallback to deal with.

      callback.onFailure(error);
     }
    }
   };

   // Start our Thread and return.
   runner.start();
  }
 };

 final Label label = new Label("Foo");
 asyn.getText(new AsyncCallback<String>() {
  public void onSuccess(String message) {
   label.setText(message);
  }

  public void onFailure(Throwable error) {
   Window.alert(error.getMessage());
  }
 });

 label.setText("Bar");
}

So you can see from the example above that “Bar” may appear on the label, but it’s not likely. Far more likely is “Hello World” coming from our “server”.

There are a good reasons why GWT only allows for this sort of call.

1. JavaScript has no threading model in place. It’s impossible to Object.wait() for something to Object.notify() you, which would be exactly how you would implement this sort of invocation in normal Java (if only under the hood)
2. Anyone who has used Swing extensively will know that doing lots of work in the event-dispatch-thread is a disaster. It stops repaints from happening, the application is basically unusable until you’re “event” in complete.In order to get around race-conditions and such multi-threading problems, JavaScript is all executed from within the browser event-queue. So if we sent a request to the server synchronously, the user wouldn’t even be able to open the file menu until the server gave us a response. “But I’m in a LAN” I’ve heard some say. GWT’s RPC mechanism is built for general consumption. Lazy Developers + Synchronous Calls + Open Internet is a recipe for disaster (and a lot of complaints on the mailing-lists), and the GWT devs know it.

Asynchronous RPC with callbacks can be considered a small price to pay for an amazing amount of power. Personally I see it as even more power, as is breaks your code into smaller modules. I often have a single AsyncCallback class handling many different invocations from the server. Using this technique helps make your code smaller to deploy, and easier to maintain.

I’m looking for my gloves

I want to go shoot some photo’s, but it’s cold.

If anyone sees my gloves, please let me know where they are.

Thanks.