Where C++ Templates Fall Down

There are many articals out there detailing comparisons between C++ Templates and Java Generics (Google around and you’ll quickly find loads of them). Almost universally they view the erasure of Java generics as a fail-point. Even the ones written by “Java Gurus” seem to favour C++ templates as being more flexable. I recently stumbled upon a simple case that I’ve been doing in Java for ages, and while trying to implement the same concept in C++ discovered that it’s impossible.

The case is a generic Visitor model:

public interface Visitor<R, P> {
    R visitFirstNodeType(FirstNodeType node, P parameter);
    R visitSecondNodeType(SecondNodeType node, P parameter);
}

public abstract class AbstractNodeType {
    public abstract <R, P> R accept(Visitor<R, P> visitor, P parameter);
}

Looks simple enough, but try and implement it in C++ and you discover that you cannot use templates on a virtual method. At first this doesn’t make sense (even to a friend of mine who lives in C++). The simple reason is the way templates are handled: they are code generation shortcuts. Therefore the compiler needs to know the exact method to run, but by declaring the method virtual you are telling the compiler to deferre that descision until runtime. In Java the generics are “erased” (although they are kept in the signature), resolving to a single method implementation. This means that there is no problem trying to decide which method to run.

C++ can create the same structure without templates, but then you loose the type-safety contract that you defined in Java. It’s not really a fail-point on C++ (rather just a product of the implementation), but it is something that seems to be overlooked on the Java side of the fence. Generic erasure buys you fairly cool compile-time contracts.

EoD SQL Version 2.1-Alpha

EoD SQL doesn’t see very much development these days (although it sees plenty of downloads and usage). Mainly it’s very stable and does what it’s supposed to, so theres very little need to change things.

That said, there are a few issues that needed some attention, so theres the start of a new release (2.1). The new features and changes are as follows:

• The engine now caches some of the more expensive construction data with SoftReferences, dramatically reducing the cost of creating a Query implementation
• @Call methods may now return updatable DataSet objects by using the new “readOnly” annotation attribute
• @Select methods now have a useful “fetchSize” hint that will be passed down to the JDBC driver
• Much better error messages for invalid / unknown types and query parse errors
• Improvements and updates to all of the JavaDocs
• Some bug fixes and corrections

Like I said: theres not that much there, but the changes that do exist make life that little bit easier. If you have any ideas of features you’d like to see in this upcomming release, leave a comment or a feature request and we’ll have a chat about it (always on the lookout for good ideas)!

Why TraceMonkey is maybe the wrong way

With Firefox 3.5 finally out the door and in use I thought it was time to write something about it that’s been bugging me for a while. TraceMonkey (the new FF JavaScript engine) employs a semi-Hotspot approach to JavaScript compilation. Although the approach is quite different under-the-hood, the basic principal is the same: interpret the code until you know what to optimize. This makes abundant sense to me: why go through the expense of native compiling a method that only ever gets run once? However: there is a hidden expense to doing things this way: your compilation unit is often much smaller. Smaller compilation-units means more compiled code that needs to go back and forth to interpreted code. It also means you can’t optimize as heavily (although that optimization has it’s own massive expense).

Where am I heading with this? I think the TraceMonkey approach adds huge value to a language like Java where an application consists of hundreds and / or thousands of classes (both those in the application and all of it’s dependent APIs), in this case: you’re loading time would be abysmal if you pre-loaded all the possibly used code and native compiled it on start-up. However in the JavaScript world we are limited by the connection, and JavaScript applications (even really huge ones like those written with GWT) are relatively small. In this case it makes more sense to me to either compile everything up front; or run everything interpreted and run a massive compilation in the background (hot-swapping the implementations when you’re finished the compile).

It’s just a thought. Maybe FF should adopt v8 and see what happens?

Simple Credit Card Fraud Checks

Credit Card Fraud is one of the hardest problems to deal with as an online business, in many cases it comes down to a human check to ensure that the credit-card in valid before accepting it. Using automatic scoring services certainly helps, but doesn’t by any means eliminate the problem (since they can sometimes be horribly inaccurate).

Here are a few additional checks you can do from inside your system:

• Blacklist card-numbers
  Some people have real plastic fake card and will re-use the number again and again. Blacklisting the number eliminates the need for a human check later down the line.
• Blacklist email addresses
  If you have an email address verification (you may require a valid email address to distribute your product) you can easily blacklist an email address. It’s not 100% effective because people can get free email addresses, but most of these sorts of fraud transactions are done by hand, and it often takes some time for the person to figure out whats gone wrong.
• Score based on previous transactions
  If you can identify the user (by name, email, or some other means) you can see how reliable they have been in the past.
• Address Verification
  There are several services out there that will provide you with the billing address registered with the bank for a card-number. This will also determine whether such a card even exists.
• Human Verification
  At the end of the day, there is no substitute for calling the person up and talking to them (and maybe getting a photo of the card faxed to you). However, this is costly and should only be done with the most suspect cards.

Using Event Handlers in GWT-1.6

I’ve already blogged about Event Handlers in GWT-1.6. Everyone coding Java knows how an event Listener works. How to attach one to an event source, how to handle the events, and how to create one yourself. Handlers are a bit different though, as an example I thought I’d show you how to write a very simple (un-styled) VerticalTabBarclass using GWT-1.6. The VerticalTabBar class has the following design points:

• It’s root Element is a simple <div> with a primary style of “VerticalTabBar”
• Each tab is a simple <div> element (wrapped in it’s own Widget class)
• Each tab has the CSS class-name “Tab”
• The selected tab has the CSS class-name “Tab Tab-Selected”
• The VerticalTabBar fires “ValueChangeEvents” with the index of the tab that has been selected

So here goes:

import java.util.List;
import java.util.ArrayList;

import com.google.gwt.dom.client.Document;
import com.google.gwt.dom.client.DivElement;

import com.google.gwt.event.dom.client.ClickEvent;
import com.google.gwt.event.dom.client.ClickHandler;

import com.google.gwt.event.shared.HandlerRegistration;

import com.google.gwt.event.logical.shared.ValueChangeEvent;
import com.google.gwt.event.logical.shared.ValueChangeHandler;
import com.google.gwt.event.logical.shared.HasValueChangeHandlers;

import com.google.gwt.user.client.Element;

import com.google.gwt.user.client.ui.Widget;
import com.google.gwt.user.client.ui.ComplexPanel;

// Unlike previous GWT implementation, implementing the Has***Handlers interface of the Event type
// your class dispatches is now mandatory, and you'll see why a little further down.
public class VerticalTabBar extends ComplexPanel
        implements HasValueChangeHandlers<Integer> {

    private final DivElement root;

    private final List<Tab> tabs = new ArrayList<Tab>();

    private int selected = -1;

    public VerticalTabBar() {
        root = Document.get().createDivElement();
        setElement(root);
        setStylePrimaryName("VerticalTabBar");
    }

    private void setTabSelected(final int tabIndex, final boolean selected) {
        if(tabIndex != -1) {
            tabs.get(tabIndex).setSelected(selected);
        }
    }

    private void setSelectedTabImpl(final int tabIndex) {
        setTabSelected(selected, false);
        setTabSelected(selected = tabIndex, true);
    }

    public void add(final String tabText) {
        add(tabText, false);
    }

    public void add(final String tabText, final boolean asHTML) {
        add(new Tab(tabText, asHTML), root.<Element>cast());
    }

    public int getSelectedTab() {
        return selected;
    }

    public void setSelectedTab(final int tabIndex) {
        if(tabIndex != selected) {
            if(tabIndex < 0 || tabIndex >= tabs.size()) {
                setSelectedTabImpl(-1);
            } else {
                setSelectedTabImpl(tabIndex);
            }

            // This is how you fire an Event with a Handler.
            // The signature of the "fire" method in ValueChangeEvent is:
            // public static <I> void fire(HasValueChangeHandlers<I> source, I value)
            // If we don't implement the HasValueChangeHandlers, we can't fire ValueChangeEvents
            ValueChangeEvent.fire(this, selected);
        }
    }

    public int getTabCount() {
        return tabs.size();
    }

    public HandlerRegistration addValueChangeHandler(
            final ValueChangeHandler<Integer> handler) {

        // This is really all we need to do to add a new Handler
        // addHandler is defined in Widget, and registers the Handler
        // with our HandlerManager
        return addHandler(handler, ValueChangeEvent.getType());
    }

    private class Tab extends Widget {
        private HandlerRegistration registration;

        private boolean selected = false;

        private Tab(final String tabText, final boolean asHTML) {
            final DivElement element = Document.get().createDivElement();

            if(asHTML) {
                element.setInnerHTML(tabText);
            } else {
                element.setInnerText(tabText);
            }

            setElement(element);
            setStylePrimaryName("Tab");
        }

        @Override
        protected void onLoad() {
            super.onLoad();

            // As you can see, you don't have to use onBrowserEvent anymore,
            // instead just register a Handler without exposing the Has***Handlers.
            // This technique only works for low-level browser (DOM) events.
            // Also not the fact that we invoke "addDomHandler" here, and not "addHandler"
            registration = addDomHandler(new ClickHandler() {
                public void onClick(final ClickEvent event) {
                    setSelectedTabImpl(tabs.indexOf(Tab.this));
                }
            }, ClickEvent.getType());
        }

        @Override
        protected void onUnload() {
            super.onUnload();

            registration.removeHandler();
            registration = null;
        }

        public void setSelected(final boolean selected) {
            if(this.selected = selected) {
                addStyleDependentName("Selected");
            } else {
                removeStyleDependentName("Selected");
            }
        }

    }

}

As you can see from this example, event dispatching and handling has become much easier with Handlers. You can invent your own Event types, and use them in the same way as all the other Handlers.

GWT’s new Event Model – Handlers in GWT 1.6

Theres a lot of concern and worry around the “new event model” in GWT 1.6: throwing out Listeners (a well know and used Java construct) and replacing them with Handlers (a GWT team invention). Let me put your mind at rest, Handlers will make your GWT life much easier.

So what are the differences between Listeners and Handlers?

Listeners	Handlers
Have one Listener per event source (ie: “mouse”, “keyboard”, etc)	Have one Handler per event type (ie: “mouse press”, “mouse down”, “mouse up”)
Are uncategorized, and the Widget is responsible for sinking / un-sinking the DOM events	Are categorized into 2 types: DOM and Logical, adding a DOM Handler to any Widget deferrers to Widget.addDomHandler which sinks events as required
Each Widget uses a ListenerCollection for each class of Listener added	All Handlers are stored with a HandlerManager which also manages dispatching any type of event (including new ones you create yourself)
Listeners are removed with the pattern Button .removeClickListener(listener);	Adding a Handler returns a HandlerRegistration object, removing a Handler is done by calling HandlerRegistration .removeHandler();
GWT Listeners accept a parameter for each bit of relative event information (ie: sender, mouse x, mouse y)	Handlers (more like normal Java Event Listeners) are passed an Event object with all the Event details as their only parameter

So what do Handlers actually buy you in code?

Possibly the most important thing to notice is they take up a lot less space when compiled to GWT. This in mainly due to the fact that their management is centralized in HandlerManager instead of requiring a separate ListenerCollection for each Listener type, and that an implementation only needs a single method overridden (instead of having to listen for events you’re not interested in).

Another point to take note of is that all DOM related Event objects extend from DomEvent. Why is this important? Because DomEvent includes the methods:

• preventDefault()
• stopPropagation()
• getNativeEvent()

I find the last (getNativeEvent()) to be the most useful, it works much the same as Event.getCurrentEvent() and returns a NativeEvent (the new parent class to Event). The advantage of this mechanism over Event.getCurrentEvent() is that you have all the access to the underlying event information (mouse button, modifiers, etc) but attached directly to the dispatched event.

Summary

Handlers are a good step forward for GWT, and make abundant sense in a Web environment where code size is very important. They also make it much easier to add you own events since you no longer need to manage your own Listener registration / deregistration cycle or firing of the events to all of the registered Listeners (HandlerManager does it all for you). I think there will be some interesting new patterns emerging with regards to managing events on generated structures (built based on data fetched from the server for example).

I’m looking forward to seeing what comes of these changes.

Performance improvements to EoD SQL 2.0

Having released EoD SQL 2.0, I thought I should take another look at the performance of the new code. Although the API does run a lot faster once you have a Query object in memory, constructing the Query in the first place is not as fast as the EoD SQL 1.0 branch. There are two reasons for this:

1. EoD SQL 1.0 Lazy-Loaded all of the method implementations (and in fact almost all of the data)
   1. Although faster to construct an object, the actual execution was much slower
   2. EoD SQL 2.0 eager builds as much data as possible
   3. EoD SQL 2.0 also works with tighter bindings under the hood than EoD SQL 1.0 did
      1. Takes the guess work out of turning a row into an object
2. EoD SQL 1.0 Cached it’s implementation objects and reused them extensively
   1. EoD SQL 2.0’s close bindings makes it much harder to cache the data for re-use
   2. However, much of the data is very lightweight to construct

With these two factors in mind, I climbed into the code and added an internal cache structure (the first part of EoD SQL 2.1). The new cache is very different to those used in EoD SQL 1.0:

• The new cache doesn’t store actual bindings, but rather all the data required to create a binding
  • Once a method is called, the implementation will store the resulting binding for that Query instance forever
  • Thus caching is not as useful, especiall since two different methods can’t share bindings anymore
• The new cache makes use of SoftReferences instead of being a permanent cache
  • Follows the caching technique used by java.lang.reflect
  • If you create static references to your Queries, the cache can be garbage collected
  • If you manage your Connections by hand, the parts of the cache that are in more use will stick around

This change marks the first part of EoD SQL 2.1, the new code is in Subversion if you want to try it out for yourself.

Easy Database Binding in GWT with EoD SQL

Connecting Java to a Database effectively can be a lot of work at the best of times. Sure JDBC does make life easy for us, but theres still turning to ResultSet into nice Java Objects. When you throw GWT into the mix, what was a lot of work becomes a bit of a nightmare (especially when you’re trying to work with JPA or Hibernate). Fortunately EoD SQL comes to the rescue!

For those of you who are either new to my blog or new to EoD SQL, go check-out this little snippet of code over on the EoD SQL homepage. You can think of EoD SQL as a very lightweight Hibernate. It’s also the perfect way to connect you GWT application to your database. Unlike Hibernate and JPA, EoD SQL requires no modification or work-arounds to get it to work perfectly with GWT.

You do need to keep to 2 very simple rules however:

1. Your Query interfaces may not return DataSet or DataIterator objects (since they require a working JDBC Connection)
2. You will still need an RPC Servlet between GWT and EoD SQL

That said, there are strong upsides to these “limitations”:

Instead of returning a DataSet or DataIterator: have your Query interfaces return single objects, or arrays of objects. Arrays are heavily optimized by EoD SQL and since your application is actually running in a browser somewhere, you want to provide it with the entire ResultSet (and not lazy-load the data, which is where the default Hibernate and JPA models fall over).

Instead of exposing your database methods directly to the client: you should place the functionality behind a RemoveServiceServlet anyways! Why? Because it gives you a good chance to verify that the user is allowed to take the given action on the database. If, for example, you wrote a forum application, and you had an admin-only “deleteForum” method that you exposed. If the RPC Servlet just deleted the forum and returned, anyone who got hold of the URL and knew anything about GWT RPC could delete every forum in the database. Not really something you want to allow.

Summary
EoD SQL works with GWT without any modifications. This is mostly due to the fact that EoD SQL is designed to be a “what you see is what you get” API. The fact that it doesn’t manage parent-child relationships automatically is also advantages in that it forces you to think about what data is going to be needed on the client side. Bare in mind that GWT by it’s very nature runs in a limited environment on the client, sending and receiving complex object graphs is never a good idea. It’s better to lazy-load child relationships when there is a strong chance the child data will not be needed.

I may consider an eod4gwt API which includes a special RPC Servlet implementation for handling EoD SQL requests (which would allow you to put the @Select and @Update annotations in your RemoteService interface). There would need to be a demand for it though

An automated JSON Encoder and Decoder for Java

I posed a little while ago that I’d implemented a JSON Encoding that worked a bit like Java Serialization. It works as such in that you don’t need to encode the objects by hand. As long as you follow some basic rules, the code will automagically turn your Java objects into JSON, and the JSON back into Java objects again. Well, I finally got my act together and the implementation is below. Consider this Public Domain code now! If it doesn’t work for you, it’s not my problem (that said, I’d like to know if it doesn’t work).

Before The implementation, heres a few things to know about it:

1. It solves the Date problem in such a way that it’s transparent to the JavaScript code!
   1. Dates are encoded as “new Date(UTCTime)”
   2. They are decoded against a collection of date-formats (listed at the top of the JSONDecoder class)
2. The encoding and decoding work against Reader and Writer objects, and thus have declared IOExceptions
   1. They expect to be closely tied to the Reader and Writer provided by the Servlet Container
3. They both conform to the Java Beans specs, so if it’s not a bean property, it’s ignored
4. The decoder can handle both the abstract Collection types and concrete types
   1. public void setUserList(List<User> users)
   2. public void setUserList(ArrayList<User> users)
   3. The Collections must have a solid type declared as their content (in the generic)
5. Map objects are encoded as JavaScript objects
   1. Map keys must therefore be Strings
   2. The Map keys should be valid JavaScript identifiers

If you are going to use this code, I would appreciate a comment here (just to let me know).

Eod SQL 2.0 Finally pushed out of the Stable

After a year of carefully removing straw from EoD SQL’s hair, I’ve given it the final push through the stable doors and into the sunlight! For those who have been working with the early access releases, you’ll be used to the very cool new features by now. But for those that have been sitting on the year-old 1.0 or 1.1 releases, heres a short-list of the awsome new features in EoD SQL 2.0:

• Much better concurrent behaviour
  • Less lazy-loading
  • Less locking
• Private fields and constructors allowed in data-objects
• Faster object binding
  • Data-Object / Query pairs are now bound together
  • Replaces the old method of creating one binding object per data-object
  • Much less work per row of a ResultSet
• Much faster array mapping
  • Different mechanisms based on JDBC driver capabilities
  • Fastest (and most common) is the array is pre-allocated and populated backwards!
  • A fall-back remains to populate forwards and resize the array as required
• A new plug-in architecture allows you to add your own annotations (like @Select, @Update or @Call)
  • This allows you to add special case structures that you need in your code
  • All of the special objects in EoD SQL remain available for you to use
    • You can use EoD SQL to auto-wrap any ResultSet object in an arbitrary Type
    • Or you can code the wrapping yourself

So why not head over and download EoD SQL 2.0! Give it a spin and tell me what you think.