Unit Testing with Mock Objects and EoD SQL

Introduction

Unit Testing has become one of the corner-stones of good software development (it has been such for quite a long time actually). However writing a Unit Test that interacts with database code (at any level) can be very challenging. The main problem being: your data should be in a “known” state when the Unit Test starts running. In the past many approaches to this problem have been taken, such as: using an in-memory database; or DbUnit. Each approach has it’s up-sides and it’s down-sides (I won’t be going into them here).

To help your unit testing along EoD SQL is growing it’s own mocking extension (currently only in Subversion however). The EoD-Mock project allows you to build mock implementations of your database connection code without any external configuration. Thats right: no dependency injection; no JNDI; no factories; nothing!

How does it work?

If the EoD-Mock JAR file is on the class-path, EoD SQL will automatically look for mock implementations of your Query interfaces instead of generating a real implementation. For example, take a simple UserQuery interface:

public interface UserQuery extends BaseQuery {
    @Select("SELECT * FROM users WHERE email = ?1")
    User selectByEmail(String email);

    @Update(sql = "INSERT INTO users (email, username, birth_date) "
    + "VALUES(?{1.email}, ?{1.username}, ?{1.birthDate})",
    keys = GeneratedKeys.RETURNED_KEYS_FIRST_COLUMN)
    User insert(User user);

    @Select("SELECT * FROM users")
    DataSet<User> selectUsers();
}

To create a Mock of this interface using EoD-Mock all you need to do is write an implementation in the same package:

public class UserQueryMock extends AbstractMockQuery implements UserQuery {

    private final List<User> users = new ArrayList<User>();

    public UserQueryMock() {
        insert(new User("joe.bloggs@nowhere.com", "Joe Bloggs", new Date(83, 3, 6)));
        insert(new User("jeff@jeffswebsite.com", "Jeff Site", new Date(76, 8, 23)));
        insert(new User("logan@murkmurk.com", "Logan Sleep", new Date(90, 4, 1)));
    }

    public User selectByEmail(final String email) {
        for(final User user : users) {
            if(user.getEmail().equals(email)) {
                return user;
            }
        }

        return null;
    }

    public User insert(final User user) {
        final long id = users.size();

        final User clonedUser = new User(
                user.getEmail(),
                user.getUsername(),
                user.getBirthDate());

        clonedUser.setId(id);

        users.add(clonedUser);

        final User idUser = new User(null, null, null);
        idUser.setId(id);

        return idUser;
    }

    public DataSet<User> selectUsers() {
        return new MockDataSet<User>(users, false, true);
    }

}

Yup, it’s really as simple as that. Now if you ask EoD-SQL for an instance of UserQuery (ie: QueryTool.getQuery(UserQuery.class)): instead of generating an implementation, it will create an instance of UserQueryMock and return that.

How much does eod-mock take care of?

  • EoD-Mock will self-register as a QueryFactory with EoDSQL (but only under Java 6 and higher) if it’s on the classpath
  • A “default” DataSource is provided automatically. The provided implementation throws exceptions instead of providing database access
  • An AbstractMockQuery class is provided to take care of the methods declared in BaseQuery
  • A MockDataSet is provided, and will attempt to behave like a real DataSet object
  • QueryTool.getQuery will automatically return mocked query objects, meaning: no changes to your data-access layer

Current State

EoD-Mock is currently only available in the Subversion repository, but will be included in the next release of EoD SQL. There is no support currently for the DataIterator class, but that will no-doubt come very soon.

EoD SQL 2.1 Released

So I finally found some time to make a 2.1 release for EoD SQL. For those who don’t already know:

EoD SQL Allows for fast, simple binding between a Relation Database Query and Java objects.

Think of it as your friend that gets Hibernate or JPA out-of-your-face and lets you get on with actually writing some code.

The 2.1 release has a long list of changes behind it (which we’ve built over the last few months).

  1. It’s much faster than EoD SQL 2.0 was (which in turn was faster than 1.0).
  2. There are a good number of bug-fixes, making it nice and stable.
  3. The 2.1 release includes batch updates
    @Update(sql=”INSERT INTO users (name, email) VALUES (?{1.name}, ?{1.email})”,batchUpdate=true)
    void batchInsertUsers(Collection users) throws SQLException;
  4. GWT developers rejoice! EoD SQL can now fetch objects in any Collection type you choose!
    @Select(“SELECT * FROM users WHERE group = ?1”)
    ArrayList getUsersInGroup(long groupId);
  5. Byte arrays are now considered primitive types, and get mapped by EoD SQL out-of-the-box (no custom TypeMapper required)

That’s far from an exhaustive list, and you should really go download the API and try out some of the cool new features.

EoD SQL 2.1 – Now in First RC Release

EoD SQL just got a 2.1 Release Candidate kicked out the door. While there are (once again) many performance improvements and a few bug fixes, the big news of this release has got to be Batch Updates!

Some people (myself included) have been waiting a very long time to see this feature in EoD SQL, and it’s taken a fair amount of discussion to decide how it would work. The final answer came from Bernd Rinn: allow parameters to be specified as various Collection types. So to add a list of User objects to your database:

@Update(sql="INSERT INTO users (name, birth_date, password, email) VALUES (?{1.name}, ?{1.birth_date}, ?{1.password}, ?{1.email})",batchUpdate=true)
void insertUsers(List<User> users);

Batch updates will work with any class extending Collection, or an array type. It can also be used with simple parameters:

@Update(sql="REPLACE INTO user_group_join (user_id, group_id) VALUES (?{1.id}, ?{2.id})",batchUpdate=true)
void ensureUserInGroups(User user, Collection<Group> groups);

A Batch Update is generally much faster than a series of normal update invocations, because EoD SQL will use PreparedStatement.addBatch() to build up all of the parameters on the client-side before sending all of the data to the database in one statement. It doesn’t mean it’s a single transaction, but it does mean that the database can execute all of the parameter variations before sending any data back.

So why not go download the new release and give it a try.

Bernd Rinn

EoD SQL 2.1-Beta Released

EoD SQL 2.1-Beta has been released, you can go download it from the Documents & Files section of the homepage. The 2.1 release is about improving speed and usability over the 2.0 release. Some of the new features included are:

  • Specific Collection implementations (including those not in the standard Java API’s) can now be returned:
    @Select("SELECT * FROM users WHERE group = ?{1.id}")
     ArrayList<User> selectUserGroup(Group group);
  • Select methods can now populate existing objects:
    @Select(sql="SELECT * FROM user_meta_info WHERE user_id = ?{1.id}",into=1)
    void selectMetaInfo(User user);
  • Faster construction of query implementations
  • Better error reporting
  • Several small bug fixes

This is of course a beta release, so there may be bugs. That said the code is pretty well unit tested and should be relatively stable. One new feature not in the release (but on the way) is batch updates. Stay tuned for more information!

EoD SQL Applied – Part 5 / 5 (GWT Applications)

The Dreaded “Not Serializable” Problem

GWT turns all of your Java code into JavaScript, but it also obfuscates everything. With this in mind, it makes serialization a bit more complex than usual. GWT generates a serializer and deserializer for each class that could be transported across the wire to an RPC service. The difficulty comes in knowing which types had code generated, and which didn’t. GWT solves this problem for itself with the Serialization Policy file, where the compiler lists all of the classes that the client code will know how to deserialize.

This however leads to another problem: what happens when something unexpected gets in the way. Hibernate is the most used, and thus most complained about when it comes to Serialization problems. Hibernate turns any Collection object into a special “lazy” implementation that will only load your data when you ask for it. All very fine and well for a server bound application, but a GWT application needs all that data up front for serialization. When GWT comes to sending that List<Message> to the client, it chokes. It knows how to serialize an ArrayList, and LinkedList (since you worked with them on the client side), but a Hibernate List or PersistentCollection is a totally unknown type, so it’s not in the Serialization Policy, so the server side throws an Exception at you.

So how does EoD SQL help with these problems? Read on to find out! 😉 Read the rest of this entry »

EoD SQL Applied – Part 4 / 5 (JavaScript)

JavaScript vs. Web Applications

So far in this series we’ve discussed using DataSets for Swing applications and DataIterators for web-applications. Why would I now bring in JavaScript as something outside of “web-application”? JavaScript applications have very different requirements to a normal web-application. Where a normal web-application has little ability to do things like preload data (like the next page), a JavaScript application may (for example) download the entire data-set and then display it in pages. This next section is about binding to JSON for JavaScript applications.

First thing to remember here is that an EoD SQL DataSet is a List and thus compatible with the Collections API. For this example we’re going to be working with the outstanding GSON API from our friends at Google. Our objective here is to minimize the amount of time spent between the Database and pushing the data to the client. Because GSON doesn’t appear to support Iterable object out-of-the-box, we’re going to start off using a DataSet.
Read the rest of this entry »

EoD SQL Applied – Part 3 / 5 (JSP / Servlets)

Static Data Display

In a Swing application with a direct connection to a database, it makes sense to leave the database to do much of the heavy lifting. The Swing client can keep a minimum amount of data in memory, while holding a scrollable Result Set open on the database (fetching more of the data as required). In a web application on the other hand, things are a little more complex. For one thing, your database isn’t exposed to the any part of the network; for another: your clients have no ability to keep Result Sets open; and finally: you generally are working with much higher loads than a Swing application.

EoD SQL has many different approaches to loading data from the database. We’ve already looked at DataSet objects, which remain connected to the database and work well with scrollable, cursor backed, Result Sets. DataSets are (in a way) a leftover from the origional EoD API from Java6-beta (where they were the only way to fetch objects from the database). In this post we’ll take a look at an EoD SQL idea: DataIterators.

Read the rest of this entry »

EoD SQL Applied – Part 2 / 5 (Swing Applications)

Swing Applications

Introduction

We start off the series with Swing Applications. Why? Although probably the application type least likely to be using EoD SQL right now, they nevertheless make use of the most commonly demonstrated EoD SQL data-structure: the DataSet. Swing Applications often have direct access to their database. They can therefore make much better use of long-lived database connections and cursor based ResultSets than web-applications can. So we open with a more detailed than-usual discussion of the DataSet interface and it’s implementations.

The DataSet is the original return type of the EoD API from Java 6 – beta. It’s a relatively thin wrapper on top of a scrollable ResultSet, and it performs best when your database that supports cursors, and your client that will be looking at large volumes of data over a longer periods of time. You’ll notice that this fits well with an intranet Swing client that has a direct connection to it’s database.

Read the rest of this entry »

Eod SQL Applied – Part 1/5 (Introduction)

Introduction

This is the first part of a series of posts, dedicated to how best to apply EoD SQL in different types of applications. Each post in this series will cover a specific type of application, and the different parts of EoD SQL that generally work best within those applications. The articles may dive fairly deep into the workings of the API, but at the end you’ll have a much better idea of where things fit in, and how they fit together.

EoD SQL well understands that (a) people like to do things differently and (b) one solution is not right for everything. While under the hood: any one part of EoD SQL works much the same way as any other part, as you climb the structural ladder, things begin to behave very differently. These behaviors have a massive performance and memory impact on your application and how you will be treating the underlying database drivers.

What we’ll be covering

  1. Swing Applications
  2. JSP / Servlets and “Related Technologies”
  3. JavaScript / JSON Applications
  4. GWT Applications

Although you could go with the “one size fits all” route (and EoD SQL would still perform wonderfully), the objective of these posts is to expose you to the different flavors of EoD SQL data structures and get you thinking about how you can mix and match them in your application to produce different results.

EoD SQL Version 2.1-Alpha

EoD SQL doesn’t see very much development these days (although it sees plenty of downloads and usage). Mainly it’s very stable and does what it’s supposed to, so theres very little need to change things.

That said, there are a few issues that needed some attention, so theres the start of a new release (2.1). The new features and changes are as follows:

  • The engine now caches some of the more expensive construction data with SoftReferences, dramatically reducing the cost of creating a Query implementation
  • @Call methods may now return updatable DataSet objects by using the new “readOnly” annotation attribute
  • @Select methods now have a useful “fetchSize” hint that will be passed down to the JDBC driver
  • Much better error messages for invalid / unknown types and query parse errors
  • Improvements and updates to all of the JavaDocs
  • Some bug fixes and corrections

Like I said: theres not that much there, but the changes that do exist make life that little bit easier. If you have any ideas of features you’d like to see in this upcomming release, leave a comment or a feature request and we’ll have a chat about it (always on the lookout for good ideas)!