Writing Express Middleware to Modify the Response

I recently had a need for some Express middleware that would track any change to a variable, and send the details of any changes in the response, but only if the response is JSON data. This turned out to be rather more interesting delve into ExpressJS than I expected, and I thought the details of it were worth a post detailing how to do it, and my findings.

The first thing you notice when writing Express middleware functions is that they take three arguments (I’m skipping error-handling middleware deliberately), the request, the response and a next() function that triggers the next step in the processing chain. The next function doesn’t typically take any arguments, instead you’re expected to modify the request and response objects to expose or capture any changes you want.

This post will show you how to capture any changes to a variable (we’re going to pretend all our users are playing a game and we have a leader-board), we want to track any changes that happen during the request and report both the changes and the current state of the variable in every response. I’m going to assume some other middleware has injected a user property onto the request. First off, this is what the code will look like:

const leaderboard = require('./leaderboard');
function leaderboardTracking(req, resp, next) {
  const user = req.user;
  const startingPosition = leaderboard.getUserPosition(user);
  const json_ = resp.json; // capture the default resp.json implementation

  resp.json = function(object) {
    const endPosition = leaderboard.getUserPosition(user);
    object['leaderboard_info'] = {
      'delta':    endPosition - startPosition,
      'position': endPosition,
      'score':    user.score
    };

    json_.call(resp, object);
  };

  next();
}

So in this example we swap out the json function with our own delegate implementation, but you’ll notice we leave the send function alone. The send function will delegate to the json function if it’s given a plain JavaScript object, and json in-turn uses send after stringifying the object.

You’ll also notice that the code doesn’t just invoke the captured json_ function, it uses the call function and specifies the response as this. The json function in Express expects to be invoked within the context of the response object, so we need to keep it that way.

That’s it really, it’s not a terribly complicated pattern, but it can be extremely powerful because it bounds the request / response cycle end-to-end.

Advertisements

Improving Perceived Performance on Android

Android is one of the strictest platform I can think of when it comes to threading. Everything seems to have a rule around whether or not it is allowed on the main thread. All user interface code (code that touches the widgets on the screen) must be run on the main thread (since it also serves as the event loop) in order to avoid concurrency issues; but at the same time networking cannot happen on the main thread. For a good reason too: networking is always slow relative to a single event (just think about NodeJS or Vert.x and the fact that all networking is offloaded away from the event loops).

Most Android applications take things one step further and try to do any local database work (typically with SQLite) on a worker thread. The most common Android method of doing this is with AsyncTask (and friends such as AsyncTaskLoader). However: these systems are often heavy-weight to implement, requiring lots of boiler-plate code. The problem there is: many things that should be on background threads are left on the main thread because it’s just too much effort to write another AsyncTask. This is not a good situation.

After a few years of developing Android, and writing a book on the subject I found myself reusing a simple pattern that at first I called a BackgroundForegroundTask. At first a massive simplification of AsyncTask (which is totally over-engineered for most purposes). Instances of BackgroundForegroundTask would just:

  1. Call onBackground on a worker thread in the background
  2. Call onForeground on the main thread with the result of the onBackground method
  3. Call onError on the main thread instead of onForeground if onBackground threw an Exception

I used this class for almost every event handler. The difference to applications was massive. Everything became silky smooth, because the main thread is almost entirely left alone to process input like that user scrolling, tapping, typing, etc.

Just forward to the present day, and I’ve modified the pattern to allow these BackgroundForegroundTask (now called ActionCommand) objects to be chained together. This enables me to write commands that only do one thing, and with no state of their own. They can be chained together, the chains can be kept and reused over and over to perform defined sets of actions:

  1. update a local user object
  2. send it to the server
  3. update the user interface

Three jobs, three ActionCommand classes:

private final ActionCommand.Chain<User, User> saveUser =
        new UpdateUserCommand(database)
            .then(new UpdateUser(serverConnector))
            .then(onForeground(user -> {
                updateUserDetails(user);
            });

In the third case above I’ve added a lambda that will be wrapped in an ActionCommand object. You can also run commands by only composing such lambdas together:

onBackground((name) -> "Hello <b>" + name + "</b>")
    .then(onBackground((msg) -> Html.fromHtml(msg)))
    .then(onForeground((msg) -> {
        Toast.makeText(this, msg, Toast.LENGTH_LONG).show();
    }))
    .exec("Jason");

Will offload the concatenation and parsing of the HTML onto a background thread, only returning to the foreground thread to create and display the Toast object. The actual execution pattern for the above code is a bit special, it actually runs like this:

  1. Run Html.fromHtml("Hello " + name + "") on background thread
  2. Run Toast.makeText(this, msg, Toast.LENGTH_LONG).show() on foreground thread

The Chain class automatically optimises the number of tasks and thread-hops when then is called with an ActionCommand returned by onBackground or onForeground. Check the code out on GitHub if you’re interested.