Development on EoD SQL Continues

A few days ago I released EoD SQL 1.0, and moved the project to Java.net. So now that 1.0 is out the door, where to next? There are several developments that will be taking place in EoD SQL land, and I’m looking for:

• People interested in developing on the codebase
• Input!
  • Bug Reports
  • Feature Requests
  • Ideas
  • What do you use EoD SQL for?

As of now, the next set of developments on EoD SQL include:

1. Writable DataSet objects
2. Internal optimizations
   1. Reuse PreparedStatements
3. Batch Processing (using the JDBC addBatch() methods)
   1. This will be an optional operation in order to cater for JDBC drivers that don’t support batches
4. Custom QueryFactory objects
   1. Based on the ServliceLoader class
   2. Only available for Java 1.6 or higher
   3. Those running 1.5 will not be able to use Custom QueryFactories
   4. Will be the base for an EoD SQL Query compiler

Let me know what you think of the upcoming feature set, what other features would you like to see, and if you’re interested in working on the EoD SQL project.

Windows; Linux and Memory

You often hear how much Linux like memory, and how much it eats up compared to Windows. It’s true! Linux eats into your memory, no matter how much you have, you never seem to have any free for applications to run in! I recently read an article where the author was complaining how Firefox ate into his memory under Vista. The sad fact of it is:

While Linux appears to eat more of your memory, it manages it much better than Windows does.

I have Firefox open right now, I have 10 tabs open with graphics intensive web sites. Firefox is claiming 200Mb of my precious memory… or is it? Actually no: real size in memory 90Mb.

The current list of the most memory intensive applications I’m running, and their “supposed” memory usage:

• Java (Netbeans) 600Mb
• Xorg 380Mb
• Firefox 200Mb
• Thunderbird 130Mb
• Amarok 120Mb
• Kwin (KDE) 120Mb

Now considering I only have 1.5Gb of memory in this machine that 50Mb to much, and considering there are a total of 120 processes running, I haven’t even started to cover the list. Now as any knowledgeable person will tell you, Linux is using some of the hard-drive and pretending that it’s memory, but since a hard-drive is slow, my computer will be running awfully slow.

Wrong! Yes, a disk is much much much slower than memory (especially a laptop drive like this one). However, my computer is running very very fast. Why? Linux’s memory manager notes what parts of memory are used frequently, and what parts are seldom accessed. This means that the parts that are used frequently stay in real memory, and the stuff no-one is using gets stored on the disk.

Translation: Linux will run much faster, and keeps things in memory in such a way as to speed things up! Under Windows, more free memory is better, under Linux it’s the opposite! Less free memory is better, since it means Linux is doing it’s job better.

EoD SQL 1.0 is Finally Here!!!

And the project has moved to eodsql.dev.java.net. New features in the 1.0 release include:

• Added the QueryTool.select method for less structured / once-off queries
• Added the DataSetCache, ArrayDataSetCache and NullDataSetCache for controlling the way a DataSet caches row values
• ConnectedDataSet’s don’t disconnect as often anymore, but are not thread-safe
• DataSourceQueryHandler’s now understand threading, and will reuse open connections if it’s safe to
• The deprecated methods in QueryTool have been removed!
• Brand new tutorial!

Go to the releases folder under Documents & Files to download the latest version!

ReadWriteLock’s on Collections

Many people don’t realize that Collections.synchronized* causes an exclusive monitor lock to be held in every method called on the specified Collection object. For example:

final List<Integer> list = Collections.synchronizedList(new ArrayList<Integer>());

Thread th1 = new Thread(new Runnable() {
	public void run() {
		for(int i = 0; i < Integer.MAX_VALUE; i++) {
			list.add(Integer.valueOf(i));
		}
	}
});

Runnable reader = new Runnable() {
	public void run() {
		for(int i = 0; i < Integer.MAX_VALUE; i++) {
			list.get(i);
		}
	}
};

Thread th2 = new Thread(reader);
Thread th3 = new Thread(reader);
Thread th4 = new Thread(reader);
th1.start();
th2.start();
th3.start();
th4.start();

I am well aware that the code for reading is very bad, but it serves the point. Four Threads, one writing to a synchronized Collection, the others reading from it. This as such is not a problem: writing to a Collection should only ever be done by one thread at a time, while no other threads are reading. However, any number of threads can read from the Collection at the same time. Collections.synchronized* acquires an exclusive lock for both read and write methods, so: only one thread will be allowed to access the Collection at a time.

The solution: Use a ReentrantReadWriteLock, which allows any number of Read locks at a time, but only one Write lock (and no Read locks while a Write lock is held).

There is an even more efficient way to make a Collection thread safe: use an inherently thread-safe collection. Just take a look in java.util.concurrent, and you’ll see that Java comes packed with plenty of them. You will need to decide which method is preferred for what you are doing. Take into account:

• How many reads vs. how many writes
• How large is the collection
• How fast does the code need to execute
• How often will the collection actually be accessed

Why Sony PSP needs Java

The PSP is awesome. I pre-ordered mine, and it still amazes me every time I play a game on it. I’ve been very picky with the games I’ve bought for it (only 4 to date… and considering the time I’ve had it, you’d expect a much larger collection… specially if you looked next to my PS2).

The whole home brew situation on the PSP had me laughing a bit at first. Especially since most of the code written was focused on ripping the games off the UMD and onto the internet. BORING!!! Homebrew is supposed to be about writing new content… not pirating[*] old content.

The PSP is not lacking for First-Party games. But there are times where I’d love to be able to download a Tetris clone for it; or Pacman; or some arb game that someone made up in their spare time. The worry is: opening up the platform could lead to losses for First-Party and paying development houses. The solution: Java.

Why? Because you can sandbox the software. A MIDP VM need have no access of any sort to the underlying file system or hardware. All the PSP needs to allow for a flurry of Java developer activity is:

• MIDP 2
• 3D Extensions
• Wireless Toolkit (for WiFI gaming)

The fact is:

• if people played free games on their PSP…
• They’d be using the device more.
• If they used it more, they’d be buying more games for it.

Java for PSP doesn’t spell the doom on First-Party titles, it’s the best way to sell more titles![*] To quote my brother: I am not a pirate, I am a preemptive maritime salvage expert.

On Netbeans; memory and speed

It’s about the most common complaint about Netbeans: It needs to much memory! *wine wine wine*
That, and: it’s so slows *cry cry cry*

You know something, there are two simple ways around this “problem”:

1. If you are running Windos, switch to a real operating system! I recommend:
   1. Kubuntu for people with permanent internet or
   2. SUSE for those who are on dial-up
2. If you are actually writing code, you need a minimum of 1Gb (preferably 1.5Gb) of RAM
   1. This applies to all software developers, regardless of you language or target platform (I included embedded developers in this statement).
   2. Software developers simply tend to have more software running at any given time
3. A dual-core CPU does make an enormous difference to IDE performance

I do 100% of my development on a 1.6Ghz Centrino laptop, with 1.5Gb of RAM (533Mhz if you must know), with a standard laptop hard-drive. With this simple configuration I am amazed at the speed when I run:

• Netbeans 5.5
• Netbeans 6 beta-2
• JBoss 4
• Thunderbird
• Firefox
• Kopete
• Konqueror
• Amarok
• Kerry-Beagle
• Many terminals
• MySQL

Spread out over 4 KDE desktops, and they all perform as if they were running by themselves. If you have the opertunity, switch to Netbeans 6, as it’s a much faster system than any previous version.