Documentation: Docs Docs Docs

One of the major reason I love programming in Java, is the excellent quality of the documentation. I can look up any class, method, field, enum, annotation, or even an entire package and get loads of great documentation on the subject. It doesn’t just tell me what it does, it also tells me valid inputs, outputs, what exceptions can be thrown, and often even the process the code goes through.

I’ve heard many people tell me: just look at the code, it’s really self-explanatory. I used to work as a trouble-shooter, so climbing into code quickly was basically my job. The fact of the matter, there’s no substitute for good documentation, none! On a side note: my IDE’s auto-completion window comes up with the field or methods documentation. Sure the code is a CTRL-Click away, but that’s an extra step, and another trawl through other code that I really don’t need.

Personally I love writing documentation explaining how my code works, where it has flaws and where it’s really cool. One annotation I wrote yesterday is a marker annotation (ie: it’s got no attributes), I wrote over 30 lines of documentation for it. It probably sounds like overkill to most people, but trust me on this, without it, people would have to spend a whole lot of extra time figuring out what my code does.

The message here is: don’t forget to document your code!

Busy busy busy busy busy

To give you an idea of how busy: I’ve been working on this project for 4 week, and have personally written over 16′000 lines of code. It’s still not complete, and our deadline is fast approaching, but today we begin with testing the project. There shouldn’t be to many things wrong, since I’ve been doing some fairly rigorous testing as I go, but there’s always little lurky bugs around this time of the project.

More on EoDSQL

In my last post about my OR (Object Relational) mapping layer, EoD SQL, I mentioned several features I wanted to add to the API. Well good news everyone! I’ve been working really hard (on other code… which uses EoD SQL), and finally found some time to work on EoD SQL again, and I’ve added several of the features I wanted:

  • @Select methods can return a lovely arrange of data-types, including simple POJO’s (great for “selectById” methods)
  • QueryTool no longer throws nasty SQLExceptions, making the API potentially “checked exception free” (unless you declare an exception of course)
  • @ResultColumn can annotate setter methods, allowing you to hide your fields away as private and have EoD SQL work you JavaBeans methods

So theres a new release (version 0.5-beta), and the actual release structure is getting much better, with a nice shiny README file, and three formats to choose from (tar.gz, tar.bz2, and zip).

I’ve also started working on some more features:

  • QueryTool will no longer throw IllegalArgumentException’s when it’s asked to construct a query implementation that doesn’t follow the rules, instead it will throw a new (unchecked) InvalidQueryException
  • A new return type is in the works, called “DataIterator” (like DataSet, but forward only)
    • It’s forward only (rather than declaring “forward-only” in your Select annotation)
    • It’s usable with the Java 5 “foreach” statement
    • It has a “rubberstamping” mode: it only constructs one data object, and modifies it each time you call “next”
      • Instead of creating one data object for each row
      • It’s great for displaying a list of data
      • It can save you lots of memory
      • It also makes your software faster
    • It automatically closes itself when it reaches the end of the data (though it can be closed before then)

So as you can see, there’s loads of new functionality on the way (and already there), watch this space (and the sourceforge project).

No more PHPBB

I know a large group of people who have been having endless trouble with PHPBB lately. As a Java programmer, I’ve naturally been looking around for a Java based alternative. The main problems for them are:

  • The email server isoverloaded with notification emails
  • Spam Bots are overrunning the forums through SQL injection
  • Their database gets turned off towards the end of the month due to overuse
  • PHPBB doesn’t properly respond to date-stamp requests from the browsers
    • This is a common problem with PHP apps that I’ve seen
      This problem means that every time you look at a page, the server regenerates it, even if your browser has a copy cached

After looking around a bit, I found JForum. It’s a page for page, image for image clone of PHPBB in Java using Servlets and Freemarker. It includes RSS feeds (no more notification emails, though it does still support them). Really easy to customize templates. Very intelligent caching of topics, including the idea of “hot topics” which should be stored in memory for reading (though they are also stored in the database, they are pre-cached in memory to reduce database load). It even supports HypersonicSQL, so you can run it without a big database like MySQL (though it also supports MySQL, PostgreSQL, Oracle and a few others).
So far I have been nothing but impressed with this forum engine, and strongly advise anyone looking for an alternative the PHPBB to take a look. For those of you who don’t know how to run a Java Servlet application, take a look at the Apache Tomcat project. It’s a Java based web server, PHP includes a Java Servlet for doing PHP within Tomcat, Jetty, JBoss and other Java web servers these days, so you don’t even have to give up your PHP sites.

Crippled Passwords are Amoung us!

I love my secure passwords. Lovely passwords like “fr!bBl3~98_m0nT4=” (no, this is not a password I use, just an example of the type). This kind of mangling makes a password more difficult to guess for password breaking programmes. No it doesn’t make them “unbreakable”, but nothing in this digital world of ours is truly “unbreakable”, but many things are “very very difficult to break”, and that’s kinda what I aim for.

When I signed up for Internet banking with my bank, I thought “nice secure password”, and typed a really horribly convoluted password, the response was “invalid password”. I picked up the phone (something I don’t do often), and called their help line, only to get told “you can only use letters and numbers in your password sir”. The frightening thing is: there are many web sites I’ve signed up to recently that have this “no symbols or spaces” password policy. What on earth is wrong with you people!?!?! You tell me to select a secure password, but then tell me the one I gave is to secure??? I see absolutely no technical (or non-technical for that matter) reason why you cannot store my horrible password.

Surely you don’t store my password as plain text in your database do you. This is a massive potential security problem. If someone breaks into their database, they own your accounts. One of the first things I do is click the “forgot my password” link on the site. If they send me the password I typed in, I change my password and get rid of my account, simple reason: they’re storing my password somewhere. If they reset my password, or send me a random one, it’s a good indication that they are storing hashed passwords, and so my data is a bit more secure.

Be careful what sites you sign up with, how secure their data is directly affects you.

EoD SQL, What next?

I’ve been really swamped with work recently, trying to put out a new product. It’s going well, me doing most of the programming, and we’ve got a really amazing web / graphics person on-board, which makes my life a lot easier. It’s also the first project I’ve been using my EoD SQL in that’s an actual, paid-for product. Quite an achievement for a project that I only started a few weeks ago.

I originally was going to use the Java Persistence Layer, but found it way over complicated. I did think about Hibernate, mainly because EoD SQL was to young, and I wasn’t sure it could cope with what I needed it to do. After the last time I used Hibernate, and got bogged down in their query language, I decided against it. It took quite a bit of discussion for me to decide to give EoD SQL a chance, and to my great surprise, it’s worked amazingly well.

That said, after working with it for some time now, I’ve started to see where the project could get some new features that weren’t in the original JDBC extensions, and I’ve put together a TODO list of sorts for myself. Here’s a first glimpse of features to come:

  •  Returning simple, non-db datatypes from @Select methods
    • Simple POJO’s
    • Arrays
    • Java Collections
    • etc.
  • @ResultColumns on setter methods
    • This will allow for JavaBeans compliant data objects
  • Commit and Rollback for Quey interfaces
  • Forward-only DataSet’s
    • Currently all DataSet’s are random-access
  • New @Join annotation
    • This will allow for more complex SQL joins, and simpler data objects

That said, I am finding the API really amazing to use, particularly since it keeps the full power of SQL very close. I’m also finding that the extensions I’ve made are very much more useful than I thought they would be.

Where did we go wrong?

Society on the verge of collapse.

It would make quite a headline, wouldn’t it? One that you would expect from a post-apocalyptic movie. If you saw it in the headlines tomorrow, you’d probably cancel your subscription to that newspaper. While we sit in our comfortable, or semi-comfortable lives however, our world is on the verge of collapse. So while we stand at the edge of oblivion, why can’t we see the end? Why isn’t there fire and torture and people running riot in the streets? I’m not going to answer those questions, think a few seconds about our society, I’m sure you have the answer.

I want to take a look at our political situation. Why are we still stuck in the 18′th century? Or maybe earlier? Why do we still see our politicians as rulers, rather than servants? We pay their salaries. When I pay someone’s salary, I generally expect them to deliver on the deadlines I set for them. Why should our governments be any different? The ancient Greek politicians were paid a basic minimum wage. This seems really intelligent to me, cause it gets rid of the idea of a “career politician”. You can’t be in it for the money if there’s no money in the job. So they’re in it for the power? Well that the next problem to solve: how do we take the power away from them? The answer to this is fairly simple: open up the process. If any law, bill, or declaration is to be made, it needs to be open and transparent. Not open to “representatives” of the public, open to the public itself. A career politician fears nothing more than the people they supposedly govern.

Society needs to ungo a massive reform if it’s going to survive.

OR Mapping layers

Having just released version 2 of my EoD SQL API (see yesterdays post) today with some massive bug fixes, I’ve been thinking about the way I code, and some of the code I’ve seen others write in my life time. Having worked in a largely “Enterprise” (note the upper-case) environment: EJB, EJB, EJB, and Very Large Database Systems, I’ve seen a lot of SQL code, a lot of OR mapping, and a lot of headaches and pain caused by both (and sometimes by a lack of coffee). So what have been my thoughts on the subject? Well this is gonna sounds really self-serving, but: most OR mapping layers I’ve worked with really suck.

JDO was one of the best (mainly cause I was working with JDO Genie), and that was mainly cause the JDO Genie work-bench let me write lovely little objects, and then built the database schema for me. Having grown up with OO as my first understanding in programming… this was really cool for me. However, once I tried to use this system with a 20 year old legacy database that had over 200 tables of data, I found myself wishing I could just use SQL instead (though my understanding of SQL at the time was almost non-existent), but the application I was working on, was not written by me… I was just the maintainer.

After trawling through several other implementations of JDO, and one or two home-grown OR mapping layers, I worked on a system that was based on Hibernate, Spring and XDoclet. At first I thought this combination was awesome, it dealt with Spring had some lovely code to deal with a CLOB restriction in Oracle’s database drivers (we were working against a 10g database). Most of the config was generated for us, and hibernate did all the mapping to the database. Wonderful! Until I started getting weird errors and bugs (even my unit tests started failing). Why? To this day I have no idea. less than 2 weeks before the project deadline, I swapped all my Hibernate, Spring and XDoclet code for direct JDBC SQL to the database in desperation. The project was released, and to my knowledge is still working fine.

I guess what all this amounts to is: I like working with SQL cause then I’m not having to talk to the tables, I can write any query I want, and not have to create 120 view’s in the database to keep my OR mapping layer working and happy. But at the end of the day, you still wind up needing objects in memory. Probably why I really loved the EoD RI from Java 6 beta, and why I started the EoD SQL project.

EoD SQL

After getting all upset when the “Ease of Development RI” was postponed for Java 1.6.

To spending 3 days re-coding said RI, cause I had several projects that depended on that code.

To having to wait 5 days for a Sourceforge project.

It’s finally mine… BWAHAHAHAHAHAHA

What is the EoD RI? It’s a simple layer on top of JDBC that generates the implementation of methods that query and update the database (based on @Select and @Update annotations), and then map the ResultSet to POJO’s. So instead of trying to map Java Objects to the structure of the database, it maps the columns found in a ResultSet to some simple POJO’s. Leaving you to do complex JOIN’s  and sub-queries how-ever you want.

Why re-write such a simple API? Because I love to be able to code SQL myself. Having spent years working with JDO, Hibernate, and even TopLink, when Java 6 beta came out and I found the “EoD RI”, I fell in love with it. It’s simple, light weight, clean and easy to work with, and best of all: it doesn’t take you to far away from your database.

Currently the new project is still in it’s early stages, mainly due to a lack of testing (I can only test so much). So I’m looking for comments and requests on the project. The more people working with it, the more bugs are likely to surface, the better the API gets. Go check it out.