Rails: The Other Foot.

It didn’t take very long at all to develop an initial version of my rails app – in fact – I’ve found rails to be immensely productive. Using yum and gem to install components also seems to be a big win. The whole development process was so ludicrously fast, in fact, that it caused me to be suspicious of what exactly I was giving up. It took me quite a while to figure it out; and in the end it came in the deployment stage.

It turns out that Rails is not thread safe. I was quite surprised to learn this, especially since ruby does in fact support threading; it’s just that Rails … doesn’t. This is a bit of a scalability issue, but it turns out that it’s not really a huge problem: I save SO much time in development, and what I was really trying to do was to throw an idea out and see if it sticks, which I am able to do. If it does stick and I need to scale it, I will gladly either hire some engineers to re-implement in Java, or buy some more servers, or probably both.

As for deployment, that isn’t fully sorted out in the same kind of way that it looks like it might be. My original thought was to go to FastCGI and plug it into Apache. Way back when (1997-1998 vintage) I worked with some large perl applications that used FastCGI, and it was always a nightmare with a bunch of processes spun up and some of them dying, turning into zombies and consuming large amounts of CPU. This experience still seems to be the state of the art, and the main reason why FastCGI is a bad idea. James Duncan Davidson has an excellent discussion on this topic on his blog. Further trips through the blog-o-sphere reveal yet more information, including the “pack-of-mongrels” approach which is detailed on Coda Hale’s blog. Thanks to both of these guys for very succinctly laying out solutions and issues on rails deployment.

The summary of this is that I have a cluster of mongrel servers running on my box, and I can deploy the application in an automated way (”cap deploy”) from my user account and have my latest code rolled out from development to production. The front-end server is apache, which seems to have grown load-balancing abilities in the form of mod_proxy_balancer. This is reasonably slick, and better than I would have had setup for a java app. Having said that – I would have been able to deploy my java app in tomcat, which is capable of handling a lot more traffic than my pair of mongrels. Of course, I don’t actually have any traffic, so much of this is mute anyway. All of this probably took about 4 hours of work or so (including some other minor deployment issues I ran into). Not bad at all.

Web Frameworks in Java

I’ve been working on a small web side project lately. Being a mostly Java oriented developer – I wanted to build a java web-app. It’s been about 2 years since I worked on a web-app full time – at which time we were using Struts 1 point something and jstl.

I did a little digging around and found a couple of blog entries and documents that talk about web framework comparisons…

• http://www.oreillynet.com/onjava/blog/2006/06/isnt_rails_supposed_to_change.html
• http://raibledesigns.com/page/rd?entry=re_what_web_application_framework
• http://themindstorms.blogspot.com/2006/06/stripes-sounds-cool.html

Unfortunately, none of these resources yielded a clear winner. The one thing everyone does go on about consistently is Ruby on Rails. Nobody seems to be happy with Java Server Faces, although there seems to be hope for the future. The middle ground seems to be filled with Tapestry, Struts 2 (the artist formerly known as WebWork) and Spring MVC (step by step guide here). There also seems to be some interest in Stripes.

Given that the Spring folks seem to have a good, clean, simple approach to things, I thought I might go that way for my app. Having said that, I quickly learned that just about all the web frameworks now use spring as their back-end container. Glowing praise indeed. I invested a couple of hours in downloading and unpacking the latest jars for frameworks, jdbc drivers etc, before I had a change of heart. Since everyone has been going on about Rails for the past year or so, why don’t I give that a try??

My first look at rails was watching the videos on the Rails website. These do a pretty good job of demonstrating how to start doing something with Rails. This blog entry assisted with getting rails installed on my Fedora server, and before long I was rocking along.

It seems like rails is basically the “most applications work like this” pattern but with automation scripts. I didn’t have to write a build.xml file, or write a task for exporting the db schema; write something to make spring load or anything else. The first thing I did was build a db table, populate it, and then build a web page to interact with it. The time I did spend was on building useful functionality: making it call Google maps and get geographical data, and then making the Javascript event model work on the client… things that I regard to be complex (or at least – highly visable). Prior to “giving up” on java, all my time had been spent plumbing. This is not to say java is bad…. it’s just that the rails framework took care of all the easy stuff by itself.

It’s obvious to me that I will pay a price for the ease of startup later on, but if you wanted to build a web application to see if it “sticks”, you don’t particularly want to pay the startup tax – you want to see if it’s worth investing any real time on it. IMHO – this is where rails is killer… from idea to implementation is much quicker than it would be with Java; and this leaves more time for what is often the harder pain: figuring out the requirements and user interaction.

The learning for the java folks IMHO is that some application layout standards need to be agreed upon, and then a standard application skeleton (or skeleton builder) needs to be published. We need to get to a point where you can do something like scrip/generate controller and have something generate a new Controller, View and supporting configuration for you, so that less time is spent doing the same old grunt work. Hopefully it’s already out there and I just haven’t found it yet .

Singletons & Double Checked Locking

I regard singletons as being perhaps the dirtiest of the GoF patterns. It’s problematic to assume there would only ever need to be one of something. As an example – the project I am currently working on has a client side application which has a series of singletons that facade server functionality. It turns out that this works very well, except there’s no way to renew these singletons if the user wants to log out of the server and log in again with a different user account. This means currently, the application has to be closed and reopened. This isn’t a show-stopper, but it is annoying.

Regardless of this, implementing thread safe, lazy initialized singletons in Java has always been problematic due some problems with the Java Memory Model. This is covered in detail in this excellent developerWorks article. I was mistakenly under the impression that this had been fixed in Java 5, but alas – it appears not. What I wasn’t aware of until today is that there is an alternative thread safe manner in which to implement a lazy-initialising singleton:
Initialize on demand holder classes.

I think the better approach nowadays however is to wire your application together using the Spring framework. Spring is very lightweight to implement, and will take care of initializing your singleton-like objects in order and wiring them together. Since spring offers full object life-cycle support, it’s easy to have it close all your objects out, and re-initialise with new ones, should you desire that.

Mail Servers and Filesystems

In 1998 I worked for a company that had a contract to process bulk email. The emails were to attempt to get leads to sign up for a trial membership and then reminders for when the trial would expire and the like. This raised some interesting questions about how to send bulk email and get the server to perform. The “best” answer to the problem was quite interesting – qmail on IRIX.

It turns out that most mail systems end up storing information in files on the file system. As an example – sendmail – generally regarded as the standard Mail Transfer Agent (MTA) stores three files on disk for each message in it’s queue. If you are sending a lot of email, that queue gets very big very fast. Most filesystems at the time used a linear scan to find directory entries which means that the average lookup order is 3n where n is the number of items in the queue. Irix’s filesystem, XFS (which is also available on Linux) uses a B+Tree to store directory entries in O(log n), which is much more efficient.

When I came to reinstall my own mail server a few years later, my principal issue was not with mail spools, but with IMAP message stores. Previous to this point I had been using sendmail and the University of Washington IMAP server which used mail spool files. I had an inbox containing about 2000 messages at the time; and operations on that box were becoming prohibitively slow. I installed the Cyrus IMAP server to replace the UW IMAP server and found that my IMAP performance dramatically improved. I was able to maintain an inbox with around 5000 messages in it before things became problematic again. This was a significant improvement, but I was now running into problems caused by the linear scan on the directory table in the ext2 filesystem.

The cyrus imap server is an interesting beast because it uses hash structures for indexes (excellent) and individual files for message content. This makes it much easier for the sever to perform operations on the mailbox. Deleting a message is a good example: for a spool file – the message must be removed, and then the whole file after the deleted message must be rewritten with the deleted message removed, which is a very expensive operation. For the cyrus server, this is implemented as a simple file delete.

Another imap server that can work in a similar manner to cyrus (and may be easier to administrate) is Dovecot. Dovecot allows it’s users to use either a Maildir (one file per message) structure, or a spool file structure. I haven’t actually worked with Dovecot so I don’t have too much to say about it other than to give it a mention. .

When I tackled the next reinstall, I installed ReiserFS. ReiserFS has some features around it’s handling of lots of very small files which make it very space efficient where lots of small files are used. It also uses B+Tree directory structures and doesn’t have an inode count which limits file count before disk space. I used Postfix instead of sendmail, which seems to be regarded as having better security than sendmail or qMail (there is contentious debate on this topic). I also continued my use of the Cyrus imap server. Cyrus’s one file per message store is particularly potent when coupled with a filesystem like Reiser. I can pick any message from the 11,000 messages in my inbox and view it close to instantaneously. Deletions and moves are similarly quick. I also have the whole system authenticating against an LDAP directory as well which makes the setup very manageable, although I haven’t taken any steps to automate management since I don’t have to do a lot of it. I’m confident this is now a very robust and scalable mail server, and it definitely outperforms any Exchange server I’ve had the pleasure of using.

NB: Wikipedia hosts a Comparison of filesystems, which is also worth a read.

String.format()

I ran across a bug in some code today where a developer had set up a template string similar to the following:

"{1}:{2}/{3}"

Subsequently they had used a series of replace commands to fill in the values.

String.replace("{1}", "foo");
String.replace("{2}", "bar");
String.replace("{3}", "baz");

It seems that this code is sub-optimal for two reasons. First, it’s not good from a performance point of view, and secondly, it could fail if the first string replace inserted {2} into the string. This would cause the second replace to do two substitutions which may or may not be desired.

I didn’t delve too deeply into the reasons for this, but the debugger showed this code as outputting a string which looked like: “foobarbaz{1}:{2}/{3}” which was the source of the bug. I went hunting for better solutions.

Knowing that Java 1.5 has had printf() functionality similar to C added to it, I wondered if there was an equivalent of sprintf(). It turns out that there is – it is a static method on String called String.format() (javadoc linked). This is a tremendously powerful tool in C, but the java version also allows more sophisticated options like pulling various components of date objects out. The full range of possibilities are documented here.

The new code looks much cleaner IMHO:

String.format("%s:%s/%s", "foo", "bar", "baz");

You should also note that printf() itself is now available on PrintWriter objects such as System.out.

AJAX in Java

Google have released their own Java API for doing AJAX stuff which compiles into javascript or something. This is some really slick stuff – I can’t wait for an opportunity to mess with it!

It Just Works.

Don’t get me wrong – I think that Apple and the Mac are an extremely good company with an extremely good product, in fact most of their products are pretty top notch.

The Apeture debacle is a mess though. Susan just got the copy she ordered. Turns out it doesn’t run on the 12 inch powerbook. Just the 15 and 17 inch. WTF? It’s certainly true that in the small print it does call this out, but this is Apple right? Doesn’t it just work? Apparently only on 60% of the Powerbook line from just a year ago. There’s even a hack which will make it work, so it’s not even a hard limitation – but having paid for the software, we expect some support, so it’s going back.

Maker Faire Pics

We had a great time at the Maker Faire looking at people’s projects. I took loads of photos. Particular favorites include the the Sterling Engine, and the burning man car with the pulsejets on it. I had no idea that it was a pulse-jet on the buzz bombs that were used in WWII.

macosxhints – A fix for a miscolored X11 cursor on Intel Macs

The only thing that doesn’t work right on my MacBook Pro is that sometimes the X11 mouse cusor is very hard to see. Here is the fix:

macosxhints – A fix for a miscolored X11 cursor on Intel Macs: “”