Fiid’s Blog

Yesterday I finally managed to do my long solo cross country flight. It went pretty well. I was planning to fly to Los Banos, Castle, Modesto and then back. The wind at Los Banos was 12 gusting to 18 knots which is beyond my solo limitations; so I skipped that and flew direct to Castle. The handoff from NorCal to Castle went well; I’m getting better at spotting airports from the air; which is helpful :). The landing at Castle was uneventful; I taxied to transient parking, shutdown the plane and headed in for the bathroom. It’s a quick hop to Modesto from Castle, and then a longer flight back to Palo Alto. All in all - everything went smoothly and I’m pretty happy with how I was able to handle the flight.

Photos! (entire album is here)

Monterey Peninsula

Unusual Attitudes?

Castle Airport - Made it!

Mount Diablo from the South East

Altamont Raceway

I always wondered why the BMW sits slightly lower on one side than on the other. I thought it might be years of sitting lop sided because of the fact that the handbrake only grips on one side. It’s caused the fender liner to get all torn up as well (I thought this was because of me driving over those annoying concrete things in parking spaces….

It turns out there is an explanation….

The spring has snapped. Damned Previous Owners!!

The truly annoying thing is that I’ve had both the BMW dealership and a tire/alignment shop examine the car to look for exactly this kind of thing, and neither of them found it. I’ll be ordering springs today :).

After flying with Hans back in January - I quickly got to thinking about how cool it was to fly, and how accessible it was to do so. I had tinkered with flight simulators a lot when I was a kid, but I always just assumed it would be utterly unthinkable to actually consider flying a real airplane.
It turns out that learning to fly is not cheap, but it is doable. Since it takes a while to learn, the cost can also be spread over a few months. There are various numbers out there, but think about between $10k to $20k to obtain a private pilot license at the current time. I also live in the San Francisco Bay Area, which increases the cost some.

The first thing I did was to figure out which of my friends had any contact with the aviation community and find a good instructor. I had worked at Viant after Vance Cochrane was there. Vance recommended Dominique Marais as an instructor at West Valley Flying Club - so I took a discovery flight with her in an old Cessna 172. Since I wanted to broaden my horizons a little, I also took a discovery flight with Tim Stingle in a Diamond Katana. My flight with Tim was much more intense than the flight with Dominique, as he took more of a “drop you in the deep end” kind of approach, plus the Katana is really a very light and maneuverable plane. I like the variety of planes that West Valley has available, and I think Dominique’s teaching style works better for me - so that’s how that decision was made. In retrospect Diamond Aviation is no longer listing the Katana’s on their site, and I more often fly out of Palo Alto, so I’m very happy with my choice, and Dominique is a great (and patient) teacher and a fun person to be around.

The next issue I ran into was getting a TSA clearance to learn to fly. Since I was still a green card holder when I started my training I first had to get a new UK passport, and then apply to the alien flight student program. You don’t need this for your first “discovery flight” with a flight school, (you can also log your discovery flight, so get your logbook first), but for any subsequent flight training in the air, you need this. I kept my frustration to a minimum by doing a lot of ground study while this was going on.

My early training was thick and fast. The learning curve is steep at the beginning - and there are a lot of things to remember, and new things are introduced almost every lesson. This makes the process very rewarding, and the first time you execute a takeoff, or a steep turn, it is all memorable and fun.

After a few months of training, Dominique felt that I was ready to solo, and my confidence level also started to improve to the point where that was possible. I left it a little late getting my Medical certificate. This was a big mistake. The fact that I am asthmatic meant that I had to go to the hospital for a ~$2000 set of lung tests before I could get my medical certificate. Having done that, I’ll have to have the same batch of tests done again this year, since they only issued me a single year certificate. The delays and issues with getting this done meant that it took about 3 months from start to finish and was quite stressful and difficult to organise.

Having gotten my medical and preparing for the solo phase check; naturally all my skills fell apart in a ball of nervousness and anxiety; so some more time was then spent correcting this.

Finally, on November 2nd, 2007 I did my first solo. 3 laps of the pattern at Palo Alto airport. This was a huge moment for me and I enjoyed it immensely. I’d always messed around with flight simulators on the computer when I was a kid. I never even considered it would be possible to actually learn to fly. I also didn’t expect it to be one of the most rewarding things I’ve done.

Boing Boing carried this story today about ridiculously long lines to get through security at Heathrow. Of course, this story is replicated anywhere in the world where there is a flight to the US. It occurred to me that the line to get through to security has now become a bigger target for a terrorist than what’s on the other side… and you don’t have to go through security to get there. As Alanis says… “Isn’t it ironic? Don’t you think?”

On Tuesday, my good friend Hans took me on a flight from San Carlos (SQL) over to Half Moon Bay, up to the Golden Gate bridge, and then back down to SQL via SFO after circling over Alcatraz for a few minutes. The flight was awesome - and I got some great photos which are in the gallery. Thanks again, Hans!

This is my favorite picture from the trip:

My friend Joe Russack has started a really nice blog dealing with architecture, building design and implementation and furniture types of things: it’s at http://www.nothingugly.com. LOL - When was the last time you “implemented” a building :). Joe is working on a building put together with shipping containers.

It didn’t take very long at all to develop an initial version of my rails app - in fact - I’ve found rails to be immensely productive. Using yum and gem to install components also seems to be a big win. The whole development process was so ludicrously fast, in fact, that it caused me to be suspicious of what exactly I was giving up. It took me quite a while to figure it out; and in the end it came in the deployment stage.

It turns out that Rails is not thread safe. I was quite surprised to learn this, especially since ruby does in fact support threading; it’s just that Rails … doesn’t. This is a bit of a scalability issue, but it turns out that it’s not really a huge problem: I save SO much time in development, and what I was really trying to do was to throw an idea out and see if it sticks, which I am able to do. If it does stick and I need to scale it, I will gladly either hire some engineers to re-implement in Java, or buy some more servers, or probably both.

As for deployment, that isn’t fully sorted out in the same kind of way that it looks like it might be. My original thought was to go to FastCGI and plug it into Apache. Way back when (1997-1998 vintage) I worked with some large perl applications that used FastCGI, and it was always a nightmare with a bunch of processes spun up and some of them dying, turning into zombies and consuming large amounts of CPU. This experience still seems to be the state of the art, and the main reason why FastCGI is a bad idea. James Duncan Davidson has an excellent discussion on this topic on his blog. Further trips through the blog-o-sphere reveal yet more information, including the “pack-of-mongrels” approach which is detailed on Coda Hale’s blog. Thanks to both of these guys for very succinctly laying out solutions and issues on rails deployment.

The summary of this is that I have a cluster of mongrel servers running on my box, and I can deploy the application in an automated way (”cap deploy”) from my user account and have my latest code rolled out from development to production. The front-end server is apache, which seems to have grown load-balancing abilities in the form of mod_proxy_balancer. This is reasonably slick, and better than I would have had setup for a java app. Having said that - I would have been able to deploy my java app in tomcat, which is capable of handling a lot more traffic than my pair of mongrels. Of course, I don’t actually have any traffic, so much of this is mute anyway. All of this probably took about 4 hours of work or so (including some other minor deployment issues I ran into). Not bad at all.

I’ve been working on a small web side project lately. Being a mostly Java oriented developer - I wanted to build a java web-app. It’s been about 2 years since I worked on a web-app full time - at which time we were using Struts 1 point something and jstl.

I did a little digging around and found a couple of blog entries and documents that talk about web framework comparisons…

• http://www.oreillynet.com/onjava/blog/2006/06/isnt_rails_supposed_to_change.html
• http://raibledesigns.com/page/rd?entry=re_what_web_application_framework
• http://themindstorms.blogspot.com/2006/06/stripes-sounds-cool.html

Unfortunately, none of these resources yielded a clear winner. The one thing everyone does go on about consistently is Ruby on Rails. Nobody seems to be happy with Java Server Faces, although there seems to be hope for the future. The middle ground seems to be filled with Tapestry, Struts 2 (the artist formerly known as WebWork) and Spring MVC (step by step guide here). There also seems to be some interest in Stripes.

Given that the Spring folks seem to have a good, clean, simple approach to things, I thought I might go that way for my app. Having said that, I quickly learned that just about all the web frameworks now use spring as their back-end container. Glowing praise indeed. I invested a couple of hours in downloading and unpacking the latest jars for frameworks, jdbc drivers etc, before I had a change of heart. Since everyone has been going on about Rails for the past year or so, why don’t I give that a try??

My first look at rails was watching the videos on the Rails website. These do a pretty good job of demonstrating how to start doing something with Rails. This blog entry assisted with getting rails installed on my Fedora server, and before long I was rocking along.

It seems like rails is basically the “most applications work like this” pattern but with automation scripts. I didn’t have to write a build.xml file, or write a task for exporting the db schema; write something to make spring load or anything else. The first thing I did was build a db table, populate it, and then build a web page to interact with it. The time I did spend was on building useful functionality: making it call Google maps and get geographical data, and then making the Javascript event model work on the client… things that I regard to be complex (or at least - highly visable). Prior to “giving up” on java, all my time had been spent plumbing. This is not to say java is bad…. it’s just that the rails framework took care of all the easy stuff by itself.

It’s obvious to me that I will pay a price for the ease of startup later on, but if you wanted to build a web application to see if it “sticks”, you don’t particularly want to pay the startup tax - you want to see if it’s worth investing any real time on it. IMHO - this is where rails is killer… from idea to implementation is much quicker than it would be with Java; and this leaves more time for what is often the harder pain: figuring out the requirements and user interaction.

The learning for the java folks IMHO is that some application layout standards need to be agreed upon, and then a standard application skeleton (or skeleton builder) needs to be published. We need to get to a point where you can do something like scrip/generate controller and have something generate a new Controller, View and supporting configuration for you, so that less time is spent doing the same old grunt work. Hopefully it’s already out there and I just haven’t found it yet :).

I regard singletons as being perhaps the dirtiest of the GoF patterns. It’s problematic to assume there would only ever need to be one of something. As an example - the project I am currently working on has a client side application which has a series of singletons that facade server functionality. It turns out that this works very well, except there’s no way to renew these singletons if the user wants to log out of the server and log in again with a different user account. This means currently, the application has to be closed and reopened. This isn’t a show-stopper, but it is annoying.

Regardless of this, implementing thread safe, lazy initialized singletons in Java has always been problematic due some problems with the Java Memory Model. This is covered in detail in this excellent developerWorks article. I was mistakenly under the impression that this had been fixed in Java 5, but alas - it appears not. What I wasn’t aware of until today is that there is an alternative thread safe manner in which to implement a lazy-initialising singleton:
Initialize on demand holder classes.

I think the better approach nowadays however is to wire your application together using the Spring framework. Spring is very lightweight to implement, and will take care of initializing your singleton-like objects in order and wiring them together. Since spring offers full object life-cycle support, it’s easy to have it close all your objects out, and re-initialise with new ones, should you desire that.

In 1998 I worked for a company that had a contract to process bulk email. The emails were to attempt to get leads to sign up for a trial membership and then reminders for when the trial would expire and the like. This raised some interesting questions about how to send bulk email and get the server to perform. The “best” answer to the problem was quite interesting - qmail on IRIX.

It turns out that most mail systems end up storing information in files on the file system. As an example - sendmail - generally regarded as the standard Mail Transfer Agent (MTA) stores three files on disk for each message in it’s queue. If you are sending a lot of email, that queue gets very big very fast. Most filesystems at the time used a linear scan to find directory entries which means that the average lookup order is 3n where n is the number of items in the queue. Irix’s filesystem, XFS (which is also available on Linux) uses a B+Tree to store directory entries in O(log n), which is much more efficient.

When I came to reinstall my own mail server a few years later, my principal issue was not with mail spools, but with IMAP message stores. Previous to this point I had been using sendmail and the University of Washington IMAP server which used mail spool files. I had an inbox containing about 2000 messages at the time; and operations on that box were becoming prohibitively slow. I installed the Cyrus IMAP server to replace the UW IMAP server and found that my IMAP performance dramatically improved. I was able to maintain an inbox with around 5000 messages in it before things became problematic again. This was a significant improvement, but I was now running into problems caused by the linear scan on the directory table in the ext2 filesystem.

The cyrus imap server is an interesting beast because it uses hash structures for indexes (excellent) and individual files for message content. This makes it much easier for the sever to perform operations on the mailbox. Deleting a message is a good example: for a spool file - the message must be removed, and then the whole file after the deleted message must be rewritten with the deleted message removed, which is a very expensive operation. For the cyrus server, this is implemented as a simple file delete.

Another imap server that can work in a similar manner to cyrus (and may be easier to administrate) is Dovecot. Dovecot allows it’s users to use either a Maildir (one file per message) structure, or a spool file structure. I haven’t actually worked with Dovecot so I don’t have too much to say about it other than to give it a mention. :).

When I tackled the next reinstall, I installed ReiserFS. ReiserFS has some features around it’s handling of lots of very small files which make it very space efficient where lots of small files are used. It also uses B+Tree directory structures and doesn’t have an inode count which limits file count before disk space. I used Postfix instead of sendmail, which seems to be regarded as having better security than sendmail or qMail (there is contentious debate on this topic). I also continued my use of the Cyrus imap server. Cyrus’s one file per message store is particularly potent when coupled with a filesystem like Reiser. I can pick any message from the 11,000 messages in my inbox and view it close to instantaneously. Deletions and moves are similarly quick. I also have the whole system authenticating against an LDAP directory as well which makes the setup very manageable, although I haven’t taken any steps to automate management since I don’t have to do a lot of it. I’m confident this is now a very robust and scalable mail server, and it definitely outperforms any Exchange server I’ve had the pleasure of using.

NB: Wikipedia hosts a Comparison of filesystems, which is also worth a read.