Friday, February 29, 2008

Rails Rocks

A couple weeks ago I mentioned that I'm working on a new venture (in my spare time), and that I'm implementing it in Ruby on Rails. After probably 30 collective hours of development time, I'm ready to say that I am really glad I picked this framework.

For the uninitiated, let me quickly point out some of the super features of rails that boost it above other web stacks. There are plenty of great tutorials out there to show you how to do this stuff in code, so I'm going to stick to pointing out the high-level attractive bits.

Active Record

Although I'm usually a strong proponent of object databases, if you're going to use a relational database you can't do much better than the ActiveRecord framework, especially if you're starting from scratch. Rails has all the necessary CRUD methods built right into the objects you'll be using, so the Data Access layer is pre-written for you. I haven't written one SQL statement since I started, and haven't so much as looked at the database. I don't even have to build the tables or change their schema as the app evolves; all I do is specify in a "Migration" script any schema changes to accomodate new fields or whatnot, and Rails will make the necessary database changes for me via a "Rake" task (and it goes both ways. Need to rollback to a previous version? Rails will rollback your DB for you as well).

MVC Architecture

The conventions of rails encourage you very strongly to follow a design pattern that encourages a seperation of concerns. Model classes store data that is persistant, View templates (.rhtml files, basically html with embedded ruby) have all the display stuff in them, and the "Controller" classes take care of the direction each request goes and all the business logic that occurs in the interim. I think this controller piece is the greatest thing to ever happen to web applications. There's no url mapping involved, it's all conventional. Each "controller" class can specify any number of "actions" (all they are is public methods, there is no configuration involved. Every time you create another public method, it's treated as an action automagically). Each action, by default, renders a view template by the same name. So if I have a "main" controller class, with a "welcome" public method, then when I type into the browser:

http://www.mydomainname.com/main/welcome

The "welcome" method will run, and after it has setup any necessary data, it will by default look for a template in the "main" folder called "welcome.rhtml". Wow, this is slick.

Ruby

part of the reason the framework is so good is that it's built upon a stellar language. It's very expressive, and comfortable to use, and it's probably currently my favorite language. If you haven't tried ruby in the past, then you won't believe me when I tell you how much fun it is to work in, and you probably will think I'm just encouraging the current "fad". I won't try to deter you, let me just point you to a good tutorial and let you make your own opinions:

http://tryruby.hobix.com/

Testing

Ruby on Rails is a framework that is built pretty much exactly the way an agile developer would want it to be. Everytime you generate a model or controller class, the relevant testing stub is built for you. Just fill in some tests and ride! Rails uses three layers of testing: unit tests for model objects, functional tests for controller objects, and integration tests for each "path" through your website. I could give you all the reasons why this rocks so much, but again, you won't believe me. The functional tests are so easy to write, that when you get back to a middle level web framework (Struts, ASP.NET) you'll be suprised at how much you miss the easy coverage that rails provides.

I'm not going to harp on this too much, but if you have never at least written a trivial app to test rails out, you owe it to yourself as a developer to give yourself the experience.

(Post Script: I am not affiliated with the Rails development team in any way (unfortunately). I have no financial interest in the popularity of this framework. It really is that great.)

Monday, February 25, 2008

Systems Administration

This week I've spent a lot of time (a LOT of time) doing something that I'm frankly fairly new to: Server Administration. At every place I've worked I've always been a software developer, and there have always been people around for babysitting the servers. Therefore my world-view used to look something like this:

premise: It is my job to create new and cool things.
premise: In order for me to impress other people with those cool things, they must be hosted on a server.
premise: I'm too important to spend any time learning how to something as mundane as server administration.

conclusion: I should treat the server administrators nicely so the servers stay up but secretly pity them for having a boring and repetative job.

After this week, I'm ready to change my opinion: Server administration is a black art that it takes skill and intelligence to master. I'd better go out there and procure some skill and intelligence.

Since I'm working on a venture that basically involves less than 5 people, and since I'm the only technologist on board, it falls to me to be the expert on all things computer related, so it's been a tough week. However, I think I've probably come to a new level of understanding regarding my primary craft (software) by spending time working on the tasks necessary to build a server and get it running my application. If you're in the position that I was a month ago, there are some skills you might want to consider picking up in order to broaden your horizons.


get comfortable with the command line

If you're used to managing everything with a graphical interface, your first experience tinkering around with a rented dedicated server will be nothing less than excruciatingly painful. I speak from experience. I'm getting a lot better, but I wish now that I had done my first programming on a *nix box rather than in a windows environment with an insane amount of IDE support. Working this way will force you to become intimately familiar with the workings of your computer, and the pain is well worth the result. In fact, I think it's the pain of struggling through the process that makes it such a great learning experience. Nothing causes information to stick like really slaving away until your task is accomplished. I'd imagine that after constructing my first server environment this way, the next one would take about half the time.

read everything available about your webserver

Are you like me? Are you used to just starting the server, dropping in an application, and watching it run? That's what I've been doing with my local WEBrick server for development, and it's been great. Well, that's all well and good in a development environment, but when you are going to have more than one person hitting the box and when performance becomes an issue your requirements change dramatically. How do you make sure everything starts properly if the machine reboots unexpectedly? How do you use the http server (Apache, for me) as a load balancer for your app servers? How do you rewrite urls on the way in to direct traffic appropriately? Where are the error logs? Where are the configuration files? You should be able to answer all of those questions by rote.

learn how to script everything

Automate everything you can, you'll be happy you did later. Early in a project, it doesn't take too much to deploy and restart everything. Because it's not too much work, it's tempting to just do it manually everytime you need to get a new version out there. DO NOT GIVE IN!!! This is the easiest time to automate your deployment, when there are only a couple things to be done (copy the code, restart the server). Later (when you are deploying to multiple servers across a cluster, versioning each one accordingly, rolling the restarts to keep service uninterrupted, running jobs to send out notifications, etc) is NOT the time to be writing your deployment script for the first time. You want to let the script evolve WITH your process. Besides, everytime you do a deployment manually you run the risk of fat-fingering some crucial part of the process and causing a major headache. Write your scripts at the beginning, you'll thank yourself later.

Most importantly, don't be scared off just because you haven't done something before. The best way to learn how to do something is to fail at it a couple times, and the more effort it takes to accomplish something the greater the feeling of accomplishment when you finally watch your creation grow legs.

Friday, February 22, 2008

Customer Participation

One of the biggest time sinks during a software project is the transmission of requirements from user to developer. Often the impression is that this should be simple. After all, the person who is going to use this software wants it to be able to do something, and the developer wants to build something that will be useful to the user, so it should be a very easy exchange. Unfortunately, we know from experience that this is not the case; you can tell because most companies have all sorts of processes in place for getting the message of "what we want" from the users to the builders. Users have been bit before by software that is basically useless, so they have lobbied for "more communication" so that developers will build the right thing. On the other side of the fence developers have been frustrated in the past by a user who asks for one thing and then complains when the project is launched (after months of development) that the way it was built "isn't what they meant", so developers ask for a "more documentation" so that they have a written record of exactly what the customer asked for in the first place.


As a result, from the synthesis of these two cries for justice comes a most unholy beast: Requirements Gathering. Obviously the way to satisfy both camps is to have a process with lots of talking (communication) and lots of writing (documentation), that way everybody will be happy and we will operate more efficiently (does anybody see the problem here?)


Usually this enlightened process goes something like this:


1) Somebody decides they need software to make their life easier. They are now the customer/user (CU).


2) That person's manager does a very managerial thing by analyzing what the CU wants, throwing it all away, and deciding what the CU REALLY wants. They are the customer's manager (CM).


3) The CM's concept for the software is passed on to some sort of customer advocate (CA) for delivery (this is called "delegation"), because the CU who will use the software is too busy doing their actual job and the CM who has the authority to approve or reject the final product is too important. The CA doesn't really understand all the hand-wavy things the CM is talking about, but doesn't want to waste the important managers time by making him repeat himself, so they interpret everything into what they expect the user REALLY wants.


4) The CA presents the idea to a business analyst (BA) who grills them for hard and fast requirements. Most of the questions the BA asks are answered with "I'm not really sure about that, let me ask the CM and I'll get back to you". In the interest of expediency the BA uses their experience with other applications to fill in the gaps with what they know the user REALLY wants.


5) The BA delivers to a project manager (PM) a list of requirements. The PM spends days turning this into a "spec", which is a iron-clad contractual document that will be shoved in the users faces later when they realize that the software is nothing like they expected. This PM has loads of experience with other similar projects, so they have adapted the requirements to better represent what the user REALLY wants.


6)The PM takes the spec to the manager of the development team (DM), who reviews at length all the technical considerations that the spec has failed to address, thus (grounded as he is in the ever-changing world of technology) he adapts the "rough draft" into a spec that represents what the user REALLY wanted to begin with.


7) the DM holds a kickoff meeting where everyone is congratulated on what a fine job they have done so far, since everything has been thoroughly communicated and documented at length. The spec is handed to the Developers, and we're off to the races.


Wasn't that much better than the way we used to write software? ; )


Now, the above is certainly a bit of an exaggeration, but sometimes hyperbole can be effectively used to prove a point. Truly, who are the important parties in the above scenario? To me it seems that the only mission critical personnel are the builders, and the users. If someone does not fit into one of those two categories, if they are not either using the software or writing the code, they are not an essential part of making the project successful.


Of course we must take a step back and say that, pragmatically, there are sometimes too many balls in the air for the users and developers to balance, so in that case perhaps a level of indirection could come in handy, so I am loathe to rule the idea of these people's job functions out entirely. But too often I feel that this process, which was devised incrementally over time in reaction to prior unsuccessful projects, has just led to more unsuccessful projects which are exponentially more frustrating because there is more confusion about just who is to blame. In my opinion, the simple solution is to address the original problems with individuals, not processes.


To the Users:

Developers are not mind readers. When they build what you describe it will very likely not fit what you were expecting the first time. The key, I think, is to not wait for the release date before that "first time" comes around. Check out how the application works every week, and your expectations will grow and change with the project as you become better able to describe exactly what it is you're looking for.


To the Developers:

Users have a right to change their mind. It's to difficult to describe software in the abstract at the beginning of a project and to fore-see all possible necessary features from the beginning. Give them a chance to give frequent feedback (and make reasonable adjustments along the way) and you won't be hit with the two-by-four of disappointment on launch day.


To everyone else involved:

You probably have a valuable contribution that you are making to the project. Never-the-less, if you're having a discussion that may change the direction of the project and there isn't at least one user and at least one developer present, you're most likely wasting a whole lot of time.

Monday, February 18, 2008

Team Integration

I got an opportunity this week to get involved with the community Big Band in my city, and this evening is the first rehearsal I'll be attending, so tonight I'll be rejoining the musical community in Columbia, MO. I'll be honest, my nerves are a little on edge. I haven't played in a little while, so my skills are definitely rusty, and more importantly I'm walking into an ensemble where I don't really know anyone. These guys already have their pecking order established, already have their relationships in order, and I'm an unknown quantity, so there's going to be a little akwardness here at the beginning for sure.

The reason this is relevant is that the emotions I'm experiencing remind me almost exactly of the emotions I experience when walking into a new job. One way or another, you've got to integrate yourself into the team, and some people are very bad at this. I've seen veritable gurus ostracized because of an inability to correctly interact with the individuals around them. The two most important skills at any job are the talent you were hired for and the social skills you can bring to bear within your team; more so within the world of software where the average participant is opinionated, arrogant, and defensive all at the same time. Play your cards wrong, and you're liable to end up at the bottom of the pile regardless of your impressive talents.

For the sake of brevity (read, "to prevent a rant"), I'll divide the topic into two possible "first day" scenarios based on your relative skill level. The first (more common) one is when you are not the big fish in the room, and the second is when you are clearly from the beginning ahead of the pack.

middle of the road

For both the musician and the hacker, it is often frustrating to be in the company of an individual who is clearly more competent than you are. Most of us like to be the smartest/most talented person in the room, and if we aren't it's easy to spend our time sniping at whoever is #1, trying to find reasons to hate that guy. The most important idea you can keep in mind at this point is that you have been given a great opportunity. Whatever your skill level is at this moment, you now have the chance to quickly gain experience by intentionally studying and interacting with the people you know are ahead of you. I can remember two very solid experiences in my life that stick out as examples of this principal.

When I was in my first year of college, I was attending as a music major and had just graduated from a high school where I was "king-of-the-hill" in the music department, so to speak. I was used to performing at a level that outshone the ensembles I was playing in, and therefore had reason to believe that I was the most amazing individual to pick up an instrument since Mozart; the messiah of modern music, if you will. Imagine my surprise when I arrived at school to find that pretty much everybody in the music program was at my level and that there were several who were head and shoulders above me. Talk about a reality check. In my four years of high school I had never been so outranked as I was that year. And you know what else? In my four years of high school I had never grown so much as I did that year. I learned quickly who the best soloists were, and I attended every concert I could. I'd watch their body language, listen to their lines, trying to pick out licks into my memory that I could take and hone during my solo practice sessions later. I'd play any pickup gig with anyone who would let me in and I'd spend half the performance watching my group mates trying to commit to muscle memory the way they played off of each other. And by the end of the year, I'd become twice the musician I was when I entered. I now firmly believe that the best way to excel is to be running with people who are already ahead of you.

The second example that really sticks out to me is from my first year writing code. Through some fortunate chain of events (and thanks to a little charisma and a STRETCHED resume) I had landed a job at a major company who was going to pay me to do something I had basically never done before. I was way out of my league. From day one I was in the pond with people who had been hacking for 10+ years, and I was sure the I was screwed. It turns out that instead I rocketed forward and learned about as much as anyone possibly can in that amount of time. The shear exposure to people who were so far ahead of me made me BURN to catch up. I was reading books every night after work, writing test programs on every weekend, drilling every one of my coworkers with so many questions I'm sure they just wanted to throw me out. By the end of that year, I was leading new projects. That never would have happened if I had been working with people who were basically at my level.

So when presented with a situation where you simply are not the Chief, do not waste the opportunity by spending all your time resenting the guy who is. You have a chance to increase your learning rate by an order of magnitude, and that's something to be happy about.

leader of the pack

So what about when you walk onto a team and you know instinctively that you are already #1 and that everyone else is positively below you? My advice to you is to stop immediately before you think any farther and GROUND YOURSELF. I am particularly bad at this, so I'm not going to pretend this is easy, but here is the mantra I use: "Each one of these people is better than me at something.". It's important to keep this in mind because the temptation for you (if you're anything like me) will be to resent those people who's skills are not quite what yours are. You'll find yourself going into mental rants about how you're surrounded by idiots and how no one else is able to accomplish anything; and besides being inaccurate, those thoughts will destroy your will to work and your joy at your job. It may be true that you're being slowed down by the hackers you're working with, but far from being a dreadful weight you can find a great opportunity here as well: the opportunity to teach.

A wise man once said that if you cannot explain something so that a 12 year old can comprehend it, you yourself do not truly understand it. This is the essence of the message I want to convey here, because a variation of this statement is that explaining a difficult topic to someone else will help you understand it better. I do not have the same anecdotal revelations that I do for the first section of this article, but I can tell you from experience in a general way that explaining anything (music, code, lawnmowing) forces you to focus on fundamentals, which in turn causes you to be more aware of those fundamentals in your own performance. Even the virtuoso musicians play their scales, even the masters of ballet warm up with basic position exercises, and even the greatest hackers write "Hello, World!" apps from time to time.

Sometimes it's hard to bring everything together at the end, but I think the salient point of my ramblings here is that all interaction is an opportunity, so don't waste time worrying about your place in the world. the best mentors are also mentored, the best teachers are also taught, and on that note I'm going to go experience a new group of individuals who I will (hopefully) learn from.

Thursday, February 14, 2008

The Right Tools

You shouldn't cut down a tree with a fish, you shouldn't hammer nails with a shoe, and you certainly travel to Canada on roller-skates. The point here, of course, is that if you want to accomplish something the first thing you need to do is make sure you are using the right tools for the task at hand.

I've been spending a good portion of this week planning a new venture, and trying to determine the best technology to work in for the idea. After great deliberation, I did come to a conclusion that I'm happy with, and I thought it might be useful to share some of the paths I took in order to help others going through the same process. I won't go into any detail about the project itself, since none of you have signed NDA's ;). But here the basic requirements that I considered while trying to pick my tehcnology stack:

1) Speed of development is king. I don't want to spend too much time reinventing the wheel on any processes not directly related to the business idea, so I want as much of the boiler-plate stuff written ahead of time as possible.

2) This will be a web-application, and it will need to have a pretty interface that uses a lot of javascript.

3) I don't want to waste my time maintaining my own in-house servers, so I need to be working in something that I can have hosted remotely on someone elses servers.

4) There will be a relational database backing the application, so good DB interaction is a must.

5) Scalability is important. Should the idea take off, I need to be able to quickly scale to a multi-server environment.

6) I am not a very good user-interface-designer. I need something that will make it easy for me to integrate the designs of a professional UI guy when the time comes.

Based on these criteria, I began a survey of the technologies I'm failiar with that might could be used for the job.

Lisp

Yeah, I know, most people would hiss at the suggestion of Lisp for anything, but I believe there is potential. There are web-frameworks available for the language(UnCommon Web, and WebLocks for example), and it's very expressive so as far as Power goes you can't beat it. That being said, I am no Lisp guru, so it probably wouldn't be the fastest technology for me to develop in. There are plenty of hosting services out there that are Lisp-friendly, so that isn't a problem, but most of the websites I looked at were vauge as to whether they provided any useful support specifically for Lisp web applications. I also don't have any experience with non-trivial appliactions in Lisp, so I would be having to teach myself along the way. Also, I know nearly nothing about running a distributed application written in Lisp, so I'd have to do some research. My instinct says that it shouldn't be a problem since functional programming doesn't suffer many of the same side-effects that stateful applications do, and HTTP by nature is essentially stateless anyway so the paradigms should match ver closely. However, I just can't justify letting the business potentially fail just because I was not technically savvy enough with the language to pull off the things I need to. I unfortunately will have to dismiss lisp out of hand based purely on my ignorance.

ASP.NET

I use ASP.NET at work all the time, and in a way I like it. Designing all the interfaces with the drag-and-drop functionality is cool, and there's a large community using the technology so there's always support available if I hit a roadblock. There's no problem from an experience standpoint, since I've been using ASP.NET for a long time, but that's actually just served to impress on me that the development cycle is not as fast as other technologies I've used. All the database functionality seems to have a bias towards people who use the data as-is out of the database (as datasets and datarows), wheras I'm usually representing the data with entity objects and there's a lot of overhead that goes into maintaining that relationship with the database, even if you use a tool like NHibernate or Ibatis. Also there are a lot of errors I run into that have to do with maintaining state between server calls. Microsoft has done a good job of making the application appear as stateful as possible, so it's almost like you're developing a desktop app, and that's what often trips me up jarring me into remembering that not everything is autmatically persisted in the viewstate. As far as hosting goes, there are lots of solutions out there, but they're all a little more expensive than other technology solutions, and (of course) they have to be on a windows box on IIS (I would prefer to be hosting on linux).

Java

Of course we must consider the other leviathan out there. At my last job I did a lot of java web-development, so I've got some good experience in the arena, which means I know one thing off the bat: it's not that fast. Don't get me wrong, I've been satisfied with the solutions I've built with java, and I definately prefer JDBC to the .NET equivilent, but I've never been able to do anything as quickly as I have been able to in Ruby or PHP. There's just a lot of configuration that goes into getting a Java app up and running. Hosting isn't a problem, since there are plenty of reputable firms out there ready to help me, and there are frameworks available to make web development easy (JSF, echo2). All together, I'd probably choose Java if I couldn't find anything else that was quicker for me to develop in...

Ruby

..but there IS something that is quicker for me to develop in. I've used Ruby on Rails before, and I really liked it. As far as speed of development goes, the ActiveRecord framework makes DB integration pretty easy and the conventional approach built into the rails framework makes building functionality go quickly and without as much overhead as Java or .Net for each new page. Hosting options are fewer, but the folks at joyent.com have put on a good showing and have convinced me that their accelerator program is the way to go for startups like this one. Now RoR is known to be slower performance-wise than other languages, so there's one drawback. There could be some tuning involved when the time comes to scale, but 43Things is out there handling over a million page-views a day on Rails, so it can certainly be done.


OK, so maybe it was pretty obvious that I've got a slight bias towards Ruby, but there's a good reason: I've built webapps with all the above technologies and my most positive experience has been with RoR. I feel like it's got all the right pieces lined up ahead of time and is just waiting for you to apply your creativity towards building something cool. The things that often take hours of work (database integration, page-controller mappings) just work out of the box, and there are lots of libraries available to do the things that are common for web applications (accepting credit cards, managing user-supplied content, etc).

So we're off and riding Rails, more updates as they arise.

Tuesday, February 12, 2008

Internal Software

I've been thinking a lot lately about why I don't much like the software I'm writing at my job right now. I'm a software consultant working on internal business applications for a large insurance company, and although I'm better compensated than I ever have been before the work just isn't doing it for me. I feel like I'm doing a pretty good job (although I'm not exactly stretching my brain to the limit or anything), and I feel that what I'm building is going to be useful to the people who asked for it, but I just can't get excited about the project and I don't enjoy it as much as other software I've written in the past. After mulling this idea around for a while, I think I've hit upon the answer: it's internal software. It is, apparently by definition, not that much fun. I could be wrong about this, because prior to this contract, I've never written internal software before; everything has been a product that some external consumer will buy or at least use. So maybe I just pulled an unlucky card in this case when I got this contract, it's possible.

That being said, I strongy suspect that this is not the case: it occurs to me that the thing I find distasteful about the software I'm writing now is common to all software that is written in-house for standard business functions. "So what is that thing?", you ask. I'll tell you, I believe it's competition.

What is that intangible force that drives the development of a piece of product software? If your experience is anything like mine, it's the energizing (gnawing?) feeling that the other guy is building your features better. The competition, whoever that might be for you, is trying to build his software better than yours and if you slow down he'll do it and potentially get all your users in the process. In order to stay viable you've got to keep reinventing yourself. Not only that, but if you DO stay sharp and you DO keep up, your users will reward you by telling their friends about your product and MORE people will use it, giving you some amount of recognition as someone who made the world a slightly better place.

What about for internal software? Where is the motivation? Well, there isn't much. The software you're working on has been mandated as THE choice for the business by management, which means that even if after months of development you hand your users a brick with "workflow application" written in sharpie on the side, they bascially have to use it anyway. No one else will deliver a better solution if you don't build yours up to snuff; you will not be beat to market even if it takes you 6 years to get out your first release. In fact, as long as you make enough progress to meet your metrics and not get fired, you will still get that same paycheck that you'd get if your application managed to meet all the users wildest dreams for their needs and made coffee on the side.

For the sake of argument, let's say that you defy motivational gravity and write a really superb business application that solves all your customer's problems and is robust and speedy despite the lake of galvanizing factors. How many people will be using your software? Will there be hordes of users clamoring for the next version of your app? No, your user base will consist of about 30 people if it's written specifically for one department; maybe several hundred if it's company-wide. There's just not much to get excited about.

I guess my main point here is that this kind of software is just not a good fit for some people. If you're somebody looking for a safe job that won't take up a lot of your free time outside of work and who doesn't want too much pressure, it's probably perfect for you and more power to you. But if you're like me and you actually like writing code at home in your free time, let me give you a warning about working anywhere where the software is not the key product for the company: DON'T!

(DISCLAIMER: yes, Eclipse was written as an internal application, so not all internal apps are bleak and soul-less, but I dare say that's the exception and not the rule)

Monday, February 11, 2008

Elegance

Last night I saw a great show put on by Wynton Marsalis at Jesse Hall in Columbia, and I'm still kind of in a state of euphoria. My first career idea for myself was to be a professional jazz musician, and although I ended up pursuing other plans I still have a latent bit of hero worship for the greats, so there will be no greater experience for me in entertainment this year.

As I was listening in the darkness of the concert hall, watching these pro's keep an audience's rapt attention with the emotion they were pushing through those horns, I got to thinking about just what makes a virtuoso so universally impressive. Most of the audience probably had very little if any musical experience, so they weren't appreciating the show from the standpoint of a student of the art, but they were no less floored by what they were watching. It seems to me that there is a universal element that we share as humans, and that is an appreciation for the elegant. I'm convinced that what really made the show so amazing was not the incredible technical skill of each of the performers (although they were superb), but the way the artists sculpted the sound in a way that gave aesthetic shape and body to the music. There's a feeling that everything was just "right" or "in the correct place".

Now, given that this is a tech blog I'm going to have to find a way to tie this back into software. Obviously a developer is not a performing artist, and people aren't going to be held in rapt attention at his keystrokes, but there's an aspect of elegance common to all crafts that is easy to neglect in the world of code. We could certainly agree that there are some ways to express code that are better than others; whole books have been written on the subject and new design paradigms and languages come out in repsone to those evolutionary ideas all the time. What I'm prepared to suggest is that the primary driving force is a pursuance of the elegant. There are, of course, pragmatic reasons for the development of "better" code: increased productivity, more accurate abstraction, reduction of duplication, speed, etc. But I think that the elegance with which code is constructed tends to corrolate proportionately with these goals (well, perhaps not always speed, but certainly the other objectives).

So what makes code elegant? Well, as Eric Raymond says in the "Art of Unix Programming",

"Elegant code does much with little. Elegant code is not only correct but visibly, transparently correct. It does not merely communicate an algorithm to a computer, but also conveys insight and assurance to the mind of a human that reads it. By seeking elegance in our code, we build better code. Learning to write transparent code is a first, long step toward learning how to write elegant code — and taking care to make code discoverable helps us learn how to make it transparent. Elegant code is both transparent and discoverable."

This is, perhaps, a little "artsy" in it's description, providing little in the way of pragmatic implementation, but I think it holds true. So let's look at what practical features of code actually contribute to this abstract idea of elegance.

Elegant code is simple

When I'm sitting down to visually parse a new codebase, the single biggest contributor to my quick comprehension of the code is it's simplicity. It's generally an agreed upon truth that simple is better, but there are conflicting ideas of what "simple" is. The naive developer begins with the mistaken idea that "simple code" means "code that expresses all things only in ways that I am already familiar with". This tends to inhibit growth both for the individual and for the codebase since there is no driving force towards improvement. A mediocre developer might define "simple code" as "code that we keep the same way I wrote it the first time because there's no point in redoing work we've already done". I think this is downright destructive. Needless duplication and hacked patches will run rampant in a codebase maintained under this doctrine because unless you're willing to constantly be refining the old code to integrate in an abstract and consistent way with new features it will inevitably degrade into some kind of Frakenstine's monster. I think a better definition for "simple code" is "code where everything is done in only one place". No, I don't mean that all functionality is in one monolithic function; I mean that there are no cases where the same functionality is being provided by two different code segments. This may mean you touch code more than once during it's evolution, and it may mean that you have to do some serious research to find a better way to do something, but when somebody else comes in later and needs to make a change, they will thank you for building something beautiful that they can understand and manipulate with only a reasonable amount of overhead.

Elegant code is expressive

I used to scoff at naming conventions as a waste of time, and I am remorseful that I ever held that opinion. Names matter in the development of code; methods should have names that indicate correctly what they do, variables should have names that indicate what they are to be used for, operators should only be overloaded to do things that make sense, and classes should have names that indicate exactly what purpose they are intended to fill. Inexpressive names (variables named "X", methods named "go()", etc) are painful. It makes a persons mind spend needless cycles maintaining a lookup table in their head mapping things names to their purpose. Save your future coworkers brains from that atrocity and name things correctly. Also, the only thing worse than inexpressive names are expressive names that are lying to you. A method called "PrintStatus()" that actually overwrites and persists data is a problem, and I use that example because I've actually run into it before. If the purpose of something changes, change the name.

Elegant code is powerful

"Power" is another one of those things that is difficult to define within the realm of programming. Never-the-less, it's a key component of good code. The best expression I can think of to describe it is to say that "powerful" code accomplishes tricky but useful goals in a way that is not akward. Now that I look at that last sentance, I feel that I still haven't correctly distilled the essence of what I'm trying to convey, but it's hard to think of anything better. Maybe another way to put it would be to say that many of the things I will want to do with that code are already built in intuitively. Again, that doesn't quite cover it. Here's an example, I think: given a "string" class, some standard functionality might be the ability to find the index of a given character, to insert or remove characters anywhere along the string, and to be able to get the length of the string at a given time. All of this functionality could be implemented in a "string" class both simply and in an expressive way, and anything that I would need to do with a string I could accomplish with those operations. If that string class also comes with a "split" method, and a "reverse" method, and an "append" method, etc, then that code is also powerful.

Long story short, beauty is not restricted to the arts. Elegance is a feature of any skilled labour, and the pursuit of it (in my opinion) will always be rewarded.

Thursday, February 7, 2008

1000 unique visitors (or, "How To Trick Out Your Blog With Google Analytics")

Today is a big day for me as a writer. According to my account with google analytics, I have recieved my 1000th visit from around the world (815 of which are unique)!

This is also my chance to say that Google Analytics is probably the coolest tool in the world (no exaggeration). It's really easy to use, and so fun to check everyday that it is more addicting than facebook to a highschooler. Here's what I see everytime I log in:


...............

Ok, if blogger had been doing a good job of uploading images, you'd be seeing a really cool dashboard showing all the stats regarding my blog and the people who view it. Then I would have described all the cool features and given some major props to google for building a cool product. Unfortunately, Google also owns Blogger, which is sucking right now by not uploading my screenshots. Therefore, Google is currently sitting at 1 and 1 on my "props and vicious ranting criticisms" scale of justice, so I'm going to skip straight to the part where I tell you, the reader, how to use analytics in your blog, afterwhich I will look into wordpress for future blogging.

1. Make sure you have a google account. You must have one in order to utilize analytics.

2.Go to google.com/analytics and login with your google account.

3. At the next screen, click the "Sign Up" button to proceed.

4. Now you'll be asked for your website's name (if you're using blogger, it would obviously be "myblogname.blogspot.com", and the alias you want to refer to as when you login. Enter the info and click continue.

5. Fill out your contact information and click continue AGAIN.

6. Agree to the terms and conditions. Yes, you have to.

7. Now you'll get to choose which script piece to include in your blog. I recommend the "New Tracking Code". It's working great for me. Just copy the little script in it's entirety, go to the "Template" section of your blogger admin, choose "Edit HTML" to see your template HTML, and paste the script in right at the bottom.

8. Done! you are now tracking your blog traffic with Google Analytics. Well done. Be sure to check back often to see where your traffic is coming from and what topics that you write about generate the most interst. It's a blast.

Wednesday, February 6, 2008

My Proudest Moment

Sorry for the couple days of absence, I've been out with a bad case of the flu. I'm back though, and my first order of business will be to publically address a comment that was left on my last blog post. "ptn" writes:

Your aspirations sound really cool to me. What's the piece of code you are most of proud of?

Although I thought that would be an easy question to answer, it's actually taken quite a bit of thought to mentally dig through my past projects and determine which ones I really felt good about.

The first one that came to mind was my final project for my assembly language class in college. I wrote a "Pong" clone in assembly for the Motorola 68000 Family, and that was a blast. I got really into modding that thing once the basics were put together. I added speed control, difficulty levels, power-ups, and all the cool extras you see on modern block breakers. I definately learned a bunch from working on it, and I think that experience points out one of the key elements of a great piece of software: the developer has to feel some personal investment. I've seen a lot of code written by people who consider software a career rather than a craft. There's nothing wrong with that, of course, anymore then there's anything wrong with violins made on an assembly line..... Now, that may have offended your sensibilities if you're a violin player, as well it should. Code written by developers of the "it's a good vocation" mindset just tends to be less elegent in the source code, and less robust when deployed. Of course that's not a hard and fast rule, but I think it's held true in enough of my experiences for me to say it's always better to have developers who are craftsmen and who care personally about the quality of their output.

One of the other items that jumped out of my memory for consideration as "code I'm most proud of" was a project I did for a Sun certification. Yes, I have those silly certifications. I am an SCJP, an SCJD, and an SCWCD, woo-hoo. Looking back, I probably wouldn't recommend picking up these kinds of credentials to most serious hackers. I felt like I needed them at the time because I didn't have any experience to put on my resume for my first programming job and I felt like no one would take me seriously otherwise, but the truth of the matter is that if you write good code, other good hackers will recognize you for it, and mentor you. The only person those credentials are going to impress is an HR rep or a manager who isn't really technical, so while they can be useful for getting your foot in the door, I don't believe they hold too much weight in the real world. Anyway, enough on that tangent. The reason I was proud of that code was that it was kind of tricky. The assigment was to write an application that would allow a client-server application to access a proprietary format datafile and do basic CRUD operations in a threadsafe way by record-locking. The client-server communication was done via java RPC. Overall it was not really that complicated of an application, but it was the first one I'd ever written that I architechted and implemented 100% myself, and I had no prior experience with RPC or with file I/O (up to that point i'd basically been filling in the blanks for more experienced developers at work). But I went for it, and I passed. I did all the necessary research to learn the things I didn't know yet, and at the end of the project it was working the way it was supposed to. At some point, I think every coder needs that kind of experience to convince them that they don't need to abandon an idea just because they don't know off the top of their head how they're going to accomplish it. That's an easy pattern to fall into, especially in a corporate environment. A lack of experience in a given technology or language can be as much of a barrier to development as a lack of developers. Good hackers have the accurate notion that they can build an program to do anything they need it to given enough time and coffee.

But even though those two experiences were intensely valuable to me, the piece of code I'm most proud of is the login page for CARFAX.com. It's very simple, just a couple textboxes for the username and password, and some other content around the borders. In fact, it may be trivial in comparison to a lot of the other things I've written. But do you know what's important about it? People use it, and that's the ultimate measure of software value. That is the first piece of code that I ever wrote that people actually use everyday, so in a way it was my first step towards the hacker I am today. I could write tons of programs that do cool things as an intellectual exercise for myself, and they do have value in that they make me better at what I do and can even do things to make my life easier as a developer, but the software I am most proud of is that which is valuable to everyone.

Saturday, February 2, 2008

The Joy is in the Journey

Programming is probably my number 1 interest, and I like to think that I'm pretty good at it. Sometimes, though, I have experiences that put my expertise into perspective.

I had a conversation today with someone who's been coding a lot longer than I have, and it was a lot of fun. I was astounded at the breadth of this person's knowledge, really taken up in his descriptions of his past projects, and perhaps even a little jealous of his accomplishments. It's a humbling realization to come to, and probably also a universal truth: The more you know, the more you realize you have to learn.

There are whole fields of knowledge within the realm of programming alone that I have yet to really experience. I've never done anything with AI. I think game-programming sounds like a blast. I want to write my own compiler. I aspire to one day be working from a code-editor that I wrote. I want to become a master at algorithmic optimization.

Will I accomplish any of those things? Probably some of them, maybe even all of them. But I have to approach that fact with the knowledge that even when I've arrived at those destinations my aspirations will have multiplied. I think that people who don't comprehend that concept are destined to be perpetually unsatisfied, since they will never be at that place where they have conquered the computational world.

The key to being a good hacker (in my humble opinion) is to realize and appreciate your condition: The joy is in the journey.

Friday, February 1, 2008

Lisp-ing, Part 6: Iteration and Recursion

This is Part 6 of my tutorial series on lisp.
<<PART 5

One of the great things about programming is that with a few very simple instructions you can tell a computer to do something tedious a thousand times over and over again. Regardless of the language you choose to use, there basically two ways to specify this kind of action: iteration and recursion. My apologies to you seasoned veterans (and even you seasoned amaetuers) who already are aware of the distinction, but on the off chance that somebody reading this is NOT in the possession of such knowledge, here is the difference.

Iteration in programming involves using a control flow statement to specify a section of code that is to be run over and over again until a specific condition is achieved. As an example, here is how you might calculate the n-th fibbonacci number in java:

public static iterativeFib(int n){
int fib1 = 1;
int fib2 = 1;

for(int i = 2; i <= n; i++){
int temp = fib1;
fib1 = fib2;
fib2 = fib2 + temp;
}

return fib2;
}

a "for" loop is an iterative construct. Everything inside that block of code will be run over and over again until the "exit condition" (i <= n, in this case) no longer holds true. At that point, control will leave the for loop and will continue with the next statement below.

Recursion, on the other hand, means that a function calls itself over and over again until the base case is met, usually using the return value from each subsequent call to perform a calculation on and return as the PREVIOUS method call's return value. To illustrate, here is the same java function written above, except using recursion instead of iteration:

public static recursiveFib(int n)
{
if(n <= 2){
return 1;
}
else {
return recursiveFib(n-1) + recursiveFib(n-2);
}
}

And here you can see the beauty of a recursive function: elegance. The "recursiveFib" function only ever returns a "1" organically, but it can calculate any number in the sequence by combining lower and lower results of recursive calls.

So to bring focus back to the lisp programming language, what is the syntax for writing iterative and recursive functions in lisp? Well, let's start by looking at a translation of the same recursiveFib function written above:

(defun recursive-fib (n)
(if (<= n 2)
1
(+ (recursive-fib (- n 1))
(recursive-fib (- n 2)))))


Recursion seems to be more common in lisp than other languages. Why is that? Paul Graham speculates in "ANSI Common Lisp" that there are three main contributing causes.

First, that Functional Programming (the primary paradigm in lisp) depends on the return value of functions rather than changes in mutable state. Recursive functions (excluding "tail recursion") build the eventual return value of the top-level call by manipulating each subsequent recursive-call-return-value on the way back up the chain, thus they lend themselves very well to a functional paradigm.

Second, Lisp's primary data structure is a List, and if you remeber from Lisping, Part 2 a list itself is a recursive collection of pairs (car and cdr, first element and a list of all other elements). Thus, understanding how lisp works is itself an exercise in recursion.

Third, and this is probably more intuitive than statistical, Hackers who take the time to learn lisp generally care more about the elegance of a program's source code, and thus are more likely to take advantage of recursion when the performance penalty is not prohibative. You might ask whether a recursive function really does look more elegant than an interative function, or whether that's a subjective call. Well, I'll leave the distinction up to you. Here is a lisp translation of the iterative version of the fibonnacci function:

(defun iterative-fib (n)
(do ((i n (- i 1))
(fib1 1 (+ fib1 fib2))
(fib2 1 fib1))
((<= i 2) fib1)))


Firstly, for those who are unfamiliar with the "do" macro in Lisp, it is the primary tool for iterative processing. As you can see, it's most common form takes two arguments: A List of symbols(each one in the format "symbol name","initial value expression", and "update value expression"), and a list that contains as it's first element a predicate function that indicates "T" if the iteration is complete (the rest of the list is the body that is executed once iteration has finished).


Now, I'd be willing to contend that this is objectively less clear and clean than the recursive implementation above. So one might ask, why would you want to use iterative structures if recursion is so much better looking? Well, for one thing not all problems lend themselves well to recursion. A common example for a task that is better handled iteratively is the printing or processing of tabular data. Why is that? Well, when you're doing anything that essentially amounts to a "for each" statement on a non-nested collection, you are still doing the same thing over and over again, but the outcome of each single execution of the code block has no dependancy upon either the prior or subsequent execution cycle excepting the information that determines whether there is any more data left to process. Conceptually this is just not a recursive problem, although I suppose you could write a recursive function to do the same work (I'm advising you right now to not do that). Also, recursion can in some cases be more expensive performance-wise than iteration. The fibonacci problem is actually a good example of this. When you execute the recursive-fib function with a parameter of 7, the trace get's pretty long. recursive-fib for 7 must calculate the recursive fib for 6 and the recursive fib for 5, 6 must calculate 5 and 4, 5 must calculate 4 and 3, etc. Basically the same computations are being done more than once. It's actually pretty inefficient. In the iterative version, the number of calculations done is linear with regard to N.

Also, when picking a direction,the size of the dataset should be considered briefly. If there are 100 million iterations you have to process, than a recursive approach may be impossible due to space constraints. After all, each recursive call is another entry into the call stack, so if your data is larger than a certain amount you'll end up overflowing the limit of what the stack can hold.

Hmm...I can't really think of a good way to land the plane, so to speak. Here's an exercise, then. If you're interested in flexing your lisp muscles a bit, try writing two implementations of a simple function to determine the size of a list; one recursively, and one iteratively.

Happy coding.