Wednesday, September 24, 2008

Honing your Craft

In my spare time (what little of it there is), I like to teach music lessons. I have several students of different skill levels, and getting to experience the various stages of musical development with them is a kind of enjoyment that you don't often get when working with code.

Here's something I've noticed as I've worked with these kids: how much they're improving is more important then how good they currently are.

Example: I have one piano student who is quite good. She's obviously had quite a bit of instruction in the past, and so we get to talk about more advanced theory and work on more complex pieces of music than my other students. However, she doesn't dedicate a lot of time to practice during the week. Thus, she improves only a very little bit from week to week. As a result, I feel somewhat ineffective and our time together is so-so.

I have a second piano student who is mostly a beginner, but practices every day and really works on the goals we set during our lessons. I can't tell you how much I look forward to lessons with her; every time we get together she makes a new discovery about common chord progressions or patterns, and because of her dedication we get to work through a prodigious repertoire of music.

Now, which student would I rather work with long-term? I think the answer should be obvious, but I'll go ahead and state it anyway: the one who is improving rapidly, even though her skills are currently well below that of the other student, is still my pick.

I think the same thing applies to software developers. The people I like to talk with and work with in the development community are those who are learners, not "experts". Don't get me wrong, I don't have anything against you if you're good at something. What I DO have something against are people who have stopped honing their craft. I've run into plenty of them in the each of the software jobs I've had, and it's kind of frustrating to talk to somebody who is obviously quite intelligent, but has decided to place artificial limits on their growth by billing themselves as an expert in a particular area and therefore feeling that they need never learn anything ever again.

The reason this is relevant is that before long (maybe weeks, maybe months) we'll be hiring fresh software talent to come work on the startup I'm currently working on, and I know what kind of person I'll be looking for when that time comes. Do I want someone with chops? Sure, but I have twice as much desire for somebody with a desire and ability to be continuously improving themselves, without needing an external motivator.

I don't want somebody who's an expert in Ruby, I want someone who's an expert in becoming an expert at whatever-the-hell they need to at any given time. That's right, a META-expert.

Why settle for a professional when you could get a guru?

Friday, September 19, 2008

Measure Twice, Cut Once

I had a problem with one of my controller actions today. For the sake of this article, we'll it "long_action".

Now, as the name implies, "long_action" was really slow to respond every time I hit it with a browser; not just a little slow, we're talking 4 second response time. Knowing this was unacceptable, I rolled up my sleeves and switched into bug-hunt mode. This is when I made the big mistake, doing the one thing you should NEVER do when faced with a performance problem: I started looking through the code line by line to figure out what was LIKELY to be causing it. Maybe I'm not sorting that array efficiently, maybe I should have stored that method result in a local variable rather than calculating it on each iteration, maybe I should have cached that database call, etc, etc, etc. An hour later, I had optimized the bejeezus out of that thing, and you know what I had? 22 new lines of code between model and controller, decreasing readability by about 50%. Want to know how much speed I'd gained? less than 1/10th of a second. Clearly I had NOT correctly guessed the source of my problem. Lucky for me, I have the new-relic plugin installed in my app (I just hadn't thought about it until that exact moment). Firing up my browser, I navigated to the performance analysis page on my local server (a Really nice feature of NewRelic RPM, I highly recommend the service), and noticed that my problem had nothing to do with anything in the code I had optimized. I saw that a database query that was taking about 3 seconds to complete, and I couldn't see anywhere in my model or controller that I was actually making that call. Confused as hell, I opened up my view template, and there it was at the very top, embedded ruby caching a collection something like this:


<% @stds = Student.find(:all,:order=>"l_name ASC,f_name ASC"] -%>


WTF? Where did THAT come from? Here's this database call that loads an entire table, sorted by two columns, into an instance variable that NEVER GETS USED! All I can think of is that I must have been trying to debug a problem at some point and dropped it in there as a test (maybe of some dummy data or something, I have no idea). In any case, all I had to do was axe that line of code, and I was back to running a clean and speedy ship.

Now the lesson here should be obvious. When you run into a performance problem, it is arrogant and ultimately harmful for you to assume that you know where it's coming from. If you were a surgeon, wouldn't you make sure that you patient's appendix was REALLY what was causing their stomach pain BEFORE you removed it? Wouldn't you want your auto-mechanic to figure out exactly what was causing that strange sound in your engine BEFORE he replaces your transmission? All it takes is a moment to hook up a profiler, and you will know FOR SURE where your performance problems are. Don't sit there thrashing away at shadows when you could be done and on to more important things. Diagnose, THEN fix; always in that order.

Thursday, September 18, 2008

Developer's Best Friend: Hoptoad

If you are a Ruby-on-Rails developer (or merb, for that matter), then I have got great news for you. Hoptoad is here to provide you with free, INTELLIGENT error notification, and it's super-easy. I read about them this morning, and have already installed and tested their plugin with my rails app; my evaluation: 10/10. All you do is sign up for a free account with hoptoad. Then you have to install their plugin in your rails app. Here's where I ran into my only speed bump: I tried to use their recommended process of running the command "sudo ruby script/install plugin [url to project]" but it always failed saying it couldn't find this plugin. I don't know what I was doing wrong there, but it doesn't matter now. I just downloaded the latest version from their public repository on github, and dropped the folder into the plugins directory of my rails application. Then after adding the following configuration file [RAILS_ROOT/config/initializers/hoptoad.rb]:

HoptoadNotifier.configure do |config|
config.api_key = '[YOUR API KEY]'
end

I was up and running. I ran the included rake task to test the plugin:

> rake hoptoad:test

and immediately had a new error logged on my account on hoptoad. From there I can choose to have it send me emails for each error, or subscribe to the errors in my RSS reader. Piece of cake! The best part? I don't get a single email for each exception (No more inbox flooding). Hoptoad will count and collect similar errors, and will notify you of each PROBLEM, not of each individual exception. Great thinking! There are additional configuration options available if you want to filter which errors you get notified about, and there's also an API you can program against to subscribe to your errors in interesting ways or to post errors from non-rails projects, really it's open to anything you want to do. To the creators of Hoptoad: Great Job guys. To current Non-Users of Hoptoad: What are you waiting for?

Wednesday, September 17, 2008

CsvRunner on Github

This post will be quick. In order to get my feet wet with git, I have started a really simple rails plugin repository on Github (which, by the way, is probably the coolest website I've seen in a year). CsvRunner is my project; it's intended to be a way to avoid rewriting code to upload ActiveRecord entities from csv files over and over again. You can see the docs through the link in the last sentence if you want to see how it works. I'll be the first one to say, it's nothing special code-wise, but it does what it's supposed to do. If anyone out there likes to refactor in their spare time, I'd love to see what kinds of changes you would make. Then again, if you're just somebody who's looking for a library to do csv uploading for you, feel free to "git" a copy (it's a public repository) and enjoy.

Tuesday, September 16, 2008

Freeze those Rails

Just a tip, today, for any rails developer who hasn't done this yet:

FREEZE YOUR RAILS GEM!

Let me set up a scenario for you (a VERY realistic one, since I experienced it recently). You read about some new feature released in the latest version of Rails that you'd like to take advantage of. Using your trusty command-line, you issue:

sudo gem update rails --include-dependencies

After a minute or two of whirring, your computer informs you that it has updated to the latest stable release of the rails gem. After hacking away for an hour or so, you're happy with your utilization of the aforementioned new feature, and you deploy your app to your production environment:

MASSIVE EXPLOSION!



Why? I'm sure you've already guessed. You wrote your app against rails version x.y.z, and your production server has the gem for version a.b.c (NOTE: variables used for simplicity; not actual rails versions). What to do? Well, you could always keep every environment you ever put your app into on the same version of the rails gem all the time. Depending on the number of machines you have, that could mean spending half a day every time you want to test out a new edge feature. The better option (MUCH better) is to "Freeze" your rails gem. If you haven't heard the term before, all it really means is that you unpack the rails gem on your local machine into the "vendor" directory of your application (much like a plugin) thus your app always carries the version of rails it uses around with it. In this state, as long as you've got a compatible version of ruby installed on any machine that you deploy the app to, you're going to be ok. When you application goes through startup, it will favor a packaged ("frozen") version of rails over the one installed in the environment it's running in.

Now, you might be thinking this sounds complicated. Stay with me here, though, because it couldn't really be any easier. All you need is the following at the command line:


/path/to/my/rails/app> rake rails:freeze:gems


This command unpacks your rails gems into the vendor directory of your app. DONE! Commit and deploy, you're now flying with a frozen version of rails in your apps back pocket.

Thursday, September 11, 2008

UI Matters

I had a phone call today from a panicked customer. "The site isn't working!", he said, "I can't figure out why!". Naturally, I immediately started working the site through my browser trying to reproduce the problem. He was telling me that a text box that was supposed to be providing auto-complete results wasn't even allowing him to type any text into it. Clearly this was a serious problem, and I couldn't imagine what was causing it. I rehearsed the same steps over and over again: Arrive at the page, press the button, start typing into the popup, pick a result, make sure the popup disappears and the chosen result shows up in the readonly box next to the button ; On every operating system, with every browser, with every add-on or extension, and everything seemed to be working fine. Every time I typed a query in, I got a result, and I just couldn't find a way to break it. What could be causing this? What's their network security like? Are they blocking something necessary? Who knows!?

As my confusion deepened, I saw the business collapsing underneath me. If I can't solve this one little problem, then this website is going to be useless, and I'm going to be a failure. I had already spent several hours on this and it was looking hopeless.

And then it struck me. "I can't type into the textbox!" The page was structured with a readonly text box, and a buton to the right of it that triggered a popup, and the popup was what ran the search. He was trying to type into the readonly textbox, and had never pressed the button. I had spent the last three hours fighting against a perfectly functional app. Awesome.

The lesson here, I think, is that your UI design matters more than you think. Just by having that button moved to the left of the textbox, rather than the right, the whole flow has becomes more intuitive, and nobody will make that mistake again. I guess it's easy, as a programmer, to get caught up in slinging code and to recoil at the word "aesthetics", but if you want to compete on anything aside from sheer speed than how your UI looks and flows is of critical importance.

Tuesday, September 9, 2008

Customer Service is King

You know what I wish? I wish that awesome technology and beautiful code could absolutely make or break a business. I wish that a cool web application with nifty helpful features could win clients over by the droves. I wish my part was more important.

That isn't the way it is, though. You want to know what really makes people talk about you? What really drives that customer loyalty thing? See the following excerpts from my day:

ME to EngineYard (our hosting provider)

Ethan Vizitei - Research To Practice, LLC
Posted On: 09 Sep 2008 12:30 PM

Hello there,

I'm in the middle of a small emergency with my production site. A certain controller action is returning a server error consistently, and I cannot replicate it on my development machine, which makes me concerned that it may be an environment difference. . Any advice or guidance would be greatly appreciated.

~Ethan Vizitei

RESPONSE FROM ENGINEYARD (NOTE TIME STAMP)

Justin Pease
Posted On: 09 Sep 2008 12:34 PM

We will look into this and report back.
--

Justin Pease, Application Support Engineer


Solution Time
Ethan Vizitei - Research To Practice, LLC
Posted On: 09 Sep 2008 01:16 PM

I think I just figured it out. Embarrassingly enough, it appears I neglected to commit some code I abstracted to a plugin. My apologies for wasting your time, and thank you for the effort and quick response. If you have a collection of "Stupid things clients have done", you can feel free to add this to it. ; )

~Ethan Vizitei

EngineYard Response (note time stamp)

J. Ryan Sobol
Posted On: 09 Sep 2008 01:18 PM

No worries, Ethan. We *all* make mistakes. :)

Cheers!

-- J. Ryan Sobol


You know what I love about EngineYard? It isn't their awesome servers, it isn't their great business model, and it sure isn't their price. I love the fact that as soon as I ask for help, they're available and taking care of me. I'm always fanatically promoting them to my developer friends, because I know that they take customer service seriously and that I can count on them when I'm having problems.

So when I got an email today from a customer asking about something on the website, here's how the conversation went:


Ethan,

I'm having a bit of a problem. When a therapist tries to add a student to her caseload, the blue search window pops up, she can type the students' name in the field but cannot submit by pressing enter, return or any other command. I installed the latest Flash and Java but still nothing.

MY RESPONSE:

Mr. -----,

I have found what the problem is. The search query seems to only check against the last name of the student, or the first name of the student. It does not check them together as a combined entity. I will make that change and will email you when it is complete and tested.

~Ethan Vizitei
Research To Practice


10 minutes later, he had a new feature on the production site, and he sent me an email telling me how much he liked me and my company. Not because the website had great features, or because of how performant it was, but because when he had a problem we listened and solved it as fast as possible.

Simple recipe, amazing results.

Friday, September 5, 2008

Ruby Enumerable#Inject Gotcha

For those of you who are Ruby users, you're probably fans of the "inject" method for arrays. If you aren't familiar with this super-useful method, observe the following for enlightenment:


>> arr = [1,5,9,5,10]
=> [1, 5, 9, 5, 10]

>> arr.inject(0){|memo,int| memo += int}
=> 30


as you can see, the inject method allows you to pass in an accumulator value, which will be returned after iterating through the entire enumerable. Now for the unexpected error:


appointments.inject({}) do |map,appt|
key = appt.user_id
map[key] = [] unless map.keys.member?(key)
map[key].push(appt)
end


The above code (I thought) would inject a hash into the iteration and create a key for each user, thus splitting the array of appointments into a hash of appointments with their user_ids as the key. Here's what happened:

NoMethodError: undefined method `keys' for #<Array:0x231c138>

So what's going on here? The short version is that whatever you inject for your memo value is replaced by the result of the last statement of the iteration block (the same way that all methods return the result of the last statement in their body). In this block, the result of pushing an appointment onto an array returned the array, which then replaced my hash as the iteration accumulator. The fix is simply to make sure that your accumulator data structure is always the last statement of the block. Therefore, the above error can be fixed like this:

appointments.inject({}) do |map,appt|
key = appt.user_id
map[key] = [] unless map.keys.member?(key)
map[key].push(appt)
map
end

Securing Passwords for your Rails Application

EDIT: Please read the comments before using this code snippet, Matthew points out some weaknesses that should be fixed before using this code (as I have done in my own application)

This morning I ran across a blog post by Ola Bin, and I immediately realized that I haven't been doing enough thinking about security for my startup application. In response to the shot in the arm Ola's post gave me, I spent the morning encrypting the passwords for users of my application. Here's the new code snippet:


require 'openssl'

def User < ActiveRecord::Base

def digest(string)
OpenSSL::Digest::SHA1.hexdigest(string)
end

def self.find_by_auth(email,password)
password = User.new().digest(password)
self.find(:first,:conditions=>["email = ? and password = ?",email,password])
end

def password=(pass)
write_attribute(:password,digest(pass))
end

LETTERS=["A","c","Z","d","i","Q","v","X","k"]
NUMBERS=["1","2","3","4","5","6","7","8","9"]
def reset_password!
new_pass = random_password
self.password= new_pass
self.save!
return new_pass
end

def random_password
pass = LETTERS.rand
pass << LETTERS.rand
pass << NUMBERS.rand
pass << LETTERS.rand
pass << NUMBERS.rand
pass << LETTERS.rand
pass << LETTERS.rand
pass << LETTERS.rand
pass << NUMBERS.rand
pass << LETTERS.rand
return pass
end
end




There you have a simple fix that makes a big difference, security wise. Since it's a one-way encryption, even if the database is compromised an intruder shouldn't be able to figure out the user's password (theoretically).

Refactoring Ruby with Blocks

Blocks is an aspect of Ruby that not all participants I've met feel ready to participate in yet, so I'm giving out an example today that is intended to show how they aren't really that scary at all. There are plenty of bloggers out there who have covered what a block is, what the difference is between a block and a proc, etc, etc, so I'm just going to show how I used one to refactor some duplication in one of my Ruby classes for my startup.

Problem

Observer the following (simplified) code and note the problematic duplication:



require 'csv'

class Person < ActiveRecord::Base

def self.csv_import(file)
@file=CSV::Reader.parse(file)
messages =[]
@ file.each do |row|
s=self.new
s.first_name=row[0]
s.middle_initial=row[1]
s.last_name=row[2]
if s.save
messages.push("#{s.full_name} imported")
else
messages.push("#{s.full_name} invalid")
end
end
return messages
end

def self.csv_validate(file)
@file=CSV::Reader.parse(file)
errors =[]
@ file.each do |row|
s=self.new
s.first_name=row[0]
s.middle_initial=row[1]
s.last_name=row[2]
if !s.valid?
errors.push "#{s.full_name} #{s.errors.full_messages[0]}"
end
end
return errors
end

end



Here we have two different methods that do very similar things. One validates the contents of a CSV file, returning errors for the rows that are not valid (but doesn't touch the database), and the other imports and saves each row of the CSV file as a new record in the Persons table. Their functionality is common except for what they do in the middle of the csv row iteration block once the person has been constructed. So, how do we DRY this up? Blocks to the rescue! observe the following new method:


def self.csv_run(file)
@file=CSV::Reader.parse(file)
accumulator =[]
@ file.each do |row|
s=self.new
s.first_name=row[0]
s.middle_initial=row[1]
s.last_name=row[2]
yield accumulator,s
end
return accumulator
end


Notice that everything common to the two original methods has been moved into this one, and the last part of the algorithm where they differ has been replaced with the "yield" keyword. The result? When the csv_run method encounters the "yield" keyword it passes control to whatever block has been given to the method call, including any arguments you hand off with it. This means I can just pass in that differing segment of the algorithm as a block from each of my original methods. Thus, my class now looks like this:


require 'csv'

class Person < ActiveRecord::Base

def self.csv_import(file)
csv_run(file) do |msgs,p|
if p.save
msgs.push("#{p.full_name} imported")
else
msgs.push("#{p.full_name} invalid")
end
end
end

def self.csv_validate(file)
csv_run(file) do |errs,p|
if !p.valid?
errs.push "#{p.full_name} #{p.errors.full_messages[0]}"
end
end
end

def self.csv_run(file)
@file=CSV::Reader.parse(file)
accumulator =[]
@ file.each do |row|
s=self.new
s.first_name=row[0]
s.middle_initial=row[1]
s.last_name=row[2]
yield accumulator,s
end
return accumulator
end

end


And there you have it. Duplication reduced.