Friday, January 30, 2009

Variable Naming Matters!

I woke up this morning with a production bug on my plate. Those of you who read this blog regularly probably know that one of the startups I'm working on is a medicaid billing software suite for school districts. The problem I was having was that there were some transactions being added to our billing batches that shouldn't have been. See, in order to bill a transaction to medicaid, several things must be in place: the student must have a prescription, the therapy provided must be one of the federally approved therapies, the therapist must have a federally issued NPI number on file, etc.

These transactions were failing validation, obviously, because they were invalid. They should never have been billed in the first place. When I dug into my code, I couldn't see the problem, although based on the title of this post you will probably see it immediately:

def get_student_detail(s)
group = s.appts.select{|sa| s.is_medicaid_eligible}
if group.size > 0
claim = Claim.new(group,{:batch_id=>self.id})
end
end


I spent a lot of time on this. I wrote more unit tests, couldn't figure out why they were failing. I built mocks and tested many underlying methods to prove that they were working properly. Eventually the only way I figured it out was when I kicked up an "irb" session on my production box and started stepping through this code one piece at a time. what an embarrassment when I realized what I had done.

Yes, I went and named my "student" object instance "s". And then I named my "StudentAppointment" object instance "sa". And they both have a method called "is_medicaid_eligible" that returns a boolean value. So, if the student is eligible, all his appointments get billed regardless of whether those appointments are eligible or not, because I mistakenly wrote "s" instead of "sa" in the block on line 2 of the above code. There are plenty of arguments out there about naming conventions and whether they really matter, and I may not be qualified to participate in many of them, but here's one lesson I won't soon forget:

if I had used the name "student" for my student variable, and "appointment" for my appoitment variable, I probably wouldn't have spent an hour trying to find this bug.

Tuesday, January 27, 2009

Breaking my own Rules

OK, I'm going to post a link to somebody elses blog post. I almost never do this, because I usually feel like there's a comment section on the other person's blog for a reason, so if you have something to say you should "Say it to their face" so to speak. But in this one case, what I read made so much sense that I had to share it with my audience:

http://www.defmacro.org/ramblings/taming-perfectionism.html

Check it out, and note especially the very last sentence.

Saturday, January 24, 2009

Permanent bug fixes

One of the startups I'm working on right now involves a lot of SMS communication. Threading and queueing messages, although not difficult, can present some edge cases that need attention. One such case was brought to me this week when the application seemed to freeze and error out after a certain message type was sent, followed immediately by a cancellation of the same message.

When presented with a bug, my first instinct is always to "think through" the code that I believe is the most probable culprit. This is usually not the best approach. Yeah, that's an opinion, and I only have anecdotal evidence to support my viewpoint, but this is MY blog so if you don't want to hear my opinion you don't have to read it.

To me there are two important components of a GOOD bug fix. The first is natural: make sure the error/undesireable/unpredictable event stops happening. The second is too often overlooked: make sure it doesn't happen again. Because of the second point, and also because you don't always have a stack trace to show you where the problem is (if, for instance, the problem is not an "error" but an undesireable outcome), I think it's important to suppress the "dig in and find it" urge, and to instead start writing tests. Why? 3 reasons:

1) unit/functional tests allow you to precisely recreate the circumstances under which a bug occured (or under which you believe it to have occured). This means that if you don't yet know what's causing the problem, you can use assertions at every step along the way to make sure every bit of state maintained in the code is as you would expect. In the case of my SMS processing bug, there were many many objects involved in the transaction (because various notifications had to be delivered) and no error was being given so I had no way to know instantly where my data was being monkeyed with. If I wanted to do an "intuitive" diagnosis (as my insticts would have me do) I'd have been dealing with maybe 60 lines of code to look through. By setting up my test case and writing assertions at every point, I was able to isolate the place where the problem was occuring (where my first assertion failed) and thus the number of lines of code I had to apply my diagnosis-thought-process to was exactly one. It's always good to reduce the problem space anyway possible, it can only make your job easier. (As a bonus, if you're using test-coverage as a metric, then you're improving your numbers WHILE bug-fixing)

2) By using tests to diagnose the problem, you also have a feedback mechanism that will tell you as soon as it's fixed. Isn't that handy? When the test is passing, the bug is dead. You might write a few more tests afterwards if this bug has opened up a new path in the code that needs coverage, but basically you won't have to go through the process of "Let's try this, deploy it, did it help? no? Ok, what about this, let me deploy the website again. How bout now? No? Damn! OK, one second". You still might be following the same path of "try this, try that" but your feedback iteration is reduced significantly. Running your test suite takes a lot less time than redeploying the code over and over again.

3) Never experience the same bug twice. With that test in place (as long as your team is disciplined about running the test suite before deployments) that bug is dead for good. Were anyone to adjust the code to the point where it would reoccur (something I've seen happen on more than one project), the test will fail, stopping the release cycle until the problem is solved. You shouldn't ever have to solve the same problem more than once.

When it comes to pest control, your number one weapon should be an up-to-date suite of unit tests.

Monday, January 19, 2009

Advanced Reporting with Excel in Rails

This is a bit of an expansion and synthesis of two previous posts, one referring to Rails Reporting in Excel, and one dealing with Processing large amounts of ActiveRecord Data. I had to combine those two skills today as our database size is increasing to the point that some of our larger user reports are having to aggregate more data than is pleasant to store in memory at once. In the interest of providing a recipe that others could use, I've pasted an example of this approach on Gist. Essentially, it uses a pagination library (paginating_find, in this case) to page through result sets for reports, limiting the amount of memory space being taken at any one time. Check it out, leave a comment if it helps you.

Friday, January 16, 2009

Simplify, Simplify

I was presented with a challenge in my work today, one which I was not immediately ready to solve. One of the startups I am working on is a google maps mashup, and a certain requirement we'd decided on earlier involved having certain "markers" on the map blink so that the user's attention would be drawn to them. The following is a sort of "stream-of-conciousness" recording as I thought about making an image on my google map blink:

"OK here's my image:"



"I need to make it blink. It's part of a GMarker object (from the google maps API), so maybe there's some sort of animation API call I don't know about yet....Let's go check the Docs [navigate to google maps API docs].....OK, there's a hide method and a show method, those might be useful.....Not much else though.....I guess I could maintain an array of all the markers that need to blink, and use the "SetInterval" javascript function to toggle their visibility once a second.......yeah, that would work, but if I have a hundred or a thousand markers it might slow down a bit.....still, workable....maybe I'll google and see if anyone's done this kind of thing before....[google for "GMarker blink"].....now we're talking...cool, there's like three open source libraries that actually add a "blink" method to the GMarker class, that's handy....OK, so I just need to plugin one of these libraries and then call the "blink" method only on the ones I want to blink, seems pretty simple....It's going to add a whole new javascript file, but hey, that's not really much of a load time drag....."

By this point I had already constructed the code in my head to an extent, and I was ready to implement it. Then, as I was downloading the library, I noticed something. As I watched the progress bar on this third party website, I was mesmirized by the little circle-spinny-thingy....you know, it was using that usual symbol for "please wait"......the segmented circle that gets highlighted constantly around and around. I always thought that was a cool effect, and I remembered that I'd created that effect a couple weeks ago for one of my other projects. It was pretty neat putting this image together, layer by layer, timing out how long each layer should show for and then saving it as an animated gif.

Then I slapped my forehead, opened up GIMP, and produced this little baby:



Moral of the story, stop and think before you dive into something complicated, there's probably a better way.

Tuesday, January 13, 2009

Crunch Time

No matter the industry, or the development methodology, or the size of your company, or the scale of your project, everyone has experienced a time of "push".  Call it a hard deadline, or non-negotiable due date,  or whatever you want, it all means the same thing: you'll be working extra hard between now and then on little else.

Well, that's where I am right now.  We're already sending out marketing information regarding a new product that is theoretically ready but perpetually not quite there, and now the race is on to put the finishing touches together.  Stress is high, task list is endless, and I ask myself "Is being part of a startup worth it?"

Certainly there are people who would answer "no" to this question, and you couldn't fault them for it.  It is perfectly valid to say "I value my home life enough that I never want my work to encroach upon it", and if that were your point of view than working in a large industry where many people share the responsibility for the success or failure of a project would make perfect sense.

I can't quite go there, myself.  It's not that I don't value my home life, it's just that I feel I'm an all around happier person when the things I do at work feel like they have impact:  when I have a stake in the outcome, so to speak.  I would not suggest that this feeling can't be found in a large company, but I guarantee that when you are the sole technical muscle behind a small and rapidly growing project, EVERYTHING you do carries weight.   The cost is just that when it comes to crunch time, you endure it alone.

A price, yes, but one I pay gladly. 

Monday, January 5, 2009

Modifying ActiveRecord fields in place

So here's a "gotcha" for Rails that had me chasing my tail for a few minutes today.

Imagine that you have a model object that is storing a string that is encoded to represent a large amount of data in a small amount of space. For instance, let's say you have a calendar item that is using a string that is 31 characters long to represent the state of each day in the month (this is purely hypothetical, of course). Let's say you want to be able to block of segments of the month at a time, labeling them as "AVAILABLE" or "UNAVAILABLE". You might have a method on your model object that looks something like this:


def mark_days(day_1,day_2,status)
marker = (status == "AVAILABLE") ? "A" : "U"
while(day_1 <= day_2)
self.availability[day_1 - 1] = marker[0]
day_1 += 1
end
self.save!
end


Cool. Now you run your unit tests, and you notice a strange thing: they're failing. Adding some debug output, you can see that the model object is indeed having it's field updated correctly, and that all the validation is passing, and that the save method is indicating that it saved properly, but when you reload the object from the database nothing has changed!

"Bummer" you might say to yourself. I know I did. After some digging into the ActiveRecord documentation, though, I have found the solution, and I hope that in the future someone who needs it comes across this blog post.

When dealing with a string field on an AR model object, the model will not save that field unless it thinks it has been changed. Usually this is done by direct assignment. For example, if you have a person object named steve, and you want to change his name to fred, you would simply say:

person.name = "Fred"

When the name attribute is assigned to, AR knows that the field has been changed, and when the next save call comes the name will be persisted. However, if you will note my example above, I'm not reassigning the field, I'm just modifying parts of it in place. This causes active record to basically say "Since the reference hasn't changed, the data hasn't changed, and I don't need to save anything".

Never fear, though, as there's an easy way around it. If you want to modify a field "in-place", you must notify ActiveRecord of the incoming change. Observe this corrected code snippet:


def mark_days(day_1,day_2,status)
marker = (status == "AVAILABLE") ? "A" : "U"
self.availability_will_change!
while(day_1 <= day_2)
self.availability[day_1 - 1] = marker[0]
day_1 += 1
end
self.save!
end


By using the "[attribute name]_will_change!" method, you tell ActiveRecord that despite all of the usual indicators, this field has changed and must be saved next time the call comes in. Problem solved, and the tests are passing. May someone benefit from this tidbit of knowledge.

Saturday, January 3, 2009

Contract Work: Blackberry Mobile Development

So my startup is coming along nicely, but now I find myself in a unique situation: we have some work that needs to be done to build a thin blackberry app that does a few basic transactions with our website. Since my plate is getting a little full, I'm interested in contracting the work out, but I don't know anyone personally who's a great mobile developer. If that's you, and if you're looking for an opportunity to make some extra cash, email me at my personal address (ethan.vizitei@gmail.com) and let's talk shop. For the right person, this relationship could turn into a full time position eventually with an equity share.

Big pluses:

-previous experience contracting for small businesses
-worked on mobile apps before, specifically on the blackberry

I know you're out there, future friend. Looking forward to hearing from you.