Wednesday, September 2, 2009

Forwarding Emails with Ruby, IMAP, and SMTP

I would hate to tell you just how much it sucked trying to figure this out.

Actually, I would LOVE to tell you. It would be totally cathartic to just unload a days worth of frustration. However, that wouldn't help you figure out how to do the one thing mentioned in the title of this post (which is probably the only reason you're here), and it sure as hell wouldn't make me seem like much of a professional.

So I'll skip the tirade except to say this: Maybe I should have given up 4 hours ago.

That out of the way, here was my problem. I have the email box where all these clients send data files to. I wanted to sort the ones that had attachments (and only ones with attachments) based on who they came from and forward them to different email addresses. There are all kinds of nice ways this could have been done if I wasn't constrained by some external requirements (like not being allowed to use forwarding rules on the server the emails were sitting on), so for the sake of this post assume this had to be done in ruby using net/imap to fetch the emails, and net/smtp to send them to wherever they needed to go.

This was not a fun place to start. These libraries work well, but I'll be honest when I say that MIME and IMAP are a little low level for my accumulated programming experience. So I started by using as a jumping off point a script I wrote for another blog post a while back when I was trying to access my gmail inbox with ruby. I got far enough to where I was polling the email inbox successfully, so now I just needed to figure out the "sending" part. No big deal, right?

I can't tell you how wrong you are.

You see, you can't just pass off a Net::IMAP::FetchData object into the net/smtp library and watch it fly off into the designated mailbox. NO SIR! You see, I had that idea, and this is what I tried:



require 'net/imap'

imap = Net::IMAP.new('mail.yourserver.org')
imap.login('username','password')
imap.select("INBOX")
imap.search(["SINCE", "8-Aug-2007"]).each do |id|
email = imap.fetch(id, "BODY[TEXT]")[0].attr["BODY[TEXT]"]
Net::SMTP.start('smtp.server.org') do |smtp|
smtp.sendmail(email, 'from',['to'])
end
end



It was not very successful. You see, my email client kept showing me the full MIME body, not just the visible components, and attachments were showing up as base64 encoded strings. This was not useful.

I decided the problem was most likely that I was just pulling incomplete MIME information from the imap library and that I needed to find the right attribute to ask for that would have the full MIME package in it's raw form. Enter "BODY[]". It might look very much like the "BODY[TEXT]" attribute I used above, but you would be deceived. "BODY[]" is the whole deal, everything that's there. If you want the raw message, that's what you need.

So that's what I tried:



require 'net/imap'

imap = Net::IMAP.new('mail.yourserver.org')
imap.login('username','password')
imap.select("INBOX")
imap.search(["SINCE", "8-Aug-2007"]).each do |id|
email = imap.fetch(id, "BODY[]")[0].attr["BODY[]"]
Net::SMTP.start('smtp.server.org') do |smtp|
smtp.sendmail(email, 'from',['to'])
end
end



Presto, it worked! No, I'm lying, same problem. So what to do now? Honestly, after the amount of reading I'd done just to arrive at the above conclusion, I was considering giving up. But I thought to myself: "This should be a solved problem. Someone out there should have already written something to take care of dealing with MIME formatting so that I DON'T HAVE TO THINK ABOUT IT!"

Enter rubymail. It looked very promising since the docs said it would parse and generate MIME format behind the scenes of a simple interface. "Great", I thought, "Let's try it":




require 'net/imap'
require 'rmail'

imap = Net::IMAP.new('mail.yourserver.org')
imap.login('username','password')
imap.select("INBOX")
imap.search(["SINCE", "8-Aug-2007"]).each do |id|
email = imap.fetch(id, "BODY[]")[0].attr["BODY[]"]
email = RMail::Parser.read(email).to_s
Net::SMTP.start('smtp.server.org') do |smtp|
smtp.sendmail(email, 'from',['to'])
end
end



This looked promising, but for an unknown reason RubyMail was parsing my emails wrong and making things that were part of the body out to be headers. It wasn't going well. Convinced I was on the right track with this whole "Let somebody else deal with the MIME" idea, though, I substituted "tmail" (another good ruby email library).



require 'net/imap'
require 'tmail'

imap = Net::IMAP.new('mail.yourserver.org')
imap.login('username','password')
imap.select("INBOX")
imap.search(["SINCE", "8-Aug-2007"]).each do |id|
email = imap.fetch(id, "BODY[]")[0].attr["BODY[]"]
email = TMail::Mail.parse(email).to_s
Net::SMTP.start('smtp.server.org') do |smtp|
smtp.sendmail(email, 'from',['to'])
end
end



there you are, scripted forwarding. May that help someone out there to not have a day like mine. :)

1 comment:

Hendrik said...

Thanks very much, it worked instantly. By the way - I like that you shortly describe your (painful) way to the solution. It helps understanding the problem...