Roll your own API
January 22, 2007
Casey Muller

I'm very lazy.

Also, the web is a very open system.

As a result, I often find myself reverse engineering little things to simplify my life. I like to think of it as discovering unofficial APIs.

Ever since I installed the web developer toolbar in Firefox, it got a lot easier.

So I thought it'd be fun to document the steps I took in doing one of these. I'm going to use likebetter as an example. It's a great site, and I hope the Br*ans don't get mad at me (any publicity is good publicity guys). Also they're good engineers, so I know I won't encounter any frustrating bad design.
(Update: it turns out you can upload zip files, so this is all academic. I still want the ability to put in an RSS feed of my photos.)

I want to upload my photos and use their cool new widget to see which are the best, but I'm way too lazy to actually go through and push them all one at a time.

Enter wget.

Step one is to login. Bring up a login form, then use the toolbar's "view generated source" to find this nice standalone form, which submits back to its own url (login/login).

Perfect, here's what we do:

wget --save-cookies cookies.txt --post-data 'username=caseymrm&password=(mypass)' http://likebetter.com/login/login

Looking in cookies.txt, we see a nice Ruby on Rails _session_id cookie. For websites that use session cookies, add --keep-session-cookies.

Okay, next step is to locate the upload form. Likebetter has kindly provided an upload from URL option, which I'm going to use to avoid having to post files.

Again, bring up the upload dialog, and view the generated source. Loading the form doesn't bring up a nice standalone page this time, so here's a few relevant excerpts:

<form action="/user/upload_personal_channel_image" enctype="multipart/form-data" method="post" onsubmit="showobject('busy-upload-form');" target="image-upload-frame"> <input name="channel_id" value="(id here)" type="hidden"> from the web (URL): <input class="txt" id="upload_url" name="upload_url" onfocus="this.select()" size="37" value="http://" type="text"> <input name="commit" value="Upload" type="submit"> </form>

I start simple:

wget --load-cookies cookies.txt --post-data 'upload_url=http://bigfiles.null-terminated.com:81/digitalcamera/2004_07_17/mediumIMG_1033.JPG' http://likebetter.com/user/upload_personal_channel_image

500 server error. I keep adding things that look useful until I find one that works:

wget --load-cookies cookies.txt --post-data 'upload_url=http://bigfiles.null-terminated.com:81/digitalcamera/2004_07_17/mediumIMG_1033.JPG&commit=Upload&channel_id=(id)' http://likebetter.com/user/upload_personal_channel_image

Bingo. I browse to my likebetter profile and see the picture I chose made it there.

At this point I notice that likebetter also lets you zip up a bunch of files and do them all at once. Doh. Better update the top of this post.

Anyway, since I already have the wget command-line figured out, let's try it on the (few) 2007 photos I've taken:

find 2007* -name "medium*" -print | xargs -iimgurl wget --load-cookies cookies.txt --post-data 'upload_url=http://bigfiles.null-terminated.com:81/digitalcamera/imgurl&commit=Upload&channel_id=(id)' http://likebetter.com/user/upload_personal_channel_image

It works!

Now I'm gonna email likebetter and see if they're (1) okay with this article and (2) okay with me uploading a few thousand pictures using this technique.
(Update: they said yes, I'm doing it now! At one every couple seconds, it's gonna take a while)

Go see the upload in progress at my likebetter channel.

Here's the widget:

previous entry:

A use page, a gear page, and some targeted advertising