At Jamglue we've always had the Apache rewrite rules set up to serve up our rails /public directory without hitting the mongrel servers, but we recently went one further and pushed all the static files there onto Amazon's S3 service.
As a result, most page loads hit our server only once, which is pretty remarkable. Everything else stays between the client and Amazon's data centers, which is fine with us.
Here's how you can do the same.
Pretty straightforward.
Technically all you need is some way of creating a bucket and sending files to rails, but if you're writing a ruby on rails app, you're probably pretty comfortable in ruby.
Personal preference, for instance: assets.example.com or static.example.com.
If you are unable to create a CNAME in your domain (get a real DNS provider, I love DNS Made Easy), you will need to choose <something>.s3.amazonaws.com instead.
I'll use assets.example.com for the rest of the steps.
You need to make an S3 bucket with your asset hostname as the key. I just used script/console, something like this:
>> require 'S3' >> AWS_ACCESS_KEY = '<your key>' >> AWS_SECRET_ACCESS_KEY = '<your key>' >> BUCKET_NAME = 'assets.example.com' >> conn = S3::AWSAuthConnection.new(AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, false) >> conn.create_bucket(BUCKET_NAME)
If you're using <whatever>.s3.amazonaws.com, the bucket name will be just the <whatever>, and needs to be all lowercase.
I like to just leave that connection open with BUCKET_NAME defined for ease of debugging. Remember to look at the response body if there are any errors.
What you need to do is create a CNAME for assets.example.com that points to s3.amazonaws.com. This allows us to use S3's Virtual Hosting of Buckets.
Skip this step if you're using <something>.s3.amazonaws.com.
You can skip this step if you like, but if you haven't used S3 before, it might be worth putting something up there and making sure it all works.
Choose an image in your public directory, and go back to the script/console where you've already created conn.
Pretending you chose public/images/menu.png, try this:
>> datafile = File.open('public/images/menu.png')
>> conn.put(BUCKET_NAME, 'images/menu.png', datafile.read,
{ "Content-Type" => 'image/png', "Content-Length" => File.size('public/images/menu.png').to_s,
"Content-Disposition"=> "inline;filename=menu.png",
"x-amz-acl" => "public-read" })
If that worked, you should be able to now go to http://assets.example.com/images/menu.png and see the image.
I wanted something easy that I could use to send new and updated files to S3, so I created script/s3commit. There are probably better (and more ruby on railsy) ways to set this kind of thing up.
My script/s3commit looks something like this:
#!/usr/bin/env ruby
require File.dirname(__FILE__) + '/../config/boot'
PUBLICDIR = File.expand_path("public", RAILS_ROOT)
require 'mime/types'
require 'S3'
AWS_ACCESS_KEY_ID = '<your key>'
AWS_SECRET_ACCESS_KEY = '<your key>'
BUCKET_NAME = 'assets.example.com'
MIME::Type.new('application/x-javascript') do |t|
t.extensions = ['js']
MIME::Types.add(t)
end
def upload_asset(path)
if File.directory?(path)
# go recursive
Dir.foreach(path) {|file|
if /^[^\.].*$/.match(file)
upload_asset("#{ path }/#{ file }")
end
}
else
# it's a file, check for validity and upload it
if /^#{ PUBLICDIR }\/(.+[^~])$/.match(path) && File.readable?(path)
key = Regexp.last_match[1]
mime = MIME::Types.type_for(key).to_s
if mime.length == 0
mime = 'text/plain'
end
puts "uploading #{ path } as #{ key } mime #{ mime }"
datafile = File.open(path)
conn = S3::AWSAuthConnection.new(AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, false)
conn.put(BUCKET_NAME, key, datafile.read,
{ "Content-Type" => mime, "Content-Length" => File.size(path).to_s,
"Content-Disposition"=> "inline;filename=#{File.basename(key).gsub(/\s/, '_')}",
"x-amz-acl" => "public-read" })
end
end
end
if ARGV.length > 0
# some files were selected
for arg in ARGV
upload_asset(File.expand_path(arg))
end
else
# no files were specified, try to upload everything public
upload_asset(PUBLICDIR)
end
Running script/s3commit by itself will re-upload everything reasonable in public to S3. Running it on one or more files or directories will upload only those, recursively.
If something like this would be better done with a rake or capistrano action or anything, I'd love to hear it.
This could be tied into SVN, but don't forget that you probably want it to happen as a sort of "update hook", not a commit hook. Otherwise if you commit (for example) new javascript and HTML that interacts, your users may see the new javascript before you get a chance to update production with the new HTML.
However you want to get the files in there, go ahead and push them now. If you use that s3commit script, just run it. For a path of the form RAILS_ROOT/public/<dir>/<file>, the S3 key should be just <dir>/<file>. More subdirectories are fine.
Test a few by going to http://assets.example.com/stylesheets/example.css, etc.
Rails makes this really easy. Just add (or uncomment) the following line in your config/environments/production.rb:
config.action_controller.asset_host = "http://assets.example.com"
And restart mongrel (or whichever you use).
Obviously you can also put it in development.rb if you want to test it there first, but eventually you probably want development running locally so you can change files without uploading them each time.
Now load some pages, and you'll see that all of your image_tag, image_path, javascript and stylesheet asset tag helpers are putting http://assets.example.com before each asset.
That's it! Make sure it works, you're done.
Of course, there are a few other little things.
Watch your Apache logs to see if any public/ files are being served up by your host. You may find a few places where somebody didn't use an AssetTagHelper.
Just think of it as equivalent to db/migrate. Develop on your dev box, and when you update your production environment, watch the list. Action in db/migrate means you have to run rake migrate, action in public means you have to run s3commit.
Your public directory will be up-to-date because that's where you're developing. If you keep your Apache config set up for static files, it's really easy to switch back.
So if S3 goes down or anything bad happens (the credit card you gave Amazon gets maxed out...), serving the assets locally is a one line change in your environments/production.rb config file.
Just make sure that if you put any flash or java files into S3, they know that all controller actions, etc will be on a different host and now require an absolute URL.
Feel free to write me, casey at jamglue.com
previous entry:
