Direct uploads to S3 with rails with delayed job

We have done many file uploading with paperclip and it has been very helpful in helping us to resize, compress, and reformat our attached files. However, we will face timeout issue when some users are trying to upload a large file or the internet is slow.
In order to solve timeout issue, we will need to implement the S3 direct upload. This is how we can implement.

Gems

  • s3_direct_upload
  • delayed_job
  • paperclip
  • aws-sdk

Flow

  1. When user submits a form with image, it will upload the image to S3 via s3_direct_upload without going through the rails app.
  2. The app will keep the temporarily attachment url from the s3_direct_upload callback.
  3. The app will submit the form and process the image via delayed_job.

Project Setup

  1. AWS S3 bucket setup for CORS configuration:
<?xml version="1.0" encoding="UTF-8"?>
<CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
  <CORSRule>
    <AllowedOrigin>https://mywebsite.com</AllowedOrigin>
    <AllowedOrigin>https://dev.mywebsite.com</AllowedOrigin>
    <AllowedMethod>PUT</AllowedMethod>
    <AllowedMethod>POST</AllowedMethod>
    <AllowedMethod>GET</AllowedMethod>
    <MaxAgeSeconds>3000</MaxAgeSeconds>
    <AllowedHeader>*</AllowedHeader>
  </CORSRule>
</CORSConfiguration>
  1. Add s3_direct_upload.rb to config/initializers
S3DirectUpload.config do |c|
  c.access_key_id = ENV['AWS_ACCESS_KEY_ID']
  c.secret_access_key = ENV['AWS_SECRET_ACCESS_KEY']
  c.bucket = ENV['AWS_DIRECT_BUCKET']
end
  1. S3 direct upload form
# Gist content from 
<%= s3_uploader_form callback_url: vessels_upload_s3_image_path(:type => "break-photograph"),  id: "s3-form" do %>
<%= file_field_tag :file, multiple: true, :class => "item-s3form-input", :accept => ".png,.jpg,.jpeg,.gif" %>
<button type="button" class="btn btn-primary add-images" data-dismiss="modal">Add Photograph of break Images</button>
<% end %>
  1. S3 form coffeescript initialize
$('#s3-form).S3Uploader
  click_submit_target: $(".add-images")
  1. The controller to process the image
@model.delay.process_images("S3_image_url")
  1. The model with delayed job method
# delayed job method on model
def process_images(urls)
  require 'open-uri'
  item = self
  image_urls   = urls.split(", ")

  image_urls.each do |url|
    file = URI.parse(url).open
    item.images.build(:cat => cat, :avatar => file)
  end
  item.save
end
	
# Image model setup for resize
has_attached_file :avatar, :styles => {
             :preview => ["150x150>",:jpg],
             :thumb => ["150x172#",:jpg],
             :large => ["100%", :jpg]
            },
           :storage => :s3,
           :bucket => ENV['AWS_BUCKET'],
           :s3_credentials => {
              :access_key_id => ENV['AWS_ACCESS_KEY_ID'],
              :secret_access_key => ENV['AWS_SECRET_ACCESS_KEY']
           }