Simple Rails APIs with Stitches

We don’t have a single monolithic application—we have lots of special purpose applications. Initially, there were just a few, managed by a few developers, and we used RubyGems to share logic between them. Now, we have over 33 developers, are a much bigger business, and have a lot more code in production. So, we’ve turned to HTTP services. Instead of detailing the virtues of this architecture, I want to talk about how we do this with Rails using a shared gem that’s less than 1,000 lines of code called stitches.

Rather than come up with our own solutions to these problems, we looked at Heroku’s HTTP API Design Guide. Their conventions seemed practical and reasonable, and had the virtue of being in use and vetted by an organization farther ahead on HTTP services than us.

We then went through and determined how to implement the guidelines in a Rails app, since we are using Rails. Fortunately, Rails embodies most of these guidelines by default, and we outlined what was important to us for all HTTP services.

Our Service Conventions

• Accept would be used for versioning (not URLs), e.g. Accept: application/json; version=2
• Security would be handled via the Authorization header, a custom realm, api keys, and SSL.
• Custom mime types didn’t seem worth it for internal apps (we’ll report later if that was a wise decision :)
• Properly use HTTP status codes, including Rails convention of using 422 for validation errors
• Namespace API resources under /api but otherwise follow Rails conventions (this allows us to serve non-API things from an API app if we needed it—which we do for resque-web and documentation).
• All exposed identifiers are UUIDs instead of monotonically increasing integer keys
• All timestamps are string-encoded using ISO 8601 in UTC.
• All error responses contain a structured error object
• All services must have human-readable, up-to-date documentation

Implicit in our conventions was that any Stitch Fix developer be able to understand the code of an HTTP service without a lot of backstory. This meant things like grape were out, since it requires an entirely new way of writing service code.

With this set of conventions, it was important that developers not feel these were optional features they could leave out to cut corners, so it seemed logical to make it as painless as possible to follow them. The result is stitches, which works as a generator and backing library. It’s not an engine or a DSL or anything complex. It’s just a bit of Rails configuration, designed to be explicit and obvious.

How Stitches Works

gem install stitches
rails g stitches:api
rake db:migrate

Now your Rails app has all of the above conventions set up! How?

Versioning and the Accept header

A stitches-powered app uses routing constraints to indicate which controllers handle which versions of a request. Our convention was to namespace controllers inside a module named for their version (e.g. V1 or V2), just so it’s clear where the code goes and how to route requests to the right place.

This excerpt for config/routes.rb describes the resource /payments that has two versions (both accessible via /api/payments):

namespace :api do
  scope module: :v1, constraints: Stitches::ApiVersionConstraint.new(1) do
    resource 'payments', only: [ :create, :index, :show ]
  end
  scope module: :v2, constraints: Stitches::ApiVersionConstraint.new(2) do
    resource 'payments', only: [ :create, :show ]
  end
end

Initially, stitches generates V1 for you, so while this may look like a lot of code, it’s not something you modify very often, so being explicit is actually preferable. Inside Api::V1::PaymentsController, you’ll just find really boring, vanilla Rails code. Code that anyone can understand, test, and modify.

There’s also a middleware configured by the generator that ensures no request that doesn’t use application/json with an explicit version gets through. This is an extension point to do more sophisticated things with mime types if we wanted to.

If there’s a topic more controversial than versioning, it’s security.

Security via the Authorization header

Per the RFC on HTTP Authentication, we decided that rather than a custom scheme, or a complex two-legged OAuth setup, we’d use API keys and a custom security realm inside the Authorization header. This is for internal server-to-server authentication only. It’s basically a shared secret, but since we are using SSL and both the client and server are trusted, this works.

Authorization: OurCustomScheme key=<<api_key>>

The ApiKey middleware (installed by the stitches generator) sets this up. Any request without this header, or with the wrong scheme, or with an unknown key gets a 401. The key is assumed to be in the Active Record class ApiClient via ApiClient.where(key: key). This is what the migration sets up for you.

If the request is good, the ApiClient instance is available in your controllers via env under a configurable key, which you can access thusly:

def create
  api_client = env[Stitches.configuration.env_var_to_hold_api_client]
  Payment.create!(payment_params.merge(api_client: api_client))
end

This is useful for attaching clients to data, so you know who created what. We make it available via current_user so it works with our logging and other shared code.

Versioning and auth are handled almost transparently, which means the Rails code is still clean and Rails-like. To use UUIDs and ISO8601 dates is similarly straightforward.

Data Serialization

Rather than require another library to serialize our objects, we’re generally fine with either to_json or using simple structs. All we need is to make sure our ids are UUIDs and encode the dates properly.

For UUIDs, we use Postgres, which supports the UUID type. You can use it instead of an int for a primary key like so:

create_table :addresses, id: :uuid do |t|
  t.string :name,               null: true
  t.string :company,            null: true
  t.string :address,            null: false
  t.string :city,               null: false
  t.column :state, 'char(2)',   null: false
  t.string :postcode, limit: 9, null: false
  t.column :created_at, 'timestamp with time zone not null'
end

This still exposes a primary key to the client, but since it’s a UUID, no one can read anything into it. This is the last you’ll have to deal with UUIDs. Getting dates working properly was a bit trickier.

In the end, we opted for monkey-patching ActiveSupport::TimeWithZone for a couple of reasons:

• No action required by users—dates just get formatted properly by default
• Our services will be small and self-contained, so will be unlikely to run up against issues where other code is assuming dates are JSON-i-fied in a different way

Error messages were a bit trickier.

Errors

It took some time to see the right way to deal with error messages, and it’s still not been a complete success. We wanted APIs to produce errors that could both be the basis for logic in the client, but also include information helpful to the programmer when understanding what went wrong on the server. We opted for a format like so:

[
  {
    "code": "not_found",
    "message": "No such user named 'Dave'"
  },
  {
    "code": "age_missing",
    "message": "Age is required"
  }
]

Basically, it’s an array of hashes that contain a code and a message. Client code can use code to write error handling logic, and message can go into a log or, in a desperate pinch, shown to a user. Note that this in conjunction with HTTP error messages, not in replacement of.

With this format seeming reasonable, we wanted an easy way to create it, as opposed to requiring everyone to remember it and make hashes in their controllers. The Errors class handles this. It can be constructed by giving it an array of Error objects (which is a simple immutable-struct around a code and message), or, more preferably, via one of its two factory methods: from_exception or from_active_record_object.

The thinking was that in a Rails app, you have two kinds of errors: validation errors from Active Record, and exceptions. Exceptions are easiest.

Exceptions

While you don’t want to use exceptions for flow control, you don’t want the user getting a 500 all the time either. You also don’t want to catch ActiveRecord::RecordNotFound in every controller just so you can create a 404. Instead, we assumed that each service would have a hierarchy of exceptions:

class BasePaymentError < StandardError
end

class NoCardOnFileError < BasePaymentError
end

class ProcessorDownError < BasePaymentError
end

In ApiController (the root of all api controllers in a stitches-based application), you can then use rescue_from on your root exception:

rescue_from BasePaymentError do |exception|
  render json: { errors: Stitches::Errors.from_exception(ex) }, status: 400
end

Stitches will look at the exception’s class to determine the code, so if your code throws NoCardOnFileError with the message “User 1234’s card is expired”, this will create an error like so:

[
  {
    "code": "no_card_on_file",
    "message": "User 1234's card is expired"
  }
]

As long as your service layer only ever throws exceptions that extend up to BasePaymentError, you get error objects more or less for free. Although the rescue_from is verbose, it’s not code you ever need to change, and it’s really explicit—anyone can see how it works. You can do the same for common errors like having ActiveRecord::RecordNotFound return a 404, so you can confidently call find(params[:id]) and never worry about dealing with the errors.

For ActiveRecord, it’s just as easy:

Active Record Errors

In your controller, you write pretty much vanilla Rails code:

person = Person.create(params)
if person.valid?
  render json: { person: person }, status: 201
else
  render json: { errors: Stitches::Errors.from_active_record_object(person)
end

What from_active_record_object will do is turn the ActiveRecord:Errors into a stitches-compliant errors hash. Suppose the person’s name is missing, and their age is invalid. You’d get this:

[
  {
    "code": "name_invalid",
    "message": "Name is required",
  },
  {
    "code": "age_invalid",
    "message": "Age must be positive"
  }
]

The code concatenates the field with the reason that field is invalid, and the message is ActiveRecord’s. It’s not perfect, but it’s good enough (callers should generally not be using services for validations, so calls like this technically shouldn’t be made).

Note that while Stitches::Errors.from_active_record_object(person) is verbose, it’s explicit and clear. Any developer can see that and look up the docs and know what to do. No DSL, no magic.

Which brings us to documentation & testing.

Documentation & Testing

Writing API documentation is pretty tricky, and it can go stale quickly if it’s written and maintained by hand. Ultimately, the api client and example code serve as the documentation for internal systems, but some real documentation is required. To solve that, we used rspec_api_documentation. It’s a fairly lightweight extension to RSpec that will both let you write and run acceptance tests, but also produce documentation in JSON that describes your API. Here’s a basic example:

resource "Payments" do
  get "/payments/:id" do
    let(:id) { FactoryGirl.create(:payment).id }
    example "GET" do
      do_request

      status.should == 200
      parsed = JSON.parse(response_body)
      expect(parsed["payment"]["id"]).to eq(id)
      # and so on
    end
  end
end

With the JSON documentation these tests output, we then use apitome to serve it up as HTML. It looks great, and shows the request and response, along with any relevant headers. You can add additional hand-written documentation as well, and it’s been great at keeping tests and docs up to date.

In Summary

Stitches is simple and explicit. It doesn’t solve every issue around services, but it helps quite a bit. I was surprised that rails-api didn’t have pretty much any of this, and I guess you could use that with stitches, but it didn’t seem worth it just to get ourselves going and try something. The main advantage of stitches is that it’s really not that much. It’s kinda boring. You just write some Rails code like normal, and call a few extra methods in your controllers every once in a while. But anyone can contribute to a stitches-powered app, and that lets us deliver value quickly and easily.