Jordi Romero2011-12-01T12:00:00-08:00jrom.nettag:jrom.net,2010:/Jordi Romerohttp://jrom.netAPI design and moretag:jrom.net,2011-12-01:/api-design-and-more<p>This is the description (and slides) from the talk I gave at the <a href='http://toster.ru/'>toster.ru</a> conference in Moscow.</p>
<p>The main point of the talk is to go through every major design decision around API design, so both API producers and consumers know what to offer or expect from a well designed <strong>REST API</strong>.</p>
<p>REST APIs are essential in any modern web application. It’s currently a trend not to build <em>Web Apps</em> but <em>Platforms</em>. A good REST API is the core of the platform, where web apps, mobile apps, 3rd party integrations, etc. can just use as its brain.</p>
<p>Some of the concepts the talk goes around are: HTTP, REST, URI, metadata, representations, security, versioning, pagination. These are all relevant aspects of a good REST API design. After that, some tips about <strong>implementation</strong> and <strong>deployment</strong> are thrown to the mix. Finally, <strong>scaling</strong> is mentioned as a key challenge a successful API will find. Horizontal scaling, HTTP cache, application cache, database replication and asynchrony are mentioned as possible methods to help the scaling of an API, although the design will still carry most of the responsability to decide if an API can scale easily or not.</p>
<p><a href='https://speakerd.s3.amazonaws.com/presentations/4ed88ddb013097005000379b/api-for-the-mobile-era.pdf'>Download the PDF</a> or see it embedded here:</p>
<script src='http://speakerdeck.com/embed/4ed88ddb013097005000379b.js' />2011-12-01T12:00:00-08:002011-12-01T12:00:00-08:00Teambox: Starting and learningtag:jrom.net,2011-11-17:/starting-and-learning<p>Slides from my talk at the <a href='http://bcndevcon.org/'>Barcelona Developer Conference 2011</a>.</p>
<script src='http://speakerdeck.com/embed/4ec50458e2f5020054007f5a.js' />2011-11-17T14:00:00-08:002011-11-17T14:00:00-08:00Stop being Paranoid, be Immortaltag:jrom.net,2011-01-16:/stop-being-paranoid-be-immortal<p>A while ago, <a href='https://github.com/saimonmoore'>Saimon</a> and I started upgrading <a href='http://teambox.com'>Teambox</a> to Rails 3. Once we got a bootable application (more on that in another post) the first big problem we faced was <code>acts_as_paranoid</code>. Its internals are a mess, so there was no chance we could just <em>fix</em> that for Rails 3. We decided we would just write a brand new gem that shared the API as much as possible.</p>
<p>The main goals were: implement almost the whole API from acts_as_paranoid (this way the impact of replacing it in Teambox would be minimal) and make it as small and lightweight as possible. We are true BDD believers, so we started writing a <a href='https://github.com/teambox/immortal/blob/29ffd7/spec/immortal_spec.rb'>spec</a> for what we needed. After that, writing the <a href='https://github.com/teambox/immortal/blob/29ffd7/lib/immortal.rb'>code</a> was quite easy.</p>
<p>In our way, we changed some assumptions from acts_as_paranoid, such as the need of storing <strong>when</strong> a row was deleted. Because of that, we no longer needed a <code>deleted_at</code> field but a boolean <code>deleted</code> field. This change demands a migration:</p>
<pre><code>class MigrateParanoidFields < ActiveRecord::Migration
def self.up
add_column :users, :deleted, :boolean
add_index :users, [:deleted]
User.update_all ["deleted = ?", true], "deleted_at IS NOT NULL"
remove_column :users, :deleted_at
end
def self.down
add_column :users, :deleted_at, :datetime
add_index :users, [:deleted_at]
User.update_all ["deleted_at = ?", Time.now], ["deleted = ?", true]
remove_column :users, :deleted
end
end</code></pre>
<p>While doing all this, at some point I regretted changing the <code>deleted</code> field to a boolean, because it looks like every relational database out there decides to implement (or not) their own boolean data type. This is where <a href='https://github.com/rails/arel'>Arel</a> comes handy, because you can just do something like this:</p>
<pre><code>default_scope where(arel_table[:deleted].eq(nil).or(arel_table[:deleted].eq(false)))</code></pre>
<p>Although there has been <a href='http://blog.semanticart.com/2009/10/13/killing-is-paranoid.html'>discussions</a> about the convenience of using <code>default_scope</code> for anything other than <code>order</code>, it just feels right to do it in this use case.</p>
<p>To sum up: the code is available at <a href='https://github.com/teambox/immortal'>github</a> or just as a <a href='https://rubygems.org/gems/immortal'>Ruby Gem</a>. Please feel free to contribute, <a href='https://github.com/unixcharles'>Charles</a> already <a href='https://github.com/teambox/immortal/commit/3bc92da646009425a45e205d0cc60e8eef205ea1'>did it</a>!</p>2011-01-16T13:00:00-08:002011-01-16T13:00:00-08:00I did it again!tag:jrom.net,2011-01-10:/i-did-it-again<p>I <a href='/i-like-coding-blogs'>said it once</a> already, but I’m going to talk about the same thing today. I enjoy rewriting or reimplementing blog engines. Most definitely more than writing blog posts.</p>
<p>The last time I did that, I kept the design and went from a <a href='https://github.com/jrom/jromrb'>sinatra app with database backend</a> to a <a href='https://github.com/jrom/jrom.net'>git-backed engine</a> (a fork of <a href='http://effectif.com/nesta'>Nesta</a>).</p>
<p>This time, i kept absolutely everything (design, html, feed, sitemap, about pages, etc.) but went to the static site generator <a href='https://github.com/mojombo/jekyll'>Jekyll</a>. The goal was to make this change transparent to the (possible) readers, search engines and feed subscribers would never notice the change, except for this post. In the last <em>migration</em> I moved all the blog posts to Markdown files, this was very nice, because I love formatting text using that markup language, and I still use it with Jekyll. The worst part were the haml and sass files. There’s no support for haml or sass on Github Pages (the generator I’m using to host this blog now), so I had to go back to plain CSS and HTML. This made me sad, but I definitely wanted everything in the same place and without using weird deployment strategies. Just git push for both posts and layout modifications. Github pages were the answer, so the move back made sense to me.</p>
<p>Other than that, I just had to replace some haml/erb/builder files with plain html/xml using <a href='http://www.liquidmarkup.org/'>Liquid</a> and make sure everything was smooth again.</p>
<p>Thanks to the awesome <a href='http://pages.github.com/'><strong>Github pages</strong></a> service, all I had to do is push to <a href='https://github.com/jrom/jrom.github.com'>github.com/jrom/jrom.github.com</a> and wait a couple of minutes.</p>
<p>Enjoy!</p>2011-01-10T01:00:00-08:002011-01-10T01:00:00-08:00Super simple authentication in Railstag:jrom.net,2010-05-07:/super-simple-authentication-in-rails<p>Here comes another short but (hopefully) useful Rails trick. In my client applications I often need to do some protected section to do backoffice stuff or similar. Sometimes I need a fully featured <strong>authentication system</strong>, then usually use <a href='http://github.com/binarylogic/authlogic'>Authlogic</a> (although I’m willing to try <a href='http://github.com/plataformatec/devise'>Devise</a>). Some other times you just need to hide a page to the public, and one password just works. And like everything in Rails, there’s a quick way of doing that.</p>
<p>All you need is some controller code to protect the private pages and some code to check the credentials. The easier way is not even creating any HTML code for that and take advantage of the HTTP Authentication. Here’s what I do in <code>app/controllers/application_controller.rb</code>:</p>
<pre><code>helper_method :logged_in?
def logged_in?
session[:login]
end
private
def authenticate
login = authenticate_or_request_with_http_basic do |username, password|
username == "username" && password == "password"
end
session[:login] = login
end
def do_logout
session[:login] = nil
end</code></pre>
<p>And now all it takes to force the user to authenticate for viewing some action, you just need to add a <code>before_filter</code> in that controller. It may look like this:</p>
<pre><code>class NewsController < ApplicationController
before_filter :authenticate, :except => [:index, :show]
# ...
end</code></pre>
<p>Here we are allowing public access to list and show a <em>New</em> but we are restricting all other actions (for example, creating, editing or destroying <em>News</em>) unless authenticated, but that’s getting <em>offtopic</em>.</p>
<p>As a small extra, there’s a helper method useful to show or hide special menus or something in the views, just put any <em>private</em> html inside an <code>if logged_in? ... end</code>.</p>
<p>That solution was really <em>quick</em>, but not very elegant. We don’t want our credentials hardcoded into a controller, and sure we don’t want our password in clear text! One better solution would be to use a <a href='/config-file-in-rails-apps'>config file</a> and hash the password, replacing the comparison line with this one:</p>
<pre><code>username == APP_CONFIG['username'] && Digest::SHA1.hexdigest(password) == APP_CONFIG['password']</code></pre>
<p>And of course storing this two variables in our config YAML file. To create the hashed version of the password for the config file the easiest way is by using the Rails console or just <code>irb</code>, and use the <code>hexdigest</code> method shown here. This will provide a quick but nice solution to our private sections.</p>2010-05-07T18:00:00-07:002010-05-07T18:00:00-07:00Gmail in Mutttag:jrom.net,2010-04-30:/gmail-in-mutt<p><img src='/files/gmail-in-mutt/mutts.png' alt='Screenshot of the Mutt mail client' /> <strong>Mutt</strong> is an open-source (<a href='http://www.gnu.org/licenses/gpl.html'>GPL</a>) text-based mail client for Unix. It may not be the best client if you receive a lot of messages formatted with HTML (most newsletters or <em>good</em> spam are), but it’s a nice program to know, because some day you may feel more hacker and want to use it.</p>
<p>To use Mutt, you must first install it. But that’s too easy: it’s on every <em><acronym title='FreeBSD's ports, Debian's dpkg, OS X's macports, homebrew, etc.'>package management system</acronym></em> after that weird name: <strong>mutt</strong>.</p>
<p>Before touching <strong>Mutt</strong>, you must enable Gmail’s IMAP for your account, going to <strong>Settings</strong> → <strong>Forwarding and POP/IMAP</strong> → <strong>Enable IMAP</strong>. <a href='/files/gmail-in-mutt/gmail-imaps.png'>Like this</a>.</p>
<p>After that comes the only tricky part: configuring Mutt to check your Gmail’s IMAP Inbox and send emails through Google’s SMTP server. And I already did the research on what’s needed to this. I’m using <a href='http://www.google.com/apps/'>Google Apps</a> for my personal domain, and it works exactly like a regular <code>@gmail.com</code> account, with the only difference of the username format. (Google Apps is absolutely a <em>must-have</em> for your domain unless you have some bigger solution).</p>
<p>You need to populate your <code>~/.muttrc</code> like this:</p>
<pre><code># Me
set from = "you@gmail_or_your_domain.com"
set realname = "Jordi Romero"
# My credentials
set smtp_url = "smtp://you@gmail_or_your_domain.com@smtp.gmail.com:587/"
set smtp_pass = "password"
set imap_user = "you@gmail_or_your_domain.com"
set imap_pass = "password"
# My mailboxes
set folder = "imaps://imap.gmail.com:993"
set spoolfile = "+INBOX"
set postponed = "+[Gmail]/Drafts"
# Where to put the stuff
set header_cache = "~/.mutt/cache/headers"
set message_cachedir = "~/.mutt/cache/bodies"
set certificate_file = "~/.mutt/certificates"
# Etc
set mail_check = 30
set move = no
set imap_keepalive = 900
set sort = threads
set editor = "vim"
# GnuPG bootstrap
# source ~/.mutt/gpg.rc</code></pre>
<p>Of course you’ll want to <code>chmod 700 ~/.muttrc</code> if you put that kind of sensitive data, or just don’t specify your passwords and will be prompted every time (for example in a non private environment). And you will also need to <code>mkdir -p ~/.mutt/cache</code> before firing mutt.</p>
<h3 id='using_mutt'>Using Mutt</h3>
<p>Now you can start it by typing <code>mutt</code> in a terminal. Read the <a href='http://www.mutt.org/doc/manual/manual-2.html'>Documentation</a> on how to move and use Mutt. The main concepts are similar to vim’s navigation shortcuts.</p>
<h3 id='encryption_with_gnupg'>Encryption with GnuPG</h3>
<p>One of the nice things about Mutt is that it has <strong>encryption</strong>/<strong>signing</strong> support out of the box. You just need to setup your keys in your environment and it will do the rest. If you already have your key in the ~/.gnupg/ directory, skip to the next point.</p>
<p>As before, you first need to install <strong>gnupg</strong> with your package management system. After that, you will need to:</p>
<pre><code># Generate your key
# You can use the default answers
# Choose a strong passphrase
gpg --gen-key
# Generate your Public key
gpg --armor --output pubkey.txt --export 'name'
# Register your public key in a public server
gpg --send-keys --keyserver hkp://subkeys.pgp.net</code></pre>
<p>And you are done. You can share your <code>pubkey.txt</code> with everybody and everybody will be able to send you encrypted mails or check your signed messages.</p>
<p>Besides that, to setup correctly the environment for Mutt, you will have to uncomment the last line of the <code>~/.muttrc</code> and copy the <a href='/files/gmail-in-mutt/gpg.rc'>gpg.rc</a> file to <code>~/.mutt/gpg.rc</code>.</p>
<p>Now you can fire <strong>Mutt</strong> again and encrypt or sign a message by pressing the <strong>p</strong> key just before sending (<strong>y</strong>). Decryption or signature verification is done behind the scenes.</p>
<h3 id='feedback_highly_appreciated'>Feedback highly appreciated</h3>
<p>I hope some of you will actually use Mutt and give back to me some tips! Until the comments are on, please use the <a href='/contact'>contact page</a>.</p>2010-04-30T18:00:00-07:002010-04-30T18:00:00-07:00Config file in Rails appstag:jrom.net,2010-04-28:/config-file-in-rails-apps<p>How many times do you find yourself hardcoding something that you now you’ll have to refactor and extract into a config file? But you don’t, because you don’t have that config file, and you don’t want to loose 10 minutes thinking (or the XXI’s century equivalent, googling) a solution. If you are nodding and feeling guilty, continue reading.</p>
<p>In an existent Rails application, just type</p>
<pre><code>rake rails:template LOCATION=http://github.com/jrom/rails-app.yml/raw/master/rails-app.yml.rb</code></pre>
<p>or if you are about to create the app, extend the rails command like this:</p>
<pre><code>rails <app> -m http://github.com/jrom/rails-app.yml/raw/master/rails-app.yml.rb</code></pre>
<p>That wasn’t hard, was it? The code that just ran is <strong>so</strong> simple that you can go <a href='http://github.com/jrom/rails-app.yml/blob/master/rails-app.yml.rb'>check it</a>. After asking you the name of that config (or assuming the default: <em>app</em>) it will create the <strong>YAML</strong> file under the <code>config</code> directory and create an initializer to load its content when the application starts. The config comes with a demo variable, defined in the three typical Rails environments: <em>production</em>, <em>development</em> and <em>test</em>. You can use <code>APP_CONFIG['variable']</code> (if the config name was app) anywhere and it will return the value for the current environment and that variable.</p>
<p>Enjoy it and use the <a href='http://github.com/jrom/rails-app.yml/issues'>issues</a> section in Github for any comment about the code.</p>
<p><strong>Bonus plus</strong>: it’s Rails 3 compatible!</p>2010-04-28T23:00:00-07:002010-04-28T23:00:00-07:00Page caching with Sinatra and nginxtag:jrom.net,2010-04-28:/page-caching-with-sinatra-and-nginx<p><img src='/files/page-caching-with-sinatra-and-nginx/sinatra.gif' alt='Sinatra' /> I’ve built some small websites with Sinatra lately (this blog itself). And in exactly all the cases I needed page caching, because the content would change once a month, a week, or a day. Page caching is <strong>easy</strong>, but sometimes you just need some tip. Let’s fix that by giving some simple and direct instructions in order to enable page caching and serving it from nginx:</p>
<p>First of all, we need <a href='http://github.com/kematzy/sinatra-cache' title='sinatra-cache on Github'>Sinatra::Cache</a>, the Sinatra extension that will make our life much nicer. You can either <code>gem install sinatra-cache</code> or add <code>gem "sinatra-cache", "0.3.2"</code> to your Gemfile.</p>
<p>After that, you will have to require and activate the extension in a <em>configure</em> block:</p>
<pre><code>require 'sinatra/cache'
module Jrom
class Application < Sinatra::Base
configure do
register(Sinatra::Cache)
set :root, File.dirname(__FILE__)
set :cache_enabled, true
set :cache_output_dir, Proc.new { File.join(root, 'public', 'cache') }
end
get '/' do
"Hi there"
end
end
end</code></pre>
<p>As you can see, first we <em>register</em> it, and then set a couple of config options. Actually, I wasn’t able to get it to work in a <em>classic style</em> app (when you just start coding <em>get</em>’s and <em>post</em>’s to the Ruby file, opposed to creating a subclass of <em>Sinatra::Base</em>), and that’s why I’m using the <em>subclass style</em>.</p>
<p>If this is the first time you see a Sinatra application using the <em>Sinatra::Base</em> thing, just remember to have a <code>config.ru</code> like this:</p>
<pre><code>require 'app'
run Jrom::Application</code></pre>
<p>Let’s continue, now we have our Sinatra app performing caching (by default it only does in production environment, check the <em>cache_environment</em> setting to change that if needed). A lot of people would stop here and go have some beers happy with an app that will be creating a cached page in <strong>every request</strong>, doing actually <strong>more work</strong> instead of less… That’s lame, but a lot of people does it.</p>
<p><img src='/files/page-caching-with-sinatra-and-nginx/nginx.gif' alt='nginx' /> The next but very important step is telling the web server (aka <strong>nginx</strong>) to serve the cached page if there is some. The following code must be added in the <code>server { ... }</code> block of the nginx config:</p>
<pre><code>if (-f $request_filename)
{
break;
}
if (-f $document_root/cache$request_filename)
{
rewrite (.*) /cache$1 break;
break;
}
if (-f $document_root/cache$request_uri.html)
{
rewrite (.*) /cache$1.html break;
break;
}
if (-f $document_root/cache$request_uri/index.html)
{
rewrite (.*) /cache$1/index.html break;
break;
}</code></pre>
<p>This code is derived from the one published by Josh Susser on this <a href='http://blog.hasmanythrough.com/2008/1/30/segregated-page-cache-storage'>blog post</a>. His code <em>as is</em> didn’t work for me, if you know why, please tell me.</p>
<p>Now our app is caching and enjoying the cached files nicely. But two small tweaks can still be done:</p>
<p>Add <code><%= cache_timestamp %></code> in your layout to print a comment with the timestamp when the page was cached, and add this <strong>Capistrano</strong> tasks to create the cache directory, link it to shared and flush the cached files when deploying:</p>
<pre><code>after "deploy:setup", "create_page_cache"
desc "Creates the cache dir"
task :create_page_cache, :roles => :app do
run "umask 02 && mkdir -p #{shared_path}/cache"
end
after "deploy:update_code","symlink_shared_dirs"
desc "Links the public/cache with the shared/cache"
task :symlink_shared_dirs, :roles => :app do
run "cd #{release_path} && ln -nfs {shared_path}/cache #{release_path}/public/cache"
end
set :flush_cache, true
task :keep_page_cache do
set :flush_cache, false
end
after "deploy:cleanup", "flush_page_cache"
desc "Empties the page cache"
task :flush_page_cache, :roles => :app do
if flush_cache
run "rm -rf #{shared_path}/cache/*"
end
end</code></pre>
<p>I think that’s enough to get a healthy Sinatra application performing page caching and flushing when deploying. You can also use the <code>cache_expire('/path')</code> to expire pages on some actions (e.g.. when an article is saved), but that’s out of this post’s subject, just read the doc of the extension.</p>
<p>I hope that was useful to somebody, and I encourage you to cache every possible page in your web applications. The performance boost will be incredible.</p>2010-04-28T01:00:00-07:002010-04-28T01:00:00-07:00I like coding blogstag:jrom.net,2010-04-27:/i-like-coding-blogs<p>As I <a href='/hi-again'>just mentioned</a>, I’ve been in a passive-aggressive relationship with my blogger alter ego. There are many reasons to that, but one is more interesting to me:</p>
<blockquote>
<p>I enjoy more writing blog engines than writing blog posts.</p>
</blockquote>
<p>I’ve been using <em>Wordpress</em> most of the time, for which I developed some themes (most of them lost, but I just keep the <a href='http://github.com/jrom/jrom09'>last</a> <a href='http://github.com/jrom/jrom10'>two</a>). Then, I’ve coded blog engines in PHP by myself (the source code is lost), using Rails (never finished) and Sinatra (first version using a database and coded from scratch <a href='http://github.com/jrom/jromrb'>here</a>). I’m sure I forgot some weekend projects done some years ago.</p>
<p>And now, this one. What’s special about this blog? Well, it’s the latest. And the prettiest. And is build as a <strong>Sinatra</strong> application using <strong>Git</strong> to store the posts. I discovered <strong><a href='http://effectif.com/nesta'>Nesta</a></strong> a couple of months ago thanks to the <a href='http://blog.peepcode.com/tutorials/2010/about-this-blog'>Peepcode blog</a>. I cloned it and built this blog on top of it. I took the look and feel I already designed in my latest Sinatra blog (the one with the database that never got to see the sunlight) and <em>voilà</em>.</p>
<p>Besides from loading all the content from static files (pushed via Git), this blog features an awesome <em>non-existent</em> admin section, some very nice page caching (using the neat <a href='http://github.com/kematzy/sinatra-cache'>sinatra-cache</a> extension by Kematzy) to prevent loading the Ruby stack, a <a href='/articles.xml'>feed</a> and a <a href='/sitemap.xml'>sitemap</a>. Oh, and the content is formatted using <a href='http://daringfireball.net/projects/markdown/'>Markdown</a> (almost everywhere) and <a href='http://haml-lang.com/'>Haml</a> (where the layout may be trickier). For the stylesheet generation, Haml’s sister is used: <a href='http://sass-lang.com/'>Sass</a>. There’s some code highlighting capabilities thanks to <a href='http://softwaremaniacs.org/soft/highlight/en/'>highlight.js</a>.</p>
<p>There is one feature I haven’t implemented yet, but will do soon: <strong>comments</strong>. This blog now is using no database to work, and everything is page-cached when the first request arrives, so a traditional comments functionality would fuck everything up a little… unless I use some service like Disqus or Itense debate. I don’t like these services (if there’s a better alternative, please <a href='/contact'>tell me</a>), so I will have to code some mini-disqus-like web-service to feed my own blog’s comments. I’ll keep you posted.</p>2010-04-27T23:00:00-07:002010-04-27T23:00:00-07:00Hi... againtag:jrom.net,2010-04-27:/hi-again<p>Hi, I’m <a href='/about'>Jordi Romero</a>. This is a blog, or it’s supposed to be, where I can publish anything I may find useful. The subjects will vary, from programming to computer architecture, or maybe some <em>sysadmin</em> trick that took me all night to find out and want to share to make the world a better place. Time will tell.</p>
<p>I’m trying to have and maintain a blog since <strong>2004</strong>! In 6 years I never published more than 20 posts, most of which never stayed online after some months. I started blogging in Catalan, my first language. Later on 2006 I deleted the blog and replaced it with a static <acronym title='Curriculum Vitae'>CV</acronym> in the front page (the first screenshot). One year later I started a <em>new</em> version of the blog (the right image) in Spanish, my second language. I published some useful Ruby on Rails tutorials, and an article about my <a href='http://en.wikipedia.org/wiki/File:Happy_Hacking_Keyboard_Professional_2.jpg' title='Happy Hacking Keyboard Professional 2'>keyboard</a> that got some attention. Sincerely, nothing worth saving or translating to this <em>newer</em> version. Some good and interesting content is out there, and I will write it down.</p>
<img class='center no-margin' src='/files/hi-again/jrom.jpg' alt='jrom.net when it was a CV in catalan' /><img class='center no-margin' src='/files/hi-again/jrom-2.jpg' alt='jrom.net when it was a blog in spanish' /><br /><br />
<p>If you just want to say something to me, make sure you check the <a href='/contact'>contact</a> page.</p>2010-04-27T22:00:00-07:002010-04-27T22:00:00-07:00