How to filter Rails requests by a given host

In this page we are going to see how to use the flexibility of request log analyzer to parse Rails requests which host is a specific one.

Request log analyzer is a very powerful tool thanks to both of its options:

• format: which let’s you define your or own format (it is, give a set of rules which will be applied by the parser).

• select: which let’s you indicate a condition to ignore or process a given line (the line will be processed if the condition is true). Exists, also, a reject option, which does the opposite to this one.

In our case, we want to parse Rails requests, so if we take a look to the Rails format file (which comes with r-l-a). A Rails request looks like this:

  Processing ReviewsController#index (for 0.0.0.0 at 2009-10-03 06:29:22) [GET]
    Parameters: {"action"=>"index", "controller"=>"reviews"}
  Rendering template within layouts/application
  Rendering site/home_not_logged
  Completed in 181ms (View: 99, DB: 9) | 200 OK [http://www.example.com/]

Our goal is to select only those requests from a given domain, let’s say www.example.com (this only makes sense when your application can run in different domains, or subdomains).

With the default Rails formatter we don’t have any variable or matcher that selects the host from the request, as can be seen in the regular expression that parses the completed line:

  RAILS_22_COMPLETED = /Completed in (\d+)ms \((?:View: (\d+), )?DB: (\d+)\) \| (\d\d\d).+\[(http.+)\]/

This line is parsed in this way:

  line.captures << { :name => :duration, :type => :duration, :unit => :msec } \
                << { :name => :view,     :type => :duration, :unit => :msec } \
                << { :name => :db,       :type => :duration, :unit => :msec } \
                << { :name => :status,   :type => :integer } \
                << { :name => :url,      :type => :string }  # 2.2 variant 

We can notice a variable named url, that will contain the complete url of the request. As this variable is not used in all the parses, we can hack it to change its function and that only match with the host of the request modifying this regular expression:

  RAILS_22_COMPLETED = /Completed in (\d+)ms \((?:View: (\d+), )?DB: (\d+)\) \| (\d\d\d).+\[http:\/\/([^\/]+)\/.+\]/

That modification should be included in a new format file, identical to the Rails one but with this small change (it would be great to have some inheritance here).

Now, we can use format and select options when we call request-log-analyzer from the command line:

  /usr/bin/request-log-analyzer production.log --format rails_host.rb --select url www.example.com

As you can see, there are a lot of possibilities to adapt request log analyzer to our requirements, but with a little bit of imagination sure you’ll get it.