Capybara HTML View Checking

I had wanted to do a decent job writing BDD tests for my views.  I was a bit surprised that the matchers were using the string form of the HTML.  I converted the string to a Capybara node in order to use the Capybara matchers.

HTML is tree-structured and my specs mirror  the HTML structure.  I check the parent node for correctness and then verify the children.  I had thought Capybara supported hierarchical specs using "within" but that doesn't seem to work for views.  It could be cockpit error on my part but Google searches didn't show any better way.

So I spent a little time adding to the RSpec DSL.  I wanted to have describe "scope into a subnode" of the current subject (the current subject being the equivalent of a DOM node).  I named the DSL "with_subnode".  With_subnode acts like describe with the ability to change the subject to a subnode.

The current subject is assumed to be a Capybara node.  with_subnode takes in the name of a Capybara::Node method (op:) to apply to the current subject plus any arguments (args:)

One starts by constructing a root_node using let( :root_node) do ... end.  This should be the string form as present in the rendered attribute.

The with_root_node method converts the root_node string into a Capybara::Node::Simple and makes it the subject of the do...end.

The example:

The code:

The code looks a bit odd. That has to do with the way RSpec builds tests. Each example/example_group is a meta-programmed class and one loses the lexical scoping. Each with_subnode clause is assigned a unique symbol. That symbol is used to retrieve the Capybara subject node from an associative array.

Unique symbols with an external associative array are used instead of let(). let() order evaluation along with inadvertent infinite recursion made that solution problematic.

The with_subnode clause is converted into two describe clauses. The inner describe clause is what the user placed in the do...end. The outer describe clause establishes the new subject.

An extra "it should_not be_nil" clause is added after the subject clause. This clause ensures the subject clause will be evaluated before the inner describe clause.

Controller Parameter Checking

Reference: rails/strong_parameters
Reference: active_record_validations.html#format
Reference: activemodel
Reference:

There are two main use cases.  The first to make sure that no extra parameters are accidentally passed into create/update operations.  This is to address any "mass assignment" vulnerability.

The second case is that RESTful controllers often have parameters not contained in the URL and hence not checked by the routing logic.  To validate those, we first construct a white-list:

nparams = params.permit( :my_param_a, :my_param_b, ...)

Note these are somewhat validated as being a basic type (see the reference above), but they aren't fully validated.

raw_parameters = { :my_param_a => "john@example.com", :my_param_b => "John", :admin => true } parameters = ActionController::Parameters.new(raw_parameters) puts "params: #{parameters}" cleaned = parameters.permit( :my_param_a,:my_param_b) puts "cleaned: #{cleaned}"

When executed:

params: {"my_param_a"=>"john@example.com", "my_param_b"=>"John", "admin"=>true} cleaned: {"my_param_a"=>"john@example.com", "my_param_b"=>"John"}

Oddly, input validation is not considered part of a controller; it seems to be model logic (see how-can-a-controller-manually-set-validation-errors-for-a-certain-field).  So we have to develop a Model that includes ActiveModel::Base in order to check the parameters.

Here is an example model which includes ActiveModel::Model

require 'active_record' require 'action_controller' class ParamCheck include ActiveModel::Model attr_accessor :my_param_a attr_accessor :my_param_b validates :my_param_a, presence: true, length: { minimum: 4 } validates :my_param_b, presence: true end

Some test code:

pc = ParamCheck.new puts pc.valid? pc.errors.full_messages.each do |err| puts err.inspect end

And the output:

false "My param a can't be blank" "My param a is too short (minimum is 4 characters)" "My param b can't be blank"

Using the cleaned parameters in the constructor:

pc2 = ParamCheck.new( cleaned) puts pc2.valid? pc2.errors.full_messages.each do |err| puts err.inspect end

Results in:

true

Adding checking to a controller

Each controller action is responsible for handling parameter white-list and required presence.

The idea is to define a nested class within the controller that is responsible for parameter checking. To avoid having a class per action, one ensures that named parameters are used consistently across all actions. For example, params[:email] should always be an e-mail address. Don't use names like :arg1 across multiple actions where it is an integer in one and an e-mail address in another.

The nested class then supplies validators for each parameter name. The validates should not check for presence as that was already handled by the action.

After the white-list and requires, one instantiates the CheckParams class using the cleaned-up parameters. One then checks to see if the instantiation is valid. If not, one produces a response with status 400 (bad request).

render :text => "400 Bad Request", :status => 400

Toggle Panel for Twitter Bootstrap

I had searched for a toggle panel widget and obtained one that sort-of worked from: where-are-collapsible-panels-in-this-framework

It was designed for JQuery and my toggle icon kept appearing on a line by itself.  I'm a relative JQuery newbie but old enough to know that writing highly chained JQuery code is someone's way of ensuring job security.

I refactored the code to use variables, added comments and substituted bootstrap icons for JQuery icons.  Here is the revised widget:

$.fn.togglepanels = function() { return this.each(function() { $(this).addClass("ui-accordion ui-accordion-icons ui-widget ui-helper-reset"); $h3 = $(this).find("h3"); $h3.addClass("ui-accordion-header ui-helper-reset ui-state-default ui-corner-top ui-corner-bottom"); // Add hover so this is apparent that it is a control $h3.hover(function() { $(this).toggleClass("ui-state-hover"); }); // Add the decorative icon $h3.prepend('<span class="toggle-panel-icon icon-plus-sign"></span>'); $h3.click(function() { $h3 = $(this); $h3.toggleClass("ui-accordion-header-active ui-state-active ui-state-default ui-corner-bottom"); $icons = $h3.children(".toggle-panel-icon"); $icons.toggleClass("icon-plus-sign icon-minus-sign"); // Stop any animations on content $content = $h3.next(); $content.stop(true, true); // Show/hide using a sliding animation $content.slideToggle(); return false; }); // Add styling classes to the content and initial hide it $content = $h3.next(); $content.addClass("ui-accordion-content ui-helper-reset ui-widget-content ui-corner-bottom"); $content.hide(); // Content is returned by default }); };

Note to self: SCSS and background images

I use background images in my .css.  When I deployed my app to Heroku, those images weren't showing up.

I used Google's developer tools and looked at the page's resources.  Those images are flagged as "not found".  Looking at the links, they were different-looking from my other precompuled assets.

Turned out that rather than url("image.png"), I needed url( image_path( "image.png")) in my .scss files.

Forms with Twiiter Bootstrap and Rails

I was using Rails's "form_for".  I had styled my form using 'form-horizontal'.  Everything looked good until I submitted a form with errors.  The form styling looked awful.

The solution can be found at the RailsApps project on GitHub.  I changed to SimpleForm using the directions provided and the presentation looks good.

Note to self: run rspec in project directory only

I noticed this when developing routing specs. Usually I run the command from the directory I'm working in. In this case, running "bundle exec rspec foo_routing_spec.rb" would fail. Running it in the parent directory as "bundle exec rspec spec/routing/foo_routing_spec.rb" works.

Templatized / Metaprogramming Testing

We often think of rules/specifications as applying to more than one object.  When I first began learning Ruby, I had followed Hartl's examples in writing tests and discovered that I was doing a lot of copy-and-paste.  That didn't feel right but I didn't know to do.  I wanted to do something like "foreach sign-in state { foreach page { verify page }}".

During the conversion of project to Cabybara 2.0, I noticed that descriptions could be tagged with additional hash key-value pairs.  They had something like:

describe "this test", :type => "feature" do

Interestingly, there is a note in rspec / rspec-core on metadata.  After some playing, I discovered a working design pattern.

  user_states = ["visiting","signed in", "admin"]

  user_states.each do |user_state|
    describe "for #{user_state} users", :user_state => user_state do
      let( :user_state) { example.metadata[:user_state] }

      let(:user) do
        user = nil
        case user_state
        when "signed in"
          user = FactoryGirl.create(:user)
        when "admin"
          user = FactoryGirl.create(:admin)
        end
        user
      end

      before do
        sign_in user unless user.nil?
      end

      all_static_page_names.each do |pg|
        describe pg, :page_name => pg do

          let(:page_name) { example.metadata[:page_name] }
          let(:pa) { PageAttributes.new( page_name) }

          before do
            # puts "          user state #{user_state}"
            pa.path = path_for_static_page( page_name)
            # puts "        pa: #{pa.inspect}"
            visit pa.path
          end

          it { should have_title( pa.title) }
          it { should_not have_title(full_title('Edit user')) }

          it { should have_link('Contact', href: contact_path) }
          it { should have_link('About', href: about_path) }
          
          if( user_state=="visiting") then
            it { should have_link('Sign in', href: signin_path) }
          else
            it { should have_link('Sign out', href: signout_path) }
          end

        end
      end

    end

  end

I have a few helper routines in my utilities. One provides the array of page names, another returns the page attributes for a page. The last returns the path to a page given a page name. I had to keep the path one separate as when I tried to include in the PageAttribute constructor, something was wrong and I couldn't access the *_path variables.

Getting my project to work with Capybara 2.0

My project was originally designed following Michael Hartl's excellent tutorial.  I was using a 1.x verison of Capybara and only had "cockpit error" types of problems - my own fault.

When I moved to Rails 4.0, my tests stopped working.  So I upgraded to Capybara 2.1 and dug in.  I must say I was surprised at the nature of the changes between 1.x and 2.1.

Some symptoms:

• Accessors like "root_path" stopped working
• visit stopped working

In general, you will need to modify your tests.

• the requests directory should be named features
• require 'spec_helper' needs to be require_relative '../spec_helper'
• I haven't figured out how to adjust the search path to allow the original construct to work.
• The outermost "describe" blocks should be "features"

I discovered that features don't nest, nor do scenarios.  See the references below.  In order to make 'visit' work, I turn the outermost 'describe' block into a feature block.

I tried using scenarios for inner blocks.  This was very painful and I went back to describe for inner blocks.

Reference links

• How to fix “Undefined method ‘visit’” with Capybara 2.0
• rspec-rails and capybara 2.0: what you need to know
• Capybara 2.0 Upgrade Guide
• Introducing Capybara 2.1
• End-to-end testing....
• Capybara project on GitHub
• Stack overflow on nested featues
• secret_key_base not defined

An incomplete list of what I changed

Added to spec_helper:

require 'capybara/rspec' require 'capybara/rails' ... config.include Rails.application.routes.url_helpers

I had several checks for title that needed to be updated. They were

it { should have_selector('title', text: 'Sign up') }

Now:

it { should have_title('Sign up') }

An update to this post will follow as i organize my notes.

Note to Self: Rails Generate Scaffold

I learned about Rails by working through Michael Hartl's tutorial.  Generating scaffolding is not something that I do every day or even every week, so I rely on my notes.

I recently shifted to Rails 4.0 and constructed a scaffold per Hartl's tutorial.  I added the pages to my so I could use the generated MVC.  I discovered that while I could create objects, the contents/attributes were blank.

I traced it back and discovered that my attributes were not added to the model as "attr_accessible".  Added those and the MVC now works.

Rubygems, Heroku, Upgrading to Rails 4.0

School of hard knocks

In a previous post, I had mentioned that I hosting my gem(s) on RubyForge. Well, that didn't work out for me as couldn't get "git" to push to the repository. I've managed to get Heroku working, starting with a project on GitHub. But no luck with RubyForge. I had opened a support ticket some time ago & no response. So I will be moving my public projects to RubyGems.
Rubygems has some tooling and documentation for Gem building. I installed the tooling and my development environment broke. Yes, likely my fault as I was lazy and ignored all of the "wax-on/wax-off" about setting up a Ruby environment.
Lesson #1: Use rvm (Ruby Version Manager). This tool installs ruby to your own private area so you don't mess up your system installation. Particularly good advice if you are on a Macintosh. For me, I'm running Fedora and I had to do the standalone install of rvm. It worked, although I had to properly modify the search path. rvm is truly private to you.
Lesson #2: Install gems locally, not into your system. When things are looking very confused and your development environment is messed up, I find it useful to delete my '~/.gem' directory and use bundle install to rebuild it.
Lesson #3: Use gem version numbers in your Gemfile. Things aren't as stable as one would like, so pinning things down is helpful.

Upgrading to Rails 4.0

I had followed this post: http://blog.barbershoplabs.com/blog/2013/02/27/upgrading-from-rails-32-to-rails-40

The biggest culprit was in my config/application.rb file.   require "active_resource/railtie" doesn't work and I just commented it out. Rake now works and I'm keeping my fingers crossed.

I'm now just cleaning up after the new set of error messages.  Those seem obvious.

Update:
I generated a new Rails 4.0 project. I wrote a small ruby script to compare and modify my existing project based upon the new reference projects. It added missing directories and copied over any missing ruby files. For ruby files that existed in both and were different, I had the script invoke the merge tool "meld". The deltas weren't all that bad to merge.

#! /usr/bin/env ruby

require 'fileutils'

ref_dir = "reference/"
src_dir = "target_project/"

# okay, let's start with directories

l4 = Dir[ "#{ref_dir}**/**/**/**" ]

# puts l4

l4.each do |n|
  result =  File.stat( n)
  if File.stat(n).directory? then

    # Take off the front part
    p = n[(ref_dir.length..-1)]
    # puts "subpath #{p}"

    d = File.join( src_dir, p)
    if !File.directory?( d) then
      puts "Directory #{d} does not exist, creating it"
      `mkdir -p #{d}`
    end

    next
  end

end


l4.each do |n|
  result =  File.stat( n)
  if File.file?(n) then

    # Take off the front part
    p = n[(ref_dir.length..-1)]
    # puts "subpath #{p}"

    d = File.join( src_dir, p)
    if !File.file?( d) then
      puts "File #{d} does not exist, copying it over"
      `cp #{n} #{d}`
    end

    next
  end

end

l4.each do |n|
  result =  File.stat( n)
  if File.file?(n) then

    # Take off the front part
    p = n[(ref_dir.length..-1)]
    # puts "subpath #{p}"

    d = File.join( src_dir, p)
    if File.file?( d) then
      if d.match( /\.rb$/) then
        result = FileUtils.cmp( n, d)
        if( !result) then
          puts "Ruby file #{d} does not match"
          r = `diff #{n} #{d}`
          `meld #{n} #{d}`
        end
      end
    end

    next
  end

end