Ruby Testing and External Dependencies

We’re strong believers in testing of all sorts. Unit testing ensures that each of your classes and functions does what you expect, in particular at edge cases. Functional tests provide a check that entire systems work from a user point of view. Thanks to test frameworks like RSpec, JUnit, and Jasmine, unit tests are usually easy to create and run, since they depend upon nothing beyond the component being tested. While there are similar tools for creating functional tests (namely, Selenium for testing webapps), running the test can be much more complicated if you have multiple parts to your application.

At Palomino Labs, we recently completed a project that involved two main pieces: a webapp written in Rails and a data storage application in Java. In our scenario, a portion of the webapp’s functionality depended upon being able to communicate with the running Java app. In the development environment, this is a minor nuisance, requiring the developer to have the Java application running. Running the test automatically using Jenkins was a bigger problem, and since we know that automated testing is integral to catching regressions quickly, the problem needed to be solved. One option would be to have a long-running version of the Java service available to the Jenkins box, but that would far from ideal because we want to make sure that the versions of the Java and Rails projects that are used for testing stay in sync with each other. Instead, we have Jenkins start and stop a fresh, up-to-date instance of the Java service for every test run.

Adding Java Convenience

While we have scripts to run our application in the production and staging environments, we usually depend on our IDE for running the application on our development machines. This is a minor issue for developers (“Why are the tests failing? Whoops, I forgot to start that other thing…”) but a show stopper for Jenkins (née Hudson) who doesn’t want to run an IDE. The first task was to create a way to run the Java application from the command line.

Using The Exec Maven Plugin

We use Maven for building our Java projects. The Exec Maven Plugin provides an easy way to run you classes from the command line with the necessary classpath. Usage begins with including it in your pom.xml

<project>

    ...

    <build>
        <plugins>
            <plugin>
                <groupId>org.codehaus.mojo</groupId>
                <artifactId>exec-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>
</project>

Actually running a main() method is easy—you just invoke mvn exec:java and pass it the class you want to run:

mvn exec:java -Dexec.mainClass="com.palominolabs.widgets.Widgeter"

Start & Stop Scripts

We start by creating a set of simple process management scripts to run the Java application in the background and stop it when we’re done testing. Our project uses modules, so this script lives within the widgeter-service module, which has the main method we want to run, at widgets/widgets-service/bin/.

Note the bashism "$(cd "$(dirname "$(dirname "$0")" )"; pwd)" which finds the directory the script lives in. This script also employs lock directories, the use of which removes a race condition when using lock files.

#!/bin/bash
WIDGETER_PATH="$(cd "$(dirname "$(dirname "$0")" )"; pwd)"
RUNDIR="$WIDGETER_PATH/run"
PIDFILE="$RUNDIR/widgeter-service.pid"
LOGFILE="$RUNDIR/widgeter-service.log"

cd $WIDGETER_PATH
if mkdir "$RUNDIR"; then
    mvn exec:java -Dexec.mainClass="com.palominolabs.widgets.Widgeter" 2>&1 > $LOGFILE &
    PID=$!
    echo -n "$PID" >> $PIDFILE
    echo "Started $PID"
else
    PID=$(cat $PIDFILE)
    echo >&2 "Lockdir $RUNDIR exists; is pid $PID still running?"
    exit 1
fi

The stop script lives in the same directory and looks similar:

#!/bin/bash
WIDGETER_PATH="$(cd "$(dirname "$(dirname "$0")" )"; pwd)"
RUNDIR="$WIDGETER_PATH/run"
PIDFILE="$RUNDIR/widgeter-service.pid"

cd $WIDGETER_PATH
if [[ -f "$PIDFILE" ]]; then
    PID=$(cat $PIDFILE)
    echo "Killing $PID"
    kill $PID

    rm -r $RUNDIR
else
    echo "Widgeter isn't running ($PIDFILE doesn't exist)"
fi

With the Java now easy to run from the command line, we just need to wire up the Rails side of things.

Cucumber Integration

We want the Widgeter to be automatically started and whenever Selenium tests (written in Cucumber & Capybara) are run—either by a developer in their IDE or by Jenkins. To do so, we edit the cucumber script provided by the cucumber gem. The one assumption made here is that the widgets project exists at the same level as the Rails project and that this script is run from the root of the rails project.

#!/usr/bin/env ruby
WIDGETS_DIR="../widgets"

system("cd $WIDGETS_DIR && ./widgeter-service/bin/widgeter-service-start.sh")
Signal.trap("EXIT") do
    system("cd $WIDGETS_DIR && ./widgeter-service/bin/widgeter-service-stop.sh")
end

cucumber_bin = Gem.bin_path('cucumber', 'cucumber')
if cucumber_bin
  load cucumber_bin
else
  require 'rubygems' unless ENV['NO_RUBYGEMS']
  require 'cucumber'
  load Cucumber::BINARY
end

Jenkins

Jenkins runs the tests using a script, jenkins-cucumber.sh. Originially, this simply called jenkins-build.sh which sources the project’s .rvmrc, executes bundle install, and performs migrations, before running cucumber under xvfb. Adding lines similar to the cucumber script above to start Widgeter before Cucumber, and stop it when the tests completed or are killed. This script is only for use by Jenkins, so we simply hardcode the location of the Widgets project.

#!/bin/bash -e
source script/jenkins-build.sh

RAILS_DIR=`pwd`
WIDGETS_DIR="/var/lib/jenkins/jobs/widgets/workspace"

cd $WIDGETS_DIR && ./widgeter-service/bin/widgeter-service-start.sh
cleanup() {
    if [[ $WIDGETER_CLEANED -eq 1 ]]; then
        exit 0
    fi
    WIDGETER_CLEANED=1
    cd $WIDGETS_DIR && ./widgeter-service/bin/widgeter-service-stop.sh
}
trap cleanup EXIT

xvfb-run bundle exec cucumber -p ci || true