I’ll be presenting on A/B testing at tonight’s raleigh.rb meetup. As developers, we use tools like RSpec and Cucumber to verify that our application is functional, but how can we verify that the layout of our home page is user-friendly? How can we determine the ideal size for our signup button? How can we maximize throughput to our signup form? A/B testing is an easy and compelling way to increase the effectiveness of our web applications. Join us tonight to learn how to leverage A/B testing in Ruby using several popular tools.
Category: Testing
-
Mass rename files in UNIX
Several of my Rails projects surface a RESTful API. I use integration tests to verify that the API calls work as expected. I also version my API calls so I can easily adapt the API to new circumstances while maintaining backwards compatibility.
To move to a new API version, I copy all of the existing integration tests and rename their prefix to the new version. Instead of renaming the files by hand, there is a nifty UNIX command that handles it for me. For example, to rename all the “v2_*.rb” files to “v3_*.rb” I would type:
for file in *; do mv "$file" "v3_${file#v2_}"; done -
test_spec_on_rails now runs on Rails 2.3
If anyone still happens to be using test_spec, you’ll be thrilled to know that the test_spec_on_rails plugin is now compatible with Rails 2.3. It has also been converted to a gem. Install with:
sudo gem install test_spec_on_rails
Add to your Rails app’s test.rb like so:
config.gem 'test_spec_on_rails'
Enjoy the goodness of test-spec helpers from inside your Rails tests. Fork and submit patches via the GitHub project. Tell your friends. Donate money. Vote for Pedro.
-
Audio interview for RubyRX 2009
Jared Richardson just posted a series of interviews in anticipation of the upcoming RubyRX/AgileRX conference taking place in Reston, Virginia in September. In my interview we discuss iPhone development, MacRuby, Git, and testing frameworks.I’m really looking forward to presenting again at RubyRX. I’ll be giving two talks this year. Git with Ruby will explore the Git source control system and how Ruby can take advantage of it. In Which Ruby Testing Framework Should I Use? we’ll briefly examine several leading testing frameworks and study the pros and cons of each. You’ll leave fully prepared to pick the best framework for your next project.
Let me know if you’re coming to the conference this year and we can link up in Reston. If you haven’t registered yet, what are you waiting for? RubyRX is a chance to network with the best and brightest developers in the area, and hear from thought leaders like Andy Hunt, Rich Kilmer, Joe O’Brien, and Chad Fowler. It’s a great way to keep your skills sharp in a down year.
-
Hey, flexmock and unit_record, play nice!
So you’re savvy with flexmock, a fine Rails plugin that lets you create mock objects in your tests. You’ve been coding up a boatload of fine tests that are elegant in their isolation thanks to your super mocks. They all run fine, but they’re kind of slow so you go Googling for techniques to speed them up.
You stumble across a wonderful plugin called unit_record that can dramatically speed up your unit tests by disconnecting them from the database. You install it, run your tests, and wham, you get a barrage of error messages like this:
ActiveRecord is disconnected; database access is unavailable in unit tests.
There could be one or two things going on here. The first possibility is that one or more of your unit tests are still attempting to access the database via a finder or a fixture. This is the most likely case if you’re only getting a few errors. If you’re getting an error for every single unit test in your suite, though, then something else is going on: flexmock is not playing nicely with unit_record.
The root of the problem is that unit_record attempts to turn off transactional fixtures for your unit tests when you run them, but flexmock hijacks unit_record by defining its own alias chains for setup/teardown.
I got around this problem by updating my unit_test_helper.rb file to include the following:
class Test::Unit::TestCase self.use_transactional_fixtures = false end
It’s unfortunate this has to be done, but it’s a better option than digging into the flexmock plugin itself… especially if you plan on upgrading to a new version of flexmock anytime soon. Don’t forget that you should be disconnecting ActiveRecord in your unit_test_helper.rb as well. This is what my entire file looks like:
require File.dirname(__FILE__) + '/../test_helper' require 'unit_record' ActiveRecord::Base.disconnect! class Test::Unit::TestCase # Unit_record is trying to do this for us, but since we're # using flexmock's test case it patches the wrong class. self.use_transactional_fixtures = false end
Hope this helps someone who was as equally stumped as I was. Happy testing!
Update on 12/1/07: After chatting with Jim Weirich (the creator of flexmock) and doing some more tweaking, it became obvious that the problem is not with flexmock but lies somewhere else. I removed all references to flexmock from my code and still had trouble. The transactional fixture switch seems to be getting set between test runs, but I can’t locate where it’s happening. If anyone else finds a solution to this problem, please let us know by posting a comment here.
-
The DMV and continuous integration
My father recently had an experience which bitterly proves that minor details can have major consequences. Last week, his insurance company mistakenly reported to the DMV that the coverage on his auto had lapsed. The DMV responded by revoking his tags within the space of 48 hours (quite speedy for them). To resolve the matter, he had to take proof of insurance to a local DMV office (in someone else’s vehicle, of course).
Here’s how the process went, in his own words:
- Go to the DMV office on Academy St.
- Find out they only do driver’s licenses there.
- Go to the DMV office at Walnut and US-1. Walk up to the desk.
- Find out that the FS-1 form the insurance company faxed me has an effective insurance date of 3/34/05 rather than 3/24/05. It’s not acceptable.
- Find a pay phone to call the insurance company for a corrected form. Seems I left my cell phone at home.
- Get the insurance company to fax the corrected form to the DMV office I’m standing outside of.
- Go back into the DMV office and finish the transaction.
- Walk out with my sticker, new registration, and the confidence that comes with obeying our state laws.
Suprisingly, aside from the nuisance of having to physically go to a DMV office, most of the blame for the mistake lies with his insurance company. First, their computers reported him as uninsured, which was, of course, incorrect. Second, when proof of insurance was faxed to him, the date on it wasn’t just off by a day or two… the date wasn’t even valid!
If we assume that the insurance company used a computer to generate the form (a safe assumption), this means that one of the following must be true:
- Their processing is so antiquated that they’re simply filling out word processor templates; the incorrect date was human error
- The software they’re using isn’t smart enough to catch invalid dates
- The software they’re using has a good, ol’ fashioned bug
If #1, they really should consider upgrading their systems to minimize the impact of human error. If #2, they need to consider investing in a more sophisticated software package. In my book, date validation is a fairly important feature! If #3, they were either sold lousy software or their in-house development group needs to do a better job of testing.
Minor details can have major consequences. The best way to ensure the integrity of minor details is to test them. The best way to test something is to always run tests. The best way to always run tests is to use a continuous integration system!
Now if only someone at my father’s insurance company would read this…
-
Practices of an Agile Developer
I recently had the honor of reviewing the Pragmatic Programmers’ latest work of art, Venkat Subramaniam and Andy Hunt’s Practices of an Agile Developer. The book doesn’t ship until March 1st, but I would certainly consider adding it to my Amazon.com wish list now, especially if I were new to agile programming or wanted to promote agile practices within my team. It was refreshing to read such a concise, yet comprehensive overview of what agility is.The book covers all the major agile practices (and several of the not-so-major ones) including minimizing up-front design, being honest even when it hurts, practicing iterative development, writing tests, letting customers make business decisions, integrating continuously, and being a mentor. The focus is not on a single agile methodology like eXtreme Programming or Scrum, but on the practices common to all of these methodologies.
One thing I really liked about this book was that it was split up into 2 to 3 page “chunks” of information. (If you’ve read Chad Fowler’s My Job Went to India, you’ll know what I’m talking about.) Such “iterative” writing makes it much easier for busy geeks to digest. I’m not forced to maintain an excessive amount of context between reads.
Another thing I liked about this book was that it wasn’t dogmatic in its approach to agility. While reading, I didn’t feel that the practices were being crammed down my throat. Venkat and Andy did a good job of espousing the benefits of the practices without becoming preachy. This is an exceptionally good thing considering how difficult it can be to explain to “traditionalists” that agility can actually help, not hinder, the craft of software development.
Venkat and Andy’s creativity really shows with their “angel and demon” motif which is used throughout the book. At the beginning of each section, a demon tells you what the typical response to a common software development problem is. At the end of the section, an angel gives you a brief overview of the agile practice which was covered and why it’s so much better than what the demon tempted you to do.
If you’re new to agile development and want a good overview of what it’s all about, I highly recommend this book. If you’re an experienced agile developer and want a handy dictionary of common practices to refer to or push into the hands of a fellow developer who comes knocking, I highly recommend this book. If you’re fed up with the way software development is done and want to know some practical steps you can take today to begin programming more intelligently, I highly recommend this book.
Most importantly, if you’re passionate about creating software and want to be encouraged to continue striving for excellence in everything that relates to it, this book will give you the boost you need.
-
Watir tips and tricks
I’ve been using Watir for a week or so now and have (naturally) run into a couple of problems. I’ve also come up with a handy way to reference fields on a page. I thought I’d share these tips so others won’t have to spend their precious time working through ’em as I had to do.
Problem: Watir doesn’t respect maxLength
The first problem I found is that Watir doesn’t respect the maxLength attribute on a text field. It will happily enter 20 characters into a text field with a maxLength of 10 without complaining. This can cause problems with assertions and such. I was able to override Watir’s set(..) method with code which will throw an exception if the string to be entered into the text field exceeds the maxLength.
Just add this code to your script if you want the same warning to show up:
class Watir::TextField
alias_method :set_value, :set
def set(value)
if (value.size > maxLength)
fail(“The length of the string < "#{value}"> exceeds ” +
“the max length (#{maxLength}) of the “#{name}” text field.”)
end
set_value(value)
end
endProblem: Watir doesn’t yet handle JavaScript alerts
Another problem I ran into was Watir’s inability to handle JavaScript alerts and pop-ups. (It can handle pop-up browser windows fine, just not windows created with JavaScript’s alert(..) function.) There currently isn’t a very good workaround for this, but the Watir folks are working on a solution for the next release. In the meantime, they suggest using a tool call AutoIt in conjunction with Watir to automate interaction with the pop-ups. I haven’t tried this method yet.
Tip: Abstract out textual references to page elements
The site I’m using Watir to test changes quite frequently. I have no guarantee that a specific page element will be named the same thing for any length of time. Why, then, would I want to scatter references to the element throughout my Watir tests and be forced to change all of the references anytime the element itself is changed? My solution was to abstract out the common attributes of the element so that each element is only defined once. Then, whenever an element changes, I only have to change a single reference.
Ruby’s “method_missing” idiom made it very, very easy to do this. I simply defined a parent for all of my test cases and added the following method definition to it:
def method_missing(method)
key = method.id2name
super unless page_objects.has_key?(key)
args = page_objects[key]
return @ie.send(args[0], args[1], args[2])
endDo you see what this does? Whenever a method which doesn’t exist in a subclass is called, method_missing is passed the name of the (non-existent) method. I take that name and use it as a key in a hash which I expect to be returned by a method called “page_objects” which is defined in a subclass. If the key exists, I grab the object from the hash and use it to call a method on Watir’s IE object (Ruby’s “send” method is the equivalent of an “exec” in other langages… it just allows strings to be interpreted as method calls and arguments).
I then define the “page_objects” method in each of my subclasses like so:
def page_objects
{
“field1” => [:text_field, :id, “my_field”],
“field2” => [:text_field, :id, “another_field”]
}
endI can then simply reference “field1” and “field2” as if they were instance variables of my subclass. For example:
def test_example
field1.setContents(“Hello world”)
assert_equal(“Booyah”, field2.getContents)
endThe calls to “field1” and “field2” are handled by “method_missing” since those methods don’t exist, and the magic then proceeds from there. If the actual ID of either field changes, I simply make the change in my “page_objects” method and my existing tests should all continue working. This indirection follows the DRY principle (Don’t Repeat Yourself).
This trick would almost impossible to accomplish in Java without a large amount of bulky reflection code. Just another example of why dynamic typing is so nice.
Hope you find this idiom as useful as I have. Happy Watir-ing!
-
Watir: Web Application Testing in Ruby
WATIR stands for “Web Application Testing in Ruby”. Watir is a free, open-source functional testing tool for automating browser-based tests of web applications. It is pronounced water.
I began experimenting with Watir over the weekend. I run a database-driven web site for a local non-profit group and have been looking for a good way to functionally test the site with a minimum time investment in a particular framework. I’ve had bad experiences with HtmlUnit and HttpUnit in the past and wanted a way to code the tests in Ruby. I’ve been told that the state of Watir a year ago was really awful, but apparently they’ve made lots of improvements since then because I didn’t have any trouble getting up and running.
As a test writer, most of your interaction with Watir takes placed through the IE object which lets you open a specific URL and then make assertions about what the page should look like. Watir’s flexibility in looking up data on the page is phenomenal. You can do lookups by name, ID, position, and so on. The pages I’ve tested thus far contain light JavaScript so I’m not sure how well Watir performs on pages with heavy scripting, but for my purposes Watir made it quick and easy to write tests that didn’t contain an enourmous amount of code, yet were powerful enough to thoroughly verify the functionality of my web app.
Couple Watir with ActiveRecord and you have a powerful way to verify that data on the backend is getting written and updated correctly from your web application.
I highly recommend you check out Watir for your next project. If you end up using it, feel free to share your thoughts about it by posting a comment here or e-mailing me directly. I’d enjoy hearing from you.