Continuous Integration with Hudson
To give you a little history
Before I discovered Selenium grid and Hudson, I put in place a system for continuous testing using Selenium scripts written in HTML, Cruise Control for continuous integration and Subversion for version control of software code.
Prerequisites:
- Subversion (http://subversion.tigris.org/getting.html)
- Selenium Grid (http://selenium-grid.seleniumhq.org/)
- Hudson (https://hudson.dev.java.net/)
Download and install Selenium Grid. Run the sample demo to get a feel of what you are working with. I decided to teach myself ruby and what better way to do that than to play with an open source system, which recommends it.
Running the test cases locally??
After downloading the selenium grid project, I renamed it to seleniumSystemTests. Then I created my testSuite directory using the code in the examples/ruby directory, as a reference. Copy the Rake file and the equivalent of the spec_helper file from the examples/ruby directory to the testSuite directory. Modify the URL for application under test, in the spec_helper file equivalent.
application_host = ENV['SELENIUM_APPLICATION_HOST'] || "www.comcast.net"
I added my test cases in the testSuite directory and was able to run the test suite locally by following the steps below
In the first terminal (Mac OS X)
- cd seleniumSystemTests
- rake hub:start
- Open http://localhost:4444/console to see the supported browsers
In the second terminal
- cd seleniumSystemTests
- rake rc:start_all
- In the console above you will see 15 remote controls available
In the third terminal
- cd seleniumSystemTests/testSuite
- rake tests:run_in_parallel
- See the tests cases run in parallel on Firefox browsers
Add the seleniumSystemTests project into subversion
svn import seleniumSystemTests https://<svn_server_url>/path/to/repo/seleniumSystemTests -m "Checking in the selenium project"??
Running the test cases remotely
Now that the test cases are running successfully with the hub and remote controls running on the same local machine, the next logical step is to run the hub and remote controls on separate machines.
Machine A:??
- Download the seleniumSystemTests project
- cd seleniumSystemTests
- rake hub:start
- Open http://localhost:4444/console to see the browsers supported
Machine B:
- Check out the seleniumSystemTests project
- cd seleniumSystemTests
- rake HOST=<MachineB_IP_address> HUB_URL=http://<MachineA_IP_address>:4444 rc:start_all
- In the console above you will see 15 remote controls available
Go back to Machine A (continuing from where we left off)
- Open http://localhost:4444/console to see the 15 browsers available
- Open another terminal to the testSuite directory and run the test cases
rake tests:run_in_parallel
FAIL!!!!!
This is because the remote control server is set to localhost. Open the testSuite directory and find the spec_helper file equivalent. Modify it to say
remote_control_server = ENV['SELENIUM_RC_HOST'] || "<machineA_url>" ?? ??
Running the test cases remotely and continuously
Now that the test cases are running remotely, the next logical step is to integrate it with Hudson to run it continuously. Sign into Hudson and configure it as below.
First create a node called Selenium Grid
- On the Hudson main page, click Manage Hudson link
- Click on Manage Nodes link
- Click New Node link
- Give the node a name, like Selenium Grid
- Choose Dumb Slave and click Ok button
- For Name, enter Selenium Grid
- For Description, enter Selenium Grid??
- For # of executors, enter 1
- For Remote FS root, enter your root directory path (/Users/<your_username>)
- Leave Labels blank
- For Usage, choose Leave this machine for tied jobs only
- For Launch method, choose Launch slave methods via JNLP
- For Availability, choose Keep this slave on-line as much as possible
- Click Save button
Secondly create a Hudson project to run the test cases regularly
- From the Hudson home page, click New Job link.
- Give the new job a name, like seleniumSystemTests??
- Choose Build a free-style software project option and click Ok button
- Tie this project to the node named ?Selenium Grid?
- Choose subversion option and enter the SVN repo URL
- Set the build to run 5 min past every hour
- Enter the commands to execute the tests
cd testSuite
rake tests:run_in_parallel
- Click ?E-mail notification? option and enter your email address
- By default, the system is set to send you an email only when the build breaks
- Click ?Save? button
Finally, click the Build Now button and sit back.??
- Under Build History find the first build run.??
- Click the currently running build link
- Click Console Output link??
- See the test cases run and the Success message appears at the end
Hope this helps. Now you will receive an email only when the build breaks. In that case, once you open the URL in the email click the Console Output link. Now you can debug the problem.
Running the test cases remotely and continuously
Now that the tests are running continuously against the web application on an hourly basis, the next logic step I am going to take is to run it against more environments. Right now the test cases were running against the Mac/FF environment. Mac/Safari, Windows/IE and Windows/FF are on my list. From my initial investigation it looks like this should work by adding more parameters, indicating the browser and operating system being used, while starting up the remote control and while running the test cases.
Further logical steps include
- When new code is checked into subversion, Hudson should automatically deploy the application build to a web server
- Once this completes, a trigger goes off in the Hudson seleniumSystemTests project that imports any new test cases added to subversion. Then it runs the test cases against the new version of the web application.
Look out for further posts when I put the final pieces in place.