Introducing TestMonkey

by Jeff Haynie on October 17, 2008 · Comments

It’s not fully ready for prime time yet but it should come together in the next few days. However, if you’d like to start following the dev, hop on over to GitHub and check out TestMonkey.

TestMonkey is an open source (Apache License) UI test framework we’re introducing today. It’s going to be fully integrated into Appcelerator (with some additional cool features on top if you’re using in an Appcelerator app). Additionally, it’s completely standalone as well and you can use on any web application – appcelerator-based or not.

Our main goal is to create a better UI framework for building out front-end test cases. We’re initially focused on unit tests. However, we’ll introduce higher level testing like use case testing soon. We’re going to offer some really neat features in the coming months to do much more advanced automated testing and quality control. So, stay tuned!

There’s a lot of great frameworks for testing out there. But, in open source style, we’ve continued to experience a lot of difficulty trying to do it with everything we’ve found so we’re going to introduce our own product. I’ll talk more in an upcoming post as we get closer to an official release about some of the differences and why we think our approach is easier, faster and ultimately, better.

Here’s some cool things you can do today with it:

1. All test suites run inside their own iframe sandbox. so, if you have any weird issues in one test suite or set of specific tests, you won’t screw up the others…

2. We’re providing a lot of convenience assertions for common UI testing .. thinks like checking for element attributes, element values, checkbox states, etc. are all as easy as pie.

3. We’re building a super cool UI on top for driving tests and the reporting of tests. Right now it’s pretty limited but we’re going to blow that out. Our goal is to provide as much information about failures, location of failures, expected results, etc. so it’s easy to figure out the issues.

4. TestMonkey has a clean API and can easily be extended, for example, to create your own assertion helper functions or test monkey plugin for handling results. In fact, our UI driver is simply an implementation of this plugin. You can easily hook into your own system if you’d like to handle results or do interesting things with them.

5. TestMonkey itself is very small and you don’t need to include anything in your application related to it. Test monkey can load your HTML files up in the sandbox and then your tests can run against the real source, no crazy includes or manually adding of test framework into the real app. It cleanly separates your tests from the real app.

And, there’s more…. And a lot more planned.

From an Appcelerator perspective, we’re essentially re-writing all our unit tests for the entire front-end in Test Monkey now for the upcoming 3.0 release. This will make it much faster to run through all the UI tests across multiple browsers and platforms. Our current web unit test framework is very manually intensive and takes about an hour to run (by a person, not a computer) for each browser + platform combo. With TestMonkey, we’ll be able to run around 90% of the unit tests on an automated basis across multiple platform concurrently. We’re also going to add this to our nightly build as well as continuous build system in the near future. This should really improve the speed and quality of the product.

Popularity: 48% [?]

If you enjoyed this post, make sure you subscribe to my RSS feed!

  • matt
    @jeff - good deal - thanks - looking forward to it!
  • How does this differ from dojo.doh ?
  • TestMonkey looks sweet! I blogged about it today at http://tinyurl.com/5ze6nb.

    Nicely done. I'm looking forward to support it.

    -Frank
  • @matt - thanks. we don't have the CI build pieces in yet but they will overlay and we should have something in place for that soon. we have a lot of additional automation capabilities we're going to be introducing soon....
  • matt
    I took a spin through the code on GitHub - looks great - quick question - automation...I'm missing the big picture here - how would I plug this into my CI build? Will there be any notion of "autodiscovery" of tests so that you don't need to manually update a manifest of test suites every time something is added? That aside - this is phenomenal - thanks for making this available!
  • Andy
    I got to see this in Chicago. Pretty good stuff. I'm kind of tired of JUnit+HttpClient
  • @itod -- thanks. i like the idea of putting it at the end, will make that optional. we can also just basically check and see if the first param is typeof object and assume it's not provided too....
  • I'm very excited to see this Jeff... I've used Selenium before and it's excellent... but I really think there's a lot of room for more automated web testing tools. Lots of web UI/JS/HTML/CSS stuff is still very difficult (impossible?) to test in an automated fashion. Go TestMonkey!

    One suggestion: I know it matches the old JUnit convention, but I wish the comment string parameter were the final param rather than the first... especially in JS, it seems nice to have optional params at the end. I know some think that the comment param should not be optional, but reasonable people will disagree on that... at least putting it last lets users follow their own preference there...

    Really looking forward to watching this! Great job!
  • @alan - selenium is a good product. if you use selenium you should consider the open source push to test stuff as well. we're just going to be doing some things that are different - in some cases maybe it will overlap, in some cases it will be very complementary.

    @roy - yeah, it's not just for testing JS and in fact, it's really primarily designed for testing out the UI (like HTML, etc).
  • Will this allow for testing of all page elements, or is it limited to JS? In other words... does it work like Grinder and Selenium?
  • What test framework were you using before? I just started using SeleniumRC and have found it to be pretty good, although they could use some more functionality around AJAX testing, but it wouldn't need much... and it's already open-source, so I am just curious which are the problems you couldn't solve that caused you to branch out with your own testing framework?

    Thanks!
    Alan
blog comments powered by Disqus

Previous post: Tough times, tough decisions

Next post: You guys…