iPad’s, iPhone’s, iThink I’m going broke

Some of you may know that over the past few years I have been a bit of a early adopter, when it comes to Apple products. A few years ago, it was considerably easier — since the rate of product releases felt much slower and the increase in technological capabilities of each of the revisions seems significantly less compelling than today.

I remember thinking, I have a Macbook Pro and an iPod — what else could Apple possibly release that would be so compelling that I had to have it. As the upgrades happened, the iPod was still really just an iPod and the laptop’s got better but not that much better that you couldn’t wait two or three releases without feeling left behind. I feel that over the last year Apple has decided to call my bluff, and I have paid dearly.

As described by a “marketing person” a month or so ago, apparently I am a “card carrying geek”, and as a result cannot help but rally behind innovative technology. Growing up to Star Trek: The Next Generation, I am star struck by multi touch, video chat, GPS positioning, etc. thus I am the perfect target for these products.

I first have to say, I do this for all of you.. I mean, I can’t count the number of people who get in touch to ask about products before buying them — and I can’t let those folks down.

My little un-boxing ritual

I bought an iPad five hours after the release downtown San Francisco. I told myself, this time I’m going to be patient — I’m not going to join the insanity and stand in line like I did for the last two iPhone releases. And for five painful hours I sat at home watching the news and occasionally calling the Apple store to see how the stock was doing. Unlike my laptop and my iPhone, the iPad is not something that I need. In fact I’m having a hard time justifying it at all, considering the amount of devices sitting around my house.

HOWEVER, the iPad is a fantastic experience for the person who doesn’t have one of those two devices. It makes for a great browsing, emailing, casual gaming experience. Skype on wifi and 3g with iPhone headphones turns it into a totally legit way to make quick calls and the apps are getting better all the time. I personally am awaiting IOS4 to really make use of it. Pretty upset about the removal of the unlimited data plans by AT&T, if I was doing much travel or didn’t work in front of a computer all day, the iPad with unlimited data and Netflix would be an even more killer combo than it is on wifi.

My vision of the iPad is that it’s a start down a road to some exponentially cooler tech. The use case I invision is currently possible, but in a very early state. The single always online device you carry with your everywhere and dock in various locations for a more extensive experience. Many would argue that this same scenario applies to the iPhone. I still find the size of the iPad very curious, as it’s not small enough to carry in your pocket, but clearly too big to want to leave sitting in any one place.

I’m just going to come out and say this, I really like the look of the iPhone 3GS and preferred the weight of the original iPhone 2G. The new design doesn’t look very traditional apple to me, and so far looks significantly more invasive to my pocket. Fortunately the feature set is pretty amazing, but minus FaceTime and the gyroscope I’m wondering how much of the new experience I’m looking for will be achievable by upgrading my 3GS to the new software? I don’t feel all that compelled to do video editing on a tiny device, no matter how clear the resolution is. I understand that the speed updates are incredible, but will my 3GS experience start feeling unreasonably slow? I doubt it.

I just wanted to point all that out, so that after I try it out and wind up upgrading I have something to look back at with my starry eyes so I can answer all those questions (thus justifying the purchase).

It’s important to look back over the last few years and remember how the iPhone has changed things, I realize that is almost exactly what Steve said in the keynote. In Portland last week, I was walking home and my phone battery ran out after using the map all day long. For a few blocks I was pretty terrified because I realized that the first of my three reactions to my situation relied on, map, google, phone call. All the sudden my brain had to start doing things that it wasn’t used to doing! I’m not saying whether this is a good change or a bad change, I’m just saying it is a DRASTIC change.

Over the past month here in the bay area, plenty of people I know have been talking about how they *may* move over to one of the new android phones. They have these great big display’s with higher resolution photos, and it’s an open platform. I would be surprised if one of those folks actually made the change after this new phone becomes available… I’m just sayin.

It’s entirely possible that I’m an Apple fan boy living in the bubble, and recently I have been seriously thinking about running Linux on my MacBook instead of MacOSX (to mix things up a bit). But I would challenge all hardware makers to produce something, that head to head (in terms of user experience) can win me over.

So this is where I would normally try to conclude by casting my opinions as facts that you should really think hard about. Instead, I propose the question – without the innovation at Apple, where would the industry be? And for those of you who think I’m inside the bubble, enlighten me – what is going on out there?

My new gig – Sauce Labs

After almost two years of working at Slide Inc, I have started my new job at Sauce Labs.

The press release can be found here: “Sauce Labs Adds Windmill Test Framework Co-Creator Adam Christian to Engineering Team“.

Slide Inc.

I had a fantastic experience and learned a ton working with the really talented team of engineers, artists and product managers over at Slide. It was incredibly educational to work in an environment where so many people use your product everyday. I built a lot of really cool features there for SuperPocus and spent a year building a test automation infrastructure, and molding Windmill to be able to test anything and everything they put in front of me.

Slide was really amazingly understanding as I went through some personal struggles over the past few months. I will miss the people the most, as Slide created a great environment enabling people to effectively work together to solve really challenging problems.

Sauce Labs

Since I moved to the Bay Area, more than once I have seen people leave jobs to goto what they deemed their “dream job”. I never really understood what they meant by that designation until now.

Sauce Labs is solving exactly the class of problems that I find the most interesting, challenging, and sought after by so many people. I’m incredibly grateful and excited to be a part of the team working to make running your tests in the cloud seamless and fast, instead of frustrating and painful. The crew of people I will be working with are second to none and I look forward to learning all I can from them.

Sauce expects everyone to work directly with customers to ensure the best experience, and I look forwarding to helping many new teams get setup with test automation.

Future

It’s hard to outline exactly what future projects I will be involved in, as the technology is moving forward incredibly fast. I do know there are so many ways that we can make the testing community stronger, and the tools better. I see NodeJS and CouchDB opening many doors to new innovations and I would like to continue improving my Python skills.

I will still be within a block from South Park, so let me know if you are in the area and want to grab lunch!

Considering in-house web automation?

Recently I have had numerous conversations with people at various tiers of companies all over the place who are toying with the idea of building their own test automation and continuous integration infrastructure. Since I have spent a considerable amount of time dealing with such undertakings I decided that it might be worth the time to brain dump some of the issues you may want to consider before you dive in.

Choosing Tools

A common first reaction is to take a couple of those old boxes sitting around to run the first “couple” tests you have. In some cases, this is the perfect solution. That is if you have a small application that rarely changes and only needs a daily test run (on one operating system and it’s available browsers). In my experience you can reasonably run one Windows VM without lagging the host machine unusable, which gives you two concurrent browser test jobs without worrying about process conflicts. Do remember that to do this correctly, you really want a machine dedicated to your CI system (which I will talk more about below).

VM’s are a great solution, however they require significant hardware overhead, continuous attention and licensing costs. Depending on the VM solution you choose you may also wind up dealing with the dreaded VM Time Drift causing problems with JavaScript and failing tests that aren’t actually failing.

“The Cloud”, is the 2009-2010 buzz word that makes all technology sound better, and in a lot of ways COULD be the ideal solution for test automation. You get the benefits of paying for only the cycles you use, having someone else manage the infrastructure and avoid those pesky licensing costs. However in my experience the setup is painful, the solutions (EC2, etc.) are slow and lacking some of the features to really do test automation well. For example, if you want to run your own CI instance and run your tests on demand in the cloud you will run into some pretty painful engineering problems. It’s not easy to instruct the “cloud” service to fire up a Windows VM and then have that VM connect to your CI instance and become an available slave. It’s doable with Linux, but last I checked – the features to do something similar with Windows simply weren’t there. Also do you really want to wait sometimes up to 15 minutes (also my experience with EC2) for the machines to come up before you can even start running your tests?

What CI System?

The point of this article is not to recommend solutions, it is to encourage questions. However, outlining all of the possible CI systems would take forever so I will simply say that I wound up using Hudson. The reasoning includes a very open and functional Open Source community, with smart contributors willing to take a few minutes to respond and help me out. I also found it possible (not simplistic) to build plugins to customize the things that I needed changed. Many people out there swear by Cruise Control, or Build Bot and I would highly encourage you to do some research and pick the solution that you feel will allow you to be the most productive. For example if you plan to use Windmill and EC2, you may want to do some reading about the Amazon EC2 and Windmill plugins available for Hudson and see what comparable tools are available.

Which test framework?

Some of you may know that I hold a mild bias when it comes to this question, but over the last year and a half I have ventured out into the land of testing frameworks and am able to see the values held by the other projects. For example, if you don’t ever need to deal with JavaScript in your application (or have browser specific functionality) — I suggest you use a tool that doesn’t require a real browser (like Twill). Tests will run faster, they will be more stable and can be run without access to your OS’s graphics layer.

When it comes to browser based web testing tools I really think you need to pick the one that fits your needs the best. A great example of this was in my needs to automate functionality contained in iframes being served over HTTPS from a different domain, the only solution (after weeks of trial and error) turned out to be WatiN. Of course, writing and building tests in C#.net wasn’t going to be an easy sell so IronWatin was invented as a means to write WatiN tests in Python.

Watir has captured a lot of the Ruby community and has recently been moving towards consolidating the separated browser  projects into one, which will significantly improve the ease of use.

Windmill has a dedicated community, focuses on dynamic JavaScript applications, boasts lots of features and goes for an ease out of the box type of experience.. at least that’s what I would like to think! Please feel free to check out the destination site or Github wiki for more information

Selenium has a thriving community, lots of available consulting support, integrates well into a Java environment and offers the Grid project. You can also avoid all of the work involved in running your own system by writing your tests in Selenium and then offloading them to a company like Sauce Labs if you are willing to pay for it.

What do we do about Flash/AS3 automation?

After unsuccessfully trying out FlexMonkey and AsUnit (don’t take my word for it), Matthew Eernisse sat down and wrote an AS3 test controller that works the way the rest of test automation works in the browser, it’s part of the Windmill codebase (codenamed FlashMill). There are two ways to run the tests, one is to hand FlashMill your tests already written in AS3 and the other is to write them in your favorite tool (or raw JavaScript) and have them call into the FlashMill API. Windmill currently has full IDE/UI integration in master to be released soon, the integration code is simple and can be viewed here (best doco at the moment until I write a better one).

Workload

Depending on the machine setup you are going with, this will vary. Obviously if you have a box and a VM on it you can manually go through the process of upgrading the browsers on each, installing patches and security fixes for the OS etc. But if you went with the VM solution, you need to come up with a way to deploy updates to all the machines in your pool. An Open Source solution that came up near the top of my search is WPKG, but like the rest of the tools on this page – there are many solutions and you will want to do your research. Some of the maintenance you will be dealing with can be done as a system job, or run by your CI system. A good example of this is to remove data, test files, source repositories etc. that accumulate on your test running machines.After a while, these files in combination with temporary internet files from the browsers and system tmp files start to slow things down.

This piece is very important to take into consideration from the beginning, because once you have 15 VM’s running tests — doing anything manually becomes a major chore. You also need to be cognizant of that fact that if you chose the cloud testing solution you will be managing your own test running images used to boot the VM’s. Every time you want to make updates or changes to that image, you get to go through the whole process of baking it and uploading it to the cloud hosting service. In my experience, this process is NOT enjoyable or quick.. so be prepared to invest some serious time.

What is the strategy for scaling and expanding?

Clearly this will be dictated by the rest of decisions you made, but I think when you get to this point, the idea of buying and manually setting up more and more physical boxes starts to break down. Buying more and more machines to sit there running tests simply seems like a bad use of resources (and desk space). VM’s allow you to quickly replicate images and expand your arsenal as long as the host machine has the hardware resources to power it without negatively effecting test run times on all the others. There are also solutions out there that allow you to boot and shutdown VM’s on the fly, which provides some interesting possibilities in juggling system resources.

I have found that since Windows is the common platform that can run all the browsers I care about, having a large pool of identical VM’s running all the time is an easy way to queue up 100′s of tests and get results in a reasonable amount of time. I do think that this is the aspect that cloud services start to become more appealing. The idea of spinning up more and more virtual machines in the cloud (with essentially endless capacity) makes the idea of scaling those tests considerably less terrifying. If you can get over the generally slow spin up times, and have come up with a strategy of dynamically harnessing and adding those machines to the pool – you may have it made!

What format/language is best for our tests?

At this point in web testing you simply need to decide what you care the most about doing. Is it manipulating the page? Or is it interacting with a database?, or a little of both? If you can get away with writing your tests fully in JavaScript, I would recommend it. At least in Windmill the JavaScript tests run exponentially faster than the Python or Ruby tests do. However the libraries for communicating with databases, email, system services etc, may make Python, Java, C#, etc your ideal solution. Each of the tools has their own ways of doing things, and to come up with the ideal language really depends upon your system, what your test developers are comfortable with, and what your application platform looks like.

What operating systems and browsers do we need?

The answer to this question should come from the metrics of your user base, and will also help you narrow down a testing framework and your test machines. Some of the available frameworks simply won’t run on Linux, or support IE6. Are all your users using Google Chrome? Then you should probably make sure the test framework you use has a Chrome launcher. Historically I have concentrated about 90% of the available VM resources on the most popular platform (usually Windows), with the most popular browsers (Firefox latest release, IE 7 or 8 depending on the users, Safari Windows latest release, and Chrome). This gives me a pool of work horse machines that can crank out tests representing the majority. The other 10% would be divided into the higher percentage minorities which probably includes a MacOSX slave running FF and Safari and a Linux machine running FF and Konqueror.

Methodology

I have found that having your own test infrastructure really makes this one easier. Since your resources will probably be on the same network as your codebase, you can easily access and run unit tests.. however as soon as you start sending your tests off to the cloud you are dealing with some security/privacy issues and engineering challenges. If inorder to unit test your code you need the entirety of your code base available, sending a copy of it off to your image on the cloud for every test (job, run or even change set) over and over could become a major strain on your system and simply sounds like a bad idea. If you are counting on unit tests you will at least want a machine on your network available as a slave from the CI system for those jobs.

How do we report results, and stay tuned into failures?

Most CI solutions have many ways they can be configured to alert you of a failure, from email, irc, jabber to a phone call you can usually find some solution that will get your attention. At this point in time the norm appears to be jUnit compatible results in XML. I’m not a huge fan, but the available tools for parsing and aggregating jUnit into something useful is very appealing. If you are okay with a true/false, that will always work out of the box, but you will need to be prepared to put in a little extra effort in both your test development and test job setup to generate the jUnit report files.

What role does automation need to play in our process?

For maximal results you need to make the leap where QA and Development both understand that a failing job in the CI system is a show stopper and must be investigated immediately. This way people honor the continual nature of continuous integration. It’s job is to catch problems shortly after they are broken, point you directly at them, and continue failing until you have solved the issue or fixed the test. If this isn’t the process you want, having continuous integration may not be the solution you are looking for.

Another effective strategy I have seen is to base release viability on the status of your CI system. Don’t let the product go out the door until the ‘thoroughly defined’ suite of functional and unit tests running in continuous integration are all running and passing. It is easy to push off the process of updating failing tests until “later”, but everyone is busy, and later is usually never.

What part of our application should we automate?

This really should be computed by time and resources, as much as I would like to say “everything” I am well aware of it’s low probability. Pick your application flows that make you money, or really mean a lot to your users likely experience. Ensure that your application loads, they can take the happy path, give you some money and then leave. At least this way you can sleep at night knowing that, they may not be able to change their profile information but they can still pay your salary.

Conclusion

Meet Kimta

The year 2010 so far has been personally very hard on me, however there has been a bit of shining light. Starting in mid January Megan and I started tossing around the idea of having an animal in our little Russian Hill cottage, after living with two awesome cats for the last two years things just didn’t feel right without the raw untainted animal presence to which we became accustomed. The land lord wasn’t keen on dogs on the property, and I can fully understand and appreciate that rule as dogs left at home for too long can cause quite a lot of noise. I was however a bit disappointed because I was really starting to get used to the idea of a Shibu Inu.

However, we were given approval to get a cat without any specification as to what kind of cat. I started googling for “cat leash walk” etc. and came upon Servals on many of the big cat websites. Unfortunately most of the write-ups concluded with “check your states legislation as to the legality of owning a big cat”. Of course California would be one of the states to completely outlaw and enforce the rules of not having a pet big cat. So then my search turned into “biggest legal cat in California” and I stumbled upon a news report in San Diego about this “new” and emerging breed called the Savannah cat. It turns out that starting in 1986 someone though it would be pretty neat to have a cat with the temperament and some of the physical features of a Serval cat without having to deal with your house getting sprayed and literally torn apart.

Don’t get me wrong, if I was living in any of the surrounding states I probably would have bought a full blown Serval cat.. but living in a small place in San Francisco it just didn’t seem like a smart move even if it was legal. So I started emailing various breeders across the entire US that I found on the Available Savannah Kittens Yahoo Group. I received a few really great responses, and had some really satisfying correspondence with very caring breeders. A few less than inspring responses simply containing a photo and a price tag and a little in between. One response didn’t show up until February 4th from Jungle Mountain Exotics based in the San Fernando Valley. February 5th was a pretty low day, and after a night of not being able to sleep and thinking about the pictures in the email I broke out my phone while still laying in bed pulled up the email and said “thats the one I want”. To make a long story short, 5 hours of phone calls later we were in the car driving to LA.

We arrived around 9AM Saturday morning at Jungle Mountain after a restless sleep at the Van Nuys Hampton. We were greeted by Bion Kirk and were led into an office with a litter box where we heard a squeak and meow from the corner of the room under a desk. After a few seconds of swinging around the feathers on a stick a little white spotted lightening bolt came cruising out from under the desk and went right up to Meg.. game over.

Bion and Temba (F1 Safari Cat)

Meg and Temba

Hybrid Wolf!

We took a tour of the compound and saw Kimta’s parents and the famous F1 Safari, as well as a hybrid wolf and various other cool animals.

We then headed back to the little office where we could watch little Kimta run around in circles while Bion told us how to take care of him, clip his nails, shots etc and then we stuck him a carrier and took him to the car. We made a quick stop by a pet stop to buy some goodies, food and litter box and then set on our way. For a little kitten he was surprisingly happy to sleep on our laps (total strangers) the whole 5 and a half hour drive up to San Francisco.

The view for 5.5 hours, who are you?

After a couple days of settling in, he pretty much runs the house and has become part of the family. I have been busily training him (or letting him train me) how to go on a walk. He currently loves to play fetch for hours and after a long day alone he has an insane amount of energy that needs to be burnt off.

As for the name, it represents a very special place to me that I spent time with my Dad as a kid on the Olympic Peninsula in the middle of the park just north of Lake Quinault. “Kimta Peak”, is a place where you rarely get good weather because it’s constantly surrounded by intense weather systems.. but when you do it opens up and is one the most amazing views you will ever see. At the base of the Peak is a small glacial lake full of freezing glacier water and ice bergs where you can take a morning dip to wake up and bask in the glory of the immensely hard work you put out to get there (on the Skyline Trail). The second I saw the pictures of the little cub the name jumped into my head.

Dad at Kimta Peak.

View Larger Map

As I have told this whole crazy story to quite a few friends and family at this point, I decided it was time to write it up before I start forgetting details. At this point I have noticed a few things about Kimta that are different from the cats I spent the last two years with. As long as I am in sight, he is okay pretty much anywhere he is (car rides, walks, new buildings/places). When he is getting ready to hunt one of his toys he will make an interesting chirping noise that sounds more like a bird than a cat. He plays in the shower and doesn’t mind getting wet (on his own terms, walking in the rain is not one of his favorite activities).

Kimta regularly head bumps me anytime he finds my head near enough to his, which I have read is normal behavior for the Serval.. but I still don’t quite know what it means. Many of the write-ups I read about Savannah’s really made them out to sound like a dog like cat, however they are still a cat and you need to keep that in mind. When walking, those giant ears are tuned to hear every single noise for miles and a screeching truck can really make them uncomfortable. I find that when walking if I continually talk to him, or have Pandora on my iPhone turned up as high as it will go (with some Mark Knopfler) that he will concentrate on that and continue his walk instead of constantly stopping to figure out what he’s hearing.

These animals attach to you, not to your house – so you will find them following you from room to room about 5 feet behind you to keep an eye on what you’re up to. They beg for scraps and go CRAZY when you have pungent cheese around, we found this out when opening a can of Cougar Gold (GO COUGS).

Kimta is a Silver Savannah with a coat similar to a Silver Tabby, however the stripes he has are thiner and more defined and he has many rosette’s (spots without an interior). I’m told that the Silver coat genetics are similar to that of the elusive white tigers. My understanding is that Kimta is somewhere around ~20% African Serval, but to me he’s the little dog cat that greets me at the door with a wagging tail.

Getting some sun after a morning walk!

Kimta on the hunt (for flying bugs)

Radio tuner ears!

A few people out there know that I have occasionally mentioned my dream of one day owning a ranch somewhere in the Northwest where I can raise some Mountain Lions.. I guess perfect practice makes perfect! It has been quite a trip so far, thanks for reading!

#OSB, #CLS and #OSCON – 09

I had the opportunity to attend the Community Leadership Summit in San Jose last weekend followed by OSCON in the same venue. The whole experience of these in conjunction gave me a lot of interesting comparison and contrast.

First I’m going to talk about my frustrations so that I can get them out of the way and talk about all the really great stuff. Like most people who have written about their experience of the weekend — I miss Portland for OSCON. I was in Portland for the Open Source Bridge, which was small, intimate, granola and very purely open sourcey. My main problem was that driving from Oakland to San Jose was absolutely 10x more painful than I imagined it would be. By the time I got down there each day to park (and pay) for that massive hot concrete bunker they call a parking garage I was in a terrible mood. It took at least an hour before I was even in the mood to talk to anyone. I’m not going to spend any more time harping on this, but please O’Reilly, if you are doing it in the bay area do it in SF, Oakland, or even Sanoma — but SJ didn’t work for me

Good things! Open Source Bridge was really small, but there was a great tight knit group of folks there. Percentage wise, there were more women attending and involved than any other conference I have been to. I think conference planner should take a look at how they managed their costs and only sprung for the necessity. I do hope that next year the attendance goes up, more talks and interesting people to talk to is the only thing I would like to see change with this conference.

Community Leadership Summit was different than any meet-up I have ever been to. The extreme un-conference format really gave it a different vibe. The idea here was that anyone involved, or interested in being involved in an OSS community could come and basically round table a whole big set of different topics (determined on the fly). And it actually worked! Many interesting folks from different organizations and projects showed up and had a lot of questions, and a lot to say. Nine out of ten of the discussions I participated in had some really great substance, and a reasonable flow of different people talking. One was pretty much two or three people talking a lot, lots of people listening, and one guy very blatantly sleeping. I guess you can’t win them all… One thing I thought was really interesting was a small almost ‘track’ of ‘social media in community building’. It seems many people want to build up their community, but are already overwhelmed with all of the different tasks involved. Their question’s were — ‘Should I use twitter/facebook etc. to promote my project?’ And if so, “How do I do it without spending all day and night on there?’ I think the answer is “Yes”, use them, as much as you reasonably can. You want to really kick ass and grow you community… give up sleeping for a couple weeks, get interns, encourage your community help you!

I’m not going to try to provide a full recap of the discussions, because they were beyond what I have time to even try to summarize — but it was great talking with all of you! Special thanks for Jono Bacon and Canonical for their support of this event.

I have to admit, by the time it became time to get involved in OSCON I was pretty beat. Since I work in SF I was trying to be involved in the conference, but also keep working — which was a bit much. I have to say for my personal growth, the best thing I did was attend the Damien Conway Speaker Workshop. I heard his great talk on “How to not suck at being a speaker”, and then had the chance to get up in front of everyone and get torn apart. It turned out to be the best speaking advise I have ever received, Tuesday I completely re-did my Slides and practiced the talk out-loud in the mirror with an audience of cats. Unfortunately part of the re-do did removed all of the loud Journey from the talk.. I may see if there is some way I can work a little of that back in for next year!

• Bigger Font, ALWAYS
• Talk to the audience, don’t preach to them
• Simple slides, simple colors
• Don’t put a logo on every slide, its annoying
• Ask questions
• Tell a story
• Be very careful with demo videos
• Talk about things you care about and know
• 4 points max on a slide, they aren’t queue cards
• Don’t be nervous (this is a bit harder to fix)

Julian Cash took this crazy picture of me.

There was a lot more than that, but those were incredibly important in making the Windmill talk more successful. The food was pretty good, the free beers were nice, walking around the Expo Hall yielded lots of cool stuff to look at and play with.

Saw the inevitable excitement over “Cloud Technology”, Scala, Lift, Closure, R, CouchDB and all of the other new and awesome things I haven’t had enough time to really dig into.

Thanks for a great conference season everyone! Monday I leave for Moscow, stand by for a report on that.

Hudson + EC2 + Windmill = Cloud Testing

This should give you a pretty good idea how to use the available tools to run your functional tests out there on the cloud.

I do plan to keep updating this post, so consider this a rough draft for the time being! I hope this provides some value.

Setting up Hudson

It would be pretty silly for me to try and re-document the Hudson installation process, so I will refer you to Meet Hudson.

Installing Hudson Plug-ins

Click the “Manage Hudson” link, and then “Manage Plugins.

Select the “Available” plugins tab.

Select both the Amazon EC2 plugin, and the windmill plugin, then click Install:

You should see the following:

Click “Restart Now”

Success!

Create your slaves

Click “Manage Hudson”

Click “Manage Nodes”, then “New Node” in the upper left.

Configure the node, we do 1 executor so that Windmill test runs don’t step on each other, c:\hudson is the standard place to store hudson job workspaces, and to keep other jobs from using our slave we specify “Leave this machine for tired jobs only”, then save.

Setup some jobs

In the upper left link menu, click “New Job”

We need to make this a “multi-configuration project”, so that we can run the same job against multiple browsers on multiple slaves. Click save.

In this example, I am using wget (which I have installed via cygwin) to pull down the test file into the workspace that we are going to run.

In this example I have selected to run this test on three different machines, each against firefox, ie and safari. This would be useful if each of these machines had different versions of the browsers. You can run each of the combination’s from one job.

Configure your Windmill test run, setting Browser to %browser% allows the job to get the browser from the configuration matrix. We also specify the url, and to run the test file that we download in the previous step. Save.

Setting up EC2

Get an account

Head to http://aws.amazon.com/ and sign up!

Setup Amazon EC2 API Tools

I found this to be relatively straight forward, however these docs look terrifying and poorly organized: http://developer.amazonwebservices.com/connect/entry.jspa?externalID=351

These were much easier on the eyes, and to the point.. plus they are OSX specific, which was nice: http://www.robertsosinski.com/2008/01/26/starting-amazon-ec2-with-mac-os-x/

Once you are all setup, allow connections to all images you launch with remote desktop by issuing the following command:

ec2-authorize default -p 3389


Launch Windmill AMI images

Navigate to the AWS Management Console

Click to Launch some Instances

Click to the “Community AMI’s” tab

My pre-maid Windmill AMI’s should appear, ‘windmillxp’ specifies that it’s the windows xp Windmill image. Click ‘select’.

Your security groups will reflect what you configured for your account, but specify the number of slaves you would like to use. Then click “Launch”.

You should get the following confirmation that your slaves have been launched.

Start the slave agents

For each of the slaves in the following list:

Check the image, and click connect.

Click “Download shortcut file”.

If you have Remote Desktop Installed, you should connect and see the following dialog. I have also found CoRD to be a great alternative on MacOSX.

You will be prompted for a username and password, use ‘Administrator’ and ‘w1ndmill’ respectively. Note: the second character of the password is the number one.

In the Window you can now access, launch a browser and navigate to the URL of your Hudson Instance and find your slave node in the list.

Click that the node to get it’s configuration screen.

Click the “Launch” button, your EC2 image is now an available Hudson slave.

Run that Test

Click the clock image on the right to start the job running. In the VM you should see the Windmill test zooming along. You can also navigate around the job to find access to the console output to see line by line what is happening on the standard out.

Other

The manual launching process can be automated using the hudson EC2 plugin for Linux, but is not yet compatible with Windows. Resolving this step would make the whole process of launching cross platform cross browser tests in the cloud a fully automated process, which we are looking forward to!

Other possible ways to handle this are by using the hudson VMWare plugin, and launch images on a VMWare server.

Windmill Plugin for Hudson

Over the last 6+ months, I have been using Hudson in conjunction with Windmill very heavily for continuous integration. For the most part using the build step specific to whatever the slave OS requires has worked sufficiently well until recently when my needs changed.

I use the ‘configuration matrix’ option to build a matrix of browsers to run the tests, this way I can have one job that represents a test run on multiple boxes and multiple browsers on each box. Drilling down allows me to see the results for each of these test runs within the job. (Configuration Matrix is awesome btw, except for one really annoying bug for, which there is a reasonable workaround)

Having many build resources, sometimes I want to move a job from one machine to another in order to equalize load on the different machines or to have a job running in ff2 and ie6 instead of ff3 and ie7. Thus the need for a uniform build step that will run the same way regardless of the machine or installed browsers, became necessary. Additionally the commands for running tests got bigger and less manageable all the time, so the time for a clean user interface finally came. This way I can automatically append arguments like ‘exit’ to keep the build step interface simple and clean.

The other simplification is that in the Hudson configuration page for the Windmill Test plug-in you can tell it to automatically call the contrib ‘clean_run.py’ script with the correct arguments (assuming clean_run.py can be found in your path).

As I am relatively new to the Java world, I struggled through the development process but thanks to some reasonable plug-in documentation, responses from the hudson dev mailing list and some IRC conversations with Kohsuke (thanks so much) I finally have something that sufficiently addresses my needs.

Screenshots

Project

The code is all available on GitHub and I welcome any improvements or input from the community as I know that continuous integration is a very important piece of the utility provided by Windmill and this pieces should be as easy as possible to setup.

The combination of this plug-in, and others like the amazon s3 plug-in or the hudson VMware plugin bring us that much closer to seamlessly creating test resources, running our functional web tests across all of our supported platforms and then throwing it away until it’s again needed (or using the CLOUD).I have also done some integration with virtual box that I have found to be very successful.

Download

• Hudson Windmill Plugin 0.4

Please log bugs and let me know what you think! If you are an avid Java and or Hudson plug-in developer and you are interested in contributing, please jump right in!

PyCon 2009 Recap

Getting back in the swing of things after conferencing for weeks can be pretty painful, thus the lateness of the post. However I think it’s important to go over some thoughts still lingering in my brain as a result.

First off, I have to say that for those of you who don’t know, PyCon is a community organized event, and amazingly well done. I was impressed by the design of the conference, the way they had four talks going on at once and they tried to keep them in a similar interest track. Every talk I attended was at least “good”, and many were “great”. You could really feel a community vibe, and for a conference that had 800+ attendees in the middle of a major recession they had every right to be excited.

There were two major themes content wise that really impressed me, the first was an amazing amount of web framework focus. Django obviously being the twinkle in the eye of the community, but there were smaller communities for each of the other projects, Pylons-Turbo Gears, web2py and lots of tools built on top of them. One that struck me with some major promise is the Pinax Project. Their goal is to make it so that I don’t ever have to deal with building user registration and in site messaging… and all the other features expected for any site that has social network functionality.

The other major theme was a mini testing conference going on within PyCon, that I was very comfortable hanging around with. We had a hugely successful Birds Of a Feather, as well as a surprisingly active Open Space talk for Windmill.

The “Using Windmill” talk turns out to be pretty successful in every aspect that I really care about. I do wish that I had been able to get a little more sleep the night before, and I have to admit the size of the venue was a bit overwhelming. I now realize watching the footage that I used the word “UM” way too much, and the demo videos must have been hard to watch from the very back of the room. But barring those two things, I am quite happy (I shall learn and practice for the next round of shameless PR at OSCON 2009).

The “Functional Testing Tools in Python” panel was very successful, and a lot of fun. I always enjoy the friendly banter between the different project owners. Everyone has a different opinion on what they care about, focus on and feel they do the best. Obviously since the only two projects represented that focused on Web Testing were Windmill and Selenium, we got a lot of attention.

Watching that panel footage I definitely think that the introductions were too long, but I still think our Journey themed – mind blowing – Windmill demo video was a great intro. At the very least, the audience had a little entertainment before the geek droning began

Slide had an awesome presence this year, a fun booth, huge banners everywhere and 6 attendees. It was fun to see all of the great responses I received about Slide from people out there in Python land.

Here are links to the videos:

• Using Windmill: http://pycon.blip.tv/file/1947236/
• Functional Testing Tools in Python Panel: http://pycon.blip.tv/file/1947342/

And some pictures:

If you are interested in seeing the new and improved version of the “Using Windmill” talk, please make it out to OSCON 2009, “Scheduled for 16:30 on 22 Jul 2009.” in San Jose, CA.

We are waiting to hear back from both Open Source Bridge and the AJAX Experience as to whether we will be participating in those conferences (fingers crossed)!

Skinning Windmill with JQuery UI Themes

I have been doing a lot of UI work on the Windmill trunk, and over the past few months I have had multiple requests for the ability to apply skins. Of course my reaction up until this point has been… alter the CSS! Which is not exactly the answer people were looking for.

Here is your answer, and it is now easier than ever.

1. Go to: http://jqueryui.com/themeroller/ and create your theme (or pick a pre-defined one)
2. Download and unzip the file. (it doesn’t matter what you select for jQuery components)
- On a mac you will find something like this: jquery-ui-1.7.1.custom.zip Folder
3. Open the contained CSS folder there will be another folder, currently windmill uses “smoothness”
4. Find windmill/html/css folder in the windmill source directory
5. Copy the specified folder in step 3 into this folder
6. Edit windmill/html/remote.html, line 10 to read like the following:

1

<link type="text/css" href="css/*your folder name*/jquery-ui-1.7.1.custom.css" rel="stylesheet" />

7. Load windmill!!

Here are a couple examples of pre-defined themes I tested out.

Humanity

UI Darkness

Happy skinning.

MozMill 1.1 UI Overview

During the Open Design session at Mozilla with Aza we were informed that we could load a HTML file with a Chrome URL, allowing me to rebuild the MozMill UI a bit more like a web page instead of using the XUL constructs that I had been struggling with. Granted it feels a lot more like a web page than it does a desktop application, but the speed that I can build new UI features by using libraries like JQuery UI have made it worth it.

The combination of writing content style HTML, and the good advice we received have come together into what I feel is a pretty usable user interface. Granted this is the first revision and will probably continually be refined to become even more user friendly, but from 1.0 to 1.1 it is a vast improvement.

Improved Editor

The first major improvement is the implementation of a full featured code editor named EditArea.

We have been keeping our eye on Bespin, which we will look more into integrating when it is a bit more modular.

For the meantime EditArea does a great job, and was *relatively* painless to integrate.

Some of the features include:

• Multiple file editing
• Syntax hilighting
• Full Screen mode
• Adjustable font properties
• Jump to line numbers
• Search and replace
• Automatic tabulation
• Toggle hi-lighting
• Toggle edit modes

EditArea implements execCommand similar to the implementation in Midas.

Reorganized Menu’s

Part of the move from XUL involved no longer relying on the toolbox, so we have reorganized the menu’s into dialog’s (don’t worry most functions have a keyboard shortcut if you are one of those people that doesn’t want to deal with that extra click).

This cleanly displays all the available options, and doesn’t clutter up the main UI. This also provides space to easily add new features that fall into these logical spaces in the future.

Improved Inspector

There was a lot of frustration when it came to getting the results from the inspector into the editor, some of this had to do with the non editable default nature of the elements we were using to draw them out in XUL, but the integration of the feature into the IDE became very messy and confusing.

This creates an obvious separation between the rest of the UI and the inspector feature and puts helper features out in front of you to simplify the process by dumping to the clipboard and then moving you to a focused editor window.

Improved Output

The output UI has been completely revamped to give you the most important information quickly, but allow you to navigate down an expandable tree to explore the output of the exception.

All of the information in the output divs can be easily selected and copied to stick in bugs etc, but also saves you a trip to the error console as it should encompass all of the information being thrown in the error object, serialized and organized into a more readable format.

Thanks to JQuery UI’s information and error boxes I was able to tweak the CSS to make some relatively attractive, but more importantly, informative UI that should quickly give you the status of your test run.

More Information

• MozMill Google Code Project
• Mikeal Rogers Release Blog Post
• Google Code XPI Download
• QMO Documentation
• MozMill on PyPi

Thanks everyone who logged bugs!

• Recent Posts

  • iPad’s, iPhone’s, iThink I’m going broke
  • My new gig – Sauce Labs
  • Considering in-house web automation?
  • Meet Kimta
  • #OSB, #CLS and #OSCON – 09

• Link

  • CompressJSDir
  • ElementsLib
  • HudsonWindmillPlugin
  • JavaScript Magazine
  • MozMill
  • MyTabs
  • TestMill Server
  • WebDevGeekly Podcast
  • Windmill Lite
  • Windmill Test Framework
  • XUSH

• Me

  LinkedIn
  Facebook
  FriendFeed
  Twitter
  Ohloh
  

• Categories

  • Automation (11)
  • Business Ideas (4)
  • Career (7)
  • Continuous Integration (6)
  • Firefox (5)
  • Hudson (3)
  • java (2)
  • JavaScript (14)
  • Life (3)
  • Location Aware (2)
  • Mozilla (8)
  • Open Source (16)
  • Plans (1)
  • Python (6)
  • Review (2)
  • Slide (8)
  • Startup (3)
  • Technology (17)
  • Uncategorized (3)
  • Web (18)
  • Windmill (14)
  • Windmill-dev (8)
  • Work (8)

• Disclaimer

  I work at Sauce Labs. The opinions expressed here are entirely my own, not those of my employer.