Adam Christian

Writing about Life, Business and Technology - the way I see it.

Entries Comments



Month: November, 2008

Diving into GIT

25 November, 2008 (20:10) | Slide, Technology, Windmill, Work, automation, continuous integration, hudson | By: adam

Over the last year, I have known that the day would come when I could no longer avoid moving from SVN (my comfort zone) to this new beast called GIT that everyone is so excited about. My first hour, which was installing it on my Mac and pulling down a repo to play with was very pleasant. My ssh keys and ~/.ssh/config was already setup the way I wanted and everything just worked.

The pain began when I started in on moving our build slaves over to GIT. Of course two of the three are running Windows because we have to run the Windmill tests against IE. In an attempt to keep things simple, I wanted to avoid installing cygwin on the machines, so I tried msysGIT. Oh WAIT I have to get ssh to work before I can actually use GIT to pull down the repo. 

After trying and failing with openSSH, I finally realized that the Windows puTTY Package was the best way to go about this. There was much frustration involved with this process because it requires that you take the ssh key you generated on your mac (and had added on the server) and convert it to a puTTY ppk. Fortunately this turns out not to be that panful using puTTYgen. The next piece to the puzzle was to get pageant to load this key automatically when the machine boots. I went through some rigamarole trying to create a shortcut in the “Startup” items and appending a string to it’s path to get it to load the correct key. This didn’t work, I’m sure it was a combination of Windows being terribly un user friendly and my brain expecting things to “Just Work TM”. Finally I just created a short cut for the actual key and stuck it in the “Startup” folder, and it works! (You still have to enter your pass phrase on boot) but its better than the alternatives. I actually found a post on a puTTY forum where someone was asking to automate this piece too and the response was basically, “No, never, die. Doing that defeats the purpose of SSH”. 

One step that I forgot to mention is that to get GIT to use the right SSH key you have to set the environment variable: GIT_SSH=path/to/putty/plink.exe

Next I went and started my git clone, which kicked off perfectly. That wasn’t so bad! 46% later at 1.99GB of the retrieval process I received two fatal errors that looked similar to file system errors. I then did some Googling to find that msysGIT only supports repository’s up to 2GB. The answer on the forums for this problem was, “have a smaller repo”. This is a problem, so I head back to the GIT homepage, where I find that the only real lasting option is to use cygwin GIT. 

After I get cygwin GIT all installed I finally get the repo fully downloaded I check to see if I can use the msysGIT to pull changes (which should never be 2gb worth), and it absolutely freaks out. So that’s not an option, and neither is adding cygwin/bin to your path because that complains about ‘exec ssh’ not existing.

Your probably wondering, why don’t you just use cygwin to do your GIT stuff? Well the issue is that we are using hudson to queue up jobs on the machines using the cmd environment to pull changes and run the tests in the repo. And it became very clear that the best way to handle this was to run the slave agent from cygwin, that way the jobs are actually running in cygwin land and this turned out to work — awesome!

To finish describing my pain, the last problem was that since I am running the tests from c:\main, (you have to be inside a git repo to pull and I want to keep these jobs simple) I am generating the log file in the wrong place, because the job looks in it’s home directory to find the jUnit Compatible XML results file. Fortunately there are some environment variables that I can use to get the files back where I want; mv *.log %WORKSPACE%

Now that life is back to normal and I’m getting used to living in the new GIT world, I can get back to important things like making Windmill more awesome and increasing test coverage.

Cheers!

Share This Post

Windmill Gets a Facelift for 1.0Beta1

19 November, 2008 (08:11) | Firefox, JavaScript, Mozilla, Open Source, Python, Technology, Web, Windmill, automation, windmill-dev | By: adam

Working up to the Windmill 1.0 Beta 1 Release, I finally had the opportunity to put some time into making the IDE (that a lot of you live in when in test writing mode) a little bit nicer to look at.

The IDE has been growing organically since 0.1 and there was a lot of functionality hacked into it that wasn’t in the original game plan, so I did what I could to improve the beauty of the CSS/Layout as well as the whole mess of code behind it.


Launching
If you have used our latest release, or are running trunk you know that we have significantly improved the load times for the Windmill IDE. By compressing the JavaScript when the service is instantiated we can simply hand the IDE window one file that contains the vast majority of the required code.

The reason that this makes such a huge performance difference is because we are loading the source via the local windmill proxy and the data size size had very little impact, the overhead was in the browser two connection limit. When you have to pull down ~30 files two at a time it takes its toll and made the IDE feel very sluggish and more like a web page loading than an IDE.

In the process of figuring out exactly what was slowing down the launch time we added some more informative messages and output so you don’t sit there staring at a twirling circle graphic wondering if anything is happening. And to make the experience even more fun, I couldn’t help but implement a progress bar.

General Layout
I removed the toolbar at the bottom of the screen, which I felt it was an irritation for test editing (especially with the drag and droppable actions). It is now in a drop down menu at the top right of the screen, with the rest of the UI access to IDE functions.

Settings and Firebug Lite Improvements
The settings dialog has continued to improve by implementing more useful defaults, adding new options, removing deprecated options and simply making it just look better. Thanks jQuery UI!

Firebug Lite has been a very popular feature since we first announced it, which has led to a handful of bug fixes over the last month. The most major of these was that the initial Windmill implementation of Firebug Lite required you to have Internet access as it was using resources that were hosted elsewhere.

These have since been copied to our source tree and are made available by the Windmill server so you can happily introspect your Web Apps JavaScript while writing tests on your Intranet.


Output and Performance
Instead of writing all the raw windmill output to the output and performance tabs there is now an array called windmill.errorArr, where all terrible errors and warnings about technical details are pushed in the case you are interested to see all that data. However, it’s more likely that you aren’t and scrolling through all that output data becomes tedious.

This is why we have implemented output in blocks with the background color representing pass/failure with green/red (white for performance). These blocks are expandable, clicking them will reveal all output (or performance information) we know about the action that was executed. This should give you a faster general overview of your results and allow you to quickly see the details you care about.


Other Worthwhile Mentions
We moved our XPath implementation from Ajax-Slt to JS-XPath, which has proven to be more accurate when it comes to resolving XPath generated in Firefox (or using Firebug) against non XPath native browsers such as IE.

Many bugs and improvements have been made to the DOM Explorer, which should now feel a lot more like the Firebug DOM inspector, but should work in any browser.

We have also put a lot of effort into improving the communication between the JavaScript Controller and the Python Service so that when a test fails you get as much detailed information in the service as you do in the IDE.

Timing and MozMill
The timing has lined up nicely as we are working on both a 1.0 release for Windmill and MozMill. MozMill is geared towards automated testing of all applications on the Mozilla Platform and functions in the trusted space providing lots of very useful flexibility.

You can currently try out MozMill 1.0rc1 as a Firefox Add-on, and keep your eyes pealed as some exciting new MozMill feature work is around the corner.

Participate
We are always trying to make life easier for the test writer, so please log your bugs and feel free to come chat with us in #windmill on FreeNode.

Share This Post