On my last few contracts I have advocated acceptance testing through UI automation, but one of the most common wishes expressed by those coding up tests was for some way to validate that our automation wrappers actually still reflected the control they wrapped in the application under test.  Something as simple as renaming a button would break our tests.  It wasn’t until the next scheduled run of our acceptance tests that the breakage would be identified.

As mentioned in my last post I am working on a Windows Automation Toolkit (WATKit), I haven’t finished it yet but I figured an early feature should be some kind of build time validation to resolve the above issues.  As of today WATKit includes an MSBuild Task and a few attributes to do just that.

WATKitBuildTask

The build task basically compares types in a test assembly that are decorated with AutomationTypeMappingAttribute against elements on types in one or more source assemblies.  The test assembly and source assemblies are specified in the build configuration file (aka VS 2010 project file).  Here is the what I added to the WATKit.Tests.csproj file in the WATKit source available on GitHub:

   1: <UsingTask TaskName="WATKIt.Build.WATKitBuildTask"

   2:             AssemblyFile="$(TargetDir)\WATKit.dll"/>

   3:   <Target Name="AfterBuild">

   4:         <WATKitBuildTask TestAssembly="$(TargetDir)WATKit.Tests.dll"

   5: SourceAssemblies="$(SolutionDir)\WATKit.TestApp.WPF\bin\Debug\WATKit.TestApp.exe" />

   6:         <WATKitBuildTask TestAssembly="$(TargetDir)WATKit.Tests.dll"

   7: SourceAssemblies="$(SolutionDir)\WATKit.TestApp.WinForms\bin\Debug\WATKit.TestApp.exe" />

   8:   </Target>

So the first thing it does is reference the build task with a <UsingTask /> element.  With this in place the task can be referenced as an element within any <Target /> element.  I simply uncommented the AfterBuild target that is included in most project files.  I have the task running to validate against the two test apps in the solution.

The build task has two required properties:

TestAssembly points to the assembly containing the wrappers to be validated.  It doesn’t have to be a test assembly, if you keep your wrappers in a separate class library then it is this library you should reference.

SourceAssemblies is a comma separated list of one or more assemblies that contain real controls and windows.  It works with WPF and WinForms applications (I haven’t tested it on other project types, I don’t see any reason it won’t work with Silverlight 4.* assemblies, but I haven’t tested it yet)

AutomationTypeMappingAttribute

In the test assembly only types with the AutomationTypeMappingAttribute are checked.  Each Property on these types is checked unless it is decorated with an IgnoreAttribute.  Methods and Fields are implicitly ignored.

This attribute has a single constructor argument that must be a fully qualified name of the type to validate against in one of the source assemblies.  For example the MainWindow wrapper control in the WATKit.Tests project is declared like this:

   1: [AutomationTypeMapping("WATKit.TestApp.MainWindow")]

   2: public class MainWindow : Window

   3: {

   4: }

This tells the build task to validate MainWindow in the test assembly against the first WATKit.TestApp.MainWindow it finds in the source assemblies (the search is carried out in the order assemblies are listed)

IgnoreAttribute

By default the build task will check every Property on the wrapper type unless you tell it otherwise by decorating a Property with this attribute.  You can optionally specify a Reason for ignoring it, so that others will know why.  For example the DynamicButton property of MainWindow mentioned above looks like this:

   1: [Ignore(Reason = "Button does not exist until run time")]

   2:  public Button DynamicButton { get { // removed for brevity } }

Many properties in the base classes in WATKit are decorated with this attribute to avoid build failures.

AutomationMemberAttribute

By default the build task will look for an exact match of the wrapper Property name with the name of a Field (child controls in WPF and WinForms are exposed as fields not properties) in the type it is validating against.  If for some reason the names do not match you can decorate the wrapper property with this attribute and specify the name to match.  For example the ChangeMyNameButton property of MainWindow mentioned above looks like this:

   1: [AutomationMemberMapping("IChangeMyNameButton")]

   2: public Button ChangeMyNameButton { get { // removed for brevity } }

This tells the build task to validate the ChangeMyNameButton property against a field named IChangeMyNameButton.

That’s it, no rocket science but it does the job and based on my experiences should help to give early feedback about breaking changes in applications being tested via UI Automation.  The build task is not limited to use with tests that use features of WATKit, it should work with any automation framework as long as you create wrappers for your controls and those wrappers expose properties representing child controls and elements exposed by the real types.

If you have any feedback or suggestions for improving this please let me know by commenting here or over at GitHub.

For those who don’t like reading long posts:  I have created a Fluent API for automated UI testing based on components in the System.Windows.Automation namespace that ship with .NET 3.0.  I have posted the early code for it on GitHub at https://github.com/MikeHanson/WATKit.  It will be fully documented on the GitHub Wiki and I will post on it here.  If you want to know how I got to this point and why I am creating another UI Automation API read on.

After an aborted attempt to retire early I am back in a new contract in central London working as part of an agile team on a WPF fat client application.  As is common in agile teams developers have to code acceptance tests that automate the UI.  As earlier posts demonstrate I have some experience of this with web apps and Silverlight but I had never had to do it with a WPF app.  Coincidentally I have started work on a desktop client for a community site I run at www.vrcyclist.com and I am doing so using BDD with SpecFlow.  Ranorex is the platform of choice at work, but this a commercial product that is outside of my budget, so I started looking at free/open source WPF automation options in my spare time.

Since v3.0 the .NET Framework has included an automation framework in the System.Windows.Automation namespace.  This is all packaged in a set of assemblies in C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\v3.0 beginning with UIAutomation.  These are not specifically for automated UI testing but make it possible.  They aren’t difficult to use but the code is quite verbose and almost immediately I wanted a more developer friendly API.  I already knew of an open source project on Codeplex called White that I had reviewed for use with Silverlight.  I looked for others but couldn’t find anything that wasn’t commercial.  So I downloaded White and started wiring it into my acceptance tests.

As far as launching my application and finding windows and controls went it worked fine, and looked promising, but, my very first really simple attempt to create a CustomUIItem failed miserably.  If you have read any of my articles on WebAii you will know I recommend creating wrappers for windows, views and controls in your application and have your tests drive these wrappers rather than finding individual elements repeatedly.  The CustomUIItem base class is mean to be the way to do this with White.  I followed a simple example in the documentation to create a wrapper for a WPF ValidationSummary control I had created, it should have worked but White could not find my control.  I posted a question on the White Codeplex site and got a couple of responses from the original author.  Google turned up a number of people with the same problem and offering workarounds but to be honest I didn’t see the point of using White for the workarounds you could achieve the same thing without the CustomUIItem base class.  Anyway not being a patient person, by the time I got a response to my post on Codeplex I had already started writing my own little API to make working with the Microsoft UI Automation framework easier, mostly a set of extension methods that made code less verbose (the route a number of the Google results indicated others had taken).

As indicated by my previous post I have become a fan of Fluent APIs like Fluent Assertions and at work I am using Fluent NHibernate for the first time.  As I developed my API I started seeing the evolution of a Fluent Automation API and the more I did the more I liked it.  Combined with Fluent Assertions my test code was reading like a Story and the more I saw this the more I liked it and have spent a significant amount of time in the last few weeks re-factoring what I had started into a more complete Fluent Automation API and will continue to do so until it is fully usable.  At first I was doubting the sense of creating another API rather than working to figure out the kinks in White, but the fluent aspect of the API changed this as I am fairly sure it makes it unique (at least amongst public offerings) and I am really enjoying working on it at the moment, so will stick with it to completion.

Enough of the blabbering, here are some tasters of what my Fluent Automation API looks like:

   1: var uat = Fluently.Launch("C:\MyApp.exe")

   2:                 .WaitUntilMainWindowIsLoaded()

   3:                 .WithDefaultMainWindow();

This is the start point and launches the application under test returning the default wrapper for the main window of the application.

   1: var uat = Fluently.Launch("C:\MyApp.exe")

   2:                 .WaitUntilMainWindowIsLoaded()

   3:                 .WithMainWindowAs<MyMainWindow>();

This is my preferred alternative that allows you to specify that a strongly typed wrapper is used for the main window.

   1: var button = aut.MainWindow

   2:         .FindControl()

   3:         .WithId("MyButton")

   4:         .IncludeDescendants()

   5:         .Now()

   6:         .As<Button>();

   1: button.Wait()

   2:     .UntilExists()

   3:     .TimeoutAfter(TimeSpan.FromSeconds(5), true);

This is how you would get a proxy to wait for the button to exist.  The second argument to TimeOutAfter indicates an exception should be thrown on timeout. The default is not to throw an exception but you can check the state of the button to identify if it is still a proxy or the real thing.

Well that is enough for now, let me know what you think and feel free to contribute ideas and comments here or on GitHub at https://github.com/MikeHanson/WATKit

NB: After I started this post Telerik posted an update to the WebAii framework that included WPF support.  I took a quick look and it, and whilst it is a welcome addition the fluent aspect of my API is still unique so I will be sticking with it.