Friday, January 30, 2009

Making your Smart Device Unit Tests run on the desktop

In spite of Smart Device Unit Testing in Visual Studio 2008 is working really good, you may want faster test runs for doing Test-Driven-Development. Actually, that’s what I want :). In that case a very good option is to perform small TDD cycles running the tests on the desktop and run them on the device after some significant progress. But, how can we make our tests run on the desktop?

Adding a Desktop Test Run Configuration

The first step is quiet easy, you just need to add (if you don’t already have one) a new .testRunConfig file. Right click on the solution and Add – New Item – Test Run Configuration file. I like to call it “Desktop.testrunconfig”. The default configuration should work, so closing the properties dialog should be enough.

The second step is even easier, we need to select Desktop.testrunconfig as the active configuration, so, go to the Test menu, Select Active Test Run Configuration and click on Desktop (desktop.testrunconfig).

image

You are now ready for run your tests on the desktop!. Assuming that all your tests pass on the device, hopefully, all of them will also pass on the desktop. But, that was not my case! My project includes 12 tests passing on the emulator, but 5 of them don’t pass on the desktop. What’s going wrong?

Handling Windows Mobile specific functionality

As we’re developing a mobile application, our unit tests may include some windows mobile specific functionality which cannot be run on the desktop. It can be using a Windows Mobile API for trying to access the Outlook Mobile Contacts, checking if a new appointment has been scheduled, sending an SMS message or checking the battery level. We need to identify those scenarios and avoid them when running on the desktop.

In two of my failing tests, the tested code uses an API which doesn’t exist on the desktop. To avoid it from failing on the desktop, I need to place it inside an IF statement asking for the current platform and executing it only if it’s the expected platform. You can use the following code snippet as a way to do it:

[TestMethod]
public void testName()
{
if (Environment.OSVersion.Platform == PlatformID.WinCE)
{
// Test body here
// ....
}
}

ExpectedException fails!


The other 3 failing tests are verifying cases where an Exception is expected. All of them use the ExpectedException attribute for this purpose, and it’s ignored when you run a smart device unit testing project on the desktop.


You have two options here: 1. Avoid the tests when running on the desktop as we did before 2. Replace ExpectedException with an alternative approach. As I want to keep most of the test running on the desktop as possible, I prefer the second one.


Actually ExpectedException already has some disadvantages even besides this smart device / desktop running issue. With ExpectedException we can not check that an specific line of the test is the responsible for throwing the exception, or we cannot validate any additional info of the exception, like the message or an InnerException, we just can check for an exception type. So, this is not only a problem, this is also an improvement opportunity, and I like the XUnit approach for this, with the Assert.Throws method. So, why we cannot just implement something similar as follows:

public static class ExtendedAssert
{
public delegate void ThrowsDelegate();

public static Exception Throws<exceptionType>(ThrowsDelegate target)
where exceptionType : Exception
{
try
{
target();
Assert.Fail("No Exception was thrown.");
}
catch(exceptionType ex)
{
return ex;
}

return null;
}
}

Now we can replace a test like this:

[TestMethod]
[ExpectedException(typeof(ArgumentOutOfRangeException))]
public void BackgroundSizeSmallerThanSizeThrowsArgumentOutOfRangeException()
{
Desktop desktop = new Desktop();
desktop.Size = new Size(20, 20);
desktop.BackgroundSize = new Size(5, 5);
}

With something like this:

[TestMethod]
public void BackgroundSizeSmallerThanSizeThrowsArgumentOutOfRangeException()
{
Desktop desktop = new Desktop();
desktop.Size = new Size(20, 20);
ExtendedAssert.Throws<ArgumentOutOfRangeException>(
delegate { desktop.BackgroundSize = new Size(5, 5); });
}

Now, the test pass on the device and also on the desktop, and at the same time, it’s clear what line should throw the exception. Enjoy!

2 comments:

Michał said...

Hi, I've tried this approach and I get tests failing with the message:

Unable to set TestContext property for the class [ClassName] Error: System.ArgumentException: Object of type 'Microsoft.VisualStudio.TestTools.TestTypes.Unit.UnitTestAdapterContext' cannot be converted to type 'Microsoft.VisualStudio.TestTools.UnitTesting.TestContext'..

So far the only solution was to create a separate project referencing the desktop Microsoft.VisualStudio.QualityTools.UnitTestFramework.dll library and add links to the test files.

Do you have any idea on how to approach this issue with a single project?

Anonymous said...

Hi,
Very informative blog. One off topic question - what tool you use to take those cool screenshots with unique border like tore page. Could you please let me know ?

thanks,
curious1