Feeds:
Posts
Comments

Posts Tagged ‘delegate’

Howdy folks,

So far, we have seen how to let Pex automatically generate unit tests, and how to generate PUT customizations while testing interfaces.

However, there are a few cases you might come across, where Pex cannot generate test cases for your code. This is the case, if you try to test code that is:

  • Environment dependent
  • Dependent on sealed classes, static methods, non-public constructors, etc.

Since these might represent non-deterministic outcomes, Pex is not able to generate test cases.

To cite the “Testability” slide of the Tutorial “Parametrized Unit Testing: Principles, Techniques, and Applications in Practice” (Tillmann, de Halleux, Schulte, Xie):

Pex understands managed .NET code only.

Pex does not understand native code.

Examples for such method calls might be:

  • Machine-dependent dates, such as calls to DateTime.Now
  • Methods that read or write from the network, hard drive, databases, etc…

Alright, so there is untestable code, but we want to test it anyway. The good news is: We can do it. Pex works tightly together with a framework called Moles (since January 15th, formerly known as Stubs framework)

1.1 A Practical Example of the Problem

We’ll show by example where the problem is: We change our Store class from the post {TODO. PEX 1}, because we also want to output the date on which the delivery started. The resultung method can be seen in Figure 1:

   1: public void DeliverToAddress(String address, Book book)

   2: {

   3:     String deliveryMsg = "Book w/ title" + book.Title +

   4:                          "delivered to " + address +

   5:                          "on " + DateTime.Now;

   6:     Console.WriteLine(deliveryMsg);

   7: }

Snippet 1: Change of the DeliverToAddress method.

Let’s look at line 5: We access DateTime.Now. Everything’s fine, you say? But not for Pex, since DateTime.Now is machine-dependent. Let’s have a look at what Pex generates, so. Right-click on DeliverToAddress and select what? Exactly:

Run Pex.

A look at the Pex UI immediately tells us, that there is something wrong with the generation of the test cases. A button appears stating that there are 4 uninstrumented methods. Figure 1 shows  the Pex UI and the newly appearing button:

image

Figure 1: Uninstrumented code/testability issues when using native code.

A click on the button reveals there have been calls to machine-time-dependent methods, which cannot be tested by Pex. Obviously, these method were (implicitly) invoked by the call to DateTime.Now. The result is shown below:

image

Figure 2: List of the testability issues, after hitting the 4 Uninstrumented Methods – Button

1.2 A Possible Solution: Detouring

Now, in order to make such code testable, the principle would be, to replace such methods with a delegate. In our case, the call to File.Open() could be replaced like that:

DateTime.Now = () => new DateTime(2010, 1, 20);

Unfortunately, this is simply not possible. Therefore, the Pex team came up with a workaround: Moles. Since we cannot replace the original classes and their methods, we construct our own classes and methods that will do the necessary detouring. There is a simple rule about the naming convention:

Prefix M + Original class name + . + Original method name + Suffix;

Now, with our own class, we can do the passing to a delegate which returns a deterministic result. In our case, DateTime.Now would become:

MDateTime.NowGet = () => new DateTime(2010, 1, 20);

Short and sweet: For every class (or struct), we can produce a Mole Class, which performs a detouring of the original call to the value we assign using a lambda expression (or anonymous method).

1.3 Generation of the Mole Classes

Do we have to generate the classes on our own? Nope. We basically create an XML file with the .stubx file extension, which states for which classes there should be generated a Mole (or a Stub). And the good news is:

The framework will generate the actual stubs and moles!

Here is how we are going to do it:

  • Right-click BookStoreTest –> Add New Item –> Stubs and Moles for Testing
  • Leave the proposed name as it is (for now) –> Add

image

Figure 3: Selecting the Stubs & Moles for Testing template

The generated Assemblies1.stubx will look like follows:

   1: <Stubs xmlns="http://schemas.microsoft.com/stubs/2008/">

   2:   <Assembly Name="Assemblies1" />

   3: </Stubs>

Snippet 2: The generated .stubx file, referring to a nonsense Assembly, Assemblies1.

As a next step, we need to tell via the .stubx file (which is nothing more than an XML file), for which classes, in which Assembly etc. we want to generate Moles. The following code snippet shows you how to do this:

   1: <Stubs xmlns="http://schemas.microsoft.com/stubs/2008/">

   2:   <Assembly Name="mscorlib" />

   3:   <StubGeneration Disable="true"></StubGeneration>

   4:   <MoleGeneration Disable="false">

   5:     <TypeFilter Namespace="System" TypeName="DateTime"/>

   6:   </MoleGeneration>

   7: </Stubs>

Snippet 3: The .stubx file defining the classes for which we want Moles to be generated.

Assemblies1.stubx, Explained (line by line)

1: The Stubs tag only defines that we are dealing with Stub/Mole generation

2: The Name attribute specifies the Assembly, in which our “problem class” resides in: DateTime belongs to mscorlib, so with add this one.

3: We turn Stub generation off, since it is not needed at this point

4: MoleGeneration: We specify our desired class by adding a <TypeFilter> tag, containing the namespace of our class and the class name.

Again, simple, right? Basically, we specified only three things:

  • Assembly
  • Namespace
  • Class name

That’s it. Once we save our file, magic happens and the Moles framework generates the Mole classes in the background. The output window shows us the current progress of the Mole class generation:

image

Once the generation is done, we can proceed to the final step: Using the Mole class to detour the actual call to DateTime.Now to our newly generated MDateTime.NowGet, or in other words: Let’s see the Mole class in action!

Therefore, we need a little modification to the StoreTest.DeliverOrders (which will then implicitly call DeliverToAddress, multiple times):

   1: [PexMethod]

   2: public void DeliverOrders([PexAssumeUnderTest]Store target)

   3: {

   4:     int count = 0;

   5:     var storage = new SIStorage()

   6:     {

   7:         DeliverToAddressStringBook = 
              (String address, Book book) => count++;

   8:     };

   9:

  10:     MDateTime.NowGet = () => new DateTime(2010, 1, 20);

  11:

  12:     target.DeliverOrders();

  13: }

Snippet 4: Excerpt of our changed DeliverOrders testing method.

The important line is line 10:

We added the Mole Class, which will now detour all calls to DateTime.Now to MDateTime.NowGet and eventually produce a new DateTime.

If we wanted to tell Pex that we want to test the code not for a specific DateTime, as shown in Snippet 4, line 10, we simply could elevate our DateTime to a parameter and pass it in the lambda expression instead of new DateTime(2010, 1, 20);

So, we are actually done for this time.

One more thing: In order to successfully be able to reference the newly generated MDateTime class, you must make use of the following namespace:

   1: using System.Stubs;

Snippet 5: Make use of the correct namespace in order to make the Mole classes visible.

Epilogue

That’s it!

I hope this short article could give you an idea of:

  • Which code can “normally” be tested using Pex
  • What you have to do when Pex encounters untestable code
  • That untestable means no more untestable

Enjoy the testing and have fun exploring the capabilities of Pex and the Mole framework!

Stay tuned for the next time, when we’ll talk about Code Contracts!

Best regards,

Martin

Advertisements

Read Full Post »

Howdy,

In reference to my previous post, Pex: Parametrized Unit Tests and their Customization, I’d like to publish an update:

I received an interesting comment by Peli, that the following code piece should be replaced by a lambda expression, in order to make it more readable. Hence, I’d advice that you should just replace the following snippet:

  28:                 DeliverToAddressStringBook =
  29:                 delegate(String address, Book book)
  30:                 {
  31:                     count++;
  32:                 }
  

with the following:  

 

28:    DeliverToAddressStringBook = (address, book) => count++;

 

And in fact, using the lambda expression we could reduce the amount of code, hence making it more readable! Thanks, Peli!

Best regards,

Martin

Read Full Post »