Feeds:
Posts
Comments

Archive for the ‘Unit testing’ Category

Howdy,

Once again about WADA, but this one will be short. I’ve added a new functionality, which is:

 

  • Filter W3WPs by related Identity (as of the application pool)

 

Thanks to my colleague Patrick  for suggesting this functionality!

The UI has changed a little, offering the new drop down for the identities:

WADA_1.1_Screenshot

The WADA 1.1 download is still available at the Visual Studio Gallery.

 

Please note: The Application Pool- and Identities-Filters are mutually exclusive.

 

Best regards,

Martin

Advertisements

Read Full Post »

Howdy people,

Me again with some more information of how both Code Contracts and Pex can be used together to write expressive tests (while still specifying a (partial) specification of the code). In the course of this article wel’ll also cover Assertions and Assumptions. In order to point out only the necessary topics, let’s introduce a new (very basic) sample project, and let’s perform the following steps:

  • Create a console application called MusicStore
  • Create the class Artist (as of Figure 1)
  • Create the class Album (as of Figure 2)
  • Enable Code Contracts checking (both static and runtime, as of figure 3)

clip_image002

Figure 1: The Artist class of the MusicStore project.

clip_image004

Figure 2: The Album class of the MusicStore project.

clip_image006

Figure 3: Enabling both Runtime Checking (Full), and Static Checking

Important: Do not forget to also check the Implicit Non-Null Obligations, as of Figure 3

The only critical point for CC should be the SellAlbum method of the Artist class. Why? Simply, because there we might have a parameter of type Album that could be NULL. And the Static Checker gives us the correct hints to it, as expected:

clip_image008

Figure 4: Expected CC’s Static Checker suggestion and warning.

So we have one suggestion to put a precondition, and an effective warning that we might be passing a parameter that could be NULL. This was predictable.

Running Pex over the method SellAlbum reveals the following results (If you need a starting point for Pex, please look at my article First steps with PEX: Automated White box Testing for .NET)

clip_image010

Figure 5: Pex results for 3 test cases, one providing the album parameter = NULL. Please note also the typical Pex exploration results’ table showing one column per parameter (in this case only album ).

What happens if we put an assertion to this critical parameter? Let’s simply add one, as shown in Figure 6:

clip_image012

Figure 6: Adding an assertion using a call to the static method Contract.Assert(bool condition) of the CC framework.

It makes sense to assert a number of AlbumsSold higher or equal to 1 (after 1 album has been sold), right? Does putting an assertion have an impact on Pex? Of course! Let’s run it again to see how:

clip_image014

Figure 7: Effects on Pex of using Code Contracts’ Assertions. Pex clearly identifies this as an assertion failure, hence letting the generated test case fail. Line 2 (highlighted the additionally created Pex test for the Code Contracts assertion).

So far, so good. Let’s look at Figure 6 again. We can also put such an assertion for the album object. This clearly makes sense – since selling an album should increase the album’s TimesSold count. Let’s do this in Figure 8:

clip_image016

Figure 8: Added another assertion for the album.TimesSold property.

And immediately, let’s look at the Static Checker warnings and the generated Pex test cases (Figures 9 and 10):

clip_image018

Figure 9: One more unproven assert warning for our newly added Assertion in the SellAlbum method. This was expected. Let’s run Pex again!

clip_image020

Figure 10: Line 3 shows the newly generated (and failed) test case of Pex. Again, this was expected.

Great coverage – but What About the Meaning?

This is a good question. Is this still an expressive test? Let’s recap the meaning. We could have an Album -type parameter passed, which could be theoretically NULL. Would it make any sense to then test its property TimesSold? I am sure, we can agree on that this would not be the case.

So, what we need, is the possibility to prevent Pex from generating test cases in such a case, hence producing only meaningful tests. And who is going to provide us with such a capability?

Code Contracts’ Assumptions

Important: Assumptions only work when the Full contract option is checked in the Code Contracts pane (under our project’s properties)

Figure 11 gives a brief example of how to put such an assumption into our code of the SellAlbum method.

clip_image022

Figure 11: Creating assumptions.

Figure 11 needs some explanations:

The first three assumptions simply protect us from getting the current object, the passed album, or the passed album’s artist as NULL. Makes sense, so far. The following two assumptions ensure that this.AlbumsSold and album.TimesSold both are non-negative values.

(Note also that now we put our assertions INSIDE the artist comparison’s block – makes sense, since we do not need to assert when there happened no increase of AlbumsSold and TimesSold properties).

What do We Expect?

Well, we pretty much thought at everything, didn’t we? So we expect Pex to not issue any errors or failures again. A simple re-run of Pex will show us the result, as can be seen in Figure 12.

clip_image024

Figure 12: Unexpected error on line 3, a failed assertion.

An unexpected failure? How could that happen? Let’s look at what the parameters looked like in the case of failure:

image

Did you see it? AlbumsSold is equal to int.MaxValue ! I guess, I do not need to explain what happens, if you try to increase an integer (in SellAlbum ) which is already equal to its maximum value! But why did that happen?

Simply put, this is one of the boundary test cases we explained in First steps with PEX: Automated White box Testing for .NET article. Pex guesses from the type of variables which are the boundary values (and also intermediate values).

To summarize shortly & sweetly today’s lecture:

Code Contracts & Pex: meaningful, expressive tests

(by preventing test case generation for assumed parameter values)

Let’s stop here today. There are more interesting topics regarding Code Contracts and Pex, and I am quite sure this will keep us busy for another while!

Enjoy playing around with Pex and Code Contracts, and see you next time!

Best regards,

Martin

Read Full Post »

Howdy all,

Here comes another post about Code Contracts. Remember my post First Steps with Code Contracts?

Let’s just recall which class we have been working on last time:

   1: namespace Vehicles

   2: {

   3:     class Car : IVehicle

   4:     {

   5:         string name;

   6:         int speed;

   7:

   8:         public Car(string name, int speed)

   9:         {

  10:             this.name = name.ToUpper();

  11:             this.speed = speed;

  12:         }

  13:

  14:         void IVehicle.Drive(int speed)

  15:         {

  16:             Console.WriteLine("Car " + name +

  17:             " is driving at a speed of " + speed);

  18:         }

  19:

  20:         public bool IsFasterThan(Car c2)

  21:         {

  22:             Console.WriteLine(name.ToUpper() +

  23:             " is faster than "

  24:             + c2.name.ToUpper());

  25:             if (this.speed > c2.speed)

  26:                 return true;

  27:             else return false;

  28:         }

  29:

  30:         [ContractInvariantMethod]

  31:         void ObjectInvariant()

  32:         {

  33:           Contract.Invariant(name != null);

  34:         }

  35:     }

  36: }

Snippet 1: The Car class from the First Steps with Code Contracts example.

After completing all the steps from that example, in the end we should have ended up with at least one warning generated by Code Contracts:

Code Contracts: Possibly calling a method on a null reference ‘c2.name’

with respect to line 24.

Now, if we review the usage of Object Invariants (First Steps with Code Contracts), we would see immediately that the purpose of Objects invariants is to provide us a mechanism to perform the checking in ObjectInvariant() in all instances (and instances of subclasses) of Car.

It seems there are no assumptions generated for the Objects Invariants of objects of the same type passed as a parameter (at least not yet, maybe it’ll come).

The workaround:

In order to avoid the warning, we can do 2 things:

  • Assumption:Contract.Assume(!String.IsNullOrEmpty(name)) or
  • Postcondition: Create a property w/ getter postcondition ensuring the result is not null

That should about do the trick!

Enjoy and see you next time!

Martin

Read Full Post »

Hello all,

Today we are going to talk about another exciting project from the Microsoft Research Labs: Code Contracts. It is a Design-By-Contract (DbC) system, that enables us as developers to formulate the expected behaviour of our code by directly putting it into the code.

The main tools of applying Code Contracts are:

  • Static checking (checks contracts at compile time)
  • Runtime checking (checks contracts at runtime)
  • Automated documentation generation (keeps code and docs in sync)

Prerequisites

Before we get started using Code Contracts. I used the following environment for carrying out the tests with Code Contracts:

  • Windows 7
  • Visual Studio 2010 Beta 2 Ultimate*
  • Code Contracts Tools*

Important: I used Visual Studio 2010 Beta 2, because it contains the .NET framework 4, which already includes the Code Contracts framework. You could also use them with VS2008, if you installed the .NET framework 4. The Code Contract Tools need to be downloaded separately. I highly recommend them, since they will add a new pane to your VS2010 which in turn lets you set various options, like enabling the static checker directly from the UI.

1.1 The Strengths of Code Contracts

Elevating assertions to the API level. At this point, the specification of code is not only somewhere deeply hidden in the method, but immediately visible to the caller (if, for example, a Contract is not fulfilled). On the other side, a caller now can know perfectly what to expect from a method.

1.2 Code Contracts for Methods

First, we are going to show how to implement Code Contracts on the method level.

We’ll start with the creation of a test project that will serve our purposes of showing Code Contracts’ capabilities:

image

Figure 1: Add a new console application and name it Vehicles

As a next step, we create a class and an interface, namely Car and IVehicle. Implement them as follows in the snippets 1 and 2. Leave the automatically generated Program class – we’ll need it later for running our examples.

   1: using System;

   2: using System.Collections.Generic;

   3: using System.Linq;

   4: using System.Text;

   5:

   6: namespace Vehicles

   7: {

   8:     interface IVehicle

   9:     {

  10:         void Drive(Int32 speed);

  11:     }

  12: }

Snippet 1: Interface IVehicle, defining one method signature

   1: namespace Vehicles

   2: {

   3:     class Car : IVehicle

   4:     {

   5:         string name;

   6:         int speed;

   7:

   8:         public Car(string name, int speed)

   9:         {

  10:             this.name = name.ToUpper();

  11:             this.speed = speed;

  12:         }

  13:

  14:         void IVehicle.Drive(int speed)

  15:         {

  16:             Console.WriteLine("Car " + name +

  17:             " is driving at a speed of " + speed);

  18:         }

  19:

  20:         public bool IsFasterThan(Car c2)

  21:         {

  22:             Console.WriteLine(name.ToUpper() +

  23:             " is faster than "

  24:             + c2.name.ToUpper());

  25:

  26:             if (this.speed > c2.speed)

  27:                 return true;

  28:             else return false;

  29:         }

  30:     }

  31: }

Snippet 2: Class Car implementing IVehicle.

So far, so good, right? Nothing special there, and the code has got very simple “functionality”. A car that can drive, alright. And check whether it is faster than another car. Compiling the project reveals a succeeding build which at first sight contains no errors.

Before we take further steps, let’s activate the Static Checking feature of Code Contracts. In order to enable this, just right-click the project Vehicles -> Properties.

The last pane on the bottom of the Properties’ window is called Code Contracts. Go for it.

clip_image004

Figure 2: The Code Contracts’ configuration in Properties. Then proceed with Figure 3.

image

Figure 3: Enable static contract checking and the checking of implicit non-null obligations.

Compile again, and in the warning sections, we’ll receive a bunch of warnings generated by Code Contracts:

image

Figure 4: Warning list, generated by Code Contracts.

Those messages are suggestions of how we should change our code in order to be (more likely to be) safe from exceptions.

The warnings, line by line:

1-2: Tell us that we should add a pre-condition in order to be sure from NullReferenceExceptions when accessing the property name (1) or passing a parameter that could be possible equal to null (2). The cool thing is: Code Contracts even provide us with the correct code in order to achieve this.

3-5: These are the actual warnings, containing possible failures.

Preconditions

A precondition is simply added by using the following method:

Contract.Requires(condition)

When debugging, we can step over such a call and verify that it is actually executed. The call is made in the method body, at any desired position. Usually it is placed at the beginning.

Postconditions

A postcondition is simply added by using the following method:

Contract.Ensures(condition)

When debugging, we cannot step over such a call (to be precise: not at the position where we put it in the code), since it is not actually executed in the method body, but only afterwards. Once we stepped over the complete method, we arrive at the Contract.Ensures call. The following snippet show how pre- and postconditions can be applied to the methods of our class Car:

   1: using System;

   2: using System.Collections.Generic;

   3: using System.Linq;

   4: using System.Text;

   5: using System.Diagnostics.Contracts;

   6:

   7: namespace Vehicles

   8: {

   9:     class Car : IVehicle

  10:     {

  11:         string name;

  12:         int speed;

  13:

  14:         public Car(string name, int speed)

  15:         {

  16:             Contract.Requires(name != null);

  17:             this.name = name.ToUpper();

  18:             this.speed = speed;

  19:         }

  20:

  21:         void IVehicle.Drive(int speed)

  22:         {

  23:             Console.WriteLine("Car " + name +

  24:             "is driving at a speed of " + speed);

  25:         }

  26:

  27:         public bool IsFasterThan(Car c2)

  28:         {

  29:             Contract.Requires(c2 != null);

  30:             Console.WriteLine(name.ToUpper() +

  31:             " is faster than " +

  32:             c2.name.ToUpper());

  33:

  34:             if (this.speed > c2.speed)

  35:                 return true;

  36:             else return false;

  37:         }

  38:     }

  39: }

Snippet 3: The Car class, using Code Contracts’ preconditions

After we added the Contract.Requires calls in IsFasterThan and in the Car ctor, we receive only 2 warnings from Code Contracts. Just compile to verify that.

Why still two warnings? We added preconditions to check that!

The reason: Only our method IsFasterThan in this particular class, Car, requires name to be non-null. But we could have for example a subclass inheriting from Car, which does not perform the same check. Hence, we need to take care for that. And in fact, Code Contracts provides us the necessary means in order to ensure that no object deriving from Car, can have name which is null:

   1: [ContractInvariantMethod]

   2: void ObjectInvariant()

   3: {

   4:     Contract.Invariant(name != null);

   5: }

Snippet 4: Object invariants

Calling again reveals that the warning concerning the name variable of class Car has gone. Good! It means, we are safe from accessing a null value – variable, and all other programmers subclassing our Car are safe, too!

This is it for now! Next time we’ll see how to deal with Code Contracts and interfaces.

Enjoy playing with Code Contracts and see you soon!

Best regards,

Martin

Read Full Post »

Howdy folks,

So far, we have seen how to let Pex automatically generate unit tests, and how to generate PUT customizations while testing interfaces.

However, there are a few cases you might come across, where Pex cannot generate test cases for your code. This is the case, if you try to test code that is:

  • Environment dependent
  • Dependent on sealed classes, static methods, non-public constructors, etc.

Since these might represent non-deterministic outcomes, Pex is not able to generate test cases.

To cite the “Testability” slide of the Tutorial “Parametrized Unit Testing: Principles, Techniques, and Applications in Practice” (Tillmann, de Halleux, Schulte, Xie):

Pex understands managed .NET code only.

Pex does not understand native code.

Examples for such method calls might be:

  • Machine-dependent dates, such as calls to DateTime.Now
  • Methods that read or write from the network, hard drive, databases, etc…

Alright, so there is untestable code, but we want to test it anyway. The good news is: We can do it. Pex works tightly together with a framework called Moles (since January 15th, formerly known as Stubs framework)

1.1 A Practical Example of the Problem

We’ll show by example where the problem is: We change our Store class from the post {TODO. PEX 1}, because we also want to output the date on which the delivery started. The resultung method can be seen in Figure 1:

   1: public void DeliverToAddress(String address, Book book)

   2: {

   3:     String deliveryMsg = "Book w/ title" + book.Title +

   4:                          "delivered to " + address +

   5:                          "on " + DateTime.Now;

   6:     Console.WriteLine(deliveryMsg);

   7: }

Snippet 1: Change of the DeliverToAddress method.

Let’s look at line 5: We access DateTime.Now. Everything’s fine, you say? But not for Pex, since DateTime.Now is machine-dependent. Let’s have a look at what Pex generates, so. Right-click on DeliverToAddress and select what? Exactly:

Run Pex.

A look at the Pex UI immediately tells us, that there is something wrong with the generation of the test cases. A button appears stating that there are 4 uninstrumented methods. Figure 1 shows  the Pex UI and the newly appearing button:

image

Figure 1: Uninstrumented code/testability issues when using native code.

A click on the button reveals there have been calls to machine-time-dependent methods, which cannot be tested by Pex. Obviously, these method were (implicitly) invoked by the call to DateTime.Now. The result is shown below:

image

Figure 2: List of the testability issues, after hitting the 4 Uninstrumented Methods – Button

1.2 A Possible Solution: Detouring

Now, in order to make such code testable, the principle would be, to replace such methods with a delegate. In our case, the call to File.Open() could be replaced like that:

DateTime.Now = () => new DateTime(2010, 1, 20);

Unfortunately, this is simply not possible. Therefore, the Pex team came up with a workaround: Moles. Since we cannot replace the original classes and their methods, we construct our own classes and methods that will do the necessary detouring. There is a simple rule about the naming convention:

Prefix M + Original class name + . + Original method name + Suffix;

Now, with our own class, we can do the passing to a delegate which returns a deterministic result. In our case, DateTime.Now would become:

MDateTime.NowGet = () => new DateTime(2010, 1, 20);

Short and sweet: For every class (or struct), we can produce a Mole Class, which performs a detouring of the original call to the value we assign using a lambda expression (or anonymous method).

1.3 Generation of the Mole Classes

Do we have to generate the classes on our own? Nope. We basically create an XML file with the .stubx file extension, which states for which classes there should be generated a Mole (or a Stub). And the good news is:

The framework will generate the actual stubs and moles!

Here is how we are going to do it:

  • Right-click BookStoreTest –> Add New Item –> Stubs and Moles for Testing
  • Leave the proposed name as it is (for now) –> Add

image

Figure 3: Selecting the Stubs & Moles for Testing template

The generated Assemblies1.stubx will look like follows:

   1: <Stubs xmlns="http://schemas.microsoft.com/stubs/2008/">

   2:   <Assembly Name="Assemblies1" />

   3: </Stubs>

Snippet 2: The generated .stubx file, referring to a nonsense Assembly, Assemblies1.

As a next step, we need to tell via the .stubx file (which is nothing more than an XML file), for which classes, in which Assembly etc. we want to generate Moles. The following code snippet shows you how to do this:

   1: <Stubs xmlns="http://schemas.microsoft.com/stubs/2008/">

   2:   <Assembly Name="mscorlib" />

   3:   <StubGeneration Disable="true"></StubGeneration>

   4:   <MoleGeneration Disable="false">

   5:     <TypeFilter Namespace="System" TypeName="DateTime"/>

   6:   </MoleGeneration>

   7: </Stubs>

Snippet 3: The .stubx file defining the classes for which we want Moles to be generated.

Assemblies1.stubx, Explained (line by line)

1: The Stubs tag only defines that we are dealing with Stub/Mole generation

2: The Name attribute specifies the Assembly, in which our “problem class” resides in: DateTime belongs to mscorlib, so with add this one.

3: We turn Stub generation off, since it is not needed at this point

4: MoleGeneration: We specify our desired class by adding a <TypeFilter> tag, containing the namespace of our class and the class name.

Again, simple, right? Basically, we specified only three things:

  • Assembly
  • Namespace
  • Class name

That’s it. Once we save our file, magic happens and the Moles framework generates the Mole classes in the background. The output window shows us the current progress of the Mole class generation:

image

Once the generation is done, we can proceed to the final step: Using the Mole class to detour the actual call to DateTime.Now to our newly generated MDateTime.NowGet, or in other words: Let’s see the Mole class in action!

Therefore, we need a little modification to the StoreTest.DeliverOrders (which will then implicitly call DeliverToAddress, multiple times):

   1: [PexMethod]

   2: public void DeliverOrders([PexAssumeUnderTest]Store target)

   3: {

   4:     int count = 0;

   5:     var storage = new SIStorage()

   6:     {

   7:         DeliverToAddressStringBook = 
              (String address, Book book) => count++;

   8:     };

   9:

  10:     MDateTime.NowGet = () => new DateTime(2010, 1, 20);

  11:

  12:     target.DeliverOrders();

  13: }

Snippet 4: Excerpt of our changed DeliverOrders testing method.

The important line is line 10:

We added the Mole Class, which will now detour all calls to DateTime.Now to MDateTime.NowGet and eventually produce a new DateTime.

If we wanted to tell Pex that we want to test the code not for a specific DateTime, as shown in Snippet 4, line 10, we simply could elevate our DateTime to a parameter and pass it in the lambda expression instead of new DateTime(2010, 1, 20);

So, we are actually done for this time.

One more thing: In order to successfully be able to reference the newly generated MDateTime class, you must make use of the following namespace:

   1: using System.Stubs;

Snippet 5: Make use of the correct namespace in order to make the Mole classes visible.

Epilogue

That’s it!

I hope this short article could give you an idea of:

  • Which code can “normally” be tested using Pex
  • What you have to do when Pex encounters untestable code
  • That untestable means no more untestable

Enjoy the testing and have fun exploring the capabilities of Pex and the Mole framework!

Stay tuned for the next time, when we’ll talk about Code Contracts!

Best regards,

Martin

Read Full Post »

Howdy,

In reference to my previous post, Pex: Parametrized Unit Tests and their Customization, I’d like to publish an update:

I received an interesting comment by Peli, that the following code piece should be replaced by a lambda expression, in order to make it more readable. Hence, I’d advice that you should just replace the following snippet:

  28:                 DeliverToAddressStringBook =
  29:                 delegate(String address, Book book)
  30:                 {
  31:                     count++;
  32:                 }
  

with the following:  

 

28:    DeliverToAddressStringBook = (address, book) => count++;

 

And in fact, using the lambda expression we could reduce the amount of code, hence making it more readable! Thanks, Peli!

Best regards,

Martin

Read Full Post »

Hello,

You might already guess it: Pex won’t let me go, and so I’d like to explain in short a few of Pex’ concepts.

1.1 Code Coverage

Pex is an abbreviation for Program Exploration. You guessed it: Nomen est omen. It analyzes the branches (if, then, else, etc…) of control flow and tries to cover each and every branch. This leads to a highly elevated code coverage, or, in other words: It helps to increase the amount of tested code.

1.2 The Main Idea of Parametrized Unit Testing using Pex

…is to elevate any values that should not matter (i.e. that should not influence the program’s runtime behaviour) into parameters. Pex will then care about how to deal with them, about creating test cases for all possible values they can assume. Short & sweet: Pex cares about your parameters and helps to ensure there are no cases where your program would fail because of them.

1.3 The Nuts and Bolts of a PUT

Figure 1 gives you an example of what a Parametrized Unit Test (PUT) looks like:

image

Figure 1: A simple PUT

We just use the [PexMethod] attribute to denote that the following method is a PUT.

Sometimes we need some of the parameters to be null, in order to get a method running at all. In such a case, we can give Pex the instruction not to generate tests which would assume that parameter as NULL. We simply do this by adding the attribute

[PexAssumeUnderTest]

in front of the parameter type. Sounds straightforward, doesn’t it?

Within the PUT method, you can (as suggested by Pex upon generationg of the tests) multiple assertions to ensure the correct behaviour of the method. What Pex does for you, is the following:

  • Detecting dereferencing of potential null values (and generates a test case)
  • Analysis of e.g. loop boundaries (and generates test cases for each assumable value, including boundary values)

There is only one drawback: This way of testing only works as long as we have implemented method bodies. Are there cases where we don’t have them? Of course: Interfaces. Let’s see how Pex deals with interfaces.

1.4 Extending our Example

In order to see how Pex generates test code for interfaces, we need to add an interface to our project BookStore (which we created in Pex – Automated White Box Unit Testing).

   1: using System;

   2: using System.Collections.Generic;

   3: using System.Linq;

   4: using System.Text;

   5:

   6: namespace BookStore

   7: {

   8:     public interface IStorage

   9:     {

  10:         void DeliverToAddress(String address, Book book);

  11:     }

  12: }

Snippet 1: The IStorage interface.

As a next step, we are going to use it somewhere, namely: We let our class Store implement IStorage.

public class Store : IStorage

Please note that the IStorage member method DeliverToAddress(…) has already been implemented in our previous post. Hence, the code should compile as soon as you implemented the changes.

What comes next? Pex cannot possibly know about our newly created interface, so we need to tell it somehow to generate stubs for the interface. And how do we do that?

Re-run Pex!

For that, right-click on the project BookStore and select Run Pex from the context menu. Pex generated the following class:

   1: public partial class SIStorage : IStorage

   2: {

   3:     Action<string, global::BookStore.Book> DeliverToAddressStringBook;

   4:     void DeliverToAddress(string address, Book book)

   5:     {

   6:         return DeliverToAddressStringBook(address, book);

   7:     }

   8: }

Snippet 2: The generated stub class for the IStorage interface: SIStorage.

As shown in Snippet 2, the generator creates a class named S + <interface name>, which implements the interface. For each method defined in the interface, a property (a delegate, in fact)will be added which resembles the method’s signature (method name + parameters’ types, camel case).

1.5 Customizing the stub

Now we’ll dig a little deeper. We modify our DeliverToAddress method from Snippet 1. Why do we do this? We simply double-check that the amount of ordered books is equal to the amount of the actually delivered book.

   1: // <copyright file="StoreTest.cs" company="Scientific Network">Copyright © Scientific Network 2010</copyright>

   2:

   3: using System;

   4: using BookStore;

   5: using Microsoft.Pex.Framework;

   6: using Microsoft.Pex.Framework.Validation;

   7: using Microsoft.VisualStudio.TestTools.UnitTesting;

   8: using BookStore.Stubs;

   9: using System.Collections.Generic;

  10:

  11: namespace BookStore

  12: {

  13:     [TestClass]

  14:     [PexClass(typeof(Store))]

  15:     [PexAllowedExceptionFromTypeUnderTest(typeof(ArgumentException),

  16:                                     AcceptExceptionSubtypes = true)]

  17:     [PexAllowedExceptionFromTypeUnderTest(

  18:             typeof(InvalidOperationException)

  19:     )]

  20:     public partial class StoreTest

  21:     {

  22:         [PexMethod]

  23:         public void DeliverOrders([PexAssumeUnderTest]Store target)

  24:         {

  25:             int count = 0;

  26:             var storage = new SIStorage()

  27:             {

  28:                 DeliverToAddressStringBook =

  29:                 delegate(String address, Book book)

  30:                 {

  31:                     count++;

  32:                 }

  33:             };

  34:             target.DeliverOrders();

  35:             Assert.AreEqual(CalculateExpectedDeliveries(target.Orders), count);

  36:             // TODO: add assertions to method StoreTest.DeliverOrders(Store)

  37:         }

  38:

  39:         private int CalculateExpectedDeliveries(List<Order> orders)

  40:         {

  41:             int deliveries = 0;

  42:             foreach (Order order in orders)

  43:             {

  44:                 deliveries += order.Quantity;

  45:             }

  46:             return deliveries;

  47:         }

  48:     }

  49: }

Snippet 3: The modified stub.

StoreTest explanantion, (line by line)

Don’t worry! It looks more complicated than it actually is. The real stuff starts only on line 25.

25: Declaring a counter variable which will serve our comparison

26-33: Declaring an anonymous method that will simply increase a counter each time DeliverToAddressStringBook is called.

34: Perform the actuall delivery, calling DeliverOrders, which itself will call DeliverToAddress.

35: Using an assertion in order to determine whether our expected amount of delilveries (CalculateExpectedDeliveries) equals the actual amount of delivered books (count).

39-47: The method CalculateExpectedDeliveries is only used to retrieve the correct amount of books that is expected to be delivered by the DeliverOrders method.

In order to let the tests run, simply right-click on the DeliverOrders method and choose Run Pex Explorations, as shown in Figure 2.

image

Figure 2: Run Pex on the PUT.

At this point, you will perhaps notice failing tests where the order.Quantity is equal to a negative number.

I’d like to show you a simple means to prevent Pex from generating and running test cases under certain circumstances: Code Contracts.

For this post, I just show you what you need for that purpose. We’ll probably cover Code Contracts in a future post, since they integrate very well with Pex (and vice versa).

Just drop a statement like

Contract.Assume(target.Orders.Count >= 0);

at the beginning of the DeliverOrders method. For Pex, this simply means to ignore all cases where the store would get a negative amount of orders – hence preventing Pex from generating nonsense test cases.

1.6 Why all this Work?

The advantage of all this customization should be clear: You can test an interface in general, without having to test all single methods that implement the interface.

I hope you are familiar now with (customizable) PUTs and interface tests.

Cheers, and see you at the next post!

Martin

Read Full Post »

Older Posts »