Posts Tagged ‘visual studio’

Howdy people,

Me again with some more information of how both Code Contracts and Pex can be used together to write expressive tests (while still specifying a (partial) specification of the code). In the course of this article wel’ll also cover Assertions and Assumptions. In order to point out only the necessary topics, let’s introduce a new (very basic) sample project, and let’s perform the following steps:

  • Create a console application called MusicStore
  • Create the class Artist (as of Figure 1)
  • Create the class Album (as of Figure 2)
  • Enable Code Contracts checking (both static and runtime, as of figure 3)


Figure 1: The Artist class of the MusicStore project.


Figure 2: The Album class of the MusicStore project.


Figure 3: Enabling both Runtime Checking (Full), and Static Checking

Important: Do not forget to also check the Implicit Non-Null Obligations, as of Figure 3

The only critical point for CC should be the SellAlbum method of the Artist class. Why? Simply, because there we might have a parameter of type Album that could be NULL. And the Static Checker gives us the correct hints to it, as expected:


Figure 4: Expected CC’s Static Checker suggestion and warning.

So we have one suggestion to put a precondition, and an effective warning that we might be passing a parameter that could be NULL. This was predictable.

Running Pex over the method SellAlbum reveals the following results (If you need a starting point for Pex, please look at my article First steps with PEX: Automated White box Testing for .NET)


Figure 5: Pex results for 3 test cases, one providing the album parameter = NULL. Please note also the typical Pex exploration results’ table showing one column per parameter (in this case only album ).

What happens if we put an assertion to this critical parameter? Let’s simply add one, as shown in Figure 6:


Figure 6: Adding an assertion using a call to the static method Contract.Assert(bool condition) of the CC framework.

It makes sense to assert a number of AlbumsSold higher or equal to 1 (after 1 album has been sold), right? Does putting an assertion have an impact on Pex? Of course! Let’s run it again to see how:


Figure 7: Effects on Pex of using Code Contracts’ Assertions. Pex clearly identifies this as an assertion failure, hence letting the generated test case fail. Line 2 (highlighted the additionally created Pex test for the Code Contracts assertion).

So far, so good. Let’s look at Figure 6 again. We can also put such an assertion for the album object. This clearly makes sense – since selling an album should increase the album’s TimesSold count. Let’s do this in Figure 8:


Figure 8: Added another assertion for the album.TimesSold property.

And immediately, let’s look at the Static Checker warnings and the generated Pex test cases (Figures 9 and 10):


Figure 9: One more unproven assert warning for our newly added Assertion in the SellAlbum method. This was expected. Let’s run Pex again!


Figure 10: Line 3 shows the newly generated (and failed) test case of Pex. Again, this was expected.

Great coverage – but What About the Meaning?

This is a good question. Is this still an expressive test? Let’s recap the meaning. We could have an Album -type parameter passed, which could be theoretically NULL. Would it make any sense to then test its property TimesSold? I am sure, we can agree on that this would not be the case.

So, what we need, is the possibility to prevent Pex from generating test cases in such a case, hence producing only meaningful tests. And who is going to provide us with such a capability?

Code Contracts’ Assumptions

Important: Assumptions only work when the Full contract option is checked in the Code Contracts pane (under our project’s properties)

Figure 11 gives a brief example of how to put such an assumption into our code of the SellAlbum method.


Figure 11: Creating assumptions.

Figure 11 needs some explanations:

The first three assumptions simply protect us from getting the current object, the passed album, or the passed album’s artist as NULL. Makes sense, so far. The following two assumptions ensure that this.AlbumsSold and album.TimesSold both are non-negative values.

(Note also that now we put our assertions INSIDE the artist comparison’s block – makes sense, since we do not need to assert when there happened no increase of AlbumsSold and TimesSold properties).

What do We Expect?

Well, we pretty much thought at everything, didn’t we? So we expect Pex to not issue any errors or failures again. A simple re-run of Pex will show us the result, as can be seen in Figure 12.


Figure 12: Unexpected error on line 3, a failed assertion.

An unexpected failure? How could that happen? Let’s look at what the parameters looked like in the case of failure:


Did you see it? AlbumsSold is equal to int.MaxValue ! I guess, I do not need to explain what happens, if you try to increase an integer (in SellAlbum ) which is already equal to its maximum value! But why did that happen?

Simply put, this is one of the boundary test cases we explained in First steps with PEX: Automated White box Testing for .NET article. Pex guesses from the type of variables which are the boundary values (and also intermediate values).

To summarize shortly & sweetly today’s lecture:

Code Contracts & Pex: meaningful, expressive tests

(by preventing test case generation for assumed parameter values)

Let’s stop here today. There are more interesting topics regarding Code Contracts and Pex, and I am quite sure this will keep us busy for another while!

Enjoy playing around with Pex and Code Contracts, and see you next time!

Best regards,


Read Full Post »


The Code Contracts User Manual holds (besides detailed information about Code Contracts’ usage, advantages and drawbacks) a few nice lists of Code Snippets that ship with the Code Contracts and will ease your life. Find below the list of available Code Snippets for the C# language, as exactly taken from the User Manual (January 12th, 2010):

cr Contract.Requires(…);
ce Contract.Ensures(…);
ci Contract.Invariant(…);
crr Contract.Result<…>()
co Contract.OldValue(…)
cim [ContractInvariantMethod]

private ObjectInvariant()




crn Contract.Requires(… != null);
cen Contract.Ensures(Contracts.Result<…>() != null);
crsn Contract.Requires( !String.IsNullOrEmpty(…) );
cesn Contract.Ensures( !String.IsNullOrEmpty(Contracts.Result<string>()) );
cca Contract.Assert(…);
cam Contract.Assume(…);
cre Contract.Requires<E>(…);
cren Contract.Requires<ArgumentNullException>(… != null);
cresn Contract.Requires<ArgumentException>( !String.IsNullOrEmpty(…) );
cintf expands to an interface template and associated contract class

If you plan to use Code Contracts effectively, this list comes in very handy. I hope it’s of use for you. In the Code Contracts User Manual, you can find a similar list also for Visual Basic.

Best regards,


Read Full Post »

Hello all,

This time we’ll do a quick exploration of how we can apply Code Contracts to interfaces. As you know from our post First Steps with Code Contracts, the preconditions and postconditions (Contract.Requires and Contract.Ensures calls, respectively) must be placed inside a method body.

Therefore, when defining an interface, we run into a problem: We do not have method bodies there. Of course we could put such calls inside each effective implementation of the interface methods. Clearly, this is not what we want, since

  • We do not want to replicate code in x classes that implement our interface and
  • Future implementations of our interface would contain the conditions we need

Luckily, the Code Contracts provide us with a powerful mechanism that allows us to define a class which implements that interface and which will do the necessary checks.

The Contract Class

Let’s see how this is done, right? For this example, please recall our example from the First Steps with Code Contracts introductory article.


   1: namespace Vehicles

   2: {

   3:     interface IVehicle

   4:     {

   5:         void Drive(Int32 speed);        

   6:     }

   7: }

Snippet 1: The IVehicle interface

Also do recall that we did not define any pre- or postconditions for our interface (how could we? – there is no method body).

This is exactly the place, where we will put a so called Contract Class, that will implement our interface. Every time the interface is called, the conditions put into our Contract Class will be injected. This holds for every implementation of our interface.

We basically need two things:

1. A class that implements our interface IVehicle that is marked as Contract Class via an attribute:


   1: [ContractClassFor(typeof(IVehicle))]

   2: public class IVehicleContract : IVehicle

   3: {

   4: }

Snippet 2: Contract Class implementing the interface we want to fulfil requirements

2. Another class that links up our interface to the ContractClass

   1: [ContractClass(typeof(IVehicleContract))]

   2: public partial interface IVehicle

   3: {

   4: }

Snippet 3: The link between the interface and our Contract Class

Hint: Don’t worry: You do not have to implement these two classes from scratch: The Code Contracts come with a bunch of predefined snippets that can be executed right away. The following snippet will do the trick for generating the Contract Classes for an interface:


cintf ->(TAB – TAB)

The next step is to implement the interface for IVehicleContract.


Figure 1: Explicitly implementing the interface IVehicle for the IVehicleContract class.

The result is shown in Snippet 4:

   1: [ContractClassFor(typeof(IVehicle))]

   2: public class IVehicleContract : IVehicle

   3: {

   4:     void IVehicle.Drive(int speed)

   5:     {

   6:         throw new NotImplementedException();

   7:     }

   8: }

As a next step, we can implement our conditions in the method body of IVehicle.Drive method in IVehicleContract, like shown in Snippet 5:

   1: [ContractClassFor(typeof(IVehicle))]

   2: public class IVehicleContract : IVehicle

   3: {

   4:     void IVehicle.Drive(int speed)

   5:     {

   6:         Contract.Requires(speed >= 0);

   7:     }

   8: }

Snippet 5: precondition in interface method implementation.

What does this mean now? Well, for every implementation of the interface IVehicle , the code precondition will be injected. In other words:


The condition we define in the Contract Class must hold for all implementations of the interface.


That was it already (for this time)! It is as simple to use as it seems.


Enjoy and stay tuned for the next time!

Best regards,


Read Full Post »

Hello folks,

It’s me again. During the last days I found an interesting new framework: Pex. On the Microsoft’s research labs’ page, it is summarized as follows:

Right from the Visual Studio code editor, Pex finds interesting input-output values of your methods, which you can save as a small test suite with high code coverage. Pex performs a systematic analysis, hunting for boundary conditions, exceptions and assertion failures, which you can debug right away. Pex enables Parameterized Unit Testing, an extension of Unit Testing that reduces test maintenance costs. […]

Sounds good, doesn’t it? We write code, and Pex cares about finding suitable testing values.

Pex is available for download in two flavours:

Both flavours come with a Visual Studio add-in. Only constraint: It will not work with the VS Express editions. However, in this case you can still use Pex from the command line.

I chose to install the academic version over an existing VS 2008 Pro installation.

1. Creating a test project

Let’s get started. After we (successfully) installed the Pex framework, we create a new Console Application, which, let’s say, simulates a basic bookstore. The following code snippets form our project’s skeleton:

   1: namespace BookStore

   2: {

   3:     public class Book

   4:     {

   5:         public String Title { get; set; }

   6:         public String Author { get; set; }

   7:         public String ISBN { get; set; }

   8:     }

   9: }

Snippet 1: The Book class

   1: namespace BookStore

   2: {

   3:     public class Client

   4:     {

   5:         public String Name { get; set; }

   6:         public String Surname { get; set; }

   7:         public String Address { get; set; }

   8:     }

   9: }

Snippet 2: The Client class

   1: namespace BookStore

   2: {

   3:     public class Order

   4:     {

   5:         public Client client { get; set; }

   6:         public Book book { get; set; }

   7:     }

   8: }

Snippet 3: The Order class

Up to now, our bookstore has no functionality at all. We’ll change that by implementing the Store class:

   1: namespace BookStore

   2: {

   3:     public class Store

   4:     {

   5:         public List<Order> Orders { get; set; }


   7:         public void AddOrderForDelivery(Order order)

   8:         {

   9:             Orders.Add(order);

  10:         }


  12:         public void DeliverOrders()

  13:         {

  14:             foreach (Order order in Orders)

  15:             {

  16:                 for (int i = 0; i < order.Quantity; i++)

  17:                 {

  18:                     DeliverToAddress(

  19:                         order.Client.Address,

  20:                         order.Book);

  21:                 }

  22:             }

  23:         }


  25:         public void DeliverToAddress(String address, Book book)

  26:         {

  27:             Console.WriteLine(

  28:                 "Book with ISBN " + book.ISBN +

  29:                 " delivered to address " + address);

  30:         }

  31:     }

  32: }

Snippet 4: The Store class

So far, so good, right? Now, clearly, our code is quite error-prone. It compiles, but we are not safe from runtime exceptions.

This is where unit tests come into play.

2. Running Pex:

You can run Pex by simply right-clicking on the method in question and selecting Run Pex from the context menu.


Figure 1: Right clicking on the DeliverOrders method reveals the newly added “Run Pex” entry in the context menu. Click on it.

Then we need to select a test framework. Since Pex generates tests, you can choose for which framework they should be generated. Choose Visual Studio Unit Test.


Figure 2: Choosing the test framework.

After confirmation the dialog in Figure 2, the Pex Exploration results window appears, showing the run test cases and indicating which values have been used to created the test case.


Figure 3: Pex exploration results: Failed tests are shown in bold, successfully tests using normal font weight.

On the right side of the Pex results window, you have a detail window. For a successful test, it will show the generated method, which you can save away by clicking the button Save Test below.


Figure 4: Test details for a successful test (here: the selected test case from Figure 3)

On the other hand, for a failed test case, other than seeing the generated method, you can see also a Stack trace of the failed code. Figure 5 shows the (expanded) Stack trace. Please note also that above the Stack trace you still have the possibility to expand the method implementation.


Figure 5: Stack trace of a failed test.

Did you notice the Allow Exception button at the bottom line? Click on it if you want to allow the exception that has arisen during the test run.

Important: You will need to re-run Pex after the allowance of an exception.

Let’s see what happens if we save the generated test:


Figure 6: Save test dialog window.

Confirm twice. The standard naming will do. You will immediately notice that in the background, there is an entire test project that is being created:


Figure 7: The created test project BookStore.Tests. Note also the reference to the Microsoft unit testing framework. This is due to our initial selection of the test framework when running Pex.

Why would we want to do that?

Because, this way you can actually debug each single test case.


Figure 8: The main window showing the Test case (StoreTest.DeliverOrders.g.cs, highlighted in the solution explorer on the right side). Please note also the Debug button below the stack trace (also highlighted). Simply click on it in order to debug the generated test case.

I’d say we stop here for now, for there’s enough to digest. in the meantime you can experiment with Pex and play around in order to get comfortable.

We’ll dig deeper with the next article on Pex.

Best regards,


Read Full Post »