First steps with PEX: Automated White box Testing for .NET

Hello folks,

It’s me again. During the last days I found an interesting new framework: Pex. On the Microsoft’s research labs’ page, it is summarized as follows:

Right from the Visual Studio code editor, Pex finds interesting input-output values of your methods, which you can save as a small test suite with high code coverage. Pex performs a systematic analysis, hunting for boundary conditions, exceptions and assertion failures, which you can debug right away. Pex enables Parameterized Unit Testing, an extension of Unit Testing that reduces test maintenance costs. […]

Sounds good, doesn’t it? We write code, and Pex cares about finding suitable testing values.

Pex is available for download in two flavours:

• Academic version (Current version: 0.20.41218.2, x86)
• Commercial version (Current version: 0.21.50114.0, x86)

Both flavours come with a Visual Studio add-in. Only constraint: It will not work with the VS Express editions. However, in this case you can still use Pex from the command line.

I chose to install the academic version over an existing VS 2008 Pro installation.

1. Creating a test project

Let’s get started. After we (successfully) installed the Pex framework, we create a new Console Application, which, let’s say, simulates a basic bookstore. The following code snippets form our project’s skeleton:

   1: namespace BookStore

   2: {

   3:     public class Book

   4:     {

   5:         public String Title { get; set; }

   6:         public String Author { get; set; }

   7:         public String ISBN { get; set; }

   8:     }

   9: }

Snippet 1: The Book class

   1: namespace BookStore

   2: {

   3:     public class Client

   4:     {

   5:         public String Name { get; set; }

   6:         public String Surname { get; set; }

   7:         public String Address { get; set; }

   8:     }

   9: }

Snippet 2: The Client class

   1: namespace BookStore

   2: {

   3:     public class Order

   4:     {

   5:         public Client client { get; set; }

   6:         public Book book { get; set; }

   7:     }

   8: }

Snippet 3: The Order class

Up to now, our bookstore has no functionality at all. We’ll change that by implementing the Store class:

   1: namespace BookStore

   2: {

   3:     public class Store

   4:     {

   5:         public List<Order> Orders { get; set; }

   6:

   7:         public void AddOrderForDelivery(Order order)

   8:         {

   9:             Orders.Add(order);

  10:         }

  11:

  12:         public void DeliverOrders()

  13:         {

  14:             foreach (Order order in Orders)

  15:             {

  16:                 for (int i = 0; i < order.Quantity; i++)

  17:                 {

  18:                     DeliverToAddress(

  19:                         order.Client.Address,

  20:                         order.Book);

  21:                 }

  22:             }

  23:         }

  24:

  25:         public void DeliverToAddress(String address, Book book)

  26:         {

  27:             Console.WriteLine(

  28:                 "Book with ISBN " + book.ISBN +

  29:                 " delivered to address " + address);

  30:         }

  31:     }

  32: }

Snippet 4: The Store class

So far, so good, right? Now, clearly, our code is quite error-prone. It compiles, but we are not safe from runtime exceptions.

This is where unit tests come into play.

2. Running Pex:

You can run Pex by simply right-clicking on the method in question and selecting Run Pex from the context menu.

Figure 1: Right clicking on the DeliverOrders method reveals the newly added “Run Pex” entry in the context menu. Click on it.

Then we need to select a test framework. Since Pex generates tests, you can choose for which framework they should be generated. Choose Visual Studio Unit Test.

Figure 2: Choosing the test framework.

After confirmation the dialog in Figure 2, the Pex Exploration results window appears, showing the run test cases and indicating which values have been used to created the test case.

Figure 3: Pex exploration results: Failed tests are shown in bold, successfully tests using normal font weight.

On the right side of the Pex results window, you have a detail window. For a successful test, it will show the generated method, which you can save away by clicking the button Save Test below.

Figure 4: Test details for a successful test (here: the selected test case from Figure 3)

On the other hand, for a failed test case, other than seeing the generated method, you can see also a Stack trace of the failed code. Figure 5 shows the (expanded) Stack trace. Please note also that above the Stack trace you still have the possibility to expand the method implementation.

Figure 5: Stack trace of a failed test.

Did you notice the Allow Exception button at the bottom line? Click on it if you want to allow the exception that has arisen during the test run.

Important: You will need to re-run Pex after the allowance of an exception.

Let’s see what happens if we save the generated test:

Figure 6: Save test dialog window.

Confirm twice. The standard naming will do. You will immediately notice that in the background, there is an entire test project that is being created:

Figure 7: The created test project BookStore.Tests. Note also the reference to the Microsoft unit testing framework. This is due to our initial selection of the test framework when running Pex.

Why would we want to do that?

Because, this way you can actually debug each single test case.

Figure 8: The main window showing the Test case (StoreTest.DeliverOrders.g.cs, highlighted in the solution explorer on the right side). Please note also the Debug button below the stack trace (also highlighted). Simply click on it in order to debug the generated test case.

I’d say we stop here for now, for there’s enough to digest. in the meantime you can experiment with Pex and play around in order to get comfortable.

We’ll dig deeper with the next article on Pex.

Best regards,

Martin

About these ads