Assert.Count in MbUnit v3.2

Assert.Count is a last-minute addition made to Gallio/MbUnit v3.2 just before passing to RC. Be sure to read the nice article that Vadim Kreynin has written about that new convenient assertion.

XML Assertions in MbUnit v3

If you have already made some attempts in the past to write unit tests for testing XML output, then you surely noticed that it is not an easy task. Usually, the initial approach is to test the resulting XML data by using a simple text equality assertion. Unfortunately, it does not work very well; mainly because XML contains insignificant whitespaces, comment tags, or self-closing empty elements. Furthermore, we would like sometimes to ignore case of element names, or to ignore the order of the attributes in the same parent element.

For example, are those two fragments equal?

<value x='123' y='456'/>

<VALUE y='456' x='123'></VALUE>

Well, it depends. The name of elements differs by the case, the attributes are in a different order, and one of element is self-closing while the later is not. Nevertheless, it's perfectly reasonable to consider they are equal. And obviously, a unit test which makes use of a text equality assertion will miserably fail because the actual strings are just different.

Fortunately, MbUnit v3.2 proposes some fresh new assertions to test XML data. Basically, the assertions parse the fragments of XML (expected and actual) and compare the resulting trees by taking in account the equality options specified by the user. This is very easy to use. Let's compare our fancy fragments by using Assert.Xml.AreEqual:

[Test]
public void MyXmlTest()
{
   var generator = new MyXmlGenerator();
   string actual = generator.ToXml();
   Assert.Xml.AreEqual("<value x='123' y='456'/>", actual, XmlOptions.Loose);
}

Remark the loose equality options which tells the assertions to ignore comment tags, case of names/values, order of attributes, etc.

More details and examples may be found the Gallio Wiki.

Equality Assertions in MbUnit v3

As .NET developers, we all know that the notion of object equality is not as simple as it looks first. Object equality is in fact probably as fundamental and difficult to understand as the famous concept of pointers in the good old C language. The fact is that writing unit tests, with Gallio/MbUnit or with any other existing framework is mostly about making equality assertions on the output of various code components. Thus chances are high you be stuck soon or later by an equality assertion which should obviously pass, but unexpectedly fails because your type does not implement any reliable equality mechanism. That's why I would like to review in that article, some features of MbUnit v3.1 which may help you using properly the powerful equality assertions.

Let's start with a simple example.

[TestFixture]
public class SolarSystemTest
{
  [Test]
  public void Number_of_planets_in_the_solar_system()
  {
    var repository = new StarSystemRepository();
    var solarSystem = repository.GetLocalSystem();
    int count = solarSystem.CountPlanets();
    Assert.AreEqual(8, count);
  }
}

We use here the well-known Assert.AreEqual to verify that the actual number of planets found in our solar system is 8, as expected. Of course, the equality assertion knows how to compare System.Int32 values. Basically, it knows how to compare any primitive like System.String or System.Double. But what happens while asserting on non-primitive types?

By default, MbUnit relies on the result returned by the overridable Object.Equals method. And by default, that method simply consists of a referential equality. Thus it returns true only if the 2 objects compared represent the same instance (EDIT: OK, let's ignore the case where objects are null for the moment.). That's why, given our implementation of the class Planet, the following test miserably fails.

public class Planet
{
  public string Name 
  { 
    get;
    private set;
  }
  
  public Planet(string name)
  {
    Name = name;
  }
}

[TestFixture]
public class SolarSystemTest
{
  [Test]
  public void Mercury_is_the_closest_planet_to_the_sun()
  {
    var repository = new StarSystemRepository();
    var solarSystem = repository.GetLocalSystem();
    IPlanet actual = solarSystem.GetClosestPlanetToTheStar();
    Assert.AreEqual(new Planet("Mercury"), actual); // Fail!?
  }
}

So how should we write the assertion to get the expected result? There are several solutions.

• Asserting the inner object properties.

  The most obvious solution is indeed to verify the inner properties of the actual object. We could just replace the failing assertion by:

  Assert.AreEqual("Mercury", actual.Name);

  For a simple scenario such as in our example, this is surely enough. But what if Planet is in fact a complex entity, with a dozen of properties, each hiding a complex tree of entities and value objects? At best, you would end up with a very large number of unmaintainable assertions. That's why you should avoid that solution if you get more than 3 or 4 assertions.

• Supporting equality.

  The second possibility consists in implementing a dedicated equality mechanism for the evaluated type. The standard way of doing it is to implement the IEquatable<T> interface.

  public class Planet : IEquatable<Planet>
  {
    public string Name
    {
        get;
        private set;
    }
  
    public Planet(string name)
    {
        Name = name;
    }
  
    public bool Equals(Planet other)
    {
        return (other != null) && (Name == other.Name); 
    }
  
    public override int GetHashCode()
    {
        return Name.GetHashCode();
    }
  
    public override bool Equals(object obj)
    {
        return Equals(obj as Planet);
    }
  }

  This may sound like a perfectly reasonable solution. In fact, if you are a lucky developer, chances are good that your class already implements such an equality mechanism. It is perhaps a needed feature of your code base. If yes, then look no further, and just use it. The initial assertion will pass. Congratulations! You made it.

  But if no, then Planet has no equality mechanism probably because it does not need of any. It means that you are about to add an unnecessary feature to your code base, just to make your tests easier to write. Remember YAGNI? You ain't gonna need it! If Planet does not need to be equatable, then why making it equatable? Your answer should never be: "To make it more testable". This is a short way to the dark side of the force, I assure you. Adding unnecessary code that will only eventually be used by your unit tests is certainly a bad practice.

• Using the comparison delegate.

  Most of the MbUnit equality assertions take a third optional parameter of the type EqualityComparison<T>. The equality comparison delegate is a function which takes two instances of the same type, and returns true if they are equal. If your scenario is simple enough, you can provide such a function to specify to the assertion how to compare the objects. With the lambda syntax, it looks very elegant.

  Assert.AreEqual(new Planet("Mercury"), actual, (x, y) => x.Name == y.Name);

  And if the object has a reasonable number of properties, that's still a good solution.

  Assert.AreEqual(new Planet("Mercury"), actual, (x, y) => 
    x.Name == y.Name &&
    x.Weight = y.Weight &&
    x.DistanceToStar == y.DistanceToStar);

  But again, if the type has too many properties, or if these properties are not more equatable than their parent, you will get a complete mess.

• Using the structural equality comparer.

  MbUnit v3.1 comes with a fantastic built-in feature which deserves to be better known (but that's the point of this post anyway). Basically, the structural equality comparer (StructuralEqualityComparer<T>) provides to the assertions a convenient way to determine whether two instances of a type are equal or not; while the type itself does not implement any relevant equality mechanism.

  Assert.AreEqual(new Planet("Mercury"), actual, 
    new StructuralEqualityComparer<Planet>
    {
      { x => x.Name }
    });

  Well, it does not look so impressive, does it? The comparer instance is populated with one comparison criterion that says to the assertion engine to use the property Name to compare two Planet instances. The syntax is not much more complicated when you have several properties.

  Assert.AreEqualnew Planet("Mercury"), actual, 
    new StructuralEqualityComparer<Planet>
    {
      { x => x.Name },
      { x => x.Weight },
      { x => x.DistanceToSun }
    });

  The true power appears when you know that each equality criterion is easily customizable, either with a comparison delegate, or with a new inner comparer.

  Assert.AreEqual(new Planet("Mercury"), actual, 
    new StructuralEqualityComparer<Planet>
    {
      { x => x.Name, (a, b) => a.Equals(b, StringComparison.OrdinalIgnoreCase) },
      { x => x.Weight },
      { x => x.DistanceToSun },
      { x => x.Revolution, (a, b) => a.Period == b.Period }
    });

  As you see, each criterion is able to define its own comparison rules. You can also nest structural equality comparers and define them as a comparison rule for an inner criterion.

  Assert.AreEqual(new Planet("Mercury"), actual, 
    new StructuralEqualityComparer<Planet>
    {
      { x => x.Name, (a, b) => a.Equals(b, StringComparison.OrdinalIgnoreCase) },
      { x => x.Weight },
      { x => x.DistanceToSun },
      { x => x.Revolution, new StructuralEqualityComparer<Revolution> { { x => x.Period } } }
    });

  The comparer works very well with enumerations too. It provides a similar result to what do Assert.AreElementsEqual and Assert.AreElementsEqualIgnoringOrder. Suppose that Planet has now a property named Satellites which returns an instance of the type IEnumerable<Satellite>. Adding them into the overall comparison structure is easy. The comparer also supports some options to ignore the order of the child elements.

  Assert.AreEqual(new Planet("Mercury"), actual, 
    new StructuralEqualityComparer<Planet>
    {
      { x => x.Name, (a, b) => a.Equals(b, StringComparison.OrdinalIgnoreCase) },
      { x => x.Weight },
      { x => x.DistanceToSun },
      { x => x.Revolution, (a, b) => a.Period == b.Period }
      { x => x.Satellites, new StructuralEqualityComparer<Satellite>
          {
            { x.Name },
            { x.DistanceToPlanet }
          }, StructuralEqualityComparerOptions.IgnoreEnumerableOrder
      }
    });

  As already explained, the structural equality comparer can be used with most of the equality assertions. It is particularly useful with the equality assertions for collections.

  Assert.AreElementsEqualIgnoringOrder(
    new[] { new Satellite("Deimos"), new Satellite("Phobos") },
    mars.Satellites,
    new StructuralEqualityComparer<Satellite> { { x => x.Name } });

See you next time!

Assert.ForAll and Assert.Exists in MbUnit v3

MbUnit v3.2 v3.1 is going to be released next week as part of the Gallio package. It contains two new very convenient assertions; So useful in fact, that they probably should have come earlier. But well... Better late than never!

Anyway, Assert.ForAll and Assert.Exists are the counterparts of the LINQ extension methods IEnumerable<T>.All and IEnumerable<T>.Any respectively.

Assert.ForAll verifies that all the elements of the sequence meet the specified condition. The following example defines a test method which verifies that every integer of the sequence is an even number.

[TestFixture]
public class MyTestFixture
{
  [Test]
  public void ForAllTest()
  {
    var data = new[] { 2, 8, 10, 6, 4, 20 };
    Assert.ForAll(data, x => x % 2 == 0); // pass!
  }
}

Assert.Exists verifies that at least one element of the sequence meets the specified condition. The assertion evaluates each element until it finds a matching one, or until it reaches the end of the enumeration, which causes the assertion to fail. The next example shows a test method which verifies that at least one integer of the sequence is an odd number. Considering the sample array, the method should obviously fail.

[TestFixture]
public class MyTestFixture
{
  [Test]
  public void ExistsTest()
  {
    var data = new[] { 2, 8, 10, 6, 4, 20 };
    Assert.Exists(data, x => x % 2 != 0); // fail!
  }
}

Assert.Sorted in MbUnit v3

Recently, we did implement in MbUnit v3 a few handy new assertions to evaluate enumerations, such as Assert.Distinct and Assert.Sorted.

Assert.Sorted verifies that the elements in an enumeration are effectively sorted.

[TestFixture]
public class MyTestFixture
{
  [Test]
  public void SortingTest1()
  {
     var array = new[] { 1, 4, 9, 9, 10 };
     Assert.Sorted(array, SortOrder.Increasing);
  }
}

As you can see, the expected sorting direction must be specified in the second parameter. It may be one of the following values:

• SortOrder.Increasing
• SortOrder.StrictlyIncreasing
• SortOrder.Decreasing
• SortOrder.StrictlyDecreasing

Internally, the assertion uses the implementation of IComparable or IComparable<T> to perform the comparisons between the elements of the enumerations. However, if the elements are of type which is not comparable, you have to provide your own mechanism to compare objects. It might be either a Comparison<T> delegate or a IComparer<T> object.

public class Foo
{
  private readonly int value;
  
  public int Value
  {
    get
    {
      return value;
    }
  }
  
  public Foo(int value)
  {
    this.value = value;
  }
}

[TestFixture]
public class MyTestFixture
{
  [Test]
  public void SortingTest2()
  {
     var array = new[] { new Foo(1), new Foo(4), new Foo(9), new Foo(9), new Foo(10) };
     Assert.Sorted(array, SortOrder.Increasing, (x, y) => x.Value.CompareTo(y.Value));
  }
}

Assert.Distinct in MbUnit v3

Assert.Distinct is a new useful assertion available in the incoming version 3.0.7 of MbUnit. The assertion verifies that the elements of the provided enumeration are distinct from each other.

Usually, the simplest overload which takes the enumeration as a single argument will be sufficient. It lets the default object comparison engine of Gallio to handle with the comparison of the elements inside the enumeration.

[TestFixture]
public class MyTestFixture
{
   [Test]
   public void MyTestMethod()
   {
      var array = new[] { 123, 456, 789 };
      Assert.Distinct(array);
   }
}

However, a couple of other overloads are available, which let you customize the way you want to compare objects together. The following example shows how to verify that an enumeration contains distinct Foo instances. Since the subject Foo type has intentionally no usual comparison feature such as the implementation of IComparable, or the override of Object.Equals, we introduce here a comparison delegate which compares two instances together.

public class Foo
{
    public int Number;
    public string Text;
}

[TestFixture]
public class MyTestFixture
{
   [Test]
   public void MyTestMethod()
   {
      var array = new[] 
      { 
         new Foo { Number = 123, Text = "ABC" }
         new Foo { Number = 456, Text = "DEF" }
         new Foo { Number = 789, Text = "GHI" }
      };
      
      Assert.Distinct(array, (x, y) => x.Number != y.Number && String.Compare(x.Text, y.Text, true) == 0);
   }
}

The assertion accepts an IEqualityComparer<T> as well. So we can use the new useful structural equality comparer of MbUnit (more info about that nice beast in a future article).

[TestFixture]
public class MyTestFixture
{
   [Test]
   public void MyTestMethod()
   {
      var array = new[] 
      { 
         new Foo { Number = 123, Text = "ABC" }
         new Foo { Number = 456, Text = "DEF" }
         new Foo { Number = 789, Text = "GHI" }
      };
      
      Assert.Distinct(array, new StructuralEqualityComparer<Foo>
      {
          { x => x.Number },
          { x => x.Text, (x, y) => String.Compare(x, y, true) == 0 }
      });
   }
}

Retry.Until in MbUnit v3

The next release of Gallio (v3.0.7) contains an impressive number of new features (VS2010 Support, Control Panel, Test With Debugger in Icarus, and many more). But we have also worked hard to extend MbUnit v3 with some new interesting capabilities. I am going to present them briefly to you in the following articles. Please remember that you are not forced to wait for the official release of Gallio v3.0.7 to play with them. You can download the latest daily build here. It is possible that you experiment some minor issues with the daily builds (let us know on the newsgroups here and there), but they are very stable already.

I will start today with the new special assertion Retry.Until. It is special because it is not of the usual form Assert.Something.

Retry.Until lets you evaluate a specific condition several times until it becomes true, or until some timeout mechanism triggers an assertion failure.

The assertion comes with a nice fluent syntax which allows some flexible configuration. It works a little bit like the famous Rhino.Mocks expectations. You can specify the number of times the condition must be evaluated (Times), a polling time between each evaluation (WithPolling), a global timeout duration for the entire operation (WithTimeout), or some custom action to execute between each cycle (DoBetween). Of course, you can finally specify the condition to evaluate (Until). So far, 3 kinds of conditions are accepted:

• A WaitHandle instance which is expected to be signaled.
• A Thread instance which is expected to be terminated.
• A versatile predicate Func<bool> which is expected to return true.

Here is a simple usage example, which asserts that an hypothetical asynchronous operation has completed within an expected timeframe.

[TestFixture]
public class MyTestFixture
{
   [Test]
   public void MyTestMethod()
   {
      var foo = new Foo();
      foo.RunSomeAsyncOperation();
      Retry.Times(5)
           .WithPolling(TimeSpan.FromSeconds(1))
           .Until(() => Foo.HasTerminated());
   }
}