2008/10/17

New Assertions for MbUnit v3

[Update 11/12/2008: As explained in a comment below, Jeff has restored the delta equality assertion with a set of methods named Assert.AreApproximatelyEqual.]

The latest version of MbUnit v3 which comes with Gallio v3.0.4, has a completely new set of assertions methods. It has been entirely re-written by Jeff and Vadim. The new MbUnit.Framework.Assert class is indeed more consistent. It makes an intensive use of generics and lambda expressions too.

Of course, it comes at the cost of some breaking changes, because many methods have been removed or renamed. But the new assertion methods are very powerful and much more flexible than before; so those breaking changes are easy to fix.

For example, the following method has been discarded:
Assert.AreEqual(double expected, double actual, double delta)
The delta parameter specified the maximum acceptable difference between the expected and the actual values. That assertion was useful when comparing the result of a calculation involving potential rounding issues. The following assertion was passing because 3.14 and 3.14159 differs less than 0.01.
Assert.AreEqual(3.14, 3.14159, 0.01);
The latest version of MbUnit proposes to use that new generic assertion method instead.
Assert.AreEqual<T>(T expectedValue, T actualValue, Func<T, T, bool> comparer)
The signature of the method seems much more complicated, especially if you are not familiar with generics. However, thanks to the C# 3.0 type inference, you do not need to specify explicitely any type parameter. So in fact, you can write the following line, which has the same effect has the example above.
Assert.AreEqual(3.14, 3.14159, (x, y) => Math.Abs(x - y) <= 0.01);

2008/10/16

Gallio v3.0.4

Gallio v3.0.4 is finally there! Download it and enjoy!

2008/10/06

Rhino.Mocks 3.5: The new generic list constraint.

Oren Eini has finally released Rhino.Mocks 3.5 RTM. As written in Oren's article (many thanks to him for the credits), I had submitted some time ago a patch, which adds a new useful constraint for generic collections. Some explanations are perhaps needed:

Suppose that you are setting up some expectation on a method that takes a collection as parameter: a list of strings, for example.
public interface IFoo
{
bool DoSomething(List<string> values);
}
With Rhino.Mocks 3.4, you can specify a constraint on the input collection. For instance, you can verify that the fifth element of the collection has the value "Hello".
Expect.Call<bool>(mockFoo.DoSomething(null))
.IgnoreArguments()
.Constraints(List.Element(4, Is.Equal("Hello"))
.Return(true);
Unfortunately, the List.Element method can only take an Int32 as first parameter. Thus, it makes the constraint only usable with Int32-indexed collections, such as lists and arrays. It is a shame if your parameter is a dictionary.

That's why I have written a new abstract constraint (KeyedListElement) and overriden the List.Element with a new generic method. The usage is very similar. It is actually a kind of generalization of the former method.
public interface IFoo
{
bool DoSomething(Dictionary<string, string> values);
}
Expect.Call<bool>(mockFoo.DoSomething(null))
.IgnoreArguments()
.Constraints(List.Element<string>("Value1", Is.Equal("Hello"))
.Return(true);
The example above defines a constraint on the parameter (a dictionary of strings keyed by strings). The value at index "Value1" must be equal to "Hello". Note the generic signature of the method, which specifies the type of the key.

Hope it helps!

2008/09/30

Gallio + PowerShell

Nobody is perfect: I am fond of PowerShell. Yes, I know, the PowerShell language syntax is a bit weird sometimes (Do you know any other languages where you use command switches to compare things?), but despite the ugly look of the scripts, this is a very powerful tool.

Gallio has a very handy snap-in for PowerShell, written by Julian Hidalgo and Jeff Brown. It adds a very useful cmdlet which controls the Gallio test runners. The snap-in is configured by the installer. You just need to ensure that the correct install options are checked.



Once Gallio is installed, start the PowerShell console, then load the snap-in in memory with the following command:
Add-PSSnapIn Gallio
Then you can run your test suite by calling Run-Gallio with the appropriate arguments.
Run-Gallio MyTests.dll
More help about the arguments can be found with the following command:
Get-Help Run-Gallio -detailed
One of the most interesting features of PowerShell is that it is entirely object oriented. In that context, the Gallio PowerShell Snap-In provides a complete report object as result. The object is of type Gallio.Runner.TestLauncherResult. Sorry, no online document is available for the moment. But you can get additional details by opening the offline API documentation. There should be a shortcut copied somewhere by the Gallio installer in the start menu. [Update 11/12/2008: Online document is now available on the Gallio website.]



It is easy to look at the result properties, and make the appropriate actions. For instance, you can decide to interrupt the execution of the script if at least one unit test has failed.
Add-PSSnapIn Gallio
$result = Run-Gallio MyTests.dll

if ($result.Statistics.FailedCount -gt 0)
{
Write-Warning "Some unit tests have failed."
Exit
}

2008/09/23

Contract Verifiers - Part IV. - The Comparison Contract Verifier

The comparison contract verifier helps in implementing correctly the generic interface IComparable. It verifies in particular that your type-specific implementation orders instances correctly.
public class Fraction : IComparable<Fration>
{
public Fraction(int numerator, int denominator)
{
// ...
}

// ...
}
As for the other contract verifiers, you must use a dedicated attribute to decorate your test fixture. And as for the equality contract verifier, your test fixture must provide classes of equivalence object instances to be consumed by the contract verifier. The only but important difference here, is that you must feed the method with ordered classes. Each class must contain instances expected to be compared strictly greater than the objects in any previous class. In other words, instances must be put in the ascending order.
[TestFixture]
[VerifyComparisonContract(typeof(Fraction))]
public class FractionTest : IEquivalenceClassProvider<Fration>
{
public EquivalenceClassCollection<Fration> GetEquivalenceClasses()
{
return EquivalenceClassCollection<Fration>.FromDistinctInstances(
new Fraction(1, 6),
new Fraction(1, 5),
new Fraction(1, 4),
new Fraction(1, 3));
};
}
}
The comparison contract verifier checks for the following features:

1. The CompareTo method works as expected. The method must return a negative value if the current object is less than the other object; a positive value if it is greater; or zero if they are equal. Furthermore, if the type is nullable, the contract verifier checks for null reference handling as well. Any non null instance should compares greater than a null reference; and two null references should be equal.
public class Fraction : IComparable<Fration>
{
public int CompareTo(Fraction other)
{
// ...
}
}
Although it is not mandatory, you might decide to override the four comparison operators. The contract verifier is able to check them too. This is an optional feature. You can disable it by setting the named parameter ImplementsOperatorOverload to false.
[TestFixture]
[VerifyComparisonContract(typeof(Fraction), ImplementsOperatorOverload = false)]
public class
FractionTest : IEquivalenceClassProvider<Fration>
{
// ...
}
2. The Greater Than (>) operator overload is correctly implemented and behaves as expected.
public class Fraction : IComparable<Fration>
{
public static bool operator >(Fraction left, Fraction right)
{
// ...
}
}
3. The Greater Than Or Equal (>=) operator overload is correctly implemented and behaves as expected.
public class Fraction : IComparable<Fration>
{
public static bool operator >=(Fraction left, Fraction right)
{
// ...
}
}
4. The Less Than (<) operator overload is correctly implemented and behaves as expected.
public class Fraction : IComparable<Fration>
{
public static bool operator <(Fraction left, Fraction right)
{
// ...
}
}
5. The Less Than Or Equal (<=) operator overload is correctly implemented and behaves as expected.
public class Fraction : IComparable<Fration>
{
public static bool operator <=(Fraction left, Fraction right)
{
// ...
}
}
It is very common to implement equality and comparison in the same time. Should this happen, you can combine several contract verifiers into the same test fixture. This is very handy.
public class Fraction : IComparable<Fration>, IEquality<Fration>
{
}
[TestFixture]
[VerifyComparisonContract(typeof(Fraction))]
[VerifyEqualityContract(typeof(Fraction))]
public class FractionTest : IEquivalenceClassProvider<Fration>
{
// ...
}

2008/09/03

Contract Verifiers - Part III. - The Equality Contract Verifier

The path that leads to a proper implementation of the equality contract has many pitfall traps. Fortunately, the equality contract verifier will help you in avoiding them. For convenience, we will consider as an example, a hypothetical class representing a simple mathematical fraction which implements IEquality<T>.
public class Fraction : IEquality<Fraction>
{
public Fraction(int numerator, int denominator)
{
// ...
}

// ...
}
In order to test the type implementing the generic equality interface, the contract verifier needs some valid type instances. That's why the test fixture must fulfill a two side parts contract.
  • It must have the attribute VerifyEqualityContract.
  • It must implement the interface IEquivalenceClassProvider<T>, which will provide the test runner with various object instances to play with.
using Gallio.Framework;
using MbUnit.Framework;
using MbUnit.Framework.ContractVerifiers;

[TestFixture]
[VerifyEqualityContract(typeof(Fraction))]
public class FractionTest : IEquivalenceClassProvider<Fraction>
{
public EquivalenceClassCollection<Fraction> GetEquivalenceClasses()
{
return EquivalenceClassCollection<Fraction>.FromDistinctInstances(
new Fraction(1, 2),
new Fraction(2, 3),
new Fraction(4, 5));
}
}
The IEquivalenceClassProvider<T> has a single method that you must use to return a collection of equivalence classes for the evaluated type. An equivalence class is a collection of object instances that are expected to be equal (The notion of equality is of course to be understood in the sense of your own equality contract). In the simplest scenarios, it is generally sufficient to provide several distinct object instances to be compared together. Use the factory method FromDistinctInstances to construct such a collection of equivalence classes. In the previous example, we provide 3 equivalence classes, each containing 1 single object.

However, in more complex scenarios, we want to verify that different object instances "representing" the same value are actually equal. For instance, the fraction 1/2 should be equal to the fraction 2/4 or 3/6. Instead of using the factory method, use the constructor to provide some equivalence classes containing a variable number of objects.
[TestFixture]
[VerifyEqualityContract(typeof(Fraction))]
public class FractionTest : IEquivalenceClassProvider<Fraction>
{
public EquivalenceClassCollection<Fraction> GetEquivalenceClasses()
{
return new EquivalenceClassCollection<Fraction>(
new EquivalenceClass<Fraction>(
new Fraction(1, 2),
new Fraction(2, 4),
new Fraction(3, 6)),
new EquivalenceClass<Fraction>(
new Fraction(2, 3),
new Fraction(4, 6)),
new EquivalenceClass<Fraction>(
new Fraction(4, 5)))
}
}
Now that we have defined our contract verifier, let's see what it will check for.

1. It verifies that the method GetHashCode returns the same value for equal objects. You should override the method and return an identical value for all the objects located in the same equivalence class. However, the contrary is not true: it is not required that the resulting hash value for unequal objects be different (although it would be a good idea, for efficiency reasons)
public class Fraction : IEquality<Fraction>
{
public override int GetHashCode()
{
// ...
}
}
2. It verifies that the method Object.Equals(object) behaves as expected. The method should return true when two instances of the same equivalence class are compared together, or when an instance is compared to itself. For nullable types (reference types, or value types nested in a Nullable class), it verifies you dealt correctly with null references. The method should return False when called against a null reference.
public class Fraction : IEquality<Fraction>
{
public override bool Equals(object obj)
{
// ...
}
}
3. Same verifications as point 2; but for the strongly typed version of the Equals method (which is actually the only method that the IEquatable<T> interface requires you to implement).
public class Fraction : IEquality<Fraction>
{
public bool Equals(Fraction other)
{
// ...
}
}
Whenever implementing IEquatable<T>, it is usually recommended to overload the two equality operators as well. By default, the equality contract verifier will evaluate them too. If your class does not override those operators, consider disabling that feature by setting to false the named parameter ImplementsOperatorOverload.
[TestFixture]
[VerifyEqualityContract(typeof(Fraction), ImplementsOperatorOverload = false)]
public class FractionTest : IEquivalenceClassProvider<Fraction>
{
// ...
}
4. It verifies that the equality operator (==) behaves as expected. The verifications are the same as for points 2 and 3. For nullable types, it is also verified that two null references are considered to be equal.
public class Fraction : IEquality<Fraction>
{
public static bool operator ==(Fraction left, Fraction right)
{
// ...
}
}
5. It verifies that the inequality operator (!=) behaves as expected.
public class Fraction : IEquality<Fraction>
{
public static bool operator !=(Fraction left, Fraction right)
{
// ...
}
}

2008/08/25

Contract Verifiers - Part II. - The Exception Contract Verifier

Designing custom exceptions is not a so easy task. The recommended guidelines are relatively strict; and writing unit tests for them is fastidious. That is why the exception contract verifier is useful. It helps testing your custom exceptions, and catching potential design and behavioral problems.

The contract verifier is available as a companion attribute for your test fixture. It adds the necessary test methods, which verify that your exception is well designed and behaves as expected.
public class MyCustomException : Exception
{
// ...
}
using Gallio.Framework;
using MbUnit.Framework;
using MbUnit.Framework.ContractVerifiers;

[TestFixture]
[VerifyExceptionContract(typeof(MyCustomException))]
public class MyCustomExceptionTest
{
}
Although the test fixture contains no explicit test methods, the test runner will execute 5 tests provided by the contract verifier attribute. Each of them corresponds to a specific guideline:

1. Provides a public default parameter-less constructor.
2. Provides a public constructor taking one single parameter for the error message .
3. Provides a public constructor taking two parameters for the error message and an inner exception.

public class MyCustomException : Exception
{
public MyCustomException()
{
// ...
}

public MyCustomException(string message)
{
// ...
}

public MyCustomException(string message, Exception innerException)
{
// ...
}
}
If for some reason, you decide that your custom exception should handle with construction differently, you can disable the 3 tests above by setting to false the named parameter ImplementsStandardConstructors.
[TestFixture]
[VerifyExceptionContract(typeof(MyCustomException), ImplementsStandardConstructors = false)]
public class MyCustomExceptionTest
{
}
4. Decorates your custom exception with the attribute System.SerializableAttribute.
[Serializable]
public class MyCustomException : Exception
{
}
5. Implements a protected constructor for data serialization.
public class MyCustomException : Exception, ISerializable
{
protected MyCustomException(SerializationInfo info, StreamingContext context)
{
// ...
}
}
Again, in some scenarios, you might decide to not support the serialization process (if your code base targets the compact framework, for instance). Disable the serialization tests by setting to false the named parameter ImplementsSerialization.
[TestFixture]
[VerifyExceptionContract(typeof(MyCustomException), ImplementsSerialization = false)]
public class MyCustomExceptionTest
{
}
When the serialization tests are enabled, the test runner will verify the preservation of data during a complete round trip serialization operation. This will be done for all the constructors; including the 3 recommended standard constructors described above.

The following sample shows the complete code for a general-purpose custom exception; and the test fixture verifying it.
[Serializable]

public class MyCustomException : Exception, ISerializable
{
public MyCustomException()
{
}

public MyCustomException(string message)
: base(message)
{
}

public MyCustomException(string message, Exception innerException)
: base(message, innerException)
{
}

protected MyCustomException(SerializationInfo info, StreamingContext context)
: base(info, context)
{
}

public override void GetObjectData(SerializationInfo info, StreamingContext context)
{
base.GetObjectData(info, context);
}
}
[TestFixture]
[VerifyExceptionContract(typeof(MyCustomException))]
public class MyCustomExceptionTest
{
}
And here is the report generated by Gallio for that test fixture: