2009/04/18

Asserting the inner exception

Asserting an expected exception is easy with MbUnit. Most of the time, the simple use of ExpectedExceptionAttribute (and similar) is sufficient:
[Test]
[ExpectedArgumentNullException]
public void DoSomething_with_null_argument_should_throw_exception()
{
var foo = new Foo();
foo.DoSomething(null);
}
This is a very handy solution. But it does not come without any inconvenient:
  1. Even if the test passes, you cannot be sure that the exception was thrown exactly where it was expected to be thrown. It is possible that another portion of the code has thrown an exception of the same type, before the expected failure point was reached. For example, a bug in the Foo constructor could have thrown the same exception; and it would make your test completely irrelevant because it would pass for a wrong reason.
  2. You cannot perform any additional assertion about the exception itself and its properties, like the message, or the inner exception.
Fortunately, MbUnit has a very useful Assert.Throws method, which solves all those limitations. It is very easy to use, in particular with the lambda expression syntax.
[Test]
public void DoSomething_with_null_argument_should_throw_exception()
{
var foo = new Foo();
Assert.Throws<ArgumentNullException>(() => foo.DoSomething(null));
}
Remark that ExpectedArgumentNullExceptionAttribute is no longer needed. Now, you are sure that the exception was thrown when passing a null reference to the DoSomething method, and not before, in an obscure part of the Foo constructor.

But wait! you can do even more! Unlike most of the other MbUnit assertion methods which just return void, Assert.Throws returns the instance of the exception which was caught. With that instance, you can make some more interesting assertions:

[Test]
public void DoSomething_with_null_argument_should_throw_exception()
{
var foo = new Foo();
var exception = Assert.Throws<ArgumentNullException>(() => foo.DoSomething(null));
Assert.AreEqual("Detected breach in space-time continuum.", exception.Message);
}
This is particularly useful for parameterized tests based on reflection. Indeed, reflective invocations encapsulate the exceptions into a TargetInvocationException instance, which makes the reason for the error a bit less discoverable:
public Foo2
{
public void SetInt32(int value) { // ... }
public void SetDouble(double value) { // ... }
public void SetInt16(short value) { // ... }
}
[Test]
[Row(typeof(int), 9999, "SetInt32")]
[Row(typeof(double), 9999.9, "SetDouble")]
[Row(typeof(short), 9999, "SetInt16")]
public void Set_too_large_value_should_throw_exception<T>(T tooLargeValue, string methodName)
{
var foo2 = new Foo2();
var method = typeof(Foo2).GetMethod(methodName);
var exception = Assert.Throws<TargetInvocationException>(() => method.Invoke(foo2, new object[] { tooLargeValue }));
Assert.IsInstanceOfType<ArgumentOutOfRangeException>(exception.InnerException);
}

2009/03/26

Contract Verifiers in MbUnit v3.0.7

Jeff Brown has announced the release of Gallio v3.0.6. This new major version contains numerous new features and fixes which have been implemented during the last 5 months mostly by Jeff and Graham. (See the release note for more details)

We made some notable additions to the contract verifier namespace as well:

  • New syntax based on the declaration of a read-only field rather than an attribute of the test fixture (breaking change)

  • Two new contract verifiers for collection-based classes (ICollection<T> and IList<T>)

  • New experimental contract verifier that checks for some aspects of immutability in a cumstom types
But the work is far for being complete. Here is what you can expect in the future v3.0.7:
  • New contract verifier for implementations of the generic IDictionary interface.

  • New property tester to easily verify the consistency of a property getter/setter.

  • Better immutability contract verifier with additional tests and more possible customizations.

  • Improved stack trace data handling (so that double-clicking a failing test will lead you directly to the declaration of your contract field)

  • Better failure messages (with a new Assert.Explain helper method to decorate inner assertion failures, so we can provide more meaningful messages)

  • More documentation in the Gallio Book.


So, let's go back to work...

2009/02/16

The Collection Contract Verifier

Gallio v3.0.6 is closed to be released. Among the numerous new features and fixes, mostly brought by Jeff and Graham, we have added a new contract verifier for the collection types.

The collection contract verifies that your implementation of the generic interface ICollection<T> behaves correctly. In particular, the contract verifier tests the addition and the deletion of items, but also the consistency of the Count and the ReadOnly properties.

The syntax for declaring a collection contract verifier is very similar to the other contracts:
[VerifyContract]
public readonly IContract CollectionTests = new CollectionContract<TCollection, TItem>
{
// Some Options...
};
You may specify some optional properties to make the contract verifier better fit your specific needs. For instance, setting AcceptEqualItems to true indicates that the collection is expected to accept duplicate items (object equality). You can also provide a custom default instance of the collection by feeding GetDefaultInstance (by default the default constructor is invoked).

To completely test a collection type is a particular painful task. The new collection contract verifier is then probably a nice time saver. However, the verifier expects your collection to have a strictly regular behavior, such as List<T>. It will probably not be suitable for a custom collection which treats addition or deletion of items with a different logic. We could have made the verifier even more flexible with additional options, but at a certain point, it is probably better to write manually the unit tests with ones own little fingers.

2009/01/28

Generic syntax for the MbUnit reflective assertions.

The MbUnit v3 assertions related to type checking have been extended with a generic syntax counterpart. The four assertion methods:
  • Assert.IsInstanceOfType(Type expectedType, object actualObject);
  • Assert.IsNotInstanceOfType(Type unexpectedType, object actualObject);
  • Assert.IsAssignableFrom(Type expectedType, object actualObject);
  • Assert.IsNotAssignableFrom(Type unexpectedType, object actualObject);
And their overloads with a custom message as well, do now exist with the following syntax:
  • Assert.IsInstanceOfType<TExpected>(object actualObject);
  • Assert.IsNotInstanceOfType<TUnexpected>(object actualObject);
  • Assert.IsAssignableFrom<TExpected>(object actualObject);
  • Assert.IsNotAssignableFrom<TUnexpected>(object actualObject);
Those new methods are available in Gallio v3.0.6.649 and later.

2009/01/07

The Immutability Contract Verifier

MbUnit v3 has a new contract verifier called VerifyImmutabilityContract. Basically, it verifies that every field of the tested type is declared with the readonly keyword. It is important to note that the verification is done recursively over all the inner field types, until some known primitive is found, such as String, Boolean, or Int32.

The original idea comes from Jeff Brown which was inspired by a post of Oren Eini.

Here is simple class that implements that sort of immutability, and a test fixture which declares the contract verifier.
public class Foo
{
private readonly int value;
private readonly string name;

// some code here...
}
[TestFixture]
public class FooTest
{
[VerifyContract]
public readonly IContract ImmutabilityTests = new ImmutabilityContract<Foo>();
}
So far, the contract verifier generates only one test method named AreAllFieldsReadOnly.

2008/12/19

New Syntax For Contract Verifiers (Release)

Today, we have commited in the Gallio development trunk, the new syntax for the MbUnit contract verifiers.

That new syntax is a major breaking change. After you update Gallio with the latest version (3.0.5.589 and later), you will need to modify the declaration of your contract verifiers to make your project compilable again. Download the latest development build here.

Please refer to that post for additional details about the new syntax.