Saturday, January 26, 2008

Expected exception

If you use the ExpectedExceptionAttribute to assert that an exception is raised in an NUnit-test the type of the thrown exception must match the actual exception's type exactly, for example if you check for an ArgumentException and an ArgumentNullException  is thrown the assertion will fail. To me this is wrong since an ArgumentNullException really is an ArgumentException. Of course you could argue that you shouldn't check for general exceptions for the same reasons as you shouldn't catch general exceptions, which I certainly agree with but I'd really leave that up to the developer.

The really big drawback of the attribute solution is that if you have code that's setting up the test objects and does other preparations for the test and this set up code by chance throws an exception of the expected type the test will succeed when it's really not even been run. 

Another issue is that when the expected exception is thrown - naturally - execution of that test is stopped resulting in execution never exits the test method. Now if you're using NCover for test coverage you don't have 100% coverage of your test cases since a line of code (the method exit) was missed.

Missed line

To avoid this I created a helper method that checks for expected exceptions, another benefit from this is that it can be used with any testing framework. It uses the template method pattern I blogged about a week or so back.

I've implemented the method in a static class called TestHelper and I show an excerpt of it here:

/// <summary>
/// Provides helper methods for testing.
/// </summary>
public static class TestHelper
{
    /// <summary>
    /// Expects the supplied method to throw an exception of the specified
    /// type {T} when invoked. If no exception is thrown or if an exception
    /// of another type is thrown an exception will be thrown making the
    /// assertion fail.
    /// </summary>
    /// <typeparam name="T">The type of <see cref="Exception" /> to expect.</typeparam>
    /// <param name="method">The method that's expected to throw an exception.</param>
    public static void ExpectException<T>(Action method) where T : Exception 
    {
        bool expectedExceptionThrown = false;

        try
        {
            method();
        }
        catch (T)
        {
            expectedExceptionThrown = true;
        }
        catch (Exception unexpectedException)
        {
            // Not correct exception type, for demonstration only.
            throw new Exception("Unexpected exception type."); 
        }

        if (!expectedExceptionThrown)
        {
            var message = string.Format(CultureInfo.InvariantCulture,
                "'{0}' was expected but no exception was thrown.", typeof(T).Name);
            // Not correct exception type, for demonstration only.
            throw new Exception(message);
        }
    }
}

Now tests where I expect exceptions are implemented as any other test without the ExpectedExceptionAttribute. For example:

[Test]
public void ToString_NullObject_Exception()
{
    // Set up code. If an exception is thrown here the
    // test will fail.
    StringBuilder testedObject;
    testedObject = null;
    
    TestHelper.ExpectException<NullReferenceException>(() => {
        // The actual test code, a NullReferenceException must
        // be thrown here or the test will fail.
        testedObject.ToString();
    });
}

No comments:

Post a Comment