Friday, December 21, 2012

JUnit: Skip test on error

I recently run into a situation where a JUnit test possibly crashed because of an OutOfMemoryError. In my case this error did not indicate that the test code or the test assumptions did not pass. It rather was a sign that the test environment (and its setup) did not fit to the test. This could happen because I run the test with different profiles (using a dedicated testing setup but also an integration setup).

The OutOfMemoryError was not generally expected in my test, so the following solution did not fit to handle the situation as it would need an OutOfMemoryError to pass the test.

    @Test(expected = OutOfMemoryError.class)
    public void doSomething() {
        // memory expensive test
    }


My first attempt to handle the situation was to catch the error and ignore it.

    @Test
    public void doSomething() {
        try {
            // memory expensive test
        } catch (OutOfMemoryError ignore) {}
    }


But this would report the test as passed 

    Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.09 sec

what is definitely a misinterpretation.

I finally found out that JUnit offers a  
  
    public static void assumeNoException(java.lang.Throwable t) 

operation that perfectly fits to my problem:

    @Test
    public void doSomething() {
        try {
           
// memory expensive test
         } catch (OutOfMemoryError ex) {
            Assume.assumeNoException(ex);
        }
    }


In test runs without the OutOfMemoryError the test is reported as run without errors/failures:

    Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.15 sec

and in test runs with the  OutOfMemoryError it will be skipped and appropriate reported:

    Tests run: 1, Failures: 0, Errors: 0, Skipped: 1