Grabbing Screenshots of Failed Selenium Tests

For the GlassFish Administration Console, we have quite a few tests (about 133 at last count). Given the nature and architecture of the application, we’ve chosen Selenium to drive our tests. One of the problems we’ve faced, though, is understanding why a test failed due to the length of time the tests take (roughly 1.5 hours to run the whole suite). Sometimes, we can look at the log and know exactly what failed, but not the why. Did the screen render correctly? Did, perhaps, the click, etc. not get performed (we’ve seen instances of that) leaving the application in a state not expected by the test? Since I usually start the tests and move on to something else, we had no way of knowing. Until now. I finally sat down and figured out how to grab a screen shot when a test fails. I’ve distilled that technique down to its essentials, which I’ll share here.

In this example, we’re going make sure example.com works correctly. Sort of. : ) What we need to do first, though, is set up our project, which we’ll configure (for simplicity’s sake) as a simple Maven-based Java application. The important pom.xml elements are these:

    <dependencies>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.8.2</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.seleniumhq.selenium</groupId>
            <artifactId>selenium-server</artifactId>
            <version>2.14.0</version>
            <exclusions>
                <exclusion>
                    <groupId>org.testng</groupId>
                    <artifactId>testng</artifactId>
                </exclusion>
            </exclusions>
        </dependency>
    </dependencies>
    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>2.3.2</version>
                <configuration>
                    <source>1.6</source>
                    <target>1.6</target>
                    <showDeprecation>true</showDeprecation>
                </configuration>
            </plugin>
        </plugins>
    </build>

Simple enough. Now, the test:

public class ScreenshotDemoTest {
    private static WebDriver driver;
    private static Selenium selenium;
    @Rule
    public ScreenshotTestRule screenshotTestRule = new ScreenshotTestRule();

    @BeforeClass
    public static void beforeClass() {
        driver = new FirefoxDriver();
        selenium = new WebDriverBackedSelenium(driver, "http://example.com");
    }

    @AfterClass
    public static void afterClass() {
        selenium.close();
    }

    @Test
    public void testThatSucceeds() {
        selenium.open("/");
        assertTrue(selenium.isTextPresent("As described"));
    }

    @Test
    public void testThatFails() {
        selenium.open("/");
        assertTrue(selenium.isTextPresent("Your test should fail here"));
    }
}

This is an extremely simple test that should make sense to those familiar with Selenium. The interesting part here is lines 4 and 5. What’s that Rule? That little nugget is a means of extending JUnit in an AOP fashion. In our case, that’s where the magic is going to happen, so let’s take a look at ScreenshotTestRule:

    class ScreenshotTestRule implements MethodRule {
        public Statement apply(final Statement statement, final FrameworkMethod frameworkMethod, final Object o) {
            return new Statement() {
                @Override
                public void evaluate() throws Throwable {
                    try {
                        statement.evaluate();
                    } catch (Throwable t) {
                        captureScreenshot(frameworkMethod.getName());
                        throw t; // rethrow to allow the failure to be reported to JUnit
                    }
                }

                public void captureScreenshot(String fileName) {
                    try {
                        new File("target/surefire-reports/").mkdirs(); // Insure directory is there
                        FileOutputStream out = new FileOutputStream("target/surefire-reports/screenshot-" + fileName + ".png");
                        out.write(((TakesScreenshot) driver).getScreenshotAs(OutputType.BYTES));
                        out.close();
                    } catch (Exception e) {
                        // No need to crash the tests if the screenshot fails
                    }
                }
            };
        }
    }

Implementations of MethodRule act as an interceptor for your tests. You can do all the usual types of things you might do in an interceptor (in fact, in GlassFish, we use this is allow us to run only specific test methods, e.g., mvn -Dtest=MyTest -Dmethod=testMethod1,testMethod3). Here, though, we want to run every test, but, in the case of failures, which present themselves as Exceptions, we wan’t to capture the screenshot. Once we’ve saved the image to a file (note the assumption that we’re running under Maven in captureScreenshot()), we rethrow the Throwable to make sure the failure is reported.

If you run these tests, you should see one success and one failure, and you should see target/surefire-reports/screenshot-testThatFails.png. How easy was that?! : )

Full source code can be found here. I hope this helps you as much as it has me. : )

Popularity: 3% [?]