Thursday, December 20, 2007

Testing Servlets with JUnit

In my day-to-day job I write a lot of unit tests, and I try to test everything I can. Recently though I had written a service that I was unable to test completely. One of the methods in the service had the job of sending an HTTP request to a remote server and responding with the results. For my project I used commons-httpclient to send the request.

@SuppressWarnings("unchecked")
public String sendHttpPost (String url, String queryString)
throws Exception
{
String result = null;

try {
HttpClient client = new HttpClient();
PostMethod post = new PostMethod(url);
post.setQueryString(queryString);
client.executeMethod(post);

result = post.getResponseBodyAsString();
post.releaseConnection();
}
catch (Exception e) {
throw new Exception("post failed", e);
}

return result;
}


If you have used commons-httpclient, this is about as simple as it gets, but I still wanted to have a unit test for it. So the problem became, "how can I test this method when it requires that I hit a web site". After some searching I found that the Jetty servlet-container has a ServletTester class just for this purpose.

In my Maven 2 I included the ServletTester using the following repository and dependency information. I always use Maven 2 as allows other developers to quickly set up their IDE and download all required JARs. If you don't use Maven, you will need to manually download all of the dependencies.

<repositories>
<repository>
<id>codehaus-release-repo</id>
<name>Codehaus Release Repo</name>
<url>http://repository.codehaus.org</url>
</repository>
</repositories>

...

<dependency>
<groupId>org.mortbay.jetty</groupId>
<artifactId>jetty-servlet-tester</artifactId>
<version>6.1.6</version>
<scope>test</scope>
</dependency>


With this in place the next step was writing the JUnit test cases. For me I wanted to initialize the servlet-container once, then run a set of tests against it. In JUnit 4 you can use the @BeforeClass and @AfterClass annotations to mark methods that should be executed before and after all of the tests.

public class HttpPostServiceTest
{
private static ServletTester tester;
private static String baseUrl;


/**
* This kicks off an instance of the Jetty
* servlet container so that we can hit it.
* We register an echo service that simply
* returns the parameters passed to it.
*/
@BeforeClass
public static void initServletContainer () throws Exception
{
tester = new ServletTester();
tester.setContextPath("/");
tester.addServlet(EchoServlet.class, "/echo");
baseUrl = tester.createSocketConnector(true);
tester.start();

}

/**
* Stops the Jetty container.
*/
@AfterClass
public static void cleanupServletContainer () throws Exception
{
tester.stop();
}
}


The code highlighted in blue is where we start an instance of the server. I created a new instance of the ServletTester, setting the context path and adding a servlet mapping. This alone does not bind the server to a port, for that you need to call createSocketConnector(true), which binds the server to a local port and returns the URL. The port used will be a high unused port. I save an instance of the ServletTester so that I can stop the service in the @AfterClass block, and I save the baseUrl so that I can target it in my tests.

The servlet I added to the container I called EchoServlet. This servlet simply echos the parameters passed to it.

public class EchoServlet extends GenericServlet
{

@SuppressWarnings("unchecked")
@Override
public void service (ServletRequest request, ServletResponse response)
throws ServletException, IOException
{
PrintWriter out = response.getWriter();
Map<String, String[]> params =
new TreeMap<String, String[]>(request.getParameterMap());

out.println("SIZE=" + params.size());
for (Entry<String, String[]> entry : params.entrySet()) {
out.println(entry.getKey() + ":::"
+ StringUtils.join(entry.getValue(), ","));
}
}
}


The servlet takes the parameter map and creates a TreeMap out of it. The TreeMap is needed so that the parameter names are returned in sorted order. Being able to predict the order of the returned keys is required in order to test against the output of the servlet.

To finish things up, I just needed to write a unit test.

@Test
public void testPost () throws Exception
{
PostService svc = new PostService();

String res = svc.sendHttpPost(baseUrl + "/echo",
"foo=bar&baz=%25%26%3D%2F");
String[] resList = StringUtils.split(res, "\n");

assertEquals(3, resList.length);
assertEquals("SIZE=2", resList[0].trim());
assertEquals("foo:::bar", resList[1].trim());
assertEquals("baz:::%&=/", resList[2].trim());
}


After that, run the test and watch it work.

31 comments:

Ajanta said...

Your instructions worked beautifully for me. I know there is a way to use the servlet filters as well though i was not able to get it working. Would appreciate if you could provide any insight on that too..
Thanks!

Robert Hanson said...

Ajanta, you can use Spring-Mock for that... even if you aren't using Spring.

Filter filter = new MyFilter();

MockHttpServletRequest req = new MockHttpServletRequest("GET", "/");
MockHttpServletResponse res = new MockHttpServletResponse();
MockFilterChain chain = new MockFilterChain();

filter.doFilter(req, res, chain);

After the filter executed you can use the methods on the mock request/response objects to see verify that the filter did what it was supposed to do.

I am not sure where this library is listed on the Spring site, but you can download it from the Maven repository here, http://repo1.maven.org/maven2/org/springframework/spring-mock/.

Ajanta said...

Hey, thanks for your quick response, I think ServletTester also provides a addFilter method and I tried using that but it didn't work. Have you used that?

Robert Hanson said...

No, I haven't tried the addFilter for ServletTester (yet), but I expect that would work well.

In my case though it is easier/better to use mock object for filter testing because I can set/inspect things in the mock objects that I couldn't do without bending over backwards in a server environment.

Erik Vonderheid said...

Have you ever tried the apache cactus framework?

Robert Hanson said...

Erik, nope, never used Cactus. I have heard about it, and probably looked at the site once or twice, but never got around to actually trying it.

I just took a look at the Quick Start guide and see one thing that immediately turns me off of using it, the part where it asks me to install Tomcat.

*I* don't have a problem installing Tomcat, but requiring that my developers do this adds time to getting them up to speed on a project... And asking my clients to do this is too much to ask.

My goal is that any developer or client can take the source or a project and run "mvn test" to automatically download and run the tests. I want Maven to be the only tool that needs to be required to test, compile, and package the app.

Of course... I don't know much about Cactus, so maybe I am making an incorrect assumption

Damian said...

Hi, I'm looking at using the new Jetty ServletTester utilities for unit testing servlets. I'm stuck attempting to mock out a HttpSession for any servlets that require one. Any ideas?

Robert Hanson said...

Damian,

You are looking for ideas on how to mock HTTPSession without using a servlet container?

Check out spring-mock, it has all of that stuff and is well maintained.

JARs from Maven repo:
http://repo1.maven.org/maven2/org/springframework/spring-mock/

Random article:
http://www.devx.com/Java/Article/30067

Mike Kaufman said...

Robert, Damian,

A shameless plug... another way you can now "mock" Servlet API objects for out-of-container testing is using my own "ObMimic" library.

This provides plain-Java classes that cover the whole Servlet API for precisely this purpose. It covers all of the Servlet API's interfaces and abstract classes (including HttpSession), and completely implements all of their methods. It does this with plain Java objects that are fully configurable, fully inspectable, and have some additional features to help in testing (e.g. record/examine the Servlet API calls made to each object, report any ambiguous or questionable Servlet API calls, switch between Servlet 2.3, 2.4, 2.5).

Because there are no unsupported or incomplete Servlet API methods it can handle even complex use of the API (e.g. request-dispatcher "forwards" and "includes", listener notifications, servlet mappings, filter chains etc), and as a result you can use it not only for pure "unit" testing but also for testing more extensive paths through your code, framework code, and other libraries.

As of May/June 2008 this is in private "beta" release, but if you want to take a look at it just let me know and we'll set you up. For more details, see my recent blog posting.

Robert Hanson said...

Mike,

> A shameless plug

Shameless plugs are welcome... just as long as it is on topic.

> As of May/June 2008 this is in
> private "beta" release, but if you
> want to take a look at it just let
> me know

I am interested, but don't currently have any projects on my plate right now that need this. If something does come up, you can definitely expect to hear from me.

Thanks for letting me(us) know about this.

toupil said...
This comment has been removed by a blog administrator.
Himanshu said...

Hi, a very nice tutorial.Please correct the font color of the code. Its appears nearly invisible as font is of white color in white background. Thanks

Anonymous said...

Hi Robert. Thanks for sharing your experience.

Basically, the ServletTester works well, but I faced the following problem and maybe you can help me:

Can you tell me which values you take for the contextPath and the DocumentBase? For both there are setters and getters, but using a contextpath like "/webapp" doesn't works when resources are to be loaded. For example, when using servlet-methods like "getServletContext().getRealPath("WEB-INF/classes")", which work when used inside a "stand-alone" container, return null when used with the ServletTester.

Thanks and best regards, Reza (from germany ;-) ).

Robert Hanson said...

Himanshu, it should be better now. Thanks for mentioning it.

Himanshu said...

That's good now n legible. Well despite repeated attempts your tutorial never worked for me and I,ve to satisfy with spring mock objects

W-Mark Kubacki said...

Indeed, saved me some time. Thanks!

But, the blue font color is not optimal, too. With black background it appears blurry.

Robert Hanson said...

> Anonymous said...
> using a contextpath like
> "/webapp" doesn't works when
> resources are to be loaded.

I really don't know. This example wasn't meant to load resources. If I needed to do that, for instance if I needed to pass a test XML file back to the caller, I would just use File to open/read the file then simply print it to the response object.

Nima said...

Hi Robert, The tutorial is great. I just wonder if you know of a way by which I can add a servlet instance to ServletTester rather than passing the servlet class. My servlet object should receive a db storage object in its constructor and thus I need to instantiate the servlet before passing it to ServletTester.

However, it seems like when testing a servlet you can only pass the servlet class to ServletTester and let it instantiate the servlet. This causes null pointer exceptions in my case which is causing problem.

Do you have any idea or suggestion on how this can be solved?

thanks a lot

Robert Hanson said...

Nima, sorry, I don't know how you might do that without either changing the ServletTester or the servlet.

Kim Hansen said...

Thanks Robert.

Your post helped me successfully set up a testing servlet context with Jetty. Something that 5 Spring books and 10 Java books and lotsa searching on the Internet failed to do!

rop said...

Hi,

I tried this, but I couldnt figure out where the class PostService comes from?

What is the package of that class?

Or which dependency/maven-coordinate do I need to include?


> PostService svc = new PostService();

rop said...

Hi,

I tried to get this working, but where does that PostService class come from?

What package/dependency do I need?

> PostService svc = new PostService();

Robert Hanson said...

PostService would be your code that you are trying to test. It could be called anything, and could use any API to make the HTTP call.

In this case it is also a reference to the sendHttpPost() method that was at the top of the post, which just happens to live in a class called PostService (which is not shown). The PostService.sendHttpPost() method (from the top of the post) uses commons-httpclient 3.1, but like I said, your service could use any API that you needed to use.

Hope that helps.

Anonymous said...

Just so you know this fits more under the umbrella of integration testing. Typically to "Unit Test" that the class is working correctly you'd simply expose enough of your inner workings of your class to your testing framework to verify that the calls to the integration point (the HTTP client library you're using in this case) are correctly configured. By attempting to ensure that the dependency you're using is actually working correctly you are in effect verifying that another class is doing something. So you're no longer testing a unit.

Anonymous said...

How can i write a junit test for a servlet that instantiates a class and then processes request to the server by getting the servlet time back. And then generates a response back to server with DATA.
eg like so [ response.getOutputstream().println(DATA + (new DATE());

Robert Hanson said...

@Anonymous,

I'm not sure I understand the question. Does it boil down to "how do I test a date value?"

For tests that involve dates I typically make use of JodaTime classes instead of java.util.Date. http://joda-time.sourceforge.net/.

JodaTime is a lot better than what comes with the JDK, and also allows you to freeze time.

// set and freeze time
DateTimeUtils.setCurrentMillisFixed(millis);

This allows you to not only freeze time but also allows you to test specific time periods that require additional testing. E.g. DST, new years, millennium bug.

After you have tested that the servlet returned the exact time that you froze time at you can then reset/unfreeze it.

DateTimeUtils.setCurrentMillisSystem();

JodaTime classes also allow conversion to/from java.util.Date. So if your code relies on java.util.Date you could mix the two, using JodaTime for constructing a new Date. E.g. "Date d = new DateTime().toDate()" instead of "Date d = new Date()".

Again... if I understand the question correctly.

Anonymous said...

I want to write a unit test for doing the following service to the servlet.
This is a rough idea what the service is doing :

response.setContentType("text/html");
StringBuffer responseBuf = new StringBuffer();
List valueList = new ArrayList();

// iterate through the valueList
for (Values ItemValue : valueList) {
responseBuf.append((String.valueOf(ItemValue.getStartTime() + "" + (String.valueOf(ItemValue.getEndTime() );

}
response.getOutputStream().println("" +(new Date().getTime()) + responseBuf.toString());
response.getOutputStream().close();

JaGoTerr said...

Awesome post, thank U so much!

PS: Cactus made me mad today, I'm glad to drop it :)

Anonymous said...

How to set values in Request Attributes. ServletTester has setAttribute which sets all the values Attribute to context. not to the request object..Is there any way to set this value?
In my Servlet I am accessing like this

request.getAttribute("name");

Krishan Gandhi said...

Hi Robert,

Please share the jar details for this class.

PostService svc = new PostService();


I have used this repository and dependency in in pom.xml



codehaus-release-repo
Codehaus Release Repo
http://repository.codehaus.org



...


org.mortbay.jetty
jetty-servlet-tester
6.1.6
test

Anonymous said...

Can't see your code samples, the background color is killing it.