Automated Testing Layers and Execution Configuration

One of the most beneficial things you can do when developing production quality applications is apply automated testing in a few different layers. In my most recent projects we’ve successfully built and deployed applications with no dedicated QA and minimal amounts of manual testing. First lets take a look at our layers.

1. Unit Tests
2. Spring Database/MVC Integration tests
3. Pre-deployment Selenium Integration Tests
4. Post-deployment Selenium Integration Tests

and then how we need to configure our tests for execution during

1. development
2. build

Unit Tests

Unit testing give us a few advantages. To put it bluntly, it forces you to design and code loosely coupled objects like we’ve always been taught, but not always practiced. This isn’t to say that other code isn’t testable, but it becomes a huge nightmare when you are testing code that does 100 different things.

Next, Unit Testing also documents our code. By writing a unit test I am telling other developers who end up working on the code “this is how it should work in these situations” and “these are my assumptions for this code”. Unit test combined with verbose self documenting code helps eliminate our need for large amounts of javadocs and other written documentations (not to say you should write no documentation, but you should understand when and where it is necessary).

The main thing to understand here is that these test do not determine if your system actually works. They just make sure the assumptions of the initial developer hold true for a given unit (class in this case).

Let’s take a look at an example unit test where we isolate our class under test using mockito to mock out its dependencies.

@RunWith(MockitoJUnitRunner.class)
public class ManifestServiceImplTest {
	private static final long FILEID = -1L;
	@InjectMocks
	private ManifestService manifestServiceImpl = new ManifestServiceImpl();
	@Mock
	private UserService userService;
	@Mock
	private MidService midService;
	@Mock
	private ManifestRepository manifestRepository;
	private Manifest manifest;
	private User user;
	private final String username = "abc";
	@Captor
	private ArgumentCaptor<Pageable> manifestPageCaptor;

	@Before
	public void setup() {
		user = new User();
		user.setUsername(username);
		when(userService.find(username)).thenReturn(user);
		manifest = new Manifest();
		when(manifestRepository.findOne(FILEID)).thenReturn(manifest);
		when(manifestRepository.save(manifest)).thenReturn(manifest);
	}

	@Test
	public void getAvailableEnvNone() {
		when(midService.hasCompletedMidCertificationStatus(username))
				.thenReturn(false);
		when(midService.hasIncompletedMidCertificationStatus(username))
				.thenReturn(false);
		assertTrue("no manifestEnvs should be returned if user has no mid",
				manifestServiceImpl.getAvailableManifestEnvs(username)
						.isEmpty());
	}

	@Test
	public void getAvailableEnvCompleteOnly() {
		when(midService.hasCompletedMidCertificationStatus(username))
				.thenReturn(true);
		when(midService.hasIncompletedMidCertificationStatus(username))
				.thenReturn(false);
		Set<ManifestEnv> manifestEnvs = manifestServiceImpl
				.getAvailableManifestEnvs(username);
		assertEquals(
				"manifestEnvs should have 2 entries when user only has completed mid cert",
				2, manifestEnvs.size());
		assertTrue("manifestEnvs should contain all ManifestEnv enums",
				manifestEnvs.containsAll(Arrays.asList(ManifestEnv.values())));
	}

	@

	Test
	public void getAvailableEnvIncompletOnly() {
		when(midService.hasCompletedMidCertificationStatus(username))
				.thenReturn(false);
		when(midService.hasIncompletedMidCertificationStatus(username))
				.thenReturn(true);
		Set<ManifestEnv> manifestEnvs = manifestServiceImpl
				.getAvailableManifestEnvs(username);
		assertEquals(
				"manifestEnvs hsould only have 1 entry when user has only incomplete mid cert",
				1, manifestEnvs.size());
		assertTrue("mainfestEnvs should only contain TEM",
				manifestEnvs.contains(ManifestEnv.TEM));
	}

	@Test
	public void getAvailableEnvBoth() {
		when(midService.hasCompletedMidCertificationStatus(username))
				.thenReturn(true);
		when(midService.hasIncompletedMidCertificationStatus(username))
				.thenReturn(true);
		Set<ManifestEnv> manifestEnvs = manifestServiceImpl
				.getAvailableManifestEnvs(username);
		assertEquals(
				"manifestEnvs should have 2 entries when user only has completed mid cert",
				2, manifestEnvs.size());
		assertTrue("manifestEnvs should contain all ManifestEnv enums",
				manifestEnvs.containsAll(Arrays.asList(ManifestEnv.values())));
	}

	@Test
	public void find() {
		when(manifestRepository.findOne(FILEID)).thenReturn(manifest);
		final Manifest returnedManifest = manifestServiceImpl.find(FILEID);
		verify(manifestRepository).findOne(FILEID);
		assertEquals("manifest should be returned when found by FILEID",
				manifest, returnedManifest);
	}

	@Test
	public void findNotFound() {
		when(manifestRepository.findOne(FILEID)).thenReturn(null);
		final Manifest returnedManifest = manifestServiceImpl.find(FILEID);
		verify(manifestRepository).findOne(FILEID);
		assertEquals(
				"null should be returned when a manifest file is not found",
				null, returnedManifest);
	}

	@Test
	public void findUserManifestHistory() {
		final Page<Manifest> page = new PageImpl<Manifest>(
				Lists.newArrayList(manifest));
		when(
				manifestRepository.findByUserUsernameOrderByCreatedTimeDesc(
						eq("abc"), isA(Pageable.class))).thenReturn(page);
		manifestServiceImpl.findUserManifestHistory("abc");
		verify(manifestRepository).findByUserUsernameOrderByCreatedTimeDesc(
				eq("abc"), manifestPageCaptor.capture());
		assertEquals("user manifest histroy should be max 7 for page size", 7,
				manifestPageCaptor.getValue().getPageSize());
		assertEquals(
				"user manifest histroy should always return the first page", 0,
				manifestPageCaptor.getValue().getPageNumber());
	}

	@Test
	public void create() {
		final Manifest returnedManifest = manifestServiceImpl.create(manifest,
				username);
		assertEquals("user should be set to manifest when creating", user,
				returnedManifest.getUser());
		verify(manifestRepository).save(manifest);
	}

}

and our class under test

@Service
public class ManifestServiceImpl implements ManifestService {

	@Autowired
	private ManifestRepository manifestRepository;

	@Autowired
	private UserService userService;

	@Autowired
	private MidService midService;

	@Override
	public Manifest create(final Manifest manifest, final String username) {
		Validate.notNull(manifest);
		Validate.notBlank(username);
		final User user = userService.find(username);
		Validate.notNull(user);
		manifest.setUser(user);
		return manifestRepository.save(manifest);
	}

	@Override
	public Set<ManifestEnv> getAvailableManifestEnvs(final String username) {
		Validate.notBlank(username);
		final Set<ManifestEnv> envs = Sets.newHashSet();
		if (midService.hasCompletedMidCertificationStatus(username)) {
			envs.add(ManifestEnv.PROD);
			envs.add(ManifestEnv.TEM);
			return envs;
		}
		if (midService.hasIncompletedMidCertificationStatus(username)) {
			envs.add(ManifestEnv.TEM);
		}
		return envs;
	}

	@Override
	@PostAuthorize("returnObject == null or returnObject.user.username == principal.username")
	public Manifest find(final long id) {
		return manifestRepository.findOne(id);
	}

	@Override
	public Page<Manifest> findUserManifestHistory(final String username) {
		Validate.notBlank(username);
		final Pageable pageable = new PageRequest(0, 7);
		return manifestRepository.findByUserUsernameOrderByCreatedTimeDesc(
				username, pageable);
	}
}

I mainly want to look at the find method so we can demonstrate what unit tests do and do not do for us.

The find method does nothing except delegate to a spring data repository interface. So we have nothing really to test except that the method got called. But even so we actually have two test methods for this method. The question is why? We can’t test much functionality here since our method isn’t doing much on its own, and code coverage would be 100% with only a single test method verifying the mocked out object had its findOne method called. But we have a found and notFound test, and this is because our interface can return a null or non null object. So here we aren’t really testing anything, but we are documenting that our interface will return null if nothing is found by the repository.

That being said, lets move on to some test that actually test our system (paritally) as a whole.

Spring Database and MVC Integration Tests

This is our first layer of integration tests. We use the spring-test framework for building these tests that focus on our spring container and down. These test spin up their own spring context and are not deployed to a container. Our test also have full access to our spring context so we can inject beans into our test class.

First lets look at a spring database integration test for our MainfestServiceImpl class to compare it with the unit tests we created.

@FlywayTest
@DBUnitSupport(loadFilesForRun = { "CLEAN_INSERT", "/dbunit/dbunit.base.xml",
		"CLEAN_INSERT", "/dbunit/dbunit.dev.base.xml", "CLEAN_INSERT",
		"/dbunit/dbunit.manifest.xml", "CLEAN_INSERT", "/dbunit/dbunit.mid.xml" })
public class ManifestServiceImplSpringDatabaseITest extends
		AbstractSpringDatabaseTest {

	@Autowired
	private ManifestService manifestService;

	@Autowired
	private UserRepository userRepository;
	private User user;

	@Before
	public void setup() {
		user = userRepository.findOne(-1L);
	}

	@Test
	@DBUnitSupport(loadFilesForRun = { "CLEAN_INSERT",
			"/dbunit/dbunit.mid.cert.incomplete.xml" })
	public void getAvailableManifestEnvsTEMOnly() {
		Set<ManifestEnv> manifestEnvs = manifestService
				.getAvailableManifestEnvs(user.getUsername());
		assertEquals(
				"only one ManifestEnv should be returned when user only has incomplete mid",
				1, manifestEnvs.size());
		assertTrue("returned manifestEnvs should contain TEM",
				manifestEnvs.contains(ManifestEnv.TEM));
	}

	@Test
	@DBUnitSupport(loadFilesForRun = { "CLEAN_INSERT",
			"/dbunit/dbunit.mid.cert.complete.xml" })
	public void getAvailableManifestEnvsTEMAndPROD() {
		Set<ManifestEnv> manifestEnvs = manifestService
				.getAvailableManifestEnvs(user.getUsername());
		assertEquals(
				"only TEM and PROD should be returned when user only has complete mid",
				2, manifestEnvs.size());
		assertTrue("returned manifestEnvs should contain TEM and PROD",
				manifestEnvs.containsAll(Arrays.asList(ManifestEnv.values())));
	}

	@Test
	public void findFound() {
		assertNotNull(manifestService.find(-1L));
	}

	@Test
	public void findNotFound() {
		assertNull("null should be returned when file not found",
				manifestService.find(-10L));
	}

	@Test
	@DBUnitSupport(loadFilesForRun = { "CLEAN_INSERT",
			"/dbunit/dbunit.manifest.xml" })
	public void findUserManifestHistory() {
		assertEquals("user should have 2 manifest in their history", 2,
				manifestService.findUserManifestHistory(user.getUsername())
						.getNumberOfElements());
	}

	@Test
	public void create() throws IOException {
		final Manifest manifest = new Manifest();
		manifest.setEnvironment(ManifestEnv.PROD);
		byte[] data = "hello".getBytes();
		final MultipartFile multipartFile = new MockMultipartFile("somefile",
				"file.txt", null, new ByteArrayInputStream(data));

		manifest.setMultipartFile(multipartFile);
		final Manifest returnManifest = manifestService.create(manifest,
				user.getUsername());
		assertTrue(returnManifest.getFileSystemResource().exists());
		assertNotNull("id should be set for saved manifest", manifest.getId());
		assertNotNull("createdTime should be set on manifest",
				manifest.getCreatedTime());
		assertNotNull("path should be set on manifest", manifest.getPath());
		assertNotNull("filename should be set on manifest",
				manifest.getFilename());
		assertEquals("file should be saved when manifest is saved",
				data.length, IOUtils.toByteArray(manifest
						.getFileSystemResource().getInputStream()).length);
	}
}

Again, similar test methods, but this time our intentions are different. For integration test we are now actually testing the application as it would be run in a live environment. We don’t mock anything here, but we do need to setup test data in our database so that our tests are reproducible.

Let’s look at the create method this time. Since our object is a hibernate entity, we expect hibernate to perform some operations for us. In this case, our entity has a prePersist method that writes a file to the file system before saving our entity to the database. That method also sets up the state of our entity by storing the path to the file, its original filename, time it was created, and hibernate assigns an id.

The @Flyway annotation handles our database lifecycle. It can be placed on the method/class level and will clean and rebuild the database. This combined with the @DBUnitSupport annotation lets us fully control the state of our database for each test. See https://github.com/flyway/flyway-test-extensions for more information.

That being said, lets take a look at the AbstractSpringDatabaseTest class that we extend so we can see how everything is configured for these tests.

@RunWith(SpringJUnit4ClassRunner.class)
@Category(IntegrationTest.class)
@ContextConfiguration(classes = { SpringTestConfig.class })
@ActiveProfiles({ "DEV_SERVICES", "TEST_DB", "DEV_SECURITY" })
@TestExecutionListeners({ DependencyInjectionTestExecutionListener.class,
		FlywayDBUnitTestExecutionListener.class,
		TransactionalTestExecutionListener.class })
@TransactionConfiguration(defaultRollback = false)
@Transactional("transactionManager")
public class AbstractSpringDatabaseTest {

}

So few things here. First is the @RunWith and @ContextConfiguration annotations setup our spring context and @ActiveProfiles sets the spring profiles we want to use while running these tests. The @TestExecutionListeners lets us register listeners that spring-test provides hooks for in our tests. DependencyInjectionTestExecutionListener allows us to inject beans directly into our test, FlywayDBUnitTestExecutionListener handles @Flyway and @DbUnitSupport annotations and TransactionalTestExecutionListener makes our tests transnational so hibernate has transactions to work within. Next for transactional support we have @TransactionConfiguration which allows us to configure our transactions and @Transactional(“transactionManager”) which actually wraps our test methods in transactions (you most likely have seen this annotation when writing transnational code).

Next we need to take a look at the SpringTestConfig class

@Configuration
@Import({ DatabaseTestConfig.class })
@ComponentScan(value = "com.company.artifact.app")
@PropertySource("classpath:test.properties")
public class SpringTestConfig {
	@Bean
	public static PropertySourcesPlaceholderConfigurer propertyPlaceholder() {
		final PropertySourcesPlaceholderConfigurer placeholder = new PropertySourcesPlaceholderConfigurer();
		placeholder.setIgnoreUnresolvablePlaceholders(true);
		return placeholder;
}

Again, this class isn’t doing too much. It tells spring to scan our base class for beans and imports another configuration class. It also imports some properties for our test.

@Configuration
@Profile("TEST_DB")
@PropertySource({ "classpath:flyway.properties" })
public class DatabaseTestConfig {

	@Value("${flyway.user}")
	private String user;
	@Value("${flyway.password}")
	private String password;
	@Value("${flyway.url}")
	private String url;
	@Value("${flyway.locations}")
	private String locations;
	@Value("${flyway.placeholdersPrefix}")
	private String prefix;
	@Value("${flyway.placeholderSuffix}")
	private String suffix;

	@Bean(destroyMethod = "close")
	public DataSource dataSource() {
		final BasicDataSource basicDataSource = new BasicDataSource();
		basicDataSource.setUsername(user);
		basicDataSource.setPassword(password);
		basicDataSource.setUrl(url);
		basicDataSource.setMaxActive(-1);
		return basicDataSource;
	}

	@Bean
	public FlywayHelperFactory flywayHelperFactory() {
		final FlywayHelperFactory factory = new FlywayHelperFactory();
		final Properties flywayProperites = new Properties();
		flywayProperites.setProperty("flyway.user", user);
		flywayProperites.setProperty("flyway.password", password);
		flywayProperites.setProperty("flyway.url", url);
		flywayProperites.setProperty("flyway.locations", locations);
		flywayProperites.setProperty("flyway.placeholderPrefix", prefix);
		flywayProperites.setProperty("flyway.placeholderSuffix", suffix);
		factory.setFlywayProperties(flywayProperites);
		return factory;
	}

	@Bean
	public Flyway flyway() {
		final Flyway flyway = flywayHelperFactory().createFlyway();
		flyway.setDataSource(dataSource());
		return flyway;
	}

	@Bean
	@Qualifier("userNumber")
	public String userNumber() {
		return "";
	}
}

We need to setup our own datasource for our tests since they won’t be run from within our container, and thus won’t have access to any jndi resources. Also here we configure flyway to use that datasource as well. flyway.properties is actually populated by defaults in our parent maven pom and can be overridden during a test run if needed. We’ll see later when we talk about the maven build how we use these properties to run against either oracle or h2 database.

Ignore the userNumber bean for now, we’ll get to that when we talk about pre and post deployment selenium tests.

Next lets look at how we extend these database test to support using spring-test for testing Spring MVC.

@WebAppConfiguration
public class AbstractSpringMvcTest extends AbstractSpringDatabaseTest {

  @Autowired
  protected WebApplicationContext webApplicationContext;

  @Autowired
  private FilterChainProxy springSecurityFilterChain;

  protected MockMvc mockMvc;

  @Before
  public void setup() {
    mockMvc =
        MockMvcBuilders.webAppContextSetup(webApplicationContext)
            .addFilter(springSecurityFilterChain).build();
  }

  protected UserDetailsRequestPostProcessor custregUser(final String username) {
    return SecurityRequestPostProcessors.userDeatilsService(username).userDetailsServiceBeanId(
        "custregUserDetailsService");
  }
}

Here we are extending our database test functionality and adding in some spring mvc test configuration. The code here is setting up springs mockMvc for us and setting a username and userDetailsService for our security so we don’t need to mock out our user. Also we annotate our configuration with @WebAppConfiguration so that we have a WebApplicationContext created for us.

Currently Spring MVC test does not support spring security, but there is an example at https://github.com/spring-projects/spring-test-mvc/blob/master/src/test/java/org/springframework/test/web/server/samples/context/SecurityRequestPostProcessors.java that we borrow from. This let’s use create some request post processors and add in our spring security information before tests run. This makes it so we don’t need to mock out our SecurityContextHolder wrapper bean since we’ll set an actual authentication object on it. This feature will most likely be added in a later version of spring test.

There’s not much configuration here outside what we’ve covered during our database tests so lets take a look at an example using Spring mvc test.

@FlywayTest
@DBUnitSupport(loadFilesForRun = { "CLEAN_INSERT", "/dbunit/dbunit.base.xml",
		"CLEAN_INSERT", "/dbunit/dbunit.dev.base.xml", "CLEAN_INSERT",
		"/dbunit/dbunit.mid.xml", "CLEAN_INSERT",
		"/dbunit/dbunit.mid.cert.complete.xml", "CLEAN_INSERT",
		"/dbunit/dbunit.manifest.xml", })
public class ManifestControllerSpringMvcITest extends AbstractSpringMvcTest {

	private User user;
	@Autowired
	private UserRepository userRepository;
	private MockMultipartFile file;

	@Before
	public void setupData() {
		user = userRepository.findOne(-1L);
		file = new MockMultipartFile("multipartFile", "orig", null,
				"bar".getBytes());
	}

	@Test
	public void index() throws Exception {
		mockMvc.perform(get("/manifests").with(custregUser(user.getUsername())))
				.andExpect(view().name(is("manifest/index")))
				.andExpect(
						model().attributeExists("manifestHistory",
								"manifestEnvironments"));
	}

	@Test
	public void uploadAjaxEnvironmentValidationErrors() throws Exception {

		mockMvc.perform(doFileUpload(file).accept(MediaType.APPLICATION_JSON))
				.andExpect(status().isBadRequest())
				.andExpect(
						jsonPath("$.fieldErrors[0].field", is("environment")))
				.andExpect(
						jsonPath("$.fieldErrors[0].error",
								is("This field cannot be null.")));
	}

	@Test
	public void uploadAjaxFileEmtpyValidationErrors() throws Exception {
		mockMvc.perform(
				doFileUpload(
						new MockMultipartFile("multipartFile", "orig", null,
								new byte[0]))
						.accept(MediaType.APPLICATION_JSON).param(
								"environment", "PROD"))
				.andExpect(content().contentType(MediaType.APPLICATION_JSON))
				.andExpect(status().isBadRequest())
				.andExpect(
						jsonPath("$.fieldErrors[0].field", is("multipartFile")))
				.andExpect(
						jsonPath("$.fieldErrors[0].error",
								is("Please select a valid file to upload.")));

	}

	@Test
	public void uploadAjaxSuccess() throws Exception {
		mockMvc.perform(
				doFileUpload(file).param("environment", "PROD").accept(
						MediaType.APPLICATION_JSON)).andExpect(status().isOk())
				.andExpect(content().contentType(MediaType.APPLICATION_JSON));

	}

	@Test
	public void uploadEnvironmentValidationErrors() throws Exception {

		mockMvc.perform(doFileUpload(file))
				.andExpect(status().isOk())
				.andExpect(model().hasErrors())
				.andExpect(
						model().attributeHasFieldErrors("manifest",
								"environment"));
	}

	@Test
	public void uploadEmptyFileValidationErrors() throws Exception {

		mockMvc.perform(
				doFileUpload(new MockMultipartFile("multipartFile", "orig",
						null, new byte[0])))
				.andExpect(status().isOk())
				.andExpect(model().hasErrors())
				.andExpect(
						model().attributeHasFieldErrors("manifest",
								"multipartFile"));
	}

	@Test
	public void uploadSuccess() throws Exception {
		mockMvc.perform(doFileUpload(file).param("environment", "PROD"))
				.andExpect(redirectedUrl("/manifests"))
				.andExpect(model().hasNoErrors());
	}

	private MockHttpServletRequestBuilder doFileUpload(
			final MockMultipartFile file) {
		return fileUpload("/manifests").file(file).with(
				custregUser(user.getUsername()));
	}
}

and the controller under test

@Controller
public class ManifestController {

	@Autowired
	private ManifestService manifestService;

	@Autowired
	private SecurityHolder securityHolder;

	@RequestMapping(value = "manifests", method = RequestMethod.GET)
	public String index(@ModelAttribute final Manifest manifest,
			final Model model) {
		setupManifestModel(model);
		return "manifest/index";
	}

	private void setupManifestModel(final Model model) {
		model.addAttribute("manifestHistory", manifestService
				.findUserManifestHistory(securityHolder.getName()));
		model.addAttribute("manifestEnvironments", manifestService
				.getAvailableManifestEnvs(securityHolder.getName()));
	}

	@RequestMapping(value = { "manifests", "api/manifests" }, method = RequestMethod.POST, produces = MediaType.APPLICATION_JSON_VALUE)
	public @ResponseBody
	Manifest uploadAjax(@Valid @ModelAttribute final Manifest manifest,
			final BindingResult bindingResult)
			throws MethodArgumentNotValidException {
		if (!manifestService.getAvailableManifestEnvs(securityHolder.getName())
				.contains(manifest.getEnvironment())) {
			bindingResult.rejectValue("environment", "invalid.manifest.env");
		}
		if (bindingResult.hasErrors()) {
			throw new MethodArgumentNotValidException(null, bindingResult);
		}
		return manifestService.create(manifest, securityHolder.getName());
	}

	@RequestMapping(value = "manifests", method = RequestMethod.POST)
	public String upload(@Valid @ModelAttribute final Manifest manifest,
			final BindingResult bindingResult, final Model model,
			final RedirectAttributes redirectAttributes) {
		if (!manifestService.getAvailableManifestEnvs(securityHolder.getName())
				.contains(manifest.getEnvironment())) {
			bindingResult.rejectValue("environment", "invalid.manifest.env");
		}
		if (bindingResult.hasErrors()) {
			setupManifestModel(model);
			return "manifest/index";
		}
		manifestService.create(manifest, securityHolder.getName());
		redirectAttributes.addFlashAttribute("flashMessage",
				"manifest.upload.success");
		return "redirect:/manifests";
	}

	@RequestMapping(value = { "manifests/{id}", "api/manifests/{id}" }, method = RequestMethod.GET, produces = MediaType.APPLICATION_OCTET_STREAM_VALUE)
	public @ResponseBody
	FileSystemResource download(@PathVariable final Long id,
			final HttpServletResponse httpServletResponse)
			throws FileNotFoundException, IOException {
		final Manifest manifest = manifestService.find(id);
		if (manifest == null)
			throw new NotFoundException();
		httpServletResponse.setHeader("Content-Disposition",
				"attachment; filename=" + manifest.getFilename());
		return manifest.getFileSystemResource();
	}

	@Autowired
	private MessageSource messageSource;

	@ExceptionHandler(MethodArgumentNotValidException.class)
	@ResponseStatus(value = HttpStatus.BAD_REQUEST)
	public @ResponseBody
	HttpValidationMessage validation(final MethodArgumentNotValidException e,
			final Locale locale) {
		return new HttpValidationMessage(e.getBindingResult(), messageSource,
				locale);
	}
}

So what are we and aren’t we testing here? First off we are not testing the UI and servlet container features. What we are testing is our HTTP API that we’ve created in spring, along with all the services and other objects involved. You’re spring code will be executed as if it had received a real requests from the servlet container.

Spring test provides us with a nice builder pattern api for creating mock http requests and having them run through spring mvc. We can easily include things like request params, content type, and more. Then spring give us access to the http response, such as content type and response codes as well as other headers, along with any spring mvc features like model and views.

Decoupling External Systems

Before we get into selenium testing, we need to talk about our application profiles. We rely on external systems for all our user/company data and security and sometimes we even create/modify data in other systems when necessary which would need to be reset. To allow for easily reproducible and fast selenium tests we need to decouple our system from these other systems. To achieve this we used spring profiles to provide database implementations of our api calls. So instead of a class using springs restOperations to make a http call, we instead just have the interface backed by a hibernate object. These database implementations are activated by our DEV_SERVICES profile which you have seen in our test configurations. We have to do something similar with our security. Instead of using the custom filter provided by the external system we use spring security’s jdbc implementation and tie that to the DEV_SECURITY profile. With this done we can control all the data from the external systems using flyway and dbunit. We can then cover the missing api calls in post deployment selenium tests or in spring tests.

Selenium Integration Tests

Now that we can start talking about our selenium tests. The idea here is to split our test into 2 categories, Pre and Post deployment.

The purpose of pre-deployment test are to test the UI and functionality of the system in a reproducible manner so that they can be run by developers before committing code and during continuous integration builds to catch any issues before we deploy our application. If your system has a regression, or a database script error, or many of the other things that could go wrong should be caught here. We are testing the 90% of application at this point, include browser, server, and database interactions. We are not testing our external services since they are backed by the database and we are not testing any production server settings/features/etc.

Now post-deployment tests are less about verifying the application features/validations work properly and more about testing that the application deployed correctly. These test will need to setup user/company data in the external systems before they run and use the custom authentication provided by them. They’ll test the happy paths of the functionality to verify that all the external apis, database connections, etc are working properly. Also you can test web server configuration here like making sure you redirect all http to https and any other type of url rewrites/proxying/whatever that would be configured in your production env but not your developer ones.

Let’s start with our ManifestController pre-deployment selenium test

/**
 * 
 * @author stephen.garlick
 * @author lindsei.berman
 * 
 */
@DBUnitSupport(loadFilesForRun = { "CLEAN_INSERT", "/dbunit/dbunit.base.xml",
		"CLEAN_INSERT", "/dbunit/dbunit.dev.base.xml", "CLEAN_INSERT",
		"/dbunit/dbunit.mid.xml", "CLEAN_INSERT",
		"/dbunit/dbunit.mid.cert.incomplete.xml" })
@FlywayTest
public class ManifestControllerSeleniumITest extends AbstractDevSeleniumTest {

	@Value("${selenium.base.url}")
	private String baseUrl;

	@Autowired
	private ManifestPage manifestPage;

	@Autowired
	private ResourceLoader resourceLoader;

	@Autowired
	private SeleniumElementVisibilityTester seleniumElementVisibilityTester;

	@Before
	@Override
	public void setup() throws Exception {
		super.setup();
		getWebDriver().get(baseUrl + "/manifests");
	}

	@Test
	@DBUnitSupport(loadFilesForRun = { "CLEAN_INSERT",
			"/dbunit/dbunit.mid.cert.incomplete.xml" })
	public void fileSizeCannotBeZero() throws IOException {
		manifestPage
				.selectFile(getAbsoluteFilePath("classpath:files/manifest-empty-test-data"));
		assertTrue(manifestPage.isFileErrorDisplayed());
	}

	@Test
	public void successfulUpload() throws IOException {

		manifestPage
				.selectFile(
						getAbsoluteFilePath("classpath:files/manifest-not-empty-test-data"))
				.submit();
		assertTrue(manifestPage.getManifestHistorySize() >= 1);
	}

	/**
	 * When Client's certification is incomplete he/she should be able to view
	 * only the Pre-Production option in the Environment selection drop down box
	 * 
	 * @throws IOException
	 */
	@Test
	@DBUnitSupport(loadFilesForRun = { "CLEAN_INSERT",
			"/dbunit/dbunit.mid.cert.incomplete.xml" })
	public void userIncompleteCertificationOnlyViewPreProduction()
			throws IOException {
		assertEquals("Pre-Production", manifestPage.getTEMEnvironmentText());

	}

	/**
	 * When Client is Certified to upload manifests he/she should be able to
	 * view both Pre-Production and Production options in the Environment
	 * selection drop down box
	 * 
	 * @throws IOException
	 */
	@Test
	@DBUnitSupport(loadFilesForRun = { "CLEAN_INSERT",
			"/dbunit/dbunit.mid.cert.complete.xml" })
	public void userCertifiedViewBothPreProductionAndProduction()
			throws IOException {
		assertEquals("user should see both prod and preprod options", 2,
				manifestPage.getNumberOfEnvironmentOptions());
	}

	/**
	 * 
	 * @throws IOException
	 * 
	 *             when the user picks a manifest using the manifest select
	 *             button. The manifest name should be displayed beside the
	 *             cancel and upload button. Then once the cancel button is
	 *             pressed the name should no longer be displayed and the
	 *             file-select should be displayed
	 **/

	@Test
	public void manifestCancelSuccessful() throws IOException {
		int before = manifestPage.getManifestHistorySize();
		manifestPage
				.selectFile(
						getAbsoluteFilePath("classpath:files/manifest-not-empty-test-data"))
				.cancel();
		assertTrue(manifestPage.isFileSelectDisplayed());
		int after = manifestPage.getManifestHistorySize();
		assertEquals(before, after);
	}

	/**
	 * After manifest select button is pressed and a file is chosen
	 * successfully(not too small) then the upload and cancel button should be
	 * visible
	 * 
	 * @throws IOException
	 */
	@Test
	public void manifestClickandFileChoosenUploadandCancelDisplayed()
			throws IOException {
		manifestPage
				.selectFile(getAbsoluteFilePath("classpath:files/manifest-not-empty-test-data"));
		List<String> buttons = Lists.newArrayList("upload-file-button",
				"cancel-file-button");
		seleniumElementVisibilityTester.testElementsDisplayedAndEnabled(
				getWebDriver(), buttons);
	}

	private String getAbsoluteFilePath(final String resource)
			throws IOException {
		return resourceLoader.getResource(resource).getFile().getAbsolutePath();
	}
}

Again you’ll see we are controlling the database with flyway and dbunit. One thing you might realize is that we require that we have a server started up to run this test. We solve this later with maven for our builds, but for development we need to have our server up when running our tests. This is solved by Arquillian which is quickly approaching production readiness. We won’t be going into that today, but look for a future post.

If you’ve done browser work you’ll notice a lot of familiar things in the above code like css selectors. Here we are able to test that specific elements on our page are visible, enabled, and anything else you could determine from within a browser. This is because of seleniums webdriver, which interacts with an actual api for each browser directly; turn on debugging and you can see http calls for each interaction you perform within the test.

Lets go in a little deeper and start looking at our base classes.

@Category(IntegrationTest.class)
@TestExecutionListeners({DependencyInjectionTestExecutionListener.class,
    FlywayDBUnitTestExecutionListener.class})
@ActiveProfiles({"TEST_DB", "TEST_DEV"})
@ContextConfiguration(classes = {DatabaseTestConfig.class, SeleniumTestConfig.class})
public class AbstractDevSeleniumTest extends AbstractSeleniumTest {

}

Most of this should look familiar. We have a new profile TEST_DEV which will discuss in a moment. Also we see a new configuration class

@Configuration
@ComponentScan("com.company.artifact.test.selenium")
@PropertySource({ "classpath:selenium/selenium.properties" })
public class SeleniumTestConfig {

	@Bean(destroyMethod = "stop", initMethod = "start")
	public ChromeDriverService chromeDriverService() throws IOException {
		final ChromeDriverService chromeDriverService = new ChromeDriverService.Builder()
				.usingDriverExecutable(
						new File(System.getProperty("user.home")
								+ "/chromedriver")).usingAnyFreePort().build();
		return chromeDriverService;
	}

	@Bean(destroyMethod = "quit")
	public ChromeDriver chromeDriver() throws IOException {
		final ChromeDriver chromeDriver = new ChromeDriver(
				chromeDriverService());
		return chromeDriver;
	}

	/**
	 * Configuration for integration tests the run during the build process.
	 * 
	 * @author stephen.garlick
	 * 
	 */
	@Configuration
	@Profile("TEST_DEV")
	@PropertySource("classpath:selenium/selenium-build.properties")
	static class BuildSeleniumConfig {

	}

	/**
	 * Configuration for integration tests that run post deployment.
	 * 
	 * @author stephen.garlick
	 * 
	 */
	@Configuration
	@Profile("TEST_SIT")
	@PropertySource("classpath:selenium/selenium-sit.properties")
	static class SitSeleniumConfig {

	}

	@Bean
	public static PropertySourcesPlaceholderConfigurer propertyPlaceholder() {
		final PropertySourcesPlaceholderConfigurer placeholder = new PropertySourcesPlaceholderConfigurer();
		placeholder.setIgnoreUnresolvablePlaceholders(true);
		return placeholder;
	}

}

Here we setup our chromeDriverService which expects the chromedriver executable and then the chromeDriver bean itself which we’ll be using to interact with the browser. Then we component scan for our reusable selenium beans and pulling in some properties.

Next let’s take a look at our base test class

@RunWith(SpringJUnit4ClassRunner.class)
public abstract class AbstractSeleniumTest {

	@Value("${bcg.user.name}")
	private String bcgUserName;
	private String userNumber;
	private String username;

	@Autowired
	@Qualifier("userNumber")
	private Provider<String> userNumberProvider;

	@Before
	public void setup() throws Exception {
		userNumber = userNumberProvider.get();
		username = bcgUserName + userNumber;
		createUserTester.createUser(webDriver, username);
		loginTester.loginIn(webDriver, username);
	}

	@After
	public void tearDown() throws Exception {
		webDriver.manage().deleteAllCookies();
	}

	@Autowired
	private WebDriver webDriver;

	public WebDriver getWebDriver() {
		return webDriver;
	}

	@Autowired
	private SeleniumLoginTester loginTester;

	@Autowired
	private SeleniumCreateUserTester createUserTester;

}

This is where a lot of the work is going on for our tests, so lets break it down.

First the setup method. This method will be run for our pre and post deployment tests but will do different things based on the TEST_DEV or TEST_SIT profiles. If you are in a TEST_DEV (pre-deployment) test then it takes the bcgUserName property adds empty string to it and then uses that as the username for our test. Next it does a createUser call, which in this case is an empty implementation since dbunit will take care of this during database setup. Next it’ll login using our dev login page we discussed earlier. Now for the TEST_SIT profile userNumber will actually pull a number from a database sequence, which we’ll see when we look at the post deployment configuration, and creatUser will actually create a user in the external system and login does nothing because we are already logged in after creating the user.

The only thing we do after each test is clear the cookies, and thus authentication info, from the webDriver. We do this instead of instantiating a new webDriver so that we can create its bean as a singleton and reduce the amount of time we spend creating/tearing down our browser.

Next lets look at our post-deployment configuration.

@Category(SitIntegrationTest.class)
@TestExecutionListeners({DependencyInjectionTestExecutionListener.class})
@ActiveProfiles({"TEST_SIT"})
@ContextConfiguration(classes = {SitIntegrationTestConfig.class, SeleniumTestConfig.class})
public class AbstractSitSeleniumTest extends AbstractSeleniumTest {

}

Again similar to our pre-deployment except for different spring profiles and configuration classes. And this time we aren’t dealing with setting up the database so all of that configuration is gone.

Lets take a look at the new configuration class

@Configuration
@Profile("TEST_SIT")
public class SitIntegrationTestConfig {

  @Bean(destroyMethod = "close")
  public DataSource dataSource() {
    final BasicDataSource basicDataSource = new BasicDataSource();
    basicDataSource.setDriverClassName("oracle.jdbc.OracleDriver");
    basicDataSource.setUsername("user");
    basicDataSource.setPassword("password");
    basicDataSource.setUrl("url");
    basicDataSource.setMaxActive(-1);
    return basicDataSource;
  }

  @Bean
  public JdbcOperations jdbcOperations() {
    return new JdbcTemplate(dataSource());
  }

  @Bean
  @Qualifier("userNumber")
  @Scope(ConfigurableBeanFactory.SCOPE_PROTOTYPE)
  public String userNumber() {
    return jdbcOperations().queryForObject("select sit_user_seq.nextval from dual", String.class);
  }
}

Here we setup a connection to a database where we have a sequence object created. Before each test our base class will pull a new userNumber bean, which will return the next number form the sequence, and add it to our username so that we can create new users in the live system for our tests without needing to update the username everytime the tests are run.

Finally remember, that the setup method on the base class can be overriden to change this default creating/loging in/etc behavior in our setup() method. This can be useful when creating helper scripts that create X amount of users in the external system for manual testing and other things.

Page Pattern Support

It’s a recommend practice to use the page pattern for selenium. I won’t be going into the pattern itself, but see Page Objects for an explanation from the selenium developers.

We are going to look at a small bit of code used to support this pattern within our tests. You may have seen this object in our selenium test earlier

/**
 * 
 * @author stephen.garlick
 * @author linsei.berman
 * 
 */
@Page
public class ManifestPage {
	@FindBy(id = "upload-file-button")
	private WebElement submitButton;
	@FindBy(id = "file-select")
	private WebElement fileSelect;
	@FindBy(id = "fileinput")
	private WebElement fileUpload;
	@FindBy(id = "cancel-file-button")
	private WebElement cancelButton;
	@FindBy(id = "file-error")
	private WebElement fileError;
	@FindBys(value = { @FindBy(id = "history-body"), @FindBy(tagName = "tr") })
	private List<WebElement> manifestHistory;
	@FindBy(xpath = "//*[@id='environment']/option[1]")
	private WebElement temEnvironmentOption;
	@FindBy(xpath = "//*[@id='environment']")
	private WebElement environmentOptions;

	public boolean isFileSelectDisplayed() {
		return fileSelect.isDisplayed();
	}

	public ManifestPage selectFile(final String filePath) {
		fileUpload.sendKeys(filePath);
		return this;
	}

	public ManifestPage submit() {
		submitButton.click();
		return this;
	}

	public int getNumberOfEnvironmentOptions() {
		return new Select(environmentOptions).getOptions().size();
	}

	public ManifestPage cancel() {
		cancelButton.click();
		return this;
	}

	public boolean isFileErrorDisplayed() {
		return fileError.isDisplayed();
	}

	public int getManifestHistorySize() {
		return manifestHistory.size();
	}

	public String getTEMEnvironmentText() {
		return temEnvironmentOption.getText();
	}
}

and the Page annotation which uses the spring @Component annotation so we can register them as beans by classpath scanning

@Target(ElementType.TYPE)
@Retention(RetentionPolicy.RUNTIME)
@Component
public @interface Page {

}

and the PageBeanPostProcessor where we check for the annotation on beans created and call PageFactory.initElements to configure the bean’s selenium annotated fields with our webDriver.

@Component
public class PageBeanPostProcessor implements BeanPostProcessor {

	@Autowired
	private WebDriver webDriver;

	@Override
	public Object postProcessBeforeInitialization(Object bean, String beanName)
			throws BeansException {
		if (bean.getClass().isAnnotationPresent(Page.class)) {
			PageFactory.initElements(webDriver, bean);
		}
		return bean;
	}

	@Override
	public Object postProcessAfterInitialization(Object bean, String beanName)
			throws BeansException {
		return bean;
	}

}

Now we don’t need to worry about initializing our page beans in each test they are used and can inject them with spring.

Decoupling the Database

By default we develop against an oracle database. But this means that our test will need an oracle instance setup in advance before our test runs. To remove this need we use an in memory database called h2 which allows for oracle syntax. While h2 is fine for projects using ORMs like hibernate, it might not be the best option if you are using a lot of vendor specific features that h2 does not have compatibility for. Just remember that when making the decision to use h2 or not.

We use maven to spawn our h2 database as a tcp server so that our websphere instance and tests can both connect to it while running in different jvms. Let’s take a look at our parent pom.

	<profile>
		<id>h2-database</id>
		<activation>
			<property>
				<name>db</name>
				<value>h2</value>
			</property>
		</activation>
		<properties>
			<flyway.url>jdbc:h2:tcp://localhost:8082/mem:test;MODE=Oracle;DB_CLOSE_DELAY=-1;DB_CLOSE_ON_EXIT=FALSE</flyway.url>
			<flyway.user>sa</flyway.user>
			<flyway.password>sa</flyway.password>
			<database.datasource.class>org.h2.jdbcx.JdbcDataSource</database.datasource.class>
			<database.driver.jar>h2-1.3.175.jar</database.driver.jar>
		</properties>
	</profile>

First we have a profile that changes all the flyway connection settings to that of our h2 database.

<plugin>
	<groupId>com.btmatthews.maven.plugins.inmemdb</groupId>
	<artifactId>inmemdb-maven-plugin</artifactId>
	<version>1.4.2</version>
	<configuration>
		<monitorPort>11527</monitorPort>
		<monitorKey>inmemdb</monitorKey>
		<daemon>true</daemon>
		<type>h2</type>
		<port>8082</port>
		<database>test</database>
		<username>${flyway.user}</username>
		<password>${flyway.password}</password>
	</configuration>
	<dependencies>
		<dependency>
			<groupId>com.h2database</groupId>
			<artifactId>h2</artifactId>
			<version>${h2.version}</version>
		</dependency>
	</dependencies>
	<executions>
		<execution>
			<id>start-db</id>
			<goals>
				<goal>run</goal>
			</goals>
			<phase>pre-integration-test</phase>
		</execution>
		<execution>
			<id>stop</id>
			<goals>
				<goal>stop</goal>
			</goals>
			<phase>post-integration-test</phase>
		</execution>
	</executions>
</plugin>

Here is our plugin configuration for spawning the h2 database on our pre-integration-test phase and stopping it in our post-ingration-test phase.

And finally with our h2 database spawned we can use the flyway plugin to do the initial migration

<plugin>
	<groupId>com.googlecode.flyway</groupId>
	<artifactId>flyway-maven-plugin</artifactId>
	<version>${plugin-version.flyway}</version>
	<executions>
		<execution>
			<phase>pre-integration-test</phase>
			<goals>
				<goal>clean</goal>
				<goal>init</goal>
				<goal>migrate</goal>
			</goals>
			<configuration>
			</configuration>
		</execution>
	</executions>
	<dependencies>
		<dependency>
			<groupId>com.oracle</groupId>
			<artifactId>ojdbc6</artifactId>
			<version>${ojdbc6.version}</version>
		</dependency>
		<dependency>
			<groupId>com.h2database</groupId>
			<artifactId>h2</artifactId>
			<version>${h2.version}</version>
		</dependency>
		<dependency>
			<groupId>${project.groupId}</groupId>
			<artifactId>db</artifactId>
			<version>${project.version}</version>
		</dependency>
	</dependencies>
	<configuration>
	</configuration>
</plugin>

Now our database is up and schema migrated and we are ready to deploy to our websphere server and start running our selenium integration tests via the maven failsafe plugin.

Bootstrapping the Websphere Liberty Profile Server

Now that we’ve gotten a database up and migrated we need a way to setup our test server, in this case websphere liberty profile, so that we can deploy the application and let our selenium tests run.

Again we are going to our pom.xml to

<plugin>
	<groupId>com.ibm.websphere.wlp.maven.plugins</groupId>
	<artifactId>liberty-maven-plugin</artifactId>
	<version>1.0</version>
	<executions>
		<execution>
			<id>pre-integration-setup</id>
			<phase>pre-integration-test</phase>
			<goals>
				<goal>start-server</goal>
				<goal>deploy</goal>
			</goals>
		</execution>
		<execution>
			<id>post-integration-setup</id>
			<phase>post-integration-test</phase>
			<goals>
				<goal>stop-server</goal>
			</goals>
		</execution>
	</executions>
	<configuration>
		<assemblyArtifact>
			<groupId>com.company</groupId>
			<artifactId>wlp-test-server</artifactId>
			<version>1.1</version>
			<type>zip</type>
		</assemblyArtifact>
		<configFile>${project.build.directory}/test-classes/server.xml</configFile>
		<appArchive>${project.build.directory}/webapp.war</appArchive>
		<timeout>60</timeout>
		<verifyTimeout>60</verifyTimeout>
	</configuration>
</plugin>

This plugin allows us to use start up websphere liberty profile server and deploy our war file automatically from maven. We’ve packaged up the server as a maven artifact and deploy it to a private repo; This server includes any necessary provided dependencies in its /lib folder before being zipped up.

Next the plugin allows us to use a server.xml file which is a configuration file for websphere. We have the following server.xml template that gets processed by maven during our build to set the correct database (h2 or oracle).

<server description="new server">
	<!-- Enable features -->

	<webContainer deferServletLoad="false" />

	<featureManager>
		<feature>jsp-2.2</feature>
		<feature>servlet-3.0</feature>
		<feature>localConnector-1.0</feature>
		<feature>jdbc-4.0</feature>
		<feature>jndi-1.0</feature>
		<feature>beanValidation-1.0</feature>
		<feature>jpa-2.0</feature>
	</featureManager>
	<httpEndpoint host="0.0.0.0" httpPort="9080" httpsPort="9443"
		id="defaultHttpEndpoint" />
	<applicationMonitor updateTrigger="mbean" />

	<dataSource id="db" jndiName="jdbc/datasource">
		<jdbcDriver libraryRef="driverLib" javax.sql.DataSource="${database.datasource.class}"/>
		<properties URL="${flyway.url}"
			password="${flyway.user}" user="${flyway.user}" />
	</dataSource>

	<library id="driverLib">
		<fileset dir="${wlp.install.dir}/lib" includes="${database.driver.jar}" />
	</library>
	<jndiEntry id="profile" jndiName="spring.profiles.active"
		value="DEV_SECURITY,DEV_SERVICES,CONTAINER_DB" />
</server> 

You’ll notice properites like database.datasource.class which were defined in our pom.xml.

Now all of our tests can be run during our build and we have no need to manually setup any databases or web servers to run our integration tests on.

Closing Comments

Now we have ways to easily developer each layer of our tests and have them run during our maven build with a single command. From here we could easily create CI jobs in jenkins to handle testing, reporting and deployments for us so that we can focus on developing our app and tests.

Brief Overview of Automation and Testing Tools

I’ve been asked recently to do a short overview of the automation and testing tools I’ve used on my current project. I’ve gotten a lot of time to play around with various test tools and configurations, as well as ways to automate the setup and configuration of a build system. We’ll be taking a quick look at what each tool provides us and list any pros/cons that I’ve run into.

Databases and tools

Flyway

What is it?

Flyway is a database versioning and migration tool. It gives us a standard way to build our database schema in a reproducible manner and thus helps us easily create reproducible integration tests. We supply it a list of folders with our scripts, and flyway executes the scripts for us and manages a “schema_version” table with the metadata about the scripts that have already run. It uses a specific file naming to achieve the script execution order, VXXXXX__script_name.sql where XXXXX is a number. Once new scripts are added, you run the migrate command and all scripts with a version number higher than the last executed script will run. It also has a maven plugin which allows us to easily manually trigger migrations and easy hook into a Jenkins build/deployment.

How we’ve used it

We’ve used Flyway to handle migrations on our DEV/SIT environments and use it to determine which scripts need to be packaged for our CAT/PROD environments since we don’t have permission to run database scripts in those. This multi env setup is made easy since flyway allows you to supply a list of folders and it’ll combine the scripts in them and run in order; we just supply different maven properties/profiles to change the list of folders (ie db-all,db-dev for dev, db-all,db-prod for prod) and database connection.

We’ve also used it to help maintain database state during integration tests. By combining, flyway, spring test, and Flyway Test Extensions we are able to clean and rebuild our database so that each test has a clean schema to run against. Dbunit also comes into play here for setting up our test data after flyway has cleaned and migrated the schema.

Pros

• Simple and easy to use
• Reproducible schemas
• Database scripts committed to version control and versioned in the database
• Scripts written in SQL
• Good support for spring, testing, and maven

Cons

• Can’t use in our upper envs (not a tool issue)
• Possible issues with multiple users creating/committing scripts due to versioning nature.
  • If someone uses the same version number as you did there will be conflicts (this is why we use timestamps to reduce this chance).
  • Also for example someone commits a script with a higher version than mine and the migration runs, and then I commit my script. My script version is now lower than the latest in schema_version table. Developers will just need to be aware of this, and update their script versions if necessary. Another option is that flyway does have an ability to run scripts out of order, so that it will go back and pickup scripts with lower version that were added after the higher version ran, but then you do run into the chance of non reproducible schemas. Whether you enable this option or not it is something that you should be aware of when using flyway.

Other options

dbmigrate
Liquibase

Oracle/Postgresql/Mysql/Etc

What is it?

In this case, everyone should be familiar with at least one of these databases as they tend to be the most used. Most applications will connect to a database to persist data collected. We’ll mostly be talking about how we’ve used this database for our integration tests.

How we’ve used it

Our project uses the Oracle database, and we originally setup our tests to run against an oracle xe install. Our tests use flyway to connect to and manage the lifecycle of the database for our integration tests.

Pros

• Same database your application will connect to in a live env
• Allows for database specific features to be used (plsql, triggers, etc)

Cons

• Additional setup required
• Need to host database externally from tests
• Slower than in memory database
• Potentially slower depending on network connection
• Database must be up and accessible to run tests

H2 Database

What is it?

H2 is a fast, lightweight sql database written in Java. It supports embedded and server modes and can be run completely in memory. It also supports syntax compatibility for a number of other databases.

How we’ve used it

Our integration test were originally setup to run against an oracle instance. This became an issue when automating our build due to poor network connection to our dev database test schema. To remedy this we updated our sql scripts to be compatible in both H2 and Oracle (this was minimal since we weren’t using any Oracle specific features and enabled H2s compatibility mode). We then added a maven profile that would use a maven plugin – inmemdb to start H2 in server mode and change the connection properties to this new H2 server instead of oracle. This way our test can be easily run against a full Oracle install or H2 with a single command line property.

We’ve also created a POC in a different branch of the code which uses Arquillian (more on that later) and H2 in embedded mode. I’ll be exploring setting up the H2 server as a spring bean instead of a maven plugin in the future.

Pros

• Quick to setup using spring
• No need to externally host
• Initialized from spring/test (embedded)
• Initialized during pre-integration maven phase or spring (server)
• In memory mode very fast for integration tests that need to reset database often

Cons

• Limited support for database compatibility (no plsql support and other database specific features)
• Additional memory required in your jvm
• Embedded mode only accessible from within same jvm its launched (remedied by running in server mode)

Testing

JUnit

What is it?

Again, this is another one most people are familiar with. JUnit is a framework for writing test code in Java. It provides the foundation of our tests, providing test lifecycle hooks, assertions, maven support, 3rd party extensions and more.

How we’ve used it

All our tests, unit and integration start off with a JUnit test and hook in the appropriate runner depending on the type of test we perform.

Our unit tests tend to use vanilla JUnit or with Mockito to allow for quick and easy dependency mocking.

We’ll go more into our integration tests setups shortly.

While JUnit has been a great tool, we’ll be looking at testNG in the future due to having better ways to parameterize tests.

Pros

• Heavy used in the Java community and simple to use
• Lots of 3rd party extensions
• Maven support

Cons

• Tends to behind most other frameworks in features

Other Options

TestNG
Spock

Spring Test

What is it?

Spring Test is a spring module meant to provide support and ease of testing spring enabled applications. It includes test runners for JUnit/TestNG, takes a spring java config/xml to build the application context, and supports spring profiles and more. It also provides extended support of testing Spring MVC by providing a way to test routing and assert various parts of the response (ie view return, status code returned, model attributes, etc)

How we’ve used it

For our database integration tests we’ve wired our test using the Spring test runner and loaded the entire core module of our project (services/repos/etc) with a spring defined datasource (normally the app uses a container supplied datasource through jndi). We then use spring profiles to ignore the jndi datasource and pickup our test datasource. We then use the flyway extension to execute flyway clean/migrate and dbunit to setup data to put our database in a known state and enable completely reproducible tests.

Pros

• Actively supported and developed by the Spring Source teams
• Test using spring, thus allowing the creation of reusable test beans
• Provides additional information to tests that is not available when unit testing (mvc test features)
• Allows for testing all spring components

Cons

• Still not inside our servlet container
• No access to container resources
• Cannot test UI
• Can’t test things controlled by the container (filters, web.xml mappings, etc)

Selenium Webdriver

What is it?

Selenium Webdriver is a tool which allows us to develop browser tests using code from just about any language. It uses the browsers internal API to find and interact with the browser; this is different from the old selenium where mouse and keyboard recording were used to interact with the browser. It also supports a wide variety of browsers, including mobile. See the selenium webdriver page for more.

How we’ve used it

We use selenium in a few different configurations so that we can tests as much as possible in our build process without deploying to a live environment to catch as many issues as possible before any deployment happens. We can also control our data using properties. We’ll discuss these configurations here.

First off, our configurations have very little duplication thanks to using Spring to wire our tests together. This allows us to create a common base class that setups spring/selenium and then creates and logs a user in for both environments. Let’s take a look at how each is setup.

Our dev environment is setup to mock out any external APIs, using database tables, to allow rapid prototyping and allow us to test the majority of the application without connecting to test environments of systems we don’t control. By doing this we can use DBUnit to populate the external data our system relies on and then run tests against. This environment configuration is controlled by Spring Profiles to enable swap out the real API calls with database writes and enable a web interface for creating mock data.

Now that you understand our dev environment we can talk about how selenium tests are run against it. As we talked about earlier you can back these tests with either an external database or H2 running in server mode since our application will be deployed into a container running in its own VM. For actually application deployment we have two options; First is that you have the application already deployed in a container running in eclipse, connected to the same database your tests will be connected to, this is general how you will develop new tests. Second is that we use maven to deploy our war during the pre-integration phase of our build. In this case we have a maven plugin that spawns a websphere liberty profile server and deploys the app before running our integration tests (our client uses websphere, but you can easily do this for other containers as well). Our spring profile then uses a NOOP implementation of the create user call in our setup (dbunit takes care of this data) and logs in using our dev panel (spring security database auth).

Now you understand how the tests are setup and executed, lets take a look at what we can and can’t test in this configuration. First we can test all UI features now as well as the majority of our functional tests (data creation, validations, etc). What we can’t test in this case are environment specific things, like connectivity between systems (firewalls/ports), webserver request modifications.

Next is our SIT configuration. In this configuration our app is deployed to our test servers (after dev integration test run/pass) and is run in its production configuration using http calls to external APIs. By the time we reach this point, the majority of our actual application testing is already covered. Since most of our integration test have already run during the build we’ll want to mainly test the happy paths to make sure all our api calls properly go through and we know the application is working properly. Again, for these tests we change our Spring profile to pickup different properties/test beans. In this profile our test is NOT connected to any database since all the data will be created through the application/external systems. So this time our create user instead of being NOOP we have it create user in the external system and then our login code is NOOP since the user is automatically logged in after creation.

We’ll discuss the test flow more when we talk about our Jenkins configuration.

Pros

• Most of the test executed during build, errors caught fast
• Using browser APIs allows for test closer to a real user experience
• Tests are fully reproducible in the dev configuration due to database lifecycle control

Cons

• Test are not run in container, and don’t have access to the container datasource and thus need to define their own that connects to the same as the container
• Server must be started up outside of the test (IDE/Maven spawned/etc)

Arquillian

What is it?

Arquillian is a new tool in development by JBoss to allow for in-container tests. Arquillian allows for you to define deployments from within our JUnit/TestNG test and have it deployed to a servlet container. It has a lot of extension support for spring, persistence, various servlet containers and extensions that add abstraction on top of selenium. Arquillian itself is still in early development stages with extensions slightly behind the core module.

How we’ve used it

We’ve used Arquillian to develop a proof of concept to allow all our integration tests to be run full self contained. We use the Tomcat container, spring extension, and persistence extension. We added a @Deployment to our selenium test base class, which packages up our war, and changed the spring runner to the arquillian runner. Combined with a embedded h2 database we are then able to run test fully self contained (in our development profile) without having to have any external resources already started.

The flow of a test in this configuration is as so.

1. Arquillian/Spring extenion packages up our app with all maven dependencies (including test/provided but this is configurable)
2. Arquillian extracts an embedded Tomcat server and starts it up
3. Arquillian deploys the war to the embedded tomcat server
4. Spring starts up and connects to the database
5. Test starts, uses the same Spring context that is running in your tomcat deployment, has access to everything running on tomcat including jndi resources
6. Test runs, passes and everything shutsdown
7. Repeat

As you can see the main benefit of this setup is that you just hit run and arquillian takes care of the rest. This reduces a lot of extra configuration and container lifecycle management on our part and lets test run as a 100% real test inside our container.

The only downside at this point is that each test class is going to redeploy the tomcat/application which greatly increases our test times (and the main reason we haven’t merged this POC into trunk). Luckily the Arquillian developers are already aware of this issue and are planning to allow for test suite support to reduce these runtimes ( see https://issues.jboss.org/browse/ARQ-567 ).

Pros

• Tests completely control their own container/test lifecycles
• No need for external resources to be in place before tests run
• Full access to spring context running on serve and inside tests
• Test have access to jndi and other container resources
• Supported by JBoss
• Lots of additional support through extensions

Cons

• Slow test times due to container redeployment per test class
• Early stages of development, thus API changes common
• Increased memory on the JVM running tests with embedded containers

Build Environment

Vagrant

What is it?

Vagrant is a tool, written in Ruby, that provides a useful abstraction for multiple VM providers (VirtualBox, VMWare, EC2, etc) and allows for easy VM configuration and startup from the commandline. It also provides hooks for provisioners (such as Chef) and other extensions in the form of plugins. It also allows us to define VMs in code allowing us to check them into version control.

How we’ve used it

We’ve used Vagrant with to setup an Ubuntu 13.04 server, mount some shared folders, and configure networking. We used the vagrant-omnibus plugin to manage our Chef install and the vagrant-berkshelf plugin to manage copying our chef dependencies to the VM before provisioning. This is all configured in a Vagrantfile and started up by running the “vagrant up” command.

After this initial setup I developed the chef cookbook which we’ll talk about in a few. This allowed me to easily develop chef scripts iteratively by adding new functionality and testing it with “vagrant provision” to rerun the cookbook. We’ll discuss this development flow more when we talk about Chef.

Pros

• Simplifies VM configuration and provisioning
• VMs defined as code
• Easy development and testing of Chef cookbooks
• Good community support

Cons

• Evolvoing API and tools

Chef

What is it?

Chef is a tool with a focus on IT infrastructure automation and management. Chef comes in two flavors, first a full server/client setup with the intent of fully managing your IT and a standalone version called chef-solo that focuses on executing chef cookbooks with out server management side. Chef provides abstractions for installing and executing scripts and other common OS operations. Its written in Ruby, and can execute Ruby code which give’s it a large additional library of useful scripting tools.

You develop Chef Cookbooks which define a idempotent procedural way of installing a piece of software. This basically means you develop cookbooks that can be rerun over and over and your system will always end up in the same state without error. This development style, combined with Vagrant, allows for us to quickly and easily develop cookbooks in small focused chunks that we can rerun over and over as we add features. This also allows you to easily deploy an update to a large number of servers at once with minimal chance of error.

Chef cookbooks can be linted, unit tested, and integration tested to verify working code. Combined with Vagrant and Jenkins you can setup a continuous integration server for chef cookbooks.

Chef also comes with the benefit that it is being used as the base of amazon web services opsworks platform so that you can execute custom chef scripts easily.

How we’ve used it

We used Chef, along with Vagrant and berkshelf (chef cookbook dependency manager), to develop a Jenkins build box for our app. The build box installs the following software to support the build/test configurations we’ve been talking about.

1. install/update apt
2. install java 7
3. setup dns-search domains and restart networking
4. install jenkins server
5. install git and svn
6. setup svn proxy
7. download and store svn certs
8. install maven
9. install chrome
10. install xvfb (headless selenium tests)
11. install chromedriver for selenium test
12. configure jenkins server

Now most of the software above has a Chef cookbook already developed that can run against multiple OSes (for example java cookbook supports debian/rhel/window with multiple versions 6/7/ibm/oracle/etc). And all cookbooks can be parameterized, allowing for high reusability.

And since everything is defined in code, we can of course check it into version control.

Pros

• Infrastructure as Code/Version Control
• Automate installation and configuration of machines
• Reduce error due to repetition
• Continuous integration/Testable
• Quickly get up identical environments test/prod mirror/etc
• Much smaller than VMs
• Large base cookbook library
• AWS Opworks support
• Strong community support
• Ruby Libraries

Cons

• Provisioning time when starting from clean VM (more of issue for things like aws autoscaling)

Jenkins

What is it?

Jenkins is a tool written in Java that provides a continuous integration environment for projects. It has a very large plugin library and provides scripting support via the Groovy language. Due to this Jenkins is one of the most popular CI software. As with most things, there is a lot more the Jenkins than what we’ll be discussing here.

The goal of Jenkins is to provide an interface for developing build/deployment pipelines, task automation, reporting and more. It has support for most major build tools including maven which we’ll discuss. Jobs can be trigger manually, scheduled, off a version control hooks, polling a version control for updates, off of over builds completing, and more.

How we’ve used it

As we talked about before we used Chef to develop our Jenkins server. This required that we figure out how Jenkins was managing its configuration. Jenkins has a number of xml/json/text files that it and its plugins use to persist configuration changes. First I inited a git repo in my Jenkins home folder and then proceeded to commit changes as I updated the Jenkins configuration. This allowed me to track and add all the configuration updates to my cookbook. This itself seems like a large hassle due to just the sheer amount of configuration files and eventually will most likely need to be changed over to using the scripting console to configure and maintain the server.

By default Jenkins comes with a plugin to perform maven builds. We took advantage of this since the project is a maven project. We have two jobs configured in jenkins.

The first job performs CI for our trunk branch. This job polls for changes in svn. Once a change is picked up, Jenkins checks out the source code and performs out maven build command to run our full (dev) integration test suite. We either end up with a successful build (all tests passed), unstable build (all unit tests passed but integration test failures), or a failed build (compilation errors or unit test failures).

The second job, uses the buildresult-trigger plugin. The goal of this job is to tag and deploy the app to our test servers. This job triggers twice a day if it sees the first job with a new build in a successful state. If so this job will handle database migration using maven/flyway, handle websphere deployment using the websphere admin clinet, and then execute our happy path integration tests to make sure everything deployed correct. If the first job is not in success status, then this job will not run and thus prevent a build with integration test errors being deployed.

Pros

• Mature CI platform
• Tons of Community support
• Support for just about anything through plugins
• Scriptable

Cons

• Text file configuration hell (most likely solved by using scripting console)
• Doesn’t seem to be a ton of info on scripting outside of the javadocs

A look at Bean Validation API scenarios.

Recently I had to start up a new Java project and opted to use the new Java EE Bean Validation, specifically the Hibernate Validation implementation. We’ll also be using it in the context of a Spring 3 application which will give us some extra support for triggering and handling validations.

First add your maven dependency, this is the lasted version implementing the 1.0 spec.

	<dependency>
		<groupId>org.hibernate</groupId>
		<artifactId>hibernate-validator</artifactId>
		<version>3.4.1.Final</version> 
	</dependency>

Next in our Spring Configuration class

@Configuration
@EnableWebMvc
public class MvcConfig extends WebMvcConfigurerAdapter {
	/**
	 * Registering our validator with spring mvc for our i18n support
	 */
	@Override
	public Validator getValidator() {
		try {
			return validator();
		} catch (final Exception e) {
			throw new BeanInitializationException(
					"exception when registering validator in "
							+ this.getClass().getName(), e);
		}
	}

	/**
	 * We use springs reloadable message resource and set it to refresh every
	 * hour.
	 * 
	 * @return
	 */
	@Bean
	public MessageSource messageSource() {
		final ReloadableResourceBundleMessageSource messageSource = new ReloadableResourceBundleMessageSource();
		messageSource.setBasenames("/WEB-INF/messages/validation",
				"/WEB-INF/messages/text", "/WEB-INF/messages/label",
				"/WEB-INF/messages/popover");
		messageSource.setCacheSeconds(3600);
		return messageSource;
	}

	/**
	 * This is the validator we will provide to spring mvc to handle message
	 * translation for the bean validation api (hibernate-validation)
	 * 
	 * @return
	 * @throws Exception
	 */
	@Bean
	public LocalValidatorFactoryBean validator() throws Exception {
		final LocalValidatorFactoryBean bean = new LocalValidatorFactoryBean();
		bean.setValidationMessageSource(messageSource());
		bean.setProviderClass(HibernateValidator.class);
		return bean;
	}
}

Here we are creating a factory bean with HibernateValidator as the provider class, and we’ve passed in our Spring messageSource bean to handle error message resolving. Finally we register the validator with spring through its WebMvcConfigurerAdapter interface and overriding its getValidator method. Now we are ready to start validating our data.

First lets take a look at our input objects and then we’ll break down whats going on.

@GroupSequenceProvider(CapsAccountFormGroupSequenceProvider.class)
public class CapsAccountForm {
	@NotNull(message = "{required}")
	private PaymentType paymentType;

	@Valid
	private BankAccount bankAccount;

	@Valid
	private AlternateContact alternateContact;

	@AssertTrue(message = "{please.agree}")
	@NotNull(message = "{required}")
	private Boolean agreement;
	//other getters and setters

public final class BankAccount {
	@BankABANumber
	private String bankAccountNumber;

	@NotBlank(message = "{required}")
	@Size(max = 17, message = "{max.seventeen.digits}")
	@Pattern(regexp = "[0-9]*", message = "{numeric.only}")
	private String checkingAccountNumber;

	@NotBlank(message = "{required}")
	@Length(max = 40, message = "{length.exceeded}")
	private String bankName;

	@Length(max = 28, message = "{length.exceeded}")
	private String city;

	private State state;

	@Pattern(regexp = "[0-9]{5}|[0-9]{9}", message = "{zip.length.exceeded}")
	private String zipCode;

	@Pattern(regexp = "[0-9]*", message = "{numeric.only}")
	@Length(max = 10, message = "{length.exceeded}")
	private String phoneNumber;

	@Pattern(regexp = "[0-9]*", message = "{numeric.only}")
	@Length(max = 10, message = "{length.exceeded}")
	private String faxNumber;

	@Length(max = 40, message = "{length.exceeded}")
	private String bankAddress;

	//getters and setters
}

public final class AlternateContact {
	@Length(max = 100, message = "{length.exceeded}")
	public String name;

	@Length(max = 20, message = "{length.exceeded}")
	@Pattern(regexp = "[0-9]*", message = "{numeric.only}")
	public String phone;
	//getters and setters.

Now the above code is used for the following business scenario. First off we are writing to a legacy table to support automation of a current multi step process in a legacy system, so most of our constraints must follow what their tables have currently defined (that should explain some of the random field length checks). Next, the original form has the ability to have two different paymentType options which then affect if you need to populate the bankAccount form or not. If the paymentType is D (Debit) then we need the bankAccount form and process its validations, and if the paymentType is C (Trust, legacy code) then we don’t need to include the bankAccount form and will not process its validations. Finally there is the alternateContact form which is optional and the agreement field which is a required to be checked/True.

First we have the basic validation annotations provided to us by bean validation and hibernate validator. Ones provided by the spec are common to all implementations and hibernate provides a few extra implementation specific ones. I’d say these are mostly self explanatory but we can take a quick look at a few.

All the annotations take a message parameter which if wrapped in curly braces will be evaluated through the spring messageSource otherwise the text given will be the message returned.

The most commonly used annotations will be the @NotNull, @NotBlank (for Strings), @NotEmpty (for Collections) @Pattern, @Length, and @Size. Each of these annotations take various parameters to be used during evaluation, for example the regex field on @Pattern is the regex pattern that the string must match to be considered valid.

Next lets take a look at how our validations are triggered and how the current Spring validation integrates.

The @Valid annotation is used to trigger the bean validation. This will generally be applied to a Spring Controller method parameters. It can also be used to trigger nested evaluations objects like we have in the above code to validate the bankAccount and alternateContact fields. Let’s take a look at the create controller method for our form.

	@RequestMapping(value = "/crids/{crid}/caps", method = RequestMethod.POST)
	public String createCaps(@PathVariable final String crid,
			@Valid @ModelAttribute final CapsAccountForm capsAccountForm,
			final BindingResult errors, final Model model,
			final RedirectAttributes redirectAttributes)

As we said above, user information is pulled from the current user and company info is pulled from the url (crid PathVariable above). Here you can see our CapsAccountForm as a parameter and it annotated with @Valid. When used in a spring controller like this any validations that are triggered will automatically be populated in the BindingResult object for us. If you’ve used spring before you know that in the past you generally implemented the spring interface and populated the BindingResults object on your own.

What if we want to use this same form, but for a restful service instead of a web form? Let’s take a look at our API’s creation endpoint.

	@RequestMapping(value = "/api/crids/{crid}/caps/", consumes = {
			"application/json", "application/xml" }, produces = {
			"application/json", "application/xml" }, method = RequestMethod.POST)
	public @ResponseBody
	CreateCapsAccountOutput createCaps(@PathVariable final String crid,
			@Valid @RequestBody final CapsAccountForm capsAccountForm ) {

As you can see this method is very similar to the previous one, with the exceptions of we’ll be accepting Json/Xml as the input and producing the same. Again the @Valid annotation will trigger and validations on our objects, but this time we have no BindingResult object to be populated, so what will happen this time? Instead it will throw a MethodArgumentNotValidException which we can then handle in a spring ExceptionHandler method.

	@ExceptionHandler(MethodArgumentNotValidException.class)
	@ResponseStatus(value = HttpStatus.BAD_REQUEST)
	public @ResponseBody
	HttpValidationMessage validation(final MethodArgumentNotValidException e,
			final Locale locale) {
		return new HttpValidationMessage(e.getBindingResult(), messageSource,
				locale);
	}

Here we catch any MethodArgumentNotValidException thrown by the given controller (or any controller if this is a global handler in a @ControllerAdvice class) and we pull out hte bindingResults object that is attached to it to populate a message to be returned with a 400 error code.

Now that we’ve seen the basics, let’s look at creating a new custom constraint. If you noticed on the above bankAccount class we are using the annotation BankABANumber which is a customer constraint a I created. Lets take a look at some of the code it is triggering.

@Target({ METHOD, FIELD, ANNOTATION_TYPE })
@Retention(RUNTIME)
@ReportAsSingleViolation
@Constraint(validatedBy = BankABANumberValidator.class)
@Documented
@NotBlank
@Size(min = 9, max = 9)
@Pattern(regexp = "[0-9]*")
public @interface BankABANumber {

	String message() default "{caps.account.aba.number.invalid}";

	Class<?>[] groups() default {};

	Class<? extends Payload>[] payload() default {};
}

public class BankABANumberValidator implements
		ConstraintValidator<BankABANumber, String> {

	private ABANumberCheckDigit abaNumberCheckDigit;

	@Override
	public void initialize(final BankABANumber arg0) {
		abaNumberCheckDigit = new ABANumberCheckDigit();
	}

	@Override
	public boolean isValid(final String arg0,
			final ConstraintValidatorContext arg1) {
		return abaNumberCheckDigit.isValid(arg0);
	}

}

First lets look at the annotation. Here you can see we’ve actually combined a few existing validations and applied the @ReportAsSingleViolation annotation. What this will do is instead of showing a different message for each validation, instead it will only report a single message, which in this case is the default we’ve supplied of {caps.account.aba.number.invalid}. So if the field is blank, incorrect size, not a digit, or not a valid routing # then we’ll see a single error message.

Next since we aren’t only wrapping existing annotations, lets look at how we actually determine the routing number is valid. As you can see on our annotation we are supplying the @Constraint with the BankABANumberValidator class as its validation implementation. This class is an implementation of the provided ConstraintValidator interface which takes generics of the annotation and field type we are validating. From here you can access any fields set on the annotation and the value of the field we are validating. We are using Apache Validation that provides us with the algorithm to validate a routing #. I won’t go into the specifics of this validation, but you can easily look it up. So on top of our standard validations, this code as gets executed during validation and simply returns a boolean.

Note: You are able to use springs aspectj @Configurable injection support to inject spring beans into this class. This would allow for dynamic checks against a database or webservice.

Finally lets take a look at one last scenario. If you remember earlier I said we only evaluate the bankAccount object if the paymentType is set to Debit. Now this scenario is slightly more complicated but definitely useful. Let’s look at the CapsAccountForm class to see what exactly we are doing to achieve this validation setup.

On the CapsAccountForm class we have the following annotation @GroupSequenceProvider(CapsAccountFormGroupSequenceProvider.class). Now lets take a look at the code for this before we get into explaining what is happening.

	public List<Class<?>> getValidationGroups(final CapsAccountForm object) {
		final List<Class<?>> groups = Lists.newArrayList();
		groups.add(CapsAccountForm.class);
		if (object.getPaymentType() != null
				&& object.getPaymentType().equals(TrustPaymentMethod.D)) {
			groups.add(BankAccount.class);
		} else {
			object.setBankAccount(null);
		}
		return groups;
	}

First let’s explain what group sequences are. Each validation annotation can take a group parameter which can be used to evaluate only specific groups, or groups in a specific order so that if an earlier group fails a validation check, then the rest are left unevaluated. If you know beforehand which set of groups you want you can use the @Validated annotation on your spring method and supply it a list of classes/interfaces which correspond to the group values.

Now, since our groups are based off the paymentType field we don’t know our groups up front, so we can use the hibernate specific @GroupSequenceProvider to determine our groups before validation the class. Let’s take a look at our sequence provider CapsAccountFormGroupSequenceProvider.

First you notice we add the CapsAccountForm to our groups, since it expects the class its evaluating to be in its group list. Next we add the BankAccount class to our groups IF our paymentType is D, otherwise we ignore it and throw away anything by setting the field to null.

Now we either have a single group, the CapsAccountForm or two groups the CapsAccountForm and BankAccount. Lets look at the path where we only have the CapsAccountForm group. Since the bankAccount field has @NotNull with the group BankAccount it will only be evaluated if the paymentType is D. We don’t have the BankAccount class in our groups this check is ignored, and the @Valid annotation is no processed since our bankAccount field should be null if the paymentType is C. Now if the paymentType is D, then we add the BankAccount group and make sure the bankAccount field is not null, and then proceed to process the remaining fields in the BankAccount object.

BVal is in its earlier stages and will continue to add more support but hopefully seeing these situations and how we adapt bval to them helps you in the future when validation more complex scenarios.

From the Browser to the Database – Using multiple frameworks to get the job done

We are going to take a look at the technologies and code used to go from a form in a web browser to saving that data in our database.

The technology stack we are using for this overview includes

• HTML5
• Bootstrap 3
• Sitemesh 2.4
• Servlet 3
• Spring 3.2
• Spring MVC 3.2
• Spring Data JPA 1
• Bean Validation 1
• JPA2/Hibernate4
• Relational Database/Oracle 11g

We are going to review some code that is used to mock out the Customer Registration API for external postal applications.

First lets design our data model and create a script to create some database objects.

DECLARE
	v_sql LONG;
BEGIN

	begin
		v_sql := 'create table dev_users (
		id number primary key,
		username varchar2(255) not null,
		password varchar2(255) not null,
		description varchar2(255) not null,
		enabled integer not null,
		first_name varchar2(255) not null,
		last_name varchar2(255) not null,
		email varchar2(255) not null,
		phone_number varchar2(10) not null,
		phone_number_ext varchar2(4),
		address_line1 varchar2(255) not null,
		address_line2 varchar2(255),
		address_line3 varchar2(255),
		city varchar2(50) not null,
		state varchar2(2) not null,
		postal_code varchar2(5) not null,
		created_time timestamp not null
	)';
	execute immediate v_sql;
	exception when others  then
		IF SQLCODE = -955 THEN
	        NULL; -- suppresses ORA-00955 exception
	      ELSE
	         RAISE;
	      END IF;
	 end;
	 begin
		v_sql := 'create unique index dev_users_username_idx on dev_users(username)';
		execute immediate v_sql;
	exception when others  then
		IF SQLCODE = -955 THEN
	        NULL; -- suppresses ORA-00955 exception
	      ELSE
	         RAISE;
	      END IF;
	 end;
	 begin
		v_sql := 'create sequence dev_users_seq CACHE 50';
		execute immediate v_sql;
	exception when others  then
		IF SQLCODE = -955 THEN
	        NULL; -- suppresses ORA-00955 exception
	      ELSE
	         RAISE;
	      END IF;
	 end;
END;
/

You should notice right away that we are wrapping our create statements in an oracle procedure. This allows us to catch object already exists exceptions and allow our script to be idempotent, which just means it can be rerun over and over without any errors or side affects.

So we create a table that holds some basic user information, it has a single integer id based off a sequence which is a best practice to use a non business related key on our data and reduces how many columns we need to join across. We then create a unique index on our natural key the username which we will generally use to query specific user’s data. Also we remember to add our not null and type size constraints in the table definition.

After you’ve migrated the database (I’ve been using flyway for database management) with the new table then we can move onto creating our JPA object to load the data. I’ve added some additional comments explaining what some of the annotations do for us. Here we’ll also see our first look at the bean validation constraints.

package com.usps.ach.dev.user;
//imports

/**
 * This is a database representation of the custreg account api.
 *
 * @author stephen.garlick
 * @deprecated for use only in the dev profile
 */
@Entity //marks this class a jpa entity
@Immutable //marks it so an entity cannot be updated
@Cacheable //enables entity caching against the @Id field
@Cache(usage = CacheConcurrencyStrategy.READ_ONLY, region = "devUser") //tells jpa to use a readonly cache and to use the devUser  cache defined in our ehcache.xml file
@NaturalIdCache //setups a cache using the fields marked with @NaturalId as the key
@Table(name = "dev_users") //override the base table name, the configured hibernate will use dev_user as default, but we like to pluralize our table names
@SequenceGenerator(sequenceName = "dev_users_seq", name = "dev_users_seq", allocationSize = 1) //sets up id generation based on the sequence we created. make sure allocationSize = 1 unless you plan on making your sequences increment more than 1 at a time.  this also makes sure hibernate doesnt use its HiLo strategy
@Deprecated //we just mark thsi deprecated so that we get a strong warning if someone tries to use this code in a non dev env.
public final class DevUser {
	@Id
	@GeneratedValue(strategy = GenerationType.SEQUENCE, generator = "dev_users_seq") //let hibernate generate our ids
	private Long id;

	@NaturalId
	@NotBlank(message = "{please.enter}") //bean validation constraint that checks if the field is null or empty string.  the message is a reference to a messagesource key used for i18n
	@Length(max = 255, message = "{length.exceeded}") //constraint that checks the fields length
	@Pattern(regexp = "[a-zA-Z0-9]*", message = "{alphanum.only}") //pattern to make sure the field only has alphanumeric characters
	@Column(unique = true)
	private String username;

	@NotBlank(message = "{please.enter}")
	@Length(max = 255, message = "{length.exceeded}")
	private String description;

	@NotBlank(message = "{please.enter}")
	@Length(max = 255, message = "{length.exceeded}")
	private String firstName;

	@NotBlank(message = "{please.enter}")
	@Length(max = 255, message = "{length.exceeded}")
	private String lastName;

	@NotBlank(message = "{please.enter}")
	@Length(max = 255, message = "{length.exceeded}")
	@Email(message = "{email.invalid}")
	private String email;

	@NotNull(message = "{please.enter}")
	@Length(min = 10, max = 10, message = "{please.enter.digit.length}")
	@Pattern(regexp = "[0-9]*", message = "{numeric.only}")
	private String phoneNumber;

	@Length(max = 4, message = "{length.exceeded}")
	@Pattern(regexp = "[0-9]*", message = "{numeric.only}")
	private String phoneNumberExt;

	@NotBlank(message = "{please.enter}")
	@Length(max = 255, message = "{length.exceeded}")
	private String addressLine1;

	@Length(max = 255, message = "{length.exceeded}")
	private String addressLine2;

	@Length(max = 255, message = "{length.exceeded}")
	private String addressLine3;

	@NotBlank(message = "{please.enter}")
	@Length(max = 50, message = "{length.exceeded}")
	@Pattern(regexp = "[a-zA-Z]*", message = "{alpha.only}")
	private String city;

	@NotNull(message = "{please.enter}")
	@Length(min = 2, max = 2, message = "{please.enter.alpha.length}")
	@Pattern(regexp = "[a-zA-Z]*", message = "{alpha.only}")
	private String state; //TODO: pull this out into an ENUM instead of a 2 char string

	@NotNull(message = "{please.enter}")
	@Length(min = 5, max = 5, message = "{please.enter.digit.length}")
	@Pattern(regexp = "[0-9]*", message = "{numeric.only}")
	private String postalCode;

	private Date createdTime;
	private String password;
	private Integer enabled;

	@OneToOne(fetch = FetchType.EAGER, cascade = CascadeType.ALL) //we create a mapping to our real user table so we can create a user object when a dev user is created
	@PrimaryKeyJoinColumn(referencedColumnName = "id") //one to one mapping using both tables primary key column (they will have same primary key in both tables)
	private User user;

	@PrePersist //jpa callback to add the time as our createdTime field
	public void createTime() {
		this.createdTime = new Date();
	}

	//make sure to override hash and equals to be against our natural key instead of using instance based.  also don't use the id field since it may not be set
	@Override
	public int hashCode() {
		final int prime = 31;
		int result = 1;
		result = prime * result
				+ ((username == null) ? 0 : username.hashCode());
		return result;
	}

	@Override
	public boolean equals(Object obj) {
		if (this == obj)
			return true;
		if (obj == null)
			return false;
		if (getClass() != obj.getClass())
			return false;
		DevUser other = (DevUser) obj;
		if (username == null) {
			if (other.username != null)
				return false;
		} else if (!username.equals(other.username))
			return false;
		return true;
	}
//getters and setters
}

Hibernate’s Improved Naming Strategy allows us to convert fields that are camel cased to database tables by converting everything to lowercase and inserting underscores before uppercase letters. Example firstName -> first_name, lastName->last_name, addressLine1 -> address_line1.

Next we use spring data to create a dynamic DAO for us, effectively eliminating the need for us to create our own dao layer

package com.usps.ach.dev.user;

import org.springframework.data.jpa.repository.JpaRepository;
import org.springframework.stereotype.Repository;

/**
 * Spring data for manipulating our {@link DevUser}
 *
 * @author stephen.garlick
 * @deprecated only for use in the dev profile
 */
@Deprecated
@Repository
public interface DevUserRepository extends JpaRepository<DevUser, String> {
	public DevUser findByUsername(String username);

}

If you look at the JpaRepository you can see that it has a standard set of methods like findAll() find(ID) save(Entity) and other CRUD methods. We can create our own set of queries by simply adding a new method to our interface and spring data will generate the necessary JPA code for us. Here since we generally will be looking up users based on their username instead of their id we add a findByUsername method. You can do this with anyfield, ex. findByField1AndField2OrField3. If you have the spring data plugin for eclipse it will let you know if you’re methods are valid. There are a lot more advance techniques you can use here such as named queries and overriding the base implementation, but for basic crud operations this is the simplest form. See the spring-data-jpa documentation if you’d like to know more.

Next we’ll take a look at our service layer. The service layer’s job is to handle our transactions, possibly caching and to handle any extra logic necessary. I’ve excluded the interface and a few methods for brevity.

package com.usps.ach.dev.user;

//imports

/**
 * See {@link DevUserService}
 *
 * @author stephen.garlick
 * @deprecated this class is only for use in the dev profile
 */
@Profile(Profiles.DEV_SERVICES) //make sure this bean is only loaded when spring.profiles.active=DEV_SERVICES
@Service //marks this class for spring to create and inject
@Transactional //tells spring all methods in this class are to be run in transactions
@Deprecated //again deprecate so we can give a strong warning if anyone uses this outside the dev profiles
public final class DevUserServiceImpl implements DevUserService {

	@Autowired //wire in our repo by type
	private DevUserRepository devUserRepository;

	@Override
	public DevUser create(final DevUser devUser) {
		devUser.setPassword(devUser.getUsername());
		devUser.setEnabled(1);
		User user = new User();
		user.setUsername(devUser.getUsername());

		devUser.setUser(user); //since we have cascading set to ALL on our DevUser object the User object will automatically be persisted on save.  if you dont have cascade of ALL/Persist //then we'll get an exception here saying User is a transient object.
		return save(devUser);
	}

	private DevUser save(final DevUser devUser) {
		return devUserRepository.save(devUser);
	}

	@Override
	public boolean exists(final DevUser devUser) {
		return null != find(devUser.getUsername()); //return a boolean if a given entity is already created
	}

	@Override
	public DevUser find(final String username) {
		return devUserRepository.findByUsername(username); //user our defined method to query for our DevUser
	}

	@Override
	public Collection<DevUser> findAll() {
		return devUserRepository.findAll(); //delegate straight to our repo
	}
}

Finally we can look at our Spring MVC controller which will handle the incoming requests from the servlet container. We’ll want to follow the basic principals of REST in that we properly use the verbs GET to retrieve, POST to create, PUT to update, and DELETE to destroy. We’ll also use the proper HTTPStatus codes where applicable (ie user not found returns 404 page).

URLs will be designed as access to our entity resources. Since this a dev form we’ll be under the /dev/ namespace. As you’ll see below we use GET /dev/users/ to show all users, GET /dev/users/{username} to access a specific user and GET /dev/users/new to a form where new users are created and a POST /dev/users with the form data will create a new user. We won’t allow editing or deleting but if we did we would map /dev/users/{username}/edit which would have a form to do a PUT on /dev/users/{username} and a DELETE on /dev/users/{username} to destroy a resource.

Also we’ll be using our DevUser entity for form binding and validation in this tier.

package com.usps.ach.dev.user;
//imports

/**
 * This class handles creation of new {@link DevUser} to be used for logging in
 * under our dev profile. it is also intended to mock out the custreg account
 * api in our dev profile
 *
 * @author stephen.garlick
 * @deprecated only for use in the dev profile
 */
@Controller //marks this to be registered by spring
@Profile(Profiles.DEV_SERVICES) //makes it only picked up in the dev profile
@Deprecated //marked deprecated to hide deprecated warnings from our dependencies
public final class DevUserController {

	@Autowired
	private DevUserService devUserService;

	@Autowired
	private DevCridService devCridService;

	@RequestMapping(value = "dev/users/new", method = RequestMethod.GET)
	public String getNew(@ModelAttribute final DevUser devUser,
			final Model model) {
		return "dev/user/new";  //return the view for this new form
	}

	@RequestMapping(value = "dev/users", method = RequestMethod.GET)
	public String getIndex(final Model model) {
		setup(model);
		return "dev/user/index"; //return the index page for our users
	}

	@RequestMapping(value = "dev/users/{username}", method = RequestMethod.GET)
	public String viewUser(@PathVariable final String username,
			@ModelAttribute DevCrid devCrid, final Model model) {
		final DevUser devUser = devUserService.find(username);
		if (null == devUser)
			throw new NotFoundException(); //if null is returned for the user then it doesnt exists and we shoudl return 404
		final Collection devCrids = devCridService.findAll();
		model.addAttribute("devUser", devUser);
		model.addAttribute("devCrids", devCrids);
		return "dev/user/view";
	}

	private void setup(Model model) {
		model.addAttribute("users", devUserService.findAll());
	}

	/**
		Here we are creating a new user.  The user data will be passed in this paramter @Valid @ModelAttribute final DevUser devUser and the @Valid annotation will trigger all the bean valdiaton on our entity object.  If errors are found then spring will automatically populate them in the final BindingResult errors for us.
	**/
	@RequestMapping(value = "dev/users", method = RequestMethod.POST)
	public String create(@Valid @ModelAttribute final DevUser devUser,
			final BindingResult errors, final Model model,
			final RedirectAttributes redirectAttributes) {
		final boolean userExists = devUserService.exists(devUser);//do a manual valdation to make sure the user does not already exists.  this is not part of bean validation spec yet
		if (userExists)
			errors.rejectValue("username", "",
					"A user already exists with this username.");
		if (errors.hasErrors()) {
			return "dev/user/new"; //return the new form and display any errors
		}
		//otherwise create the user and redirect to its newly created view page.  Generally you want to redirect after a non GET operation.
		DevUser newUser = devUserService.create(devUser);
		redirectAttributes.addAttribute("username", newUser.getUsername());
		return "redirect:/dev/users/{username}";
	}
}

You may have noticed that all the classes we’ve covered are in the same package. This benefits us by allowing all related classes with a piece of functionality to be logically organized and makes it easier to find classes that are related to each other. This package naming scheme is preferable over the style of using packages to group components together (ie xx.xx.xx.controller xx.xx.xx.model etc).

Finally we can take a look at the views that we are returning from our spring controller which will allow the user to actually enter the information in the browser and submit it to our server.

users index /dev/users

<%@include file="/WEB-INF/jsp/shared/common-taglibs.jsp"%>
<!DOCTYPE html>
<html>
<head>
<title>Development Users</title>
</head>
<body id="users">
<div>
		<a href="<c:url value="/dev/users/new"/>"
			class="btn btn-primary btn-sm">Create New User</a></div>
<div>
<table class="table">
<thead>
<tr>
<td>Username</td>
<td>Description</td>
<td>Login</td>
</tr>
</thead>
<tbody>
				<c:forEach items="${users}" var="devUser">
<tr>
<td><a class="btn btn-primary btn-sm"
							href="<c:url value="/dev/users/${devUser.username}"/>">${devUser.username}</a></td>
<td>${devUser.description}</td>
<td><form action="login/process" method="POST"><input name="j_username" value="${devUser.username}" type="hidden"/><input name="j_password" value="${devUser.username}" type="hidden"/><button class="btn btn-primary btn-sm">Login</button></form></td>
</tr>
</c:forEach></tbody>
</table>
</div>
</body>
</html>

users new /dev/users/new

<%@include file="/WEB-INF/jsp/shared/common-taglibs.jsp"%>

<!DOCTYPE html>
<html>
<head>
<title>Development Create New User</title>
</head>
<body id="users">
<h3>Create a New User</h3>
<h5>The password will be the same as the username</h5>
<form:form commandName="devUser" method="POST" servletRelativeAction="/dev/users/"
			cssClass="form-horizontal">
			<t:bootstrap-text-input path="username" maxLength="255" label="Username" required="true"/>
			<t:bootstrap-text-input path="description" maxLength="255" label="Description" required="true"/>
			<t:bootstrap-text-input path="firstName" maxLength="255" label="First Name" required="true"/>
			<t:bootstrap-text-input path="lastName" maxLength="255" label="Last Name" required="true"/>
			<t:bootstrap-text-input path="email" maxLength="255" label="Email Address" required="true"/>
			<t:bootstrap-text-input path="phoneNumber" maxLength="10" label="Phone Number" required="true"/>
			<t:bootstrap-text-input path="phoneNumberExt" maxLength="4" label="Phone Number Extension"/>
			<t:bootstrap-text-input path="addressLine1" maxLength="255" label="Address Line 1" required="true"/>
			<t:bootstrap-text-input path="addressLine2" maxLength="255" label="Address Line 2"/>
			<t:bootstrap-text-input path="addressLine3" maxLength="255" label="Address Line 3"/>
			<t:bootstrap-text-input path="city" maxLength="50" label="City" required="true"/>
			<t:bootstrap-text-input path="state" maxLength="2" label="State" required="true"/>
			<t:bootstrap-text-input path="postalCode" maxLength="5" label="Postal Code" required="true"/>
<div class="col-lg-8">
				<form:button class="btn btn-primary btn-sm pull-right">Create</form:button></div>
</form:form>
</body>
</html>

user view /dev/users/{username}

<%@include file="/WEB-INF/jsp/shared/common-taglibs.jsp"%>
<!DOCTYPE html>
<html>
<head>
<title>Development User View</title>
</head>
<body id="users">
<div class="form-horizontal">
<h3>Development User</h3>
<t:bootstrap-text-input disabled="true" path="devUser.username"
			maxLength="255" label="Username" required="true" />
		<t:bootstrap-text-input disabled="true" path="devUser.description"
			maxLength="255" label="Description" required="true" />
		<t:bootstrap-text-input disabled="true" path="devUser.firstName"
			maxLength="255" label="First Name" required="true" />
		<t:bootstrap-text-input disabled="true" path="devUser.lastName"
			maxLength="255" label="Last Name" required="true" />
		<t:bootstrap-text-input disabled="true" path="devUser.email"
			maxLength="255" label="Email Address" required="true" />
		<t:bootstrap-text-input disabled="true" path="devUser.phoneNumber"
			maxLength="10" label="Phone Number" required="true" />
		<t:bootstrap-text-input disabled="true" path="devUser.phoneNumberExt"
			maxLength="4" label="Phone Number Extension" /></div>
<div>
<table class="table" id="user-crids">
<thead>
<tr>
<td>CRID</td>
<td>Company Name</td>
<td>Link Toggle</td>
</tr>
</thead>
<tbody>
				<c:forEach items="${devCrids}" var="crid">
<tr>
<td><a class="btn btn-primary btn-sm"
							href="<c:url value="/dev/crids/${crid.crid}"/>">${crid.crid}</a></td>
<td>${crid.companyName}</td>
<td><c:choose>
								<c:when test="${crid.achCrid.users.contains(devUser.user)}">
									<form:form servletRelativeAction="/dev/users/${devUser.username}/crids/${crid.crid}" method="DELETE"
										commandName="devCrid">
										<form:button class="btn btn-primary btn-sm">Unlink</form:button>
									</form:form>
								</c:when>
								<c:otherwise>
									<form:form servletRelativeAction="/dev/users/${devUser.username}/crids" method="POST"
										commandName="devCrid">
										<form:button class="btn btn-primary btn-sm">Link</form:button>
										<form:hidden path="crid" value="${crid.crid}" />
									</form:form>
								</c:otherwise>
							</c:choose></td>
</tr>
</c:forEach></tbody>
</table>
</div>
</body>
</html>

and all these views are wrapped in a sitemesh template

<%@include file="/WEB-INF/jsp/shared/common-taglibs.jsp"%>

<!DOCTYPE html>
<html>
<head>
<meta name="viewport" content="width=device-width, initial-scale=1.0"
	http-equiv="Content-Type" content="text/html; charset=UTF-8">
<!-- Bootstrap -->
	<link href="<c:url value="/resources/css/bootstrap.min.css"/>"
	rel="stylesheet" media="screen">

<!-- HTML5 shim and Respond.js IE8 support of HTML5 elements and media queries -->
<!--[if lt IE 9]>
      <script src="../../assets/js/html5shiv.js"></script>
      <script src="../../assets/js/respond.min.js"></script>
    <![endif]-->

<decorator:head />
<title><decorator:title /></title>
</head>
<body>
<div class="container">
<div class="container">
<div id="header" class="col-lg-12">
				<page:applyDecorator name="dev-header">
					<page:param name="section">
						<decorator:getProperty property="body.id" />
					</page:param>
				</page:applyDecorator></div>
<div class="clearfix"></div>
<div class="col-lg-12">
				<decorator:body /></div>
</div>
</div>
<!-- jQuery (necessary for Bootstrap's JavaScript plugins) -->
	<script src="<c:url value="/resources/js/jquery-1.10.2.min.js"/>"></script>
	<!-- Include all compiled plugins (below), or include individual files as needed -->
	<script src="<c:url value="/resources/js/bootstrap.min.js"/>"></script>
	<script src="<c:url value="/resources/js/bootstrap-triggers.js"/>"></script>
</body>
</html>

and our sitemesh header for dev panel

<%@include file="/WEB-INF/jsp/shared/common-taglibs.jsp"%>
<c:set var="activeTab"><decorator:getProperty property="section"/></c:set>
<nav class="navbar navbar-default">
<div class="navbar-header">
		<button type="button" class="navbar-toggle" data-toggle="collapse"
			data-target=".navbar-ex1-collapse">
			<span class="sr-only">Toggle navigation</span> <span class="icon-bar"></span>
			<span class="icon-bar"></span> <span class="icon-bar"></span>
		</button>
		<a href="<c:url value="/dev/login"/>" class="navbar-brand">Developer Panel</a></div>
<!-- Collect the nav links, forms, and other content for toggling -->
<div class="collapse navbar-collapse navbar-ex1-collapse">
<ul class="nav navbar-nav">
	<li ${activeTab eq 'login' ? 'class="active"' : '' }><a href="<c:url value="/dev/login"/>">Login</a></li>
	<li ${activeTab eq 'users' ? 'class="active"' : '' }><a href="<c:url value="/dev/users"/>">Users</a></li>
	<li ${activeTab eq 'crids' ? 'class="active"' : '' }><a href="<c:url value="/dev/crids"/>">CRIDs</a></li>
	<li ${activeTab eq 'permits' ? 'class="active"' : '' }><a href="<c:url value="/dev/permits"/>">Permits</a></li>
</ul>
</div>
<!-- /.navbar-collapse -->
</nav>

our common-taglibs file

<%-- common taglibs to include for jsps --%>
<%@ page language="java" contentType="text/html; charset=UTF-8" 	pageEncoding="UTF-8" isErrorPage="false"%>
<%@ taglib uri="http://www.springframework.org/security/tags" 	prefix="sec"%>
<%@ taglib uri="http://www.springframework.org/tags" prefix="spring"%>
<%@ taglib prefix="form" uri="http://www.springframework.org/tags/form"%>
<%@ taglib uri="http://java.sun.com/jsp/jstl/core" prefix="c"%>
<%@ taglib uri="http://java.sun.com/jsp/jstl/fmt" prefix="fmt"%>
<%@ taglib uri="http://www.opensymphony.com/sitemesh/decorator" 	prefix="decorator"%>
<%@ taglib uri="http://www.opensymphony.com/sitemesh/page" prefix="page"%>
<%@ taglib prefix="fn" uri="http://java.sun.com/jsp/jstl/functions"%>
<%@taglib prefix="t" tagdir="/WEB-INF/tags" %>

and our custom bootstrap tags for text input

<%@tag
	description="Extended input tag to allow for sophisticated errors"
	pageEncoding="UTF-8"%>
<%@taglib prefix="spring" uri="http://www.springframework.org/tags"%>
<%@taglib prefix="c" uri="http://java.sun.com/jsp/jstl/core"%>
<%@taglib prefix="form" uri="http://www.springframework.org/tags/form"%>
<%@taglib prefix="fn" uri="http://java.sun.com/jsp/jstl/functions"%>
<%@attribute name="path" required="true" type="java.lang.String"%>
<%@attribute name="labelCssClass" required="false"
	type="java.lang.String"%>
<%@attribute name="formCssClass" required="false"
	type="java.lang.String"%>
<%@attribute name="errorCssClass" required="false"
	type="java.lang.String"%>
<%@attribute name="label" required="true" type="java.lang.String"%>
<%@attribute name="maxLength" required="true" type="java.lang.Long"%>
<%@attribute name="placeholder" required="false" type="java.lang.String"%>
<%@attribute name="required" required="false" type="java.lang.Boolean"%>
<%@attribute name="disabled" required="false" type="java.lang.Boolean"%>
<%@attribute name="readonly" required="false" type="java.lang.Boolean"%>
<c:set var="errors"><form:errors path="${path}"/></c:set>
<div class="form-group ${not empty errors ? 'has-error' : '' }">
	<label
		class="control-label ${empty labelCssClass ? 'col-lg-2' : labelCssClass}"
		for="${path}">${label}<c:if test="${required}">
			<span class="text-danger">*</span>
		</c:if></label>
<div class="${empty formCssClass ? 'col-lg-6' : formCssClass }">
		<form:input path="${path}" disabled="${disabled}" cssClass="form-control" maxlength="${maxLength}" readonly="${readonly}" placeholder="${empty placeholder ? label : placeholder}" /></div>
<c:if test="${not empty errors}">
		<span class="text-danger ${empty errorCssClass ? 'col-lg-4' : errorCssClass }">${errors}</span>
	</c:if></div>

Bootstrap is twitters browser framework that give us a grid for easily making layouts, lots of base css classes for styling, and some dynamic javascript with plugin support as well. I’ve created these custom tags to allow for simple reusable and consistent form on our page. I won’t go too much into boostrap here but all classes you see in the given views and tags are provided by boostrap. I’ve written 0 custom css and javascript so far.

Of course there are some obvious things missing that we will cover in the future such as testing each layer with unit and integration tests and all the necessary configuration to actually get the frameworks in place but the main goal is getting used to developing end to end functionality using mutliple frameworks to handle all the non business stuff for us.

Spring Java Configuration in Servlet 3.x containers

As stated on the SpringSource site

Spring is the most popular application development framework for enterprise Java™. Millions of developers use Spring to create high performing, easily testable, reusable code without any lock-in.

We are going to take a look at getting Spring+Spring MVC setup in a servlet 3.x container such as Tomcat, WebSphere, Jboss or any of the many other JavaEE/Servlet servers.

Maven Dependencies

First thing we need to do is add Maven dependencies to our pom files.  We’ll be using a multi-module project as described in an earlier post.  We’ll have three pom files, one parent, one core, and one webapp.  In our Parent POM add the following artifacts to the dependencyManagement section

<dependency>
 <groupId>javax.servlet</groupId>
 <artifactId>javax.servlet-api</artifactId>
 <version>${servlet.version}</version>
 <scope>provided</scope>
 </dependency>
 <dependency>
 <groupId>javax.servlet</groupId>
 <artifactId>jstl</artifactId>
 <version>${jstl.version}</version>
 <scope>provided</scope>
 </dependency>
 <dependency>
 <groupId>javax.servlet.jsp</groupId>
 <artifactId>javax.servlet.jsp-api</artifactId>
 <version>${jsp.version}</version>
 <scope>provided</scope>
 </dependency>
 <dependency>
 <groupId>javax.el</groupId>
 <artifactId>javax.el-api</artifactId>
 <version>${el.version}</version>
 <scope>provided</scope>
 </dependency></pre>
<!-- Web application development utilities applicable to both Servlet
 and Portlet Environments (depends on spring-core, spring-beans, spring-context)
 Define this if you use Spring MVC, or wish to use Struts, JSF, or another
 web framework with Spring (org.springframework.web.*) -->
 <dependency>
 <groupId>org.springframework</groupId>
 <artifactId>spring-web</artifactId>
 <version>${org.springframework.version}</version>
 </dependency>

<!-- Spring MVC for Servlet Environments (depends on spring-core, spring-beans,
 spring-context, spring-web) Define this if you use Spring MVC with a Servlet
 Container such as Apache Tomcat (org.springframework.web.servlet.*) -->
 <dependency>
 <groupId>org.springframework</groupId>
 <artifactId>spring-webmvc</artifactId>
 <version>${org.springframework.version}</version>
 </dependency>

<!-- Application Context (depends on spring-core, spring-expression, spring-aop,
 spring-beans) This is the central artifact for Spring's Dependency Injection
 Container and is generally always defined -->
 <dependency>
 <groupId>org.springframework</groupId>
 <artifactId>spring-context</artifactId>
 <version>${org.springframework.version}</version>
 <exclusions>
 <exclusion>
 <groupId>commons-logging</groupId>
 <artifactId>commons-logging</artifactId>
 </exclusion>
 </exclusions>
 </dependency>

<dependency>
 <groupId>org.springframework</groupId>
 <artifactId>spring-core</artifactId>
 <version>${org.springframework.version}</version>
 <exclusions>
 <exclusion>
 <groupId>commons-logging</groupId>
 <artifactId>commons-logging</artifactId>
 </exclusion>
 </exclusions>
 </dependency>

<!-- Expression Language (depends on spring-core) Define this if you use
 Spring Expression APIs (org.springframework.expression.*) -->
 <dependency>
 <groupId>org.springframework</groupId>
 <artifactId>spring-expression</artifactId>
 <version>${org.springframework.version}</version>
 </dependency>

<!-- Bean Factory and JavaBeans utilities (depends on spring-core) Define
 this if you use Spring Bean APIs (org.springframework.beans.*) -->
 <dependency>
 <groupId>org.springframework</groupId>
 <artifactId>spring-beans</artifactId>
 <version>${org.springframework.version}</version>
 </dependency>

Note the provided scope on the servlet dependencies.  This means we are expecting our servlet container to already have these dependencies available to our application so we don’t need to package them up, but still need them for compiling.  The default scope for our other artifacts is compile which means they’ll be used to compiled and packaged up with our project

Also we are excluding commons-logging from spring core in favor of a slf4j implementation.

Next in the Parent POM add our versions under the properties section.  The servlet version should correspond to what your container provides and the spring dependencies should be the newest stable version that works with your servlet version.  These should work for 3.0.x containers.

<el.version>2.2.4</el.version>
 <jsp.version>2.2.1</jsp.version>
 <jstl.version>1.2</jstl.version>
 <org.springframework.version>3.2.4.RELEASE</org.springframework.version>
 <servlet.version>3.0.1</servlet.version>

Now we need to add the actual depenencies to our project modules for use.

<dependencies>
<dependency>
 <groupId>javax.servlet</groupId>
 <artifactId>javax.servlet-api</artifactId>
 </dependency>
 <dependency>
 <groupId>javax.servlet</groupId>
 <artifactId>jstl</artifactId>
 </dependency>
 <dependency>
 <groupId>javax.servlet.jsp</groupId>
 <artifactId>javax.servlet.jsp-api</artifactId>
 </dependency>
 <dependency>
 <groupId>javax.el</groupId>
 <artifactId>javax.el-api</artifactId>
 </dependency>
 <!-- Web application development utilities applicable to both Servlet and
 Portlet Environments (depends on spring-core, spring-beans, spring-context)
 Define this if you use Spring MVC, or wish to use Struts, JSF, or another
 web framework with Spring (org.springframework.web.*) -->
 <dependency>
 <groupId>org.springframework</groupId>
 <artifactId>spring-web</artifactId>
 </dependency>

<!-- Spring MVC for Servlet Environments (depends on spring-core, spring-beans,
 spring-context, spring-web) Define this if you use Spring MVC with a Servlet
 Container such as Apache Tomcat (org.springframework.web.servlet.*) -->
 <dependency>
 <groupId>org.springframework</groupId>
 <artifactId>spring-webmvc</artifactId>
 </dependency>
 <dependency>
 <groupId>${project.groupId}</groupId>
 <artifactId>core</artifactId>
 </dependency>
 </dependencies>

Adding this to the webapp pom give’s us our servlet and spring web/mvc dependencies as well as our core project module if you didn’t already add it.

 <dependency>
 <groupId>org.springframework</groupId>
 <artifactId>spring-context</artifactId>
 </dependency>

<dependency>
 <groupId>org.springframework</groupId>
 <artifactId>spring-core</artifactId>
 </dependency>

<dependency>
 <groupId>org.springframework</groupId>
 <artifactId>spring-expression</artifactId>
 </dependency>

<dependency>
 <groupId>org.springframework</groupId>
 <artifactId>spring-beans</artifactId>
 </dependency>

These pull the rest of the Spring dependencies into our core project which then get pull transtively to our webapp project.

Configuration

We are going to save why we use Spring, and other IoC containers for Java applications, for a later post and jump right into some configuration code.  Containers that implement the Java EE Servlet Spec 3.x enable us to skip the standard /WEB-INF/web.xml file in favor of simpler java class where we can pragmatically configure servlets.  So lets delete the web.xml file generated by our maven webapp archetype (keep the /WEB-INF/ folder) and look at how we use Java to register our servlets.

First just to stay, all the Spring documentation is very well written and complete. Looking at http://static.springsource.org/spring/docs/3.2.x/spring-framework-reference/html/new-in-3.1.html#new-in-3.1-servlet-3-code-config we see a reference to WebApplicationInitializer where I’ve slightly modified the code under the A 100% code-based approach to configuration

public class ServletConfig implements WebApplicationInitializer {

@Override
 public void onStartup(ServletContext servletContext)
 throws ServletException {

// Create the 'root' Spring application context and register our
 // MvcConfig spring class
 AnnotationConfigWebApplicationContext rootContext = new AnnotationConfigWebApplicationContext();
 rootContext.register(MvcConfig.class);

// Manage the lifecycle of the root application context
 servletContext.addListener(new ContextLoaderListener(rootContext));

// Create the dispatcher servlet's Spring application context
 AnnotationConfigWebApplicationContext dispatcherContext = new AnnotationConfigWebApplicationContext();

// Register and map the dispatcher servlet
 ServletRegistration.Dynamic dispatcher = servletContext.addServlet(
 "dispatcher", new DispatcherServlet(dispatcherContext));
 dispatcher.setLoadOnStartup(1);
 dispatcher.addMapping("/");
 }

}

You can create this class anywhere in your project and your servlet 3 container will pick it up and execute the code when you start up your application.  I’ve gone with root.package.path.config for all my servlet and spring configuration classes that aren’t specific to application business functionality.

How is Spring doing this with Servlet 3?

So this new configuration method is built on top of Servlet 3 dependency, so let’s take a quick look on how spring is achieving this auto detection and configuration.  First looks take a look at the SpringServletContainerInitializer class.  Here we see the class definition as

@HandlesTypes(WebApplicationInitializer.class)
public class SpringServletContainerInitializer implements ServletContainerInitializer

If you look at the Javadoc for ServletContainerInitializer you can see it says that there must be a file under META-INF/services which declares all implementations of the interface.  If we look at spring-web/META-INF/service/javax.servlet.ServletContainerInitializer file we see one entry org.springframework.web.SpringServletContainerInitializer

Now we know how spring registers it’s SpringServletContainerInitializer with the container let’s take a look at how it picks up our WebApplicationInitializer implementations.  You’ll notice @HandlesTypes(WebApplicationInitializer.class) annotation on SpringServletContainerInitializer.  This annotation tells the container to scan for all classes implementing that interface and pass them in as the first parameter in the method

public interface ServletContainerInitializer {

 public void onStartup(Set<Class<?>> c, ServletContext ctx)
 throws ServletException;
}

Which will then pass in all your WebApplicationInitializer implementations for Spring to invoke and handle the onStartup methods.

Finally Our Servlet Configuration

Now that we know how our container is loading our servlet information lets go back to our original configuration and take a look at the Spring classes we are registering.

We need to configure two things, Springs implementation of ServletContextListener which allows Spring to startup and shutdown with our application.  The second thing we need to register is Spring’s DispatcherServlet which will let us use Spring MVC which we’ll discuss more later.

Listener

// Create the 'root' Spring application context and register our
 // MvcConfig spring class
 AnnotationConfigWebApplicationContext rootContext = new AnnotationConfigWebApplicationContext();
 rootContext.register(MvcConfig.class);

// Manage the lifecycle of the root application context
 servletContext.addListener(new ContextLoaderListener(rootContext));

Serlvet

// Create the dispatcher servlet's Spring application context
 AnnotationConfigWebApplicationContext dispatcherContext = new AnnotationConfigWebApplicationContext();

// Register and map the dispatcher servlet
 ServletRegistration.Dynamic dispatcher = servletContext.addServlet(
 "dispatcher", new DispatcherServlet(dispatcherContext));
 dispatcher.setLoadOnStartup(1);
 dispatcher.addMapping("/");

As you may have notice we configure multiple AnnotationConfigWebApplicationContext, one for the listener and another for the servlet.  Springs AnnotationConfigWebApplicationContext is hierarchical so the root context belongs to the listener, and then the context for the servlet can override beans from the root context for that servlet. Since we are keeping our application very simple and will have no need for this functionality we only register our Spring @Configuration class MvcConfig.class in the root context.

In our DispatcherServlet we overrider the default servlet ‘/’ mapping so that Spring MVC can handle all our incoming request, note that this is different than mapping ‘/*’ which will cause problems later when using Spring MVC.

Let’s take a quick peak at our MvcConfig.class just to see how Spring is taking over at this point.

@Configuration
@EnableWebMvc
@ComponentScan(basePackages = "com.company.artifact")
// do a full scan of the root package to pickup annotationed classes
public class MvcConfig extends WebMvcConfigurerAdapter

The @Configuration annotation tells Spring that this is a configuration class (like the old .xml application context files).  For the  @ComponentScan annotation we pass in our root package name so that Spring will automatically scan and pickup any other annotated classes, including other @Configuration classes and beans.

That’s it

Now with very little code we have Spring hooked into our web application.