Fluent Builder method ordering

Classic simple fluent builder usually suffers from some annoyances.

Let's look a this example:

ComplexClass cc = ComplexClass.Builder()
  .addThis(42).setThat("I'm that").addSomethingOther("I'm other")
  .addYetAnother("yet yet yet").mixPudding(true).setChickenFeedingDevice(device)
  .addThis(99).withTimeout(5000).setThat("I'm another that").build()

Disregarding silly and inconsistent method naming...

• large number of builder methods confuse user
• builder method can be mistakenly called multiple times

Having way to enforce order into method chaining will allow to build something more like wizzard or workflow which wiil simplify Builder usage greatly.

Let's introduce some interfaces according following rules

• Any interface declares only subset of Builder methods
• Interface method return value is another interface instead of Builder instance
• Builder itself implements all interfaces

Together this basicaly forms very simple example of formal grammar where order in interface chaining represents production rules

To demonstrate idea just described, I built Selenium2/WebDriver WaitBuilder. It uses WbBegin interface as initial building point, WbAnd interface allowing to add multiple multiple conditions and finaly WbEnd with .seconds(int seconds) method instead of traditional .build().

Selenium has WebDriverWait class allowing conditional waits, which is very useful for testing pages, where elements appear dynamicaly or to perform assertion of Post/Redirect/Get (redirect after form submission) in time-boxed manner. SeleniumWaitBuilder allows to combine multiple conditions together.

It enables to write such cool chains such as...

//pass test if "results-table" element will appear in 5 seconds, fail otherwise
SeleniumWaitBuilder.with(driver)
 .passOn().element(By.id("results-table")).seconds(5);

//pass test if title become "Example Domain" and in 5 seconds or fail immediately if it happen to contain "error" string
SeleniumWaitBuilder.with(driver).passOn()
  .title().equals("Example Domain")
 .and().failOn()
  .title().contains("error")
 .seconds(5);

SeleniumWaitBuilder.with(driver).passOn()
  .title().endsWith("Example Domain")
  .element(By.id("result-table"))
 .and().failOn()
  .title().contains("error")
  .url().contains("/500")
 .seconds(5);

And finally, hero of today's blog post - mighty SeleniumWaitBuilder itself!

I admit that this is overkill for most of the builders but still, it is neat...

	package com.nature.quickstep.test;
	import java.util.ArrayList;
	import java.util.List;
	import org.openqa.selenium.By;
	import org.openqa.selenium.WebDriver;
	import org.openqa.selenium.WebDriverException;
	import org.openqa.selenium.WebElement;
	import org.openqa.selenium.support.ui.ExpectedCondition;
	import org.openqa.selenium.support.ui.ExpectedConditions;
	import org.openqa.selenium.support.ui.WebDriverWait;
	/**
	*
	* @author martin.vanek
	*
	*/
	public class SeleniumWaitBuilder {
	public static WbBegin with(WebDriver driver) {
	return new WaitBuilder(driver);
	}
	public interface WbBegin {
	/**
	* Positive condition outcome pass
	*/
	public WbCondition passOn();
	/**
	* Negative condition outcome kill
	*/
	public WbCondition failOn();
	}
	/**
	* Terminal - execute conditional waiting
	*/
	public interface WbEnd {
	public void seconds(int seconds);
	}
	public interface WbAnd {
	public WbBegin and();
	}
	public interface WbNext extends WbAnd, WbCondition, WbEnd {
	}
	public interface WbCondition {
	/**
	* Wait for presence of element
	*/
	public WbNext element(By locator);
	/**
	* Wait for url becomes...
	*
	* public WaitTime url(StringConditionType type, String value);
	*/
	public StringCondition url();
	public StringCondition title();
	/**
	* Wait for root tag become $lt;hmtl$gt; - use to skip redirects, because Selenium hides http status codes
	*/
	public WbNext html();
	/**
	* Generic condition method
	*/
	public WbEnd condition(ExpectedCondition<?> condition);
	}
	public interface StringCondition {
	public WbNext equals(String value);
	public WbNext contains(String value);
	public WbNext endsWith(String value);
	public WbNext matches(String value);
	}
	public static class WaitBuilder implements WbBegin, WbCondition, WbNext, WbEnd {
	private final WebDriver driver;
	private boolean positive;
	private ConditionsList pass = new ConditionsList(true);
	private ConditionsList fail = new ConditionsList(false);
	public WaitBuilder(WebDriver driver) {
	this.positive = true; //important
	this.driver = driver;
	}
	@Override
	public WbCondition passOn() {
	this.positive = true;
	return this;
	}
	@Override
	public WbCondition failOn() {
	this.positive = false;
	return this;
	}
	@Override
	public WbBegin and() {
	return this;
	}
	public WbNext element(By locator) {
	if (locator == null) {
	throw new IllegalArgumentException("Null element locator");
	}
	ExpectedCondition<WebElement> condition = ExpectedConditions.presenceOfElementLocated(locator);
	condition(condition);
	return this;
	}
	public WbNext html() {
	ExpectedCondition<WebElement> condition = ExpectedConditions.presenceOfElementLocated(By.xpath("/html"));
	condition(condition);
	return this;
	}
	public StringCondition url() {
	return new WaitUrlImpl();
	}
	public StringCondition title() {
	return new WaitTitleImpl();
	}
	@Override
	public WbEnd condition(ExpectedCondition<?> condition) {
	if (condition == null) {
	throw new IllegalArgumentException("Null condition");
	}
	if (positive) {
	pass.add(condition);
	} else {
	fail.add(condition);
	}
	return this;
	}
	@Override
	public void seconds(int seconds) {
	WebDriverWait wait = new WebDriverWait(driver, seconds);
	if (pass.size() != 0) {
	if (fail.size() != 0) {
	wait.until(new DualityCollectionCondition(pass, fail));//true or ErrorConditionException or TimeoutException
	} else {
	wait.until(pass); //true or TimeoutException
	}
	} else {
	if (fail.size() != 0) {
	try {
	wait.until(fail); //ErrorConditionException or TimeoutException
	} catch (org.openqa.selenium.TimeoutException tx) {
	//SUCCESS - Error condition did not happened so we can leave
	}
	} else {
	throw new IllegalStateException("Neither pass nor fail conditions");
	}
	}
	}
	class WaitUrlImpl implements StringCondition {
	@Override
	public WbNext equals(String value) {
	URLCondition condition = new URLCondition(value, StringConditionType.EQUALS);
	condition(condition);
	return WaitBuilder.this;
	}
	@Override
	public WbNext contains(String value) {
	URLCondition condition = new URLCondition(value, StringConditionType.CONTAINS);
	condition(condition);
	return WaitBuilder.this;
	}
	@Override
	public WbNext endsWith(String value) {
	URLCondition condition = new URLCondition(value, StringConditionType.ENDS_WITH);
	condition(condition);
	return WaitBuilder.this;
	}
	@Override
	public WbNext matches(String value) {
	URLCondition condition = new URLCondition(value, StringConditionType.MATCH_REGEX);
	condition(condition);
	return WaitBuilder.this;
	}
	}
	class WaitTitleImpl implements StringCondition {
	@Override
	public WbNext equals(String value) {
	TitleCondition condition = new TitleCondition(value, StringConditionType.EQUALS);
	condition(condition);
	return WaitBuilder.this;
	}
	@Override
	public WbNext contains(String value) {
	TitleCondition condition = new TitleCondition(value, StringConditionType.CONTAINS);
	condition(condition);
	return WaitBuilder.this;
	}
	@Override
	public WbNext endsWith(String value) {
	TitleCondition condition = new TitleCondition(value, StringConditionType.ENDS_WITH);
	condition(condition);
	return WaitBuilder.this;
	}
	@Override
	public WbNext matches(String value) {
	TitleCondition condition = new TitleCondition(value, StringConditionType.MATCH_REGEX);
	condition(condition);
	return WaitBuilder.this;
	}
	}
	}
	static class DualityCollectionCondition implements ExpectedCondition<Boolean> {
	private ConditionsList passOn;
	private ConditionsList failOn;
	public DualityCollectionCondition(ConditionsList passOn, ConditionsList failOn) {
	this.passOn = passOn;
	this.failOn = failOn;
	}
	@Override
	public Boolean apply(WebDriver driver) {
	failOn.apply(driver); //false or ErrorConditionException
	return passOn.apply(driver); //true or TimeoutException
	}
	@Override
	public String toString() {
	StringBuilder sb = new StringBuilder();
	if (passOn.conditions.size() == 1) {
	sb.append(passOn.conditions.iterator().next());
	} else {
	sb.append(passOn.conditions);
	}
	if (failOn.conditions.size() != 0) {
	if (sb.length() != 0) {
	sb.append(" and");
	}
	sb.append(" not ");
	if (failOn.conditions.size() == 1) {
	sb.append(failOn.conditions.iterator().next());
	} else {
	sb.append(failOn.conditions);
	}
	}
	return sb.toString();
	}
	}
	static class ConditionsList implements ExpectedCondition<Boolean> {
	private final boolean positive;
	private List<ExpectedCondition<?>> conditions = new ArrayList<ExpectedCondition<?>>();
	public ConditionsList(boolean positive) {
	this.positive = positive;
	}
	public int size() {
	return conditions.size();
	}
	public boolean isPositive() {
	return positive;
	}
	@Override
	public String toString() {
	return "ConditionsList [positive=" + positive + ", conditions=" + conditions + "]";
	}
	public void add(ExpectedCondition<?> condition) {
	if (condition == null) {
	throw new IllegalArgumentException("Null condition");
	}
	conditions.add(condition);
	}
	@Override
	public Boolean apply(WebDriver driver) {
	if (positive) {
	return applyPositive(driver);
	} else {
	return applyNegative(driver);
	}
	}
	/**
	* All conditions must be satisfied to return true
	*/
	private boolean applyPositive(WebDriver driver) {
	for (ExpectedCondition<?> condition : conditions) {
	Object apply = condition.apply(driver);
	if (apply == null) {
	return false; //expected element not returned
	} else {
	if (apply instanceof Boolean) {
	if (!(Boolean) apply) {
	return false; //well false is false
	}
	}
	}
	}
	return true; //nobody said no -> go!
	}
	/**
	* return false or throw ErrorConditionException
	*/
	private boolean applyNegative(WebDriver driver) {
	for (ExpectedCondition<?> condition : conditions) {
	Object result = condition.apply(driver);
	if (result != null) {
	if (result instanceof Boolean) {
	if ((Boolean) result) {
	throw new ErrorConditionException(condition);
	} else {
	return false; //keep checking
	}
	} else {
	//anything else not null
	throw new ErrorConditionException(result, condition);
	}
	} else {
	return false;
	}
	}
	return false; //nobody failed
	}
	}
	public static class ErrorConditionException extends WebDriverException {
	private static final long serialVersionUID = 1L;
	private final ExpectedCondition<?> condition;
	public ErrorConditionException(ExpectedCondition<?> condition) {
	super(String.valueOf(condition));
	this.condition = condition;
	}
	public ErrorConditionException(Object applyResult, ExpectedCondition<?> condition) {
	super("Result: " + applyResult + " for: " + condition);
	this.condition = condition;
	}
	public ExpectedCondition<?> getCondition() {
	return condition;
	}
	}
	public static enum StringConditionType {
	EQUALS, CONTAINS, ENDS_WITH, MATCH_REGEX;
	}
	static abstract class AbstractStringCondition implements ExpectedCondition<Boolean> {
	private final String name;
	private final String expectedValue;
	private final StringConditionType type;
	private final boolean positive;
	private transient String currentValue;
	protected abstract String getCurrentValue(WebDriver driver);
	public AbstractStringCondition(String name, String value, StringConditionType type) {
	this(name, value, type, true);
	}
	public AbstractStringCondition(String name, String value, StringConditionType type, boolean positive) {
	if (name == null || name.length() == 0) {
	throw new IllegalArgumentException("Blank condition name " + name);
	}
	this.name = name;
	if (value == null || value.length() == 0) {
	throw new IllegalArgumentException("Blank condition value " + value);
	}
	this.expectedValue = value;
	if (type == null) {
	throw new IllegalArgumentException("Null condition type");
	}
	this.type = type;
	this.positive = positive;
	}
	@Override
	public Boolean apply(WebDriver driver) {
	currentValue = getCurrentValue(driver);
	if (currentValue == null) {
	return false;
	}
	boolean result;
	switch (type) {
	case EQUALS:
	result = currentValue.equals(expectedValue);
	break;
	case CONTAINS:
	result = currentValue.contains(expectedValue);
	break;
	case ENDS_WITH:
	result = currentValue.endsWith(currentValue);
	break;
	case MATCH_REGEX:
	result = currentValue.matches(expectedValue);
	break;
	default:
	throw new IllegalStateException("Unsupported type: " + type);
	}
	return positive ? result : !result;
	}
	@Override
	public String toString() {
	return String.format("%s \"%s\" %s \"%s\"", name, currentValue, type, expectedValue);
	}
	}
	static class URLCondition extends AbstractStringCondition {
	public URLCondition(String value, StringConditionType type, boolean positive) {
	super("URL", value, type, positive);
	}
	public URLCondition(String value, StringConditionType type) {
	super("URL", value, type, true);
	}
	@Override
	protected String getCurrentValue(WebDriver driver) {
	return driver.getCurrentUrl();
	}
	}
	static class TitleCondition extends AbstractStringCondition {
	public TitleCondition(String value, StringConditionType type, boolean positive) {
	super("Title", value, type, positive);
	}
	public TitleCondition(String value, StringConditionType type) {
	super("Title", value, type, true);
	}
	@Override
	protected String getCurrentValue(WebDriver driver) {
	return driver.getTitle();
	}
	}
	}

view raw SeleniumWaitBuilder.java hosted with ❤ by GitHub

Happy contitional waiting!

Java cloud hosted continuous integration services

Last year I resigned to maintain my own VPS hosted CI server with Git Repos, Jenkins, Sonar, Maven repository, LDAP, etc... and moved source code repositories to GitHub. I'm uploading build artifacts into Sonatype OSSRH in case of libraries and with web applications deployments, I'm still experimenting with Openshift and some other services. But I also lost possibility do build and deploy project on demand. Fortunately many cloud hosted CI services popped up last year or two.

CircleCI

Predefined, not very extensive toolset for Java projects is avaliable. It has recognized my pom.xml automaticaly without any configuration. Uses circle.yml configuration file. It is does not have free plan (only trial), cheapest Solo plan is for $19/month.

Travis-CI

Java is quite well supported. It is cofigured using .travis.yml. There is free (unlimited) plan and also quite expensive paid plans

Codeship

Neat interface, but you are allowed to use only few preinstalled java tools. Free plan is limited to 50 builds per month, while Basic plan will cost you $49 per month.

drone.io

Simplistic with basic java support. Free unlimited plan.

Cloudbees DEV@Cloud

Jenkins in the cloud with all it's awesomeness. You can choose pretty much any Java, Maven or Gradle version you can imagine plus zillions of Jenkins plugins. Pricing is trickier here because billing is build time based and offering includes application hosting (you might not be interested in). Free plan includes 100 build minutes and there is also FOSS plan. Starter plan with 40 hours of build per month will cost you $60 + $17 = $77 per month

Another Cloudbees service is BuildHive where you will get unlimited number of builds, but you will use shared and slower Jenkins instance with very limited configuration and only for GitHub repositories.

Test drive with Phanbedder

I have recently built little library named Phanbedder and blogged about it few days ago. While very tiny in java code size, it is pretty unusual because it launches separate processes of bundeled PhantomJS native binary during the tests. This makes it perfect candidate for testing cloud CI services.

Good news is that every above mentioned service managed to compile and test it using mvn clean test -Denforcer.skip=true command. I use maven-enforcer-plugin to enforce Java 6 to be used for compilation. But because many of CIs offers only Java 7, I had to switch enforcer off...

Trickier part is execution of mvn deploy. I'm uploading maven snapshot artifacts into Sonatype OSSRH and also release versions later into Maven central. For snapshot deployment into Sonatype OSS Nexus, username and password must be provided to make maven-deploy-plugin work, otherwise...

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-deploy-plugin:2.8.1:deploy (default-deploy) on project phanbedder-1.9.7: Failed to deploy artifacts: Could not transfer artifact net.anthavio:phanbedder-1.9.7:jar:1.0.1-20140422.184656-7 from/to sonatype-oss-snapshots (https://oss.sonatype.org/content/repositories/snapshots): Failed to transfer file: https://oss.sonatype.org/content/repositories/snapshots/net/anthavio/phanbedder-1.9.7/1.0.1-SNAPSHOT/phanbedder-1.9.7-1.0.1-20140422.184656-7.jar. Return code is: 401, ReasonPhrase: Unauthorized. -> [Help 1]

Normaly, deployment credentials are stored inside your personal maven settings.xml, but obviously this file is not present on cloud CI server. Workarounds exists for Travis-ci. Cloudbees DEV@Cloud has most elegant solution, but I guess no deployment with undisclosed credentials can be done from BuildHive and others.

Now, here goes links to public success build logs for Travis-CI, drone.io, BuildHive and DEV@Cloud. Sadly CircleCI and Codeship does not seem to support public projects.

Mentioned CI services can be webhook triggered, provide integrated browser testing and deployment into popular cloud hosting services like Heroku or Google App Engine. They evolve pretty quickly as they add more integrations as I write... Listing and comparing features here is not worth the effort so check documentation pages.

Everybody has different needs. All my projects are open source hobby projects so I guess I'll go with webhook triggered snapshot builds on BuildHive or I might use Travis-CI to have it with snapshot deployments.

For Maven central release deployment builds, I see only one option - Cloudbees DEV@Cloud. In some next blog post, I'll describe how fully automatic deployment can be achieved using Cloudbees DEV@Cloud, maven-release-plugin and Sonatype OSSRH.

Happy cloud hosted CI builds!

PhantomJS embedder for Selenium GhostDriver

There is quite a few Drivers for Selenium2/WebDriver. Two of them are particulary interesting, because they allow fast headless browser tests - HtmlUnitDriver and PhantomJSDriver.

Using HtmlUnitDriver is piece of cake, because HtmlUnit is pure java library, but disadvantage is limited JavaScript execution support, making it usable for mostly static html sites only.

PhantomJSDriver from GhostDriver project allows to employ PhantomJS, which is a headless WebKit, much much closer to real web browser. As native binary having it's dependencies staticaly linked, PhantomJS does not need any installation. Just unzip it somewhere.

Selenium 2 has annoying habit of needing to specify full path to browser binary when Driver instace is being created. For most common browsers like Chrome or Firefox, some basic discovery is performed, but it usualy fails for me and it is same story with PhantomJS now.

Most probably, you will get folowing exception

java.lang.IllegalStateException: The path to the driver executable must be set by the phantomjs.binary.path capability/system property/PATH variable; for more information, see https://github.com/ariya/phantomjs/wiki. The latest version can be downloaded from http://phantomjs.org/download.html
 at com.google.common.base.Preconditions.checkState(Preconditions.java:177)
 at org.openqa.selenium.phantomjs.PhantomJSDriverService.findPhantomJS(PhantomJSDriverService.java:237)
 at org.openqa.selenium.phantomjs.PhantomJSDriverService.createDefaultService(PhantomJSDriverService.java:182)
 at org.openqa.selenium.phantomjs.PhantomJSDriver.<init>(PhantomJSDriver.java:99)

Another obstacle usualy is different Operating System between developer's machine (MacOS, Windows) and continuous integration server (Linux). Because PhantomJS is native application, therefore every OS needs it's own binary to execute and you'll need to distribute right versions to every host that will ever build/test your project.

To escape from both annoyances, I've created Phanbedder. Tiny library that bundles PhantomJS binaries and unpacks right one for you on any of supported platform - Linux, Windows and Mac OSX.

File phantomjs = Phanbedder.unpack(); //Phanbedder to the rescue!
DesiredCapabilities dcaps = new DesiredCapabilities();
dcaps.setCapability(PhantomJSDriverService.PHANTOMJS_EXECUTABLE_PATH_PROPERTY, phantomjs.getAbsolutePath());
PhantomJSDriver driver = new PhantomJSDriver(dcaps);

//Usual Selenium stuff follows
try {
 driver.get("https://www.google.com");
 WebElement query = driver.findElement(By.name("q"));
 query.sendKeys("Phanbedder");
 query.submit();
 Assertions.assertThat(driver.getTitle()).contains("Phanbedder");
} finally {
 driver.quit();
}

To run code above you'll need following dependencies. Number 1.9.7. in artifactId stands for PhantomJS version bundeled inside.

    
      net.anthavio
      phanbedder-1.9.7
      1.0.0
    

    
      com.github.detro.ghostdriver
      phantomjsdriver
      1.1.0
    

Because this library is targeting various OSes, testing is tricky. I have tested it on Mac OS X 10.6, Windows 7 and I've also gave it a spin on travis-ci (good) and Cloudbees Jenkins (great). Source code is hosted on GitHub and deployed into maven central again using Cloudbees Cloud@DEV

Happy Phanbedding!

How many Base64 encoders is in JDK/JRE

Base64 variants

Base64 encoding is an algorithm that converts binary data into ASCII. Resulting string consist of characters A-Z,a-z,0-9 and two extra '+' (plus) and '/' (slash) and also padding character '=' (equals). Conversion does not happens the same every time, there is few variants of it.

- Simple (basic) encoding creates single longlonglonglonglonglonglonglonglonglonglonglooooong= base64 encoded line.

- Fixed line lenght encoding, sometimes also called Mime base64 encoding or chunked encoding or simply line folding. Instead of single long line, it produces multiple lines usualy 76 characters long. It is quite important, because it is mandatory in some use scenarios (Binary attachments) , while harmful in others (BASIC authentication header).

- URL (safe) encoding produce string that can be used as parameter value in URL. Because '+' and '/' are not allowed, they are encoded as '-' and '_' while '=' padding character is usually removed.

Now back to initial question...

How many Base64 encoders/decoders is present in Oracle (Sun) JDK?

I found 6 of them

sun.misc.BASE64Encoder Since Java 1.0? Well we all know that we should not touch anything from sun.* or com.sun.* packages. So we don't.

javax.xml.bind.DatatypeConverter Since Java 1.6 - This one actually works, but allows you only basic encoding. No mime or url encoding.

java.util.Base64 Since Java 1.8 - Finaly generaly usable Base64 encoder/decoder allowing basic, mime and url safe encoding.

And finaly some curiosities illustrating how even Sun/Oracle JDK/JRE contributors were missing Base64 encoder, so they created their own.

java.util.prefs.Base64 Since Java 1.4, but has default (package) visibility, therefore not usable

com.sun.net.httpserver.Base64 Since Java 1.6, but has default (package) visibility, therefore not usable

com.sun.org.apache.xml.internal.security.utils.Base64 - Similar story as sun.misc.BASE64Encoder, it also internaly uses XMLUtils.ignoreLineBreaks() to perform line folding...

Let's see some encoding results

Both commons-codec 1.6+ and Java8 java.util.Base64 can produce and consume any of mentioned base64 variants, but beware of quite different encoding results. I think that a lot of headaches is coming because of that.

In following test, commons-codec 1.9 and Java8u5 is used

Mime (chunked) encoding

import org.apache.commons.codec.binary.Base64;

String string = "This string encoded will be longer that 76 characters and cause MIME base64 line folding";
 
byte[] encodeBase64Chunked = Base64.encodeBase64Chunked(string.getBytes());
System.out.println("commons-codec Base64.encodeBase64Chunked\n" + new String(encodeBase64Chunked));

String encodeMimeToString = java.util.Base64.getMimeEncoder().encodeToString(string.getBytes());
System.out.println("java.util.Base64.getMimeEncoder().encodeToString\n" + encodeMimeToString);

prints

commons-codec Base64.encodeBase64Chunked
VGhpcyBzdHJpbmcgZW5jb2RlZCB3aWxsIGJlIGxvbmdlciB0aGF0IDc2IGNoYXJhY3RlcnMgYW5k
IGNhdXNlIE1JTUUgYmFzZTY0IGxpbmUgZm9sZGluZw==

java.util.Base64.getMimeEncoder().encodeToString
VGhpcyBzdHJpbmcgZW5jb2RlZCB3aWxsIGJlIGxvbmdlciB0aGF0IDc2IGNoYXJhY3RlcnMgYW5k
IGNhdXNlIE1JTUUgYmFzZTY0IGxpbmUgZm9sZGluZw==

Java8 mime Encoder ends with '==' padding and does not add last newline (CR/LF) after that!

URL (safe) encoding

String string = "ůůůůů";

String encodeUrlToString = java.util.Base64.getUrlEncoder().encodeToString(string.getBytes());
System.out.println("java.util.Base64.getUrlEncoder().encodeToString\n" + encodeUrlToString);

String encodeBase64URLSafeString = Base64.encodeBase64URLSafeString(string.getBytes());
System.out.println("commons-codec Base64.encodeBase64URLSafeString\n" + encodeBase64URLSafeString);

prints

java.util.Base64.getUrlEncoder().encodeToString
xa_Fr8Wvxa_Frxc=
commons-codec Base64.encodeBase64URLSafeString
xa_Fr8Wvxa_Frxc

Java8 url Encoder leaves padding '=' at the end of the result, which makes it unusable as URL parameter value!

UPDATE: This was reported while ago and it has turned out, that any Encoder can be switched into non-padding using withoutPadding() method.

String string = "ůůůůů";
String encodeUrlToString = Base64.getUrlEncoder().withoutPadding().encodeToString(string.getBytes());
System.out.println("java.util.Base64.getUrlEncoder().withoutPadding().encodeToString\n" + encodeUrlToString);

prints

java.util.Base64.getUrlEncoder().withoutPadding().encodeToString
xa_Fr8Wvxa_Frw

Note: In quite old commons-codec 1.4 chunking was incostitently turned on by default for encode() method, resulting in nasty surprises. See Jira ticket.

Happy Base64 encoding

Spring MockMvc tests

MVC Test Framework arrived with Spring 3.2. It allows to write integration tests (well... almost) for your Spring MVC @Controller(s)

One of server-side cornerstones is MockMvc class. It allows to execute requests on your Controllers very easily, but it needs to be initialized before it is used. Threre are two ways to initialize MockMvc and choice depends on how broad integration with other Spring MVC components your tests need. In my case, it was custom view resolver and few others I removed for sake of simplicity.

First is meant for simple single Controller testing

    @Test
    public void foo() {
        TestedController controller = new TestedController();
        MyCustomViewResolver resolver = new MyCustomViewResolver();
        MockMvc mockMvc = MockMvcBuilders.standaloneSetup(controller).setViewResolvers(resolver).build();

        MvcResult result = this.mockMvc.perform(MockMvcRequestBuilders.get("/foo")).andReturn();
        //parse and assert response.getContentAsString() as view was resolved by MyCustomViewResolver
    }

You can employ extensive set of spring-mvc usual suspects, like ConversionService, ViewResolvers, MessageConverters, ... see StandaloneMockMvcBuilder javadoc

All this manual assembling makes me feel little unconfortable. Normaly all those beans are wired together in Spring Dispatcher context.

Another way to initlialize MockMvc is using @WebAppConfiguration and MockMvcBuilders.webAppContextSetup(webAppContext)

@RunWith(SpringJUnit4ClassRunner.class)
@WebAppConfiguration
@ContextConfiguration(classes = TestMvcSpringConfig.class)
public class FooTests {

    @Autowired
    private WebApplicationContext webAppContext;

    private MockMvc mockMvc;

    @Before
    public void setup() {
        this.mockMvc = MockMvcBuilders.webAppContextSetup(webAppContext).build();
    }

    @Test
    public void foo() {
        MvcResult result = this.mockMvc.perform(MockMvcRequestBuilders.get("/foo")).andReturn();
        MockHttpServletResponse response = result.getResponse();
        //parse and assert response.getContentAsString() as view was resolved by MyCustomViewResolver
    }

    @EnableWebMvc
    @Configuration
    public static class TestMvcSpringConfig extends WebMvcConfigurerAdapter {

        @Override
        public void configureDefaultServletHandling(DefaultServletHandlerConfigurer configurer) {
            configurer.enable();
        }

        @Bean
        public TestedController getController() {
            return new TestedController();
        }

        @Bean
        public ViewResolver viewResolver() {
            MyCustomViewResolver resolver = new MyViewCustomResolver();
            return resolver;
        }
    }

For some unknown reason (that disappeared as quickly as it appeared) Spring was failing to create @EnableWebMvc annotated @Configuration, complaining about missing servlet context. I had to get my hands dirty and roll my own Spring web context. So for completeness, here is how it was done using MockServletContext. May be useful sometime.

    @Test
    public void test() throws Exception {
        MockServletContext servletContext = new MockServletContext();

        AnnotationConfigWebApplicationContext springContext = new AnnotationConfigWebApplicationContext();
        springContext.setServletContext(servletContext);
        springContext.register(TestMvcSpringConfig.class);
        springContext.refresh();

        MockMvc mockMvc = MockMvcBuilders.webAppContextSetup(springContext).build();

        MvcResult result = mockMvc.perform(MockMvcRequestBuilders.get("/foo")).andReturn();
    }

Happy Spring MVC testing!

RPM upgrade and embedded Jetty

Java web application I'm working on recently (let's name it lobster) is embedding and packaging Jetty inside itself. This makes application artifact more self-contained and independent, because it does not require preinstalled and preconfigured server.

To simplify application deployment even more, it is packaged as RPM file, which is uploaded into Nexus serving as Yum repository.

Deployment then is simple matter of executing sudo yum install lobster and thanks to RPM scriptlets, server is automaticaly restarted as a part or installation.

Everything went nice and smoothly, until second release. When yum update happend, we have found how amazingly wierd thing RPM upgrade is.

sudo yum update lobster execution completed, but application failed to start and logfile contained strange exceptions

Caused by: java.io.FileNotFoundException: /opt/lobster/lib/lobster-core-0.4.0.jar (No such file or directory)
 at java.util.zip.ZipFile.open(Native Method) ~[na:1.7.0_40]
 at java.util.zip.ZipFile.(ZipFile.java:215) ~[na:1.7.0_40]
 at java.util.zip.ZipFile.(ZipFile.java:145) ~[na:1.7.0_40]
 at java.util.jar.JarFile.(JarFile.java:153) ~[na:1.7.0_40]
 at java.util.jar.JarFile.(JarFile.java:90) ~[na:1.7.0_40]
 at sun.net.www.protocol.jar.URLJarFile.(URLJarFile.java:93) ~[na:1.7.0_40]
 at sun.net.www.protocol.jar.URLJarFile.getJarFile(URLJarFile.java:69) ~[na:1.7.0_40]
 at sun.net.www.protocol.jar.JarFileFactory.get(JarFileFactory.java:99) ~[na:1.7.0_40]
 at sun.net.www.protocol.jar.JarURLConnection.connect(JarURLConnection.java:122) ~[na:1.7.0_40]
 at sun.net.www.protocol.jar.JarURLConnection.getJarFile(JarURLConnection.java:89) ~[na:1.7.0_40]

What was wierd even more, that it was lobster version 0.5.0 installation and 0.4.0 stated in stacktrace, actualy was previous version!

To make story short, after some googling I've found RPM upgrade sequence.

1. execute new version %pre [ $1 >= 2 ]
2. unpack new version files (both old and new are now mixed together)
3. execute new version %post [ $1 >= 2 ]
4. execute old version %preun [ $1 == 1 ]
5. remove old files (only new and unchanged stays)
6. execute old verion %postun [ $1 == 1 ]

Important is, that between step 2 and 5, mixture of old and new version jar files is present in installation directory! Attempt to start server java process in %post or %preun scriptlet can only result in disaster same as we experienced. Old version jars will be deleted right after scriptlet execution.

Here comes working install/upgrade/uninstall solution

%pre scriptlet

#!/bin/sh
# rpm %pre scriptlet
#
# parameter $1 means
# $1 == 1 ~ initial installation
# $1 >= 2 ~ version upgrade
# never executed for uninstall

echo "rpm: pre-install $1"

# failsafe commands - can't break anything

# make sure that user exist
id -u lobster &>/dev/null || useradd lobster

# make sure that application is not runnig
if [ -f /etc/init.d/lobster ]; then
 /sbin/service lobster stop
fi

%post scriptlet

Important is NOT to start application on rpm upgrade

#!/bin/sh
# rpm %post scriptlet
#
# parameter $1 means
# $1 == 1 ~ initial installation
# $1 >= 2 ~ version upgrade
# never executed for uninstall

echo "rpm: post-install $1"

# initial install
if [ "$1" -eq "1" ]; then
 /sbin/chkconfig --add lobster
 /sbin/service lobster start
fi

%preun scriptlet

#!/bin/sh
# rpm %preun scriptlet
#
# parameter $1 means
# $1 == 0 ~ uninstall last
# $1 == 1 ~ version upgrade
# never executed for first install

echo "rpm: pre-uninstall $1"

# uninstall last
if [ "$1" -eq "0" ]; then
 /sbin/service lobster stop
 /sbin/chkconfig --del lobster
fi

%postun scriptlet

Here is right place to start application on rpm upgrade

#!/bin/sh
# rpm %postun scriptlet
#
# parameter $1 means
# $1 == 0 ~ uninstall last
# $1 == 1 ~ version upgrade
# never executed for first install

# console output is surpressed for postun% !
echo "rpm: post-uninstall $1"

# upgrade
if [ "$1" -ge "1" ]; then
 /sbin/service lobster start
fi

Useful RPM scriptlet documentation is in Fedora Wiki also how to integrate with SysV Init Scripts

Happy RPM deployments!