Thursday 28 November 2013

Generic throws

Suprisingly few developers really master Java generics in their full extent. As apparently not being one of them, generic throws feature surprised me lately.

Following code snippet shows the idea - generic <X extends Exception> is used in throws part of go() method. Implementing class defines Exception subclass that will be allowed to be thrown.

import java.io.FileNotFoundException;
interface Something<X extends Exception> {
  
 public abstract void go() throws X;
  
}
 
class SomethingImpl implements Something<FileNotFoundException> {
  
 public void go() throws FileNotFoundException {
  throw new FileNotFoundException("Yeah! Not found!");
 }
}

Interesting trick, but how can this be useful? Well, for example, you can pimp Strategy pattern with cunning Exception passing/handling!

In Strategy pattern, you basicaly have two choices how to deal with exceptions
  • Add throws Exception on both strategy interface and context method to pass every exception back to client - especially poor choice
  • Add throws Exception on strategy interface method, catch it inside context and rethrow it wrapped inside RuntimeExcetpion back to client - usual solution, but it mixes strategy and context exceptions together
But with generic throws, aditional options are available.

This is interface allowing Client to implement his own LoaderStrategy and it also allows him to choose Exception which can be thrown from his LoaderStrategy via generic throws X

/**
 * Strategy interface - To be implemented by client
 */
interface LoaderStrategy<X extends Exception> {

 public String load(long id) throws X;

}

And this is simple Context using LoaderStrategy interface. It is also passing chosen exceptions from strategy back to client, again via generic throws X

/**
 * Context using LoaderStrategy, passing selected exceptions (X)
 */
class StrategyContext {

 public <X extends Exception> void perform(LoaderStrategy<X> oader) throws X {

  long resourceId = getIdFromSomewhere();

  String data = loader.load(resourceId); //throws X - don't have to hadle exception here

  try {
   sendDataToRemoteSystem(data);
  } catch (RemoteException rx) {
   //Other checked Exception must be wrapped inside ContextException 
   throw new ContextException("Failed to send data for resourceId " + resourceId, rx);
  }
 }

 private long getIdFromSomewhere() {
  return System.currentTimeMillis(); //CurrentTimeMillis is best id ever!
 }

 private void sendDataToRemoteSystem(String data) throws RemoteException {
  //invoke some remoting operation here...
 }
}

/**
 * Context exception wrapper - delivers other then LoaderStrategy exceptions to client 
 */
class ContextException extends RuntimeException {

 public ContextException(String string, Exception cause) {
  super(string, cause);
 }
}

Use case 1 - Passing checked Exceptions

This is client's LoaderStrategy implementation. It throws IOException
/**
 * Client provided LoaderStrategy implementation throwing IOException
 */
class ReaderLoader implements LoaderStrategy<IOException> {

 private Reader reader;

 public ReaderLoader(Reader reader) {
  this.reader = reader;
 }

 public String load(long id) throws IOException {
  String sid = String.valueOf(id);
  BufferedReader br = new BufferedReader(reader);
  String line = null;
  while ((line = br.readLine()) != null) {
   if (line.startsWith(sid)) {
    return line;
   }
  }
  return null;
 }
}
Then IOException is propagated back into client code to be handeled here
 StrategyContext context = new StrategyContext();

 ReaderLoader readerLoader = new ReaderLoader(new StringReader("Blah! Blah! Blah!"));
 try {
  context.perform(readerLoader);
 } catch (IOException iox) {
  //ReaderLoader thrown IOException is propagated through StrategyContext and delivered here
 } catch (ContextException cx) {
  //StrategyContext thrown exception wrapper - All sorts of NON ReaderLoader originated Exceptions
  Throwable cause = cx.getCause();
 }
This does not brings much value, but in client code we can easily distinguish our ReaderLoader originated exceptions from Context originated ones and handle them differently. Simple two catch clauses.

Use case 2 - Customized checked Exceptions

Client also provides custom Exception he want to be thrown from his Strategy and propagated through StrategyContext back to client
/**
 * Client provided custom Exception
 */
class InvalidRecordCountException extends Exception {

 private long recordId;

 public InvalidRecordCountException(long recordId, String message) {
  super(message);
  this.recordId = recordId;
 }

 public InvalidRecordCountException(long recordId, SQLException exception) {
  super(exception);
  this.recordId = recordId;
 }

 public long getRecordId() {
  return recordId;
 }
}

/**
 * Client provided LoaderStrategy implementation throwing custom InvalidRecordCountException
 */
class JdbcLoader implements LoaderStrategy<InvalidRecordCountException> {

 private DataSource dataSource;

 private String select;

 public JdbcLoader(DataSource dataSource, String select) {
  this.dataSource = dataSource;
  this.select = select;
 }

 @Override
 public String load(long id) throws InvalidRecordCountException {
  Connection connection = null;
  PreparedStatement statement = null;
  ResultSet resultSet = null;
  try {
   connection = dataSource.getConnection();
   statement = connection.prepareStatement(select);
   statement.setLong(1, id);
   resultSet = statement.executeQuery();
   if (resultSet.next()) {
    String string = resultSet.getString(1);
    if (resultSet.next()) {
     //here we go...
     throw new InvalidRecordCountException(id, "Too many somethings in somewhere!");
    }
    return string;
   } else {
    //here we go...
    throw new InvalidRecordCountException(id, "Not a single something in somewhere!");
   }
  } catch (SQLException sqlx) {
   //here we go...
   throw new InvalidRecordCountException(id, sqlx);
  } finally {
   //TODO close resultSet, statement, connection
  }

 }
}
Then SQLExceptions are handled while InvalidRecordCountException propagated
 StrategyContext context = new StrategyContext();

 DataSource dataSource = null; //get it from somewhere
 JdbcLoader jdbcLoader = new JdbcLoader(dataSource, "SELECT something FROM somewhere WHERE column = ?");
 try {
  context.perform(jdbcLoader);
 } catch (InvalidRecordCountException ircx) {
  //JdbcLoader thrown InvalidRecordCountException is propagated through StrategyContext and delivered here
  long badRecordId = ircx.getRecordId(); //we can take some action when knowing failing record id
 } catch (ContextException cx) {
  //StrategyContext thrown exception wrapper - SQLException will be cause propably...
  Throwable cause = cx.getCause();
 }
Now we can easily pass additional error information inside custom exception.

Well, I understand that nobody loves to write lots of single purpose exceptions, but adopting "generic throws idiom" quite increases exception reusability.

All used code can by found in kitchensink repository

Wednesday 27 November 2013

How many javax.annotation jars is out there?!?!

Today I've me very strange Exception. Here comes it's mighty stacktrace.

Caused by: java.lang.SecurityException: class "javax.annotation.Nullable"'s signer information does not match signer information of other classes in the same package
 at java.lang.ClassLoader.checkCerts(ClassLoader.java:806)
 at java.lang.ClassLoader.preDefineClass(ClassLoader.java:487)
 at java.lang.ClassLoader.defineClassCond(ClassLoader.java:625)
 at java.lang.ClassLoader.defineClass(ClassLoader.java:615)
 at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
 at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
 at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
 at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
 at java.security.AccessController.doPrivileged(Native Method)
 at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
 at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
 at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
 at java.lang.ClassLoader.loadClass(ClassLoader.java:247)


Application is Maven based and there are loads of transitive dependencies. Time for good classpath hunt!


Going through classpath revealed that only one javax.annotation.Nullable class is present in com.google.code.findbugs:jsr305:1.3.9 dependency. Aparently it is not a case of having multiple class definition, which happens quite often with javax.servlet API classes.
Searching further I found another dependency, with same javax.annotation package, but with diferent (jsr250) classes inside - org.eclipse.jetty.orbit:javax.annotation:1.1.0.v201108011116 This jar is special. It is (jarsigner) signed with certificate to guard it's content.
Case is clear now, package javax.annotation from inside of signed jsr250 jar is protected from being tampered and classes from unsigned jsr305 jar, having same package, are considered malicious as they are trying to sneak in signed jar's package. JVM performs check when such classes are loaded and eventually throws SecurityException.

My solution was simply to exclude com.google.code.findbugs:jsr305 dependency, because it was not really needed. When this is not an option, then signed orbit org.eclipse.jetty.orbit:javax.annotation dependency must be excluded and use some another (unsigned) added instead. Truly annoying and googling around revealed that poor package javax.annnotation is almost same transitive classpath blight as infamous commons-logging.jar

Short summary of javax.annnotation libraries

JSR 250: Common Annotations for the JavaTM Platform

  • javax.annotation (Generated, ManagedBean , PostConstruct, PreDestroy, Resource, Resources)
  • javax.annotation.security (DeclareRoles, DenyAll, PermitAll, RolesAllowed, RunAs) 
  • javax.annotation.sql (DataSourceDefinition, DataSourceDefinitions) - since version 1.1 
Usual Maven suspects:
  • jsr250-api.jar - http://mvnrepository.com/artifact/javax.annotation/jsr250-api
  • javax.annotation-api.jar - http://mvnrepository.com/artifact/javax.annotation/javax.annotation-api
  • Geronimo - http://mvnrepository.com/artifact/org.apache.geronimo.specs/geronimo-annotation_1.0_spec
  • Glassfish - http://mvnrepository.com/artifact/org.glassfish/javax.annotation
  • Jetty Orbit - http://mvnrepository.com/artifact/org.eclipse.jetty.orbit/javax.annotation
  • And loads of others - http://mavenhub.com/c/javax/annotation/postconstruct

JSR 305: Annotations for Software Defect Detection

  • javax.annotation (CheckForNull, ... , Nonnull, Nullable, ... , WillNotClose)
  • javax.annotation.concurrent (GuardedBy, ... , ThreadSafe)
  • javax.annotation.meta (Exclusive, ... , When)
Usual Maven suspects:
Oddly enough, Findbugs packaged jar is only Maven distributed version
  • jsr305.jar http://mavenhub.com/c/javax/annotation/nullable/jar 


Early version of jsr299 (WebBeans, later CDI) tried to jump into javax.annotation package too, but fortunately were kicked out
  • http://mavenhub.com/c/javax/annotation/named
  • javax.annotation.Named
  • javax.annotation.NonBinding
  • javax.annotation.Stereotype 
But wait, there is more!
For example obscure jsr308. And starting with Java 6, stub of jsr250 (Generated, PostConstruct, PreDestroy,Resource, Resources) is part of standard library -http://docs.oracle.com/javase/6/docs/api/javax/annotation/package-summary.html
Well this is real mess

Thursday 25 April 2013

JDOM2 namespaces

JDOM2 is relatively new (2012) and it is replacing ancient, pre Java 5 generic, JDOM1 version. You can find nice set of basic examples here.

My requirement was to generate xml with single namespace without prefix. Quite common task you can say, but this happend to be little bit tricky. I thought that namespace, once set on Element, should be inherited by child Elements. Of course unless you intentionaly set another namespace on some child. I was wrong!

You must explicitly set very same namespace over and over again on every Element you create. Very annoying indeed. There is a page dedicated to namespace scoping but call me stupid, it haven't help me single bit.

Luckily, after you are finished with creating elements, you can "namespacize" them in one shot with following simple recursive method...

import org.jdom2.Element;
import org.jdom2.Namespace;

public static void setNamespace(Element element, Namespace namespace) {
 element.setNamespace(namespace);
 List children = element.getChildren();
 for (Element child : children) {
  setNamespace(child, namespace);
 }
}
Happy new namespaces!

Thursday 18 April 2013

Building Fluent Builder

Recently I've rediscovered another wheel again, but this one is pretty interesting I think...

Simple Fluent Builder

Builder is rather simple desing pattern. It accumulates bunch of values and then uses them to construct some complex product.

Fluent builder is a builder with fluent interface. That means it returns reference to self from every method (except the build() method that returns product) so it allows method chaining.
Nice and simple example is StringBuilder from java standard library.

String hello = new StringBuilder()
  .append("h").append("e").append("l").append("l").append("o")
  .toString();

Generic getThis() trick

When Builder is single standalone class, using return this is dead simple solution, but when inheritance comes into play, builder base class methods cannot return this, because we need concrete builder to be returned to allow method chaining.

Solution is the getThis() trick, that allows to return reference to concrete builder even from base class that builder is extending. The price is that every concrete builder must implement getThis() method.

Here is a nice article with some background and explanation

Generic parent trick

I've found another slightly different scenario, when similar generic trick can be used. It employs two builders and uses generic parent (instead of generic self) to keep track of builders used in chaining. It is quite simple JSON builder.

JSON message is built on two structures: object and array, therefore we will have two concrete builders working together. We don't want builder user to be confused by offering object building methods to him when he constructs array and contrariwise. When user completes constructing array or object, previous builder must be restored and returned to user.

This builder completely prevents user from making wrong method call at any point of building.

JSR-353 - JSON-P

Brand new Java API for JSON processing is going to be part of the upcoming JEE 7 specification also contains JSON builder.
Surprisinly to me they came with simplest builder that is throwing exception when user calls object/array builder method out of correct object/array building context.
For example this code
JsonGenerator jg = javax.json.Json.createGenerator(System.out);
jg.writeStartObject().write("array_element").writeEnd();
executed will throw
javax.json.stream.JsonGenerationException: write(String) can only be called in array context
Very error prone API indeed! I've filed improvement into Jira, but since JSR already passed the Final Approval Ballot, I guess there is no hope.

Tuesday 5 March 2013

nslookup vs /etc/hosts

TLDR: nslookup command does NOT use /etc/hosts for domain name to IP resolution.

Recently I needed to simulate outage of Disqus server providing REST endpoints for commenting.

Because client application, using this service, was deployed into horrible CMS server, changing URL for Disqus service was quite painful. OS on the server is Red Hat Enterprise Linux, so I decided to simply add entry into /etc/hosts translating "disqus.com" domain name to 127.0.0.1 (localhost). Purpose was to hijack DNS resolution into real public Disqus IP address, obtained from DNS server, to make disqus.com calls fail. Dead simple.

I've added line 127.0.0.1 disqus.com into /etc/hosts and executed

nslookup disqus.com
expecting to see the difference, but there wasn't any and nslookup was still returning Disqus public IP address. Suspecting DNS caching for causing this, I've started looking for a way to flush DNS translation cache. Linux standard way seems to have nscd daemon running and refreshing it
nscd -I hosts
should do the trick. But surprisingly, nscd was not running on that server. Neither did any of named, bind, rscd, dnsmasq...

Probably only way to flush dns cache in this situation could be restart of whole networking subsystem (/etc/init.d/network restart), but that wasn't something I could do on that server.

Then, just to be double sure, I executed

ping disqus.com
and spotted that it is pinging 127.0.0.1 as I wanted! But nslookup, executed again, was still showing public Disqus IP address!
As it has turned out that, nslookup is always doing DNS server lookup and it is ignoring your /etc/hosts file

Lesson learned - Don't test your /etc/hosts changes with nslookup. Use ping instead.

It is still some DNS caching done in Java, because ping reflected IP changes immediately, but application running in Java server changed IP after while. I haven't measured it precisely, but it seemed to be about one minute. As it turned out, this delay was actually caused by Varnish and has nothing to do with Java... sorry

All of this happened of RHEL 5.7 (cat /etc/*-release)

PS: Just for completeness. To make sure that /etc/hosts file will take precedence over DNS servers, in process of hostname resolution, following should be true
  • File /etc/host.conf should contain line order hosts,bind
  • File /etc/nsswitch.conf should contain line hosts: files dns

Monday 4 March 2013

ConnectTimeoutTest

From time to time I need to simulate one special type of timeout and that is connect timeout.

It differs from more common read timeout, which may happen when connection is already opened, request is sent, but it takes too long to get response. Connection refused is also different beast, meaning that server actively refused your request, probably while you have been accessing wrong host or port. You should not ignore this connect timeout just because it is less common. When it hits, it will get you down you down as quickly as read timeout.

Connect timeout can occur during connection opening process. That is after connection attempt was not refused and before any possible read timeout.

Fairly rare occasions lead to this type of timeout. It may be firewall misconfiguration silently throwing away your packets, network failure or server can be dying down being unable to even to open socket correctly. Any of those conditions is quite hard to meet so when you need to write some piece of software anticipating this type of timeout, some deterministic way to induce it on demand will come handy.

Basic idea is to create ServerSocket with one position long backlog. Backlog is basically request queue and if we make artificial request to fill it up, any consequent request is doomed.

Update: Backlog queue implementation is differs between platforms. This does work on Mac OS X and does NOT on Linux or Windows. Damn it!

Test method uses standard java HttpURLConnection to make the request and it's setConnectionTimeout to set connection timeout to some rasonable value.

Here comes the complete test case:

Oddly enough, different exception (java.net.ConnectException: Operation timed out) is thrown after 75 seconds in case that connection timeout is NOT set, comparing to case when it IS set (java.net.SocketTimeoutException: connect timed out).

Bonus code! Setup code for Apache Httpclient might look like:

HttpConnectionParamBean connectionBean = new HttpConnectionParamBean(httpParams);
connectionBean.setConnectionTimeout(getConnectTimeout());//httpParams.setParameter(CoreConnectionPNames.CONNECTION_TIMEOUT, 1000L);
connectionBean.setSoTimeout(getReadTimeout());//httpParams.setParameter(CoreConnectionPNames.SO_TIMEOUT, 5000L);
DefaultHttpClient httpClient = new DefaultHttpClient(httpParams);

Enjoy your timeouts!