Monday 20 June 2016

Maven, SBT, Gradle local repository sharing

I've run into environment where different project were using different build tools - Maven, Gradle and SBT. Problem was how to reuse build artifacts between them and here goes small summary of my investigation. I will cover only local file repositories and NOT publishing/retrieving to/from remote repositories.

TL;DR every tool is able to use publish into and retrieve from Maven local repository

I'm using today's latest versions of Maven 3.3.9, Gradle 2.14, SBT 0.13.11 While Maven dependency management and artifact publishing is pretty stable, SBT and Gradle are still evolving quite a lot so check your versions carefully.

All build tools (unless you override default configuration) use your home directory to keep local repository inside of it. It is different on various operating systems and with my user name mvanek, then it will be

  • Linux ${USER_HOME} is /home/mvanek
  • Mac OS X ${USER_HOME} is /Users/mvanek
  • Windows 10 %USERPROFILE% is C:\Users\mvanek

Maven 3.3.9

  • mvn package - Operates inside target subdirectory of your project
  • mvn install - Also copy jar into ${USER_HOME}/.m2/repository

SBT 0.13.11

sbt build operates inside target subdirectory of your project

Let's have following simple build.sbt:
organization := "bt"
name := "bt-sbt"
version := "1.0.0-SNAPSHOT"

scalaVersion := "2.11.7"
sbtVersion := "0.13.11"

resolvers += Resolver.mavenLocal // Also use $HOME/.m2/repository

libraryDependencies += "commons-codec" % "commons-codec" % "1.10" 
libraryDependencies += "org.scalatest" %% "scalatest" % "2.2.6" % "test"

SBT -> Ivy

SBT is using Apache Ivy internally so executing sbt publish-local (which same as sbt publishLocal) will build and store produced jars into ${USER_HOME}/.ivy2/local/bt/bt-sbt_2.11 subdirectories Because Scala applications are bound to Scala version they are compiled with, Ivy module/Maven artifactId/Jar file name will be bt-sbt_2.11 to reflect that.

SBT -> Maven

Since SBT 0.13.0 publishing into Maven local repo is trivial. Execute sbt publish-m2 which is same as sbt publishM2 to build and store compiled binary, source and javadoc jars into ${USER_HOME}/.m2/repository/bt/bt-sbt_2.11/1.0.0-SNAPSHOT

Note that for file cache of remote repository's artifacts SBT uses ${USER_HOME}/.ivy2/cache directory.

Gradle 2.14

Gradle by itself does not have concept of local repository. gradle build - Operates inside build subdirectory of your project

Gradle's build behaviour changes with plugins applied inside build.gradle. Let's assume only java plugin is used in simple build.gradle file
apply plugin: 'java'

group = 'bt'
version = '1.0.0-SNAPSHOT'

task sourceJar(type: Jar) {
    from sourceSets.main.allJava
}

repositories {
    mavenCentral()
}

dependencies {
 compile group: 'commons-codec', name: 'commons-codec', version: '1.10'
 testCompile group: 'junit', name: 'junit', version: '4.12'
}

Name of the directory that contains Gradle project files is taken project name. This is important because project name will become Maven artifactId and it cannot be redefined in build.gradle. You can redefine it using settings.gradle file with line rootProject.name = 'bt-gradle'. There are also options to configure it just for individual plugins but I'd not recommend it because then you need to configure it for every plugin that might be affected (some publish plugin for example)

Gradle -> Maven

For sharing with Maven, probably easiest choice is maven plugin

Apply plugin in your build.gradle apply plugin: 'maven' then execute build via gradle install and your jar will land in ${USER_HOME}/.m2/repository/bt/bt-gradle/1.0.0-SNAPSHOT/bt-gradle-1.0.0-SNAPSHOT.jar

Another Gradle -> Maven option is maven-publish plugin which might be good choice if you also plan to publish artifacts into remote repository as well.

Add into build.gradle

apply plugin: 'maven-publish'

publishing {
    publications {
        mavenJava(MavenPublication) {
            from components.java
        }
    }
}
and execute build using gradle publishToMavenLocal

Gradle -> Ivy

Because SBT is using Ivy distribution instead of Maven, ivy-publish plugin should be weapon of choice for this.
apply plugin: 'ivy-publish'

publishing {
    publications {
        ivyJava(IvyPublication) {
            from components.java
        }
    }
    
    repositories {
        ivy {
            url "${System.properties['user.home']}/.ivy2/local"
        } 
    }
}
then execute buid using gradle publishIvyJavaPublicationToIvyRepository and jar will appear in ${USER_HOME}/.ivy2/local/bt/gradle-built/1.0.0-SNAPSHOT/gradle-built-1.0.0-SNAPSHOT.jar. This is unfortunately wrong as it is Maven layout, not Ivy. You can add layout explicitly
    repositories {
        ivy {
            layout "ivy"
            url "${System.properties['user.home']}/.ivy2/local"
        } 
    }
which is better and will work for simple jar build but it will fail when you attach sources. Yet another reason why I'll never use Gradle unless I'm tortured for a week at least.

Note that for file cache of remote repository's artifacts Gradle uses ${USER_HOME}/.gradle/caches/modules-2/files-2.1 directory

Happy local jar sharing!