jenkins - optimizing maven build perfomance -


i have maven web application. taking time build(30-40 mins).i reduce less 10 mins. pom.xml below.

<?xml version="1.0" encoding="utf-8"?> <project xmlns="http://maven.apache.org/pom/4.0.0" xmlns:xsi="http://www.w3.org/2001/xmlschema-instance" xsi:schemalocation="http://maven.apache.org/pom/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">     <modelversion>4.0.0</modelversion>     <parent>         <groupid>se.mysite.etc.settings</groupid>         <artifactid>projects</artifactid>         <version>1.5</version>     </parent>     <groupid>se.myweb.portal</groupid>     <artifactid>myweb-se-main</artifactid>     <name>myweb-se-main</name>     <version>3.1.81-b2_forv-snapshot</version>     <packaging>pom</packaging>     <inceptionyear>2009</inceptionyear>     <properties>         <release.version>${project.version}</release.version>         <acc.version>2.3.42-test-maint-snapshot</acc.version>         <cxf.version>2.5.3</cxf.version>         <spring.version>3.0.3.release</spring.version>         <spring.security.version>2.0.4</spring.security.version>         <spring.webflow.version>2.1.1.release</spring.webflow.version>         <commonportal.version>1.9.9-spring3</commonportal.version>                   <junit.version>4.5</junit.version>         <java.source.version>1.6</java.source.version>         <cobertura.maxmem>1024m</cobertura.maxmem>         <project.build.sourceencoding>utf-8</project.build.sourceencoding>     </properties>     <scm>         <connection>scm:hg:http://test1/myweb</connection>         <developerconnection>scm:hg:http://test1/myweb</developerconnection>         <tag/>         <url>http://test1/myweb</url>     </scm>     <profiles>         <profile>             <id>dev</id>             <dependencies>                 <!--some dependencies-->             </dependencies>             <modules>                 <module>../project1</module>                 <module>../project2</module>                 <module>../project3</module>                 <module>../project4</module>             </modules>         </profile>         <profile>             <id>dist</id>             <activation>                 <activebydefault>true</activebydefault>             </activation>             <dependencies>                 <!--some dependencies-->             </dependencies>             <modules>                 <module>../project1</module>                 <module>../project2</module>                 <module>../project3</module>                 <module>../project4</module>                 <module>../project5</module>                 <module>../project6</module>             </modules>         </profile>         <profile>             <id>backend</id>             <dependencies>                 <!--some dependencies-->             <modules>                 <module>../project3</module>                 <module>../project1</module>                 <module>../project2</module>                 <module>../project4</module>                 <module>../project9</module>             </modules>         </profile>         <profile>             <id>frontend</id>             <!--some dependencies-->             <modules>                 <module>../project10</module>                 <module>../project11</module>             </modules>             <build>                 <plugins>                     <plugin>                         <groupid>org.apache.maven.plugins</groupid>                         <artifactid>maven-surefire-plugin</artifactid>                         <version>2.12.4</version>                         <configuration>                             <forkmode>always</forkmode>                             <argline>-xms512m -xmx2048m -xx:maxpermsize=2048m</argline>                         </configuration>                     </plugin>                 </plugins>             </build>         </profile>         <profile>             <id>webtest</id>             <build>                 <finalname>myweb-web-test</finalname>                 <plugins>                     <plugin>                         <groupid>org.apache.maven.plugins</groupid>                         <artifactid>maven-surefire-plugin</artifactid>                         <configuration>                             <argline>-xx:maxpermsize=512m</argline>                         </configuration>                     </plugin>                 </plugins>             </build>             <!--some dependencies-->             <modules>                 <module>../myweb-web-test</module>             </modules>         </profile>         <profile>             <id>redeploy_web_app</id>             <activation>                 <property>                     <name>redeploywebapp</name>                 </property>             </activation>             <properties>                 <user.tomcat.home>${env.dest_dir}</user.tomcat.home>             </properties>             <build>                 <pluginmanagement>                     <plugins>                         <plugin>                             <groupid>org.apache.maven.plugins</groupid>                             <artifactid>maven-antrun-plugin</artifactid>                             <version>1.6</version>                             <dependencies>                                 <!--some dependencies-->                             </dependencies>                             <executions>                                 <execution>                                     <id>delete_project_artifact</id>                                     <phase>prepare-package</phase>                                     <configuration>                                         <tasks>                                             <taskdef classpathref="maven.plugin.classpath" resource="net/sf/antcontrib/antcontrib.properties"/>                                             <if>                                                 <equals arg1="${project.packaging}" arg2="war"/>                                                 <then>                                                     <echo message="removing ${project.artifactid}.${project.packaging} ${user.tomcat.home}/webapps"/>                                                     <delete dir="${user.tomcat.home}/webapps/${project.artifactid}"/>                                                     <delete file="${user.tomcat.home}/webapps/${project.artifactid}.${project.packaging}"/>                                                 </then>                                             </if>                                         </tasks>                                     </configuration>                                     <goals>                                         <goal>run</goal>                                     </goals>                                 </execution>                                 <execution>                                     <id>copy_project_artifact</id>                                     <phase>package</phase>                                     <configuration>                                         <tasks>                                             <taskdef classpathref="maven.plugin.classpath" resource="net/sf/antcontrib/antcontrib.properties"/>                                             <if>                                                 <equals arg1="${project.packaging}" arg2="war"/>                                                 <then>                                                     <echo message="copying ${project.build.finalname}.${project.packaging}"/>                                                     <copy file="${project.build.directory}/${project.build.finalname}.${project.packaging}" overwrite="true" tofile="${user.tomcat.home}/webapps/${project.artifactid}.${project.packaging}"/>                                                 </then>                                             </if>                                         </tasks>                                     </configuration>                                     <goals>                                         <goal>run</goal>                                     </goals>                                 </execution>                             </executions>                         </plugin>                     </plugins>                 </pluginmanagement>                 <plugins>                     <plugin>                         <groupid>org.apache.maven.plugins</groupid>                         <artifactid>maven-antrun-plugin</artifactid>                     </plugin>                 </plugins>             </build>         </profile>     </profiles>     <!-- general dependencies -->       <build>         <plugins>             <plugin>                 <groupid>org.codehaus.mojo</groupid>                 <artifactid>cobertura-maven-plugin</artifactid>                 <version>2.2</version>                 <configuration>                     <instrumentation>                         <excludes>                             <exclude>se.mysite/**/test*.class</exclude>                         </excludes>                     </instrumentation>                 </configuration>                 <executions>                     <execution>                         <goals>                             <goal>clean</goal>                         </goals>                     </execution>                 </executions>             </plugin>             <plugin>                 <groupid>org.apache.maven.plugins</groupid>                 <artifactid>maven-compiler-plugin</artifactid>                 <configuration>                     <!-- todo: use utf-8 here or not??? -->                     <encoding>utf-8</encoding>                     <!-- encoding>iso-8859-1</encoding-->                     <source>${java.source.version}</source>                     <target>${java.source.version}</target>                     <showdeprecation>true</showdeprecation>                 </configuration>             </plugin>             <plugin>                 <groupid>org.apache.maven.plugins</groupid>                 <artifactid>maven-release-plugin</artifactid>                 <version>2.1</version>                 <configuration>                     <scmcommentprefix>ii    release:</scmcommentprefix>                 </configuration>             </plugin>         </plugins>     </build>     <organization>         <name>mysite ab</name>         <url>http://www.mysite.se</url>     </organization> </project> 

my compile command below.

mvn -u -dartifactstarget=$dest_dir/lib/tpp -p dev -dmaven.test.failure.ignore=true -dtesting.teststoexclude=**/test*.java -f pom.xml $* -dredeploywebapp install 

it taking 30 mins build.

i using maven 3.0.4. explored using following options can optimize build time.

  1. using maven parallel processing
  2. using plexus compiler plugin
  3. using maven power shell

please let me know if need other changes pom.xml or other techniques optimize performance.

optimizing maven build of java project comes down 3 factors:

  1. compilation , tests
  2. complexity of maven build
  3. optimizing maven (binary) performance

compilation , tests

if, example, have gwt in application , you're compiling 20 different permutations problem if you're doing whole time.

there ways around maybe doing full build when need to, , otherwise limiting build using draftcompile or optimizationlevel options.

the same thing applies tests: if have hundreds of tests, length of time each test takes starts become significant. common ways increase performance in tests are:

  1. don't use thread.sleep(..), use mutex's , locks.
  2. share configuration (persistence units, mocks) across multiple tests instead of doing setup per test.
  3. make unit test instead of integration test, unless need to. many integration tests bad code smell, in case.

complexity of maven build

if have 10's or hundreds of modules, complex dependency tree, maven need time calculate dependency tree. use `dependency:analyze' figure out whether or not there unused dependencies , remove them.

also, number of plugins used in each module's build increase build time.

optimizing maven (binary) performance

i use following options:

export maven_opts="-dmaven.wagon.provider.http=httpclient -dmaven.artifact.threads=12 -dhttp.tcp.nodelay=false -xmx2048m -xss256m -xx:+cmsclassunloadingenabled -xx:-usegcoverheadlimit -xx:maxpermsize=256m -t2c"

  • -dmaven.wagon.provider.http=httpclient: changes http connection mechanism, theory apache httpclient library faster , better default java httpurlconnection class
  • -dmaven.artifact.threads=12: number of threads used download artifacts.
  • -dhttp.tcp.nodelay=false: turns off nagle's algorithm increasing performance (but bandwidth used). wagon property.
  • -t2c: expiremental build introduced in maven 3: number of threads per build. see parallel builds in maven 3

please feel edit post if have other tips!


Comments