Setup continuous integration (CI) with Jenkins / Hudson for TestFlight

Steps taken between Nov 12, 2012 and Mar 7, 2013 to finally get it going:

1. Started with a tutorial.
2. Then went here to download Jenkins for mac.
3. I tried 5 builds so far via Jenkins, all resulted in failure, solved a few configuration issues to resolve problems, and now I'm facing an issue that has to do with keychains. Tried to follow this thread to figure out the appropriate action.
4. Decided to uninstall Jenkins using these instructions
5. Instead used a new installer that circumvents the keychain access issues.
6. I let it checkout my project into its workspace once at:

   /Users/pulkitsinghal/.jenkins/jobs/my_project/workspace

   But afterwards I deleted that workspace and soft-linked it with the actual directory which I already have setup for development:

   cd /Users/pulkitsinghal/.jenkins/jobs/my_project
   ln -s ~/dev/my_project/ ./workspace

   And I completely turned off the git checkout process in the Jenkin's job configuration just to be safe ... although I should have been able to get away by simply ignoring submodules too but why checkout when I don't need to.
7. Then I bumped my head on this:

   fatal error: 'RestKit/RestKit.h' file not found
   #import
   ^
   1 error generated.

8. It should have already been working based on the directions from the RestKit website: Add the following Header Search Paths (including the quotes):

   "$(BUILT_PRODUCTS_DIR)/../../Headers"

   But I suppose for some reason that meant one thing to Xcode and something entirely different to jenkins-xcode-plugin. So through trial and error and watching the logs, I figured out that I needed to add:

   "$(BUILT_PRODUCTS_DIR)/../my_project/Build/Headers

   in order for jenkins-xcode-plugin to pick it up and work with it properly.
9. Ran into another road-block ... posted question to stackoverflow.
10. Then got stuck due to this issue.
    • Agreed to pay $5 bucks to fast-forward the resolution of this bug on Jenkins.
    • While waiting ... I simply removed the following because the jenkins xcode plugin could not process it:
      

Hudson, Tomcat and cacerts

Setting up a secure connection for any application, doesn't really require too much effort in terms of the process involved. It is supposed to be simple. Yet one often gets stuck on the simplest of steps simply because there is no tool or no simple answer other than trial & error.

What if someone asks you to import the server certificate into your application server's truststore. Simple right? Well no! There never seems to be a simple and straightforward answer to the question: "Well, where exactly is the current truststore?"

Where is the cacerts being used by a Tomcat instance that runs Hudson?
Navigating to Hudson > Manage Hudson > System Information will enlighten you to two possibilities:
1) the value for java.home (C:\Program Files\Java\jre6) under the "System Properties" table is a starting point, and
2) the value for JAVA_HOME (C:\Program Files\Java\jdk1.6.X_XX) under the "Environment Variables" table is another.

But not knowing which one exactly can be a bit frustrating because you need to stop and start your Tomcat again and again until you get it right. One could just import the server certificate into the cacerts file under both these locations but then you will still not know which was the right one.

Well if you are starting Tomcat as a Windows Service then the answer is use the file at ${java.home}\lib\security\cacerts. Otherwise if you use the start/stop scripts then the ${JAVA_HOME}\jre\lib\security\cacerts is your best bet.

Hudson: Deploy to container based on a schedule

Challenges:
1) The schedules in Hudson jobs apply to Build Triggers. There isn't any separate place to specify a schedule for Post Build Actions like Deploy war/ear to a container.
2) A job without any build information and only deployment info, will not run the Post Build Actions.
3) Cloning an existing job only for the sake of changing the schedule for deployment will mean throwing off the Hudson build numbers placed inside of your artifacts. Even if you un-check the Archive the artifacts option in the cloned job's configuration it will be meaningless ... the clone's artifacts will continue to be published/installed into your local repository over the original ones by maven itself.
4) The clone might get associated as a downstream job and therefore would end up running whenever your original job had finished running. This would make the custom schedule pointless!

Solutions:

1. Clone the job for which you want to schedule deployments.
2. If you are using a pom.xml which is set up to use child modules then just directly have the cloned configuration reference the pom.xml of the child which actually puts together the war/ear artifact ... instead of the main parent project's pom.xml file.
3. Uncheck the following option from the clone's configuration to decouple the upstream/downstream relationship if there is any:
   Build whenever a SNAPSHOT dependency is built
4. Within the pom.xml file which actually generates your war/ear artifact, place the cargo plugin and its configuration to deploy remotely.
   
   1. It wouldn't hurt to start with the original documentation for reference on how to do this exactly. But it doesn't talk about the most crucial part which is the use of the cargo.tomcat.manager.url property which is covered in this blog. It is a very good example on how to do this.
      
   2. Since you are going to commit the altered pom.xml file you may be worried about the side-effects on your original job that performs the build. Well if you build only runs up to the install phase, you don't have to worry about anything as cargo goals are not part of the maven lifecycle up until that point. Everything about your original build will continue to work as it did.
   3. You can also check This build is parameterized and specify string parameters so that you don't have to go changing your pom.xml everytime that your source location of the war/ear file changes or the tomcat that you want to deploy to changes. Here's a sample (+/-)
      
        <properties>
          <artifactFileLocation>${ARTIFACT_FILE_LOCATION}</artifactFileLocation>
          <remoteTomcat>${REMOTE_TOMCAT_URL}</remoteTomcat>
        </properties>
        ...
            <plugin>
              <groupId>org.codehaus.cargo</groupId>
              <artifactId>cargo-maven2-plugin</artifactId>
              <configuration>
                <!-- Container configuration -->
                <container>
                  <containerId>tomcat6x</containerId>
                  <type>remote</type>
                </container>
                <!-- Configuration to use with the container -->
                <configuration>
                  <type>runtime</type>
                  <properties>
                    <cargo.tomcat.manager.url>${remoteTomcat}/manager</cargo.tomcat.manager.url>
                    <cargo.remote.username>username</cargo.remote.username>
                    <cargo.remote.password>password</cargo.remote.password>
                  </properties>
                </configuration>
                <!-- Deployer configuration -->
                <deployer>
                  <type>remote</type>
                  <deployables>
                    <deployable>
                      <location>${artifactFileLocation}</location>
                      <pingURL>${remoteTomcat}</pingURL>
                    </deployable>
                  </deployables>
                </deployer>
              </configuration>
            </plugin>
   4. Come to think of it, username and password would also benefit from being parameterized.
5. Take advantage of the new configuration in your artifact generating pom.xml file by configuring the following goal in Hudson for the cloned job:
   org.codehaus.cargo:cargo-maven2-plugin:deployer-redeploy
   

Versioning Flex Applications: Displaying Hudson, Maven, SVN build or revision numbers

Hudson sets the values for the following environment variables:

1. BUILD_NUMBER
2. SVN_REVISION

In order to show the build and revision numbers in the UI, these variables could be written out to a properties file via a maven project's pom.xml file using a plugin. And then if one did this early enough in the maven lifecycle, your flex application could be compiled to read the version information as key-value pairs from the properties file and show the info on screen when it runs.

Alas there is no decent maven plugin out there to write environment variables out to a file. I found the following plugins out there but none of them (+/-) did the trick.

1. org.codehaus.mojo:properties-maven-plugin can be found in the following repo You can find proper documentation here. It never printed anything but the Java system level variables so it was useless when it came to tracking the environment variables set by Hudson.
2. org.sonatype.plugins:maven-properties-plugin can be found in the following repo but this isn't the one you want to use. It has only one goal called filter-file and I cannot find any page documenting its usage no matter how hard I try.
3. Even placing the info generated by the mvn help:system command-line invocation seemed like a decent idea. But after configuring the same command to be invoked inside a pom file(+/-) I realized that it spit out some header lines that weren't commented out in a manner suited for a properties file.
   
         <plugin>
           <groupId>org.apache.maven.plugins</groupId>
           <artifactId>maven-help-plugin</artifactId>
           <version>2.1.1</version>
           <executions>
             <execution>
               <phase>generate-resources</phase>
               <goals>
                 <goal>system</goal>
               </goals>
               <configuration>
                 <output>${basedir}/src/locale/app.properties</output>
               </configuration>
             </execution>
           </executions>
         </plugin>

You can resort to usind the ant plugin inside Maven to do a macro-style replace technique. You can read more about it here.

Or the best way is to have a template file (+-) and leverage Maven model like this (+-)

<?xml version="1.0" encoding="UTF-8"?>
<root>
<buildNumber>${buildNumber}</buildNumber>
<svnRevisionNumber>${svnRevision}</svnRevisionNumber>
</root>

<project >
  ...
  <!-- Map the values provided by the Hudson build to local variables -->
  <properties>
    <buildNumber>${BUILD_NUMBER}</buildNumber>
    <svnRevision>${SVN_REVISION}</svnRevision>
  </properties>
  ...
  <build>
    ...
    <!-- Update version.xml file with the versioning data -->
    <resources>
      <resource>
        <targetPath>${project.build.directory}/${project.artifactId}-${project.version}</targetPath>
        <filtering>true</filtering>
        <directory>${basedir}/src/main/resources</directory>
        <includes><include>version.xml</include></includes>
      </resource>
    </resources>
    ...
  <build>
  ...
<project >

Also as a sidenote: The Hudson variables can be placed inside the manifest file if your artifact being built happens to be a war or jar file. Read more on that here.

Links:
Home / Using Flex 4 / Developer tools / Flex compilers / Using mxmlc, the application compiler / Passing Strings
How to configure flexmojos to pass in compile time variables:
gmail thread
official docs

Trobleshooting Hudson and Ant

[catalog-server] $ cmd.exe /C '"ant.bat -file build.xml install && exit %%ERRORLEVEL%%"'
'ant.bat' is not recognized as an internal or external command, operable program or batch file.
Finished: FAILURE

1) The machine which runs Hudson might not have ANT installed on it.
2) If ANT is installed on it then you might not have set the ANT_HOME variable.
3) If ANT_HOME is set, then you might not have set %ANT_HOME%\bin as a part of the system PATH variable.
4) If that is also set, then you may have started your application server (tomcat\hudson) before you performed steps 1-3 so the changes haven't taken effect yet, try restarting it.
5) No good? You may be starting or stopping the tomcat container as a windows service BUT the user under whom the service actually runs may be a different one! This means you need to log that user out and then back-in for him/her to pick up the changes made to the system variables like ANT_HOME or PATH. You can simply reboot your machine if you don't know who that user is or what their password happens to be.
6) No luck? Goto Hudson > Manage Hudson > Configure System > Ant and fill out a value for the ANT_HOME variable and then click Save.
7) Still no luck? Well go to the Job in question and then goto Configure > Build > Invoke Ant > Ant Version and change it from (Default) to the one you configured in the previous step.

If this is not enough try having a look at: http://blog.mxunit.org/2009/05/ci-with-hudson-cf-and-mxunit.html