Why doesn't Groovy's power assert work in a Jenkins job

Power assertions work in the /script, but do not work in a Jenkinsfile driven job. Why? Is there a way to get it to work?

In Jenkinsfile job:

assert 1 == 2  at org.codehaus.groovy.runtime.InvokerHelper.assertFailed(InvokerHelper.java:404) ... wall of stack trace 

In /script window

Assertion failed:   assert 1 == 2          |          false 

In this trivial example, it is easy to figure out what is going on. In practice, one or both of the operands of "==" will be a variable. In the /script version, it will display the values (see the link above). In the Jenkinsfile console log you just get the assert statement as is with no hints.

Added after @daggett question:

node ()   {   stage('assert')     {     try        {       two = 2       assert 1==two        }     catch(Throwable t)       {       println t       error "assert failed"       }     }   } 

output:

Assertion failed:   assert 1==two 

2 Answers

Answers 1

Not possible as far as I can see. Jenkins pipeline groovy DSL uses custom CPS interpreter during interpretation of groovy. This means that it bypasses/overrides a lot of the standard implementation of groovy, and thus also the assert implementation. The assert implementation for Jenkins Pipeline CPS can be found here, while the real groovy implementation uses this class during the assert evaluation in order to record values and print proper exception.

In order to get similar behaviour in Jenkins pipeline CPS one would either need to refactor the groovy code base and Jenkins CPS code or duplicate a lot of the functionality in the AssertionWriter class linked above.

Answers 2

This is an example of a hack that sort of works:

def ASSERT_EQ (def arg1, def arg2)   {   if (arg1 != arg2)     {     println ('arg1 = ' + arg1 + " " + arg1.getClass())     println ('arg2 = ' + arg2 + " " + arg2.getClass())     }   assert arg1 == arg2   } 

It is not nearly as robust as the normal groovy power assert.

Read More

Am I able to bypass Jenkins tsl errors and continue with the build?

My question is related to Jenkins and linting errors.

When I run npm run build locally on my machine, I am successfully able to build my code base, without any errors.

However, when I run npm run build on Jenkins, the job FAILS due to linting errors, usually related to case sensitive issues, and varialbes of type any.

Here is an example:

ERROR in {jenkins-path-to-tsx-file} [tsl] ERROR in {jenkins-path-to-tsx-file}(3,35) TS2307: Cannot find module './ResetPasswordForm'.

And

ERROR in {jenkins-path-to-tsx-file} [tsl] ERROR in {jenkins-path-to-tsx-file}(38,29) TS7006: Parameter 'res' implicitly has an 'any' type.

I understand how to fix these errors in my application code to remove these errors.

My question is, am I able to stop the build failing due to these errors, and display them as warnings, or continue building the Jenkins job despite the errors above?

0 Answers

Read More

Retrieve parameter from a Jenkins REST query

The following REST query will return parameters of the last successful build of a job: https://localhost/job/test1/lastSuccessfulBuild/api/json
I'd be interested to retrieve one of the parameters of this build, the BUILD_VERSION:

{      "_class": "org.jenkinsci.plugins.workflow.job.WorkflowRun",     "actions": [         {             "_class": "hudson.model.CauseAction",             "causes": [                 {                     "_class": "hudson.model.Cause$UpstreamCause",                     "shortDescription": "Started by upstream project \"continuous-testing-pipeline-for-nightly\" build number 114",                     "upstreamBuild": 114,                     "upstreamProject": "continuous-testing-pipeline-for-nightly",                     "upstreamUrl": "job/continuous-testing-pipeline-for-nightly/"                 }             ]         },         { },         {             "_class": "hudson.model.ParametersAction",             "parameters": [                  {                     "_class": "hudson.model.StringParameterValue",                     "name": "BUILD_VERSION",                     "value": "1.1.15" 

Is there a way to retrieve the BUILD_VERSION (1.1.15) directly using the REST Api or do I have to parse manually the json string ? Thanks

3 Answers

Answers 1

Short answer: No.

Easiest way to programmatically access any attribute exposed via the JSON API is to take the JSON from one of Jenkins supported JSON APIs (in your case: https://localhost/job/<jobname>/lastSuccessfulBuild/api/json)

1. Copy the resultant JSON into http://json2csharp.com
2. Generate the corresponding C# code. Don't forget to create a meaningful name for top level class.
3. Call RestAPI programmatically from C# using RestSharp.
4. Deserialise the json to the C# class you defined in 2 above.

Wammo, you have access to the entire object tree and all its values.

I used this approach to write an MVC5 ASP.NET site I called "BuildDashboard" to provide all the information a development team could want and answered every question they had.

Answers 2

Yeah you can get the value,But it will only work for XML API :(
The JSON API will return a simplified json object using Tree :)

So Jenkins provides you with api (XML,JSON,PYTHON) from which you can read the Jenkins related data of any project. Documentation in detail is provide in https://localhost/job/test1/lastSuccessfulBuild/api

In that it clearly states that

1. XML API - Use XPath to control the fragment you want.For example, ../api/xml?xpath=//[0]

2. JSON API - Use tree

3. Python API - Use st.literal_eval(urllib.urlopen("...").read())

All the above can be used to get a specific fragment/piece from the entire messy data that you get from the API.

In your case, we will use tree for obvious reasons :)

Syntax : tree=keyname[field1,field2,subkeyname[subfield1]]

In order to retrieve BUILD_VERSION i.e. value

//jenkins/job/myjob/../api/json?tree=lastSuccessfulBuild[parameters[value]] 

The above should get you what you want, but a bit of trail and error is required :)

You can also refer here for a better understanding of how to use Tree in JSON API https://www.cloudbees.com/blog/taming-jenkins-json-api-depth-and-tree

Hope it helps :)

Answers 3

Here is an example with a public jenkins instance and one of its builds in order to get "candidate_revision" parameter for "lastSuccessfulBuild" build:

https://jenkins.qa.ubuntu.com/view/All/job/account-plugins-vivid-i386-ci/lastSuccessfulBuild/parameters/

https://jenkins.qa.ubuntu.com/view/All/job/account-plugins-vivid-i386-ci/lastSuccessfulBuild/api/xml?xpath=/freeStyleBuild/action/parameter[name=%22candidate_revision%22]/value

Read More

Karma Chrome Headless not working on Jenkins

When I run the below setup with Docker locally on my mac everything works fine.

But same setup does not work on Jenkins running on Ubuntu 16.04

ChromiumHeadless have not captured in 60000 ms, killing.

Following error log is from Jenkins console:

25 05 2018 06:35:09.076:INFO [karma]: Karma v2.0.2 server started at http://0.0.0.0:9222/ 25 05 2018 06:35:09.079:INFO [launcher]: Launching browser Chromium_no_sandbox with unlimited concurrency 25 05 2018 06:35:09.090:INFO [launcher]: Starting browser ChromiumHeadless 25 05 2018 06:36:09.128:WARN [launcher]: ChromiumHeadless have not captured in 60000 ms, killing. 25 05 2018 06:36:09.139:INFO [launcher]: Trying to start ChromiumHeadless again (1/2). 25 05 2018 06:37:09.140:WARN [launcher]: ChromiumHeadless have not captured in 60000 ms, killing. 25 05 2018 06:37:09.147:INFO [launcher]: Trying to start ChromiumHeadless again (2/2). 

Package.json ... "testProd": "./node_modules/karma/bin/karma start karma.conf-prod.js --single-run",

Dockerfile

FROM zenika/alpine-node:latest LABEL name="product-web"  # Update apk repositories RUN echo "http://dl-2.alpinelinux.org/alpine/edge/main" > /etc/apk/repositories RUN echo "http://dl-2.alpinelinux.org/alpine/edge/community" >> /etc/apk/repositories RUN echo "http://dl-2.alpinelinux.org/alpine/edge/testing" >> /etc/apk/repositories  # Install chromium RUN apk -U --no-cache \     --allow-untrusted add \     zlib-dev \     chromium \     xvfb \     wait4ports \     xorg-server \     dbus \     ttf-freefont \     mesa-dri-swrast \     grep \     udev \     && apk del --purge --force linux-headers binutils-gold gnupg zlib-dev libc-utils \     && rm -rf /var/lib/apt/lists/* \     /var/cache/apk/* \     /usr/share/man \     /tmp/* \     /usr/lib/node_modules/npm/man \     /usr/lib/node_modules/npm/doc \     /usr/lib/node_modules/npm/html \     /usr/lib/node_modules/npm/scripts  WORKDIR /home/dev/code COPY . .  #RUN rm -rf node_modules && npm cache clear --force  ENV CHROME_BIN=/usr/bin/chromium-browser ENV CHROME_PATH=/usr/lib/chromium/  RUN npm install RUN npm run testProd && npm run buildProd 

karma.conf-prod.js

const path = require('path'); module.exports = function(config) {     config.set({         basePath: '',         browsers: ['ChromeHeadlessNoSandbox'],     customLaunchers: {         ChromeHeadlessNoSandbox: {             base: 'ChromeHeadless',             flags: [                 '--no-sandbox',                 '--user-data-dir=/tmp/chrome-test-profile',                 '--disable-web-security'             ]         }     },         frameworks: ['mocha', 'chai'],         captureConsole: true,         files: [             'node_modules/babel-polyfill/dist/polyfill.js',             'test/root.js'         ],         preprocessors: {             'src/index.js': ['webpack', 'sourcemap'],             'test/root.js': ['webpack']         },         webpack: {             devtool: 'inline-source-map',             module: {                 loaders: [                     {                         test: /\.js$/,                         loader: 'babel-loader',                         exclude: path.resolve(__dirname, 'node_modules'),                         query: {                             plugins: ['transform-decorators-legacy', 'transform-regenerator'],                             presets: ['env', 'stage-1', 'react']                         }                     },                     {                         test: /\.json$/,                         loader: 'json-loader',                     },                 ]             },             externals: {                 'react/addons': true,                 'react/lib/ExecutionEnvironment': true,                 'react/lib/ReactContext': true             }         },         webpackServer: {             noInfo: true         },         reporters: ['spec'],         port: 9222,         logLevel: config.LOG_INFO     }); }; 

I even tried with logLevel: config.LOG_DEBUG but did not show anything missing or unusual.

2 Answers

Answers 1

Based on issue Karma 1.6 breaks Headless support for Chrome created on github, it is related to the slower machine and happens, because it took > 60 seconds before test bundle was parsed and executed by Chrome and therefore test run was started and communicated back to Karma server. Reasons why it may take long vary.

There are 2 ways to handle timeout:

Investigate why your test bundle loads >60 seconds and make sure it loads faster.

1. Increase browserNoActivityTimeout to highter value, so test bundle has enough time to load.
2. This particular appearance of timeout does not seem to be a Karma issue, but rather problem in the project or misconfiguration.

Based on the Derek's comment

There was a connection that was disconnecting too soon.

He found that in /static/karma.js, when the socket was created, there was a timeout value that is hardcoded to 2 seconds (see below). He just added another 0 to make it 20 seconds and the connection stayed open long enough for the server to respond to the initial request. karma/client/main.js

Lines 14 to 20 in e79463b

var socket = io(location.host, {     reconnectionDelay: 500,     reconnectionDelayMax: Infinity,     timeout: 2000,     path: KARMA_PROXY_PATH + KARMA_URL_ROOT.substr(1) + 'socket.io',     'sync disconnect on unload': true   })  

The next problem he faced was that Karma thought there was no activity even though there was traffic going back and forth on the socket. To fix that he just added browserNoActivityTimeout: 60000 to the Karma configuration.

You need to change the timeout configuration more then that is in the configuration file.

Answers 2

This is not a problem with Jenkins or from Docker according to my current understanding of the situation.

Read More

How to deploy node app to remote host from Jenkins?

This is Jenkins file in the root of node app directory:

pipeline {     agent any     triggers {         pollSCM('* * * * *')     }     stages {         stage("deploy") {             steps {             sh "scp"             }         }     }  } 

I configured Jenkins to connect to remote gitlab node proj repo to checkout node project along with Jenkinsfile and run the project's Jenkinsfile. This part works fine but what to do now to perform (note that Jenkins server and the server on which node js is running as well as gitlab repo are all remote to each other):

run these commands on remote server on which node app is running  cd ~/mynodeproj   pm2 stop mynodeproj   copy project source files from Jenkins server to remote server where  node app is running   npm install   export NODE_ENV=production   pm2 start mynodeproj 

How to achieve this?

Do I need to setup private/public keypair on server running jenkins so that jenkins server can do scp to copy file to remote server running node app?

3 Answers

Answers 1

I'm suggest we can use Rocketeer for this case.

Here is Rocketeer tree on my Jenkins Server for NodeJS App

$ tree .rocketeer/ .rocketeer/ ├── config.php ├── hooks.php ├── logs │   ├── develop--20170613.log │   ├── develop--20170614.log │   ├── develop--20170616.log │   ├── staging--20180323.log │   ├── staging--20180324.log │   ├── staging--20180326.log │   ├── production--20180223.log │   ├── production--20180226.log │   ├── production--20180227.log │   ├── production--20180227.log │   └── custom-environment--20180328.log ├── paths.php ├── remote.php ├── scm.php ├── stages.php └── strategies.php 

• You can manage remote environment of NodeJS App: Develop, Staging, Production (at config.php file)
• It's will pull newest source code on your Gitlab and keep releases version like Capistrano (at remote.php file)
• It's can run your pm2 command line after deployed newest source code (at hooks.php file)
• It's already help run npm install NodeJS packages.

---

So here is my Jenkins Job setting:

Source Code Management

Build Triggers

Build

#!/bin/bash -el cd $JENKINS_HOME/app-deploy/app-socketio rocketeer deploy --on="develop" 

develop mean connect to Develop Remote Server (at .rocketeer\config.php file)

'connections'      => [     'develop' => [         'host'      => '35.xx.xx.xx',         'username'  => 'ec2-user',         'password'  => '',         'key'       => '/var/lib/jenkins/.ssh/foo.pem',         'keyphrase' => '',         'agent'     => '',         'db_role'   => true,     ],     'staging' => [         'host'      => '34.xx.xx.xx',         'username'  => 'ec2-user',         'password'  => '',         'key'       => '/var/lib/jenkins/.ssh/bar.pem',         'keyphrase' => '',         'agent'     => '',         'db_role'   => true,      ],     'production' => [         'host'      => '18.xx.xx.xx:63612',         'username'  => 'ec2-user',         'password'  => '',         'key'       => '/var/lib/jenkins/.ssh/woot.pem',         'keyphrase' => '',         'agent'     => '',         'db_role'   => true,     ],     'custom-environment' => [         'host'      => '13.xx.xx.xx:63612',         'username'  => 'ec2-user',         'password'  => '',         'key'       => '/var/lib/jenkins/.ssh/test.pem',         'keyphrase' => '',         'agent'     => '',         'db_role'   => true,     ], ], 

And run pm2 command line configure at hooks.php file

'after'  => [     'setup'   => [],     'deploy'  => [             "pm2 delete mynodeproj", //Delete old pm2 task             "pm2 start src/mynodeproj.js", //Start new mynodeproj     ],     'cleanup' => [], ], 

Hope this can help you!!

Answers 2

What to do now to perform (note that Jenkins server and the server on which node js is running as well as gitlab repo are all remote to each other):

Those command should be in a file part of your Git repo, meaning checked out in the workspace used by the pipeline.

From there, you can scp that script to the server:

sh 'scp -r script user@server:/path/server' 

And execute it through ssh, provided the private/public key of the remote user is deployed on the agent (or passed through the ssh-agent plugin):

sh 'ssh user@server /path/server/script' 

The trick though, as mentioned here, is that the PATH of an ssh session is fairly limited, and might not include node, if node is not installed in a standard path like /usr/bin.
So you might want to modify your script in order to use absolute paths for your executable (like /absolute/path/to/node)

Answers 3

Your question is how to deploy your application to remote host from jenkins

easiest solution

1.using Jenkins File

If your using a pipeline project, then you will notice a Pipeline Syntax option, just select ssh Publisher:send artifacts over SSH (you'llneed the plugin Publish over SSH plugin)

Add an SSH server

• Source Files :mention the files you wish to transfer (**/*.war takes the WAR file only)
• Remove prefix : target(WAR files will generally be located in a target folder so i'm telling I don't want the target folder to be copied only my WAR)
• Remote Directory : Leave it blank( since your taking it from your project)
• Exec command : Commands to be executed after copying the files( like starting tomcat or unzip any properties files and place at a particular location - in your case the commands to start node from your server)

Finally just click Generate Pipeline Script and BOOM, you get the script (it internally does an scp) just copy and paste it into your Pipeline file and your done :)

)

2.Publish over SSH

You can do the same thing( follow the above steps adding details only) in a normal Jenkins project in the Post Build Action If you don't have a Pipeline Project (helps if you don't have/know a Pipeline Project)

Do I need to setup private/public keypair on server running jenkins so that jenkins server can do scp to copy file to remote server running node app?

Yes its required(Recommended) you can add the deatils as shown below in Jenkins under Manage Jenkins, Configure Jenkins Under Publish over SSH and you can test your connection .

Hope this was useful :)

Read More

What's the best way to save test data to a database from jenkins/hudson?

I'm trying to make a dashboard for test results (dynamically filtered by os/hardware/ect tags) from jenkins.

What's the best way for jenkins to save test results from an xml file to a central db(preferably postgres)?

The sql plugins seem to be designed to be used by other plugins.

Should I just call java sql apis from jenkins.groovy script?

3 Answers

Answers 1

Several possibilities:

Use an existing java sql library with Grab

There are several world class java sql libs out there (https://commons.apache.org/proper/commons-dbutils/, https://github.com/jOOQ/jOOQ) which you can use in your pipeline in conjunction with Grab. Personally I love jooq. Then you write your specific code in groovy.

@Grab(group='org.jooq', module='jooq', version='3.10.5') import ... 

Use an existing java library and put it on the classpath

Like the solution above. But you don't import the library with Grab. Instead starting jenkins from the war directly, you maintain your own jetty installation (https://wiki.jenkins.io/display/JENKINS/Jetty). This provides you the freedom to add additional libraries to the classpath, which can be used then in the pipelines.

Write a java library and use it with Grab or classpath

Instead of using a general purpose library and code your specific requirements in groovy. You create a library, which provides an API for your needs. You import it with Grab or you add it to the classpath of the application server. In your groovy scripts you just have to call the specific API and pass the parameters.

Write your own plugin

You can write a plugin for your purpose which is not so difficult https://jenkins.io/doc/developer/plugin-development/pipeline-integration/. The plugin would take the file path as parameter. The contract will be the structure (xml schema) of the xml file. You could also add parameters to control the import.

Personal opinion:

I would go with "Write a java library and use it with Grab or classpath":

• good testing possibilities on unit level, because of simplicity
• no dependency of the implementation of the specific requirements to jenkins, means can also be used without jenkins
• easy to distribute, can be deployed to any maven compliant repository and fetched from there @GrabResolver(name='custom', root='http://customserver/repo', m2Compatible='true')

Answers 2

What we are using at the company (and at some other projects I've involved) is the FluentD. You can find a well-maintained Jenkins plugin here or here.

The main features are: sending any valid json, which can be analyzed later using your own tool, something like ELK stack (you can find a lot HOWTO articles), or some other stacks.

Why FluentD? I guess it clearly states on the author's github page: " Our choice was to use Fluentd as looks mature enough to handle a lot of data and it supports a lot of destination endpoints (DBs, file, http, etc)." The data outputs could be anythong you wantL Hadoop, Mongo, S3, AWS/Azure services, or any existing database (https://www.fluentd.org/dataoutputs), and a lot of visualization.

I assume you can ask "why not push it directly to the database?". The answer is - you will need to push it in a specific format and the data won't be easily formatted to another one. I.e. You've built your own tool, but later realized you can use Kibana instead, so you'll need to migrate over. If you start using the FluentD, the only thing you should think about is how to format the json, which is much more trivial, than designing a database. Also, FluentD is highly scalable and supports robust failovers.

One more potential issue is converting XML, you've mentioned you have as test output, to json. First of all, a lot of test runners support json formatting. Secondly, there's a common method of generating json out of xml and I would suggest something like "xml2json" tool, which does a pretty good job:

XML JSON

<e/>                             "e": null <e>text</e>                      "e": "text" <e name="value" />               "e": { "@name": "value" } <e name="value">text</e>         "e": { "@name": "value", "#text": "text" } <e> <a>text</a ><b>text</b> </e> "e": { "a": "text", "b": "text" } <e> <a>text</a> <a>text</a> </e> "e": { "a": ["text", "text"] } <e> text <a>text</a> </e>        "e": { "#text": "text", "a": "text" } 

Answers 3

Also you can do it via database REST API. Just call sh in your pipeline and put an artifact in a database.

sh("curl http://database:8080/api --verbose -X PUT \                   -H "Content-Type: application/xml"          ... ) 

Read More

Jenkins multibranch pipeline build number automatically reset

While working with jenkins multibranch pipeline, the build number automatically (silently) reset itself back to 1 and dropped all other build history up until that point.

In the syslog I see no evidence of updateNextBuildNumber being run to set it back to 1.

The only thing different about the specific branch of the multipipe that got reset back to one was that I had manually set the build number high (1300) when I created the pipeline due to version tagging collisions from an old build.

Has anyone experienced this bug? If so any idea what happened? This cannot happen since it will overwrite production build numbers.

EDIT: I am seeing errors about unauthorized people attempting to make REST API calls from bulgaria and failure to register webhooks on the multibranch pipeline.

EDIT2: Here is a deduplicated document of all errors and warnings encountered in the 3 day window the reset could have happned. The full tracebacks have been removed for brevity.

Dec 10, 2017 9:33:31 AM hudson.init.impl.InstallUncaughtExceptionHandler$DefaultUncaughtExceptionHandler uncaughtException SEVERE: A thread (TCP agent connection handler #41 with /5.188.10.105:41480/70225) died unexpectedly due to an uncaught exception, this may leave your Jenkins in a bad way and is usually indicative of a bug in the code. java.lang.SecurityException: Rejected: sun.reflect.annotation.AnnotationInvocationHandler  Dec 10, 2017 1:23:47 PM hudson.remoting.SynchronousCommandTransport$ReaderThread run SEVERE: I/O error in channel HTTP full-duplex channel 093ea142-17d1-4aa3-b57c-98899dfb7964 hudson.remoting.DiagnosedStreamCorruptionException Read back: 0xac 0xed 0x00 0x05 'sr' 0x00 '/org.apache.commons.collections.map.ReferenceMap' 0x15 0x94 0xca 0x03 0x98 'I' 0x08 0xd7 0x03 0x00 0x00 'xpw' 0x11 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x01 0x00 '?@' 0x00 0x00 0x00 0x00 0x00 0x10 'sr' 0x00 '(java.util.concurrent.CopyOnWriteArraySetK' 0xbd 0xd0 0x92 0x90 0x15 'i' 0xd7 0x02 0x00 0x01 'L' 0x00 0x02 'alt' 0x00 '+Ljava/util/concurrent/CopyOnWriteArrayList;xpsr' 0x00 ')java.util.concurrent.CopyOnWriteArrayListx]' 0x9f 0xd5 'F' 0xab 0x90 0xc3 0x03 0x00 0x00 'xpw' 0x04 0x00 0x00 0x00 0x02 'sr' 0x00 '*java.util.concurrent.ConcurrentSkipListSet' 0xdd 0x98 'Py' 0xbd 0xcf 0xf1 '[' 0x02 0x00 0x01 'L' 0x00 0x01 'mt' 0x00 '-Ljava/util/concurrent/ConcurrentNavigableMap;xpsr' 0x00 '*java.util.concurrent.ConcurrentSkipListMap' 0x88 'Fu' 0xae 0x06 0x11 'F' 0xa7 0x03 0x00 0x01 'L' 0x00 0x0a 'comparatort' 0x00 0x16 'Ljava/util/Comparator;xppsr' 0x00 0x1a 'java.security.SignedObject' 0x09 0xff 0xbd 'h*<' 0xd5 0xff 0x02 0x00 0x03 '[' 0x00 0x07 'contentt' 0x00 0x02 '[B[' 0x00 0x09 'signatureq' 0x00 '~' 0x00 0x0e 'L' 0x00 0x0c 'thealgorithmt' 0x00 0x12 'Ljava/lang/String;' Read ahead: 'xpur' 0x00 0x02 '[B' 0xac 0xf3 0x17 0xf8 0x06 0x08 'T' 0xe0 0x02 0x00 0x00 'xp' 0x00 0x00 0x05 0x09 0xac 0xed 0x00 0x05 'sr' 0x00 0x11 'java.util.HashSet' 0xba 'D' 0x85 0x95 0x96 0xb8 0xb7 '4' 0x03 0x00 0x00 'xpw' 0x0c 0x00 0x00 0x00 0x02 '?@' 0x00 0x00 0x00 0x00 0x00 0x01 'sr' 0x00 '4org.apache.commons.collections.keyvalue.TiedMapEntry' 0x8a 0xad 0xd2 0x9b '9' 0xc1 0x1f 0xdb 0x02 0x00 0x02 'L' 0x00 0x03 'keyt' 0x00 0x12 'Ljava/lang/Object;L' 0x00 0x03 'mapt' 0x00 0x0f 'Ljava/util/Map;xpt' 0x00 0x03 'foosr' 0x00 '*org.apache.commons.collections.map.LazyMapn' 0xe5 0x94 0x82 0x9e 'y' 0x10 0x94 0x03 0x00 0x01 'L' 0x00 0x07 'factoryt' 0x00 ',Lorg/apache/commons/collections/Transformer;xpsr' 0x00 ':org.apache.commons.collections.functors.ChainedTransformer0' 0xc7 0x97 0xec '(z' 0x97 0x04 0x02 0x00 0x01 '[' 0x00 0x0d 'iTransformerst' 0x00 '-[Lorg/apache/commons/collections/Transformer;xpur' 0x00 '-[Lorg.apache.commons.collections.Transformer;' 0xbd 'V*' 0xf1 0xd8 '4' 0x18 0x99 0x02 0x00 0x00 'xp' 0x00 0x00 0x00 0x05 'sr' 0x00 ';org.apache.commons.collections.functors.ConstantTransformerXv' 0x90 0x11 'A' 0x02 0xb1 0x94 0x02 0x00 0x01 'L' 0x00 0x09 'iConstantq' 0x00 '~' 0x00 0x03 'xpvr' 0x00 0x11 'java.lang.Runtime' 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 'xpsr' 0x00 ':org.apache.commons.collections.functors.InvokerTransformer' 0x87 0xe8 0xff 'k{|' 0xce '8' 0x02 0x00 0x03 '[' 0x00 0x05 'iArgst' 0x00 0x13 '[Ljava/lang/Object;L' 0x00 0x0b 'iMethodNamet' 0x00 0x12 'Ljava/lang/String;[' 0x00 0x0b 'iParamTypest' 0x00 0x12 '[Ljava/lang/Class;xpur' 0x00 0x13 '[Ljava.lang.Object;' 0x90 0xce 'X' 0x9f 0x10 's)l' 0x02 0x00 0x00 'xp' 0x00 0x00 0x00 0x02 't' 0x00 0x0a 'getRuntimeur' 0x00 0x12 '[Ljava.lang.Class;' 0xab 0x16 0xd7 0xae 0xcb 0xcd 'Z' 0x99 0x02 0x00 0x00 'xp' 0x00 0x00 0x00 0x00 't' 0x00 0x09 'getMethoduq' 0x00 '~' 0x00 0x1b 0x00 0x00 0x00 0x02 'vr' 0x00 0x10 'java.lang.String' 0xa0 0xf0 0xa4 '8z;' 0xb3 'B' 0x02 0x00 0x00 'xpvq' 0x00 '~' 0x00 0x1b 'sq' 0x00 '~' 0x00 0x13 'uq' 0x00 '~' 0x00 0x18 0x00 0x00 0x00 0x02 'puq' 0x00 '~' 0x00 0x18 0x00 0x00 0x00 0x00 't' 0x00 0x06 'invokeuq' 0x00 '~' 0x00 0x1b 0x00 0x00 0x00 0x02 'vr' 0x00 0x10 'java.lang.Object' 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 'xpvq' 0x00 '~' 0x00 0x18 'sq' 0x00 '~' 0x00 0x13 'ur' 0x00 0x13 '[Ljava.lang.String;' 0xad 0xd2 'V' 0xe7 0xe9 0x1d '{G' 0x02 0x00 0x00 'xp' 0x00 0x00 0x00 0x01 't' 0x00 0x10 'rm -f /tmp/.64/*t' 0x00 0x04 'execuq' 0x00 '~' 0x00 0x1b 0x00 0x00 0x00 0x01 'q' 0x00 '~' 0x00 ' sq' 0x00 '~' 0x00 0x0f 'sr' 0x00 0x11 'java.lang.Integer' 0x12 0xe2 0xa0 0xa4 0xf7 0x81 0x87 '8' 0x02 0x00 0x01 'I' 0x00 0x05 'valuexr' 0x00 0x10 'java.lang.Number' 0x86 0xac 0x95 0x1d 0x0b 0x94 0xe0 0x8b 0x02 0x00 0x00 'xp' 0x00 0x00 0x00 0x01 'sr' 0x00 0x11 'java.util.HashMap' 0x05 0x07 0xda 0xc1 0xc3 0x16 '`' 0xd1 0x03 0x00 0x02 'F' 0x00 0x0a 'loadFactorI' 0x00 0x09 'thresholdxp?@' 0x00 0x00 0x00 0x00 0x00 0x00 'w' 0x08 0x00 0x00 0x00 0x10 0x00 0x00 0x00 0x00 'xxxuq' 0x00 '~' 0x00 0x11 0x00 0x00 0x00 '.0,' 0x02 0x14 '=' 0x14 'a~' 0xe1 0xe8 0x82 0xc9 '~' 0x92 0xd9 0xb9 'dy,' 0xd9 0x13 'H' 0x96 '+' 0x02 0x14 'j' 0xf6 '=' 0x9d 0x97 '.' 0x1f 0x13 0xf5 'h' 0x09 0xeb '}' 0x91 '4' 0xf2 0x97 0x1a 0x05 0xfe 't' 0x00 0x03 'DSAsr' 0x00 0x11 'java.lang.Boolean' 0xcd ' r' 0x80 0xd5 0x9c 0xfa 0xee 0x02 0x00 0x01 'Z' 0x00 0x05 'valuexp' 0x01 'pxsr' 0x00 '1org.apache.commons.collections.set.ListOrderedSet' 0xfc 0xd3 0x9e 0xf6 0xfa 0x1c 0xed 'S' 0x02 0x00 0x01 'L' 0x00 0x08 'setOrdert' 0x00 0x10 'Ljava/util/List;xr' 0x00 'Corg.apache.commons.collections.set.AbstractSerializableSetDecorator' 0x11 0x0f 0xf4 'k' 0x96 0x17 0x0e 0x1b 0x03 0x00 0x00 'xpsr' 0x00 0x15 'net.sf.json.JSONArray]' 0x01 'To\(r' 0xd2 0x02 0x00 0x02 'Z' 0x00 0x0e 'expandElementsL' 0x00 0x08 'elementsq' 0x00 '~' 0x00 0x18 'xr' 0x00 0x18 'net.sf.json.AbstractJSON' 0xe8 0x8a 0x13 0xf4 0xf6 0x9b '?' 0x82 0x02 0x00 0x00 'xp' 0x00 'sr' 0x00 0x13 'java.util.ArrayListx' 0x81 0xd2 0x1d 0x99 0xc7 'a' 0x9d 0x03 0x00 0x01 'I' 0x00 0x04 'sizexp' 0x00 0x00 0x00 0x01 'w' 0x04 0x00 0x00 0x00 0x01 't' 0x00 0x04 'asdfxxsq' 0x00 '~' 0x00 0x1e 0x00 0x00 0x00 0x00 'w' 0x04 0x00 0x00 0x00 0x00 'xxq' 0x00 '~' 0x00 ' sq' 0x00 '~' 0x00 0x02 'sq' 0x00 '~' 0x00 0x05 'w' 0x04 0x00 0x00 0x00 0x02 'q' 0x00 '~' 0x00 0x1a 'q' 0x00 '~' 0x00 0x09 'xq' 0x00 '~' 0x00 ' px'  Dec 10, 2017 6:47:41 PM org.eclipse.jetty.util.log.JavaUtilLog warn WARNING: Illegal character 0x3 in state=START for buffer HeapByteBuffer@5a0cb7be[p=1,l=42,c=16384,r=41]={\x03<<<\x00\x00*%\xE0\x00\x00\x00\x00\x00Cookie:...=Test\r\n\x01\x00\x08\x00\x03\x00\x00\x00>>>80\r\nConnection: k...to add resource} Dec 10, 2017 6:47:41 PM org.eclipse.jetty.util.log.JavaUtilLog warn WARNING: badMessage: 400 Illegal character 0x3 for HttpChannelOverHttp@54cb428f{r=0,c=false,a=IDLE,uri=} Dec 10, 2017 6:47:42 PM org.eclipse.jetty.util.log.JavaUtilLog warn WARNING: Illegal character 0x3 in state=START for buffer HeapByteBuffer@50f4dcce[p=1,l=42,c=16384,r=41]={\x03<<<\x00\x00*%\xE0\x00\x00\x00\x00\x00Cookie:...=Test\r\n\x01\x00\x08\x00\x03\x00\x00\x00>>>s.map.ReferenceMa...ns.slack.SlackN} Dec 10, 2017 6:47:42 PM org.eclipse.jetty.util.log.JavaUtilLog warn WARNING: badMessage: 400 Illegal character 0x3 for HttpChannelOverHttp@15f648b4{r=0,c=false,a=IDLE,uri=}  Dec 10, 2017 11:23:04 PM hudson.ExpressionFactory2$JexlExpression evaluate WARNING: Caught exception evaluating: h.filterDescriptors(it,attrs.descriptors) in /job/csp-multipipe/configure. Reason: java.lang.NullPointerException java.lang.NullPointerException  Dec 11, 2017 3:18:54 AM org.jenkinsci.plugins.durabletask.ProcessLiveness isAlive WARNING: hudson.Launcher$LocalLauncher@595604e5 on hudson.remoting.LocalChannel@47f4f94f does not seem able to determine whether processes are alive or not Dec 11, 2017 3:18:55 AM org.jenkinsci.plugins.durabletask.ProcessLiveness isAlive WARNING: hudson.Launcher$LocalLauncher@2ede7d5c on hudson.remoting.LocalChannel@47f4f94f does not seem able to determine whether processes are alive or not Dec 11, 2017 3:18:55 AM org.jenkinsci.plugins.durabletask.ProcessLiveness isAlive WARNING: hudson.Launcher$LocalLauncher@3b8ca3d5 on hudson.remoting.LocalChannel@47f4f94f does not seem able to determine whether processes are alive or not Dec 11, 2017 3:18:55 AM org.jenkinsci.plugins.durabletask.ProcessLiveness isAlive WARNING: hudson.Launcher$LocalLauncher@46bfcd1b on hudson.remoting.LocalChannel@47f4f94f does not seem able to determine whether processes are alive or not Dec 11, 2017 3:18:55 AM org.jenkinsci.plugins.durabletask.ProcessLiveness isAlive WARNING: hudson.Launcher$LocalLauncher@14c307e9 on hudson.remoting.LocalChannel@47f4f94f does not seem able to determine whether processes are alive or not Dec 11, 2017 3:18:56 AM org.jenkinsci.plugins.durabletask.ProcessLiveness isAlive WARNING: hudson.Launcher$LocalLauncher@6f4b5525 on hudson.remoting.LocalChannel@47f4f94f does not seem able to determine whether processes are alive or not Dec 11, 2017 3:18:56 AM org.jenkinsci.plugins.durabletask.ProcessLiveness isAlive WARNING: hudson.Launcher$LocalLauncher@23f674cc on hudson.remoting.LocalChannel@47f4f94f does not seem able to determine whether processes are alive or not  Dec 11, 2017 6:34:05 AM hudson.ExpressionFactory2$JexlExpression evaluate WARNING: Caught exception evaluating: request.getSession() in /cli. Reason: java.lang.IllegalStateException: Response is committed java.lang.IllegalStateException: Response is committed  Dec 11, 2017 7:11:00 AM org.eclipse.jetty.util.log.JavaUtilLog warn WARNING: Illegal character 0x16 in state=START for buffer HeapByteBuffer@5a0cb7be[p=1,l=177,c=16384,r=176]={\x16<<<\x03\x01\x00\xAc\x01\x00\x00\xA8\x03\x03M\x04\x8e\x01w\xAf\xFb...\x03\x02\x03\x03\x02\x01\x02\x02\x02\x03\x00\x0f\x00\x01\x01>>>ent: Go-http-clie...to add resource} Dec 11, 2017 7:11:00 AM org.eclipse.jetty.util.log.JavaUtilLog warn WARNING: badMessage: 400 Illegal character 0x16 for HttpChannelOverHttp@8cd8337{r=0,c=false,a=IDLE,uri=}  Dec 11, 2017 9:31:42 AM hudson.init.impl.InstallUncaughtExceptionHandler$DefaultUncaughtExceptionHandler uncaughtException SEVERE: A thread (TCP agent connection handler #43 with /5.188.10.105:51004/86524) died unexpectedly due to an uncaught exception, this may leave your Jenkins in a bad way and is usually indicative of a bug in the code. java.lang.SecurityException: Rejected: sun.reflect.annotation.AnnotationInvocationHandler   Dec 11, 2017 10:07:17 AM com.cloudbees.jenkins.plugins.bitbucket.hooks.WebhookAutoRegisterListener$1 doRun WARNING: Could not register hooks for csp-multipipe java.io.IOException: I/O error when parsing response from URL: https://bitbucket.org/api/2.0/repositories/nubeva/service_platform/hooks?page=1&pagelen=50  Dec 11, 2017 10:45:30 AM hudson.ExpressionFactory2$JexlExpression evaluate WARNING: Caught exception evaluating: request.getSession() in /cli. Reason: java.lang.IllegalStateException: Response is committed java.lang.IllegalStateException: Response is committed  Dec 12, 2017 4:34:28 AM hudson.ExpressionFactory2$JexlExpression evaluate WARNING: Caught exception evaluating: h.filterDescriptors(it,attrs.descriptors) in /job/csp-multipipe/configure. Reason: java.lang.NullPointerException java.lang.NullPointerException  Dec 12, 2017 9:29:15 AM hudson.TcpSlaveAgentListener$ConnectionHandler run WARNING: Connection #54 failed java.io.EOFException  Dec 12, 2017 1:14:58 PM hudson.TcpSlaveAgentListener$ConnectionHandler run WARNING: Connection #60 failed java.io.EOFException: unexpected stream termination  Dec 12, 2017 6:51:40 PM org.eclipse.jetty.util.log.JavaUtilLog warn WARNING: Illegal character 0x4 in state=START for buffer HeapByteBuffer@50f4dcce[p=1,l=10,c=16384,r=9]={\x04<<<\x01\x00P\xC0c\xF660\x00>>>Controller_Endpoi...backend.%22%2C+} Dec 12, 2017 6:51:40 PM org.eclipse.jetty.util.log.JavaUtilLog warn WARNING: badMessage: 400 Illegal character 0x4 for HttpChannelOverHttp@754b5fd6{r=0,c=false,a=IDLE,uri=} Dec 12, 2017 6:51:40 PM org.eclipse.jetty.util.log.JavaUtilLog warn WARNING: Illegal character 0x4 in state=START for buffer HeapByteBuffer@62f58182[p=1,l=10,c=16384,r=9]={\x04<<<\x01\x00P\xC0c\xF660\x00>>>/httpheader.net/ ...e+change+repos+} Dec 12, 2017 6:51:40 PM org.eclipse.jetty.util.log.JavaUtilLog warn WARNING: badMessage: 400 Illegal character 0x4 for HttpChannelOverHttp@3049df1b{r=0,c=false,a=IDLE,uri=} Dec 12, 2017 6:51:41 PM org.eclipse.jetty.util.log.JavaUtilLog warn WARNING: Illegal character 0x5 in state=START for buffer HeapByteBuffer@50f4dcce[p=1,l=3,c=16384,r=2]={\x05<<<\x01\x00>>>P\xC0c\xF660\x00Controller...backend.%22%2C+} Dec 12, 2017 6:51:41 PM org.eclipse.jetty.util.log.JavaUtilLog warn WARNING: badMessage: 400 Illegal character 0x5 for HttpChannelOverHttp@5639cbb0{r=0,c=false,a=IDLE,uri=}  Dec 12, 2017 11:21:00 PM org.apache.commons.httpclient.HttpMethodBase getResponseBody WARNING: Going to buffer response body of large or unknown size. Using getResponseBodyAsStream instead is recommended. 

1 Answers

Answers 1

This is not something I'll do, but if you're willing to use a hacky workaround, you can setup a parameter e.g. called OFFSET in your job config, then pass the OFFSET value along with BUILD_NUMBER environment variable. Subtract the OFFSET from the BUILD_NUMBER for the last digit in your version number. E.g. BUILD_NUMBER is 2691, OFFSET 2690, the number you want is 2691 - 2690 = 1. For the next build 2692, the number will be 2. And so on so forth. At one point, when you want to reset it back to 1, if the next build number is 3214, then you can simply change the offset to 3213, then you get 1, 2, 3... and so on again.

Read More

How to unlock jenkins on windows 10

I installed jenkins on my window 10 machine by using the msi I downloaded. It installed fine and has now launched the browser prompting me to unlock jenkins with the initialadminpassword. I can't find this file anywhere. I looked in the install directory C:\Program Files (x86)\Jenkins and I have checked the log files and no password was written to it. I even did a search on my whole C: for initialadminpassword and nothing came up. I do have a file called secret.key in my Jenkins install directory but the key in this file isn't working. Any ideas on how I can get around this would be very helpful

Thanks

2 Answers

Answers 1

In the Jenkins Home directory there should be a secrets subfolder. In that directory there should be a file called initialAdminPassword (no extension).

This file contains the password it is referring to.

Answers 2

In addition to "How to “Unlock Jenkins”?", I use a groovy script in order to make sure an admin account was created.

That means:

• I copy the groovy script in <Jenkins>/ref/init.groovy.d/security.groovy
• I launch jenkins.war with -Djenkins.install.runSetupWizard=false

In that script, replace the path of the files with the username/password by a Windows path one:

def user = new File("/run/secrets/jenkins-adm-name").text.trim() def pass = new File("/run/secrets/jenkins-adm-pass").text.trim() 

That way:

• no wizard
• one admin account with a username/password of my choosing.

Read More

Tcl script started from jenkins turns the command to lowercase

I'm having a strange issue when running a tcl command in jenkins.

The tcl script has the following lines (pay attention to the uppercase I in Id):

foreach name $docker_names {   set name "TestName"   puts $name   set command "docker inspect --format='{{.Id}}' ${name} > /home/temp/id.txt"   send -- "$command\n"   expect "$" } 

In the jeknins log I see that the job fails because what is sent in the second iteration of the loop is the command above but in lowercase. I need the I in Id to be uppercase.

This is what is sent in the second iteration of the loop:

docker inspect --format='{{.id}}' testname > /home/temp/id.txt 

NOTE: In the first iteration, everything is sent correctly.

Anyone has an idea of why this is happening?

Thanks!

0 Answers

Read More

jenkins ignores “inclusive regions”

I have two jenkins projects

1) one that polls over git repo

2) second that triggers java application

I want to configure (1) to poll on a specific file only, so i did it under "inclusive regions". However, i see the project is triggered after every change in the repo, not only when the specific file is changed.

Has anyone experienced the same?

Other work around?

1 Answers

Answers 1

Following the question "Jenkins Git plugin included regions not working", make sure that:

• your "included region" is a path relative to the root folder of your Git repo.

  a/b/c...bal/sdk.conf 

• "Force polling using workspace" under "Source-Code-Management - Additional Behaviours"

Note: there are still open bugs relative to "included regions":

• "JENKINS-28032: Included Regions do not work when monitoring all branches"

Make sure you are not monitoring all branches.

Read More

Parse JSON message in Logstash

I am sending my jenkins logs to logstash with following config:

 redis {      host => "localhost"      key => "logstash"      data_type => "list"      codec => json      }

This works as smooth as expected, now i see follwoing message in KIBANA:

{    "_index": "logstash-2015.12.18",    "_type": "logs",    "_id": "AVG1BN5LXZBIbp7HE4xN",    "_score": null,    "_source": {      "data": {        "id": "965",        "projectName": "NicePJ",        "displayName": "#965",        "fullDisplayName": "NicePJ",        "url": "job/NIcePJ/965/",        "buildHost": "Jenkins",        "buildLabel": "master",        "buildNum": 965,        "buildDuration": 1,        "rootProjectName": "NicePJ",        "rootProjectDisplayName": "#965",        "rootBuildNum": 965,        "buildVariables": {          "target_SUT": "0201",          "report_warnings": "false",          "product": "Ours",          "testsuite": "Exciting_stuff5",          "qft_version": "current",          "target_task": "t324",          "branch": "test",          "testcase": "",          "revision": "HEAD",          "node": "hsqs960",          "client": "Desktop",          "run_specific_test": "false",          "user": "xxxxx"        }      },      "message": [        "A         This is a message XYZ"      ],      "source": "jenkins",      "source_host": "http://serverXL:8080/",      "@timestamp": "2015-12-18T12:16:02.000Z",      "@version": 1    },    "fields": {      "@timestamp": [        1450440962000      ]    },    "sort": [      1450440962000    ]  }

Now i want to filter the message field for certain messages, but i cant get it work. How can i filter the message field and how can i access the buildHost field to use it in an if statement in the pipeline?

Following i tried after many examples:

 if[data][buildHost]== "jenkins"    {           grok           {             match => { "message[0]"  => "\[exec\]\s*\<%{GREEDYDATA:test}\s*\[%{GREEDYDATA:result}\]" }           }    }

But this is just not working at all, please help me out.

1 Answers

Answers 1

Conditional

The == compares simple string and case sensitive, so "jenkins" will not match as your data shows ("buildHost": "Jenkins",):

if[data][buildHost]== "jenkins" 

But following does:

if[data][buildHost]== "Jenkins" 

If you need match both, you can either use || or regex =~.

Grok

The grok is a filter to parse message with regex pattern. You can test your regex pattern with

• online grok debugger
• Kibana dev tools's grok debugger

Read More

Setting build args for dockerfile agent using a Jenkins declarative pipeline

I'm using the declarative pipeline syntax to do some CI work inside a docker container.

I've noticed that the Docker plugin for Jenkins runs a container using the user id and group id of the jenkins user in the host (ie if the jenkins user has user id 100 and group id 111 it would run the pipeline creating a container with the command docker run -u 100:111 ...).

I had some problems with this, as the container will run with a non existing user (particularly I ran into some issues with the user not having a home dir). So I thought of creating a Dockerfile that will receive the user id and group id as build arguments and creater a proper jenkins user inside the container. The Dockerfile looks like this:

FROM ubuntu:trusty ARG user_id ARG group_id  # Add jenkins user RUN groupadd -g ${group_id} jenkins RUN useradd jenkins -u ${user_id} -g jenkins --shell /bin/bash --create-home USER jenkins  ... 

The dockerfile agent has an additionalBuildArgs property, so I can read the user id and group id of the jenkins user in the host and send those as build aguments, but the problem I have now is that it seems that there is no way of executing those commands in a declarative pipeline before specifying the agent. I want my Jenkinsfile to be something like this:

// THIS WON'T WORK def user_id = sh(returnStdout: true, script: 'id -u').trim() def group_id = sh(returnStdout: true, script: 'id -g').trim()  pipeline {   agent {     dockerfile {       additionalBuildArgs "--build-arg user_id=${user_id} --build-arg group_id=${group_id}"     }   }   stages {     stage('Foo') {       steps {         ...       }     }     stage('Bar') {       steps {         ...       }     }     stage('Baz') {       steps {         ..       }     }     ...   } } 

I there is any way to achieve this? I've also tried wrapping the pipeline directive inside a node, but the pipeline needs to be at the root of the file.

2 Answers

Answers 1

I verified that trying to assign user_id and group_id without a node didn't work, as you found, but this worked for me to assign these values and later access them:

def user_id def group_id node {   user_id = sh(returnStdout: true, script: 'id -u').trim()   group_id = sh(returnStdout: true, script: 'id -g').trim() }  pipeline {   agent { label 'docker' }   stages {     stage('commit_stage') {       steps {         echo 'user_id'         echo user_id         echo 'group_id'         echo group_id       }     }   } } 

Hopefully these will also work in your additionalBuildArgs statement.

In a comment, you pointed out what is most likely a critical flaw with the approach that figures out the user_id and group_id outside the declarative pipeline before using it to configure the dockerfile: the slave on which it discovers the user_id will not necessarily match up with the slave that it uses to kick off the docker-based build. i don't there is any way around this while also keeping the declarative Jenkinsfile constraint.

You can guarantee one slave for all stages by using a global agent declaration: Jenkins declarative pipeline: What workspace is associated with a stage when the agent is set only for the pipeline?

But multiple node references with the same label don't guarantee the same workspace: Jenkins declarative pipeline: What workspace is associated with a stage when the agent is set only for the pipeline?

Answers 2

if you have admin access to Jenkins you can add these two script approvals:

staticMethod org.codehaus.groovy.runtime.DefaultGroovyMethods execute java.lang.String staticMethod org.codehaus.groovy.runtime.ProcessGroovyMethods getText java.lang.Process 

in this URI: http://${jenkins_host:port}/jenkins/scriptApproval/

which will allow you to execute a shell command in the master in this way:

def user = 'id -u'.execute().text node {    echo "Hello World ${user}" } 

Read More

Don't execute jenkins job if svn polling failed

I have a jenkins job, that is polling svn every 5 minutes and executing my unittests if some changes occured.

My probleme is, the svn polling fails randomly due to a unreachable proxy.

org.tmatesoft.svn.core.SVNAuthenticationException: svn: E170001: HTTP proxy authorization failed 

I guess this problem is related to some issues with the proxy we use and not the configuration of my job or machine.

My question now is, can I skip the job if the svn poll is failing and only execute if it was succesful? So that I don't have failed builds in my job list because of the proxy issue.

Or does anyhow have an idea why this random error can occure?

Fyi, I don't want the proxy problem itself fixed, as this is probably happening due to network problems, but I just want to skip the execution of the job if the svn poll fails.

2 Answers

Answers 1

Instead of polling svn, you can try a post-commit hook so that svn notifies Jenkins of changes; see https://wiki.jenkins-ci.org/display/JENKINS/Subversion+Plugin?focusedCommentId=43352266

Answers 2

In order to prevent running next action when the previous action is failed, add set +e to the top of your shell script. -e option is exit immediately when any action returns 1(which means failed). And also. @mikep's answer is useful thought. Instead of polling, Post-commit hook is more efficient.

Read More

Ad-Hoc codesigning for device succeeds in Studio, Fails in Jenkins

I have a Xamarin Forms application that supports Android and iOS. I've generated Jenkins builds to compile them. All of the Android builds work. The iOS Debug build compiles fine. The Ad-Hoc build, however, fails to build completely for an iPhone target. It appears to be failing during codesigning. It works if I target the iPhoneSimulator, but if I target iPhone device it fails.

Tool /usr/bin/codesign execution started with arguments: -v --force --sign 81088F8E194139DC4C6CE640716944E41FB0709F --entitlements "/Users/Shared/Jenkins/.jenkins/workspace/{project path}/obj/iPhone/Ad-Hoc/Entitlements.xcent" --deep "/Users/Shared/Jenkins/.jenkins/workspace/{project path}/bin/iPhone/Ad-Hoc/AppName.app" bin/iPhone/Ad-Hoc/AppName.app : error : /Users/Shared/Jenkins/.jenkins/workspace/{project path}/bin/iPhone/Ad-Hoc/AppName.app: unknown error -1=ffffffffffffffff [/Users/Shared/Jenkins/.jenkins/workspace/{project path}/iDriverMobile.iOS.csproj]

If I open up the Solution in Visual Studio, right in the Jenkins workspace folder so it's using the exact same files, then compilations works fine, which is really frustrating.

Looking at differences between the two outputs, it seems that the working build (from Studio) has AOT output for all of the assemblies that looks like this:

Mono Ahead of Time compiler - compiling assembly /Users/Shared/Jenkins/.jenkins/workspace/{project path}/obj/iPhone/Ad-Hoc/mtouch-cache/32/Build/OpenNETCF.Google.Analytics.dll

The failing build has none of those. Instead, it has a couple lines that look like this:

MTOUCH : warning MT0095: Aot files could not be copied to the destination directory /Users/Shared/Jenkins/.jenkins/workspace/{project path}/obj/iPhone/Ad-Hoc/mtouch-cache/64/Build/Msym/Msym/tmp: Could not start process. [/Users/Shared/Jenkins/.jenkins/workspace/{project path}/AppName.csproj]

The worst part of all of this is that these builds did work, but then I restarted the Mac Mini that Jenkins is running on and things went downhill. I can't figure out what the difference is between what Studio is doing and the command line call to msbuild. They both point to the same binaries.

Additional Information This still fails with the latest updates as of today (5/24/17). This is the environment:

• Mac OS X 10.12.5
• List item
• XCode 8.3.2
• Xamarin.iOS 10.10.0.36
• Visual Studio 2017 Community for Mac 7.0.1 (build 24)
• Mono 5.0.1.1

What doesn't fix it:

• Creating a new Jenkins build
• Changing the Jenkins workspace path
• Opening up permissions (777) to the entire Jenkins folder
• Enabling LLVM
• Disabling all linking
• Completely uninstalling and re-installing Jenkins
• Using xbuild instead of msbuild
• Swearing a lot
• My middle finger

1 Answers

Answers 1

Try to delete the derived data folder in DerivedData of your app. It looks like YourAPP_ dasfdsfsdafdsasfdsaf, according to this from Apple Developer Forum.

The DerivedData data folder is located at ~/Library/Developer/Xcode/DerivedData/

If this does not work, all the symptoms point to a signing certificate (also called, signing identity) issue.

It seems like when it was compiled from command line, /usr/bin/codesign can not access signing identity 81088F8E194139DC4C6CE640716944E41FB0709F. It could be many different reasons, unfortunately:

• keychain was locked
• codesign is not allowed to access the signing identity.
• multiple identities exist in keychain and wrong signing identity was selected
• Wrong provision profile was matched for Ad Hoc build.

Try to add following code snippets before running msbuild, assuming your signing identity is in keychain ~/Library/Keychains/login.keychain:

security unlock-keychain -p <password> ~/Library/Keychains/login.keychain security set-keychain-settings -l -u -t 3600 ~/Library/Keychains/login.keychain security set-key-partition-list -S apple-tool:,apple:,codesign: -s -k -p <password> ~/Library/Keychains/login.keychain 

It is not a good idea to have keychain password stored in the build script, you can follow this guide to hide them.

Read More

Dynamic parameter based on previous string parameter

In Jenkins I have a string parameter HOST. After this parameter I have a dynamic parameter URL. This parameter needs to be a concatenation of strings, including the previously defined HOST parameter.

I have been unable to find out how to do this.

The motivation for this is subsequent parameters will also use this HOST parameter.

2 Answers

Answers 1

Just make sure I understand your requirement correctly: you have two parameters and the later one is depends on the value of the first one?

If it's so I think plugin Parameterized Build is enough for this:

• Job configuration:

• Build with specified value:

• Build result:

• If you change your HOST value at build, the URL will also changed.

Hope this helps :-)

Answers 2

The dynamic parameter plugin might not be the right choice for your task. If you want the URL parameter to be updated according to the changes in the HOST parameter, you should consider using the Active Choices Plugin.

In your case, create a String parameter for the host, and an active choice reactive parameter for the url. this parameter reads the host and updates the url. example:

(you can find another example here: http://stackoverflow.com/a/42871619/947784)

Read More

JUnit test with Embedded tomcat server , how to specify automatic ports for both http and https connectors?

Description

I have made a JUnit test that focus on trying to test a call to a SOAP web service.

I am using an embedded tomcat server for my test in order to run my test with a mock server.

I am also using both http and https connectors.

I need to use automatic ports for both these connectors because the test is running on a Jenkins server and i can't just use port 443 or 8443 as they are already taken.

I understand that using the port 0 as standard port will result in tomcat using automatic port allocation but I can't manage to use it with both connectors.

Expected behavior

I'd like to use automatic port allocation also for my custom ssl connector.

Is it possible to do so in some way ?

Sample code

Here is the code for my tomcat instance :

@Before public void setup() throws Throwable {      File tomcatWorkingDir = new File(mWorkingDir);      //Empty the target/tomcat-working-dir directory if it exist     //Create the directory otherwise     if(tomcatWorkingDir.exists() && tomcatWorkingDir.isDirectory()){         LOGGER.info("cleaning tomcat-working-dir directory");         FileUtils.cleanDirectory(new File(mWorkingDir));      } else {         LOGGER.info("create tomcat-working-dir directory");         tomcatWorkingDir.mkdir();     }      LOGGER.info("disabling ssl certification validation");     //Disable JVM ssl sockets connection     disableJVMCertificate();      //Add server certificate     createServerCertificate();      //Custom SSL Connector     Connector SSLConnector = getSSLConnector();      mTomcat = new Tomcat();      //Standard http startup port     mTomcat.setPort(0);      //Set up base directory      //Otherwise, tomcat would use the current directory     mTomcat.setBaseDir(mWorkingDir);      LOGGER.info("setting the ssl connector in TOMCAT");     Service service = mTomcat.getService();     service.addConnector(SSLConnector);      //Redirect current port     Connector defaultConnector = mTomcat.getConnector();     defaultConnector.setRedirectPort(SERVER_HTTPS_PORT);      //Configure the way WAR are managed by the engine     mTomcat.getHost().setAutoDeploy(true);     mTomcat.getHost().setDeployOnStartup(true);      //Add mock server into our webApp     String servletName = "/server";     File webApp = new File(mWorkingDir,"../../../ws-mock-server/src/main/webapp");      mTomcat.addWebapp(mTomcat.getHost(), servletName, webApp.getAbsolutePath());      //start tomcat     LOGGER.info("starting TOMCAT");      mTomcat.start();   } 

and here for my custom ssl connector.

    private static Connector getSSLConnector(){     Connector connector = new Connector();     connector.setPort(SERVER_HTTPS_PORT);     connector.setSecure(true);      //Http protocol Http11AprProtocol     connector.setAttribute("protocol", "org.apache.coyote.http11.Http11AprProtocol");      //Maximum threads allowedd on this instance of tomcat     connector.setAttribute("maxThreads","200");     connector.setAttribute("SSLEnabled", true);      //No client Authentification is required in order to connect     connector.setAttribute("clientAuth", false);      //SSL TLSv1 protocol     connector.setAttribute("sslProtocol","TLS");      //Ciphers configuration describing how server will encrypt his messages     //A common cipher suite need to exist between server and client in an ssl     //communication in order for the handshake to succeed     connector.setAttribute("ciphers","TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA");      LOGGER.info("setting keystore file");     //Here an absolute file path is needed in order to properly set up the keystore attribute     connector.setAttribute("keystoreFile",new File(".").getAbsolutePath().replace("\\", "/")+"/"+mWorkingDir+"/server.jks");      LOGGER.info("setting keystore pass");     connector.setAttribute("keystorePass","changeit");      return connector; } 

0 Answers

Read More

CodeSign Issues: resource-rules (deprecated in Mac OS X >= 10.10)

Firstly, to ease the assumptions of "possible duplicate", I have ran accross these two posts here: Jenkins vs. Xcode plugin - codesign troubles

Xcode 6.1 error while building IPA

And subsequently this apple tech note

These solutions have not solved anything. I recently upgraded a Mac box from a much older OS X to El Capitan along with the latest and greatest XCode version (7.something or other) and is running Jenkins. Projects that were building before the upgrades are now not building.

First error was the one from my title (from first jenkins build after upgrade):

error: /usr/bin/codesign --force --preserve-metadata=identifier,entitlements,resource-rules --sign Stuff_I_Dont_think_I_canShow/TheProject.app failed with error 1. Output: Warning: usage of --preserve-metadata with option "resource-rules" (deprecated in Mac OS X >= 10.10)!.

Tried the solutions from the two posts, pushed the changes, but it failed again with:

Code Sign error: No code signing identities found: No valid signing identities (i.e. certificate and private key pair) matching the team ID “SOMEID123” were found. (again, not sure I can show team ID, this stuff is new to me, and I'm not sure what is and what is not sensitive info)

Did some more digging and ran across this answer here: --resource-rules has been deprecated in mac os x >= 10.10. Which cited:

Since Xcode 7, the Code Signing Resource Rules Path build setting must be left empty

Ok. Well I have Xcode 7. So I removed the changes the previous solutions cited and now I get the same error still: Code Sign error: No code signing identities found: No valid signing identities (i.e. certificate and private key pair) matching the team ID “SOMEID123” were found.

What's going on here? I have wasted many hours trouble shooting this with no results. Do I have to somehow explicitly remove the usage of resource-rules in some obscure location? Any knowledge sharing would be appreciated.

Here is the packaging and signing part of the script (censored to a degree):

# Build & package app cd $PROJECT_DIR/src/xcode xcodebuild -target MyApp -configuration Release clean build SYMROOT=$BUILD_DIR_BASE xcrun -sdk iphoneos PackageApplication -v $BUILD_DIR/Mobile_MyApp.app -o $BUILD_DIR/Mobile_MyApp.ipa --sign "iPhone Distribution: Company" --embed "$CERT_DIR/hf83uw90-i987-21j0-5t6y-f897e2384957.mobileprovision" 

To add another tidbit of information, this is a git project and before the packaging, retrieval of the project takes place, then access to repos.

1 Answers

Answers 1

So your provision file (PROVISIONFILE) is $CERT_DIR/hf83uw90-i987-21j0-5t6y-f897e2384957.mobileprovision and your distribution name (DISTRIBUTION_NAME) is iPhone Distribution: Company.

I'd add this to the script:

PROFILE_UUID=`grep "<key>UUID</key>" $PROVISIONFILE -A 1 --binary-files=text | sed -E -e "/<key>/ d" -e "s/(^.*<string>)//" -e "s/(<.*)//"` 

and then on the xcodebuild line

PROVISIONING_PROFILE=$PROFILE_UUID CODE_SIGN_IDENTITY="$DISTRIBUTION_NAME" 

Read More

Notify about / check for SCM poll failure in Jenkins

I would like to check for or get notifications about SCM poll failures in Jenkins (for example, when the repository URL had changed, or branch got deleted). I thought about these:

a) A Jenkins console script, which would list such faulty jobs

b) Configuring/installing plugin for Jenkins to notify me somehow about that fact (e-mail, anything)

c) External script/executable (bash, python, ...), which would list builds which failed in last X hours due to SCM poll failure

0 Answers

Read More

Trigger workflow on Github push - Pipeline plugin - Multibranch configuration

We are using the pipeline plugin with multibranch configuration for our CD. We have checked in the Jenkinsfile which works off git.

git url: "$url",credentialsId:'$credentials' 

The job works fine, but does not auto trigger when a change is pushed to github. I have set up the GIT web hooks correctly.

Interestingly, when I go into a branch of the multibranch job and I click "View Configuration", I see that the "Build when a change is pushed to Github" is unchecked. There is no way to check it since I can not modify the configuration of the job (since it takes from parent) and the same option is not there in parent.

Any ideas how to fix this?

0 Answers

Read More

Jenkins: Link between Stacktrace and GitLab

We use Jenkins and GitLab in our company. We use py.test to create the XML output for Jenkins which gets rendered by the jUnit Jenkins Plugin.

The stacktraces of exceptions are pure ascii up to now. It would be really great if we could hook into Jenkins somehow and show hyperlinks to our gitlab server instead of a html pre block.

In our case we it would be enough to filter the every line of the test output and use a regular expression on it....

Example output of the Jenkins jUnit Plugin:

File "/home/modwork_ems_d66/src/foo/foo/utils/testutils.py", line 975, in wrapped     return fn(*args, **kwargs) File "/home/modwork_ems_d66/src/foo/foo/tests/FooTest.py", line 641, in test_empty_models_if_unused     models)) 

I want a hyperlink to the server hosting the git repo of "foo". In our case this would be "https://source/repos/foo/files/master/foo/foo/tests/FooTest.py"

How to get this done? I know how to use regular expressions. The question is how to hook the regex into jenkins.

Here is a screenshot to visualize it

I want to this link on the TestReport of a single test. The URL pattern of these pages:

https://jenkins/job/ci_foo_bar/lastCompletedBuild/testReport/src.myapp.myapp.tests.test_tree/TestCase/test_something/ 

1 Answers

Answers 1

If you can acquire exception output after testing process you can implement a parser which prepare a simple html report for you as a build-step.

A a simplest way you can implement just a regular expression with sed like

sed -e 's@"/home/modwork_ems_d66/src\(.*\?\)"@<a href="https://source/repos/foo/files/master\1">\1</a>@' exceptions.txt 

which operates on the exception file.

Read More