Jenkins Webhook with Parameters

Send parameters through a Git webhook to Jenkins

Background

Someone asked me over the weekend how to trigger a build in Jenkins through a Git webhook and send parameters to the build at the same time. I don’t know why anyone would pass parameters like that (from SCM as opposed to a script), but since I’ve already spent the time figuring out how to do it, here it is.

In this article, I’ll go over the steps taken by the person who asked for help and then talk about where their assumptions went wrong and how to actually achieve what they wanted. While I only tested this solution on Gogs, I can’t see why the general idea wouldn’t work for Github, Bitbucket etc.

Create new Gogs repository

We’ll start from the basics: a dummy repository in Gogs. In my case, I created a brand new one (webhook-with-parameters), containing a README.md file and nothing else. Make sure you set the Jenkins user (jenkins) as a collaborator in your repo, otherwise Jenkins won’t be able to pull code from it.

/img/2018/10/2018-10-09-jenkins-webhook-with-parameters/01-gogs-jenkins-collaborator-thumb.jpg

Jenkins user as repository collaborator.

Create new Jenkins project

The Jenkins project I’m creating is a good old freestyle project1 and I’m calling it webhook-with-parameters as well. When you get to the configuration page, go to Gogs Webhook and check the box This project is parameterized. Then, select Add Parameter and fill in the form with whatever you want to use. I want to keep things simple for myself, so I’m sticking with a String Parameter (from_webhook).

/img/2018/10/2018-10-09-jenkins-webhook-with-parameters/02-parameterized-project-thumb.jpg

Parameterized project configuration.

Then, under Source Code Management, select Git and fill in the form with your repository’s information and the SSH credentials you’ve previously set for the default Jenkins user.

/img/2018/10/2018-10-09-jenkins-webhook-with-parameters/03-source-code-management-thumb.jpg

Source code management with global SSH credentials.

Under Build Triggers, check the box Build when a change is pushed to Gogs. This will cause the Jenkins project to start a new build whenever a new commit is sent to the Git repository.

/img/2018/10/2018-10-09-jenkins-webhook-with-parameters/04-build-on-push-thumb.jpg

Build when a change is pushed to Gogs.

Next, under Build, click on Add build step and select Execute shell. The shell script I’m using is just to test the value of the variable I’ve set up above (from_webhook).

/img/2018/10/2018-10-09-jenkins-webhook-with-parameters/05-execute-shell-thumb.jpg

Shell script to be executed during build.

Nothing fancy so far; all we did was set up a vanilla freestyle project with Git webhook integration. Save the project now and go back to Gogs.

Set up the webhook

In Gogs, go to your repo’s Settings tab and click on Webhooks. Then, click on Add Webhook and choose Gogs from the drop down menu. The only thing you need to change is the Payload URL field, which you’ll fill in with the Gogs webhook for your Jenkins project.2 Mine is:

http://jenkins.cwtf.local:8080/gogs-webhook/?job=webhook-with-parameters
/img/2018/10/2018-10-09-jenkins-webhook-with-parameters/06-gogs-webhook-settings-thumb.jpg

Gogs webhook settings.

Once the webhook has been created, you can open its configuration page and click on Test Delivery to see if Gogs is able to send a payload to Jenkins and get a successful response back. If it does, you can move on to the next step.

/img/2018/10/2018-10-09-jenkins-webhook-with-parameters/07-gogs-test-delivery-thumb.jpg

Test webhook delivery.

Testing the webhook

Testing the Gogs webhook like that will effectively trigger a build in Jenkins. When I go to my Jenkins project to check the console output, I can see that the build succeeds and the variable’s default value is printed on the Jenkins console output.

/img/2018/10/2018-10-09-jenkins-webhook-with-parameters/08-console-output-thumb.jpg

Jenkins console outputs variable's default value.

That’s what I was expecting from a regular build—one where the parameter was not explicitly passed to the Jenkins project. Let’s now add the parameter (from_webhook), send a new delivery test to the payload URL and see what happens.

http://jenkins.cwtf.local:8080/gogs-webhook/?job=webhook-with-parameters&from_webhook=new-value
/img/2018/10/2018-10-09-jenkins-webhook-with-parameters/09-gogs-webhook-settings-thumb.jpg

Gogs webhook with URL parameter.

Sending another delivery test from Gogs still triggers a new build in Jenkins, but the from_webhook variable keeps being assigned the default value… Jenkins doesn’t seem to be accepting the custom parameter passed in the URL payload. WTF?!?

/img/2018/10/2018-10-09-jenkins-webhook-with-parameters/10-console-output-thumb.jpg

Jenkins console still outputs variable's default value.

Build on SCM push vs. Build remotely

This is where my acquaintance got stuck. What seems obvious to an experienced Jenkins user is not quite so to someone new who’s just trying to get a freaking job running. They assumed Jenkins would happily take whatever they passed from that Gogs payload, but the butler knows better than that, amigo. To cut a long story short, when we configure a build to be triggered on SCM push, Jenkins seems to ignore any extra parameters that are sent to it from the Git server. I couldn’t find anything that confirms my hypothesis in the official documentation, but I believe that’s a security measure.

So, we can’t rely on the Build when a change is pushed to Gogs option in this scenario; instead, we’re going to resort to the Jenkins super powerful API. To that end, the first thing we need is a Jenkins API token:

API tokens offer a way to make authenticated CLI or REST API calls. See our wiki for more details. The username associated with each token is your Jenkins username.

You can create one by going to {JENKINS_URI}/user/{YOUR_JENKINS_USER}/configure. For example: http://jenkins.cwtf.local:8080/user/cwtf/configure. Under API Token, click Add New Token and then Generate. You should copy the auto-generated token somewhere, because as soon as you leave that page you won’t have access to it anymore.

/img/2018/10/2018-10-09-jenkins-webhook-with-parameters/11-api-token-thumb.jpg

Generate new API token.

Now we can use that token to authenticate against the Jenkins API and trigger builds from Gogs. Before we do that, though, we need to let Jenkins know that our project will be built remotely (through a raw call to its API, not from an SCM push), otherwise it will keep ignoring our custom parameter. On the project configuration page in Jenkins, under Build Triggers, check the box that says Trigger builds remotely (e.g., from scripts) and complete its input field (Authentication Token) with any string value—preferably one that’s hard to guess. You can also uncheck the Build when a change is pushed to Gogs box now, because it just became pointless. You can save the project once that’s done.

/img/2018/10/2018-10-09-jenkins-webhook-with-parameters/12-trigger-builds-remotely-thumb.jpg

Trigger builds remotely.

The last step is to update the Gogs webhook. We need to add to our URL the username and API token generated from the Jenkins config page; specify the token we’ve created manually in the previous step; and use a different endpoint altogether (the buildWithParameters endpoint). Something like this (note that the endpoint is completely different now):

{http|https}://{YOUR_JENKINS_USER}:{YOUR_JENKINS_API_TOKEN}@{JENKINS_URI}/job/{PATH_TO_JOB}/buildWithParameters?token={TRIGGER_BUILD_REMOTELY_TOKEN}&{PARAMETER_NAME}={PARAMETER_VALUE}

This is how my payload URL looks like:

http://cwtf:xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx@jenkins.cwtf.local:8080/job/webhook-with-parameters/buildWithParameters?token=super-secure-token-trust-me-its-very-secure&from_webhook=new-value

Paste that thing in the Gogs payload URL field, save it and you’re good to go. If you click on Test Delivery on Gogs now or push a new commit to your repository, the Jenkins variable should finally get assigned the value (new-value) passed from the Gogs URL parameter.

/img/2018/10/2018-10-09-jenkins-webhook-with-parameters/13-console-output-thumb.jpg

Output of custom parameter from external payload.

Conclusion

We usually call the Jenkins API from scripts or the command line (with curl), not from a source control webhook like that. I didn’t have the chance to ask the aforementioned person why they went with this approach, but regardless of their reasons, the solution works. And, as much as I love Jenkins, I have to admit that the butler is sometimes hard to make sense of and I completely understand when people get mad at him. With a little patience and fuzzing around, though, there’s probably at least one way of doing what you want. Cup o’ tea?


  1. I tested this solution with a pipeline project and it worked as well. [return]
  2. See the Gogs Webhook Plugin’s documentation for more information. [return]
api  ci-cd  git  gogs  jenkins