User Tools

Site Tools


OPNFV CI Infrastructure

The information presented here is yet to be discussed within Octopus Team and should be seen as input to discussion unless noted otherwise.

There are several tools and hardware resources that make it possible for us to develop/establish CI Framework for OPNFV.

Most of the tools needed for CI are already provided by Linux Foundation. We need to setup/configure/utilize these tools.

OPNFV hardware resources will be delivered to LF during coming week(s) so first CI POCs can/will be done on community donated hardware if things don't move on fast enough.

We also need to identify what other tools/resources we need with the help from wider OPNFV Community and make them available with the help from LF.

Please note that the information on this page will be detailed/evolved while we move on.


Below diagram tries to capture some of the tools/hardware resources needed for Octopus and does not mean how the things work.

See this page for CI Flows.


Tool/Resource Name Status Tasks Left to Do Notes
CI Engine Jenkins Pending configuration See below Link to general info
SCM Git Pending configuration See below Link to general info
Code Review Gerrit Pending configuration See below Link to general info
Artifact Repository TBD TBD See below Link to general info
Build Servers N/A TBD See below N/A
Test/Deployment Servers N/A TBD See below N/A


This section lists tasks to be handled in order to setup and configure tools and hardware resources.

Tasks that are not strikethroughed are pending.


Summary Priority Responsible Notes
Create CI Service/Functional User High LF Will be used for connecting LF/community Jenkins slaves, running jobs, voting in Gerrit, mail notification, chat, etc.
Suggested id: octopus
Configure CI Service/Functional User in Jenkins High LF SSH keys etc. setup on Jenkins.
Configure Jenkins master/apply configuration to Jenkins Global Configuration High LF Whatever is necessary.
Create domains Medium LF Whatever is necessary.
Install Jenkins Plugins and configure them High LF Please see below or click here for initial list of plugins.
Connect Jenkins to Gerrit High LF Need to be able to trigger jobs on Gerrit events.
Please see here.
Connect/configure slaves High LF Can/will start with echoing "Hello World!" by jobs running on master.
And then connect/use community(Ericsson/Intel BGS) hosting until LF resources are ready.
Create pools/attach labels on slaves so build and test/deployment resources are properly used High LF/Octopus Team N/A
Configure mail notification Low LF This could be handled like opt-in if people don't want to be bugged.
Make Jenkins talk/chat Low LF Like it is done with #opnfv-gerrit chatroom.
Please see here and here.
Create basic jobs to evaluate what can be done without proper HW High Octopus Team Create and run some jobs to see how urgent to get real HW for real Gerrit Verification flow etc.
Create basic jobs to verify the Gerrit connection High Octopus Team Do simple commits and check if things are triggered/voted on Gerrit/mail notification sent.
Decide/agree which repos to monitor/trigger jobs on High Octopus Team/OPNFV Community We start with the repos with some code/activity in them, such as BGS.


Summary Priority Responsible Notes
Configure CI Service/Functional User in Gerrit High LF SSH keys etc. setup on Gerrit.
Also it should be granted to Gerrit stream events.
Please see here.
Configure Mail Notification High LF This could be duplicate to Jenkins Mail notification and can be opted in.
But in general, people should be notified if a certain commit passed Gerrit verification so they can go and do the code review on Gerrit.
Enable Git/Gerrit hooks for large file check Medium LF/Octopus Team/OPNFV Community This is still under discussion but it would be good if we can start with some initial setting so we don't mess up with the size of the repos right in the beginning.
This obviously requires discussion/agreement with other OPNFV projects who have Git repos.
Please see here.
Do simple commit/push/wait for verification/review stuff High Octopus Team In order to evaluate if things are setup fine. Similar to Jenkins basic job creation task above.

Artifact Repository

Summary Priority Responsible Notes
Agree on if we need artifact repository or not High Octopus Team/OPNFV Community SSH keys etc. setup on Gerrit.
Also it should be granted to Gerrit stream events.
Please see here.
Decide which artifact repository to use with the guidance from LF High Octopus Team/LF/OPNFV Community Swift/Artifactory/Nexus
Guide OPNFV Community regarding artifact repository selection High LF Given that LF already hosts repositories for other OSS projects.
Decide and create repositories Medium Octopus Team/LF We can start with "a" repo and evolve later on.
Decide retention policy Medium Octopus Team/LF If storage becomes the bottleneck, we may need to start cleaning up based on the policy.
Acquire/provide storage if storage becomes the bottleneck even after enforcing the the retention policy Low OPNFV Management This needs to be evaluated while we move on and raised by the community.

Hardware Resources

Summary Priority Responsible Notes
Configure/connect/evaluate Ericsson hosting until LF resources arrive High LF/Octopus Team/Ericsson OPNFV Contributors This requires discussions.
Configure/connect/evaluate Intel hosting until LF resources arrive High LF/Octopus Team/Intel OPNFV Contributors This requires discussions.
Assist Ericsson/Intel OPNFV Contributors to securely connect hosting to Jenkins High LF This requires discussions.
Configure/connect LF resources High LF/Octopus Team N/A

Jenkins Job Builder

During Octopus Team Meeting on 2015-02-16, it has been suggested to give a try and perhaps use OpenStack Jenkins Job Builder for OPNFV CI.

OpenDaylight also uses Jenkins Job Builder for Jenkins Jobs so this is a common way of handling Jenkins related things. As OPNFV uses OpenStack and OpenDaylight as upstream projects for the components themselves, using/doing CI in a similar way is also logical.

As a first step, LF will help us setting up some simple jobs using Jenkins Job Builder and they will be put in Octopus repo on OPNFV Gerrit. Then we will continue with the rest of the evaluation but this looks quite promising as the things stand.

Jenkins Job Builder helps us to develop/maintain/handle Jenkins job configurations like the source code; changes to jobs go through regular Gerrit review process and makes things visible to greater OPNFV Community. Having job configurations version controlled has its own benefits which is not needed to be listed here. Deploying those jobs could also be automated.

Apart from all the good things listed in previous sentence, it also relieves us from creating/modifying/maintaining Jenkins jobs from Jenkins GUI, potentially reducing chance of mistakes and annoyance of clicking things around.

Here are some links

List of Plugins to Install

This is the initial list of plugins to install and new ones will probably be added based on Octopus/Community feedback/needs.

The ones that are marked as low priority can be skipped for the timebeing to see/evaluate the load on Jenkins master when other plugins are installed. If the load looks fine, they can be installed as well.

Plugin Purpose Responsible Priority Link
SSH Slaves Plugin Connecting slaves to Jenkins with SSH LF High SSH Slaves Plugin
Git Plugin Use Git as SCM LF High Git Plugin
Gerrit Plugin Integrate Gerrit with Jenkins LF High Gerrit Plugin
Gerrit Trigger Plugin Integrate Gerrit with Jenkins to trigger jobs LF High Gerrit Trigger Plugin
Parameterized Trigger Plugin Create jobs that can trigger other jobs and take action depending on result LF High Parameterized Trigger Plugin
Multijob Plugin Create complex/hierarchical jobs LF High Multijob Plugin
Job DSL Plugin Create jobs programmatically LF Medium Job DSL Plugin
Workflow Plugin Create workflows LF Low Workflow Plugin
Robot Framework Plugin Publishes/stores Robot FW test reports LF High Robot Framework plugin
Throttle Concurrent Builds Plugin Throttle concurrent builds LFHigh Throttle Concurrent Builds Plugin
Email-ext Plugin Send customized emails LFMedium Email-ext Plugin
Instant Messaging/Jabber Plugin Make Jenkins talk/chat LFLow Instant Messaging Plugin, Jabber Plugin
eXtreme Feedback Plugin Show quick build status LFHigh eXtreme Feedback Plugin
Build Pipeline Plugin Create and visualize pipelines LFMedium Build Pipeline Plugin
Green Balls Use Green Balls for successful builds LFHigh Green Balls
Plot Plugin Create plots LF Low Plot Plugin
Priority Sorter Plugin Adjust priorities of jobs LF Medium Priority Sorter Plugin
Dashboard View Plugin Create dashboards LF Medium Dashboard View
Static Code Analysis Plugin Provide Static Code Analysis utilities LF High Static Code Analysis Plugin
Artifact Deployer Plugin Deploy artifacts from workspace to output directories LF Medium ArtifactDeployer Plugin
Promoted builds plugin Mark builds as promoted LF Medium Promoted Builds Plugin
Build Failure Analyser Present causes of failures LF Low Build Failure Analyser
Build Timeout Plugin Set timeout so builds don't get stuck LF High Build Timeout Plugin
EnvInject Plugin Inject env variables to builds LF High EnvInject Plugin
Git Parameter Plugin Choose revisions or tags LF Medium Git Parameter Plugin
Job Config History Plugin Track job configuration history LF High JobConfigHistory Plugin
Log Parser Plugin Parse console logs LF High Log Parser Plugin
Mask Passwords Plugin Mask passwords LF High Mask Passwords Plugin
Post Build Task Plugin Do post build tasks LF High Post Build Task Plugin
Show Build Parameters Plugin Show parameters used for a certain build LF High Show Build Parameters plugin
Timestamper PluginAdd timestamps to console LF High Timestamper Plugin
Delivery Pipeline Plugin Visualize delivery pipeline LF Low Delivery Pipeline Plugin
Robot Framework Plugin Publishes/stores Robot FW test reports LF High Robot Framework plugin
octopus/infrastructure.txt · Last modified: 2015/02/17 16:21 by Fatih Degirmenci