Differences

This shows you the differences between two versions of the page.

--- joid:ci_integration [2015/12/09 17:51]
David Blaisonneau [Cleanup step]
+++ joid:ci_integration [2016/01/26 14:21] (current)
David Blaisonneau [Jenkins defaults]
@@ Line 2: / Line 2: @@
 ==== Prerequisites ====
+=== Jenkins install ===
+Please follow HOW TO here: https://wiki.opnfv.org/wiki/jenkins#how_to_connect_servers_from_labs_to_opnfv_jenkins
 === Jump host preparation ===
@@ Line 8: / Line 12: @@
 The following things must be set on the jumphost before starting any deploy:
-  * One extra repo (repository ppa:juju/stable)
   * applications must be installed
-    * ''libvirt-bin''
     * ''git''
-    * ''iptables-persistent''
     * ''bridge-utils''
     * ''vlan''
@@ Line 20: / Line 21: @@
     * ''brStorage''
-===Pod parameters===
+All those pre-requisits can be managed by Master MAAS.
-The pod admin must also set a few parameters
+Now you can launch jenkins
-<code>
+===== Jenkins parameters =====
-mkdir ~/joid_config
-cat << EOF > ~/joid_config/config.sh
-### Set Pod name
+Jenkins fixes those parameters:
-export POD_DC=orange
-export POD_NUM=pod2
-export POD_NAME=\$POD_DC\$POD_NUM
-### Set passwords
+====Jenkins defaults====
-export MAAS_USER=ubuntu
+<code>
-export MAAS_PASSWORD=ubuntu
+# Slave name on Jenkins
-export OS_ADMIN_PASSWORD=openstack
+export SLAVE_NAME=orange-fr-pod2
+export NODE_NAME=$SLAVE_NAME
-### Set CEPH parameters
+# Git URL to use on this Jenkins Slave
-# disks to use by ceph (comma separated list)
+export GIT_BASE=https://gerrit.opnfv.org/gerrit/$PROJECT
-export CEPH_DISKS=/dev/sdb,/dev/sdc
-# format or not disk before using ceph (must be done the first time)
-export CEPH_REFORMAT=false
-EOF
+# Project Name
-</code>
+export PROJECT=joid
-===Get initial joid source===
+# Place of jenkins workspace
+export WORKSPACE=$HOME/workspace/joid-deploy-<scenario>/
+# in case of local script testing
+export WORKSPACE=$HOME/joid/
-<code>
-cd ~/
-git clone http://gerrit.opnfv.org/gerrit/joid.git
 </code>
-===== Jenkins parameters =====
-Jenkins need to set those vars for joid installation:
+====Projects default====
+The file containing installer defaults is **releng/jjb/opnfv/installer-params.yml**
 <code>
-        - string:
+# IP of the installer (Not used for Joid)
-            name: INSTALLER
+export INSTALLER_IP=192.168.Y.Y
-            default: '{installer}'
-            description: "Installer to use."
-        - string:
-            name: JOID_MODE
-            default: 'ha'
-            description: "High availability mode: <ha|nonha|tip>"
-        - string:
-            name: JOID_OS_RELEASE
-            default: 'liberty'
-            description: "Openstack release: <juno|liberty>"
-        - string:
-            name: JOID_SDN_CONTROLLER
-            default: 'odl'
-            description: "Joid SDN controller: <odl|opencontrail|onos|none>"
-        - string:
-            name: JOID_LOCAL_CONFIG_FOLDER
-            default: '~/joid_config'
-            description: "Joid local pod config"
-        - string:
-            name: JOID_ADMIN_OPENRC
-            default: '~/joid_config/admin-openrc'
-            description: "Joid local pod config"
-</code>
-===== Cleanup steps =====
+# Installer used for deploying OPNFV on this POD
+export INSTALLER_TYPE=joid
-Cleanup can be on Juju only or also on the jumphost (destroying MAAS and Bootstrap VM)
+# Scenario to deploy and test
-So we can split that step in 2 templates:
+# Based on scenario naming we can get joid options
+# naming convention:
+#    os-<controller>-<nfvfeature>-<mode>[-<extrastuff>]
+# With parameters:
+#    controller=(nosdn|odl_l3|odl_l2|onos|ocl)
+#       No odl_l3 today
+#    nfvfeature=(kvm|ovs|dpdk|nofeature)
+#       '_' list separated.
+#    mode=(ha|noha)
+#    extrastuff=(none)
+#       Optional field - Not used today
+export DEPLOY_SCENARIO=none
-=== [joid-clean] Juju destroy environmnet ====
+# OpenStack release (kilo|liberty)
+export OS_RELEASE=liberty
-<code>
+# External network to create (name;type;first ip;last ip; gateway;network)
-#!/bin/bash
+export EXTERNAL_NETWORK="ext-net;flat;192.168.0.2;192.168.0.253;192.168.0.1;192.168.0.0/24"
-set +e
-cd ~/joid/ci
-echo "------ Backup Juju environment ------"
-cp environment.yaml $JOID_LOCAL_CONFIG_FOLDER
-echo "------ Clean MAAS and Juju ------"
-./clean.sh
-</code>
-To enhance for 'juju not deployed' case
-=== [joid-destroy-vm] Destroy MAAS and Bootstrap VM ===
+# Disks to use by ceph by default (space separated list)
+export CEPH_DISKS=/srv
-<code>
+# Disks to use by ceph by controllers (empty if same as CEPH_DISKS)
-#!/bin/bash
+export CEPH_DISKS_CONTROLLERS=""
-set +e
-echo "------ VM List ------"
-virsh list
-echo "------ Destroy Bootstrap ------"
-VM=$(virsh list --name |grep bootstrap)
-if [ "z$VM" = "z" ]
-    then "echo NO Bootstrap VM"
-    else {
-        virsh destroy $VM
-        virsh undefine $VM
-        virsh vol-delete --pool default /var/lib/libvirt/images/$VM.img
-    }
-fi
-echo "------ Destroy MAAS ------"
-VM=$(virsh list --name |grep maas)
-if [ "z$VM" = "z" ]
-    then "echo NO MAAS VM"
-    else {
-        virsh destroy $VM
-        virsh undefine $VM
-        virsh vol-delete --pool default /var/lib/libvirt/images/$VM-root.img
-        virsh vol-delete --pool default /var/lib/libvirt/images/$VM-seed.img
-    }
-fi
-</code>
+# Local lab config and Openstack openrc location
+export LAB_CONFIG=$HOME/joid_config
-To be completed
+# Format or not disk before using ceph [true/false] (must be done the first time)
+export CEPH_REFORMAT=false
-===== Build =====
-No build needed
-===== Install =====
-Installation can be automatically done following those steps:
-  - [joid-clean] see previous section
-  - [joid-destroy-vm] see previous section
-  - [joid-gitupdate] fetch the last version of joid
-  - [joid-configure] load local pod parameters (local login/passwords) and load jenkins parameters (ha mode, sdn controller, open stack release)
-  - [joid-prepare] prepare infra by installing maas and juju
-  - [joid-deploy] deploy joid
-  - [joid-adminrc] prepare adminrc for other jenkins jobs (functest...)
-All those steps can be done in one jenkins template
-  - [joid-daily-full] this template is also destroying MAAS and Bootstrap VM
-  - [joid-daily] this one is not destroying MAAS and Bootstrap VM (no [joid-destroy-vm] and [joid-prepare])
+# Reinstall MAAS and Bootstrap before deploy [true/false]
+export MAAS_REINSTALL=false
+</code>
-==== [joid-gitupdate] fetch the last version of joid ====
+==== Pod defaults====
+The file containing pod default is **releng/jjb/opnfv/slave-params.yml**
 <code>
-cd ~/
+# Version of the installer to deploy
-source joid_local_config
+export INSTALLER_VERSION=latest
-rm -rf joid
-git clone http://gerrit.opnfv.org/gerrit/joid.git
-cd ~/joid/ci
 </code>
-==== [joid-configure] load parameters ====
+Those parameters shall be present in the pod default.
-load local pod parameters and jenkins parameters to modify joid templates
 <code>
-#!/bin/bash
+# Disks to use by ceph by default (space separated list)
-set +e
+export CEPH_DISKS=/srv
-source $JOID_LOCAL_CONFIG_FOLDER/config.sh
-cd ~/joid/ci
-# Get juju deployer file
+# Disks to use by ceph by controllers (empty if same as CEPH_DISKS)
-if [ "$JOID_MODE" == 'nonha' ]
+export CEPH_DISKS_CONTROLLERS=/srv
-    then SRCBUNDLE=~/joid/ci/$JOID_SDN_CONTROLLER/juju-deployer/ovs-$JOID_SDN_CONTROLLER.yaml
+</code>
-    else SRCBUNDLE=~/joid/ci/$JOID_SDN_CONTROLLER/juju-deployer/ovs-$JOID_SDN_CONTROLLER-$JOID_MODE.yaml
-    fi
-# Get MAAS config file
+===== Cleanup steps =====
-MAASCONFIG=~/joid/ci/maas/$POD_DC/$POD_NUM/deployment.yaml
-# Modify files
+The cleanup is done at each deploy in the script
-echo "------ Set MAAS password ------"
+===== HOW TO =====
-sed -i -- 's/user: ubuntu/user: $MAAS_USER/' $MAASCONFIG
-sed -i -- 's/password: ubuntu/user: $MAAS_PASSWORD/' $MAASCONFIG
-echo "------ Set openstack password ------"
+==== Run CI deploy script without jenkins ====
-sed -i -- "s/\"admin-password\": openstack/\"admin-password\": $OS_ADMIN_PASSWORD/" $SRCBUNDLE
-echo "------ Set ceph disks ------"
+  * Copy all jenkins parameters (see list 'Jenkins parameters' section) in a file (~/deploy_params.sh) and set them according to your platform (**don't forget to set DEPLOY_SCENARIO value !**)
-sed -i -- "s/\"osd-devices: \/srv/osd-devices: $CEPH_DISKS/" $SRCBUNDLE
+  * Load this file
-if [ "$CEPH_REFORMAT" == 'true' ]
+<code>source ~/deploy_params.sh</code>
-    then sed -i -- "s/osd-reformat: 'no'/osd-reformat: 'yes'/" $SRCBUNDLE
+  * Get the script
+<code>wget https://git.opnfv.org/cgit/releng/plain/jjb/joid/joid-deploy.sh
+chmod +x joid-deploy.sh</code>
+  * Run the script
+<code>./joid-deploy.sh</code>
-</code>
+==== Add my pod deployed by jenkins ====
+  - Edit Releng **jjb/joid/joid-ci-jobs.yml**
+     - to add your node to pod list in the project section
+     - to add your trigger macro at the end of the file
+  - Add your pod description in **jjb/opnfv/slave-params.yml**
+  - Setup (or not) your local config file, as described in "Set specific pod parameters".
+  - If your $POD_NAME is not formated like this: <POD_DATA_CENTER>-<POD_ID> as set in joid/ci/maas/<POD_DATA_CENTER>/<POD_ID> subfolders, please modify section "Set Joid pod config name" in **releng/jjb/joid/joid-deploy.sh**
+  - Ask jenkins to deploy :)
-==== [joid-prepare] install maas and juju ====
+==== Set specific pod parameters ====
-<code>
+The pod admin can set a few parameters
-MAASCONFIG=~/joid/ci/maas/$POD_DC/$POD_NUM/deployment.yaml
-sed -i -- 's/user: ubuntu/user: $MAAS_USER/' $MAASCONFIG
-sed -i -- 's/password: ubuntu/user: $MAAS_PASSWORD/' $MAASCONFIG
-./02-maasdeploy.sh $POD_NAME
-</code>
-==== [joid-deploy] Deploy joid ====
-in the case of a non full install (without MAAS destroy) we need to recover the juju env file
 <code>
-#!/bin/bash
+sudo -s jenkins
-set +e
+mkdir ~/joid_config
-cd ~/joid/ci/
+cat << EOF > ~/joid_config/config.sh
-echo "------ Recover Juju environment to use MAAS ------"
+### Set passwords
-cp $JOID_LOCAL_CONFIG_FOLDER/environment.yaml .
+export MAAS_USER=ubuntu
+export MAAS_PASSWORD=ubuntu
+export OS_ADMIN_PASSWORD=openstack
+EOF
 </code>
-<code>
+==== Deploy or not MAAS at each time ====
-./deploy.sh -t $JOID_MODE -o $JOID_RELEASE -s $JOID_SDN_CONTROLLER -l $POD_NAME
-</code>
+Testing MAAS deploy is not the core of CI tests, so to gain time, MAAS reinstall can be disabled by setting local parameter MAAS_REINSTALL=false. Juju will be cleaned and all nodes managed will return in ready mode.
-==== [joid-adminrc] Set admin-openrc ====
-<code>
-cat << EOF > ~/admin-openrc
-export OS_USERNAME=admin
-export OS_PASSWORD=$OS_ADMIN_PASSWORD
-export OS_TENANT_NAME=admin
-export OS_AUTH_URL=http://192.168.2.23:5000/v2.0
-export OS_REGION_NAME=Canonical
-EOF
-</code>

Wiki

User Tools

Site Tools

Differences

Page Tools