This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision | ||
brahmaputra_testing_page [2016/01/13 15:58] Gerald Kunzmann [Table] |
brahmaputra_testing_page [2016/02/05 19:38] (current) Fatih Degirmenci [Jenkins Job Naming] |
||
---|---|---|---|
Line 66: | Line 66: | ||
| armband | - | Need an ARM-based lab (ANEA) | - | - | - | - | - | | | armband | - | Need an ARM-based lab (ANEA) | - | - | - | - | - | | ||
| copper | - | - | - | - | - | - | associated with functest | | | copper | - | - | - | - | - | - | associated with functest | | ||
- | | doctor | - | A.1 | - | - | - | - | associated with functest | | + | | doctor | - | A.* | - | - | - | - | associated with functest | |
| functest | need to detect the controller to run either odl/onos/ovno suite | *.* | - | - | Y | Y | - | | | functest | need to detect the controller to run either odl/onos/ovno suite | *.* | - | - | Y | Y | - | | ||
| ha | - | - | - | - | - | - | - | | | ha | - | - | - | - | - | - | - | | ||
Line 74: | Line 74: | ||
| ovsnfv | custom OVS | F.11 | - | - | - | - | - | | | ovsnfv | custom OVS | F.11 | - | - | - | - | - | | ||
| ovno | opencontrail | *.6 | - | - | - | - | associated with functest | | | ovno | opencontrail | *.6 | - | - | - | - | associated with functest | | ||
- | | promise | - | F.*, J.1 | - | - | - | - | associated with functest | | + | | promise | - | F.*, J.* | - | - | - | - | associated with functest | |
| sdnvpn | ODL needs VPN service karaf feature enabled | *.* | - | - | Y | - | associated with functest and yardstick | | | sdnvpn | ODL needs VPN service karaf feature enabled | *.* | - | - | Y | - | associated with functest and yardstick | | ||
| sfc | Needs ODL Beryllium with SFC, GBP, and OVSDB SB configured. Also needs Tack installed via post-install scripts. | F.9 | - | - | Y | - | associated with functest and yardstick | | | sfc | Needs ODL Beryllium with SFC, GBP, and OVSDB SB configured. Also needs Tack installed via post-install scripts. | F.9 | - | - | Y | - | associated with functest and yardstick | | ||
Line 81: | Line 81: | ||
| yardstick | - | *.* | - | - | - | - | - | | | yardstick | - | *.* | - | - | - | - | - | | ||
| bottlenecks | - | - | - | Y | - | | | | | bottlenecks | - | - | - | Y | - | | | | ||
+ | |||
Line 101: | Line 102: | ||
===== Available Scenarios ===== | ===== Available Scenarios ===== | ||
- | Available scenarios mean that the scenarios listed on below page are currently running on CI. Click Jenkins link to access console log of latest build. [[https://build.opnfv.org/ci/view/OPNFV%20Platform%20CI%20-%20Alternative%20View/|Jenkins view per PODs]] | + | Available scenarios mean that the scenarios listed on below page are currently running on CI. |
- | * bm: bare metal | + | Click Jenkins link to access console log of latest build. [[https://build.opnfv.org/ci/view/OPNFV%20Platform%20CI%20-%20Alternative%20View/|Jenkins view per PODs]] |
- | * virt: virtual | + | |
- | + | ||
- | Note: all the Apex scenario are based on Centos 7, all the scenario on compass, fuel and joid are based on Ubuntu 14.04 | + | |
- | + | ||
- | Scenario indication shall be understand as follow: [X].[Y], with | + | |
- | + | ||
- | * X = installer (apex|compass|fuel|joid) | + | |
- | * Y = scenario id as defined in the scenario section (e.g. 1 = bm + odl i.e. Arno) | + | |
- | + | ||
- | + | ||
- | ^ ID ^ Description ^ [[https://build.opnfv.org/ci/view/apex|Apex]] |^ [[https://build.opnfv.org/ci/view/compass4nfv|Compass]] |^ [[https://build.opnfv.org/ci/view/fuel|Fuel]] |^ [[https://build.opnfv.org/ci/view/joid|Joid]] |^ | + | |
- | | | | Owner | Status | Owner | Status | Owner | Status | Owner | Status | | + | |
- | | 1 | bm odl_l3-ha | Apex | [[https://build.opnfv.org/ci/job/apex-daily-master/ | Master]] | | | Fuel | [[https://build.opnfv.org/ci/job/fuel-os_odl-l3_ha-opnfv-jump-2-daily-brahmaputra/|Stable]] \\ [[https://build.opnfv.org/ci/job/fuel-os_odl-l3_ha-ericsson-pod2-daily-master/ | Master]] | Joid | [[https://build.opnfv.org/ci/job/joid-odl-intel-pod5-daily-brahmaputra/ | Stable]] \\ [[ https://build.opnfv.org/ci/job/joid-odl-intel-pod6-daily-master/| Master]] | | + | |
- | | 2 | virt odl_l3-ha | Apex | [[https://build.opnfv.org/ci/view/apex/job/apex-deploy-virtual-master/| Intel POD2]] \\ [[https://build.opnfv.org/ci/view/functest/job/functest-apex-intel-us-deploy-virtual-2-daily-master/|Functest]] \\ yardstick: | Compass | [[https://build.opnfv.org/ci/view/compass4nfv/job/compass-deploy-virtual-odl-master/|Huawei virtual odl]] | | | | | | + | |
- | | 3 | bm odl_l2-ha | | | | | Fuel | [[https://build.opnfv.org/ci/job/fuel-os_odl-l2_ha-opnfv-jump-2-daily-brahmaputra/| Stable]] \\ [[https://build.opnfv.org/ci/job/fuel-os_odl-l2_ha-ericsson-pod2-daily-master/ | Master]] | | | | + | |
- | | 5 | virt onos-ha | | | Compass | [[https://build.opnfv.org/ci/view/compass4nfv/job/compass-deploy-virtual-onos-master/|Huawei virtual onos]] | | | | | + | |
- | | 14 | bm onos-ha | | | Compass | [[https://build.opnfv.org/ci/view/OPNFV%20Platform%20CI%20-%20Alternative%20View/job/compass-onos-intel-pod8-daily-master/ | Stable ]] \\ [[https://build.opnfv.org/ci/view/OPNFV%20Platform%20CI%20-%20Alternative%20View/job/compass-onos-huawei-us-deploy-bare-1-daily-brahmaputra/| Master]] | Fuel | [[https://build.opnfv.org/ci/job/fuel-os_onos_ha-opnfv-jump-2-daily-brahmaputra/ | Stable]] \\ [[ https://build.opnfv.org/ci/job/fuel-os_onos_ha-ericsson-pod2-daily-master/ | Master]] | Joid | [[https://build.opnfv.org/ci/job/joid-onos-intel-pod5-daily-brahmaputra/ | Stable]] \\ [[https://build.opnfv.org/ci/job/joid-onos-intel-pod6-daily-master/ | Master]] | | + | |
- | | 15 | bm nosdn-ovs-ha | | | | | Fuel | [[https://build.opnfv.org/ci/job/fuel-os_ovs_ha-opnfv-jump-2-daily-brahmaputra/|Stable]] \\ [[https://build.opnfv.org/ci/job/fuel-os_ovs_ha-ericsson-pod2-daily-master/ | Master]] | | | | + | |
- | | 16 | bm nosdn-kvm-ha | | | | | Fuel | [[https://build.opnfv.org/ci/view/OPNFV%20Platform%20CI%20-%20Alternative%20View/job/fuel-os_kvm_ha-opnfv-jump-2-daily-brahmaputra/|Stable]] \\ [[https://build.opnfv.org/ci/view/OPNFV%20Platform%20CI%20-%20Alternative%20View/job/fuel-os_kvm_ha-ericsson-pod2-daily-master/ | + | |
- | /|master]] | | | | + | |
- | | 17 | bm nosdn-ovs_kvm-ha | | | | | Fuel | [[https://build.opnfv.org/ci/view/OPNFV%20Platform%20CI%20-%20Alternative%20View/job/fuel-os_kvm_ovs_ha-opnfv-jump-2-daily-brahmaputra/|Stable]] \\ [[https://build.opnfv.org/ci/view/OPNFV%20Platform%20CI%20-%20Alternative%20View/job/fuel-os_kvm_ovs_ha-ericsson-pod2-daily-master/|master]] | | | | + | |
Line 132: | Line 112: | ||
NB: each scenario needs a leader.. | NB: each scenario needs a leader.. | ||
- | ^ ID ^ Description ^ [[https://build.opnfv.org/ci/view/apex|Apex]] |^ [[https://build.opnfv.org/ci/view/compass4nfv|Compass]] |^ [[https://build.opnfv.org/ci/view/fuel|Fuel]] |^ [[https://build.opnfv.org/ci/view/joid|Joid]] || | + | ^ ID ^ Description ^ [[https://build.opnfv.org/ci/view/apex|Apex]] |^ [[https://build.opnfv.org/ci/view/compass4nfv|Compass]] |^ [[https://build.opnfv.org/ci/view/fuel|Fuel]] |^ [[https://build.opnfv.org/ci/view/joid|Joid]] || |
- | | | | Owner | Status | Owner | Status | Owner | Status | Owner | Status | | + | | | | Owner | Status | Owner | Status | Owner | Status | Owner | Status | |
- | | 1 | bm odl_l3-ha | Apex | | Compass | | Fuel | | | | | + | | 1 | bm odl_l3-ha | Apex | | Compass | | Fuel | | | | |
- | | 2 | virt odl_l3-ha | | | | | Fuel | | Joid | | | + | | 2 | virt odl_l3-ha | | | | | Fuel | | Joid | | |
- | | 3 | bm odl_l2-ha | IPv6 | GENESIS-72 to bypass the gap in ODL Lithium that doesn't support IPv6 L3 Routing | IPv6 | see Apex comment | | | IPv6 | see Apex comment | | + | | 3 | bm odl_l2-ha | IPv6 | GENESIS-72 to bypass the gap in ODL Lithium that doesn't support IPv6 L3 Routing | IPv6 | see Apex comment | | | IPv6 | see A.3 note; dropped for B-release | |
- | | 4 | virt odl_l2-ha | IPv6 | see Apex comment on scenario 3 | IPv6 | see Apex comment on scenario 3 | IPv6 | see Apex comment on scenario 3 | IPv6 | see A.3 note | | + | | 4 | virt odl_l2-ha | IPv6 | see Apex comment on scenario 3 | IPv6 | see Apex comment on scenario 3 | IPv6 | see Apex comment on scenario 3 | IPv6 | see A.3 note; dropped for B-release | |
- | | 5 | virt onos-ha | ONOSFW | | | | ONOSFW | | ONOSFW | | | + | | 9 | virt ODL Beryllium with SFC, GBP, and OVSDB SouthBound. OVS with NSH. | Apex | | | | Fuel/SFC | | | | |
- | | 9 | virt ODL Beryllium with SFC, GBP, and OVSDB SouthBound. OVS with NSH. | | | | | Fuel/SFC | | | | | + | | 10 | bm VSPERF | | | | | Fuel | | | | |
- | | 10 | bm VSPERF | | | | | Fuel | | | | | + | | 11 | bm, nosdn-dpdk_ovs * Ubuntu 14.0.4 * No ODL Backend * networking-ovs-dpdk mech driver * libvirt > 1.2.10 * Qemu 2.1+ * One NIC must be provided solely for OVS-DPDK for the private network (and I guess in case DVR is enabled two interfaces needs to be reserved – Public and private - not sure yet). | | | | | OVSNFV | FuncTest tests only (initially) | | | |
- | | 11 | bm, nosdn-dpdk_ovs | | | | | OVSNFV | FuncTest tests only (initially) | | | | + | | 12 | bm, Ubuntu 14, ODL with VPN Service enabled, BGPVPN with ODL backend | | | | | Fuel/SDNVPN | | | | |
- | | 12 | bm, Ubuntu 14, ODL with VPN Service enabled, BGPVPN with ODL backend | | | | | Fuel/SDNVPN | | | | | + | | 13 | virt, Ubuntu 14, ODL with VPN Service enabled, BGPVPN with ODL backend | | | | | Fuel/SDNVPN | | | | |
- | | 13 | virt, Ubuntu 14, ODL with VPN Service enabled, BGPVPN with ODL backend | | | | | Fuel/SDNVPN | | | | | + | | 15 | bm nosdn-ovs-ha | | dropped for B-release | | dropped for B-release | | dropped for B-release | | dropped for B-release | |
+ | | 16 | bm nosdn-kvm-ha | | dropped for B-release | | dropped for B-release | | dropped for B-release | | dropped for B-release | | ||
+ | | 17 | bm nosdn-ovs_kvm-ha | | dropped for B-release | | dropped for B-release | | dropped for B-release | | dropped for B-release | | ||
+ | | 18 | virt os-opencontrail-nofeature-ha | Apex | | Compass | dropped for B-release | N/A | | JOID | dropped for B-release | | ||
+ | | 19 | bm os-opencontrail-nofeature-ha | Apex | | Compass | dropped for B-release | N/A | | JOID | dropped for B-release | | ||
+ | | 20 | virt, Ubuntu 14, ONOS with OpenStack neutron-l3-agent being disabled | | | ONOSFW | | | | | | | ||
+ | | 21 | bm, Ubuntu 14, ONOS with OpenStack neutron-l3-agent being disabled | | | ONOSFW | | | | | | | ||
+ | | 22 | virt, CentOS 7, with ONOS neutron-l3-agent being disabled | ONOSFW | | | | | | | | | ||
+ | | 23 | virt, Ubuntu 14, ONOS with OpenStack neutron-l3-agent being disabled | | | | | ONOSFW | | ONOSFW | | | ||
+ | |||
Line 154: | Line 144: | ||
* X.9: virt, Centos/Ubuntu 14, | * X.9: virt, Centos/Ubuntu 14, | ||
* X.10: bm, Centos 7/Ubuntu 14, VSPERF | * X.10: bm, Centos 7/Ubuntu 14, VSPERF | ||
+ | |||
+ | ====== Scenario and Jenkins Job Naming Scheme ======= | ||
+ | |||
+ | It has been proposed to construct and align short scenario names in order to ease the Releng and Test Projects work to proceed with CI work and release verification. | ||
+ | |||
+ | An important aspect to point out is that even though short scenario names can give an idea regarding what a specific scenario is about, it is always important/necessary to look at and work with actual configuration files. | ||
+ | |||
+ | This naming convention is only valid for short scenario names and the details (config files, etc.) are handled within Genesis. | ||
+ | |||
+ | Both naming scenarios and Jenkins jobs have been touched upon during Daily Release Meeting on 2016-01-13 and there were no objections. See [[http://ircbot.wl.linuxfoundation.org/meetings/opnfv-release/2016/opnfv-release.2016-01-13-14.30.html|this link]] for minutes. | ||
+ | |||
+ | ===== Scenario Naming ===== | ||
+ | |||
+ | Short scenario names should follow the scheme below. | ||
+ | |||
+ | **os-[controller]-[feature]-[mode]-[option]** | ||
+ | |||
+ | Details of the fields are | ||
+ | * **os**: mandatory | ||
+ | * possible value: os | ||
+ | * please note that this field is needed in order to select parent jobs to list and do blocking relations between them. | ||
+ | * **[controller]**: mandatory | ||
+ | * example values: nosdn, ocl, odl, onos | ||
+ | * **[feature]**: mandatory | ||
+ | * example values: nofeature, kvm, ovs | ||
+ | * **[mode]**: mandatory | ||
+ | * possible values: ha, noha | ||
+ | * **[option]**: optional | ||
+ | * used for the scenarios those do not fit into naming scheme. | ||
+ | * optional field in the short scenario name should not be included if there is no optional scenario. | ||
+ | |||
+ | Examples are | ||
+ | * os-nosdn-kvm-noha | ||
+ | * os-odl_l2-nofeature-ha | ||
+ | * os-onos-kvm_ovs-noha | ||
+ | |||
+ | Please note that the dashes (-) are the separator between fields. | ||
+ | Underscores (_) can be used for separating the values/words if a field contains more than one value (ie kvm_ovs). | ||
+ | |||
+ | **This scenario naming scheme is currently in use/followed by compass, fuel, and joid.** | ||
+ | |||
+ | ===== Jenkins Job Structure ===== | ||
+ | |||
+ | Jenkins jobs use Freestyle project type. | ||
+ | |||
+ | Please take a look at the examples from [[https://build.opnfv.org/ci/job/fuel-os-odl_l2-nofeature-ha-opnfv-jump-2-daily-brahmaputra/ | this]] or [[https://build.opnfv.org/ci/job/joid-os-ocl-nofeature-ha-intel-pod6-daily-master/ | this]] link to see job structuring. | ||
+ | |||
+ | In this new job structure, there is 1 main job per scenario-pod-branch, controlling the executions of deploy, functest, and yardstick jobs. Main jobs are used for setting triggers, triggering jobs manually or automatically. | ||
+ | |||
+ | There is 1 job for each of the deploy, functest, and yardstick for pod-branch and these jobs take scenario short names from parent job and do what they are supposed to do with given scenario on the POD. | ||
+ | |||
+ | **This job structure is currently in use/followed by compass, fuel and joid.** | ||
+ | |||
+ | ===== Jenkins Job Naming ===== | ||
+ | |||
+ | Names of Jenkins Jobs require alignment as well in order to ease the triggering, configuration, management, and checking logs. | ||
+ | |||
+ | The jobs on Jenkins should be named according to below naming scheme. | ||
+ | |||
+ | * **Parent/Main Jobs: [installer]-[scenario]-[pod]-[loop]-[branch]** | ||
+ | * [installer]: apex, compass, fuel, joid | ||
+ | * [scenario]: only the scenario names that fit into scenario naming scheme above. | ||
+ | * [pod]: all the PODs that are assigned to specific installer and currently connected to Jenkins. | ||
+ | * [loop]: daily | ||
+ | * [branch]: master or brahmaputra | ||
+ | * **Deploy Jobs: [installer]-deploy-[pod]-[loop]-[branch]** | ||
+ | * [installer]: apex, compass, fuel, joid | ||
+ | * [pod]: all the PODs that are assigned to specific installer and currently connected to Jenkins. | ||
+ | * [loop]: daily | ||
+ | * [branch]: master or brahmaputra | ||
+ | * **Test Jobs: [testproject]-[installer]-[pod]-[loop]-[branch] ** | ||
+ | * [testproject]: functest, yardstick, bottlenecks, qtip | ||
+ | * [installer]: apex, compass, fuel, joid | ||
+ | * [pod]: all the PODs that are assigned to specific installer and currently connected to Jenkins. | ||
+ | * [loop]: daily or suite | ||
+ | * [branch]: master or brahmaputra | ||
+ | |||
+ | Example Parent/Main Job names are | ||
+ | * [[https://build.opnfv.org/ci/job/joid-os-ocl-nofeature-ha-intel-pod6-daily-master/|joid-os-ocl-nofeature-ha-intel-pod6-daily-master]] | ||
+ | * [[https://build.opnfv.org/ci/job/fuel-os-odl_l2-nofeature-ha-opnfv-jump-2-daily-brahmaputra/|fuel-os-odl_l2-nofeature-ha-opnfv-jump-2-daily-brahmaputra]] | ||
+ | |||
+ | Example Deploy Job names are | ||
+ | * [[https://build.opnfv.org/ci/job/joid-deploy-intel-pod6-daily-master/|joid-deploy-intel-pod6-daily-master]] | ||
+ | * [[https://build.opnfv.org/ci/job/fuel-deploy-opnfv-jump-2-daily-brahmaputra/|fuel-deploy-opnfv-jump-2-daily-brahmaputra]] | ||
+ | |||
+ | Example Test Job names are | ||
+ | * [[https://build.opnfv.org/ci/job/yardstick-joid-intel-pod6-daily-master/|yardstick-joid-intel-pod6-daily-master]] | ||
+ | * [[https://build.opnfv.org/ci/job/functest-fuel-opnfv-jump-2-daily-brahmaputra/|functest-fuel-opnfv-jump-2-daily-brahmaputra]] | ||
+ | |||
+ | See all the OPNFV Platform CI jobs on [[https://build.opnfv.org/ci/view/OPNFV%20Platform%20CI%20-%20Alternative%20View/|this link]]. | ||
+ | |||
+ | **This Jenkins Job naming scheme is currently in use/followed by compass, fuel and joid.** | ||
====== Brahmaputra Testing using non-CI production PODs ====== | ====== Brahmaputra Testing using non-CI production PODs ====== | ||
Line 173: | Line 255: | ||
| QTIP | QTIP test suite | Dell POD3 | Fuel | Daily Jenkins Job for QTIP test suite. | | | QTIP | QTIP test suite | Dell POD3 | Fuel | Daily Jenkins Job for QTIP test suite. | | ||
| VSPERF | VSPERF test suite | Intel POD3 | None | Daily Jenkins Job for VSPERF test suite. | | | VSPERF | VSPERF test suite | Intel POD3 | None | Daily Jenkins Job for VSPERF test suite. | | ||
+ | | bottlenecks | rubbos and VSTF test suite | Huawei | Compass | Daily Jenkins Job; Responsible for testing: bottlenecks; Responsible for documentation: bottlenecks | |