Wiki

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== Status of OPNFV Lab Resources ======

This Pharos page explains the current status of **Labs** for support of development, test and CI activities. For future we plan to have dashboards with //live// information. 

//Tracking actual capabilities, status and ongoing usage of our diverse community lab resources is non-trivial ... however its needed to support projects and releases with accurate planning. To achieve this we need easy visibility of our lab resources, their capabilities as well as current and past usage.// 

Information on **Linux Foundation lab** access and usage is found here https://wiki.opnfv.org/pharos/lf_lab 

Information on project needs from labs was collected from lab owners by a survey (closed 16 Oct 2015) ... https://www.surveymonkey.com/results/SM-CYC35WH2


  * 11 community labs currently are used by projects and releng activities
  * Labs that are not Pharos compliant may still provide servers as dev resources
  * Some labs still need to publish developer access process and guides
  * Some labs are not yet setup under Jenkins control
  * Labs that suffer from low bandwidth or unstable connections can still be used for specific project activities

===== Status of production CI resources for release B =====

  * List of resources that are connected to OPNFV Jenkins for the use of the CI: https://wiki.opnfv.org/pharos/hw_resource_allocation
  * Information on test combinations and allocations / combinations 
    * https://wiki.opnfv.org/brahmaputra_testing_page
    * https://etherpad.opnfv.org/p/brahmaputra_release_testing 
  * Configuration for Pod-layout/Build/Deploy/Test https://wiki.opnfv.org/genesis/configuration-files-discussion

See OPNFV CI Process Proposal for Brahmaputra https://wiki.opnfv.org/octopus/brahmaputra_ci_process


The references "stable" and "latest" mean the branches. PODs marked as "stable" will run against //stable/brahmaputra// branch and PODs marked as "latest" will run agains //master// branch. The testing will be done against both branches but the results from stable/brahmaputra counts when it comes to release candidate evaluation.

^ POD             ^ Installer              ^ Stable/Latest  ^ POD/Resources available & Tested with installer                                                                                                      ^  POD has Jenkins configured for automated deploys and tested  ^ Notes                                                                                                                  ^
^ LF POD 1        | Apex                   | stable         | Yes - Fully operational \\ [[https://build.opnfv.org/ci/view/apex/job/apex-deploy-baremetal-master/|Jenkins: apex-deploy-baremetal-master]]          | Jenkins: Yes, Deploy: No, Test: No                            | Still running against latest                                                                                           |
^ Huawei SC POD1  | Compass                | stable         | Yes - Fully operational                                                                                                                              | Jenkins: Yes, Deploy: No, Test: No                            | N/A                                                                                                                    |
^ Ericsson POD2   | Fuel                   | stable         | Yes - Fully operational                                                                                                                              | Jenkins: Yes, Deploy: Yes, Test: Yes                          | N/A                                                                                                                    |
^ Intel POD 5     | Joid                   | stable         | Yes - Fully operational \\ [[https://build.opnfv.org/ci/job/joid-odl-intel-pod5-daily-brahmaputra/|Jenkins: joid-odl-intel-pod5-daily-brahmaputra]]  | Jenkins: Yes, Deploy: Yes, Test: Yes                          | N/A                                                                                                                    |
^ Intel POD 2     | Apex moved to POD 7    | dev            | Yes - Fully operational                                                                                                                              | Jenkins: No,  Deploy: No, Test: No                            | Apex moved to a new environment (Intel POD 7). POD 2 is being used by CPERF                                            |
^ Intel POD 7     | Apex                   | latest         | Yes - Fully operational. Moved Apex environment from POD 2. https://jira.opnfv.org/browse/PHAROS-90                                                  | Jenkins: Yes,  Deploy: No, Test: No                           | Apex is currently using LF POD 1 for all jobs. Waiting for Apex to verify issues experienced with POD 2 are resolved.  |
^ Intel POD 8     | Compass                | latest         | Yes - Fully operational                                                                                                                              | Jenkins: Yes,  Deploy: No, Test: No                           | N/A                                                                                                                    |
^ LF POD2         | Fuel                   | latest         | Yes - Fuel8 reconfig is ongoing                                                                                                                      | Jenkins: Yes, Deploy: Yes, Test: Yes                          | N/A                                                                                                                    |
^ Intel POD 6     | Joid                   | latest         | Yes - Fully operational \\ [[https://build.opnfv.org/ci/job/joid-odl-intel-pod6-daily-master/|Jenkins: joid-odl-intel-pod6-daily-master]]            | Jenkins: Yes, Deploy: Yes, Test: Yes                          | N/A                                                                                                                    |
^ Intel POD 4     | x5 Standalone Servers  | N/A            | Yes - Fully operational (Investigating increasing Internet bandwidth which is an issue for downloading large images frequently)                      | Jenkins: Yes, Deploy: N/A, Test: N/A                          | Build & virtual deploy resources                                                                                       |
^ LF POD 3        | x4 Standalone servers  | N/A            | No  - WIP with 1 server                                                                                                                              | Jenkins: No,  Deploy: N/A, Test: N/A                          | Build & virtual deploy resources \\ 1 server is reserved for Joid virtual and WIP to take it into operation            |










See also: [[https://build.opnfv.org/ci/|Jenkins OPNFV Platform CI main page]]

===== Production CI Resources Reservation =====

Test projects raised the need to access and work on CI Production PODs for troubleshooting purposes. The table below tries to start the discussion around how this can be done.

In ideal case, the PODs should be left in the state as it was failed but due to large number of scenarios, this will probably not be possible most of the times so that's why it is important to leave time for test projects on PODs to have the troubleshooting done on the POD where the problem occurred.

All the communication regarding PODs should be done in respective IRC Channels: #opnfv-testperf, #opnfv-yardstick, #opnfv-apex, #opnfv-fuel, #opnfv-compass, #opnfv-joid.

^ POD             ^ Installer  ^ Branch  ^ Reserved for CI      ^ Reserved for Troubleshooting                                                                                          ^ Notes                                                            ^
^ LF POD1         | Apex       | master  | 01:00UTC - 13:00UTC  | 16:00UTC - 01:00UTC    \\ Booked for Yardstick: 16:00 UTC - 18:00UTC \\ Booked for Functest: 18:00 UTC - 20:00 UTC    |                                                                  |
^ Huawei SC POD1  | Compass    | stable  | 10:00UTC - 22:00UTC  | 22:00UTC - 10:00UTC    \\ Booked for Yardstick: 02:00 UTC - 04:00 UTC                                                 |                                                                  |
^ LF POD2         | Fuel       | stable  | 17:00UTC - 08:00UTC  | 11:00UTC - 17:00UTC    \\ Booked for Yardstick: 13:00 UTC - 15:00 UTC  \\ Booked for Functest: 15:00 UTC - 17:00 UTC  | **2016-01-28: Dedicated for Fuel8 Uplift until further notice**  |
^ Intel POD5      | Joid       | stable  | 17:00UTC - 05:00UTC  | 05:00UTC - 17:00UTC    \\ Booked for Yardstick: 14:00 UTC - 16:00 UTC  \\ Booked for Functest: 16:00 UTC - 17:00 UTC  |                                                                  |
^ Intel POD2      | Apex       | N/A     | N/A                  | N/A                                                                                                                   |                                                                  |
^ Intel POD8      | Compass    | master  | 22:00UTC - 10:00UTC  | 10:00UTC - 22:00UTC    \\ Booked for Yardstick: 10:00 UTC - 12:00 UTC  \\ Booked for Functest: 13:00 UTC - 15:00 UTC  |                                                                  |
^ Ericsson POD2   | Fuel       | master  | 05:00UTC - 20:00UTC  | 20:00UTC - 05:00UTC    \\ Booked for Yardstick: 20:00 UTC - 22:00 UTC                                                 | **2016-01-28: Dedicated for Fuel8 Uplift until further notice**  |
^ Intel POD6      | Joid       | master  | 05:00UTC - 17:00UTC  | 17:00UTC - 05:00UTC    \\ Booked for Yardstick: 17:00 UTC - 19:00 UTC \\ Booked for Functest: 19:00 UTC - 20:00 UTC   |                                                                  |



Notes:
  * It is assumed that the CI will run 3 scenarios per Installer/POD until the code freeze.
  * The reservations are done based on the locations of active contributors of installer and test projects.
  * The table tries to leave 1 POD per installer for troubleshooting purposes mainly to be used by test projects.
  * Approximate time to run full loop per scenario (deploy + functest + yardstick). These times are taken out from Jenkins jobs and might obviously be longer or shorter.
    * Apex   : 4 hours per scenario
    * Compass: 4 hours per scenario 
    * Fuel   : 5 hours per scenario
    * Joid   : 4 hours per scenario

Once the code freeze hits, time to do troubleshooting on the PODs dedicated for stable branch will go down drastically given the number of scenarios. Due to this, the troubleshooting work needs to be done on PODs running against master if the problem in question can be troubleshooted there.

It is also possible that installer teams might prefer test teams to use a certain POD longer than what is stated above and test teams get more time to work on certain POD for troubleshooting. But this should not result in not running CI for the branch on the POD in question or else this will increase the possibility of introducing faults to stable branch. The PODs must not turn into development resources!

Timezones
  * Beijing: UTC+8
  * CET    : UTC+1
  * PDT    : UTC-7

===== Community Lab Resources - Operational =====


Labs listed here are operational and being actively used for either development (features and/or tools), test (feature testing or dedicated to test project), production (dedicated with 24/7 SLA). Note a community lab can provide one or more of these capabilities.

For allocation of PODs for testing release B (there are 4 installers) see https://wiki.opnfv.org/brahmaputra_testing_page 



^  Lab                    ^  Pharos compliant PODs  ^  Stand-alone servers  ^  Type / Usage                                                                                                                                                                                                                                                                                                   ^  Production CI operational  ^  Project CI (Test) operational  ^  Developer resources with access guide and process published  ^
| China Mobile (Beijing)  | 1                       |                       | POD 1 (Fuel) - dev for Functest and Yardstick tools                                                                                                                                                                                                                                                             | No                          | Yes                             | Yes                                                           |
| Dell (Santa Clara)      | 3                       |                       | POD 1 - available for CI allocation; POD 2 Qtip dev and test                                                                                                                                                                                                                                                    | No (TBD)                    | Yes                             | Yes                                                           |
| Enea                    | 1                       |                       | HA server config with ARM servers                                                                                                                                                                                                                                                                               | No                          | No                              | No                                                            |
| Ericsson (Montreal)     | 2                       | 4 Blades              | POD 1 (Fuel) - Yardstick testing (proj CI); POD 2 (Fuel) - production CI; N servers used for Dev based on Demand                                                                                                                                                                                                | Yes                         | Yes                             | Yes                                                           |
| Huawei (Santa Clara)    | 2                       | 2                     | POD 1 (Compass) - production CI; POD 2 - available for dev allocation; 2x servers - VMs available for dev allocation                                                                                                                                                                                            | No (TBD)                    | Yes                             | No                                                            |
| Huawei (Xia-an)         | 2                       | 4                     | POD 1 - available for proj allocation; CI POD 2 - available for proj CI allocation; 4x servers - dev for Compass                                                                                                                                                                                                | No                          | No                              | No                                                            |
| Intel (Portland)        | 9                       | 3 (POD 3 - VSPERF)    | POD 1 - Apex dev/test; POD 2 - Apex production CI; POD 3 - VSPERF test environment with Ixia traffic generator, OVSNFV, kvm4nfv; POD 4 - CI builds and virt deploys; POD 5 (JOID) - production CI; POD 6 - JOID dev/test; POD 7 - Fuel dev/test; POD 8 - Compass production CI; POD 9 - STORPERF / Bottlenecks  | Yes                         | Yes                             | Yes                                                           |
| Orange (Paris)          | 1                       |                       | POD 1 (Fuel) - used for PoCs (vCPE), Functest dev, proj CI                                                                                                                                                                                                                                                      | No                          | Yes                             | Yes                                                           |
| Orange (Lanion)         | 1                       |                       | POD 1 (odentified as Orange POD 2 in jenkins) - dev for JOID and Functest, temp use as CI POD for joid for B release                                                                                                                                                                                            | Yes                         | Yes                             | No                                                            |
| Spirent (San Jose)      |                         | 12                    | Individual servers are used for various dev activities                                                                                                                                                                                                                                                          | No                          | No                              | Yes                                                           |
| ZTE (Nan Jing)          | 1                       |                       | POD 1 - dev/test for Functest and Yardstick                                                                                                                                                                                                                                                                     | No                          | No                              | Yes                                                           |













===== Community Lab Resources - Upcoming =====




The following organization are actively working on providing additional OPNFV community labs ...  

  1. Cable labs – currently have 2 PODs that are currently private resources  
  2. CENGN – waiting on hardware to complete build out
Wiki

User Tools

Site Tools

Page Tools
