==== Test and Performance Weekly Meeting ====

This weekly meeting is for topics common across projects focused on testing … this is a good place to start if you want to get high-level status of the individual projects and learn about hot topics related to testing OPNFV
  * Test projects should try to have a rep attend this meeting
  * New test related projects should be discussed in this forum to gain feedback from the OPNFV testing community

  * Agenda varies but typically is structured ... 
    * 15 min report out / sharing by projects (high level status / plans)
    * 20 min discussion on test project collaboration topic/s 
    * 20 min – share new test projects or other test topics

  * There is a Test and Performance Operational Meeting immediately after (topics include dashboards, tools, KPIs, traffic profiles, etc.) See https://wiki.opnfv.org/meetings


  * When: **Every Thursday 7:00-8:00 <del>PDT</del> PST (<del>14:00-15:00</del> 15:00-16:00 UTC)**
    * [[http://www.timeanddate.com/worldclock/fixedtime.html?msg=OPNFV-Test and performance &iso=20151105T15&ah=1|Your local time]] and Every two weeks on Wednesday 8h30 UTC (Asian/European slot)

    * Meeting notes (MeetBot) ... http://meetbot.opnfv.org/meetings/opnfv-meeting/
    * **Etherpad:** https://etherpad.opnfv.org/p/testperf

    * **IRC channel:** ''#opnfv-meeting'' @ Freenode

    * Join the meeting from your computer, tablet or smartphone 
      * Thursday GTM: https://global.gotomeeting.com/join/305553637
        * Access Code: 305-553-637
        * Phone numbers: https://global.gotomeeting.com/305553637/numbersdisplay.html
          * United States : +1 (626) 521-0010
          * Canada : +1 (647) 497-9371
          * Finland : +358 (0) 942 41 5770
          * France : +33 (0) 182 880 159
          * Germany : +49 (0) 692 5736 7208
          * Sweden : +46 (0) 852 500 179
          * ...
      * Wednesday GTM: https://global.gotomeeting.com/join/644444853
        * Access Code: 644-444-853  
        * Phone numbers: https://global.gotomeeting.com/644444853/numbersdisplay.html
          * Germany : 0 800 723 5274
          * China : 4007 160008
          * Korea, République de: 0806180880
          * Spain: 800 900 593
          * Russia: 8 800 100 6216
          * Finland: 0 800 94473
          * France: 0 805 541 052
          * India: 000 800 100 8227
          * Japan: 0 120 242 200
          * Sweden: 0 200 330 924

----

**Useful links ...**

[[:testing| Testing overview page with links to testing projects]]

[[pharos| Pharos project page]]

------------------------------------------

**__//Upcoming Agenda Topics//__ ...**



==== Proposed Agenda - March 31st, 2016 (55 min) ====

  * ?


----


**__//Meetings//__ ...**

==== Agenda - March 24th, 2016 (55 min) ====

    * Plugfest
    * Resources for release C


==== Agenda - March 10th, 2016 (55 min) ====

    * Benchmarking VNF/VNFIs - 30 min (Al)
    * Storperf - 15 min (Mark)
    * Pharos - 10 min (Trevor)


==== Agenda - March 3rd, 2016 (55 min) ====

  * Continue from last week with what Test Projects delivered in release B ...

    * VSPERF - 15 min (Maryam)
    * Bottlenecks - 15 min (Hongbo / Yaoguang)
    * Storperf - 15 min (Mark)

    * Time allowing -> Release C common topics across test projects including test cycles, test strategy, infrastructure, coverage, automation, etc.


==== Agenda - February 25th, 2016 (55 min) ====

  * Test projects - what was delivered in release B ...

    * Qtip demo - 15 min
    * Functest - 15 min
    * Yardstick - 15 min
    * VSPERF - 15 min
    * Bottlenecks - 15 min
    * Storperf - 15 min

May run over into Yardstick meeting (or continue next week)

==== Agenda - February 18th, 2016 (55 min) ====


  * Plugfest test-cases (30 min)
  * Testing plans for release C (25 min)



==== Agenda - February 4th, 2016 (55 min) ====

//Meeting Cancelled
//

==== Agenda - January 28th, 2016 (55 min) ====


  - Plugfest testplan
  - Brahmaputra test readiness status
  - Test success criteria (now that scenarios clarified)
  - Plan for the visualization of the results (alternative to LF dashboard)
  - Organization Asian/European slot
  - Troubleshooting/debugging slot


==== Agenda - January 21st, 2016 (55 min) ====

  - Brahmaputra release (35 min)
     * https://wiki.opnfv.org/brahmaputra_testing_page
     * https://wiki.opnfv.org/releases/brahmaputra/release_plan#brahmaputra_release_participating_projects
    - Test project status (5 min)
      * yardstick, functest, bottleneck, vsperf, qtip
      * Storperf     
      * Companion projects
    - Slots for testing / responsibility to trigger redeployments (5 min)
    - Migration of test DB (5 min)
    - Results display / dashboards (10 min)
    - Troubleshooting organization/planning (10 min)
      * Success criteria / SLA 
      * https://wiki.opnfv.org/vping_brahmaputra_page
      * https://wiki.opnfv.org/tempest_brahmaputra_page 
  - Test meeting organization (5 min)
  - Test project participation in Dovetail / plugfest (10 min)

==== Agenda - January 14th, 2016 (55 min) ====

  * Test dashboard demo (Marion, Andrew - LF) - 15 min

  * Update on test scenarios - 15 min
    * https://wiki.opnfv.org/brahmaputra_testing_page

  * Test artifacts for Rls B - 15 min
    * Test results
    * Test results sample / template
    * Other test documents?

  * Test projects status updates - as time allows (few min per project)
    * https://wiki.opnfv.org/brahmaputra_testing_page
    * https://wiki.opnfv.org/releases/brahmaputra/release_plan#brahmaputra_release_participating_projects
      * pharos
      * yardstick
      * functest
      * bottleneck
      * vsperf
      * qtip


==== Agenda - January 7th, 2016 (55 min) ====

  * Test dashboard - 10 min
    * Update on portal prototype status and roadmap (Marion, Andrew - LF)
    * Next steps and anything needed from other projects (Morgan)

  * Test projects status for Rls B - 15 min (2 min per project)
    * https://wiki.opnfv.org/brahmaputra_testing_page
    * https://wiki.opnfv.org/releases/brahmaputra/release_plan#brahmaputra_release_participating_projects
      * pharos
      * yardstick
      * functest
      * bottleneck
      * vsperf
      * qtip

  * Test Scenarios for rls B - 15 min
    * Review ... https://wiki.opnfv.org/brahmaputra_testing_page
    * Clarify how test projects are managed from CI perspective
    * Questions on CI integration of feature projects / testing

  * Documentation for Rls B - expectations and status - 15 min
    * Test result documentation - define and agree what to document
    * Pharos documents / templates




==== Agenda - December 17th, 2015 (55 min) ====

  * Status of the test projects (Morgan) 20 min
    * Functest ... CI labs - Fuel, Joid, Compass, Apex
    * Yardstick
    * Other test projects
  * Retro planning attempt for B-Release from now to .... 20min
  * Showcasing OPNFV test results using Kibana (Frank Brockners and Peter Bandzi) - 15 min


==== Agenda - December 10th, 2015 (55 min) ====

As part of test meeting, we’ll also continue the discussion on common configuration files – with the plan to evolve things to closure asap.

Agenda items for this topic include:
  * finalize common inventory config file: https://gerrit.opnfv.org/gerrit/#/c/4079/ 
  * review common network config proposal (see Jonas proposal) - https://gerrit.opnfv.org/gerrit/#/c/4095/ 
  * review common install/scenario config proposal (Tim will create one today – taking into account input from Jonas, Narinder, Weidong).
  * Updates on feature integration guides for installers (i.e. a HOWTO integrate your feature into a specific installer) – See the section on https://wiki.opnfv.org/genesis 




==== Agenda - December 3rd, 2015 (55 min) ====
  * **This meeting slot will be used for discussion on release plan for build/deploy/test **
  * CI needs to know which configuration to run and which tests to execute
  * Need common configuration files for ...
    * Build config
    * Deploy config
    * POD config
    * Test config
  * Question 0: what are the combinations of installers/controllers/other sw component versions (compatibility matrix)?
  * Question 1: how many deploy/test environments are needed?
  * Question 2: what community labs will volunteer to provide needed resources?


==== Agenda - November 26th, 2015 (55 min) ====
  * Release D milestone - status for test projects (10 min)
  * Dashboard strategy and plan (45 min)

==== Agenda - November 19th, 2015 (55 min) ====
  * Digital Rebar demo (this will be recorded) (30 min)
  * Summit discussions re. lab resources (allocation and open issues) (10 min)
  * Yardstick update (10 min)
  * Asia meeting slot (5 min)

==== Agenda - November 5th, 2015 (55 min) ====
  * Preparing/organizing test related activities at OPNFV Summit next week (30 min)
  * OpenStack Summit last week - what was of interest to the testing group? (10 min)
  * Update on Functest (15 min)

==== Agenda - October 22nd, 2015 (55 min) ====
  * Test projects - announcements (5 min)
  * MAAS PoC demo (30 min) - will be recorded
  * Test lab requirements https://wiki.opnfv.org/pharos_rls_b_spec (10 min)
  * Project requirements for lab resources https://wiki.opnfv.org/projectreqonpharos (10 min) 

==== Agenda - October 15th, 2015 (55 min) ====
  * Test projects - notices / sync-up (5 min)
  * Test definitions https://wiki.opnfv.org/testing (10 min)
  * OPNFV tester journey https://wiki.opnfv.org/_media/devtest_journey.pdf (10 min)
  * Test project requirements for Pharos labs (30 min)
      * Types of labs
      * Collecting project requirements https://wiki.opnfv.org/projectreqonpharos 
      * Pharos lab ready criteria (test perspective)
      * etc.


==== Agenda - October 8th, 2015 (55 min) ====
  * Rally: presentation of the new features  (20 min)
  * Pharos: Test project support (20 min)
  * Yardstick Docker discussion (15 min)
  * Individual test projects status (time allowing)

==== Agenda - October 1st, 2015 (55 min) ====
  * Test projects status (10 min)
  * Demo & discussion Test Dashboard (15 min)
  * Yardstick demo - virtual Traffic Classifier (30 min)

==== Agenda - September 24th, 2015 (55 min) ====
  * Test projects status (10 min)
  * Discussions/decisions on the test deliverable for R2 (20 min)
  * Discussions/decisions on the dependencies towards features and other test projects (20 min)

==== Agenda - September 17th, 2015 (55 min) ====
  * Bottlenecks status (5 min) - Hongbo
  * Transformer status (5 min) - Michael
  * ODL related testing activities (45 min, Morgan)


==== Agenda - September 10th, 2015 (55 min) ====

  * Test projects status (only projects not represented last week) - 2 min per project

    1. Bottlenecks - Find system bottlenecks (Huawei) - Hongbo
    2. CPERF - Controller Performance Testing (Red hat) – Daniel
    3. Policytest - Network policy testing (CISCO) - Keith
    4. Affirm - validation criteria for use of OPNFV trademarks (Ericsson) – Chris
    Projects Not yet approved by TSC ...
    5. STORPERF - Storage Performance Benchmarking for NFVI (EMC) - Edgar
    6. Transformer - Carrier Grade Requirements (Ericsson) - Michael

  * Logistics for Test projects ”operational” meeting (5 min)

  * Milestone C guidelines (15 min)
    * What are the expectations from Test Projects
    * Test needs from projects and how to integrate (test specs were supposed to have been defined end of Sep)

  * Testing dashboard - update and first demo (15 min)

  * Test definitions for rls B (10 min)
    * What are we testing for each installer
    * Definition of failure / severity





==== Agenda - September 3rd, 2015 (55 min) ====

  * Test projects status - all test project leads - 20 min  
  * Yardstick - Demo of Virtual Traffic Classifier VNF (Ana, Ericsson) - 30 min
  * Qtip update (Vikram, Dell) - 15 min

**Test projects**

Projects approved by TSC
  1. Functest - Base system functionality testing (Orange) - Morgan
  2. Yardstick - Infrastructure Verification (Ericsson) – Ana
  3. VSPERF -Characterize vSwitch Performance for Telco (Intel) - Maryam
  4. Qtip - Platform Performance Benchmarking (Dell) - Vikram
  5. Bottlenecks - Find system bottlenecks (Huawei) - Hongbo
  6. CPerf - Controller Performance Testing (upstream) – Daniel Farrell
  7. Policytest - Network policy testing (CISCO) - Keith
  8. Affirm - validation criteria for use of OPNFV trademarks (Ericsson) – Chris

Projects Not yet approved by TSC ...
  9. STORPERF - Storage Performance Benchmarking for NFVI (EMC) - Edgar
  10. Transformer - Carrier Grade Requirements (Ericsson) - Michael

==== Agenda - August 27, 2015 (55 min) ====

  * Yardstick - Discussion on planned deliverables for Mobility Traffic Profile (Ana, Ericsson) - 20 min
  * VSPERF - update and demo (Maryam, Intel) - 30 min
  * Test dashboard (wiki) (Morgan, Orange) - 10 min


==== Agenda - August 20, 2015 (55 min) ====

  * Yardstick - Demo of new functionality (Ana, Ericsson) - 30 min
  * Bottlenecks - https://wiki.opnfv.org/project_proposals/bottlenecks (Hongbo Tian, Huawei) - 15 min
  * Testing dashboard (Ray, Andrew - Linux Foundation) - 10 min

---------------------------------
==== Agenda - August 13, 2015 ====

  * Show the community how MAAS can deploy Arno release of OPNFV (Narinder Gupta, Canonical)
  * Proposed Storage Benchmark project (STORPERF) https://wiki.opnfv.org/storperf (Edgar StPierre, EMC)
  * "Policy Test" project proposal discussion https://wiki.opnfv.org/policytest/policytest_project_proposal (Keith Burns)

---------------------------------

==== Agenda for August 6, 2015 ====

Meeting canceled

---------------------------------



==== Agenda for July 30, 2015 ====

Meeting canceled

---------------------------------

==== Agenda for July 23, 2015 ====

Proposed Agenda (60 min): 

**Qtip** 45min
  * Testing strategy, planning for Brahmaputra (Nauman Ahad / Wenjing Chu)

**Pharos** 5 min
  * Status update

**Functest** 5 min
  * Status Update
 
**Yardstick** 5 min
  * Status update

---------------------------------



==== Agenda for July 16, 2015 ====

Proposed Agenda (60 min): 

**Pharos** 20 min
  * Release B deliverables
  * LF support plan
  * Near term needs
  * Lab status and availability
  * MAAS follow up

**Functest** 5 min
  * Status and Update - Morgan is out this week
 
**Yardstick** 30 min
  * B-release deliverables 
  * PODs to use for test case verification
  * Definition of Done for a test case
  * SLAs for OPNFV infrastructure
  * Weekly meeting for follow up on Jira tasks

**Other topics** 5min
  * Pharos meeting at ODL Summit


---------------------------------


==== Agenda for July 9, 2015 ====

Proposed Agenda (60 min): 

**Pharos**
  * Planning (15 min)
      * Plans presented to TSC
      * Logistics for Pharos dedicated meetings
  * MAAS demo and discussion - Narinder Gupta - narinder.gupta@canonical.com (30 min)

**Functest** 
  * Status and Update (5 min)

**Yardstick**
  * Status and Update (5 min)

**Other topics**
  * Standing agenda (5 min)

---------------------------------


==== Agenda for July 2, 2015 ====

Proposed Agenda (60 min): 

**Functest**
  * Demo Functest 

**Common topics**
   * Discussion on test coverage (if time allows), see proposal in wiki: https://wiki.opnfv.org/testing 


----
==== Agenda for June 25, 2015 ====

Proposed Agenda (60 min): 

  * Discuss test strategy for Rls 2

  * See proposal for Test deliverables and strategy for release 2 ... https://wiki.opnfv.org/_media/test_strategy_v0.2.pptx



-------------------------------------------


==== Agenda for June 18, 2015 ====

Proposed Agenda (60 min): 

**Pharos**
  * No agenda topics

**Functest**
  * No agenda topics

** Yardstick** 
  - Demo of the tool
  - Discussion on use of the tool and how to serve wider needs of testing projects

-------------------------------------------


==== Agenda for June 11, 2015 ====

* Due to Arno retrospective this week meeting is changed to 14h30 UCT (7:30 am Pacific Daylight Time) and will be 30 min long

* Yardstick demo is postponed to the following week (Thursday 18th June)

Proposed Agenda (30 min): 

**Pharos**
  * No agenda topics planned

**Functest**
  * Test strategy for Release 2

** Yardstick** 
  - Demo postponed to next week

-------------------------------------------


==== Agenda for June 4, 2015 ====

PLEASE NOTE MEETING HAS BEEN CANCELLED AS A NUMBER OF ACTIVE PARTICIPANTS CANNOT ATTEND (YARDSTICK DEMO WILL BE RESCHEDULED FOR NEXT WEEK)


-------------------------------------------



==== Agenda for May 28, 2015 ====

Proposed Agenda (1hr): 

**OVS characterization test framework**
  - Overview of test tools
  - Demo
  - Status and plans

**Functest**
  * Introduction to Functest
    - Arno release status
    - Project organization
    - Git description
    - Integration with CI
    - Results analysis
    - New test case integration

** Yardstick** 
  - Status
  - Demo June 4th
  - Backlog

-------------------------------------------

==== Agenda for May 14, 2015 ====

Proposed Agenda (1hr): 

**Pharos** (15 min)
  * Revised Pharos spec for Arno 
  * Lab definitions and tracking 

**Functest** (15 min)
  * Organization format (meeting, IRC,..)
  * Beyond R1
    * new test cases
    * new tools
    * analytics on bench results
    * synchro with ETSI Test

** Yardstick** (30 min)
  * Questions / discussion from project proposal
  * Project scope & synergies with other test projects
  * Project members
  * Organization format (meeting, IRC,..)


------------------------------

==== Agenda for May 7, 2015 ====

Meeting this week will be 30 min to accommodate Arno readiness call at 7:30 PDT. We will likely only have time for Yardstick topics in the call.

**Pharos**
  * No agenda items this week

**Functest**
  * No agenda items this week

**Yardstick (30 min)**
  * Project scope & synergies with other test projects
  * Project members
  * Organization format (meeting, IRC,..)

-------------------------

==== Agenda for April 30, 2015 ====

**Functest**
  * Release 1
  * Validation new committers
  * Review organization format (meeting, IRC,..)
  * Beyond R1
    * new test cases
    * new tools
      * synergies with yardstick
      * analytics on bench results
    * synchro with ETSI Test
 
**Pharos**
  * Release 1 documents

-------------------------------------------------------------


==== Agenda for April 23, 2015 ====

** Meeting canceled - prioritizing release activities and meetings **

---------------------------------------


==== Agenda for April 16, 2015 ====

** Meeting canceled - prioritize release candidate meeting **

--------------------------------------
==== Agenda for April 9, 2015 ====

** Meeting canceled - prioritize release candidate meeting **

** Meeting time UTC 14:00-15:00 / 7:00am-8:00am PDT **
[[http://www.timeanddate.com/worldclock/fixedtime.html?msg=OPNFV-TEST-PERFORMANCE&iso=20150319T14&ah=1|Your local time]]

  * Pharos (20 min)
    * Test-bed status
    * Pharos specification and work for release one

  * Functest (30 min)
    * Voting for functest leader
    * Working session re. status and activities for release one

  * Wiki and tool links 
    * Pharos project https://wiki.opnfv.org/pharos
    * Pharos specification https://wiki.opnfv.org/pharos/pharos_specification
    * Functest https://wiki.opnfv.org/opnfv_functional_testing
    * Documentation project https://wiki.opnfv.org/documentation_projects/opnfv_documentation
    * Jira https://jira.opnfv.org/secure/Dashboard.jspa
    * OPNFV Release page:  https://wiki.opnfv.org/releases 
    * Arno Release Description:  https://wiki.opnfv.org/releases/arno 
    * Arno Release Candidate plan:  https://wiki.opnfv.org/releases/arno/releasecandidates 


==== Agenda for March 19, 2015 ====

** Meeting time UTC 14:00-15:00 / 7:00am-8:00am PDT **
[[http://www.timeanddate.com/worldclock/fixedtime.html?msg=OPNFV-TEST-PERFORMANCE&iso=20150319T14&ah=1|Your local time]]


  * Functest (35 min)
    * status and plan for rls one
    * CI integration
    * Documentation
    * Update on vIMS proposal – Metaswitch

  * Test Tool initiative (25 min) - Ericsson 

  * Wiki and tool links 
    * Pharos specification https://wiki.opnfv.org/pharos/pharos_specification
    * Functest https://wiki.opnfv.org/opnfv_functional_testing
    * Documentation https://wiki.opnfv.org/documentation_projects/opnfv_documentation
    * Jira https://jira.opnfv.org/secure/Dashboard.jspa





==== Agenda for March 12, 2015 ====

** Meeting time UTC 14:00-15:00 / 7:00am-8:00am PDT **
[[http://www.timeanddate.com/worldclock/fixedtime.html?msg=OPNFV-TEST-PERFORMANCE&iso=20150312T14&ah=1|Your local time]]

  * Functest
    * Progress with test tools and CI integration
    * vIMS proposal from Metaswitch
  * Rls 1 deliverables
    * Pharos specification https://wiki.opnfv.org/pharos/pharos_specification
    * Documentation https://wiki.opnfv.org/documentation_projects/opnfv_documentation
    * Functest https://wiki.opnfv.org/opnfv_functional_testing
  * Tools
    * Jira https://jira.opnfv.org/secure/Dashboard.jspa
    * Test tools - Tempest / Rally





==== Agenda for March 05, 2015 ====

** Meeting time UTC 15:00-16:00 / 7:00am-9:00am PST **
[[http://www.timeanddate.com/worldclock/fixedtime.html?msg=OPNFV-TEST-PERFORMANCE&iso=20150305T15&ah=1|Your local time]]

  * Wiki updates - meetings https://wiki.opnfv.org/meetings/test
  * Rls 1 deliverables
    * Pharos specification https://wiki.opnfv.org/pharos/pharos_specification
    * Documentation https://wiki.opnfv.org/documentation_projects/opnfv_documentation
    * Functest https://wiki.opnfv.org/opnfv_functional_testing
    * vIMS
  * Tools
    * Jira https://jira.opnfv.org/secure/Dashboard.jspa
    * Test tools - Tempest / Rally



=== Minutes ===

[[http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2015/opnfv-meeting.2015-12-03-15.04.html| 2015-12-03]]