See OPNFV meetings page https://wiki.opnfv.org/meetings/ … with link to https://wiki.opnfv.org/meetings/test

* Do not place meeting related pages under the ''/wiki'' or ''/project-name'' folders.
* ALSO: Project info pages should not be under the ''/wiki'' folder. 
* Project related pages should be located under the ''/'' (root) folder.

Purpose: Meeting logistics, Agenda and Notes for the "Test and Performance" sub-group that was initiated at TSC F2F meeting on Oct 1st/2nd

• https://wiki.opnfv.org/notes_brainstorming_testbed_f2f_meeting_2-10-2014

1. Prague hackfest meeting - review plans

2. Test Framework - discuss requirements and short terms needs (Palani)

3. Deliverables for Rls 1 - any updates

4. Any project report outs (time allowing)?

Meeting Notes:

• http://meetbot.opnfv.org/meetings/opnfv-testperf

1. Prague hackfest meeting

1. Pharos plan … https://wiki.opnfv.org/pharos
2. ETSI NFV / OPNFV collaboration

2. Test deliverables and dates

3. Review Ericsson Proposal re carrier grade (posted today) … TRANSFORM https://wiki.opnfv.org/transformer_-_carrier_grade_requirements_for_network_transformation

4. Project report outs

1. Base system functionality testing
2. vIMS

Meeting Notes:

• http://meetbot.opnfv.org/meetings/opnfv-testperf

1. Pharos … https://wiki.opnfv.org/pharos

1. Diagram showing relationships
2. Governance text draft (Ericsson description, ODL description, Cable Labs)
3. Dashboard

2. Qtip … https://wiki.opnfv.org/platform_performance_benchmarking

1. Tempest and ODL tests, esp.
2. test plan template
3. priority: what would be the most urgent performance questions?
4. Next steps?

3. Hackfest plans

4. Base system functionality testing

5. vIMS

Meeting Notes:

• http://meetbot.opnfv.org/meetings/opnfv-testperf

1. Wiki structure for Pharos project execution … https://wiki.opnfv.org/pharos

- Clarification of articulation between Pharos and testing projects.

1. Address TSC questions (Morgan to explain)
2. Qtip question - how it relates to Pharos and FuncTest (Wenjing)

- Sub menus (FAQ, Task follow-up, ..)

- list of community platforms

- Governance aspects, … there are a number of sources need to converge (Ericsson description, ODL description, Cable Labs)

- Project draft release

- Brainstorm on what we could propose for the hackfest in Prague

- Discuss sharing and reporting between the projects and test-beds

- Dashboard to track operational status and features of test-beds

2. Any time allowing … Project updates and plans to work together

1. vSwitch
2. vPE
3. vIMS
4. Carrier grade requirements

Meeting Notes:

• http://meetbot.opnfv.org/meetings/opnfv-meeting/2014/

1. Pharos - approval, progress (20 min)

1. Governance
2. HW/SW specs
3. Tooling

2. Status / Plans of hosted test environments (20 min)

3. Project updates and plans to work together (20 min)

1. vSwitch
2. vPE
3. vIMS
4. Carrier grade requirements

Meeting Notes:

• http://meetbot.opnfv.org/meetings/opnfv-meeting/2014/

1. Characterize vSwitch Performance for Telco NFV Use Cases - how to work together (10 min)

2. Spirent - test tools intend to make available for open-source projects in testbed infrastructures (20 min)

3. Ericsson Proposal - Carrier Grade (10 min)

4. Review status of activities and projects … review progress of Wiki clean-up (10 min)

Meeting Notes:

• http://meetbot.opnfv.org/meetings/opnfv-meeting/2014/

• Agenda and opens (5 min)
  • Meeting schedule over the holidays

• Inform and discuss
  • Spirent Test Tools (15 min)

• New proposals
  • No agenda items

• Ongoing - discussions / work items / project status
  • Status of projects submitted to TSC …
    • Base system functionality testing (5 min)
      • Review contributors / committers
      • https://wiki.opnfv.org/opnfv_functional_testing
    • Testbed Infrastructure (Pharos) (5 min)
      • Review contributors / committers
      • https://wiki.opnfv.org/opnfv_testbed_infrastructure
  • Qtip - Platform Performance Benchmarking (5 min)
    • https://wiki.opnfv.org/platform_performance_benchmarking
  • Working with Characterize vSwitch Performance for Telco NFV Use Cases (5min)
    • https://wiki.opnfv.org/characterize_vswitch_performance_for_telco_nfv_use_cases
• Test-bed Tools (20 min)
• Agenda topics for future meetings - 5 min

Meeting Notes:

• http://meetbot.opnfv.org/meetings/opnfv-meeting/2014/

• Roll call (5 min)
• Agenda and opens (5 min)

• Inform and discuss
  • No agenda items

• New proposals
  • No agenda items

• Ongoing - discussions / work items / project status
  • Review OPNFV Base system functionality testing Project Proposal - 15 min
    • https://wiki.opnfv.org/opnfv_functional_testing
  • Review OPNFV Testbed Infrastructure Project Proposal - 35 min
    • https://wiki.opnfv.org/opnfv_testbed_infrastructure

• Agenda topics for next meeting - 0 min

Meeting Notes:

• http://meetbot.opnfv.org/meetings/opnfv-meeting/2014/

• Roll call (5 min)
• Agenda and opens (5 min)

• Inform and discuss
  • Benchmarking Methodology Working Group (BMWG) at IETF working NFV (Al Morton - AT&T) - 30 min
  • Considerations for Benchmarking VNFs and their Infrastructure
  • ETSI NFV test plans/activities - 5 min

• New proposals
  • No agenda items

• Ongoing - discussions / work items / project status
  • OPNFV Testbed Infrastructure Project - 5 min
    • Status of project proposal
  • OPNFV Platform Characterization Projects - 5 min
    • VNF Use Cases
    • Platform Performance Benchmarking

• Timelines for projects

• Agenda topics for next meeting - 5 min

Meeting Notes:

• http://meetbot.opnfv.org/meetings/opnfv-meeting/2014/

• Roll call (5 min)
• Agenda and opens (5 min)

• New topics/proposals
  • Proposal for vSBC use-case - Martin Taylor (Metaswitch)

• Continue discussions / report status re. ongoing work items
  • OPNFV Testbed Infrastructure Project
    • Governance and policy - Morgan (Orange)
    • Reference platform HW/SW definition - Trevor (Intel)
    • Test-bed Tooling - Wenjing (Dell)

• OPNFV Platform Characterization Projects
  • VNF Use Cases
    • vPE - Palani (CISCO)
  • Platform Performance Benchmarking
    • No agenda topic today

• Agenda items for next meeting

Meeting Notes:

• Metaswitch presentation
• http://meetbot.opnfv.org/meetings/opnfv-meeting/2014/

• Roll call / IRC (5 min)
• Agenda and any open items (5 min)

• Discuss top level project proposal/s for submitting to TSC (10 min)

• New T&P sub-group proposals
  • Characterize vSwitch Performance for Telco NFV Use Cases - Intel / Nokia Networks (10 min)
  • Ericsson proposals - VNF Use Cases / Platform Benchmarking (10 min)

• Continue discussions / report status re. ongoing projects
  • Platform benchmarking - Ixia / Dell (5 min)
  • Test-bed Governance and policy - Orange / Cable Labs (5 min)
  • Test-bed Tooling - Dell (5 min)
  • VNF Use Cases - CISCO (5 min)

• Plan agenda for next meeting (time allowing … if not by email)

Meeting Notes:

• Plan to use IRC … https://freenode.net/ … channel is opnfv-meeting
• See OPNFV site under "developers" for record of meeting notes … MeetBot http://meetbot.opnfv.org/meetings/

• Wiki has been updated to reflect two top-level projects that will be submitted to TSC for approval (once approved sub-projects will propose in T&P subgroup for discussion/input before being initiated) …
  • 1. OPNVF Testbed Infrastructure
    • Testbed governance & policy
    • Reference platform HW&SW definition
    • Platform tooling
  • 2. OPNFV Platform Characterization
    • VNF Deployment Test Cases
    • Platform Performance Benchmarking

• New T&P sub-group proposals presented/discussed …
  • Characterize vSwitch Performance for Telco NFV Use Cases - Intel / Nokia Networks … Mike Lynch (Intel)
    • General agreement and interest (Spirent can help support)
    • Need to make sure OPNFV work stays in sync with ETSI NFV … there are test/performance projects/proposals being developed
  • Ericsson proposals - VNF Use Cases / Platform Benchmarking … Sheng-Ann Yu (Ericsson)
    • Ericsson presentation
  • May make sense to create a 3rd track … "Carrier Grade" for stability/reliability class of requirements

• Continue discussions / report status re. ongoing projects
  • Platform benchmarking - Michael/Wenjing will pull in Spirent and others working on proposal
  • Test-bed Governance and policy - info from existing docs, looking for review of progress next week (Morgan?)
  • Test-bed Tooling - Wenjing has updated Wiki re tools, looking for further input
  • VNF Use Cases - Palani working on detailed proposal of first use-case … review next week.

• Roll call (5 min)
• Review agenda and any open agenda items (5 min)
• VNF Use Cases (led by Palani)
• Progress on other work packages
• Project proposal expectations
• Agenda for next meeting (5 min)

Meeting Notes:

• Trevor setup IRC for note taking AR
• Sheng-Ann - Ericsson would like to contribute ideas on carrier grade performance and performance testing
• Palani (CISCO) led discussion on VNF deployment use-cases
• Collecting info on traffic profiles … e.g. ATT providing info for Virtual Provider Edge … also defining metrics
• Has been a suggestion to replace CDN use-case with IMS (Palani identified a VNF)
• Objective is to characterize the OPNFV platform (capabilities, performance limits, etc.) … we are not validating performance of vendor VNFs (there are many different approaches to deploying VNFs to achieve particular service level requirements)
• Two approaches have been proposed (likely will craft 2 separate proposals around these … there will be overlap as testing evolves and each approach maps to the other )
  • Use-case (top down) approach - Palani has initated outline and will continue working with others on this
  • Platform benchmarking (bottom up) approach - Michael Haugh / Wenjing will initiate by creating an outline
    • Output is test spec (includes platform configuration i.e. controlled environment, key metrics, test cases, …)
    • Characterize platform components … e.g. scaling, throughput, etc.
    • Start with performance at each level … NIC, vSwitch, hypervisor, … define metrics
    • Map platform performance metrics to high level use-cases
    • Then configure/deploy a VNF … do service assessment with relevant metrics (successfully deployed … basic performance e.g. throughput, latency …)
    • Performance can include availability/allocation of platform resources (e.g. CPU cores) for a system performance requirement
• Palani - Setup Etherpads for each approach to start collecting input for crafting proposal AR

• Roll call (5 min)
• Review agenda and any open agenda items (5 min)
• Organization of future standing meetings (10 min) … Agendas, Roles … organizer, notes (IRC?), etc, Timeslot / frequency
• Organization of streams/projects on Wiki - discussion (5 min)
• https://wiki.opnfv.org/integration_testing_projects
• Project status and plans (led by leads identified from the F2F)
  1. Testbed governance & policy - Morgan Richomme (Orange) + Chris Donley (Cable Labs) (15 min)
  2. Reference platform HW&SW definition - Trevor Cooper (Intel) (15 min)
  3. Testbed tooling / infrastructure - Wenjing Chu (Dell) (15 min)
  4. Use-cases - Palani Chinnakannan (CISCO) (15 min)
• Agenda for next meeting (5 min)

Meeting Notes:

• https://wiki.opnfv.org/opnfv_test_and_performance_group_meeting_minutes_10-23-2014?&#platform_tooling

Notes from brainstorming to kick-off Test and Performance initiative https://wiki.opnfv.org/notes_brainstorming_testbed_f2f_meeting_2-10-2014