• NOTE: Meeting info pages for each project should be moved to the https://wiki.opnfv.org/meetings/ folder.
• Do not place meeting related pages under the /wiki or /project-name folders.
• ALSO: Project info pages should not be under the /wiki folder.
• Project related pages should be located under the / (root) folder.
• To move your page you should create a new page and update this one with a note stating it has been moved.

Purpose: Meeting logistics, Agenda and Notes for the "Test and Performance" sub-group that was initiated at TSC F2F meeting on Oct 1st/2nd

Project Proposals

• https://wiki.opnfv.org/notes_brainstorming_testbed_f2f_meeting_2-10-2014

Logistics

When: Every Thursday at 7:00am - 8:00am Pacific Time (UTC -8:00) http://www.timeanddate.com/worldclock/fixedtime.html?iso=20141030T07&p1=283).

Call logistics (GoToMeeting):

• Please join the meeting from your computer, tablet or smartphone: https://global.gotomeeting.com/join/305553637
• You can also call in using your phone. US number (toll): +1 (626) 521-0010
• Access Code: 305-553-637
• Audio PIN: Shown after joining the session
• Additional access numbers …
  • Australia (Long distance): +61 2 8355 1031
  • Belgium (Long distance): +32 (0) 28 08 4342
  • Austria (Long distance): +43 (0) 7 2088 1033
  • Norway (Long distance): +47 21 54 82 21
  • Switzerland (Long distance): +41 (0) 435 0167 65
  • Denmark (Long distance): +45 (0) 69 91 89 21
  • Ireland (Long distance): +353 (0) 19 030 050
  • Canada (Long distance): +1 (647) 497-9371
  • Netherlands (Long distance): +31 (0) 208 080 384
  • Italy (Long distance): +39 0 693 38 75 50
  • Finland (Long distance): +358 (0) 942 41 5770
  • France (Long distance): +33 (0) 182 880 159
  • Germany (Long distance): +49 (0) 692 5736 7208
  • Spain (Long distance): +34 911 82 9890
  • New Zealand (Long distance): +64 (0) 9 925 0313
  • Sweden (Long distance): +46 (0) 852 500 179
  • United Kingdom (Long distance): +44 (0) 330 221 0096

Meeting notes (IRC / MeetBot):

• IRC
  • freenode https://freenode.net/
  • IRC channel: #opnfv * http://webchat.freenode.net/?channels=opnfv-testperf
• Link to meeting notes parsed and stored by MeetBot
  • http://meetbot.opnfv.org/meetings/opnfv-testperf/

1. Prague hackfest meeting - review plans

2. Test Framework - discuss requirements and short terms needs (Palani)

3. Deliverables for Rls 1 - any updates

4. Any project report outs (time allowing)?

Meeting Notes:

• http://meetbot.opnfv.org/meetings/opnfv-testperf

1. Prague hackfest meeting

1. Pharos plan … https://wiki.opnfv.org/pharos
2. ETSI NFV / OPNFV collaboration

2. Test deliverables and dates

3. Review Ericsson Proposal re carrier grade (posted today) … TRANSFORM https://wiki.opnfv.org/transformer_-_carrier_grade_requirements_for_network_transformation

4. Project report outs

1. Base system functionality testing
2. vIMS

Meeting Notes:

• http://meetbot.opnfv.org/meetings/opnfv-testperf

1. Pharos … https://wiki.opnfv.org/pharos

1. Diagram showing relationships
2. Governance text draft (Ericsson description, ODL description, Cable Labs)
3. Dashboard

2. Qtip … https://wiki.opnfv.org/platform_performance_benchmarking

1. Tempest and ODL tests, esp.
2. test plan template
3. priority: what would be the most urgent performance questions?
4. Next steps?

3. Hackfest plans

4. Base system functionality testing

5. vIMS

Meeting Notes:

• http://meetbot.opnfv.org/meetings/opnfv-testperf

1. Wiki structure for Pharos project execution … https://wiki.opnfv.org/pharos

- Clarification of articulation between Pharos and testing projects.

1. Address TSC questions (Morgan to explain)
2. Qtip question - how it relates to Pharos and FuncTest (Wenjing)

- Sub menus (FAQ, Task follow-up, ..)

- list of community platforms

- Governance aspects, … there are a number of sources need to converge (Ericsson description, ODL description, Cable Labs)

- Project draft release

- Brainstorm on what we could propose for the hackfest in Prague

- Discuss sharing and reporting between the projects and test-beds

- Dashboard to track operational status and features of test-beds

2. Any time allowing … Project updates and plans to work together

1. vSwitch
2. vPE
3. vIMS
4. Carrier grade requirements

Meeting Notes:

• http://meetbot.opnfv.org/meetings/opnfv-meeting/2014/

1. Pharos - approval, progress (20 min)

1. Governance
2. HW/SW specs
3. Tooling

2. Status / Plans of hosted test environments (20 min)

3. Project updates and plans to work together (20 min)

1. vSwitch
2. vPE
3. vIMS
4. Carrier grade requirements

Meeting Notes:

• http://meetbot.opnfv.org/meetings/opnfv-meeting/2014/

1. Characterize vSwitch Performance for Telco NFV Use Cases - how to work together (10 min)

2. Spirent - test tools intend to make available for open-source projects in testbed infrastructures (20 min)

3. Ericsson Proposal - Carrier Grade (10 min)

4. Review status of activities and projects … review progress of Wiki clean-up (10 min)

Meeting Notes:

• http://meetbot.opnfv.org/meetings/opnfv-meeting/2014/

• Agenda and opens (5 min)
  • Meeting schedule over the holidays

• Inform and discuss
  • Spirent Test Tools (15 min)

• New proposals
  • No agenda items

• Ongoing - discussions / work items / project status
  • Status of projects submitted to TSC …
    • Base system functionality testing (5 min)
      • Review contributors / committers
      • https://wiki.opnfv.org/opnfv_functional_testing
    • Testbed Infrastructure (Pharos) (5 min)
      • Review contributors / committers
      • https://wiki.opnfv.org/opnfv_testbed_infrastructure
  • Qtip - Platform Performance Benchmarking (5 min)
    • https://wiki.opnfv.org/platform_performance_benchmarking
  • Working with Characterize vSwitch Performance for Telco NFV Use Cases (5min)
    • https://wiki.opnfv.org/characterize_vswitch_performance_for_telco_nfv_use_cases
• Test-bed Tools (20 min)
• Agenda topics for future meetings - 5 min

Meeting Notes:

• http://meetbot.opnfv.org/meetings/opnfv-meeting/2014/

• Roll call (5 min)
• Agenda and opens (5 min)

• Inform and discuss
  • No agenda items

• New proposals
  • No agenda items

• Ongoing - discussions / work items / project status
  • Review OPNFV Base system functionality testing Project Proposal - 15 min
    • https://wiki.opnfv.org/opnfv_functional_testing
  • Review OPNFV Testbed Infrastructure Project Proposal - 35 min
    • https://wiki.opnfv.org/opnfv_testbed_infrastructure

• Agenda topics for next meeting - 0 min

Meeting Notes:

• http://meetbot.opnfv.org/meetings/opnfv-meeting/2014/

• Roll call (5 min)
• Agenda and opens (5 min)

• Inform and discuss
  • Benchmarking Methodology Working Group (BMWG) at IETF working NFV (Al Morton - AT&T) - 30 min
  • Considerations for Benchmarking VNFs and their Infrastructure
  • ETSI NFV test plans/activities - 5 min

• New proposals
  • No agenda items

• Ongoing - discussions / work items / project status
  • OPNFV Testbed Infrastructure Project - 5 min
    • Status of project proposal
  • OPNFV Platform Characterization Projects - 5 min
    • VNF Use Cases
    • Platform Performance Benchmarking

• Timelines for projects

• Agenda topics for next meeting - 5 min

Meeting Notes:

• http://meetbot.opnfv.org/meetings/opnfv-meeting/2014/

• Roll call (5 min)
• Agenda and opens (5 min)

• New topics/proposals
  • Proposal for vSBC use-case - Martin Taylor (Metaswitch)

• Continue discussions / report status re. ongoing work items
  • OPNFV Testbed Infrastructure Project
    • Governance and policy - Morgan (Orange)
    • Reference platform HW/SW definition - Trevor (Intel)
    • Test-bed Tooling - Wenjing (Dell)

• OPNFV Platform Characterization Projects
  • VNF Use Cases
    • vPE - Palani (CISCO)
  • Platform Performance Benchmarking
    • No agenda topic today

• Agenda items for next meeting

Meeting Notes:

• Metaswitch presentation
• http://meetbot.opnfv.org/meetings/opnfv-meeting/2014/

• Roll call / IRC (5 min)
• Agenda and any open items (5 min)

• Discuss top level project proposal/s for submitting to TSC (10 min)

• New T&P sub-group proposals
  • Characterize vSwitch Performance for Telco NFV Use Cases - Intel / Nokia Networks (10 min)
  • Ericsson proposals - VNF Use Cases / Platform Benchmarking (10 min)

• Continue discussions / report status re. ongoing projects
  • Platform benchmarking - Ixia / Dell (5 min)
  • Test-bed Governance and policy - Orange / Cable Labs (5 min)
  • Test-bed Tooling - Dell (5 min)
  • VNF Use Cases - CISCO (5 min)

• Plan agenda for next meeting (time allowing … if not by email)

Meeting Notes:

• Plan to use IRC … https://freenode.net/ … channel is opnfv-meeting
• See OPNFV site under "developers" for record of meeting notes … MeetBot http://meetbot.opnfv.org/meetings/

• Wiki has been updated to reflect two top-level projects that will be submitted to TSC for approval (once approved sub-projects will propose in T&P subgroup for discussion/input before being initiated) …
  • 1. OPNVF Testbed Infrastructure
    • Testbed governance & policy
    • Reference platform HW&SW definition
    • Platform tooling
  • 2. OPNFV Platform Characterization
    • VNF Deployment Test Cases
    • Platform Performance Benchmarking

• New T&P sub-group proposals presented/discussed …
  • Characterize vSwitch Performance for Telco NFV Use Cases - Intel / Nokia Networks … Mike Lynch (Intel)
    • General agreement and interest (Spirent can help support)
    • Need to make sure OPNFV work stays in sync with ETSI NFV … there are test/performance projects/proposals being developed
  • Ericsson proposals - VNF Use Cases / Platform Benchmarking … Sheng-Ann Yu (Ericsson)
    • Ericsson presentation
  • May make sense to create a 3rd track … "Carrier Grade" for stability/reliability class of requirements

• Continue discussions / report status re. ongoing projects
  • Platform benchmarking - Michael/Wenjing will pull in Spirent and others working on proposal
  • Test-bed Governance and policy - info from existing docs, looking for review of progress next week (Morgan?)
  • Test-bed Tooling - Wenjing has updated Wiki re tools, looking for further input
  • VNF Use Cases - Palani working on detailed proposal of first use-case … review next week.

• Roll call (5 min)
• Review agenda and any open agenda items (5 min)
• VNF Use Cases (led by Palani)
• Progress on other work packages
• Project proposal expectations
• Agenda for next meeting (5 min)

Meeting Notes:

• Trevor setup IRC for note taking AR
• Sheng-Ann - Ericsson would like to contribute ideas on carrier grade performance and performance testing
• Palani (CISCO) led discussion on VNF deployment use-cases
• Collecting info on traffic profiles … e.g. ATT providing info for Virtual Provider Edge … also defining metrics
• Has been a suggestion to replace CDN use-case with IMS (Palani identified a VNF)
• Objective is to characterize the OPNFV platform (capabilities, performance limits, etc.) … we are not validating performance of vendor VNFs (there are many different approaches to deploying VNFs to achieve particular service level requirements)
• Two approaches have been proposed (likely will craft 2 separate proposals around these … there will be overlap as testing evolves and each approach maps to the other )
  • Use-case (top down) approach - Palani has initated outline and will continue working with others on this
  • Platform benchmarking (bottom up) approach - Michael Haugh / Wenjing will initiate by creating an outline
    • Output is test spec (includes platform configuration i.e. controlled environment, key metrics, test cases, …)
    • Characterize platform components … e.g. scaling, throughput, etc.
    • Start with performance at each level … NIC, vSwitch, hypervisor, … define metrics
    • Map platform performance metrics to high level use-cases
    • Then configure/deploy a VNF … do service assessment with relevant metrics (successfully deployed … basic performance e.g. throughput, latency …)
    • Performance can include availability/allocation of platform resources (e.g. CPU cores) for a system performance requirement
• Palani - Setup Etherpads for each approach to start collecting input for crafting proposal AR

• Roll call (5 min)
• Review agenda and any open agenda items (5 min)
• Organization of future standing meetings (10 min) … Agendas, Roles … organizer, notes (IRC?), etc, Timeslot / frequency
• Organization of streams/projects on Wiki - discussion (5 min)
• https://wiki.opnfv.org/integration_testing_projects
• Project status and plans (led by leads identified from the F2F)
  1. Testbed governance & policy - Morgan Richomme (Orange) + Chris Donley (Cable Labs) (15 min)
  2. Reference platform HW&SW definition - Trevor Cooper (Intel) (15 min)
  3. Testbed tooling / infrastructure - Wenjing Chu (Dell) (15 min)
  4. Use-cases - Palani Chinnakannan (CISCO) (15 min)
• Agenda for next meeting (5 min)

Meeting Notes:

• https://wiki.opnfv.org/opnfv_test_and_performance_group_meeting_minutes_10-23-2014?&#platform_tooling

Notes from brainstorming to kick-off Test and Performance initiative https://wiki.opnfv.org/notes_brainstorming_testbed_f2f_meeting_2-10-2014