OPNFV program

Using the PMI definition, the traditional interpretation of project is: "It's a temporary endeavor undertaken to create a unique product, service or result. A project is temporary in that it has a defined beginning and end in time, and therefore defined scope and resources. And a project is unique in that it is not a routine operation, but a specific set of operations designed to accomplish a singular goal."

After defining project, we can say that program is a group of related projects being managed together.

As OPNFV is an open source community the term project gets a slightly different interpretation. The difference comes from not defining explicitly the end time, but focusing on the joint development effort on a specific product. This product is built, maintained and improved by the community of developers and it can have many releases.

In this sense the OPNFV program consists of several open source projects, the set of projects may vary among releases.

The activities of OPNFV are organized into several projects, which are targeting different areas with different focus points. The projects are working in collaboration to cover the integration points between the areas and on platform level.

As OPNFV is a mid-stream community we have to cover an end to end process. This means that we have to identify what is missing from the upstream projects we are using, we need to address those gaps. In parallel we are also working on integrating these open source components and then validating and verifying the platform we got.

Besides these main activities each project in OPNFV has a scope that identifies an area it is working on.

Projects are now categorized into three main groups:

• Continuous Build and Integration
• Continuous Deployment and Testing
• New Requirements and Features

These categories mean more a labelling to see on which part of the end to end flow the projects are active.

High level (sub-)project lifecycle for the current categories:

As you can see on the above figure the steps for each current category are the same, on high level the above mentioned categorization does not add any differentiation in the meaning of how we are using these terms. On the other hand today we are not following the process of having sub-projects for the different types of activities, but we are more aiming for covering all these under the umbrella of one single project.

The categorization does not define anything about the lifecycle of the projects contained by each group either. In case of each and every project we have phases as:

• Requirement definition
• Implementation
• Testing
• Documentation

As we are working in an Agile environment these phases are not parts of a waterfall way of working! This means that the requirements specification phase does not have to be completed before starting the implementation of one or more items on the list.

Basic Agile flow:

Requirements can target deployment, testing of the platform or gaps in upstream projects. It is the responsibility of the project to cover the implementation, testing and documentation in the appropriate repository (repositories).

By removing categorization we would like to encourage projects to follow the Agile flow. Having only generic projects we also remove the need of an additional administration step, which was required for Requirements Projects to move into the Collaborative Development category.

The lifecycle process without categories looks like the following:

We haven't followed the lifecycle process depicted on the above figure during the Arno release. It is still an open question of how to handle these stages currently identified for a project and also whether we need all the steps. It is also a question whether we should aim more for a Big Tent-like structure rather than strict review processes.

Requirements projects are one group of projects, which are focusing on identifying and addressing gaps in upstream projects like OpenStack, OpenDaylight, etc. As this marker does not differentiate these projects from any other from life cycle point of view these projects have to provide an end to end coverage from requirements to implementation (including tests and documentation). Even with this criteria a project should not be rejected because of the lack of developers who could cover the implementation at the beginning. Projects should be able to cover each phase iteratively in a step by step process. On the other hand as we are an open source community we need to keep our doors open for new comers.

Each project has an area, well defined scope to work on. Overlapping between the scope has to be considered as integration points between modules, it is not necessarily an issue that would need to be avoided. These situations have to be investigated and addressed in collaboration between the projects:

• What do we mean by overlapping?
  • How to differentiate between real overlapping and integration points?
• How to identify overlapping early and how to handle it?
  • Have liasons in the projects to focus on existing and/or possible collaboration?

These projects have a unique nature by the need of defining the use cases they plan to cover and analysing the upstream components to find gaps or areas to improve/extend. This means a longer planning activity at the beginning of the project life cycle.

The use cases, gap analysis, requirements and architecture and implementation plans should be stored in a document. This document can then be both for having a view about the scope of the project and for using it as a reference, when interacting with other open source communities to cover the implementation phase. Below you can see the process for the creation and maintenance of these documents.

The following projects are related to development of requirements for the OPNFV reference platform. Add your proposal for a requirement centric OPNFV project according to the template on the OPNFV Project Proposals page.

Copper (Virtual Infrastructure Deployment Policies)

Promise (Resource Management)

Doctor (Fault Mangement and Maintenance)

High availability for VNFs

  Proposal for Requirements projects processes here for consistency

The requirement handling process in OPNFV needs to provide a set of documents. The starting point is to clarify the purpose of the requirements projects and define use cases, which can be referred later when the requirements are defined towards the upstream projects. The process should end in a deliverable document, which describes the problem area (the motivation for the features needed in OPNFV), use cases, desired architecture and requirements with reference to associated blueprints or upstream development proposals needed.

In order to get to the deliverable artifact iteratively and continuously available for review it is important to use a structure and format, which works well with Git/Gerrit and fits into the toolchains created by the documentation project. The suggested format is .rst to follow the existing patterns from open source communities like OpenStack. In case of this exact community this format is used for design document (blueprint) proposal, therefore using this format in OPNFV means that the design documents created within the requirements projects does not have to be converted.

In order to separate the requirement forming activity from the actual design proposals it is recommended to use two separate folders for the different type of documents:

• The OPNFV Requirement description document:

repository/requirements/deliverable_document.rst

• The OPNFV design reference documents:

repository/design_docs/reference_document_1.rst
repository/design_docs/reference_document_2.rst
      ...

As the deliverable document, which describes the requirements, consists of several chapters in order to capture the full picture it is hard to handle it one large file. In order to make both the review and the document editing process easier multiple files should be used, from which the final version of the document can be built. The starting point should be to split the document by the chapters:

repository/requirements/introduction.rst
repository/requirements/use_cases.rst
...

The chapters, which are considered as mandatory can be identified and will be used as a template structure. This structure will consist of a main file, referred as 'deliverable_document.rst' and will include a file for each of the recommended chapters. By having multiple files as a starting point multiple authors can work on the document in the same time and also smaller patches can be proposed to Gerrit for review, which makes the review process easier. The template structure is intended to be a baseline as opposed to an exact requirement, which means that the chapters should be adapted to each requirement project we have. To have a template in place helps in building the documents in a consistent way and helps the committers to set up their environment and start composing their documents.

As a starting point a document template is provided with the chapters a deliverable document should contain and the required configuration files for a successful document build. Please keep the line length below 120 characters in the .rst files.

You can generate HTML or LaTeX output from the source files, for which you can use for example Sphinx. For further guidance please see the links below.

The above referred 'reference_document.rst' files are intended to contain the design description for the identified gaps and requirements towards the upstream projects. The desired format of these documents are the one that is used in the targeted project. An example of this is the OpenStack blueprint process which uses specific OpenStack blueprint templates that differs for each project: Nova Template Celiometer Template Neutron Template Sahara Template made available by the community.

The intention is not to perform development analysis in OPNFV, but to ensure when we engage the upstream communities we have sufficient clarity in the proposal that a design discussion can occur.

• The Doctor team initially started working on the document in Word
  • Good way of drafting if
    • People are working on different sections
    • Sharing document within the team in private mails
  • Also sharing draft versions uploaded to the wiki
  • It worked until the community review:
    • We should not depend on proprietary solutions.
    • There are people not using Microsoft office.
    • Difficult to and keep track of multiple reviews
    • Requires sending back and forth the .doc community-wide.

• For community review moved to .rst and git/gerrit
  • Reviewers could easily read the document, even in plain text (except for figures and big tables), and the team could keep track of all changes and comments on a per-line basis.
  • The process worked well:
    • Is that a prerequisite that the document should be already in a certain state with agreements within the team?
    • How difficult would be starting drafting directly on Gerrit?
    • Gerrit workflow possible issues:
      • New changes have to be quickly reviewed and merged to avoid merge conflicts.
      • Tables with multiple columns are not trivial to handle for two reasons:
        • Limited to 80 characters per line, though not a technology constraint, but widely used in OpenStack for blueprints for instance and when reviewing at Gerrit;
        • Alignment of columns with text in rows has to match;
        • As a workaround we could use the list-table directive (http://docutils.sourceforge.net/docs/ref/rst/directives.html#list-table). Less readable as plane text…
    • Should we allow to merge documents with open review comments?
      • In general no
      • Document versioning guideline is needed
      • We need to define rules in this area in order to sync with the release cycle
    • How to review/comment already merged changes?
      • JIRA ticket?
      • Comment on old merged patch?
    • Gerrit does not allow other people than the original committer/author to submit new patch sets for the same Change-Id. This is not a limitation of Gerrit as platform itself but of its current configuration – OpenStack’s Gerrit has this feature enabled. This is very useful if someone wants to help out!!!

• Good hints on RST syntax and an online rendered can be found here: reStructuredText Primer
• To test out your formatting (Online): http://rst.ninjs.org
• Simple web based Ascii flow diagram drawing tool: http://asciiflow.com/
• Or, you can use Ascii editor: http://www.jave.de/

• TSC supervision?
  • Milestones
  • Quality check
  • Based on certain items:
    • The scope of the project is still in line with OPNFV
    • The project artifacts are in line with ETSI NFV
• Provide
  • an on hold status
  • an exit strategy for projects
• Is there a finished/victory state?
• Should requirements projects maintain links to their upstream project dependencies?
  • Yes, pointers to top level items, from which the sub-items can be identified
• Is there a comparison to specific differences from existing process?
  • Not more than the retrospective above
• Are we treating Collaboration & Development projects as Requirements Project?
  • We are handling each project as generic open source projects
• Requirements was considered as Carrier input and not Gaps with respect to Upstream Projects and thus the suggested end-to-end pipeline brings the risk of Collaboration & Development projects outside driving the agenda rather than NFV reference architecture
  • This brings in more the question of who and how should supervise of the project artifacts
• This means that scope of OPNFV being aligned with NFV reference architecture will need an Architecture Review Board(ARB) to ensure that Collaboration and Development Projects as well all Requirement project keep alignment with NFV reference architecture