The main labs of interest for the community will be Pharos Labs.

This page includes an informal list of Pharos Labs within the community. It is not intended to replace the Pharos Lab directory anchored on the Pharos Project, but rather to provide a summary, intent, and status of the labs.

This page also includes descriptions of ad-hoc labs and/or ideas and recommendations about resources that members of the community can pursue in order to work with ARM 64-bit servers.

Enea is building an ARM-based Pharos Lab in Kista (Stockholm) Sweden:

https://wiki.opnfv.org/enea-pharos-lab.

The lab will focus first on bringing up the Fuel Installer, running on an x86-based Jump Host and deploying to 3 Applied Micro ARM64 servers as controller nodes and 2 Cavium ThunderX servers as compute nodes. Ubuntu is the target OS to be installed on the bare metal Cavium and Applied Micro servers, based on Cavium's preference.

The reasons for starting with Fuel are:

• It has a mature feature set (as an original Arno installer)
• Fuel supports VLANs (Apex appears to require individual NICs for each network), and there are a limited number of NICs on the ARM64 servers

In addition to considering Apex (formerly Foreman/Quickstack), the lab will consider using the Compass and JOID installer projects as well. It should be understood that this

The lab will initially use Ubuntu as the target OS to be installed on top of the bare metal ARM64 servers. This is mainly based on Cavium's recommendation/preference for Ubuntu.

Note: adding lab status at the request of folks on the ARMband call on Nov 20, 2015. It's dated, so please ping joe.kidder@enea.com if it hasn't been updated by Nov 27, 2015:).

Two Cavium servers are connected to the network as described in the Enea Pharos page (follow link above). These two servers will initially be configured in a non-HA configuration, one as a controller and the other as a compute.

The Applied Micro servers will ship to the lab just prior to the end of November. When they arrive the lab will become 2 Cavium computes and 3 Applied Micro controllers in an HA configuration.

Developers are currently porting the Fuel Installer to run with ARM64 targets.

The first step, which is in progress, is building an ARM64 bootstrap target image (on which nailgun agent runs) to replace the x86 bootstrap target image that is bundled with Fuel. Since the bootstrap has to be built for ubuntu rather than centos, the team is first building an x86 ubuntu-based bootstrap image to ensure that we have the tooling correct to build the bootstrap (then we will repeat the build for ARM64). We are in the build/test/debug phase with this image.

There are two main activities at the center of the Armband project.

1. Porting Fuel Installer to support a Pharos Lab that is populated with arm64 servers as targets.

There are three major porting components to this effort:

a. Creating an initial arm64-based bootstrap image that is served to bare metal servers that boot on a Fuel Master’s admin network

   Short Description: Build a Fuel Bootstrap Linux Image to enable the discovery of bare metal servers
Status:  
   This has been built for Fuel 6.1 and has been successfully used on the ARM servers in the Enea Pharos Lab (Cavium, Applied Micro, and SoftIron/AMD) 	We are currently building this again for Fuel 7 (and soon for Fuel 8).
   This is in good shape. It will be included in the Fuel “ISO” for arm64, in similar fashion to the bootstrap image included for x86.

b. Providing a set of repositories, both standard upstream Ubuntu (or Centos) repos as well as MOS (Mirantis Open Stack) repos, that contain arm64-based packages for all necessary target configurations (controller, compute, ODL, HA, etc) that can provide for both the base target image that is booted on deployed targets and for the puppet-based installation/configuration that occurs after the initial target images are booted on controller and compute nodes.

 Short Description: Preparing arm64 repositories for all the layers of packages, from standard Ubuntu to MOS to specials for OPNFV
 Status: 
 We have assembled these packages by hand (pointing at an ubuntu repo and building the MOS packages) for a combo of public and private repos.
 We will rebuild the arm64 packages (particularly those in the MOS repo) and put them in a public mirror hosted by Enea for the time being (until Mirantis consumes and supports this repo in their upstream MOS repo for Fuel).
 We are currently setting this up, and targeting building the repo for the Fuel 7 and soon Fuel 8 versions of the MOS repo.

c. Porting the Fuel business-logic code that creates initial target distros and drives the installation/configuration of the various Openstack/ODL/OPNFV parts when Fuel does the install.

      Short Description: This is the machinery that does what Fuel does – we change the “x86” or “amd64” specifiers for packages to also support “arm64”.
Status: 
We have done this for Fuel 6.1 (what’s in Arno SR1) in an exploratory manner (I.e. Not necessarily optimized to push upstream). We are starting this arm64 porting exercise for Fuel 7 (and eventually 8) and will be doing it in a fashion more suited to upstream.

d. Overall Status:

i. We have deployed Arno SR1 using Fuel on both a 2-node Cavium server environment and a 2-node Applied Micro (APM) server environment.

ii. We have deployed Arno SR1 with ODL using Fuel on a 2-node APM environment.

iii. We have not yet deployed Arno SR1 with HA controllers on a 5-node arm64 server environment

iv. We are putting 4 nodes together to do a 3-controller in HA configuration with 1 compute to test HA controllers on arm64. This started today and will continue tomorrow and into early next week.

v. We are currently bringing up Fuel 7 and are putting in place a public mirror to host arm64 repos for things like MOS that are not already available publicly for ARM. Note: the MOS (Mirantis OpenStack) is just deviations/patches applied to Openstack and its components by Mirantis to “improve” Openstack. This is done currently (for Arno as well as Brahmaputra) when using the Fuel installer.

2.	Porting Functest to arm64 

-a. We have run a number of functest tests against an “opnfv-like” install (I.e. Not installed by an installer) of Openstack and ODL in our dev lab.

b. We will next run functest against our 4-node HA pharos environment (installed by Fuel 6.1) when we have it running.

c. We will also run functest against a 2-node non-HA with ODL pharos environment.

d. Portability issues here are so far mainly related to the test image that is launched, which is by default a small x86-based image called “cirros”. We are replacing this with a similar small arm64-based image.

e. The status for the Functest effort is not as detailed as for Fuel. We’ve been practicing with Functest both on x86-based deployments and on non-Fuel-installed (I.e. “by hand”) arm64 deployments.

Once we have the two functions above (Fuel for arm64 and Functest for arm64), the Pharos Lab should be ready for Jenkins integration. It’s possible we can start on that effort earlier, but we think we've gotten the feedback that we should start the integration after Fuel/Functest are ready. If that’s not the case, we are up for trying to start Jenkins integration sooner.

This is a place holder for other ARM-based Pharos Labs that are brewing. A recommendation is to provide a link to a Pharos Lab description page (the type that hangs from Pharos Lab list), and add a brief description of the lab.

Here is a low-cost ARMv8 platform described at 96boards.com at this link: https://www.96boards.org/products/ce/dragonboard410c/

We [at Enea] have powered this up (it comes with Android) and installed Ubuntu. It's certainly a challenging form-factor, as there's no built-in Ethernet, but it has wifi and we have added a 100mb USB enet dongle. This isn't awesome, but it's much closer to an OPNFV-capable ARMv8 SoC than BBBrC or rPi. One blocker with this device is that getting KVM running on it is not simple due to some bootstrap challenges. Anyone with interest or experience, please comment as well!