Differences

This shows you the differences between two versions of the page.

--- wiki:sandbox [2014/11/27 10:40]
Christopher Price
+++ wiki:sandbox [2015/02/27 18:51] (current)
Aric Gardner
@@ Line 1: / Line 1: @@
-====== OPNFV Sandbox ======
+====== Packstack OPNFV Sandbox ======
+This OPNFV SandBox is intended to be a laptop compatible dev environment.  It is currently under development.
-The OPNFV SandBox is intended to be a laptop compatible dev environment.  The SandBox is currently under development.
 Get involved, download it and help evaluate, debug and improve!
 [[https://github.com/Aricg/PackStackSandBox]]
-SandBox currently exists as a template for further sandbox work with packstack, it brings up two nodes on a machine with at least 8GB of memory. One controller and One Compute/Networking node. It is intended to be easy to modify to meet our/your needs.
+SandBox currently exists as a template for further sandbox work with packstack, it brings up two nodes on a machine with at least 6GB of memory. One controller and One Compute/Networking node. It is intended to be easy to modify to meet our/your needs.
 The nodes can be reached after vagrant up with vagrant ssh controller vagrant ssh compute
 ===== SandBox getting started =====
+Can be run in bridged mode or nat mode, see below for details \\
+It's possible that the documentation on the github page will be more current https://github.com/Aricg/PackStackSandBox
-==== Set up your machine ====
+nodes can be reached after vagrant up with
+    vagrant ssh controller
+    vagrant ssh compute
-You will need the right tools to get the SandBox up and running, the following are the key items needed to bring up the SandBox using vagrant and VirtualBox.
+Requirements
+============
+Get VirtualBox https://www.virtualbox.org/wiki/Downloads
-Get VirtualBox [[https://www.virtualbox.org/wiki/Downloads]]
+Get Vagrant https://www.vagrantup.com/downloads.html
-Get Vagrant [[https://www.vagrantup.com/downloads.html]]
+Install vagrant-vbguest
-Install Ruby [[https://www.ruby-lang.org/en/installation]]
+    vagrant plugin install vagrant-vbguest
-  $: ruby -v
-  ruby 2.0.0p481 (2014-05-08 revision 45883) [universal.x86_64-darwin14]
-Install vagrant-vbguest
+Get this repo
-  vagrant plugin install vagrant-vbguest
+    git clone https://github.com/Aricg/PackStackSandBox.git && cd PackStackSandBox
-Enable bridged mode on your local machine
-  $ brctl show
+Nat Mode
-  bridge name     bridge id               STP enabled     interfaces
+========
-  docker0         8000.28d244719f30       no              enp0s25
-Get the SandBox repo
+Copy Vagrantfile.yml.template.natmode to Vagrantfile.yml
-  git clone git@github.com:Aricg/PackStackSandBox.git && cd PackStackSandBox
+Nat networking will provide the gateway to the internet as well as connectivity between hosts throught the vboxnetX interface created by vagrant
-==== Setting up the SandBox ====
+Setup Masquerade/Forwarding on your host to you vboxnet interface
-Once you have the right pieces in place you need to set up the SandBox configuration to match your local environment.  This is done by configuring the Vagrantfile.yml file with the environment parameters will use for your SandBox environment.
+Linux:
-=== Vagrantfile.yml ===
+make sure these are set in /etc/sysctl.d
-Modify Vagrantfile.yml to reflect the network avaliable to you. Note that the default configuration is based on having a /22 avaliable on my home network, you will need to reserve a /24 section of whatever network you are on so that we can create a route to the neutron router we later create. eg:
+    net.ipv4.ip_forward = 1
+    net.ipv4.conf.all.proxy_arp = 1
-  route add -net 192.168.x.0 netmask 255.255.255.0 gw 192.168.x.1
+And loaded
-Without this, you will not be able to route to your VMs. (Outbound traffic will still work)
+    sudo sysctl -p
-Default working config:
+In my example my hosts interface for internet connetiviy is docker0 (yours might be eth0 for example) and my the vboxnet brought up by vagrant up is vboxnet4 and the subnet I have set for the sandbox machines in the vagrantfile.yaml is 10.0.20.0/22
-  bridge: docker0
+    iptables -A FORWARD -o docker0 -i vboxnet4 -s 10.0.20.0/22 -m conntrack --ctstate NEW -j ACCEPT
-  netmask: 255.255.252.0
+    iptables -A FORWARD -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
-  gateway: 192.168.0.1
+    iptables -A POSTROUTING -t nat -j MASQUERADE
-  neutron_router_start: 192.168.3.1
-  neutron_router_end: 192.168.3.128
-  controller:
-    bridged_ip: 192.168.1.91
-    private_ip: 192.168.22.92
-  compute:
-    bridged_ip: 192.168.1.93
-    private_ip: 192.168.22.94
-=== Vagrantfile.yml Explanation ===
+In this example we have set the vboxnet to  the 10.0.20.0/22 range.
+OSX:
-**bridge:** name of your bridge interface ($ brctl show )
+Don't have a mac, this is untested
-**netmask:** netmask of your private subnet, probably given to you via dhcp. you can see this with ifconfig, however on osx if will be in the unreadble format, something like 0xffffff00 Refer here for a table that human can read. http://www.pawprint.net/designresources/netmask-converter.php Most home networks only give out a /24 you will need to log into your router and change your range to at least a /23 so that we an properly route to the router that neutron creates.
+    /usr/sbin/natd -interface en0
+    /sbin/ipfw -f flush
+    /sbin/ipfw add divert natd all from any to any via en0
+    /sbin/ipfw add pass all from any to any
+    sudo sysctl -w net.inet.ip.forwarding=1
-**gateway:** Your workstations gateway to the internet (your routers ip, this is also the ip you go to to increase your network size ) you can check this with ip r on linux or netstat -nr on osx
+Vagrant ssh into the compute and the controller node and set the default route to vboxnet0 rather than the nat device that vagrant sets at default
-**neutron_router_start:** This will be the start of your openstack dhcp, I also use this to guess your neutron router gateway. Make it something that is routable but that none of your computers are using. eg: If my workstation and gateway(router) were in the 192.168.0.0/24 range I could make the neutron range inside 192.168.1.0/24 eg: 192.168.1.1-192.168.1.254
+TODO automate this.
-**neutron_router_end:** the end of the range explained above
+    ip route del default
+    ip route add default via 10.0.20.1 (the gateway set it your Vagrantfile.yml) dev eth1
-**controller:**
+You are now ready to "Launch Vagrant" (see below)
-**bridged_ip:** this interface should be given an ip on the same /24 as your workstation.
+Bridged Mode
+===========
-**private_ip:** this interface can have any ip you want, virtualbox deals with the routing.
+If you are able to configure and use a bridge we can bring up openstack VMs on your local network. you will need a netmask of 23 or below.
-**compute:**
+My bridge in this readme is called docker0
-**bridged_ip:** same rules as the controller bridged_ip but unique
+    $ brctl show
+    bridge name     bridge id               STP enabled     interfaces
+    docker0
-**private_ip:** same rules as controller: private_ip but unique
-==== Start the SandBox ====
+To start copy Vagrantfile.yml.template.bridgemode to Vagrantfile.yml to reflect the network avaliable to you. In this example I have a /22 avaliable on my home network, Later we reserve a /24 section of my /22 network for the neutron router we create.
-Bring up the SandBox by launching the VM's through vagrant.
+My example config:
-  $ vagrant up
+    bridge: docker0
+    netmask: 255.255.252.0
+    gateway: 192.168.0.1
+    neutron_router_start: 192.168.3.1
+    neutron_router_end: 192.168.3.128
+    controller:
+      bridged_ip: 192.168.1.91
+      private_ip: 10.2.20.2
+    compute:
+      bridged_ip: 192.168.1.93
+      private_ip: 10.2.20.3
+You are now ready to "Launch Vagrant" (see below)
-=== Set up your control node: ===
+Vagrantfile.yml Explanation
+==========================
+Warning, make sure there are no trailing white spaces in this file
+**nat_mode:** set to yes for nat mode, leave blank for bridge mode
+**bridge:** name of your bridge interface ($ brctl show ) leave blank for nat mode
+**netmask:** netmask of your private subnet, probably given to you via dhcp. you can see this with ifconfig, however on osx if will be in the unreadble format, something like 0xffffff00 Refer here for a table that human can read. http://www.pawprint.net/designresources/netmask-converter.php Most home networks only give out a /24 you will need to log into your router and change your range to at least a /23 so that we an properly route to the router that neutron creates.
+**gateway:**  For bridged mode Your workstations gateway to the internet (your routers ip, this is also the ip you go to to increase your network size
+) you can check this with ip r on linux or netstat -nr on osx
+For nat mode set this to the first ip in the range you are choosing for private_ip
+**neutron_router_start:** This will be the start of your openstack dhcp, I also use this as your neutron router gateway. give neutron its own /24 range
+**neutron_router_end:** the end of the range explained above
+controller:
+  **bridged_ip:** this interface should be given an ip on the same /24 as your workstation.
+  **private_ip:** this interface can have any ip you want, virtualbox deals with the routing.
+compute:
+  **bridged_ip:** same rules as the controller bridged_ip but unique
+  **private_ip:** same rules as controller: private_ip but unique
+for nat mode set the bridged_ip and private_ip to the same values for each host (as seen in Vagrantfile.yml.template.natmode)
+Launch Vagrant
+==============
+    vagrant up
 ssh into the vagrant controller (password is vagrant)
-  $ vagrant ssh controller
+    vagrant ssh controller
-Once you have connected to the controller VM you will need run packstack, the environment variables you have prepared in the Vagrantfile.yml will be carried forward by Vagrant and used in the packstack scripts.
+run packstack (for nat mode complete steps below first)
-  [vagrant@controller]# cd /vagrant
+    cd /vagrant
-  [vagrant@compute]# sudo bash
+    sudo bash
-  [root@compute]# packstack  --answer-file=ans.txt && cp /root/keystonerc_admin /vagrant
+    packstack  --answer-file=ans.txt && yes|cp /root/keystonerc_admin /vagrant
-=== Set up your Network node ===
+the answerfile is generated from ans.template or ans.NAT.template when you run vagrant up.
+packstack should now prompt you for the root password of both nodes. The password is "vagrant"
+if packstack fails for some reason, just run it again.
+Networking
+==========
 To setup networking, and launch the cirros minimal VM you must wait for the above operations to complete. (packstack and copying the keystonerc_admin) Once those are done, vagrant ssh into the networking (compute node):
-  $ vagrant ssh compute
+    vagrant ssh compute
-  [vagrant@compute]# sudo bash
+    [vagrant@compute]# sudo bash
-  [root@compute ]# cd /vagrant && ./SetupComputeNode
+    [root@compute ]# cd /vagrant && ./SetupComputeNode
+That's it everything should work now.
+Testing
+=======
+Dashboard
+Natmode:
+http://localhost:8080/dashboard/
+Bridgemode:
+http://compute.bridged.ip from your vagtantfile.yml
+CirrosVM:
+ssh into the CirrosVM spawned by ./SetupComputeNode and ping the outside world
+    [root@compute vagrant]# source keystonerc_admin
+    [root@compute vagrant(keystone_admin)]# neutron floatingip-list
+    +--------------------------------------+------------------+---------------------+--------------------------------------+
+    | id                                   | fixed_ip_address | floating_ip_address | port_id                              |
+    +--------------------------------------+------------------+---------------------+--------------------------------------+
+    | ea3d5757-e646-4d6e-9c0d-e6304cee3ff0 | 172.17.0.2       | 10.0.23.2           | 53157795-741e-479c-afb6-1ceb26fd500e |
+    +--------------------------------------+------------------+---------------------+--------------------------------------+
+    [root@compute vagrant(keystone_admin)]# ssh cirros@10.0.23.2
+    cirros@10.0.23.2's password: cubswin:)
+    $ ping 8.8.8.8
+    PING 8.8.8.8 (8.8.8.8): 56 data bytes
+bytes from 8.8.8.8: seq=0 ttl=50 time=24.782 ms
+bytes from 8.8.8.8: seq=1 ttl=50 time=23.527 ms
+Tools
+=====
+Ideally this sandbox will be loaded with usefull tools enumerated here. Right now there are some scripts that I use to setup the networking node
+SetupNeutron: This sets up neutron with a router for external connectivity for your VM's, this file is generated by ./build_SetupNeutron
+SwitchToQemu: KVM is not supported inside virtualbox, this script switches to qemu
+LaunchCirrosVM: Launches a vm with the name $1
+DeleteNetwork: Runs throught some loops and removes all openstack networking, must be run on the compute node
+Trouble-shooting
+================
+When restarting netwoking, the neutron switch become unresponsive, you'll need to restart various neutron components
+    service network restart
+    for i in dhcp-agent l3-agent metadata-agent openvswitch-agent; \
+    do service neutron-$i restart; done
+    neutron agent-list
+    #takes me 38 seconds before I can ping a the router
+Vagrant exits with a syntax error
+    Message: undefined method `[]' for nil:NilClass
+Try running the included ./testyaml you may need to install the ruby yaml library
+Vagrant Can't download the box on OSX
+    vagrant box add --name controller https://build.opnfv.org/downloads/controller.box
+    vagrant init controller
-===== Trouble-shooting =====
+This will help you debug some wierd permission erros that we've seen on osx
-=== Neutron switch is unresponsive after restart ===
+Wierd locale issue.
-When restarting networking, the neutron switch become unresponsive, you'll need to restart various neutron components
-  service network restart
+    ERROR : Error appeared during Puppet run: 10.0.20.2_glance.pp
-  for i in dhcp-agent l3-agent metadata-agent openvswitch-agent; \
+    Notice: /Stage[main]/Glance::Registry/Exec[glance-manage db_sync]/returns: ValueError: unknown locale: UTF-8
-  do service neutron-$i restart; done
-  neutron agent-list
-  #takes me 38 seconds before I can ping a the router
-=== Vagrant fails to execute when running vagrant up ===
+Edit your /etc/ssh_config file on your Mac OS X system and remove LC_CTYPE from SendEnv. This will cause the ssh client to stop propagating LC_CTYPE to the ssh servers.
-Vagrant may exit with a syntax error
-  Message: undefined method `[]' for nil:NilClass
-Try running the included ./testyaml you may need to install the ruby yaml library.
+Caveats
-On OSX this has not yet been solved and the workaround is to enter the configuration parameters in the Vagrantfile.yaml directly into the declarations at the start of the Vagrantfile.
+======================
+Vagrant reconfigures the network device eth1 on boot.
+You will need to run /vagrant/SetupComputeNodeAfterReboot each time the compute node is rebooted.
-=== Vagrant Can't download the box on OSX ===
-If you see this error when running vagrant up the first time.
-  ==> controller: Importing base box 'controller'...
-  Progress: 90%/Applications/Vagrant/embedded/gems/gems/vagrant-1.6.5/lib/vagrant/machine.rb:259:in `initialize': Permission denied - /PackStackSandBox/.vagrant/machines/controller/virtualbox/id (Errno::EACCES)
-This has something (not sure what) to do with write authority on OSX, the workaround is to manually add the box using vagrant...
-  vagrant box add --name controller https://build.opnfv.org/downloads/controller.box
-  vagrant init controller
-=== Vagrant is unable to bring up the VM's ===
+Contribute!
-You may see this error when running //vagrant up//: (noticed on OSX)
+===========
-  PackStackSandBox$ vagrant up
-  Bringing machine 'controller' up with 'virtualbox' provider...
-  Bringing machine 'compute' up with 'virtualbox' provider...
-  /Applications/Vagrant/embedded/gems/gems/vagrant-1.6.5/lib/vagrant/machine.rb:295:in `unlink': Permission denied - /PackStackSandBox/.vagrant/machines/controller/virtualbox/id (Errno::EACCES)
-In this case you can get around the problem by running as root:
-  PackStackSandBox$ sudo vagrant up
-==== Things that still suck ====
+Fork this repo
+Create your feature branch (git checkout -b my-new-feature)
+Commit your changes (git commit -am 'Add some feature')
+Push to the branch (git push origin my-new-feature)
+Create new Pull Request
-Vagrant reconfigures the network device eth1 on boot. even tho I have managed set to false. this breaks everything. Gar.

Wiki

User Tools

Site Tools

Differences

Page Tools