===== Multisite Project Meeting ===== ==== Logistics ==== * When: **Every Thursday 8:00-9:00 UTC** * To join the meeting: * **IRC channel:** ''#opnfv-meeting'' @ Freenode [[http://webchat.freenode.net/?channels=opnfv-meeting|Web Chat]]; * **Web Conferencing**: ''https://global.gotomeeting.com/join/959683557'' ==== Agenda of the week ==== * Agenda is sent in the OPNFV mail-list every week: opnfv-tech-discuss@lists.opnfv.org * Please subscribe the mail-list for the agenda ==== Past Meeting Agenda and Minutes ==== Mar.10, 2016: Agenda & Minutes * Agenda * Kingbird Release, follow OpenStack Mitaka or OPNFV Release C? * Release C preparation. * Other topics * Minutes * 1. Finilize work in KB for the first mile-stone with OpenStack mitaka (joehuang, 08:42:36) * 1.5 CURL FT test (joehuang, 08:42:57) * 2. Tag for mitaka (joehuang, 08:43:05) * 3. Work on integration in OPNFV (joehuang, 08:43:13) * 4. preparation of pypi/tarball for a release (joehuang, 08:49:00) * Links * Minutes: http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2016/opnfv-meeting.2016-03-10-08.03.html * Minutes (text): http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2016/opnfv-meeting.2016-03-10-08.03.txt * Log: http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2016/opnfv-meeting.2016-03-10-08.03.log.html Mar.3, 2016: Agenda & Minutes * Agenda * Kingbird quota management - API * Other topics * Minutes * http://developer.openstack.org/api-ref-blockstorage-v2.html#os-quota-sets-v2 (sorantis, 08:25:11) * https://review.openstack.org/#/c/284137/2/kingbird/api/README.rst (SAshish, 08:26:29) * remove post API request (SAshish, 08:29:50) * Links * Minutes: http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2016/opnfv-meeting.2016-03-03-08.04.html * Minutes (text): http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2016/opnfv-meeting.2016-03-03-08.04.txt * Log: http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2016/opnfv-meeting.2016-03-03-08.04.log.html Feb.25, 2016: Agenda & Minutes * Agenda * Multisite installation/configuration/test in Release C * Geo-site disaster recovery leveraging volume replication * Oslo configure sample generation, currently two configuration sample file maintained manually * Other topics in Kingbird quota management * Minutes * ACTION: contact test team for multi-region env. establishment and test (joehuang, 08:40:59) * https://specs.openstack.org/openstack/cinder-specs/specs/mitaka/cheesecake.html (joehuang, 08:43:48) * https://review.openstack.org/#/c/275797/ (joehuang, 08:44:08) * https://blueprints.launchpad.net/cinder/+spec/replication (SAshish, 08:53:31) * Links * Minutes: http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2016/opnfv-meeting.2016-02-25-08.17.html * Minutes (text): http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2016/opnfv-meeting.2016-02-25-08.17.txt * Log: http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2016/opnfv-meeting.2016-02-25-08.17.log.html Feb.18, 2016: Agenda & Minutes * Agenda * Periodic quota sync. in Kingbird Engine. * Kingbird API structure for quota limits, synchronisation, etc. * Geo-site disaster recovery leveraging volume replication * Multisite installation/configuration/test in Release C * Minutes * https://review.openstack.org/#/c/277489/ (SAshish, 08:10:54) * AGREED: configuration about quota sync timer is ok, but need guide that not too much nodes enbaled, 2 or 3 nodes are ok (joehuang, 08:20:54) * http://developer.openstack.org/api-ref-compute-v2.1.html (joehuang, 08:27:29) * AGREED: similar url structure like Nova/Cinder for quota management (joehuang, 08:45:30) * http://127.0.0.1:8118/v1.0/{admin_tenant_id}/quota/sync/{tenant_id} (SAshish, 08:47:25) * AGREED: sync /v1/{project_id}/quotas/sync (joehuang, 08:52:04) * https://review.openstack.org/#/c/277802/ (SAshish, 08:54:01) * rework Quota Sync APIs (SAshish, 09:09:08) * Links * Minutes: http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2016/opnfv-meeting.2016-02-18-08.03.html * Minutes (text): http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2016/opnfv-meeting.2016-02-18-08.03.txt * Log: http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2016/opnfv-meeting.2016-02-18-08.03.log.html Jan.28, 2016: Agenda & Minutes * Agenda of Jan. 28, 2016 * Quota management implementation option discussion. * Minutes of Jan. 28, 2016 * https://github.com/svinota/pyroute2/issues/99 (SAshish, 08:31:56) * http://stackoverflow.com/questions/3044580/multiprocessing-vs-threading-python (SAshish, 08:35:39) * ACTION: Ashish to check Senlin’s implementation of Actions (sorantis, 08:42:53) * ACTION: zhiyuan do a test see if eventlet works (sorantis, 08:43:02) * ACTION: Ashish check ThreadGroupManager and eventlets (SAshish, 08:55:31) * https://github.com/openstack/senlin/blob/master/senlin/engine/scheduler.py (sorantis, 08:56:50) * https://github.com/openstack/senlin/blob/master/senlin/engine/scheduler.py (sorantis, 09:00:38) * https://github.com/openstack/senlin/blob/master/senlin/engine/scheduler.py (sorantis, 09:00:58) * Link * Minutes: http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2016/opnfv-meeting.2016-01-28-08.11.html * Minutes (text): http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2016/opnfv-meeting.2016-01-28-08.11.txt * Log: http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2016/opnfv-meeting.2016-01-28-08.11.log.html Jan.21, 2016: Agenda & Minutes * Agenda of Jan. 21, 2016 * Quota management implementation * Multi-region devstack setup * Multisite use cases collection for next release * Minutes of Jan. 21, 2016 * http://www.openstack.org/assets/telecoms-and-nfv/OpenStack-Foundation-NFV-Report.pdf (joehuang, 08:10:37) * ACTION: have beer (zhipeng, 08:17:19) * ACTION: send a mail for inquiry to other teams for multisite requirements, but at the same time we mainly focus on current use cases (joehuang, 08:18:06) * kingbird design (joehuang, 08:19:41) * ACTION: summary of different idea and comparation of quota management implementation (joehuang, 08:43:00) * Link: * Minutes: http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2016/opnfv-meeting.2016-01-21-08.08.html * Minutes (text): http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2016/opnfv-meeting.2016-01-21-08.08.txt * Log: http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2016/opnfv-meeting.2016-01-21-08.08.log.html Jan.14, 2016: Agenda & Minutes * Agenda of Jan. 14, 2016 * Discussion of Kingbird BPs and progress. * Next release discussion. Goal of kingbird in OPNFV release C * Open to openstack community for more contributor? * Minutes of Jan. 14, 2016 * https://blueprints.launchpad.net/kingbird/+spec/kingbird-openstack-driver (SAshish, 08:10:43) * ACTION: Ashish, add your patches under the BP whiteborad and change the BP status (SAshish, 08:13:22) * http://stackalytics.com/?project_type=all&module=kingbird (joehuang_, 08:15:25) * ACTION: Ashish So I think from the configured one would be the right one (SAshish, 08:21:31) * https://github.com/openstack/tricircle (joehuang_, 08:50:34) * Link: * Minutes: http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2016/opnfv-meeting.2016-01-14-08.08.html * Minutes (text): http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2016/opnfv-meeting.2016-01-14-08.08.txt * Log: http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2016/opnfv-meeting.2016-01-14-08.08.log.html Dec.10, 2015: Agenda & Minutes * Agenda of Dec. 10, 2015 * Discussion of Kingbird BPs and progress. * Minutes of Dec. 10, 2015 * Discussion of Kingbird BPs and progress * all projects' quota will be synced batch by batch (SAshish, 08:29:44) * ACTION: update the BP for the driver (joehuang, 08:38:56) * ACTION: change openstackclient usage (sorantis, 08:40:05) * Link: * Minutes: http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2015/opnfv-meeting.2015-12-10-08.10.html * Minutes (text): http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2015/opnfv-meeting.2015-12-10-08.10.txt * Log: http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2015/opnfv-meeting.2015-12-10-08.10.log.html Dec.3, 2015: Agenda & Minutes * Agenda of Dec. 3, 2015 * Documentation task in OPNFV Multisite * Discussion of Kingbird BPs and progress. * Minutes of Dec. 3, 2015 * 1. documentation (joehuang, 08:06:19) * https://wiki.opnfv.org/documentation/tools (r-mibu, 08:08:45) * 2. kingbird (joehuang, 08:13:25) * a.AGREED: native clients as of now for the features which are not present in openstackclient (joehuang, 08:30:14) * b.AGREED: for all supported features, we will use OSclient (joehuang, 08:30:23) * c.ACTION: check tox error (joehuang, 08:44:00) * d.we can always submit a bug though or a feature request, but we need to focus on kingbird itself (joehuang, 08:50:18) * Link: * Minutes: http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2015/opnfv-meeting.2015-12-03-08.05.html * Minutes (text): http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2015/opnfv-meeting.2015-12-03-08.05.txt * Log: http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2015/opnfv-meeting.2015-12-03-08.05.log.html Nov.19, 2015: Agenda & Minutes * Agenda * Discussion of Kingbird BPs and progress * 1. discussion Kingbird BPs and progress (joehuang, 08:09:18) * https://docs.google.com/document/d/1aYmhfxdlKVhv3j1NGrrfSXnyonfKv12jv6KURdwMMKI/edit?usp=sharing (sorantis, 08:42:08) * ACTION: ashish to make a spec (sorantis, 08:45:02) * ACTION: for the openstack client (sorantis, 08:46:10) * ACTION: Dimitri to commit unit test execution script (sorantis, 08:49:19) * https://github.com/openstack/kingbird/tree/master/doc/source (joehuang, 08:53:55) * have a readme file for each submodule describing how to run/use it (joehuang, 08:55:49) * 2. Link: * Minutes: http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2015/opnfv-meeting.2015-11-19-08.07.html * Minutes (text): http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2015/opnfv-meeting.2015-11-19-08.07.txt * Log: http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2015/opnfv-meeting.2015-11-19-08.07.log.html Nov.5, 2015: Agenda & Minutes * Agenda * Use case 1&2&3 progress in OpenStack summit and close in review * Jira ticket of configuration / user guide for use case 1&2&3 decomposition * Kingbird team establishment and operation in OpenStack. * link * Meeting log: * http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2015/opnfv-meeting.2015-11-05-09.10.log.html Oct.14, 2015: Agenda & Minutes * Kingbird framework and BP * Ref: https://launchpad.net/kingbird * Ref: https://blueprints.launchpad.net/kingbird/+spec/kingbird-framework * Ref: https://blueprints.launchpad.net/kingbird/+spec/quota-sync-api * JIRA ticket follow up * Minutes * Kingbird framework and BP (joehuang, 08:06:50) * ACTION: add Kingbird to the OpenStack CI system (sorantis, 08:12:23) * http://docs.openstack.org/infra/manual/creators.html#adding-the-project-to-the-ci-system (sorantis, 08:14:05) * https://wiki.opnfv.org/wiki/weekly_technical_discussion (Malla, 08:32:30) * https://jira.opnfv.org/browse/MULTISITE-10 (sorantis, 08:40:38) * https://jira.opnfv.org/browse/MULTISITE-9 (sorantis, 08:40:47) * ACTION: contact Genesis about keystone deployment in multisite scenario (joehuang, 08:48:00) * https://wiki.opnfv.org/multisite/bug_bp and mention that we’ve already identified some issues (sorantis, 08:52:39) * ACTION: ask Heather if Kingbird can reuse the OPNFV logo (sorantis, 08:56:30) * ACTION: address the Jira tickets from Chris. Describe the first three use cases. (sorantis, 08:57:31) * ACTION: also mention the identified issues from https://wiki.opnfv.org/multisite/bug_bp (sorantis, 08:58:03) * ACTION: write the quota management documentation part (sorantis, 08:58:38) * #agreed start with quota management use case * link * Meeting log: * Minutes: http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2015/opnfv-meeting.2015-10-15-08.04.html * Minutes (text): http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2015/opnfv-meeting.2015-10-15-08.04.txt * Log: http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2015/opnfv-meeting.2015-10-15-08.04.log.html Oct.1, 2015: Agenda & Minutes * Agenda * Use case 4/5 Centralized service in multisite * Ref :https://etherpad.opnfv.org/p/centralized_quota_management * Ref :https://etherpad.opnfv.org/p/multisite_centralized_service * Ref :http://docs.kingbird.apiary.io/# * JIRA ticket follow up * Minutes * AGREED: the deadline for usecase 1/2/3 review is Oct. 25 (joehuang, 08:12:23) * usecase 4 (joehuang, 08:15:32) * https://etherpad.opnfv.org/p/multisite_centralized_service (joehuang, 08:17:36) * http://docs.kingbird.apiary.io/# (joehuang, 08:18:01) * https://etherpad.opnfv.org/p/centralized_quota_management (joehuang, 08:18:22) * ACTION: update the api definition for quota of the Kingbird service (joehuang, 08:33:53) * ACTION: open new project under OpenStack, start from OPNFV first (joehuang, 08:38:09) * https://github.com/opnfv (Malla, 08:38:19) * link * Meeting log: * [16:57] Minutes: http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2015/opnfv-meeting.2015-10-01-08.02.html * [16:57] Minutes (text): http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2015/opnfv-meeting.2015-10-01-08.02.txt * [16:57] Log: http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2015/opnfv-meeting.2015-10-01-08.02.log.html Sept 24, 2015: Agenda & Minutes * Agenda * JIRA ticket follow up and sprint plan for B release * Use case 4: Centralized service for resources management and/or replication (sync tenant resources like images, ssh-keys, security groups, etc) * one solution for quota management "https://etherpad.opnfv.org/p/centralized_quota_management". * Minutes * jira ticket and sprint (joehuang, 08:06:02) * could we finish review for use case 1/2/3 before the OCT 25, so that we finish the sprint 1, after that we spend 2 month for use case 4/5 review * [16:08] and we are now discussing use case 4 and later use case 5 * [16:08] there are use case 1/3 in review/approve process in gerrit * [16:09] and because Colin has some issue in access gira, I will help him prepare use case 2 for review and approve * [16:09] I’ll review the commits asap * use case 4 discussion (joehuang, 08:11:56) * https://etherpad.opnfv.org/p/multisite_usecase_collection (joehuang, 08:13:33) * https://etherpad.opnfv.org/p/centralized_quota_management (joehuang, 08:16:38) * http://openstack-in-production.blogspot.se/2015/03/nova-quota-usage-synchronization.html (sorantis, 08:26:05) * https://docs.google.com/document/d/19BXf0RhkH8wEEymE2eHHqoDZ67gnzgvpr3atk4qwdGs/edit (joehuang, 08:45:05) * https://wiki.openstack.org/wiki/Tricircle (joehuang, 08:48:35) * [16:53] I also have another idea to have a standalone service for distributed cloud, used for post control for quota , but group resource view and proactive on demand replication for ssh keys/ image/seg * [16:54] but service provisioning for VM/volume/network will be directly called to each region seperately * [16:55] ok, that’s what I’m also aiming for. Have a service working aside, quite transparent, which has zero impact on the openstack codebase * [16:55] even for ceilometer part ( use case 5 ), view will be generated with task maner and collect information on demand * [16:57] the centralized service will collect usage from each region periodicly, and send alarm to tenant if the quota is exceeded * [16:57] this will be post control * [16:58] without any action taken? * [16:59] if need, then your proposal is a good compliment * [17:01] good. will you have time to describe your idea on etherpad before the next meeting ? * [17:01] I'll * link * Meeting log: * Minutes: http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2015/opnfv-meeting.2015-09-24-08.04.html * Minutes (text): http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2015/opnfv-meeting.2015-09-24-08.04.txt * Log: http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2015/opnfv-meeting.2015-09-24-08.04.log.html Sept 10, 2015: Agenda & Minutes * Agenda * 1. Jira issue follow up * a. we have 3 agenda today (joehuang, 08:06:57) * b. Jira issue follow up (joehuang, 08:07:14) * c. 2 collboration with HA team (joehuang, 08:07:36) * d. 3 use case 4 discussion (joehuang, 08:07:47) * e. we have 8 issues registered now (joehuang, 08:08:22) * f. MULTISITE-5 resolved (sorantis, 08:10:45) * g. I just updated the commiter list, Dimitri ( sorantis ) is added to the list of committer (joehuang, 08:12:37) * 2. colloboration with HA team (joehuang, 08:19:29) * a. https://gerrit.opnfv.org/gerrit/#/c/1072/ (fuqiao, 08:21:54) * b. https://wiki.opnfv.org/_media/multisite/multisite_bp-bug_review_0910.pdf (joehuang, 08:29:05) * c. action 1: review multisite gerrit and provide comments (fuqiao, 08:30:35) * d. action 2: analyse multisite usecases, include it in the usecase doc, and refer to the multisite project work (fuqiao, 08:31:05) * e. action 3: analyse multisite usecases, see if we need to add any other requirements for the other components other than openstack(fuqiao, 08:31:51) * f. https://etherpad.opnfv.org/p/VNF_Geo_site_redundancy (sorantis, 08:38:21) * 3. use case 4 (joehuang, 08:53:02) * a. https://etherpad.opnfv.org/p/centralized_quota_management (sorantis, 08:56:07) * 4. Meeting log: * Minutes: http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2015/opnfv-meeting.2015-09-10-08.04.html * Minutes (text): http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2015/opnfv-meeting.2015-09-10-08.04.txt * Log: http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2015/opnfv-meeting.2015-09-10-08.04.log.html Sept 3, 2015: Agenda & Minutes * Agenda * Use case 3: VNF Geo-site Redundancy * JIRA issue track * Minutes * ACTION: update the keystone commit, joehuang (joehuang, 08:05:51) * VNF Geo site redundancy (joehuang, 08:06:11) * https://etherpad.opnfv.org/p/VNF_Geo_site_redundancy (joehuang, 08:06:44) * three proposal were presented in the etherpad for VNF geo site redundancy (joehuang, 08:07:18) * the purpose to expose the quiesce/unquiesce API directly is to make transactional snapshot of a group of VMs is possible (joehuang, 08:15:37) * …in Nova (sorantis, 08:16:10) * ACTION: update requirements to openstack for 3 options, joehuang (joehuang, 08:50:40) * AGREED: requirements perspective we could recommend all three options, that it is an operator choice (joehuang, 08:53:09) * ACTION: add example, sorantis #1, joehuang #2, colin #3 (joehuang, 08:55:32) * link * Minutes: http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2015/opnfv-meeting.2015-09-03-08.02.html * Minutes (text): http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2015/opnfv-meeting.2015-09-03-08.02.txt * Log: http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2015/opnfv-meeting.2015-09-03-08.02.log.html Aug 27, 2015: Agenda & Minutes * Agenda * Use case 3: VNF Geo-site Redundancy * using JIRA for action item * related BP&bug: https://wiki.opnfv.org/multisite/bug_bp * * Minutes * for the use case 1, after the prototype, one bug regirstered (joehuang, 08:05:58) * https://bugs.launchpad.net/keystonemiddleware/+bug/1488347 (joehuang, 08:06:25) * after the bug solved, we at least one way to achieve use case 1 goal (joehuang, 08:07:29) * we'll not put more focus on use case 1 then. Only other requirement or issue found (joehuang, 08:08:06) * for use case 2, conclusion in the meeting is that there are cross Neutron L2/L3 networking requirement to OpenStack (joehuang, 08:09:09) * and for use case 2, two RFE( request for exception ) were registered (joehuang, 08:09:47) * shall we co-sign on the BP so it shows it is from collective effort from Multisite project? (zhipeng, 08:13:08) * https://wiki.opnfv.org/multisite/bug_bp (zhipeng, 08:13:54) * co-author for BP/coding (joehuang, 08:14:02) * or register new BP for same purpose (joehuang, 08:14:23) * ACTION: Colin to rephrase the use case, and a JIRA item will be created for this (zhipeng, 08:18:04) * ACTION: zhipeng to contact HA team about DR discussion (zhipeng, 08:21:15) * the rough idea is to leverage the volume replication and manage/un-manag function of cinder (joehuang, 08:25:31) * link * [16:58] Minutes:http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2015/opnfv-meeting.2015-08-27-08.01.html * [16:58] Minutes (text): http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2015/opnfv-meeting.2015-08-27-08.01.txt * [16:58] Log: http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2015/opnfv-meeting.2015-08-27-08.01.log.html Aug 20, 2015: Agenda & Minutes * Agenda * short summary of use case 1 and use case 2 * Use case 3: VNF Geo-site Redundancy * Minutes * ACTION: send mail about keystonemiddleware in openstack-dev * link * Log: http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2015/opnfv-meeting.2015-08-20-08.12.log.html * Minutes: http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2015/opnfv-meeting.2015-08-20-08.12.html * Minutes (text): http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2015/opnfv-meeting.2015-08-20-08.12.txt Jul 23, 2015: Agenda & Minutes * Agenda * Use case 1: multisite identity service management(short summary of the prototype) * Architecture proposal discussion, 1st time * Use case 3: VNF Geo-site Redundancy * reference https://etherpad.opnfv.org/p/multisite_identity_management, https://etherpad.opnfv.org/p/multisite_usecase_collection * Minutes * I have done the prototype of identity management (joehuang, 08:09:33) * based on hafe's work (joehuang, 08:09:45) * the asrnc replication between mysql galera cluster works (joehuang, 08:10:27) * that means fully distributed keystone service for fernet token is feasible (joehuang, 08:11:47) * but I prefer the new idea of the replication, see the candidate solution 3 (joehuang, 08:12:36) * https://etherpad.opnfv.org/p/multisite_identity_management (joehuang, 08:13:46) * shall we include hybrid cloud scenario in the identity management use case? (joehuang, 08:31:21) * it would be better if we have firmer examples (joehuang, 08:31:56) * resume weekly meeting from Aug.20 after summer vacation season. (joehuang, 08:42:41) * meeting-log * http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2015/opnfv-meeting.2015-07-23-08.05.html * http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2015/opnfv-meeting.2015-07-23-08.05.txt * http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2015/opnfv-meeting.2015-07-23-08.05.log.html Jul 16, 2015: Agenda & Minutes * Agenda * Use case 1 multisite identity service management(If the prototype finished in time) * Use case 2 VNF high availability across VIM( focus on requirements to OpenStack ) * reference https://etherpad.opnfv.org/p/multisite_identity_management, https://etherpad.opnfv.org/p/multisite_usecase_collection * Minutes * use case 1 identity service management (joehuang, 08:06:19) * 08:16:41 My point is that we should be clear about what changes we want to allow if the network is partitioned, and then critically how the resulting system converges when the partition ends. * 08:17:23 Many telcos have a very hard requirement that geographic sites should be able to operate as isolated entities to deal with earthquake. flood, fire, etc knocking out interconnects. * 08:17:53 They normally need to ability to make changes in the isolated site to deal with changing circumstances * if not each site installed with KeyStone service, we have to process "Escape from site level KeyStone failure" use case. I'll go on the prototype, and let's mark what colin's concerns and see how to address it (zhipeng, 08:24:52) * use case 2. VNF high availability across VIM * 08:32:37 #agreed #2 is intrasite clouds, #3 is intersite clouds * 08:38:13 To my the biggest difference between #2 & #3, is that #2 is all about maintaining media/signalling and calls (which requires IP transfer), whilst #3 is about restoring/continuing service but most likely not calls. * 08:38:49 (use case)#3 does not require special openstack networking support, (use case)but #2 does. * 08:39:32 #agreed inter-cloud intra-site L2 and L3 networking enhancement is one requirement from OPNFV Multisite to OpenStack * 08:42:06 Agree, but we need to describe that from two aspect, one is for VNF communication to other VNF (inter-VNF), another one is for VNF internal communition for heart-beat, session replication(intra-VNF) * For L2 the major requirements relate to config/management of those networks. For L2 do you need to use provider networks? Exactly how do you disable anti-spoof support? etc. For L3 it might make sense to have a common neutron api for the take IP/ free IP support, which can then be plumbed onto multiple underlying technologies. In fact it (zhipeng, 08:43:59) * may even make sense to use the same API for L2, just have it trigger GARP. (fzdarsky, 08:44:42) * 08:46:12 The latter leads onto cross-cloud tenant networks * 08:46:47 agree, it's cross-cloud tenant networks for intra-VNF traffic * 08:57:06 #agreed second req out of use case #2 given intra-site inter-cloud enhancement on L2/L3 IP traffic transfer (inter and intra VNF) is a requirement from OPNFV Multisite to OpenStack * ACTION: colintd_ to reword the req to be more accurate :) (zhipeng, 08:57:26) * link(meetbot doesn't work well, only log linked here, refer to "Minutes" section for the summary): * [17:00] <@collabot> Log: http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2015/opnfv-meeting.2015-07-16-08.01.log.html Jul 9, 2015: Agenda & Minutes * Agenda * Use case 1 multisite identity service management * reference https://etherpad.opnfv.org/p/multisite_identity_management * Minutes * multisite identity service management (zhipeng, 08:03:35) * https://etherpad.opnfv.org/p/multisite_identity_management (joehuang, 08:03:43) * http://openstack-in-production.blogspot.se/2014/10/kerberos-and-single-sign-on-with.html (hafe, 08:31:39) * http://indico.cern.ch/event/283833/contribution/0/attachments/523615/722236/summitHK2013_All_v2.pdf (joehuang, 08:33:09) * AGREED: "a user should, using a single authentication point be able to manage virtual resources spread over multiple OpenStack regions" (zhipeng, 08:40:29) * clarification for keystone federation application scenario, and PKI token discussion * solution 2 prototype, Fernet token works if single slave is supposed to be read only * link: * http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2015/opnfv-meeting.2015-07-09-08.00.log.html * http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2015/opnfv-meeting.2015-07-09-08.00.html * http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2015/opnfv-meeting.2015-07-09-14.00.txt Jul 2, 2015: Agenda & Minutes * Agenda * use cases prioritization * B-release goal discussion * minimum viable use cases * want but at risk use cases * out plan use cases * Minutes * use cases merged: * Use case 1: multisite identity service management * Use case 2. VNF high availability across VIM * Use case 3. VNF Geo-site Redundancy * Use case 4. Centralized service for resources management and/or replication (sync tenant resources like images, flavor, ssh-keys, security groups, etc) * Use case 5. centralized monitoring service * proposed use case prioritization, 1, (2), 3,4,5. But the diversity is still on the use case 2, for those who need this deployment scenario, it's the highest priority, but for those who don't want, it's the lowest one. * at least deliver use cases/requirements/gap analysis for B-release, for BP/spec/code approvement, it's up to OpenStack * [[https://wiki.opnfv.org/multisite/minutes/log-07-02|meeting log]] Jun 25, 2015: Agenda & Minutes * Agenda * use case 2 discussion: * IP movement between OpenStack instances * Cross Neutron L2/L3 networking for heart-beat/state(session)replication * image replication between OpenStack instances * Minutes (the meetbot doesn't work today) * [16:07] #info today's agenda: IP movement between OpenStack instances, Cross Neutron L2/L3 networking for heart-beat/state(session)replication * [16:08] #info if we have time, then image replication between OpenStack instances * [16:10] How about address-pair to be used in the IP movement to archieve the HA for VNF across OpenStack * [16:15] the IP address to communicate with other VNF/PNF should be an IP address from provider network * [16:15] because the VNF has to talk to PNF at the pratical deployment * [16:26] but do we necessarily need to automate this with provider network devices? * [16:29] I guess, you can configure the physical router prior to, let’s say MME deployment? and then provide the configured IP addresses upon deployment of MME. * [16:31] you can have a separate tool working with that specific provider physical hardware and producing these IP addresses, that later on could be used as deployment parameters for MME * [16:31] if you mean that this tool is part of VNFM then yes * [16:32] on the other hand why should VNFM be concerned about this. * [16:32] A VNF can expect some input parameters. The values for this parameters can come from anywhere, as long as they are correct for deployment * [16:33] so I wouldn’t necessarily harden this function into VNFM * [16:33] just want to clarify whether these IP address management should be the responsibility of VIM * [16:34] we can exclude this functionality (IP address management, which is used for interaction among VNF/PNF) from VIM, i.e from OpenStack * [16:35] i think so * [16:35] i mean at least for this case * [16:36] Sorry to interrupt: some SDN solutions want to manage the IPs for the VNFs, so this should be included * [16:37] can SDN be used for that? * [16:37] these SDN controller running above OpenStack, and manipulate VNF, isn't it * [16:39] SDN controller interacts with network devices, no? sorry, I’m not very familiar with SDN * [16:39] to tallgren: can you eloborate how to do that * [16:40] When your VNF starts and creates a port, it gets an IP from the SDN controller * [16:41] SDN controller acting as DHCP server * [16:41] but currently, the IP/mac is allocated in Neutron, or with one driver * [16:41] so can we say that this is the SDN controller’s responsibility to provide an IP address for the VNF? * [16:42] that would be an SDN controller plugged under Neutron, right? * [16:42] yes * [16:43] With PNFs, the OSS/BSS currently configures FQDNs (via the EM) and the PNFs then resolve to IP via DNS. * [16:44] When you create a network in OpenStack (Neutron), you need to know how the IP management will be done * [16:44] The IP being assigned from a managed address pool by DHCP. * [16:44] Yes * [16:45] so the IP is from a VLAN provider network, or as a floating IP from external network * [16:46] from your description, it's from a VLAN provider network? * [16:49] ok, how the standby will get the IP before it become the master, if the mater failed * [16:49] Is there a figure somewhere that illustrates the use case? * [16:49] we can draw a figure after the meeting, but not now * [16:50] BTW, I am not sure if this is a side track, but floating IPs do not really work with IPv6 * [16:50] sorry, I change a littel bit the subject of discussion, is there any VNF which should be d eployed across multiple site or multiple openstack instances? * [16:50] #action joehuang draw a figure for the use case 2, especially the IP address https://wiki.opnfv.org/_media/multisite/vnf_ha_across_vim.png * [16:51] this is the use case 2. two openstack instances in one site, and the master in one OpenStack, the standby in the other one * [16:51] I would not deploy a VNF across OpenStack instances * [16:51] xiaolong, not a single VNF instance, but one active and one standby instance. * [16:59] ok, thanks for the explanation. I think we need more clear definition about the use case * [16:59] If we talk about failover across regions that's a different thing * [17:00] failover between AZs technically means that you have to VNF instances running in the same cloud * [17:00] --> different API endpoints, no coordination between regions * [17:00] It's difficult to draw a conclution today to include the IP address management or not. Let's continue discuss next time. Before next meeting, hope that we can discuss in mailist. And whether the use case shoudl be addressed * [17:00] I also need to understand if we need to have a so complicated architecture : multi-AZ, multi-region inside multi-site Jun 18, 2015: Agenda & Minutes * Agenda * Gap analysis for use case 2 * Minutes * I'm was focussing on the need to be able to redirect external traffic between application instances sitting in one or more "local" clouds. (colintd, 08:36:58) * Where the example was suggested (I missed who) of a VNF implementing VRRP, with one instance in each of two clouds, and the question being what config/changes are needed to allow this to work ( is anti-spoof an issue) (zhipeng, 08:38:17) * I'm was focussing on the need to be able to redirect external traffic between application instances sitting in one or more "local" clouds. (colintd, 08:38:43) * We also talked about SND controllers, and how in many telco deployments these control much broader end-to-end traffice than simply intra-cloud. They might however be implemented using multiple redundant control nodes, say one per cloud, but providing a global function. (colintd, 08:39:46) * In this case neutron may be used to "connect" to those networks, but isn't the major control interface for the whole system, just a "joining" interface (colintd, 08:40:17) * Finally, returning to traffic failover, we talked about how for L2 failover can be triggered by apps just using GARP, but L3 requires protocol level (BGP/SDN) API. We also talked about how L3 convergence times (say BGP) might be too slow by default, especially in error cases (loss of node) as opposed to managed failover. (colintd, 08:42:03) * Moving the IP address is required when you have a core node serving lots of remote endpoints in the external network (e.g. voip phones with RTP streams). On failover it is too slow to resignal all of those, so you need to redirect traffic to the new node. Given routing is based on IP address, this needs to be moved * HA with VRRP http://blog.aaronorosen.com/implementing-high-availability-instances-with-neutron-using-vrrp/ (sorantis, 08:57:36) * We are looking at similar function between clouds, for which this may or may not need to be extended. May also be interplay with provider networks. (colintd, 08:58:18) * I will investigate before the next meeting. (colintd, 08:58:27) * link: * http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2015/opnfv-meeting.2015-06-18-08.07.html * http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2015/opnfv-meeting.2015-06-18-08.07.log.html * http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2015/opnfv-meeting.2015-06-18-08.07.txt Jun 11, 2015: Agenda & Minutes * Agenda * Gap analysis for use case 2, use case 4.1.2, use case 4.1.3, If we have enough time, then discuss use case 4.1.1 and 1 * Minutes * https://etherpad.opnfv.org/p/multisite_usecase_collection (zhipeng, 08:06:47) * Centralized resource management required for multi-site resource management. (Malla, 08:09:40) * Keystone supports regions concept as part of an endpoint. This enables communicating with all the registered regions via corresponding endpoints (sorantis_, 08:16:50) * promise project is working on resource management, maybe we can get some information if discuss with promise project folks. (Malla, 08:17:23) * does promise support multi-site? * to make things cear, let's talk about a concret use case: a user wants to know his total virtual resources (cpu, ram, disk) across multiple regions and multiple openstack instances, how can he do that? * can the user use a "for" loop? * no, if it needs that the user programs himself, it is a problem * I think creating another layer of APIs just for the sake for hiding a for loop is overkill * some "for" loop multi-tenancy discussion here * let's think more about the user cases, without talk about the presentation layer, is there any more operation we should do beside the simple "for loop", at least, aggregation (sum), sort, select? * Is there any one want to use Cells with shared Nova,Cinder,Neutron...in multisite? * cells probably are best to use withing a large datacenter * AGREED: : cells probably are best to use withing a large datacenter (joehuang, 08:46:21) * for me the stated use-case easily be addressed with a simple iteration * But if you look at the use case 4.1.4 * The resource utilization also should be controlled by quatos * Quota and usage discussion * Total resource view means your total quota in multi-site * if we user the same definitions as in nova, neutron, cinder, etc. then quota is a limit you apply on a resource type. Quota usage is the amount of resources currently in use * link: * Log: http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2015/opnfv-meeting.2015-06-11-08.02.log.html Jun 4, 2015: Agenda & Minutes * Agenda * Gap, Requirement, Use case discussion https://etherpad.opnfv.org/p/multisite_usecase_collection * Time schedule for folks from North America,etc to join * Minutes * Colin explains on the Geo-Redundancy (zhipeng, 08:09:57) * Colin mentions that the current cinder and swift replication scheme risks fault propagation (zhipeng, 08:26:49) * Joe explains various currently available OpenStack solutions for multi site scenario (zhipeng, 08:34:34) * Dimitri mentions that Nova Cell is now built in and may take momentum (zhipeng, 08:37:09) * Dimitri says CERN, Rachspace, Godady all use Cell, although with modification (zhipeng, 08:46:08) * link: * Minutes: http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2015/opnfv-meeting.2015-06-04-08.05.html * Minutes (text): http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2015/opnfv-meeting.2015-06-04-08.05.txt * Log: http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2015/opnfv-meeting.2015-06-04-08.05.log.html May 28, 2015: Agenda & Minutes * Agenda * Use case discussion https://etherpad.opnfv.org/p/multisite_usecase_collection * How to work with Large_Deployment_Team in OpenStack operators who have already multi-site in production * Minutes * Joe explains the Keystone multisite failover use case (zhipeng, 08:15:33) * no question for Keystone Use Case, Joe go on to explain the VNF HA multisite Use Case (zhipeng, 08:20:15) * for HA Use Case, two main gaps are targeted (zhipeng, 08:25:06) * overlay l2 network and shared floating IP, across multiple sites (zhipeng, 08:25:32) * Jun Li (zhipeng, 08:26:49) * Xiaolong suggests to have requirements, besides use cases (zhipeng, 08:28:33) * Xiaolong suggests to have a centralized platform for mgmt, so item 4 and 5 maybe should be merged (zhipeng, 08:30:17) * Malla question about Keystone (zhipeng, 08:32:50) * Malla asks it seems the current Keystone solution is already distributed, and can support the requirement (zhipeng, 08:36:44) * Joe answers that even if Keystone could be deployed distributed across multisite, it would be almost impossible to manage the DBs (zhipeng, 08:38:14) * Malla asks for VNF HA use case, how to hand over to the standby VM seemlessly (zhipeng, 08:39:09) * Joe answers that when standby VM detects the failure, or an arbitrator detect the failure, then the handover process starts (zhipeng, 08:40:40) * Joe asks Xiaolong which token type would he prefer, UUID, pki or fernet (zhipeng, 08:44:18) * Xiaolong expressed maybe pki are prefered (zhipeng, 08:44:43) * Resource quota is very difficult in multi-site openstack. (malla, 08:49:33) * Malla asks for Telco WG, if they've alreay started the same work (zhipeng, 08:49:41) * Joe answers that there is no similar work currently (zhipeng, 08:51:12) * Malla suggest to invite OpenStack multisite guys over to the meeting (zhipeng, 08:52:50) * maybe a special meeting with North America guys in OpenStack (zhipeng, 08:53:46) * link: * Minutes: http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2015/opnfv-meeting.2015-05-28-08.04.html * Minutes (text): http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2015/opnfv-meeting.2015-05-28-08.04.txt * Log: http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2015/opnfv-meeting.2015-05-28-08.04.log.html May 07, 2015: Agenda & Minutes * Agenda * Self introduction * Discussion of work plan * Minutes * Xiaolong propose to have a wiki for collboration (zhipeng, 08:19:48) * Uli propose to use etherpad for early discussion (zhipeng, 08:20:39) * AGREED: use Etherpad for early discussion, wiki editing will begin as soon as any conclusion reached at the Etherpad (zhipeng, 08:24:03) * AGREED: gather use cases from each committer first, and then do the categorization and owner appointment (zhipeng, 08:29:54) * link: * Minutes: http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2015/opnfv-meeting.2015-05-07-08.09.html * Minutes (text): http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2015/opnfv-meeting.2015-05-07-08.09.txt * Log: http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2015/opnfv-meeting.2015-05-07-08.09.log.html