From patchwork Sat Jan 20 03:55:13 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ansis Atteka X-Patchwork-Id: 863921 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3zNkP94yq2z9t3B for ; Sat, 20 Jan 2018 14:55:25 +1100 (AEDT) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id C7BD01076; Sat, 20 Jan 2018 03:55:21 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 8EA1B1072 for ; Sat, 20 Jan 2018 03:55:20 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from relay2-d.mail.gandi.net (relay2-d.mail.gandi.net [217.70.183.194]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id B29441C0 for ; Sat, 20 Jan 2018 03:55:17 +0000 (UTC) X-Originating-IP: 208.91.1.34 Received: from aatteka-Precision-Tower-5810.eng.vmware.com (unknown [208.91.1.34]) (Authenticated sender: aatteka@ovn.org) by relay2-d.mail.gandi.net (Postfix) with ESMTPSA id E9C2DC5A4F; Sat, 20 Jan 2018 04:55:14 +0100 (CET) From: Ansis Atteka To: dev@openvswitch.org Date: Fri, 19 Jan 2018 19:55:13 -0800 Message-Id: <20180120035513.7268-1-aatteka@ovn.org> X-Mailer: git-send-email 2.14.1 X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_LOW autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Cc: Ansis Atteka Subject: [ovs-dev] [PATCH] poc: Introduce Proof of Concepts (Package building) X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org From: Ansis Atteka This patch sets up foundations for Proof of Concepts that simply materialize documentation into Ansible instructions executed in virtualized Vagrant environment. This Proof of Concept allows to easily build: 1. *.deb packages on Ubuntu 16.04; AND 2. *.rpm packages on CentOS 7.4. It also sets up DEB and RPM repository over HTTP that can be used to pull these openvswitch packages with apt-get or yum from another host. This particular Proof of Concept is intended to address following use-cases: 1. for new OVS users to see how debian and rpm packages are built; 2. for developers to easily check for packaging build regressions; 3. for developers to easily share their sandbox builds into QE setups (opposed to manually copying binaries); 4. for developers to add other Proof of Concepts that possibly may require full end-to-end integration with other thirdparty projects (e.g. DPI, libvirt, IPsec) and need Open vSwitch packages. Signed-off-by: Ansis Atteka Tested-by: Greg Rose Reviewed-by: Greg Rose --- .gitignore | 2 + Documentation/topics/testing.rst | 46 ++++++++++++++++++ Makefile.am | 3 ++ poc/builders/Vagrantfile | 35 ++++++++++++++ poc/playbook-centos-builder.yml | 100 +++++++++++++++++++++++++++++++++++++++ poc/playbook-ubuntu-builder.yml | 66 ++++++++++++++++++++++++++ 6 files changed, 252 insertions(+) create mode 100644 poc/builders/Vagrantfile create mode 100644 poc/playbook-centos-builder.yml create mode 100644 poc/playbook-ubuntu-builder.yml diff --git a/.gitignore b/.gitignore index 8019bee41..81faf270d 100644 --- a/.gitignore +++ b/.gitignore @@ -16,6 +16,7 @@ *.lib *.pdb *.pyc +*.retry *.so *.suo **/*.sym @@ -29,6 +30,7 @@ .dirstamp .libs .tmp_versions +.vagrant .gitattributes /Makefile /Makefile.in diff --git a/Documentation/topics/testing.rst b/Documentation/topics/testing.rst index a49336b79..4d93944c7 100644 --- a/Documentation/topics/testing.rst +++ b/Documentation/topics/testing.rst @@ -389,3 +389,49 @@ validate the suitability of different vSwitch implementations in a telco deployment environment. More information can be found on the `OPNFV wiki`_. .. _OPNFV wiki: https://wiki.opnfv.org/display/vsperf/VSperf+Home + +Proof of Concepts +~~~~~~~~~~~~~~~~~ + +Proof of Concepts are documentation materialized into Ansible recipes +executed in Virtualbox environment orchastrated by Vagrant. Proof of +Concepts allow developers to create small virtualized setups that +demonstrate how certain Open vSwitch features are intended to work, +especially when integration with thirdparty software is involved. + +The host where Vagrant runs does not need to have any special software +installed besides vagrant, virtualbox and ansible. + +The following Proof of Concepts are supported: + +Builders +++++++++ + +This Proof of Concept demonsrtates integration with Debian and RPM +packaging tools: + + $ cd ./poc/builders + + # When setting up Proof of Concept for the first time run: + $ vagrant up + +Once that command finished you can get packages from /var/www/html +directory. Since those hosts are also configured as repositories then +you can add them to /etc/apt/sources.list.d or /etc/yum.repos.d +configuration files. + + # When you have made changes to OVS and want to rebuild packages run: + $ git commit -a + $ vagrant rsync && vagrant provision + +Each packages are rebuilt the Open vSwitch release number increases +by one and you can simply upgrade them by running yum or apt-get. + +Once you are done with Particular Proof of Concept run + + $ vagrant destroy + +Or simply go to VirtualBox management console and delete those VMs. + +Sometime deployment of Proof of Concept may fail if for example VMs +don't have network reachability to the Internet. diff --git a/Makefile.am b/Makefile.am index ed4b7fd7c..174a421e6 100644 --- a/Makefile.am +++ b/Makefile.am @@ -90,6 +90,9 @@ EXTRA_DIST = \ build-aux/sodepends.py \ build-aux/soexpand.py \ build-aux/xml2nroff \ + poc/builders/Vagrantfile \ + poc/playbook-centos-builder.yml \ + poc/playbook-ubuntu-builder.yml \ $(MAN_FRAGMENTS) \ $(MAN_ROOTS) \ Vagrantfile \ diff --git a/poc/builders/Vagrantfile b/poc/builders/Vagrantfile new file mode 100644 index 000000000..00b114475 --- /dev/null +++ b/poc/builders/Vagrantfile @@ -0,0 +1,35 @@ +# -*- mode: ruby -*- +# vi: set ft=ruby : + +VAGRANTFILE_API_VERSION = "2" + +Vagrant.configure(VAGRANTFILE_API_VERSION) do |config| + config.ssh.insert_key = false + config.vm.provider :virtualbox do |vb| + vb.customize ["modifyvm", :id, "--memory", "1024"] + vb.customize ["modifyvm", :id, "--cpus", "4"] + end + + # Centos-7.4 builder host + config.vm.define "centosbuilder" do |builder| + builder.vm.hostname = "centosbuilder.dev" + builder.vm.box = "bento/centos-7.4" + builder.vm.synced_folder "../../", "/git/ovs", type: "rsync", rsync__args: ["--verbose", "--archive", "--delete", "-z"] + builder.vm.provision "builder", type: "ansible" do |ansible| + ansible.playbook = "../playbook-centos-builder.yml" + ansible.sudo = true + end + end + + # Ubuntu-16.04 builder host + config.vm.define "ubuntubuilder" do |builder| + builder.vm.hostname = "ubuntubuilder.dev" + builder.vm.box = "bento/ubuntu-16.04" + builder.vm.synced_folder "../../", "/git/ovs", type: "rsync", rsync__args: ["--verbose", "--archive", "--delete", "-z"] + builder.vm.provision "builder", type: "ansible" do |ansible| + ansible.playbook = "../playbook-ubuntu-builder.yml" + ansible.sudo = true + end + end + +end diff --git a/poc/playbook-centos-builder.yml b/poc/playbook-centos-builder.yml new file mode 100644 index 000000000..b9b6e0776 --- /dev/null +++ b/poc/playbook-centos-builder.yml @@ -0,0 +1,100 @@ +--- +- hosts: all + become: true + name: builder + tasks: + + - name: Create Ansible Local Facts Directory + file: path=/etc/ansible/facts.d state=directory + + - name: Initiate Build Numbering + copy: + content: '{ "build_counter":"1" }' + dest: "/etc/ansible/facts.d/builder.fact" + force: no + + - name: Reload Ansible Local Facts + setup: filter=ansible_local + + - name: Install "yum-utils", "rpmdevtools", "createrepo", "httpd", "git" + yum: update_cache=yes name={{item}} state=present + with_items: + - yum-utils + - rpmdevtools + - createrepo + - httpd + - git + + - name: Remove untracked files from Open vSwitch GIT repository + command: chdir=/git/ovs/ git clean -xdf + + - name: Reset Open vSwitch GIT repository to last comitted state + command: chdir=/git/ovs/ git reset --hard + + - name: Generate "openvswitch.spec" and "openvswitch-kmod-rhel6.spec" files for easy build dependency retrieval + shell: sed -e 's/@VERSION@/0.0.1/' {{item}}.in > /tmp/{{item}} + args: + chdir: /git/ovs/rhel + with_items: + - openvswitch.spec + - openvswitch-kmod-rhel6.spec + + - name: Install build dependencies specified in "openvswitch.spec" and "openvswitch-kmod-rhel6.spec" + shell: echo "y" | yum-builddep /tmp/{{item}} + with_items: + - openvswitch.spec + - openvswitch-kmod-rhel6.spec + + - name: Create rpm dev tree + command: rpmdev-setuptree + + - name: Run "./boot.sh" + command: chdir=/git/ovs/ ./boot.sh + + - name: Run "./configure" + command: chdir=/git/ovs/ ./configure + + - name: Run "make dist" + command: chdir=/git/ovs/ make dist + + - name: Parse out Open vSwitch version from "configure.ac" + command: chdir=/git/ovs autoconf -t AC_INIT:'$2' + register: version + + - name: Copy source tarball to rpm dev tree + command: cp /git/ovs/openvswitch-{{version.stdout}}.tar.gz /root/rpmbuild/SOURCES/ + + - name: Unarchive openvswitch source tarball + unarchive: + src: /root/rpmbuild/SOURCES/openvswitch-{{version.stdout}}.tar.gz + dest: /root/rpmbuild/SOURCES + remote_src: yes + + - name: Update release number in "openvswitch.spec" and "openvswitch-kmod-rhel6.spec" + lineinfile: + path: /root/rpmbuild/SOURCES/openvswitch-{{version.stdout}}/rhel/{{item}} + regexp: '^Release:' + line: "Release: {{ ansible_local.builder.build_counter }}" + with_items: + - openvswitch.spec + - openvswitch-kmod-rhel6.spec + + - name: Build Open vSwitch user space rpms + command: chdir=/root/rpmbuild/SOURCES/openvswitch-{{version.stdout}} rpmbuild -bb --without check rhel/openvswitch.spec + + - name: Build Open vSwitch kmod rpms (only for currently loaded kernel) + command: chdir=/root/rpmbuild/SOURCES/openvswitch-{{version.stdout}} rpmbuild -bb --without check rhel/openvswitch-kmod-rhel6.spec + + - name: Copy RPM packages to /var/www/html + command: cp -r /root/rpmbuild/RPMS/ /var/www/html + + - name: Create RPM Package index file for repository + command: chdir=/var/www/html createrepo /var/www/html + + - name: Make sure Apache is running + systemd: state=started name=httpd + + - name: Bump up Build Number + copy: + content: '{ "build_counter":"{{ansible_local.builder.build_counter|int+1}}" }' + dest: "/etc/ansible/facts.d/builder.fact" diff --git a/poc/playbook-ubuntu-builder.yml b/poc/playbook-ubuntu-builder.yml new file mode 100644 index 000000000..c6f5b6eb0 --- /dev/null +++ b/poc/playbook-ubuntu-builder.yml @@ -0,0 +1,66 @@ +--- +- hosts: all + become: true + name: builder + tasks: + + - name: Create Ansible Local Facts Directory + file: path=/etc/ansible/facts.d state=directory + + - name: Initiate Build Numbering + copy: + content: '{ "build_counter":"1" }' + dest: "/etc/ansible/facts.d/builder.fact" + force: no + + - name: Reload Ansible Local Facts + setup: filter=ansible_local + + - name: Install "devscripts", "equivs", "apache2", "autoconf" + apt: update_cache=yes name={{item}} state=present + with_items: + - devscripts + - equivs + - apache2 + - autoconf + + - name: Remove untracked files from Open vSwitch GIT repository + command: chdir=/git/ovs/ git clean -xdf + + - name: Reset Open vSwitch GIT repository to last comitted state + command: chdir=/git/ovs/ git reset --hard + + - name: Parse out Open vSwitch version from "configure.ac" + command: chdir=/git/ovs autoconf -t AC_INIT:'$2' + register: version + + - name: Concatenate full version + set_fact: + full_version: "{{version.stdout}}-{{ansible_local.builder.build_counter}}" + + - name: Update Open vSwitch version to {{full_version}} + command: chdir=/git/ovs/ dch -b -v {{full_version}} Vagrant Build + + - name: Build debian package with Open vSwitch build dependencies + command: chdir=/git/ovs/ mk-build-deps -B debian/control + + - name: Install Open vSwitch {{full_version}} build dependencies + apt: deb=/git/ovs/openvswitch-build-deps-depends_{{full_version}}_all.deb + + - name: Build Open vSwitch {{full_version}} debian packages + shell: DEB_BUILD_OPTIONS='parallel=4 nocheck' fakeroot debian/rules binary + args: + chdir: /git/ovs/ + + - name: Move debian packages to /var/www/html + shell: mv /git/*.deb /var/www/html/ + + - name: Create Debian Package index file for repository + shell: dpkg-scanpackages . | gzip -9c > Packages.gz + args: + chdir: /var/www/html + + - name: Bump up Build Number + copy: + content: '{ "build_counter":"{{ansible_local.builder.build_counter|int+1}}" }' + dest: "/etc/ansible/facts.d/builder.fact"