| Message ID | 20180120035513.7268-1-aatteka@ovn.org |
|---|---|
| State | Accepted |
| Headers | show |
| Series | [ovs-dev] poc: Introduce Proof of Concepts (Package building) | expand |
On 1/19/2018 7:55 PM, Ansis Atteka wrote: > From: Ansis Atteka <ansisatteka@gmail.com> > > This patch sets up foundations for Proof of Concepts that > simply materialize documentation into Ansible instructions > executed in virtualized Vagrant environment. > > This Proof of Concept allows to easily build: > 1. *.deb packages on Ubuntu 16.04; AND > 2. *.rpm packages on CentOS 7.4. > It also sets up DEB and RPM repository over HTTP that can > be used to pull these openvswitch packages with apt-get > or yum from another host. > > This particular Proof of Concept is intended to address > following use-cases: > 1. for new OVS users to see how debian and rpm packages are > built; > 2. for developers to easily check for packaging build > regressions; > 3. for developers to easily share their sandbox builds > into QE setups (opposed to manually copying binaries); > 4. for developers to add other Proof of Concepts > that possibly may require full end-to-end integration > with other thirdparty projects (e.g. DPI, libvirt, IPsec) > and need Open vSwitch packages. > > Signed-off-by: Ansis Atteka <aatteka@ovn.org> > --- > .gitignore | 2 + > Documentation/topics/testing.rst | 46 ++++++++++++++++++ > Makefile.am | 3 ++ > poc/builders/Vagrantfile | 35 ++++++++++++++ > poc/playbook-centos-builder.yml | 100 +++++++++++++++++++++++++++++++++++++++ > poc/playbook-ubuntu-builder.yml | 66 ++++++++++++++++++++++++++ > 6 files changed, 252 insertions(+) > create mode 100644 poc/builders/Vagrantfile > create mode 100644 poc/playbook-centos-builder.yml > create mode 100644 poc/playbook-ubuntu-builder.yml > > diff --git a/.gitignore b/.gitignore > index 8019bee41..81faf270d 100644 > --- a/.gitignore > +++ b/.gitignore > @@ -16,6 +16,7 @@ > *.lib > *.pdb > *.pyc > +*.retry > *.so > *.suo > **/*.sym > @@ -29,6 +30,7 @@ > .dirstamp > .libs > .tmp_versions > +.vagrant > .gitattributes > /Makefile > /Makefile.in > diff --git a/Documentation/topics/testing.rst b/Documentation/topics/testing.rst > index a49336b79..4d93944c7 100644 > --- a/Documentation/topics/testing.rst > +++ b/Documentation/topics/testing.rst > @@ -389,3 +389,49 @@ validate the suitability of different vSwitch implementations in a telco > deployment environment. More information can be found on the `OPNFV wiki`_. > > .. _OPNFV wiki: https://wiki.opnfv.org/display/vsperf/VSperf+Home > + > +Proof of Concepts > +~~~~~~~~~~~~~~~~~ > + > +Proof of Concepts are documentation materialized into Ansible recipes > +executed in Virtualbox environment orchastrated by Vagrant. Proof of > +Concepts allow developers to create small virtualized setups that > +demonstrate how certain Open vSwitch features are intended to work, > +especially when integration with thirdparty software is involved. s/thirdparty/third party > + > +The host where Vagrant runs does not need to have any special software > +installed besides vagrant, virtualbox and ansible. Seem it also requires Apache? > + > +The following Proof of Concepts are supported: > + > +Builders > +++++++++ > + > +This Proof of Concept demonsrtates integration with Debian and RPM > +packaging tools: > + > + $ cd ./poc/builders > + > + # When setting up Proof of Concept for the first time run: > + $ vagrant up > + > +Once that command finished you can get packages from /var/www/html > +directory. Since those hosts are also configured as repositories then > +you can add them to /etc/apt/sources.list.d or /etc/yum.repos.d > +configuration files. > + > + # When you have made changes to OVS and want to rebuild packages run: > + $ git commit -a > + $ vagrant rsync && vagrant provision > + > +Each packages are rebuilt the Open vSwitch release number increases Maybe "When packages are rebuilt" ? There are some checkpatch warnings about lines too long but I'm not worried about those. There are a few other checkpatch warnings about lines with trailing whitespace. Those should probably be cleaned up. The rest of the patch seems fine. I applied it and ran the simple test case you outline and it worked as advertised. I have a bit of a learning curve to get over for me to make further use of this but looks promising! Thanks Ansis! Tested-by: Greg Rose <gvrose8192@gmail.com> Reviewed-by: Greg Rose <gvrose8192@gmail.com> > +by one and you can simply upgrade them by running yum or apt-get. > + > +Once you are done with Particular Proof of Concept run > + > + $ vagrant destroy > + > +Or simply go to VirtualBox management console and delete those VMs. > + > +Sometime deployment of Proof of Concept may fail if for example VMs > +don't have network reachability to the Internet. > diff --git a/Makefile.am b/Makefile.am > index ed4b7fd7c..174a421e6 100644 > --- a/Makefile.am > +++ b/Makefile.am > @@ -90,6 +90,9 @@ EXTRA_DIST = \ > build-aux/sodepends.py \ > build-aux/soexpand.py \ > build-aux/xml2nroff \ > + poc/builders/Vagrantfile \ > + poc/playbook-centos-builder.yml \ > + poc/playbook-ubuntu-builder.yml \ > $(MAN_FRAGMENTS) \ > $(MAN_ROOTS) \ > Vagrantfile \ > diff --git a/poc/builders/Vagrantfile b/poc/builders/Vagrantfile > new file mode 100644 > index 000000000..00b114475 > --- /dev/null > +++ b/poc/builders/Vagrantfile > @@ -0,0 +1,35 @@ > +# -*- mode: ruby -*- > +# vi: set ft=ruby : > + > +VAGRANTFILE_API_VERSION = "2" > + > +Vagrant.configure(VAGRANTFILE_API_VERSION) do |config| > + config.ssh.insert_key = false > + config.vm.provider :virtualbox do |vb| > + vb.customize ["modifyvm", :id, "--memory", "1024"] > + vb.customize ["modifyvm", :id, "--cpus", "4"] > + end > + > + # Centos-7.4 builder host > + config.vm.define "centosbuilder" do |builder| > + builder.vm.hostname = "centosbuilder.dev" > + builder.vm.box = "bento/centos-7.4" > + builder.vm.synced_folder "../../", "/git/ovs", type: "rsync", rsync__args: ["--verbose", "--archive", "--delete", "-z"] > + builder.vm.provision "builder", type: "ansible" do |ansible| > + ansible.playbook = "../playbook-centos-builder.yml" > + ansible.sudo = true > + end > + end > + > + # Ubuntu-16.04 builder host > + config.vm.define "ubuntubuilder" do |builder| > + builder.vm.hostname = "ubuntubuilder.dev" > + builder.vm.box = "bento/ubuntu-16.04" > + builder.vm.synced_folder "../../", "/git/ovs", type: "rsync", rsync__args: ["--verbose", "--archive", "--delete", "-z"] > + builder.vm.provision "builder", type: "ansible" do |ansible| > + ansible.playbook = "../playbook-ubuntu-builder.yml" > + ansible.sudo = true > + end > + end > + > +end > diff --git a/poc/playbook-centos-builder.yml b/poc/playbook-centos-builder.yml > new file mode 100644 > index 000000000..b9b6e0776 > --- /dev/null > +++ b/poc/playbook-centos-builder.yml > @@ -0,0 +1,100 @@ > +--- > +- hosts: all > + become: true > + name: builder > + tasks: > + > + - name: Create Ansible Local Facts Directory > + file: path=/etc/ansible/facts.d state=directory > + > + - name: Initiate Build Numbering > + copy: > + content: '{ "build_counter":"1" }' > + dest: "/etc/ansible/facts.d/builder.fact" > + force: no > + > + - name: Reload Ansible Local Facts > + setup: filter=ansible_local > + > + - name: Install "yum-utils", "rpmdevtools", "createrepo", "httpd", "git" > + yum: update_cache=yes name={{item}} state=present > + with_items: > + - yum-utils > + - rpmdevtools > + - createrepo > + - httpd > + - git > + > + - name: Remove untracked files from Open vSwitch GIT repository > + command: chdir=/git/ovs/ git clean -xdf > + > + - name: Reset Open vSwitch GIT repository to last comitted state > + command: chdir=/git/ovs/ git reset --hard > + > + - name: Generate "openvswitch.spec" and "openvswitch-kmod-rhel6.spec" files for easy build dependency retrieval > + shell: sed -e 's/@VERSION@/0.0.1/' {{item}}.in > /tmp/{{item}} > + args: > + chdir: /git/ovs/rhel > + with_items: > + - openvswitch.spec > + - openvswitch-kmod-rhel6.spec > + > + - name: Install build dependencies specified in "openvswitch.spec" and "openvswitch-kmod-rhel6.spec" > + shell: echo "y" | yum-builddep /tmp/{{item}} > + with_items: > + - openvswitch.spec > + - openvswitch-kmod-rhel6.spec > + > + - name: Create rpm dev tree > + command: rpmdev-setuptree > + > + - name: Run "./boot.sh" > + command: chdir=/git/ovs/ ./boot.sh > + > + - name: Run "./configure" > + command: chdir=/git/ovs/ ./configure > + > + - name: Run "make dist" > + command: chdir=/git/ovs/ make dist > + > + - name: Parse out Open vSwitch version from "configure.ac" > + command: chdir=/git/ovs autoconf -t AC_INIT:'$2' > + register: version > + > + - name: Copy source tarball to rpm dev tree > + command: cp /git/ovs/openvswitch-{{version.stdout}}.tar.gz /root/rpmbuild/SOURCES/ > + > + - name: Unarchive openvswitch source tarball > + unarchive: > + src: /root/rpmbuild/SOURCES/openvswitch-{{version.stdout}}.tar.gz > + dest: /root/rpmbuild/SOURCES > + remote_src: yes > + > + - name: Update release number in "openvswitch.spec" and "openvswitch-kmod-rhel6.spec" > + lineinfile: > + path: /root/rpmbuild/SOURCES/openvswitch-{{version.stdout}}/rhel/{{item}} > + regexp: '^Release:' > + line: "Release: {{ ansible_local.builder.build_counter }}" > + with_items: > + - openvswitch.spec > + - openvswitch-kmod-rhel6.spec > + > + - name: Build Open vSwitch user space rpms > + command: chdir=/root/rpmbuild/SOURCES/openvswitch-{{version.stdout}} rpmbuild -bb --without check rhel/openvswitch.spec > + > + - name: Build Open vSwitch kmod rpms (only for currently loaded kernel) > + command: chdir=/root/rpmbuild/SOURCES/openvswitch-{{version.stdout}} rpmbuild -bb --without check rhel/openvswitch-kmod-rhel6.spec > + > + - name: Copy RPM packages to /var/www/html > + command: cp -r /root/rpmbuild/RPMS/ /var/www/html > + > + - name: Create RPM Package index file for repository > + command: chdir=/var/www/html createrepo /var/www/html > + > + - name: Make sure Apache is running > + systemd: state=started name=httpd > + > + - name: Bump up Build Number > + copy: > + content: '{ "build_counter":"{{ansible_local.builder.build_counter|int+1}}" }' > + dest: "/etc/ansible/facts.d/builder.fact" > diff --git a/poc/playbook-ubuntu-builder.yml b/poc/playbook-ubuntu-builder.yml > new file mode 100644 > index 000000000..c6f5b6eb0 > --- /dev/null > +++ b/poc/playbook-ubuntu-builder.yml > @@ -0,0 +1,66 @@ > +--- > +- hosts: all > + become: true > + name: builder > + tasks: > + > + - name: Create Ansible Local Facts Directory > + file: path=/etc/ansible/facts.d state=directory > + > + - name: Initiate Build Numbering > + copy: > + content: '{ "build_counter":"1" }' > + dest: "/etc/ansible/facts.d/builder.fact" > + force: no > + > + - name: Reload Ansible Local Facts > + setup: filter=ansible_local > + > + - name: Install "devscripts", "equivs", "apache2", "autoconf" > + apt: update_cache=yes name={{item}} state=present > + with_items: > + - devscripts > + - equivs > + - apache2 > + - autoconf > + > + - name: Remove untracked files from Open vSwitch GIT repository > + command: chdir=/git/ovs/ git clean -xdf > + > + - name: Reset Open vSwitch GIT repository to last comitted state > + command: chdir=/git/ovs/ git reset --hard > + > + - name: Parse out Open vSwitch version from "configure.ac" > + command: chdir=/git/ovs autoconf -t AC_INIT:'$2' > + register: version > + > + - name: Concatenate full version > + set_fact: > + full_version: "{{version.stdout}}-{{ansible_local.builder.build_counter}}" > + > + - name: Update Open vSwitch version to {{full_version}} > + command: chdir=/git/ovs/ dch -b -v {{full_version}} Vagrant Build > + > + - name: Build debian package with Open vSwitch build dependencies > + command: chdir=/git/ovs/ mk-build-deps -B debian/control > + > + - name: Install Open vSwitch {{full_version}} build dependencies > + apt: deb=/git/ovs/openvswitch-build-deps-depends_{{full_version}}_all.deb > + > + - name: Build Open vSwitch {{full_version}} debian packages > + shell: DEB_BUILD_OPTIONS='parallel=4 nocheck' fakeroot debian/rules binary > + args: > + chdir: /git/ovs/ > + > + - name: Move debian packages to /var/www/html > + shell: mv /git/*.deb /var/www/html/ > + > + - name: Create Debian Package index file for repository > + shell: dpkg-scanpackages . | gzip -9c > Packages.gz > + args: > + chdir: /var/www/html > + > + - name: Bump up Build Number > + copy: > + content: '{ "build_counter":"{{ansible_local.builder.build_counter|int+1}}" }' > + dest: "/etc/ansible/facts.d/builder.fact"
On 26 January 2018 at 09:13, Gregory Rose <gvrose8192@gmail.com> wrote: > On 1/19/2018 7:55 PM, Ansis Atteka wrote: >> >> From: Ansis Atteka <ansisatteka@gmail.com> >> >> This patch sets up foundations for Proof of Concepts that >> simply materialize documentation into Ansible instructions >> executed in virtualized Vagrant environment. >> >> This Proof of Concept allows to easily build: >> 1. *.deb packages on Ubuntu 16.04; AND >> 2. *.rpm packages on CentOS 7.4. >> It also sets up DEB and RPM repository over HTTP that can >> be used to pull these openvswitch packages with apt-get >> or yum from another host. >> >> This particular Proof of Concept is intended to address >> following use-cases: >> 1. for new OVS users to see how debian and rpm packages are >> built; >> 2. for developers to easily check for packaging build >> regressions; >> 3. for developers to easily share their sandbox builds >> into QE setups (opposed to manually copying binaries); >> 4. for developers to add other Proof of Concepts >> that possibly may require full end-to-end integration >> with other thirdparty projects (e.g. DPI, libvirt, IPsec) >> and need Open vSwitch packages. >> >> Signed-off-by: Ansis Atteka <aatteka@ovn.org> >> --- >> .gitignore | 2 + >> Documentation/topics/testing.rst | 46 ++++++++++++++++++ >> Makefile.am | 3 ++ >> poc/builders/Vagrantfile | 35 ++++++++++++++ >> poc/playbook-centos-builder.yml | 100 >> +++++++++++++++++++++++++++++++++++++++ >> poc/playbook-ubuntu-builder.yml | 66 ++++++++++++++++++++++++++ >> 6 files changed, 252 insertions(+) >> create mode 100644 poc/builders/Vagrantfile >> create mode 100644 poc/playbook-centos-builder.yml >> create mode 100644 poc/playbook-ubuntu-builder.yml >> >> diff --git a/.gitignore b/.gitignore >> index 8019bee41..81faf270d 100644 >> --- a/.gitignore >> +++ b/.gitignore >> @@ -16,6 +16,7 @@ >> *.lib >> *.pdb >> *.pyc >> +*.retry >> *.so >> *.suo >> **/*.sym >> @@ -29,6 +30,7 @@ >> .dirstamp >> .libs >> .tmp_versions >> +.vagrant >> .gitattributes >> /Makefile >> /Makefile.in >> diff --git a/Documentation/topics/testing.rst >> b/Documentation/topics/testing.rst >> index a49336b79..4d93944c7 100644 >> --- a/Documentation/topics/testing.rst >> +++ b/Documentation/topics/testing.rst >> @@ -389,3 +389,49 @@ validate the suitability of different vSwitch >> implementations in a telco >> deployment environment. More information can be found on the `OPNFV >> wiki`_. >> .. _OPNFV wiki: https://wiki.opnfv.org/display/vsperf/VSperf+Home >> + >> +Proof of Concepts >> +~~~~~~~~~~~~~~~~~ >> + >> +Proof of Concepts are documentation materialized into Ansible recipes >> +executed in Virtualbox environment orchastrated by Vagrant. Proof of >> +Concepts allow developers to create small virtualized setups that >> +demonstrate how certain Open vSwitch features are intended to work, >> +especially when integration with thirdparty software is involved. > > s/thirdparty/third party > >> + >> +The host where Vagrant runs does not need to have any special software >> +installed besides vagrant, virtualbox and ansible. > > > Seem it also requires Apache? Since the Ansible recipes are executed only on the guest (and not host) then only guests need Apache/httpd installed. Let me know if I am missing something and you still had to install Apache on host? Apache is required only to have the packages distributed via repository. > >> + >> +The following Proof of Concepts are supported: >> + >> +Builders >> +++++++++ >> + >> +This Proof of Concept demonsrtates integration with Debian and RPM >> +packaging tools: >> + >> + $ cd ./poc/builders >> + >> + # When setting up Proof of Concept for the first time run: >> + $ vagrant up >> + >> +Once that command finished you can get packages from /var/www/html >> +directory. Since those hosts are also configured as repositories then >> +you can add them to /etc/apt/sources.list.d or /etc/yum.repos.d >> +configuration files. >> + >> + # When you have made changes to OVS and want to rebuild packages run: >> + $ git commit -a >> + $ vagrant rsync && vagrant provision >> + >> +Each packages are rebuilt the Open vSwitch release number increases > > > Maybe "When packages are rebuilt" ? You are right. Will send V2. > > There are some checkpatch warnings about lines too long but I'm not worried > about those. There > are a few other checkpatch warnings about lines with trailing whitespace. > Those should probably > be cleaned up. Agree about whitespace errors, not sure how I missed them... Will look into truncating lines to 80 characters. There are some lines that require file-system paths so it may require a little more creativity to truncate them without sacrificing readability. > > The rest of the patch seems fine. I applied it and ran the simple test case > you outline > and it worked as advertised. I have a bit of a learning curve to get over > for me to make > further use of this but looks promising! > > Thanks Ansis! Great. I am somewhat curious if we should switch from Virtualbox to KVM/libvirt, because it appears that is what most developers are using here. Will let others chime in as well. > > Tested-by: Greg Rose <gvrose8192@gmail.com> > Reviewed-by: Greg Rose <gvrose8192@gmail.com> > > >> +by one and you can simply upgrade them by running yum or apt-get. >> + >> +Once you are done with Particular Proof of Concept run >> + >> + $ vagrant destroy >> + >> +Or simply go to VirtualBox management console and delete those VMs. >> + >> +Sometime deployment of Proof of Concept may fail if for example VMs >> +don't have network reachability to the Internet. >> diff --git a/Makefile.am b/Makefile.am >> index ed4b7fd7c..174a421e6 100644 >> --- a/Makefile.am >> +++ b/Makefile.am >> @@ -90,6 +90,9 @@ EXTRA_DIST = \ >> build-aux/sodepends.py \ >> build-aux/soexpand.py \ >> build-aux/xml2nroff \ >> + poc/builders/Vagrantfile \ >> + poc/playbook-centos-builder.yml \ >> + poc/playbook-ubuntu-builder.yml \ >> $(MAN_FRAGMENTS) \ >> $(MAN_ROOTS) \ >> Vagrantfile \ >> diff --git a/poc/builders/Vagrantfile b/poc/builders/Vagrantfile >> new file mode 100644 >> index 000000000..00b114475 >> --- /dev/null >> +++ b/poc/builders/Vagrantfile >> @@ -0,0 +1,35 @@ >> +# -*- mode: ruby -*- >> +# vi: set ft=ruby : >> + >> +VAGRANTFILE_API_VERSION = "2" >> + >> +Vagrant.configure(VAGRANTFILE_API_VERSION) do |config| >> + config.ssh.insert_key = false >> + config.vm.provider :virtualbox do |vb| >> + vb.customize ["modifyvm", :id, "--memory", "1024"] >> + vb.customize ["modifyvm", :id, "--cpus", "4"] >> + end >> + >> + # Centos-7.4 builder host >> + config.vm.define "centosbuilder" do |builder| >> + builder.vm.hostname = "centosbuilder.dev" >> + builder.vm.box = "bento/centos-7.4" >> + builder.vm.synced_folder "../../", "/git/ovs", type: "rsync", >> rsync__args: ["--verbose", "--archive", "--delete", "-z"] >> + builder.vm.provision "builder", type: "ansible" do |ansible| >> + ansible.playbook = "../playbook-centos-builder.yml" >> + ansible.sudo = true >> + end >> + end >> + >> + # Ubuntu-16.04 builder host >> + config.vm.define "ubuntubuilder" do |builder| >> + builder.vm.hostname = "ubuntubuilder.dev" >> + builder.vm.box = "bento/ubuntu-16.04" >> + builder.vm.synced_folder "../../", "/git/ovs", type: "rsync", >> rsync__args: ["--verbose", "--archive", "--delete", "-z"] >> + builder.vm.provision "builder", type: "ansible" do |ansible| >> + ansible.playbook = "../playbook-ubuntu-builder.yml" >> + ansible.sudo = true >> + end >> + end >> + >> +end >> diff --git a/poc/playbook-centos-builder.yml >> b/poc/playbook-centos-builder.yml >> new file mode 100644 >> index 000000000..b9b6e0776 >> --- /dev/null >> +++ b/poc/playbook-centos-builder.yml >> @@ -0,0 +1,100 @@ >> +--- >> +- hosts: all >> + become: true >> + name: builder >> + tasks: >> + >> + - name: Create Ansible Local Facts Directory >> + file: path=/etc/ansible/facts.d state=directory >> + >> + - name: Initiate Build Numbering >> + copy: >> + content: '{ "build_counter":"1" }' >> + dest: "/etc/ansible/facts.d/builder.fact" >> + force: no >> + >> + - name: Reload Ansible Local Facts >> + setup: filter=ansible_local >> + >> + - name: Install "yum-utils", "rpmdevtools", "createrepo", "httpd", >> "git" >> + yum: update_cache=yes name={{item}} state=present >> + with_items: >> + - yum-utils >> + - rpmdevtools >> + - createrepo >> + - httpd >> + - git >> + >> + - name: Remove untracked files from Open vSwitch GIT repository >> + command: chdir=/git/ovs/ git clean -xdf >> + >> + - name: Reset Open vSwitch GIT repository to last comitted state >> + command: chdir=/git/ovs/ git reset --hard >> + >> + - name: Generate "openvswitch.spec" and "openvswitch-kmod-rhel6.spec" >> files for easy build dependency retrieval >> + shell: sed -e 's/@VERSION@/0.0.1/' {{item}}.in > /tmp/{{item}} >> + args: >> + chdir: /git/ovs/rhel >> + with_items: >> + - openvswitch.spec >> + - openvswitch-kmod-rhel6.spec >> + >> + - name: Install build dependencies specified in "openvswitch.spec" and >> "openvswitch-kmod-rhel6.spec" >> + shell: echo "y" | yum-builddep /tmp/{{item}} >> + with_items: >> + - openvswitch.spec >> + - openvswitch-kmod-rhel6.spec >> + >> + - name: Create rpm dev tree >> + command: rpmdev-setuptree >> + >> + - name: Run "./boot.sh" >> + command: chdir=/git/ovs/ ./boot.sh >> + >> + - name: Run "./configure" >> + command: chdir=/git/ovs/ ./configure >> + >> + - name: Run "make dist" >> + command: chdir=/git/ovs/ make dist >> + >> + - name: Parse out Open vSwitch version from "configure.ac" >> + command: chdir=/git/ovs autoconf -t AC_INIT:'$2' >> + register: version >> + >> + - name: Copy source tarball to rpm dev tree >> + command: cp /git/ovs/openvswitch-{{version.stdout}}.tar.gz >> /root/rpmbuild/SOURCES/ >> + >> + - name: Unarchive openvswitch source tarball >> + unarchive: >> + src: /root/rpmbuild/SOURCES/openvswitch-{{version.stdout}}.tar.gz >> + dest: /root/rpmbuild/SOURCES >> + remote_src: yes >> + >> + - name: Update release number in "openvswitch.spec" and >> "openvswitch-kmod-rhel6.spec" >> + lineinfile: >> + path: >> /root/rpmbuild/SOURCES/openvswitch-{{version.stdout}}/rhel/{{item}} >> + regexp: '^Release:' >> + line: "Release: {{ ansible_local.builder.build_counter }}" >> + with_items: >> + - openvswitch.spec >> + - openvswitch-kmod-rhel6.spec >> + >> + - name: Build Open vSwitch user space rpms >> + command: chdir=/root/rpmbuild/SOURCES/openvswitch-{{version.stdout}} >> rpmbuild -bb --without check rhel/openvswitch.spec >> + >> + - name: Build Open vSwitch kmod rpms (only for currently loaded kernel) >> + command: chdir=/root/rpmbuild/SOURCES/openvswitch-{{version.stdout}} >> rpmbuild -bb --without check rhel/openvswitch-kmod-rhel6.spec >> + >> + - name: Copy RPM packages to /var/www/html >> + command: cp -r /root/rpmbuild/RPMS/ /var/www/html >> + >> + - name: Create RPM Package index file for repository >> + command: chdir=/var/www/html createrepo /var/www/html >> + >> + - name: Make sure Apache is running >> + systemd: state=started name=httpd >> + >> + - name: Bump up Build Number >> + copy: >> + content: '{ >> "build_counter":"{{ansible_local.builder.build_counter|int+1}}" }' >> + dest: "/etc/ansible/facts.d/builder.fact" >> diff --git a/poc/playbook-ubuntu-builder.yml >> b/poc/playbook-ubuntu-builder.yml >> new file mode 100644 >> index 000000000..c6f5b6eb0 >> --- /dev/null >> +++ b/poc/playbook-ubuntu-builder.yml >> @@ -0,0 +1,66 @@ >> +--- >> +- hosts: all >> + become: true >> + name: builder >> + tasks: >> + >> + - name: Create Ansible Local Facts Directory >> + file: path=/etc/ansible/facts.d state=directory >> + >> + - name: Initiate Build Numbering >> + copy: >> + content: '{ "build_counter":"1" }' >> + dest: "/etc/ansible/facts.d/builder.fact" >> + force: no >> + >> + - name: Reload Ansible Local Facts >> + setup: filter=ansible_local >> + >> + - name: Install "devscripts", "equivs", "apache2", "autoconf" >> + apt: update_cache=yes name={{item}} state=present >> + with_items: >> + - devscripts >> + - equivs >> + - apache2 >> + - autoconf >> + >> + - name: Remove untracked files from Open vSwitch GIT repository >> + command: chdir=/git/ovs/ git clean -xdf >> + >> + - name: Reset Open vSwitch GIT repository to last comitted state >> + command: chdir=/git/ovs/ git reset --hard >> + >> + - name: Parse out Open vSwitch version from "configure.ac" >> + command: chdir=/git/ovs autoconf -t AC_INIT:'$2' >> + register: version >> + >> + - name: Concatenate full version >> + set_fact: >> + full_version: >> "{{version.stdout}}-{{ansible_local.builder.build_counter}}" >> + >> + - name: Update Open vSwitch version to {{full_version}} >> + command: chdir=/git/ovs/ dch -b -v {{full_version}} Vagrant Build >> + >> + - name: Build debian package with Open vSwitch build dependencies >> + command: chdir=/git/ovs/ mk-build-deps -B debian/control >> + >> + - name: Install Open vSwitch {{full_version}} build dependencies >> + apt: >> deb=/git/ovs/openvswitch-build-deps-depends_{{full_version}}_all.deb >> + >> + - name: Build Open vSwitch {{full_version}} debian packages >> + shell: DEB_BUILD_OPTIONS='parallel=4 nocheck' fakeroot debian/rules >> binary >> + args: >> + chdir: /git/ovs/ >> + >> + - name: Move debian packages to /var/www/html >> + shell: mv /git/*.deb /var/www/html/ >> + >> + - name: Create Debian Package index file for repository >> + shell: dpkg-scanpackages . | gzip -9c > Packages.gz >> + args: >> + chdir: /var/www/html >> + >> + - name: Bump up Build Number >> + copy: >> + content: '{ >> "build_counter":"{{ansible_local.builder.build_counter|int+1}}" }' >> + dest: "/etc/ansible/facts.d/builder.fact" > >
diff --git a/.gitignore b/.gitignore index 8019bee41..81faf270d 100644 --- a/.gitignore +++ b/.gitignore @@ -16,6 +16,7 @@ *.lib *.pdb *.pyc +*.retry *.so *.suo **/*.sym @@ -29,6 +30,7 @@ .dirstamp .libs .tmp_versions +.vagrant .gitattributes /Makefile /Makefile.in diff --git a/Documentation/topics/testing.rst b/Documentation/topics/testing.rst index a49336b79..4d93944c7 100644 --- a/Documentation/topics/testing.rst +++ b/Documentation/topics/testing.rst @@ -389,3 +389,49 @@ validate the suitability of different vSwitch implementations in a telco deployment environment. More information can be found on the `OPNFV wiki`_. .. _OPNFV wiki: https://wiki.opnfv.org/display/vsperf/VSperf+Home + +Proof of Concepts +~~~~~~~~~~~~~~~~~ + +Proof of Concepts are documentation materialized into Ansible recipes +executed in Virtualbox environment orchastrated by Vagrant. Proof of +Concepts allow developers to create small virtualized setups that +demonstrate how certain Open vSwitch features are intended to work, +especially when integration with thirdparty software is involved. + +The host where Vagrant runs does not need to have any special software +installed besides vagrant, virtualbox and ansible. + +The following Proof of Concepts are supported: + +Builders +++++++++ + +This Proof of Concept demonsrtates integration with Debian and RPM +packaging tools: + + $ cd ./poc/builders + + # When setting up Proof of Concept for the first time run: + $ vagrant up + +Once that command finished you can get packages from /var/www/html +directory. Since those hosts are also configured as repositories then +you can add them to /etc/apt/sources.list.d or /etc/yum.repos.d +configuration files. + + # When you have made changes to OVS and want to rebuild packages run: + $ git commit -a + $ vagrant rsync && vagrant provision + +Each packages are rebuilt the Open vSwitch release number increases +by one and you can simply upgrade them by running yum or apt-get. + +Once you are done with Particular Proof of Concept run + + $ vagrant destroy + +Or simply go to VirtualBox management console and delete those VMs. + +Sometime deployment of Proof of Concept may fail if for example VMs +don't have network reachability to the Internet. diff --git a/Makefile.am b/Makefile.am index ed4b7fd7c..174a421e6 100644 --- a/Makefile.am +++ b/Makefile.am @@ -90,6 +90,9 @@ EXTRA_DIST = \ build-aux/sodepends.py \ build-aux/soexpand.py \ build-aux/xml2nroff \ + poc/builders/Vagrantfile \ + poc/playbook-centos-builder.yml \ + poc/playbook-ubuntu-builder.yml \ $(MAN_FRAGMENTS) \ $(MAN_ROOTS) \ Vagrantfile \ diff --git a/poc/builders/Vagrantfile b/poc/builders/Vagrantfile new file mode 100644 index 000000000..00b114475 --- /dev/null +++ b/poc/builders/Vagrantfile @@ -0,0 +1,35 @@ +# -*- mode: ruby -*- +# vi: set ft=ruby : + +VAGRANTFILE_API_VERSION = "2" + +Vagrant.configure(VAGRANTFILE_API_VERSION) do |config| + config.ssh.insert_key = false + config.vm.provider :virtualbox do |vb| + vb.customize ["modifyvm", :id, "--memory", "1024"] + vb.customize ["modifyvm", :id, "--cpus", "4"] + end + + # Centos-7.4 builder host + config.vm.define "centosbuilder" do |builder| + builder.vm.hostname = "centosbuilder.dev" + builder.vm.box = "bento/centos-7.4" + builder.vm.synced_folder "../../", "/git/ovs", type: "rsync", rsync__args: ["--verbose", "--archive", "--delete", "-z"] + builder.vm.provision "builder", type: "ansible" do |ansible| + ansible.playbook = "../playbook-centos-builder.yml" + ansible.sudo = true + end + end + + # Ubuntu-16.04 builder host + config.vm.define "ubuntubuilder" do |builder| + builder.vm.hostname = "ubuntubuilder.dev" + builder.vm.box = "bento/ubuntu-16.04" + builder.vm.synced_folder "../../", "/git/ovs", type: "rsync", rsync__args: ["--verbose", "--archive", "--delete", "-z"] + builder.vm.provision "builder", type: "ansible" do |ansible| + ansible.playbook = "../playbook-ubuntu-builder.yml" + ansible.sudo = true + end + end + +end diff --git a/poc/playbook-centos-builder.yml b/poc/playbook-centos-builder.yml new file mode 100644 index 000000000..b9b6e0776 --- /dev/null +++ b/poc/playbook-centos-builder.yml @@ -0,0 +1,100 @@ +--- +- hosts: all + become: true + name: builder + tasks: + + - name: Create Ansible Local Facts Directory + file: path=/etc/ansible/facts.d state=directory + + - name: Initiate Build Numbering + copy: + content: '{ "build_counter":"1" }' + dest: "/etc/ansible/facts.d/builder.fact" + force: no + + - name: Reload Ansible Local Facts + setup: filter=ansible_local + + - name: Install "yum-utils", "rpmdevtools", "createrepo", "httpd", "git" + yum: update_cache=yes name={{item}} state=present + with_items: + - yum-utils + - rpmdevtools + - createrepo + - httpd + - git + + - name: Remove untracked files from Open vSwitch GIT repository + command: chdir=/git/ovs/ git clean -xdf + + - name: Reset Open vSwitch GIT repository to last comitted state + command: chdir=/git/ovs/ git reset --hard + + - name: Generate "openvswitch.spec" and "openvswitch-kmod-rhel6.spec" files for easy build dependency retrieval + shell: sed -e 's/@VERSION@/0.0.1/' {{item}}.in > /tmp/{{item}} + args: + chdir: /git/ovs/rhel + with_items: + - openvswitch.spec + - openvswitch-kmod-rhel6.spec + + - name: Install build dependencies specified in "openvswitch.spec" and "openvswitch-kmod-rhel6.spec" + shell: echo "y" | yum-builddep /tmp/{{item}} + with_items: + - openvswitch.spec + - openvswitch-kmod-rhel6.spec + + - name: Create rpm dev tree + command: rpmdev-setuptree + + - name: Run "./boot.sh" + command: chdir=/git/ovs/ ./boot.sh + + - name: Run "./configure" + command: chdir=/git/ovs/ ./configure + + - name: Run "make dist" + command: chdir=/git/ovs/ make dist + + - name: Parse out Open vSwitch version from "configure.ac" + command: chdir=/git/ovs autoconf -t AC_INIT:'$2' + register: version + + - name: Copy source tarball to rpm dev tree + command: cp /git/ovs/openvswitch-{{version.stdout}}.tar.gz /root/rpmbuild/SOURCES/ + + - name: Unarchive openvswitch source tarball + unarchive: + src: /root/rpmbuild/SOURCES/openvswitch-{{version.stdout}}.tar.gz + dest: /root/rpmbuild/SOURCES + remote_src: yes + + - name: Update release number in "openvswitch.spec" and "openvswitch-kmod-rhel6.spec" + lineinfile: + path: /root/rpmbuild/SOURCES/openvswitch-{{version.stdout}}/rhel/{{item}} + regexp: '^Release:' + line: "Release: {{ ansible_local.builder.build_counter }}" + with_items: + - openvswitch.spec + - openvswitch-kmod-rhel6.spec + + - name: Build Open vSwitch user space rpms + command: chdir=/root/rpmbuild/SOURCES/openvswitch-{{version.stdout}} rpmbuild -bb --without check rhel/openvswitch.spec + + - name: Build Open vSwitch kmod rpms (only for currently loaded kernel) + command: chdir=/root/rpmbuild/SOURCES/openvswitch-{{version.stdout}} rpmbuild -bb --without check rhel/openvswitch-kmod-rhel6.spec + + - name: Copy RPM packages to /var/www/html + command: cp -r /root/rpmbuild/RPMS/ /var/www/html + + - name: Create RPM Package index file for repository + command: chdir=/var/www/html createrepo /var/www/html + + - name: Make sure Apache is running + systemd: state=started name=httpd + + - name: Bump up Build Number + copy: + content: '{ "build_counter":"{{ansible_local.builder.build_counter|int+1}}" }' + dest: "/etc/ansible/facts.d/builder.fact" diff --git a/poc/playbook-ubuntu-builder.yml b/poc/playbook-ubuntu-builder.yml new file mode 100644 index 000000000..c6f5b6eb0 --- /dev/null +++ b/poc/playbook-ubuntu-builder.yml @@ -0,0 +1,66 @@ +--- +- hosts: all + become: true + name: builder + tasks: + + - name: Create Ansible Local Facts Directory + file: path=/etc/ansible/facts.d state=directory + + - name: Initiate Build Numbering + copy: + content: '{ "build_counter":"1" }' + dest: "/etc/ansible/facts.d/builder.fact" + force: no + + - name: Reload Ansible Local Facts + setup: filter=ansible_local + + - name: Install "devscripts", "equivs", "apache2", "autoconf" + apt: update_cache=yes name={{item}} state=present + with_items: + - devscripts + - equivs + - apache2 + - autoconf + + - name: Remove untracked files from Open vSwitch GIT repository + command: chdir=/git/ovs/ git clean -xdf + + - name: Reset Open vSwitch GIT repository to last comitted state + command: chdir=/git/ovs/ git reset --hard + + - name: Parse out Open vSwitch version from "configure.ac" + command: chdir=/git/ovs autoconf -t AC_INIT:'$2' + register: version + + - name: Concatenate full version + set_fact: + full_version: "{{version.stdout}}-{{ansible_local.builder.build_counter}}" + + - name: Update Open vSwitch version to {{full_version}} + command: chdir=/git/ovs/ dch -b -v {{full_version}} Vagrant Build + + - name: Build debian package with Open vSwitch build dependencies + command: chdir=/git/ovs/ mk-build-deps -B debian/control + + - name: Install Open vSwitch {{full_version}} build dependencies + apt: deb=/git/ovs/openvswitch-build-deps-depends_{{full_version}}_all.deb + + - name: Build Open vSwitch {{full_version}} debian packages + shell: DEB_BUILD_OPTIONS='parallel=4 nocheck' fakeroot debian/rules binary + args: + chdir: /git/ovs/ + + - name: Move debian packages to /var/www/html + shell: mv /git/*.deb /var/www/html/ + + - name: Create Debian Package index file for repository + shell: dpkg-scanpackages . | gzip -9c > Packages.gz + args: + chdir: /var/www/html + + - name: Bump up Build Number + copy: + content: '{ "build_counter":"{{ansible_local.builder.build_counter|int+1}}" }' + dest: "/etc/ansible/facts.d/builder.fact"