[5/7] Allow use of Docker for development

Message ID	1470097486-15269-6-git-send-email-dja@axtens.net
State	Superseded
Headers	show Return-Path: <patchwork-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org> From: Daniel Axtens <dja@axtens.net> To: patchwork@lists.ozlabs.org Subject: [PATCH 5/7] Allow use of Docker for development Date: Tue, 2 Aug 2016 10:24:44 +1000 Message-Id: <1470097486-15269-6-git-send-email-dja@axtens.net> In-Reply-To: <1470097486-15269-1-git-send-email-dja@axtens.net> References: <1470097486-15269-1-git-send-email-dja@axtens.net> Precedence: list MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: patchwork-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Patchwork" <patchwork-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org>

diff --git a/.dockerignore b/.dockerignore new file mode 100644 index 000000000000..24473bbe1237 --- /dev/null +++ b/.dockerignore @@ -0,0 +1,3 @@ +.vagrant +docker/db + diff --git a/README.md b/README.md index c83e7d31f1db..6f889c09def0 100644 --- a/README.md +++ b/README.md @@ -20,7 +20,7 @@ subsystems of the Linux kernel. Although Patchwork has been developed with the kernel workflow in mind, the aim is to be flexible enough to suit the majority of community projects. -# Development Installation +# Development Installation using Vagrant 1. Install [**Vagrant**][ref-vagrant] 2. Clone this repo: @@ -32,6 +32,42 @@ of community projects. $ cd patchwork $ vagrant up +# Development Installation using Docker + +1. Install Docker and docker-compose. +2. Clone this repo, as with vagrant. +3. Build the images. This will download ~200MB from the internet: + + $ docker-compose build + +3. Run as follows: + + * Regular server: + + $ docker-compose up + + This will be visible on http://localhost:8000/. + + * Shell: + + $ docker-compose run --rm web --shell + + * Quick test (not including selenium UI interaction tests): + + $ docker-compose run --rm web --quick-test + + * Full tests, including selenium, run headlessly: + + $ docker-compose run --rm web --test + + * To reset the database before beginning, add `--reset` to the command line after `web` and before any other arguments. + + * If you want to run non-headless tests, you'll need something like this ugly hack: + + $ docker run -it --rm -v (pwd):/home/patchwork/patchwork/ --link patchwork_db_1:db -p 8000:8000 -v /tmp/.X11-unix:/tmp/.X11-unix -e PW_TEST_DB_HOST=db -e DISPLAY patchwork_web bash + +With both vagrant and docker, any edits to the project files made locally are immediately visible to the VM/container, and so should be picked up by the Django auto-reloader. + # Talks and Presentations * [**A New Patchwork**][pdf-fosdem] - FOSDEM 2016 diff --git a/docker-compose.yml b/docker-compose.yml new file mode 100644 index 000000000000..80d2571a2dbc --- /dev/null +++ b/docker-compose.yml @@ -0,0 +1,23 @@ +# the version of docker-compose shipped in ubuntu 16.04 is +# 1.5.2, which doesn't support version 2 syntax. Yay! +# also, v1 doesn't support explicit build args, so if you're not +# uid 1000, you will either need to manually hack the Dockerfile +# or upgrade to v2 and use the build-arg to override it. + +db: + build: docker/db + volumes: + - ./docker/db/data:/var/lib/mysql +web: + build: . + dockerfile: ./docker/Dockerfile + command: python3 manage.py runserver 0.0.0.0:8000 + volumes: + - .:/home/patchwork/patchwork/ + ports: + - "8000:8000" + links: + - db + environment: + - PW_TEST_DB_HOST=db + - PW_TEST_DB_PORT=3306 \ No newline at end of file diff --git a/docker/Dockerfile b/docker/Dockerfile new file mode 100644 index 000000000000..507e6c843fe2 --- /dev/null +++ b/docker/Dockerfile @@ -0,0 +1,50 @@ +FROM ubuntu + +ARG UID=1000 + +ENV PROJECT_HOME /home/patchwork/patchwork + +ENV db_user root +ENV db_pass password + +ENV DJANGO_SETTINGS_MODULE patchwork.settings.dev +ENV DEBIAN_FRONTEND noninteractive +ENV PYTHONUNBUFFERED 1 + +ENV chromedriver_version 2.22 + +# System +COPY docker/google-chrome.list /etc/apt/sources.list.d/ +# we aren't adding the key for google here so use --allow-unauthenticated +# hicolor-icon-theme is 'recommended' but a theme is required for chrome +# to install succesfully +RUN apt-get update -qq && \ + apt-get install -y --allow-unauthenticated --no-install-recommends \ + python python3-dev python3-pip python3-setuptools python3-wheel \ + libmysqlclient-dev mysql-client curl unzip xvfb google-chrome-stable \ + hicolor-icon-theme build-essential && \ + curl http://chromedriver.storage.googleapis.com/$chromedriver_version/chromedriver_linux64.zip -o /tmp/chromedriver.zip && \ + cd /usr/local/bin && unzip /tmp/chromedriver.zip + +# User +RUN useradd --uid=$UID --create-home patchwork + +# Python requirements. +# If you update requrirements, you should rebuild the container. +COPY requirements-*.txt /tmp/ +RUN pip3 install virtualenv tox && \ + pip3 install -r /tmp/requirements-dev.txt +# we deliberately leave the requirements files in tmp so we can +# ping the user if they change it! + +COPY docker/bashrc /tmp/bashrc + +# we put the code in ~/patchwork rather than ~ so that we +# can put in these bashrc snippets +RUN cat /tmp/bashrc >> /home/patchwork/.bashrc + +COPY docker/entrypoint.sh /usr/local/bin/entrypoint.sh + +ENTRYPOINT ["/usr/local/bin/entrypoint.sh"] +USER patchwork +WORKDIR /home/patchwork/patchwork diff --git a/docker/bashrc b/docker/bashrc new file mode 100644 index 000000000000..eb2ed7dddb85 --- /dev/null +++ b/docker/bashrc @@ -0,0 +1,5 @@ +# This snippet is appended to ~/.bashrc when the container is created + +alias runserver='python3 $PROJECT_HOME/manage.py runserver 0.0.0.0:8000' +alias createsu='python3 $PROJECT_HOME/manage.py createsuperuser' + diff --git a/docker/db/.dockerignore b/docker/db/.dockerignore new file mode 100644 index 000000000000..1269488f7fb1 --- /dev/null +++ b/docker/db/.dockerignore @@ -0,0 +1 @@ +data diff --git a/docker/db/.gitignore b/docker/db/.gitignore new file mode 100644 index 000000000000..60baa9cb833f --- /dev/null +++ b/docker/db/.gitignore @@ -0,0 +1 @@ +data/* diff --git a/docker/db/Dockerfile b/docker/db/Dockerfile new file mode 100644 index 000000000000..5df9b5acb486 --- /dev/null +++ b/docker/db/Dockerfile @@ -0,0 +1,10 @@ +FROM mysql:5.7 + +ENV MYSQL_ROOT_PASSWORD password +ENV MYSQL_USER patchwork +ENV MYSQL_PASSWORD password + +# We don't want to use the MYSQL_DATABASE env here because +# we want to be able to create the database with UTF-8 explictly. +# We also can't load in the data because it's in XML, yay. + diff --git a/docker/entrypoint.sh b/docker/entrypoint.sh new file mode 100755 index 000000000000..7a42165455b1 --- /dev/null +++ b/docker/entrypoint.sh @@ -0,0 +1,76 @@ +#!/bin/bash +set -euo pipefail + +# functions +test_db_connection() { + mysqladmin -h $PW_TEST_DB_HOST -u patchwork --password=password ping > /dev/null 2> /dev/null +} + +reset_data() { + mysql -u$db_user -p$db_pass -h $PW_TEST_DB_HOST << EOF +DROP DATABASE IF EXISTS patchwork; +CREATE DATABASE patchwork CHARACTER SET utf8; +GRANT ALL ON patchwork.* TO 'patchwork' IDENTIFIED BY 'password'; +FLUSH PRIVILEGES; +EOF + + # load initial data + python3 $PROJECT_HOME/manage.py migrate #> /dev/null + python3 $PROJECT_HOME/manage.py loaddata default_tags #> /dev/null + python3 $PROJECT_HOME/manage.py loaddata default_states #> /dev/null + python3 $PROJECT_HOME/manage.py loaddata default_projects #> /dev/null +} + + +# The script begins! + +# check if patchwork is mounted. Checking if we exist is a +# very good start! +if [ ! -f ~patchwork/patchwork/docker/entrypoint.sh ]; then + echo "The patchwork directory doesn't seem to be mounted!" + echo "Are you using docker-compose?" + echo "If not, you need -v PATH_TO_PATCHWORK:/home/patchwork/patchwork" + exit 1 +fi + +# check if we need to rebuild because requirements changed +for x in /tmp/requirements-*.txt; do + if ! cmp $x ~/patchwork/$(basename $x); then + echo "A requirements file has changed." + echo "Please rebuild the patchwork image:" + echo " docker-compose build web" + exit 1 + fi +done + +# check if mysql is connected +if ! test_db_connection; then + echo "MySQL seems not to be connected, or the patchwork user is broken" + echo "MySQL may still be starting. Waiting 5 seconds." + sleep 5 + if ! test_db_connection; then + echo "Still cannot connect to MySQL." + echo "Are you using docker-compose? If not, have you set up the link correctly?" + exit 1 + fi +fi + +# rebuild mysql db +# do this on --reset or if the db doesn't exist +if [[ "$1" == "--reset" ]]; then + shift + reset_data +elif ! ( echo ';' | mysql -h db -u patchwork -ppassword patchwork 2> /dev/null ); then + reset_data +fi + +if [ "$1" == "--shell" ]; then + exec bash +elif [ "$1" == "--quick-test" ]; then + export PW_SKIP_BROWSER_TESTS=yes + python3 manage.py test +elif [ "$1" == "--test" ]; then + xvfb-run --server-args='-screen 0, 1024x768x16' python3 manage.py test +else # run whatever CMD is set to + $@ +fi diff --git a/docker/google-chrome.list b/docker/google-chrome.list new file mode 100644 index 000000000000..7c60c5415db8 --- /dev/null +++ b/docker/google-chrome.list @@ -0,0 +1 @@ +deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main

[5/7] Allow use of Docker for development

Commit Message

Comments

Patch