From patchwork Fri May 3 22:21:08 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stephen Finucane X-Patchwork-Id: 1095066 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [203.11.71.2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 44wmpS23JPz9sD4 for ; Sat, 4 May 2019 08:22:24 +1000 (AEST) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=that.guru Authentication-Results: ozlabs.org; dkim=fail reason="key not found in DNS" (0-bit key; unprotected) header.d=that.guru header.i=@that.guru header.b="nmwTsmK2"; dkim-atps=neutral Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 44wmpR3kH2zDqbY for ; Sat, 4 May 2019 08:22:23 +1000 (AEST) X-Original-To: patchwork@lists.ozlabs.org Delivered-To: patchwork@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=none (mailfrom) smtp.mailfrom=that.guru (client-ip=199.181.239.197; helo=relay0197.mxlogin.com; envelope-from=stephen@that.guru; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=that.guru Authentication-Results: lists.ozlabs.org; dkim=fail reason="key not found in DNS" (0-bit key; unprotected) header.d=that.guru header.i=@that.guru header.b="nmwTsmK2"; dkim-atps=neutral Received: from relay0197.mxlogin.com (relay0197.mxlogin.com [199.181.239.197]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 44wmnm6swFzDqYM for ; Sat, 4 May 2019 08:21:48 +1000 (AEST) Received: from filter001.mxrelay.co (unknown [116.203.155.46]) by relay0197.mxlogin.com (Postfix) with ESMTP id D4D8FCC8024F; Fri, 3 May 2019 17:21:21 -0500 (CDT) Received: from one.mxroute.com (one.mxroute.com [195.201.59.211]) by filter001.mxrelay.co (Postfix) with ESMTPS id 13F93100A01; Fri, 3 May 2019 22:21:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=that.guru; s=default; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=qlC1y1u0tAfXMpQ85q9A2FDHN4XzNMJpQK/yRXDIPaE=; b=nmwTsmK2RJjdmXaGCBeCIWUCTx 4zTdW1Mr52G16M48ZXUZRaFQYN/qJ5RoQ9B3TlK67zfczAuOY11up+rl6BP2sLRv4LjsLAFpnJ9s3 jPfZohNS/hp087hxfWoKR+V+Nx2HdrwB+Ks4WoSahZGyeocZxiXYfug/DOpDjInqVPe97HSqHoYM9 1vEnCysa0nr4D/OVdA5ptivDzgwLErmkoWmd9WfWS9NYzx+5coM5S/PRRurabjRvIbxqqMpmwfDj8 9xhKxzDHSv/C031yoHRHEju2FqsQ84Uw9RmxzobBxiFwRD7p7+K7n7Va66pleu2KMjNuD/imvFCKs zfQuO65w==; From: Stephen Finucane To: patchwork@lists.ozlabs.org Subject: [PATCH 2/4] tox: Integrate tox-docker Date: Fri, 3 May 2019 16:21:08 -0600 Message-Id: <20190503222110.562-3-stephen@that.guru> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190503222110.562-1-stephen@that.guru> References: <20190503222110.562-1-stephen@that.guru> MIME-Version: 1.0 X-AuthUser: stephen@that.guru X-BeenThere: patchwork@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Patchwork development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: patchwork-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Patchwork" This eliminates the need to use docker-compose for most use cases. Instead, we can now do: tox -e py27-django111-postgres If you're using a locally configured PostgreSQL or MySQL instance, you simply omit the last factor and things behave as before: tox -e py27-django111 Signed-off-by: Stephen Finucane Cc: Daniel Axtens --- docs/development/contributing.rst | 28 +++++++--------------------- docs/development/installation.rst | 6 ------ patchwork/settings/dev.py | 12 ++++++++++++ requirements-dev.txt | 2 ++ requirements-test.txt | 2 -- tox.ini | 27 ++++++++++++++++++++++++++- 6 files changed, 47 insertions(+), 30 deletions(-) diff --git a/docs/development/contributing.rst b/docs/development/contributing.rst index 5089bba8..f713f872 100644 --- a/docs/development/contributing.rst +++ b/docs/development/contributing.rst @@ -31,29 +31,15 @@ Testing ------- Patchwork includes a `tox`_ script to automate testing. This requires a -functional database and some Python requirements like *tox*. Refer to -:doc:`installation` for information on how to configure these. - -You may also need to install *tox*. If so, do this now: +functional database and some Python requirements like *tox*. These can be +installed using :command:`pip`: .. code-block:: shell - $ pip install --user tox - -.. tip:: - - If you're using Docker, you may not need to install *tox* - locally. Instead, it will already be installed inside the - container. For Docker, you can run *tox* like so: - - .. code-block:: shell - - $ docker-compose run --rm web tox [ARGS...] - - For more information, refer to :ref:`installation-docker`. + $ pip install --user tox tox-docker -Assuming these requirements are met, actually testing Patchwork is quite easy -to do. To start, you can show the default targets like so: +Once installed, actually testing Patchwork is quite easy to do. To start, you +can show the default targets like so: .. code-block:: shell @@ -66,7 +52,7 @@ parameter: .. code-block:: shell - $ tox -e py27-django18 + $ tox -e py36-django21-mysql In the case of the unit tests targets, you can also run specific tests by passing the fully qualified test name as an additional argument to this @@ -74,7 +60,7 @@ command: .. code-block:: shell - $ tox -e py27-django18 patchwork.tests.SubjectCleanUpTest + $ tox -e py36-django21-mysql patchwork.tests.SubjectCleanUpTest Because Patchwork support multiple versions of Django, it's very important that you test against all supported versions. When run without argument, tox will do diff --git a/docs/development/installation.rst b/docs/development/installation.rst index 0ab755f4..433c3a41 100644 --- a/docs/development/installation.rst +++ b/docs/development/installation.rst @@ -86,12 +86,6 @@ To run unit tests against the system Python packages, run: $ docker-compose run --rm web python manage.py test -To run unit tests for multiple versions using ``tox``, run: - -.. code-block:: shell - - $ docker-compose run --rm web tox - To reset the database before any of these commands, add ``--reset`` to the command line after ``web`` and before any other arguments: diff --git a/patchwork/settings/dev.py b/patchwork/settings/dev.py index 53fa58f6..400a8d70 100644 --- a/patchwork/settings/dev.py +++ b/patchwork/settings/dev.py @@ -19,6 +19,18 @@ try: except ImportError: debug_toolbar = None +# +# tox-docker settings +# + +if 'POSTGRES_5432_TCP' in os.environ: + os.environ['PW_TEST_DB_HOST'] = 'localhost' + os.environ['PW_TEST_DB_PORT'] = os.environ['POSTGRES_5432_TCP'] +elif 'MYSQL_3306_TCP' in os.environ: + os.environ['PW_TEST_DB_HOST'] = 'localhost' + os.environ['PW_TEST_DB_PORT'] = os.environ['MYSQL_3306_TCP'] + + # # Core settings # https://docs.djangoproject.com/en/1.11/ref/settings/#core-settings diff --git a/requirements-dev.txt b/requirements-dev.txt index 563e24e2..0efdf886 100644 --- a/requirements-dev.txt +++ b/requirements-dev.txt @@ -5,4 +5,6 @@ django-filter==2.1.0; python_version >= '3.4' django-filter==1.1.0; python_version < '3.0' # pyup: ignore django-debug-toolbar==1.11 django-dbbackup==3.2.0 +psycopg2-binary==2.8.2 +mysqlclient==1.3.14 -r requirements-test.txt diff --git a/requirements-test.txt b/requirements-test.txt index 0ad3c8aa..b05bc15b 100644 --- a/requirements-test.txt +++ b/requirements-test.txt @@ -1,5 +1,3 @@ -mysqlclient==1.3.14 -psycopg2-binary==2.8.2 sqlparse==0.3.0 python-dateutil==2.8.0 openapi-core==0.8.0 diff --git a/tox.ini b/tox.ini index 78b57f78..ba5d5104 100644 --- a/tox.ini +++ b/tox.ini @@ -1,6 +1,6 @@ [tox] minversion = 2.0 -envlist = pep8,docs,py{27,34}-django111,py{35,36}-django{111,20,21} +envlist = pep8,docs,py{27,34}-django111-{mysql,postgres},py{35,36}-django{111,20,21}-{mysql,postgres} skipsdist = True [testenv] @@ -14,6 +14,11 @@ deps = django21: django>=2.1,<2.2 django{20,21}: djangorestframework>=3.7,<3.10 django{20,21}: django-filter>=2.0,<3.0 + postgres: psycopg2-binary==2.7.7 + mysql: mysqlclient==1.3.14 +docker = + postgres: postgres:9.6 + mysql: mysql:5.7 setenv = DJANGO_SETTINGS_MODULE = patchwork.settings.dev PYTHONDONTWRITEBYTECODE = 1 @@ -21,15 +26,31 @@ setenv = py27: PYTHONWARNINGS = once py{34,36}:PYTHONWARNINGS = once,ignore::ImportWarning:backports py35:PYTHONWARNINGS = once,ignore::ResourceWarning:unittest.suite,ignore::ImportWarning:backports + postgres: PW_TEST_DB_TYPE = postgres + postgres: PW_TEST_DB_USER = postgres + postgres: PW_TEST_DB_PASS = password + postgres: PGPASSWORD = password passenv = http_proxy HTTP_PROXY https_proxy HTTPS_PROXY no_proxy NO_PROXY PW_TEST_DB_TYPE PW_TEST_DB_USER PW_TEST_DB_PASS PW_TEST_DB_HOST PW_TEST_DB_PORT +dockerenv = + postgres: POSTGRES_PASSWORD=password + mysql: MYSQL_ROOT_PASSWORD=password + mysql: MYSQL_USER=patchwork + mysql: MYSQL_PASSWORD=password$ +whitelist_externals = + sleep +# If we're using the docker containers, we need to sleep to ensure the +# container has time to startup commands = + {postgres,mysql}: sleep 5 python {toxinidir}/manage.py test --noinput '{posargs:patchwork}' [testenv:bashate] deps = bashate +docker = +dockerenv = whitelist_externals = bash commands = bash -c "find {toxinidir} \ @@ -40,6 +61,8 @@ commands = [testenv:pep8] basepython = python2.7 deps = flake8 +docker = +dockerenv = commands = flake8 {posargs} patchwork patchwork/bin/pwclient [flake8] @@ -54,6 +77,8 @@ exclude = ./patchwork/migrations [testenv:docs] deps = -r{toxinidir}/docs/requirements.txt +docker = +dockerenv = commands = sphinx-build -E -W -b dirhtml -d docs/_build/doctrees docs docs/_build/html