From patchwork Sat Jun 8 17:36:21 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stephen Finucane X-Patchwork-Id: 1112509 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [203.11.71.2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 45Lmnf2JqJz9sBb for ; Sun, 9 Jun 2019 03:37:58 +1000 (AEST) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=that.guru Authentication-Results: ozlabs.org; dkim=fail reason="key not found in DNS" (0-bit key; unprotected) header.d=that.guru header.i=@that.guru header.b="OV+RIHif"; dkim-atps=neutral Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 45Lmnf1LWjzDqxM for ; Sun, 9 Jun 2019 03:37:58 +1000 (AEST) X-Original-To: patchwork@lists.ozlabs.org Delivered-To: patchwork@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=none (mailfrom) smtp.mailfrom=that.guru (client-ip=172.82.139.229; helo=qrelay229.mxroute.com; envelope-from=stephen@that.guru; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=that.guru Authentication-Results: lists.ozlabs.org; dkim=fail reason="key not found in DNS" (0-bit key; unprotected) header.d=that.guru header.i=@that.guru header.b="OV+RIHif"; dkim-atps=neutral Received: from qrelay229.mxroute.com (qrelay229.mxroute.com [172.82.139.229]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 45Lmm44WW4zDqv8 for ; Sun, 9 Jun 2019 03:36:36 +1000 (AEST) Received: from filter001.mxrelay.co (unknown [116.203.155.46]) by qrelay229.mxroute.com (Postfix) with ESMTP id 2971B2A0C4A; Sat, 8 Jun 2019 13:36:33 -0400 (EDT) Received: from one.mxroute.com (one.mxroute.com [195.201.59.211]) by filter001.mxrelay.co (Postfix) with ESMTPS id 73CA71009C0; Sat, 8 Jun 2019 17:36:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=that.guru; s=default; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=ncibkGgT508Y+lbZFEl3RtGqCiTs4AE/1MbrEKrFNoM=; b=OV+RIHif0n6KpH3Jg8xhsgMoJX T/FMxF5qObTFPR1GJQLXc4PsEKBYXwL5e6AWXgw3EzSWquSjiIrB7QyLlN5uhl67F6zP93ORXzoV6 Q8Y/zVH1gibCqWw7AGDHyDvz1MRi11s/pQafg/fvxPdKmxXlzjFy3kb5CfT0Y86Gppx4nw+ua03FW awTgMZ1vHeUavf7YidLOgG5QVASGE4ln9TTNC67lltu18+qWa2G672TeY/r1qa1NO8s77QgruvmRH jIPoEGqRg/64GQ5fdSiPz1rdWQl1XEqBWtFfpTqBmseOnYOEpOvAdCCP3H0fXTjLSi9pJ4JQ0A5ps XHd4Hl+w==; From: Stephen Finucane To: patchwork@lists.ozlabs.org Subject: [PATCH v3 4/5] tox: Integrate tox-docker Date: Sat, 8 Jun 2019 18:36:21 +0100 Message-Id: <20190608173622.6711-5-stephen@that.guru> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190608173622.6711-1-stephen@that.guru> References: <20190608173622.6711-1-stephen@that.guru> MIME-Version: 1.0 X-AuthUser: stephen@that.guru X-BeenThere: patchwork@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Patchwork development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: patchwork-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Patchwork" This eliminates the need to use docker-compose for most use cases. Instead, we can now do: tox -e py27-django111-postgres If you're using a locally configured PostgreSQL or MySQL instance, you simply omit the last factor and things behave as before: tox -e py27-django111 We removed the 'venv' environment, since it was never actually that useful and is even less so now (you'd need to have a local DB set up) and add the 'skip_missing_interpreters' flag for folks on an OS that doesn't provide all the Python versions under the sun like Fedora does. Signed-off-by: Stephen Finucane Cc: Daniel Axtens --- v3: - Bump the mysql container version to 8.0 to work around an issue with tox-docker (https://github.com/tox-dev/tox-docker/issues/35) - Use the 'root' user for the mysql container since it simplifies things greatly (we do this for Travis already) - Revert changes to requirements.txt which turned out to be unnecessary and broke Travis - Remove the venv environment instead of attempting to hack it into working --- docs/development/contributing.rst | 28 +++++++--------------------- docs/development/installation.rst | 6 ------ patchwork/settings/dev.py | 12 ++++++++++++ tox.ini | 25 ++++++++++++++++++++----- 4 files changed, 39 insertions(+), 32 deletions(-) diff --git a/docs/development/contributing.rst b/docs/development/contributing.rst index 5089bba8..f713f872 100644 --- a/docs/development/contributing.rst +++ b/docs/development/contributing.rst @@ -31,29 +31,15 @@ Testing ------- Patchwork includes a `tox`_ script to automate testing. This requires a -functional database and some Python requirements like *tox*. Refer to -:doc:`installation` for information on how to configure these. - -You may also need to install *tox*. If so, do this now: +functional database and some Python requirements like *tox*. These can be +installed using :command:`pip`: .. code-block:: shell - $ pip install --user tox - -.. tip:: - - If you're using Docker, you may not need to install *tox* - locally. Instead, it will already be installed inside the - container. For Docker, you can run *tox* like so: - - .. code-block:: shell - - $ docker-compose run --rm web tox [ARGS...] - - For more information, refer to :ref:`installation-docker`. + $ pip install --user tox tox-docker -Assuming these requirements are met, actually testing Patchwork is quite easy -to do. To start, you can show the default targets like so: +Once installed, actually testing Patchwork is quite easy to do. To start, you +can show the default targets like so: .. code-block:: shell @@ -66,7 +52,7 @@ parameter: .. code-block:: shell - $ tox -e py27-django18 + $ tox -e py36-django21-mysql In the case of the unit tests targets, you can also run specific tests by passing the fully qualified test name as an additional argument to this @@ -74,7 +60,7 @@ command: .. code-block:: shell - $ tox -e py27-django18 patchwork.tests.SubjectCleanUpTest + $ tox -e py36-django21-mysql patchwork.tests.SubjectCleanUpTest Because Patchwork support multiple versions of Django, it's very important that you test against all supported versions. When run without argument, tox will do diff --git a/docs/development/installation.rst b/docs/development/installation.rst index 0ab755f4..433c3a41 100644 --- a/docs/development/installation.rst +++ b/docs/development/installation.rst @@ -86,12 +86,6 @@ To run unit tests against the system Python packages, run: $ docker-compose run --rm web python manage.py test -To run unit tests for multiple versions using ``tox``, run: - -.. code-block:: shell - - $ docker-compose run --rm web tox - To reset the database before any of these commands, add ``--reset`` to the command line after ``web`` and before any other arguments: diff --git a/patchwork/settings/dev.py b/patchwork/settings/dev.py index 53fa58f6..cfb9256c 100644 --- a/patchwork/settings/dev.py +++ b/patchwork/settings/dev.py @@ -19,6 +19,18 @@ try: except ImportError: debug_toolbar = None +# +# tox-docker settings +# + +if 'POSTGRES_5432_TCP' in os.environ: + os.environ['PW_TEST_DB_HOST'] = os.environ['POSTGRES_HOST'] + os.environ['PW_TEST_DB_PORT'] = os.environ['POSTGRES_5432_TCP'] +elif 'MYSQL_3306_TCP' in os.environ: + os.environ['PW_TEST_DB_HOST'] = os.environ['MYSQL_HOST'] + os.environ['PW_TEST_DB_PORT'] = os.environ['MYSQL_3306_TCP'] + + # # Core settings # https://docs.djangoproject.com/en/1.11/ref/settings/#core-settings diff --git a/tox.ini b/tox.ini index d4c34e1c..3c71dafd 100644 --- a/tox.ini +++ b/tox.ini @@ -1,6 +1,7 @@ [tox] minversion = 2.0 -envlist = pep8,docs,py{27,34}-django111,py{35,36}-django{111,20,21,22} +envlist = pep8,docs,py{27,34}-django111-{mysql,postgres},py{35,36}-django{111,20,21,22}-{mysql,postgres} +skip_missing_interpreters = True skipsdist = True [testenv] @@ -17,6 +18,9 @@ deps = django22: django>=2.2,<2.3 django22: djangorestframework>=3.9.2 django22: django-filter>=2.1,<3.0 +docker = + postgres: postgres:9.6 + mysql: mysql:8.0 setenv = DJANGO_SETTINGS_MODULE = patchwork.settings.dev PYTHONDONTWRITEBYTECODE = 1 @@ -24,15 +28,25 @@ setenv = py27: PYTHONWARNINGS = once py{34,36}:PYTHONWARNINGS = once,ignore::ImportWarning:backports py35:PYTHONWARNINGS = once,ignore::ResourceWarning:unittest.suite,ignore::ImportWarning:backports + postgres: PW_TEST_DB_TYPE = postgres + postgres: PW_TEST_DB_USER = postgres + postgres: PW_TEST_DB_PASS = password + mysql: PW_TEST_DB_USER = root + mysql: PW_TEST_DB_PASS = password passenv = http_proxy HTTP_PROXY https_proxy HTTPS_PROXY no_proxy NO_PROXY PW_TEST_DB_TYPE PW_TEST_DB_USER PW_TEST_DB_PASS PW_TEST_DB_HOST PW_TEST_DB_PORT +dockerenv = + postgres: POSTGRES_PASSWORD=password + mysql: MYSQL_ROOT_PASSWORD=password commands = - python {toxinidir}/manage.py test --noinput '{posargs:patchwork}' + python {toxinidir}/manage.py test --noinput {posargs:patchwork} [testenv:bashate] deps = bashate +docker = +dockerenv = whitelist_externals = bash commands = bash -c "find {toxinidir} \ @@ -43,6 +57,8 @@ commands = [testenv:pep8] basepython = python2.7 deps = flake8 +docker = +dockerenv = commands = flake8 {posargs} patchwork patchwork/bin/pwclient [flake8] @@ -57,6 +73,8 @@ exclude = ./patchwork/migrations [testenv:docs] deps = -r{toxinidir}/docs/requirements.txt +docker = +dockerenv = commands = sphinx-build -E -W -b dirhtml -d docs/_build/doctrees docs docs/_build/html @@ -67,9 +85,6 @@ deps = -r{toxinidir}/requirements-prod.txt commands = pylint patchwork --rcfile=pylint.rc -[testenv:venv] -commands = {posargs} - [testenv:coverage] basepython = python2.7 deps =