From patchwork Fri May 3 22:21:07 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stephen Finucane X-Patchwork-Id: 1095064 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [203.11.71.2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 44wmp23nJLz9sD4 for ; Sat, 4 May 2019 08:22:02 +1000 (AEST) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=that.guru Authentication-Results: ozlabs.org; dkim=fail reason="key not found in DNS" (0-bit key; unprotected) header.d=that.guru header.i=@that.guru header.b="oPXn0PFJ"; dkim-atps=neutral Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 44wmp22t39zDqZP for ; Sat, 4 May 2019 08:22:02 +1000 (AEST) X-Original-To: patchwork@lists.ozlabs.org Delivered-To: patchwork@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=none (mailfrom) smtp.mailfrom=that.guru (client-ip=199.181.239.166; helo=relay0166.mxlogin.com; envelope-from=stephen@that.guru; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=that.guru Authentication-Results: lists.ozlabs.org; dkim=fail reason="key not found in DNS" (0-bit key; unprotected) header.d=that.guru header.i=@that.guru header.b="oPXn0PFJ"; dkim-atps=neutral Received: from relay0166.mxlogin.com (relay0166.mxlogin.com [199.181.239.166]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 44wmnl2RjJzDqY2 for ; Sat, 4 May 2019 08:21:47 +1000 (AEST) Received: from filter001.mxrelay.co (unknown [94.130.183.33]) by relay0166.mxlogin.com (Postfix) with ESMTP id 3DCE4CC50323; Fri, 3 May 2019 17:21:20 -0500 (CDT) Received: from one.mxroute.com (one.mxroute.com [195.201.59.211]) by filter001.mxrelay.co (Postfix) with ESMTPS id 0B36E1009F0; Fri, 3 May 2019 22:21:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=that.guru; s=default; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=qD6K+7U32kVhQY+BaF+AsOtCTAsvoJ3A/j+XAibxpNM=; b=oPXn0PFJuBsQuORu+Zlg20CQeO /SmR0t143e2EXxHUP1BwRWUgSolOXSFtD3QCdTeC+5Fge8qMN+HhVJx2eb0G7Z6JnI9POs3QVVaHa 7GhZ7wmIydJ+IEsJnx+xiG+bepL6fzUzW3/7+uwVSw6+9HSMaiwy9lsc/8QigEe21YlSv2a1psizg hphmUGKqYlAk1E5OA2OH2e5tSJ1kqDnpQtHrreYpPEQB2LTLFlQ+A1qrBZKBZkaoPWCdWF0JpcKDB m8+TJJtUWCfjHpZ+gL27maRvy9bFPCppSkxugivUO2mcWaIdwEvO80OFo28Dk3xGmyqHIJl7Bavk/ lsLd34Ng==; From: Stephen Finucane To: patchwork@lists.ozlabs.org Subject: [PATCH 1/4] forms: Don't attempt to evaluate State at startup Date: Fri, 3 May 2019 16:21:07 -0600 Message-Id: <20190503222110.562-2-stephen@that.guru> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190503222110.562-1-stephen@that.guru> References: <20190503222110.562-1-stephen@that.guru> MIME-Version: 1.0 X-AuthUser: stephen@that.guru X-BeenThere: patchwork@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Patchwork development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: patchwork-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Patchwork" As was designed, starting the interpreter would cause the State model and its entries to be evaluated. This was an issue if, for example, the model had been modified and you were attempting to apply the migration. Traceback (most recent call last): File "manage.py", line 11, in execute_from_command_line(sys.argv) ... File "/usr/local/lib/python2.7/dist-packages/django/forms/models.py", line 1199, in _set_queryset self.widget.choices = self.choices File "/home/patchwork/patchwork/patchwork/forms.py", line 157, in _get_choices super(OptionalModelChoiceField, self)._get_choices()) File "/usr/local/lib/python2.7/dist-packages/django/forms/models.py", line 1143, in __len__ return (len(self.queryset) + (1 if self.field.empty_label is not None else 0)) File "/usr/local/lib/python2.7/dist-packages/django/db/models/query.py", line 232, in __len__ self._fetch_all() File "/usr/local/lib/python2.7/dist-packages/django/db/models/query.py", line 1118, in _fetch_all self._result_cache = list(self._iterable_class(self)) File "/usr/local/lib/python2.7/dist-packages/django/db/models/query.py", line 53, in __iter__ results = compiler.execute_sql(chunked_fetch=self.chunked_fetch) File "/usr/local/lib/python2.7/dist-packages/django/db/models/sql/compiler.py", line 899, in execute_sql raise original_exception django.db.utils.OperationalError: (1054, "Unknown column 'patchwork_state.slug' in 'field list'") Resolve this by moving the evaluation into '__init__', meaning it will only occur when a new form is created. Signed-off-by: Stephen Finucane --- patchwork/forms.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/patchwork/forms.py b/patchwork/forms.py index 5d4c920a..5690eb01 100644 --- a/patchwork/forms.py +++ b/patchwork/forms.py @@ -165,7 +165,6 @@ class OptionalBooleanField(forms.TypedChoiceField): class MultiplePatchForm(forms.Form): action = 'update' - state = OptionalModelChoiceField(queryset=State.objects.all()) archived = OptionalBooleanField( choices=[('*', 'no change'), ('True', 'Archived'), ('False', 'Unarchived')], @@ -176,6 +175,8 @@ class MultiplePatchForm(forms.Form): super(MultiplePatchForm, self).__init__(*args, **kwargs) self.fields['delegate'] = OptionalModelChoiceField( queryset=_get_delegate_qs(project=project), required=False) + self.fields['state'] = OptionalModelChoiceField( + queryset=State.objects.all()) def save(self, instance, commit=True): opts = instance.__class__._meta From patchwork Fri May 3 22:21:08 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stephen Finucane X-Patchwork-Id: 1095066 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [203.11.71.2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 44wmpS23JPz9sD4 for ; Sat, 4 May 2019 08:22:24 +1000 (AEST) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=that.guru Authentication-Results: ozlabs.org; dkim=fail reason="key not found in DNS" (0-bit key; unprotected) header.d=that.guru header.i=@that.guru header.b="nmwTsmK2"; dkim-atps=neutral Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 44wmpR3kH2zDqbY for ; Sat, 4 May 2019 08:22:23 +1000 (AEST) X-Original-To: patchwork@lists.ozlabs.org Delivered-To: patchwork@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=none (mailfrom) smtp.mailfrom=that.guru (client-ip=199.181.239.197; helo=relay0197.mxlogin.com; envelope-from=stephen@that.guru; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=that.guru Authentication-Results: lists.ozlabs.org; dkim=fail reason="key not found in DNS" (0-bit key; unprotected) header.d=that.guru header.i=@that.guru header.b="nmwTsmK2"; dkim-atps=neutral Received: from relay0197.mxlogin.com (relay0197.mxlogin.com [199.181.239.197]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 44wmnm6swFzDqYM for ; Sat, 4 May 2019 08:21:48 +1000 (AEST) Received: from filter001.mxrelay.co (unknown [116.203.155.46]) by relay0197.mxlogin.com (Postfix) with ESMTP id D4D8FCC8024F; Fri, 3 May 2019 17:21:21 -0500 (CDT) Received: from one.mxroute.com (one.mxroute.com [195.201.59.211]) by filter001.mxrelay.co (Postfix) with ESMTPS id 13F93100A01; Fri, 3 May 2019 22:21:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=that.guru; s=default; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=qlC1y1u0tAfXMpQ85q9A2FDHN4XzNMJpQK/yRXDIPaE=; b=nmwTsmK2RJjdmXaGCBeCIWUCTx 4zTdW1Mr52G16M48ZXUZRaFQYN/qJ5RoQ9B3TlK67zfczAuOY11up+rl6BP2sLRv4LjsLAFpnJ9s3 jPfZohNS/hp087hxfWoKR+V+Nx2HdrwB+Ks4WoSahZGyeocZxiXYfug/DOpDjInqVPe97HSqHoYM9 1vEnCysa0nr4D/OVdA5ptivDzgwLErmkoWmd9WfWS9NYzx+5coM5S/PRRurabjRvIbxqqMpmwfDj8 9xhKxzDHSv/C031yoHRHEju2FqsQ84Uw9RmxzobBxiFwRD7p7+K7n7Va66pleu2KMjNuD/imvFCKs zfQuO65w==; From: Stephen Finucane To: patchwork@lists.ozlabs.org Subject: [PATCH 2/4] tox: Integrate tox-docker Date: Fri, 3 May 2019 16:21:08 -0600 Message-Id: <20190503222110.562-3-stephen@that.guru> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190503222110.562-1-stephen@that.guru> References: <20190503222110.562-1-stephen@that.guru> MIME-Version: 1.0 X-AuthUser: stephen@that.guru X-BeenThere: patchwork@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Patchwork development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: patchwork-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Patchwork" This eliminates the need to use docker-compose for most use cases. Instead, we can now do: tox -e py27-django111-postgres If you're using a locally configured PostgreSQL or MySQL instance, you simply omit the last factor and things behave as before: tox -e py27-django111 Signed-off-by: Stephen Finucane Cc: Daniel Axtens --- docs/development/contributing.rst | 28 +++++++--------------------- docs/development/installation.rst | 6 ------ patchwork/settings/dev.py | 12 ++++++++++++ requirements-dev.txt | 2 ++ requirements-test.txt | 2 -- tox.ini | 27 ++++++++++++++++++++++++++- 6 files changed, 47 insertions(+), 30 deletions(-) diff --git a/docs/development/contributing.rst b/docs/development/contributing.rst index 5089bba8..f713f872 100644 --- a/docs/development/contributing.rst +++ b/docs/development/contributing.rst @@ -31,29 +31,15 @@ Testing ------- Patchwork includes a `tox`_ script to automate testing. This requires a -functional database and some Python requirements like *tox*. Refer to -:doc:`installation` for information on how to configure these. - -You may also need to install *tox*. If so, do this now: +functional database and some Python requirements like *tox*. These can be +installed using :command:`pip`: .. code-block:: shell - $ pip install --user tox - -.. tip:: - - If you're using Docker, you may not need to install *tox* - locally. Instead, it will already be installed inside the - container. For Docker, you can run *tox* like so: - - .. code-block:: shell - - $ docker-compose run --rm web tox [ARGS...] - - For more information, refer to :ref:`installation-docker`. + $ pip install --user tox tox-docker -Assuming these requirements are met, actually testing Patchwork is quite easy -to do. To start, you can show the default targets like so: +Once installed, actually testing Patchwork is quite easy to do. To start, you +can show the default targets like so: .. code-block:: shell @@ -66,7 +52,7 @@ parameter: .. code-block:: shell - $ tox -e py27-django18 + $ tox -e py36-django21-mysql In the case of the unit tests targets, you can also run specific tests by passing the fully qualified test name as an additional argument to this @@ -74,7 +60,7 @@ command: .. code-block:: shell - $ tox -e py27-django18 patchwork.tests.SubjectCleanUpTest + $ tox -e py36-django21-mysql patchwork.tests.SubjectCleanUpTest Because Patchwork support multiple versions of Django, it's very important that you test against all supported versions. When run without argument, tox will do diff --git a/docs/development/installation.rst b/docs/development/installation.rst index 0ab755f4..433c3a41 100644 --- a/docs/development/installation.rst +++ b/docs/development/installation.rst @@ -86,12 +86,6 @@ To run unit tests against the system Python packages, run: $ docker-compose run --rm web python manage.py test -To run unit tests for multiple versions using ``tox``, run: - -.. code-block:: shell - - $ docker-compose run --rm web tox - To reset the database before any of these commands, add ``--reset`` to the command line after ``web`` and before any other arguments: diff --git a/patchwork/settings/dev.py b/patchwork/settings/dev.py index 53fa58f6..400a8d70 100644 --- a/patchwork/settings/dev.py +++ b/patchwork/settings/dev.py @@ -19,6 +19,18 @@ try: except ImportError: debug_toolbar = None +# +# tox-docker settings +# + +if 'POSTGRES_5432_TCP' in os.environ: + os.environ['PW_TEST_DB_HOST'] = 'localhost' + os.environ['PW_TEST_DB_PORT'] = os.environ['POSTGRES_5432_TCP'] +elif 'MYSQL_3306_TCP' in os.environ: + os.environ['PW_TEST_DB_HOST'] = 'localhost' + os.environ['PW_TEST_DB_PORT'] = os.environ['MYSQL_3306_TCP'] + + # # Core settings # https://docs.djangoproject.com/en/1.11/ref/settings/#core-settings diff --git a/requirements-dev.txt b/requirements-dev.txt index 563e24e2..0efdf886 100644 --- a/requirements-dev.txt +++ b/requirements-dev.txt @@ -5,4 +5,6 @@ django-filter==2.1.0; python_version >= '3.4' django-filter==1.1.0; python_version < '3.0' # pyup: ignore django-debug-toolbar==1.11 django-dbbackup==3.2.0 +psycopg2-binary==2.8.2 +mysqlclient==1.3.14 -r requirements-test.txt diff --git a/requirements-test.txt b/requirements-test.txt index 0ad3c8aa..b05bc15b 100644 --- a/requirements-test.txt +++ b/requirements-test.txt @@ -1,5 +1,3 @@ -mysqlclient==1.3.14 -psycopg2-binary==2.8.2 sqlparse==0.3.0 python-dateutil==2.8.0 openapi-core==0.8.0 diff --git a/tox.ini b/tox.ini index 78b57f78..ba5d5104 100644 --- a/tox.ini +++ b/tox.ini @@ -1,6 +1,6 @@ [tox] minversion = 2.0 -envlist = pep8,docs,py{27,34}-django111,py{35,36}-django{111,20,21} +envlist = pep8,docs,py{27,34}-django111-{mysql,postgres},py{35,36}-django{111,20,21}-{mysql,postgres} skipsdist = True [testenv] @@ -14,6 +14,11 @@ deps = django21: django>=2.1,<2.2 django{20,21}: djangorestframework>=3.7,<3.10 django{20,21}: django-filter>=2.0,<3.0 + postgres: psycopg2-binary==2.7.7 + mysql: mysqlclient==1.3.14 +docker = + postgres: postgres:9.6 + mysql: mysql:5.7 setenv = DJANGO_SETTINGS_MODULE = patchwork.settings.dev PYTHONDONTWRITEBYTECODE = 1 @@ -21,15 +26,31 @@ setenv = py27: PYTHONWARNINGS = once py{34,36}:PYTHONWARNINGS = once,ignore::ImportWarning:backports py35:PYTHONWARNINGS = once,ignore::ResourceWarning:unittest.suite,ignore::ImportWarning:backports + postgres: PW_TEST_DB_TYPE = postgres + postgres: PW_TEST_DB_USER = postgres + postgres: PW_TEST_DB_PASS = password + postgres: PGPASSWORD = password passenv = http_proxy HTTP_PROXY https_proxy HTTPS_PROXY no_proxy NO_PROXY PW_TEST_DB_TYPE PW_TEST_DB_USER PW_TEST_DB_PASS PW_TEST_DB_HOST PW_TEST_DB_PORT +dockerenv = + postgres: POSTGRES_PASSWORD=password + mysql: MYSQL_ROOT_PASSWORD=password + mysql: MYSQL_USER=patchwork + mysql: MYSQL_PASSWORD=password$ +whitelist_externals = + sleep +# If we're using the docker containers, we need to sleep to ensure the +# container has time to startup commands = + {postgres,mysql}: sleep 5 python {toxinidir}/manage.py test --noinput '{posargs:patchwork}' [testenv:bashate] deps = bashate +docker = +dockerenv = whitelist_externals = bash commands = bash -c "find {toxinidir} \ @@ -40,6 +61,8 @@ commands = [testenv:pep8] basepython = python2.7 deps = flake8 +docker = +dockerenv = commands = flake8 {posargs} patchwork patchwork/bin/pwclient [flake8] @@ -54,6 +77,8 @@ exclude = ./patchwork/migrations [testenv:docs] deps = -r{toxinidir}/docs/requirements.txt +docker = +dockerenv = commands = sphinx-build -E -W -b dirhtml -d docs/_build/doctrees docs docs/_build/html From patchwork Fri May 3 22:21:09 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stephen Finucane X-Patchwork-Id: 1095065 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [203.11.71.2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 44wmpF0sWpz9sD4 for ; Sat, 4 May 2019 08:22:13 +1000 (AEST) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=that.guru Authentication-Results: ozlabs.org; dkim=fail reason="key not found in DNS" (0-bit key; unprotected) header.d=that.guru header.i=@that.guru header.b="tC6bDAkw"; dkim-atps=neutral Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 44wmpD73fczDqYX for ; Sat, 4 May 2019 08:22:12 +1000 (AEST) X-Original-To: patchwork@lists.ozlabs.org Delivered-To: patchwork@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=none (mailfrom) smtp.mailfrom=that.guru (client-ip=199.181.239.147; helo=relay0147.mxlogin.com; envelope-from=stephen@that.guru; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=that.guru Authentication-Results: lists.ozlabs.org; dkim=fail reason="key not found in DNS" (0-bit key; unprotected) header.d=that.guru header.i=@that.guru header.b="tC6bDAkw"; dkim-atps=neutral Received: from relay0147.mxlogin.com (relay0147.mxlogin.com [199.181.239.147]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 44wmnm3m7jzDqY2 for ; Sat, 4 May 2019 08:21:48 +1000 (AEST) Received: from filter002.mxroute.com (unknown [94.130.183.33]) by relay0147.mxlogin.com (Postfix) with ESMTP id 8CF57CC3032C; Fri, 3 May 2019 17:21:21 -0500 (CDT) Received: from one.mxroute.com (one.mxroute.com [195.201.59.211]) by filter002.mxroute.com (Postfix) with ESMTPS id 89BB73F0B9; Fri, 3 May 2019 22:21:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=that.guru; s=default; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=W3kxe2HAQMy4/aLJUzgUL2JxBPEkV/cer0RXS6fx+js=; b=tC6bDAkw5vk3AOxVdRAMZEp4m9 g2LKSH3s+V5IbuGyvfq4eQD1pj0h/A2WMu2dpTdHPU4fxSodsMii2IVjldEaAA7q71O5P9JUFBhpa qRDD2cOkBXLPPkIGd+rVXHusMfTVUDjVQHjzHNU7C99kE2O+b0GhVy3bqg4/zyzEb4pq5eoAZNVG4 bc3PUeOLXRzyjkKtp456w6IvHYKGfISpaXY+e30kCQt0CIlKZzrclY9yRIodK8rinVs+ceWZc/NRo M8O2MnCa+DKfDOIeIhLHdGx2wAmE04zecSD+eNX0XkGaiMShj57W5QpUJaqPV1439EVfJoAD6yy/f yegZ1gbA==; From: Stephen Finucane To: patchwork@lists.ozlabs.org Subject: [PATCH 3/4] requirements: Switch to PyMySQL Date: Fri, 3 May 2019 16:21:09 -0600 Message-Id: <20190503222110.562-4-stephen@that.guru> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190503222110.562-1-stephen@that.guru> References: <20190503222110.562-1-stephen@that.guru> MIME-Version: 1.0 X-AuthUser: stephen@that.guru X-BeenThere: patchwork@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Patchwork development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: patchwork-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Patchwork" This is pure Python, which means there's no need for those pesky MySQL development libs. Signed-off-by: Stephen Finucane --- manage.py | 12 ++++++++++-- requirements-dev.txt | 2 +- tox.ini | 2 +- 3 files changed, 12 insertions(+), 4 deletions(-) diff --git a/manage.py b/manage.py index fe6189ee..86bd67e9 100755 --- a/manage.py +++ b/manage.py @@ -2,9 +2,17 @@ import os import sys +try: + # From https://github.com/PyMySQL/PyMySQL/wiki/WhyPyMySQL + import pymysql + pymysql.install_as_MySQLdb() +except ImportError: + pass + + if __name__ == "__main__": - os.environ.setdefault("DJANGO_SETTINGS_MODULE", - "patchwork.settings.production") + os.environ.setdefault( + "DJANGO_SETTINGS_MODULE", "patchwork.settings.production") from django.core.management import execute_from_command_line diff --git a/requirements-dev.txt b/requirements-dev.txt index 0efdf886..06eefd99 100644 --- a/requirements-dev.txt +++ b/requirements-dev.txt @@ -6,5 +6,5 @@ django-filter==1.1.0; python_version < '3.0' # pyup: ignore django-debug-toolbar==1.11 django-dbbackup==3.2.0 psycopg2-binary==2.8.2 -mysqlclient==1.3.14 +PyMySQL==0.9.3 -r requirements-test.txt diff --git a/tox.ini b/tox.ini index ba5d5104..86fb62e1 100644 --- a/tox.ini +++ b/tox.ini @@ -15,7 +15,7 @@ deps = django{20,21}: djangorestframework>=3.7,<3.10 django{20,21}: django-filter>=2.0,<3.0 postgres: psycopg2-binary==2.7.7 - mysql: mysqlclient==1.3.14 + mysql: PyMySQL==0.9.3 docker = postgres: postgres:9.6 mysql: mysql:5.7 From patchwork Fri May 3 22:21:10 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stephen Finucane X-Patchwork-Id: 1095067 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [203.11.71.2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 44wmpg2jLGz9sD4 for ; Sat, 4 May 2019 08:22:35 +1000 (AEST) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=that.guru Authentication-Results: ozlabs.org; dkim=fail reason="key not found in DNS" (0-bit key; unprotected) header.d=that.guru header.i=@that.guru header.b="sNxDthcp"; dkim-atps=neutral Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 44wmpg1dVvzDqGJ for ; Sat, 4 May 2019 08:22:35 +1000 (AEST) X-Original-To: patchwork@lists.ozlabs.org Delivered-To: patchwork@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=none (mailfrom) smtp.mailfrom=that.guru (client-ip=199.181.239.188; helo=relay0188.mxlogin.com; envelope-from=stephen@that.guru; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=that.guru Authentication-Results: lists.ozlabs.org; dkim=fail reason="key not found in DNS" (0-bit key; unprotected) header.d=that.guru header.i=@that.guru header.b="sNxDthcp"; dkim-atps=neutral Received: from relay0188.mxlogin.com (relay0188.mxlogin.com [199.181.239.188]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 44wmnq4GGxzDqZ2 for ; Sat, 4 May 2019 08:21:51 +1000 (AEST) Received: from filter001.mxrelay.co (unknown [116.203.155.46]) by relay0188.mxlogin.com (Postfix) with ESMTP id 6B442CC70314; Fri, 3 May 2019 17:21:24 -0500 (CDT) Received: from one.mxroute.com (one.mxroute.com [195.201.59.211]) by filter001.mxrelay.co (Postfix) with ESMTPS id A0E6910003F; Fri, 3 May 2019 22:21:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=that.guru; s=default; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=px91YGuNdOKbVMnmfCBqcMeaZHS+srZ3vH2ejV2B1/o=; b=sNxDthcp2svuuX1gFLID4UAMDJ PjhBKvYVNz2SbsheEaKnDYruK5dXNchsV620rsJ418wxNLO9WIXf+5xOBfSqT6NMnx4WLXTj7d0Ob kZdldX0UJp4swahTKJFWxCph953Y8Q//dRPkYeegB0obC38SnH5rtrgLIn2ICXnVGZybtCGuPpRop bhCD9kfhR/nqMSFTjqrLdtk1efPd480cY9PS5H6DFLW93JGmEOeU9iSQfnEeAOGzM9fKatfUveasO RPpWcuM5GysnU5HrfaTRAPKKto3rWX2SvB+GYbLVFV79lu7HuP2XwIm8BjPA2TWXXNjNNf2laIiN3 lYodiDtw==; From: Stephen Finucane To: patchwork@lists.ozlabs.org Subject: [PATCH 4/4] docker-compose: Massively simplify Date: Fri, 3 May 2019 16:21:10 -0600 Message-Id: <20190503222110.562-5-stephen@that.guru> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190503222110.562-1-stephen@that.guru> References: <20190503222110.562-1-stephen@that.guru> MIME-Version: 1.0 X-AuthUser: stephen@that.guru X-BeenThere: patchwork@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Patchwork development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: patchwork-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Patchwork" Now that we can run tox on the localhost, we no longer need to install the variety of Python versions we previously did and can keep this purely for the 'manage.py' use cases. Note that we're also able to remove the 'libmysqlclient-dev' and 'mysql-client' packages thanks to our switch to 'PyMySQL', which is pure Python. Signed-off-by: Stephen Finucane --- tools/docker/Dockerfile | 22 ++-------------------- tools/docker/entrypoint.sh | 4 ++-- tools/docker/trusty-ports.list | 3 --- tools/docker/trusty.list | 3 --- tools/docker/xenial-ports.list | 3 --- tools/docker/xenial.list | 3 --- tox.ini | 15 ++++++++++++++- 7 files changed, 18 insertions(+), 35 deletions(-) delete mode 100644 tools/docker/trusty-ports.list delete mode 100644 tools/docker/trusty.list delete mode 100644 tools/docker/xenial-ports.list delete mode 100644 tools/docker/xenial.list diff --git a/tools/docker/Dockerfile b/tools/docker/Dockerfile index 76bb6b2b..5653ec89 100644 --- a/tools/docker/Dockerfile +++ b/tools/docker/Dockerfile @@ -17,26 +17,10 @@ ENV PYTHONUNBUFFERED 1 # System -# trusty and findutils is for python3.4; xenial is for python3.5 -# TODO(stephenfin): Are curl, unzip required? -COPY tools/docker/*.list /etc/apt/sources.list.d/ - -RUN cd /etc/apt/sources.list.d; \ - echo $(uname -m) > /tmp/arch; \ - if [ $(cat /tmp/arch) != 'x86_64' ] && grep -q -v "i.86" /tmp/arch; then \ - mv trusty-ports.list trusty.list; \ - mv xenial-ports.list xenial.list; \ - else \ - rm *-ports.list; \ - fi - RUN apt-get update -qq && \ apt-get install -y --no-install-recommends --allow-downgrades \ - python-dev python-pip python-setuptools python-wheel \ - python3.5-dev python3-pip python3-setuptools python3-wheel \ - python3.4-dev findutils=4.4.2-7 python3.6-dev \ - libmysqlclient-dev mysql-client curl unzip build-essential \ - git postgresql-client tzdata libpq-dev + python3-dev python3-pip python3-setuptools python3-wheel \ + postgresql-client tzdata # User RUN useradd --uid=$UID --create-home patchwork @@ -51,8 +35,6 @@ RUN rm /etc/localtime; ln -s /usr/share/zoneinfo/$TZ /etc/localtime COPY requirements-*.txt /tmp/ RUN pip3 install virtualenv tox && \ pip3 install -r /tmp/requirements-dev.txt -RUN pip2 install virtualenv tox && \ - pip2 install -r /tmp/requirements-dev.txt # we deliberately leave the requirements files in tmp so we can # ping the user in entrypoint.sh if the change them! diff --git a/tools/docker/entrypoint.sh b/tools/docker/entrypoint.sh index 32f7132a..b28de5a0 100755 --- a/tools/docker/entrypoint.sh +++ b/tools/docker/entrypoint.sh @@ -124,8 +124,8 @@ elif [ "$1" == "--test" ] || [ "$1" == "--quick-test" ]; then shift python3 manage.py test $@ elif [ "$1" == "--tox" ] || [ "$1" == "--quick-tox" ]; then - shift - tox $@ + echo "tox is no longer installed here; use e.g. 'tox -e py36-django21-mysql' instead" + exit 0 else # run whatever CMD is set to $@ fi diff --git a/tools/docker/trusty-ports.list b/tools/docker/trusty-ports.list deleted file mode 100644 index ebcf4fa4..00000000 --- a/tools/docker/trusty-ports.list +++ /dev/null @@ -1,3 +0,0 @@ -deb http://ports.ubuntu.com/ubuntu-ports/ trusty main -deb http://ports.ubuntu.com/ubuntu-ports/ trusty-updates main -deb http://ports.ubuntu.com/ubuntu-ports/ trusty-security main diff --git a/tools/docker/trusty.list b/tools/docker/trusty.list deleted file mode 100644 index 8bb92c09..00000000 --- a/tools/docker/trusty.list +++ /dev/null @@ -1,3 +0,0 @@ -deb http://archive.ubuntu.com/ubuntu/ trusty main -deb http://archive.ubuntu.com/ubuntu/ trusty-updates main -deb http://security.ubuntu.com/ubuntu trusty-security main diff --git a/tools/docker/xenial-ports.list b/tools/docker/xenial-ports.list deleted file mode 100644 index d84641fa..00000000 --- a/tools/docker/xenial-ports.list +++ /dev/null @@ -1,3 +0,0 @@ -deb http://ports.ubuntu.com/ubuntu-ports/ xenial main -deb http://ports.ubuntu.com/ubuntu-ports/ xenial-updates main -deb http://ports.ubuntu.com/ubuntu-ports/ xenial-security main diff --git a/tools/docker/xenial.list b/tools/docker/xenial.list deleted file mode 100644 index a70ff56a..00000000 --- a/tools/docker/xenial.list +++ /dev/null @@ -1,3 +0,0 @@ -deb http://archive.ubuntu.com/ubuntu/ xenial main -deb http://archive.ubuntu.com/ubuntu/ xenial-updates main -deb http://security.ubuntu.com/ubuntu xenial-security main diff --git a/tox.ini b/tox.ini index 86fb62e1..22291748 100644 --- a/tox.ini +++ b/tox.ini @@ -90,7 +90,20 @@ deps = commands = pylint patchwork --rcfile=pylint.rc [testenv:venv] -commands = {posargs} +deps = + -rrequirements-dev.txt +docker = + postgres:9.6 +setenv = + PW_TEST_DB_TYPE = postgres + PW_TEST_DB_USER = postgres + PW_TEST_DB_PASS = password + PGPASSWORD = password +dockerenv = + POSTGRES_PASSWORD=password +commands = + sleep 5 + {posargs} [testenv:coverage] basepython = python2.7