From patchwork Tue Dec 7 13:15:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: You-Sheng Yang X-Patchwork-Id: 1564703 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=A0LhVFNH; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=112.213.38.117; helo=lists.ozlabs.org; envelope-from=patchwork-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=) Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4J7glC147Jz9s1l for ; Wed, 8 Dec 2021 00:15:19 +1100 (AEDT) Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4J7glB71QGz2yg4 for ; Wed, 8 Dec 2021 00:15:18 +1100 (AEDT) Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=A0LhVFNH; dkim-atps=neutral X-Original-To: patchwork@lists.ozlabs.org Delivered-To: patchwork@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gmail.com (client-ip=2607:f8b0:4864:20::102b; helo=mail-pj1-x102b.google.com; envelope-from=vicamo@gmail.com; receiver=) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=A0LhVFNH; dkim-atps=neutral Received: from mail-pj1-x102b.google.com (mail-pj1-x102b.google.com [IPv6:2607:f8b0:4864:20::102b]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4J7gl24sqNz2yb5 for ; Wed, 8 Dec 2021 00:15:09 +1100 (AEDT) Received: by mail-pj1-x102b.google.com with SMTP id n15-20020a17090a160f00b001a75089daa3so1901379pja.1 for ; Tue, 07 Dec 2021 05:15:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=vs/Mb4ac1wHnJs4db9D6ww7uaHkzJYH3ylSoXWwOekM=; b=A0LhVFNHtgdet4J/e3dIBz+QELN+kZv+SlJwWtYzc7pMDZ3P+iCvHy9b8fmxdQqmdO b2x9VxRKjanLen0VQPTmRG6Wvj1RAVbyy61TswscIFf646//+9Ub6iiazr90JSE/XjPg 0jF5xoU8aNKbdTFI8LvHY6kC4lfJKlvh/SiIkZoqwSng7dmaDSKWr7E7a07YwMW3XXRD USnD7g5VFMXd10dU2T6QzvqBcX6eara5O3ocHVYjQUeqFAkXjl5aFSWnnvWj+UBt7O9i HomivMmB8xduTMP73AJtUQYvakR1XDv5osYxHTywFQ6KmcvmtwsSEyn683IvEOdu2+QZ 3E8g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=vs/Mb4ac1wHnJs4db9D6ww7uaHkzJYH3ylSoXWwOekM=; b=B9WCiYd9qrlI/I6MPcqZAFnADhjna4Wou3az4fyo4l1oUqWai9YEEQKCOy40/sjNgv 1qLsdBEiEQr7WhBH/i92cNY6EZCS2r49IZAuX7+CfTRLybLGl8wZ/Tp3jNcWPkPj3tZI xRWMzQQ0Zy10UaMiNqztr5S4/5V9DJO5Lpk7mlWa/psB1mcLnY4guPZYim9xpx6oQQa+ v9sHrQiqOZSKTGjE01DYJMgg3uUVrMRcZf550Sk54gcuJKsXuhBBfrxpHNu2deako9zl NfDybszG10bUZNJmX8ozeSyBBKZ39JkmKkju14zmUW9BYtxH8DEPecPf9q0UpkKJ9oEa 8U2Q== X-Gm-Message-State: AOAM5331R1A2K9jOvjjjpOqbGdT0155NmnJd9XMOsiI7CR58Ykr95vGA Pc65gth1c1nmYB+Ub7vZS2EAfxDjnzI= X-Google-Smtp-Source: ABdhPJwMVqaWinCTzM/O20r18THIyiJBhNhqHI+lHNifimHKwp29M9phXoq0Zsabgl9/mFiTFVER9A== X-Received: by 2002:a17:902:82c9:b0:142:401f:dc9 with SMTP id u9-20020a17090282c900b00142401f0dc9mr51999105plz.43.1638882907318; Tue, 07 Dec 2021 05:15:07 -0800 (PST) Received: from localhost (218-173-141-247.dynamic-ip.hinet.net. [218.173.141.247]) by smtp.gmail.com with ESMTPSA id ls14sm3137899pjb.49.2021.12.07.05.15.06 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Dec 2021 05:15:07 -0800 (PST) From: You-Sheng Yang To: patchwork@lists.ozlabs.org Subject: [PATCH v2 1/4] settings: unify database connection variables Date: Tue, 7 Dec 2021 21:15:00 +0800 Message-Id: <20211207131503.416945-2-vicamo@gmail.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211207131503.416945-1-vicamo@gmail.com> References: <20211207131503.416945-1-vicamo@gmail.com> MIME-Version: 1.0 X-BeenThere: patchwork@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Patchwork development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: patchwork-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Patchwork" This changeset replaces all PW_TEST_DB_* with corresponding DATABASE_* variables, fixes inconsistent DATABASE_PASS/DATABASE_PASSWORD usages, and moves database settings into base.py for sharing/syncing between dev and production setup. Signed-off-by: You-Sheng Yang --- .github/workflows/ci.yaml | 40 +++++++++++-------- docker-compose-pg.yml | 12 +++--- docker-compose.yml | 10 +++-- docs/deployment/installation.rst | 4 +- docs/development/installation.rst | 13 +++--- lib/uwsgi/patchwork.ini | 2 +- patchwork/settings/base.py | 39 ++++++++++++++++++ patchwork/settings/dev.py | 28 +------------ patchwork/settings/production.example.py | 17 -------- tools/docker/entrypoint.sh | 51 ++++++++++++++++-------- tox.ini | 4 +- 11 files changed, 123 insertions(+), 97 deletions(-) diff --git a/.github/workflows/ci.yaml b/.github/workflows/ci.yaml index f4a33b3..d37cdbf 100644 --- a/.github/workflows/ci.yaml +++ b/.github/workflows/ci.yaml @@ -25,13 +25,20 @@ jobs: matrix: python: [3.6, 3.7, 3.8, 3.9] db: [postgres, mysql] + env: + DATABASE_TYPE: "${{ matrix.db }}" + DATABASE_HOST: "127.0.0.1" + DATABASE_NAME: patchwork + DATABASE_USER: patchwork + DATABASE_PASSWORD: password + MYSQL_ROOT_PASSWORD: root services: postgres: image: postgres:latest env: - POSTGRES_DB: patchwork - POSTGRES_PASSWORD: patchwork - POSTGRES_USER: patchwork + POSTGRES_DB: ${{ env.DATABASE_NAME }} + POSTGRES_PASSWORD: ${{ env.DATABASE_PASSWORD }} + POSTGRES_USER: ${{ env.DATABASE_USER }} ports: - 5432:5432 options: >- @@ -42,10 +49,10 @@ jobs: mysql: image: mysql:latest env: - MYSQL_DATABASE: patchwork - MYSQL_USER: patchwork - MYSQL_PASSWORD: patchwork - MYSQL_ROOT_PASSWORD: root + MYSQL_DATABASE: ${{ env.DATABASE_NAME }} + MYSQL_USER: ${{ env.DATABASE_USER }} + MYSQL_PASSWORD: ${{ env.DATABASE_PASSWORD }} + MYSQL_ROOT_PASSWORD: ${{ env.MYSQL_ROOT_PASSWORD }} ports: - 3306:3306 options: >- @@ -64,22 +71,23 @@ jobs: run: python -m pip install tox tox-gh-actions codecov - name: Log database configuration (mysql) if: ${{ matrix.db == 'mysql' }} - run: mysql -h 127.0.0.1 -e "SELECT VERSION(), CURRENT_USER();" -uroot -proot patchwork + run: | + mysql -h 127.0.0.1 -e "SELECT VERSION(), CURRENT_USER();" \ + -uroot -p${MYSQL_ROOT_PASSWORD} ${DATABASE_NAME} - name: Log database configuration (postgres) if: ${{ matrix.db == 'postgres' }} - run: psql -h 127.0.0.1 -c "SELECT VERSION(), CURRENT_USER, current_database()" -U patchwork -d patchwork + run: | + psql -h 127.0.0.1 -c "SELECT VERSION(), CURRENT_USER, current_database()" \ + -U ${DATABASE_USER} -d ${DATABASE_NAME} env: - PGPASSWORD: patchwork + PGPASSWORD: ${{ env.DATABASE_PASSWORD }} - name: Modify database user permissions (mysql) if: ${{ matrix.db == 'mysql' }} - run: mysql -h 127.0.0.1 -e "GRANT ALL ON \`test\\_patchwork%\`.* to 'patchwork'@'%';" -uroot -proot + run: | + mysql -h 127.0.0.1 -e "GRANT ALL ON \`test\\_${DATABASE_NAME}%\`.* to '${DATABASE_USER}'@'%';" \ + -uroot -p${MYSQL_ROOT_PASSWORD} - name: Run unit tests (via tox) run: tox - env: - PW_TEST_DB_TYPE: "${{ matrix.db }}" - PW_TEST_DB_USER: "patchwork" - PW_TEST_DB_PASS: "patchwork" - PW_TEST_DB_HOST: "127.0.0.1" docs: name: Build docs runs-on: ubuntu-latest diff --git a/docker-compose-pg.yml b/docker-compose-pg.yml index 44c49fb..7398929 100644 --- a/docker-compose-pg.yml +++ b/docker-compose-pg.yml @@ -24,9 +24,9 @@ services: environment: - UID - GID - - PW_TEST_DB_HOST=db - - PW_TEST_DB_PORT=5432 - - PW_TEST_DB_TYPE=postgres - - PW_TEST_DB_USER=postgres - - PW_TEST_DB_PASS=password - - PGPASSWORD=password + - DATABASE_TYPE=postgres + - DATABASE_HOST=db + - DATABASE_PORT=5432 + - DATABASE_NAME=patchwork + - DATABASE_USER=postgres + - DATABASE_PASSWORD=password diff --git a/docker-compose.yml b/docker-compose.yml index 1d49c51..103f19f 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -5,7 +5,7 @@ services: volumes: - ./tools/docker/db/data:/var/lib/mysql environment: - - MYSQL_ROOT_PASSWORD=password + - MYSQL_ROOT_PASSWORD=root - MYSQL_USER=patchwork - MYSQL_PASSWORD=password @@ -26,5 +26,9 @@ services: environment: - UID - GID - - PW_TEST_DB_HOST=db - - PW_TEST_DB_PORT=3306 + # skip DATABASE_TYPE explicitly as mysql should be the default type. + - DATABASE_HOST=db + - DATABASE_PORT=3306 + - DATABASE_USER=patchwork + - DATABASE_PASSWORD=password + - MYSQL_ROOT_PASSWORD=root diff --git a/docs/deployment/installation.rst b/docs/deployment/installation.rst index 1abf187..14ad746 100644 --- a/docs/deployment/installation.rst +++ b/docs/deployment/installation.rst @@ -64,7 +64,7 @@ used to ease deployment: Username that the Patchwork web application will access the database with. We will use ``www-data``, for reasons described later in this guide. -``DATABASE_PASS=`` +``DATABASE_PASSWORD=`` Password that the Patchwork web application will access the database with. As we're going to use *peer* authentication (more on this later), this will be unset. @@ -323,7 +323,7 @@ Once done, we should be able to check that all requirements are met using the $ sudo -u www-data \ --preserve-env=DATABASE_NAME \ --preserve-env=DATABASE_USER \ - --preserve-env=DATABASE_PASS \ + --preserve-env=DATABASE_PASSWORD \ --preserve-env=DATABASE_HOST \ --preserve-env=DATABASE_PORT \ --preserve-env=STATIC_ROOT \ diff --git a/docs/development/installation.rst b/docs/development/installation.rst index d16177c..3507f43 100644 --- a/docs/development/installation.rst +++ b/docs/development/installation.rst @@ -316,7 +316,7 @@ if necessary: The ``patchwork`` username and ``password`` password are the defaults expected by the provided ``dev`` settings files. If using something - different, export the ``PW_TEST_DB_USER`` and ``PW_TEST_DB_PASS`` variables + different, export the ``DATABASE_USER`` and ``DATABASE_PASSWORD`` variables described in the :ref:`Environment Variables ` section below. Alternatively, you can create your own settings file with these variables hardcoded and change the value of ``DJANGO_SETTINGS_MODULE`` as described @@ -470,17 +470,16 @@ __ https://django-dbbackup.readthedocs.io/en/stable/ Environment Variables --------------------- -The following environment variables are available to configure settings when -using the provided ``dev`` settings file. +The following environment variables are available to configure settings. -``PW_TEST_DB_NAME=patchwork`` +``DATABASE_NAME=patchwork`` Name of the database -``PW_TEST_DB_USER=patchwork`` +``DATABASE_USER=patchwork`` Username to access the database with -``PW_TEST_DB_PASS=password`` +``DATABASE_PASSWORD=password`` Password to access the database with< -``PW_TEST_DB_TYPE=mysql`` +``DATABASE_TYPE=mysql`` Type of database to use. Options: ``mysql``, ``postgres`` diff --git a/lib/uwsgi/patchwork.ini b/lib/uwsgi/patchwork.ini index 95a0613..dbff508 100644 --- a/lib/uwsgi/patchwork.ini +++ b/lib/uwsgi/patchwork.ini @@ -10,7 +10,7 @@ url = / # env = DJANGO_SECRET_KEY= # env = DATABASE_NAME= # env = DATABASE_USER= -# env = DATABASE_PASS= +# env = DATABASE_PASSWORD= # env = DATABASE_HOST= # env = DATABASE_PORT= # env = STATIC_ROOT= diff --git a/patchwork/settings/base.py b/patchwork/settings/base.py index ff14d91..5e200f0 100644 --- a/patchwork/settings/base.py +++ b/patchwork/settings/base.py @@ -105,6 +105,45 @@ STATICFILES_DIRS = [ os.path.join(ROOT_DIR, 'htdocs'), ] +# Database +# +# If you're using a postgres database, connecting over a local unix-domain +# socket, then the following setting should work for you. Otherwise, +# see https://docs.djangoproject.com/en/2.2/ref/settings/#databases + +if os.getenv('DATABASE_TYPE', None) == 'postgres': + DATABASES = { + 'default': { + 'ENGINE': 'django.db.backends.postgresql_psycopg2', + 'HOST': os.environ.get('DATABASE_HOST', 'localhost'), + 'PORT': os.environ.get('DATABASE_PORT', ''), + 'NAME': os.environ.get('DATABASE_NAME', 'patchwork'), + 'USER': os.environ.get('DATABASE_USER', 'patchwork'), + 'PASSWORD': os.environ.get('DATABASE_PASSWORD', 'password'), + }, + } +elif os.getenv('DATABASE_TYPE', None) == 'sqlite3': + DATABASES = { + 'default': { + 'ENGINE': 'django.db.backends.sqlite3', + 'NAME': os.environ.get('DATABASE_NAME', ''), + }, + } +else: + DATABASES = { + 'default': { + 'ENGINE': 'django.db.backends.mysql', + 'HOST': os.getenv('DATABASE_HOST', 'localhost'), + 'PORT': os.getenv('DATABASE_PORT', ''), + 'NAME': os.getenv('DATABASE_NAME', 'patchwork'), + 'USER': os.getenv('DATABASE_USER', 'patchwork'), + 'PASSWORD': os.getenv('DATABASE_PASSWORD', 'password'), + 'TEST': { + 'CHARSET': 'utf8', + }, + }, + } + # # Third-party application settings # diff --git a/patchwork/settings/dev.py b/patchwork/settings/dev.py index 2b7fc95..cb4cb19 100644 --- a/patchwork/settings/dev.py +++ b/patchwork/settings/dev.py @@ -7,8 +7,6 @@ Design based on: http://www.revsys.com/blog/2014/nov/21/recommended-django-project-layout/ """ -import os - from .base import * # noqa try: @@ -36,30 +34,8 @@ SECRET_KEY = '00000000000000000000000000000000000000000000000000' # noqa DEBUG = True -DATABASES = { - 'default': { - 'ENGINE': 'django.db.backends.mysql', - 'HOST': os.getenv('PW_TEST_DB_HOST', 'localhost'), - 'PORT': os.getenv('PW_TEST_DB_PORT', ''), - 'USER': os.getenv('PW_TEST_DB_USER', 'patchwork'), - 'PASSWORD': os.getenv('PW_TEST_DB_PASS', 'password'), - 'NAME': os.getenv('PW_TEST_DB_NAME', 'patchwork'), - 'TEST': { - 'CHARSET': 'utf8', - }, - }, -} - -if os.getenv('PW_TEST_DB_TYPE', None) == 'postgres': - DATABASES['default']['ENGINE'] = 'django.db.backends.postgresql_psycopg2' - DATABASES['default']['HOST'] = os.getenv('PW_TEST_DB_HOST', '') -elif os.getenv('PW_TEST_DB_TYPE', None) == 'sqlite': - DATABASES['default']['ENGINE'] = 'django.db.backends.sqlite3' - DATABASES['default']['NAME'] = '/dev/shm/patchwork.test.db.sqlite3' - del DATABASES['default']['HOST'] - del DATABASES['default']['PORT'] - del DATABASES['default']['USER'] - del DATABASES['default']['PASSWORD'] +if DATABASES['default']['ENGINE'] == 'mysql': # noqa: F405 + DATABASES['default']['TEST'] = {'CHARSET': 'utf8'} # noqa: F405 EMAIL_BACKEND = 'django.core.mail.backends.console.EmailBackend' diff --git a/patchwork/settings/production.example.py b/patchwork/settings/production.example.py index caaf429..ff05629 100644 --- a/patchwork/settings/production.example.py +++ b/patchwork/settings/production.example.py @@ -48,23 +48,6 @@ ADMINS = ( # ('Jeremy Kerr', 'jk@ozlabs.org'), ) -# Database -# -# If you're using a postgres database, connecting over a local unix-domain -# socket, then the following setting should work for you. Otherwise, -# see https://docs.djangoproject.com/en/2.2/ref/settings/#databases - -DATABASES = { - 'default': { - 'ENGINE': 'django.db.backends.postgresql_psycopg2', - 'NAME': os.environ.get('DATABASE_NAME', ''), - 'USER': os.environ.get('DATABASE_USER', ''), - 'PASSWORD': os.environ.get('DATABASE_PASSWORD', ''), - 'HOST': os.environ.get('DATABASE_HOST', ''), - 'PORT': os.environ.get('DATABASE_PORT', ''), - }, -} - # # Static files settings # https://docs.djangoproject.com/en/2.2/ref/settings/#static-files diff --git a/tools/docker/entrypoint.sh b/tools/docker/entrypoint.sh index 5450a53..faa36b3 100755 --- a/tools/docker/entrypoint.sh +++ b/tools/docker/entrypoint.sh @@ -1,45 +1,62 @@ #!/bin/bash set -euo pipefail -PW_TEST_DB_TYPE=${PW_TEST_DB_TYPE:-mysql} +export DATABASE_HOST=${DATABASE_HOST:-} +export DATABASE_PORT=${DATABASE_PORT:-} +export DATABASE_NAME=${DATABASE_NAME:-patchwork} +export DATABASE_USER=${DATABASE_USER:-patchwork} +export DATABASE_PASSWORD=${DATABASE_PASSWORD:-password} + +case "${DATABASE_TYPE:-}" in +postgres) + export PGPORT=${DATABASE_PORT} + export PGPASSWORD=${DATABASE_PASSWORD} + psql_args=( ${DATABASE_HOST:+--host=${DATABASE_HOST}} "--username=${DATABASE_USER}" ) + ;; +*) + export DATABASE_TYPE=mysql + mysql_args=( ${DATABASE_HOST:+--host=${DATABASE_HOST}} ${DATABASE_PORT:+--port=${DATABASE_PORT}} "--user=${DATABASE_USER}" "--password=${DATABASE_PASSWORD}" ) + mysql_root_args=( ${DATABASE_HOST:+--host=${DATABASE_HOST}} ${DATABASE_PORT:+--port=${DATABASE_PORT}} "--user=root" "--password=${MYSQL_ROOT_PASSWORD:-}" ) + ;; +esac # functions test_db_connection() { - if [ ${PW_TEST_DB_TYPE} = "postgres" ]; then - echo ';' | psql -h $PW_TEST_DB_HOST -U postgres 2> /dev/null > /dev/null + if [ ${DATABASE_TYPE} = "postgres" ]; then + echo ';' | psql "${psql_args[@]}" 2> /dev/null > /dev/null else - mysqladmin -h $PW_TEST_DB_HOST -u patchwork --password=password ping > /dev/null 2> /dev/null + mysqladmin "${mysql_root_args[@]}" ping > /dev/null 2> /dev/null fi } test_database() { - if [ ${PW_TEST_DB_TYPE} = "postgres" ]; then - echo ';' | psql -h $PW_TEST_DB_HOST -U postgres patchwork 2> /dev/null + if [ ${DATABASE_TYPE} = "postgres" ]; then + echo ';' | psql "${psql_args[@]}" "${DATABASE_NAME}" 2> /dev/null else - echo ';' | mysql -h $PW_TEST_DB_HOST -u patchwork -ppassword patchwork 2> /dev/null + echo ';' | mysql "${mysql_args[@]}" "${DATABASE_NAME}" 2> /dev/null fi } reset_data_mysql() { - mysql -uroot -ppassword -h $PW_TEST_DB_HOST << EOF -DROP DATABASE IF EXISTS patchwork; -CREATE DATABASE patchwork CHARACTER SET utf8; -GRANT ALL ON patchwork.* TO 'patchwork' IDENTIFIED BY 'password'; -GRANT ALL ON \`test\\_patchwork%\`.* to 'patchwork'@'%'; + mysql "${mysql_root_args[@]}" << EOF +DROP DATABASE IF EXISTS ${DATABASE_NAME}; +CREATE DATABASE ${DATABASE_NAME} CHARACTER SET utf8; +GRANT ALL ON ${DATABASE_NAME}.* TO '${DATABASE_USER}' IDENTIFIED BY '${DATABASE_PASSWORD}'; +GRANT ALL ON \`test\\_${DATABASE_NAME}%\`.* to '${DATABASE_USER}'@'%'; FLUSH PRIVILEGES; EOF } reset_data_postgres() { - psql -h $PW_TEST_DB_HOST -U postgres < X-Patchwork-Id: 1564704 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=F4TZoGDJ; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=2404:9400:2:0:216:3eff:fee1:b9f1; helo=lists.ozlabs.org; envelope-from=patchwork-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=) Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2404:9400:2:0:216:3eff:fee1:b9f1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4J7glL3wzMz9s1l for ; Wed, 8 Dec 2021 00:15:26 +1100 (AEDT) Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4J7glL2tW1z3bd0 for ; Wed, 8 Dec 2021 00:15:26 +1100 (AEDT) Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=F4TZoGDJ; dkim-atps=neutral X-Original-To: patchwork@lists.ozlabs.org Delivered-To: patchwork@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gmail.com (client-ip=2607:f8b0:4864:20::1036; helo=mail-pj1-x1036.google.com; envelope-from=vicamo@gmail.com; receiver=) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=F4TZoGDJ; dkim-atps=neutral Received: from mail-pj1-x1036.google.com (mail-pj1-x1036.google.com [IPv6:2607:f8b0:4864:20::1036]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4J7gl35ZJSz2yb5 for ; Wed, 8 Dec 2021 00:15:11 +1100 (AEDT) Received: by mail-pj1-x1036.google.com with SMTP id cq22-20020a17090af99600b001a9550a17a5so1896189pjb.2 for ; Tue, 07 Dec 2021 05:15:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=D1yj2DLpb5lYhbt06AmRHq5JrsVE03nHxKDPSdnYCd0=; b=F4TZoGDJQb9RY3FMOUz1evqYDlVvnAUOIcat/PdTJSJM2Pm1DrMDUSVUkv1NrmwFDM Nxbg0U9T9D+C8OGjwKlRybcDS/zI8+t+uI1clVHq/CD2Px4Jje/v5ncL1jWT2vTBDWP8 sFh8QjrR8mhKGgYCwWIp0ZA8RVRfQKIfOSVDPcLHVFikSSB1Vec0fsCjlXGPvfAPZY7B csxpmQeUvnS6cKpCpa6L+AIRMlK4MjBWHIw45VolIa3QpT4Gr3lA+SHyItXX3mAFjCuv X5jm/oR2dnAoFK/Ob4hjRezSiFaLOj7xKbxBAet4xJ7k8dUGmBk9a3k3KAXgTUSkPDP7 lFLQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=D1yj2DLpb5lYhbt06AmRHq5JrsVE03nHxKDPSdnYCd0=; b=m3sXHeWz6sfzK350alxacF4t4od+FT/W3KxN1R+3DrfSk8p3MdaS5R9rdXJHbcHJz5 jWcrQO+QkNF9T42fzq2MLnz0h9nx7oQQIQmm+RHYp31tBPeX4ZeDmSK/xO6pDOkZ6fI2 SYjTELKB1k+GZzRHuIl82zfexkmeXXXeRHaeUvNOsCkZFdJ0kS5hC6o7pl4NPpmtbWIs CPUKdLriSsdtiNpcL7idsqVDuEkYb2Wvx3W6T2lCfLU9E/korGUuCC+lPcs04j5Awx+E OYPOKNLZRQwpn/S/F3/UenUZoAJYy77OfmD5bBzK7/C9R9nAO8kSdC9QokAmmf3UuDJA wOTQ== X-Gm-Message-State: AOAM533fUvIc11UXqDoMALSu7UaN66kUNoGs763A4E5Ey+mPiPH/0PI1 9Ddd7a8MCiStAxRNoPnCElVrphJ1aM4= X-Google-Smtp-Source: ABdhPJyHd6kLyJ5rE4y1vDNVwCdE6clPG73I5iy8gs5NlkSOzRrrK3g6mMDkGbZsYxFw2MQ+zGfhCw== X-Received: by 2002:a17:902:9694:b0:143:6f27:29ad with SMTP id n20-20020a170902969400b001436f2729admr51839680plp.46.1638882908949; Tue, 07 Dec 2021 05:15:08 -0800 (PST) Received: from localhost (218-173-141-247.dynamic-ip.hinet.net. [218.173.141.247]) by smtp.gmail.com with ESMTPSA id c15sm2954355pjg.53.2021.12.07.05.15.08 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Dec 2021 05:15:08 -0800 (PST) From: You-Sheng Yang To: patchwork@lists.ozlabs.org Subject: [PATCH v2 2/4] docker: drop database maintenance stuff Date: Tue, 7 Dec 2021 21:15:01 +0800 Message-Id: <20211207131503.416945-3-vicamo@gmail.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211207131503.416945-1-vicamo@gmail.com> References: <20211207131503.416945-1-vicamo@gmail.com> MIME-Version: 1.0 X-BeenThere: patchwork@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Patchwork development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: patchwork-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Patchwork" Trying to re-create database inside a client docker could be risky and error prone. The previous handling process may fail service startup when an empty database was previously created. `test_database()` will always succeed as it tests only the existence of the database, not any table inside. Without migrations being called on such empty database, patchwork cannot work normally. Checking the existence of a patchwork specific table is also invalid under the empty database scenario. And even the inspecting table does exist, migration process might be still necessary. As a result, we shall test if the database exist and do migration always. This change removes database maintenance functions and those marked deprecated long ago. Signed-off-by: You-Sheng Yang --- docker-compose-pg.yml | 5 ++- docker-compose.yml | 4 +- tools/docker/Dockerfile | 1 + tools/docker/entrypoint.sh | 79 +++++--------------------------------- 4 files changed, 16 insertions(+), 73 deletions(-) diff --git a/docker-compose-pg.yml b/docker-compose-pg.yml index 7398929..9129f32 100644 --- a/docker-compose-pg.yml +++ b/docker-compose-pg.yml @@ -5,6 +5,8 @@ services: volumes: - ./tools/docker/db/postdata:/var/lib/postgresql/data environment: + - POSTGRES_DB=patchwork + - POSTGRES_USER=patchwork - POSTGRES_PASSWORD=password web: @@ -16,7 +18,6 @@ services: - GID depends_on: - db - command: python3 manage.py runserver 0.0.0.0:8000 volumes: - .:/home/patchwork/patchwork/ ports: @@ -28,5 +29,5 @@ services: - DATABASE_HOST=db - DATABASE_PORT=5432 - DATABASE_NAME=patchwork - - DATABASE_USER=postgres + - DATABASE_USER=patchwork - DATABASE_PASSWORD=password diff --git a/docker-compose.yml b/docker-compose.yml index 103f19f..d824436 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -6,6 +6,7 @@ services: - ./tools/docker/db/data:/var/lib/mysql environment: - MYSQL_ROOT_PASSWORD=root + - MYSQL_DATABASE=patchwork - MYSQL_USER=patchwork - MYSQL_PASSWORD=password @@ -18,7 +19,6 @@ services: - GID depends_on: - db - command: python3 manage.py runserver 0.0.0.0:8000 volumes: - .:/home/patchwork/patchwork/ ports: @@ -29,6 +29,6 @@ services: # skip DATABASE_TYPE explicitly as mysql should be the default type. - DATABASE_HOST=db - DATABASE_PORT=3306 + - DATABASE_NAME=patchwork - DATABASE_USER=patchwork - DATABASE_PASSWORD=password - - MYSQL_ROOT_PASSWORD=root diff --git a/tools/docker/Dockerfile b/tools/docker/Dockerfile index fe0cd59..71167e0 100644 --- a/tools/docker/Dockerfile +++ b/tools/docker/Dockerfile @@ -61,5 +61,6 @@ RUN pip install -r /opt/requirements-dev.txt COPY tools/docker/entrypoint.sh /usr/local/bin/entrypoint.sh ENTRYPOINT ["/usr/local/bin/entrypoint.sh"] +CMD ["python3", "manage.py", "runserver", "0.0.0.0:8000"] USER patchwork WORKDIR /home/patchwork/patchwork diff --git a/tools/docker/entrypoint.sh b/tools/docker/entrypoint.sh index faa36b3..d1bad82 100755 --- a/tools/docker/entrypoint.sh +++ b/tools/docker/entrypoint.sh @@ -16,20 +16,11 @@ postgres) *) export DATABASE_TYPE=mysql mysql_args=( ${DATABASE_HOST:+--host=${DATABASE_HOST}} ${DATABASE_PORT:+--port=${DATABASE_PORT}} "--user=${DATABASE_USER}" "--password=${DATABASE_PASSWORD}" ) - mysql_root_args=( ${DATABASE_HOST:+--host=${DATABASE_HOST}} ${DATABASE_PORT:+--port=${DATABASE_PORT}} "--user=root" "--password=${MYSQL_ROOT_PASSWORD:-}" ) ;; esac # functions -test_db_connection() { - if [ ${DATABASE_TYPE} = "postgres" ]; then - echo ';' | psql "${psql_args[@]}" 2> /dev/null > /dev/null - else - mysqladmin "${mysql_root_args[@]}" ping > /dev/null 2> /dev/null - fi -} - test_database() { if [ ${DATABASE_TYPE} = "postgres" ]; then echo ';' | psql "${psql_args[@]}" "${DATABASE_NAME}" 2> /dev/null @@ -38,37 +29,6 @@ test_database() { fi } -reset_data_mysql() { - mysql "${mysql_root_args[@]}" << EOF -DROP DATABASE IF EXISTS ${DATABASE_NAME}; -CREATE DATABASE ${DATABASE_NAME} CHARACTER SET utf8; -GRANT ALL ON ${DATABASE_NAME}.* TO '${DATABASE_USER}' IDENTIFIED BY '${DATABASE_PASSWORD}'; -GRANT ALL ON \`test\\_${DATABASE_NAME}%\`.* to '${DATABASE_USER}'@'%'; -FLUSH PRIVILEGES; -EOF -} - -reset_data_postgres() { - psql "${psql_args[@]}" < /dev/null - python manage.py loaddata default_tags #> /dev/null - python manage.py loaddata default_states #> /dev/null - python manage.py loaddata default_projects #> /dev/null -} - # the script begins! # check if patchwork is mounted. Checking if we exist is a @@ -103,20 +63,20 @@ done set -e # check if db is connected -if ! test_db_connection; then +if ! test_database; then echo "The database seems not to be connected, or the ${DATABASE_USER} user is broken" echo "MySQL/Postgres may still be starting. Waiting 5 seconds." sleep 5 - if ! test_db_connection; then + if ! test_database; then echo "Still cannot connect to database." echo "Maybe you are starting the db for the first time. Waiting up to 60 seconds." for i in {0..9}; do sleep 5 - if test_db_connection; then + if test_database; then break fi done - if ! test_db_connection; then + if ! test_database; then echo "Still cannot connect to database. Giving up." echo "Are you using docker-compose? If not, have you set up the link correctly?" exit 1 @@ -124,29 +84,10 @@ if ! test_db_connection; then fi fi -# rebuild db -# do this on --reset or if the db doesn't exist -if [[ "$1" == "--reset" ]]; then - shift - reset_data -elif ! test_database; then - reset_data -fi +# load initial data +python manage.py migrate #> /dev/null +python manage.py loaddata default_tags #> /dev/null +python manage.py loaddata default_states #> /dev/null +python manage.py loaddata default_projects #> /dev/null -# TODO(stephenfin): Deprecated the --test, --tox, --quick-test and --quick-tox -# flags in a future release -if [ $# -eq 0 ]; then - # we probably ran with --reset and nothing else - # just exit cleanly - exit 0 -elif [ "$1" == "--shell" ]; then - exec bash -elif [ "$1" == "--test" ] || [ "$1" == "--quick-test" ]; then - shift - python manage.py test $@ -elif [ "$1" == "--tox" ] || [ "$1" == "--quick-tox" ]; then - shift - tox $@ -else # run whatever CMD is set to - $@ -fi +exec "$@" From patchwork Tue Dec 7 13:15:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: You-Sheng Yang X-Patchwork-Id: 1564705 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=nOL7V0yQ; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=2404:9400:2:0:216:3eff:fee1:b9f1; helo=lists.ozlabs.org; envelope-from=patchwork-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=) Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2404:9400:2:0:216:3eff:fee1:b9f1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4J7glV41Rfz9s1l for ; Wed, 8 Dec 2021 00:15:34 +1100 (AEDT) Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4J7glV2cfqz2yg4 for ; Wed, 8 Dec 2021 00:15:34 +1100 (AEDT) Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=nOL7V0yQ; dkim-atps=neutral X-Original-To: patchwork@lists.ozlabs.org Delivered-To: patchwork@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gmail.com (client-ip=2607:f8b0:4864:20::530; helo=mail-pg1-x530.google.com; envelope-from=vicamo@gmail.com; receiver=) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=nOL7V0yQ; dkim-atps=neutral Received: from mail-pg1-x530.google.com (mail-pg1-x530.google.com [IPv6:2607:f8b0:4864:20::530]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4J7gl51FxRz2ync for ; Wed, 8 Dec 2021 00:15:13 +1100 (AEDT) Received: by mail-pg1-x530.google.com with SMTP id f125so13869596pgc.0 for ; Tue, 07 Dec 2021 05:15:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=B/eTTAjqF7eLigR7XIh6c0o32E9kOLUMLN1H9XH3ybY=; b=nOL7V0yQOyizHOxMRaOyYYhxy6+nGkqMxd5071QO3zAtVlVYgG9+5OOSB5niD/Qfhb EYjJpEE3CyKU+Nsk21jSaiyP5rQVa3F2WuexYBaIM3N8Pc1qUI49Q+hzWpdreBKK9b2E +DGpr1sf6WBi0s5rQcZpmOA4gohYhCUbVpdmTL+grSozwndEo1H/wW4dLUvWj+uORIyF wGNWUdXSWOXRaTG2ZW3V4epHsaiEPOkT/r+a6VElixYVntKS1Q2OSHH9HuLc2Vl5ROeL BqQHi0SixbqfxzUX3ja6WJEX2zyc9+ZOuD/SJ5bPy+4pe5XK8jdP7pBtJI4Wcp2BYABS e6LA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=B/eTTAjqF7eLigR7XIh6c0o32E9kOLUMLN1H9XH3ybY=; b=htBxlvK4iQsmzqtP+55cmGkbysOvoCT+EIAbAdplGMiV9zooYj8Vmbv+sx7+DP/7SQ dM55QXYR08x99aCORD7v+Iq1GO1sdYRgpwr+RKUCUbcE5dJuWszgupoqZ5CRNbk9JuOA VKW9fYWxBV/L3SpFHURFzCROnGeVjXkb2VKqmFh3rBKGT7fEbEPLse1EXtdOKo+eXf0K YNkwUfO5PiZtxEOB+XBvHr1phLP1TDjg/CmKVIhTdy76RCplwA5eGcGDiUQbwvlEez9m v92EBZoHt50b74rmzKTIdyXzto0Fkl+RS1F7JUB+e8SRAGyfKtuGUtwpVOBtqBd8DEyG FzEA== X-Gm-Message-State: AOAM533igD0F+qLb0CyDw/BmEdR4KowZhFtuVnz65Eu0lj2dWaHXb4GD lsYraUK/51odCVVHgEg0fJAlZcMOUeg= X-Google-Smtp-Source: ABdhPJxP3Om2+PAjDQBNa6qn6RxJAXpRVs11qnyjMW95cjHfihFbEjReKzPuP+ktpx9CbwleQeaQ4A== X-Received: by 2002:a63:703:: with SMTP id 3mr6927717pgh.128.1638882910609; Tue, 07 Dec 2021 05:15:10 -0800 (PST) Received: from localhost (218-173-141-247.dynamic-ip.hinet.net. [218.173.141.247]) by smtp.gmail.com with ESMTPSA id p15sm2771474pjh.1.2021.12.07.05.15.10 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Dec 2021 05:15:10 -0800 (PST) From: You-Sheng Yang To: patchwork@lists.ozlabs.org Subject: [PATCH v2 3/4] ci: add tests for sqlite3 Date: Tue, 7 Dec 2021 21:15:02 +0800 Message-Id: <20211207131503.416945-4-vicamo@gmail.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211207131503.416945-1-vicamo@gmail.com> References: <20211207131503.416945-1-vicamo@gmail.com> MIME-Version: 1.0 X-BeenThere: patchwork@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Patchwork development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: patchwork-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Patchwork" Signed-off-by: You-Sheng Yang --- .github/workflows/ci.yaml | 8 ++++---- docker-compose-sqlite3.yml | 19 +++++++++++++++++++ tools/docker/Dockerfile | 1 + tools/docker/entrypoint.sh | 19 +++++++++++++------ 4 files changed, 37 insertions(+), 10 deletions(-) create mode 100644 docker-compose-sqlite3.yml diff --git a/.github/workflows/ci.yaml b/.github/workflows/ci.yaml index d37cdbf..00c2ecf 100644 --- a/.github/workflows/ci.yaml +++ b/.github/workflows/ci.yaml @@ -24,11 +24,11 @@ jobs: strategy: matrix: python: [3.6, 3.7, 3.8, 3.9] - db: [postgres, mysql] + db: [postgres, mysql, sqlite3] env: DATABASE_TYPE: "${{ matrix.db }}" DATABASE_HOST: "127.0.0.1" - DATABASE_NAME: patchwork + DATABASE_NAME: ${{ matrix.db != 'sqlite3' && 'patchwork' || '/dev/shm/patchwork.test.db.sqlite3' }} DATABASE_USER: patchwork DATABASE_PASSWORD: password MYSQL_ROOT_PASSWORD: root @@ -115,7 +115,7 @@ jobs: runs-on: ubuntu-latest strategy: matrix: - db: [postgres, mysql] + db: [postgres, mysql, sqlite3] steps: - name: Checkout source code uses: actions/checkout@v2 @@ -127,7 +127,7 @@ jobs: python-version: 3.9 - name: Bring up docker-compose service env: - COMPOSE_FILE: ${{ matrix.db == 'mysql' && 'docker-compose.yml' || 'docker-compose-pg.yml' }} + COMPOSE_FILE: ${{ matrix.db == 'mysql' && 'docker-compose.yml' || (matrix.db == 'postgres' && 'docker-compose-pg.yml') || 'docker-compose-sqlite3.yml' }} run: | docker-compose build --build-arg UID=$(id -u) --build-arg GID=$(id -g) docker-compose up --detach diff --git a/docker-compose-sqlite3.yml b/docker-compose-sqlite3.yml new file mode 100644 index 0000000..d4c6659 --- /dev/null +++ b/docker-compose-sqlite3.yml @@ -0,0 +1,19 @@ +version: "3" +services: + web: + build: + context: . + dockerfile: ./tools/docker/Dockerfile + args: + - UID + - GID + command: python3 manage.py runserver 0.0.0.0:8000 + volumes: + - .:/home/patchwork/patchwork/ + ports: + - "8000:8000" + environment: + - UID + - GID + - DATABASE_TYPE=sqlite3 + - DATABASE_NAME=/home/patchwork/patchwork/tools/docker/db/db.sqlite3 diff --git a/tools/docker/Dockerfile b/tools/docker/Dockerfile index 71167e0..d9d0905 100644 --- a/tools/docker/Dockerfile +++ b/tools/docker/Dockerfile @@ -38,6 +38,7 @@ RUN apt-get update && apt-get install -y --no-install-recommends \ libssl-dev \ mysql-client \ postgresql-client \ + sqlite3 \ tzdata \ zlib1g-dev \ && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* diff --git a/tools/docker/entrypoint.sh b/tools/docker/entrypoint.sh index d1bad82..96a59ed 100755 --- a/tools/docker/entrypoint.sh +++ b/tools/docker/entrypoint.sh @@ -3,18 +3,22 @@ set -euo pipefail export DATABASE_HOST=${DATABASE_HOST:-} export DATABASE_PORT=${DATABASE_PORT:-} -export DATABASE_NAME=${DATABASE_NAME:-patchwork} export DATABASE_USER=${DATABASE_USER:-patchwork} export DATABASE_PASSWORD=${DATABASE_PASSWORD:-password} case "${DATABASE_TYPE:-}" in postgres) + export DATABASE_NAME=${DATABASE_NAME:-patchwork} export PGPORT=${DATABASE_PORT} export PGPASSWORD=${DATABASE_PASSWORD} psql_args=( ${DATABASE_HOST:+--host=${DATABASE_HOST}} "--username=${DATABASE_USER}" ) ;; +sqlite3) + export DATABASE_NAME=${DATABASE_NAME:-} + ;; *) export DATABASE_TYPE=mysql + export DATABASE_NAME=${DATABASE_NAME:-patchwork} mysql_args=( ${DATABASE_HOST:+--host=${DATABASE_HOST}} ${DATABASE_PORT:+--port=${DATABASE_PORT}} "--user=${DATABASE_USER}" "--password=${DATABASE_PASSWORD}" ) ;; esac @@ -22,11 +26,14 @@ esac # functions test_database() { - if [ ${DATABASE_TYPE} = "postgres" ]; then - echo ';' | psql "${psql_args[@]}" "${DATABASE_NAME}" 2> /dev/null - else - echo ';' | mysql "${mysql_args[@]}" "${DATABASE_NAME}" 2> /dev/null - fi + case "${DATABASE_TYPE}" in + "postgres") + echo ';' | psql "${psql_args[@]}" "${DATABASE_NAME}" 2> /dev/null ;; + "mysql") + echo ';' | mysql "${mysql_args[@]}" "${DATABASE_NAME}" 2> /dev/null ;; + "sqlite3") + echo ';' | sqlite3 "${DATABASE_NAME}" > /dev/null 2> /dev/null ;; + esac } # the script begins! From patchwork Tue Dec 7 13:15:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: You-Sheng Yang X-Patchwork-Id: 1564706 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=RMStb9vf; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=112.213.38.117; helo=lists.ozlabs.org; envelope-from=patchwork-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=) Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4J7gld4dKJz9s1l for ; Wed, 8 Dec 2021 00:15:41 +1100 (AEDT) Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4J7gld3VC7z2ypX for ; Wed, 8 Dec 2021 00:15:41 +1100 (AEDT) Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=RMStb9vf; dkim-atps=neutral X-Original-To: patchwork@lists.ozlabs.org Delivered-To: patchwork@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gmail.com (client-ip=2607:f8b0:4864:20::1030; helo=mail-pj1-x1030.google.com; envelope-from=vicamo@gmail.com; receiver=) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=RMStb9vf; dkim-atps=neutral Received: from mail-pj1-x1030.google.com (mail-pj1-x1030.google.com [IPv6:2607:f8b0:4864:20::1030]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4J7gl70qSLz2yPD for ; Wed, 8 Dec 2021 00:15:15 +1100 (AEDT) Received: by mail-pj1-x1030.google.com with SMTP id nh10-20020a17090b364a00b001a69adad5ebso2667444pjb.2 for ; Tue, 07 Dec 2021 05:15:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=OzE6zUPXyUYmO9mBfLlqDk40tnJKpU4OzsA7sr1Uj3E=; b=RMStb9vfahpG1aXGSx77ztC8mRQTAASB/KKIyhMV0jz15mkwpHNJv6izAE1Z4ABLxx 1AZBqIfh3wlHHmUPyAxu1MgytvhEUwBOFDTpQkTnENVGtRtyax3jBKdijKsOhFFodvij 0nDsL0eiV3mozUpQNtGccosbStx7rDxtLnBGnvYbifFxKvjFRZ8CGMAfQAbmbX1wv2ww yejbpVwpwtgRki4kRQoK4ihc9IhgT5R7lEw44WBF0nCm4jqDx8HwIqw4BK+9cMzC7BkA ayGGKaURvE1H/pizyBkLCvPAndGmjqzAVNCyoqu1fX/DCssD19bcAdw+H6IhOjfPUXnU KsIQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=OzE6zUPXyUYmO9mBfLlqDk40tnJKpU4OzsA7sr1Uj3E=; b=OrF0Ue4KN+MYz4WpArwE/4qcH1QX5k2fj/+REcSEG98hKteOYSi0L7Ge/XzlTW8sh/ pVEj/LGBR0sQzoZuIcyTCih9NyP9CtTDFrfjYwS5D+hQyM/1ljQ0z4NR56z8uPVK5M46 KhXefr7I0VbgbHd7D9E7hlxeUOm3xmDsUlOzgVQAbU4CoipNcLG20tSA06lr/OS48sYu Wzbe8KVtJelgvtFghSxlhYNqHZZ/jiQpOG+h06jv4bk9aS0/NYsW8I3qpONO69pMXPV5 /DhdPnjA/Vjkg5o9/DpWFm+U1fiNLWjqow+t4qYXOPQZdYKuSB1FiEA/VNLFIt3S1utE 3SNA== X-Gm-Message-State: AOAM531DyJUJBSwf7f0dRRGvjTW7zo5i0YYuPHSXWe3BXfICM9P28VdT 3GFw+cbfylpI2B0KOFNvOuwYVqRVtc0= X-Google-Smtp-Source: ABdhPJyrTkeq9HnZTjiR74njvx951gnHC1qQ5+DvC3MUBorTt1J2UgHWLYzmT2ceBvdAkUEe638q2w== X-Received: by 2002:a17:90b:4f45:: with SMTP id pj5mr6629022pjb.70.1638882912745; Tue, 07 Dec 2021 05:15:12 -0800 (PST) Received: from localhost (218-173-141-247.dynamic-ip.hinet.net. [218.173.141.247]) by smtp.gmail.com with ESMTPSA id mu4sm3066256pjb.8.2021.12.07.05.15.12 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Dec 2021 05:15:12 -0800 (PST) From: You-Sheng Yang To: patchwork@lists.ozlabs.org Subject: [PATCH v2 4/4] ci: randomize database credentials a bit Date: Tue, 7 Dec 2021 21:15:03 +0800 Message-Id: <20211207131503.416945-5-vicamo@gmail.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211207131503.416945-1-vicamo@gmail.com> References: <20211207131503.416945-1-vicamo@gmail.com> MIME-Version: 1.0 X-BeenThere: patchwork@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Patchwork development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: patchwork-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Patchwork" Signed-off-by: You-Sheng Yang --- .github/workflows/ci.yaml | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/.github/workflows/ci.yaml b/.github/workflows/ci.yaml index 00c2ecf..2d4a45f 100644 --- a/.github/workflows/ci.yaml +++ b/.github/workflows/ci.yaml @@ -28,10 +28,10 @@ jobs: env: DATABASE_TYPE: "${{ matrix.db }}" DATABASE_HOST: "127.0.0.1" - DATABASE_NAME: ${{ matrix.db != 'sqlite3' && 'patchwork' || '/dev/shm/patchwork.test.db.sqlite3' }} - DATABASE_USER: patchwork - DATABASE_PASSWORD: password - MYSQL_ROOT_PASSWORD: root + DATABASE_NAME: ${{ matrix.db != 'sqlite3' && format('patchwork-db-{0}', github.run_id) || '/dev/shm/patchwork.test.db.sqlite3' }} + DATABASE_USER: patchwork-user-${{ github.run_id }} + DATABASE_PASSWORD: password-${{ github.run_id }} + MYSQL_ROOT_PASSWORD: root-${{ github.run_id }} services: postgres: image: postgres:latest