From patchwork Fri Dec 9 14:10:02 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Petr Vorel X-Patchwork-Id: 1714249 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.linux.it (client-ip=213.254.12.146; helo=picard.linux.it; envelope-from=ltp-bounces+incoming=patchwork.ozlabs.org@lists.linux.it; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=suse.cz header.i=@suse.cz header.a=rsa-sha256 header.s=susede2_rsa header.b=gzI8Kkkq; dkim=fail reason="signature verification failed" header.d=suse.cz header.i=@suse.cz header.a=ed25519-sha256 header.s=susede2_ed25519 header.b=PAkhuMjL; dkim-atps=neutral Received: from picard.linux.it (picard.linux.it [213.254.12.146]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4NTCbJ2X83z23ns for ; Sat, 10 Dec 2022 01:10:20 +1100 (AEDT) Received: from picard.linux.it (localhost [IPv6:::1]) by picard.linux.it (Postfix) with ESMTP id 6BDE93CBFD0 for ; Fri, 9 Dec 2022 15:10:18 +0100 (CET) X-Original-To: ltp@lists.linux.it Delivered-To: ltp@picard.linux.it Received: from in-3.smtp.seeweb.it (in-3.smtp.seeweb.it [217.194.8.3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by picard.linux.it (Postfix) with ESMTPS id 3FE913CBFA6 for ; Fri, 9 Dec 2022 15:10:17 +0100 (CET) Received: from smtp-out1.suse.de (smtp-out1.suse.de [IPv6:2001:67c:2178:6::1c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by in-3.smtp.seeweb.it (Postfix) with ESMTPS id 7B5FF1A00CBD for ; Fri, 9 Dec 2022 15:10:14 +0100 (CET) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id ED34F337AE; Fri, 9 Dec 2022 14:10:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1670595012; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding; bh=lb6vrLQBJMNx5rnsXRn+a68LpAd9OXcH4J+VMxXOtl8=; b=gzI8Kkkq8mrn0v+a8ltgrBvHE3G4zz410fceDAWC9qhF63hglW47AbEf8z8qejlEVmMZRw n4ekPwtvBrGB5ZmLC4RuEBGIYWVNvb7dRTUsBh2B/t7uBjSO/bKMMoXTIXxsTFT3DssIoX pNJwLGl3I14CVJKP0CRSJjJxHPn3m0s= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1670595012; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding; bh=lb6vrLQBJMNx5rnsXRn+a68LpAd9OXcH4J+VMxXOtl8=; b=PAkhuMjLoXYHY0XOxzgGF8g4Fy5mj69csZ7UPvjLU3+DlYUq+iiZPMVzapW2ORplJYHqnR kZBqmsbCtduS3dBQ== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id B9443138E0; Fri, 9 Dec 2022 14:10:12 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id hDL/K8RBk2PmBAAAMHmgww (envelope-from ); Fri, 09 Dec 2022 14:10:12 +0000 From: Petr Vorel To: ltp@lists.linux.it Date: Fri, 9 Dec 2022 15:10:02 +0100 Message-Id: <20221209141002.15551-1-pvorel@suse.cz> X-Mailer: git-send-email 2.38.1 MIME-Version: 1.0 X-Virus-Scanned: clamav-milter 0.102.4 at in-3.smtp.seeweb.it X-Virus-Status: Clean X-Spam-Status: No, score=0.1 required=7.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE,SPF_SOFTFAIL autolearn=disabled version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on in-3.smtp.seeweb.it Subject: [LTP] [PATCH v2 1/1] ci: Add hook to mirror docparse to homepage X-BeenThere: ltp@lists.linux.it X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux Test Project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Aleks L , Richard Palethorpe Errors-To: ltp-bounces+incoming=patchwork.ozlabs.org@lists.linux.it Sender: "ltp" GitHub Actions git push hook generates metadata HTML and push it to LTP homepage. Hook pushes only if there are actual changes in generated doc. NOTE: this change requires to add: 1) Personal Access Token (PAT) to any developer which has write access to homepage git repository [3]. In Developer settings -> Personal access tokens -> Tokens (classic) [4]), where set: Note: GH_PERSONAL_ACCESS_TOKEN Select scopes: public_repo (minimal permission) Expiration: either never or regularly renew. 2) Allow PAT in LTP organisation (I dared to already set it) Iin linux-test-project group -> Settings -> Third-party Access -> Personal access tokens -> Settings [5] select: Allow access via personal access tokens (classic) API and Git access will be allowed using an organization member's personal access token (classic) 3) Add repository action secret to ltp repository IN Settings -> Actions -> New repository secret [6]: name: GH_PERSONAL_ACCESS_TOKEN value: the value of previously created token. Because using token, default permission is just read. Reviewed-by: Aleks L Signed-off-by: Petr Vorel --- Changes v1->v2: * permission write => read (the real permission is done with token) * remove unused env Testing: https://github.com/foo-pevik/ltp_foo/actions/runs/3656966820 https://github.com/foo-pevik/linux-test-project.github.com_foo/commit/785150ba13a9934d6ce7325abecd05e3643a00ed No generation with irrelevant change: https://github.com/foo-pevik/ltp_foo/actions/runs/3656968653 .github/workflows/metadata-mirror.yml | 139 ++++++++++++++++++++++++++ 1 file changed, 139 insertions(+) create mode 100644 .github/workflows/metadata-mirror.yml diff --git a/.github/workflows/metadata-mirror.yml b/.github/workflows/metadata-mirror.yml new file mode 100644 index 000000000..296b3286c --- /dev/null +++ b/.github/workflows/metadata-mirror.yml @@ -0,0 +1,139 @@ +# SPDX-License-Identifier: GPL-2.0-or-later +# Copyright (c) 2022 Petr Vorel + +name: "Mirror metadata doc to homepage" + +on: + push: + branches: + - master + +permissions: {} +jobs: + metadata-mirror: + permissions: + contents: read + + runs-on: ubuntu-latest + if: ${{ github.repository == 'linux-test-project/ltp' }} + steps: + - name: Check secret + env: + GH_PERSONAL_ACCESS_TOKEN: ${{ secrets.GH_PERSONAL_ACCESS_TOKEN }} + run: | + if [ -z "$GH_PERSONAL_ACCESS_TOKEN" ]; then + echo "::error::GH_PERSONAL_ACCESS_TOKEN environment variable is not set" + exit 1 + fi + + - name: Checkout LTP + uses: actions/checkout@v3 + with: + path: ltp + # we need to fetch whole history to get 'git describe' working for correct version in docs + fetch-depth: 0 + + - name: Checkout LTP homepage + uses: actions/checkout@v3 + with: + repository: "linux-test-project/linux-test-project.github.com" + path: linux-test-project.github.com + persist-credentials: false + + - name: Check metadata need to be updated + run: | + cd "$GITHUB_WORKSPACE/ltp/testcases/" + + # check for changes + cd "$GITHUB_WORKSPACE/linux-test-project.github.com" + git grep '

Version:' metadata/metadata.nightly.html + sed -ne 's/.*

Version<\/strong>: \(.*\)<\/p>/\1/p' metadata/metadata.nightly.html + old_commit=$(sed -ne 's/.*

Version<\/strong>: \(.*\)<\/p>/\1/p' metadata/metadata.nightly.html) + old_commit=$(echo "$old_commit" | sed 's/.*-g\(.*\)/\1/') + echo "::notice::old commit in linux-test-project.github.com: '$old_commit'" + + echo "git log $old_commit..HEAD testcases/ | wc -l" + git log $old_commit..HEAD testcases/ | wc -l + + echo "git log $old_commit..HEAD testcases/ 2>&1 | wc -l" + git log $old_commit..HEAD testcases/ 2>&1 | wc -l + + if [ "$(git log $old_commit..HEAD testcases/ 2>&1 | wc -l)" = 0 ]; then + echo "::notice::no file changes in testcases/ directory since '$old_commit' => exit without commit and push" + exit 0 + fi + + - name: Install dependencies + run: | + apt="apt install -y --no-install-recommends" + sudo $apt asciidoctor autoconf automake libjson-perl libwww-perl make + + echo "which asciidoctor" + which asciidoctor || { echo "::error::missing asciidoctor"; exit 1; } + + - name: Configure LTP + run: | + cd "$GITHUB_WORKSPACE/ltp/" + make autotools && ./configure --with-metadata-generator=asciidoctor && make Version || { echo "::error::LTP configure failed"; exit 1; } + + - name: Generate html metadata doc + run: | + cd "$GITHUB_WORKSPACE/ltp/metadata/" + echo "going to generate metadata" + make -j$(nproc) + + - name: Check possible LTP homepage update + run: | + cd "$GITHUB_WORKSPACE/ltp/" + ltp_commit=$(git log --pretty=format:"%h" -1 .) + echo "::notice::Search for $ltp_commit in metadata/metadata.nightly.html" + + cd "$GITHUB_WORKSPACE/linux-test-project.github.com" + grep "

Version:" metadata/metadata.nightly.html + git pull + grep "

Version:" metadata/metadata.nightly.html + if grep "

Version: .*${ltp_commit}.*

" metadata/metadata.nightly.html; then + echo "::notice::$ltp_commit already in metadata/metadata.nightly.html => exit without commit and push" + exit 0 + fi + + - name: Push generated html metadata to LTP homepage + env: + GH_PERSONAL_ACCESS_TOKEN: ${{ secrets.GH_PERSONAL_ACCESS_TOKEN }} + run: | + git config --global user.email "actions@github.com" + git config --global user.name "Metadata doc mirror" + + cd "$GITHUB_WORKSPACE/ltp/" + commit=$(git log --pretty=format:"%h" -1 .) + commit_desc=$(git log --pretty=format:"%h (\"%s\")" -1 .) + echo "commit_desc: '$commit_desc'" + + cd "$GITHUB_WORKSPACE/linux-test-project.github.com" + + # pull in case previous hook pushed later than checkout here had started + git pull + + cp -v "$GITHUB_WORKSPACE/ltp/docparse/metadata.html" metadata/metadata.nightly.html + + # skip if there is only different version and time + echo "Check for changes" + if ! git diff --unified=0 --no-color | grep -Po '(?<=^\+)(?!\+\+).*' | grep -v -e '^

Version:' -e '^Last updated'; then + echo "::notice::No changes in metadata/metadata.nightly.html => exit without commit and push" + exit 0 + fi + + git add . + + # only commit if there are changes + if git diff-index --quiet HEAD --; then + echo "::notice::No files changed in $GITHUB_WORKSPACE/linux-test-project.github.com => exit without commit and push" + git status + exit 0 + fi + + printf "metadata.nightly.html: Update to $commit\n\nUpdate metadata.nightly.html to $commit_desc\n" > /tmp/msg + git commit -F /tmp/msg . + + echo "::notice::GH_PERSONAL_ACCESS_TOKEN: $GH_PERSONAL_ACCESS_TOKEN" + git push https://${GH_PERSONAL_ACCESS_TOKEN}@github.com/linux-test-project/linux-test-project.github.com.git