diff mbox series

[1/1] ci: Add hook to mirror docparse to homepage

Message ID 20221209100359.6275-1-pvorel@suse.cz
State Changes Requested
Headers show
Series [1/1] ci: Add hook to mirror docparse to homepage | expand

Commit Message

Petr Vorel Dec. 9, 2022, 10:03 a.m. UTC
GitHub Actions git push hook generates metadata HTML and push it wo LTP
homepage.

Hook pushes only if there are actual changes in generated doc.

NOTE: This requires to add Personal access tokens (classic)
named as GH_PERSONAL_ACCESS_TOKEN with permissions to public_repo.

Signed-off-by: Petr Vorel <pvorel@suse.cz>
---
Hi,

automatic updates of our docparse documentation to our homepage [1]
(linked in "nightly test documentation" [2]).

This does push to our homepage git repository [3].

This patchset required to add:

1) Personal Access Token (PAT) to any developer which has write access
to homepage git repository [3]. In Developer settings -> Personal access
tokens -> Tokens (classic) [4]), where set:
Note: GH_PERSONAL_ACCESS_TOKEN
Select scopes: public_repo (minimal permission)
Expiration: either never or regularly renew.

2) Allow PAT in LTP organisation (I dared to already set it)
Iin linux-test-project group -> Settings -> Third-party Access -> Personal
access tokens -> Settings [5]
select:
Allow access via personal access tokens (classic)
API and Git access will be allowed using an organization member's personal access token (classic)

3) Add repository action secret to ltp repository
IN Settings -> Actions -> New repository secret [6]:
name: GH_PERSONAL_ACCESS_TOKEN
value: the value of previously created token.

I haven't tested fine-grained personal access tokens, maybe they are
safer. Using proper GitHub App would be safer (would not include PAT),
but I have spent already quite a lot of time with this, thus unless
anybody objects I'd merge this way.

I did example of this run on organisation which I created just for
testing purposes:

* working example
action run [7], resulted in commit [8]. Example of unrelated commit [9]
which does not trigger doc generation [10].

* example of error, when token not set
https://github.com/foo-pevik/ltp_foo/actions/runs/3650567244
GH_PERSONAL_ACCESS_TOKEN environment variable is not set

* example of invalid GH_PERSONAL_ACCESS_TOKEN
https://github.com/foo-pevik/ltp_foo/actions/runs/3650579488/jobs/6166734733
Notice: GH_PERSONAL_ACCESS_TOKEN: ***
fatal: could not read Password for 'https://***@github.com': No such device or address
Error: Process completed with exit code 128.

Credits:
* Inspiration from SwiftDocOrg [11], pointed out by Cyril
* Thanks a lot to Aleks for hint with "persist-credentials: false".

Kind regards,
Petr

[1] http://linux-test-project.github.io/
[2] http://linux-test-project.github.io/metadata/metadata.nightly.html
[3] https://github.com/linux-test-project/linux-test-project.github.com
[4] https://github.com/settings/tokens/new
[5] https://github.com/organizations/linux-test-project/settings/personal-access-tokens
[6] https://github.com/foo-pevik/ltp_foo/settings/secrets/actions/new
[7] https://github.com/foo-pevik/ltp_foo/actions/runs/3651475124
[8] https://github.com/foo-pevik/linux-test-project.github.com_foo/commit/c2b94862da2c5741f08282988dcc0302e0847a69
[9] https://github.com/foo-pevik/ltp_foo/commit/36a66a2f304b449e1c17d72bad1b1db46a30f36f
[10] https://github.com/foo-pevik/ltp_foo/actions/runs/3651475552
[11] https://github.com/SwiftDocOrg/github-wiki-publish-action

 .github/workflows/metadata-mirror.yml | 141 ++++++++++++++++++++++++++
 1 file changed, 141 insertions(+)
 create mode 100644 .github/workflows/metadata-mirror.yml

Comments

Aleks L Dec. 9, 2022, 11:05 a.m. UTC | #1
Hi Peter, two comments:

+  metadata-mirror:
+    permissions:
+      contents: write # for git push

Since you are using GH_PERSONAL_ACCESS_TOKEN and not the default
GITHUB_TOKEN for push, I think "contents: read" should be sufficient. Could
you please try it?

+      - name: Checkout LTP homepage
+        uses: actions/checkout@v3
+        with:
+          repository: "linux-test-project/linux-test-project.github.com"
+          path: linux-test-project.github.com
+          persist-credentials: false
+        env:
+          GH_PERSONAL_ACCESS_TOKEN: ${{ secrets.GH_PERSONAL_ACCESS_TOKEN }}

I think the GH_PERSONAL_ACCESS_TOKEN env variable here is not used and can
be deleted. The checkout uses the standard GITHUB_TOKEN with read access to
public repos.

Otherwise LGTM

On Fri, Dec 9, 2022 at 11:04 AM Petr Vorel <pvorel@suse.cz> wrote:

> GitHub Actions git push hook generates metadata HTML and push it wo LTP
> homepage.
>
> Hook pushes only if there are actual changes in generated doc.
>
> NOTE: This requires to add Personal access tokens (classic)
> named as GH_PERSONAL_ACCESS_TOKEN with permissions to public_repo.
>
> Signed-off-by: Petr Vorel <pvorel@suse.cz>
> ---
> Hi,
>
> automatic updates of our docparse documentation to our homepage [1]
> (linked in "nightly test documentation" [2]).
>
> This does push to our homepage git repository [3].
>
> This patchset required to add:
>
> 1) Personal Access Token (PAT) to any developer which has write access
> to homepage git repository [3]. In Developer settings -> Personal access
> tokens -> Tokens (classic) [4]), where set:
> Note: GH_PERSONAL_ACCESS_TOKEN
> Select scopes: public_repo (minimal permission)
> Expiration: either never or regularly renew.
>
> 2) Allow PAT in LTP organisation (I dared to already set it)
> Iin linux-test-project group -> Settings -> Third-party Access -> Personal
> access tokens -> Settings [5]
> select:
> Allow access via personal access tokens (classic)
> API and Git access will be allowed using an organization member's personal
> access token (classic)
>
> 3) Add repository action secret to ltp repository
> IN Settings -> Actions -> New repository secret [6]:
> name: GH_PERSONAL_ACCESS_TOKEN
> value: the value of previously created token.
>
> I haven't tested fine-grained personal access tokens, maybe they are
> safer. Using proper GitHub App would be safer (would not include PAT),
> but I have spent already quite a lot of time with this, thus unless
> anybody objects I'd merge this way.
>
> I did example of this run on organisation which I created just for
> testing purposes:
>
> * working example
> action run [7], resulted in commit [8]. Example of unrelated commit [9]
> which does not trigger doc generation [10].
>
> * example of error, when token not set
> https://github.com/foo-pevik/ltp_foo/actions/runs/3650567244
> GH_PERSONAL_ACCESS_TOKEN environment variable is not set
>
> * example of invalid GH_PERSONAL_ACCESS_TOKEN
>
> https://github.com/foo-pevik/ltp_foo/actions/runs/3650579488/jobs/6166734733
> Notice: GH_PERSONAL_ACCESS_TOKEN: ***
> fatal: could not read Password for 'https://***@github.com': No such
> device or address
> Error: Process completed with exit code 128.
>
> Credits:
> * Inspiration from SwiftDocOrg [11], pointed out by Cyril
> * Thanks a lot to Aleks for hint with "persist-credentials: false".
>
> Kind regards,
> Petr
>
> [1] http://linux-test-project.github.io/
> [2] http://linux-test-project.github.io/metadata/metadata.nightly.html
> [3] https://github.com/linux-test-project/linux-test-project.github.com
> [4] https://github.com/settings/tokens/new
> [5]
> https://github.com/organizations/linux-test-project/settings/personal-access-tokens
> [6] https://github.com/foo-pevik/ltp_foo/settings/secrets/actions/new
> [7] https://github.com/foo-pevik/ltp_foo/actions/runs/3651475124
> [8]
> https://github.com/foo-pevik/linux-test-project.github.com_foo/commit/c2b94862da2c5741f08282988dcc0302e0847a69
> [9]
> https://github.com/foo-pevik/ltp_foo/commit/36a66a2f304b449e1c17d72bad1b1db46a30f36f
> [10] https://github.com/foo-pevik/ltp_foo/actions/runs/3651475552
> [11] https://github.com/SwiftDocOrg/github-wiki-publish-action
>
>  .github/workflows/metadata-mirror.yml | 141 ++++++++++++++++++++++++++
>  1 file changed, 141 insertions(+)
>  create mode 100644 .github/workflows/metadata-mirror.yml
>
> diff --git a/.github/workflows/metadata-mirror.yml
> b/.github/workflows/metadata-mirror.yml
> new file mode 100644
> index 000000000..9182c554d
> --- /dev/null
> +++ b/.github/workflows/metadata-mirror.yml
> @@ -0,0 +1,141 @@
> +# SPDX-License-Identifier: GPL-2.0-or-later
> +# Copyright (c) 2022 Petr Vorel <pvorel@suse.cz>
> +
> +name: "Mirror metadata doc to homepage"
> +
> +on:
> +  push:
> +    branches:
> +      - master
> +
> +permissions: {}
> +jobs:
> +  metadata-mirror:
> +    permissions:
> +      contents: write # for git push
> +
> +    runs-on: ubuntu-latest
> +    if: ${{ github.repository == 'linux-test-project/ltp' }}
> +    steps:
> +      - name: Check secret
> +        env:
> +          GH_PERSONAL_ACCESS_TOKEN: ${{ secrets.GH_PERSONAL_ACCESS_TOKEN
> }}
> +        run: |
> +          if [ -z "$GH_PERSONAL_ACCESS_TOKEN" ]; then
> +            echo "::error::GH_PERSONAL_ACCESS_TOKEN environment variable
> is not set"
> +            exit 1
> +          fi
> +
> +      - name: Checkout LTP
> +        uses: actions/checkout@v3
> +        with:
> +          path: ltp
> +          # we need to fetch whole history to get 'git describe' working
> for correct version in docs
> +          fetch-depth: 0
> +
> +      - name: Checkout LTP homepage
> +        uses: actions/checkout@v3
> +        with:
> +          repository: "linux-test-project/linux-test-project.github.com"
> +          path: linux-test-project.github.com
> +          persist-credentials: false
> +        env:
> +          GH_PERSONAL_ACCESS_TOKEN: ${{ secrets.GH_PERSONAL_ACCESS_TOKEN
> }}
> +
> +      - name: Check metadata need to be updated
> +        run: |
> +          cd "$GITHUB_WORKSPACE/ltp/testcases/"
> +
> +          # check for changes
> +          cd "$GITHUB_WORKSPACE/linux-test-project.github.com"
> +          git grep '<p><strong>Version</strong>:'
> metadata/metadata.nightly.html
> +          sed -ne 's/.*<p><strong>Version<\/strong>: \(.*\)<\/p>/\1/p'
> metadata/metadata.nightly.html
> +          old_commit=$(sed -ne 's/.*<p><strong>Version<\/strong>:
> \(.*\)<\/p>/\1/p' metadata/metadata.nightly.html)
> +          old_commit=$(echo "$old_commit" | sed 's/.*-g\(.*\)/\1/')
> +          echo "::notice::old commit in linux-test-project.github.com:
> '$old_commit'"
> +
> +          echo "git log $old_commit..HEAD testcases/ | wc -l"
> +          git log $old_commit..HEAD testcases/ | wc -l
> +
> +          echo "git log $old_commit..HEAD testcases/ 2>&1 | wc -l"
> +          git log $old_commit..HEAD testcases/ 2>&1 | wc -l
> +
> +          if [ "$(git log $old_commit..HEAD testcases/ 2>&1 | wc -l)" = 0
> ]; then
> +            echo "::notice::no file changes in testcases/ directory since
> '$old_commit' => exit without commit and push"
> +            exit 0
> +          fi
> +
> +      - name: Install dependencies
> +        run: |
> +          apt="apt install -y --no-install-recommends"
> +          sudo $apt asciidoctor autoconf automake libjson-perl
> libwww-perl make
> +
> +          echo "which asciidoctor"
> +          which asciidoctor || { echo "::error::missing asciidoctor";
> exit 1; }
> +
> +      - name: Configure LTP
> +        run: |
> +          cd "$GITHUB_WORKSPACE/ltp/"
> +          make autotools && ./configure
> --with-metadata-generator=asciidoctor && make Version || { echo
> "::error::LTP configure failed"; exit 1; }
> +
> +      - name: Generate html metadata doc
> +        run: |
> +          cd "$GITHUB_WORKSPACE/ltp/metadata/"
> +          echo "going to generate metadata"
> +          make -j$(nproc)
> +
> +      - name: Check possible LTP homepage update
> +        run: |
> +          cd "$GITHUB_WORKSPACE/ltp/"
> +          ltp_commit=$(git log --pretty=format:"%h" -1 .)
> +          echo "::notice::Search for $ltp_commit in
> metadata/metadata.nightly.html"
> +
> +          cd "$GITHUB_WORKSPACE/linux-test-project.github.com"
> +          grep "<p><strong>Version</strong>:"
> metadata/metadata.nightly.html
> +          git pull
> +          grep "<p><strong>Version</strong>:"
> metadata/metadata.nightly.html
> +          if grep "<p><strong>Version</strong>: .*${ltp_commit}.*</p>"
> metadata/metadata.nightly.html; then
> +            echo "::notice::$ltp_commit already in
> metadata/metadata.nightly.html => exit without commit and push"
> +            exit 0
> +          fi
> +
> +      - name: Push generated html metadata to LTP homepage
> +        env:
> +          GH_PERSONAL_ACCESS_TOKEN: ${{ secrets.GH_PERSONAL_ACCESS_TOKEN
> }}
> +        run: |
> +          git config --global user.email "actions@github.com"
> +          git config --global user.name "Metadata doc mirror"
> +
> +          cd "$GITHUB_WORKSPACE/ltp/"
> +          commit=$(git log --pretty=format:"%h" -1 .)
> +          commit_desc=$(git log --pretty=format:"%h (\"%s\")" -1 .)
> +          echo "commit_desc: '$commit_desc'"
> +
> +          cd "$GITHUB_WORKSPACE/linux-test-project.github.com"
> +
> +          # pull in case previous hook pushed later than checkout here
> had started
> +          git pull
> +
> +          cp -v "$GITHUB_WORKSPACE/ltp/docparse/metadata.html"
> metadata/metadata.nightly.html
> +
> +          # skip if there is only different version and time
> +          echo "Check for changes"
> +          if ! git diff --unified=0 --no-color | grep -Po
> '(?<=^\+)(?!\+\+).*' | grep -v -e '^<p><strong>Version</strong>:' -e '^Last
> updated'; then
> +            echo "::notice::No changes in metadata/metadata.nightly.html
> => exit without commit and push"
> +            exit 0
> +          fi
> +
> +          git add .
> +
> +          # only commit if there are changes
> +          if git diff-index --quiet HEAD --; then
> +            echo "::notice::No files changed in $GITHUB_WORKSPACE/
> linux-test-project.github.com => exit without commit and push"
> +            git status
> +            exit 0
> +          fi
> +
> +          printf "metadata.nightly.html: Update to $commit\n\nUpdate
> metadata.nightly.html to $commit_desc\n" > /tmp/msg
> +          git commit -F /tmp/msg .
> +
> +          echo "::notice::GH_PERSONAL_ACCESS_TOKEN:
> $GH_PERSONAL_ACCESS_TOKEN"
> +          git push https://${
> GH_PERSONAL_ACCESS_TOKEN}@github.com/linux-test-project/linux-test-project.github.com.git
> <http://GH_PERSONAL_ACCESS_TOKEN%7D@github.com/linux-test-project/linux-test-project.github.com.git>
> --
> 2.38.1
>
>
Petr Vorel Dec. 9, 2022, 11:29 a.m. UTC | #2
> Hi Peter, two comments:

> +  metadata-mirror:
> +    permissions:
> +      contents: write # for git push

> Since you are using GH_PERSONAL_ACCESS_TOKEN and not the default
> GITHUB_TOKEN for push, I think "contents: read" should be sufficient. Could
> you please try it?

Sure, I'll test it.

> +      - name: Checkout LTP homepage
> +        uses: actions/checkout@v3
> +        with:
> +          repository: "linux-test-project/linux-test-project.github.com"
> +          path: linux-test-project.github.com
> +          persist-credentials: false
> +        env:
> +          GH_PERSONAL_ACCESS_TOKEN: ${{ secrets.GH_PERSONAL_ACCESS_TOKEN }}

> I think the GH_PERSONAL_ACCESS_TOKEN env variable here is not used and can
> be deleted. The checkout uses the standard GITHUB_TOKEN with read access to
> public repos.

You're most likely right, testing!

Kind regards,
Petr

> Otherwise LGTM

> On Fri, Dec 9, 2022 at 11:04 AM Petr Vorel <pvorel@suse.cz> wrote:

> > GitHub Actions git push hook generates metadata HTML and push it wo LTP
> > homepage.

> > Hook pushes only if there are actual changes in generated doc.

> > NOTE: This requires to add Personal access tokens (classic)
> > named as GH_PERSONAL_ACCESS_TOKEN with permissions to public_repo.

> > Signed-off-by: Petr Vorel <pvorel@suse.cz>
> > ---
> > Hi,

> > automatic updates of our docparse documentation to our homepage [1]
> > (linked in "nightly test documentation" [2]).

> > This does push to our homepage git repository [3].

> > This patchset required to add:

> > 1) Personal Access Token (PAT) to any developer which has write access
> > to homepage git repository [3]. In Developer settings -> Personal access
> > tokens -> Tokens (classic) [4]), where set:
> > Note: GH_PERSONAL_ACCESS_TOKEN
> > Select scopes: public_repo (minimal permission)
> > Expiration: either never or regularly renew.

> > 2) Allow PAT in LTP organisation (I dared to already set it)
> > Iin linux-test-project group -> Settings -> Third-party Access -> Personal
> > access tokens -> Settings [5]
> > select:
> > Allow access via personal access tokens (classic)
> > API and Git access will be allowed using an organization member's personal
> > access token (classic)

> > 3) Add repository action secret to ltp repository
> > IN Settings -> Actions -> New repository secret [6]:
> > name: GH_PERSONAL_ACCESS_TOKEN
> > value: the value of previously created token.

> > I haven't tested fine-grained personal access tokens, maybe they are
> > safer. Using proper GitHub App would be safer (would not include PAT),
> > but I have spent already quite a lot of time with this, thus unless
> > anybody objects I'd merge this way.

> > I did example of this run on organisation which I created just for
> > testing purposes:

> > * working example
> > action run [7], resulted in commit [8]. Example of unrelated commit [9]
> > which does not trigger doc generation [10].

> > * example of error, when token not set
> > https://github.com/foo-pevik/ltp_foo/actions/runs/3650567244
> > GH_PERSONAL_ACCESS_TOKEN environment variable is not set

> > * example of invalid GH_PERSONAL_ACCESS_TOKEN

> > https://github.com/foo-pevik/ltp_foo/actions/runs/3650579488/jobs/6166734733
> > Notice: GH_PERSONAL_ACCESS_TOKEN: ***
> > fatal: could not read Password for 'https://***@github.com': No such
> > device or address
> > Error: Process completed with exit code 128.

> > Credits:
> > * Inspiration from SwiftDocOrg [11], pointed out by Cyril
> > * Thanks a lot to Aleks for hint with "persist-credentials: false".

> > Kind regards,
> > Petr

> > [1] http://linux-test-project.github.io/
> > [2] http://linux-test-project.github.io/metadata/metadata.nightly.html
> > [3] https://github.com/linux-test-project/linux-test-project.github.com
> > [4] https://github.com/settings/tokens/new
> > [5]
> > https://github.com/organizations/linux-test-project/settings/personal-access-tokens
> > [6] https://github.com/foo-pevik/ltp_foo/settings/secrets/actions/new
> > [7] https://github.com/foo-pevik/ltp_foo/actions/runs/3651475124
> > [8]
> > https://github.com/foo-pevik/linux-test-project.github.com_foo/commit/c2b94862da2c5741f08282988dcc0302e0847a69
> > [9]
> > https://github.com/foo-pevik/ltp_foo/commit/36a66a2f304b449e1c17d72bad1b1db46a30f36f
> > [10] https://github.com/foo-pevik/ltp_foo/actions/runs/3651475552
> > [11] https://github.com/SwiftDocOrg/github-wiki-publish-action

> >  .github/workflows/metadata-mirror.yml | 141 ++++++++++++++++++++++++++
> >  1 file changed, 141 insertions(+)
> >  create mode 100644 .github/workflows/metadata-mirror.yml

> > diff --git a/.github/workflows/metadata-mirror.yml
> > b/.github/workflows/metadata-mirror.yml
> > new file mode 100644
> > index 000000000..9182c554d
> > --- /dev/null
> > +++ b/.github/workflows/metadata-mirror.yml
> > @@ -0,0 +1,141 @@
> > +# SPDX-License-Identifier: GPL-2.0-or-later
> > +# Copyright (c) 2022 Petr Vorel <pvorel@suse.cz>
> > +
> > +name: "Mirror metadata doc to homepage"
> > +
> > +on:
> > +  push:
> > +    branches:
> > +      - master
> > +
> > +permissions: {}
> > +jobs:
> > +  metadata-mirror:
> > +    permissions:
> > +      contents: write # for git push
> > +
> > +    runs-on: ubuntu-latest
> > +    if: ${{ github.repository == 'linux-test-project/ltp' }}
> > +    steps:
> > +      - name: Check secret
> > +        env:
> > +          GH_PERSONAL_ACCESS_TOKEN: ${{ secrets.GH_PERSONAL_ACCESS_TOKEN
> > }}
> > +        run: |
> > +          if [ -z "$GH_PERSONAL_ACCESS_TOKEN" ]; then
> > +            echo "::error::GH_PERSONAL_ACCESS_TOKEN environment variable
> > is not set"
> > +            exit 1
> > +          fi
> > +
> > +      - name: Checkout LTP
> > +        uses: actions/checkout@v3
> > +        with:
> > +          path: ltp
> > +          # we need to fetch whole history to get 'git describe' working
> > for correct version in docs
> > +          fetch-depth: 0
> > +
> > +      - name: Checkout LTP homepage
> > +        uses: actions/checkout@v3
> > +        with:
> > +          repository: "linux-test-project/linux-test-project.github.com"
> > +          path: linux-test-project.github.com
> > +          persist-credentials: false
> > +        env:
> > +          GH_PERSONAL_ACCESS_TOKEN: ${{ secrets.GH_PERSONAL_ACCESS_TOKEN
> > }}
> > +
> > +      - name: Check metadata need to be updated
> > +        run: |
> > +          cd "$GITHUB_WORKSPACE/ltp/testcases/"
> > +
> > +          # check for changes
> > +          cd "$GITHUB_WORKSPACE/linux-test-project.github.com"
> > +          git grep '<p><strong>Version</strong>:'
> > metadata/metadata.nightly.html
> > +          sed -ne 's/.*<p><strong>Version<\/strong>: \(.*\)<\/p>/\1/p'
> > metadata/metadata.nightly.html
> > +          old_commit=$(sed -ne 's/.*<p><strong>Version<\/strong>:
> > \(.*\)<\/p>/\1/p' metadata/metadata.nightly.html)
> > +          old_commit=$(echo "$old_commit" | sed 's/.*-g\(.*\)/\1/')
> > +          echo "::notice::old commit in linux-test-project.github.com:
> > '$old_commit'"
> > +
> > +          echo "git log $old_commit..HEAD testcases/ | wc -l"
> > +          git log $old_commit..HEAD testcases/ | wc -l
> > +
> > +          echo "git log $old_commit..HEAD testcases/ 2>&1 | wc -l"
> > +          git log $old_commit..HEAD testcases/ 2>&1 | wc -l
> > +
> > +          if [ "$(git log $old_commit..HEAD testcases/ 2>&1 | wc -l)" = 0
> > ]; then
> > +            echo "::notice::no file changes in testcases/ directory since
> > '$old_commit' => exit without commit and push"
> > +            exit 0
> > +          fi
> > +
> > +      - name: Install dependencies
> > +        run: |
> > +          apt="apt install -y --no-install-recommends"
> > +          sudo $apt asciidoctor autoconf automake libjson-perl
> > libwww-perl make
> > +
> > +          echo "which asciidoctor"
> > +          which asciidoctor || { echo "::error::missing asciidoctor";
> > exit 1; }
> > +
> > +      - name: Configure LTP
> > +        run: |
> > +          cd "$GITHUB_WORKSPACE/ltp/"
> > +          make autotools && ./configure
> > --with-metadata-generator=asciidoctor && make Version || { echo
> > "::error::LTP configure failed"; exit 1; }
> > +
> > +      - name: Generate html metadata doc
> > +        run: |
> > +          cd "$GITHUB_WORKSPACE/ltp/metadata/"
> > +          echo "going to generate metadata"
> > +          make -j$(nproc)
> > +
> > +      - name: Check possible LTP homepage update
> > +        run: |
> > +          cd "$GITHUB_WORKSPACE/ltp/"
> > +          ltp_commit=$(git log --pretty=format:"%h" -1 .)
> > +          echo "::notice::Search for $ltp_commit in
> > metadata/metadata.nightly.html"
> > +
> > +          cd "$GITHUB_WORKSPACE/linux-test-project.github.com"
> > +          grep "<p><strong>Version</strong>:"
> > metadata/metadata.nightly.html
> > +          git pull
> > +          grep "<p><strong>Version</strong>:"
> > metadata/metadata.nightly.html
> > +          if grep "<p><strong>Version</strong>: .*${ltp_commit}.*</p>"
> > metadata/metadata.nightly.html; then
> > +            echo "::notice::$ltp_commit already in
> > metadata/metadata.nightly.html => exit without commit and push"
> > +            exit 0
> > +          fi
> > +
> > +      - name: Push generated html metadata to LTP homepage
> > +        env:
> > +          GH_PERSONAL_ACCESS_TOKEN: ${{ secrets.GH_PERSONAL_ACCESS_TOKEN
> > }}
> > +        run: |
> > +          git config --global user.email "actions@github.com"
> > +          git config --global user.name "Metadata doc mirror"
> > +
> > +          cd "$GITHUB_WORKSPACE/ltp/"
> > +          commit=$(git log --pretty=format:"%h" -1 .)
> > +          commit_desc=$(git log --pretty=format:"%h (\"%s\")" -1 .)
> > +          echo "commit_desc: '$commit_desc'"
> > +
> > +          cd "$GITHUB_WORKSPACE/linux-test-project.github.com"
> > +
> > +          # pull in case previous hook pushed later than checkout here
> > had started
> > +          git pull
> > +
> > +          cp -v "$GITHUB_WORKSPACE/ltp/docparse/metadata.html"
> > metadata/metadata.nightly.html
> > +
> > +          # skip if there is only different version and time
> > +          echo "Check for changes"
> > +          if ! git diff --unified=0 --no-color | grep -Po
> > '(?<=^\+)(?!\+\+).*' | grep -v -e '^<p><strong>Version</strong>:' -e '^Last
> > updated'; then
> > +            echo "::notice::No changes in metadata/metadata.nightly.html
> > => exit without commit and push"
> > +            exit 0
> > +          fi
> > +
> > +          git add .
> > +
> > +          # only commit if there are changes
> > +          if git diff-index --quiet HEAD --; then
> > +            echo "::notice::No files changed in $GITHUB_WORKSPACE/
> > linux-test-project.github.com => exit without commit and push"
> > +            git status
> > +            exit 0
> > +          fi
> > +
> > +          printf "metadata.nightly.html: Update to $commit\n\nUpdate
> > metadata.nightly.html to $commit_desc\n" > /tmp/msg
> > +          git commit -F /tmp/msg .
> > +
> > +          echo "::notice::GH_PERSONAL_ACCESS_TOKEN:
> > $GH_PERSONAL_ACCESS_TOKEN"
> > +          git push https://${
> > GH_PERSONAL_ACCESS_TOKEN}@github.com/linux-test-project/linux-test-project.github.com.git
> > <http://GH_PERSONAL_ACCESS_TOKEN%7D@github.com/linux-test-project/linux-test-project.github.com.git>
> > --
> > 2.38.1
diff mbox series

Patch

diff --git a/.github/workflows/metadata-mirror.yml b/.github/workflows/metadata-mirror.yml
new file mode 100644
index 000000000..9182c554d
--- /dev/null
+++ b/.github/workflows/metadata-mirror.yml
@@ -0,0 +1,141 @@ 
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2022 Petr Vorel <pvorel@suse.cz>
+
+name: "Mirror metadata doc to homepage"
+
+on:
+  push:
+    branches:
+      - master
+
+permissions: {}
+jobs:
+  metadata-mirror:
+    permissions:
+      contents: write # for git push
+
+    runs-on: ubuntu-latest
+    if: ${{ github.repository == 'linux-test-project/ltp' }}
+    steps:
+      - name: Check secret
+        env:
+          GH_PERSONAL_ACCESS_TOKEN: ${{ secrets.GH_PERSONAL_ACCESS_TOKEN }}
+        run: |
+          if [ -z "$GH_PERSONAL_ACCESS_TOKEN" ]; then
+            echo "::error::GH_PERSONAL_ACCESS_TOKEN environment variable is not set"
+            exit 1
+          fi
+
+      - name: Checkout LTP
+        uses: actions/checkout@v3
+        with:
+          path: ltp
+          # we need to fetch whole history to get 'git describe' working for correct version in docs
+          fetch-depth: 0
+
+      - name: Checkout LTP homepage
+        uses: actions/checkout@v3
+        with:
+          repository: "linux-test-project/linux-test-project.github.com"
+          path: linux-test-project.github.com
+          persist-credentials: false
+        env:
+          GH_PERSONAL_ACCESS_TOKEN: ${{ secrets.GH_PERSONAL_ACCESS_TOKEN }}
+
+      - name: Check metadata need to be updated
+        run: |
+          cd "$GITHUB_WORKSPACE/ltp/testcases/"
+
+          # check for changes
+          cd "$GITHUB_WORKSPACE/linux-test-project.github.com"
+          git grep '<p><strong>Version</strong>:' metadata/metadata.nightly.html
+          sed -ne 's/.*<p><strong>Version<\/strong>: \(.*\)<\/p>/\1/p' metadata/metadata.nightly.html
+          old_commit=$(sed -ne 's/.*<p><strong>Version<\/strong>: \(.*\)<\/p>/\1/p' metadata/metadata.nightly.html)
+          old_commit=$(echo "$old_commit" | sed 's/.*-g\(.*\)/\1/')
+          echo "::notice::old commit in linux-test-project.github.com: '$old_commit'"
+
+          echo "git log $old_commit..HEAD testcases/ | wc -l"
+          git log $old_commit..HEAD testcases/ | wc -l
+
+          echo "git log $old_commit..HEAD testcases/ 2>&1 | wc -l"
+          git log $old_commit..HEAD testcases/ 2>&1 | wc -l
+
+          if [ "$(git log $old_commit..HEAD testcases/ 2>&1 | wc -l)" = 0 ]; then
+            echo "::notice::no file changes in testcases/ directory since '$old_commit' => exit without commit and push"
+            exit 0
+          fi
+
+      - name: Install dependencies
+        run: |
+          apt="apt install -y --no-install-recommends"
+          sudo $apt asciidoctor autoconf automake libjson-perl libwww-perl make
+
+          echo "which asciidoctor"
+          which asciidoctor || { echo "::error::missing asciidoctor"; exit 1; }
+
+      - name: Configure LTP
+        run: |
+          cd "$GITHUB_WORKSPACE/ltp/"
+          make autotools && ./configure --with-metadata-generator=asciidoctor && make Version || { echo "::error::LTP configure failed"; exit 1; }
+
+      - name: Generate html metadata doc
+        run: |
+          cd "$GITHUB_WORKSPACE/ltp/metadata/"
+          echo "going to generate metadata"
+          make -j$(nproc)
+
+      - name: Check possible LTP homepage update
+        run: |
+          cd "$GITHUB_WORKSPACE/ltp/"
+          ltp_commit=$(git log --pretty=format:"%h" -1 .)
+          echo "::notice::Search for $ltp_commit in metadata/metadata.nightly.html"
+
+          cd "$GITHUB_WORKSPACE/linux-test-project.github.com"
+          grep "<p><strong>Version</strong>:" metadata/metadata.nightly.html
+          git pull
+          grep "<p><strong>Version</strong>:" metadata/metadata.nightly.html
+          if grep "<p><strong>Version</strong>: .*${ltp_commit}.*</p>" metadata/metadata.nightly.html; then
+            echo "::notice::$ltp_commit already in metadata/metadata.nightly.html => exit without commit and push"
+            exit 0
+          fi
+
+      - name: Push generated html metadata to LTP homepage
+        env:
+          GH_PERSONAL_ACCESS_TOKEN: ${{ secrets.GH_PERSONAL_ACCESS_TOKEN }}
+        run: |
+          git config --global user.email "actions@github.com"
+          git config --global user.name "Metadata doc mirror"
+
+          cd "$GITHUB_WORKSPACE/ltp/"
+          commit=$(git log --pretty=format:"%h" -1 .)
+          commit_desc=$(git log --pretty=format:"%h (\"%s\")" -1 .)
+          echo "commit_desc: '$commit_desc'"
+
+          cd "$GITHUB_WORKSPACE/linux-test-project.github.com"
+
+          # pull in case previous hook pushed later than checkout here had started
+          git pull
+
+          cp -v "$GITHUB_WORKSPACE/ltp/docparse/metadata.html" metadata/metadata.nightly.html
+
+          # skip if there is only different version and time
+          echo "Check for changes"
+          if ! git diff --unified=0 --no-color | grep -Po '(?<=^\+)(?!\+\+).*' | grep -v -e '^<p><strong>Version</strong>:' -e '^Last updated'; then
+            echo "::notice::No changes in metadata/metadata.nightly.html => exit without commit and push"
+            exit 0
+          fi
+
+          git add .
+
+          # only commit if there are changes
+          if git diff-index --quiet HEAD --; then
+            echo "::notice::No files changed in $GITHUB_WORKSPACE/linux-test-project.github.com => exit without commit and push"
+            git status
+            exit 0
+          fi
+
+          printf "metadata.nightly.html: Update to $commit\n\nUpdate metadata.nightly.html to $commit_desc\n" > /tmp/msg
+          git commit -F /tmp/msg .
+
+          echo "::notice::GH_PERSONAL_ACCESS_TOKEN: $GH_PERSONAL_ACCESS_TOKEN"
+          git push https://${GH_PERSONAL_ACCESS_TOKEN}@github.com/linux-test-project/linux-test-project.github.com.git