diff mbox series

[kteam-tools,1/3] stable/docs: commit copy of kernel-sru-workflow

Message ID 20220217160154.240691-2-dimitri.ledkov@canonical.com
State New
Headers show
Series Updates to kernel-sru-workflow | expand

Commit Message

Dimitri John Ledkov Feb. 17, 2022, 4:01 p.m. UTC
Signed-off-by: Dimitri John Ledkov <dimitri.ledkov@canonical.com>
---
 stable/docs/kernel-sru-workflow.moin | 136 +++++++++++++++++++++++++++
 1 file changed, 136 insertions(+)
 create mode 100644 stable/docs/kernel-sru-workflow.moin
diff mbox series

Patch

diff --git a/stable/docs/kernel-sru-workflow.moin b/stable/docs/kernel-sru-workflow.moin
new file mode 100644
index 0000000000..2e5f652e11
--- /dev/null
+++ b/stable/docs/kernel-sru-workflow.moin
@@ -0,0 +1,136 @@ 
+#### This is a copy of https://wiki.ubuntu.com/Kernel/kernel-sru-workflow
+#### For reference, and ease of patching/preparing update to the page
+####
+
+## page was renamed from Kernel/kernel-sru-workflow.html
+
+= IMPORTANT NOTE =
+
+{{{#!wiki red/solid
+The workflow processes described on this page are embodied in workflow tools
+which perform automated updates to process tasks.
+
+No changes to the processes on this page should be made without discussion
+among the teams affected, including the Release Manager, Stable Kernel Team,
+Security Team, QA Team, and Certification Team.
+}}}
+
+== Kernel SRU Workflow ==
+
+The kernel release tracking bug is going to be changed to facilitate better communication between the
+responsible parties and clearer handoffs as the release progresses.
+
+We are taking advantage of existing Launchpad capabilities. We will be using project series to represent
+individual workflow tasks. The meaning of status is overloaded to indicate the state of the task.
+
+When a kernel release tracking bug is created, it is created against the relevant kernel source package
+and nominated for the related Ubuntu series. The new process will target the bug against an additional
+project, the "Kernel SRU Workflow" project and nominate it for all the series that are defined for that
+project.
+
+The kernel release tracking bug can be identified on the kernel SRU report page at:
+http://kernel.ubuntu.com/sru/sru-report.html
+
+The "Kernel SRU Workflow" project has a number of custom "series" created for it that represent the
+different stages of the kernel cadence. A "series" represents a task to be accomplished by a
+team/person. The different tasks will be assigned to the team/person responsible for that stage. The
+assignee will set the status of the tasks they are working.
+
+An automated script will run periodically to monitor the current state of the
+different tasks and change status when necessary. This script will be referred
+to below as the '''Workflow Mgr.''' The kernel team will develop this bot.
+
+=== The Workflow: ===
+ 1. The kernel team or '''Workflow Mgr.''' creates a tracking bug, all tasks will be set to their initial state (status: '''New''') and be assigned to the appropriate team.
+
+ 1. If there is an '''Upload-to-ppa''' task in the tracking bug:
+
+    1. The assignee of '''Prepare-package''' task sets it to the in-progress state (status: '''In Progress'''). Then he/she prepares the branch in git repository for the necessary packages, and when release is closed in the branch, updates the version in the tracking bug title and description. If there is no abi change, the assignee must invalidate any '''Prepare-package-*''' tasks (status: '''Invalid'''). When all this is ready, the assignee changes '''Prepare-package''' to completed (status: '''Fix Released''').
+
+    1. '''Workflow Mgr.''' detects that the state of the '''Prepare-package''' task is now completed and changes the state of the '''Upload-to-ppa''' task to its ready-to-start state (status: '''Confirmed''').
+
+    1. The kernel team sets '''Upload-to-ppa''' task to the in-progress state (status: '''In Progress'''). If build of other dependent packages is needed ('''Prepare-package-*''' tasks not invalid), kernel team sets '''Prepare-package-*''' tasks to the in-progress state (status: '''In Progress''').
+
+ 1. If no '''Upload-to-ppa''' task is present:
+
+   1. The kernel team sets '''Prepare-package''' task to the in-progress state (status: '''In Progress'''). If build of other dependent packages is needed, kernel team sets any required '''Prepare-package-*''' tasks to the in-progress state, or keeps/sets them to invalid if it isn't needed (status: '''In Progress''' or '''Invalid''').
+
+ 1. The kernel team builds and uploads all the necessary packages. For all '''Prepare-package-*''' tasks that are not Invalid and are In Progress, '''Workflow Mgr.''' detects when the package successfully builds on c-k-t ppa and is ready to be copied to proposed, then it changes their state to completed (status: '''Fix Released'''). When all '''Prepare-package-*''' tasks present are finished (status: '''Fix Released''' or '''Invalid'''), '''Workflow Mgr.''' detects when the main source package is built and ready to be copied to proposed, and when it happens then:
+
+   1. If '''Upload-to-ppa''' is present, '''Workflow Mgr.''' sets it to completed (status: '''Fix Released''').
+
+   1. If no '''Upload-to-ppa''' task is present: '''Workflow Mgr.''' sets the '''Prepare-package''' task to completed (status: '''Fix Released''').
+
+ 1. When all required packages are built, after '''Workflow Mgr.''' changes the state of the '''Upload-to-ppa''' to completed, or if it isn't present and it changes the state of '''Prepare-package''' to completed, then '''Workflow Mgr.''' creates if needed new tracking bugs for derivative (topic branches for this package) and backport packages, and changes the '''Promote-to-proposed''' task to its ready-to-start state (status: '''Confirmed''').
+
+ 1. The kernel team or assignee of '''Prepare-package''' runs the script copy-proposed-kernel.py from lp:ubuntu-archive-tools for each source package uploaded, so the built packages are added to the upload queue used by archive admins to sync the packages to proposed.
+
+ 1. An archive admin sets the '''Promote-to-proposed''' task to in-progress (status: '''In Progress''') and syncs the packages from the queue to the '''proposed''' pocket in the archive. The task is reassigned to the individual working the task.
+
+ 1. Once the packages have been copied, the archive admin (or exceptionally an stable release team member) sets the '''Promote-to-proposed''' task to completed (status: '''Fix Released''').
+
+ 1. '''Workflow Mgr.''' detects that the state of the '''Promote-to-proposed''' task is now completed. It may do additional checks if the packages are really in the archive and in the proper components. If it's completed and everything is ok, it changes the state of the '''Verification-testing''' task to the in-progress state (status: '''In Progress''').
+
+ 1. The kernel team tags all bugs needing verification with '''verification-needed-<series>''', and adds a request for testing/verification.
+
+ 1. Once all the bugs, requiring verification, listed in the changelog have been marked '''verification-done-<series>''', the kernel team changes the state of the '''Verification-testing''' task to completed (status: '''Fix Released'''). '''Workflow Mgr.''' then detects this and sets the '''Certification-testing''', '''Regression-testing''' and '''Security-signoff''' tasks to the ready-to-start state (status: '''Confirmed''').
+
+ 1. When the HW Certification team detects that the '''Certification-testing''' task is in the ready-to-start state (status: '''Confirmed''') they may start [[Kernel/kernel-sru-workflow/CertificationTestSuite|certification testing]]. If the HW Certification team will not perform testing on this package, the '''Certification-testing''' task may be set to '''Invalid''', but if they are testing then they change the tasks state to in-progress (status: '''In Progress'''). The task is reassigned to the individual working the task. (Note - the task may actually be set to '''Invalid''' while the task is still in the '''New''' state if desired)
+
+ 1. When the QA team detects that the '''Regression-testing''' task is in the ready-to-start state (status: '''Confirmed''') and they start testing, they change the tasks state to in-progress (status: '''In Progress'''). The task is reassigned to the individual working the task.
+
+ 1. When the security team detects that the '''Security-signoff''' task is in the ready-to-start state (status: '''Confirmed'''), they change the tasks state to in-progress (status: '''In Progress'''). The task is reassigned to the individual working the task. They take care of any tasks they deem necessary prior to having an archive admin copy the release to the '''security''' pocket. If there are no CVEs, the security team sets the '''Security-signoff''' task to the not-needed state (status: '''Invalid'''). If there are CVEs in the release and the security team has signed-off on the release being promoted to the security pocket they change the status of the '''Security-signoff''' task to completed (status: '''Fix Released''').
+
+ 1. If certification testing is being performed, once certification testing completes, the HW certification team changes the state of the '''Certification-testing''' task to completed (status: '''Fix Released'''). If the testing was successful the certification team adds a '''certification-testing-passed''' tag otherwise they add a '''certification-testing-failed''' tag. These tags are not required if the '''Certification-testing''' task is set to '''Invalid'''.
+
+ 1. Once regression testing completes, the QA team changes the state of the '''Regression-testing''' task to completed (status:'''Fix Released'''). If the testing was successful the QA team adds a '''qa-testing-passed''' tag, otherwise they add a '''qa-testing-failed''' tag.
+
+ 1. When the '''Certification-testing''' and '''Regression-testing''' tasks have been set to completed states (status: '''Fix Released''' or optionally '''Invalid''' for certification) and both the '''certification-testing-passed''' and '''qa-testing-passed''' tags have been added by the appropriate team, and when the status of the '''Security-Signoff''' task has been set to either '''Invalid''' or '''Fix Released''', the '''Workflow Mgr.''' is ready to change the state of the promote to update/security tasks to be started (status:'''Confirmed'''). However, '''Workflow Mgr.''' will only change these tasks if it not runs nearby an weekend (between 18:00 UTC Friday - 21:00 UTC Sunday), as there are less resources on weekends to fix anything if problems happen on packages copied to -updates/-security. Therefore, if promote to updates/security are ready to be started, and '''Workflow Mgr.''' isn't running near or on an weekend, it'll set '''Promote-to-updates''' task to be ready to be started (status:'''Confirmed'''), and do the same to the '''Promote-to-security''' task if '''Security-signoff''' task isn't invalid (status:'''Invalid'''). If '''Security-signoff''' is invalid, '''Promote-to-security''' is set to invalid as well.
+
+ 1. If the '''Promote-to-updates''' task is set to the ready-to-start state (status: '''Confirmed'''), an stable release team member copies the packages from '''proposed''' to the '''updates''' pocket in the archive and sets the '''Promote-to-updates''' task to completed (status: '''Fix Released'''). The task is reassigned to the individual working the task.
+
+ 1. If the '''Promote-to-security''' task is set to the ready-to-start state (status: '''Confirmed'''), an archive admin copies the packages to the '''security''' pocket and sets the state of the '''Promote-to-security''' task to completed (status: '''Fix Released'''). The task is reassigned to the individual working the task.
+
+ 1. After all tasks are completed, '''Workflow Mgr.''' sets the main kernel-sru-workflow project task to completed (status: '''Fix Released'''), and sends all desired notifications about the new update.
+
+'''Note:''' Some tasks can move from '''Confirmed''' straight to '''Fix Released''' depending on the amount of time/effort involved in the task.
+
+=== An Example Tracking Bug ===
+https://bugs.launchpad.net/kernel-sru-workflow/+bug/677021
+
+=== Tasks ===
+||'''Series'''               || '''Owner'''           || '''Description''' ||
+||Prepare-package            || Assigned maintainer of the package branch                               || The assignee has prepared or uploaded the source package for the release to the kernel team's ppa. ||
+||Upload-to-ppa              || [[https://launchpad.net/~canonical-kernel-team|Kernel Team]]            || The kernel team has uploaded the source package for the release to the kernel team's ppa. ||
+||Prepare-package-lbm        || [[https://launchpad.net/~canonical-kernel-team|Kernel Team]]            || The kernel team has uploaded the linux-backport-modules source package for the release to the kernel team's ppa. ||
+||Prepare-package-lrm        || [[https://launchpad.net/~canonical-kernel-team|Kernel Team]]            || The kernel team has uploaded the linux-restricted-modules source package for the release to the kernel team's ppa. ||
+||Prepare-package-lum        || [[https://launchpad.net/~canonical-kernel-team|Kernel Team]]            || The kernel team has uploaded the linux-ubuntu-modules source package for the release to the kernel team's ppa. ||
+||Prepare-package-meta       || [[https://launchpad.net/~canonical-kernel-team|Kernel Team]]            || The kernel team has uploaded the linux-*-meta source package for the release to the kernel team's ppa. ||
+||Prepare-package-ports-meta || [[https://launchpad.net/~canonical-kernel-team|Kernel Team]]            || The kernel team has uploaded the linux-*-ports-meta source package for the release to the kernel team's ppa. ||
+||Prepare-package-signed || [[https://launchpad.net/~canonical-kernel-team|Kernel Team]]            || The kernel team has uploaded the linux-signed source package for the release to the kernel team's ppa. ||
+||Promote-to-proposed        || [[https://launchpad.net/~ubuntu-sru|SRU Team]]                          || The package in the kernel team's ppa is copied to the '''proposed''' pocket in the archive. ||
+||Verification-testing       || [[https://launchpad.net/~canonical-kernel-team|Kernel Team]]            || The bugs related to the release are being verified as having been fixed by the appropriate community member. ||
+||Certification-testing      || [[https://launchpad.net/~hardware-certification|HW Certification Team]] || The kernel in '''proposed''' is tested via the certification tests. ||
+||Regression-testing         || [[https://launchpad.net/~canonical-platform-qa/|QA Team]]                         || The kernel in '''proposed''' is tested for regressions. ||
+||Security-signoff           || [[https://launchpad.net/~canonical-security|Security Team]]             || The security team does any validation they deem necessary and declares if Promote-to-security is needed ||
+||Promote-to-updates         || [[https://launchpad.net/~ubuntu-sru|SRU Team]]                          || The package is copied from the '''proposed''' pocket to the '''updates''' pocket in the archive. ||
+||Promote-to-security        || [[https://launchpad.net/~ubuntu-sru|SRU Team]]                          || The package is copied from the '''proposed''' pocket to the '''security''' pocket in the archive, if needed. ||
+
+=== Status ===
+||'''Status'''           || '''Description''' ||
+||New                    || The initial state of the task. This is not ready for the assigned team/person to begin working on that task. ||
+||Confirmed              || This indicates that the prerequisites for the task to begin have been met and the task is ready to be started. ||
+||In Progress            || The assigned team/person has begun the work associated with the given task. ||
+||Invalid                || The process state is not appropriate for the given kernel release. The individual that changed the state to Invalid should reassign the task to themselves. Examples of where this would be used are: When there are no CVEs fixed in the release and it is not necessary to copy the release.||
+||Fix Released           || The assigned team/person has finished the task. ||
+
+=== Implementation Notes ===
+In order to track some meta-information about the overall status of the tracking bug, the kernel-sru-workflow task is used by the software bot that automates process steps. The status of this task should never be changed manually unless the implications are well understood.
+
+Status for this task maps as follows:
+||'''Status'''           || '''Description''' ||
+||New                    || Just created. ||
+||In Progress            || Actively being processed by '''Workflow Mgr.''' This is the only state in which '''Workflow Mgr''' will make changes to the bug. ||
+||Incomplete             || Error, exception, or invalid condition - Processing by '''Workflow Mgr''' is suspended. ||
+||Invalid                || The bug has been replaced by a new tracking bug.||
+||Fix Released           || All good and we're done. ||