diff mbox series

[SRU,PULL,linux-firmware,Lunar] UBUNTU: [Packaging] scripts: add back list-lts-update-files

Message ID 20221213130718.1727795-1-vicamo.yang@canonical.com
State New
Headers show
Series [SRU,PULL,linux-firmware,Lunar] UBUNTU: [Packaging] scripts: add back list-lts-update-files | expand

Commit Message

You-Sheng Yang Dec. 13, 2022, 1:07 p.m. UTC
BugLink: https://bugs.launchpad.net/bugs/1999406

[Impact]

When migrating to a new hwe kernel or introduced a oem kernel of a
newer version, the firmware blobs might not match the expectation of
the new kernel. For example, a hwe-5.15 on Focal may include new
drivers that relies on firmware blobs that are only available in Jammy.

[Fix]

A previously obsoleted (and removed in Kinetic and Lunar) script
`list-lts-update-files` is used to enumerate firmware blobs to be
backported. It's retored and revised with additional support to
pick the right blobs especially in regard to ath and iwlwifi drivers.

On Focal, the commits in need are enumerated by:

  $ debian/scripts/list-lts-update-files focal jammy \
        ../jammy/debian.master/abi/fwinfo | \
        xargs git log --graph --oneline focal..jammy --
  * 36f2ea9f7 ath11k: WCN6855 hw2.0: add WLAN.HSP.1.1-03125-QCAHSPSWPL_V1_V2_SILICONZ_LITE-3
  * 05b5dc001 ath11k: WCN6855 hw2.0: add board-2.bin and regdb.bin
  * 7a3005059 brcm: Add 43455 based AP6255 NVRAM for the ACEPC T8 Mini PC
  * ad185afa1 cypress: update firmware for cyw4373 sdio
  * 77d3eb8ef cypress: update firmware for cyw43570 pcie
  * 92e9acdbd cypress: update firmware for cyw4356 sdio
  * 5f88084be cypress: update firmware for cyw4354 sdio
  * f97e31677 cypress: update firmware for cyw43455 sdio
  * 3df9ea0b9 cypress: update firmware for cyw43430 sdio
  * 6150015cf cypress: update firmware for cyw43012 sdio
  * 2548d065b brcm: Add nvram for the Chuwi Hi8 (CWI509) tablet
  * e45c137e7 brcm: Add nvram for the Predia Basic tablet
  * d52886242 brcm: Add NVRAM for Vamrs 96boards Rock960
  *   b503c9660 Merge branch 'ath10k-20201023' of git://git.kernel.org/pub/scm/linux/kernel/git/kvalo/linux-firmware into main
  |\
  | * 34cb5fce2 ath11k: IPQ8074 hw2.0: add to WLAN.HK.2.1.0.1-01238-QCAHKSWPL_SILICONZ-2
  | * c0a8efd24 ath11k: IPQ8074 hw2.0: add board-2.bin
  | * ac7f5e93f ath11k: IPQ6018 hw1.0: add to WLAN.HK.2.1.0.1-01238-QCAHKSWPL_SILICONZ-2
  | * 2594e510a ath11k: IPQ6018 hw1.0: add board-2.bin
  * 04f71fe56 cypress: add Cypress firmware and clm_blob files

Commit 05b5dc001 and 36f2ea9f7 include updates to other existing files,
so they are dropped from this SRU. An additional commit that modifies
only WHENCE, commit 0b558e8a7, found when resolving conflicts, were also
added.

On Jammy,

  $ debian/scripts/list-lts-update-files jammy kinetic \
        ../kinetic/debian.master/abi/fwinfo | \
        xargs git log --graph --oneline jammy..kinetic --
  * 35f8de521 UBUNTU: Add pre-compiled echoaudio firmware
  * c954892f6 Add initial AzureWave AW-CM256SM NVRAM file
  * 86f0c5642 ath10k: WCN3990 hw1.0: add board-2.bin
  *   c3624ebd6 Merge branch 'ath10k-20220423' of git://git.kernel.org/pub/scm/linux/kernel/git/kvalo/linux-firmware into main
  |\
  | * 1962cbab3 ath10k: QCA99X0 hw2.0: add board-2.bin
  | * 0d5e9f7e0 ath11k: WCN6750 hw1.0: add to WLAN.MSL.1.0.1-00887-QCAMSLSWPLZ-1
  | * a50132f7f ath11k: WCN6750 hw1.0: add board-2.bin
  | * 97f8b7563 ath11k: QCN9074 hw1.0: add to WLAN.HK.2.5.0.1-01208-QCAHKSWPL_SILICONZ-1
  | * f56505fe2 ath11k: QCN9074 hw1.0: add board-2.bin
  * | dfa3c4c30 qcom: add firmware files for Adreno a420 & related generations
  * | 4a43f1a84 qcom: add firmware files for Adreno a330
  * | ac21ab5d1 Mellanox: Add lc_ini_bundle for xx.2010.1006
  |/
  * 4ffcf980a brcm: rename Rock960 NVRAM to AP6356S and link devices to it

Commit 4ffcf980a involves rename existing files in WHENCE, so ignored.

While Jammy has 3 additional oem kernels, they were also tested:

  $ debian/scripts/list-lts-update-files jammy kinetic \
        ../oem-5.17/debian.oem/abi/fwinfo | \
        xargs git log --graph --oneline jammy..kinetic --
  (no new commits)

  $ debian/scripts/list-lts-update-files jammy lunar \
        ../oem-6.0/debian.oem/abi/fwinfo | \
        xargs git log --graph --oneline jammy..lunar --
  * 06dbfbc74 iwlwifi: add new FWs from core69-81 release

  $ debian/scripts/list-lts-update-files jammy lunar \
        ../oem-6.1/debian.oem/abi/fwinfo | \
        xargs git log --graph --oneline jammy..lunar --
  * 51fff4e69 i915: Add versionless HuC files for current platforms

On Lunar, while it's not an LTS series and there will not be any hwe/oem
kernel backported, the only change proposed is the script itself.

[Test Case]

To ensure no exiting bits being overwritten, the changes are reviewed
manually.

[Where problems could occur]

These firmware blobs are only referenced in the new hwe/oem kernels, and
shall not have side effect. On Focal and Jammy, while we have rolled out
oem and hwe kernels a long time ago, these changes will enable the
corresponding hardware that were meant to be enabled along with the
oem/hwe kernel migration at the time.

[Other Info]

While there is no hwe/oem kernel planned for Kinetic, and unlike the
devel series Lunar, adding this script to it is quite meaningless, so
it's not nominated for fix here.

----------------------------------------------------------------
The following changes since commit e3ff59307071a7867b3436f48bf69a7156abd2f1:

  UBUNTU: Ubuntu-20221212.git0707b2f2-0ubuntu1 (2022-12-12 08:51:34 +0100)

are available in the Git repository at:

  https://git.launchpad.net/~vicamo/ubuntu/+source/linux-firmware bug-1999406/update-fw-for-hwe-kernels/lunar

for you to fetch changes up to 834d95c984cd90dba9c6e9d497732fa08dd9a920:

  UBUNTU: [Packaging] scripts: add back list-lts-update-files (2022-12-13 20:05:46 +0800)

----------------------------------------------------------------
You-Sheng Yang (vicamo) (1):
      UBUNTU: [Packaging] scripts: add back list-lts-update-files

 debian/scripts/list-lts-update-files | 248 +++++++++++++++++++++++++++++++++++
 1 file changed, 248 insertions(+)
 create mode 100755 debian/scripts/list-lts-update-files

Comments

Juerg Haefliger Jan. 20, 2023, 3:54 p.m. UTC | #1
Applied to linux-firmware lunar branch.

Note that your submission email also contains the patch content inline.
Please don't do that.

I also just realized that the patch author email and SOB are different (gmail
vs. canonical). I accept it this time but only because I missed it in your
jammy submission. In the future, please make sure the SOB and author
name/email are identical for patches that you author yourself.

...Juerg


> BugLink: https://bugs.launchpad.net/bugs/1999406
> 
> [Impact]
> 
> When migrating to a new hwe kernel or introduced a oem kernel of a
> newer version, the firmware blobs might not match the expectation of
> the new kernel. For example, a hwe-5.15 on Focal may include new
> drivers that relies on firmware blobs that are only available in Jammy.
> 
> [Fix]
> 
> A previously obsoleted (and removed in Kinetic and Lunar) script
> `list-lts-update-files` is used to enumerate firmware blobs to be
> backported. It's retored and revised with additional support to
> pick the right blobs especially in regard to ath and iwlwifi drivers.
> 
> On Focal, the commits in need are enumerated by:
> 
>   $ debian/scripts/list-lts-update-files focal jammy \
>         ../jammy/debian.master/abi/fwinfo | \
>         xargs git log --graph --oneline focal..jammy --
>   * 36f2ea9f7 ath11k: WCN6855 hw2.0: add WLAN.HSP.1.1-03125-QCAHSPSWPL_V1_V2_SILICONZ_LITE-3
>   * 05b5dc001 ath11k: WCN6855 hw2.0: add board-2.bin and regdb.bin
>   * 7a3005059 brcm: Add 43455 based AP6255 NVRAM for the ACEPC T8 Mini PC
>   * ad185afa1 cypress: update firmware for cyw4373 sdio
>   * 77d3eb8ef cypress: update firmware for cyw43570 pcie
>   * 92e9acdbd cypress: update firmware for cyw4356 sdio
>   * 5f88084be cypress: update firmware for cyw4354 sdio
>   * f97e31677 cypress: update firmware for cyw43455 sdio
>   * 3df9ea0b9 cypress: update firmware for cyw43430 sdio
>   * 6150015cf cypress: update firmware for cyw43012 sdio
>   * 2548d065b brcm: Add nvram for the Chuwi Hi8 (CWI509) tablet
>   * e45c137e7 brcm: Add nvram for the Predia Basic tablet
>   * d52886242 brcm: Add NVRAM for Vamrs 96boards Rock960
>   *   b503c9660 Merge branch 'ath10k-20201023' of git://git.kernel.org/pub/scm/linux/kernel/git/kvalo/linux-firmware into main
>   |\
>   | * 34cb5fce2 ath11k: IPQ8074 hw2.0: add to WLAN.HK.2.1.0.1-01238-QCAHKSWPL_SILICONZ-2
>   | * c0a8efd24 ath11k: IPQ8074 hw2.0: add board-2.bin
>   | * ac7f5e93f ath11k: IPQ6018 hw1.0: add to WLAN.HK.2.1.0.1-01238-QCAHKSWPL_SILICONZ-2
>   | * 2594e510a ath11k: IPQ6018 hw1.0: add board-2.bin
>   * 04f71fe56 cypress: add Cypress firmware and clm_blob files
> 
> Commit 05b5dc001 and 36f2ea9f7 include updates to other existing files,
> so they are dropped from this SRU. An additional commit that modifies
> only WHENCE, commit 0b558e8a7, found when resolving conflicts, were also
> added.
> 
> On Jammy,
> 
>   $ debian/scripts/list-lts-update-files jammy kinetic \
>         ../kinetic/debian.master/abi/fwinfo | \
>         xargs git log --graph --oneline jammy..kinetic --
>   * 35f8de521 UBUNTU: Add pre-compiled echoaudio firmware
>   * c954892f6 Add initial AzureWave AW-CM256SM NVRAM file
>   * 86f0c5642 ath10k: WCN3990 hw1.0: add board-2.bin
>   *   c3624ebd6 Merge branch 'ath10k-20220423' of git://git.kernel.org/pub/scm/linux/kernel/git/kvalo/linux-firmware into main
>   |\
>   | * 1962cbab3 ath10k: QCA99X0 hw2.0: add board-2.bin
>   | * 0d5e9f7e0 ath11k: WCN6750 hw1.0: add to WLAN.MSL.1.0.1-00887-QCAMSLSWPLZ-1
>   | * a50132f7f ath11k: WCN6750 hw1.0: add board-2.bin
>   | * 97f8b7563 ath11k: QCN9074 hw1.0: add to WLAN.HK.2.5.0.1-01208-QCAHKSWPL_SILICONZ-1
>   | * f56505fe2 ath11k: QCN9074 hw1.0: add board-2.bin
>   * | dfa3c4c30 qcom: add firmware files for Adreno a420 & related generations
>   * | 4a43f1a84 qcom: add firmware files for Adreno a330
>   * | ac21ab5d1 Mellanox: Add lc_ini_bundle for xx.2010.1006
>   |/
>   * 4ffcf980a brcm: rename Rock960 NVRAM to AP6356S and link devices to it
> 
> Commit 4ffcf980a involves rename existing files in WHENCE, so ignored.
> 
> While Jammy has 3 additional oem kernels, they were also tested:
> 
>   $ debian/scripts/list-lts-update-files jammy kinetic \
>         ../oem-5.17/debian.oem/abi/fwinfo | \
>         xargs git log --graph --oneline jammy..kinetic --
>   (no new commits)
> 
>   $ debian/scripts/list-lts-update-files jammy lunar \
>         ../oem-6.0/debian.oem/abi/fwinfo | \
>         xargs git log --graph --oneline jammy..lunar --
>   * 06dbfbc74 iwlwifi: add new FWs from core69-81 release
> 
>   $ debian/scripts/list-lts-update-files jammy lunar \
>         ../oem-6.1/debian.oem/abi/fwinfo | \
>         xargs git log --graph --oneline jammy..lunar --
>   * 51fff4e69 i915: Add versionless HuC files for current platforms
> 
> On Lunar, while it's not an LTS series and there will not be any hwe/oem
> kernel backported, the only change proposed is the script itself.
> 
> [Test Case]
> 
> To ensure no exiting bits being overwritten, the changes are reviewed
> manually.
> 
> [Where problems could occur]
> 
> These firmware blobs are only referenced in the new hwe/oem kernels, and
> shall not have side effect. On Focal and Jammy, while we have rolled out
> oem and hwe kernels a long time ago, these changes will enable the
> corresponding hardware that were meant to be enabled along with the
> oem/hwe kernel migration at the time.
> 
> [Other Info]
> 
> While there is no hwe/oem kernel planned for Kinetic, and unlike the
> devel series Lunar, adding this script to it is quite meaningless, so
> it's not nominated for fix here.
> 
> ----------------------------------------------------------------
> The following changes since commit e3ff59307071a7867b3436f48bf69a7156abd2f1:
> 
>   UBUNTU: Ubuntu-20221212.git0707b2f2-0ubuntu1 (2022-12-12 08:51:34 +0100)
> 
> are available in the Git repository at:
> 
>   https://git.launchpad.net/~vicamo/ubuntu/+source/linux-firmware bug-1999406/update-fw-for-hwe-kernels/lunar
> 
> for you to fetch changes up to 834d95c984cd90dba9c6e9d497732fa08dd9a920:
> 
>   UBUNTU: [Packaging] scripts: add back list-lts-update-files (2022-12-13 20:05:46 +0800)
> 
> ----------------------------------------------------------------
> You-Sheng Yang (vicamo) (1):
>       UBUNTU: [Packaging] scripts: add back list-lts-update-files
> 
>  debian/scripts/list-lts-update-files | 248 +++++++++++++++++++++++++++++++++++
>  1 file changed, 248 insertions(+)
>  create mode 100755 debian/scripts/list-lts-update-files
> 
> diff --git a/debian/scripts/list-lts-update-files b/debian/scripts/list-lts-update-files
> new file mode 100755
> index 000000000..46fa3dfff
> --- /dev/null
> +++ b/debian/scripts/list-lts-update-files
> @@ -0,0 +1,248 @@
> +#!/bin/bash
> +#
> +# Used to get a list of firmware files needed to update a LTS
> +# linux-firmware package for a backport kernel. For example to
> +# get a list of files for trusty to support the linux-lts-xenial
> +# kernel run:
> +#
> +#  debian/scripts/list-lts-update-files [--dump-filtered] trusty xenial /path/to/xenial/fwinfo
> +#
> +# This will generate a list of files that satisfy all of these
> +# conditions:
> +#
> +#  1) In the xenial branch of linux-firmware
> +#  2) Not in the trusty branch of linux-firmware
> +#  3) In the specified fwinfo file
> +#
> +# This can be used as a starting point for finding upstream
> +# commits needed to add the missing firmware files.
> +#
> +# The fwinfo file can be omitted, in which case the output is a list
> +# of firmware files in @newbranch but not in @oldbranch.
> +
> +#set -ex
> +
> +dump_filtered=
> +if [ $# -ge 1 ]; then
> +	case "$1" in
> +	--dump-filtered)
> +		dump_filtered=1; shift;;
> +	*) ;;
> +	esac
> +fi
> +
> +if [ $# -lt 2 ]; then
> +	echo "Usage $0 [--dump-filtered] <oldbranch> <newbranch> [<fwinfo>]"
> +	exit 1
> +fi
> +
> +OLDBRANCH="$1"
> +NEWBRANCH="$2"
> +FWINFO="$3"
> +
> +TMPDIR=$(mktemp -d)
> +
> +function cleanup {
> +	rm -r "$TMPDIR"
> +}
> +trap cleanup EXIT
> +
> +function normalize_link {
> +	# handle relative paths: a/b/c/../../d/e -> a/d/e
> +	local s="$1"
> +	s="${s#./}"
> +	while true; do
> +		case "$s" in
> +		*/../*)
> +			s=$(echo "$s" | sed 's,[^/]\+/../,,')
> +			;;
> +		*)
> +			break ;;
> +		esac
> +	done
> +	echo "$s"
> +}
> +
> +function grep_links {
> +	grep ^WHENCE "$1" | sed "s,^,$2:," | xargs git show | grep ^Link:
> +}
> +
> +if ! git ls-tree -r --name-only "$OLDBRANCH" >"$TMPDIR/files.tmp"
> +then
> +	exit 1
> +fi
> +cat "$TMPDIR/files.tmp" | sort >"$TMPDIR/oldbranch-files"
> +
> +if ! git ls-tree -r --name-only "$NEWBRANCH" >"$TMPDIR/files.tmp"
> +then
> +	exit 1
> +fi
> +cat "$TMPDIR/files.tmp" | sort >"$TMPDIR/newbranch-files"
> +
> +# Get files in @newbranch not in @oldbranch
> +comm -13 "$TMPDIR/oldbranch-files" "$TMPDIR/newbranch-files" >"$TMPDIR/new-files"
> +
> +if [ "$3" == "" ]; then
> +	cat "$TMPDIR/new-files"
> +	exit 0
> +fi
> +
> +if ! cat $FWINFO | head -n1 | grep "firmware:       " > /dev/null
> +then
> +	echo "Invliad fwinfo file $FWINFO"
> +	exit 1
> +fi
> +
> +declare -A oldbranch_files
> +while read of; do
> +	oldbranch_files["$of"]="old"
> +done < "$TMPDIR/oldbranch-files"
> +
> +while read dc1 ol dc2 of; do
> +	if [ -z "${oldbranch_files[$ol]:-}" ]; then
> +		oldbranch_files["$ol"]="link"
> +	fi
> +done < <(grep_links "$TMPDIR/oldbranch-files" "$OLDBRANCH")
> +
> +declare -A new_files
> +while read nf; do
> +	if [ -n "${oldbranch_files["$nf"]:-}" ]; then
> +		# new file but matches a previous link
> +		new_files["$nf"]="duplicated"
> +	elif [ -n "$(git log "$OLDBRANCH" -1 -- "$nf")" ]; then
> +		new_files["$nf"]="deleted"
> +	else
> +		new_files["$nf"]="unused"
> +	fi
> +done < "$TMPDIR/new-files"
> +
> +declare -A newlink_files
> +while read dc1 nl dc2 nf; do
> +
> +	nf="$(normalize_link "$(dirname "$nl")/$nf")"
> +	newlink_files["$nl"]="$nf"
> +done < <(grep_links "$TMPDIR/newbranch-files" "$NEWBRANCH")
> +
> +function check_duplicated {
> +	if [ -n "${oldbranch_files["$1"]:-}" ]; then
> +		return 0
> +	fi
> +
> +	if [ -n "${newlink_files["$1"]:-}" ]; then
> +		if check_duplicated "${newlink_files["$1"]}"; then
> +			return 0
> +		fi
> +	fi
> +
> +	return 1
> +}
> +
> +function mark_if_unused {
> +	if [ -n "${new_files["$1"]:-}" ]; then
> +		if [ "${new_files["$1"]}" = "unused" ]; then
> +			new_files["$1"]="$2"
> +		fi
> +
> +		return 0
> +	fi
> +
> +	if [ -n "${newlink_files["$1"]:-}" ]; then
> +		if mark_if_unused "${newlink_files["$1"]}" "$2"; then
> +			return 0
> +		fi
> +	fi
> +
> +	return 1
> +}
> +
> +while read fw_pattern; do
> +	fws=()
> +
> +	case "$fw_pattern" in
> +	*\**)
> +		# pattern with wildcard
> +		for nf in "${!new_files[@]}"; do
> +			case "$nf" in
> +			$fw_pattern)
> +				fws+=("$nf");;
> +			*)
> +				;;
> +			esac
> +		done
> +		;;
> +	*)
> +		fws+=("$fw_pattern")
> +		;;
> +	esac
> +
> +	for fw in "${fws[@]}"; do
> +	case "$fw" in
> +	iwlwifi-*.ucode)
> +		# check if new_files contains one with lower-equal FW API
> +		prefix="${fw%-*}"
> +		max_fw_api="${fw##*-}"
> +		max_fw_api="${max_fw_api%.ucode}"
> +		found=
> +		for api in $(seq $max_fw_api -1 1); do
> +			nf="$prefix-$api.ucode"
> +
> +			if [ -n "$found" ]; then
> +				mark_if_unused "$nf" "iwlwifi fwapi skipped" || true
> +				continue
> +			fi
> +
> +			if check_duplicated "$nf"; then
> +				mark_if_unused "$nf" "duplicated" || true
> +				found=1
> +			elif mark_if_unused "$nf" "new"; then
> +				found=1
> +			fi
> +		done
> +		;;
> +	*)
> +		if check_duplicated "$fw"; then
> +			mark_if_unused "$fw" "duplicated" || true
> +		else
> +			mark_if_unused "$fw" "new" || true
> +		fi
> +		;;
> +	esac
> +	done
> +done < <(cat "$FWINFO" | sed 's/firmware: \+//' | sort)
> +
> +declare -A ath_hidden_fw
> +for nf in "${!new_files[@]}"; do
> +	[ "${new_files[$nf]}" = "unused" ] || continue
> +
> +	case "$nf" in
> +	ath*k/*)
> +		model=${nf#*/}
> +		model=${model%%/*}
> +		if [ -z "${ath_hidden_fw[$model]}" ]; then
> +			if ! grep -q "/$model/" "$FWINFO"; then
> +				ath_hidden_fw["$model"]=1
> +			fi
> +		fi
> +
> +		# ath*k build firmware path at runtime, so set as new if not
> +		# previously marked duplicated.
> +		if [ -n "${ath_hidden_fw[$model]:-}" ]; then
> +			if check_duplicated "$nf"; then
> +				mark_if_unused "$nf" "duplicated" || true
> +			else
> +				mark_if_unused "$nf" "new" || true
> +			fi
> +		fi
> +		;;
> +	esac
> +done
> +
> +for nf in "${!new_files[@]}"; do
> +	[ "${new_files[$nf]}" = "new" ] && echo "$nf"
> +done | sort
> +
> +if [ -n "$dump_filtered" ]; then
> +	for nf in "${!new_files[@]}"; do
> +		[ "${new_files[$nf]}" = "new" ] || echo "# $nf: ${new_files[$nf]}"
> +	done | sort
> +fi
>
diff mbox series

Patch

diff --git a/debian/scripts/list-lts-update-files b/debian/scripts/list-lts-update-files
new file mode 100755
index 000000000..46fa3dfff
--- /dev/null
+++ b/debian/scripts/list-lts-update-files
@@ -0,0 +1,248 @@ 
+#!/bin/bash
+#
+# Used to get a list of firmware files needed to update a LTS
+# linux-firmware package for a backport kernel. For example to
+# get a list of files for trusty to support the linux-lts-xenial
+# kernel run:
+#
+#  debian/scripts/list-lts-update-files [--dump-filtered] trusty xenial /path/to/xenial/fwinfo
+#
+# This will generate a list of files that satisfy all of these
+# conditions:
+#
+#  1) In the xenial branch of linux-firmware
+#  2) Not in the trusty branch of linux-firmware
+#  3) In the specified fwinfo file
+#
+# This can be used as a starting point for finding upstream
+# commits needed to add the missing firmware files.
+#
+# The fwinfo file can be omitted, in which case the output is a list
+# of firmware files in @newbranch but not in @oldbranch.
+
+#set -ex
+
+dump_filtered=
+if [ $# -ge 1 ]; then
+	case "$1" in
+	--dump-filtered)
+		dump_filtered=1; shift;;
+	*) ;;
+	esac
+fi
+
+if [ $# -lt 2 ]; then
+	echo "Usage $0 [--dump-filtered] <oldbranch> <newbranch> [<fwinfo>]"
+	exit 1
+fi
+
+OLDBRANCH="$1"
+NEWBRANCH="$2"
+FWINFO="$3"
+
+TMPDIR=$(mktemp -d)
+
+function cleanup {
+	rm -r "$TMPDIR"
+}
+trap cleanup EXIT
+
+function normalize_link {
+	# handle relative paths: a/b/c/../../d/e -> a/d/e
+	local s="$1"
+	s="${s#./}"
+	while true; do
+		case "$s" in
+		*/../*)
+			s=$(echo "$s" | sed 's,[^/]\+/../,,')
+			;;
+		*)
+			break ;;
+		esac
+	done
+	echo "$s"
+}
+
+function grep_links {
+	grep ^WHENCE "$1" | sed "s,^,$2:," | xargs git show | grep ^Link:
+}
+
+if ! git ls-tree -r --name-only "$OLDBRANCH" >"$TMPDIR/files.tmp"
+then
+	exit 1
+fi
+cat "$TMPDIR/files.tmp" | sort >"$TMPDIR/oldbranch-files"
+
+if ! git ls-tree -r --name-only "$NEWBRANCH" >"$TMPDIR/files.tmp"
+then
+	exit 1
+fi
+cat "$TMPDIR/files.tmp" | sort >"$TMPDIR/newbranch-files"
+
+# Get files in @newbranch not in @oldbranch
+comm -13 "$TMPDIR/oldbranch-files" "$TMPDIR/newbranch-files" >"$TMPDIR/new-files"
+
+if [ "$3" == "" ]; then
+	cat "$TMPDIR/new-files"
+	exit 0
+fi
+
+if ! cat $FWINFO | head -n1 | grep "firmware:       " > /dev/null
+then
+	echo "Invliad fwinfo file $FWINFO"
+	exit 1
+fi
+
+declare -A oldbranch_files
+while read of; do
+	oldbranch_files["$of"]="old"
+done < "$TMPDIR/oldbranch-files"
+
+while read dc1 ol dc2 of; do
+	if [ -z "${oldbranch_files[$ol]:-}" ]; then
+		oldbranch_files["$ol"]="link"
+	fi
+done < <(grep_links "$TMPDIR/oldbranch-files" "$OLDBRANCH")
+
+declare -A new_files
+while read nf; do
+	if [ -n "${oldbranch_files["$nf"]:-}" ]; then
+		# new file but matches a previous link
+		new_files["$nf"]="duplicated"
+	elif [ -n "$(git log "$OLDBRANCH" -1 -- "$nf")" ]; then
+		new_files["$nf"]="deleted"
+	else
+		new_files["$nf"]="unused"
+	fi
+done < "$TMPDIR/new-files"
+
+declare -A newlink_files
+while read dc1 nl dc2 nf; do
+
+	nf="$(normalize_link "$(dirname "$nl")/$nf")"
+	newlink_files["$nl"]="$nf"
+done < <(grep_links "$TMPDIR/newbranch-files" "$NEWBRANCH")
+
+function check_duplicated {
+	if [ -n "${oldbranch_files["$1"]:-}" ]; then
+		return 0
+	fi
+
+	if [ -n "${newlink_files["$1"]:-}" ]; then
+		if check_duplicated "${newlink_files["$1"]}"; then
+			return 0
+		fi
+	fi
+
+	return 1
+}
+
+function mark_if_unused {
+	if [ -n "${new_files["$1"]:-}" ]; then
+		if [ "${new_files["$1"]}" = "unused" ]; then
+			new_files["$1"]="$2"
+		fi
+
+		return 0
+	fi
+
+	if [ -n "${newlink_files["$1"]:-}" ]; then
+		if mark_if_unused "${newlink_files["$1"]}" "$2"; then
+			return 0
+		fi
+	fi
+
+	return 1
+}
+
+while read fw_pattern; do
+	fws=()
+
+	case "$fw_pattern" in
+	*\**)
+		# pattern with wildcard
+		for nf in "${!new_files[@]}"; do
+			case "$nf" in
+			$fw_pattern)
+				fws+=("$nf");;
+			*)
+				;;
+			esac
+		done
+		;;
+	*)
+		fws+=("$fw_pattern")
+		;;
+	esac
+
+	for fw in "${fws[@]}"; do
+	case "$fw" in
+	iwlwifi-*.ucode)
+		# check if new_files contains one with lower-equal FW API
+		prefix="${fw%-*}"
+		max_fw_api="${fw##*-}"
+		max_fw_api="${max_fw_api%.ucode}"
+		found=
+		for api in $(seq $max_fw_api -1 1); do
+			nf="$prefix-$api.ucode"
+
+			if [ -n "$found" ]; then
+				mark_if_unused "$nf" "iwlwifi fwapi skipped" || true
+				continue
+			fi
+
+			if check_duplicated "$nf"; then
+				mark_if_unused "$nf" "duplicated" || true
+				found=1
+			elif mark_if_unused "$nf" "new"; then
+				found=1
+			fi
+		done
+		;;
+	*)
+		if check_duplicated "$fw"; then
+			mark_if_unused "$fw" "duplicated" || true
+		else
+			mark_if_unused "$fw" "new" || true
+		fi
+		;;
+	esac
+	done
+done < <(cat "$FWINFO" | sed 's/firmware: \+//' | sort)
+
+declare -A ath_hidden_fw
+for nf in "${!new_files[@]}"; do
+	[ "${new_files[$nf]}" = "unused" ] || continue
+
+	case "$nf" in
+	ath*k/*)
+		model=${nf#*/}
+		model=${model%%/*}
+		if [ -z "${ath_hidden_fw[$model]}" ]; then
+			if ! grep -q "/$model/" "$FWINFO"; then
+				ath_hidden_fw["$model"]=1
+			fi
+		fi
+
+		# ath*k build firmware path at runtime, so set as new if not
+		# previously marked duplicated.
+		if [ -n "${ath_hidden_fw[$model]:-}" ]; then
+			if check_duplicated "$nf"; then
+				mark_if_unused "$nf" "duplicated" || true
+			else
+				mark_if_unused "$nf" "new" || true
+			fi
+		fi
+		;;
+	esac
+done
+
+for nf in "${!new_files[@]}"; do
+	[ "${new_files[$nf]}" = "new" ] && echo "$nf"
+done | sort
+
+if [ -n "$dump_filtered" ]; then
+	for nf in "${!new_files[@]}"; do
+		[ "${new_files[$nf]}" = "new" ] || echo "# $nf: ${new_files[$nf]}"
+	done | sort
+fi