[0/1,bionic:linux] nr_writeback memory leak in kernel 4.15.0-137+

BugLink: https://bugs.launchpad.net/bugs/1926081

[Impact]

Ubuntu 18.04.5 4.15.0 LTS kernels at version 4.15.0-137 and above contain a
memory leak due to the inclusion of patch from the upstream kernel, but not
the fix for that patch which was released later.

Bad patch in bionic:linux 2c17fa778db85644458b52a7df8eacc402cbc1ef mm:
memcontrol: fix excessive complexity in memory.stat reporting

[Test Plan]

See bug report for extended discussion. Test results are positive.

The following is enough to permanently increase the value of `nr_writeback`.

cat > breakme.sh <<EOF
date
grep nr_writeback /proc/vmstat
mkdir -p /docker/testfiles/{1..5}

seq -w 1 100000 | xargs -n1 -I% sh -c 'dd if=/dev/urandom of=/docker/testfiles/1/file.% bs=4k count=10 status=none' &
seq -w 1 100000 | xargs -n1 -I% sh -c 'dd if=/dev/urandom of=/docker/testfiles/2/file.% bs=4k count=10 status=none' &
seq -w 1 100000 | xargs -n1 -I% sh -c 'dd if=/dev/urandom of=/docker/testfiles/3/file.% bs=4k count=10 status=none' &
seq -w 1 100000 | xargs -n1 -I% sh -c 'dd if=/dev/urandom of=/docker/testfiles/4/file.% bs=4k count=10 status=none' &
seq -w 1 100000 | xargs -n1 -I% sh -c 'dd if=/dev/urandom of=/docker/testfiles/5/file.% bs=4k count=10 status=none' &

wait $(jobs -p)
grep nr_writeback /proc/vmstat
date
EOF

[Where problems could occur]

Memory leakage could continue. The new spinlocks could cause some performance degradation
or even deadlock.

[Other Info]

These patches have been accepted to v4.14.y

On Wed, Apr 28, 2021 at 05:58:49AM -0600, Tim Gardner wrote:
> BugLink: https://bugs.launchpad.net/bugs/1926081
> 
> [Impact]
> 
> Ubuntu 18.04.5 4.15.0 LTS kernels at version 4.15.0-137 and above contain a
> memory leak due to the inclusion of patch from the upstream kernel, but not
> the fix for that patch which was released later.
> 
> Bad patch in bionic:linux 2c17fa778db85644458b52a7df8eacc402cbc1ef mm:
> memcontrol: fix excessive complexity in memory.stat reporting
> 
> [Test Plan]
> 
> See bug report for extended discussion. Test results are positive.
> 
> The following is enough to permanently increase the value of `nr_writeback`.
> 
> cat > breakme.sh <<EOF
> date
> grep nr_writeback /proc/vmstat
> mkdir -p /docker/testfiles/{1..5}
> 
> seq -w 1 100000 | xargs -n1 -I% sh -c 'dd if=/dev/urandom of=/docker/testfiles/1/file.% bs=4k count=10 status=none' &
> seq -w 1 100000 | xargs -n1 -I% sh -c 'dd if=/dev/urandom of=/docker/testfiles/2/file.% bs=4k count=10 status=none' &
> seq -w 1 100000 | xargs -n1 -I% sh -c 'dd if=/dev/urandom of=/docker/testfiles/3/file.% bs=4k count=10 status=none' &
> seq -w 1 100000 | xargs -n1 -I% sh -c 'dd if=/dev/urandom of=/docker/testfiles/4/file.% bs=4k count=10 status=none' &
> seq -w 1 100000 | xargs -n1 -I% sh -c 'dd if=/dev/urandom of=/docker/testfiles/5/file.% bs=4k count=10 status=none' &
> 
> wait $(jobs -p)
> grep nr_writeback /proc/vmstat
> date
> EOF
> 
> [Where problems could occur]
> 
> Memory leakage could continue. The new spinlocks could cause some performance degradation
> or even deadlock.
> 
> [Other Info]
> 
> These patches have been accepted to v4.14.y
> 

There is a nice test case here. These are clean cherry picks.

I see two fixups for the first patch, though:

e81bf9793b1861d74953ef041b4f6c7faecc2dbd ("mem_cgroup: make sure moving_account, move_lock_task and stat_cpu in the same cacheline")
fe6bdfc8e1e131720abbe77a2eb990c94c9024cb ("mm: fix oom_kill event handling")

The first one seems fine to apply.

The seconde one is a nice to have.

But I wonder if we should reconsider the approach and keep bionic as it is, and
skip the first commit. It doesn't seem to me that reverting the backport of
a983b5ebee57209c99f68c8327072f25e0e6e3da would be fine, as it fixes a big
scalability problem.

So, I would rather add these two extra commits than apply these two as is.

Thanks.
Cascardo.

On Wed, Apr 28, 2021 at 8:59 AM Tim Gardner <tim.gardner@canonical.com> wrote:
>
> BugLink: https://bugs.launchpad.net/bugs/1926081
>
> [Impact]
>
> Ubuntu 18.04.5 4.15.0 LTS kernels at version 4.15.0-137 and above contain a
> memory leak due to the inclusion of patch from the upstream kernel, but not
> the fix for that patch which was released later.
>
> Bad patch in bionic:linux 2c17fa778db85644458b52a7df8eacc402cbc1ef mm:
> memcontrol: fix excessive complexity in memory.stat reporting
>
> [Test Plan]
>
> See bug report for extended discussion. Test results are positive.
>
> The following is enough to permanently increase the value of `nr_writeback`.
>
> cat > breakme.sh <<EOF
> date
> grep nr_writeback /proc/vmstat
> mkdir -p /docker/testfiles/{1..5}
>
> seq -w 1 100000 | xargs -n1 -I% sh -c 'dd if=/dev/urandom of=/docker/testfiles/1/file.% bs=4k count=10 status=none' &
> seq -w 1 100000 | xargs -n1 -I% sh -c 'dd if=/dev/urandom of=/docker/testfiles/2/file.% bs=4k count=10 status=none' &
> seq -w 1 100000 | xargs -n1 -I% sh -c 'dd if=/dev/urandom of=/docker/testfiles/3/file.% bs=4k count=10 status=none' &
> seq -w 1 100000 | xargs -n1 -I% sh -c 'dd if=/dev/urandom of=/docker/testfiles/4/file.% bs=4k count=10 status=none' &
> seq -w 1 100000 | xargs -n1 -I% sh -c 'dd if=/dev/urandom of=/docker/testfiles/5/file.% bs=4k count=10 status=none' &
>
> wait $(jobs -p)
> grep nr_writeback /proc/vmstat
> date
> EOF
>
> [Where problems could occur]
>
> Memory leakage could continue. The new spinlocks could cause some performance degradation
> or even deadlock.
>
> [Other Info]
>
> These patches have been accepted to v4.14.y
>
>

Thanks Tim! LGTM:
Acked-by: Guilherme G. Piccoli <gpiccoli@canonical.com>

On 4/28/21 5:58 AM, Tim Gardner wrote:
> BugLink: https://bugs.launchpad.net/bugs/1926081
> 
> [Impact]
> 
> Ubuntu 18.04.5 4.15.0 LTS kernels at version 4.15.0-137 and above contain a
> memory leak due to the inclusion of patch from the upstream kernel, but not
> the fix for that patch which was released later.
> 
> Bad patch in bionic:linux 2c17fa778db85644458b52a7df8eacc402cbc1ef mm:
> memcontrol: fix excessive complexity in memory.stat reporting
> 
> [Test Plan]
> 
> See bug report for extended discussion. Test results are positive.
> 
> The following is enough to permanently increase the value of `nr_writeback`.
> 
> cat > breakme.sh <<EOF
> date
> grep nr_writeback /proc/vmstat
> mkdir -p /docker/testfiles/{1..5}
> 
> seq -w 1 100000 | xargs -n1 -I% sh -c 'dd if=/dev/urandom of=/docker/testfiles/1/file.% bs=4k count=10 status=none' &
> seq -w 1 100000 | xargs -n1 -I% sh -c 'dd if=/dev/urandom of=/docker/testfiles/2/file.% bs=4k count=10 status=none' &
> seq -w 1 100000 | xargs -n1 -I% sh -c 'dd if=/dev/urandom of=/docker/testfiles/3/file.% bs=4k count=10 status=none' &
> seq -w 1 100000 | xargs -n1 -I% sh -c 'dd if=/dev/urandom of=/docker/testfiles/4/file.% bs=4k count=10 status=none' &
> seq -w 1 100000 | xargs -n1 -I% sh -c 'dd if=/dev/urandom of=/docker/testfiles/5/file.% bs=4k count=10 status=none' &
> 
> wait $(jobs -p)
> grep nr_writeback /proc/vmstat
> date
> EOF
> 
> [Where problems could occur]
> 
> Memory leakage could continue. The new spinlocks could cause some performance degradation
> or even deadlock.
> 
> [Other Info]
> 
> These patches have been accepted to v4.14.y
> 
Respinning with an additional 2 commits based on feedback from Thadeu 
Lima de Souza Cascardo <cascardo@canonical.com>

rtg
-----------
Tim Gardner
Canonical, Inc