| Message ID | alpine.LFD.2.02.1103081849430.6414@zhanghuan.nrchpc.ac.cn |
|---|---|
| State | Accepted, archived |
| Headers | show |
On Tue, 8 Mar 2011, Zhang Huan wrote: > Hi all, > > There is potential memory leak of journal head in function > jbd2_journal_commit_transaction. The problem is that JBD2 will not > reclaim the journal head of commit record if error occurs or journal is > abotred. > > I use the following script to reproduce this issue, on a RHEL6 system. I > found it very easy to reproduce with async commit enabled. > > mount /dev/sdb /mnt -o journal_checksum,journal_async_commit > touch /mnt/xxx > echo offline > /sys/block/sdb/device/state > sync > umount /mnt > rmmod ext4 > rmmod jbd2 > > Removal of the jbd2 module will make slab complaining that > "cache `jbd2_journal_head': can't free all objects". > > > Here is my fix for this issue. The commit record should be reclaimed no > matter error occurs or not. > > diff --git a/fs/jbd2/commit.c b/fs/jbd2/commit.c > index f3ad159..37a973a 100644 > --- a/fs/jbd2/commit.c > +++ b/fs/jbd2/commit.c > @@ -105,6 +105,8 @@ static int journal_submit_commit_record(journal_t *journal, > int ret; > struct timespec now = current_kernel_time(); > > + *cbh = NULL; > + > if (is_journal_aborted(journal)) > return 0; > > @@ -808,7 +810,7 @@ wait_for_iobuf: > if (err) > __jbd2_journal_abort_hard(journal); > } > - if (!err && !is_journal_aborted(journal)) > + if (cbh) Hi, I wonder if we could do rather this: if (!err && !is_journal_aborted(journal)) err = journal_wait_on_commit_record(journal, cbh); else if (cbh) { put_bh(cbh); jbd2_journal_put_journal_head(bh2jh(cbh)); } I think this is more readable... Thanks! -Lukas > err = journal_wait_on_commit_record(journal, cbh); > if (JBD2_HAS_INCOMPAT_FEATURE(journal, > JBD2_FEATURE_INCOMPAT_ASYNC_COMMIT) && > > > > PS: Just out of curiosity, why would journal_submit_commit_record return a > value of 1 instead of an error number if get descriptor buffer is failed. > > > Zhang Huan > -- > To unsubscribe from this list: send the line "unsubscribe linux-ext4" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html >
On Tue, 8 Mar 2011, Lukas Czerner wrote: > On Tue, 8 Mar 2011, Zhang Huan wrote: > >> Hi all, >> >> There is potential memory leak of journal head in function >> jbd2_journal_commit_transaction. The problem is that JBD2 will not >> reclaim the journal head of commit record if error occurs or journal is >> abotred. >> >> I use the following script to reproduce this issue, on a RHEL6 system. I >> found it very easy to reproduce with async commit enabled. >> >> mount /dev/sdb /mnt -o journal_checksum,journal_async_commit >> touch /mnt/xxx >> echo offline > /sys/block/sdb/device/state >> sync >> umount /mnt >> rmmod ext4 >> rmmod jbd2 >> >> Removal of the jbd2 module will make slab complaining that >> "cache `jbd2_journal_head': can't free all objects". >> >> >> Here is my fix for this issue. The commit record should be reclaimed no >> matter error occurs or not. >> >> diff --git a/fs/jbd2/commit.c b/fs/jbd2/commit.c >> index f3ad159..37a973a 100644 >> --- a/fs/jbd2/commit.c >> +++ b/fs/jbd2/commit.c >> @@ -105,6 +105,8 @@ static int journal_submit_commit_record(journal_t *journal, >> int ret; >> struct timespec now = current_kernel_time(); >> >> + *cbh = NULL; >> + >> if (is_journal_aborted(journal)) >> return 0; >> >> @@ -808,7 +810,7 @@ wait_for_iobuf: >> if (err) >> __jbd2_journal_abort_hard(journal); >> } >> - if (!err && !is_journal_aborted(journal)) >> + if (cbh) > > Hi, > > I wonder if we could do rather this: > > if (!err && !is_journal_aborted(journal)) > err = journal_wait_on_commit_record(journal, cbh); > else if (cbh) { > put_bh(cbh); > jbd2_journal_put_journal_head(bh2jh(cbh)); > } > > I think this is more readable... Hi, I don't think it is a good idea. Anyway, you need to wait for buffer to complete, and then release journal head on it. That is exactly what journal_wait_on_commit_record does. > > Thanks! > -Lukas > >> err = journal_wait_on_commit_record(journal, cbh); >> if (JBD2_HAS_INCOMPAT_FEATURE(journal, >> JBD2_FEATURE_INCOMPAT_ASYNC_COMMIT) && >> >> >> >> PS: Just out of curiosity, why would journal_submit_commit_record return a >> value of 1 instead of an error number if get descriptor buffer is failed. >> >> >> Zhang Huan >> -- >> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in >> the body of a message to majordomo@vger.kernel.org >> More majordomo info at http://vger.kernel.org/majordomo-info.html >> > > -- > Zhang Huan -- To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Tue, 8 Mar 2011, Zhang Huan wrote: > > On Tue, 8 Mar 2011, Lukas Czerner wrote: > > > On Tue, 8 Mar 2011, Zhang Huan wrote: > > > > > Hi all, > > > > > > There is potential memory leak of journal head in function > > > jbd2_journal_commit_transaction. The problem is that JBD2 will not > > > reclaim the journal head of commit record if error occurs or journal is > > > abotred. > > > > > > I use the following script to reproduce this issue, on a RHEL6 system. I > > > found it very easy to reproduce with async commit enabled. > > > > > > mount /dev/sdb /mnt -o journal_checksum,journal_async_commit > > > touch /mnt/xxx > > > echo offline > /sys/block/sdb/device/state > > > sync > > > umount /mnt > > > rmmod ext4 > > > rmmod jbd2 > > > > > > Removal of the jbd2 module will make slab complaining that > > > "cache `jbd2_journal_head': can't free all objects". > > > > > > > > > Here is my fix for this issue. The commit record should be reclaimed no > > > matter error occurs or not. > > > > > > diff --git a/fs/jbd2/commit.c b/fs/jbd2/commit.c > > > index f3ad159..37a973a 100644 > > > --- a/fs/jbd2/commit.c > > > +++ b/fs/jbd2/commit.c > > > @@ -105,6 +105,8 @@ static int journal_submit_commit_record(journal_t > > > *journal, > > > int ret; > > > struct timespec now = current_kernel_time(); > > > > > > + *cbh = NULL; > > > + > > > if (is_journal_aborted(journal)) > > > return 0; > > > > > > @@ -808,7 +810,7 @@ wait_for_iobuf: > > > if (err) > > > __jbd2_journal_abort_hard(journal); > > > } > > > - if (!err && !is_journal_aborted(journal)) > > > + if (cbh) > > > > Hi, > > > > I wonder if we could do rather this: > > > > if (!err && !is_journal_aborted(journal)) > > err = journal_wait_on_commit_record(journal, cbh); > > else if (cbh) { > > put_bh(cbh); > > jbd2_journal_put_journal_head(bh2jh(cbh)); > > } > > > > I think this is more readable... > > Hi, > > I don't think it is a good idea. Anyway, you need to wait for buffer to > complete, and then release journal head on it. That is exactly what > journal_wait_on_commit_record does. Oh, I see. Ok then ignore me :) Thanks! -Lukas > > > > > Thanks! > > -Lukas > > > > > err = journal_wait_on_commit_record(journal, cbh); > > > if (JBD2_HAS_INCOMPAT_FEATURE(journal, > > > JBD2_FEATURE_INCOMPAT_ASYNC_COMMIT) && > > > > > > > > > > > > PS: Just out of curiosity, why would journal_submit_commit_record return a > > > value of 1 instead of an error number if get descriptor buffer is failed. > > > > > > > > > Zhang Huan > > > -- > > > To unsubscribe from this list: send the line "unsubscribe linux-ext4" in > > > the body of a message to majordomo@vger.kernel.org > > > More majordomo info at http://vger.kernel.org/majordomo-info.html > > > > > > > -- > > > > Zhang Huan >
Hi, I would like to know if there is more comment on this? On Tue, Mar 8, 2011 at 19:40, Zhang Huan <zhhuan@gmail.com> wrote: > Hi all, > > There is potential memory leak of journal head in function > jbd2_journal_commit_transaction. The problem is that JBD2 will not > reclaim the journal head of commit record if error occurs or journal is > abotred. > > I use the following script to reproduce this issue, on a RHEL6 system. I > found it very easy to reproduce with async commit enabled. > > mount /dev/sdb /mnt -o journal_checksum,journal_async_commit > touch /mnt/xxx > echo offline > /sys/block/sdb/device/state > sync > umount /mnt > rmmod ext4 > rmmod jbd2 > > Removal of the jbd2 module will make slab complaining that > "cache `jbd2_journal_head': can't free all objects". > > > Here is my fix for this issue. The commit record should be reclaimed no > matter error occurs or not. > > diff --git a/fs/jbd2/commit.c b/fs/jbd2/commit.c > index f3ad159..37a973a 100644 > --- a/fs/jbd2/commit.c > +++ b/fs/jbd2/commit.c > @@ -105,6 +105,8 @@ static int journal_submit_commit_record(journal_t *journal, > int ret; > struct timespec now = current_kernel_time(); > > + *cbh = NULL; > + > if (is_journal_aborted(journal)) > return 0; > > @@ -808,7 +810,7 @@ wait_for_iobuf: > if (err) > __jbd2_journal_abort_hard(journal); > } > - if (!err && !is_journal_aborted(journal)) > + if (cbh) > err = journal_wait_on_commit_record(journal, cbh); > if (JBD2_HAS_INCOMPAT_FEATURE(journal, > JBD2_FEATURE_INCOMPAT_ASYNC_COMMIT) && > > > > PS: Just out of curiosity, why would journal_submit_commit_record return a > value of 1 instead of an error number if get descriptor buffer is failed. > > > Zhang Huan > Zhang Huan -- To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Hi Zhang, Sorry for the delay in looking at it. One thing which is missing from your patch is a signed-off-by header. From section 12 of the Documentation/SubmittingPatches file in the Linux source tree: >The sign-off is a simple line at the end of the explanation for the >patch, which certifies that you wrote it or otherwise have the right to >pass it on as a open-source patch. The rules are pretty simple: if you >can certify the below: > > Developer's Certificate of Origin 1.1 > > By making a contribution to this project, I certify that: > > (a) The contribution was created in whole or in part by me and I > have the right to submit it under the open source license > indicated in the file; or > > (b) The contribution is based upon previous work that, to the best > of my knowledge, is covered under an appropriate open source > license and I have the right under that license to submit that > work with modifications, whether created in whole or in part > by me, under the same open source license (unless I am > permitted to submit under a different license), as indicated > in the file; or > > (c) The contribution was provided directly to me by some other > person who certified (a), (b) or (c) and I have not modified > it. > > (d) I understand and agree that this project and the contribution > are public and that a record of the contribution (including all > personal information I submit with it, including my sign-off) is > maintained indefinitely and may be redistributed consistent with > this project or the open source license(s) involved. > >then you just add a line saying > > Signed-off-by: Random J Developer <random@developer.example.org> > >using your real name (sorry, no pseudonyms or anonymous contributions.) If you could send me an acknowledgement that it would be OK to add a Signed-off-by: Zhang Huan <zhhuan@gmail.com> line to your patch, it would be much appreciated. - Ted -- To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Tue, Apr 5, 2011 at 06:10, Ted Ts'o <tytso@mit.edu> wrote: > > If you could send me an acknowledgement that it would be OK to add a > > Signed-off-by: Zhang Huan <zhhuan@gmail.com> > > line to your patch, it would be much appreciated. > Yes. please go ahead. Zhang Huan -- To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Tue, Apr 05, 2011 at 08:35:35PM +0800, Zhang Huan wrote: > On Tue, Apr 5, 2011 at 06:10, Ted Ts'o <tytso@mit.edu> wrote: > > > > If you could send me an acknowledgement that it would be OK to add a > > > > Signed-off-by: Zhang Huan <zhhuan@gmail.com> > > > > line to your patch, it would be much appreciated. > > > > Yes. please go ahead. Thanks, I've added your patch to the ext4 patch queue. - Ted -- To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
diff --git a/fs/jbd2/commit.c b/fs/jbd2/commit.c index f3ad159..37a973a 100644 --- a/fs/jbd2/commit.c +++ b/fs/jbd2/commit.c @@ -105,6 +105,8 @@ static int journal_submit_commit_record(journal_t *journal, int ret; struct timespec now = current_kernel_time(); + *cbh = NULL; + if (is_journal_aborted(journal)) return 0; @@ -808,7 +810,7 @@ wait_for_iobuf: if (err) __jbd2_journal_abort_hard(journal); } - if (!err && !is_journal_aborted(journal)) + if (cbh) err = journal_wait_on_commit_record(journal, cbh); if (JBD2_HAS_INCOMPAT_FEATURE(journal, JBD2_FEATURE_INCOMPAT_ASYNC_COMMIT) &&