From patchwork Thu Jul 17 11:17:11 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Mason X-Patchwork-Id: 371066 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 3BE7814008B for ; Thu, 17 Jul 2014 21:16:28 +1000 (EST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756294AbaGQLQ1 (ORCPT ); Thu, 17 Jul 2014 07:16:27 -0400 Received: from smtp23.services.sfr.fr ([93.17.128.19]:42621 "EHLO smtp23.services.sfr.fr" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754379AbaGQLQZ (ORCPT ); Thu, 17 Jul 2014 07:16:25 -0400 Received: from filter.sfr.fr (localhost [79.84.138.1]) by msfrf2319.sfr.fr (SMTP Server) with ESMTP id 4214070000E9; Thu, 17 Jul 2014 13:16:23 +0200 (CEST) Authentication-Results: sfrmc.priv.atos.fr; dkim=none (no signature); dkim-adsp=none (no policy) header.from=mpeg.blue@free.fr Received: from [223.200.200.29] (1.138.84.79.rev.sfr.net [79.84.138.1]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by msfrf2319.sfr.fr (SMTP Server) with ESMTP id 8436D7000043; Thu, 17 Jul 2014 13:16:21 +0200 (CEST) X-SFR-UUID: 20140717111621541.8436D7000043@msfrf2319.sfr.fr Message-ID: <53C7B0B7.9030007@free.fr> Date: Thu, 17 Jul 2014 13:17:11 +0200 From: Mason User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:29.0) Gecko/20100101 Firefox/29.0 SeaMonkey/2.26.1 MIME-Version: 1.0 To: =?iso-8859-2?b?THVr4bk=?= Czerner CC: Andreas Dilger , Ext4 Developers List , linux-fsdevel Subject: Re: After unlinking a large file on ext4, the process stalls for a long time References: <53C687B1.30809@free.fr> <21446.38705.190786.631403@quad.stoffel.home> <53C6B38A.3000100@free.fr> <59C3F41A-6AFD-418E-BCE6-2361B8140D9A@dilger.ca> <53C7A5CA.4050903@free.fr> In-Reply-To: Sender: linux-ext4-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org Lukáš Czerner wrote: > So it really does not seem to be stalling in fallocate, nor unlink. > Can you add close() before unlink, just to be sure what's happening > there ? Doh! Good catch! Unlinking was fast because the ref count didn't drop to 0 on unlink, it did so on the implicit close done on exit, which would explain why the process stalled "at the end". If I unlink a closed file, it is indeed unlink that stalls. [BTW, some of the e2fsprogs devs may be reading this. I suppose you already know, but the cross-compile build was broken in 1.4.10. I wrote a trivial patch to fix it (cf. the end of this message) although I'm not sure I did it the canonical way.] # time strace -T ./foo /mnt/hdd/xxx 300 2> strace.out posix_fallocate(fd, 0, size_in_GiB << 30): 0 [412 ms] close(fd): 0 [0 ms] unlink(filename): 0 [111481 ms] open("/mnt/hdd/xxx", O_WRONLY|O_CREAT|O_EXCL|O_LARGEFILE, 0600) = 3 <0.000456> clock_gettime(CLOCK_MONOTONIC, {82152, 251657385}) = 0 <0.000085> SYS_4320() = 0 <0.411628> clock_gettime(CLOCK_MONOTONIC, {82152, 664179762}) = 0 <0.000089> fstat64(1, {st_mode=S_IFCHR|0755, st_rdev=makedev(4, 64), ...}) = 0 <0.000094> ioctl(1, TIOCNXCL, {B115200 opost isig icanon echo ...}) = 0 <0.000128> old_mmap(NULL, 65536, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x773e4000 <0.000195> write(1, "posix_fallocate(fd, 0, size_in_G"..., 54) = 54 <0.000281> clock_gettime(CLOCK_MONOTONIC, {82152, 668413115}) = 0 <0.000077> close(3) = 0 <0.000119> clock_gettime(CLOCK_MONOTONIC, {82152, 669249479}) = 0 <0.000129> write(1, "close(fd): 0 [0 ms]\n", 20) = 20 <0.000145> clock_gettime(CLOCK_MONOTONIC, {82152, 670361133}) = 0 <0.000078> unlink("/mnt/hdd/xxx") = 0 <111.479283> clock_gettime(CLOCK_MONOTONIC, {82264, 150551496}) = 0 <0.000080> write(1, "unlink(filename): 0 [111481 ms]\n", 32) = 32 <0.000225> exit_group(0) = ? 0.01user 111.48system 1:51.99elapsed 99%CPU (0avgtext+0avgdata 772maxresident)k 0inputs+0outputs (0major+434minor)pagefaults 0swaps For reference, here's my minimal test case: #define _FILE_OFFSET_BITS 64 #include #include #include #include #include #define BENCH(op) do { \ struct timespec t0; clock_gettime(CLOCK_MONOTONIC, &t0); \ int err = op; \ struct timespec t1; clock_gettime(CLOCK_MONOTONIC, &t1); \ int ms = (t1.tv_sec-t0.tv_sec)*1000 + (t1.tv_nsec-t0.tv_nsec)/1000000; \ printf("%s: %d [%d ms]\n", #op, err, ms); } while(0) int main(int argc, char **argv) { if (argc != 3) { puts("Usage: prog filename size"); return 42; } char *filename = argv[1]; int fd = open(filename, O_CREAT | O_EXCL | O_WRONLY, 0600); if (fd < 0) { perror("open"); return 1; } long long size_in_GiB = atoi(argv[2]); BENCH(posix_fallocate(fd, 0, size_in_GiB << 30)); BENCH(close(fd)); BENCH(unlink(filename)); return 0; } $ cat e2fsprogs-1.42.10.patch diff -ur a/util/Makefile.in b/util/Makefile.in --- a/util/Makefile.in 2014-05-15 19:04:08.000000000 +0200 +++ b/util/Makefile.in 2014-07-10 15:31:04.819352596 +0200 @@ -15,7 +15,7 @@ .c.o: $(E) " CC $<" - $(Q) $(BUILD_CC) -c $(BUILD_CFLAGS) $< -o $@ + $(Q) $(BUILD_CC) $(CPPFLAGS) -c $(BUILD_CFLAGS) $< -o $@ $(Q) $(CHECK_CMD) $(ALL_CFLAGS) $< PROGS= subst symlinks