From patchwork Tue Sep 1 12:31:08 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Martin_Li=C5=A1ka?= X-Patchwork-Id: 1354927 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=2620:52:3:1:0:246e:9693:128c; helo=sourceware.org; envelope-from=gcc-patches-bounces@gcc.gnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=suse.cz Received: from sourceware.org (server2.sourceware.org [IPv6:2620:52:3:1:0:246e:9693:128c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4Bgmdh063Pz9sTM for ; Tue, 1 Sep 2020 22:31:18 +1000 (AEST) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 3F5EB3842438; Tue, 1 Sep 2020 12:31:15 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from mx2.suse.de (mx2.suse.de [195.135.220.15]) by sourceware.org (Postfix) with ESMTPS id BFE2D385783D for ; Tue, 1 Sep 2020 12:31:09 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org BFE2D385783D Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=suse.cz Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=mliska@suse.cz X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id 3743FAF34; Tue, 1 Sep 2020 12:31:09 +0000 (UTC) To: GCC Patches From: =?utf-8?q?Martin_Li=C5=A1ka?= Subject: [PATCH 1/3] Support new mallinfo2 function. Message-ID: Date: Tue, 1 Sep 2020 14:31:08 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.11.0 MIME-Version: 1.0 Content-Language: en-US X-Spam-Status: No, score=-9.7 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_STATUS, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Jan Hubicka Errors-To: gcc-patches-bounces@gcc.gnu.org Sender: "Gcc-patches" Hey. I've just applied to patches to glibc introducing a new mallinfo2 function. Limitation of the current function mallinfo is usage of int type which overflows for allocation > 2GB. The patch adds configure detection and usage of the new one. And it prints heap usage in MiB. Ready to be installed after tests? Thanks, Martin From 031ae63018a32c9ac10fab58f4a6c8b5a9166060 Mon Sep 17 00:00:00 2001 From: Martin Liska Date: Tue, 1 Sep 2020 14:14:45 +0200 Subject: [PATCH 1/3] Support new mallinfo2 function. gcc/ChangeLog: * config.in: Regenerate. * configure: Likewise. * configure.ac: Detect for mallinfo2. * ggc-common.c (defined): Use it and display info in MiB. * system.h: Handle also HAVE_MALLINFO2. --- gcc/config.in | 16 ++++++++++++++-- gcc/configure | 4 ++-- gcc/configure.ac | 4 ++-- gcc/ggc-common.c | 12 +++++++++--- gcc/system.h | 2 +- 5 files changed, 28 insertions(+), 10 deletions(-) diff --git a/gcc/config.in b/gcc/config.in index 478e74fac02..1832c112ed9 100644 --- a/gcc/config.in +++ b/gcc/config.in @@ -983,13 +983,19 @@ #endif -/* Define to 1 if we found a declaration for 'mallinfo', otherwise define to - 0. */ +/* Define to 1 if we found a declaration for 'mallinfo */ #ifndef USED_FOR_TARGET #undef HAVE_DECL_MALLINFO #endif +/* Define to 1 if we found a declaration for 'mallinfo2', otherwise define to + 0. */ +#ifndef USED_FOR_TARGET +#undef HAVE_DECL_MALLINFO2 +#endif + + /* Define to 1 if we found a declaration for 'malloc', otherwise define to 0. */ #ifndef USED_FOR_TARGET @@ -1665,6 +1671,12 @@ #endif +/* Define to 1 if you have the `mallinfo2' function. */ +#ifndef USED_FOR_TARGET +#undef HAVE_MALLINFO2 +#endif + + /* Define to 1 if you have the header file. */ #ifndef USED_FOR_TARGET #undef HAVE_MALLOC_H diff --git a/gcc/configure b/gcc/configure index 0f7a8dbe0f9..b8b9bd3505b 100755 --- a/gcc/configure +++ b/gcc/configure @@ -10120,7 +10120,7 @@ fi for ac_func in times clock kill getrlimit setrlimit atoq \ popen sysconf strsignal getrusage nl_langinfo \ gettimeofday mbstowcs wcswidth mmap setlocale \ - clearerr_unlocked feof_unlocked ferror_unlocked fflush_unlocked fgetc_unlocked fgets_unlocked fileno_unlocked fprintf_unlocked fputc_unlocked fputs_unlocked fread_unlocked fwrite_unlocked getchar_unlocked getc_unlocked putchar_unlocked putc_unlocked madvise mallinfo + clearerr_unlocked feof_unlocked ferror_unlocked fflush_unlocked fgetc_unlocked fgets_unlocked fileno_unlocked fprintf_unlocked fputc_unlocked fputs_unlocked fread_unlocked fwrite_unlocked getchar_unlocked getc_unlocked putchar_unlocked putc_unlocked madvise mallinfo mallinfo2 do : as_ac_var=`$as_echo "ac_cv_func_$ac_func" | $as_tr_sh` ac_fn_cxx_check_func "$LINENO" "$ac_func" "$as_ac_var" @@ -11549,7 +11549,7 @@ fi done -for ac_func in mallinfo +for ac_func in mallinfo, mallinfo2 do ac_tr_decl=`$as_echo "HAVE_DECL_$ac_func" | $as_tr_cpp` { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether $ac_func is declared" >&5 diff --git a/gcc/configure.ac b/gcc/configure.ac index 0f11238c19f..18640fdb8a5 100644 --- a/gcc/configure.ac +++ b/gcc/configure.ac @@ -1408,7 +1408,7 @@ define(gcc_UNLOCKED_FUNCS, clearerr_unlocked feof_unlocked dnl AC_CHECK_FUNCS(times clock kill getrlimit setrlimit atoq \ popen sysconf strsignal getrusage nl_langinfo \ gettimeofday mbstowcs wcswidth mmap setlocale \ - gcc_UNLOCKED_FUNCS madvise mallinfo) + gcc_UNLOCKED_FUNCS madvise mallinfo mallinfo2) if test x$ac_cv_func_mbstowcs = xyes; then AC_CACHE_CHECK(whether mbstowcs works, gcc_cv_func_mbstowcs_works, @@ -1488,7 +1488,7 @@ gcc_AC_CHECK_DECLS(getrlimit setrlimit getrusage, , ,[ #endif ]) -gcc_AC_CHECK_DECLS(mallinfo, , ,[ +gcc_AC_CHECK_DECLS([mallinfo, mallinfo2], , ,[ #include "ansidecl.h" #include "system.h" #ifdef HAVE_MALLOC_H diff --git a/gcc/ggc-common.c b/gcc/ggc-common.c index 94da02f1185..f6be5472bd5 100644 --- a/gcc/ggc-common.c +++ b/gcc/ggc-common.c @@ -1008,13 +1008,19 @@ ggc_prune_overhead_list (void) } } -/* Return memory used by heap in kb, 0 if this info is not available. */ +/* Print memory used by heap in MiB if this info is not available. */ void report_heap_memory_use () { -#ifdef HAVE_MALLINFO +#if defined(HAVE_MALLINFO) || defined(HAVE_MALLINFO2) +#ifdef HAVE_MALLINFO2 + #define MALLINFO_FN mallinfo2 +#else + #define MALLINFO_FN mallinfo +#endif if (!quiet_flag) - fprintf (stderr," {heap %luk}", (unsigned long)(mallinfo().arena / 1024)); + fprintf (stderr," {heap %lu MiB}", + (unsigned long) MALLINFO_FN ().arena / ONE_M); #endif } diff --git a/gcc/system.h b/gcc/system.h index 3c543a005d8..4f0482be25d 100644 --- a/gcc/system.h +++ b/gcc/system.h @@ -732,7 +732,7 @@ extern int vsnprintf (char *, size_t, const char *, va_list); #endif #ifdef INCLUDE_MALLOC_H -#ifdef HAVE_MALLINFO +#if defined(HAVE_MALLINFO) || defined(HAVE_MALLINFO2) #include #endif #endif -- 2.28.0 From patchwork Tue Sep 1 12:33:04 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Martin_Li=C5=A1ka?= X-Patchwork-Id: 1354929 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=2620:52:3:1:0:246e:9693:128c; helo=sourceware.org; envelope-from=gcc-patches-bounces@gcc.gnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=suse.cz Received: from sourceware.org (server2.sourceware.org [IPv6:2620:52:3:1:0:246e:9693:128c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4Bgmgp2RmMz9sTd for ; Tue, 1 Sep 2020 22:33:10 +1000 (AEST) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id F3D0F3950C41; Tue, 1 Sep 2020 12:33:07 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from mx2.suse.de (mx2.suse.de [195.135.220.15]) by sourceware.org (Postfix) with ESMTPS id C34C13950C41 for ; Tue, 1 Sep 2020 12:33:05 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org C34C13950C41 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=suse.cz Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=mliska@suse.cz X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id 39F14AEF8; Tue, 1 Sep 2020 12:33:05 +0000 (UTC) Subject: [PATCH 3/3] Use more ONE_? in GGC functions. From: =?utf-8?q?Martin_Li=C5=A1ka?= To: GCC Patches References: Message-ID: <11c9c279-e14f-3d68-95bc-2f2a0d9ed882@suse.cz> Date: Tue, 1 Sep 2020 14:33:04 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.11.0 MIME-Version: 1.0 In-Reply-To: Content-Language: en-US X-Spam-Status: No, score=-9.7 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_STATUS, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Jan Hubicka Errors-To: gcc-patches-bounces@gcc.gnu.org Sender: "Gcc-patches" The last patch is a refactoring using ONE_* macros. Thoughts? Thanks, Martin From 999a1969e7325efa0072d0ee893fcb6aa7178997 Mon Sep 17 00:00:00 2001 From: Martin Liska Date: Tue, 1 Sep 2020 14:24:20 +0200 Subject: [PATCH 3/3] Use more ONE_? in GGC functions. gcc/ChangeLog: * ggc-common.c (ggc_rlimit_bound): Use ONE_? macros. (ggc_min_expand_heuristic): Likewise. (ggc_min_heapsize_heuristic): Likewise. * ggc-page.c (ggc_collect): Likewise. * system.h (ONE_G): Likewise. --- gcc/ggc-common.c | 16 ++++++++-------- gcc/ggc-page.c | 2 +- gcc/system.h | 1 + 3 files changed, 10 insertions(+), 9 deletions(-) diff --git a/gcc/ggc-common.c b/gcc/ggc-common.c index f6be5472bd5..a22f746e645 100644 --- a/gcc/ggc-common.c +++ b/gcc/ggc-common.c @@ -742,7 +742,7 @@ ggc_rlimit_bound (double limit) appears to be ignored. Ignore such silliness. If a limit this small was actually effective for mmap, GCC wouldn't even start up. */ - && rlim.rlim_cur >= 8 * 1024 * 1024) + && rlim.rlim_cur >= 8 * ONE_M) limit = rlim.rlim_cur; # endif /* RLIMIT_AS or RLIMIT_DATA */ #endif /* HAVE_GETRLIMIT */ @@ -761,7 +761,7 @@ ggc_min_expand_heuristic (void) /* The heuristic is a percentage equal to 30% + 70%*(RAM/1GB), yielding a lower bound of 30% and an upper bound of 100% (when RAM >= 1GB). */ - min_expand /= 1024*1024*1024; + min_expand /= ONE_G; min_expand *= 70; min_expand = MIN (min_expand, 70); min_expand += 30; @@ -776,8 +776,8 @@ ggc_min_heapsize_heuristic (void) double phys_kbytes = physmem_total (); double limit_kbytes = ggc_rlimit_bound (phys_kbytes * 2); - phys_kbytes /= 1024; /* Convert to Kbytes. */ - limit_kbytes /= 1024; + phys_kbytes /= ONE_K; /* Convert to Kbytes. */ + limit_kbytes /= ONE_K; /* The heuristic is RAM/8, with a lower bound of 4M and an upper bound of 128M (when RAM >= 1GB). */ @@ -790,7 +790,7 @@ ggc_min_heapsize_heuristic (void) struct rlimit rlim; if (getrlimit (RLIMIT_RSS, &rlim) == 0 && rlim.rlim_cur != (rlim_t) RLIM_INFINITY) - phys_kbytes = MIN (phys_kbytes, rlim.rlim_cur / 1024); + phys_kbytes = MIN (phys_kbytes, rlim.rlim_cur / ONE_K); } # endif @@ -798,12 +798,12 @@ ggc_min_heapsize_heuristic (void) *next* GC would be within 20Mb of the limit or within a quarter of the limit, whichever is larger. If GCC does hit the data limit, compilation will fail, so this tries to be conservative. */ - limit_kbytes = MAX (0, limit_kbytes - MAX (limit_kbytes / 4, 20 * 1024)); + limit_kbytes = MAX (0, limit_kbytes - MAX (limit_kbytes / 4, 20 * ONE_K)); limit_kbytes = (limit_kbytes * 100) / (110 + ggc_min_expand_heuristic ()); phys_kbytes = MIN (phys_kbytes, limit_kbytes); - phys_kbytes = MAX (phys_kbytes, 4 * 1024); - phys_kbytes = MIN (phys_kbytes, 128 * 1024); + phys_kbytes = MAX (phys_kbytes, 4 * ONE_K); + phys_kbytes = MIN (phys_kbytes, 128 * ONE_K); return phys_kbytes; } diff --git a/gcc/ggc-page.c b/gcc/ggc-page.c index 0a43a7cdc0e..e5988ab1839 100644 --- a/gcc/ggc-page.c +++ b/gcc/ggc-page.c @@ -2184,7 +2184,7 @@ ggc_collect (void) total allocations haven't expanded much since the last collection. */ float allocated_last_gc = - MAX (G.allocated_last_gc, (size_t)param_ggc_min_heapsize * 1024); + MAX (G.allocated_last_gc, (size_t)param_ggc_min_heapsize * ONE_K); /* It is also good time to get memory block pool into limits. */ memory_block_pool::trim (); diff --git a/gcc/system.h b/gcc/system.h index 4f0482be25d..b0f3f1dd019 100644 --- a/gcc/system.h +++ b/gcc/system.h @@ -1237,6 +1237,7 @@ void gcc_stablesort (void *, size_t, size_t, #define ONE_K 1024 #define ONE_M (ONE_K * ONE_K) +#define ONE_G (ONE_K * ONE_M) /* Display a number as an integer multiple of either: - 1024, if said integer is >= to 10 K (in base 2) -- 2.28.0