Message ID | 5788E12C.9010205@cs.ucla.edu |
---|---|
State | New |
Headers | show |
On Fri, Jul 15, 2016 at 6:42 PM, Paul Eggert <eggert@cs.ucla.edu> wrote: > On 07/15/2016 12:52 PM, Florian Weimer wrote: >> >> The problem is that this happens at dump time only, while the issue occurs >> when running the dumped Emacs. Furthermore, the personality system call (or >> at least ADDR_NO_RANDOMIZE) only applies to new process images, it does not >> affected the randomization decisions made at load time for the current >> process. > > > I wrote a revised patch to emacs-25 (attached) which I hope addresses the > former issue, by disabling randomization in both the undumped and dumped > Emacs when Emacs uses its own allocator. Emacs already addresses the latter > issue by re-execing itself when it changes personalities, so this shouldn't > be an issue. After applying this patch in emacs-25.0.95, it builds and run fine on ppc64(le), Fedora rawhide. It no longer consumes memory in GB ! For reference, scratch build link is http://ppc.koji.fedoraproject.org/koji/taskinfo?taskID=3538651
On Fri, Jul 15, 2016 at 7:49 PM, Sinny Kumari <ksinny@gmail.com> wrote: > On Fri, Jul 15, 2016 at 6:42 PM, Paul Eggert <eggert@cs.ucla.edu> wrote: >> On 07/15/2016 12:52 PM, Florian Weimer wrote: >>> >>> The problem is that this happens at dump time only, while the issue occurs >>> when running the dumped Emacs. Furthermore, the personality system call (or >>> at least ADDR_NO_RANDOMIZE) only applies to new process images, it does not >>> affected the randomization decisions made at load time for the current >>> process. >> >> >> I wrote a revised patch to emacs-25 (attached) which I hope addresses the >> former issue, by disabling randomization in both the undumped and dumped >> Emacs when Emacs uses its own allocator. Emacs already addresses the latter >> issue by re-execing itself when it changes personalities, so this shouldn't >> be an issue. > > After applying this patch in emacs-25.0.95, it builds and run fine on ppc64(le), > Fedora rawhide. It no longer consumes memory in GB ! > For reference, scratch build link is > http://ppc.koji.fedoraproject.org/koji/taskinfo?taskID=3538651 Also, output of "cat /proc/self/personality" inside emacs shell contains value 00040000
On 07/15/2016 04:28 PM, Sinny Kumari wrote: > On Fri, Jul 15, 2016 at 7:49 PM, Sinny Kumari <ksinny@gmail.com> wrote: >> On Fri, Jul 15, 2016 at 6:42 PM, Paul Eggert <eggert@cs.ucla.edu> wrote: >>> On 07/15/2016 12:52 PM, Florian Weimer wrote: >>>> >>>> The problem is that this happens at dump time only, while the issue occurs >>>> when running the dumped Emacs. Furthermore, the personality system call (or >>>> at least ADDR_NO_RANDOMIZE) only applies to new process images, it does not >>>> affected the randomization decisions made at load time for the current >>>> process. >>> >>> >>> I wrote a revised patch to emacs-25 (attached) which I hope addresses the >>> former issue, by disabling randomization in both the undumped and dumped >>> Emacs when Emacs uses its own allocator. Emacs already addresses the latter >>> issue by re-execing itself when it changes personalities, so this shouldn't >>> be an issue. >> >> After applying this patch in emacs-25.0.95, it builds and run fine on ppc64(le), >> Fedora rawhide. It no longer consumes memory in GB ! >> For reference, scratch build link is >> http://ppc.koji.fedoraproject.org/koji/taskinfo?taskID=3538651 > > Also, output of "cat /proc/self/personality" inside emacs shell > contains value 00040000 Thanks! This means that new processes spawned by Emacs will have ASLR disable. Paul, I think the patch should call personality again after it is active for the Emacs process, to disable it for subprocesses. Florian
On 07/15/2016 04:33 PM, Florian Weimer wrote: > > This means that new processes spawned by Emacs will have ASLR disable. Yes, on second thought that's too drastic. The patch I just now installed into master (and sent via email to this list) doesn't do that. It disables ASLR only in temacs, not in ordinary Emacs invocations. (Enabling ASLR in ordinary Emacs broke the Emacs build on Fedora 23 x86-64.) This is how Emacs has worked for many years; although you indicated that it might not work with bleeding-edge glibc in some situations, I don't understand why not, and would like to understand it better before changing this aspect of Emacs..
On Fri, Jul 15, 2016 at 4:00 PM, Paul Eggert <eggert@cs.ucla.edu> wrote: > On 07/15/2016 04:33 PM, Florian Weimer wrote: >> >> This means that new processes spawned by Emacs will have ASLR disable. > > Yes, on second thought that's too drastic. The patch I just now installed > into master (and sent via email to this list) doesn't do that. It disables > ASLR only in temacs, not in ordinary Emacs invocations. (Enabling ASLR in > ordinary Emacs broke the Emacs build on Fedora 23 x86-64.) This is how Emacs > has worked for many years; although you indicated that it might not work > with bleeding-edge glibc in some situations, I don't understand why not, and > would like to understand it better before changing this aspect of Emacs.. I don't have affected hardware to test this on, but the phenomenon seems pretty clear to me. Emacs' bundled malloc assumes that the data segment is contiguous with, or at least close to, the memory region controlled by brk(). This is not true with ASLR enabled, on certain architectures (e.g. ppc64). That assumption is a fundamental design constraint, rather than something done to support dumping. Therefore, if the bundled malloc is used in the "ordinary Emacs", ASLR must be disabled in the "ordinary Emacs"; disabling it in temacs is not enough. It looks like HYBRID_MALLOC mode would avoid the problem, but from upthread, that mode isn't ready for prime time yet...? zw
On 07/15/2016 11:19 PM, Zack Weinberg wrote: > Emacs' bundled malloc assumes that the data > segment is contiguous with, or at least close to, the memory region > controlled by brk(). This is not true with ASLR enabled, on certain > architectures (e.g. ppc64). Unfortunately Emacs with glibc 2.22 malloc seems to be making the opposite assumption, at least on Fedora 23 x86-64. That is, if I disable ASLR in an ordinary (dumped) Emacs with glibc 2.22, Emacs stops working (I don't know why). This leads me to be leery of disabling ASLR in an ordinary Emacs, except for platforms like ppc64 where it seems to be necessary. The patch I just sent in therefore has a special case for __PPC64__. Quite a hack, but without knowing the problem better I don't know what other hack to try. > It looks like > HYBRID_MALLOC mode would avoid the problem, but from upthread, that > mode isn't ready for prime time yet...? Yes, my sense is that HYBRID_MALLOC should work but it is not ready for emacs-25. It is already enabled in the master branch and we should be able to get it to work with ppc64 too, at least assuming the patch I just emailed here works on emacs-25.
From 6c7e7b23abe495da5ac83be6cf8e714d3160af45 Mon Sep 17 00:00:00 2001 From: Paul Eggert <eggert@cs.ucla.edu> Date: Fri, 15 Jul 2016 13:07:09 +0200 Subject: [PATCH] Port to glibc 2.24 (pre-release) + ppc64 Inspired by a suggestion by Florian Weimer in: https://sourceware.org/ml/libc-alpha/2016-07/msg00425.html * configure.ac (HAVE_PERSONALITY_ADDR_NO_RANDOMIZE): Rename from HAVE_PERSONALITY_LINUX32, and check for ADDR_NO_RANDOMIZE (the crucial thing) instead of for LINUX32. All uses changed. * src/emacs.c (main) [HAVE_PERSONALITY_ADDR_NO_RANDOMIZE]: Use ADDR_NO_RANDOMIZE from personality.h rather than inventing the flag ourselves. Just set that flag, rather than also setting the persona. Do all this earlier, so as to avoid problems with calls to brk in the interim. When doing it, avoid functions like putenv that may allocate memory. --- admin/CPP-DEFINES | 2 +- configure.ac | 20 +++++++++++--------- src/emacs.c | 55 ++++++++++++++++++++++++++++++++----------------------- 3 files changed, 44 insertions(+), 33 deletions(-) diff --git a/admin/CPP-DEFINES b/admin/CPP-DEFINES index 796b57d..d404dee 100644 --- a/admin/CPP-DEFINES +++ b/admin/CPP-DEFINES @@ -237,7 +237,7 @@ HAVE_NET_IF_DL_H HAVE_NET_IF_H HAVE_NLIST_H HAVE_OTF_GET_VARIATION_GLYPHS -HAVE_PERSONALITY_LINUX32 +HAVE_PERSONALITY_ADDR_NO_RANDOMIZE HAVE_PNG HAVE_PNG_H HAVE_POSIX_MEMALIGN diff --git a/configure.ac b/configure.ac index 678e98e..9da23d1 100644 --- a/configure.ac +++ b/configure.ac @@ -1615,15 +1615,17 @@ AC_CHECK_HEADERS_ONCE( sys/resource.h sys/utsname.h pwd.h utmp.h util.h) -AC_MSG_CHECKING(if personality LINUX32 can be set) -AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[#include <sys/personality.h>]], [[personality (PER_LINUX32)]])], - emacs_cv_personality_linux32=yes, - emacs_cv_personality_linux32=no) -AC_MSG_RESULT($emacs_cv_personality_linux32) - -if test $emacs_cv_personality_linux32 = yes; then - AC_DEFINE(HAVE_PERSONALITY_LINUX32, 1, - [Define to 1 if personality LINUX32 can be set.]) +AC_CACHE_CHECK([if personality ADDR_NO_RANDOMIZE flag exists], + [emacs_cv_personality_addr_no_randomize], + [AC_COMPILE_IFELSE( + [AC_LANG_PROGRAM([[#include <sys/personality.h>]], + [[personality (personality (0xffffffff) + | ADDR_NO_RANDOMIZE)]])], + [emacs_cv_personality_addr_no_randomize=yes], + [emacs_cv_personality_addr_no_randomize=no])]) +if test $emacs_cv_personality_addr_no_randomize = yes; then + AC_DEFINE([HAVE_PERSONALITY_ADDR_NO_RANDOMIZE], [1], + [Define to 1 if personality flag ADDR_NO_RANDOMIZE exists.]) fi # Note that Solaris has sys/sysinfo.h which defines struct diff --git a/src/emacs.c b/src/emacs.c index 5c187e7..901bc0c 100644 --- a/src/emacs.c +++ b/src/emacs.c @@ -106,7 +106,7 @@ extern void moncontrol (int mode); #include <sys/resource.h> #endif -#ifdef HAVE_PERSONALITY_LINUX32 +#ifdef HAVE_PERSONALITY_ADDR_NO_RANDOMIZE #include <sys/personality.h> #endif @@ -674,6 +674,37 @@ main (int argc, char **argv) stack_base = &dummy; + dumping = !initialized && (strcmp (argv[argc - 1], "dump") == 0 + || strcmp (argv[argc - 1], "bootstrap") == 0); + +#ifdef HAVE_PERSONALITY_ADDR_NO_RANDOMIZE + + /* True if address randomization interferes with memory allocaiton. */ +# ifdef HYBRID_MALLOC + bool disable_aslr = dumping; +# elif defined SYSTEM_MALLOC || defined DOUG_LEA_MALLOC + bool disable_aslr = false; +# else + bool disable_aslr = true; +# endif + + if (disable_aslr) + { + int pers = personality (0xffffffff); + if (! (pers & ADDR_NO_RANDOMIZE) + && 0 <= personality (pers | ADDR_NO_RANDOMIZE)) + { + /* Address randomization was enabled, but is now disabled. + Re-execute Emacs to get a clean slate. */ + execvp (argv[0], argv); + + /* If the exec fails, warn the user and then try without a + clean slate. */ + fprintf (stderr, "%s: %s\n", argv[0], strerror (errno)); + } + } +#endif + #ifndef CANNOT_DUMP might_dump = !initialized; #endif @@ -781,28 +812,6 @@ main (int argc, char **argv) } } - dumping = !initialized && (strcmp (argv[argc - 1], "dump") == 0 - || strcmp (argv[argc - 1], "bootstrap") == 0); - -#ifdef HAVE_PERSONALITY_LINUX32 - if (dumping && ! getenv ("EMACS_HEAP_EXEC")) - { - /* Set this so we only do this once. */ - xputenv ("EMACS_HEAP_EXEC=true"); - - /* A flag to turn off address randomization which is introduced - in linux kernel shipped with fedora core 4 */ -#define ADD_NO_RANDOMIZE 0x0040000 - personality (PER_LINUX32 | ADD_NO_RANDOMIZE); -#undef ADD_NO_RANDOMIZE - - execvp (argv[0], argv); - - /* If the exec fails, try to dump anyway. */ - emacs_perror (argv[0]); - } -#endif /* HAVE_PERSONALITY_LINUX32 */ - #if defined (HAVE_SETRLIMIT) && defined (RLIMIT_STACK) && !defined (CYGWIN) /* Extend the stack space available. Don't do that if dumping, since some systems (e.g. DJGPP) might define a smaller stack -- 2.5.5