diff mbox

"ss -p" segfaults

Message ID 20150715151204.GB28525@angus-think.wlc.globallogic.com
State RFC, archived
Delegated to: stephen hemminger
Headers show

Commit Message

Vadym Kochan July 15, 2015, 3:12 p.m. UTC
On Wed, Jul 15, 2015 at 04:09:03PM +0200, Marc Dietrich wrote:
> Hi,
> 
> ss -p segfaults here with some kind of memory corruption:
> 
> *** Error in `/work/iproute2/misc/ss': free(): invalid pointer: 
> 0x0000000000623000 ***
> ======= Backtrace: =========
> /lib64/libc.so.6(+0x71c6d)[0x7ffff7885c6d]
> /lib64/libc.so.6(+0x771be)[0x7ffff788b1be]
> /lib64/libc.so.6(+0x7799b)[0x7ffff788b99b]
> /work/iproute2/misc/ss[0x403de1]
> /work/iproute2/misc/ss[0x408247]
> /work/iproute2/misc/ss[0x403295]
> /lib64/libc.so.6(__libc_start_main+0xf0)[0x7ffff7834790]
> /work/iproute2/misc/ss[0x4037f9]
> ======= Memory map: ========
> 00400000-00416000 r-xp 00000000 00:33 4207305                            
> /work/iproute2/misc/ss
> 00616000-00617000 r--p 00016000 00:33 4207305                            
> /work/iproute2/misc/ss
> 00617000-0061b000 rw-p 00017000 00:33 4207305                            
> /work/iproute2/misc/ss
> 0061b000-0065f000 rw-p 00000000 00:00 0                                  
> [heap]
> 7ffff6f6d000-7ffff6f83000 r-xp 00000000 00:21 16154175                   
> /lib64/libgcc_s.so.1
> 7ffff6f83000-7ffff7182000 ---p 00016000 00:21 16154175                   
> /lib64/libgcc_s.so.1
> 7ffff7182000-7ffff7183000 r--p 00015000 00:21 16154175                   
> /lib64/libgcc_s.so.1
> 7ffff7183000-7ffff7184000 rw-p 00016000 00:21 16154175                   
> /lib64/libgcc_s.so.1
> 7ffff7184000-7ffff719c000 r-xp 00000000 00:21 16694826                   
> /lib64/libpthread-2.21.so
> 7ffff719c000-7ffff739b000 ---p 00018000 00:21 16694826                   
> /lib64/libpthread-2.21.so
> 7ffff739b000-7ffff739c000 r--p 00017000 00:21 16694826                   
> /lib64/libpthread-2.21.so
> 7ffff739c000-7ffff739d000 rw-p 00018000 00:21 16694826                   
> /lib64/libpthread-2.21.so
> 7ffff739d000-7ffff73a1000 rw-p 00000000 00:00 0 
> 7ffff73a1000-7ffff73a4000 r-xp 00000000 00:21 16694804                   
> /lib64/libdl-2.21.so
> 7ffff73a4000-7ffff75a3000 ---p 00003000 00:21 16694804                   
> /lib64/libdl-2.21.so
> 7ffff75a3000-7ffff75a4000 r--p 00002000 00:21 16694804                   
> /lib64/libdl-2.21.so
> 7ffff75a4000-7ffff75a5000 rw-p 00003000 00:21 16694804                   
> /lib64/libdl-2.21.so
> 7ffff75a5000-7ffff7613000 r-xp 00000000 00:21 16153198                   
> /usr/lib64/libpcre.so.1.2.5
> 7ffff7613000-7ffff7812000 ---p 0006e000 00:21 16153198                   
> /usr/lib64/libpcre.so.1.2.5
> 7ffff7812000-7ffff7813000 r--p 0006d000 00:21 16153198                   
> /usr/lib64/libpcre.so.1.2.5
> 7ffff7813000-7ffff7814000 rw-p 0006e000 00:21 16153198                   
> /usr/lib64/libpcre.so.1.2.5
> 7ffff7814000-7ffff79ad000 r-xp 00000000 00:21 16694798                   
> /lib64/libc-2.21.so
> 7ffff79ad000-7ffff7bac000 ---p 00199000 00:21 16694798                   
> /lib64/libc-2.21.so
> 7ffff7bac000-7ffff7bb1000 r--p 00198000 00:21 16694798                   
> /lib64/libc-2.21.so
> 7ffff7bb1000-7ffff7bb3000 rw-p 0019d000 00:21 16694798                   
> /lib64/libc-2.21.so
> 7ffff7bb3000-7ffff7bb7000 rw-p 00000000 00:00 0 
> 7ffff7bb7000-7ffff7bd8000 r-xp 00000000 00:21 16155991                   
> /lib64/libselinux.so.1
> 7ffff7bd8000-7ffff7dd7000 ---p 00021000 00:21 16155991                   
> /lib64/libselinux.so.1
> 7ffff7dd7000-7ffff7dd8000 r--p 00020000 00:21 16155991                   
> /lib64/libselinux.so.1
> 7ffff7dd8000-7ffff7dd9000 rw-p 00021000 00:21 16155991                   
> /lib64/libselinux.so.1
> 7ffff7dd9000-7ffff7ddb000 rw-p 00000000 00:00 0 
> 7ffff7ddb000-7ffff7dfc000 r-xp 00000000 00:21 16694791                   
> /lib64/ld-2.21.so
> 7ffff7fb5000-7ffff7fba000 rw-p 00000000 00:00 0 
> 7ffff7ff5000-7ffff7ff8000 rw-p 00000000 00:00 0 
> 7ffff7ff8000-7ffff7ffa000 r--p 00000000 00:00 0                          
> [vvar]
> 7ffff7ffa000-7ffff7ffc000 r-xp 00000000 00:00 0                          
> [vdso]
> 7ffff7ffc000-7ffff7ffd000 r--p 00021000 00:21 16694791                   
> /lib64/ld-2.21.so
> 7ffff7ffd000-7ffff7ffe000 rw-p 00022000 00:21 16694791                   
> /lib64/ld-2.21.so
> 7ffff7ffe000-7ffff7fff000 rw-p 00000000 00:00 0 
> 7ffffffdd000-7ffffffff000 rw-p 00000000 00:00 0                          
> [stack]
> ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  
> [vsyscall]
> 
> Program received signal SIGABRT, Aborted.
> 0x00007ffff7847638 in raise () from /lib64/libc.so.6
> Missing separate debuginfos, use: zypper install libgcc_s1-
> debuginfo-5.1.1+r224716-1.2.x86_64 libpcre1-debuginfo-8.37-1.18.x86_64 
> libselinux1-debuginfo-2.3-5.18.x86_64
> (gdb) bt full
> #0  0x00007ffff7847638 in raise () from /lib64/libc.so.6
> No symbol table info available.
> #1  0x00007ffff7848a1a in abort () from /lib64/libc.so.6
> No symbol table info available.
> #2  0x00007ffff7885c72 in __libc_message () from /lib64/libc.so.6
> No symbol table info available.
> #3  0x00007ffff788b1be in malloc_printerr () from /lib64/libc.so.6
> No symbol table info available.
> #4  0x00007ffff788b99b in _int_free () from /lib64/libc.so.6
> No symbol table info available.
> #5  0x0000000000403de1 in unix_list_free (list=0x6251a0, list@entry=0x645b50) 
> at ss.c:2516
>         s = 0x623010
>         name = 0x272 <error: Cannot access memory at address 0x272>
> #6  0x0000000000408247 in unix_show (f=0x61cdf0 <current_filter>) at ss.c:2798
>         buf = "ffff880205a15b00: 00000003 00000000 00000000 0001 03 
> 84307\n\000/tmp/.X11-
> unix/X0\n\000/stdout\n\000adiserver.socket\n\000\n\000c\n\000cket\n", '\000' 
> <repeats 12 times>, 
> "q\017A\000\000\000\000\000p\205\201\367\377\177\000\000x\323\377\377\377\177\000\000\067\v\000\000\000\000\000\000h\323\377\377\377\177\000\000\060\374\336\367\377\177\000\000\000U\000\000\005\000\000\000\277\000\000\000;
> \212\000\000\000\003\034\177\025\004\000\001"...
>         name = "\000/tmp/.X11-
> unix/X0\000l/stdout\000nadiserver.socket\000\071\000ec\000ocket\000\021@\000\000\000\000\000\377\377\377\377\000\000\000\000\b\321\377\377\377\177\000\000p\205\201\367\377\177\000\000Д\373\367\377\177\000\000\330|
> \335\367\377\177\000\000\226\226\204\367\377\177\000\000\370\n@\000\000\000\000\000\070\254a\000\000\000\000"
>         newformat = 0
>         cnt = 734
>         list = <optimized out>
> #7  0x0000000000403295 in main (argc=<optimized out>, argv=0x7fffffffd378) at 
> ss.c:3921
>         saw_states = <optimized out>
>         saw_query = 0
>         do_summary = <optimized out>
>         dump_tcpdiag = 0x0
>         filter_fp = 0x0
>         ch = <optimized out>
>         state_filter = 2871
> (gdb) 
> 
> git bisect shows bad commit ec4d0d8 (ss: Replace unixstat struct by new 
> sockstat struct)
> 
> This is with a 4.1.2 kernel. Strange thing is, that this segfault does not 
> happen on my distro kernel (also v4.1) (openSUSE). Seems to be some random 
> stuff or kernel config problem maybe.
> 
> Marc
Would you please check this fix ?

--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Comments

Rustad, Mark D July 15, 2015, 4:49 p.m. UTC | #1
> On Jul 15, 2015, at 8:12 AM, Vadim Kochan <vadim4j@gmail.com> wrote:
> Would you please check this fix ?
> 
> diff --git a/misc/ss.c b/misc/ss.c
> index 03f92fa..3a826e4 100644
> --- a/misc/ss.c
> +++ b/misc/ss.c
> @@ -683,8 +683,8 @@ static inline void sock_addr_set_str(inet_prefix *prefix, char **ptr)
> 
> static inline char *sock_addr_get_str(const inet_prefix *prefix)
> {
> -    char *tmp ;
> -    memcpy(&tmp, prefix->data, sizeof(char *));
> +    char *tmp;
> +    memcpy(&tmp, &prefix->data[0], sizeof(char *));
>     return tmp;
> }

That surely is not a fix! The destination of the memcpy is the address of an uninitialized stack variable! Both versions are equally bad.

--
Mark Rustad, Networking Division, Intel Corporation
Rustad, Mark D July 15, 2015, 6:52 p.m. UTC | #2
> On Jul 15, 2015, at 9:49 AM, Rustad, Mark D <mark.d.rustad@intel.com> wrote:
> 
>> On Jul 15, 2015, at 8:12 AM, Vadim Kochan <vadim4j@gmail.com> wrote:
>> Would you please check this fix ?
>> 
>> diff --git a/misc/ss.c b/misc/ss.c
>> index 03f92fa..3a826e4 100644
>> --- a/misc/ss.c
>> +++ b/misc/ss.c
>> @@ -683,8 +683,8 @@ static inline void sock_addr_set_str(inet_prefix *prefix, char **ptr)
>> 
>> static inline char *sock_addr_get_str(const inet_prefix *prefix)
>> {
>> -    char *tmp ;
>> -    memcpy(&tmp, prefix->data, sizeof(char *));
>> +    char *tmp;
>> +    memcpy(&tmp, &prefix->data[0], sizeof(char *));
>>    return tmp;
>> }
> 
> That surely is not a fix! The destination of the memcpy is the address of an uninitialized stack variable! Both versions are equally bad.

I probably over-reacted, but using memcpy to access a pointer in this way is just ugly. For one thing, it circumvents any sanity-checking that the compiler can do. And changing the prefix->data to &prefix->data[0] should be exactly the same thing and therefore should not fix anything. Anyway, never mind that.

Looking at more of the code, it looks to me like the the string pointer in data can sometimes point to a literal string instead of allocated memory when proc is in use. Free would not be happy with that. Look at the use of variable peer in function unix_stats_print.

--
Mark Rustad, Networking Division, Intel Corporation
Vadym Kochan July 15, 2015, 6:57 p.m. UTC | #3
On Wed, Jul 15, 2015 at 06:52:49PM +0000, Rustad, Mark D wrote:
> > On Jul 15, 2015, at 9:49 AM, Rustad, Mark D <mark.d.rustad@intel.com> wrote:
> > 
> >> On Jul 15, 2015, at 8:12 AM, Vadim Kochan <vadim4j@gmail.com> wrote:
> >> Would you please check this fix ?
> >> 
> >> diff --git a/misc/ss.c b/misc/ss.c
> >> index 03f92fa..3a826e4 100644
> >> --- a/misc/ss.c
> >> +++ b/misc/ss.c
> >> @@ -683,8 +683,8 @@ static inline void sock_addr_set_str(inet_prefix *prefix, char **ptr)
> >> 
> >> static inline char *sock_addr_get_str(const inet_prefix *prefix)
> >> {
> >> -    char *tmp ;
> >> -    memcpy(&tmp, prefix->data, sizeof(char *));
> >> +    char *tmp;
> >> +    memcpy(&tmp, &prefix->data[0], sizeof(char *));
> >>    return tmp;
> >> }
> > 
> > That surely is not a fix! The destination of the memcpy is the address of an uninitialized stack variable! Both versions are equally bad.
> 
> I probably over-reacted, but using memcpy to access a pointer in this way is just ugly. For one thing, it circumvents any sanity-checking that the compiler can do. And changing the prefix->data to &prefix->data[0] should be exactly the same thing and therefore should not fix anything. Anyway, never mind that.
> 
> Looking at more of the code, it looks to me like the the string pointer in data can sometimes point to a literal string instead of allocated memory when proc is in use. Free would not be happy with that. Look at the use of variable peer in function unix_stats_print.
> 
Yes that right, I am already looking on this ...
> --
> Mark Rustad, Networking Division, Intel Corporation
> 
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Vadym Kochan July 15, 2015, 10:22 p.m. UTC | #4
On Wed, Jul 15, 2015 at 09:57:51PM +0300, Vadim Kochan wrote:
> On Wed, Jul 15, 2015 at 06:52:49PM +0000, Rustad, Mark D wrote:
> > > On Jul 15, 2015, at 9:49 AM, Rustad, Mark D <mark.d.rustad@intel.com> wrote:
> > > 
> > >> On Jul 15, 2015, at 8:12 AM, Vadim Kochan <vadim4j@gmail.com> wrote:
> > >> Would you please check this fix ?
> > >> 
> > >> diff --git a/misc/ss.c b/misc/ss.c
> > >> index 03f92fa..3a826e4 100644
> > >> --- a/misc/ss.c
> > >> +++ b/misc/ss.c
> > >> @@ -683,8 +683,8 @@ static inline void sock_addr_set_str(inet_prefix *prefix, char **ptr)
> > >> 
> > >> static inline char *sock_addr_get_str(const inet_prefix *prefix)
> > >> {
> > >> -    char *tmp ;
> > >> -    memcpy(&tmp, prefix->data, sizeof(char *));
> > >> +    char *tmp;
> > >> +    memcpy(&tmp, &prefix->data[0], sizeof(char *));
> > >>    return tmp;
> > >> }
> > > 
> > > That surely is not a fix! The destination of the memcpy is the address of an uninitialized stack variable! Both versions are equally bad.
> > 
> > I probably over-reacted, but using memcpy to access a pointer in this way is just ugly. For one thing, it circumvents any sanity-checking that the compiler can do. And changing the prefix->data to &prefix->data[0] should be exactly the same thing and therefore should not fix anything. Anyway, never mind that.
> > 
> > Looking at more of the code, it looks to me like the the string pointer in data can sometimes point to a literal string instead of allocated memory when proc is in use. Free would not be happy with that. Look at the use of variable peer in function unix_stats_print.
> > 
> Yes that right, I am already looking on this ...
> > --
> > Mark Rustad, Networking Division, Intel Corporation
> > 

I did partially revert of the buggy commit and it does not crash, but I will do
more testing, and after will send the patch and will try to prepare some
test scripts for ss.

The crash appears only if to dump processes info from /proc, which might
be caused that netlink stats returned error, probably by wrong request
(not supported attribute or flag ?).
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Marc Dietrich July 16, 2015, 6:37 a.m. UTC | #5
Am Donnerstag, 16. Juli 2015, 01:22:38 schrieb Vadim Kochan:
> On Wed, Jul 15, 2015 at 09:57:51PM +0300, Vadim Kochan wrote:
> > On Wed, Jul 15, 2015 at 06:52:49PM +0000, Rustad, Mark D wrote:
> > > > On Jul 15, 2015, at 9:49 AM, Rustad, Mark D <mark.d.rustad@intel.com> 
wrote:
> > > >> On Jul 15, 2015, at 8:12 AM, Vadim Kochan <vadim4j@gmail.com> wrote:
> > > >> Would you please check this fix ?
> > > >> 
> > > >> diff --git a/misc/ss.c b/misc/ss.c
> > > >> index 03f92fa..3a826e4 100644
> > > >> --- a/misc/ss.c
> > > >> +++ b/misc/ss.c
> > > >> @@ -683,8 +683,8 @@ static inline void sock_addr_set_str(inet_prefix
> > > >> *prefix, char **ptr)
> > > >> 
> > > >> static inline char *sock_addr_get_str(const inet_prefix *prefix)
> > > >> {
> > > >> -    char *tmp ;
> > > >> -    memcpy(&tmp, prefix->data, sizeof(char *));
> > > >> +    char *tmp;
> > > >> +    memcpy(&tmp, &prefix->data[0], sizeof(char *));
> > > >> 
> > > >>    return tmp;
> > > >> 
> > > >> }
> > > > 
> > > > That surely is not a fix! The destination of the memcpy is the address
> > > > of an uninitialized stack variable! Both versions are equally bad.> > 
> > > I probably over-reacted, but using memcpy to access a pointer in this
> > > way is just ugly. For one thing, it circumvents any sanity-checking
> > > that the compiler can do. And changing the prefix->data to
> > > &prefix->data[0] should be exactly the same thing and therefore should
> > > not fix anything. Anyway, never mind that.
> > > 
> > > Looking at more of the code, it looks to me like the the string pointer
> > > in data can sometimes point to a literal string instead of allocated
> > > memory when proc is in use. Free would not be happy with that. Look at
> > > the use of variable peer in function unix_stats_print.> 
> > Yes that right, I am already looking on this ...
> > 
> > > --
> > > Mark Rustad, Networking Division, Intel Corporation
> 
> I did partially revert of the buggy commit and it does not crash, but I will
> do more testing, and after will send the patch and will try to prepare some
> test scripts for ss.
> 
> The crash appears only if to dump processes info from /proc, which might
> be caused that netlink stats returned error, probably by wrong request
> (not supported attribute or flag ?).

the reason it uses proc for me is my self compiled (and trimmed) kernel which 
disabled all *_diag modules which seem to be required by ss. I didn't know 
this. On the other hand, I managed to find a bug this way :-)

Marc
diff mbox

Patch

diff --git a/misc/ss.c b/misc/ss.c
index 03f92fa..3a826e4 100644
--- a/misc/ss.c
+++ b/misc/ss.c
@@ -683,8 +683,8 @@  static inline void sock_addr_set_str(inet_prefix *prefix, char **ptr)
 
 static inline char *sock_addr_get_str(const inet_prefix *prefix)
 {
-    char *tmp ;
-    memcpy(&tmp, prefix->data, sizeof(char *));
+    char *tmp;
+    memcpy(&tmp, &prefix->data[0], sizeof(char *));
     return tmp;
 }