diff mbox

tproxy: nf_tproxy_assign_sock() can handle tw sockets

Message ID 1278695580.2696.55.camel@edumazet-laptop
State Not Applicable, archived
Delegated to: David Miller
Headers show

Commit Message

Eric Dumazet July 9, 2010, 5:13 p.m. UTC
Le vendredi 09 juillet 2010 à 12:03 -0300, Felipe W Damasio a écrit :
> Hi,
> 
> 2010/7/8 Eric Dumazet <eric.dumazet@gmail.com>:
> > Please try to reproduce a new report.
> >
> > It looks like a memory corruption, and it would be good to see if a
> > common pattern is occurring.
> 
> I'm trying..the thing is the freeze occured on the machine that sits
> on a 200Mbps ISP in bridge-mode. Since the machine frooze, and the
> whole ISP went down for a few minutes, I'm not allowed to run any
> tests on it.
> 
> I've setup the same scenario on a lab, but since last night been
> unable to reproduce the bug. Maybe there's a clue on the this crash
> below that can help me write some program to trigger the problem?
> 

Reviewing tproxy stuff I spotted a problem in nf_tproxy_assign_sock()
but I could not see how it could explain your crash.

We can read uninitialized memory and trigger a fault in
nf_tproxy_assign_sock(), not later in tcp_recvmsg()...

David, Patrick, what do you think ?

Thanks

[PATCH] tproxy: nf_tproxy_assign_sock() can handle tw sockets

transparent field of a socket is either inet_twsk(sk)->tw_transparent
for timewait sockets, or inet_sk(sk)->transparent for other sockets
(TCP/UDP).

Signed-off-by: Eric Dumazet <eric.dumazet@gmail.com>
---


--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Comments

David Miller July 9, 2010, 5:53 p.m. UTC | #1
From: Eric Dumazet <eric.dumazet@gmail.com>
Date: Fri, 09 Jul 2010 19:13:00 +0200

> Reviewing tproxy stuff I spotted a problem in nf_tproxy_assign_sock()
> but I could not see how it could explain your crash.
> 
> We can read uninitialized memory and trigger a fault in
> nf_tproxy_assign_sock(), not later in tcp_recvmsg()...
> 
> David, Patrick, what do you think ?
> 
> Thanks
> 
> [PATCH] tproxy: nf_tproxy_assign_sock() can handle tw sockets
> 
> transparent field of a socket is either inet_twsk(sk)->tw_transparent
> for timewait sockets, or inet_sk(sk)->transparent for other sockets
> (TCP/UDP).
> 
> Signed-off-by: Eric Dumazet <eric.dumazet@gmail.com>

Looks fine to me:

Acked-by: David S. Miller <davem@davemloft.net>
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Felipe W Damasio July 9, 2010, 6:16 p.m. UTC | #2
Hi,

2010/7/9 Eric Dumazet <eric.dumazet@gmail.com>:
> Reviewing tproxy stuff I spotted a problem in nf_tproxy_assign_sock()
> but I could not see how it could explain your crash.
>
> We can read uninitialized memory and trigger a fault in
> nf_tproxy_assign_sock(), not later in tcp_recvmsg()...

Well, since I can't reproduce the bug, if you think this patch solves
my problem, I'll tell my bosses that we can put back the production
machine online.

But anyway, if there's test you think I can run, I have same the same
hardware on a lab using the same setup as the production
environment...

Thanks,

Felipe Damasio
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Felipe W Damasio July 10, 2010, 3:18 a.m. UTC | #3
Hi Mr. Dumazet,

2010/7/9 Eric Dumazet <eric.dumazet@gmail.com>:
> Reviewing tproxy stuff I spotted a problem in nf_tproxy_assign_sock()
> but I could not see how it could explain your crash.
>
> We can read uninitialized memory and trigger a fault in
> nf_tproxy_assign_sock(), not later in tcp_recvmsg()...
>
> David, Patrick, what do you think ?

But do you think that the bug that squid triggered was caused by the
TProxy code?

Or is related to the network-stack in some other point.

I don't know if this helps, but I'm using ebtables to remove the
packets from the bridge, and iptables to redirect the traffic to
squid.

ebtables rules are:

-p IPv4 -i eth0 --ip-proto tcp --ip-dport 80 -j redirect  --redirect-target DROP
-p IPv4 -i eth1 --ip-proto tcp --ip-sport 80 -j redirect  --redirect-target DROP


iptables -t mangle -L -n is:

iptables -t mangle -L -n
Chain PREROUTING (policy ACCEPT)
target     prot opt source               destination
DIVERT     tcp  --  0.0.0.0/0            0.0.0.0/0           socket
extrachain  tcp  --  0.0.0.0/0            0.0.0.0/0           tcp
dpt:80 ctstate NEW
TPROXY     tcp  --  0.0.0.0/0           !201.40.162.5        tcp
dpt:80 connmark match 0x0 TPROXY redirect 127.0.0.1:3127 mark 0x1/0x1
TPROXY     tcp  --  0.0.0.0/0           !201.40.162.5        tcp
dpt:80 connmark match 0x1 TPROXY redirect 127.0.0.1:3128 mark 0x1/0x1
TPROXY     tcp  --  0.0.0.0/0           !201.40.162.5        tcp
dpt:80 connmark match 0x2 TPROXY redirect 127.0.0.1:3129 mark 0x1/0x1

Chain DIVERT (1 references)
target     prot opt source               destination
MARK       all  --  0.0.0.0/0            0.0.0.0/0           MARK xset
0x1/0xffffffff
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0

Chain extrachain (1 references)
target     prot opt source               destination
CONNMARK   all  --  0.0.0.0/0            0.0.0.0/0           statistic
mode nth every 35 CONNMARK and 0x0
CONNMARK   all  --  0.0.0.0/0            0.0.0.0/0           statistic
mode nth every 35 packet 1 CONNMARK xset 0x1/0xffffffff
CONNMARK   all  --  0.0.0.0/0            0.0.0.0/0           statistic
mode nth every 35 packet 2 CONNMARK xset 0x2/0xffffffff

Don't know if the code on these can be traced back to tcp_recvmsg()
accessing some wrong memory address...

Cheers,

Felipe Damasio
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Eric Dumazet July 10, 2010, 6:17 a.m. UTC | #4
Le samedi 10 juillet 2010 à 00:18 -0300, Felipe W Damasio a écrit :
> Hi Mr. Dumazet,
> 
> 2010/7/9 Eric Dumazet <eric.dumazet@gmail.com>:
> > Reviewing tproxy stuff I spotted a problem in nf_tproxy_assign_sock()
> > but I could not see how it could explain your crash.
> >
> > We can read uninitialized memory and trigger a fault in
> > nf_tproxy_assign_sock(), not later in tcp_recvmsg()...
> >
> > David, Patrick, what do you think ?
> 
> But do you think that the bug that squid triggered was caused by the
> TProxy code?
> 

I dont think so, but I was asking David or Patrick another point of
view.

Strange thing with your crash report is CR2 value, with unexpected value
of 000000000b388000 while RAX value is dce8dce85d415d41

Faulting instruction is :

48 83 b8 b0 00 00 00 00   cmpq   $0x0,0xb0(%rax)

So I would have expected CR2 being RAX+0xb0, but its not.



> Or is related to the network-stack in some other point.
> 
> I don't know if this helps, but I'm using ebtables to remove the
> packets from the bridge, and iptables to redirect the traffic to
> squid.
> 
> ebtables rules are:
> 
> -p IPv4 -i eth0 --ip-proto tcp --ip-dport 80 -j redirect  --redirect-target DROP
> -p IPv4 -i eth1 --ip-proto tcp --ip-sport 80 -j redirect  --redirect-target DROP
> 
> 
> iptables -t mangle -L -n is:
> 
> iptables -t mangle -L -n
> Chain PREROUTING (policy ACCEPT)
> target     prot opt source               destination
> DIVERT     tcp  --  0.0.0.0/0            0.0.0.0/0           socket
> extrachain  tcp  --  0.0.0.0/0            0.0.0.0/0           tcp
> dpt:80 ctstate NEW
> TPROXY     tcp  --  0.0.0.0/0           !201.40.162.5        tcp
> dpt:80 connmark match 0x0 TPROXY redirect 127.0.0.1:3127 mark 0x1/0x1
> TPROXY     tcp  --  0.0.0.0/0           !201.40.162.5        tcp
> dpt:80 connmark match 0x1 TPROXY redirect 127.0.0.1:3128 mark 0x1/0x1
> TPROXY     tcp  --  0.0.0.0/0           !201.40.162.5        tcp
> dpt:80 connmark match 0x2 TPROXY redirect 127.0.0.1:3129 mark 0x1/0x1
> 
> Chain DIVERT (1 references)
> target     prot opt source               destination
> MARK       all  --  0.0.0.0/0            0.0.0.0/0           MARK xset
> 0x1/0xffffffff
> ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0
> 
> Chain extrachain (1 references)
> target     prot opt source               destination
> CONNMARK   all  --  0.0.0.0/0            0.0.0.0/0           statistic
> mode nth every 35 CONNMARK and 0x0
> CONNMARK   all  --  0.0.0.0/0            0.0.0.0/0           statistic
> mode nth every 35 packet 1 CONNMARK xset 0x1/0xffffffff
> CONNMARK   all  --  0.0.0.0/0            0.0.0.0/0           statistic
> mode nth every 35 packet 2 CONNMARK xset 0x2/0xffffffff
> 
> Don't know if the code on these can be traced back to tcp_recvmsg()
> accessing some wrong memory address...
> 
> Cheers,
> 
> Felipe Damasio


--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
David Miller July 10, 2010, 7:30 p.m. UTC | #5
From: Eric Dumazet <eric.dumazet@gmail.com>
Date: Sat, 10 Jul 2010 08:17:29 +0200

> Strange thing with your crash report is CR2 value, with unexpected value
> of 000000000b388000 while RAX value is dce8dce85d415d41
> 
> Faulting instruction is :
> 
> 48 83 b8 b0 00 00 00 00   cmpq   $0x0,0xb0(%rax)
> 
> So I would have expected CR2 being RAX+0xb0, but its not.

It could be corruption from elsewhere.  Those last four hex
digits (0x5d415d41) are "]A]A" in ascii, but that could just
be coincidence.

--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Felipe W Damasio July 11, 2010, 3:11 a.m. UTC | #6
Hi Mr. Miller,

2010/7/10 David Miller <davem@davemloft.net>:
> It could be corruption from elsewhere.  Those last four hex
> digits (0x5d415d41) are "]A]A" in ascii, but that could just
> be coincidence.

What do you mean "from elsewhere"? You mean elsewhere on the network code?

Since the function that had the problem was tcp_recvmsg and we're
talking about a squid process, we're either talking about a typical
webserver-objet response, or about about an incorrect/faulty http
request from the user.

Like I told Mr. Dumazet, since on the squid logs I got a:

2010/07/08 14:51:10| clientTryParseRequest: FD 6088
(187.16.240.122:2035) Invalid Request

Only a second before the bug entry on syslog, I suppose that this
invalid request caused the problem (more like a guess, really).

If you think there's a way I can help reproduce/trigger and fix this
bug, please let me know, since the production machine is down until I
can ensure my bosses that this particular crash won't happen again.

Thanks,

Felipe Damasio
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Avi Kivity July 11, 2010, 5:19 a.m. UTC | #7
On 07/10/2010 09:17 AM, Eric Dumazet wrote:
>
> Strange thing with your crash report is CR2 value, with unexpected value
> of 000000000b388000 while RAX value is dce8dce85d415d41
>
> Faulting instruction is :
>
> 48 83 b8 b0 00 00 00 00   cmpq   $0x0,0xb0(%rax)
>
> So I would have expected CR2 being RAX+0xb0, but its not.
>    

Nothing strange about it.  You only get page faults and valid cr2 for 
canonical addresses (17 high order bits all equal).  In this case 
rax+0xb0 is not a canonical address, so you got a general protection 
fault instead, with cr2 unchanged.
Felipe W Damasio July 11, 2010, 7:11 a.m. UTC | #8
Hi,

2010/7/11 Felipe W Damasio <felipewd@gmail.com>:
> What do you mean "from elsewhere"? You mean elsewhere on the network code?
>
> Since the function that had the problem was tcp_recvmsg and we're
> talking about a squid process, we're either talking about a typical
> webserver-objet response, or about about an incorrect/faulty http
> request from the user.

I'm trying to understand the network code to see if I can help track
this down...

So I looked at the code of tcp_recvmsg, and I saw that the function
that calls __kfree_skb is sk_eat_skb, which is called in tcp_recvmsg:

                if (tcp_hdr(skb)->fin)
                        goto found_fin_ok;
                if (!(flags & MSG_PEEK)) {
                        sk_eat_skb(sk, skb, copied_early);
                        copied_early = 0;
                }
                continue;

        found_fin_ok:
                /* Process the FIN. */
                ++*seq;
                if (!(flags & MSG_PEEK)) {
                        sk_eat_skb(sk, skb, copied_early);
                        copied_early = 0;
                }
                break;


  Now, I'm no kernel programmer, but I saw the "Process the FIN" thing
on the code. I'm tuning these proc parameters....maybe some of these
are triggering the bug?

   The production machine has 8GB of RAM:


echo 1 >  /proc/sys/net/ipv4/ip_forward
echo 0 > /proc/sys/net/ipv4/conf/lo/rp_filter
echo 1 > /proc/sys/net/ipv4/ip_nonlocal_bind
echo 1 >  /proc/sys/net/ipv4/tcp_low_latency

echo 0 > /proc/sys/net/ipv4/conf/eth2/rp_filter
echo 0 > /proc/sys/net/ipv4/conf/eth1/rp_filter
echo 0 > /proc/sys/net/ipv4/conf/eth0/rp_filter
echo 0 > /proc/sys/net/ipv4/conf/br0/rp_filter

echo 1 > /proc/sys/net/ipv4/conf/all/forwarding
echo 1 > /proc/sys/net/ipv4/conf/all/send_redirects
echo 1 > /proc/sys/net/ipv4/conf/eth0/send_redirects

echo 16384 > /proc/sys/net/ipv4/neigh/default/gc_thresh1
echo 32768 > /proc/sys/net/ipv4/neigh/default/gc_thresh2
echo 65535 > /proc/sys/net/ipv4/neigh/default/gc_thresh3

echo 1024 65535 > /proc/sys/net/ipv4/ip_local_port_range

echo 0 > /proc/sys/net/ipv4/conf/all/rp_filter
echo 0 > /proc/sys/net/ipv4/tcp_ecn
echo 1 > /proc/sys/net/ipv4/tcp_low_latency
echo 100000 > /proc/sys/net/core/netdev_max_backlog
echo 409600  > /proc/sys/net/ipv4/tcp_max_syn_backlog


echo 3 > /proc/sys/net/ipv4/tcp_fin_timeout
echo 15 > /proc/sys/net/ipv4/tcp_keepalive_intvl
echo 3 > /proc/sys/net/ipv4/tcp_keepalive_probes
echo 65536 > /proc/sys/vm/min_free_kbytes
echo "262144 1024000 4194304" > /proc/sys/net/ipv4/tcp_rmem
echo "262144 1024000 4194304" > /proc/sys/net/ipv4/tcp_wmem
echo "1024000" > /proc/sys/net/core/rmem_max
echo "1024000" > /proc/sys/net/core/wmem_max
echo "512000" > /proc/sys/net/core/rmem_default
echo "512000" > /proc/sys/net/core/wmem_default
echo "524288" > /proc/sys/net/ipv4/netfilter/ip_conntrack_max

echo "2" > /proc/sys/net/ipv4/tcp_synack_retries
echo "2" > /proc/sys/net/ipv4/tcp_syn_retries
echo "262144" > /proc/sys/net/ipv4/tcp_max_orphans
echo "262144" > /proc/sys/net/core/somaxconn

  Maybe these can help?

  Cheers,

Felipe Damasio
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Felipe W Damasio July 11, 2010, 7:13 a.m. UTC | #9
2010/7/11 Felipe W Damasio <felipewd@gmail.com>:
>   The production machine has 8GB of RAM:

  I'm sorry, this is not right. The production machine has 16GB of RAM.

  Don't know if that matters regarding those proc parameters, though.

  Cheers,

Felipe Damasio
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Eric Dumazet July 11, 2010, 8:02 a.m. UTC | #10
Le dimanche 11 juillet 2010 à 08:19 +0300, Avi Kivity a écrit :
> On 07/10/2010 09:17 AM, Eric Dumazet wrote:
> >
> > Strange thing with your crash report is CR2 value, with unexpected value
> > of 000000000b388000 while RAX value is dce8dce85d415d41
> >
> > Faulting instruction is :
> >
> > 48 83 b8 b0 00 00 00 00   cmpq   $0x0,0xb0(%rax)
> >
> > So I would have expected CR2 being RAX+0xb0, but its not.
> >    
> 
> Nothing strange about it.  You only get page faults and valid cr2 for 
> canonical addresses (17 high order bits all equal).  In this case 
> rax+0xb0 is not a canonical address, so you got a general protection 
> fault instead, with cr2 unchanged.
> 

OK, thanks Avi for this information, as I was not aware of this.

So something overwrote sk->sk_prot pointer (or skb->sk pointer) with
some data.

tcp sockets are allocated from a dedicated kmem_cache (because of
SLAB_DESTROY_RCU attribute). Their sk->sk_prot should never change in
normal operation, since underlying memory cannot be reused by another
object type in kernel. It should be NULL or &tcp_prot

Felipe, please describe your configuration as much as possible.
It might be a driver bug with with special kind of network frames.

lsmod
lspci -v
ethtool -k eth0
ethtool -k eth1 (if applicable)



--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Eric Dumazet July 11, 2010, 8:36 a.m. UTC | #11
Le samedi 10 juillet 2010 à 12:30 -0700, David Miller a écrit :
> From: Eric Dumazet <eric.dumazet@gmail.com>
> Date: Sat, 10 Jul 2010 08:17:29 +0200
> 
> > Strange thing with your crash report is CR2 value, with unexpected value
> > of 000000000b388000 while RAX value is dce8dce85d415d41
> > 
> > Faulting instruction is :
> > 
> > 48 83 b8 b0 00 00 00 00   cmpq   $0x0,0xb0(%rax)
> > 
> > So I would have expected CR2 being RAX+0xb0, but its not.
> 
> It could be corruption from elsewhere.  Those last four hex
> digits (0x5d415d41) are "]A]A" in ascii, but that could just
> be coincidence.
> 

x86 being litle endian, string is "A]A]" followed by another "XYXY"
pattern (non ASCII chars : 0xE8, 0xDC, 0xE8, 0xDC, "èÜèÜ" in ISO8859)



--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Felipe W Damasio July 12, 2010, 12:52 a.m. UTC | #12
Hi Mr. Dumazet,

2010/7/11 Eric Dumazet <eric.dumazet@gmail.com>:
> Felipe, please describe your configuration as much as possible.
> It might be a driver bug with with special kind of network frames.
>
> lsmod
> lspci -v
> ethtool -k eth0
> ethtool -k eth1 (if applicable)

Sure.

lsmod is empty. Though .config is attached.

We're using eth1 facing the user and eth2 facing the internet.

ethtool -k eth1
Offload parameters for eth1:
rx-checksumming: on
tx-checksumming: on
scatter-gather: on
tcp segmentation offload: on
udp fragmentation offload: off
generic segmentation offload: on


ethtool -k eth2
Offload parameters for eth2:
rx-checksumming: on
tx-checksumming: on
scatter-gather: on
tcp segmentation offload: on
udp fragmentation offload: off
generic segmentation offload: on


Dmesg output from a normal boot is also attached.

Cheers,

Felipe Damasio
Initializing cgroup subsys cpuset
Initializing cgroup subsys cpu
Linux version 2.6.34 (root@livecd) (gcc version 4.3.4 (Gentoo 4.3.4 p1.0, pie-10.1.5) ) #1 SMP Thu Jul 1 18:47:08 BRT 2010
Command line: root=/dev/sda1
BIOS-provided physical RAM map:
BIOS-e820: 0000000000000000 - 000000000008f000 (usable)
BIOS-e820: 000000000008f000 - 0000000000090000 (reserved)
BIOS-e820: 0000000000090000 - 000000000009d000 (usable)
BIOS-e820: 000000000009d000 - 00000000000a0000 (reserved)
BIOS-e820: 00000000000e0000 - 0000000000100000 (reserved)
BIOS-e820: 0000000000100000 - 00000000cf4bd000 (usable)
BIOS-e820: 00000000cf4bd000 - 00000000cf4bf000 (reserved)
BIOS-e820: 00000000cf4bf000 - 00000000cf4c4000 (usable)
BIOS-e820: 00000000cf4c4000 - 00000000cf7bf000 (ACPI NVS)
BIOS-e820: 00000000cf7bf000 - 00000000cf7df000 (usable)
BIOS-e820: 00000000cf7df000 - 00000000cf7ff000 (ACPI data)
BIOS-e820: 00000000cf7ff000 - 00000000cf800000 (usable)
BIOS-e820: 00000000cf800000 - 00000000d0000000 (reserved)
BIOS-e820: 00000000f8000000 - 00000000fd000000 (reserved)
BIOS-e820: 00000000ffe00000 - 0000000100000000 (reserved)
BIOS-e820: 0000000100000000 - 0000000430000000 (usable)
NX (Execute Disable) protection: active
DMI 2.5 present.
e820 update range: 0000000000000000 - 0000000000001000 (usable) ==> (reserved)
e820 remove range: 00000000000a0000 - 0000000000100000 (usable)
No AGP bridge found
last_pfn = 0x430000 max_arch_pfn = 0x400000000
MTRR default type: uncachable
MTRR fixed ranges enabled:
00000-9FFFF write-back
A0000-BFFFF uncachable
C0000-DFFFF write-protect
E0000-FFFFF uncachable
MTRR variable ranges enabled:
0 base 000000000 mask F80000000 write-back
1 base 080000000 mask FC0000000 write-back
2 base 0C0000000 mask FF0000000 write-back
3 base 100000000 mask F00000000 write-back
4 base 200000000 mask E00000000 write-back
5 base 400000000 mask FE0000000 write-back
6 base 420000000 mask FF0000000 write-back
7 base 0FFFF0000 mask FFFFF0000 write-protect
x86 PAT enabled: cpu 0, old 0x7040600070406, new 0x7010600070106
last_pfn = 0xcf800 max_arch_pfn = 0x400000000
e820 update range: 0000000000001000 - 0000000000010000 (usable) ==> (reserved)
Scanning 1 areas for low memory corruption
modified physical RAM map:
modified: 0000000000000000 - 0000000000010000 (reserved)
modified: 0000000000010000 - 000000000008f000 (usable)
modified: 000000000008f000 - 0000000000090000 (reserved)
modified: 0000000000090000 - 000000000009d000 (usable)
modified: 000000000009d000 - 00000000000a0000 (reserved)
modified: 00000000000e0000 - 0000000000100000 (reserved)
modified: 0000000000100000 - 00000000cf4bd000 (usable)
modified: 00000000cf4bd000 - 00000000cf4bf000 (reserved)
modified: 00000000cf4bf000 - 00000000cf4c4000 (usable)
modified: 00000000cf4c4000 - 00000000cf7bf000 (ACPI NVS)
modified: 00000000cf7bf000 - 00000000cf7df000 (usable)
modified: 00000000cf7df000 - 00000000cf7ff000 (ACPI data)
modified: 00000000cf7ff000 - 00000000cf800000 (usable)
modified: 00000000cf800000 - 00000000d0000000 (reserved)
modified: 00000000f8000000 - 00000000fd000000 (reserved)
modified: 00000000ffe00000 - 0000000100000000 (reserved)
modified: 0000000100000000 - 0000000430000000 (usable)
initial memory mapped : 0 - 20000000
init_memory_mapping: 0000000000000000-00000000cf800000
0000000000 - 00cf800000 page 2M
kernel direct mapping tables up to cf800000 @ 16000-1b000
init_memory_mapping: 0000000100000000-0000000430000000
0100000000 - 0430000000 page 2M
kernel direct mapping tables up to 430000000 @ 19000-2b000
ACPI: RSDP 00000000000fe020 00024 (v02 INTEL )
ACPI: XSDT 00000000cf7fe120 00074 (v01 INTEL  DX58SO   000011F6      01000013)
ACPI: FACP 00000000cf7fd000 000F4 (v03 INTEL  DX58SO   000011F6 MSFT 0100000D)
ACPI Warning: Optional field Pm2ControlBlock has zero address or length: 0000000000000450/0 (20100121/tbfadt-557)
ACPI Warning: Invalid length for Pm2ControlBlock: 0, using default 8 (20100121/tbfadt-607)
ACPI: DSDT 00000000cf7f8000 04641 (v02 INTEL  DX58SO   000011F6 MSFT 0100000D)
ACPI: FACS 00000000cf728000 00040
ACPI: APIC 00000000cf7f7000 00138 (v02 INTEL  DX58SO   000011F6 MSFT 0100000D)
ACPI: WDDT 00000000cf7f6000 00040 (v01 INTEL  DX58SO   000011F6 MSFT 0100000D)
ACPI: MCFG 00000000cf7f5000 0003C (v01 INTEL  DX58SO   000011F6 MSFT 0100000D)
ACPI: ASF! 00000000cf7f4000 000AC (v32 INTEL  DX58SO   000011F6 MSFT 0100000D)
ACPI: HPET 00000000cf7f3000 00038 (v01 INTEL  DX58SO   000011F6 MSFT 0100000D)
ACPI: SSDT 00000000cf7e2000 0E134 (v01 INTEL  SSDT  PM 000011F6 MSFT 0100000D)
ACPI: DMAR 00000000cf7df000 00140 (v01 INTEL  DX58SO   000011F6 MSFT 0100000D)
ACPI: WDTT 00000000cf7f1000 0020C (v02 INTEL  DX58SO   000011F6 MSFT 0100000D)
ACPI: ASPT 00000000cf7f2000 00034 (v04 INTEL  PerfTune 000011F6 MSFT 0100000D)
ACPI: Local APIC address 0xfee00000
[ffffea0000000000-ffffea000ebfffff] PMD -> [ffff880002000000-ffff8800101fffff] on node 0
Zone PFN ranges:
DMA      0x00000010 -> 0x00001000
DMA32    0x00001000 -> 0x00100000
Normal   0x00100000 -> 0x00430000
Movable zone start PFN for each node
early_node_map[7] active PFN ranges
0: 0x00000010 -> 0x0000008f
0: 0x00000090 -> 0x0000009d
0: 0x00000100 -> 0x000cf4bd
0: 0x000cf4bf -> 0x000cf4c4
0: 0x000cf7bf -> 0x000cf7df
0: 0x000cf7ff -> 0x000cf800
0: 0x00100000 -> 0x00430000
On node 0 totalpages: 4191343
DMA zone: 56 pages used for memmap
DMA zone: 0 pages reserved
DMA zone: 3924 pages, LIFO batch:0
DMA32 zone: 14280 pages used for memmap
DMA32 zone: 830747 pages, LIFO batch:31
Normal zone: 45696 pages used for memmap
Normal zone: 3296640 pages, LIFO batch:31
ACPI: PM-Timer IO Port: 0x408
ACPI: Local APIC address 0xfee00000
ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
ACPI: LAPIC (acpi_id[0x01] lapic_id[0x02] enabled)
ACPI: LAPIC (acpi_id[0x02] lapic_id[0x04] enabled)
ACPI: LAPIC (acpi_id[0x03] lapic_id[0x06] enabled)
ACPI: LAPIC (acpi_id[0x04] lapic_id[0x01] enabled)
ACPI: LAPIC (acpi_id[0x05] lapic_id[0x03] enabled)
ACPI: LAPIC (acpi_id[0x06] lapic_id[0x05] enabled)
ACPI: LAPIC (acpi_id[0x07] lapic_id[0x07] enabled)
ACPI: LAPIC (acpi_id[0x01] lapic_id[0x10] disabled)
ACPI: LAPIC (acpi_id[0x03] lapic_id[0x12] disabled)
ACPI: LAPIC (acpi_id[0x05] lapic_id[0x14] disabled)
ACPI: LAPIC (acpi_id[0x07] lapic_id[0x16] disabled)
ACPI: LAPIC (acpi_id[0x09] lapic_id[0x11] disabled)
ACPI: LAPIC (acpi_id[0x0b] lapic_id[0x13] disabled)
ACPI: LAPIC (acpi_id[0x0d] lapic_id[0x15] disabled)
ACPI: LAPIC (acpi_id[0x0f] lapic_id[0x17] disabled)
ACPI: LAPIC_NMI (acpi_id[0x00] high level lint[0x1])
ACPI: LAPIC_NMI (acpi_id[0x01] high level lint[0x1])
ACPI: LAPIC_NMI (acpi_id[0x02] high level lint[0x1])
ACPI: LAPIC_NMI (acpi_id[0x03] high level lint[0x1])
ACPI: LAPIC_NMI (acpi_id[0x04] high level lint[0x1])
ACPI: LAPIC_NMI (acpi_id[0x05] high level lint[0x1])
ACPI: LAPIC_NMI (acpi_id[0x06] high level lint[0x1])
ACPI: LAPIC_NMI (acpi_id[0x07] high level lint[0x1])
ACPI: LAPIC_NMI (acpi_id[0x08] high level lint[0x1])
ACPI: LAPIC_NMI (acpi_id[0x09] high level lint[0x1])
ACPI: LAPIC_NMI (acpi_id[0x0a] high level lint[0x1])
ACPI: LAPIC_NMI (acpi_id[0x0b] high level lint[0x1])
ACPI: LAPIC_NMI (acpi_id[0x0c] high level lint[0x1])
ACPI: LAPIC_NMI (acpi_id[0x0d] high level lint[0x1])
ACPI: LAPIC_NMI (acpi_id[0x0e] high level lint[0x1])
ACPI: LAPIC_NMI (acpi_id[0x0f] high level lint[0x1])
ACPI: IOAPIC (id[0x08] address[0xfec00000] gsi_base[0])
IOAPIC[0]: apic_id 8, version 32, address 0xfec00000, GSI 0-23
ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
ACPI: IRQ0 used by override.
ACPI: IRQ2 used by override.
ACPI: IRQ9 used by override.
Using ACPI (MADT) for SMP configuration information
ACPI: HPET id: 0x8086a301 base: 0xfed00000
16 Processors exceeds NR_CPUS limit of 8
SMP: Allowing 8 CPUs, 0 hotplug CPUs
nr_irqs_gsi: 24
early_res array is doubled to 64 at [26000 - 267ff]
PM: Registered nosave memory: 000000000008f000 - 0000000000090000
PM: Registered nosave memory: 000000000009d000 - 00000000000a0000
PM: Registered nosave memory: 00000000000a0000 - 00000000000e0000
PM: Registered nosave memory: 00000000000e0000 - 0000000000100000
PM: Registered nosave memory: 00000000cf4bd000 - 00000000cf4bf000
PM: Registered nosave memory: 00000000cf4c4000 - 00000000cf7bf000
PM: Registered nosave memory: 00000000cf7df000 - 00000000cf7ff000
PM: Registered nosave memory: 00000000cf800000 - 00000000d0000000
PM: Registered nosave memory: 00000000d0000000 - 00000000f8000000
PM: Registered nosave memory: 00000000f8000000 - 00000000fd000000
PM: Registered nosave memory: 00000000fd000000 - 00000000ffe00000
PM: Registered nosave memory: 00000000ffe00000 - 0000000100000000
Allocating PCI resources starting at d0000000 (gap: d0000000:28000000)
setup_percpu: NR_CPUS:8 nr_cpumask_bits:8 nr_cpu_ids:8 nr_node_ids:1
PERCPU: Embedded 26 pages/cpu @ffff880001800000 s75880 r8192 d22424 u262144
early_res array is doubled to 128 at [26800 - 277ff]
pcpu-alloc: s75880 r8192 d22424 u262144 alloc=1*2097152
pcpu-alloc: [0] 0 1 2 3 4 5 6 7 
Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 4131311
Kernel command line: root=/dev/sda1
PID hash table entries: 4096 (order: 3, 32768 bytes)
Dentry cache hash table entries: 2097152 (order: 12, 16777216 bytes)
Inode-cache hash table entries: 1048576 (order: 11, 8388608 bytes)
Checking aperture...
No AGP bridge found
Subtract (66 early reservations)
#1 [0001000000 - 0001751214]   TEXT DATA BSS
#2 [000009d000 - 0000100000]   BIOS reserved
#3 [0001752000 - 000175215e]             BRK
#4 [0000010000 - 0000012000]      TRAMPOLINE
#5 [0000012000 - 0000016000]     ACPI WAKEUP
#6 [0000016000 - 0000019000]         PGTABLE
#7 [0000019000 - 0000026000]         PGTABLE
#8 [0001752180 - 0001753180]         BOOTMEM
#9 [0001751240 - 0001751e40]         BOOTMEM
#10 [0001f54000 - 0001f55000]         BOOTMEM
#11 [0001f55000 - 0001f56000]         BOOTMEM
#12 [0002000000 - 0010200000]        MEMMAP 0
#13 [0001751e40 - 0001751fc0]         BOOTMEM
#14 [0001753180 - 000176b180]         BOOTMEM
#15 [000176b180 - 0001783180]         BOOTMEM
#16 [0001784000 - 0001785000]         BOOTMEM
#17 [0001783180 - 00017831c1]         BOOTMEM
#18 [0001783200 - 0001783243]         BOOTMEM
#19 [0001783280 - 0001783638]         BOOTMEM
#20 [0001783640 - 00017836a8]         BOOTMEM
#21 [00017836c0 - 0001783728]         BOOTMEM
#22 [0001783740 - 00017837a8]         BOOTMEM
#23 [00017837c0 - 0001783828]         BOOTMEM
#24 [0001783840 - 00017838a8]         BOOTMEM
#25 [00017838c0 - 0001783928]         BOOTMEM
#26 [0001783940 - 00017839a8]         BOOTMEM
#27 [00017839c0 - 0001783a28]         BOOTMEM
#28 [0001783a40 - 0001783aa8]         BOOTMEM
#29 [0001783ac0 - 0001783b28]         BOOTMEM
#30 [0001783b40 - 0001783ba8]         BOOTMEM
#31 [0001783bc0 - 0001783c28]         BOOTMEM
#32 [0001783c40 - 0001783ca8]         BOOTMEM
#33 [0001783cc0 - 0001783d28]         BOOTMEM
#34 [0001783d40 - 0001783da8]         BOOTMEM
#35 [0001783dc0 - 0001783e28]         BOOTMEM
#36 [0001751fc0 - 0001751fe0]         BOOTMEM
#37 [0001783e40 - 0001783e60]         BOOTMEM
#38 [0001783e80 - 0001783ea0]         BOOTMEM
#39 [0001783ec0 - 0001783ee0]         BOOTMEM
#40 [0001783f00 - 0001783f20]         BOOTMEM
#41 [0001783f40 - 0001783f60]         BOOTMEM
#42 [0001783f80 - 0001783f8f]         BOOTMEM
#43 [0001783fc0 - 0001783fcf]         BOOTMEM
#44 [0001800000 - 000181a000]         BOOTMEM
#45 [0001840000 - 000185a000]         BOOTMEM
#46 [0001880000 - 000189a000]         BOOTMEM
#47 [00018c0000 - 00018da000]         BOOTMEM
#48 [0001900000 - 000191a000]         BOOTMEM
#49 [0001940000 - 000195a000]         BOOTMEM
#50 [0001980000 - 000199a000]         BOOTMEM
#51 [00019c0000 - 00019da000]         BOOTMEM
#52 [0001787000 - 0001787008]         BOOTMEM
#53 [0001787040 - 0001787048]         BOOTMEM
#54 [0001787080 - 00017870a0]         BOOTMEM
#55 [00017870c0 - 0001787100]         BOOTMEM
#56 [0001787100 - 0001787220]         BOOTMEM
#57 [0001787240 - 0001787288]         BOOTMEM
#58 [00017872c0 - 0001787308]         BOOTMEM
#59 [0001787340 - 000178f340]         BOOTMEM
#60 [0010200000 - 0011200000]         BOOTMEM
#61 [0011200000 - 0011a00000]         BOOTMEM
#62 [0011a00000 - 0015a00000]         BOOTMEM
#63 [000178f340 - 00017af340]         BOOTMEM
#64 [00017af340 - 00017ef340]         BOOTMEM
#65 [0000027800 - 000002f800]         BOOTMEM
Memory: 16434752k/17563648k available (4275k kernel code, 798276k absent, 330620k reserved, 2219k data, 444k init)
SLUB: Genslabs=13, HWalign=64, Order=0-3, MinObjects=0, CPUs=8, Nodes=1
Hierarchical RCU implementation.
NR_IRQS:4352 nr_irqs:472
Extended CMOS year: 2000
Console: colour VGA+ 80x25
console [tty0] enabled
hpet clockevent registered
Fast TSC calibration using PIT
Detected 2666.892 MHz processor.
Calibrating delay loop (skipped), value calculated using timer frequency.. 5333.78 BogoMIPS (lpj=2666892)
Security Framework initialized
SELinux:  Initializing.
SELinux:  Starting in permissive mode
Mount-cache hash table entries: 256
Initializing cgroup subsys ns
Initializing cgroup subsys cpuacct
Initializing cgroup subsys freezer
CPU: Physical Processor ID: 0
CPU: Processor Core ID: 0
mce: CPU supports 9 MCE banks
CPU0: Thermal monitoring enabled (TM1)
using mwait in idle threads.
Performance Events: Nehalem/Corei7 events, Intel PMU driver.
... version:                3
... bit width:              48
... generic registers:      4
... value mask:             0000ffffffffffff
... max period:             000000007fffffff
... fixed-purpose events:   3
... event mask:             000000070000000f
ACPI: Core revision 20100121
Setting APIC routing to flat
..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
CPU0: Intel(R) Core(TM) i7 CPU         920  @ 2.67GHz stepping 05
Booting Node   0, Processors  #1 #2 #3 #4 #5 #6 #7 Ok.
Brought up 8 CPUs
Total of 8 processors activated (42661.40 BogoMIPS).
NET: Registered protocol family 16
ACPI FADT declares the system doesn't support PCIe ASPM, so disable it
ACPI: bus type pci registered
PCI: MMCONFIG for domain 0000 [bus 00-3f] at [mem 0xf8000000-0xfbffffff] (base 0xf8000000)
PCI: MMCONFIG at [mem 0xf8000000-0xfbffffff] reserved in E820
PCI: Using configuration type 1 for base access
bio: create slab <bio-0> at 0
ACPI: EC: Look up EC in DSDT
ACPI: Interpreter enabled
ACPI: (supports S0 S1 S3 S4 S5)
ACPI: Using IOAPIC for interrupt routing
ACPI: No dock devices found.
PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug
ACPI: PCI Root Bridge [PCI0] (0000:00)
pci_root PNP0A08:00: host bridge window [io  0x0000-0x0cf7]
pci_root PNP0A08:00: host bridge window [io  0x0d00-0xffff]
pci_root PNP0A08:00: host bridge window [mem 0x000a0000-0x000bffff]
pci_root PNP0A08:00: host bridge window [mem 0x000c4000-0x000cbfff]
pci_root PNP0A08:00: host bridge window [mem 0xfed40000-0xfedfffff]
pci_root PNP0A08:00: host bridge window [mem 0xd0000000-0xf7ffffff]
pci 0000:00:01.0: PME# supported from D0 D3hot D3cold
pci 0000:00:01.0: PME# disabled
pci 0000:00:03.0: PME# supported from D0 D3hot D3cold
pci 0000:00:03.0: PME# disabled
pci 0000:00:07.0: PME# supported from D0 D3hot D3cold
pci 0000:00:07.0: PME# disabled
pci 0000:00:19.0: reg 10: [mem 0xd0500000-0xd051ffff]
pci 0000:00:19.0: reg 14: [mem 0xd0523000-0xd0523fff]
pci 0000:00:19.0: reg 18: [io  0x5100-0x511f]
pci 0000:00:19.0: PME# supported from D0 D3hot D3cold
pci 0000:00:19.0: PME# disabled
pci 0000:00:1a.0: reg 20: [io  0x50e0-0x50ff]
pci 0000:00:1a.1: reg 20: [io  0x50c0-0x50df]
pci 0000:00:1a.2: reg 20: [io  0x50a0-0x50bf]
pci 0000:00:1a.7: reg 10: [mem 0xd0522000-0xd05223ff]
pci 0000:00:1a.7: PME# supported from D0 D3hot D3cold
pci 0000:00:1a.7: PME# disabled
pci 0000:00:1b.0: reg 10: [mem 0xf0000000-0xf0003fff 64bit]
pci 0000:00:1b.0: PME# supported from D0 D3hot D3cold
pci 0000:00:1b.0: PME# disabled
pci 0000:00:1c.0: PME# supported from D0 D3hot D3cold
pci 0000:00:1c.0: PME# disabled
pci 0000:00:1c.1: PME# supported from D0 D3hot D3cold
pci 0000:00:1c.1: PME# disabled
pci 0000:00:1c.4: PME# supported from D0 D3hot D3cold
pci 0000:00:1c.4: PME# disabled
pci 0000:00:1d.0: reg 20: [io  0x5080-0x509f]
pci 0000:00:1d.1: reg 20: [io  0x5060-0x507f]
pci 0000:00:1d.2: reg 20: [io  0x5040-0x505f]
pci 0000:00:1d.7: reg 10: [mem 0xd0521000-0xd05213ff]
pci 0000:00:1d.7: PME# supported from D0 D3hot D3cold
pci 0000:00:1d.7: PME# disabled
pci 0000:00:1f.0: quirk: [io  0x0400-0x047f] claimed by ICH6 ACPI/GPIO/TCO
pci 0000:00:1f.0: quirk: [io  0x0500-0x053f] claimed by ICH6 GPIO
pci 0000:00:1f.0: ICH7 LPC Generic IO decode 1 PIO at 0680 (mask 007f)
pci 0000:00:1f.0: ICH7 LPC Generic IO decode 2 PIO at 0810 (mask 007f)
pci 0000:00:1f.2: reg 10: [io  0x5128-0x512f]
pci 0000:00:1f.2: reg 14: [io  0x5134-0x5137]
pci 0000:00:1f.2: reg 18: [io  0x5120-0x5127]
pci 0000:00:1f.2: reg 1c: [io  0x5130-0x5133]
pci 0000:00:1f.2: reg 20: [io  0x5020-0x503f]
pci 0000:00:1f.2: reg 24: [mem 0xd0520000-0xd05207ff]
pci 0000:00:1f.2: PME# supported from D3hot
pci 0000:00:1f.2: PME# disabled
pci 0000:00:1f.3: reg 10: [mem 0xf0004000-0xf00040ff 64bit]
pci 0000:00:1f.3: reg 20: [io  0x5000-0x501f]
pci 0000:01:00.0: reg 10: [mem 0xd0480000-0xd049ffff]
pci 0000:01:00.0: reg 14: [mem 0xd0400000-0xd047ffff]
pci 0000:01:00.0: reg 18: [io  0x4000-0x401f]
pci 0000:01:00.0: reg 1c: [mem 0xd04a0000-0xd04a3fff]
pci 0000:01:00.0: PME# supported from D0 D3hot D3cold
pci 0000:01:00.0: PME# disabled
pci 0000:00:01.0: PCI bridge to [bus 01-01]
pci 0000:00:01.0:   bridge window [io  0x4000-0x4fff]
pci 0000:00:01.0:   bridge window [mem 0xd0400000-0xd04fffff]
pci 0000:00:01.0:   bridge window [mem 0xfff00000-0x000fffff pref] (disabled)
pci 0000:00:03.0: PCI bridge to [bus 02-02]
pci 0000:00:03.0:   bridge window [io  0xf000-0x0000] (disabled)
pci 0000:00:03.0:   bridge window [mem 0xfff00000-0x000fffff] (disabled)
pci 0000:00:03.0:   bridge window [mem 0xfff00000-0x000fffff pref] (disabled)
pci 0000:03:00.0: reg 10: [mem 0xe0000000-0xefffffff 64bit pref]
pci 0000:03:00.0: reg 18: [mem 0xd0300000-0xd030ffff 64bit]
pci 0000:03:00.0: reg 20: [io  0x3000-0x30ff]
pci 0000:03:00.0: reg 30: [mem 0xfffe0000-0xffffffff pref]
pci 0000:03:00.0: supports D1 D2
pci 0000:03:00.1: reg 10: [mem 0xd0310000-0xd0313fff 64bit]
pci 0000:03:00.1: supports D1 D2
pci 0000:00:07.0: PCI bridge to [bus 03-03]
pci 0000:00:07.0:   bridge window [io  0x3000-0x3fff]
pci 0000:00:07.0:   bridge window [mem 0xd0300000-0xd03fffff]
pci 0000:00:07.0:   bridge window [mem 0xe0000000-0xefffffff 64bit pref]
pci 0000:04:00.0: reg 10: [mem 0xd0280000-0xd029ffff]
pci 0000:04:00.0: reg 14: [mem 0xd0200000-0xd027ffff]
pci 0000:04:00.0: reg 18: [io  0x2000-0x201f]
pci 0000:04:00.0: reg 1c: [mem 0xd02a0000-0xd02a3fff]
pci 0000:04:00.0: PME# supported from D0 D3hot D3cold
pci 0000:04:00.0: PME# disabled
pci 0000:00:1c.0: PCI bridge to [bus 04-04]
pci 0000:00:1c.0:   bridge window [io  0x2000-0x2fff]
pci 0000:00:1c.0:   bridge window [mem 0xd0200000-0xd02fffff]
pci 0000:00:1c.0:   bridge window [mem 0xfff00000-0x000fffff pref] (disabled)
pci 0000:00:1c.1: PCI bridge to [bus 05-05]
pci 0000:00:1c.1:   bridge window [io  0xf000-0x0000] (disabled)
pci 0000:00:1c.1:   bridge window [mem 0xfff00000-0x000fffff] (disabled)
pci 0000:00:1c.1:   bridge window [mem 0xfff00000-0x000fffff pref] (disabled)
pci 0000:06:00.0: reg 10: [io  0x1018-0x101f]
pci 0000:06:00.0: reg 14: [io  0x1024-0x1027]
pci 0000:06:00.0: reg 18: [io  0x1010-0x1017]
pci 0000:06:00.0: reg 1c: [io  0x1020-0x1023]
pci 0000:06:00.0: reg 20: [io  0x1000-0x100f]
pci 0000:06:00.0: reg 24: [mem 0xd0100000-0xd01003ff]
pci 0000:06:00.0: supports D1
pci 0000:06:00.0: PME# supported from D0 D1 D3hot
pci 0000:06:00.0: PME# disabled
pci 0000:00:1c.4: PCI bridge to [bus 06-06]
pci 0000:00:1c.4:   bridge window [io  0x1000-0x1fff]
pci 0000:00:1c.4:   bridge window [mem 0xd0100000-0xd01fffff]
pci 0000:00:1c.4:   bridge window [mem 0xfff00000-0x000fffff pref] (disabled)
pci 0000:07:03.0: reg 10: [mem 0xd0004000-0xd00047ff]
pci 0000:07:03.0: reg 14: [mem 0xd0000000-0xd0003fff]
pci 0000:07:03.0: supports D1 D2
pci 0000:07:03.0: PME# supported from D0 D1 D2 D3hot
pci 0000:07:03.0: PME# disabled
pci 0000:00:1e.0: PCI bridge to [bus 07-07] (subtractive decode)
pci 0000:00:1e.0:   bridge window [io  0xf000-0x0000] (disabled)
pci 0000:00:1e.0:   bridge window [mem 0xd0000000-0xd00fffff]
pci 0000:00:1e.0:   bridge window [mem 0xfff00000-0x000fffff pref] (disabled)
pci 0000:00:1e.0:   bridge window [io  0x0000-0x0cf7] (subtractive decode)
pci 0000:00:1e.0:   bridge window [io  0x0d00-0xffff] (subtractive decode)
pci 0000:00:1e.0:   bridge window [mem 0x000a0000-0x000bffff] (subtractive decode)
pci 0000:00:1e.0:   bridge window [mem 0x000c4000-0x000cbfff] (subtractive decode)
pci 0000:00:1e.0:   bridge window [mem 0xfed40000-0xfedfffff] (subtractive decode)
pci 0000:00:1e.0:   bridge window [mem 0xd0000000-0xf7ffffff] (subtractive decode)
pci_bus 0000:00: on NUMA node 0
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.P32_._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PEG1._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PEG3._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PEG7._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PEX0._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PEX1._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PEX4._PRT]
ACPI: PCI Interrupt Link [LNKA] (IRQs 3 4 5 6 7 9 10 *11 12 14 15)
ACPI: PCI Interrupt Link [LNKB] (IRQs 3 4 5 6 7 9 *10 11 12 14 15)
ACPI: PCI Interrupt Link [LNKC] (IRQs 3 4 *5 6 7 9 10 11 12 14 15)
ACPI: PCI Interrupt Link [LNKD] (IRQs 3 4 5 6 7 9 *10 11 12 14 15)
ACPI: PCI Interrupt Link [LNKE] (IRQs 3 4 *5 6 7 9 10 11 12 14 15)
ACPI: PCI Interrupt Link [LNKF] (IRQs 3 4 5 6 7 9 10 *11 12 14 15)
ACPI: PCI Interrupt Link [LNKG] (IRQs 3 4 5 6 7 9 10 *11 12 14 15)
ACPI: PCI Interrupt Link [LNKH] (IRQs 3 4 5 6 7 9 *10 11 12 14 15)
vgaarb: device added: PCI:0000:03:00.0,decodes=io+mem,owns=io+mem,locks=none
vgaarb: loaded
SCSI subsystem initialized
libata version 3.00 loaded.
usbcore: registered new interface driver usbfs
usbcore: registered new interface driver hub
usbcore: registered new device driver usb
PCI: Using ACPI for IRQ routing
PCI: pci_cache_line_size set to 64 bytes
reserve RAM buffer: 000000000008f000 - 000000000008ffff 
reserve RAM buffer: 000000000009d000 - 000000000009ffff 
reserve RAM buffer: 00000000cf4bd000 - 00000000cfffffff 
reserve RAM buffer: 00000000cf4c4000 - 00000000cfffffff 
reserve RAM buffer: 00000000cf7df000 - 00000000cfffffff 
reserve RAM buffer: 00000000cf800000 - 00000000cfffffff 
HPET: 4 timers in total, 0 timers will be used for per-cpu timer
hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0, 0
hpet0: 4 comparators, 64-bit 14.318180 MHz counter
Switching to clocksource tsc
pnp: PnP ACPI init
ACPI: bus type pnp registered
pnp: PnP ACPI: found 9 devices
ACPI: ACPI bus type pnp unregistered
system 00:05: [io  0x0500-0x053f] has been reserved
system 00:05: [io  0x0400-0x047f] has been reserved
system 00:05: [io  0x0680-0x06ff] has been reserved
pci 0000:03:00.0: no compatible bridge window for [mem 0xfffe0000-0xffffffff pref]
pci 0000:00:01.0: PCI bridge to [bus 01-01]
pci 0000:00:01.0:   bridge window [io  0x4000-0x4fff]
pci 0000:00:01.0:   bridge window [mem 0xd0400000-0xd04fffff]
pci 0000:00:01.0:   bridge window [mem pref disabled]
pci 0000:00:03.0: PCI bridge to [bus 02-02]
pci 0000:00:03.0:   bridge window [io  disabled]
pci 0000:00:03.0:   bridge window [mem disabled]
pci 0000:00:03.0:   bridge window [mem pref disabled]
pci 0000:03:00.0: BAR 6: assigned [mem 0xd0320000-0xd033ffff pref]
pci 0000:00:07.0: PCI bridge to [bus 03-03]
pci 0000:00:07.0:   bridge window [io  0x3000-0x3fff]
pci 0000:00:07.0:   bridge window [mem 0xd0300000-0xd03fffff]
pci 0000:00:07.0:   bridge window [mem 0xe0000000-0xefffffff 64bit pref]
pci 0000:00:1c.0: PCI bridge to [bus 04-04]
pci 0000:00:1c.0:   bridge window [io  0x2000-0x2fff]
pci 0000:00:1c.0:   bridge window [mem 0xd0200000-0xd02fffff]
pci 0000:00:1c.0:   bridge window [mem pref disabled]
pci 0000:00:1c.1: PCI bridge to [bus 05-05]
pci 0000:00:1c.1:   bridge window [io  disabled]
pci 0000:00:1c.1:   bridge window [mem disabled]
pci 0000:00:1c.1:   bridge window [mem pref disabled]
pci 0000:00:1c.4: PCI bridge to [bus 06-06]
pci 0000:00:1c.4:   bridge window [io  0x1000-0x1fff]
pci 0000:00:1c.4:   bridge window [mem 0xd0100000-0xd01fffff]
pci 0000:00:1c.4:   bridge window [mem pref disabled]
pci 0000:00:1e.0: PCI bridge to [bus 07-07]
pci 0000:00:1e.0:   bridge window [io  disabled]
pci 0000:00:1e.0:   bridge window [mem 0xd0000000-0xd00fffff]
pci 0000:00:1e.0:   bridge window [mem pref disabled]
alloc irq_desc for 16 on node -1
alloc kstat_irqs on node -1
pci 0000:00:01.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
pci 0000:00:01.0: setting latency timer to 64
pci 0000:00:03.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
pci 0000:00:03.0: setting latency timer to 64
pci 0000:00:07.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
pci 0000:00:07.0: setting latency timer to 64
alloc irq_desc for 17 on node -1
alloc kstat_irqs on node -1
pci 0000:00:1c.0: PCI INT A -> GSI 17 (level, low) -> IRQ 17
pci 0000:00:1c.0: setting latency timer to 64
pci 0000:00:1c.1: PCI INT B -> GSI 16 (level, low) -> IRQ 16
pci 0000:00:1c.1: setting latency timer to 64
pci 0000:00:1c.4: PCI INT A -> GSI 17 (level, low) -> IRQ 17
pci 0000:00:1c.4: setting latency timer to 64
pci 0000:00:1e.0: setting latency timer to 64
pci_bus 0000:00: resource 4 [io  0x0000-0x0cf7]
pci_bus 0000:00: resource 5 [io  0x0d00-0xffff]
pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff]
pci_bus 0000:00: resource 7 [mem 0x000c4000-0x000cbfff]
pci_bus 0000:00: resource 8 [mem 0xfed40000-0xfedfffff]
pci_bus 0000:00: resource 9 [mem 0xd0000000-0xf7ffffff]
pci_bus 0000:01: resource 0 [io  0x4000-0x4fff]
pci_bus 0000:01: resource 1 [mem 0xd0400000-0xd04fffff]
pci_bus 0000:03: resource 0 [io  0x3000-0x3fff]
pci_bus 0000:03: resource 1 [mem 0xd0300000-0xd03fffff]
pci_bus 0000:03: resource 2 [mem 0xe0000000-0xefffffff 64bit pref]
pci_bus 0000:04: resource 0 [io  0x2000-0x2fff]
pci_bus 0000:04: resource 1 [mem 0xd0200000-0xd02fffff]
pci_bus 0000:06: resource 0 [io  0x1000-0x1fff]
pci_bus 0000:06: resource 1 [mem 0xd0100000-0xd01fffff]
pci_bus 0000:07: resource 1 [mem 0xd0000000-0xd00fffff]
pci_bus 0000:07: resource 4 [io  0x0000-0x0cf7]
pci_bus 0000:07: resource 5 [io  0x0d00-0xffff]
pci_bus 0000:07: resource 6 [mem 0x000a0000-0x000bffff]
pci_bus 0000:07: resource 7 [mem 0x000c4000-0x000cbfff]
pci_bus 0000:07: resource 8 [mem 0xfed40000-0xfedfffff]
pci_bus 0000:07: resource 9 [mem 0xd0000000-0xf7ffffff]
NET: Registered protocol family 2
IP route cache hash table entries: 524288 (order: 10, 4194304 bytes)
TCP established hash table entries: 262144 (order: 10, 4194304 bytes)
TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
TCP: Hash tables configured (established 262144 bind 65536)
TCP reno registered
UDP hash table entries: 8192 (order: 6, 262144 bytes)
UDP-Lite hash table entries: 8192 (order: 6, 262144 bytes)
NET: Registered protocol family 1
RPC: Registered udp transport module.
RPC: Registered tcp transport module.
RPC: Registered tcp NFSv4.1 backchannel transport module.
pci 0000:00:1a.0: uhci_check_and_reset_hc: legsup = 0x0f10
pci 0000:00:1a.0: Performing full reset
pci 0000:00:1a.1: uhci_check_and_reset_hc: legsup = 0x0010
pci 0000:00:1a.1: Performing full reset
pci 0000:00:1a.2: uhci_check_and_reset_hc: legsup = 0x0010
pci 0000:00:1a.2: Performing full reset
pci 0000:00:1d.0: uhci_check_and_reset_hc: legsup = 0x0f10
pci 0000:00:1d.0: Performing full reset
pci 0000:00:1d.1: uhci_check_and_reset_hc: legsup = 0x0010
pci 0000:00:1d.1: Performing full reset
pci 0000:00:1d.2: uhci_check_and_reset_hc: legsup = 0x0010
pci 0000:00:1d.2: Performing full reset
pci 0000:03:00.0: Boot video device
PCI: CLS 64 bytes, default 64
PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
Placing 64MB software IO TLB between ffff880011a00000 - ffff880015a00000
software IO TLB at phys 0x11a00000 - 0x15a00000
microcode: CPU0 sig=0x106a5, pf=0x2, revision=0x11
microcode: CPU1 sig=0x106a5, pf=0x2, revision=0x11
microcode: CPU2 sig=0x106a5, pf=0x2, revision=0x11
microcode: CPU3 sig=0x106a5, pf=0x2, revision=0x11
microcode: CPU4 sig=0x106a5, pf=0x2, revision=0x11
microcode: CPU5 sig=0x106a5, pf=0x2, revision=0x11
microcode: CPU6 sig=0x106a5, pf=0x2, revision=0x11
microcode: CPU7 sig=0x106a5, pf=0x2, revision=0x11
microcode: Microcode Update Driver: v2.00 <tigran@aivazian.fsnet.co.uk>, Peter Oruba
Scanning for low memory corruption every 60 seconds
audit: initializing netlink socket (disabled)
type=2000 audit(1278110631.614:1): initialized
HugeTLB registered 2 MB page size, pre-allocated 0 pages
SGI XFS with security attributes, large block/inode numbers, no debug enabled
msgmni has been set to 32099
SELinux:  Registering netfilter hooks
alg: No test for stdrng (krng)
Block layer SCSI generic (bsg) driver version 0.4 loaded (major 253)
io scheduler noop registered
io scheduler deadline registered
io scheduler cfq registered (default)
pcieport 0000:00:01.0: setting latency timer to 64
alloc irq_desc for 24 on node -1
alloc kstat_irqs on node -1
pcieport 0000:00:01.0: irq 24 for MSI/MSI-X
pcieport 0000:00:03.0: setting latency timer to 64
alloc irq_desc for 25 on node -1
alloc kstat_irqs on node -1
pcieport 0000:00:03.0: irq 25 for MSI/MSI-X
pcieport 0000:00:07.0: setting latency timer to 64
alloc irq_desc for 26 on node -1
alloc kstat_irqs on node -1
pcieport 0000:00:07.0: irq 26 for MSI/MSI-X
pcieport 0000:00:1c.0: setting latency timer to 64
alloc irq_desc for 27 on node -1
alloc kstat_irqs on node -1
pcieport 0000:00:1c.0: irq 27 for MSI/MSI-X
pcieport 0000:00:1c.1: setting latency timer to 64
alloc irq_desc for 28 on node -1
alloc kstat_irqs on node -1
pcieport 0000:00:1c.1: irq 28 for MSI/MSI-X
pcieport 0000:00:1c.4: setting latency timer to 64
alloc irq_desc for 29 on node -1
alloc kstat_irqs on node -1
pcieport 0000:00:1c.4: irq 29 for MSI/MSI-X
aer 0000:00:01.0:pcie02: service driver aer loaded
aer 0000:00:03.0:pcie02: service driver aer loaded
aer 0000:00:07.0:pcie02: service driver aer loaded
pci_hotplug: PCI Hot Plug PCI Core version: 0.5
input: Sleep Button as /devices/LNXSYSTM:00/LNXSYBUS:00/PNP0C0E:00/input/input0
ACPI: Sleep Button [SLPB]
input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input1
ACPI: Power Button [PWRF]
Monitor-Mwait will be used to enter C-1 state
Non-volatile memory driver v1.3
Linux agpgart interface v0.103
Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
serial8250: ttyS1 at I/O 0x2f8 (irq = 3) is a NS16550A
brd: module loaded
loop: module loaded
ahci 0000:00:1f.2: version 3.0
alloc irq_desc for 19 on node -1
alloc kstat_irqs on node -1
ahci 0000:00:1f.2: PCI INT A -> GSI 19 (level, low) -> IRQ 19
alloc irq_desc for 30 on node -1
alloc kstat_irqs on node -1
ahci 0000:00:1f.2: irq 30 for MSI/MSI-X
ahci 0000:00:1f.2: AHCI 0001.0200 32 slots 6 ports 3 Gbps 0x3f impl SATA mode
ahci 0000:00:1f.2: flags: 64bit ncq sntf pm led clo pio slum part ccc ems 
ahci 0000:00:1f.2: setting latency timer to 64
scsi0 : ahci
scsi1 : ahci
scsi2 : ahci
scsi3 : ahci
scsi4 : ahci
scsi5 : ahci
ata1: SATA max UDMA/133 abar m2048@0xd0520000 port 0xd0520100 irq 30
ata2: SATA max UDMA/133 abar m2048@0xd0520000 port 0xd0520180 irq 30
ata3: SATA max UDMA/133 abar m2048@0xd0520000 port 0xd0520200 irq 30
ata4: SATA max UDMA/133 abar m2048@0xd0520000 port 0xd0520280 irq 30
ata5: SATA max UDMA/133 abar m2048@0xd0520000 port 0xd0520300 irq 30
ata6: SATA max UDMA/133 abar m2048@0xd0520000 port 0xd0520380 irq 30
ahci 0000:06:00.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
ahci 0000:06:00.0: controller can't do NCQ, turning off CAP_NCQ
ahci 0000:06:00.0: controller can't do PMP, turning off CAP_PMP
ahci 0000:06:00.0: MV_AHCI HACK: port_map 7 -> 3
ahci 0000:06:00.0: Disabling your PATA port. Use the boot option 'ahci.marvell_enable=0' to avoid this.
ahci: SSS flag set, parallel bus scan disabled
ahci 0000:06:00.0: AHCI 0001.0000 32 slots 3 ports 3 Gbps 0x3 impl IDE mode
ahci 0000:06:00.0: flags: 64bit stag led slum part 
ahci 0000:06:00.0: setting latency timer to 64
scsi6 : ahci
scsi7 : ahci
scsi8 : ahci
ata7: SATA max UDMA/133 abar m1024@0xd0100000 port 0xd0100100 irq 16
ata8: SATA max UDMA/133 abar m1024@0xd0100000 port 0xd0100180 irq 16
ata9: DUMMY
Intel(R) PRO/1000 Network Driver - version 7.3.21-k5-NAPI
Copyright (c) 1999-2006 Intel Corporation.
e1000e: Intel(R) PRO/1000 Network Driver - 1.0.2-k2
e1000e: Copyright (c) 1999 - 2009 Intel Corporation.
alloc irq_desc for 20 on node -1
alloc kstat_irqs on node -1
e1000e 0000:00:19.0: PCI INT A -> GSI 20 (level, low) -> IRQ 20
e1000e 0000:00:19.0: setting latency timer to 64
alloc irq_desc for 31 on node -1
alloc kstat_irqs on node -1
e1000e 0000:00:19.0: irq 31 for MSI/MSI-X
0000:00:19.0: eth0: (PCI Express:2.5GB/s:Width x1) 00:1c:c0:b2:1d:a2
0000:00:19.0: eth0: Intel(R) PRO/1000 Network Connection
0000:00:19.0: eth0: MAC: 7, PHY: 8, PBA No: ffffff-0ff
e1000e 0000:01:00.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
e1000e 0000:01:00.0: setting latency timer to 64
alloc irq_desc for 32 on node -1
alloc kstat_irqs on node -1
e1000e 0000:01:00.0: irq 32 for MSI/MSI-X
alloc irq_desc for 33 on node -1
alloc kstat_irqs on node -1
e1000e 0000:01:00.0: irq 33 for MSI/MSI-X
alloc irq_desc for 34 on node -1
alloc kstat_irqs on node -1
e1000e 0000:01:00.0: irq 34 for MSI/MSI-X
e1000e 0000:01:00.0: Disabling ASPM L0s 
0000:01:00.0: eth1: (PCI Express:2.5GB/s:Width x1) 00:22:64:89:75:e6
0000:01:00.0: eth1: Intel(R) PRO/1000 Network Connection
0000:01:00.0: eth1: MAC: 3, PHY: 8, PBA No: e56869-003
e1000e 0000:04:00.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
e1000e 0000:04:00.0: setting latency timer to 64
alloc irq_desc for 35 on node -1
alloc kstat_irqs on node -1
e1000e 0000:04:00.0: irq 35 for MSI/MSI-X
alloc irq_desc for 36 on node -1
alloc kstat_irqs on node -1
e1000e 0000:04:00.0: irq 36 for MSI/MSI-X
alloc irq_desc for 37 on node -1
alloc kstat_irqs on node -1
e1000e 0000:04:00.0: irq 37 for MSI/MSI-X
e1000e 0000:04:00.0: Disabling ASPM L0s 
0000:04:00.0: eth2: (PCI Express:2.5GB/s:Width x1) 00:22:64:89:75:dc
0000:04:00.0: eth2: Intel(R) PRO/1000 Network Connection
0000:04:00.0: eth2: MAC: 3, PHY: 8, PBA No: e56869-003
sky2: driver version 1.27
console [netcon0] enabled
netconsole: network logging started
ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
ehci_hcd: block sizes: qh 104 qtd 96 itd 192 sitd 96
alloc irq_desc for 18 on node -1
alloc kstat_irqs on node -1
ehci_hcd 0000:00:1a.7: PCI INT C -> GSI 18 (level, low) -> IRQ 18
ehci_hcd 0000:00:1a.7: setting latency timer to 64
ehci_hcd 0000:00:1a.7: EHCI Host Controller
drivers/usb/core/inode.c: creating file 'devices'
drivers/usb/core/inode.c: creating file '001'
ehci_hcd 0000:00:1a.7: new USB bus registered, assigned bus number 1
ehci_hcd 0000:00:1a.7: reset hcs_params 0x103206 dbg=1 cc=3 pcc=2 ordered !ppc ports=6
ehci_hcd 0000:00:1a.7: reset hcc_params 16871 thresh 7 uframes 1024 64 bit addr
ehci_hcd 0000:00:1a.7: debug port 1
ehci_hcd 0000:00:1a.7: reset command 080002 (park)=0 ithresh=8 period=1024 Reset HALT
ehci_hcd 0000:00:1a.7: cache line size of 64 is not supported
ehci_hcd 0000:00:1a.7: supports USB remote wakeup
ehci_hcd 0000:00:1a.7: irq 18, io mem 0xd0522000
ehci_hcd 0000:00:1a.7: reset command 080002 (park)=0 ithresh=8 period=1024 Reset HALT
ehci_hcd 0000:00:1a.7: init command 010001 (park)=0 ithresh=1 period=1024 RUN
ata2: SATA link down (SStatus 0 SControl 300)
ata4: SATA link down (SStatus 0 SControl 300)
ata1: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
ata6: SATA link down (SStatus 0 SControl 300)
ata5: SATA link down (SStatus 0 SControl 300)
ata3: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
ata1.00: ATA-8: ST3500418AS, CC38, max UDMA/133
ata1.00: 976773168 sectors, multi 0: LBA48 NCQ (depth 31/32)
ata3.00: ATA-8: ST31000528AS, CC38, max UDMA/133
ata3.00: 1953525168 sectors, multi 0: LBA48 NCQ (depth 31/32)
ata7: SATA link down (SStatus 0 SControl 300)
ata1.00: configured for UDMA/133
ata3.00: configured for UDMA/133
ehci_hcd 0000:00:1a.7: USB 2.0 started, EHCI 1.00
usb usb1: default language 0x0409
usb usb1: udev 1, busnum 1, minor = 0
usb usb1: New USB device found, idVendor=1d6b, idProduct=0002
usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
usb usb1: Product: EHCI Host Controller
usb usb1: Manufacturer: Linux 2.6.34 ehci_hcd
usb usb1: SerialNumber: 0000:00:1a.7
usb usb1: usb_probe_device
usb usb1: configuration #1 chosen from 1 choice
usb usb1: adding 1-0:1.0 (config #1, interface 0)
hub 1-0:1.0: usb_probe_interface
hub 1-0:1.0: usb_probe_interface - got id
hub 1-0:1.0: USB hub found
hub 1-0:1.0: 6 ports detected
hub 1-0:1.0: standalone hub
hub 1-0:1.0: no power switching (usb 1.0)
hub 1-0:1.0: individual port over-current protection
hub 1-0:1.0: power on to power good time: 20ms
hub 1-0:1.0: local power source is good
hub 1-0:1.0: trying to enable port power on non-switchable hub
drivers/usb/core/inode.c: creating file '001'
alloc irq_desc for 23 on node -1
alloc kstat_irqs on node -1
ehci_hcd 0000:00:1d.7: PCI INT A -> GSI 23 (level, low) -> IRQ 23
ehci_hcd 0000:00:1d.7: setting latency timer to 64
ehci_hcd 0000:00:1d.7: EHCI Host Controller
drivers/usb/core/inode.c: creating file '002'
ehci_hcd 0000:00:1d.7: new USB bus registered, assigned bus number 2
ehci_hcd 0000:00:1d.7: reset hcs_params 0x103206 dbg=1 cc=3 pcc=2 ordered !ppc ports=6
ehci_hcd 0000:00:1d.7: reset hcc_params 16871 thresh 7 uframes 1024 64 bit addr
ehci_hcd 0000:00:1d.7: debug port 1
ehci_hcd 0000:00:1d.7: reset command 080002 (park)=0 ithresh=8 period=1024 Reset HALT
ehci_hcd 0000:00:1d.7: cache line size of 64 is not supported
ehci_hcd 0000:00:1d.7: supports USB remote wakeup
ehci_hcd 0000:00:1d.7: irq 23, io mem 0xd0521000
ehci_hcd 0000:00:1d.7: reset command 080002 (park)=0 ithresh=8 period=1024 Reset HALT
ehci_hcd 0000:00:1d.7: init command 010001 (park)=0 ithresh=1 period=1024 RUN
scsi 0:0:0:0: Direct-Access     ATA      ST3500418AS      CC38 PQ: 0 ANSI: 5
sd 0:0:0:0: [sda] 976773168 512-byte logical blocks: (500 GB/465 GiB)
sd 0:0:0:0: Attached scsi generic sg0 type 0
scsi 2:0:0:0: Direct-Access     ATA      ST31000528AS     CC38 PQ: 0 ANSI: 5
sd 0:0:0:0: [sda] Write Protect is off
sd 2:0:0:0: [sdb] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB)
sd 2:0:0:0: [sdb] Write Protect is off
sd 2:0:0:0: [sdb] Mode Sense: 00 3a 00 00
sd 2:0:0:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
sd 2:0:0:0: Attached scsi generic sg1 type 0
sdb:
sd 0:0:0:0: [sda] Mode Sense: 00 3a 00 00
sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
sda:
ehci_hcd 0000:00:1d.7: USB 2.0 started, EHCI 1.00
usb usb2: default language 0x0409
usb usb2: udev 1, busnum 2, minor = 128
usb usb2: New USB device found, idVendor=1d6b, idProduct=0002
usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
usb usb2: Product: EHCI Host Controller
usb usb2: Manufacturer: Linux 2.6.34 ehci_hcd
usb usb2: SerialNumber: 0000:00:1d.7
usb usb2: usb_probe_device
usb usb2: configuration #1 chosen from 1 choice
usb usb2: adding 2-0:1.0 (config #1, interface 0)
hub 2-0:1.0: usb_probe_interface
hub 2-0:1.0: usb_probe_interface - got id
hub 2-0:1.0: USB hub found
hub 2-0:1.0: 6 ports detected
hub 2-0:1.0: standalone hub
hub 2-0:1.0: no power switching (usb 1.0)
hub 2-0:1.0: individual port over-current protection
hub 2-0:1.0: power on to power good time: 20ms
hub 2-0:1.0: local power source is good
hub 2-0:1.0: trying to enable port power on non-switchable hub
drivers/usb/core/inode.c: creating file '001'
ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
ohci_hcd: block sizes: ed 80 td 96
uhci_hcd: USB Universal Host Controller Interface driver
uhci_hcd 0000:00:1a.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
uhci_hcd 0000:00:1a.0: setting latency timer to 64
uhci_hcd 0000:00:1a.0: UHCI Host Controller
drivers/usb/core/inode.c: creating file '003'
uhci_hcd 0000:00:1a.0: new USB bus registered, assigned bus number 3
uhci_hcd 0000:00:1a.0: detected 2 ports
uhci_hcd 0000:00:1a.0: uhci_check_and_reset_hc: cmd = 0x0000
uhci_hcd 0000:00:1a.0: Performing full reset
uhci_hcd 0000:00:1a.0: supports USB remote wakeup
uhci_hcd 0000:00:1a.0: irq 16, io base 0x000050e0
usb usb3: default language 0x0409
usb usb3: udev 1, busnum 3, minor = 256
usb usb3: New USB device found, idVendor=1d6b, idProduct=0001
usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1
usb usb3: Product: UHCI Host Controller
usb usb3: Manufacturer: Linux 2.6.34 uhci_hcd
usb usb3: SerialNumber: 0000:00:1a.0
unknown partition table
usb usb3: usb_probe_device
usb usb3: configuration #1 chosen from 1 choice
usb usb3: adding 3-0:1.0 (config #1, interface 0)
hub 3-0:1.0: usb_probe_interface
hub 3-0:1.0: usb_probe_interface - got id
hub 3-0:1.0: USB hub found
sd 2:0:0:0: [sdb] Attached SCSI disk
hub 3-0:1.0: 2 ports detected
hub 3-0:1.0: standalone hub
hub 3-0:1.0: no power switching (usb 1.0)
hub 3-0:1.0: individual port over-current protection
hub 3-0:1.0: power on to power good time: 2ms
hub 3-0:1.0: local power source is good
hub 3-0:1.0: trying to enable port power on non-switchable hub
drivers/usb/core/inode.c: creating file '001'
ehci_hcd 0000:00:1a.7: HS companion for 0000:00:1a.0
alloc irq_desc for 21 on node -1
alloc kstat_irqs on node -1
uhci_hcd 0000:00:1a.1: PCI INT B -> GSI 21 (level, low) -> IRQ 21
uhci_hcd 0000:00:1a.1: setting latency timer to 64
uhci_hcd 0000:00:1a.1: UHCI Host Controller
drivers/usb/core/inode.c: creating file '004'
uhci_hcd 0000:00:1a.1: new USB bus registered, assigned bus number 4
uhci_hcd 0000:00:1a.1: detected 2 ports
uhci_hcd 0000:00:1a.1: uhci_check_and_reset_hc: cmd = 0x0000
uhci_hcd 0000:00:1a.1: Performing full reset
uhci_hcd 0000:00:1a.1: supports USB remote wakeup
uhci_hcd 0000:00:1a.1: irq 21, io base 0x000050c0
usb usb4: default language 0x0409
usb usb4: udev 1, busnum 4, minor = 384
usb usb4: New USB device found, idVendor=1d6b, idProduct=0001
usb usb4: New USB device strings: Mfr=3, Product=2, SerialNumber=1
usb usb4: Product: UHCI Host Controller
usb usb4: Manufacturer: Linux 2.6.34 uhci_hcd
usb usb4: SerialNumber: 0000:00:1a.1
sda1 sda2
usb usb4: usb_probe_device
usb usb4: configuration #1 chosen from 1 choice
usb usb4: adding 4-0:1.0 (config #1, interface 0)
hub 4-0:1.0: usb_probe_interface
hub 4-0:1.0: usb_probe_interface - got id
hub 4-0:1.0: USB hub found
sd 0:0:0:0: [sda] Attached SCSI disk
hub 4-0:1.0: 2 ports detected
hub 4-0:1.0: standalone hub
hub 4-0:1.0: no power switching (usb 1.0)
hub 4-0:1.0: individual port over-current protection
hub 4-0:1.0: power on to power good time: 2ms
hub 4-0:1.0: local power source is good
hub 4-0:1.0: trying to enable port power on non-switchable hub
drivers/usb/core/inode.c: creating file '001'
ehci_hcd 0000:00:1a.7: HS companion for 0000:00:1a.1
uhci_hcd 0000:00:1a.2: PCI INT D -> GSI 19 (level, low) -> IRQ 19
uhci_hcd 0000:00:1a.2: setting latency timer to 64
uhci_hcd 0000:00:1a.2: UHCI Host Controller
drivers/usb/core/inode.c: creating file '005'
uhci_hcd 0000:00:1a.2: new USB bus registered, assigned bus number 5
uhci_hcd 0000:00:1a.2: detected 2 ports
uhci_hcd 0000:00:1a.2: uhci_check_and_reset_hc: cmd = 0x0000
uhci_hcd 0000:00:1a.2: Performing full reset
uhci_hcd 0000:00:1a.2: supports USB remote wakeup
uhci_hcd 0000:00:1a.2: irq 19, io base 0x000050a0
usb usb5: default language 0x0409
usb usb5: udev 1, busnum 5, minor = 512
usb usb5: New USB device found, idVendor=1d6b, idProduct=0001
usb usb5: New USB device strings: Mfr=3, Product=2, SerialNumber=1
usb usb5: Product: UHCI Host Controller
usb usb5: Manufacturer: Linux 2.6.34 uhci_hcd
usb usb5: SerialNumber: 0000:00:1a.2
usb usb5: usb_probe_device
usb usb5: configuration #1 chosen from 1 choice
usb usb5: adding 5-0:1.0 (config #1, interface 0)
hub 5-0:1.0: usb_probe_interface
hub 5-0:1.0: usb_probe_interface - got id
hub 5-0:1.0: USB hub found
hub 5-0:1.0: 2 ports detected
hub 5-0:1.0: standalone hub
hub 5-0:1.0: no power switching (usb 1.0)
hub 5-0:1.0: individual port over-current protection
hub 5-0:1.0: power on to power good time: 2ms
hub 5-0:1.0: local power source is good
hub 5-0:1.0: trying to enable port power on non-switchable hub
drivers/usb/core/inode.c: creating file '001'
ehci_hcd 0000:00:1a.7: HS companion for 0000:00:1a.2
uhci_hcd 0000:00:1d.0: PCI INT A -> GSI 23 (level, low) -> IRQ 23
uhci_hcd 0000:00:1d.0: setting latency timer to 64
uhci_hcd 0000:00:1d.0: UHCI Host Controller
drivers/usb/core/inode.c: creating file '006'
uhci_hcd 0000:00:1d.0: new USB bus registered, assigned bus number 6
uhci_hcd 0000:00:1d.0: detected 2 ports
uhci_hcd 0000:00:1d.0: uhci_check_and_reset_hc: cmd = 0x0000
uhci_hcd 0000:00:1d.0: Performing full reset
uhci_hcd 0000:00:1d.0: supports USB remote wakeup
uhci_hcd 0000:00:1d.0: irq 23, io base 0x00005080
usb usb6: default language 0x0409
usb usb6: udev 1, busnum 6, minor = 640
usb usb6: New USB device found, idVendor=1d6b, idProduct=0001
usb usb6: New USB device strings: Mfr=3, Product=2, SerialNumber=1
usb usb6: Product: UHCI Host Controller
usb usb6: Manufacturer: Linux 2.6.34 uhci_hcd
usb usb6: SerialNumber: 0000:00:1d.0
usb usb6: usb_probe_device
usb usb6: configuration #1 chosen from 1 choice
usb usb6: adding 6-0:1.0 (config #1, interface 0)
hub 6-0:1.0: usb_probe_interface
hub 6-0:1.0: usb_probe_interface - got id
hub 6-0:1.0: USB hub found
hub 6-0:1.0: 2 ports detected
hub 6-0:1.0: standalone hub
hub 6-0:1.0: no power switching (usb 1.0)
hub 6-0:1.0: individual port over-current protection
hub 6-0:1.0: power on to power good time: 2ms
hub 6-0:1.0: local power source is good
hub 6-0:1.0: trying to enable port power on non-switchable hub
drivers/usb/core/inode.c: creating file '001'
ehci_hcd 0000:00:1d.7: HS companion for 0000:00:1d.0
uhci_hcd 0000:00:1d.1: PCI INT B -> GSI 19 (level, low) -> IRQ 19
uhci_hcd 0000:00:1d.1: setting latency timer to 64
uhci_hcd 0000:00:1d.1: UHCI Host Controller
drivers/usb/core/inode.c: creating file '007'
uhci_hcd 0000:00:1d.1: new USB bus registered, assigned bus number 7
uhci_hcd 0000:00:1d.1: detected 2 ports
uhci_hcd 0000:00:1d.1: uhci_check_and_reset_hc: cmd = 0x0000
uhci_hcd 0000:00:1d.1: Performing full reset
uhci_hcd 0000:00:1d.1: supports USB remote wakeup
uhci_hcd 0000:00:1d.1: irq 19, io base 0x00005060
usb usb7: default language 0x0409
usb usb7: udev 1, busnum 7, minor = 768
usb usb7: New USB device found, idVendor=1d6b, idProduct=0001
usb usb7: New USB device strings: Mfr=3, Product=2, SerialNumber=1
usb usb7: Product: UHCI Host Controller
usb usb7: Manufacturer: Linux 2.6.34 uhci_hcd
usb usb7: SerialNumber: 0000:00:1d.1
usb usb7: usb_probe_device
usb usb7: configuration #1 chosen from 1 choice
usb usb7: adding 7-0:1.0 (config #1, interface 0)
hub 7-0:1.0: usb_probe_interface
hub 7-0:1.0: usb_probe_interface - got id
hub 7-0:1.0: USB hub found
hub 7-0:1.0: 2 ports detected
hub 7-0:1.0: standalone hub
hub 7-0:1.0: no power switching (usb 1.0)
hub 7-0:1.0: individual port over-current protection
hub 7-0:1.0: power on to power good time: 2ms
hub 7-0:1.0: local power source is good
hub 7-0:1.0: trying to enable port power on non-switchable hub
drivers/usb/core/inode.c: creating file '001'
ehci_hcd 0000:00:1d.7: HS companion for 0000:00:1d.1
uhci_hcd 0000:00:1d.2: PCI INT C -> GSI 18 (level, low) -> IRQ 18
uhci_hcd 0000:00:1d.2: setting latency timer to 64
uhci_hcd 0000:00:1d.2: UHCI Host Controller
drivers/usb/core/inode.c: creating file '008'
uhci_hcd 0000:00:1d.2: new USB bus registered, assigned bus number 8
uhci_hcd 0000:00:1d.2: detected 2 ports
uhci_hcd 0000:00:1d.2: uhci_check_and_reset_hc: cmd = 0x0000
uhci_hcd 0000:00:1d.2: Performing full reset
uhci_hcd 0000:00:1d.2: supports USB remote wakeup
uhci_hcd 0000:00:1d.2: irq 18, io base 0x00005040
usb usb8: default language 0x0409
usb usb8: udev 1, busnum 8, minor = 896
usb usb8: New USB device found, idVendor=1d6b, idProduct=0001
usb usb8: New USB device strings: Mfr=3, Product=2, SerialNumber=1
usb usb8: Product: UHCI Host Controller
usb usb8: Manufacturer: Linux 2.6.34 uhci_hcd
usb usb8: SerialNumber: 0000:00:1d.2
usb usb8: usb_probe_device
usb usb8: configuration #1 chosen from 1 choice
usb usb8: adding 8-0:1.0 (config #1, interface 0)
hub 8-0:1.0: usb_probe_interface
hub 8-0:1.0: usb_probe_interface - got id
hub 8-0:1.0: USB hub found
hub 8-0:1.0: 2 ports detected
hub 8-0:1.0: standalone hub
hub 8-0:1.0: no power switching (usb 1.0)
hub 8-0:1.0: individual port over-current protection
hub 8-0:1.0: power on to power good time: 2ms
hub 8-0:1.0: local power source is good
hub 8-0:1.0: trying to enable port power on non-switchable hub
drivers/usb/core/inode.c: creating file '001'
ehci_hcd 0000:00:1d.7: HS companion for 0000:00:1d.2
usbcore: registered new interface driver usblp
Initializing USB Mass Storage driver...
usbcore: registered new interface driver usb-storage
USB Mass Storage support registered.
usbcore: registered new interface driver libusual
PNP: No PS/2 controller found. Probing ports directly.
serio: i8042 KBD port at 0x60,0x64 irq 1
serio: i8042 AUX port at 0x60,0x64 irq 12
mice: PS/2 mouse device common for all mice
rtc_cmos 00:02: RTC can wake from S4
rtc_cmos 00:02: rtc core: registered rtc_cmos as rtc0
rtc0: alarms up to one month, y3k, 114 bytes nvram, hpet irqs
i2c /dev entries driver
i801_smbus 0000:00:1f.3: PCI INT B -> GSI 18 (level, low) -> IRQ 18
device-mapper: ioctl: 4.17.0-ioctl (2010-03-05) initialised: dm-devel@redhat.com
cpuidle: using governor ladder
cpuidle: using governor menu
usbcore: registered new interface driver hiddev
usbcore: registered new interface driver usbhid
usbhid: USB HID core driver
Netfilter messages via NETLINK v0.30.
nf_conntrack version 0.5.0 (16384 buckets, 65536 max)
ctnetlink v0.93: registering with nfnetlink.
NF_TPROXY: Transparent proxy support initialized, version 4.1.0
NF_TPROXY: Copyright (c) 2006-2007 BalaBit IT Ltd.
xt_time: kernel timezone is -0000
IPv4 over IPv4 tunneling driver
GRE over IPv4 tunneling driver
ip_tables: (C) 2000-2006 Netfilter Core Team
NET: Registered protocol family 17
Bridge firewalling registered
Ebtables v2.0 registered
802.1Q VLAN Support v1.8 Ben Greear <greearb@candelatech.com>
All bugs added by David S. Miller <davem@redhat.com>
registered taskstats version 1
ehci_hcd 0000:00:1a.7: GetStatus port 2 status 001403 POWER sig=k CSC CONNECT
hub 1-0:1.0: port 2: status 0501 change 0001
hub 2-0:1.0: state 7 ports 6 chg 0000 evt 0000
uhci_hcd 0000:00:1a.0: port 2 portsc 008a,00
hub 4-0:1.0: state 7 ports 2 chg 0000 evt 0000
hub 5-0:1.0: state 7 ports 2 chg 0000 evt 0000
hub 6-0:1.0: state 7 ports 2 chg 0000 evt 0000
hub 7-0:1.0: state 7 ports 2 chg 0000 evt 0000
hub 8-0:1.0: state 7 ports 2 chg 0000 evt 0000
hub 1-0:1.0: state 7 ports 6 chg 0004 evt 0000
hub 1-0:1.0: port 2, status 0501, change 0000, 480 Mb/s
ehci_hcd 0000:00:1a.7: port 2 low speed --> companion
ehci_hcd 0000:00:1a.7: GetStatus port 2 status 003002 POWER OWNER sig=se0 CSC
hub 1-0:1.0: state 7 ports 6 chg 0000 evt 0004
hub 3-0:1.0: state 7 ports 2 chg 0000 evt 0004
uhci_hcd 0000:00:1a.0: port 2 portsc 01a3,00
hub 3-0:1.0: port 2, status 0301, change 0001, 1.5 Mb/s
ata8: SATA link down (SStatus 0 SControl 300)
hub 3-0:1.0: debounce: port 2: total 100ms stable 100ms status 0x301
md: Waiting for all devices to be available before autodetect
md: If you don't use raid, use raid=noautodetect
md: Autodetecting RAID arrays.
md: Scanned 0 and added 0 devices.
md: autorun ...
md: ... autorun DONE.
usb 3-2: new low speed USB device using uhci_hcd and address 2
XFS mounting filesystem sda1
Ending clean XFS mount for filesystem: sda1
VFS: Mounted root (xfs filesystem) readonly on device 8:1.
Freeing unused kernel memory: 444k freed
usb 3-2: skipped 1 descriptor after interface
usb 3-2: skipped 1 descriptor after interface
usb 3-2: udev 2, busnum 3, minor = 257
usb 3-2: New USB device found, idVendor=04d9, idProduct=1203
usb 3-2: New USB device strings: Mfr=0, Product=0, SerialNumber=0
usb 3-2: usb_probe_device
usb 3-2: configuration #1 chosen from 1 choice
usb 3-2: adding 3-2:1.0 (config #1, interface 0)
usbhid 3-2:1.0: usb_probe_interface
usbhid 3-2:1.0: usb_probe_interface - got id
input: HID 04d9:1203 as /devices/pci0000:00/0000:00:1a.0/usb3/3-2/3-2:1.0/input/input2
uhci_hcd 0000:00:1a.0: reserve dev 2 ep81-INT, period 8, phase 4, 118 us
generic-usb 0003:04D9:1203.0001: input,hidraw0: USB HID v1.11 Keyboard [HID 04d9:1203] on usb-0000:00:1a.0-2/input0
usb 3-2: adding 3-2:1.1 (config #1, interface 1)
usbhid 3-2:1.1: usb_probe_interface
usbhid 3-2:1.1: usb_probe_interface - got id
input: HID 04d9:1203 as /devices/pci0000:00/0000:00:1a.0/usb3/3-2/3-2:1.1/input/input3
uhci_hcd 0000:00:1a.0: reserve dev 2 ep82-INT, period 8, phase 4, 118 us
generic-usb 0003:04D9:1203.0002: input,hidraw1: USB HID v1.11 Device [HID 04d9:1203] on usb-0000:00:1a.0-2/input1
drivers/usb/core/inode.c: creating file '002'
hub 3-0:1.0: state 7 ports 2 chg 0000 evt 0004
usb usb4: suspend_rh (auto-stop)
usb usb5: suspend_rh (auto-stop)
usb usb6: suspend_rh (auto-stop)
usb usb7: suspend_rh (auto-stop)
usb usb8: suspend_rh (auto-stop)
udev: starting version 146
Adding 4988176k swap on /dev/sda2.  Priority:-1 extents:1 across:4988176k
00:00.0 Host bridge: Intel Corporation X58 I/O Hub to ESI Port (rev 13)
	Subsystem: Intel Corporation Device 4f53
	Flags: fast devsel, IRQ 11
	Capabilities: [40] #00 [0000]

00:01.0 PCI bridge: Intel Corporation 5520/5500/X58 I/O Hub PCI Express Root Port 1 (rev 13) (prog-if 00 [Normal decode])
	Flags: bus master, fast devsel, latency 0
	Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
	I/O behind bridge: 00003000-00003fff
	Memory behind bridge: d0200000-d02fffff
	Capabilities: [40] Subsystem: Intel Corporation Device 4f53
	Capabilities: [60] MSI: Enable+ Count=1/2 Maskable+ 64bit-
	Capabilities: [90] Express Root Port (Slot+), MSI 00
	Capabilities: [e0] Power Management version 3
	Capabilities: [100] Advanced Error Reporting
	Capabilities: [150] Access Control Services
	Capabilities: [160] Vendor Specific Information <?>
	Kernel driver in use: pcieport

00:03.0 PCI bridge: Intel Corporation 5520/5500/X58 I/O Hub PCI Express Root Port 3 (rev 13) (prog-if 00 [Normal decode])
	Flags: bus master, fast devsel, latency 0
	Bus: primary=00, secondary=02, subordinate=02, sec-latency=0
	Capabilities: [40] Subsystem: Intel Corporation Device 4f53
	Capabilities: [60] MSI: Enable+ Count=1/2 Maskable+ 64bit-
	Capabilities: [90] Express Root Port (Slot+), MSI 00
	Capabilities: [e0] Power Management version 3
	Capabilities: [100] Advanced Error Reporting
	Capabilities: [150] Access Control Services
	Capabilities: [160] Vendor Specific Information <?>
	Kernel driver in use: pcieport

00:07.0 PCI bridge: Intel Corporation 5520/5500/X58 I/O Hub PCI Express Root Port 7 (rev 13) (prog-if 00 [Normal decode])
	Flags: bus master, fast devsel, latency 0
	Bus: primary=00, secondary=03, subordinate=03, sec-latency=0
	I/O behind bridge: 00002000-00002fff
	Memory behind bridge: d0100000-d01fffff
	Prefetchable memory behind bridge: 00000000e0000000-00000000efffffff
	Capabilities: [40] Subsystem: Intel Corporation Device 4f53
	Capabilities: [60] MSI: Enable+ Count=1/2 Maskable+ 64bit-
	Capabilities: [90] Express Root Port (Slot+), MSI 00
	Capabilities: [e0] Power Management version 3
	Capabilities: [100] Advanced Error Reporting
	Capabilities: [150] Access Control Services
	Capabilities: [160] Vendor Specific Information <?>
	Kernel driver in use: pcieport

00:10.0 PIC: Intel Corporation 5520/5500/X58 Physical and Link Layer Registers Port 0 (rev 13) (prog-if 00 [8259])
	Flags: fast devsel
	Capabilities: [50] Vendor Specific Information <?>

00:10.1 PIC: Intel Corporation 5520/5500/X58 Routing and Protocol Layer Registers Port 0 (rev 13) (prog-if 00 [8259])
	Flags: fast devsel

00:14.0 PIC: Intel Corporation 5520/5500/X58 I/O Hub System Management Registers (rev 13) (prog-if 00 [8259])
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00

00:14.1 PIC: Intel Corporation 5520/5500/X58 I/O Hub GPIO and Scratch Pad Registers (rev 13) (prog-if 00 [8259])
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00

00:14.2 PIC: Intel Corporation 5520/5500/X58 I/O Hub Control Status and RAS Registers (rev 13) (prog-if 00 [8259])
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00

00:14.3 PIC: Intel Corporation 5520/5500/X58 I/O Hub Throttle Registers (rev 13) (prog-if 00 [8259])
	Flags: fast devsel

00:19.0 Ethernet controller: Intel Corporation 82567LM-2 Gigabit Network Connection
	Subsystem: Intel Corporation Device 0000
	Flags: bus master, fast devsel, latency 0, IRQ 30
	Memory at d0300000 (32-bit, non-prefetchable) [size=128K]
	Memory at d0323000 (32-bit, non-prefetchable) [size=4K]
	I/O ports at 4100 [size=32]
	Capabilities: [c8] Power Management version 2
	Capabilities: [d0] MSI: Enable+ Count=1/1 Maskable- 64bit+
	Kernel driver in use: e1000e

00:1a.0 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #4 (prog-if 00 [UHCI])
	Subsystem: Intel Corporation Device 4f53
	Flags: bus master, medium devsel, latency 0, IRQ 16
	I/O ports at 40e0 [size=32]
	Capabilities: [50] PCI Advanced Features
	Kernel driver in use: uhci_hcd

00:1a.1 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #5 (prog-if 00 [UHCI])
	Subsystem: Intel Corporation Device 4f53
	Flags: bus master, medium devsel, latency 0, IRQ 21
	I/O ports at 40c0 [size=32]
	Capabilities: [50] PCI Advanced Features
	Kernel driver in use: uhci_hcd

00:1a.2 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #6 (prog-if 00 [UHCI])
	Subsystem: Intel Corporation Device 4f53
	Flags: bus master, medium devsel, latency 0, IRQ 19
	I/O ports at 40a0 [size=32]
	Capabilities: [50] PCI Advanced Features
	Kernel driver in use: uhci_hcd

00:1a.7 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB2 EHCI Controller #2 (prog-if 20 [EHCI])
	Subsystem: Intel Corporation Device 4f53
	Flags: bus master, medium devsel, latency 0, IRQ 18
	Memory at d0322000 (32-bit, non-prefetchable) [size=1K]
	Capabilities: [50] Power Management version 2
	Capabilities: [58] Debug port: BAR=1 offset=00a0
	Kernel driver in use: ehci_hcd

00:1c.0 PCI bridge: Intel Corporation 82801JI (ICH10 Family) PCI Express Port 1 (prog-if 00 [Normal decode])
	Flags: bus master, fast devsel, latency 0
	Bus: primary=00, secondary=04, subordinate=04, sec-latency=0
	I/O behind bridge: 00001000-00001fff
	Memory behind bridge: d0000000-d00fffff
	Capabilities: [40] Express Root Port (Slot-), MSI 00
	Capabilities: [80] MSI: Enable+ Count=1/1 Maskable- 64bit-
	Capabilities: [90] Subsystem: Intel Corporation Device 4f53
	Capabilities: [a0] Power Management version 2
	Capabilities: [100] Virtual Channel <?>
	Capabilities: [180] Root Complex Link <?>
	Kernel driver in use: pcieport

00:1c.1 PCI bridge: Intel Corporation 82801JI (ICH10 Family) PCI Express Port 2 (prog-if 00 [Normal decode])
	Flags: bus master, fast devsel, latency 0
	Bus: primary=00, secondary=05, subordinate=05, sec-latency=0
	Capabilities: [40] Express Root Port (Slot-), MSI 00
	Capabilities: [80] MSI: Enable+ Count=1/1 Maskable- 64bit-
	Capabilities: [90] Subsystem: Intel Corporation Device 4f53
	Capabilities: [a0] Power Management version 2
	Capabilities: [100] Virtual Channel <?>
	Capabilities: [180] Root Complex Link <?>
	Kernel driver in use: pcieport

00:1d.0 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #1 (prog-if 00 [UHCI])
	Subsystem: Intel Corporation Device 4f53
	Flags: bus master, medium devsel, latency 0, IRQ 23
	I/O ports at 4080 [size=32]
	Capabilities: [50] PCI Advanced Features
	Kernel driver in use: uhci_hcd

00:1d.1 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #2 (prog-if 00 [UHCI])
	Subsystem: Intel Corporation Device 4f53
	Flags: bus master, medium devsel, latency 0, IRQ 19
	I/O ports at 4060 [size=32]
	Capabilities: [50] PCI Advanced Features
	Kernel driver in use: uhci_hcd

00:1d.2 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #3 (prog-if 00 [UHCI])
	Subsystem: Intel Corporation Device 4f53
	Flags: bus master, medium devsel, latency 0, IRQ 18
	I/O ports at 4040 [size=32]
	Capabilities: [50] PCI Advanced Features
	Kernel driver in use: uhci_hcd

00:1d.7 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB2 EHCI Controller #1 (prog-if 20 [EHCI])
	Subsystem: Intel Corporation Device 4f53
	Flags: bus master, medium devsel, latency 0, IRQ 23
	Memory at d0321000 (32-bit, non-prefetchable) [size=1K]
	Capabilities: [50] Power Management version 2
	Capabilities: [58] Debug port: BAR=1 offset=00a0
	Kernel driver in use: ehci_hcd

00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev 90) (prog-if 01 [Subtractive decode])
	Flags: bus master, fast devsel, latency 0
	Bus: primary=00, secondary=06, subordinate=06, sec-latency=32
	Capabilities: [50] Subsystem: Intel Corporation Device 4f53

00:1f.0 ISA bridge: Intel Corporation 82801JIR (ICH10R) LPC Interface Controller
	Subsystem: Intel Corporation Device 4f53
	Flags: bus master, medium devsel, latency 0
	Capabilities: [e0] Vendor Specific Information <?>

00:1f.2 SATA controller: Intel Corporation 82801JI (ICH10 Family) SATA AHCI Controller (prog-if 01 [AHCI 1.0])
	Subsystem: Intel Corporation Device 4f53
	Flags: bus master, 66MHz, medium devsel, latency 0, IRQ 29
	I/O ports at 4128 [size=8]
	I/O ports at 4134 [size=4]
	I/O ports at 4120 [size=8]
	I/O ports at 4130 [size=4]
	I/O ports at 4020 [size=32]
	Memory at d0320000 (32-bit, non-prefetchable) [size=2K]
	Capabilities: [80] MSI: Enable+ Count=1/16 Maskable- 64bit-
	Capabilities: [70] Power Management version 3
	Capabilities: [a8] SATA HBA <?>
	Kernel driver in use: ahci

00:1f.3 SMBus: Intel Corporation 82801JI (ICH10 Family) SMBus Controller
	Subsystem: Intel Corporation Device 4f53
	Flags: medium devsel, IRQ 18
	Memory at f0000000 (64-bit, non-prefetchable) [size=256]
	I/O ports at 4000 [size=32]
	Kernel driver in use: i801_smbus

01:00.0 Ethernet controller: Intel Corporation 82574L Gigabit Network Connection
	Subsystem: Hewlett-Packard Company NC112T PCI Express single Port Gigabit Server Adapter
	Flags: bus master, fast devsel, latency 0, IRQ 16
	Memory at d0280000 (32-bit, non-prefetchable) [size=128K]
	Memory at d0200000 (32-bit, non-prefetchable) [size=512K]
	I/O ports at 3000 [size=32]
	Memory at d02a0000 (32-bit, non-prefetchable) [size=16K]
	Capabilities: [c8] Power Management version 2
	Capabilities: [d0] MSI: Enable- Count=1/1 Maskable- 64bit+
	Capabilities: [e0] Express Endpoint, MSI 00
	Capabilities: [a0] MSI-X: Enable+ Count=5 Masked-
	Capabilities: [100] Advanced Error Reporting
	Capabilities: [140] Device Serial Number 00-22-64-ff-ff-89-75-e6
	Kernel driver in use: e1000e

03:00.0 VGA compatible controller: ATI Technologies Inc RV710 [Radeon HD 4350] (prog-if 00 [VGA controller])
	Subsystem: XFX Pine Group Inc. Device 2461
	Flags: bus master, fast devsel, latency 0, IRQ 11
	Memory at e0000000 (64-bit, prefetchable) [size=256M]
	Memory at d0100000 (64-bit, non-prefetchable) [size=64K]
	I/O ports at 2000 [size=256]
	Expansion ROM at d0120000 [disabled] [size=128K]
	Capabilities: [50] Power Management version 3
	Capabilities: [58] Express Legacy Endpoint, MSI 00
	Capabilities: [a0] MSI: Enable- Count=1/1 Maskable- 64bit+
	Capabilities: [100] Vendor Specific Information <?>

03:00.1 Audio device: ATI Technologies Inc R700 Audio Device [Radeon HD 4000 Series]
	Subsystem: XFX Pine Group Inc. Device aa38
	Flags: bus master, fast devsel, latency 0, IRQ 10
	Memory at d0110000 (64-bit, non-prefetchable) [size=16K]
	Capabilities: [50] Power Management version 3
	Capabilities: [58] Express Legacy Endpoint, MSI 00
	Capabilities: [a0] MSI: Enable- Count=1/1 Maskable- 64bit+
	Capabilities: [100] Vendor Specific Information <?>

04:00.0 Ethernet controller: Intel Corporation 82574L Gigabit Network Connection
	Subsystem: Hewlett-Packard Company NC112T PCI Express single Port Gigabit Server Adapter
	Flags: bus master, fast devsel, latency 0, IRQ 16
	Memory at d0080000 (32-bit, non-prefetchable) [size=128K]
	Memory at d0000000 (32-bit, non-prefetchable) [size=512K]
	I/O ports at 1000 [size=32]
	Memory at d00a0000 (32-bit, non-prefetchable) [size=16K]
	Capabilities: [c8] Power Management version 2
	Capabilities: [d0] MSI: Enable- Count=1/1 Maskable- 64bit+
	Capabilities: [e0] Express Endpoint, MSI 00
	Capabilities: [a0] MSI-X: Enable+ Count=5 Masked-
	Capabilities: [100] Advanced Error Reporting
	Capabilities: [140] Device Serial Number 00-22-64-ff-ff-89-75-dc
	Kernel driver in use: e1000e
Felipe W Damasio July 12, 2010, 6:49 p.m. UTC | #13
Hi Mr. Dumazet,

2010/7/11 Felipe W Damasio <felipewd@gmail.com>:
> We're using eth1 facing the user and eth2 facing the internet.

Here's the result using ethtool-2.6.34:

./ethtool -k eth1

Offload parameters for eth1:
rx-checksumming: on
tx-checksumming: on
scatter-gather: on
tcp-segmentation-offload: on
udp-fragmentation-offload: off
generic-segmentation-offload: on
generic-receive-offload: off
large-receive-offload: off
ntuple-filters: off
receive-hashing: off


./ethtool -k eth2

Offload parameters for eth2:
rx-checksumming: on
tx-checksumming: on
scatter-gather: on
tcp-segmentation-offload: on
udp-fragmentation-offload: off
generic-segmentation-offload: on
generic-receive-offload: off
large-receive-offload: off
ntuple-filters: off
receive-hashing: off
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Felipe W Damasio July 13, 2010, 2:24 p.m. UTC | #14
Hi Mr. Dumazet,

2010/7/12 Felipe W Damasio <felipewd@gmail.com>:
> Here's the result using ethtool-2.6.34:
>
> ./ethtool -k eth1
>
> Offload parameters for eth1:
> rx-checksumming: on
> tx-checksumming: on
> scatter-gather: on
> tcp-segmentation-offload: on
> udp-fragmentation-offload: off
> generic-segmentation-offload: on
> generic-receive-offload: off
> large-receive-offload: off
> ntuple-filters: off
> receive-hashing: off
>
>
> ./ethtool -k eth2
>
> Offload parameters for eth2:
> rx-checksumming: on
> tx-checksumming: on
> scatter-gather: on
> tcp-segmentation-offload: on
> udp-fragmentation-offload: off
> generic-segmentation-offload: on
> generic-receive-offload: off
> large-receive-offload: off
> ntuple-filters: off
> receive-hashing: off

Did these help you track down the issue?

Sorry to insist, it's just that my bosses are kind of pressuring me to
solve the problem and put the squid machine back online :-)

Is there a test I can run to try and trigger the issue?

I have the same scenario (hardware and network setup) on my lab...

Cheers,

Felipe Damasio
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Eric Dumazet July 13, 2010, 2:40 p.m. UTC | #15
Le mardi 13 juillet 2010 à 11:24 -0300, Felipe W Damasio a écrit :
> Hi Mr. Dumazet,
> 
> 2010/7/12 Felipe W Damasio <felipewd@gmail.com>:
> > Here's the result using ethtool-2.6.34:
> >
> > ./ethtool -k eth1
> >
> > Offload parameters for eth1:
> > rx-checksumming: on
> > tx-checksumming: on
> > scatter-gather: on
> > tcp-segmentation-offload: on
> > udp-fragmentation-offload: off
> > generic-segmentation-offload: on
> > generic-receive-offload: off
> > large-receive-offload: off
> > ntuple-filters: off
> > receive-hashing: off
> >
> >
> > ./ethtool -k eth2
> >
> > Offload parameters for eth2:
> > rx-checksumming: on
> > tx-checksumming: on
> > scatter-gather: on
> > tcp-segmentation-offload: on
> > udp-fragmentation-offload: off
> > generic-segmentation-offload: on
> > generic-receive-offload: off
> > large-receive-offload: off
> > ntuple-filters: off
> > receive-hashing: off
> 
> Did these help you track down the issue?
> 
> Sorry to insist, it's just that my bosses are kind of pressuring me to
> solve the problem and put the squid machine back online :-)
> 
> Is there a test I can run to try and trigger the issue?
> 
> I have the same scenario (hardware and network setup) on my lab...
> 

I currently have no fresh ideas. If you want this problem to be solved,
its important to setup in your lab a workload to trigger again and again
the bug, in order to provide us more crash information.

After code review doesnt spot obvious bugs, this is time for brute force
hunting, using git bisection for example...



--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Felipe W Damasio July 13, 2010, 2:49 p.m. UTC | #16
Hi Mr. Dumazet,

2010/7/13 Eric Dumazet <eric.dumazet@gmail.com>:
> I currently have no fresh ideas. If you want this problem to be solved,
> its important to setup in your lab a workload to trigger again and again
> the bug, in order to provide us more crash information.

 Right. I've been running non-stop since the first bug happened, but
so far the problem hasn't surfaced again :-(

 I've been using the kernel with the patch that you provided me
(nf_tproxy.c). Is there a chance that patch fixed the problem?

 Cheers,

Felipe Damasio
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Eric Dumazet July 13, 2010, 3:49 p.m. UTC | #17
Le mardi 13 juillet 2010 à 11:49 -0300, Felipe W Damasio a écrit :
> Hi Mr. Dumazet,
> 
> 2010/7/13 Eric Dumazet <eric.dumazet@gmail.com>:
> > I currently have no fresh ideas. If you want this problem to be solved,
> > its important to setup in your lab a workload to trigger again and again
> > the bug, in order to provide us more crash information.
> 
>  Right. I've been running non-stop since the first bug happened, but
> so far the problem hasn't surfaced again :-(
> 
>  I've been using the kernel with the patch that you provided me
> (nf_tproxy.c). Is there a chance that patch fixed the problem?

This is a real bug, but I dont think it can fix your problem.

Looking again at your crash, we see RCX=0x720, decimal 1824

As its skb->len, we are freeing an skb that was collapsed or something
like that, since 1824 > 1460 (the normal MSS on ethernet)

GRO is off on your machine.

But coincidently 0x0720 is also a blank char for VGA screen...
(0x20 : ASCII space, 0x07 : default attribute)

So maybe you hit a corruption outside of network stack.



--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Felipe W Damasio July 13, 2010, 8:55 p.m. UTC | #18
Hi Mr. Dumazet,

I used the patched kernel on the production machine and squid frooze again.

This is the dmesg message:


general protection fault: 0000 [#1] SMP
last sysfs file: /sys/devices/pci0000:00/0000:00:1f.3/i2c-0/name
CPU 1
Modules linked in:

Pid: 5533, comm: squid Not tainted 2.6.34 #6 DX58SO/
RIP: 0010:[<ffffffff81369b2a>]  [<ffffffff81369b2a>] sock_rfree+0x26/0x37
RSP: 0018:ffff88042287fc20  EFLAGS: 00010206
RAX: 66c86f938964c696 RBX: ffff88034e8f9a00 RCX: 0000000000000720
RDX: ffff8803f0ce05c0 RSI: ffff8803d441960c RDI: ffff88034e8f9a00
RBP: ffff8803f0ee05c0 R08: ffffea000dcb9998 R09: 0000000000000000
R10: 000000000003d830 R11: ffff8803f0ee05c0 R12: 00000000000005a8
R13: 00000000000005a8 R14: 0000000000004378 R15: 0000000000000000
FS:  00007f4cf33ee710(0000) GS:ffff880001840000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000021d5fd0 CR3: 0000000422872000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process squid (pid: 5533, threadinfo ffff88042287e000, task ffff88042eb61a40)
Stack:
 ffffffff8136ecda ffff88034e8f9a00 ffffffff8136ea8c ffff88034e8f9a00
<0> ffffffff813ab142 00000000000000d0 ffffffff8136f9f9 000000000eec60e2
<0> ffff88042eb61a40 ffff88042eb61a40 ffff88042eb61a40 00000000edca7300
Call Trace:
 [<ffffffff8136ecda>] ? skb_release_head_state+0x6d/0xb7
 [<ffffffff8136ea8c>] ? __kfree_skb+0x9/0x7d
 [<ffffffff813ab142>] ? tcp_recvmsg+0x6a3/0x89a
 [<ffffffff8136f9f9>] ? __alloc_skb+0x5e/0x14e
 [<ffffffff81369dde>] ? sock_common_recvmsg+0x30/0x45
 [<ffffffff81367b0f>] ? sock_aio_read+0xdd/0xf1
 [<ffffffff813b6c97>] ? tcp_write_xmit+0x93e/0x96c
 [<ffffffff810ac500>] ? do_sync_read+0xb0/0xf2
 [<ffffffff810acf32>] ? vfs_read+0xb9/0xff
 [<ffffffff810ad034>] ? sys_read+0x45/0x6e
 [<ffffffff8100292b>] ? system_call_fastpath+0x16/0x1b
Code: ff ff ff ff c3 48 8b 57 18 8b 87 d8 00 00 00 48 8d 8a ac 00 00
00 f0 29 82 ac 00 00 00 48 8b 57 18 8b 8f d8 00 00 00 48 8b 42 38 <48>
83 b8 b0 00 00 00 00 74 06 01 8a f4 00 00 00 c3 41 57 41 89
RIP  [<ffffffff81369b2a>] sock_rfree+0x26/0x37
 RSP <ffff88042287fc20>
---[ end trace 22e6ca9ef825c0e6 ]---


Seems to be the same issue, right?

Cheers,

Felipe Damasio





2010/7/13 Eric Dumazet <eric.dumazet@gmail.com>:
> Le mardi 13 juillet 2010 à 11:49 -0300, Felipe W Damasio a écrit :
>> Hi Mr. Dumazet,
>>
>> 2010/7/13 Eric Dumazet <eric.dumazet@gmail.com>:
>> > I currently have no fresh ideas. If you want this problem to be solved,
>> > its important to setup in your lab a workload to trigger again and again
>> > the bug, in order to provide us more crash information.
>>
>>  Right. I've been running non-stop since the first bug happened, but
>> so far the problem hasn't surfaced again :-(
>>
>>  I've been using the kernel with the patch that you provided me
>> (nf_tproxy.c). Is there a chance that patch fixed the problem?
>
> This is a real bug, but I dont think it can fix your problem.
>
> Looking again at your crash, we see RCX=0x720, decimal 1824
>
> As its skb->len, we are freeing an skb that was collapsed or something
> like that, since 1824 > 1460 (the normal MSS on ethernet)
>
> GRO is off on your machine.
>
> But coincidently 0x0720 is also a blank char for VGA screen...
> (0x20 : ASCII space, 0x07 : default attribute)
>
> So maybe you hit a corruption outside of network stack.
>
>
>
>
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Felipe W Damasio July 13, 2010, 9:06 p.m. UTC | #19
Hi,

2010/7/13 Felipe W Damasio <felipewd@gmail.com>:
> This is the dmesg message:

This error happened at 17:19:59.

A few seconds earlier, I have these on squid's logs:

2010/07/13 17:19:50| clientTryParseRequest: FD 1690
(189.113.65.55:52681) Invalid Request
2010/07/13 17:19:50| clientTryParseRequest: FD 5056
(189.113.74.101:2420) Invalid Request
2010/07/13 17:19:52| clientTryParseRequest: FD 26923
(189.113.74.101:2419) Invalid Request
2010/07/13 17:19:57| parseHttpRequest: Unsupported method 'PASS'
2010/07/13 17:19:57| clientTryParseRequest: FD 17786
(189.113.69.194:2422) Invalid Request
2010/07/13 17:19:57| parseHttpRequest: Unsupported method '<D2>Yk'
2010/07/13 17:19:57| clientTryParseRequest: FD 22554
(189.113.79.151:4225) Invalid Request
2010/07/13 17:19:59| httpReadReply: Excess data from "GET
http://webcs.msg.yahoo.com/crossdomain.xml"

 Maybe with these two errors we can create some kind of trigger
program to help us duplicate this on the lab setup?

 Cheers,

Felipe Damasio
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Eric Dumazet July 14, 2010, 3:21 a.m. UTC | #20
Le mardi 13 juillet 2010 à 17:55 -0300, Felipe W Damasio a écrit :
> Hi Mr. Dumazet,
> 
> I used the patched kernel on the production machine and squid frooze again.
> 
> This is the dmesg message:
> 
> 
> general protection fault: 0000 [#1] SMP
> last sysfs file: /sys/devices/pci0000:00/0000:00:1f.3/i2c-0/name
> CPU 1
> Modules linked in:
> 
> Pid: 5533, comm: squid Not tainted 2.6.34 #6 DX58SO/
> RIP: 0010:[<ffffffff81369b2a>]  [<ffffffff81369b2a>] sock_rfree+0x26/0x37
> RSP: 0018:ffff88042287fc20  EFLAGS: 00010206
> RAX: 66c86f938964c696 RBX: ffff88034e8f9a00 RCX: 0000000000000720
> RDX: ffff8803f0ce05c0 RSI: ffff8803d441960c RDI: ffff88034e8f9a00
> RBP: ffff8803f0ee05c0 R08: ffffea000dcb9998 R09: 0000000000000000
> R10: 000000000003d830 R11: ffff8803f0ee05c0 R12: 00000000000005a8
> R13: 00000000000005a8 R14: 0000000000004378 R15: 0000000000000000
> FS:  00007f4cf33ee710(0000) GS:ffff880001840000(0000) knlGS:0000000000000000
> CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> CR2: 00000000021d5fd0 CR3: 0000000422872000 CR4: 00000000000006e0
> DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> Process squid (pid: 5533, threadinfo ffff88042287e000, task ffff88042eb61a40)
> Stack:
>  ffffffff8136ecda ffff88034e8f9a00 ffffffff8136ea8c ffff88034e8f9a00
> <0> ffffffff813ab142 00000000000000d0 ffffffff8136f9f9 000000000eec60e2
> <0> ffff88042eb61a40 ffff88042eb61a40 ffff88042eb61a40 00000000edca7300
> Call Trace:
>  [<ffffffff8136ecda>] ? skb_release_head_state+0x6d/0xb7
>  [<ffffffff8136ea8c>] ? __kfree_skb+0x9/0x7d
>  [<ffffffff813ab142>] ? tcp_recvmsg+0x6a3/0x89a
>  [<ffffffff8136f9f9>] ? __alloc_skb+0x5e/0x14e
>  [<ffffffff81369dde>] ? sock_common_recvmsg+0x30/0x45
>  [<ffffffff81367b0f>] ? sock_aio_read+0xdd/0xf1
>  [<ffffffff813b6c97>] ? tcp_write_xmit+0x93e/0x96c
>  [<ffffffff810ac500>] ? do_sync_read+0xb0/0xf2
>  [<ffffffff810acf32>] ? vfs_read+0xb9/0xff
>  [<ffffffff810ad034>] ? sys_read+0x45/0x6e
>  [<ffffffff8100292b>] ? system_call_fastpath+0x16/0x1b
> Code: ff ff ff ff c3 48 8b 57 18 8b 87 d8 00 00 00 48 8d 8a ac 00 00
> 00 f0 29 82 ac 00 00 00 48 8b 57 18 8b 8f d8 00 00 00 48 8b 42 38 <48>
> 83 b8 b0 00 00 00 00 74 06 01 8a f4 00 00 00 c3 41 57 41 89
> RIP  [<ffffffff81369b2a>] sock_rfree+0x26/0x37
>  RSP <ffff88042287fc20>
> ---[ end trace 22e6ca9ef825c0e6 ]---
> 
> 
> Seems to be the same issue, right?
> 

Exactly the same. Only RAX value is different, its another chain.

BTW, 0x720 is not skb->len like I said earlier, but skb->truesize, and
0x720 is OK on a 64 bit machine for a regular packet.

48 8b 57 18             mov    0x18(%rdi),%rdx     skb->sk
8b 87 d8 00 00 00       mov    0xd8(%rdi),%eax     skb->truesize
48 8d 8a ac 00 00 00    lea 0xac(%rdx),%rcx
f0 29 82 ac 00 00 00    lock sub %eax,0xac(%rdx)
48 8b 57 18             mov    0x18(%rdi),%rdx     skb->sk
8b 8f d8 00 00 00       mov    0xd8(%rdi),%ecx     skb->truesize
48 8b 42 38                  mov    0x38(%rdx),%rax  sk->sk_prot
<48> 83 b8 b0 00 00 00 00    cmpq   $0x0,0xb0(%rax)
74 06 					     je     .+6
01 8a fa 00 00 00       add    %ecx,0xfa(%rdx)


One thing to notice are the RDX and RBP values:

RDX: ffff8803f0ce05c0 
RBP: ffff8803f0ee05c0

RDX being the sk pointer (and sk+0x38 contains the corrupted "sk_prot" value)
, we notice RBP contains same "sk" value + 0x200000  (2 Mbytes).

(same remark on your initial bug report)

Could you enable CONFIG_FRAME_POINTER=y in your config ?


--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Felipe W Damasio July 14, 2010, 3:27 a.m. UTC | #21
Hi Mr. Dumazet,

2010/7/14 Eric Dumazet <eric.dumazet@gmail.com>:
> RDX being the sk pointer (and sk+0x38 contains the corrupted "sk_prot" value)
> , we notice RBP contains same "sk" value + 0x200000  (2 Mbytes).
>
> (same remark on your initial bug report)
>
> Could you enable CONFIG_FRAME_POINTER=y in your config ?

I can, but my bosses will kick my ass if I bring down the ISP again :)

If you think it's the only way to find the problem I'll tell them that
I need to do it. In this case, please tell me what other config
options/tools I can use to get as much info as possible...since I'll
probably be able to test this only once more on the production
environment for debugging purposes.

Cheers,

Felipe Damasio
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Eric Dumazet July 14, 2010, 3:43 a.m. UTC | #22
Le mercredi 14 juillet 2010 à 00:27 -0300, Felipe W Damasio a écrit :
> Hi Mr. Dumazet,
> 
> 2010/7/14 Eric Dumazet <eric.dumazet@gmail.com>:
> > RDX being the sk pointer (and sk+0x38 contains the corrupted "sk_prot" value)
> > , we notice RBP contains same "sk" value + 0x200000  (2 Mbytes).
> >
> > (same remark on your initial bug report)
> >
> > Could you enable CONFIG_FRAME_POINTER=y in your config ?
> 
> I can, but my bosses will kick my ass if I bring down the ISP again :)
> 

I have no guarantee at all, even if we find the bug.

> If you think it's the only way to find the problem I'll tell them that
> I need to do it. In this case, please tell me what other config
> options/tools I can use to get as much info as possible...since I'll
> probably be able to test this only once more on the production
> environment for debugging purposes.
> 

You really should try to setup a lab to trigger the bug, and not doing
experiments on production :)



--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Felipe W Damasio July 14, 2010, 3:51 a.m. UTC | #23
Hi,

2010/7/14 Eric Dumazet <eric.dumazet@gmail.com>:
>> I can, but my bosses will kick my ass if I bring down the ISP again :)
>
> I have no guarantee at all, even if we find the bug.

Ok :-)

>> If you think it's the only way to find the problem I'll tell them that
>> I need to do it. In this case, please tell me what other config
>> options/tools I can use to get as much info as possible...since I'll
>> probably be able to test this only once more on the production
>> environment for debugging purposes.
>
> You really should try to setup a lab to trigger the bug, and not doing
> experiments on production :)

Right, I'm trying.

The thing is: The ISP is a 200Mbps network with 10,000 users. The
first time it took around 2 minutes to trigger the bug. The second
time it took around 17 minutes.

So I *think* it's some TCP flag with some weird content...but I can't
find out what it is so I can trigger it on the lab.

So my only guess is to enable every possible debug flag I can think of
to track the bug down on the production environment. Any hints here
would be appreciated :)

Cheers,

Felipe Damasio
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Bill Fink July 14, 2010, 6:56 a.m. UTC | #24
On Wed, 14 Jul 2010, Felipe W Damasio wrote:

> Hi,
> 
> 2010/7/14 Eric Dumazet <eric.dumazet@gmail.com>:
> >> I can, but my bosses will kick my ass if I bring down the ISP again :)
> >
> > I have no guarantee at all, even if we find the bug.
> 
> Ok :-)
> 
> >> If you think it's the only way to find the problem I'll tell them that
> >> I need to do it. In this case, please tell me what other config
> >> options/tools I can use to get as much info as possible...since I'll
> >> probably be able to test this only once more on the production
> >> environment for debugging purposes.
> >
> > You really should try to setup a lab to trigger the bug, and not doing
> > experiments on production :)
> 
> Right, I'm trying.
> 
> The thing is: The ISP is a 200Mbps network with 10,000 users. The
> first time it took around 2 minutes to trigger the bug. The second
> time it took around 17 minutes.
> 
> So I *think* it's some TCP flag with some weird content...but I can't
> find out what it is so I can trigger it on the lab.
> 
> So my only guess is to enable every possible debug flag I can think of
> to track the bug down on the production environment. Any hints here
> would be appreciated :)

Is it possible for you to mirror the production traffic to another
port, and then do a tcpdump capture to a series of files, so that
you might possibly be able to correlate the kernel crash to the
actual packets on the wire (and the Invalid Request squid errors)?
Just a suggestion.

						-Bill
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Patrick McHardy July 14, 2010, 11:41 a.m. UTC | #25
On 09.07.2010 19:13, Eric Dumazet wrote:
> Le vendredi 09 juillet 2010 à 12:03 -0300, Felipe W Damasio a écrit :
>> Hi,
>>
>> 2010/7/8 Eric Dumazet <eric.dumazet@gmail.com>:
>>> Please try to reproduce a new report.
>>>
>>> It looks like a memory corruption, and it would be good to see if a
>>> common pattern is occurring.
>>
>> I'm trying..the thing is the freeze occured on the machine that sits
>> on a 200Mbps ISP in bridge-mode. Since the machine frooze, and the
>> whole ISP went down for a few minutes, I'm not allowed to run any
>> tests on it.
>>
>> I've setup the same scenario on a lab, but since last night been
>> unable to reproduce the bug. Maybe there's a clue on the this crash
>> below that can help me write some program to trigger the problem?
>>
> 
> Reviewing tproxy stuff I spotted a problem in nf_tproxy_assign_sock()
> but I could not see how it could explain your crash.
> 
> We can read uninitialized memory and trigger a fault in
> nf_tproxy_assign_sock(), not later in tcp_recvmsg()...
> 
> David, Patrick, what do you think ?
> 
> Thanks
> 
> [PATCH] tproxy: nf_tproxy_assign_sock() can handle tw sockets
> 
> transparent field of a socket is either inet_twsk(sk)->tw_transparent
> for timewait sockets, or inet_sk(sk)->transparent for other sockets
> (TCP/UDP).

I don't see anything preventing use of timewait sockets, so the
patch looks correct to me.

Applied to nf-2.6.git, thanks Eric.
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Felipe W Damasio July 16, 2010, 3:41 p.m. UTC | #26
Hi All,

2010/7/14 Felipe W Damasio <felipewd@gmail.com>:
> Hi Mr. Dumazet,
>
> 2010/7/14 Eric Dumazet <eric.dumazet@gmail.com>:
>> RDX being the sk pointer (and sk+0x38 contains the corrupted "sk_prot" value)
>> , we notice RBP contains same "sk" value + 0x200000  (2 Mbytes).
>>
>> (same remark on your initial bug report)
>>
>> Could you enable CONFIG_FRAME_POINTER=y in your config ?

I did, this is the new bug:

general protection fault: 0000 [#1] SMP
last sysfs file: /sys/devices/pci0000:00/0000:00:1f.3/i2c-0/name
CPU 2
Modules linked in: e1000e

Pid: 4209, comm: squid Not tainted 2.6.34 #4 DX58SO/
RIP: 0010:[<ffffffff8137a887>]  [<ffffffff8137a887>] sock_rfree+0x2a/0x3c
RSP: 0018:ffff88042d781ba8  EFLAGS: 00010282
RAX: 9a7e7f4602400d48 RBX: ffff88034c918e00 RCX: 0000000000000720
RDX: ffff880413a82e00 RSI: ffff8804161e5e2a RDI: ffff88034c918e00
RBP: ffff88042d781ba8 R08: ffff88042d781b98 R09: 0000000000000000
R10: 0000000000040570 R11: 0000000000000000 R12: ffff880413882e00
R13: 00000000000005a8 R14: 00000000000005a8 R15: 000000000000a84d
FS:  00007f9aa0007710(0000) GS:ffff880001a80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f11f831f020 CR3: 000000042d5f8000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process squid (pid: 4209, threadinfo ffff88042d780000, task ffff88042e325620)
Stack:
 ffff88042d781bc8 ffffffff8137fda0 ffff880413882e00 ffff88034c918e00
<0> ffff88042d781be8 ffffffff8137fb3b ffff88034c918e00 ffff88034c918e00
<0> ffff88042d781cd8 ffffffff813be69b ffff88042d781c38 ffffffff813c76e4
Call Trace:
 [<ffffffff8137fda0>] skb_release_head_state+0x75/0xc0
 [<ffffffff8137fb3b>] __kfree_skb+0x11/0x86
 [<ffffffff813be69b>] tcp_recvmsg+0x6b9/0x8be
 [<ffffffff813c76e4>] ? tcp_current_mss+0x46/0x65
 [<ffffffff8137ab89>] sock_common_recvmsg+0x32/0x47
 [<ffffffff811bafb6>] ? selinux_socket_recvmsg+0x1d/0x1f
 [<ffffffff81378752>] __sock_recvmsg+0x6a/0x76
 [<ffffffff81378847>] sock_aio_read+0xe9/0x102
 [<ffffffff811b9d68>] ? avc_has_perm+0x4e/0x60
 [<ffffffff810b63f6>] do_sync_read+0xc7/0x10d
 [<ffffffff811bdb17>] ? selinux_file_permission+0xa5/0xb2
 [<ffffffff811b7917>] ? security_file_permission+0x11/0x13
 [<ffffffff810b6e7b>] vfs_read+0xbb/0x102
 [<ffffffff810b6f86>] sys_read+0x47/0x70
 [<ffffffff810029eb>] system_call_fastpath+0x16/0x1b
Code: c3 48 8b 57 18 55 8b 87 d8 00 00 00 48 89 e5 48 8d 8a ac 00 00
00 f0 29 82 ac 00 00 00 48 8b 57 18 8b 8f d8 00 00 00 48 8b 42 38 <48>
83 b8 b0 00 00 00 00 74 06 01 8a f4 00 00 00 c9 c3 55 48 89
RIP  [<ffffffff8137a887>] sock_rfree+0x2a/0x3c
 RSP <ffff88042d781ba8>
---[ end trace 8932efc1ba58ce6e ]---

Does this tell you anything?

Cheers,

Felipe Damasio
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Eric Dumazet July 16, 2010, 3:52 p.m. UTC | #27
Le vendredi 16 juillet 2010 à 12:41 -0300, Felipe W Damasio a écrit :
> Hi All,
> 
> 2010/7/14 Felipe W Damasio <felipewd@gmail.com>:
> > Hi Mr. Dumazet,
> >
> > 2010/7/14 Eric Dumazet <eric.dumazet@gmail.com>:
> >> RDX being the sk pointer (and sk+0x38 contains the corrupted "sk_prot" value)
> >> , we notice RBP contains same "sk" value + 0x200000  (2 Mbytes).
> >>
> >> (same remark on your initial bug report)
> >>
> >> Could you enable CONFIG_FRAME_POINTER=y in your config ?
> 
> I did, this is the new bug:
> 
> general protection fault: 0000 [#1] SMP
> last sysfs file: /sys/devices/pci0000:00/0000:00:1f.3/i2c-0/name
> CPU 2
> Modules linked in: e1000e
> 
> Pid: 4209, comm: squid Not tainted 2.6.34 #4 DX58SO/
> RIP: 0010:[<ffffffff8137a887>]  [<ffffffff8137a887>] sock_rfree+0x2a/0x3c
> RSP: 0018:ffff88042d781ba8  EFLAGS: 00010282
> RAX: 9a7e7f4602400d48 RBX: ffff88034c918e00 RCX: 0000000000000720
> RDX: ffff880413a82e00 RSI: ffff8804161e5e2a RDI: ffff88034c918e00
> RBP: ffff88042d781ba8 R08: ffff88042d781b98 R09: 0000000000000000
> R10: 0000000000040570 R11: 0000000000000000 R12: ffff880413882e00
> R13: 00000000000005a8 R14: 00000000000005a8 R15: 000000000000a84d
> FS:  00007f9aa0007710(0000) GS:ffff880001a80000(0000) knlGS:0000000000000000
> CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> CR2: 00007f11f831f020 CR3: 000000042d5f8000 CR4: 00000000000006e0
> DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> Process squid (pid: 4209, threadinfo ffff88042d780000, task ffff88042e325620)
> Stack:
>  ffff88042d781bc8 ffffffff8137fda0 ffff880413882e00 ffff88034c918e00
> <0> ffff88042d781be8 ffffffff8137fb3b ffff88034c918e00 ffff88034c918e00
> <0> ffff88042d781cd8 ffffffff813be69b ffff88042d781c38 ffffffff813c76e4
> Call Trace:
>  [<ffffffff8137fda0>] skb_release_head_state+0x75/0xc0
>  [<ffffffff8137fb3b>] __kfree_skb+0x11/0x86
>  [<ffffffff813be69b>] tcp_recvmsg+0x6b9/0x8be
>  [<ffffffff813c76e4>] ? tcp_current_mss+0x46/0x65
>  [<ffffffff8137ab89>] sock_common_recvmsg+0x32/0x47
>  [<ffffffff811bafb6>] ? selinux_socket_recvmsg+0x1d/0x1f
>  [<ffffffff81378752>] __sock_recvmsg+0x6a/0x76
>  [<ffffffff81378847>] sock_aio_read+0xe9/0x102
>  [<ffffffff811b9d68>] ? avc_has_perm+0x4e/0x60
>  [<ffffffff810b63f6>] do_sync_read+0xc7/0x10d
>  [<ffffffff811bdb17>] ? selinux_file_permission+0xa5/0xb2
>  [<ffffffff811b7917>] ? security_file_permission+0x11/0x13
>  [<ffffffff810b6e7b>] vfs_read+0xbb/0x102
>  [<ffffffff810b6f86>] sys_read+0x47/0x70
>  [<ffffffff810029eb>] system_call_fastpath+0x16/0x1b
> Code: c3 48 8b 57 18 55 8b 87 d8 00 00 00 48 89 e5 48 8d 8a ac 00 00
> 00 f0 29 82 ac 00 00 00 48 8b 57 18 8b 8f d8 00 00 00 48 8b 42 38 <48>
> 83 b8 b0 00 00 00 00 74 06 01 8a f4 00 00 00 c9 c3 55 48 89
> RIP  [<ffffffff8137a887>] sock_rfree+0x2a/0x3c
>  RSP <ffff88042d781ba8>
> ---[ end trace 8932efc1ba58ce6e ]---
> 
> Does this tell you anything?
> 

Could you privatly send me the vmlinux file ?


--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
diff mbox

Patch

diff --git a/net/netfilter/nf_tproxy_core.c b/net/netfilter/nf_tproxy_core.c
index 5490fc3..daab8c4 100644
--- a/net/netfilter/nf_tproxy_core.c
+++ b/net/netfilter/nf_tproxy_core.c
@@ -70,7 +70,11 @@  nf_tproxy_destructor(struct sk_buff *skb)
 int
 nf_tproxy_assign_sock(struct sk_buff *skb, struct sock *sk)
 {
-	if (inet_sk(sk)->transparent) {
+	bool transparent = (sk->sk_state == TCP_TIME_WAIT) ?
+				inet_twsk(sk)->tw_transparent :
+				inet_sk(sk)->transparent;
+
+	if (transparent) {
 		skb_orphan(skb);
 		skb->sk = sk;
 		skb->destructor = nf_tproxy_destructor;