From patchwork Sat Mar 14 20:16:51 2009 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Boutcher X-Patchwork-Id: 24455 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from vger.kernel.org (vger.kernel.org [209.132.176.167]) by ozlabs.org (Postfix) with ESMTP id 2B3F1DDFE2 for ; Sun, 15 Mar 2009 07:18:18 +1100 (EST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756469AbZCNUQ5 (ORCPT ); Sat, 14 Mar 2009 16:16:57 -0400 Received: (majordomo@vger.kernel.org) by vger.kernel.org id S1752579AbZCNUQ4 (ORCPT ); Sat, 14 Mar 2009 16:16:56 -0400 Received: from yw-out-2324.google.com ([74.125.46.31]:41644 "EHLO yw-out-2324.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1757872AbZCNUQy (ORCPT ); Sat, 14 Mar 2009 16:16:54 -0400 Received: by yw-out-2324.google.com with SMTP id 5so582209ywh.1 for ; Sat, 14 Mar 2009 13:16:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:date:message-id:subject :from:to:content-type:content-transfer-encoding; bh=W3qG4flKB7p2gr5fCJ2/M5CSNPePJR+R3/7xL+S9upg=; b=Co02sePKjZhoM5UF0GbIeFEs0CjxRHb7GOqbjdCcN53+no8YmQetoV1+qVma30BaPb 7Gany2xnI/MlX+xD0ZLUWo701FVG+g7jB/dGwYj30PEfhrikOKOg61n37M9tWAflwoNr HIaZIIjdix/8G9ibN9mg+F4eAPKmE3gvI/L6E= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type :content-transfer-encoding; b=KWIftTjFnmrSxokc+vzmwpmqjQMfH20O+xa/vJZ6urp0UbMZeq1EMdm+OSlzPrWDO/ hS867PkfnsfDbTfG7/TxWWnFUn0v9iIgK0iF04kA+zePRmsrMgRzy/pcs/H/uh1KJYww LpolRRvaNRxBtzf2kukbQo6ma+kbGmqLaLKt0= MIME-Version: 1.0 Received: by 10.100.58.18 with SMTP id g18mr1786012ana.125.1237061811836; Sat, 14 Mar 2009 13:16:51 -0700 (PDT) Date: Sat, 14 Mar 2009 15:16:51 -0500 Message-ID: <91bdcedb0903141316j2dbf4160wb348a5a9e3bde8ad@mail.gmail.com> Subject: IGMP Join dropping multicast packets From: Dave Boutcher To: netdev@vger.kernel.org Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org I'm running into an interesting problem with joining multiple multicast feeds. If you join multiple multicast feeds using setsockopt(...,IP_ADD_MEMBERSHIP...) it causes packets on UNRELATED multicast feeds to get dropped. We have a multicast feed on a rock solid network, and we were very surprised to see dropped packets. The cause was a different process/program being run by a different user joining a bunch of mulitcast feeds. I can recreate this with a fairly simple testcase (attached below.) The problem doesn't happen with unicast UDP data, and it doesn't happen with loopback, so you need at least two systems to run this (and what subscriber to netdev doesn't have at least two systems.) To recreate, run "receiver" on one system, "sender", on another, and then "joiner" on the receiving system. You should see a message pop out saying that packets have been dropped. I've recreated this on a few different kernel versions (the latest being 2.6.28) and a few different sets off hardware. I HAVEN"T recreated it if the system doing the IP_ADD_MEMBERSHIP specifies a specific interface rather than INADDR_ANY. I'm not sure if that is core to the issue or not. You may also need to bump the value in /proc/sys/net/ipv4/igmp_max_memberships (though that hasn't seemed necessary for me.) I poked around in igmp.c, but its mojo exceeds my threshold. If anyone has any ideas or questions I'd be happy to hear them. diff -uNr null/joiner.c multicast/joiner.c --- null/joiner.c 1969-12-31 18:00:00.000000000 -0600 +++ multicast/joiner.c 2009-03-14 15:04:10.000000000 -0500 @@ -0,0 +1,44 @@ +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#define NUMSOCK 55 + +int main(int argc, char **argv) +{ + struct ip_mreq mreq; + int i; + int sd; + char ipaddr[64]; + + for (i=0; i +#include +#include +#include +#include +#include +#include +#include +#include + +#include "mctest.h" + +int main(int argc, char **argv) +{ + int sd = socket(PF_INET, SOCK_DGRAM, IPPROTO_UDP); + int on = 1; + struct sockaddr_in addr; + uint32_t seq = 1; + int bytes; + struct ip_mreq mreq; + + struct mcdata data; + + if (sd < 0) { + perror("socket"); + exit(0); + } + + mreq.imr_multiaddr.s_addr = inet_addr("239.192.1.1"); + mreq.imr_interface.s_addr = htonl(INADDR_ANY); + + if (setsockopt(sd, IPPROTO_IP, IP_ADD_MEMBERSHIP, &mreq, sizeof(mreq))) { + perror("IP_ADD_MEMBERSHIP"); + exit(0); + } + + if (setsockopt(sd, SOL_SOCKET, SO_REUSEADDR, &on, sizeof(int))) { + perror("SO_REUSEADDR"); + exit(0); + } + + bzero(&addr, sizeof(addr)); + addr.sin_family = AF_INET; + addr.sin_port = ntohs(60604); + addr.sin_addr.s_addr = inet_addr("239.192.1.1"); + + if (bind(sd, (struct sockaddr*)&addr, sizeof(addr))) { + perror("bind"); + exit(0); + } + + while(1) { + bytes = recv(sd, &data, sizeof(data), 0); + if (bytes != sizeof(data)) { + printf("recv got %d, expected %lu\n", + bytes, sizeof(data)); + exit(0); + } + + if ((ntohl(data.seq1) != seq) || + (ntohl(data.seq2) != seq)) { + printf("Mismatched seq! Expected %u, got %u/%u\n", + seq, ntohl(data.seq1), ntohl(data.seq2)); + } + + seq = ntohl(data.seq1)+1; + if (seq % 10000 == 0) + printf("got seq %u\n",seq); + } +} + diff -uNr null/sender.c multicast/sender.c --- null/sender.c 1969-12-31 18:00:00.000000000 -0600 +++ multicast/sender.c 2009-03-14 14:47:13.000000000 -0500 @@ -0,0 +1,55 @@ +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "mctest.h" + +int main(int argc, char **argv) +{ + int sd = socket(PF_INET, SOCK_DGRAM, IPPROTO_UDP); + int on = 1; + struct sockaddr_in addr; + uint32_t seq = 1; + int bytes; + + struct mcdata data; + + if (sd < 0) { + perror("socket"); + exit(0); + } + + if (setsockopt(sd, IPPROTO_IP, IP_MULTICAST_LOOP, &on, sizeof(int))) { + perror("IO_MULTICAST_LOOP"); + exit(0); + } + + bzero(&addr, sizeof(addr)); + addr.sin_family = AF_INET; + addr.sin_port = ntohs(60604); + addr.sin_addr.s_addr = inet_addr("239.192.1.1"); + + memset(data.data, 0xdb, sizeof(data.data)); + + while (1) { + data.seq1 = data.seq2 = htonl(seq); + + bytes = sendto(sd, &data, sizeof(data), 0, + (struct sockaddr *)&addr, sizeof(addr)); + if (bytes != sizeof(data)) { + perror("send"); + exit(0); + } + + seq++; + usleep(1000); + } + + return 0; +}