[{"id":1768310,"web_url":"http://patchwork.ozlabs.org/comment/1768310/","msgid":"<0d7068aa-7331-0ce0-145d-8afe1e83a8ec@ozlabs.ru>","date":"2017-09-14T02:45:06","subject":"Re: [PATCH kernel] powerpc/powernv: Update comment about shifting\n\tIOV BAR","submitter":{"id":7621,"url":"http://patchwork.ozlabs.org/api/people/7621/","name":"Alexey Kardashevskiy","email":"aik@ozlabs.ru"},"content":"On 31/08/17 13:34, Alexey Kardashevskiy wrote:\n> From: Benjamin Herrenschmidt \n\nOops, this was not right :)\n\nAnyway, Ben, please comment. Thanks.\n\n\n> \n> From: Alexey Kardashevskiy \n> \n> This updates the comment about creating a hole in /proc/iomem which\n> should not be normally happening but it does in the powernv platform\n> due the way MMIO M64 BARs are organised in the IODA2-capable hardware.\n> \n> Signed-off-by: Alexey Kardashevskiy \n> ---\n> \n> It has been mentioned multiple times (last one -\n> https://www.spinics.net/lists/linux-pci/msg64084.html ) that the comment\n> is not informative enough for people not particularly familiar with\n> the POWER8 IO hardware.\n> \n> This attempt aims to:\n> 1. explain why we shift the resource\n> 2. explain why nothing can use that hole as a resource while it is \"free\"\n> (I am not sure that this is the case actually)\n> \n> Please comment, everyone, let's have this very well documented while\n> I remember these bits :) Thanks.\n> ---\n> arch/powerpc/platforms/powernv/pci-ioda.c | 10 +++++++---\n> 1 file changed, 7 insertions(+), 3 deletions(-)\n> \n> diff --git a/arch/powerpc/platforms/powernv/pci-ioda.c b/arch/powerpc/platforms/powernv/pci-ioda.c\n> index 48de308224d6..c4a36ae78c95 100644\n> --- a/arch/powerpc/platforms/powernv/pci-ioda.c\n> +++ b/arch/powerpc/platforms/powernv/pci-ioda.c\n> @@ -1002,9 +1002,13 @@ static int pnv_pci_vf_resource_shift(struct pci_dev *dev, int offset)\n> \t}\n> \n> \t/*\n> -\t * After doing so, there would be a \"hole\" in the /proc/iomem when\n> -\t * offset is a positive value. It looks like the device return some\n> -\t * mmio back to the system, which actually no one could use it.\n> +\t * Since M64 BAR shares segments among all possible 256 PEs,\n> +\t * we have to shift the beginning of PF IOV BAR to make it start from\n> +\t * the segment which belongs to the PE number assigned to the first VF.\n> +\t * This creates a \"hole\" in the /proc/iomem which could be used for\n> +\t * allocating other resources, however this is not expected to happen\n> +\t * on IODA as the only possibility would be a PCI hotplug and IODA\n> +\t * hardware only allows it on a slot with dedicated PHB.\n> \t */\n> \tfor (i = 0; i < PCI_SRIOV_NUM_BARS; i++) {\n> \t\tres = &dev->resource[i + PCI_IOV_RESOURCES];\n>","headers":{"Return-Path":"","X-Original-To":["patchwork-incoming@ozlabs.org","linuxppc-dev@lists.ozlabs.org"],"Delivered-To":["patchwork-incoming@ozlabs.org","linuxppc-dev@lists.ozlabs.org"],"Received":["from lists.ozlabs.org (lists.ozlabs.org [103.22.144.68])\n\t(using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xt2x43xjSz9t2f\n\tfor ;\n\tThu, 14 Sep 2017 12:46:48 +1000 (AEST)","from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3])\n\tby lists.ozlabs.org (Postfix) with ESMTP id 3xt2x42p2TzDqkv\n\tfor ;\n\tThu, 14 Sep 2017 12:46:48 +1000 (AEST)","from mail-pg0-x241.google.com (mail-pg0-x241.google.com\n\t[IPv6:2607:f8b0:400e:c05::241])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128\n\tbits)) (No client certificate requested)\n\tby lists.ozlabs.org (Postfix) with ESMTPS id 3xt2vJ5TZRzDqYQ\n\tfor ;\n\tThu, 14 Sep 2017 12:45:15 +1000 (AEST)","by mail-pg0-x241.google.com with SMTP id j16so992485pga.2\n\tfor ;\n\tWed, 13 Sep 2017 19:45:15 -0700 (PDT)","from [10.61.2.175] ([122.99.82.10])\n\tby smtp.googlemail.com with ESMTPSA id\n\tb75sm26554802pfc.29.2017.09.13.19.45.08\n\t(version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);\n\tWed, 13 Sep 2017 19:45:11 -0700 (PDT)"],"Authentication-Results":["ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=ozlabs-ru.20150623.gappssmtp.com\n\theader.i=@ozlabs-ru.20150623.gappssmtp.com\n\theader.b=\"uwWRS1C3\"; dkim-atps=neutral","lists.ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=ozlabs-ru.20150623.gappssmtp.com\n\theader.i=@ozlabs-ru.20150623.gappssmtp.com\n\theader.b=\"uwWRS1C3\"; dkim-atps=neutral","ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=ozlabs.ru\n\t(client-ip=2607:f8b0:400e:c05::241; helo=mail-pg0-x241.google.com;\n\tenvelope-from=aik@ozlabs.ru; receiver=)","lists.ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=ozlabs-ru.20150623.gappssmtp.com\n\theader.i=@ozlabs-ru.20150623.gappssmtp.com header.b=\"uwWRS1C3\"; \n\tdkim-atps=neutral"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=ozlabs-ru.20150623.gappssmtp.com; s=20150623;\n\th=subject:to:cc:references:from:message-id:date:user-agent\n\t:mime-version:in-reply-to:content-language:content-transfer-encoding; \n\tbh=W8ybXUrFH4oJh2WWlPxHDeksYrpW6sZ/3S1IYN55vUM=;\n\tb=uwWRS1C38JCgQ6ocfHZnCPxtjaKsikpOsso0Q5Qeh9vpNFM0PpiYYhqQEfpCcUcrdT\n\tFVU2X6mjLA8Bm5Lx3Zum79kNblUbHrO4EEi9hycPla7moSmJRQ2wnQIsn+DaY4SBX6T8\n\tH0XuH4slvltu66XDDQRuqAk8a4cmIo2HDI/ScoRoQ2uCsjshWe0pT9RnpaCUpseyBV2H\n\tffCLxpeDDWCjJ///PcpskNAzOANmZY0pxfwSm4j2MLUtmKWNK56vZpQ3SFlY4tGgjcSJ\n\tAY6ggmCI0xlOP8gKWd0lkHMoIH0KhNZKqLGQO2/Bl21d20ksWE/IjJYUmU88x7EEdip9\n\tTlmA==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:subject:to:cc:references:from:message-id:date\n\t:user-agent:mime-version:in-reply-to:content-language\n\t:content-transfer-encoding;\n\tbh=W8ybXUrFH4oJh2WWlPxHDeksYrpW6sZ/3S1IYN55vUM=;\n\tb=Ivg196iRV4dNFsIX7FWPaxyZZM7zrSG3YmuON4Rr/UGNRN1QVpi/gRf7aufUmzuGmm\n\tGD/hHIj5BfgsfDLAFe+pscG04oQHYSVPMBXxS1e+FmPf4mhFcz751XOLpEd0rSWAuBTq\n\t7HKo5vyBqvvSBJBetBSrfU6eOREQgKLyg5yKqhMOyyIjFhfW5X8hIstSdC0Il7qohk3z\n\tu22XYloV/n5x9vCcWwzWVyItD6+X1KfjVwJfacgjm+c+iScqUpDsqOk9+dvyIS1vDpjQ\n\tlk4bx+/qQTe0JwODUjrQeYmYg+ENVbHTQ3KQJg2lpvUQCuk9uXiq6BdG3b+EixslwqT9\n\tlQaA==","X-Gm-Message-State":"AHPjjUgJ5MpYmKPJ4ZejdgJSENa+sY4jRdS29rt4HAlDqr8WBxusLtC/\n\twbM25knYJLqGKuj7","X-Google-Smtp-Source":"ADKCNb54+dye6KjMNZPoUV1SHY0TQLClSYxCLQsNtO+D0FEFywLqF97z35aqCOIt+Pl9bXT3CVwVfQ==","X-Received":"by 10.99.126.87 with SMTP id o23mr20301268pgn.213.1505357112566; \n\tWed, 13 Sep 2017 19:45:12 -0700 (PDT)","Subject":"Re: [PATCH kernel] powerpc/powernv: Update comment about shifting\n\tIOV BAR","To":"Benjamin Herrenschmidt ","References":"<20170831033412.1971-1-aik@ozlabs.ru>","From":"Alexey Kardashevskiy ","Message-ID":"<0d7068aa-7331-0ce0-145d-8afe1e83a8ec@ozlabs.ru>","Date":"Thu, 14 Sep 2017 12:45:06 +1000","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.2.1","MIME-Version":"1.0","In-Reply-To":"<20170831033412.1971-1-aik@ozlabs.ru>","Content-Type":"text/plain; charset=utf-8","Content-Language":"en-AU","Content-Transfer-Encoding":"7bit","X-BeenThere":"linuxppc-dev@lists.ozlabs.org","X-Mailman-Version":"2.1.24","Precedence":"list","List-Id":"Linux on PowerPC Developers Mail List\n\t","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Alistair Popple ,\n\tBjorn Helgaas , \n\tshan.gavin@gmail.com, Paul Mackerras ,\n\tlinuxppc-dev@lists.ozlabs.org, David Gibson ","Errors-To":"linuxppc-dev-bounces+patchwork-incoming=ozlabs.org@lists.ozlabs.org","Sender":"\"Linuxppc-dev\"\n\t"}},{"id":1768313,"web_url":"http://patchwork.ozlabs.org/comment/1768313/","msgid":"<1505358454.12628.185.camel@kernel.crashing.org>","date":"2017-09-14T03:07:34","subject":"Re: [PATCH kernel] powerpc/powernv: Update comment about shifting\n\tIOV BAR","submitter":{"id":38,"url":"http://patchwork.ozlabs.org/api/people/38/","name":"Benjamin Herrenschmidt","email":"benh@kernel.crashing.org"},"content":"On Thu, 2017-09-14 at 12:45 +1000, Alexey Kardashevskiy wrote:\n> On 31/08/17 13:34, Alexey Kardashevskiy wrote:\n> > From: Benjamin Herrenschmidt \n> \n> Oops, this was not right :)\n> \n> Anyway, Ben, please comment. Thanks.\n\nThis is incorrect, we can do hotplug behind switches afaik.\n> \n> > \n> > From: Alexey Kardashevskiy \n> > \n> > This updates the comment about creating a hole in /proc/iomem which\n> > should not be normally happening but it does in the powernv platform\n> > due the way MMIO M64 BARs are organised in the IODA2-capable hardware.\n> > \n> > Signed-off-by: Alexey Kardashevskiy \n> > ---\n> > \n> > It has been mentioned multiple times (last one -\n> > https://www.spinics.net/lists/linux-pci/msg64084.html ) that the comment\n> > is not informative enough for people not particularly familiar with\n> > the POWER8 IO hardware.\n> > \n> > This attempt aims to:\n> > 1. explain why we shift the resource\n> > 2. explain why nothing can use that hole as a resource while it is \"free\"\n> > (I am not sure that this is the case actually)\n> > \n> > Please comment, everyone, let's have this very well documented while\n> > I remember these bits :) Thanks.\n> > ---\n> > arch/powerpc/platforms/powernv/pci-ioda.c | 10 +++++++---\n> > 1 file changed, 7 insertions(+), 3 deletions(-)\n> > \n> > diff --git a/arch/powerpc/platforms/powernv/pci-ioda.c b/arch/powerpc/platforms/powernv/pci-ioda.c\n> > index 48de308224d6..c4a36ae78c95 100644\n> > --- a/arch/powerpc/platforms/powernv/pci-ioda.c\n> > +++ b/arch/powerpc/platforms/powernv/pci-ioda.c\n> > @@ -1002,9 +1002,13 @@ static int pnv_pci_vf_resource_shift(struct pci_dev *dev, int offset)\n> > \t}\n> > \n> > \t/*\n> > -\t * After doing so, there would be a \"hole\" in the /proc/iomem when\n> > -\t * offset is a positive value. It looks like the device return some\n> > -\t * mmio back to the system, which actually no one could use it.\n> > +\t * Since M64 BAR shares segments among all possible 256 PEs,\n> > +\t * we have to shift the beginning of PF IOV BAR to make it start from\n> > +\t * the segment which belongs to the PE number assigned to the first VF.\n> > +\t * This creates a \"hole\" in the /proc/iomem which could be used for\n> > +\t * allocating other resources, however this is not expected to happen\n> > +\t * on IODA as the only possibility would be a PCI hotplug and IODA\n> > +\t * hardware only allows it on a slot with dedicated PHB.\n> > \t */\n> > \tfor (i = 0; i < PCI_SRIOV_NUM_BARS; i++) {\n> > \t\tres = &dev->resource[i + PCI_IOV_RESOURCES];\n> > \n> \n>","headers":{"Return-Path":"","X-Original-To":["patchwork-incoming@ozlabs.org","linuxppc-dev@lists.ozlabs.org"],"Delivered-To":["patchwork-incoming@ozlabs.org","linuxppc-dev@lists.ozlabs.org"],"Received":["from lists.ozlabs.org (lists.ozlabs.org [103.22.144.68])\n\t(using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xt3TM3CRdz9sP1\n\tfor ;\n\tThu, 14 Sep 2017 13:11:19 +1000 (AEST)","from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3])\n\tby lists.ozlabs.org (Postfix) with ESMTP id 3xt3TM1prWzDrCb\n\tfor ;\n\tThu, 14 Sep 2017 13:11:19 +1000 (AEST)","from gate.crashing.org (gate.crashing.org [63.228.1.57])\n\t(using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby lists.ozlabs.org (Postfix) with ESMTPS id 3xt3Pn6McPzDqvS\n\tfor ;\n\tThu, 14 Sep 2017 13:08:13 +1000 (AEST)","from localhost (localhost.localdomain [127.0.0.1])\n\tby gate.crashing.org (8.14.1/8.13.8) with ESMTP id v8E37aQm001747;\n\tWed, 13 Sep 2017 22:07:44 -0500"],"Authentication-Results":"ozlabs.org; spf=permerror (mailfrom)\n\tsmtp.mailfrom=kernel.crashing.org (client-ip=63.228.1.57;\n\thelo=gate.crashing.org; envelope-from=benh@kernel.crashing.org;\n\treceiver=)","Message-ID":"<1505358454.12628.185.camel@kernel.crashing.org>","Subject":"Re: [PATCH kernel] powerpc/powernv: Update comment about shifting\n\tIOV BAR","From":"Benjamin Herrenschmidt ","To":"Alexey Kardashevskiy ","Date":"Thu, 14 Sep 2017 13:07:34 +1000","In-Reply-To":"<0d7068aa-7331-0ce0-145d-8afe1e83a8ec@ozlabs.ru>","References":"<20170831033412.1971-1-aik@ozlabs.ru>\n\t<0d7068aa-7331-0ce0-145d-8afe1e83a8ec@ozlabs.ru>","Content-Type":"text/plain; charset=\"UTF-8\"","X-Mailer":"Evolution 3.24.5 (3.24.5-1.fc26) ","Mime-Version":"1.0","Content-Transfer-Encoding":"7bit","X-BeenThere":"linuxppc-dev@lists.ozlabs.org","X-Mailman-Version":"2.1.24","Precedence":"list","List-Id":"Linux on PowerPC Developers Mail List\n\t","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Alistair Popple ,\n\tBjorn Helgaas , \n\tshan.gavin@gmail.com, Paul Mackerras ,\n\tlinuxppc-dev@lists.ozlabs.org, David Gibson ","Errors-To":"linuxppc-dev-bounces+patchwork-incoming=ozlabs.org@lists.ozlabs.org","Sender":"\"Linuxppc-dev\"\n\t"}},{"id":1768315,"web_url":"http://patchwork.ozlabs.org/comment/1768315/","msgid":"<18f3462e-09ee-ce02-f9c6-eafad68cb9c6@ozlabs.ru>","date":"2017-09-14T03:18:17","subject":"Re: [PATCH kernel] powerpc/powernv: Update comment about shifting\n\tIOV BAR","submitter":{"id":7621,"url":"http://patchwork.ozlabs.org/api/people/7621/","name":"Alexey Kardashevskiy","email":"aik@ozlabs.ru"},"content":"On 14/09/17 13:07, Benjamin Herrenschmidt wrote:\n> On Thu, 2017-09-14 at 12:45 +1000, Alexey Kardashevskiy wrote:\n>> On 31/08/17 13:34, Alexey Kardashevskiy wrote:\n>>> From: Benjamin Herrenschmidt \n>>\n>> Oops, this was not right :)\n>>\n>> Anyway, Ben, please comment. Thanks.\n> \n> This is incorrect, we can do hotplug behind switches afaik.\n\nDo we have an actual system which allows this? Anyway, what we do now is\nwrong and it needs what? Reserve that hole? I'd like to update the comment\nfor now, at least, and state what bad thing can happen and what we expect.\n\n\n>>\n>>>\n>>> From: Alexey Kardashevskiy \n>>>\n>>> This updates the comment about creating a hole in /proc/iomem which\n>>> should not be normally happening but it does in the powernv platform\n>>> due the way MMIO M64 BARs are organised in the IODA2-capable hardware.\n>>>\n>>> Signed-off-by: Alexey Kardashevskiy \n>>> ---\n>>>\n>>> It has been mentioned multiple times (last one -\n>>> https://www.spinics.net/lists/linux-pci/msg64084.html ) that the comment\n>>> is not informative enough for people not particularly familiar with\n>>> the POWER8 IO hardware.\n>>>\n>>> This attempt aims to:\n>>> 1. explain why we shift the resource\n>>> 2. explain why nothing can use that hole as a resource while it is \"free\"\n>>> (I am not sure that this is the case actually)\n>>>\n>>> Please comment, everyone, let's have this very well documented while\n>>> I remember these bits :) Thanks.\n>>> ---\n>>> arch/powerpc/platforms/powernv/pci-ioda.c | 10 +++++++---\n>>> 1 file changed, 7 insertions(+), 3 deletions(-)\n>>>\n>>> diff --git a/arch/powerpc/platforms/powernv/pci-ioda.c b/arch/powerpc/platforms/powernv/pci-ioda.c\n>>> index 48de308224d6..c4a36ae78c95 100644\n>>> --- a/arch/powerpc/platforms/powernv/pci-ioda.c\n>>> +++ b/arch/powerpc/platforms/powernv/pci-ioda.c\n>>> @@ -1002,9 +1002,13 @@ static int pnv_pci_vf_resource_shift(struct pci_dev *dev, int offset)\n>>> \t}\n>>> \n>>> \t/*\n>>> -\t * After doing so, there would be a \"hole\" in the /proc/iomem when\n>>> -\t * offset is a positive value. It looks like the device return some\n>>> -\t * mmio back to the system, which actually no one could use it.\n>>> +\t * Since M64 BAR shares segments among all possible 256 PEs,\n>>> +\t * we have to shift the beginning of PF IOV BAR to make it start from\n>>> +\t * the segment which belongs to the PE number assigned to the first VF.\n>>> +\t * This creates a \"hole\" in the /proc/iomem which could be used for\n>>> +\t * allocating other resources, however this is not expected to happen\n>>> +\t * on IODA as the only possibility would be a PCI hotplug and IODA\n>>> +\t * hardware only allows it on a slot with dedicated PHB.\n>>> \t */\n>>> \tfor (i = 0; i < PCI_SRIOV_NUM_BARS; i++) {\n>>> \t\tres = &dev->resource[i + PCI_IOV_RESOURCES];\n>>>\n>>\n>>","headers":{"Return-Path":"","X-Original-To":["patchwork-incoming@ozlabs.org","linuxppc-dev@lists.ozlabs.org"],"Delivered-To":["patchwork-incoming@ozlabs.org","linuxppc-dev@lists.ozlabs.org"],"Received":["from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3])\n\t(using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xt3gG22mJz9sP1\n\tfor ;\n\tThu, 14 Sep 2017 13:19:54 +1000 (AEST)","from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3])\n\tby lists.ozlabs.org (Postfix) with ESMTP id 3xt3gG0s4dzDqjL\n\tfor ;\n\tThu, 14 Sep 2017 13:19:54 +1000 (AEST)","from mail-pg0-x241.google.com (mail-pg0-x241.google.com\n\t[IPv6:2607:f8b0:400e:c05::241])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128\n\tbits)) (No client certificate requested)\n\tby lists.ozlabs.org (Postfix) with ESMTPS id 3xt3dZ22P3zDqh4\n\tfor ;\n\tThu, 14 Sep 2017 13:18:26 +1000 (AEST)","by mail-pg0-x241.google.com with SMTP id v5so1027932pgn.4\n\tfor ;\n\tWed, 13 Sep 2017 20:18:25 -0700 (PDT)","from [10.61.2.175] ([122.99.82.10])\n\tby smtp.googlemail.com with ESMTPSA id\n\td126sm24804211pgc.85.2017.09.13.20.18.19\n\t(version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);\n\tWed, 13 Sep 2017 20:18:23 -0700 (PDT)"],"Authentication-Results":["ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=ozlabs-ru.20150623.gappssmtp.com\n\theader.i=@ozlabs-ru.20150623.gappssmtp.com\n\theader.b=\"U49Tf4zn\"; dkim-atps=neutral","lists.ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=ozlabs-ru.20150623.gappssmtp.com\n\theader.i=@ozlabs-ru.20150623.gappssmtp.com\n\theader.b=\"U49Tf4zn\"; dkim-atps=neutral","ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=ozlabs.ru\n\t(client-ip=2607:f8b0:400e:c05::241; helo=mail-pg0-x241.google.com;\n\tenvelope-from=aik@ozlabs.ru; receiver=)","lists.ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=ozlabs-ru.20150623.gappssmtp.com\n\theader.i=@ozlabs-ru.20150623.gappssmtp.com header.b=\"U49Tf4zn\"; \n\tdkim-atps=neutral"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=ozlabs-ru.20150623.gappssmtp.com; s=20150623;\n\th=subject:to:cc:references:from:message-id:date:user-agent\n\t:mime-version:in-reply-to:content-language:content-transfer-encoding; \n\tbh=wuuDEXWVJK0IryZ/0kVr8iOhR9javDS4AbyUaIkrK4k=;\n\tb=U49Tf4zn4lahC0aq6D89RzukZX/Yae+HFeyn9NWS1lqip+6AsaaI6Y7nREjZ1L0Ohc\n\t6TVWED205ppGijhhXt41si6gmLloN2Nc/7ls9L/0JaPvoRcTu0kID8RVIEttxckOLSFR\n\tXJoj6R7JxbWp4PWkMjqx2NiDJ8PLLdjb53n0WftVdRLhBpa66g6I9/+EpwD+Cclx13R7\n\tFTmMD5YZE95Rhovnuq1u/fu8+fRKWW9e4ACgkLKqFNwg5g5ar69Mk2fZTU1BwWVOpdeB\n\txXcpCtnaXz6c4j6/rUDtrlOvSFBJeMe3VaP9rFd2xNW73fbS754V+mor1M/5ZY4YQVv7\n\tksnA==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:subject:to:cc:references:from:message-id:date\n\t:user-agent:mime-version:in-reply-to:content-language\n\t:content-transfer-encoding;\n\tbh=wuuDEXWVJK0IryZ/0kVr8iOhR9javDS4AbyUaIkrK4k=;\n\tb=UcEZaOTGvp+DJasLxpDX1C8aWeOe1CDoZCB9NIP5AVCxB4f81BN+Ajgg+aDCZK4kQH\n\trShawRO3vRh9AbGiK/VZ17LP3fvv9GH9yYnRLVT+FEaivKsRiI5w7yoCywb78Sjg+CYs\n\tgEzl58oG+glKh2WSnF6OQkGZAbFmn7reeZvOJZFM3xRsel5O0NjSU1lhiM0iTemQRpBl\n\tP54i2JPGyQf6v+I9Kx7H+CZNI6ixMvD74FsGyuq7SDkch+RmP/LSEdFe3qJaAuXjCJpH\n\tJxVpBY1CYcv35yQQLyshpoSIzhrmnC75/mdqrI9xXuLQcbiPCFUPvvbsmJfUXlX7Urx8\n\tcFwA==","X-Gm-Message-State":"AHPjjUgquDUF/5suNU5Vq71ZD6RPmc91BsjmJ84U2AdGUAsD/8b46Mnd\n\turIpf0b6uaCjtnKI","X-Google-Smtp-Source":"ADKCNb7f/VOp1TTHEKsGuDfVXLF9qzRdC5glWicVikF3N0lr9vPU3qZbi6FkVJSxaNNvlEVMgJHH6w==","X-Received":"by 10.84.217.138 with SMTP id p10mr22634099pli.400.1505359104276;\n\tWed, 13 Sep 2017 20:18:24 -0700 (PDT)","Subject":"Re: [PATCH kernel] powerpc/powernv: Update comment about shifting\n\tIOV BAR","To":"Benjamin Herrenschmidt ","References":"<20170831033412.1971-1-aik@ozlabs.ru>\n\t<0d7068aa-7331-0ce0-145d-8afe1e83a8ec@ozlabs.ru>\n\t<1505358454.12628.185.camel@kernel.crashing.org>","From":"Alexey Kardashevskiy ","Message-ID":"<18f3462e-09ee-ce02-f9c6-eafad68cb9c6@ozlabs.ru>","Date":"Thu, 14 Sep 2017 13:18:17 +1000","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.2.1","MIME-Version":"1.0","In-Reply-To":"<1505358454.12628.185.camel@kernel.crashing.org>","Content-Type":"text/plain; charset=utf-8","Content-Language":"en-AU","Content-Transfer-Encoding":"7bit","X-BeenThere":"linuxppc-dev@lists.ozlabs.org","X-Mailman-Version":"2.1.24","Precedence":"list","List-Id":"Linux on PowerPC Developers Mail List\n\t","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Alistair Popple ,\n\tBjorn Helgaas , \n\tshan.gavin@gmail.com, Paul Mackerras ,\n\tlinuxppc-dev@lists.ozlabs.org, David Gibson ","Errors-To":"linuxppc-dev-bounces+patchwork-incoming=ozlabs.org@lists.ozlabs.org","Sender":"\"Linuxppc-dev\"\n\t"}},{"id":1768323,"web_url":"http://patchwork.ozlabs.org/comment/1768323/","msgid":"<1505360385.12628.187.camel@kernel.crashing.org>","date":"2017-09-14T03:39:45","subject":"Re: [PATCH kernel] powerpc/powernv: Update comment about shifting\n\tIOV BAR","submitter":{"id":38,"url":"http://patchwork.ozlabs.org/api/people/38/","name":"Benjamin Herrenschmidt","email":"benh@kernel.crashing.org"},"content":"On Thu, 2017-09-14 at 13:18 +1000, Alexey Kardashevskiy wrote:\n> On 14/09/17 13:07, Benjamin Herrenschmidt wrote:\n> > On Thu, 2017-09-14 at 12:45 +1000, Alexey Kardashevskiy wrote:\n> > > On 31/08/17 13:34, Alexey Kardashevskiy wrote:\n> > > > From: Benjamin Herrenschmidt \n> > > \n> > > Oops, this was not right :)\n> > > \n> > > Anyway, Ben, please comment. Thanks.\n> > \n> > This is incorrect, we can do hotplug behind switches afaik.\n> \n> Do we have an actual system which allows this? \n\nTuleta no ?\n\n> Anyway, what we do now is\n> wrong and it needs what? Reserve that hole? I'd like to update the comment\n> for now, at least, and state what bad thing can happen and what we expect.\n\nThe hole should be reserved unless another SR-IOV device can use it ...\n\n> \n> > > \n> > > > \n> > > > From: Alexey Kardashevskiy \n> > > > \n> > > > This updates the comment about creating a hole in /proc/iomem which\n> > > > should not be normally happening but it does in the powernv platform\n> > > > due the way MMIO M64 BARs are organised in the IODA2-capable hardware.\n> > > > \n> > > > Signed-off-by: Alexey Kardashevskiy \n> > > > ---\n> > > > \n> > > > It has been mentioned multiple times (last one -\n> > > > https://www.spinics.net/lists/linux-pci/msg64084.html ) that the comment\n> > > > is not informative enough for people not particularly familiar with\n> > > > the POWER8 IO hardware.\n> > > > \n> > > > This attempt aims to:\n> > > > 1. explain why we shift the resource\n> > > > 2. explain why nothing can use that hole as a resource while it is \"free\"\n> > > > (I am not sure that this is the case actually)\n> > > > \n> > > > Please comment, everyone, let's have this very well documented while\n> > > > I remember these bits :) Thanks.\n> > > > ---\n> > > > arch/powerpc/platforms/powernv/pci-ioda.c | 10 +++++++---\n> > > > 1 file changed, 7 insertions(+), 3 deletions(-)\n> > > > \n> > > > diff --git a/arch/powerpc/platforms/powernv/pci-ioda.c b/arch/powerpc/platforms/powernv/pci-ioda.c\n> > > > index 48de308224d6..c4a36ae78c95 100644\n> > > > --- a/arch/powerpc/platforms/powernv/pci-ioda.c\n> > > > +++ b/arch/powerpc/platforms/powernv/pci-ioda.c\n> > > > @@ -1002,9 +1002,13 @@ static int pnv_pci_vf_resource_shift(struct pci_dev *dev, int offset)\n> > > > \t}\n> > > > \n> > > > \t/*\n> > > > -\t * After doing so, there would be a \"hole\" in the /proc/iomem when\n> > > > -\t * offset is a positive value. It looks like the device return some\n> > > > -\t * mmio back to the system, which actually no one could use it.\n> > > > +\t * Since M64 BAR shares segments among all possible 256 PEs,\n> > > > +\t * we have to shift the beginning of PF IOV BAR to make it start from\n> > > > +\t * the segment which belongs to the PE number assigned to the first VF.\n> > > > +\t * This creates a \"hole\" in the /proc/iomem which could be used for\n> > > > +\t * allocating other resources, however this is not expected to happen\n> > > > +\t * on IODA as the only possibility would be a PCI hotplug and IODA\n> > > > +\t * hardware only allows it on a slot with dedicated PHB.\n> > > > \t */\n> > > > \tfor (i = 0; i < PCI_SRIOV_NUM_BARS; i++) {\n> > > > \t\tres = &dev->resource[i + PCI_IOV_RESOURCES];\n> > > > \n> > > \n> > > \n> \n>","headers":{"Return-Path":"","X-Original-To":["patchwork-incoming@ozlabs.org","linuxppc-dev@lists.ozlabs.org"],"Delivered-To":["patchwork-incoming@ozlabs.org","linuxppc-dev@lists.ozlabs.org"],"Received":["from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3])\n\t(using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xt47x1lt5z9t3f\n\tfor ;\n\tThu, 14 Sep 2017 13:41:17 +1000 (AEST)","from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3])\n\tby lists.ozlabs.org (Postfix) with ESMTP id 3xt47x0rKXzDrWB\n\tfor ;\n\tThu, 14 Sep 2017 13:41:17 +1000 (AEST)","from gate.crashing.org (gate.crashing.org [63.228.1.57])\n\t(using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby lists.ozlabs.org (Postfix) with ESMTPS id 3xt46Z1Sm2zDqwr\n\tfor ;\n\tThu, 14 Sep 2017 13:40:05 +1000 (AEST)","from localhost (localhost.localdomain [127.0.0.1])\n\tby gate.crashing.org (8.14.1/8.13.8) with ESMTP id v8E3dj6l005103;\n\tWed, 13 Sep 2017 22:39:46 -0500"],"Authentication-Results":"ozlabs.org; spf=permerror (mailfrom)\n\tsmtp.mailfrom=kernel.crashing.org (client-ip=63.228.1.57;\n\thelo=gate.crashing.org; envelope-from=benh@kernel.crashing.org;\n\treceiver=)","Message-ID":"<1505360385.12628.187.camel@kernel.crashing.org>","Subject":"Re: [PATCH kernel] powerpc/powernv: Update comment about shifting\n\tIOV BAR","From":"Benjamin Herrenschmidt ","To":"Alexey Kardashevskiy ","Date":"Thu, 14 Sep 2017 13:39:45 +1000","In-Reply-To":"<18f3462e-09ee-ce02-f9c6-eafad68cb9c6@ozlabs.ru>","References":"<20170831033412.1971-1-aik@ozlabs.ru>\n\t<0d7068aa-7331-0ce0-145d-8afe1e83a8ec@ozlabs.ru>\n\t<1505358454.12628.185.camel@kernel.crashing.org>\n\t<18f3462e-09ee-ce02-f9c6-eafad68cb9c6@ozlabs.ru>","Content-Type":"text/plain; charset=\"UTF-8\"","X-Mailer":"Evolution 3.24.5 (3.24.5-1.fc26) ","Mime-Version":"1.0","Content-Transfer-Encoding":"7bit","X-BeenThere":"linuxppc-dev@lists.ozlabs.org","X-Mailman-Version":"2.1.24","Precedence":"list","List-Id":"Linux on PowerPC Developers Mail List\n\t","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Alistair Popple ,\n\tBjorn Helgaas , \n\tshan.gavin@gmail.com, Paul Mackerras ,\n\tlinuxppc-dev@lists.ozlabs.org, David Gibson ","Errors-To":"linuxppc-dev-bounces+patchwork-incoming=ozlabs.org@lists.ozlabs.org","Sender":"\"Linuxppc-dev\"\n\t"}},{"id":1768451,"web_url":"http://patchwork.ozlabs.org/comment/1768451/","msgid":"<063D6719AE5E284EB5DD2968C1650D6DD0076B19@AcuExch.aculab.com>","date":"2017-09-14T09:27:52","subject":"RE: [PATCH kernel] powerpc/powernv: Update comment about shifting\n\tIOV BAR","submitter":{"id":6689,"url":"http://patchwork.ozlabs.org/api/people/6689/","name":"David Laight","email":"David.Laight@ACULAB.COM"},"content":"From: Benjamin Herrenschmidt\r\n> Sent: 14 September 2017 04:40\r\n> On Thu, 2017-09-14 at 13:18 +1000, Alexey Kardashevskiy wrote:\r\n> > On 14/09/17 13:07, Benjamin Herrenschmidt wrote:\r\n> > > On Thu, 2017-09-14 at 12:45 +1000, Alexey Kardashevskiy wrote:\r\n> > > > On 31/08/17 13:34, Alexey Kardashevskiy wrote:\r\n> > > > > From: Benjamin Herrenschmidt \r\n> > > >\r\n> > > > Oops, this was not right :)\r\n> > > >\r\n> > > > Anyway, Ben, please comment. Thanks.\r\n> > >\r\n> > > This is incorrect, we can do hotplug behind switches afaik.\r\n> >\r\n> > Do we have an actual system which allows this?\r\n> \r\n> Tuleta no ?\r\n\r\nYou can logically 'hotplug' PCI(e) on any system [1].\r\n\r\nThe 'problem' is that whatever enumerates the PCI(e) at system\r\npowerup doesn't normally assign extra resources to bridges to allow\r\nfor devices that aren't present at boot time.\r\nSo you can normally only replace cards with ones that use the same\r\n(or less) resources, or that are not behind any bridges.\r\nThis is problematic if you have a docking station connected via\r\na bridge.\r\n\r\n[1] Apart from some annoying x86 Dell servers we have which generate\r\nan NMI when the PCIe link goes down (when we reprogram the fpga).\r\nThey also fail to boot if a link doesn't come up...\r\n\r\n\tDavid","headers":{"Return-Path":"","X-Original-To":["patchwork-incoming@ozlabs.org","linuxppc-dev@lists.ozlabs.org"],"Delivered-To":["patchwork-incoming@ozlabs.org","linuxppc-dev@lists.ozlabs.org"],"Received":["from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3])\n\t(using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xtCsZ4hClz9sRW\n\tfor ;\n\tThu, 14 Sep 2017 19:29:22 +1000 (AEST)","from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3])\n\tby lists.ozlabs.org (Postfix) with ESMTP id 3xtCsZ26QGzDrWF\n\tfor ;\n\tThu, 14 Sep 2017 19:29:22 +1000 (AEST)","from smtp-out4.electric.net (smtp-out4.electric.net\n\t[192.162.216.185])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby lists.ozlabs.org (Postfix) with ESMTPS id 3xtCr50yWkzDqr8\n\tfor ;\n\tThu, 14 Sep 2017 19:28:03 +1000 (AEST)","from 1dsQR7-0001xg-TO by out4a.electric.net with emc1-ok (Exim\n\t4.87) (envelope-from )\n\tid 1dsQRA-00029p-TN; Thu, 14 Sep 2017 02:27:56 -0700","by emcmailer; Thu, 14 Sep 2017 02:27:56 -0700","from [156.67.243.126] (helo=AcuExch.aculab.com)\n\tby out4a.electric.net with esmtps (TLSv1:AES128-SHA:128) (Exim 4.87)\n\t(envelope-from )\n\tid 1dsQR7-0001xg-TO; Thu, 14 Sep 2017 02:27:53 -0700","from ACUEXCH.Aculab.com ([::1]) by AcuExch.aculab.com ([::1]) with\n\tmapi id 14.03.0123.003; Thu, 14 Sep 2017 10:27:52 +0100"],"Authentication-Results":"ozlabs.org;\n\tspf=softfail (mailfrom) smtp.mailfrom=aculab.com\n\t(client-ip=192.162.216.185; helo=smtp-out4.electric.net;\n\tenvelope-from=david.laight@aculab.com; receiver=)","From":"David Laight ","To":"'Benjamin Herrenschmidt' , Alexey Kardashevskiy\n\t","Subject":"RE: [PATCH kernel] powerpc/powernv: Update comment about shifting\n\tIOV BAR","Thread-Topic":"[PATCH kernel] powerpc/powernv: Update comment about shifting\n\tIOV BAR","Thread-Index":"AQHTLQtO90+0q2RzREez+loEaOQ1EqK0G5vw","Date":"Thu, 14 Sep 2017 09:27:52 +0000","Message-ID":"<063D6719AE5E284EB5DD2968C1650D6DD0076B19@AcuExch.aculab.com>","References":"<20170831033412.1971-1-aik@ozlabs.ru>\n\t<0d7068aa-7331-0ce0-145d-8afe1e83a8ec@ozlabs.ru>\n\t<1505358454.12628.185.camel@kernel.crashing.org>\n\t<18f3462e-09ee-ce02-f9c6-eafad68cb9c6@ozlabs.ru>\n\t<1505360385.12628.187.camel@kernel.crashing.org>","In-Reply-To":"<1505360385.12628.187.camel@kernel.crashing.org>","Accept-Language":"en-GB, en-US","Content-Language":"en-US","X-MS-Has-Attach":"","X-MS-TNEF-Correlator":"","x-originating-ip":"[10.202.99.200]","Content-Type":"text/plain; charset=\"utf-8\"","Content-Transfer-Encoding":"base64","MIME-Version":"1.0","X-Outbound-IP":"156.67.243.126","X-Env-From":"David.Laight@ACULAB.COM","X-Proto":"esmtps","X-Revdns":"","X-HELO":"AcuExch.aculab.com","X-TLS":"TLSv1:AES128-SHA:128","X-Authenticated_ID":"","X-PolicySMART":"3396946, 3397078","X-Virus-Status":["Scanned by VirusSMART (c)","Scanned by VirusSMART (s)"],"X-BeenThere":"linuxppc-dev@lists.ozlabs.org","X-Mailman-Version":"2.1.24","Precedence":"list","List-Id":"Linux on PowerPC Developers Mail List\n\t","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Alistair Popple , Paul\n\tMackerras ,\n\t\"shan.gavin@gmail.com\" , \n\tBjorn Helgaas ,\n\t\"linuxppc-dev@lists.ozlabs.org\" ,\n\tDavid Gibson ","Errors-To":"linuxppc-dev-bounces+patchwork-incoming=ozlabs.org@lists.ozlabs.org","Sender":"\"Linuxppc-dev\"\n\t"}},{"id":1768490,"web_url":"http://patchwork.ozlabs.org/comment/1768490/","msgid":"<1505383610.12628.193.camel@kernel.crashing.org>","date":"2017-09-14T10:06:50","subject":"Re: [PATCH kernel] powerpc/powernv: Update comment about shifting\n\tIOV BAR","submitter":{"id":38,"url":"http://patchwork.ozlabs.org/api/people/38/","name":"Benjamin Herrenschmidt","email":"benh@kernel.crashing.org"},"content":"On Thu, 2017-09-14 at 09:27 +0000, David Laight wrote:\n> You can logically 'hotplug' PCI(e) on any system [1].\n> \n> The 'problem' is that whatever enumerates the PCI(e) at system\n> powerup doesn't normally assign extra resources to bridges to allow\n> for devices that aren't present at boot time.\n> So you can normally only replace cards with ones that use the same\n> (or less) resources, or that are not behind any bridges.\n> This is problematic if you have a docking station connected via\n> a bridge.\n\nThere's also the problem of Max Payload Size. If you can hotplug behind\na bridge then the standard algorithm of finding the max of all devices\nbehind a host bridge doesn't work anymore and you have to clamp\neverybody to 128 bytes.\n\n> [1] Apart from some annoying x86 Dell servers we have which generate\n> an NMI when the PCIe link goes down (when we reprogram the fpga).\n> They also fail to boot if a link doesn't come up...\n> \n> David","headers":{"Return-Path":"","X-Original-To":["patchwork-incoming@ozlabs.org","linuxppc-dev@lists.ozlabs.org"],"Delivered-To":["patchwork-incoming@ozlabs.org","linuxppc-dev@lists.ozlabs.org"],"Received":["from lists.ozlabs.org (lists.ozlabs.org [103.22.144.68])\n\t(using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xtDkb3hSCz9sRg\n\tfor ;\n\tThu, 14 Sep 2017 20:08:23 +1000 (AEST)","from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3])\n\tby lists.ozlabs.org (Postfix) with ESMTP id 3xtDkb2glCzDrVk\n\tfor ;\n\tThu, 14 Sep 2017 20:08:23 +1000 (AEST)","from gate.crashing.org (gate.crashing.org [63.228.1.57])\n\t(using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby lists.ozlabs.org (Postfix) with ESMTPS id 3xtDjK0kKDzDr13\n\tfor ;\n\tThu, 14 Sep 2017 20:07:16 +1000 (AEST)","from localhost (localhost.localdomain [127.0.0.1])\n\tby gate.crashing.org (8.14.1/8.13.8) with ESMTP id v8EA6oxE030316;\n\tThu, 14 Sep 2017 05:06:52 -0500"],"Authentication-Results":"ozlabs.org; spf=permerror (mailfrom)\n\tsmtp.mailfrom=kernel.crashing.org (client-ip=63.228.1.57;\n\thelo=gate.crashing.org; envelope-from=benh@kernel.crashing.org;\n\treceiver=)","Message-ID":"<1505383610.12628.193.camel@kernel.crashing.org>","Subject":"Re: [PATCH kernel] powerpc/powernv: Update comment about shifting\n\tIOV BAR","From":"Benjamin Herrenschmidt ","To":"David Laight , Alexey Kardashevskiy\n\t","Date":"Thu, 14 Sep 2017 20:06:50 +1000","In-Reply-To":"<063D6719AE5E284EB5DD2968C1650D6DD0076B19@AcuExch.aculab.com>","References":"<20170831033412.1971-1-aik@ozlabs.ru>\n\t<0d7068aa-7331-0ce0-145d-8afe1e83a8ec@ozlabs.ru>\n\t<1505358454.12628.185.camel@kernel.crashing.org>\n\t<18f3462e-09ee-ce02-f9c6-eafad68cb9c6@ozlabs.ru>\n\t<1505360385.12628.187.camel@kernel.crashing.org>\n\t<063D6719AE5E284EB5DD2968C1650D6DD0076B19@AcuExch.aculab.com>","Content-Type":"text/plain; charset=\"UTF-8\"","X-Mailer":"Evolution 3.24.5 (3.24.5-1.fc26) ","Mime-Version":"1.0","Content-Transfer-Encoding":"7bit","X-BeenThere":"linuxppc-dev@lists.ozlabs.org","X-Mailman-Version":"2.1.24","Precedence":"list","List-Id":"Linux on PowerPC Developers Mail List\n\t","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Alistair Popple ,\n\tPaul Mackerras , \n\t\"shan.gavin@gmail.com\" ,\n\tBjorn Helgaas ,\n\t\"linuxppc-dev@lists.ozlabs.org\" ,\n\tDavid Gibson ","Errors-To":"linuxppc-dev-bounces+patchwork-incoming=ozlabs.org@lists.ozlabs.org","Sender":"\"Linuxppc-dev\"\n\t"}}]