diff mbox

2.6.34-rc3 Disable R_OK (Early ACK) on SII 3726/4726 PMP

Message ID m2gda824cf31004131746o15be77b1n7af6fac42ce25280@mail.gmail.com
State Not Applicable
Delegated to: David Miller
Headers show

Commit Message

Grant Grundler April 14, 2010, 12:46 a.m. UTC
In 2009, While running "cache read" performance test of drives behind
SII PMP we encountered a "all 5 drives" timeout on more than 30% of the
machines under test.  This patch reduces the rate by a factor of about 70.
Low enough that we didn't care to further investigate the issue.

Performance impact with any sort of "normal" use was ~2%+ CPU and less
than 1% throughput degradation.  Worst case impact (cached read) was
6% IOPS reduction. This is with NCQ off (q=1) but I believe FIS based
switching enabled in the SATA driver.

The patch disables "Early ACK" in the 3726/4726 port multiplier.
"Early ACK" is issued when device sends a FIS to the host (via PMP)
and the PMP sends an ACK immediately back to the device - well before
the host gets the response. Under worst case IOPs load (cached read
test) and more than 2 PMPs connected to a 4-port SATA controller,
I suspect the time to service all of the PMPs is exceeding the PMPs
ability to keep track of outstanding FIS it owes the Host. Reducing
the number of PMPs to 2 (or 1) reduces the frequency by several orders
of magnitude. Kudos to Gwendal for initial debugging of this issue.
[Any errors in the description are mine, not his.]

Patch is currently in production on Google servers.

Signed-off-by: Grant Grundler <grundler@google.com>
Signed-off-by: Gwendal Grignou <gwendal@google.com>

---
And my apologies in advance for using gmail. Patch appended below
is also attached since I expect gmail will mangle the inlined content.

Comments

Tejun Heo April 15, 2010, 12:31 a.m. UTC | #1
Hello,

On 04/14/2010 09:46 AM, Grant Grundler wrote:
> In 2009, While running "cache read" performance test of drives behind
> SII PMP we encountered a "all 5 drives" timeout on more than 30% of the
> machines under test.  This patch reduces the rate by a factor of about 70.
> Low enough that we didn't care to further investigate the issue.

Interesting.

> Performance impact with any sort of "normal" use was ~2%+ CPU and less
> than 1% throughput degradation.  Worst case impact (cached read) was
> 6% IOPS reduction. This is with NCQ off (q=1) but I believe FIS based
> switching enabled in the SATA driver.

Given the general flakiness of the device, I'm more than willing to
pay some overhead for stability and the performance hit seems pretty
small.

> @@ -260,12 +262,28 @@ static int sata_pmp_configure(struct ata_device
> *dev, int print_info)
>                 goto fail;
>         }
> 
> +       /* For Sil3726, disable sending Early R_OK */
> +       if (vendor == 0x1095 && devid == 0x3726) {

Isn't this also applicable to 4723 and 4726?

> diff --git a/include/linux/ata.h b/include/linux/ata.h
> index 700c5b9..18fc208 100644
> --- a/include/linux/ata.h
> +++ b/include/linux/ata.h
> @@ -386,6 +386,7 @@ enum {
>         SATA_PMP_GSCR_ERROR_EN  = 33,
>         SATA_PMP_GSCR_FEAT      = 64,
>         SATA_PMP_GSCR_FEAT_EN   = 96,
> +       SATA_PMP_GSCR_SII_POL   = 129,  /* VENDOR : Silicon Image 3726/4726 */

Can you plesae define it inside libata-pmp.c?  ata.h only contains
stuff defined by the ATA standard.

Other than the above nitpicks,

Acked-by: Tejun Heo <tj@kernel.org>

Thanks.
Grant Grundler April 15, 2010, 12:57 a.m. UTC | #2
On Wed, Apr 14, 2010 at 5:31 PM, Tejun Heo <tj@kernel.org> wrote:
> Hello,
>
> On 04/14/2010 09:46 AM, Grant Grundler wrote:
>> In 2009, While running "cache read" performance test of drives behind
>> SII PMP we encountered a "all 5 drives" timeout on more than 30% of the
>> machines under test.  This patch reduces the rate by a factor of about 70.
>> Low enough that we didn't care to further investigate the issue.
>
> Interesting.
>
>> Performance impact with any sort of "normal" use was ~2%+ CPU and less
>> than 1% throughput degradation.  Worst case impact (cached read) was
>> 6% IOPS reduction. This is with NCQ off (q=1) but I believe FIS based
>> switching enabled in the SATA driver.
>
> Given the general flakiness of the device, I'm more than willing to
> pay some overhead for stability and the performance hit seems pretty
> small.
>
>> @@ -260,12 +262,28 @@ static int sata_pmp_configure(struct ata_device
>> *dev, int print_info)
>>                 goto fail;
>>         }
>>
>> +       /* For Sil3726, disable sending Early R_OK */
>> +       if (vendor == 0x1095 && devid == 0x3726) {
>
> Isn't this also applicable to 4723 and 4726?

Sorry, I don't know why I included 4726 in the Subject line since I
only tested with 3726.

I expect it would apply to 4726 but someone who knows the internals of
both would have to confirm that.

I don't know anything about 4723.


>
>> diff --git a/include/linux/ata.h b/include/linux/ata.h
>> index 700c5b9..18fc208 100644
>> --- a/include/linux/ata.h
>> +++ b/include/linux/ata.h
>> @@ -386,6 +386,7 @@ enum {
>>         SATA_PMP_GSCR_ERROR_EN  = 33,
>>         SATA_PMP_GSCR_FEAT      = 64,
>>         SATA_PMP_GSCR_FEAT_EN   = 96,
>> +       SATA_PMP_GSCR_SII_POL   = 129,  /* VENDOR : Silicon Image 3726/4726 */
>
> Can you plesae define it inside libata-pmp.c?

Yes. I'll resubmit with "SII_POL" defined "locally".

> ata.h only contains stuff defined by the ATA standard.

Ah ok. I didn't realize that.

>
> Other than the above nitpicks,
>
> Acked-by: Tejun Heo <tj@kernel.org>

Thanks for the Ack/Review!
grant

>
> Thanks.
>
> --
> tejun
>
--
To unsubscribe from this list: send the line "unsubscribe linux-ide" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
diff mbox

Patch

diff --git a/drivers/ata/libata-pmp.c b/drivers/ata/libata-pmp.c
index 00305f4..69bb203 100644
--- a/drivers/ata/libata-pmp.c
+++ b/drivers/ata/libata-pmp.c
@@ -235,6 +235,8 @@  static int sata_pmp_configure(struct ata_device
*dev, int print_info)
 {
        struct ata_port *ap = dev->link->ap;
        u32 *gscr = dev->gscr;
+       u16 vendor = sata_pmp_gscr_vendor(gscr);
+       u16 devid = sata_pmp_gscr_devid(gscr);
        unsigned int err_mask = 0;
        const char *reason;
        int nr_ports, rc;
@@ -260,12 +262,28 @@  static int sata_pmp_configure(struct ata_device
*dev, int print_info)
                goto fail;
        }

+       /* For Sil3726, disable sending Early R_OK */
+       if (vendor == 0x1095 && devid == 0x3726) {
+               u32 reg;
+               err_mask = sata_pmp_read(&ap->link,
SATA_PMP_GSCR_SII_POL, &reg);
+               if (err_mask) {
+                       rc = -EIO;
+                       reason = "failed to read Sil3726 Private Register";
+                       goto fail;
+               }
+               reg &= ~0x1;
+               err_mask = sata_pmp_write(&ap->link,
SATA_PMP_GSCR_SII_POL, reg);
+               if (err_mask) {
+                       rc = -EIO;
+                       reason = "failed to write Sil3726 Private Register";
+                       goto fail;
+               }
+       }
+
        if (print_info) {
                ata_dev_printk(dev, KERN_INFO, "Port Multiplier %s, "
                               "0x%04x:0x%04x r%d, %d ports, feat 0x%x/0x%x\n",
-                              sata_pmp_spec_rev_str(gscr),
-                              sata_pmp_gscr_vendor(gscr),
-                              sata_pmp_gscr_devid(gscr),
+                              sata_pmp_spec_rev_str(gscr), vendor, devid,
                               sata_pmp_gscr_rev(gscr),
                               nr_ports, gscr[SATA_PMP_GSCR_FEAT_EN],
                               gscr[SATA_PMP_GSCR_FEAT]);
diff --git a/include/linux/ata.h b/include/linux/ata.h
index 700c5b9..18fc208 100644
--- a/include/linux/ata.h
+++ b/include/linux/ata.h
@@ -386,6 +386,7 @@  enum {
        SATA_PMP_GSCR_ERROR_EN  = 33,
        SATA_PMP_GSCR_FEAT      = 64,
        SATA_PMP_GSCR_FEAT_EN   = 96,
+       SATA_PMP_GSCR_SII_POL   = 129,  /* VENDOR : Silicon Image 3726/4726 */

        SATA_PMP_PSCR_STATUS    = 0,
        SATA_PMP_PSCR_ERROR     = 1,