From patchwork Wed Feb 15 05:49:04 2012 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qiang Liu X-Patchwork-Id: 141239 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from ozlabs.org (localhost [IPv6:::1]) by ozlabs.org (Postfix) with ESMTP id C73FA1007D6 for ; Wed, 15 Feb 2012 16:53:47 +1100 (EST) Received: from DB3EHSOBE005.bigfish.com (db3ehsobe002.messaging.microsoft.com [213.199.154.140]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (Client CN "mail.global.frontbridge.com", Issuer "Microsoft Secure Server Authority" (not verified)) by ozlabs.org (Postfix) with ESMTPS id 785DC1007D5 for ; Wed, 15 Feb 2012 16:52:55 +1100 (EST) Received: from mail48-db3-R.bigfish.com (10.3.81.232) by DB3EHSOBE005.bigfish.com (10.3.84.25) with Microsoft SMTP Server id 14.1.225.23; Wed, 15 Feb 2012 05:52:49 +0000 Received: from mail48-db3 (localhost [127.0.0.1]) by mail48-db3-R.bigfish.com (Postfix) with ESMTP id A7F46600A9; Wed, 15 Feb 2012 05:52:48 +0000 (UTC) X-SpamScore: 0 X-BigFish: VS0(zz13e6Kc8kzz1202hzz8275bh84d07hz2dh2a8h668h839h) X-Forefront-Antispam-Report: CIP:70.37.183.190; KIP:(null); UIP:(null); IPV:NLI; H:mail.freescale.net; RD:none; EFVD:NLI Received: from mail48-db3 (localhost.localdomain [127.0.0.1]) by mail48-db3 (MessageSwitch) id 1329285166595561_32649; Wed, 15 Feb 2012 05:52:46 +0000 (UTC) Received: from DB3EHSMHS007.bigfish.com (unknown [10.3.81.248]) by mail48-db3.bigfish.com (Postfix) with ESMTP id 8281580045; Wed, 15 Feb 2012 05:52:46 +0000 (UTC) Received: from mail.freescale.net (70.37.183.190) by DB3EHSMHS007.bigfish.com (10.3.87.107) with Microsoft SMTP Server (TLS) id 14.1.225.23; Wed, 15 Feb 2012 05:52:46 +0000 Received: from az33smr02.freescale.net (10.64.34.200) by 039-SN1MMR1-002.039d.mgd.msft.net (10.84.1.15) with Microsoft SMTP Server id 14.1.355.3; Tue, 14 Feb 2012 23:52:44 -0600 Received: from localhost (rock.ap.freescale.net [10.193.20.106]) by az33smr02.freescale.net (8.13.1/8.13.0) with ESMTP id q1F5qfnE021827; Tue, 14 Feb 2012 23:52:42 -0600 (CST) From: Qiang Liu To: , Subject: [PATCH V2 RESEND] fsl-sata: I/O load balancing Date: Wed, 15 Feb 2012 13:49:04 +0800 Message-ID: <1329284944-17943-1-git-send-email-qiang.liu@freescale.com> X-Mailer: git-send-email 1.7.5.1 MIME-Version: 1.0 X-OriginatorOrg: freescale.com Cc: Qiang Liu , linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: linuxppc-dev-bounces+patchwork-incoming=ozlabs.org@lists.ozlabs.org Sender: linuxppc-dev-bounces+patchwork-incoming=ozlabs.org@lists.ozlabs.org Reduce interrupt signals through reset Interrupt Coalescing Control Reg. Provide dynamic method to adjust interrupt signals and timer ticks by sysfs. It is a tradeoff for different applications. Signed-off-by: Qiang Liu --- change for V2 support dynamic config interrupt coalescing register by /sysfs test random small file with iometer adjust kernel source baseline for apply upstream Description: 1. fsl-sata interrupt will be raised 130 thousand times when write 8G file (dd if=/dev/zero of=/dev/sda2 bs=128K count=65536); 2. most of interrupts raised because of only 1-4 commands completed; 3. only 30 thousand times will be raised after set max interrupt threshold, more interrupts are coalesced as the description of ICC; Test methods and results: 1. test sequential large file performance, [root@p2020ds root]# echo 31 524287 > \ /sys/devices/soc.0/ffe18000.sata/intr_coalescing [root@p2020ds root]# dd if=/dev/zero of=/dev/sda2 bs=128K count=65536 & [root@p2020ds root]# top CPU % | dd | flush-8:0 | softirq --------------------------------------- before | 20-22 | 17-19 | 7 --------------------------------------- after | 18-21 | 15-16 | 5 --------------------------------------- 2. test random small file with iometer, iometer paramters: 4 I/Os burst length, 1MB transfer request size, 100% write, 2MB file size as default configuration of interrupt coalescing register, 1 interrupts and no timeout config, total write performance is 119MB per second, after config with the maximum value, write performance is 110MB per second. After compare the test results, a configuable interrupt coalescing should be better when cope with flexible context. drivers/ata/sata_fsl.c | 111 ++++++++++++++++++++++++++++++++++++++++++++++-- 1 files changed, 107 insertions(+), 4 deletions(-) -- 1.6.4 diff --git a/drivers/ata/sata_fsl.c b/drivers/ata/sata_fsl.c index 0120b0d..d6577b9 100644 --- a/drivers/ata/sata_fsl.c +++ b/drivers/ata/sata_fsl.c @@ -6,7 +6,7 @@ * Author: Ashish Kalra * Li Yang * - * Copyright (c) 2006-2007, 2011 Freescale Semiconductor, Inc. + * Copyright (c) 2006-2007, 2011-2012 Freescale Semiconductor, Inc. * * This program is free software; you can redistribute it and/or modify it * under the terms of the GNU General Public License as published by the @@ -26,6 +26,15 @@ #include #include +static unsigned int intr_coalescing_count; +module_param(intr_coalescing_count, int, S_IRUGO); +MODULE_PARM_DESC(intr_coalescing_count, + "INT coalescing count threshold (1..31)"); + +static unsigned int intr_coalescing_ticks; +module_param(intr_coalescing_ticks, int, S_IRUGO); +MODULE_PARM_DESC(intr_coalescing_ticks, + "INT coalescing timer threshold in AHB ticks"); /* Controller information */ enum { SATA_FSL_QUEUE_DEPTH = 16, @@ -83,6 +92,16 @@ enum { }; /* + * Interrupt Coalescing Control Register bitdefs */ +enum { + ICC_MIN_INT_COUNT_THRESHOLD = 1, + ICC_MAX_INT_COUNT_THRESHOLD = ((1 << 5) - 1), + ICC_MIN_INT_TICKS_THRESHOLD = 0, + ICC_MAX_INT_TICKS_THRESHOLD = ((1 << 19) - 1), + ICC_SAFE_INT_TICKS = 1, +}; + +/* * Host Controller command register set - per port */ enum { @@ -263,8 +282,65 @@ struct sata_fsl_host_priv { void __iomem *csr_base; int irq; int data_snoop; + struct device_attribute intr_coalescing; }; +static void fsl_sata_set_irq_coalescing(struct ata_host *host, + unsigned int count, unsigned int ticks) +{ + struct sata_fsl_host_priv *host_priv = host->private_data; + void __iomem *hcr_base = host_priv->hcr_base; + + if (count > ICC_MAX_INT_COUNT_THRESHOLD) + count = ICC_MAX_INT_COUNT_THRESHOLD; + else if (count < ICC_MIN_INT_COUNT_THRESHOLD) + count = ICC_MIN_INT_COUNT_THRESHOLD; + + if (ticks > ICC_MAX_INT_TICKS_THRESHOLD) + ticks = ICC_MAX_INT_TICKS_THRESHOLD; + else if ((ICC_MIN_INT_TICKS_THRESHOLD == ticks) && + (count > ICC_MIN_INT_COUNT_THRESHOLD)) + ticks = ICC_SAFE_INT_TICKS; + + spin_lock(&host->lock); + iowrite32((count << 24 | ticks), hcr_base + ICC); + + intr_coalescing_count = count; + intr_coalescing_ticks = ticks; + spin_unlock(&host->lock); + + DPRINTK("intrrupt coalescing, count = 0x%x, ticks = %x\n", + intr_coalescing_count, intr_coalescing_ticks); + DPRINTK("ICC register status: (hcr base: 0x%x) = 0x%x\n", + hcr_base, ioread32(hcr_base + ICC)); +} + +static ssize_t fsl_sata_intr_coalescing_show(struct device *dev, + struct device_attribute *attr, char *buf) +{ + return sprintf(buf, "%d %d\n", + intr_coalescing_count, intr_coalescing_ticks); +} + +static ssize_t fsl_sata_intr_coalescing_store(struct device *dev, + struct device_attribute *attr, + const char *buf, size_t count) +{ + unsigned int coalescing_count, coalescing_ticks; + + if (sscanf(buf, "%d%d", + &coalescing_count, + &coalescing_ticks) != 2) { + printk(KERN_ERR "fsl-sata: wrong parameter format.\n"); + return -EINVAL; + } + + fsl_sata_set_irq_coalescing(dev_get_drvdata(dev), + coalescing_count, coalescing_ticks); + + return strlen(buf); +} + static inline unsigned int sata_fsl_tag(unsigned int tag, void __iomem *hcr_base) { @@ -346,10 +422,10 @@ static unsigned int sata_fsl_fill_sg(struct ata_queued_cmd *qc, void *cmd_desc, (unsigned long long)sg_addr, sg_len); /* warn if each s/g element is not dword aligned */ - if (sg_addr & 0x03) + if (unlikely(sg_addr & 0x03)) ata_port_err(qc->ap, "s/g addr unaligned : 0x%llx\n", (unsigned long long)sg_addr); - if (sg_len & 0x03) + if (unlikely(sg_len & 0x03)) ata_port_err(qc->ap, "s/g len unaligned : 0x%x\n", sg_len); @@ -1245,6 +1321,13 @@ static int sata_fsl_init_controller(struct ata_host *host) iowrite32(0x00000FFFF, hcr_base + CE); iowrite32(0x00000FFFF, hcr_base + DE); + /* + * reset the number of command complete bits which will cause the + * interrupt to be signaled + */ + fsl_sata_set_irq_coalescing(host, intr_coalescing_count, + intr_coalescing_ticks); + /* * host controller will be brought on-line, during xx_port_start() * callback, that should also initiate the OOB, COMINIT sequence @@ -1309,7 +1392,7 @@ static int sata_fsl_probe(struct platform_device *ofdev) void __iomem *csr_base = NULL; struct sata_fsl_host_priv *host_priv = NULL; int irq; - struct ata_host *host; + struct ata_host *host = NULL; u32 temp; struct ata_port_info pi = sata_fsl_port_info[0]; @@ -1356,6 +1439,10 @@ static int sata_fsl_probe(struct platform_device *ofdev) /* allocate host structure */ host = ata_host_alloc_pinfo(&ofdev->dev, ppi, SATA_FSL_MAX_PORTS); + if (!host) { + retval = -ENOMEM; + goto error_exit_with_cleanup; + } /* host->iomap is not used currently */ host->private_data = host_priv; @@ -1373,10 +1460,24 @@ static int sata_fsl_probe(struct platform_device *ofdev) dev_set_drvdata(&ofdev->dev, host); + host_priv->intr_coalescing.show = fsl_sata_intr_coalescing_show; + host_priv->intr_coalescing.store = fsl_sata_intr_coalescing_store; + sysfs_attr_init(&host_priv->intr_coalescing.attr); + host_priv->intr_coalescing.attr.name = "intr_coalescing"; + host_priv->intr_coalescing.attr.mode = S_IRUGO | S_IWUSR; + retval = device_create_file(host->dev, &host_priv->intr_coalescing); + if (retval) + goto error_exit_with_cleanup; + return 0; error_exit_with_cleanup: + if (host) { + dev_set_drvdata(&ofdev->dev, NULL); + ata_host_detach(host); + } + if (hcr_base) iounmap(hcr_base); if (host_priv) @@ -1390,6 +1491,8 @@ static int sata_fsl_remove(struct platform_device *ofdev) struct ata_host *host = dev_get_drvdata(&ofdev->dev); struct sata_fsl_host_priv *host_priv = host->private_data; + device_remove_file(&ofdev->dev, &host_priv->intr_coalescing); + ata_host_detach(host); dev_set_drvdata(&ofdev->dev, NULL);