From patchwork Fri Aug 4 19:50:26 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: dann frazier X-Patchwork-Id: 797986 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) by ozlabs.org (Postfix) with ESMTP id 3xPHbm1bQ3z9sN7; Sat, 5 Aug 2017 05:51:00 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.76) (envelope-from ) id 1ddicb-0004Na-D5; Fri, 04 Aug 2017 19:50:57 +0000 Received: from complete.lackof.org ([198.49.126.79]) by huckleberry.canonical.com with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.76) (envelope-from ) id 1ddicU-0004Ju-8B for kernel-team@lists.ubuntu.com; Fri, 04 Aug 2017 19:50:50 +0000 Received: from localhost (c-107-2-141-92.hsd1.co.comcast.net [107.2.141.92]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by complete.lackof.org (Postfix) with ESMTPSA id 0153733E00B3 for ; Fri, 4 Aug 2017 13:50:47 -0600 (MDT) From: dann frazier To: kernel-team@lists.ubuntu.com Subject: [PATCH 4/4][Unstable][Artful][SRU Zesty] scsi: hisi_sas: optimise DMA slot memory Date: Fri, 4 Aug 2017 13:50:26 -0600 Message-Id: <20170804195026.32028-5-dann.frazier@canonical.com> X-Mailer: git-send-email 2.13.3 In-Reply-To: <20170804195026.32028-1-dann.frazier@canonical.com> References: <20170804195026.32028-1-dann.frazier@canonical.com> X-Spam-Status: No, score=0.0 required=5.0 tests=UNPARSEABLE_RELAY autolearn=unavailable version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on complete.lackof.org X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.14 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: kernel-team-bounces@lists.ubuntu.com From: Xiaofei Tan BugLink: https://bugs.launchpad.net/bugs/1708734 Currently we allocate 3 sets of DMA memories from separate pools for each slot. This is inefficient in terms of memory usage (buffers are less than 1 page in size, so we lose due to alignment), and also time spent in doing 3 allocations + de-allocations per slot, instead of 1. To optimise, combine the 3 DMA buffers into a single buffer from a single pool. Signed-off-by: Xiaofei Tan Signed-off-by: John Garry Signed-off-by: Martin K. Petersen (backported from commit f557e32c0023ea0d67cdaa81b3398550dc1e4876) Signed-off-by: dann frazier --- drivers/scsi/hisi_sas/hisi_sas.h | 47 ++++++++++++++------- drivers/scsi/hisi_sas/hisi_sas_main.c | 75 +++++++++------------------------- drivers/scsi/hisi_sas/hisi_sas_v1_hw.c | 31 +++++++------- drivers/scsi/hisi_sas/hisi_sas_v2_hw.c | 37 ++++++++--------- 4 files changed, 86 insertions(+), 104 deletions(-) diff --git a/drivers/scsi/hisi_sas/hisi_sas.h b/drivers/scsi/hisi_sas/hisi_sas.h index 1410faf7970d..0503cf7d668e 100644 --- a/drivers/scsi/hisi_sas/hisi_sas.h +++ b/drivers/scsi/hisi_sas/hisi_sas.h @@ -33,10 +33,24 @@ #define HISI_SAS_MAX_DEVICES HISI_SAS_MAX_ITCT_ENTRIES #define HISI_SAS_RESET_BIT 0 -#define HISI_SAS_STATUS_BUF_SZ \ - (sizeof(struct hisi_sas_err_record) + 1024) -#define HISI_SAS_COMMAND_TABLE_SZ \ - (((sizeof(union hisi_sas_command_table)+3)/4)*4) +#define HISI_SAS_STATUS_BUF_SZ (sizeof(struct hisi_sas_status_buffer)) +#define HISI_SAS_COMMAND_TABLE_SZ (sizeof(union hisi_sas_command_table)) + +#define hisi_sas_status_buf_addr(buf) \ + (buf + offsetof(struct hisi_sas_slot_buf_table, status_buffer)) +#define hisi_sas_status_buf_addr_mem(slot) hisi_sas_status_buf_addr(slot->buf) +#define hisi_sas_status_buf_addr_dma(slot) \ + hisi_sas_status_buf_addr(slot->buf_dma) + +#define hisi_sas_cmd_hdr_addr(buf) \ + (buf + offsetof(struct hisi_sas_slot_buf_table, command_header)) +#define hisi_sas_cmd_hdr_addr_mem(slot) hisi_sas_cmd_hdr_addr(slot->buf) +#define hisi_sas_cmd_hdr_addr_dma(slot) hisi_sas_cmd_hdr_addr(slot->buf_dma) + +#define hisi_sas_sge_addr(buf) \ + (buf + offsetof(struct hisi_sas_slot_buf_table, sge_page)) +#define hisi_sas_sge_addr_mem(slot) hisi_sas_sge_addr(slot->buf) +#define hisi_sas_sge_addr_dma(slot) hisi_sas_sge_addr(slot->buf_dma) #define HISI_SAS_MAX_SSP_RESP_SZ (sizeof(struct ssp_frame_hdr) + 1024) #define HISI_SAS_MAX_SMP_RESP_SZ 1028 @@ -132,14 +146,10 @@ struct hisi_sas_slot { int cmplt_queue_slot; int idx; int abort; + void *buf; + dma_addr_t buf_dma; void *cmd_hdr; dma_addr_t cmd_hdr_dma; - void *status_buffer; - dma_addr_t status_buffer_dma; - void *command_table; - dma_addr_t command_table_dma; - struct hisi_sas_sge_page *sge_page; - dma_addr_t sge_page_dma; struct work_struct abort_slot; struct timer_list internal_abort_timer; }; @@ -220,10 +230,8 @@ struct hisi_hba { int queue_count; - struct dma_pool *sge_page_pool; + struct dma_pool *buffer_pool; struct hisi_sas_device devices[HISI_SAS_MAX_DEVICES]; - struct dma_pool *command_table_pool; - struct dma_pool *status_buffer_pool; struct hisi_sas_cmd_hdr *cmd_hdr[HISI_SAS_MAX_QUEUES]; dma_addr_t cmd_hdr_dma[HISI_SAS_MAX_QUEUES]; void *complete_hdr[HISI_SAS_MAX_QUEUES]; @@ -335,7 +343,7 @@ struct hisi_sas_command_table_stp { #define HISI_SAS_SGE_PAGE_CNT SG_CHUNK_SIZE struct hisi_sas_sge_page { struct hisi_sas_sge sge[HISI_SAS_SGE_PAGE_CNT]; -}; +} __aligned(16); struct hisi_sas_command_table_ssp { struct ssp_frame_hdr hdr; @@ -354,6 +362,17 @@ union hisi_sas_command_table { struct hisi_sas_command_table_ssp ssp; struct hisi_sas_command_table_smp smp; struct hisi_sas_command_table_stp stp; +} __aligned(16); + +struct hisi_sas_status_buffer { + struct hisi_sas_err_record err; + u8 iu[1024]; +} __aligned(16); + +struct hisi_sas_slot_buf_table { + struct hisi_sas_status_buffer status_buffer; + union hisi_sas_command_table command_header; + struct hisi_sas_sge_page sge_page; }; extern struct hisi_sas_port *to_hisi_sas_port(struct asd_sas_port *sas_port); diff --git a/drivers/scsi/hisi_sas/hisi_sas_main.c b/drivers/scsi/hisi_sas/hisi_sas_main.c index 7ae3e12c48ba..f9af8c9c2b50 100644 --- a/drivers/scsi/hisi_sas/hisi_sas_main.c +++ b/drivers/scsi/hisi_sas/hisi_sas_main.c @@ -28,8 +28,10 @@ void hisi_sas_sata_done(struct sas_task *task, { struct task_status_struct *ts = &task->task_status; struct ata_task_resp *resp = (struct ata_task_resp *)ts->buf; - struct dev_to_host_fis *d2h = slot->status_buffer + - sizeof(struct hisi_sas_err_record); + struct hisi_sas_status_buffer *status_buf = + hisi_sas_status_buf_addr_mem(slot); + u8 *iu = &status_buf->iu[0]; + struct dev_to_host_fis *d2h = (struct dev_to_host_fis *)iu; resp->frame_len = sizeof(struct dev_to_host_fis); memcpy(&resp->ending_fis[0], d2h, sizeof(struct dev_to_host_fis)); @@ -109,17 +111,9 @@ void hisi_sas_slot_task_free(struct hisi_hba *hisi_hba, struct sas_task *task, atomic64_dec(&sas_dev->running_req); } - if (slot->command_table) - dma_pool_free(hisi_hba->command_table_pool, - slot->command_table, slot->command_table_dma); + if (slot->buf) + dma_pool_free(hisi_hba->buffer_pool, slot->buf, slot->buf_dma); - if (slot->status_buffer) - dma_pool_free(hisi_hba->status_buffer_pool, - slot->status_buffer, slot->status_buffer_dma); - - if (slot->sge_page) - dma_pool_free(hisi_hba->sge_page_pool, slot->sge_page, - slot->sge_page_dma); list_del_init(&slot->entry); slot->task = NULL; @@ -288,24 +282,15 @@ static int hisi_sas_task_prep(struct sas_task *task, struct hisi_sas_dq task->lldd_task = slot; INIT_WORK(&slot->abort_slot, hisi_sas_slot_abort); - slot->status_buffer = dma_pool_alloc(hisi_hba->status_buffer_pool, - GFP_ATOMIC, - &slot->status_buffer_dma); - if (!slot->status_buffer) { + slot->buf = dma_pool_alloc(hisi_hba->buffer_pool, + GFP_ATOMIC, &slot->buf_dma); + if (!slot->buf) { rc = -ENOMEM; goto err_out_slot_buf; } - memset(slot->status_buffer, 0, HISI_SAS_STATUS_BUF_SZ); - - slot->command_table = dma_pool_alloc(hisi_hba->command_table_pool, - GFP_ATOMIC, - &slot->command_table_dma); - if (!slot->command_table) { - rc = -ENOMEM; - goto err_out_status_buf; - } - memset(slot->command_table, 0, HISI_SAS_COMMAND_TABLE_SZ); memset(slot->cmd_hdr, 0, sizeof(struct hisi_sas_cmd_hdr)); + memset(hisi_sas_cmd_hdr_addr_mem(slot), 0, HISI_SAS_COMMAND_TABLE_SZ); + memset(hisi_sas_status_buf_addr_mem(slot), 0, HISI_SAS_STATUS_BUF_SZ); switch (task->task_proto) { case SAS_PROTOCOL_SMP: @@ -328,9 +313,7 @@ static int hisi_sas_task_prep(struct sas_task *task, struct hisi_sas_dq if (rc) { dev_err(dev, "task prep: rc = 0x%x\n", rc); - if (slot->sge_page) - goto err_out_sge; - goto err_out_command_table; + goto err_out_buf; } list_add_tail(&slot->entry, &sas_dev->list); @@ -345,15 +328,9 @@ static int hisi_sas_task_prep(struct sas_task *task, struct hisi_sas_dq return 0; -err_out_sge: - dma_pool_free(hisi_hba->sge_page_pool, slot->sge_page, - slot->sge_page_dma); -err_out_command_table: - dma_pool_free(hisi_hba->command_table_pool, slot->command_table, - slot->command_table_dma); -err_out_status_buf: - dma_pool_free(hisi_hba->status_buffer_pool, slot->status_buffer, - slot->status_buffer_dma); +err_out_buf: + dma_pool_free(hisi_hba->buffer_pool, slot->buf, + slot->buf_dma); err_out_slot_buf: /* Nothing to be done */ err_out_tag: @@ -1508,16 +1485,9 @@ static int hisi_sas_alloc(struct hisi_hba *hisi_hba, struct Scsi_Host *shost) goto err_out; } - s = HISI_SAS_STATUS_BUF_SZ; - hisi_hba->status_buffer_pool = dma_pool_create("status_buffer", - dev, s, 16, 0); - if (!hisi_hba->status_buffer_pool) - goto err_out; - - s = HISI_SAS_COMMAND_TABLE_SZ; - hisi_hba->command_table_pool = dma_pool_create("command_table", - dev, s, 16, 0); - if (!hisi_hba->command_table_pool) + s = sizeof(struct hisi_sas_slot_buf_table); + hisi_hba->buffer_pool = dma_pool_create("dma_buffer", dev, s, 16, 0); + if (!hisi_hba->buffer_pool) goto err_out; s = HISI_SAS_MAX_ITCT_ENTRIES * sizeof(struct hisi_sas_itct); @@ -1552,11 +1522,6 @@ static int hisi_sas_alloc(struct hisi_hba *hisi_hba, struct Scsi_Host *shost) if (!hisi_hba->slot_index_tags) goto err_out; - hisi_hba->sge_page_pool = dma_pool_create("status_sge", dev, - sizeof(struct hisi_sas_sge_page), 16, 0); - if (!hisi_hba->sge_page_pool) - goto err_out; - s = sizeof(struct hisi_sas_initial_fis) * HISI_SAS_MAX_PHYS; hisi_hba->initial_fis = dma_alloc_coherent(dev, s, &hisi_hba->initial_fis_dma, GFP_KERNEL); @@ -1602,9 +1567,7 @@ static void hisi_sas_free(struct hisi_hba *hisi_hba) hisi_hba->complete_hdr_dma[i]); } - dma_pool_destroy(hisi_hba->status_buffer_pool); - dma_pool_destroy(hisi_hba->command_table_pool); - dma_pool_destroy(hisi_hba->sge_page_pool); + dma_pool_destroy(hisi_hba->buffer_pool); s = HISI_SAS_MAX_ITCT_ENTRIES * sizeof(struct hisi_sas_itct); if (hisi_hba->itct) diff --git a/drivers/scsi/hisi_sas/hisi_sas_v1_hw.c b/drivers/scsi/hisi_sas/hisi_sas_v1_hw.c index 7d7d2a7e690d..adcb2287197d 100644 --- a/drivers/scsi/hisi_sas/hisi_sas_v1_hw.c +++ b/drivers/scsi/hisi_sas/hisi_sas_v1_hw.c @@ -939,6 +939,7 @@ static int prep_prd_sge_v1_hw(struct hisi_hba *hisi_hba, struct scatterlist *scatter, int n_elem) { + struct hisi_sas_sge_page *sge_page = hisi_sas_sge_addr_mem(slot); struct device *dev = &hisi_hba->pdev->dev; struct scatterlist *sg; int i; @@ -949,13 +950,8 @@ static int prep_prd_sge_v1_hw(struct hisi_hba *hisi_hba, return -EINVAL; } - slot->sge_page = dma_pool_alloc(hisi_hba->sge_page_pool, GFP_ATOMIC, - &slot->sge_page_dma); - if (!slot->sge_page) - return -ENOMEM; - for_each_sg(scatter, sg, n_elem, i) { - struct hisi_sas_sge *entry = &slot->sge_page->sge[i]; + struct hisi_sas_sge *entry = &sge_page->sge[i]; entry->addr = cpu_to_le64(sg_dma_address(sg)); entry->page_ctrl_0 = entry->page_ctrl_1 = 0; @@ -963,7 +959,7 @@ static int prep_prd_sge_v1_hw(struct hisi_hba *hisi_hba, entry->data_off = 0; } - hdr->prd_table_addr = cpu_to_le64(slot->sge_page_dma); + hdr->prd_table_addr = cpu_to_le64(hisi_sas_sge_addr_dma(slot)); hdr->sg_len = cpu_to_le32(n_elem << CMD_HDR_DATA_SGL_LEN_OFF); @@ -1026,7 +1022,7 @@ static int prep_smp_v1_hw(struct hisi_hba *hisi_hba, hdr->transfer_tags = cpu_to_le32(slot->idx << CMD_HDR_IPTT_OFF); hdr->cmd_table_addr = cpu_to_le64(req_dma_addr); - hdr->sts_buffer_addr = cpu_to_le64(slot->status_buffer_dma); + hdr->sts_buffer_addr = cpu_to_le64(hisi_sas_status_buf_addr_dma(slot)); return 0; @@ -1107,10 +1103,11 @@ static int prep_ssp_v1_hw(struct hisi_hba *hisi_hba, } hdr->data_transfer_len = cpu_to_le32(task->total_xfer_len); - hdr->cmd_table_addr = cpu_to_le64(slot->command_table_dma); - hdr->sts_buffer_addr = cpu_to_le64(slot->status_buffer_dma); + hdr->cmd_table_addr = cpu_to_le64(hisi_sas_cmd_hdr_addr_dma(slot)); + hdr->sts_buffer_addr = cpu_to_le64(hisi_sas_status_buf_addr_dma(slot)); - buf_cmd = slot->command_table + sizeof(struct ssp_frame_hdr); + buf_cmd = hisi_sas_cmd_hdr_addr_mem(slot) + + sizeof(struct ssp_frame_hdr); if (task->ssp_task.enable_first_burst) { fburst = (1 << 7); dw2 |= 1 << CMD_HDR_FIRST_BURST_OFF; @@ -1147,7 +1144,8 @@ static void slot_err_v1_hw(struct hisi_hba *hisi_hba, struct hisi_sas_slot *slot) { struct task_status_struct *ts = &task->task_status; - struct hisi_sas_err_record_v1 *err_record = slot->status_buffer; + struct hisi_sas_err_record_v1 *err_record = + hisi_sas_status_buf_addr_mem(slot); struct device *dev = &hisi_hba->pdev->dev; switch (task->task_proto) { @@ -1364,8 +1362,11 @@ static int slot_complete_v1_hw(struct hisi_hba *hisi_hba, switch (task->task_proto) { case SAS_PROTOCOL_SSP: { - struct ssp_response_iu *iu = slot->status_buffer + - sizeof(struct hisi_sas_err_record); + struct hisi_sas_status_buffer *status_buffer = + hisi_sas_status_buf_addr_mem(slot); + struct ssp_response_iu *iu = (struct ssp_response_iu *) + &status_buffer->iu[0]; + sas_ssp_task_response(dev, task, iu); break; } @@ -1382,7 +1383,7 @@ static int slot_complete_v1_hw(struct hisi_hba *hisi_hba, dma_unmap_sg(dev, &task->smp_task.smp_req, 1, DMA_TO_DEVICE); memcpy(to + sg_resp->offset, - slot->status_buffer + + hisi_sas_status_buf_addr_mem(slot) + sizeof(struct hisi_sas_err_record), sg_dma_len(sg_resp)); kunmap_atomic(to); diff --git a/drivers/scsi/hisi_sas/hisi_sas_v2_hw.c b/drivers/scsi/hisi_sas/hisi_sas_v2_hw.c index 658ef5a59fb6..8e157940f8d0 100644 --- a/drivers/scsi/hisi_sas/hisi_sas_v2_hw.c +++ b/drivers/scsi/hisi_sas/hisi_sas_v2_hw.c @@ -1498,6 +1498,7 @@ static int prep_prd_sge_v2_hw(struct hisi_hba *hisi_hba, struct scatterlist *scatter, int n_elem) { + struct hisi_sas_sge_page *sge_page = hisi_sas_sge_addr_mem(slot); struct device *dev = &hisi_hba->pdev->dev; struct scatterlist *sg; int i; @@ -1508,13 +1509,8 @@ static int prep_prd_sge_v2_hw(struct hisi_hba *hisi_hba, return -EINVAL; } - slot->sge_page = dma_pool_alloc(hisi_hba->sge_page_pool, GFP_ATOMIC, - &slot->sge_page_dma); - if (!slot->sge_page) - return -ENOMEM; - for_each_sg(scatter, sg, n_elem, i) { - struct hisi_sas_sge *entry = &slot->sge_page->sge[i]; + struct hisi_sas_sge *entry = &sge_page->sge[i]; entry->addr = cpu_to_le64(sg_dma_address(sg)); entry->page_ctrl_0 = entry->page_ctrl_1 = 0; @@ -1522,7 +1518,7 @@ static int prep_prd_sge_v2_hw(struct hisi_hba *hisi_hba, entry->data_off = 0; } - hdr->prd_table_addr = cpu_to_le64(slot->sge_page_dma); + hdr->prd_table_addr = cpu_to_le64(hisi_sas_sge_addr_dma(slot)); hdr->sg_len = cpu_to_le32(n_elem << CMD_HDR_DATA_SGL_LEN_OFF); @@ -1586,7 +1582,7 @@ static int prep_smp_v2_hw(struct hisi_hba *hisi_hba, hdr->transfer_tags = cpu_to_le32(slot->idx << CMD_HDR_IPTT_OFF); hdr->cmd_table_addr = cpu_to_le64(req_dma_addr); - hdr->sts_buffer_addr = cpu_to_le64(slot->status_buffer_dma); + hdr->sts_buffer_addr = cpu_to_le64(hisi_sas_status_buf_addr_dma(slot)); return 0; @@ -1660,10 +1656,11 @@ static int prep_ssp_v2_hw(struct hisi_hba *hisi_hba, } hdr->data_transfer_len = cpu_to_le32(task->total_xfer_len); - hdr->cmd_table_addr = cpu_to_le64(slot->command_table_dma); - hdr->sts_buffer_addr = cpu_to_le64(slot->status_buffer_dma); + hdr->cmd_table_addr = cpu_to_le64(hisi_sas_cmd_hdr_addr_dma(slot)); + hdr->sts_buffer_addr = cpu_to_le64(hisi_sas_status_buf_addr_dma(slot)); - buf_cmd = slot->command_table + sizeof(struct ssp_frame_hdr); + buf_cmd = hisi_sas_cmd_hdr_addr_mem(slot) + + sizeof(struct ssp_frame_hdr); memcpy(buf_cmd, &task->ssp_task.LUN, 8); if (!is_tmf) { @@ -1890,7 +1887,8 @@ static void slot_err_v2_hw(struct hisi_hba *hisi_hba, int err_phase) { struct task_status_struct *ts = &task->task_status; - struct hisi_sas_err_record_v2 *err_record = slot->status_buffer; + struct hisi_sas_err_record_v2 *err_record = + hisi_sas_status_buf_addr_mem(slot); u32 trans_tx_fail_type = cpu_to_le32(err_record->trans_tx_fail_type); u32 trans_rx_fail_type = cpu_to_le32(err_record->trans_rx_fail_type); u16 dma_tx_err_type = cpu_to_le16(err_record->dma_tx_err_type); @@ -2279,8 +2277,10 @@ slot_complete_v2_hw(struct hisi_hba *hisi_hba, struct hisi_sas_slot *slot) switch (task->task_proto) { case SAS_PROTOCOL_SSP: { - struct ssp_response_iu *iu = slot->status_buffer + - sizeof(struct hisi_sas_err_record); + struct hisi_sas_status_buffer *status_buffer = + hisi_sas_status_buf_addr_mem(slot); + struct ssp_response_iu *iu = (struct ssp_response_iu *) + &status_buffer->iu[0]; sas_ssp_task_response(dev, task, iu); break; @@ -2298,7 +2298,7 @@ slot_complete_v2_hw(struct hisi_hba *hisi_hba, struct hisi_sas_slot *slot) dma_unmap_sg(dev, &task->smp_task.smp_req, 1, DMA_TO_DEVICE); memcpy(to + sg_resp->offset, - slot->status_buffer + + hisi_sas_status_buf_addr_mem(slot) + sizeof(struct hisi_sas_err_record), sg_dma_len(sg_resp)); kunmap_atomic(to); @@ -2475,12 +2475,11 @@ static int prep_ata_v2_hw(struct hisi_hba *hisi_hba, return rc; } - hdr->data_transfer_len = cpu_to_le32(task->total_xfer_len); - hdr->cmd_table_addr = cpu_to_le64(slot->command_table_dma); - hdr->sts_buffer_addr = cpu_to_le64(slot->status_buffer_dma); + hdr->cmd_table_addr = cpu_to_le64(hisi_sas_cmd_hdr_addr_dma(slot)); + hdr->sts_buffer_addr = cpu_to_le64(hisi_sas_status_buf_addr_dma(slot)); - buf_cmd = slot->command_table; + buf_cmd = hisi_sas_cmd_hdr_addr_mem(slot); if (likely(!task->ata_task.device_control_reg_update)) task->ata_task.fis.flags |= 0x80; /* C=1: update ATA cmd reg */