From patchwork Wed Jul 3 15:59:39 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxim Levitsky X-Patchwork-Id: 1126939 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 45f5sg03hWz9s4Y for ; Thu, 4 Jul 2019 02:19:31 +1000 (AEST) Received: from localhost ([::1]:37301 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hihyi-0004Qt-T6 for incoming@patchwork.ozlabs.org; Wed, 03 Jul 2019 12:19:28 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54990) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hihg9-0007Qa-1u for qemu-devel@nongnu.org; Wed, 03 Jul 2019 12:00:18 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hihg7-0002Lk-0B for qemu-devel@nongnu.org; Wed, 03 Jul 2019 12:00:16 -0400 Received: from mx1.redhat.com ([209.132.183.28]:53778) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hihfy-0002DO-OA; Wed, 03 Jul 2019 12:00:08 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 485D985A04; Wed, 3 Jul 2019 16:00:05 +0000 (UTC) Received: from maximlenovopc.usersys.redhat.com (dhcp-4-67.tlv.redhat.com [10.35.4.67]) by smtp.corp.redhat.com (Postfix) with ESMTP id DCD72519C2; Wed, 3 Jul 2019 15:59:53 +0000 (UTC) From: Maxim Levitsky To: qemu-devel@nongnu.org Date: Wed, 3 Jul 2019 18:59:39 +0300 Message-Id: <20190703155944.9637-2-mlevitsk@redhat.com> In-Reply-To: <20190703155944.9637-1-mlevitsk@redhat.com> References: <20190703155944.9637-1-mlevitsk@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.26]); Wed, 03 Jul 2019 16:00:05 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 1/6] block/nvme: don't touch the completion entries X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , Kevin Wolf , qemu-block@nongnu.org, Max Reitz , Paolo Bonzini , Maxim Levitsky , John Snow Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Completion entries are meant to be only read by the host and written by the device. The driver is supposed to scan the completions from the last point where it left, and until it sees a completion with non flipped phase bit. Signed-off-by: Maxim Levitsky Reviewed-by: Max Reitz --- block/nvme.c | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/block/nvme.c b/block/nvme.c index 73ed5fa75f..6d4e7f3d83 100644 --- a/block/nvme.c +++ b/block/nvme.c @@ -315,7 +315,7 @@ static bool nvme_process_completion(BDRVNVMeState *s, NVMeQueuePair *q) while (q->inflight) { int16_t cid; c = (NvmeCqe *)&q->cq.queue[q->cq.head * NVME_CQ_ENTRY_BYTES]; - if (!c->cid || (le16_to_cpu(c->status) & 0x1) == q->cq_phase) { + if ((le16_to_cpu(c->status) & 0x1) == q->cq_phase) { break; } q->cq.head = (q->cq.head + 1) % NVME_QUEUE_SIZE; @@ -339,10 +339,7 @@ static bool nvme_process_completion(BDRVNVMeState *s, NVMeQueuePair *q) qemu_mutex_unlock(&q->lock); req.cb(req.opaque, nvme_translate_error(c)); qemu_mutex_lock(&q->lock); - c->cid = cpu_to_le16(0); q->inflight--; - /* Flip Phase Tag bit. */ - c->status = cpu_to_le16(le16_to_cpu(c->status) ^ 0x1); progress = true; } if (progress) { From patchwork Wed Jul 3 15:59:40 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxim Levitsky X-Patchwork-Id: 1126941 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 45f5zp63Stz9s3Z for ; Thu, 4 Jul 2019 02:24:54 +1000 (AEST) Received: from localhost ([::1]:37350 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hii3w-0007Ju-Th for incoming@patchwork.ozlabs.org; Wed, 03 Jul 2019 12:24:52 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55159) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hihgQ-0007eV-PT for qemu-devel@nongnu.org; Wed, 03 Jul 2019 12:00:35 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hihgO-0002X4-RB for qemu-devel@nongnu.org; Wed, 03 Jul 2019 12:00:34 -0400 Received: from mx1.redhat.com ([209.132.183.28]:35792) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hihgI-0002S0-I6; Wed, 03 Jul 2019 12:00:30 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 81B2681DE9; Wed, 3 Jul 2019 16:00:21 +0000 (UTC) Received: from maximlenovopc.usersys.redhat.com (dhcp-4-67.tlv.redhat.com [10.35.4.67]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9A1ED18963; Wed, 3 Jul 2019 16:00:05 +0000 (UTC) From: Maxim Levitsky To: qemu-devel@nongnu.org Date: Wed, 3 Jul 2019 18:59:40 +0300 Message-Id: <20190703155944.9637-3-mlevitsk@redhat.com> In-Reply-To: <20190703155944.9637-1-mlevitsk@redhat.com> References: <20190703155944.9637-1-mlevitsk@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.25]); Wed, 03 Jul 2019 16:00:21 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 2/6] block/nvme: fix doorbell stride X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , Kevin Wolf , qemu-block@nongnu.org, Max Reitz , Paolo Bonzini , Maxim Levitsky , John Snow Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Fix the math involving non standard doorbell stride Signed-off-by: Maxim Levitsky Reviewed-by: Max Reitz --- block/nvme.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/block/nvme.c b/block/nvme.c index 6d4e7f3d83..52798081b2 100644 --- a/block/nvme.c +++ b/block/nvme.c @@ -217,7 +217,7 @@ static NVMeQueuePair *nvme_create_queue_pair(BlockDriverState *bs, error_propagate(errp, local_err); goto fail; } - q->cq.doorbell = &s->regs->doorbells[idx * 2 * s->doorbell_scale + 1]; + q->cq.doorbell = &s->regs->doorbells[(idx * 2 + 1) * s->doorbell_scale]; return q; fail: From patchwork Wed Jul 3 15:59:41 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxim Levitsky X-Patchwork-Id: 1126944 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 45f64d4QgKz9s8m for ; Thu, 4 Jul 2019 02:29:05 +1000 (AEST) Received: from localhost ([::1]:37414 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hii7z-0004rq-LW for incoming@patchwork.ozlabs.org; Wed, 03 Jul 2019 12:29:03 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55166) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hihgR-0007fy-9U for qemu-devel@nongnu.org; Wed, 03 Jul 2019 12:00:36 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hihgP-0002Xd-Va for qemu-devel@nongnu.org; Wed, 03 Jul 2019 12:00:35 -0400 Received: from mx1.redhat.com ([209.132.183.28]:35848) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hihgM-0002UG-Ff; Wed, 03 Jul 2019 12:00:30 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 6141C81DF1; Wed, 3 Jul 2019 16:00:29 +0000 (UTC) Received: from maximlenovopc.usersys.redhat.com (dhcp-4-67.tlv.redhat.com [10.35.4.67]) by smtp.corp.redhat.com (Postfix) with ESMTP id D5DB73781; Wed, 3 Jul 2019 16:00:21 +0000 (UTC) From: Maxim Levitsky To: qemu-devel@nongnu.org Date: Wed, 3 Jul 2019 18:59:41 +0300 Message-Id: <20190703155944.9637-4-mlevitsk@redhat.com> In-Reply-To: <20190703155944.9637-1-mlevitsk@redhat.com> References: <20190703155944.9637-1-mlevitsk@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.25]); Wed, 03 Jul 2019 16:00:29 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 3/6] block/nvme: support larger that 512 bytes sector devices X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , Kevin Wolf , qemu-block@nongnu.org, Max Reitz , Paolo Bonzini , Maxim Levitsky , John Snow Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Currently the driver hardcodes the sector size to 512, and doesn't check the underlying device. Fix that. Also fail if underlying nvme device is formatted with metadata as this needs special support. Signed-off-by: Maxim Levitsky --- block/nvme.c | 45 ++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 40 insertions(+), 5 deletions(-) diff --git a/block/nvme.c b/block/nvme.c index 52798081b2..1f0d09349f 100644 --- a/block/nvme.c +++ b/block/nvme.c @@ -102,8 +102,11 @@ typedef struct { size_t doorbell_scale; bool write_cache_supported; EventNotifier irq_notifier; + uint64_t nsze; /* Namespace size reported by identify command */ int nsid; /* The namespace id to read/write data. */ + size_t blkshift; + uint64_t max_transfer; bool plugged; @@ -415,8 +418,9 @@ static void nvme_identify(BlockDriverState *bs, int namespace, Error **errp) BDRVNVMeState *s = bs->opaque; NvmeIdCtrl *idctrl; NvmeIdNs *idns; + NvmeLBAF *lbaf; uint8_t *resp; - int r; + int r, hwsect_size; uint64_t iova; NvmeCmd cmd = { .opcode = NVME_ADM_CMD_IDENTIFY, @@ -463,7 +467,22 @@ static void nvme_identify(BlockDriverState *bs, int namespace, Error **errp) } s->nsze = le64_to_cpu(idns->nsze); + lbaf = &idns->lbaf[NVME_ID_NS_FLBAS_INDEX(idns->flbas)]; + + if (lbaf->ms) { + error_setg(errp, "Namespaces with metadata are not yet supported"); + goto out; + } + + hwsect_size = 1 << lbaf->ds; + + if (hwsect_size < BDRV_SECTOR_BITS || hwsect_size > s->page_size) { + error_setg(errp, "Namespace has unsupported block size (%d)", + hwsect_size); + goto out; + } + s->blkshift = lbaf->ds; out: qemu_vfio_dma_unmap(s->vfio, resp); qemu_vfree(resp); @@ -782,8 +801,22 @@ fail: static int64_t nvme_getlength(BlockDriverState *bs) { BDRVNVMeState *s = bs->opaque; + return s->nsze << s->blkshift; +} - return s->nsze << BDRV_SECTOR_BITS; +static int64_t nvme_get_blocksize(BlockDriverState *bs) +{ + BDRVNVMeState *s = bs->opaque; + assert(s->blkshift >= 9); + return 1 << s->blkshift; +} + +static int nvme_probe_blocksizes(BlockDriverState *bs, BlockSizes *bsz) +{ + int64_t blocksize = nvme_get_blocksize(bs); + bsz->phys = blocksize; + bsz->log = blocksize; + return 0; } /* Called with s->dma_map_lock */ @@ -914,13 +947,14 @@ static coroutine_fn int nvme_co_prw_aligned(BlockDriverState *bs, BDRVNVMeState *s = bs->opaque; NVMeQueuePair *ioq = s->queues[1]; NVMeRequest *req; - uint32_t cdw12 = (((bytes >> BDRV_SECTOR_BITS) - 1) & 0xFFFF) | + + uint32_t cdw12 = (((bytes >> s->blkshift) - 1) & 0xFFFF) | (flags & BDRV_REQ_FUA ? 1 << 30 : 0); NvmeCmd cmd = { .opcode = is_write ? NVME_CMD_WRITE : NVME_CMD_READ, .nsid = cpu_to_le32(s->nsid), - .cdw10 = cpu_to_le32((offset >> BDRV_SECTOR_BITS) & 0xFFFFFFFF), - .cdw11 = cpu_to_le32(((offset >> BDRV_SECTOR_BITS) >> 32) & 0xFFFFFFFF), + .cdw10 = cpu_to_le32((offset >> s->blkshift) & 0xFFFFFFFF), + .cdw11 = cpu_to_le32(((offset >> s->blkshift) >> 32) & 0xFFFFFFFF), .cdw12 = cpu_to_le32(cdw12), }; NVMeCoData data = { @@ -1151,6 +1185,7 @@ static BlockDriver bdrv_nvme = { .bdrv_file_open = nvme_file_open, .bdrv_close = nvme_close, .bdrv_getlength = nvme_getlength, + .bdrv_probe_blocksizes = nvme_probe_blocksizes, .bdrv_co_preadv = nvme_co_preadv, .bdrv_co_pwritev = nvme_co_pwritev, From patchwork Wed Jul 3 15:59:42 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxim Levitsky X-Patchwork-Id: 1126955 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 45f6FY1GLQz9s8m for ; Thu, 4 Jul 2019 02:36:49 +1000 (AEST) Received: from localhost ([::1]:37514 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hiiFR-0003Ju-JS for incoming@patchwork.ozlabs.org; Wed, 03 Jul 2019 12:36:47 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55222) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hihgb-0007rc-IF for qemu-devel@nongnu.org; Wed, 03 Jul 2019 12:00:47 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hihgZ-0002bA-JA for qemu-devel@nongnu.org; Wed, 03 Jul 2019 12:00:45 -0400 Received: from mx1.redhat.com ([209.132.183.28]:42324) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hihgT-0002Z6-CA; Wed, 03 Jul 2019 12:00:37 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id A97BB21BAA; Wed, 3 Jul 2019 16:00:31 +0000 (UTC) Received: from maximlenovopc.usersys.redhat.com (dhcp-4-67.tlv.redhat.com [10.35.4.67]) by smtp.corp.redhat.com (Postfix) with ESMTP id B59BB183B2; Wed, 3 Jul 2019 16:00:29 +0000 (UTC) From: Maxim Levitsky To: qemu-devel@nongnu.org Date: Wed, 3 Jul 2019 18:59:42 +0300 Message-Id: <20190703155944.9637-5-mlevitsk@redhat.com> In-Reply-To: <20190703155944.9637-1-mlevitsk@redhat.com> References: <20190703155944.9637-1-mlevitsk@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Wed, 03 Jul 2019 16:00:31 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 4/6] block/nvme: add support for image creation X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , Kevin Wolf , qemu-block@nongnu.org, Max Reitz , Paolo Bonzini , Maxim Levitsky , John Snow Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Tesed on a nvme device like that: # create preallocated qcow2 image $ qemu-img create -f qcow2 nvme://0000:06:00.0/1 10G -o preallocation=metadata Formatting 'nvme://0000:06:00.0/1', fmt=qcow2 size=10737418240 cluster_size=65536 preallocation=metadata lazy_refcounts=off refcount_bits=16 # create an empty qcow2 image $ qemu-img create -f qcow2 nvme://0000:06:00.0/1 10G -o preallocation=off Formatting 'nvme://0000:06:00.0/1', fmt=qcow2 size=10737418240 cluster_size=65536 preallocation=off lazy_refcounts=off refcount_bits=16 Signed-off-by: Maxim Levitsky --- block/nvme.c | 108 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 108 insertions(+) diff --git a/block/nvme.c b/block/nvme.c index 1f0d09349f..152d27b07f 100644 --- a/block/nvme.c +++ b/block/nvme.c @@ -1148,6 +1148,90 @@ static void nvme_aio_unplug(BlockDriverState *bs) } } +static int coroutine_fn nvme_co_create_opts(const char *filename, + QemuOpts *opts, Error **errp) +{ + + int64_t total_size = 0; + char *buf = NULL; + BlockDriverState *bs; + QEMUIOVector local_qiov; + int ret = 0; + int64_t blocksize; + QDict *options; + Error *local_err = NULL; + PreallocMode prealloc; + + total_size = ROUND_UP(qemu_opt_get_size_del(opts, BLOCK_OPT_SIZE, 0), + BDRV_SECTOR_SIZE); + + buf = qemu_opt_get_del(opts, BLOCK_OPT_PREALLOC); + prealloc = qapi_enum_parse(&PreallocMode_lookup, buf, + PREALLOC_MODE_OFF, &local_err); + g_free(buf); + + if (prealloc != PREALLOC_MODE_OFF) { + error_setg(errp, "Only prealloc=off is supported"); + return -EINVAL; + } + + options = qdict_new(); + qdict_put_str(options, "driver", "nvme"); + nvme_parse_filename(filename, options, &local_err); + + if (local_err) { + error_propagate(errp, local_err); + qobject_unref(options); + return -EINVAL; + } + + bs = bdrv_open(NULL, NULL, options, + BDRV_O_RDWR | BDRV_O_RESIZE | BDRV_O_PROTOCOL, errp); + if (bs == NULL) { + return -EIO; + } + + if (nvme_getlength(bs) < total_size) { + error_setg(errp, "Device is too small"); + bdrv_unref(bs); + qobject_unref(options); + return -ENOSPC; + } + + blocksize = nvme_get_blocksize(bs); + buf = qemu_try_blockalign0(bs, blocksize); + qemu_iovec_init(&local_qiov, 1); + qemu_iovec_add(&local_qiov, buf, blocksize); + + ret = nvme_co_prw_aligned(bs, 0, blocksize, + &local_qiov, true, BDRV_REQ_FUA); + if (ret) { + error_setg(errp, "Write error to sector 0"); + } + + qemu_vfree(buf); + bdrv_unref(bs); + return ret; +} + + +static int coroutine_fn nvme_co_truncate(BlockDriverState *bs, int64_t offset, + PreallocMode prealloc, Error **errp) +{ + if (prealloc != PREALLOC_MODE_OFF) { + error_setg(errp, "Preallocation mode '%s' unsupported nvme devices", + PreallocMode_str(prealloc)); + return -ENOTSUP; + } + + if (offset > nvme_getlength(bs)) { + error_setg(errp, "Cannot grow nvme devices"); + return -EINVAL; + } + + return 0; +} + static void nvme_register_buf(BlockDriverState *bs, void *host, size_t size) { int ret; @@ -1169,6 +1253,7 @@ static void nvme_unregister_buf(BlockDriverState *bs, void *host) qemu_vfio_dma_unmap(s->vfio, host); } + static const char *const nvme_strong_runtime_opts[] = { NVME_BLOCK_OPT_DEVICE, NVME_BLOCK_OPT_NAMESPACE, @@ -1176,6 +1261,25 @@ static const char *const nvme_strong_runtime_opts[] = { NULL }; + +static QemuOptsList nvme_create_opts = { + .name = "nvme-create-opts", + .head = QTAILQ_HEAD_INITIALIZER(nvme_create_opts.head), + .desc = { + { + .name = BLOCK_OPT_SIZE, + .type = QEMU_OPT_SIZE, + .help = "Virtual disk size" + }, + { + .name = BLOCK_OPT_PREALLOC, + .type = QEMU_OPT_STRING, + .help = "Preallocation mode (allowed values: off)", + }, + { /* end of list */ } + } +}; + static BlockDriver bdrv_nvme = { .format_name = "nvme", .protocol_name = "nvme", @@ -1187,6 +1291,10 @@ static BlockDriver bdrv_nvme = { .bdrv_getlength = nvme_getlength, .bdrv_probe_blocksizes = nvme_probe_blocksizes, + .bdrv_co_create_opts = nvme_co_create_opts, + .bdrv_co_truncate = nvme_co_truncate, + .create_opts = &nvme_create_opts, + .bdrv_co_preadv = nvme_co_preadv, .bdrv_co_pwritev = nvme_co_pwritev, .bdrv_co_flush_to_disk = nvme_co_flush, From patchwork Wed Jul 3 15:59:43 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxim Levitsky X-Patchwork-Id: 1126952 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 45f69b3tR7z9s8m for ; Thu, 4 Jul 2019 02:33:19 +1000 (AEST) Received: from localhost ([::1]:37482 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hiiC2-0000LZ-Q1 for incoming@patchwork.ozlabs.org; Wed, 03 Jul 2019 12:33:14 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55210) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hihgX-0007qa-O0 for qemu-devel@nongnu.org; Wed, 03 Jul 2019 12:00:43 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hihgV-0002Zu-9S for qemu-devel@nongnu.org; Wed, 03 Jul 2019 12:00:41 -0400 Received: from mx1.redhat.com ([209.132.183.28]:41054) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hihgR-0002YP-L2; Wed, 03 Jul 2019 12:00:35 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id E56A031628E7; Wed, 3 Jul 2019 16:00:34 +0000 (UTC) Received: from maximlenovopc.usersys.redhat.com (dhcp-4-67.tlv.redhat.com [10.35.4.67]) by smtp.corp.redhat.com (Postfix) with ESMTP id 047FE503EA; Wed, 3 Jul 2019 16:00:31 +0000 (UTC) From: Maxim Levitsky To: qemu-devel@nongnu.org Date: Wed, 3 Jul 2019 18:59:43 +0300 Message-Id: <20190703155944.9637-6-mlevitsk@redhat.com> In-Reply-To: <20190703155944.9637-1-mlevitsk@redhat.com> References: <20190703155944.9637-1-mlevitsk@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.41]); Wed, 03 Jul 2019 16:00:35 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 5/6] block/nvme: add support for write zeros X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , Kevin Wolf , qemu-block@nongnu.org, Max Reitz , Paolo Bonzini , Maxim Levitsky , John Snow Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Maxim Levitsky --- block/nvme.c | 69 +++++++++++++++++++++++++++++++++++++++++++- block/trace-events | 1 + include/block/nvme.h | 19 +++++++++++- 3 files changed, 87 insertions(+), 2 deletions(-) diff --git a/block/nvme.c b/block/nvme.c index 152d27b07f..02e0846643 100644 --- a/block/nvme.c +++ b/block/nvme.c @@ -110,6 +110,8 @@ typedef struct { uint64_t max_transfer; bool plugged; + bool supports_write_zeros; + CoMutex dma_map_lock; CoQueue dma_flush_queue; @@ -457,6 +459,8 @@ static void nvme_identify(BlockDriverState *bs, int namespace, Error **errp) s->max_transfer = MIN_NON_ZERO(s->max_transfer, s->page_size / sizeof(uint64_t) * s->page_size); + s->supports_write_zeros = (idctrl->oncs & NVME_ONCS_WRITE_ZEROS) != 0; + memset(resp, 0, 4096); cmd.cdw10 = 0; @@ -469,6 +473,11 @@ static void nvme_identify(BlockDriverState *bs, int namespace, Error **errp) s->nsze = le64_to_cpu(idns->nsze); lbaf = &idns->lbaf[NVME_ID_NS_FLBAS_INDEX(idns->flbas)]; + if (NVME_ID_NS_DLFEAT_WRITE_ZEROS(idns->dlfeat) && + NVME_ID_NS_DLFEAT_READ_BEHAVIOR(idns->dlfeat) == + NVME_ID_NS_DLFEAT_READ_BEHAVIOR_ZEROS) + bs->supported_write_flags |= BDRV_REQ_MAY_UNMAP; + if (lbaf->ms) { error_setg(errp, "Namespaces with metadata are not yet supported"); goto out; @@ -763,6 +772,8 @@ static int nvme_file_open(BlockDriverState *bs, QDict *options, int flags, int ret; BDRVNVMeState *s = bs->opaque; + bs->supported_write_flags = BDRV_REQ_FUA; + opts = qemu_opts_create(&runtime_opts, NULL, 0, &error_abort); qemu_opts_absorb_qdict(opts, options, &error_abort); device = qemu_opt_get(opts, NVME_BLOCK_OPT_DEVICE); @@ -791,7 +802,6 @@ static int nvme_file_open(BlockDriverState *bs, QDict *options, int flags, goto fail; } } - bs->supported_write_flags = BDRV_REQ_FUA; return 0; fail: nvme_close(bs); @@ -1085,6 +1095,60 @@ static coroutine_fn int nvme_co_flush(BlockDriverState *bs) } +static coroutine_fn int nvme_co_pwrite_zeroes(BlockDriverState *bs, + int64_t offset, + int bytes, + BdrvRequestFlags flags) +{ + BDRVNVMeState *s = bs->opaque; + NVMeQueuePair *ioq = s->queues[1]; + NVMeRequest *req; + + if (!s->supports_write_zeros) { + return -ENOTSUP; + } + + uint32_t cdw12 = ((bytes >> s->blkshift) - 1) & 0xFFFF; + + NvmeCmd cmd = { + .opcode = NVME_CMD_WRITE_ZEROS, + .nsid = cpu_to_le32(s->nsid), + .cdw10 = cpu_to_le32((offset >> s->blkshift) & 0xFFFFFFFF), + .cdw11 = cpu_to_le32(((offset >> s->blkshift) >> 32) & 0xFFFFFFFF), + }; + + NVMeCoData data = { + .ctx = bdrv_get_aio_context(bs), + .ret = -EINPROGRESS, + }; + + if (flags & BDRV_REQ_MAY_UNMAP) { + cdw12 |= (1 << 25); + } + + if (flags & BDRV_REQ_FUA) { + cdw12 |= (1 << 30); + } + + cmd.cdw12 = cpu_to_le32(cdw12); + + trace_nvme_write_zeros(s, offset, bytes, flags); + assert(s->nr_queues > 1); + req = nvme_get_free_req(ioq); + assert(req); + + nvme_submit_command(s, ioq, req, &cmd, nvme_rw_cb, &data); + + data.co = qemu_coroutine_self(); + while (data.ret == -EINPROGRESS) { + qemu_coroutine_yield(); + } + + trace_nvme_rw_done(s, true, offset, bytes, data.ret); + return data.ret; +} + + static int nvme_reopen_prepare(BDRVReopenState *reopen_state, BlockReopenQueue *queue, Error **errp) { @@ -1297,6 +1361,9 @@ static BlockDriver bdrv_nvme = { .bdrv_co_preadv = nvme_co_preadv, .bdrv_co_pwritev = nvme_co_pwritev, + + .bdrv_co_pwrite_zeroes = nvme_co_pwrite_zeroes, + .bdrv_co_flush_to_disk = nvme_co_flush, .bdrv_reopen_prepare = nvme_reopen_prepare, diff --git a/block/trace-events b/block/trace-events index 9ccea755da..12f363bb44 100644 --- a/block/trace-events +++ b/block/trace-events @@ -148,6 +148,7 @@ nvme_submit_command_raw(int c0, int c1, int c2, int c3, int c4, int c5, int c6, nvme_handle_event(void *s) "s %p" nvme_poll_cb(void *s) "s %p" nvme_prw_aligned(void *s, int is_write, uint64_t offset, uint64_t bytes, int flags, int niov) "s %p is_write %d offset %"PRId64" bytes %"PRId64" flags %d niov %d" +nvme_write_zeros(void *s, uint64_t offset, uint64_t bytes, int flags) "s %p offset %"PRId64" bytes %"PRId64" flags %d" nvme_qiov_unaligned(const void *qiov, int n, void *base, size_t size, int align) "qiov %p n %d base %p size 0x%zx align 0x%x" nvme_prw_buffered(void *s, uint64_t offset, uint64_t bytes, int niov, int is_write) "s %p offset %"PRId64" bytes %"PRId64" niov %d is_write %d" nvme_rw_done(void *s, int is_write, uint64_t offset, uint64_t bytes, int ret) "s %p is_write %d offset %"PRId64" bytes %"PRId64" ret %d" diff --git a/include/block/nvme.h b/include/block/nvme.h index 3ec8efcc43..65eb65c740 100644 --- a/include/block/nvme.h +++ b/include/block/nvme.h @@ -653,12 +653,29 @@ typedef struct NvmeIdNs { uint8_t mc; uint8_t dpc; uint8_t dps; - uint8_t res30[98]; + + uint8_t nmic; + uint8_t rescap; + uint8_t fpi; + uint8_t dlfeat; + + uint8_t res30[94]; NvmeLBAF lbaf[16]; uint8_t res192[192]; uint8_t vs[3712]; } NvmeIdNs; + +/*Deallocate Logical Block Features*/ +#define NVME_ID_NS_DLFEAT_GUARD_CRC(dlfeat) ((dlfeat) & 0x10) +#define NVME_ID_NS_DLFEAT_WRITE_ZEROS(dlfeat) ((dlfeat) & 0x04) + +#define NVME_ID_NS_DLFEAT_READ_BEHAVIOR(dlfeat) ((dlfeat) & 0x3) +#define NVME_ID_NS_DLFEAT_READ_BEHAVIOR_UNDEFINED 0 +#define NVME_ID_NS_DLFEAT_READ_BEHAVIOR_ZEROS 1 +#define NVME_ID_NS_DLFEAT_READ_BEHAVIOR_ONES 2 + + #define NVME_ID_NS_NSFEAT_THIN(nsfeat) ((nsfeat & 0x1)) #define NVME_ID_NS_FLBAS_EXTENDED(flbas) ((flbas >> 4) & 0x1) #define NVME_ID_NS_FLBAS_INDEX(flbas) ((flbas & 0xf)) From patchwork Wed Jul 3 15:59:44 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxim Levitsky X-Patchwork-Id: 1126938 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 45f5pP6wX8z9s3Z for ; Thu, 4 Jul 2019 02:16:45 +1000 (AEST) Received: from localhost ([::1]:37256 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hihw3-00019i-0T for incoming@patchwork.ozlabs.org; Wed, 03 Jul 2019 12:16:43 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55276) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hihgj-0007tu-RY for qemu-devel@nongnu.org; Wed, 03 Jul 2019 12:00:56 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hihgh-0002eA-GR for qemu-devel@nongnu.org; Wed, 03 Jul 2019 12:00:53 -0400 Received: from mx1.redhat.com ([209.132.183.28]:14163) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hihgX-0002aB-MI; Wed, 03 Jul 2019 12:00:43 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 8396C81DE5; Wed, 3 Jul 2019 16:00:39 +0000 (UTC) Received: from maximlenovopc.usersys.redhat.com (dhcp-4-67.tlv.redhat.com [10.35.4.67]) by smtp.corp.redhat.com (Postfix) with ESMTP id 43D293781; Wed, 3 Jul 2019 16:00:35 +0000 (UTC) From: Maxim Levitsky To: qemu-devel@nongnu.org Date: Wed, 3 Jul 2019 18:59:44 +0300 Message-Id: <20190703155944.9637-7-mlevitsk@redhat.com> In-Reply-To: <20190703155944.9637-1-mlevitsk@redhat.com> References: <20190703155944.9637-1-mlevitsk@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.25]); Wed, 03 Jul 2019 16:00:39 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 6/6] block/nvme: add support for discard X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , Kevin Wolf , qemu-block@nongnu.org, Max Reitz , Paolo Bonzini , Maxim Levitsky , John Snow Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Maxim Levitsky --- block/nvme.c | 81 ++++++++++++++++++++++++++++++++++++++++++++++ block/trace-events | 2 ++ 2 files changed, 83 insertions(+) diff --git a/block/nvme.c b/block/nvme.c index 02e0846643..f8bcf1ffb6 100644 --- a/block/nvme.c +++ b/block/nvme.c @@ -111,6 +111,7 @@ typedef struct { bool plugged; bool supports_write_zeros; + bool supports_discard; CoMutex dma_map_lock; CoQueue dma_flush_queue; @@ -460,6 +461,7 @@ static void nvme_identify(BlockDriverState *bs, int namespace, Error **errp) s->page_size / sizeof(uint64_t) * s->page_size); s->supports_write_zeros = (idctrl->oncs & NVME_ONCS_WRITE_ZEROS) != 0; + s->supports_discard = (idctrl->oncs & NVME_ONCS_DSM) != 0; memset(resp, 0, 4096); @@ -1149,6 +1151,84 @@ static coroutine_fn int nvme_co_pwrite_zeroes(BlockDriverState *bs, } +static int coroutine_fn nvme_co_pdiscard(BlockDriverState *bs, + int64_t offset, + int bytes) +{ + BDRVNVMeState *s = bs->opaque; + NVMeQueuePair *ioq = s->queues[1]; + NVMeRequest *req; + NvmeDsmRange *buf; + QEMUIOVector local_qiov; + int r; + + NvmeCmd cmd = { + .opcode = NVME_CMD_DSM, + .nsid = cpu_to_le32(s->nsid), + .cdw10 = 0, /*number of ranges - 0 based*/ + .cdw11 = cpu_to_le32(1 << 2), /*deallocate bit*/ + }; + + NVMeCoData data = { + .ctx = bdrv_get_aio_context(bs), + .ret = -EINPROGRESS, + }; + + if (!s->supports_discard) { + return -ENOTSUP; + } + + assert(s->nr_queues > 1); + + buf = qemu_try_blockalign0(bs, 4096); + if (!buf) { + return -ENOMEM; + } + + buf->nlb = bytes >> s->blkshift; + buf->slba = offset >> s->blkshift; + buf->cattr = 0; + + qemu_iovec_init(&local_qiov, 1); + qemu_iovec_add(&local_qiov, buf, 4096); + + req = nvme_get_free_req(ioq); + assert(req); + + qemu_co_mutex_lock(&s->dma_map_lock); + r = nvme_cmd_map_qiov(bs, &cmd, req, &local_qiov); + qemu_co_mutex_unlock(&s->dma_map_lock); + + if (r) { + req->busy = false; + return r; + } + + trace_nvme_dsm(s, offset, bytes); + + nvme_submit_command(s, ioq, req, &cmd, nvme_rw_cb, &data); + + data.co = qemu_coroutine_self(); + while (data.ret == -EINPROGRESS) { + qemu_coroutine_yield(); + } + + qemu_co_mutex_lock(&s->dma_map_lock); + r = nvme_cmd_unmap_qiov(bs, &local_qiov); + qemu_co_mutex_unlock(&s->dma_map_lock); + if (r) { + return r; + } + + trace_nvme_dsm_done(s, offset, bytes, data.ret); + + qemu_iovec_destroy(&local_qiov); + qemu_vfree(buf); + return data.ret; + +} + + static int nvme_reopen_prepare(BDRVReopenState *reopen_state, BlockReopenQueue *queue, Error **errp) { @@ -1363,6 +1443,7 @@ static BlockDriver bdrv_nvme = { .bdrv_co_pwritev = nvme_co_pwritev, .bdrv_co_pwrite_zeroes = nvme_co_pwrite_zeroes, + .bdrv_co_pdiscard = nvme_co_pdiscard, .bdrv_co_flush_to_disk = nvme_co_flush, .bdrv_reopen_prepare = nvme_reopen_prepare, diff --git a/block/trace-events b/block/trace-events index 12f363bb44..f763f79d99 100644 --- a/block/trace-events +++ b/block/trace-events @@ -152,6 +152,8 @@ nvme_write_zeros(void *s, uint64_t offset, uint64_t bytes, int flags) "s %p offs nvme_qiov_unaligned(const void *qiov, int n, void *base, size_t size, int align) "qiov %p n %d base %p size 0x%zx align 0x%x" nvme_prw_buffered(void *s, uint64_t offset, uint64_t bytes, int niov, int is_write) "s %p offset %"PRId64" bytes %"PRId64" niov %d is_write %d" nvme_rw_done(void *s, int is_write, uint64_t offset, uint64_t bytes, int ret) "s %p is_write %d offset %"PRId64" bytes %"PRId64" ret %d" +nvme_dsm(void *s, uint64_t offset, uint64_t bytes) "s %p offset %"PRId64" bytes %"PRId64"" +nvme_dsm_done(void *s, uint64_t offset, uint64_t bytes, int ret) "s %p offset %"PRId64" bytes %"PRId64" ret %d" nvme_dma_map_flush(void *s) "s %p" nvme_free_req_queue_wait(void *q) "q %p" nvme_cmd_map_qiov(void *s, void *cmd, void *req, void *qiov, int entries) "s %p cmd %p req %p qiov %p entries %d"