From patchwork Sun Jan 24 02:54:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Minwoo Im X-Patchwork-Id: 1430855 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=P+CEUH6L; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4DNd683P32z9sW4 for ; Sun, 24 Jan 2021 14:00:32 +1100 (AEDT) Received: from localhost ([::1]:59612 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1l3Vde-0006Dx-2d for incoming@patchwork.ozlabs.org; Sat, 23 Jan 2021 22:00:30 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:55372) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1l3VYQ-0000Fm-Dq; Sat, 23 Jan 2021 21:55:06 -0500 Received: from mail-pj1-x102f.google.com ([2607:f8b0:4864:20::102f]:53442) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1l3VYO-0003fz-FL; Sat, 23 Jan 2021 21:55:06 -0500 Received: by mail-pj1-x102f.google.com with SMTP id p15so6230372pjv.3; Sat, 23 Jan 2021 18:55:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=sJy8j7so+eEZG2Um9lFCjm7LQCU9N8yZbI2MukMkgVs=; b=P+CEUH6LiphsA9n+aismd2rSYkiHahj3PjXx9Kv/u0kAJ4W3fcB50rLHJxGMJ6JEG4 7PmUkw5ji3zujsp5QrGnWntsb3NKZ/yYUGC/vnPgzLnJ6oSRBkifUtXR3jWmAMzPx3VL gJsCtekgyQsp9fL8EabikA9W/t1fAINJeBNnUmY0WyqMu5DFRVIKPAA/HE+jqomjh/On gKq/EQIoNYBuQa/7xbaojWQ0KuHoXsks5aIfSWsx87LTUahEiI65kRf8/WQ7/hSLxKAU f/8eW7Wt7y8R9dVpJZhBDG47gEYbfXgKM3T2JPVLMxEiaaiYdNbnfouJ0HEcVcnDHtDf fcGw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=sJy8j7so+eEZG2Um9lFCjm7LQCU9N8yZbI2MukMkgVs=; b=mdeOnoYWmqklp1v8BgmhFIY0ffxGinQBv9iSaLXjH8rhPjhVTsXjjLp5Zya8zVpE07 aZgEOR0CUBpwbdSpkUrQLRiXo4odVRZuzxrYhlJoQkkEYGomQ5mx0Bwh5tCO5itDA2TB 3bBZ7Q7DFaYJh46cEhEuJ2FR8ZAYGP1bJh8T54IikELDBR4kGvB6WSjHv7byBsR/4lL8 i+lbrNv68KXSNumEAlgGyYRCq5wFdbLdCUuxddj9AT2G/zIJdoZPObehwMIWrEniV1V9 j8ghv70JVDg7U3acckWjtWp8cKL2ujo7PppL45OxtZcQu3cY/jnvy6sQkOykJv5z/Vnm yvAQ== X-Gm-Message-State: AOAM532l2RrcU2B2AV+PbHP4Hcc+Vc9/qPyYXq+0QWc6/rFu1E8gqMZQ 4MXK0GF3K3ZHeFy+RtP5pAvh877Mff5XWg== X-Google-Smtp-Source: ABdhPJy/RPXwn9outj4TngUVAj0z15G57R2Eh9BOt3bXYjjfrv6Ca1tlwbeloqgRcqg6NeUNZWq3uA== X-Received: by 2002:a17:90a:ad01:: with SMTP id r1mr14775873pjq.197.1611456902688; Sat, 23 Jan 2021 18:55:02 -0800 (PST) Received: from localhost.localdomain ([211.108.35.36]) by smtp.gmail.com with ESMTPSA id 77sm314842pfx.130.2021.01.23.18.55.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 23 Jan 2021 18:55:02 -0800 (PST) From: Minwoo Im To: qemu-devel@nongnu.org, qemu-block@nongnu.org Subject: [PATCH V6 1/6] hw/block/nvme: introduce nvme-subsys device Date: Sun, 24 Jan 2021 11:54:45 +0900 Message-Id: <20210124025450.11071-2-minwoo.im.dev@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210124025450.11071-1-minwoo.im.dev@gmail.com> References: <20210124025450.11071-1-minwoo.im.dev@gmail.com> Received-SPF: pass client-ip=2607:f8b0:4864:20::102f; envelope-from=minwoo.im.dev@gmail.com; helo=mail-pj1-x102f.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Keith Busch , Klaus Jensen , Minwoo Im , Kevin Wolf , Max Reitz Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" To support multi-path in QEMU NVMe device model, We need to have NVMe subsystem hierarchy to map controllers and namespaces to a NVMe subsystem. This patch introduced a simple nvme-subsys device model. The subsystem will be prepared with subsystem NQN with provided in nvme-subsys device: ex) -device nvme-subsys,id=subsys0: nqn.2019-08.org.qemu:subsys0 Signed-off-by: Minwoo Im --- hw/block/meson.build | 2 +- hw/block/nvme-subsys.c | 60 ++++++++++++++++++++++++++++++++++++++++++ hw/block/nvme-subsys.h | 25 ++++++++++++++++++ hw/block/nvme.c | 3 +++ 4 files changed, 89 insertions(+), 1 deletion(-) create mode 100644 hw/block/nvme-subsys.c create mode 100644 hw/block/nvme-subsys.h diff --git a/hw/block/meson.build b/hw/block/meson.build index 602ca6c8541d..83ea2d37978d 100644 --- a/hw/block/meson.build +++ b/hw/block/meson.build @@ -13,7 +13,7 @@ softmmu_ss.add(when: 'CONFIG_SSI_M25P80', if_true: files('m25p80.c')) softmmu_ss.add(when: 'CONFIG_SWIM', if_true: files('swim.c')) softmmu_ss.add(when: 'CONFIG_XEN', if_true: files('xen-block.c')) softmmu_ss.add(when: 'CONFIG_SH4', if_true: files('tc58128.c')) -softmmu_ss.add(when: 'CONFIG_NVME_PCI', if_true: files('nvme.c', 'nvme-ns.c')) +softmmu_ss.add(when: 'CONFIG_NVME_PCI', if_true: files('nvme.c', 'nvme-ns.c', 'nvme-subsys.c')) specific_ss.add(when: 'CONFIG_VIRTIO_BLK', if_true: files('virtio-blk.c')) specific_ss.add(when: 'CONFIG_VHOST_USER_BLK', if_true: files('vhost-user-blk.c')) diff --git a/hw/block/nvme-subsys.c b/hw/block/nvme-subsys.c new file mode 100644 index 000000000000..aa82911b951c --- /dev/null +++ b/hw/block/nvme-subsys.c @@ -0,0 +1,60 @@ +/* + * QEMU NVM Express Subsystem: nvme-subsys + * + * Copyright (c) 2021 Minwoo Im + * + * This code is licensed under the GNU GPL v2. Refer COPYING. + */ + +#include "qemu/units.h" +#include "qemu/osdep.h" +#include "qemu/uuid.h" +#include "qemu/iov.h" +#include "qemu/cutils.h" +#include "qapi/error.h" +#include "hw/qdev-properties.h" +#include "hw/qdev-core.h" +#include "hw/block/block.h" +#include "block/aio.h" +#include "block/accounting.h" +#include "sysemu/sysemu.h" +#include "hw/pci/pci.h" +#include "nvme.h" +#include "nvme-subsys.h" + +static void nvme_subsys_setup(NvmeSubsystem *subsys) +{ + snprintf((char *)subsys->subnqn, sizeof(subsys->subnqn), + "nqn.2019-08.org.qemu:%s", subsys->parent_obj.id); +} + +static void nvme_subsys_realize(DeviceState *dev, Error **errp) +{ + NvmeSubsystem *subsys = NVME_SUBSYS(dev); + + nvme_subsys_setup(subsys); +} + +static void nvme_subsys_class_init(ObjectClass *oc, void *data) +{ + DeviceClass *dc = DEVICE_CLASS(oc); + + set_bit(DEVICE_CATEGORY_STORAGE, dc->categories); + + dc->realize = nvme_subsys_realize; + dc->desc = "Virtual NVMe subsystem"; +} + +static const TypeInfo nvme_subsys_info = { + .name = TYPE_NVME_SUBSYS, + .parent = TYPE_DEVICE, + .class_init = nvme_subsys_class_init, + .instance_size = sizeof(NvmeSubsystem), +}; + +static void nvme_subsys_register_types(void) +{ + type_register_static(&nvme_subsys_info); +} + +type_init(nvme_subsys_register_types) diff --git a/hw/block/nvme-subsys.h b/hw/block/nvme-subsys.h new file mode 100644 index 000000000000..40f06a4c7db0 --- /dev/null +++ b/hw/block/nvme-subsys.h @@ -0,0 +1,25 @@ +/* + * QEMU NVM Express Subsystem: nvme-subsys + * + * Copyright (c) 2021 Minwoo Im + * + * This code is licensed under the GNU GPL v2. Refer COPYING. + */ + +#ifndef NVME_SUBSYS_H +#define NVME_SUBSYS_H + +#define TYPE_NVME_SUBSYS "nvme-subsys" +#define NVME_SUBSYS(obj) \ + OBJECT_CHECK(NvmeSubsystem, (obj), TYPE_NVME_SUBSYS) + +#define NVME_SUBSYS_MAX_CTRLS 32 + +typedef struct NvmeCtrl NvmeCtrl; +typedef struct NvmeNamespace NvmeNamespace; +typedef struct NvmeSubsystem { + DeviceState parent_obj; + uint8_t subnqn[256]; +} NvmeSubsystem; + +#endif /* NVME_SUBSYS_H */ diff --git a/hw/block/nvme.c b/hw/block/nvme.c index 21aec90637fa..aabccdf36f4b 100644 --- a/hw/block/nvme.c +++ b/hw/block/nvme.c @@ -25,6 +25,7 @@ * mdts=,zoned.append_size_limit= \ * -device nvme-ns,drive=,bus=,nsid=,\ * zoned= + * -device nvme-subsys,id= * * Note cmb_size_mb denotes size of CMB in MB. CMB is assumed to be at * offset 0 in BAR2 and supports only WDS, RDS and SQS for now. By default, the @@ -38,6 +39,8 @@ * * The PMR will use BAR 4/5 exclusively. * + * To place controller(s) and namespace(s) to a subsystem, then provide + * nvme-subsys device as above. * * nvme device parameters * ~~~~~~~~~~~~~~~~~~~~~~ From patchwork Sun Jan 24 02:54:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Minwoo Im X-Patchwork-Id: 1430851 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=oAMTgFZY; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4DNd2F5M7Pz9sW4 for ; Sun, 24 Jan 2021 13:57:09 +1100 (AEDT) Received: from localhost ([::1]:47140 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1l3VaN-0000vc-LB for incoming@patchwork.ozlabs.org; Sat, 23 Jan 2021 21:57:07 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:55384) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1l3VYS-0000Lt-V8; Sat, 23 Jan 2021 21:55:09 -0500 Received: from mail-pj1-x1029.google.com ([2607:f8b0:4864:20::1029]:37024) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1l3VYQ-0003hp-T3; Sat, 23 Jan 2021 21:55:08 -0500 Received: by mail-pj1-x1029.google.com with SMTP id g15so6376346pjd.2; Sat, 23 Jan 2021 18:55:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=YwBe4YYkS5JT7ZD/trCcY77b8jZIwtzVoLSSmJTJ9Po=; b=oAMTgFZYtC0X5fo09UtwlftvbsBs80UZPl83J4YF7zZ3DAIZuqcZ4ID3J+XRlEDmBW 3fsLHTK67x5e9dHHbKzcO1eGKnBWehnokc/BrxJrpKFll9ASB6/2L9+MIb2yQsGpv9FZ GBzrNRHS985bMj7LPYORg8iK5n4lB6z6j0xnH5EiZ3bsQFle/A+N9mH4hEoDnw7VB2l9 8Zgu3PcQkFkH/aBOya2BZP2MLUCEAEF+0VJYPUjLKtjJz0Of5+JsGLl7Wn4BxoO3bOol AThPoIGmF/e2kY6ZfG7MUY54ztclfDDTpIz4AlyjRrY4IjKsqoOhGdqxBNa4aQFh9V/I 8Wgg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=YwBe4YYkS5JT7ZD/trCcY77b8jZIwtzVoLSSmJTJ9Po=; b=h2EuSzJoo6RFiqZHBrT6/JcxZMSMcWZ72lEl2I1n8GlkTdKj7QYPipB7MPES49/WBg VdrZchltOOmFd6WPbaf8B4cKlE3ZUd8EtqIS/ojzue3ofC7i/qQYfwLTsIdNae3VvDxo qjQUIySPaAzfOi7fpewnHnnBa0Q/4vEgBZQPigsm6g4UStU3X3pRig/0lKQ0WuXe1We9 TwdTRCP7Lx7AZ1UhCOzDQoLNWwO4rHUH3Wdmgi4hJ78ujV9e97h27l7NbISYRaJJsugz upf+LHsrmdLacpcSjfiqKbmmHCV9Sdc/VHYFe3CrcXixI3Y0pypoRDS9aKWnJseGwvEu M9wQ== X-Gm-Message-State: AOAM532HyfkHTDSEBPtDctbVoO66jxCCrEq9nffARuBIXiOnJ2+70rFz dFNrsf/rx7LRbFJ02r1nqncTTs+VqCVNeQ== X-Google-Smtp-Source: ABdhPJzdxha8bzQ6gtsalAcHLkBect559mZzxo6ClM41mXyn0j9XyusNMmlAfl1mRXpW5LlFQxSPFA== X-Received: by 2002:a17:902:e741:b029:de:9cd1:35c8 with SMTP id p1-20020a170902e741b02900de9cd135c8mr12728767plf.18.1611456904964; Sat, 23 Jan 2021 18:55:04 -0800 (PST) Received: from localhost.localdomain ([211.108.35.36]) by smtp.gmail.com with ESMTPSA id 77sm314842pfx.130.2021.01.23.18.55.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 23 Jan 2021 18:55:04 -0800 (PST) From: Minwoo Im To: qemu-devel@nongnu.org, qemu-block@nongnu.org Subject: [PATCH V6 2/6] hw/block/nvme: support to map controller to a subsystem Date: Sun, 24 Jan 2021 11:54:46 +0900 Message-Id: <20210124025450.11071-3-minwoo.im.dev@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210124025450.11071-1-minwoo.im.dev@gmail.com> References: <20210124025450.11071-1-minwoo.im.dev@gmail.com> Received-SPF: pass client-ip=2607:f8b0:4864:20::1029; envelope-from=minwoo.im.dev@gmail.com; helo=mail-pj1-x1029.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Keith Busch , Klaus Jensen , Minwoo Im , Kevin Wolf , Max Reitz Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" nvme controller(nvme) can be mapped to a NVMe subsystem(nvme-subsys). This patch maps a controller to a subsystem by adding a parameter 'subsys' to the nvme device. To map a controller to a subsystem, we need to put nvme-subsys first and then maps the subsystem to the controller: -device nvme-subsys,id=subsys0 -device nvme,serial=foo,id=nvme0,subsys=subsys0 If 'subsys' property is not given to the nvme controller, then subsystem NQN will be created with serial (e.g., 'foo' in above example), Otherwise, it will be based on subsys id (e.g., 'subsys0' in above example). Signed-off-by: Minwoo Im --- hw/block/nvme.c | 30 +++++++++++++++++++++++++----- hw/block/nvme.h | 3 +++ 2 files changed, 28 insertions(+), 5 deletions(-) diff --git a/hw/block/nvme.c b/hw/block/nvme.c index aabccdf36f4b..b525fca14103 100644 --- a/hw/block/nvme.c +++ b/hw/block/nvme.c @@ -22,7 +22,8 @@ * [pmrdev=,] \ * max_ioqpairs=, \ * aerl=, aer_max_queued=, \ - * mdts=,zoned.append_size_limit= \ + * mdts=,zoned.append_size_limit=, \ + * subsys= \ * -device nvme-ns,drive=,bus=,nsid=,\ * zoned= * -device nvme-subsys,id= @@ -44,6 +45,13 @@ * * nvme device parameters * ~~~~~~~~~~~~~~~~~~~~~~ + * - `subsys` + * NVM Subsystem device. If given, a subsystem NQN will be initialized with + * given. Otherwise, will be taken for subsystem NQN. + * Also, it will enable multi controller capability represented in Identify + * Controller data structure in CMIC (Controller Multi-path I/O and Namesapce + * Sharing Capabilities), if given. + * * - `aerl` * The Asynchronous Event Request Limit (AERL). Indicates the maximum number * of concurrently outstanding Asynchronous Event Request commands support @@ -4404,11 +4412,23 @@ static int nvme_init_pci(NvmeCtrl *n, PCIDevice *pci_dev, Error **errp) return 0; } +static void nvme_init_subnqn(NvmeCtrl *n) +{ + NvmeSubsystem *subsys = n->subsys; + NvmeIdCtrl *id = &n->id_ctrl; + + if (!subsys) { + snprintf((char *)id->subnqn, sizeof(id->subnqn), + "nqn.2019-08.org.qemu:%s", n->params.serial); + } else { + pstrcpy((char *)id->subnqn, sizeof(id->subnqn), (char*)subsys->subnqn); + } +} + static void nvme_init_ctrl(NvmeCtrl *n, PCIDevice *pci_dev) { NvmeIdCtrl *id = &n->id_ctrl; uint8_t *pci_conf = pci_dev->config; - char *subnqn; id->vid = cpu_to_le16(pci_get_word(pci_conf + PCI_VENDOR_ID)); id->ssvid = cpu_to_le16(pci_get_word(pci_conf + PCI_SUBSYSTEM_VENDOR_ID)); @@ -4455,9 +4475,7 @@ static void nvme_init_ctrl(NvmeCtrl *n, PCIDevice *pci_dev) id->sgls = cpu_to_le32(NVME_CTRL_SGLS_SUPPORT_NO_ALIGN | NVME_CTRL_SGLS_BITBUCKET); - subnqn = g_strdup_printf("nqn.2019-08.org.qemu:%s", n->params.serial); - strpadcpy((char *)id->subnqn, sizeof(id->subnqn), subnqn, '\0'); - g_free(subnqn); + nvme_init_subnqn(n); id->psd[0].mp = cpu_to_le16(0x9c4); id->psd[0].enlat = cpu_to_le32(0x10); @@ -4545,6 +4563,8 @@ static Property nvme_props[] = { DEFINE_BLOCK_PROPERTIES(NvmeCtrl, namespace.blkconf), DEFINE_PROP_LINK("pmrdev", NvmeCtrl, pmr.dev, TYPE_MEMORY_BACKEND, HostMemoryBackend *), + DEFINE_PROP_LINK("subsys", NvmeCtrl, subsys, TYPE_NVME_SUBSYS, + NvmeSubsystem *), DEFINE_PROP_STRING("serial", NvmeCtrl, params.serial), DEFINE_PROP_UINT32("cmb_size_mb", NvmeCtrl, params.cmb_size_mb, 0), DEFINE_PROP_UINT32("num_queues", NvmeCtrl, params.num_queues, 0), diff --git a/hw/block/nvme.h b/hw/block/nvme.h index dee6092bd45f..04d4684601fd 100644 --- a/hw/block/nvme.h +++ b/hw/block/nvme.h @@ -2,6 +2,7 @@ #define HW_NVME_H #include "block/nvme.h" +#include "nvme-subsys.h" #include "nvme-ns.h" #define NVME_MAX_NAMESPACES 256 @@ -170,6 +171,8 @@ typedef struct NvmeCtrl { uint8_t zasl; + NvmeSubsystem *subsys; + NvmeNamespace namespace; NvmeNamespace *namespaces[NVME_MAX_NAMESPACES]; NvmeSQueue **sq; From patchwork Sun Jan 24 02:54:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Minwoo Im X-Patchwork-Id: 1430856 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=TFuvV/aw; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4DNd9R38fRz9sW4 for ; Sun, 24 Jan 2021 14:03:23 +1100 (AEDT) Received: from localhost ([::1]:36630 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1l3VgP-0000Cc-As for incoming@patchwork.ozlabs.org; Sat, 23 Jan 2021 22:03:21 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:55396) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1l3VYU-0000Mx-J0; Sat, 23 Jan 2021 21:55:10 -0500 Received: from mail-pl1-x629.google.com ([2607:f8b0:4864:20::629]:41634) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1l3VYS-0003i3-Vr; Sat, 23 Jan 2021 21:55:10 -0500 Received: by mail-pl1-x629.google.com with SMTP id h15so3045231pli.8; Sat, 23 Jan 2021 18:55:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=gq79wtNdkMEoBleQb6W3xZAzvk0xvFQyqyLanl+JX3c=; b=TFuvV/awoo+rp9WaAkpJPvN3hx4SevBVdVAfyrSASz/8INtpKjYJEmRg8hbx5kgKJB hmXKQfr3npmafMOzwobxzb08J5GkfXtSlObcrnXS9s4m0VMP1rKnx6JV+UG/e9vF4Xcp gTQcwtC7Eyy7iZnLdUKc/h4OaksXjhP6NvVN+B5c0Vt8Am4q5aw05/+lhm5kFAtnuXwY Dx5kipQ0tYabxlBu4nzCAa0jCdzc2ucO+vufKPDYBu4UyGR1wwGl6IdGH1zYY7+7GbpM uSSWWS/2C5h/HwpxV/kiDo0dIdvgaVFRpwNgKCHj6GggzGMV8a8cSEPOEgULEtFTxoP1 Dy/w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=gq79wtNdkMEoBleQb6W3xZAzvk0xvFQyqyLanl+JX3c=; b=GnD2FgNMDi4g61S+UIW5pyfaqLCkNQhqk9VbbqxvJntTDgMajW4f2n9guR5ufFy4Nl FnKdrbxq0bExoClYc6Wgws0pJRKupwYKigoH0lyF3L3phY6u4g40iJtbRH9sobFtsVPi RKbcqH1q3BtdFOW6uq7FrXlbPmqyadBaBsXdKG0unvBV0OCO0WC3dp+Y6zJ/8qaGqhbt QIDIzzB35OeMlWXmY7Oyn1SxeeP4fN6M5vTLmGYBk0n/joUt4xiBlF21s5nWlA7ZBCaC 3dc7GVPbaqpWxx7uu7Y5aVjPXLA+ExupFG1Lhp88h/Sd2XM44LJQEGF5t2glW78fs2wI vCiA== X-Gm-Message-State: AOAM532DQGUR4Q44vJFp4I28kkTceFlp+MajlFab1vq2ZIYLU93zTz6b +fr9fVSZ3BqzY4kwD4QenI4Pb0dAt8IShA== X-Google-Smtp-Source: ABdhPJz7GyjSe7TDLKmfROwceETBnrjtlbO40EJez2KQgWtKsQyVqxhsBepuxFhUg0r8S3jUIjFKgw== X-Received: by 2002:a17:903:181:b029:df:c7e5:8e39 with SMTP id z1-20020a1709030181b02900dfc7e58e39mr12632676plg.25.1611456907256; Sat, 23 Jan 2021 18:55:07 -0800 (PST) Received: from localhost.localdomain ([211.108.35.36]) by smtp.gmail.com with ESMTPSA id 77sm314842pfx.130.2021.01.23.18.55.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 23 Jan 2021 18:55:06 -0800 (PST) From: Minwoo Im To: qemu-devel@nongnu.org, qemu-block@nongnu.org Subject: [PATCH V6 3/6] hw/block/nvme: add CMIC enum value for Identify Controller Date: Sun, 24 Jan 2021 11:54:47 +0900 Message-Id: <20210124025450.11071-4-minwoo.im.dev@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210124025450.11071-1-minwoo.im.dev@gmail.com> References: <20210124025450.11071-1-minwoo.im.dev@gmail.com> Received-SPF: pass client-ip=2607:f8b0:4864:20::629; envelope-from=minwoo.im.dev@gmail.com; helo=mail-pl1-x629.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Keith Busch , Klaus Jensen , Minwoo Im , Kevin Wolf , Max Reitz Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Added Controller Multi-path I/O and Namespace Sharing Capabilities (CMIC) field to support multi-controller in the following patches. This field is in Identify Controller data structure in [76]. Signed-off-by: Minwoo Im --- include/block/nvme.h | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/include/block/nvme.h b/include/block/nvme.h index e4b918064df9..d6415a869c1c 100644 --- a/include/block/nvme.h +++ b/include/block/nvme.h @@ -1034,6 +1034,10 @@ enum NvmeIdCtrlLpa { NVME_LPA_EXTENDED = 1 << 2, }; +enum NvmeIdCtrlCmic { + NVME_CMIC_MULTI_CTRL = 1 << 1, +}; + #define NVME_CTRL_SQES_MIN(sqes) ((sqes) & 0xf) #define NVME_CTRL_SQES_MAX(sqes) (((sqes) >> 4) & 0xf) #define NVME_CTRL_CQES_MIN(cqes) ((cqes) & 0xf) From patchwork Sun Jan 24 02:54:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Minwoo Im X-Patchwork-Id: 1430857 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=Wk3tvhR1; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4DNdBC6PTmz9sW4 for ; Sun, 24 Jan 2021 14:04:03 +1100 (AEDT) Received: from localhost ([::1]:38138 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1l3Vh3-0000pS-Rb for incoming@patchwork.ozlabs.org; Sat, 23 Jan 2021 22:04:01 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:55412) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1l3VYZ-0000Qh-CS; Sat, 23 Jan 2021 21:55:15 -0500 Received: from mail-pg1-x535.google.com ([2607:f8b0:4864:20::535]:33247) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1l3VYW-0003j2-Uq; Sat, 23 Jan 2021 21:55:15 -0500 Received: by mail-pg1-x535.google.com with SMTP id n25so6627567pgb.0; Sat, 23 Jan 2021 18:55:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=DTj2aGM1/xjfCWZbhFHeSQReQnHkNso3QEjVPVOb3W4=; b=Wk3tvhR1rlp+6I4KgsIw2kbTXOKJnbP4ffDMPLdX+f+bvt+7q21I8bQAoY+ys7ejX7 JQ6gwB2uduKyGFOsqU3nBFSe/vwncFnBZ5MkPiELolMaePFREFg/NV/PSo4bi1P8+HsQ uih1jAPAwceeKMN2gaOhndsEI8WB1BYcvTw0siawBlAMmH7mLiNhYW8r4h/blfXwtoUg lRzpgllci3LusG01rE1Qf4xo6r3ymxsgdKn5vOZVVmo1/M/jHZ699StvRFN9levsVJ9D lUjz6OqWzPVFvNlL7dzMYJC309mPGz46nBGLUl5DkIM0Ar/S0K/QXVZ+oaofWt2Lz6Ys ++Og== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=DTj2aGM1/xjfCWZbhFHeSQReQnHkNso3QEjVPVOb3W4=; b=COuCYHx8rTCnmSGNk2CJSQVDH+mESiA6GlHRudg8Ij7jHSjhNIH69cy7ofhrTcwSIa b1nno5L9g6h1jm2ZOF4llGyE5wmuylIsg7XoXTBAGzfv4hu05h0zkPsjs+dZi3QBofb5 ww/0rzTa2scH8qHmfCBSxMr9FylWLOMAKssdp+is1kU8ZllGzhMQOMlRDa+EL1bzLohJ YYyP56ruUyTIdlwphSov+AcOCdHdi43k/17U4Gh7t1ZBtgCTnV+gFUyhhaFfn8/iy5lU mbS3iO9e13ZSdJ1cxlp7G7++wvcKZcsI3O30ia63t7xcm9j/z3ybtKn9rDYrgFBHt4hN hUdg== X-Gm-Message-State: AOAM533+MqPXnI+vVse44OIlGbRsO4+gne+sjFmaYbBvTMzqCENNfG9d +hhxVMGjJvTSh1aQIhAS5iWaFcWOMbD1Bg== X-Google-Smtp-Source: ABdhPJy9kOCM0z61zrfTqDEDFWaqoRD+bQgsrgSM5TIhOex4XdzezLaqLLk06YxT1HpUr4r1kYz9AA== X-Received: by 2002:a62:ee03:0:b029:1a9:cc29:7d1f with SMTP id e3-20020a62ee030000b02901a9cc297d1fmr138089pfi.24.1611456909563; Sat, 23 Jan 2021 18:55:09 -0800 (PST) Received: from localhost.localdomain ([211.108.35.36]) by smtp.gmail.com with ESMTPSA id 77sm314842pfx.130.2021.01.23.18.55.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 23 Jan 2021 18:55:09 -0800 (PST) From: Minwoo Im To: qemu-devel@nongnu.org, qemu-block@nongnu.org Subject: [PATCH V6 4/6] hw/block/nvme: support for multi-controller in subsystem Date: Sun, 24 Jan 2021 11:54:48 +0900 Message-Id: <20210124025450.11071-5-minwoo.im.dev@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210124025450.11071-1-minwoo.im.dev@gmail.com> References: <20210124025450.11071-1-minwoo.im.dev@gmail.com> Received-SPF: pass client-ip=2607:f8b0:4864:20::535; envelope-from=minwoo.im.dev@gmail.com; helo=mail-pg1-x535.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Keith Busch , Klaus Jensen , Minwoo Im , Kevin Wolf , Max Reitz Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" We have nvme-subsys and nvme devices mapped together. To support multi-controller scheme to this setup, controller identifier(id) has to be managed. Earlier, cntlid(controller id) used to be always 0 because we didn't have any subsystem scheme that controller id matters. This patch introduced 'cntlid' attribute to the nvme controller instance(NvmeCtrl) and make it allocated by the nvme-subsys device mapped to the controller. If nvme-subsys is not given to the controller, then it will always be 0 as it was. Added 'ctrls' array in the nvme-subsys instance to manage attached controllers to the subsystem with a limit(32). This patch didn't take list for the controllers to make it seamless with nvme-ns device. Signed-off-by: Minwoo Im --- hw/block/nvme-subsys.c | 21 +++++++++++++++++++++ hw/block/nvme-subsys.h | 4 ++++ hw/block/nvme.c | 29 +++++++++++++++++++++++++++++ hw/block/nvme.h | 1 + 4 files changed, 55 insertions(+) diff --git a/hw/block/nvme-subsys.c b/hw/block/nvme-subsys.c index aa82911b951c..e9d61c993c90 100644 --- a/hw/block/nvme-subsys.c +++ b/hw/block/nvme-subsys.c @@ -22,6 +22,27 @@ #include "nvme.h" #include "nvme-subsys.h" +int nvme_subsys_register_ctrl(NvmeCtrl *n, Error **errp) +{ + NvmeSubsystem *subsys = n->subsys; + int cntlid; + + for (cntlid = 0; cntlid < ARRAY_SIZE(subsys->ctrls); cntlid++) { + if (!subsys->ctrls[cntlid]) { + break; + } + } + + if (cntlid == ARRAY_SIZE(subsys->ctrls)) { + error_setg(errp, "no more free controller id"); + return -1; + } + + subsys->ctrls[cntlid] = n; + + return cntlid; +} + static void nvme_subsys_setup(NvmeSubsystem *subsys) { snprintf((char *)subsys->subnqn, sizeof(subsys->subnqn), diff --git a/hw/block/nvme-subsys.h b/hw/block/nvme-subsys.h index 40f06a4c7db0..4eba50d96a1d 100644 --- a/hw/block/nvme-subsys.h +++ b/hw/block/nvme-subsys.h @@ -20,6 +20,10 @@ typedef struct NvmeNamespace NvmeNamespace; typedef struct NvmeSubsystem { DeviceState parent_obj; uint8_t subnqn[256]; + + NvmeCtrl *ctrls[NVME_SUBSYS_MAX_CTRLS]; } NvmeSubsystem; +int nvme_subsys_register_ctrl(NvmeCtrl *n, Error **errp); + #endif /* NVME_SUBSYS_H */ diff --git a/hw/block/nvme.c b/hw/block/nvme.c index b525fca14103..7138389be4bd 100644 --- a/hw/block/nvme.c +++ b/hw/block/nvme.c @@ -4435,6 +4435,9 @@ static void nvme_init_ctrl(NvmeCtrl *n, PCIDevice *pci_dev) strpadcpy((char *)id->mn, sizeof(id->mn), "QEMU NVMe Ctrl", ' '); strpadcpy((char *)id->fr, sizeof(id->fr), "1.0", ' '); strpadcpy((char *)id->sn, sizeof(id->sn), n->params.serial, ' '); + + id->cntlid = cpu_to_le16(n->cntlid); + id->rab = 6; id->ieee[0] = 0x00; id->ieee[1] = 0x02; @@ -4481,6 +4484,10 @@ static void nvme_init_ctrl(NvmeCtrl *n, PCIDevice *pci_dev) id->psd[0].enlat = cpu_to_le32(0x10); id->psd[0].exlat = cpu_to_le32(0x4); + if (n->subsys) { + id->cmic |= NVME_CMIC_MULTI_CTRL; + } + NVME_CAP_SET_MQES(n->bar.cap, 0x7ff); NVME_CAP_SET_CQR(n->bar.cap, 1); NVME_CAP_SET_TO(n->bar.cap, 0xf); @@ -4495,6 +4502,24 @@ static void nvme_init_ctrl(NvmeCtrl *n, PCIDevice *pci_dev) n->bar.intmc = n->bar.intms = 0; } +static int nvme_init_subsys(NvmeCtrl *n, Error **errp) +{ + int cntlid; + + if (!n->subsys) { + return 0; + } + + cntlid = nvme_subsys_register_ctrl(n, errp); + if (cntlid < 0) { + return -1; + } + + n->cntlid = cntlid; + + return 0; +} + static void nvme_realize(PCIDevice *pci_dev, Error **errp) { NvmeCtrl *n = NVME(pci_dev); @@ -4515,6 +4540,10 @@ static void nvme_realize(PCIDevice *pci_dev, Error **errp) return; } + if (nvme_init_subsys(n, errp)) { + error_propagate(errp, local_err); + return; + } nvme_init_ctrl(n, pci_dev); /* setup a namespace if the controller drive property was given */ diff --git a/hw/block/nvme.h b/hw/block/nvme.h index 04d4684601fd..b8f5f2d6ffb8 100644 --- a/hw/block/nvme.h +++ b/hw/block/nvme.h @@ -134,6 +134,7 @@ typedef struct NvmeCtrl { NvmeBus bus; BlockConf conf; + uint16_t cntlid; bool qs_created; uint32_t page_size; uint16_t page_bits; From patchwork Sun Jan 24 02:54:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Minwoo Im X-Patchwork-Id: 1430858 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=QiFGTGfv; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4DNdDr41mnz9sW4 for ; Sun, 24 Jan 2021 14:06:20 +1100 (AEDT) Received: from localhost ([::1]:43060 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1l3VjF-0002xI-BJ for incoming@patchwork.ozlabs.org; Sat, 23 Jan 2021 22:06:17 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:55436) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1l3VYc-0000T1-28; Sat, 23 Jan 2021 21:55:19 -0500 Received: from mail-pj1-x1034.google.com ([2607:f8b0:4864:20::1034]:36024) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1l3VYY-0003jD-IX; Sat, 23 Jan 2021 21:55:17 -0500 Received: by mail-pj1-x1034.google.com with SMTP id gx1so659125pjb.1; Sat, 23 Jan 2021 18:55:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=Lkc05bhxPEaurDuvorYROlMZGsONdTZ57rQZ1y9IAE0=; b=QiFGTGfvAiUlc9YJMxM4xb0EfsY44lW9T2VCWzwYoGimvJSZLb0ZVz1mFPk1l+QaON yaixGviK8ggOz9Xj8ww1WVXTv9cYtieTpVtn0NyRFI5+vdXnU6Q4OHmuBxyewRE4FI4H nD25C+1a25E5C3BOv82Q7twqLOm1RllITlNkxglsp2HyrdKS+oTYPOgIxxJKHnbJE0x2 JPNXokaVeYaG/xUMN/LiwoVMUs+QLqaY73tLDvfKaxH/J5x49MY+xSqymzmMlmRg8dIE 8aOaUkP5eIoxs6AH7Rw5wMt0xhtOf2sFWhea4TO/TyQvKGRjyK3nCxuh0FSPNeMkdhKR lPUw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=Lkc05bhxPEaurDuvorYROlMZGsONdTZ57rQZ1y9IAE0=; b=I0KRQElJaQyet+kixtQZwZeWjcw66eloTs+C005yKYe1+i5XtzrvjW9CylQ3sHxbg5 CXFBPS1qiehTVsniS6qqif7HSDfgKzRWET7cIfzi48KVM26GJkudVjnrXRkvqRuAwTc/ J0r3BF13Mo0ARcKIpoQ9XfzqPm7VF5akbfnX6gYPNL6k464yiTCA9dCHhEN+T26KHLoB y7YuRWDVXjFW6/K64Trzr4Y2UIrsn+x9q3Lva973KwMjmPIK295AXis8CGAxR+cWJMRo dGAsqEqKKyOFRRaeM0ENdrbzXcePwouuYcKUXM7H+9yiVROj7Xg0RkH8MXW5NgpdEYME MsDA== X-Gm-Message-State: AOAM533LyiMIbV+geJMtrGFbEi/eZQNfKe93xSOhNmb4afYBPg8aji4U kbeONpoNZldpZVsyrPYi20tf9RWK5Uskgw== X-Google-Smtp-Source: ABdhPJzEVuRZLlN90nt7AdTS8A4j8fWPOT7ECnL2LzHsBe2SE1B110wngtVVSS3z9LbfRcxUZtCHuA== X-Received: by 2002:a17:90a:5993:: with SMTP id l19mr121428pji.203.1611456911891; Sat, 23 Jan 2021 18:55:11 -0800 (PST) Received: from localhost.localdomain ([211.108.35.36]) by smtp.gmail.com with ESMTPSA id 77sm314842pfx.130.2021.01.23.18.55.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 23 Jan 2021 18:55:11 -0800 (PST) From: Minwoo Im To: qemu-devel@nongnu.org, qemu-block@nongnu.org Subject: [PATCH V6 5/6] hw/block/nvme: add NMIC enum value for Identify Namespace Date: Sun, 24 Jan 2021 11:54:49 +0900 Message-Id: <20210124025450.11071-6-minwoo.im.dev@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210124025450.11071-1-minwoo.im.dev@gmail.com> References: <20210124025450.11071-1-minwoo.im.dev@gmail.com> Received-SPF: pass client-ip=2607:f8b0:4864:20::1034; envelope-from=minwoo.im.dev@gmail.com; helo=mail-pj1-x1034.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Keith Busch , Klaus Jensen , Minwoo Im , Kevin Wolf , Max Reitz Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Added Namespace Multi-path I/O and Namespace Sharing Capabilities (NMIC) field to support shared namespace from controller(s). This field is in Identify Namespace data structure in [30]. Signed-off-by: Minwoo Im --- include/block/nvme.h | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/include/block/nvme.h b/include/block/nvme.h index d6415a869c1c..ad68cdc2b92d 100644 --- a/include/block/nvme.h +++ b/include/block/nvme.h @@ -1203,6 +1203,10 @@ enum NvmeNsIdentifierType { NVME_NIDT_CSI = 0x04, }; +enum NvmeIdNsNmic { + NVME_NMIC_NS_SHARED = 1 << 0, +}; + enum NvmeCsi { NVME_CSI_NVM = 0x00, NVME_CSI_ZONED = 0x02, From patchwork Sun Jan 24 02:54:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Minwoo Im X-Patchwork-Id: 1430859 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=r8MUMjTI; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4DNdGd2SfHz9sWK for ; Sun, 24 Jan 2021 14:07:53 +1100 (AEDT) Received: from localhost ([::1]:47450 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1l3Vkl-0004nJ-BB for incoming@patchwork.ozlabs.org; Sat, 23 Jan 2021 22:07:51 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:55446) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1l3VYe-0000Td-Vd; Sat, 23 Jan 2021 21:55:23 -0500 Received: from mail-pj1-x102a.google.com ([2607:f8b0:4864:20::102a]:54306) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1l3VYa-0003kG-39; Sat, 23 Jan 2021 21:55:19 -0500 Received: by mail-pj1-x102a.google.com with SMTP id cq1so6220518pjb.4; Sat, 23 Jan 2021 18:55:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=iwWbqZVh/h3mcBCrlvmjw4AKW/qVCosULhSCkOcZ/8c=; b=r8MUMjTINTanxDD2peQ2KAThEwwSqaK1TgeT0Z3CaWgIbYGzY9XJMmQ2pI8IwRkU/s 9tiQaoTB7oMCZhqnRYzasfMwJiFAFoKwYZRWvwvRmK6MK3Z10utqZ0ZzUKgpDuDTtV0L UjDjTrnmCNEAHDBJdCQBANnYi/l0AHw29PNneREPGlMbOf+5r6eh38SRUJOyjIm9c/pK zLGlBQudUSV8FuIlWFvaX0wpMhDNK5vIfN06E/06N/CTYOsiMmbLpFTIqFur8WpihTlA uEJf2P/APQZA/m668rCzK5GaGnxjLxcEtUPWnjADF5pDpB6vhR2v+vP8hyKq67+w3thb Ln2Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=iwWbqZVh/h3mcBCrlvmjw4AKW/qVCosULhSCkOcZ/8c=; b=CzrV1XemomHwM51lxuXenWjiO3fr+xDZj1vmXyQZfzuqljpqcTJG6D8YmXrUKL9ID4 sKUi78YrMLESdlZQWL87CVoGdQepvSomiqQy/anb8Rc1vx9xhJGGQAcBY9ScNuq1blQu moPRqX3yx8hb+efw+25qDh57lrCjiAQsDxYNFn3TuWmwtqQXRUuIQCw5r/eJ3uSGGPP7 JMyr0Lp3w0bt5X5YbeO0tflCqleViO4TdIfoVWbRTV/AYTt9cdKLepOvVgEfyTcxLTyz v6gTHWDuffUznzjljzTSFsC4HuJZ9rOoSYGF6K7vxLqjHcDVGCSE+7dUS2I2wPWJ7yCt 4mDQ== X-Gm-Message-State: AOAM5308A75dS5kkwiyZmYjrHA+89/XXw8rO7ybxsgCaUILCIUj5Ry6T BNQD4y+E4Z5/A+QHvIaUUNGW7v+xx6i4VA== X-Google-Smtp-Source: ABdhPJyBUMO6j2+TLHgmpvAUxChpir1Z3h9Cj9biK9ZTkrKrdnKyXRraYebzYNOgoDGj/Y76V09TFw== X-Received: by 2002:a17:903:22c2:b029:dd:f952:e341 with SMTP id y2-20020a17090322c2b02900ddf952e341mr3600211plg.67.1611456914285; Sat, 23 Jan 2021 18:55:14 -0800 (PST) Received: from localhost.localdomain ([211.108.35.36]) by smtp.gmail.com with ESMTPSA id 77sm314842pfx.130.2021.01.23.18.55.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 23 Jan 2021 18:55:13 -0800 (PST) From: Minwoo Im To: qemu-devel@nongnu.org, qemu-block@nongnu.org Subject: [PATCH V6 6/6] hw/block/nvme: support for shared namespace in subsystem Date: Sun, 24 Jan 2021 11:54:50 +0900 Message-Id: <20210124025450.11071-7-minwoo.im.dev@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210124025450.11071-1-minwoo.im.dev@gmail.com> References: <20210124025450.11071-1-minwoo.im.dev@gmail.com> Received-SPF: pass client-ip=2607:f8b0:4864:20::102a; envelope-from=minwoo.im.dev@gmail.com; helo=mail-pj1-x102a.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Keith Busch , Klaus Jensen , Minwoo Im , Kevin Wolf , Max Reitz Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" nvme-ns device is registered to a nvme controller device during the initialization in nvme_register_namespace() in case that 'bus' property is given which means it's mapped to a single controller. This patch introduced a new property 'subsys' just like the controller device instance did to map a namespace to a NVMe subsystem. If 'subsys' property is given to the nvme-ns device, it will belong to the specified subsystem and will be attached to all controllers in that subsystem by enabling shared namespace capability in NMIC(Namespace Multi-path I/O and Namespace Capabilities) in Identify Namespace. Usage: -device nvme-subsys,id=subsys0 -device nvme,serial=foo,id=nvme0,subsys=subsys0 -device nvme,serial=bar,id=nvme1,subsys=subsys0 -device nvme,serial=baz,id=nvme2,subsys=subsys0 -device nvme-ns,id=ns1,drive=,nsid=1,subsys=subsys0 # Shared -device nvme-ns,id=ns2,drive=,nsid=2,bus=nvme2 # Non-shared In the above example, 'ns1' will be shared to 'nvme0' and 'nvme1' in the same subsystem. On the other hand, 'ns2' will be attached to the 'nvme2' only as a private namespace in that subsystem. All the namespace with 'subsys' parameter will attach all controllers in the subsystem to the namespace by default. Signed-off-by: Minwoo Im --- hw/block/nvme-ns.c | 23 ++++++++++++++++++----- hw/block/nvme-ns.h | 7 +++++++ hw/block/nvme-subsys.c | 25 +++++++++++++++++++++++++ hw/block/nvme-subsys.h | 3 +++ hw/block/nvme.c | 10 +++++++++- 5 files changed, 62 insertions(+), 6 deletions(-) diff --git a/hw/block/nvme-ns.c b/hw/block/nvme-ns.c index 62b25cf69bfa..9b493f2ead03 100644 --- a/hw/block/nvme-ns.c +++ b/hw/block/nvme-ns.c @@ -63,6 +63,10 @@ static int nvme_ns_init(NvmeNamespace *ns, Error **errp) id_ns->npda = id_ns->npdg = npdg - 1; + if (nvme_ns_shared(ns)) { + id_ns->nmic |= NVME_NMIC_NS_SHARED; + } + return 0; } @@ -365,16 +369,25 @@ static void nvme_ns_realize(DeviceState *dev, Error **errp) return; } - if (nvme_register_namespace(n, ns, errp)) { - error_propagate_prepend(errp, local_err, - "could not register namespace: "); - return; + if (ns->subsys) { + if (nvme_subsys_register_ns(ns, errp)) { + error_propagate_prepend(errp, local_err, + "could not setup namespace to subsys: "); + return; + } + } else { + if (nvme_register_namespace(n, ns, errp)) { + error_propagate_prepend(errp, local_err, + "could not register namespace: "); + return; + } } - } static Property nvme_ns_props[] = { DEFINE_BLOCK_PROPERTIES(NvmeNamespace, blkconf), + DEFINE_PROP_LINK("subsys", NvmeNamespace, subsys, TYPE_NVME_SUBSYS, + NvmeSubsystem *), DEFINE_PROP_UINT32("nsid", NvmeNamespace, params.nsid, 0), DEFINE_PROP_UUID("uuid", NvmeNamespace, params.uuid), DEFINE_PROP_BOOL("zoned", NvmeNamespace, params.zoned, false), diff --git a/hw/block/nvme-ns.h b/hw/block/nvme-ns.h index 293ac990e3f6..929e78861903 100644 --- a/hw/block/nvme-ns.h +++ b/hw/block/nvme-ns.h @@ -47,6 +47,8 @@ typedef struct NvmeNamespace { const uint32_t *iocs; uint8_t csi; + NvmeSubsystem *subsys; + NvmeIdNsZoned *id_ns_zoned; NvmeZone *zone_array; QTAILQ_HEAD(, NvmeZone) exp_open_zones; @@ -77,6 +79,11 @@ static inline uint32_t nvme_nsid(NvmeNamespace *ns) return -1; } +static inline bool nvme_ns_shared(NvmeNamespace *ns) +{ + return !!ns->subsys; +} + static inline NvmeLBAF *nvme_ns_lbaf(NvmeNamespace *ns) { NvmeIdNs *id_ns = &ns->id_ns; diff --git a/hw/block/nvme-subsys.c b/hw/block/nvme-subsys.c index e9d61c993c90..641de33e99fc 100644 --- a/hw/block/nvme-subsys.c +++ b/hw/block/nvme-subsys.c @@ -43,6 +43,31 @@ int nvme_subsys_register_ctrl(NvmeCtrl *n, Error **errp) return cntlid; } +int nvme_subsys_register_ns(NvmeNamespace *ns, Error **errp) +{ + NvmeSubsystem *subsys = ns->subsys; + NvmeCtrl *n; + int i; + + if (subsys->namespaces[nvme_nsid(ns)]) { + error_setg(errp, "namespace %d already registerd to subsy %s", + nvme_nsid(ns), subsys->parent_obj.id); + return -1; + } + + subsys->namespaces[nvme_nsid(ns)] = ns; + + for (i = 0; i < ARRAY_SIZE(subsys->ctrls); i++) { + n = subsys->ctrls[i]; + + if (n && nvme_register_namespace(n, ns, errp)) { + return -1; + } + } + + return 0; +} + static void nvme_subsys_setup(NvmeSubsystem *subsys) { snprintf((char *)subsys->subnqn, sizeof(subsys->subnqn), diff --git a/hw/block/nvme-subsys.h b/hw/block/nvme-subsys.h index 4eba50d96a1d..ccf6a71398d3 100644 --- a/hw/block/nvme-subsys.h +++ b/hw/block/nvme-subsys.h @@ -14,6 +14,7 @@ OBJECT_CHECK(NvmeSubsystem, (obj), TYPE_NVME_SUBSYS) #define NVME_SUBSYS_MAX_CTRLS 32 +#define NVME_SUBSYS_MAX_NAMESPACES 32 typedef struct NvmeCtrl NvmeCtrl; typedef struct NvmeNamespace NvmeNamespace; @@ -22,8 +23,10 @@ typedef struct NvmeSubsystem { uint8_t subnqn[256]; NvmeCtrl *ctrls[NVME_SUBSYS_MAX_CTRLS]; + NvmeNamespace *namespaces[NVME_SUBSYS_MAX_NAMESPACES]; } NvmeSubsystem; int nvme_subsys_register_ctrl(NvmeCtrl *n, Error **errp); +int nvme_subsys_register_ns(NvmeNamespace *ns, Error **errp); #endif /* NVME_SUBSYS_H */ diff --git a/hw/block/nvme.c b/hw/block/nvme.c index 7138389be4bd..8259dbf48ec0 100644 --- a/hw/block/nvme.c +++ b/hw/block/nvme.c @@ -25,7 +25,8 @@ * mdts=,zoned.append_size_limit=, \ * subsys= \ * -device nvme-ns,drive=,bus=,nsid=,\ - * zoned= + * zoned=, \ + * subsys= * -device nvme-subsys,id= * * Note cmb_size_mb denotes size of CMB in MB. CMB is assumed to be at @@ -70,6 +71,13 @@ * data size being in effect. By setting this property to 0, users can make * ZASL to be equal to MDTS. This property only affects zoned namespaces. * + * nvme namespace device parameters + * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + * - `subsys` + * NVM Subsystem device. If given, this namespace will be attached to all + * controllers in the subsystem. Otherwise, `bus` must be given to attach + * this namespace to a specified single controller as a non-shared namespace. + * * Setting `zoned` to true selects Zoned Command Set at the namespace. * In this case, the following namespace properties are available to configure * zoned operation: