From patchwork Fri Jan 22 12:07:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Minwoo Im X-Patchwork-Id: 1430327 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=DHMKwkbq; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4DMdN20qN3z9s2g for ; Fri, 22 Jan 2021 23:09:06 +1100 (AEDT) Received: from localhost ([::1]:46904 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1l2vFQ-0008TE-1p for incoming@patchwork.ozlabs.org; Fri, 22 Jan 2021 07:09:04 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:42364) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1l2vEV-0008P4-CC; Fri, 22 Jan 2021 07:08:07 -0500 Received: from mail-pg1-x52f.google.com ([2607:f8b0:4864:20::52f]:46094) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1l2vEK-0001t5-0T; Fri, 22 Jan 2021 07:08:05 -0500 Received: by mail-pg1-x52f.google.com with SMTP id c22so3525699pgg.13; Fri, 22 Jan 2021 04:07:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=sJy8j7so+eEZG2Um9lFCjm7LQCU9N8yZbI2MukMkgVs=; b=DHMKwkbqJuopmA6kdf3W6R3uyo+gGVr6IuMIjmBFGK1odRGBapAlBf91ieau4XVgVD 8PLb0LPU0OjKvs2i++NTEEtcZd/BPyhVkcyj7TrQcr+dq4a5v4zblLkksf78cn0zCtDp 0evhJCxJwkTwz3iMwwHJqtCcS7R3JYRG08tM7JY2Rb1UDEye1sPNIybaBljTukHP8gxo jEYzGiEpIB+BxwkPshfAEWxjSB5Yj50zQoja+yE+UJvdKLOqKXFlyT7aLGM+EPq5FwEU +V+fO3cxcb3r+t4vbJvSl1YatblMTr9AvMooJY0qIR9NFuhfroRpnmC0WWwwFOrKAAy/ FOcg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=sJy8j7so+eEZG2Um9lFCjm7LQCU9N8yZbI2MukMkgVs=; b=irV20xgMO7TjOWCgoNRbNEtA/QIzc2Zp+uT7rdHS7xDIGasqfR3h6AC2LU2BMP5yW0 3rpTQhuD/ImBtqjqASfHpxRBr7z7jbDj8GgM7PiMQ4VSt3pcFxHoPFF73gTwYDE3rFFt Rnhqtckqb8ANFIWcNnpwcG55VDERLYxxT6sPWj/HMv02SdFsnRoIRrAlkOrIXncP16sU bJDRwK9aw6ctUlI8wRJDxlRq435Noqz2ye19kUaYGGGgf1mfbJMgSZiJ4/basWiPUedd xHWabKHiVBvGCPIuE8HSWxjm2DniZjo7qxa0/mazBr6k3y9eCszaODpzZbrsGWgx1kJO H43w== X-Gm-Message-State: AOAM531kPx0OUO6n6sXXYsPCX0PjhEmUeZgTT7w/QrQGcgz2sf27Gm0L l8CdBG7S3rjHwrZs9cCAaU1q98m5+IdSSw== X-Google-Smtp-Source: ABdhPJyi40H9B6EQFywoUGbj6EYVlY292ibsO6wdzHPZCG6dlXKEPdioHvDewnzoSR/LsR/uUNMntg== X-Received: by 2002:a62:25c7:0:b029:156:72a3:b0c0 with SMTP id l190-20020a6225c70000b029015672a3b0c0mr4413943pfl.59.1611317273754; Fri, 22 Jan 2021 04:07:53 -0800 (PST) Received: from localhost.localdomain ([211.108.35.36]) by smtp.gmail.com with ESMTPSA id mv17sm9104440pjb.17.2021.01.22.04.07.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 22 Jan 2021 04:07:53 -0800 (PST) From: Minwoo Im To: qemu-devel@nongnu.org, qemu-block@nongnu.org Subject: [PATCH V5 1/6] hw/block/nvme: introduce nvme-subsys device Date: Fri, 22 Jan 2021 21:07:31 +0900 Message-Id: <20210122120736.5242-2-minwoo.im.dev@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210122120736.5242-1-minwoo.im.dev@gmail.com> References: <20210122120736.5242-1-minwoo.im.dev@gmail.com> Received-SPF: pass client-ip=2607:f8b0:4864:20::52f; envelope-from=minwoo.im.dev@gmail.com; helo=mail-pg1-x52f.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Keith Busch , Klaus Jensen , Minwoo Im , Kevin Wolf , Max Reitz Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" To support multi-path in QEMU NVMe device model, We need to have NVMe subsystem hierarchy to map controllers and namespaces to a NVMe subsystem. This patch introduced a simple nvme-subsys device model. The subsystem will be prepared with subsystem NQN with provided in nvme-subsys device: ex) -device nvme-subsys,id=subsys0: nqn.2019-08.org.qemu:subsys0 Signed-off-by: Minwoo Im --- hw/block/meson.build | 2 +- hw/block/nvme-subsys.c | 60 ++++++++++++++++++++++++++++++++++++++++++ hw/block/nvme-subsys.h | 25 ++++++++++++++++++ hw/block/nvme.c | 3 +++ 4 files changed, 89 insertions(+), 1 deletion(-) create mode 100644 hw/block/nvme-subsys.c create mode 100644 hw/block/nvme-subsys.h diff --git a/hw/block/meson.build b/hw/block/meson.build index 602ca6c8541d..83ea2d37978d 100644 --- a/hw/block/meson.build +++ b/hw/block/meson.build @@ -13,7 +13,7 @@ softmmu_ss.add(when: 'CONFIG_SSI_M25P80', if_true: files('m25p80.c')) softmmu_ss.add(when: 'CONFIG_SWIM', if_true: files('swim.c')) softmmu_ss.add(when: 'CONFIG_XEN', if_true: files('xen-block.c')) softmmu_ss.add(when: 'CONFIG_SH4', if_true: files('tc58128.c')) -softmmu_ss.add(when: 'CONFIG_NVME_PCI', if_true: files('nvme.c', 'nvme-ns.c')) +softmmu_ss.add(when: 'CONFIG_NVME_PCI', if_true: files('nvme.c', 'nvme-ns.c', 'nvme-subsys.c')) specific_ss.add(when: 'CONFIG_VIRTIO_BLK', if_true: files('virtio-blk.c')) specific_ss.add(when: 'CONFIG_VHOST_USER_BLK', if_true: files('vhost-user-blk.c')) diff --git a/hw/block/nvme-subsys.c b/hw/block/nvme-subsys.c new file mode 100644 index 000000000000..aa82911b951c --- /dev/null +++ b/hw/block/nvme-subsys.c @@ -0,0 +1,60 @@ +/* + * QEMU NVM Express Subsystem: nvme-subsys + * + * Copyright (c) 2021 Minwoo Im + * + * This code is licensed under the GNU GPL v2. Refer COPYING. + */ + +#include "qemu/units.h" +#include "qemu/osdep.h" +#include "qemu/uuid.h" +#include "qemu/iov.h" +#include "qemu/cutils.h" +#include "qapi/error.h" +#include "hw/qdev-properties.h" +#include "hw/qdev-core.h" +#include "hw/block/block.h" +#include "block/aio.h" +#include "block/accounting.h" +#include "sysemu/sysemu.h" +#include "hw/pci/pci.h" +#include "nvme.h" +#include "nvme-subsys.h" + +static void nvme_subsys_setup(NvmeSubsystem *subsys) +{ + snprintf((char *)subsys->subnqn, sizeof(subsys->subnqn), + "nqn.2019-08.org.qemu:%s", subsys->parent_obj.id); +} + +static void nvme_subsys_realize(DeviceState *dev, Error **errp) +{ + NvmeSubsystem *subsys = NVME_SUBSYS(dev); + + nvme_subsys_setup(subsys); +} + +static void nvme_subsys_class_init(ObjectClass *oc, void *data) +{ + DeviceClass *dc = DEVICE_CLASS(oc); + + set_bit(DEVICE_CATEGORY_STORAGE, dc->categories); + + dc->realize = nvme_subsys_realize; + dc->desc = "Virtual NVMe subsystem"; +} + +static const TypeInfo nvme_subsys_info = { + .name = TYPE_NVME_SUBSYS, + .parent = TYPE_DEVICE, + .class_init = nvme_subsys_class_init, + .instance_size = sizeof(NvmeSubsystem), +}; + +static void nvme_subsys_register_types(void) +{ + type_register_static(&nvme_subsys_info); +} + +type_init(nvme_subsys_register_types) diff --git a/hw/block/nvme-subsys.h b/hw/block/nvme-subsys.h new file mode 100644 index 000000000000..40f06a4c7db0 --- /dev/null +++ b/hw/block/nvme-subsys.h @@ -0,0 +1,25 @@ +/* + * QEMU NVM Express Subsystem: nvme-subsys + * + * Copyright (c) 2021 Minwoo Im + * + * This code is licensed under the GNU GPL v2. Refer COPYING. + */ + +#ifndef NVME_SUBSYS_H +#define NVME_SUBSYS_H + +#define TYPE_NVME_SUBSYS "nvme-subsys" +#define NVME_SUBSYS(obj) \ + OBJECT_CHECK(NvmeSubsystem, (obj), TYPE_NVME_SUBSYS) + +#define NVME_SUBSYS_MAX_CTRLS 32 + +typedef struct NvmeCtrl NvmeCtrl; +typedef struct NvmeNamespace NvmeNamespace; +typedef struct NvmeSubsystem { + DeviceState parent_obj; + uint8_t subnqn[256]; +} NvmeSubsystem; + +#endif /* NVME_SUBSYS_H */ diff --git a/hw/block/nvme.c b/hw/block/nvme.c index 21aec90637fa..aabccdf36f4b 100644 --- a/hw/block/nvme.c +++ b/hw/block/nvme.c @@ -25,6 +25,7 @@ * mdts=,zoned.append_size_limit= \ * -device nvme-ns,drive=,bus=,nsid=,\ * zoned= + * -device nvme-subsys,id= * * Note cmb_size_mb denotes size of CMB in MB. CMB is assumed to be at * offset 0 in BAR2 and supports only WDS, RDS and SQS for now. By default, the @@ -38,6 +39,8 @@ * * The PMR will use BAR 4/5 exclusively. * + * To place controller(s) and namespace(s) to a subsystem, then provide + * nvme-subsys device as above. * * nvme device parameters * ~~~~~~~~~~~~~~~~~~~~~~ From patchwork Fri Jan 22 12:07:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Minwoo Im X-Patchwork-Id: 1430332 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=QFguUmHb; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4DMdSX4TJyz9s2g for ; Fri, 22 Jan 2021 23:13:00 +1100 (AEDT) Received: from localhost ([::1]:55086 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1l2vJC-0004GD-JQ for incoming@patchwork.ozlabs.org; Fri, 22 Jan 2021 07:12:58 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:42368) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1l2vEV-0008PV-CS; Fri, 22 Jan 2021 07:08:07 -0500 Received: from mail-pg1-x52e.google.com ([2607:f8b0:4864:20::52e]:40084) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1l2vEP-0001u5-P0; Fri, 22 Jan 2021 07:08:07 -0500 Received: by mail-pg1-x52e.google.com with SMTP id 15so3546890pgx.7; Fri, 22 Jan 2021 04:07:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=YwBe4YYkS5JT7ZD/trCcY77b8jZIwtzVoLSSmJTJ9Po=; b=QFguUmHbRCLDd9Bzlki01BtEQolBg4Q9WWzgFGvalMRo7O+RbenzDWLX5INmsuliRk X9jTuHq14vizCrDm6Qo9EJDZ51RN94xsc2CpWkOYlHhsJdjfiaDbFVI+v19Ag54OGDyc 1vSX0UsZrPJmRuvlHl2QhtTT3pXCF9BGbWdr2TmSdpR6whE4FaASmY+kLbkEbzRprxl0 1udnK65iK2YatrcTkGVFTe7Ld46tQMZezM3gDu9R5PNpcK/xaVvkH2oGMDYl5I5agZEo gyj/f+bmsZBnDHZxe9GJv2W0A8MJgwq4uzhqqgVVttIzLYWT3ZFm08cbbagTa7XvblBO ClfQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=YwBe4YYkS5JT7ZD/trCcY77b8jZIwtzVoLSSmJTJ9Po=; b=QTXlQ0TZZNcFMMEhf+0fWXd/oSkqsTY68iB6LR2Qy8GihW/1Y7hkSpPoHKIwZfgAKR oCKZvKV7bluxqX54feI3Kt1xnn/fCWVGzYXgBU/a8ML/E9SIyGgNwmgfMYQeRv7QZlZR Q5L2fTsaLm9HA0fgVYV1R+h2q25B+jTHWLtOuzIho56ACZbpBiD55veij+9J+Nv2aIQz 5aY2r0pZPNpblML0SzwrsQDuw/1X1vEndmYhA9W1y5PhdSrUlBG2I6qRAK+aY5ETe4vh js6L0nDDqskqdSVnYVJj0Z78XkXHAVZjrAOBBVmJKcXSSPwpXw4axnHDLRWxHx3GCHIC HfeA== X-Gm-Message-State: AOAM532kAqYh/tXsyIAk1vKl3KuhdZCSZouFzytOg56fq1xbnWy/qutV r4AeCggJg1FvSq89f/+iTJ+ZcoSbnuR5tA== X-Google-Smtp-Source: ABdhPJxX3z9a8Oevf/3DuvbA5ObDbSyHCyp1+caBfncxHLkbQZBUCA2j92uLf5OLAaRdb+SiRdOflg== X-Received: by 2002:a62:838d:0:b029:1ba:9b85:2eac with SMTP id h135-20020a62838d0000b02901ba9b852eacmr4458892pfe.36.1611317276553; Fri, 22 Jan 2021 04:07:56 -0800 (PST) Received: from localhost.localdomain ([211.108.35.36]) by smtp.gmail.com with ESMTPSA id mv17sm9104440pjb.17.2021.01.22.04.07.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 22 Jan 2021 04:07:55 -0800 (PST) From: Minwoo Im To: qemu-devel@nongnu.org, qemu-block@nongnu.org Subject: [PATCH V5 2/6] hw/block/nvme: support to map controller to a subsystem Date: Fri, 22 Jan 2021 21:07:32 +0900 Message-Id: <20210122120736.5242-3-minwoo.im.dev@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210122120736.5242-1-minwoo.im.dev@gmail.com> References: <20210122120736.5242-1-minwoo.im.dev@gmail.com> Received-SPF: pass client-ip=2607:f8b0:4864:20::52e; envelope-from=minwoo.im.dev@gmail.com; helo=mail-pg1-x52e.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Keith Busch , Klaus Jensen , Minwoo Im , Kevin Wolf , Max Reitz Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" nvme controller(nvme) can be mapped to a NVMe subsystem(nvme-subsys). This patch maps a controller to a subsystem by adding a parameter 'subsys' to the nvme device. To map a controller to a subsystem, we need to put nvme-subsys first and then maps the subsystem to the controller: -device nvme-subsys,id=subsys0 -device nvme,serial=foo,id=nvme0,subsys=subsys0 If 'subsys' property is not given to the nvme controller, then subsystem NQN will be created with serial (e.g., 'foo' in above example), Otherwise, it will be based on subsys id (e.g., 'subsys0' in above example). Signed-off-by: Minwoo Im --- hw/block/nvme.c | 30 +++++++++++++++++++++++++----- hw/block/nvme.h | 3 +++ 2 files changed, 28 insertions(+), 5 deletions(-) diff --git a/hw/block/nvme.c b/hw/block/nvme.c index aabccdf36f4b..b525fca14103 100644 --- a/hw/block/nvme.c +++ b/hw/block/nvme.c @@ -22,7 +22,8 @@ * [pmrdev=,] \ * max_ioqpairs=, \ * aerl=, aer_max_queued=, \ - * mdts=,zoned.append_size_limit= \ + * mdts=,zoned.append_size_limit=, \ + * subsys= \ * -device nvme-ns,drive=,bus=,nsid=,\ * zoned= * -device nvme-subsys,id= @@ -44,6 +45,13 @@ * * nvme device parameters * ~~~~~~~~~~~~~~~~~~~~~~ + * - `subsys` + * NVM Subsystem device. If given, a subsystem NQN will be initialized with + * given. Otherwise, will be taken for subsystem NQN. + * Also, it will enable multi controller capability represented in Identify + * Controller data structure in CMIC (Controller Multi-path I/O and Namesapce + * Sharing Capabilities), if given. + * * - `aerl` * The Asynchronous Event Request Limit (AERL). Indicates the maximum number * of concurrently outstanding Asynchronous Event Request commands support @@ -4404,11 +4412,23 @@ static int nvme_init_pci(NvmeCtrl *n, PCIDevice *pci_dev, Error **errp) return 0; } +static void nvme_init_subnqn(NvmeCtrl *n) +{ + NvmeSubsystem *subsys = n->subsys; + NvmeIdCtrl *id = &n->id_ctrl; + + if (!subsys) { + snprintf((char *)id->subnqn, sizeof(id->subnqn), + "nqn.2019-08.org.qemu:%s", n->params.serial); + } else { + pstrcpy((char *)id->subnqn, sizeof(id->subnqn), (char*)subsys->subnqn); + } +} + static void nvme_init_ctrl(NvmeCtrl *n, PCIDevice *pci_dev) { NvmeIdCtrl *id = &n->id_ctrl; uint8_t *pci_conf = pci_dev->config; - char *subnqn; id->vid = cpu_to_le16(pci_get_word(pci_conf + PCI_VENDOR_ID)); id->ssvid = cpu_to_le16(pci_get_word(pci_conf + PCI_SUBSYSTEM_VENDOR_ID)); @@ -4455,9 +4475,7 @@ static void nvme_init_ctrl(NvmeCtrl *n, PCIDevice *pci_dev) id->sgls = cpu_to_le32(NVME_CTRL_SGLS_SUPPORT_NO_ALIGN | NVME_CTRL_SGLS_BITBUCKET); - subnqn = g_strdup_printf("nqn.2019-08.org.qemu:%s", n->params.serial); - strpadcpy((char *)id->subnqn, sizeof(id->subnqn), subnqn, '\0'); - g_free(subnqn); + nvme_init_subnqn(n); id->psd[0].mp = cpu_to_le16(0x9c4); id->psd[0].enlat = cpu_to_le32(0x10); @@ -4545,6 +4563,8 @@ static Property nvme_props[] = { DEFINE_BLOCK_PROPERTIES(NvmeCtrl, namespace.blkconf), DEFINE_PROP_LINK("pmrdev", NvmeCtrl, pmr.dev, TYPE_MEMORY_BACKEND, HostMemoryBackend *), + DEFINE_PROP_LINK("subsys", NvmeCtrl, subsys, TYPE_NVME_SUBSYS, + NvmeSubsystem *), DEFINE_PROP_STRING("serial", NvmeCtrl, params.serial), DEFINE_PROP_UINT32("cmb_size_mb", NvmeCtrl, params.cmb_size_mb, 0), DEFINE_PROP_UINT32("num_queues", NvmeCtrl, params.num_queues, 0), diff --git a/hw/block/nvme.h b/hw/block/nvme.h index dee6092bd45f..04d4684601fd 100644 --- a/hw/block/nvme.h +++ b/hw/block/nvme.h @@ -2,6 +2,7 @@ #define HW_NVME_H #include "block/nvme.h" +#include "nvme-subsys.h" #include "nvme-ns.h" #define NVME_MAX_NAMESPACES 256 @@ -170,6 +171,8 @@ typedef struct NvmeCtrl { uint8_t zasl; + NvmeSubsystem *subsys; + NvmeNamespace namespace; NvmeNamespace *namespaces[NVME_MAX_NAMESPACES]; NvmeSQueue **sq; From patchwork Fri Jan 22 12:07:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Minwoo Im X-Patchwork-Id: 1430328 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=eB60VHeM; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4DMdN72pJ5z9sS8 for ; Fri, 22 Jan 2021 23:09:11 +1100 (AEDT) Received: from localhost ([::1]:47414 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1l2vFV-0000Jq-9h for incoming@patchwork.ozlabs.org; Fri, 22 Jan 2021 07:09:09 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:42398) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1l2vEW-0008US-Rh; Fri, 22 Jan 2021 07:08:08 -0500 Received: from mail-pg1-x52a.google.com ([2607:f8b0:4864:20::52a]:37168) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1l2vER-0001ur-3Y; Fri, 22 Jan 2021 07:08:08 -0500 Received: by mail-pg1-x52a.google.com with SMTP id z21so3562487pgj.4; Fri, 22 Jan 2021 04:08:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=gq79wtNdkMEoBleQb6W3xZAzvk0xvFQyqyLanl+JX3c=; b=eB60VHeMgiayMqvcPMxVYb7zNXdNTCajrlM0JCGl8msaYM0HpTQHdYMRo3jPrqs142 gy+Bpg0en0tqCckqNMmTDgEaUMEQruYi2J6FVZb0OJY+4wN+lcQzVHBFrXZGKkN+sn23 PZLtUU9szttMUWuXx80Rv2Z2yxi5nyn6IG4Yne+kXjGjshbC+zIbY0QGcrrQOdHi6/uK /BJk+w7+HYBrDNYFyMlT362JCPIIq01wWUGse/+oSVF+rrKHDa0teK0HRssPtSDiVib6 JIon8z9Z72OuMNBEOjP+GyEmlPHxNsh4mutVtxdWaM0Dru9M/G/nWBBbt9vOJZ6tVgr7 TMaQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=gq79wtNdkMEoBleQb6W3xZAzvk0xvFQyqyLanl+JX3c=; b=s1Xxcie3biZml4PQRC6eoLZMiIInfVUYXN45iLPl39JAHUq1AlUvgM7d5YjUvfsOHV 3FLy2QSCSKczAt7GO+hrOiwJkA9cEgEcF9H3q293Sap28FXriPLa0atlH0wMLIJBJuuB 0WbMPOK3WGddtRAAbhJrdjxXde4UoD5vQpysKdrfqiN65rSJrLq0nWHKdlBeZOlyhlwN 2c2z/POI5Q2Qj3bnOzcrBgod5Yqe2bTz+HsaSm35oInSnYlKRRmajz7NtHx9j/W6BhqS UFlv+iOlYuO4Q5KZGT1vDFA0rqx7vPUY7LkwbFZcHBTwfPMDmZTKWr/niMrtFcwrLES0 ihgg== X-Gm-Message-State: AOAM532KepttqN4KR9Elv3HLFbM9SFFYFQ+ULRUN+GCkfcDnJoSDuMYz bUboES4Kfuqwn1XWWNHJr1mr6aGWXVhcog== X-Google-Smtp-Source: ABdhPJwFiaY7EnsPSjcglCUlteONHKED7Jm3ZyZkomPFlw5Y345+w/sZR/NtTjCx/RrlcRKZMCk6Ig== X-Received: by 2002:a62:794f:0:b029:1b4:59d1:df8d with SMTP id u76-20020a62794f0000b02901b459d1df8dmr4471583pfc.14.1611317279163; Fri, 22 Jan 2021 04:07:59 -0800 (PST) Received: from localhost.localdomain ([211.108.35.36]) by smtp.gmail.com with ESMTPSA id mv17sm9104440pjb.17.2021.01.22.04.07.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 22 Jan 2021 04:07:58 -0800 (PST) From: Minwoo Im To: qemu-devel@nongnu.org, qemu-block@nongnu.org Subject: [PATCH V5 3/6] hw/block/nvme: add CMIC enum value for Identify Controller Date: Fri, 22 Jan 2021 21:07:33 +0900 Message-Id: <20210122120736.5242-4-minwoo.im.dev@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210122120736.5242-1-minwoo.im.dev@gmail.com> References: <20210122120736.5242-1-minwoo.im.dev@gmail.com> Received-SPF: pass client-ip=2607:f8b0:4864:20::52a; envelope-from=minwoo.im.dev@gmail.com; helo=mail-pg1-x52a.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Keith Busch , Klaus Jensen , Minwoo Im , Kevin Wolf , Max Reitz Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Added Controller Multi-path I/O and Namespace Sharing Capabilities (CMIC) field to support multi-controller in the following patches. This field is in Identify Controller data structure in [76]. Signed-off-by: Minwoo Im --- include/block/nvme.h | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/include/block/nvme.h b/include/block/nvme.h index e4b918064df9..d6415a869c1c 100644 --- a/include/block/nvme.h +++ b/include/block/nvme.h @@ -1034,6 +1034,10 @@ enum NvmeIdCtrlLpa { NVME_LPA_EXTENDED = 1 << 2, }; +enum NvmeIdCtrlCmic { + NVME_CMIC_MULTI_CTRL = 1 << 1, +}; + #define NVME_CTRL_SQES_MIN(sqes) ((sqes) & 0xf) #define NVME_CTRL_SQES_MAX(sqes) (((sqes) >> 4) & 0xf) #define NVME_CTRL_CQES_MIN(cqes) ((cqes) & 0xf) From patchwork Fri Jan 22 12:07:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Minwoo Im X-Patchwork-Id: 1430330 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=jYdA/nen; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4DMdR60CvSz9s2g for ; Fri, 22 Jan 2021 23:11:46 +1100 (AEDT) Received: from localhost ([::1]:51662 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1l2vHu-0002dC-0q for incoming@patchwork.ozlabs.org; Fri, 22 Jan 2021 07:11:38 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:42382) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1l2vEV-0008RJ-QP; Fri, 22 Jan 2021 07:08:07 -0500 Received: from mail-pf1-x431.google.com ([2607:f8b0:4864:20::431]:41757) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1l2vER-0001v1-Kb; Fri, 22 Jan 2021 07:08:07 -0500 Received: by mail-pf1-x431.google.com with SMTP id q20so3542063pfu.8; Fri, 22 Jan 2021 04:08:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=UyIGV4RaYW8+BtNKgF8fPnPcHXbvYoyUV1nS/YXwuoE=; b=jYdA/nenv5oY/qG2kU1fZef+VbSXAT8GEkN9U8M00gvXVVBqTETZCYMqT0NxukDIWr 7zYwBNhnL44cZIe6KIKqQ6pOdmFI5aDKne2hfeTCqmUEQ1LScA6dPOJODgiRYuzopW93 y06I9YtLybibI3EkD3YVNRNpPh/jtwphz2NTw7qODzU03Onq+oao4a7DkBDLy0JkVSQh /bNfFGeBCeV2i0VqDvyi+/6EOXT+LF+ChCG2l1g4AW91jG2KCWjx/dJhhyNWnC1xp0q4 EvBDD3TUlrf1mSmj2OxPP8nawr4x9C/8ICnsHvnbwM2EVI3KFwdqHX9DBZ5I4V04XRuq YGMg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=UyIGV4RaYW8+BtNKgF8fPnPcHXbvYoyUV1nS/YXwuoE=; b=r2sH8BKFSX407PD9cTcQx8qg1umxhIvTG82vVivtCJHz1hUtc79+9YuVisRUB2IsDO ovw80l3EjTCWVGYJlOBv2bwYqj4LWkJwBWE/qiVkJ/JYiTEa7SB3LOouq8jSQuvaO9JE z0r29JpiIcNMD/87g7yAy1qELn/9I7ajW5xCfluuNBuc9876j33DKIjXMq1ZlVAW7Yi2 v9s+mSqJHyzzKUz42aGp15ZaA+/UQ0q3x9vYoQx0E4w/uhsAMLG3mGtr0RCok2yRepNo mxW6RE5nyQdgl7C0AU9nHED38dM3mBkGa+Kq1pab1crbCDBbNAiNxbmuvkxl0AtiipB4 DTag== X-Gm-Message-State: AOAM531vQtABPD9FAGAUM5W528N5dWFnOQBM1vXU2v5aBpJGCBzzDEQL d+z7NCAGKZPXqxcBzV9wgykLOqQ5mB4zdw== X-Google-Smtp-Source: ABdhPJxjKxSvihhABLv56I3DwdRRN4JwmTMJ0nPcIU2VtqA/kDK4X9v2yrwHI2O4QHkwfDLPZmTXPw== X-Received: by 2002:a62:2e86:0:b029:1a6:5f94:2cb with SMTP id u128-20020a622e860000b02901a65f9402cbmr4770257pfu.19.1611317281652; Fri, 22 Jan 2021 04:08:01 -0800 (PST) Received: from localhost.localdomain ([211.108.35.36]) by smtp.gmail.com with ESMTPSA id mv17sm9104440pjb.17.2021.01.22.04.07.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 22 Jan 2021 04:08:01 -0800 (PST) From: Minwoo Im To: qemu-devel@nongnu.org, qemu-block@nongnu.org Subject: [PATCH V5 4/6] hw/block/nvme: support for multi-controller in subsystem Date: Fri, 22 Jan 2021 21:07:34 +0900 Message-Id: <20210122120736.5242-5-minwoo.im.dev@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210122120736.5242-1-minwoo.im.dev@gmail.com> References: <20210122120736.5242-1-minwoo.im.dev@gmail.com> Received-SPF: pass client-ip=2607:f8b0:4864:20::431; envelope-from=minwoo.im.dev@gmail.com; helo=mail-pf1-x431.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Keith Busch , Klaus Jensen , Minwoo Im , Kevin Wolf , Max Reitz Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" We have nvme-subsys and nvme devices mapped together. To support multi-controller scheme to this setup, controller identifier(id) has to be managed. Earlier, cntlid(controller id) used to be always 0 because we didn't have any subsystem scheme that controller id matters. This patch introduced 'cntlid' attribute to the nvme controller instance(NvmeCtrl) and make it allocated by the nvme-subsys device mapped to the controller. If nvme-subsys is not given to the controller, then it will always be 0 as it was. Added 'ctrls' array in the nvme-subsys instance to manage attached controllers to the subsystem with a limit(32). This patch didn't take list for the controllers to make it seamless with nvme-ns device. Signed-off-by: Minwoo Im --- hw/block/nvme-subsys.c | 21 +++++++++++++++++++++ hw/block/nvme-subsys.h | 4 ++++ hw/block/nvme.c | 29 +++++++++++++++++++++++++++++ hw/block/nvme.h | 1 + 4 files changed, 55 insertions(+) diff --git a/hw/block/nvme-subsys.c b/hw/block/nvme-subsys.c index aa82911b951c..e9d61c993c90 100644 --- a/hw/block/nvme-subsys.c +++ b/hw/block/nvme-subsys.c @@ -22,6 +22,27 @@ #include "nvme.h" #include "nvme-subsys.h" +int nvme_subsys_register_ctrl(NvmeCtrl *n, Error **errp) +{ + NvmeSubsystem *subsys = n->subsys; + int cntlid; + + for (cntlid = 0; cntlid < ARRAY_SIZE(subsys->ctrls); cntlid++) { + if (!subsys->ctrls[cntlid]) { + break; + } + } + + if (cntlid == ARRAY_SIZE(subsys->ctrls)) { + error_setg(errp, "no more free controller id"); + return -1; + } + + subsys->ctrls[cntlid] = n; + + return cntlid; +} + static void nvme_subsys_setup(NvmeSubsystem *subsys) { snprintf((char *)subsys->subnqn, sizeof(subsys->subnqn), diff --git a/hw/block/nvme-subsys.h b/hw/block/nvme-subsys.h index 40f06a4c7db0..4eba50d96a1d 100644 --- a/hw/block/nvme-subsys.h +++ b/hw/block/nvme-subsys.h @@ -20,6 +20,10 @@ typedef struct NvmeNamespace NvmeNamespace; typedef struct NvmeSubsystem { DeviceState parent_obj; uint8_t subnqn[256]; + + NvmeCtrl *ctrls[NVME_SUBSYS_MAX_CTRLS]; } NvmeSubsystem; +int nvme_subsys_register_ctrl(NvmeCtrl *n, Error **errp); + #endif /* NVME_SUBSYS_H */ diff --git a/hw/block/nvme.c b/hw/block/nvme.c index b525fca14103..3dedefb8ebba 100644 --- a/hw/block/nvme.c +++ b/hw/block/nvme.c @@ -4435,6 +4435,9 @@ static void nvme_init_ctrl(NvmeCtrl *n, PCIDevice *pci_dev) strpadcpy((char *)id->mn, sizeof(id->mn), "QEMU NVMe Ctrl", ' '); strpadcpy((char *)id->fr, sizeof(id->fr), "1.0", ' '); strpadcpy((char *)id->sn, sizeof(id->sn), n->params.serial, ' '); + + id->cntlid = n->cntlid; + id->rab = 6; id->ieee[0] = 0x00; id->ieee[1] = 0x02; @@ -4481,6 +4484,10 @@ static void nvme_init_ctrl(NvmeCtrl *n, PCIDevice *pci_dev) id->psd[0].enlat = cpu_to_le32(0x10); id->psd[0].exlat = cpu_to_le32(0x4); + if (n->subsys) { + id->cmic |= NVME_CMIC_MULTI_CTRL; + } + NVME_CAP_SET_MQES(n->bar.cap, 0x7ff); NVME_CAP_SET_CQR(n->bar.cap, 1); NVME_CAP_SET_TO(n->bar.cap, 0xf); @@ -4495,6 +4502,24 @@ static void nvme_init_ctrl(NvmeCtrl *n, PCIDevice *pci_dev) n->bar.intmc = n->bar.intms = 0; } +static int nvme_init_subsys(NvmeCtrl *n, Error **errp) +{ + int cntlid; + + if (!n->subsys) { + return 0; + } + + cntlid = nvme_subsys_register_ctrl(n, errp); + if (cntlid < 0) { + return -1; + } + + n->cntlid = cntlid; + + return 0; +} + static void nvme_realize(PCIDevice *pci_dev, Error **errp) { NvmeCtrl *n = NVME(pci_dev); @@ -4515,6 +4540,10 @@ static void nvme_realize(PCIDevice *pci_dev, Error **errp) return; } + if (nvme_init_subsys(n, errp)) { + error_propagate(errp, local_err); + return; + } nvme_init_ctrl(n, pci_dev); /* setup a namespace if the controller drive property was given */ diff --git a/hw/block/nvme.h b/hw/block/nvme.h index 04d4684601fd..b8f5f2d6ffb8 100644 --- a/hw/block/nvme.h +++ b/hw/block/nvme.h @@ -134,6 +134,7 @@ typedef struct NvmeCtrl { NvmeBus bus; BlockConf conf; + uint16_t cntlid; bool qs_created; uint32_t page_size; uint16_t page_bits; From patchwork Fri Jan 22 12:07:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Minwoo Im X-Patchwork-Id: 1430333 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=peK59r+D; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4DMdTX6gRhz9s2g for ; Fri, 22 Jan 2021 23:13:52 +1100 (AEDT) Received: from localhost ([::1]:59492 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1l2vK2-00067g-RQ for incoming@patchwork.ozlabs.org; Fri, 22 Jan 2021 07:13:50 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:42394) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1l2vEW-0008Tp-Js; Fri, 22 Jan 2021 07:08:08 -0500 Received: from mail-pl1-x633.google.com ([2607:f8b0:4864:20::633]:34163) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1l2vET-0001vp-RV; Fri, 22 Jan 2021 07:08:08 -0500 Received: by mail-pl1-x633.google.com with SMTP id t6so3091585plq.1; Fri, 22 Jan 2021 04:08:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=Lkc05bhxPEaurDuvorYROlMZGsONdTZ57rQZ1y9IAE0=; b=peK59r+DyPmQsmELzHNYcddiwPw7jzm4o7xdUohKOuLhPwAOahOMQ6sCM+VEdbr3ie X0CmoYbGTaCyGoIaQTulJ2Ds2KDcq1lo1kIL4fROQRMQtdjnHoIWMfnqp8+souXlI5Kw 5EERzynrFs0kGF1uMY3TpNVlwh7LuWs6z8Hgfic0hkvez7Sw43l9RE/U/6MrXp5BpPjS dDekNgBX6jNS8OtYV92DgmohOHw6wu9i9VDF1dO2ZHj1vyZqON3gMgw7XYWu7/2ldW3e GaTAi9fpFyWiJqDwDiHjjjtLMtyKm8wyl77hsRzMGYttmyxvw0agEFscwHx7aJgNm0Wg jeXg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=Lkc05bhxPEaurDuvorYROlMZGsONdTZ57rQZ1y9IAE0=; b=BcNr4PWeL6ETlKoW0hc14PseLJz+a9JCeOGvfkoZGY8kdPSvxJxU9L0BU5jSanvyRU w03TrBdheIkmrVJSDJmyYbKTNG6z49Ydh98YBpDVDqh0y09/dxrbxxgOyuknTjFVXk5C PLElXX2nTxbW+UVyOc0ItQw0CFN5EanzOk5WKTTqMqlVT1rulND73Z6tyQt7bBqNAqxp aPZYp1xRV8uuoxcAZbVBkHFsZ2lVrQi/bnSfaU25udOpWUe2nh4USDXoH6geeQYh3mXd eyE57WixE9lYzeT00Ai1IPepPU9V/jUOy5sPmOl9O0sToYKWi1S8SdfgIRjIGQ4p8ti3 XAYw== X-Gm-Message-State: AOAM533W6BI+WN1nmqQllIzqNB/rL+8R2LiLRl/+A1TkgzxEnZpDIKWT VSW0Hz5jLo53h8lGJktvZsOM9qFtJBc1IQ== X-Google-Smtp-Source: ABdhPJxxyZfTbAQVk7EWku3FmKPQds6B9G9Z+TfxXygLO2/vJ+Cvj4zboDbWWutT6wTmyxACzjm7ng== X-Received: by 2002:a17:90b:228f:: with SMTP id kx15mr5207817pjb.32.1611317284147; Fri, 22 Jan 2021 04:08:04 -0800 (PST) Received: from localhost.localdomain ([211.108.35.36]) by smtp.gmail.com with ESMTPSA id mv17sm9104440pjb.17.2021.01.22.04.08.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 22 Jan 2021 04:08:03 -0800 (PST) From: Minwoo Im To: qemu-devel@nongnu.org, qemu-block@nongnu.org Subject: [PATCH V5 5/6] hw/block/nvme: add NMIC enum value for Identify Namespace Date: Fri, 22 Jan 2021 21:07:35 +0900 Message-Id: <20210122120736.5242-6-minwoo.im.dev@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210122120736.5242-1-minwoo.im.dev@gmail.com> References: <20210122120736.5242-1-minwoo.im.dev@gmail.com> Received-SPF: pass client-ip=2607:f8b0:4864:20::633; envelope-from=minwoo.im.dev@gmail.com; helo=mail-pl1-x633.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Keith Busch , Klaus Jensen , Minwoo Im , Kevin Wolf , Max Reitz Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Added Namespace Multi-path I/O and Namespace Sharing Capabilities (NMIC) field to support shared namespace from controller(s). This field is in Identify Namespace data structure in [30]. Signed-off-by: Minwoo Im --- include/block/nvme.h | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/include/block/nvme.h b/include/block/nvme.h index d6415a869c1c..ad68cdc2b92d 100644 --- a/include/block/nvme.h +++ b/include/block/nvme.h @@ -1203,6 +1203,10 @@ enum NvmeNsIdentifierType { NVME_NIDT_CSI = 0x04, }; +enum NvmeIdNsNmic { + NVME_NMIC_NS_SHARED = 1 << 0, +}; + enum NvmeCsi { NVME_CSI_NVM = 0x00, NVME_CSI_ZONED = 0x02, From patchwork Fri Jan 22 12:07:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Minwoo Im X-Patchwork-Id: 1430331 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=aNcPqt3W; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4DMdSL60gMz9s2g for ; Fri, 22 Jan 2021 23:12:50 +1100 (AEDT) Received: from localhost ([::1]:54944 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1l2vJ2-0004Ci-Nq for incoming@patchwork.ozlabs.org; Fri, 22 Jan 2021 07:12:48 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:42430) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1l2vEY-00007w-I5; Fri, 22 Jan 2021 07:08:10 -0500 Received: from mail-pl1-x636.google.com ([2607:f8b0:4864:20::636]:34166) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1l2vEW-0001wv-GK; Fri, 22 Jan 2021 07:08:10 -0500 Received: by mail-pl1-x636.google.com with SMTP id t6so3091642plq.1; Fri, 22 Jan 2021 04:08:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=BEHM7Br+OBI0cEf9P25bLe7++Rkcp9QFD23+Ri5JUXE=; b=aNcPqt3W1aJlNVLWKC7u26bHIK7UCNzMFWBceE1X4FbGEwHfXjNyGPWyLwYWhp3JZr ZRm7n5ZmGqf/9tyRa4PrxDa+eGDjKTDkSSeaHqZd3dm39AvuYfu1VZaJH6TmuuSJawgx S8tNTs7DfORIz2aqmK3j4p8bHnCbj5wHZ99hhTcCa1aizDcL7GPcJcua0QsNc7ZxFrjU KmALMhkV1KM4DzN+4CJJyOMDESsfnD/6DKClM//+oYxSg46juy35oh5AqYWjXp1N4LUI AKE96iJQ8Riq8Wyiccc0AFY1wLwooI4dRlfPUwF06UBPLeRpaZXIWr1gc6HcH8IEzeQr eirQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=BEHM7Br+OBI0cEf9P25bLe7++Rkcp9QFD23+Ri5JUXE=; b=nBtdVVCkRyrhdOqE9jK7wQa95+JvR9/Z3Ijc1482xjt0ilXxncoJKnrrPVfsLx8QpR mXohlwiZZLyYgb14VDrZgS6yvFud/HJWO9l42G/dJjh1KvBfbCJblhmPri0rcr9y44nv afLCwmfJNO4lIQKv5tblBw3rYZZgmNnizhqp3VHN2zriZdT3FuC78yu0CNeus8bpNqOi cKjU+0JbWGsMLiOBfZv2MpRMrXO7iuh3CpIaQTDtOHg1iDzTj6CGV+104s6vkbrGiKmP 4M65ssN2aRMfgHrnnPMzGkM8jWhiR9rD/qX1+FEY4PxGGvMOpk4qIiFABWvs8wLeHANy gZBg== X-Gm-Message-State: AOAM530xi5VXYTsPhS8AMxqxAW2GwjrG7OkQbfPzsTtaJrsOvo64ZLHg jABVFCIdN2Lp6cQ/d2LLgRtRM3rjwIflkw== X-Google-Smtp-Source: ABdhPJyJUal5ZHAmzXd3mA1SyT2Ceitox/Jg9fU84FwewxO5FV0KM4fU+bZH0psbhMK3dDZEpeAkHw== X-Received: by 2002:a17:90a:fa18:: with SMTP id cm24mr5058171pjb.180.1611317286557; Fri, 22 Jan 2021 04:08:06 -0800 (PST) Received: from localhost.localdomain ([211.108.35.36]) by smtp.gmail.com with ESMTPSA id mv17sm9104440pjb.17.2021.01.22.04.08.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 22 Jan 2021 04:08:06 -0800 (PST) From: Minwoo Im To: qemu-devel@nongnu.org, qemu-block@nongnu.org Subject: [PATCH V5 6/6] hw/block/nvme: support for shared namespace in subsystem Date: Fri, 22 Jan 2021 21:07:36 +0900 Message-Id: <20210122120736.5242-7-minwoo.im.dev@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210122120736.5242-1-minwoo.im.dev@gmail.com> References: <20210122120736.5242-1-minwoo.im.dev@gmail.com> Received-SPF: pass client-ip=2607:f8b0:4864:20::636; envelope-from=minwoo.im.dev@gmail.com; helo=mail-pl1-x636.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Keith Busch , Klaus Jensen , Minwoo Im , Kevin Wolf , Max Reitz Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" nvme-ns device is registered to a nvme controller device during the initialization in nvme_register_namespace() in case that 'bus' property is given which means it's mapped to a single controller. This patch introduced a new property 'subsys' just like the controller device instance did to map a namespace to a NVMe subsystem. If 'subsys' property is given to the nvme-ns device, it will belong to the specified subsystem and will be attached to all controllers in that subsystem by enabling shared namespace capability in NMIC(Namespace Multi-path I/O and Namespace Capabilities) in Identify Namespace. Usage: -device nvme-subsys,id=subsys0 -device nvme,serial=foo,id=nvme0,subsys=subsys0 -device nvme,serial=bar,id=nvme1,subsys=subsys0 -device nvme,serial=baz,id=nvme2,subsys=subsys0 -device nvme-ns,id=ns1,drive=,nsid=1,subsys=subsys0 # Shared -device nvme-ns,id=ns2,drive=,nsid=2,bus=nvme2 # Non-shared In the above example, 'ns1' will be shared to 'nvme0' and 'nvme1' in the same subsystem. On the other hand, 'ns2' will be attached to the 'nvme2' only as a private namespace in that subsystem. All the namespace with 'subsys' parameter will attach all controllers in the subsystem to the namespace by default. Signed-off-by: Minwoo Im --- hw/block/nvme-ns.c | 23 ++++++++++++++++++----- hw/block/nvme-ns.h | 7 +++++++ hw/block/nvme-subsys.c | 25 +++++++++++++++++++++++++ hw/block/nvme-subsys.h | 3 +++ hw/block/nvme.c | 10 +++++++++- 5 files changed, 62 insertions(+), 6 deletions(-) diff --git a/hw/block/nvme-ns.c b/hw/block/nvme-ns.c index 62b25cf69bfa..9b493f2ead03 100644 --- a/hw/block/nvme-ns.c +++ b/hw/block/nvme-ns.c @@ -63,6 +63,10 @@ static int nvme_ns_init(NvmeNamespace *ns, Error **errp) id_ns->npda = id_ns->npdg = npdg - 1; + if (nvme_ns_shared(ns)) { + id_ns->nmic |= NVME_NMIC_NS_SHARED; + } + return 0; } @@ -365,16 +369,25 @@ static void nvme_ns_realize(DeviceState *dev, Error **errp) return; } - if (nvme_register_namespace(n, ns, errp)) { - error_propagate_prepend(errp, local_err, - "could not register namespace: "); - return; + if (ns->subsys) { + if (nvme_subsys_register_ns(ns, errp)) { + error_propagate_prepend(errp, local_err, + "could not setup namespace to subsys: "); + return; + } + } else { + if (nvme_register_namespace(n, ns, errp)) { + error_propagate_prepend(errp, local_err, + "could not register namespace: "); + return; + } } - } static Property nvme_ns_props[] = { DEFINE_BLOCK_PROPERTIES(NvmeNamespace, blkconf), + DEFINE_PROP_LINK("subsys", NvmeNamespace, subsys, TYPE_NVME_SUBSYS, + NvmeSubsystem *), DEFINE_PROP_UINT32("nsid", NvmeNamespace, params.nsid, 0), DEFINE_PROP_UUID("uuid", NvmeNamespace, params.uuid), DEFINE_PROP_BOOL("zoned", NvmeNamespace, params.zoned, false), diff --git a/hw/block/nvme-ns.h b/hw/block/nvme-ns.h index 293ac990e3f6..929e78861903 100644 --- a/hw/block/nvme-ns.h +++ b/hw/block/nvme-ns.h @@ -47,6 +47,8 @@ typedef struct NvmeNamespace { const uint32_t *iocs; uint8_t csi; + NvmeSubsystem *subsys; + NvmeIdNsZoned *id_ns_zoned; NvmeZone *zone_array; QTAILQ_HEAD(, NvmeZone) exp_open_zones; @@ -77,6 +79,11 @@ static inline uint32_t nvme_nsid(NvmeNamespace *ns) return -1; } +static inline bool nvme_ns_shared(NvmeNamespace *ns) +{ + return !!ns->subsys; +} + static inline NvmeLBAF *nvme_ns_lbaf(NvmeNamespace *ns) { NvmeIdNs *id_ns = &ns->id_ns; diff --git a/hw/block/nvme-subsys.c b/hw/block/nvme-subsys.c index e9d61c993c90..641de33e99fc 100644 --- a/hw/block/nvme-subsys.c +++ b/hw/block/nvme-subsys.c @@ -43,6 +43,31 @@ int nvme_subsys_register_ctrl(NvmeCtrl *n, Error **errp) return cntlid; } +int nvme_subsys_register_ns(NvmeNamespace *ns, Error **errp) +{ + NvmeSubsystem *subsys = ns->subsys; + NvmeCtrl *n; + int i; + + if (subsys->namespaces[nvme_nsid(ns)]) { + error_setg(errp, "namespace %d already registerd to subsy %s", + nvme_nsid(ns), subsys->parent_obj.id); + return -1; + } + + subsys->namespaces[nvme_nsid(ns)] = ns; + + for (i = 0; i < ARRAY_SIZE(subsys->ctrls); i++) { + n = subsys->ctrls[i]; + + if (n && nvme_register_namespace(n, ns, errp)) { + return -1; + } + } + + return 0; +} + static void nvme_subsys_setup(NvmeSubsystem *subsys) { snprintf((char *)subsys->subnqn, sizeof(subsys->subnqn), diff --git a/hw/block/nvme-subsys.h b/hw/block/nvme-subsys.h index 4eba50d96a1d..ccf6a71398d3 100644 --- a/hw/block/nvme-subsys.h +++ b/hw/block/nvme-subsys.h @@ -14,6 +14,7 @@ OBJECT_CHECK(NvmeSubsystem, (obj), TYPE_NVME_SUBSYS) #define NVME_SUBSYS_MAX_CTRLS 32 +#define NVME_SUBSYS_MAX_NAMESPACES 32 typedef struct NvmeCtrl NvmeCtrl; typedef struct NvmeNamespace NvmeNamespace; @@ -22,8 +23,10 @@ typedef struct NvmeSubsystem { uint8_t subnqn[256]; NvmeCtrl *ctrls[NVME_SUBSYS_MAX_CTRLS]; + NvmeNamespace *namespaces[NVME_SUBSYS_MAX_NAMESPACES]; } NvmeSubsystem; int nvme_subsys_register_ctrl(NvmeCtrl *n, Error **errp); +int nvme_subsys_register_ns(NvmeNamespace *ns, Error **errp); #endif /* NVME_SUBSYS_H */ diff --git a/hw/block/nvme.c b/hw/block/nvme.c index 3dedefb8ebba..2752039b5df1 100644 --- a/hw/block/nvme.c +++ b/hw/block/nvme.c @@ -25,7 +25,8 @@ * mdts=,zoned.append_size_limit=, \ * subsys= \ * -device nvme-ns,drive=,bus=,nsid=,\ - * zoned= + * zoned=, \ + * subsys= * -device nvme-subsys,id= * * Note cmb_size_mb denotes size of CMB in MB. CMB is assumed to be at @@ -70,6 +71,13 @@ * data size being in effect. By setting this property to 0, users can make * ZASL to be equal to MDTS. This property only affects zoned namespaces. * + * nvme namespace device parameters + * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + * - `subsys` + * NVM Subsystem device. If given, this namespace will be attached to all + * controllers in the subsystem. Otherwise, `bus` must be given to attach + * this namespace to a specified single controller as a non-shared namespace. + * * Setting `zoned` to true selects Zoned Command Set at the namespace. * In this case, the following namespace properties are available to configure * zoned operation: