| Message ID | 20220303230131.2103-8-shameerali.kolothum.thodi@huawei.com |
|---|---|
| State | New |
| Headers | show |
| Series | vfio/hisilicon: add ACC live migration driver | expand |
> From: Longfang Liu <liulongfang@huawei.com>> > We use VF QM state register to record the status of the QM configuration > state. This will be used in the ACC migration driver to determine whether > we can safely save and restore the QM data. > > Signed-off-by: Longfang Liu <liulongfang@huawei.com> > Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com> Acked-by: Zhou Wang <wangzhou1@hisilicon.com> Best, Zhou > --- > drivers/crypto/hisilicon/qm.c | 8 ++++++++ > include/linux/hisi_acc_qm.h | 6 ++++++ > 2 files changed, 14 insertions(+) > > diff --git a/drivers/crypto/hisilicon/qm.c b/drivers/crypto/hisilicon/qm.c > index c88e013371af..6a8776db38b5 100644 > --- a/drivers/crypto/hisilicon/qm.c > +++ b/drivers/crypto/hisilicon/qm.c > @@ -3492,6 +3492,12 @@ static void hisi_qm_pci_uninit(struct hisi_qm *qm) > pci_disable_device(pdev); > } > > +static void hisi_qm_set_state(struct hisi_qm *qm, u8 state) > +{ > + if (qm->ver > QM_HW_V2 && qm->fun_type == QM_HW_VF) > + writel(state, qm->io_base + QM_VF_STATE); > +} > + > /** > * hisi_qm_uninit() - Uninitialize qm. > * @qm: The qm needed uninit. > @@ -3520,6 +3526,7 @@ void hisi_qm_uninit(struct hisi_qm *qm) > dma_free_coherent(dev, qm->qdma.size, > qm->qdma.va, qm->qdma.dma); > } > + hisi_qm_set_state(qm, QM_NOT_READY); > up_write(&qm->qps_lock); > > qm_irq_unregister(qm); > @@ -3745,6 +3752,7 @@ int hisi_qm_start(struct hisi_qm *qm) > if (!ret) > atomic_set(&qm->status.flags, QM_START); > > + hisi_qm_set_state(qm, QM_READY); > err_unlock: > up_write(&qm->qps_lock); > return ret; > diff --git a/include/linux/hisi_acc_qm.h b/include/linux/hisi_acc_qm.h > index 00f2a4db8723..177f7b7cd414 100644 > --- a/include/linux/hisi_acc_qm.h > +++ b/include/linux/hisi_acc_qm.h > @@ -67,6 +67,7 @@ > #define QM_DB_RAND_SHIFT_V2 16 > #define QM_DB_INDEX_SHIFT_V2 32 > #define QM_DB_PRIORITY_SHIFT_V2 48 > +#define QM_VF_STATE 0x60 > > /* qm cache */ > #define QM_CACHE_CTL 0x100050 > @@ -162,6 +163,11 @@ enum qm_debug_file { > DEBUG_FILE_NUM, > }; > > +enum qm_vf_state { > + QM_READY = 0, > + QM_NOT_READY, > +}; > + > struct qm_dfx { > atomic64_t err_irq_cnt; > atomic64_t aeq_irq_cnt; >
> From: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com> > Sent: Friday, March 4, 2022 7:01 AM > > From: Longfang Liu <liulongfang@huawei.com> > > We use VF QM state register to record the status of the QM configuration > state. This will be used in the ACC migration driver to determine whether > we can safely save and restore the QM data. Can you say something about what QM is and how it is related to the VF state to be migrated? It might be obvious to acc driver people but not so to to others in vfio community like me. 😊 Thanks Kevin
> -----Original Message----- > From: Tian, Kevin [mailto:kevin.tian@intel.com] > Sent: 08 March 2022 06:31 > To: Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>; > kvm@vger.kernel.org; linux-kernel@vger.kernel.org; > linux-crypto@vger.kernel.org > Cc: linux-pci@vger.kernel.org; alex.williamson@redhat.com; jgg@nvidia.com; > cohuck@redhat.com; mgurtovoy@nvidia.com; yishaih@nvidia.com; Linuxarm > <linuxarm@huawei.com>; liulongfang <liulongfang@huawei.com>; Zengtao (B) > <prime.zeng@hisilicon.com>; Jonathan Cameron > <jonathan.cameron@huawei.com>; Wangzhou (B) <wangzhou1@hisilicon.com> > Subject: RE: [PATCH v8 7/9] crypto: hisilicon/qm: Set the VF QM state register > > > From: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com> > > Sent: Friday, March 4, 2022 7:01 AM > > > > From: Longfang Liu <liulongfang@huawei.com> > > > > We use VF QM state register to record the status of the QM configuration > > state. This will be used in the ACC migration driver to determine whether > > we can safely save and restore the QM data. > > Can you say something about what QM is and how it is related to the VF state > to be migrated? It might be obvious to acc driver people but not so to to > others in vfio community like me. 😊 Ok, understand that. I will lift a description from the QM driver patch :). QM stands for Queue Management which is a generic IP used by ACC devices. It provides a general PCIe interface for the CPU and the ACC devices to share a group of queues. QM integrated into an accelerator provides queue management service. Queues can be assigned to PF and VFs, and queues can be controlled by unified mailboxes and doorbells. The QM driver(drivers/crypto/hisilicon/qm.c) provides generic interfaces to ACC drivers to manage the QM. I will add some of this in either this commit msg or in the actual file somewhere. Thanks, Shameer
> From: Shameerali Kolothum Thodi > <shameerali.kolothum.thodi@huawei.com> > Sent: Tuesday, March 8, 2022 4:46 PM > > > > > -----Original Message----- > > From: Tian, Kevin [mailto:kevin.tian@intel.com] > > Sent: 08 March 2022 06:31 > > To: Shameerali Kolothum Thodi > <shameerali.kolothum.thodi@huawei.com>; > > kvm@vger.kernel.org; linux-kernel@vger.kernel.org; > > linux-crypto@vger.kernel.org > > Cc: linux-pci@vger.kernel.org; alex.williamson@redhat.com; > jgg@nvidia.com; > > cohuck@redhat.com; mgurtovoy@nvidia.com; yishaih@nvidia.com; > Linuxarm > > <linuxarm@huawei.com>; liulongfang <liulongfang@huawei.com>; > Zengtao (B) > > <prime.zeng@hisilicon.com>; Jonathan Cameron > > <jonathan.cameron@huawei.com>; Wangzhou (B) > <wangzhou1@hisilicon.com> > > Subject: RE: [PATCH v8 7/9] crypto: hisilicon/qm: Set the VF QM state > register > > > > > From: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com> > > > Sent: Friday, March 4, 2022 7:01 AM > > > > > > From: Longfang Liu <liulongfang@huawei.com> > > > > > > We use VF QM state register to record the status of the QM configuration > > > state. This will be used in the ACC migration driver to determine whether > > > we can safely save and restore the QM data. > > > > Can you say something about what QM is and how it is related to the VF > state > > to be migrated? It might be obvious to acc driver people but not so to to > > others in vfio community like me. 😊 > > Ok, understand that. I will lift a description from the QM driver patch :). > > QM stands for Queue Management which is a generic IP used by ACC devices. > It provides a general PCIe interface for the CPU and the ACC devices to share > a group of queues. > > QM integrated into an accelerator provides queue management service. > Queues can be assigned to PF and VFs, and queues can be controlled by > unified mailboxes and doorbells. The QM > driver(drivers/crypto/hisilicon/qm.c) > provides generic interfaces to ACC drivers to manage the QM. > > I will add some of this in either this commit msg or in the actual file > somewhere. > Yes, such information is helpful
diff --git a/drivers/crypto/hisilicon/qm.c b/drivers/crypto/hisilicon/qm.c index c88e013371af..6a8776db38b5 100644 --- a/drivers/crypto/hisilicon/qm.c +++ b/drivers/crypto/hisilicon/qm.c @@ -3492,6 +3492,12 @@ static void hisi_qm_pci_uninit(struct hisi_qm *qm) pci_disable_device(pdev); } +static void hisi_qm_set_state(struct hisi_qm *qm, u8 state) +{ + if (qm->ver > QM_HW_V2 && qm->fun_type == QM_HW_VF) + writel(state, qm->io_base + QM_VF_STATE); +} + /** * hisi_qm_uninit() - Uninitialize qm. * @qm: The qm needed uninit. @@ -3520,6 +3526,7 @@ void hisi_qm_uninit(struct hisi_qm *qm) dma_free_coherent(dev, qm->qdma.size, qm->qdma.va, qm->qdma.dma); } + hisi_qm_set_state(qm, QM_NOT_READY); up_write(&qm->qps_lock); qm_irq_unregister(qm); @@ -3745,6 +3752,7 @@ int hisi_qm_start(struct hisi_qm *qm) if (!ret) atomic_set(&qm->status.flags, QM_START); + hisi_qm_set_state(qm, QM_READY); err_unlock: up_write(&qm->qps_lock); return ret; diff --git a/include/linux/hisi_acc_qm.h b/include/linux/hisi_acc_qm.h index 00f2a4db8723..177f7b7cd414 100644 --- a/include/linux/hisi_acc_qm.h +++ b/include/linux/hisi_acc_qm.h @@ -67,6 +67,7 @@ #define QM_DB_RAND_SHIFT_V2 16 #define QM_DB_INDEX_SHIFT_V2 32 #define QM_DB_PRIORITY_SHIFT_V2 48 +#define QM_VF_STATE 0x60 /* qm cache */ #define QM_CACHE_CTL 0x100050 @@ -162,6 +163,11 @@ enum qm_debug_file { DEBUG_FILE_NUM, }; +enum qm_vf_state { + QM_READY = 0, + QM_NOT_READY, +}; + struct qm_dfx { atomic64_t err_irq_cnt; atomic64_t aeq_irq_cnt;