Patchwork [PATCHv2,vringh,1/3] remoteproc: Add support for vringh (Host vrings)

login
register
mail settings
Submitter sjur.brandeland@stericsson.com
Date Feb. 12, 2013, 11:49 a.m.
Message ID <1360669793-6921-2-git-send-email-sjur.brandeland@stericsson.com>
Download mbox | patch
Permalink /patch/219830/
State Not Applicable
Delegated to: David Miller
Headers show

Comments

sjur.brandeland@stericsson.com - Feb. 12, 2013, 11:49 a.m.
From: Sjur Brændeland <sjur.brandeland@stericsson.com>

Add functions for creating, deleting and kicking host-side virtio rings.

The host ring is not integrated with virtiqueues and cannot be managed
through virtio-config. Remoteproc must export functions for handling the
host-side virtio rings.

The functions rproc_virtio_get_vringh(), rproc_virtio_del_vringh(),
rproc_virtio_kick_vringh() are added to remoteproc_virtio.c. The
existing functions rproc_vq_interrupt() and rproc_virtio_find_vqs()
are updated to handle the new vhost rings.

Signed-off-by: Sjur Brændeland <sjur.brandeland@stericsson.com>
cc: Ohad Ben-Cohen <ohad@wizery.com>
cc: Rusty Russell <rusty@rustcorp.com.au>
cc: Ido Yariv <ido@wizery.com>
cc: Erwan Yvin <erwan.yvin@stericsson.com>
---
 drivers/remoteproc/Kconfig             |    3 +
 drivers/remoteproc/remoteproc_virtio.c |  127 ++++++++++++++++++++++++++++++--
 include/linux/remoteproc.h             |   14 ++++
 3 files changed, 137 insertions(+), 7 deletions(-)
Ohad Ben-Cohen - Feb. 20, 2013, 4:05 p.m.
Hi Sjur,

On Tue, Feb 12, 2013 at 1:49 PM,  <sjur.brandeland@stericsson.com> wrote:
> From: Sjur Brændeland <sjur.brandeland@stericsson.com>
>
> Add functions for creating, deleting and kicking host-side virtio rings.
>
> The host ring is not integrated with virtiqueues and cannot be managed
> through virtio-config.

Is that an inherent design/issue of vringh or just a description of
the current vringh code ?

> Remoteproc must export functions for handling the host-side virtio rings.

Have you considered exporting this via virtio instead ?

> The functions rproc_virtio_get_vringh(), rproc_virtio_del_vringh(),
> rproc_virtio_kick_vringh() are added to remoteproc_virtio.c.

I wonder if this is the way we want things to work.

Following this design, virtio drivers that use these rproc_* functions
will be coupled with the remoteproc framework.

One issue with this is what happens if, e.g., a VIRTIO_ID_CAIF vdev is
added by other than remoteproc (e.g. by virtio_pci or virtio_mmio).
Not sure how probable this really is, and whether there's anything
that prevents this, but things will go awry if this happens.

But maybe the important aspect to consider is whether we really want
to couple virtio drivers (such as the upcoming caif one) with the
remoteproc framework.

If you'll take a look at the rpmsg virtio driver, there's nothing
there which couples it with remoteproc. It's just a standard virtio
driver, that can be easily used with traditional virtio hosts as well.

This is possible of course thanks to the abstraction provided by
virtio: remoteproc only implements a set of callbacks which virtio
invokes when needed.

Do we not want to follow a similar design scheme with vringh ?

I have some other questions as well but maybe it's better to discuss
first the bigger picture.

Thanks!
Ohad.
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
sjur.brandeland@stericsson.com - Feb. 20, 2013, 11:01 p.m.
On Wed, Feb 20, 2013 at 5:05 PM, Ohad Ben-Cohen <ohad@wizery.com> wrote:
> Hi Sjur,
>
> On Tue, Feb 12, 2013 at 1:49 PM,  <sjur.brandeland@stericsson.com> wrote:
>> From: Sjur Brændeland <sjur.brandeland@stericsson.com>
>>
>> Add functions for creating, deleting and kicking host-side virtio rings.
>>
>> The host ring is not integrated with virtiqueues and cannot be managed
>> through virtio-config.
>
> Is that an inherent design/issue of vringh or just a description of
> the current vringh code ?
>
>> Remoteproc must export functions for handling the host-side virtio rings.
>
> Have you considered exporting this via virtio instead ?

Rusty should comment on this...
I asked Rusty the same question a while a go, see
http://lkml.org/lkml/2013/1/11/559
AFAIK, using the vringh API directly is a deliberate design choice.

[Sjur:]
>> How do you see the in-kernel API for this? I would like to see
>> something similar to my previous patches, where we extend
>> the virtqueue API. E.g. something like this:
>> struct virtqueue *vring_new_virtqueueh(...)...

[Rusty:]
>I was just going to create _kernel variants of all the _user helpers,
>and let you drive it directly like that.
>
>If we get a second in-kernel user, we create wrappers (I'd prefer not to
>overload struct virtqueue though).


>> The functions rproc_virtio_get_vringh(), rproc_virtio_del_vringh(),
>> rproc_virtio_kick_vringh() are added to remoteproc_virtio.c.
>
> I wonder if this is the way we want things to work.
>
> Following this design, virtio drivers that use these rproc_* functions
> will be coupled with the remoteproc framework.
>
> One issue with this is what happens if, e.g., a VIRTIO_ID_CAIF vdev is
> added by other than remoteproc (e.g. by virtio_pci or virtio_mmio).
> Not sure how probable this really is, and whether there's anything
> that prevents this, but things will go awry if this happens.

Yes, if you insert a "malicious device" like that you can make it crash,
but wouldn't most drivers do if you try to register a malicious device...?

If we really want to protect from this, we could perhaps validate the vdev
pointer in function rproc_virtio_new_vringh() by looking through the vdevs
of the registered rprocs.

> But maybe the important aspect to consider is whether we really want
> to couple virtio drivers (such as the upcoming caif one) with the
> remoteproc framework.

I'm not sure this is an issue for the CAIF driver. It would be very nice
if someone else could make use of it, but right now cannot see the CAIF
driver being used outside the remoteproc framework. This driver is
designed specifically to work with the STE-modem using the CAIF
protocol over a shared memory interface.

> If you'll take a look at the rpmsg virtio driver, there's nothing
> there which couples it with remoteproc. It's just a standard virtio
> driver, that can be easily used with traditional virtio hosts as well.
>
> This is possible of course thanks to the abstraction provided by
> virtio: remoteproc only implements a set of callbacks which virtio
> invokes when needed.

Yes, and generalizing the use of virtio devices in remoteproc
has been useful. It has enabled me to let remoteproc manage both
virtio_serial and virtio_caif devices :-)

>
> Do we not want to follow a similar design scheme with vringh ?

I know some of my colleagues has been working on symmetric vring
for rpmsg. Last I heard from them they were going to use the same
approach I've done for CAIF, by "reversing" the direction of the rings.
AFAIK this means that the current API I have proposed will work for
them as well.

If some other driver is showing up using the vringh kernel API where
the current API don't fit, I guess it's time to create some abstractions
and wrappers... But I hope the current approach will do for now?

> I have some other questions as well but maybe it's better to discuss
> first the bigger picture.

OK, but please don't hesitate to address this. I'm still aiming for this
to go into 3.9.

Regards,
Sjur
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Rusty Russell - Feb. 21, 2013, 6:37 a.m.
Ohad Ben-Cohen <ohad@wizery.com> writes:
> Hi Sjur,
>
> On Tue, Feb 12, 2013 at 1:49 PM,  <sjur.brandeland@stericsson.com> wrote:
>> From: Sjur Brændeland <sjur.brandeland@stericsson.com>
>>
>> Add functions for creating, deleting and kicking host-side virtio rings.
>>
>> The host ring is not integrated with virtiqueues and cannot be managed
>> through virtio-config.
>
> Is that an inherent design/issue of vringh or just a description of
> the current vringh code ?

It's by design.  The producer (virtqueue) and consumer (vringh) are two
sides of the same coin, but they do different things.

virtqueue is a slightly higher level abstraction which assumes a
virtio_device, because every user so far has had one.  vringh doesn't,
because it's also aimed to underlie vhost.c which doesn't really have
one.

> This is possible of course thanks to the abstraction provided by
> virtio: remoteproc only implements a set of callbacks which virtio
> invokes when needed.
>
> Do we not want to follow a similar design scheme with vringh ?

Hmm... I clearly jumped the gun, assuming consensus was already reached.
I have put these patches *back* into pending-rebases, and they will not
be merged this merge window.

Cheers,
Rusty.
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Ohad Ben-Cohen - Feb. 21, 2013, 1:47 p.m.
On Thu, Feb 21, 2013 at 1:01 AM, Sjur Brændeland
<sjur.brandeland@stericsson.com> wrote:
> [Sjur:]
>>> How do you see the in-kernel API for this? I would like to see
>>> something similar to my previous patches, where we extend
>>> the virtqueue API. E.g. something like this:
>>> struct virtqueue *vring_new_virtqueueh(...)...
>
> [Rusty:]
>>I was just going to create _kernel variants of all the _user helpers,
>>and let you drive it directly like that.
>>
>>If we get a second in-kernel user, we create wrappers (I'd prefer not to
>>overload struct virtqueue though).

The wrappers suggestion sounds good.

One of the things we truly enjoyed about virtio is how easy it was to
reuse its drivers for communication with a remote processor.

It'd be great if we can stick to this design and keep vringh virtio
drivers decoupled from the low level implementation they are developed
with (specifically caif and remoteproc in this case).

> Yes, if you insert a "malicious device" like that you can make it crash,
> but wouldn't most drivers do if you try to register a malicious device...?
>
> If we really want to protect from this, we could perhaps validate the vdev
> pointer in function rproc_virtio_new_vringh() by looking through the vdevs
> of the registered rprocs.

It sounds like we can instead just avoid this altogether if we follow
Rusty's wrappers suggestion.

The result would probably look better, and I suspect it might be very
minimal code.

> If some other driver is showing up using the vringh kernel API where
> the current API don't fit, I guess it's time to create some abstractions
> and wrappers... But I hope the current approach will do for now?

Unless I'm missing something here, adding wrappers should be straight
forward and quick. Coupling a virtio driver with remoteproc doesn't
look great to me, probably because I appreciate so much the elegance
of virtio and how easy we could have reused its vanilla drivers with a
use case it wasn't originally designed to support.

>> I have some other questions as well but maybe it's better to discuss
>> first the bigger picture.
>
> OK, but please don't hesitate to address this. I'm still aiming for this
> to go into 3.9.

I was wondering - can you please explain your motivation for using
vringh in caif ?

We have internally discussed supporting multiple remote processors
talking to each other using rpmsg, and in that scenario using vringh
can considerably simplifies the solution (no need to decide in advance
which side is the "guest" and which is the "host"). Was this the
general incentive in using vringh in caif too or something else?

Thanks,
Ohad.
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Ohad Ben-Cohen - Feb. 21, 2013, 1:53 p.m.
On Thu, Feb 21, 2013 at 8:37 AM, Rusty Russell <rusty@rustcorp.com.au> wrote:
> Hmm... I clearly jumped the gun, assuming consensus was already reached.
> I have put these patches *back* into pending-rebases, and they will not
> be merged this merge window.

Thanks.

What do you think about creating some virtio-level wrappers for the
vringh handlers?

I don't think we're going to stop with caif as the only vringh user,
and it could be nice if we follow the virtio spirit of decoupling the
drivers from the low level implementation. It sure did prove itself
when the remoteproc use cases started showing up, and it's neat.

Thanks,
Ohad.
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Sjur Brændeland - Feb. 21, 2013, 5:28 p.m.
Hi Ohad,

> I was wondering - can you please explain your motivation for using
> vringh in caif ?
>
> We have internally discussed supporting multiple remote processors
> talking to each other using rpmsg, and in that scenario using vringh
> can considerably simplifies the solution (no need to decide in advance
> which side is the "guest" and which is the "host"). Was this the
> general incentive in using vringh in caif too or something else?

The motivation for using vringh was to avoid copying buffers
when sending data from the modem to the host. By using
vringh the modem can transfer datagrams to host only by
transfering buffer pointers via the rings.

We use a carveout to declare a memory area that is passed to the
modem internal memory allocator. This way we get both modem
and host to work on the shared memory region.

For TX traffic we use normal virtio queues for the same reason.
TX buffers can be handled without copying as well.

We are seeing a nice performance gain on the modem side after
implementing this.

Host size zero-copy is more tricky. It's hard to handle ownership of buffers,
if the modem crashes. But we might look into this in the future as well.

Regards,
Sjur
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Ohad Ben-Cohen - Feb. 21, 2013, 5:55 p.m.
Hi Sjur,

On Thu, Feb 21, 2013 at 7:28 PM, Sjur Brændeland <sjurbren@gmail.com> wrote:
> The motivation for using vringh was to avoid copying buffers
> when sending data from the modem to the host.

I may be missing something here, but why do you need vringh for that?

With rpmsg (which uses two regular vrings) both ends send huge amount
of data without copying it - we just send the pointers across.

Thanks,
Ohad.
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Sjur Brændeland - Feb. 21, 2013, 7:36 p.m.
Hi Ohad,

On Thu, Feb 21, 2013 at 6:55 PM, Ohad Ben-Cohen <ohad@wizery.com> wrote:
> On Thu, Feb 21, 2013 at 7:28 PM, Sjur Brændeland <sjurbren@gmail.com> wrote:
>> The motivation for using vringh was to avoid copying buffers
>> when sending data from the modem to the host.
>
> I may be missing something here, but why do you need vringh for that?
>
> With rpmsg (which uses two regular vrings) both ends send huge amount
> of data without copying it - we just send the pointers across.

OK, We did carefully consider using the normal vrings, but concluded it was
not doable. I'll try to give you some of the background that I can
recall from top of my head.
(I can dig out more if you're still not convinced :)

The modem we're integrating with is a complex beast (multi mode modem
LTE, HSPA+, etc).
When a packet is received deep down in the radio stack, it allocates packet
buffers using the internal slab allocator.The packet may contain
anything from control, voice,
or IP packets. If the packet contains IP-payload it will travel to a
different asymmetric CPU that
handles the IPC towards the modem. If the packet is not a IP-packet it will be
processed internally and eventually freed.

What we have done to integrate virtio is to inject the carveout into the
the modem internal slab-allocator.

Using the reversed ring allows us to introduce zero-copy for virtio
with virtually no
impact on the radio-stack, only a small change on the modem slab allocator.
It supports dynamic size buffer allocation, and modem internal
messaging using data
allocated from the shared region. It also allows modem to manage it's
own memory,
without any dependency on host side allocators.


Thanks,
Sjur
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Rusty Russell - Feb. 22, 2013, 12:42 a.m.
Ohad Ben-Cohen <ohad@wizery.com> writes:
> On Thu, Feb 21, 2013 at 8:37 AM, Rusty Russell <rusty@rustcorp.com.au> wrote:
>> Hmm... I clearly jumped the gun, assuming consensus was already reached.
>> I have put these patches *back* into pending-rebases, and they will not
>> be merged this merge window.
>
> Thanks.
>
> What do you think about creating some virtio-level wrappers for the
> vringh handlers?
>
> I don't think we're going to stop with caif as the only vringh user,
> and it could be nice if we follow the virtio spirit of decoupling the
> drivers from the low level implementation. It sure did prove itself
> when the remoteproc use cases started showing up, and it's neat.

The problem space is a bit different.  My immediate concern is getting
vhost (and thus vhost_net/blk) to use vringh: I wanted to unify the
in-userspace and in-kernelspace ring implementations.  We don't have
that issue in virtqueue.c.

vhost is (will be) the higher abstraction for in-userspace rings,
perhaps we want an equivalent for in-kernelspace rings.  I'm happy to
look at patches, but I don't immediately see what it would look like...

Thanks,
Rusty.
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Ohad Ben-Cohen - Feb. 23, 2013, 9:49 a.m.
On Thu, Feb 21, 2013 at 9:36 PM, Sjur Brændeland <sjurbren@gmail.com> wrote:
> OK, We did carefully consider using the normal vrings, but concluded it was
> not doable. I'll try to give you some of the background that I can
> recall from top of my head.
> (I can dig out more if you're still not convinced :)
>
> The modem we're integrating with is a complex beast (multi mode modem
> LTE, HSPA+, etc).
> When a packet is received deep down in the radio stack, it allocates packet
> buffers using the internal slab allocator.The packet may contain
> anything from control, voice,
> or IP packets. If the packet contains IP-payload it will travel to a
> different asymmetric CPU that
> handles the IPC towards the modem. If the packet is not a IP-packet it will be
> processed internally and eventually freed.
>
> What we have done to integrate virtio is to inject the carveout into the
> the modem internal slab-allocator.
>
> Using the reversed ring allows us to introduce zero-copy for virtio
> with virtually no
> impact on the radio-stack, only a small change on the modem slab allocator.
> It supports dynamic size buffer allocation, and modem internal
> messaging using data
> allocated from the shared region. It also allows modem to manage it's
> own memory,
> without any dependency on host side allocators.

Thanks for the description, Sjur.

It sounds like this mostly simplifies your modem-side code, especially
if it talks to other cores as well. My impression is that persistently
sticking to guest vrings is also possible, but it makes things
cumbersome.

We will most probably adopt vringh in rpmsg too when a real multicore
use case shows up.

Ohad.
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Patch

diff --git a/drivers/remoteproc/Kconfig b/drivers/remoteproc/Kconfig
index 96ce101..c7d1d36 100644
--- a/drivers/remoteproc/Kconfig
+++ b/drivers/remoteproc/Kconfig
@@ -7,6 +7,9 @@  config REMOTEPROC
 	depends on HAS_DMA
 	select FW_CONFIG
 	select VIRTIO
+	select VHOST_RING
+
+source drivers/vhost/Kconfig
 
 config OMAP_REMOTEPROC
 	tristate "OMAP remoteproc support"
diff --git a/drivers/remoteproc/remoteproc_virtio.c b/drivers/remoteproc/remoteproc_virtio.c
index 9e198e5..fa7bf7b 100644
--- a/drivers/remoteproc/remoteproc_virtio.c
+++ b/drivers/remoteproc/remoteproc_virtio.c
@@ -23,6 +23,7 @@ 
 #include <linux/virtio_config.h>
 #include <linux/virtio_ids.h>
 #include <linux/virtio_ring.h>
+#include <linux/vringh.h>
 #include <linux/err.h>
 #include <linux/kref.h>
 #include <linux/slab.h>
@@ -60,10 +61,15 @@  irqreturn_t rproc_vq_interrupt(struct rproc *rproc, int notifyid)
 	dev_dbg(&rproc->dev, "vq index %d is interrupted\n", notifyid);
 
 	rvring = idr_find(&rproc->notifyids, notifyid);
-	if (!rvring || !rvring->vq)
+	if (!rvring)
 		return IRQ_NONE;
 
-	return vring_interrupt(0, rvring->vq);
+	if (rvring->vringh && rvring->vringh_cb)
+		return rvring->vringh_cb(&rvring->rvdev->vdev, rvring->vringh);
+	else if (rvring->vq)
+		return vring_interrupt(0, rvring->vq);
+	else
+		return IRQ_NONE;
 }
 EXPORT_SYMBOL(rproc_vq_interrupt);
 
@@ -149,14 +155,21 @@  static int rproc_virtio_find_vqs(struct virtio_device *vdev, unsigned nvqs,
 		       const char *names[])
 {
 	struct rproc *rproc = vdev_to_rproc(vdev);
-	int i, ret;
+	struct rproc_vdev *rvdev = vdev_to_rvdev(vdev);
+	int rng, id, ret, nrings = ARRAY_SIZE(rvdev->vring);
+
+	for (id = 0, rng = 0; rng < nrings; ++rng) {
+		struct rproc_vring *rvring = &rvdev->vring[rng];
+		/* Skip the host side rings */
+		if (rvring->vringh)
+			continue;
 
-	for (i = 0; i < nvqs; ++i) {
-		vqs[i] = rp_find_vq(vdev, i, callbacks[i], names[i]);
-		if (IS_ERR(vqs[i])) {
-			ret = PTR_ERR(vqs[i]);
+		vqs[id] = rp_find_vq(vdev, rng, callbacks[id], names[id]);
+		if (IS_ERR(vqs[id])) {
+			ret = PTR_ERR(vqs[id]);
 			goto error;
 		}
+		++id;
 	}
 
 	/* now that the vqs are all set, boot the remote processor */
@@ -173,6 +186,106 @@  error:
 	return ret;
 }
 
+/**
+ * rproc_virtio_new_vringh() - create a reversed virtio ring.
+ * @vdev: the virtio device
+ * @index: the virtio ring index
+ * @cb: callback for the reversed virtio ring
+ *
+ * This function should be called by the virtio-driver
+ * before calling find_vqs(). It returns a struct vringh for
+ * accessing the virtio ring.
+ *
+ * Return: struct vhost, or NULL upon error.
+ */
+struct vringh *
+rproc_virtio_new_vringh(struct virtio_device *vdev, unsigned index,
+			irqreturn_t (*cb)(struct virtio_device *vdev,
+					  struct vringh *vring))
+{
+	struct rproc_vdev *rvdev = vdev_to_rvdev(vdev);
+	struct rproc_vring *rvring;
+	struct vringh *vrh;
+	int err;
+
+	if (index > ARRAY_SIZE(rvdev->vring)) {
+		dev_err(&rvdev->vdev.dev, "bad vring index: %d\n", index);
+		return NULL;
+	}
+
+	vrh = kzalloc(sizeof(*vrh), GFP_KERNEL);
+	if (!vrh)
+		return NULL;
+
+	err = rproc_alloc_vring(rvdev, index);
+	if (err)
+		goto free_vring;
+
+
+	rvring = &rvdev->vring[index];
+	/* zero vring */
+	memset(rvring->va, 0, vring_size(rvring->len, rvring->align));
+	vring_init(&vrh->vring, rvring->len, rvring->va, rvring->align);
+
+	rvring->vringh_cb = cb;
+	rvring->vringh = vrh;
+
+	err = vringh_init_kern(vrh,
+			       rvdev->dfeatures,
+			       rvring->len,
+			       false,
+			       vrh->vring.desc,
+			       vrh->vring.avail,
+			       vrh->vring.used);
+	if (!err)
+		return vrh;
+
+	dev_err(&vdev->dev, "failed to create vhost: %d\n", err);
+	rproc_free_vring(rvring);
+free_vring:
+	kfree(vrh);
+	return NULL;
+}
+EXPORT_SYMBOL(rproc_virtio_new_vringh);
+
+/**
+ * rproc_virtio_del_vringh() - release a reversed virtio ring.
+ * @vdev: the virtio device
+ * @index: the virtio ring index
+ *
+ * This function release the reversed virtio ring.
+ */
+void rproc_virtio_del_vringh(struct virtio_device *vdev, unsigned index)
+{
+	struct rproc_vdev *rvdev = vdev_to_rvdev(vdev);
+	struct rproc_vring *rvring = &rvdev->vring[index];
+	kfree(rvring->vringh);
+	rproc_free_vring(rvring);
+	rvring->vringh_cb = NULL;
+	rvring->vringh = NULL;
+}
+EXPORT_SYMBOL(rproc_virtio_del_vringh);
+
+/**
+ * rproc_virtio_kick_vringh() - kick the remote processor.
+ * @vdev: the virtio device
+ * @index: the virtio ring index
+ *
+ * kick the remote processor, and let it know which vring to poke at
+ */
+void rproc_virtio_kick_vringh(struct virtio_device *vdev, unsigned index)
+{
+	struct rproc_vdev *rvdev = vdev_to_rvdev(vdev);
+	struct rproc_vring *rvring = &rvdev->vring[index];
+	struct rproc *rproc = rvring->rvdev->rproc;
+	int notifyid = rvring->notifyid;
+
+	dev_dbg(&rproc->dev, "kicking vq index: %d\n", notifyid);
+
+	rproc->ops->kick(rproc, notifyid);
+}
+EXPORT_SYMBOL(rproc_virtio_kick_vringh);
+
 /*
  * We don't support yet real virtio status semantics.
  *
diff --git a/include/linux/remoteproc.h b/include/linux/remoteproc.h
index faf3332..414a1fd 100644
--- a/include/linux/remoteproc.h
+++ b/include/linux/remoteproc.h
@@ -39,7 +39,9 @@ 
 #include <linux/klist.h>
 #include <linux/mutex.h>
 #include <linux/virtio.h>
+#include <linux/vringh.h>
 #include <linux/completion.h>
+#include <linux/interrupt.h>
 #include <linux/idr.h>
 
 /**
@@ -444,6 +446,8 @@  struct rproc {
  * @notifyid: rproc-specific unique vring index
  * @rvdev: remote vdev
  * @vq: the virtqueue of this vring
+ * @vringh_cb: callback used when device has kicked
+ * @vringh: the reversed host-side vring
  */
 struct rproc_vring {
 	void *va;
@@ -454,6 +458,9 @@  struct rproc_vring {
 	int notifyid;
 	struct rproc_vdev *rvdev;
 	struct virtqueue *vq;
+	irqreturn_t (*vringh_cb)(struct virtio_device *vdev,
+				 struct vringh *vring);
+	struct vringh *vringh;
 };
 
 /**
@@ -485,6 +492,13 @@  int rproc_boot(struct rproc *rproc);
 void rproc_shutdown(struct rproc *rproc);
 void rproc_report_crash(struct rproc *rproc, enum rproc_crash_type type);
 
+struct vringh *
+rproc_virtio_new_vringh(struct virtio_device *vdev, unsigned index,
+			irqreturn_t (*cb)(struct virtio_device *vdev,
+					  struct vringh *vring));
+void rproc_virtio_del_vringh(struct virtio_device *vdev, unsigned index);
+void rproc_virtio_kick_vringh(struct virtio_device *vdev, unsigned index);
+
 static inline struct rproc_vdev *vdev_to_rvdev(struct virtio_device *vdev)
 {
 	return container_of(vdev, struct rproc_vdev, vdev);