| Message ID | 1584598473.9256.15248.camel@hbabu-laptop (mailing list archive) |
|---|---|
| State | Superseded |
| Headers | show |
| Series | powerpc/vas: Page fault handling for user space NX requests | expand |
| Context | Check | Description |
|---|---|---|
| snowpatch_ozlabs/apply_patch | success | Successfully applied on branch powerpc/merge (8a445cbcb9f5090cb07ec6cbb89a8a1fc99a0ff7) |
| snowpatch_ozlabs/checkpatch | warning | total: 0 errors, 0 warnings, 4 checks, 67 lines checked |
| snowpatch_ozlabs/needsstable | success | Patch has no Fixes tags |
Haren Myneni's on March 19, 2020 4:14 pm: > > Alloc IRQ and get trigger port address for each VAS instance. Kernel > register this IRQ per VAS instance and sets this port for each send > window. NX interrupts the kernel when it sees page fault. Again, should cc Cedric and Greg for XIVE / interrupt stuff. And for patch 2/14. The changelogs could use a bit of work. They're hard to read, and it can be a bit hard to decipher "why". Allocate a xive irq on each chip with a vas instance. The NX coprocessor raises a host CPU interrupt via vas if it encounters a page fault on an effective address. Subsequent patches register the trigger port with the NX coprocessor, and create a vas fault handler for this interrupt mapping. Don't know if the technical details are correct, but something like that in structure. Thanks, Nick
On 3/19/20 7:14 AM, Haren Myneni wrote: > > Alloc IRQ and get trigger port address for each VAS instance. Kernel > register this IRQ per VAS instance and sets this port for each send > window. NX interrupts the kernel when it sees page fault. I don't understand why this is not done by the OPAL driver for each VAS of the system. Is the VAS unit very different from OpenCAPI regarding the fault ? C. > > Signed-off-by: Haren Myneni <haren@linux.ibm.com> > --- > arch/powerpc/platforms/powernv/vas.c | 34 ++++++++++++++++++++++++++++------ > arch/powerpc/platforms/powernv/vas.h | 2 ++ > 2 files changed, 30 insertions(+), 6 deletions(-) > > diff --git a/arch/powerpc/platforms/powernv/vas.c b/arch/powerpc/platforms/powernv/vas.c > index ed9cc6d..168ab68 100644 > --- a/arch/powerpc/platforms/powernv/vas.c > +++ b/arch/powerpc/platforms/powernv/vas.c > @@ -15,6 +15,7 @@ > #include <linux/of_address.h> > #include <linux/of.h> > #include <asm/prom.h> > +#include <asm/xive.h> > > #include "vas.h" > > @@ -25,10 +26,12 @@ > > static int init_vas_instance(struct platform_device *pdev) > { > - int rc, cpu, vasid; > - struct resource *res; > - struct vas_instance *vinst; > struct device_node *dn = pdev->dev.of_node; > + struct vas_instance *vinst; > + uint32_t chipid, irq; > + struct resource *res; > + int rc, cpu, vasid; > + uint64_t port; > > rc = of_property_read_u32(dn, "ibm,vas-id", &vasid); > if (rc) { > @@ -36,6 +39,12 @@ static int init_vas_instance(struct platform_device *pdev) > return -ENODEV; > } > > + rc = of_property_read_u32(dn, "ibm,chip-id", &chipid); > + if (rc) { > + pr_err("No ibm,chip-id property for %s?\n", pdev->name); > + return -ENODEV; > + } > + > if (pdev->num_resources != 4) { > pr_err("Unexpected DT configuration for [%s, %d]\n", > pdev->name, vasid); > @@ -69,9 +78,22 @@ static int init_vas_instance(struct platform_device *pdev) > > vinst->paste_win_id_shift = 63 - res->end; > > - pr_devel("Initialized instance [%s, %d], paste_base 0x%llx, " > - "paste_win_id_shift 0x%llx\n", pdev->name, vasid, > - vinst->paste_base_addr, vinst->paste_win_id_shift); > + rc = xive_native_alloc_get_irq_info(chipid, &irq, &port); > + if (rc) > + return rc; > + > + vinst->virq = irq_create_mapping(NULL, irq); > + if (!vinst->virq) { > + pr_err("Inst%d: Unable to map global irq %d\n", > + vinst->vas_id, irq); > + return -EINVAL; > + } > + > + vinst->irq_port = port; > + pr_devel("Initialized instance [%s, %d] paste_base 0x%llx paste_win_id_shift 0x%llx IRQ %d Port 0x%llx\n", > + pdev->name, vasid, vinst->paste_base_addr, > + vinst->paste_win_id_shift, vinst->virq, > + vinst->irq_port); > > for_each_possible_cpu(cpu) { > if (cpu_to_chip_id(cpu) == of_get_ibm_chip_id(dn)) > diff --git a/arch/powerpc/platforms/powernv/vas.h b/arch/powerpc/platforms/powernv/vas.h > index 5574aec..598608b 100644 > --- a/arch/powerpc/platforms/powernv/vas.h > +++ b/arch/powerpc/platforms/powernv/vas.h > @@ -313,6 +313,8 @@ struct vas_instance { > u64 paste_base_addr; > u64 paste_win_id_shift; > > + u64 irq_port; > + int virq; > struct mutex mutex; > struct vas_window *rxwin[VAS_COP_TYPE_MAX]; > struct vas_window *windows[VAS_WINDOWS_PER_CHIP]; >
On 3/23/20 10:06 AM, Cédric Le Goater wrote: > On 3/19/20 7:14 AM, Haren Myneni wrote: >> >> Alloc IRQ and get trigger port address for each VAS instance. Kernel >> register this IRQ per VAS instance and sets this port for each send >> window. NX interrupts the kernel when it sees page fault. > > I don't understand why this is not done by the OPAL driver for each VAS > of the system. Is the VAS unit very different from OpenCAPI regarding > the fault ? I checked the previous patchsets and I see that v3 was more like I expected it: one interrupt for faults allocated by the skiboot driver and exposed in the DT. What made you change your mind ? This version is hijacking the lowlevel routines of the XIVE irqchip which is not the best approach. OCXL is doing that because it needs to allocate interrupts for the user space processes using the AFU and we should rework that part. However, the translation fault interrupt is allocated by skiboot. Sorry for the noise, I would like to understand more how this works. I also have passthrough in mind. C.
On Mon, 2020-03-23 at 10:27 +0100, Cédric Le Goater wrote: > On 3/23/20 10:06 AM, Cédric Le Goater wrote: > > On 3/19/20 7:14 AM, Haren Myneni wrote: > >> > >> Alloc IRQ and get trigger port address for each VAS instance. Kernel > >> register this IRQ per VAS instance and sets this port for each send > >> window. NX interrupts the kernel when it sees page fault. > > > > I don't understand why this is not done by the OPAL driver for each VAS > > of the system. Is the VAS unit very different from OpenCAPI regarding > > the fault ? > > I checked the previous patchsets and I see that v3 was more like I expected > it: one interrupt for faults allocated by the skiboot driver and exposed > in the DT. > > What made you change your mind ? > > This version is hijacking the lowlevel routines of the XIVE irqchip which > is not the best approach. OCXL is doing that because it needs to allocate > interrupts for the user space processes using the AFU and we should rework > that part. > > However, the translation fault interrupt is allocated by skiboot. Sorry my mistake. I should have CC you earlier. Each VAS instance will generate fault interrupt which is per chip. There won't be other job completion interrupts. Correct, V3 used allocating interrupts per chip in skiboot and exposed in DT. Since XIVE code has similar feature, exploited this approach so that we do not need skiboot changes. Thanks Haren > > Sorry for the noise, I would like to understand more how this works. I also > have passthrough in mind. > > C. >
On Mon, Mar 23, 2020 at 8:28 PM Cédric Le Goater <clg@kaod.org> wrote: > > On 3/23/20 10:06 AM, Cédric Le Goater wrote: > > On 3/19/20 7:14 AM, Haren Myneni wrote: > >> > >> Alloc IRQ and get trigger port address for each VAS instance. Kernel > >> register this IRQ per VAS instance and sets this port for each send > >> window. NX interrupts the kernel when it sees page fault. > > > > I don't understand why this is not done by the OPAL driver for each VAS > > of the system. Is the VAS unit very different from OpenCAPI regarding > > the fault ? > > I checked the previous patchsets and I see that v3 was more like I expected > it: one interrupt for faults allocated by the skiboot driver and exposed > in the DT. > > What made you change your mind ? From init_vas_inst() in arch/powerpc/platforms/powernv/vas.c: if (pdev->num_resources != 4) { pr_err("Unexpected DT configuration for [%s, %d]\n", pdev->name, vasid); return -ENODEV; } This code should never have been written, but here we are. Due to the above adding an interrupt in the DT makes the driver unable to bind on older kernels. In an older version of the patches (don't think it was posted) Haren was using a non-standard interrupt property and we could work around the problem by going back to that. However, we already have the OPAL calls for allocating / freeing hardware interrupt numbers so why not do that? If we ever want to take advantage of the job completion interrupts we'd want to have the ability to allocate them since the completion interrupts are per-window rather than per-VAS. > This version is hijacking the lowlevel routines of the XIVE irqchip which > is not the best approach. OCXL is doing that because it needs to allocate > interrupts for the user space processes using the AFU and we should rework > that part. What'd you have in mind for the reworking the oxcl interrupt allocation? I didn't find it that objectionable since it's more or less the same as what happens when allocating IPIs. > However, the translation fault interrupt is allocated by skiboot. > > Sorry for the noise, I would like to understand more how this works. I also > have passthrough in mind. > > C.
On 3/23/20 8:02 PM, Haren Myneni wrote: > On Mon, 2020-03-23 at 10:27 +0100, Cédric Le Goater wrote: >> On 3/23/20 10:06 AM, Cédric Le Goater wrote: >>> On 3/19/20 7:14 AM, Haren Myneni wrote: >>>> >>>> Alloc IRQ and get trigger port address for each VAS instance. Kernel >>>> register this IRQ per VAS instance and sets this port for each send >>>> window. NX interrupts the kernel when it sees page fault. >>> >>> I don't understand why this is not done by the OPAL driver for each VAS >>> of the system. Is the VAS unit very different from OpenCAPI regarding >>> the fault ? >> >> I checked the previous patchsets and I see that v3 was more like I expected >> it: one interrupt for faults allocated by the skiboot driver and exposed >> in the DT. >> >> What made you change your mind ? >> >> This version is hijacking the lowlevel routines of the XIVE irqchip which >> is not the best approach. OCXL is doing that because it needs to allocate >> interrupts for the user space processes using the AFU and we should rework >> that part. >> >> However, the translation fault interrupt is allocated by skiboot. > > Sorry my mistake. I should have CC you earlier. > > Each VAS instance will generate fault interrupt which is per chip. There > won't be other job completion interrupts. That's a very good reason to set everything in the skiboot driver and advertise the interrupt number in the DT. The interrupt will be mapped automatically by OF routines and the driver will only have to install an interrupt handler. > Correct, V3 used allocating interrupts per chip in skiboot and exposed > in DT. Since XIVE code has similar feature, exploited this approach so > that we do not need skiboot changes. It's not the same. These are the low level (OPAL) interface used by the XIVE driver. The exception is the KVM XIVE device which needs a finer grain. C.
On 3/24/20 3:26 AM, Oliver O'Halloran wrote: > On Mon, Mar 23, 2020 at 8:28 PM Cédric Le Goater <clg@kaod.org> wrote: >> >> On 3/23/20 10:06 AM, Cédric Le Goater wrote: >>> On 3/19/20 7:14 AM, Haren Myneni wrote: >>>> >>>> Alloc IRQ and get trigger port address for each VAS instance. Kernel >>>> register this IRQ per VAS instance and sets this port for each send >>>> window. NX interrupts the kernel when it sees page fault. >>> >>> I don't understand why this is not done by the OPAL driver for each VAS >>> of the system. Is the VAS unit very different from OpenCAPI regarding >>> the fault ? >> >> I checked the previous patchsets and I see that v3 was more like I expected >> it: one interrupt for faults allocated by the skiboot driver and exposed >> in the DT. >> >> What made you change your mind ? > > From init_vas_inst() in arch/powerpc/platforms/powernv/vas.c: > > if (pdev->num_resources != 4) { > pr_err("Unexpected DT configuration for [%s, %d]\n", > pdev->name, vasid); > return -ENODEV; > } > > This code should never have been written, but here we are. Due to the > above adding an interrupt in the DT makes the driver unable to bind on > older kernels. In an older version of the patches (don't think it was > posted) Haren was using a non-standard interrupt property and we could > work around the problem by going back to that. ok ... :/ I didn't know. Don't we have a rule on LinuxPPC for such things ? Such as, the culprit should send a croissant to everyone involved. > However, we already have the OPAL calls for allocating / freeing > hardware interrupt numbers so why not do that? It's a good way to work around the problem but we are bypassing the irqchip which does other things for the driver. > If we ever want to take > advantage of the job completion interrupts we'd want to have the > ability to allocate them since the completion interrupts are > per-window rather than per-VAS. Yes. That's what I thought it was about to begin with. OCXL has a first implementation of such interrupts. >> This version is hijacking the lowlevel routines of the XIVE irqchip which >> is not the best approach. OCXL is doing that because it needs to allocate >> interrupts for the user space processes using the AFU and we should rework >> that part. > > What'd you have in mind for the reworking the oxcl interrupt allocation? > I didn't find it that objectionable since it's more or less the same as > what happens when allocating IPIs. I think we need to work a bit more on the concepts, on the interfaces, internal at the platform kernel level and at the user space level, and on the configuration, with chip affinity in mind. There are bunch of information on the sources that are retrieved from the firmware or hypervisor that we care about. An irqchip might be the best option for the moment. At the same time, it would be good to keep in mind user interrupts. C. > >> However, the translation fault interrupt is allocated by skiboot. >> >> Sorry for the noise, I would like to understand more how this works. I also >> have passthrough in mind. >> >> C.
On 3/19/20 7:14 AM, Haren Myneni wrote: > > Alloc IRQ and get trigger port address for each VAS instance. Kernel > register this IRQ per VAS instance and sets this port for each send > window. NX interrupts the kernel when it sees page fault. > > Signed-off-by: Haren Myneni <haren@linux.ibm.com> > --- > arch/powerpc/platforms/powernv/vas.c | 34 ++++++++++++++++++++++++++++------ > arch/powerpc/platforms/powernv/vas.h | 2 ++ > 2 files changed, 30 insertions(+), 6 deletions(-) > > diff --git a/arch/powerpc/platforms/powernv/vas.c b/arch/powerpc/platforms/powernv/vas.c > index ed9cc6d..168ab68 100644 > --- a/arch/powerpc/platforms/powernv/vas.c > +++ b/arch/powerpc/platforms/powernv/vas.c > @@ -15,6 +15,7 @@ > #include <linux/of_address.h> > #include <linux/of.h> > #include <asm/prom.h> > +#include <asm/xive.h> > > #include "vas.h" > > @@ -25,10 +26,12 @@ > > static int init_vas_instance(struct platform_device *pdev) > { > - int rc, cpu, vasid; > - struct resource *res; > - struct vas_instance *vinst; > struct device_node *dn = pdev->dev.of_node; > + struct vas_instance *vinst; > + uint32_t chipid, irq; > + struct resource *res; > + int rc, cpu, vasid; > + uint64_t port; > > rc = of_property_read_u32(dn, "ibm,vas-id", &vasid); > if (rc) { > @@ -36,6 +39,12 @@ static int init_vas_instance(struct platform_device *pdev) > return -ENODEV; > } > > + rc = of_property_read_u32(dn, "ibm,chip-id", &chipid); > + if (rc) { > + pr_err("No ibm,chip-id property for %s?\n", pdev->name); > + return -ENODEV; > + } > + > if (pdev->num_resources != 4) { > pr_err("Unexpected DT configuration for [%s, %d]\n", > pdev->name, vasid); > @@ -69,9 +78,22 @@ static int init_vas_instance(struct platform_device *pdev) > > vinst->paste_win_id_shift = 63 - res->end; > > - pr_devel("Initialized instance [%s, %d], paste_base 0x%llx, " > - "paste_win_id_shift 0x%llx\n", pdev->name, vasid, > - vinst->paste_base_addr, vinst->paste_win_id_shift); > + rc = xive_native_alloc_get_irq_info(chipid, &irq, &port); > + if (rc) > + return rc; > + > + vinst->virq = irq_create_mapping(NULL, irq); > + if (!vinst->virq) { > + pr_err("Inst%d: Unable to map global irq %d\n", > + vinst->vas_id, irq); > + return -EINVAL; > + } > + > + vinst->irq_port = port; I would prefer something like this : hwirq = xive_native_alloc_irq_on_chip(chip_id); vinst->virq = irq_create_mapping(NULL, hwirq); if (!vinst->virq) { ... } { struct irq_data *d = irq_get_irq_data(vinst->virq); struct xive_irq_data *xd = irq_data_get_irq_handler_data(d); vinst->irq_port = xd->trig_page; } and dump xive_native_alloc_get_irq_info(). C. > + pr_devel("Initialized instance [%s, %d] paste_base 0x%llx paste_win_id_shift 0x%llx IRQ %d Port 0x%llx\n", > + pdev->name, vasid, vinst->paste_base_addr, > + vinst->paste_win_id_shift, vinst->virq, > + vinst->irq_port); > > for_each_possible_cpu(cpu) { > if (cpu_to_chip_id(cpu) == of_get_ibm_chip_id(dn)) > diff --git a/arch/powerpc/platforms/powernv/vas.h b/arch/powerpc/platforms/powernv/vas.h > index 5574aec..598608b 100644 > --- a/arch/powerpc/platforms/powernv/vas.h > +++ b/arch/powerpc/platforms/powernv/vas.h > @@ -313,6 +313,8 @@ struct vas_instance { > u64 paste_base_addr; > u64 paste_win_id_shift; > > + u64 irq_port; > + int virq; > struct mutex mutex; > struct vas_window *rxwin[VAS_COP_TYPE_MAX]; > struct vas_window *windows[VAS_WINDOWS_PER_CHIP]; >
On Tue, 2020-03-24 at 15:48 +0100, Cédric Le Goater wrote: > On 3/19/20 7:14 AM, Haren Myneni wrote: > > > > Alloc IRQ and get trigger port address for each VAS instance. Kernel > > register this IRQ per VAS instance and sets this port for each send > > window. NX interrupts the kernel when it sees page fault. > > > > Signed-off-by: Haren Myneni <haren@linux.ibm.com> > > --- > > arch/powerpc/platforms/powernv/vas.c | 34 ++++++++++++++++++++++++++++------ > > arch/powerpc/platforms/powernv/vas.h | 2 ++ > > 2 files changed, 30 insertions(+), 6 deletions(-) > > > > diff --git a/arch/powerpc/platforms/powernv/vas.c b/arch/powerpc/platforms/powernv/vas.c > > index ed9cc6d..168ab68 100644 > > --- a/arch/powerpc/platforms/powernv/vas.c > > +++ b/arch/powerpc/platforms/powernv/vas.c > > @@ -15,6 +15,7 @@ > > #include <linux/of_address.h> > > #include <linux/of.h> > > #include <asm/prom.h> > > +#include <asm/xive.h> > > > > #include "vas.h" > > > > @@ -25,10 +26,12 @@ > > > > static int init_vas_instance(struct platform_device *pdev) > > { > > - int rc, cpu, vasid; > > - struct resource *res; > > - struct vas_instance *vinst; > > struct device_node *dn = pdev->dev.of_node; > > + struct vas_instance *vinst; > > + uint32_t chipid, irq; > > + struct resource *res; > > + int rc, cpu, vasid; > > + uint64_t port; > > > > rc = of_property_read_u32(dn, "ibm,vas-id", &vasid); > > if (rc) { > > @@ -36,6 +39,12 @@ static int init_vas_instance(struct platform_device *pdev) > > return -ENODEV; > > } > > > > + rc = of_property_read_u32(dn, "ibm,chip-id", &chipid); > > + if (rc) { > > + pr_err("No ibm,chip-id property for %s?\n", pdev->name); > > + return -ENODEV; > > + } > > + > > if (pdev->num_resources != 4) { > > pr_err("Unexpected DT configuration for [%s, %d]\n", > > pdev->name, vasid); > > @@ -69,9 +78,22 @@ static int init_vas_instance(struct platform_device *pdev) > > > > vinst->paste_win_id_shift = 63 - res->end; > > > > - pr_devel("Initialized instance [%s, %d], paste_base 0x%llx, " > > - "paste_win_id_shift 0x%llx\n", pdev->name, vasid, > > - vinst->paste_base_addr, vinst->paste_win_id_shift); > > + rc = xive_native_alloc_get_irq_info(chipid, &irq, &port); > > + if (rc) > > + return rc; > > + > > + vinst->virq = irq_create_mapping(NULL, irq); > > + if (!vinst->virq) { > > + pr_err("Inst%d: Unable to map global irq %d\n", > > + vinst->vas_id, irq); > > + return -EINVAL; > > + } > > + > > + vinst->irq_port = port; > > > I would prefer something like this : > > hwirq = xive_native_alloc_irq_on_chip(chip_id); > > vinst->virq = irq_create_mapping(NULL, hwirq); > if (!vinst->virq) { > ... > } > > { > struct irq_data *d = irq_get_irq_data(vinst->virq); > struct xive_irq_data *xd = irq_data_get_irq_handler_data(d); > > vinst->irq_port = xd->trig_page; > } > > > and dump xive_native_alloc_get_irq_info(). Thanks Cedric, I will remove patch which adds this function. > > C. >
diff --git a/arch/powerpc/platforms/powernv/vas.c b/arch/powerpc/platforms/powernv/vas.c index ed9cc6d..168ab68 100644 --- a/arch/powerpc/platforms/powernv/vas.c +++ b/arch/powerpc/platforms/powernv/vas.c @@ -15,6 +15,7 @@ #include <linux/of_address.h> #include <linux/of.h> #include <asm/prom.h> +#include <asm/xive.h> #include "vas.h" @@ -25,10 +26,12 @@ static int init_vas_instance(struct platform_device *pdev) { - int rc, cpu, vasid; - struct resource *res; - struct vas_instance *vinst; struct device_node *dn = pdev->dev.of_node; + struct vas_instance *vinst; + uint32_t chipid, irq; + struct resource *res; + int rc, cpu, vasid; + uint64_t port; rc = of_property_read_u32(dn, "ibm,vas-id", &vasid); if (rc) { @@ -36,6 +39,12 @@ static int init_vas_instance(struct platform_device *pdev) return -ENODEV; } + rc = of_property_read_u32(dn, "ibm,chip-id", &chipid); + if (rc) { + pr_err("No ibm,chip-id property for %s?\n", pdev->name); + return -ENODEV; + } + if (pdev->num_resources != 4) { pr_err("Unexpected DT configuration for [%s, %d]\n", pdev->name, vasid); @@ -69,9 +78,22 @@ static int init_vas_instance(struct platform_device *pdev) vinst->paste_win_id_shift = 63 - res->end; - pr_devel("Initialized instance [%s, %d], paste_base 0x%llx, " - "paste_win_id_shift 0x%llx\n", pdev->name, vasid, - vinst->paste_base_addr, vinst->paste_win_id_shift); + rc = xive_native_alloc_get_irq_info(chipid, &irq, &port); + if (rc) + return rc; + + vinst->virq = irq_create_mapping(NULL, irq); + if (!vinst->virq) { + pr_err("Inst%d: Unable to map global irq %d\n", + vinst->vas_id, irq); + return -EINVAL; + } + + vinst->irq_port = port; + pr_devel("Initialized instance [%s, %d] paste_base 0x%llx paste_win_id_shift 0x%llx IRQ %d Port 0x%llx\n", + pdev->name, vasid, vinst->paste_base_addr, + vinst->paste_win_id_shift, vinst->virq, + vinst->irq_port); for_each_possible_cpu(cpu) { if (cpu_to_chip_id(cpu) == of_get_ibm_chip_id(dn)) diff --git a/arch/powerpc/platforms/powernv/vas.h b/arch/powerpc/platforms/powernv/vas.h index 5574aec..598608b 100644 --- a/arch/powerpc/platforms/powernv/vas.h +++ b/arch/powerpc/platforms/powernv/vas.h @@ -313,6 +313,8 @@ struct vas_instance { u64 paste_base_addr; u64 paste_win_id_shift; + u64 irq_port; + int virq; struct mutex mutex; struct vas_window *rxwin[VAS_COP_TYPE_MAX]; struct vas_window *windows[VAS_WINDOWS_PER_CHIP];
Alloc IRQ and get trigger port address for each VAS instance. Kernel register this IRQ per VAS instance and sets this port for each send window. NX interrupts the kernel when it sees page fault. Signed-off-by: Haren Myneni <haren@linux.ibm.com> --- arch/powerpc/platforms/powernv/vas.c | 34 ++++++++++++++++++++++++++++------ arch/powerpc/platforms/powernv/vas.h | 2 ++ 2 files changed, 30 insertions(+), 6 deletions(-)