| Message ID | 1416901802-24211-39-git-send-email-jiang.liu@linux.intel.com |
|---|---|
| State | Not Applicable |
| Headers | show |
On Tue, Nov 25, 2014 at 12:50 AM, Jiang Liu <jiang.liu@linux.intel.com> wrote: > Parameter vector_alloc should be set to an integer with: > bit 0: enable allocating CPU vector from CPUs on device local node. > That's to allocate from cpumask_of_node(irq_data->node). > bit 1: enable the default policy, which is to allocate from > apic->target_cpus(). > > When allocating vectors, it tries all enabled policies from lower bit > position to higher bit position. > > This option could be use to optimize interrupt distribution on large > system such as NumaChip etc. Why can't we figure this out automatically? Having a kernel parameter is a pain in the neck for users. > Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com> > Cc: Daniel J Blueman <daniel@numascale.com> > --- > Documentation/kernel-parameters.txt | 6 ++++++ > arch/x86/kernel/apic/vector.c | 11 +++++++++++ > 2 files changed, 17 insertions(+) > > diff --git a/Documentation/kernel-parameters.txt b/Documentation/kernel-parameters.txt > index 4c81a860cc2b..a175c5016954 100644 > --- a/Documentation/kernel-parameters.txt > +++ b/Documentation/kernel-parameters.txt > @@ -3709,6 +3709,12 @@ bytes respectively. Such letter suffixes can also be entirely omitted. > vector= [IA-64,SMP] > vector=percpu: enable percpu vector domain > > + vector_alloc= [x86,SMP] > + vector_alloc=policy: policy is a bitmap, bit 0 > + for allocating CPU vector from CPUs on device local > + node; bit 1 for the default policy to allocating from > + apic->target_cpus(). All higher bits are reserved. > + > video= [FB] Frame buffer configuration > See Documentation/fb/modedb.txt. > > diff --git a/arch/x86/kernel/apic/vector.c b/arch/x86/kernel/apic/vector.c > index 16de8906ee1e..1158843551c7 100644 > --- a/arch/x86/kernel/apic/vector.c > +++ b/arch/x86/kernel/apic/vector.c > @@ -79,6 +79,17 @@ void set_vector_alloc_policy(unsigned int policy) > x86_vector_alloc_policy = policy | X86_VECTOR_POL_CALLER; > } > > +static int __init apic_parse_vector_policy(char *str) > +{ > + int policy; > + > + if (get_option(&str, &policy) == 1) > + set_vector_alloc_policy(policy); > + > + return 1; > +} > +__setup("vector_alloc=", apic_parse_vector_policy); > + > static struct apic_chip_data *alloc_apic_chip_data(int node) > { > struct apic_chip_data *data; > -- > 1.7.10.4 > -- To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On 12/01/2014 01:49 PM, Bjorn Helgaas wrote: > On Tue, Nov 25, 2014 at 12:50 AM, Jiang Liu <jiang.liu@linux.intel.com> wrote: >> Parameter vector_alloc should be set to an integer with: >> bit 0: enable allocating CPU vector from CPUs on device local node. >> That's to allocate from cpumask_of_node(irq_data->node). >> bit 1: enable the default policy, which is to allocate from >> apic->target_cpus(). >> >> When allocating vectors, it tries all enabled policies from lower bit >> position to higher bit position. >> >> This option could be use to optimize interrupt distribution on large >> system such as NumaChip etc. > > Why can't we figure this out automatically? Having a kernel parameter > is a pain in the neck for users. For the Numachip case, it would be sufficient to add a call to set_vector_alloc_policy in the APIC driver, as this will always be preferred for local MSI handling. I'll follow up with a patch in the next day or so; further support for other systems can be added later as and when tested. >> Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com> >> Cc: Daniel J Blueman <daniel@numascale.com> >> --- >> Documentation/kernel-parameters.txt | 6 ++++++ >> arch/x86/kernel/apic/vector.c | 11 +++++++++++ >> 2 files changed, 17 insertions(+) >> >> diff --git a/Documentation/kernel-parameters.txt b/Documentation/kernel-parameters.txt >> index 4c81a860cc2b..a175c5016954 100644 >> --- a/Documentation/kernel-parameters.txt >> +++ b/Documentation/kernel-parameters.txt >> @@ -3709,6 +3709,12 @@ bytes respectively. Such letter suffixes can also be entirely omitted. >> vector= [IA-64,SMP] >> vector=percpu: enable percpu vector domain >> >> + vector_alloc= [x86,SMP] >> + vector_alloc=policy: policy is a bitmap, bit 0 >> + for allocating CPU vector from CPUs on device local >> + node; bit 1 for the default policy to allocating from >> + apic->target_cpus(). All higher bits are reserved. >> + >> video= [FB] Frame buffer configuration >> See Documentation/fb/modedb.txt. >> >> diff --git a/arch/x86/kernel/apic/vector.c b/arch/x86/kernel/apic/vector.c >> index 16de8906ee1e..1158843551c7 100644 >> --- a/arch/x86/kernel/apic/vector.c >> +++ b/arch/x86/kernel/apic/vector.c >> @@ -79,6 +79,17 @@ void set_vector_alloc_policy(unsigned int policy) >> x86_vector_alloc_policy = policy | X86_VECTOR_POL_CALLER; >> } >> >> +static int __init apic_parse_vector_policy(char *str) >> +{ >> + int policy; >> + >> + if (get_option(&str, &policy) == 1) >> + set_vector_alloc_policy(policy); >> + >> + return 1; >> +} >> +__setup("vector_alloc=", apic_parse_vector_policy); >> + >> static struct apic_chip_data *alloc_apic_chip_data(int node) >> { >> struct apic_chip_data *data;
On 2014/12/2 2:49, Bjorn Helgaas wrote: > On Tue, Nov 25, 2014 at 12:50 AM, Jiang Liu <jiang.liu@linux.intel.com> wrote: >> Parameter vector_alloc should be set to an integer with: >> bit 0: enable allocating CPU vector from CPUs on device local node. >> That's to allocate from cpumask_of_node(irq_data->node). >> bit 1: enable the default policy, which is to allocate from >> apic->target_cpus(). >> >> When allocating vectors, it tries all enabled policies from lower bit >> position to higher bit position. >> >> This option could be use to optimize interrupt distribution on large >> system such as NumaChip etc. > > Why can't we figure this out automatically? Having a kernel parameter > is a pain in the neck for users. Hi Bjorn, I'm thinking of automatically turning on local vector allocation if any host bridge has an assigned(valid) NUMA node id. By this way, user doesn't need to specify the kernel parameter for most cases. How about this solution? Thanks! Gerry > >> Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com> >> Cc: Daniel J Blueman <daniel@numascale.com> >> --- >> Documentation/kernel-parameters.txt | 6 ++++++ >> arch/x86/kernel/apic/vector.c | 11 +++++++++++ >> 2 files changed, 17 insertions(+) >> >> diff --git a/Documentation/kernel-parameters.txt b/Documentation/kernel-parameters.txt >> index 4c81a860cc2b..a175c5016954 100644 >> --- a/Documentation/kernel-parameters.txt >> +++ b/Documentation/kernel-parameters.txt >> @@ -3709,6 +3709,12 @@ bytes respectively. Such letter suffixes can also be entirely omitted. >> vector= [IA-64,SMP] >> vector=percpu: enable percpu vector domain >> >> + vector_alloc= [x86,SMP] >> + vector_alloc=policy: policy is a bitmap, bit 0 >> + for allocating CPU vector from CPUs on device local >> + node; bit 1 for the default policy to allocating from >> + apic->target_cpus(). All higher bits are reserved. >> + >> video= [FB] Frame buffer configuration >> See Documentation/fb/modedb.txt. >> >> diff --git a/arch/x86/kernel/apic/vector.c b/arch/x86/kernel/apic/vector.c >> index 16de8906ee1e..1158843551c7 100644 >> --- a/arch/x86/kernel/apic/vector.c >> +++ b/arch/x86/kernel/apic/vector.c >> @@ -79,6 +79,17 @@ void set_vector_alloc_policy(unsigned int policy) >> x86_vector_alloc_policy = policy | X86_VECTOR_POL_CALLER; >> } >> >> +static int __init apic_parse_vector_policy(char *str) >> +{ >> + int policy; >> + >> + if (get_option(&str, &policy) == 1) >> + set_vector_alloc_policy(policy); >> + >> + return 1; >> +} >> +__setup("vector_alloc=", apic_parse_vector_policy); >> + >> static struct apic_chip_data *alloc_apic_chip_data(int node) >> { >> struct apic_chip_data *data; >> -- >> 1.7.10.4 >> -- To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Mon, Dec 1, 2014 at 4:45 PM, Jiang Liu <jiang.liu@linux.intel.com> wrote: > > > On 2014/12/2 2:49, Bjorn Helgaas wrote: >> On Tue, Nov 25, 2014 at 12:50 AM, Jiang Liu <jiang.liu@linux.intel.com> wrote: >>> Parameter vector_alloc should be set to an integer with: >>> bit 0: enable allocating CPU vector from CPUs on device local node. >>> That's to allocate from cpumask_of_node(irq_data->node). >>> bit 1: enable the default policy, which is to allocate from >>> apic->target_cpus(). >>> >>> When allocating vectors, it tries all enabled policies from lower bit >>> position to higher bit position. >>> >>> This option could be use to optimize interrupt distribution on large >>> system such as NumaChip etc. >> >> Why can't we figure this out automatically? Having a kernel parameter >> is a pain in the neck for users. > Hi Bjorn, > I'm thinking of automatically turning on local vector > allocation if any host bridge has an assigned(valid) NUMA node id. > By this way, user doesn't need to specify the kernel parameter for most > cases. How about this solution? I'm not an IRQ person, but your idea sounds like figuring this out automatically, and I'm in favor of that :) >>> Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com> >>> Cc: Daniel J Blueman <daniel@numascale.com> >>> --- >>> Documentation/kernel-parameters.txt | 6 ++++++ >>> arch/x86/kernel/apic/vector.c | 11 +++++++++++ >>> 2 files changed, 17 insertions(+) >>> >>> diff --git a/Documentation/kernel-parameters.txt b/Documentation/kernel-parameters.txt >>> index 4c81a860cc2b..a175c5016954 100644 >>> --- a/Documentation/kernel-parameters.txt >>> +++ b/Documentation/kernel-parameters.txt >>> @@ -3709,6 +3709,12 @@ bytes respectively. Such letter suffixes can also be entirely omitted. >>> vector= [IA-64,SMP] >>> vector=percpu: enable percpu vector domain >>> >>> + vector_alloc= [x86,SMP] >>> + vector_alloc=policy: policy is a bitmap, bit 0 >>> + for allocating CPU vector from CPUs on device local >>> + node; bit 1 for the default policy to allocating from >>> + apic->target_cpus(). All higher bits are reserved. >>> + >>> video= [FB] Frame buffer configuration >>> See Documentation/fb/modedb.txt. >>> >>> diff --git a/arch/x86/kernel/apic/vector.c b/arch/x86/kernel/apic/vector.c >>> index 16de8906ee1e..1158843551c7 100644 >>> --- a/arch/x86/kernel/apic/vector.c >>> +++ b/arch/x86/kernel/apic/vector.c >>> @@ -79,6 +79,17 @@ void set_vector_alloc_policy(unsigned int policy) >>> x86_vector_alloc_policy = policy | X86_VECTOR_POL_CALLER; >>> } >>> >>> +static int __init apic_parse_vector_policy(char *str) >>> +{ >>> + int policy; >>> + >>> + if (get_option(&str, &policy) == 1) >>> + set_vector_alloc_policy(policy); >>> + >>> + return 1; >>> +} >>> +__setup("vector_alloc=", apic_parse_vector_policy); >>> + >>> static struct apic_chip_data *alloc_apic_chip_data(int node) >>> { >>> struct apic_chip_data *data; >>> -- >>> 1.7.10.4 >>> -- To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
diff --git a/Documentation/kernel-parameters.txt b/Documentation/kernel-parameters.txt index 4c81a860cc2b..a175c5016954 100644 --- a/Documentation/kernel-parameters.txt +++ b/Documentation/kernel-parameters.txt @@ -3709,6 +3709,12 @@ bytes respectively. Such letter suffixes can also be entirely omitted. vector= [IA-64,SMP] vector=percpu: enable percpu vector domain + vector_alloc= [x86,SMP] + vector_alloc=policy: policy is a bitmap, bit 0 + for allocating CPU vector from CPUs on device local + node; bit 1 for the default policy to allocating from + apic->target_cpus(). All higher bits are reserved. + video= [FB] Frame buffer configuration See Documentation/fb/modedb.txt. diff --git a/arch/x86/kernel/apic/vector.c b/arch/x86/kernel/apic/vector.c index 16de8906ee1e..1158843551c7 100644 --- a/arch/x86/kernel/apic/vector.c +++ b/arch/x86/kernel/apic/vector.c @@ -79,6 +79,17 @@ void set_vector_alloc_policy(unsigned int policy) x86_vector_alloc_policy = policy | X86_VECTOR_POL_CALLER; } +static int __init apic_parse_vector_policy(char *str) +{ + int policy; + + if (get_option(&str, &policy) == 1) + set_vector_alloc_policy(policy); + + return 1; +} +__setup("vector_alloc=", apic_parse_vector_policy); + static struct apic_chip_data *alloc_apic_chip_data(int node) { struct apic_chip_data *data;
Parameter vector_alloc should be set to an integer with: bit 0: enable allocating CPU vector from CPUs on device local node. That's to allocate from cpumask_of_node(irq_data->node). bit 1: enable the default policy, which is to allocate from apic->target_cpus(). When allocating vectors, it tries all enabled policies from lower bit position to higher bit position. This option could be use to optimize interrupt distribution on large system such as NumaChip etc. Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com> Cc: Daniel J Blueman <daniel@numascale.com> --- Documentation/kernel-parameters.txt | 6 ++++++ arch/x86/kernel/apic/vector.c | 11 +++++++++++ 2 files changed, 17 insertions(+)