| Message ID | 20211120000250.1663391-13-ben.widawsky@intel.com |
|---|---|
| State | New |
| Headers | show |
| Series | Add drivers for CXL ports and mem devices | expand |
On Fri, 19 Nov 2021 16:02:39 -0800 Ben Widawsky <ben.widawsky@intel.com> wrote: > Endpoints have decoders too. It is useful to share the same > infrastructure from cxl_core. Endpoints do not have dports (downstream > targets), only the underlying physical medium. As a result, some special > casing is needed. > > There is no functional change introduced yet as endpoints don't actually > enumerate decoders yet. > > Signed-off-by: Ben Widawsky <ben.widawsky@intel.com> I'm not a fan of special values like using 0 here to indicate endpoint device. I'd rather see a base cxl_decode_alloc(..., bool ep) and possibly wrappers for the non ep case and ep one. Jonathan > --- > drivers/cxl/core/bus.c | 41 +++++++++++++++++++++++++++++++++-------- > 1 file changed, 33 insertions(+), 8 deletions(-) > > diff --git a/drivers/cxl/core/bus.c b/drivers/cxl/core/bus.c > index 1ee12a60f3f4..16b15f54fb62 100644 > --- a/drivers/cxl/core/bus.c > +++ b/drivers/cxl/core/bus.c > @@ -187,6 +187,12 @@ static const struct attribute_group *cxl_decoder_switch_attribute_groups[] = { > NULL, > }; > > +static const struct attribute_group *cxl_decoder_endpoint_attribute_groups[] = { > + &cxl_decoder_base_attribute_group, > + &cxl_base_attribute_group, > + NULL, > +}; > + > static void cxl_decoder_release(struct device *dev) > { > struct cxl_decoder *cxld = to_cxl_decoder(dev); > @@ -196,6 +202,12 @@ static void cxl_decoder_release(struct device *dev) > kfree(cxld); > } > > +static const struct device_type cxl_decoder_endpoint_type = { > + .name = "cxl_decoder_endpoint", > + .release = cxl_decoder_release, > + .groups = cxl_decoder_endpoint_attribute_groups, > +}; > + > static const struct device_type cxl_decoder_switch_type = { > .name = "cxl_decoder_switch", > .release = cxl_decoder_release, > @@ -208,6 +220,11 @@ static const struct device_type cxl_decoder_root_type = { > .groups = cxl_decoder_root_attribute_groups, > }; > > +static bool is_endpoint_decoder(struct device *dev) > +{ > + return dev->type == &cxl_decoder_endpoint_type; > +} > + > bool is_root_decoder(struct device *dev) > { > return dev->type == &cxl_decoder_root_type; > @@ -499,7 +516,9 @@ static int decoder_populate_targets(struct cxl_decoder *cxld, > * cxl_decoder_alloc - Allocate a new CXL decoder > * @port: owning port of this decoder > * @nr_targets: downstream targets accessible by this decoder. All upstream > - * ports and root ports must have at least 1 target. > + * ports and root ports must have at least 1 target. Endpoint > + * devices will have 0 targets. Callers wishing to register an > + * endpoint device should specify 0. > * > * A port should contain one or more decoders. Each of those decoders enable > * some address space for CXL.mem utilization. A decoder is expected to be > @@ -516,7 +535,7 @@ struct cxl_decoder *cxl_decoder_alloc(struct cxl_port *port, > struct device *dev; > int rc = 0; > > - if (nr_targets > CXL_DECODER_MAX_INTERLEAVE || nr_targets == 0) > + if (nr_targets > CXL_DECODER_MAX_INTERLEAVE) > return ERR_PTR(-EINVAL); > > cxld = kzalloc(struct_size(cxld, target, nr_targets), GFP_KERNEL); > @@ -535,8 +554,11 @@ struct cxl_decoder *cxl_decoder_alloc(struct cxl_port *port, > dev->parent = &port->dev; > dev->bus = &cxl_bus_type; > > + /* Endpoints don't have a target list */ > + if (nr_targets == 0) > + dev->type = &cxl_decoder_endpoint_type; > /* root ports do not have a cxl_port_type parent */ > - if (port->dev.parent->type == &cxl_port_type) > + else if (port->dev.parent->type == &cxl_port_type) > dev->type = &cxl_decoder_switch_type; > else > dev->type = &cxl_decoder_root_type; > @@ -579,12 +601,15 @@ int cxl_decoder_add(struct cxl_decoder *cxld, int *target_map) > if (cxld->interleave_ways < 1) > return -EINVAL; > > - port = to_cxl_port(cxld->dev.parent); > - rc = decoder_populate_targets(cxld, port, target_map); > - if (rc) > - return rc; > - > dev = &cxld->dev; > + > + port = to_cxl_port(cxld->dev.parent); > + if (!is_endpoint_decoder(dev)) { > + rc = decoder_populate_targets(cxld, port, target_map); > + if (rc) > + return rc; > + } > + > rc = dev_set_name(dev, "decoder%d.%d", port->id, cxld->id); > if (rc) > return rc;
On 21-11-22 16:20:39, Jonathan Cameron wrote: > On Fri, 19 Nov 2021 16:02:39 -0800 > Ben Widawsky <ben.widawsky@intel.com> wrote: > > > Endpoints have decoders too. It is useful to share the same > > infrastructure from cxl_core. Endpoints do not have dports (downstream > > targets), only the underlying physical medium. As a result, some special > > casing is needed. > > > > There is no functional change introduced yet as endpoints don't actually > > enumerate decoders yet. > > > > Signed-off-by: Ben Widawsky <ben.widawsky@intel.com> > > I'm not a fan of special values like using 0 here to indicate endpoint > device. I'd rather see a base cxl_decode_alloc(..., bool ep) > and possibly wrappers for the non ep case and ep one. > > Jonathan > My inclination is the opposite. However, I think you and Dan both brought up something to this effect in the previous RFCs. Dan, do you have a preference here? > > --- > > drivers/cxl/core/bus.c | 41 +++++++++++++++++++++++++++++++++-------- > > 1 file changed, 33 insertions(+), 8 deletions(-) > > > > diff --git a/drivers/cxl/core/bus.c b/drivers/cxl/core/bus.c > > index 1ee12a60f3f4..16b15f54fb62 100644 > > --- a/drivers/cxl/core/bus.c > > +++ b/drivers/cxl/core/bus.c > > @@ -187,6 +187,12 @@ static const struct attribute_group *cxl_decoder_switch_attribute_groups[] = { > > NULL, > > }; > > > > +static const struct attribute_group *cxl_decoder_endpoint_attribute_groups[] = { > > + &cxl_decoder_base_attribute_group, > > + &cxl_base_attribute_group, > > + NULL, > > +}; > > + > > static void cxl_decoder_release(struct device *dev) > > { > > struct cxl_decoder *cxld = to_cxl_decoder(dev); > > @@ -196,6 +202,12 @@ static void cxl_decoder_release(struct device *dev) > > kfree(cxld); > > } > > > > +static const struct device_type cxl_decoder_endpoint_type = { > > + .name = "cxl_decoder_endpoint", > > + .release = cxl_decoder_release, > > + .groups = cxl_decoder_endpoint_attribute_groups, > > +}; > > + > > static const struct device_type cxl_decoder_switch_type = { > > .name = "cxl_decoder_switch", > > .release = cxl_decoder_release, > > @@ -208,6 +220,11 @@ static const struct device_type cxl_decoder_root_type = { > > .groups = cxl_decoder_root_attribute_groups, > > }; > > > > +static bool is_endpoint_decoder(struct device *dev) > > +{ > > + return dev->type == &cxl_decoder_endpoint_type; > > +} > > + > > bool is_root_decoder(struct device *dev) > > { > > return dev->type == &cxl_decoder_root_type; > > @@ -499,7 +516,9 @@ static int decoder_populate_targets(struct cxl_decoder *cxld, > > * cxl_decoder_alloc - Allocate a new CXL decoder > > * @port: owning port of this decoder > > * @nr_targets: downstream targets accessible by this decoder. All upstream > > - * ports and root ports must have at least 1 target. > > + * ports and root ports must have at least 1 target. Endpoint > > + * devices will have 0 targets. Callers wishing to register an > > + * endpoint device should specify 0. > > * > > * A port should contain one or more decoders. Each of those decoders enable > > * some address space for CXL.mem utilization. A decoder is expected to be > > @@ -516,7 +535,7 @@ struct cxl_decoder *cxl_decoder_alloc(struct cxl_port *port, > > struct device *dev; > > int rc = 0; > > > > - if (nr_targets > CXL_DECODER_MAX_INTERLEAVE || nr_targets == 0) > > + if (nr_targets > CXL_DECODER_MAX_INTERLEAVE) > > return ERR_PTR(-EINVAL); > > > > cxld = kzalloc(struct_size(cxld, target, nr_targets), GFP_KERNEL); > > @@ -535,8 +554,11 @@ struct cxl_decoder *cxl_decoder_alloc(struct cxl_port *port, > > dev->parent = &port->dev; > > dev->bus = &cxl_bus_type; > > > > + /* Endpoints don't have a target list */ > > + if (nr_targets == 0) > > + dev->type = &cxl_decoder_endpoint_type; > > /* root ports do not have a cxl_port_type parent */ > > - if (port->dev.parent->type == &cxl_port_type) > > + else if (port->dev.parent->type == &cxl_port_type) > > dev->type = &cxl_decoder_switch_type; > > else > > dev->type = &cxl_decoder_root_type; > > @@ -579,12 +601,15 @@ int cxl_decoder_add(struct cxl_decoder *cxld, int *target_map) > > if (cxld->interleave_ways < 1) > > return -EINVAL; > > > > - port = to_cxl_port(cxld->dev.parent); > > - rc = decoder_populate_targets(cxld, port, target_map); > > - if (rc) > > - return rc; > > - > > dev = &cxld->dev; > > + > > + port = to_cxl_port(cxld->dev.parent); > > + if (!is_endpoint_decoder(dev)) { > > + rc = decoder_populate_targets(cxld, port, target_map); > > + if (rc) > > + return rc; > > + } > > + > > rc = dev_set_name(dev, "decoder%d.%d", port->id, cxld->id); > > if (rc) > > return rc; >
On Mon, Nov 22, 2021 at 11:38 AM Ben Widawsky <ben.widawsky@intel.com> wrote: > > On 21-11-22 16:20:39, Jonathan Cameron wrote: > > On Fri, 19 Nov 2021 16:02:39 -0800 > > Ben Widawsky <ben.widawsky@intel.com> wrote: > > > > > Endpoints have decoders too. It is useful to share the same > > > infrastructure from cxl_core. Endpoints do not have dports (downstream > > > targets), only the underlying physical medium. As a result, some special > > > casing is needed. > > > > > > There is no functional change introduced yet as endpoints don't actually > > > enumerate decoders yet. > > > > > > Signed-off-by: Ben Widawsky <ben.widawsky@intel.com> > > > > I'm not a fan of special values like using 0 here to indicate endpoint > > device. I'd rather see a base cxl_decode_alloc(..., bool ep) > > and possibly wrappers for the non ep case and ep one. > > > > Jonathan > > > > My inclination is the opposite. However, I think you and Dan both brought up > something to this effect in the previous RFCs. > > Dan, do you have a preference here? I was thinking something along the lines of what Jonathan wants, explicit per-type APIs, but internal / private to the core can use heuristics like nr_targets == 0 == endpoint. So unexport cxl_decoder_alloc() and have separate: cxl_root_decoder_alloc() cxl_switch_decoder_alloc() cxl_endpoint_decoder_alloc() ...apis that use a static cxl_decoder_alloc() internally. Probably also wants a cxl_endpoint_decoder_add() that drops the need to pass a NULL @target_map.
On 21-11-24 16:07:23, Dan Williams wrote: > On Mon, Nov 22, 2021 at 11:38 AM Ben Widawsky <ben.widawsky@intel.com> wrote: > > > > On 21-11-22 16:20:39, Jonathan Cameron wrote: > > > On Fri, 19 Nov 2021 16:02:39 -0800 > > > Ben Widawsky <ben.widawsky@intel.com> wrote: > > > > > > > Endpoints have decoders too. It is useful to share the same > > > > infrastructure from cxl_core. Endpoints do not have dports (downstream > > > > targets), only the underlying physical medium. As a result, some special > > > > casing is needed. > > > > > > > > There is no functional change introduced yet as endpoints don't actually > > > > enumerate decoders yet. > > > > > > > > Signed-off-by: Ben Widawsky <ben.widawsky@intel.com> > > > > > > I'm not a fan of special values like using 0 here to indicate endpoint > > > device. I'd rather see a base cxl_decode_alloc(..., bool ep) > > > and possibly wrappers for the non ep case and ep one. > > > > > > Jonathan > > > > > > > My inclination is the opposite. However, I think you and Dan both brought up > > something to this effect in the previous RFCs. > > > > Dan, do you have a preference here? > > I was thinking something along the lines of what Jonathan wants, > explicit per-type APIs, but internal / private to the core can use > heuristics like nr_targets == 0 == endpoint. > > So unexport cxl_decoder_alloc() and have separate: > > cxl_root_decoder_alloc() > cxl_switch_decoder_alloc() > cxl_endpoint_decoder_alloc() > > ...apis that use a static cxl_decoder_alloc() internally. Probably > also wants a cxl_endpoint_decoder_add() that drops the need to pass a > NULL @target_map. Would you a like a prep patch to set up the APIs first, or just do it all in one?
On Mon, Nov 29, 2021 at 12:05 PM Ben Widawsky <ben.widawsky@intel.com> wrote: > > On 21-11-24 16:07:23, Dan Williams wrote: > > On Mon, Nov 22, 2021 at 11:38 AM Ben Widawsky <ben.widawsky@intel.com> wrote: > > > > > > On 21-11-22 16:20:39, Jonathan Cameron wrote: > > > > On Fri, 19 Nov 2021 16:02:39 -0800 > > > > Ben Widawsky <ben.widawsky@intel.com> wrote: > > > > > > > > > Endpoints have decoders too. It is useful to share the same > > > > > infrastructure from cxl_core. Endpoints do not have dports (downstream > > > > > targets), only the underlying physical medium. As a result, some special > > > > > casing is needed. > > > > > > > > > > There is no functional change introduced yet as endpoints don't actually > > > > > enumerate decoders yet. > > > > > > > > > > Signed-off-by: Ben Widawsky <ben.widawsky@intel.com> > > > > > > > > I'm not a fan of special values like using 0 here to indicate endpoint > > > > device. I'd rather see a base cxl_decode_alloc(..., bool ep) > > > > and possibly wrappers for the non ep case and ep one. > > > > > > > > Jonathan > > > > > > > > > > My inclination is the opposite. However, I think you and Dan both brought up > > > something to this effect in the previous RFCs. > > > > > > Dan, do you have a preference here? > > > > I was thinking something along the lines of what Jonathan wants, > > explicit per-type APIs, but internal / private to the core can use > > heuristics like nr_targets == 0 == endpoint. > > > > So unexport cxl_decoder_alloc() and have separate: > > > > cxl_root_decoder_alloc() > > cxl_switch_decoder_alloc() > > cxl_endpoint_decoder_alloc() > > > > ...apis that use a static cxl_decoder_alloc() internally. Probably > > also wants a cxl_endpoint_decoder_add() that drops the need to pass a > > NULL @target_map. > > Would you a like a prep patch to set up the APIs first, or just do it all in > one? Prep patch to switch over the current usages to the new style before introducing more helpers sounds good to me.
On 21-11-29 12:07:00, Dan Williams wrote: > On Mon, Nov 29, 2021 at 12:05 PM Ben Widawsky <ben.widawsky@intel.com> wrote: > > > > On 21-11-24 16:07:23, Dan Williams wrote: > > > On Mon, Nov 22, 2021 at 11:38 AM Ben Widawsky <ben.widawsky@intel.com> wrote: > > > > > > > > On 21-11-22 16:20:39, Jonathan Cameron wrote: > > > > > On Fri, 19 Nov 2021 16:02:39 -0800 > > > > > Ben Widawsky <ben.widawsky@intel.com> wrote: > > > > > > > > > > > Endpoints have decoders too. It is useful to share the same > > > > > > infrastructure from cxl_core. Endpoints do not have dports (downstream > > > > > > targets), only the underlying physical medium. As a result, some special > > > > > > casing is needed. > > > > > > > > > > > > There is no functional change introduced yet as endpoints don't actually > > > > > > enumerate decoders yet. > > > > > > > > > > > > Signed-off-by: Ben Widawsky <ben.widawsky@intel.com> > > > > > > > > > > I'm not a fan of special values like using 0 here to indicate endpoint > > > > > device. I'd rather see a base cxl_decode_alloc(..., bool ep) > > > > > and possibly wrappers for the non ep case and ep one. > > > > > > > > > > Jonathan > > > > > > > > > > > > > My inclination is the opposite. However, I think you and Dan both brought up > > > > something to this effect in the previous RFCs. > > > > > > > > Dan, do you have a preference here? > > > > > > I was thinking something along the lines of what Jonathan wants, > > > explicit per-type APIs, but internal / private to the core can use > > > heuristics like nr_targets == 0 == endpoint. > > > > > > So unexport cxl_decoder_alloc() and have separate: > > > > > > cxl_root_decoder_alloc() > > > cxl_switch_decoder_alloc() > > > cxl_endpoint_decoder_alloc() > > > > > > ...apis that use a static cxl_decoder_alloc() internally. Probably > > > also wants a cxl_endpoint_decoder_add() that drops the need to pass a > > > NULL @target_map. > > > > Would you a like a prep patch to set up the APIs first, or just do it all in > > one? > > Prep patch to switch over the current usages to the new style before > introducing more helpers sounds good to me. Thanks for the suggestions... Looking again, I think it makes sense to squash it into the patch before this which documents and tightens up this exact API.
diff --git a/drivers/cxl/core/bus.c b/drivers/cxl/core/bus.c index 1ee12a60f3f4..16b15f54fb62 100644 --- a/drivers/cxl/core/bus.c +++ b/drivers/cxl/core/bus.c @@ -187,6 +187,12 @@ static const struct attribute_group *cxl_decoder_switch_attribute_groups[] = { NULL, }; +static const struct attribute_group *cxl_decoder_endpoint_attribute_groups[] = { + &cxl_decoder_base_attribute_group, + &cxl_base_attribute_group, + NULL, +}; + static void cxl_decoder_release(struct device *dev) { struct cxl_decoder *cxld = to_cxl_decoder(dev); @@ -196,6 +202,12 @@ static void cxl_decoder_release(struct device *dev) kfree(cxld); } +static const struct device_type cxl_decoder_endpoint_type = { + .name = "cxl_decoder_endpoint", + .release = cxl_decoder_release, + .groups = cxl_decoder_endpoint_attribute_groups, +}; + static const struct device_type cxl_decoder_switch_type = { .name = "cxl_decoder_switch", .release = cxl_decoder_release, @@ -208,6 +220,11 @@ static const struct device_type cxl_decoder_root_type = { .groups = cxl_decoder_root_attribute_groups, }; +static bool is_endpoint_decoder(struct device *dev) +{ + return dev->type == &cxl_decoder_endpoint_type; +} + bool is_root_decoder(struct device *dev) { return dev->type == &cxl_decoder_root_type; @@ -499,7 +516,9 @@ static int decoder_populate_targets(struct cxl_decoder *cxld, * cxl_decoder_alloc - Allocate a new CXL decoder * @port: owning port of this decoder * @nr_targets: downstream targets accessible by this decoder. All upstream - * ports and root ports must have at least 1 target. + * ports and root ports must have at least 1 target. Endpoint + * devices will have 0 targets. Callers wishing to register an + * endpoint device should specify 0. * * A port should contain one or more decoders. Each of those decoders enable * some address space for CXL.mem utilization. A decoder is expected to be @@ -516,7 +535,7 @@ struct cxl_decoder *cxl_decoder_alloc(struct cxl_port *port, struct device *dev; int rc = 0; - if (nr_targets > CXL_DECODER_MAX_INTERLEAVE || nr_targets == 0) + if (nr_targets > CXL_DECODER_MAX_INTERLEAVE) return ERR_PTR(-EINVAL); cxld = kzalloc(struct_size(cxld, target, nr_targets), GFP_KERNEL); @@ -535,8 +554,11 @@ struct cxl_decoder *cxl_decoder_alloc(struct cxl_port *port, dev->parent = &port->dev; dev->bus = &cxl_bus_type; + /* Endpoints don't have a target list */ + if (nr_targets == 0) + dev->type = &cxl_decoder_endpoint_type; /* root ports do not have a cxl_port_type parent */ - if (port->dev.parent->type == &cxl_port_type) + else if (port->dev.parent->type == &cxl_port_type) dev->type = &cxl_decoder_switch_type; else dev->type = &cxl_decoder_root_type; @@ -579,12 +601,15 @@ int cxl_decoder_add(struct cxl_decoder *cxld, int *target_map) if (cxld->interleave_ways < 1) return -EINVAL; - port = to_cxl_port(cxld->dev.parent); - rc = decoder_populate_targets(cxld, port, target_map); - if (rc) - return rc; - dev = &cxld->dev; + + port = to_cxl_port(cxld->dev.parent); + if (!is_endpoint_decoder(dev)) { + rc = decoder_populate_targets(cxld, port, target_map); + if (rc) + return rc; + } + rc = dev_set_name(dev, "decoder%d.%d", port->id, cxld->id); if (rc) return rc;
Endpoints have decoders too. It is useful to share the same infrastructure from cxl_core. Endpoints do not have dports (downstream targets), only the underlying physical medium. As a result, some special casing is needed. There is no functional change introduced yet as endpoints don't actually enumerate decoders yet. Signed-off-by: Ben Widawsky <ben.widawsky@intel.com> --- drivers/cxl/core/bus.c | 41 +++++++++++++++++++++++++++++++++-------- 1 file changed, 33 insertions(+), 8 deletions(-)