| Message ID | 20260818213218.875883-1-pawandeep.oza@oss.qualcomm.com |
|---|---|
| Headers | show |
| Series | MPXY interdomain bridge driver | expand |
On Wed, Aug 19, 2026 at 3:02 AM Pawandeep Oza <pawandeep.oza@oss.qualcomm.com> wrote: > > From: Oza Pawandeep <pawandeep.oza@oss.qualcomm.com> > > This series introduces two new capabilities to OpenSBI: > > 1. A domain registration notifier mechanism > 2. RPMI REQUEST_FORWARD Service Group definations > 3. An FDT-configured MPXY bridge driver for inter-domain message passing > > --- Motivation --- > > Modern SBI deployments increasingly require cooperation between multiple > domains sharing the same physical hart — for example a trusted bare-metal > domain and an untrusted domain running on a single > core. While OpenSBI's domain context switching infrastructure provides the > mechanism for hart multiplexing between domains, there was no existing > facility for domains to exchange messages across domain boundaries. > > This series addresses that gap by implementing a bridge over the MPXY > (Message Proxy) extension that uses domain context switching as its > transport. Overall, the series is in right direction but I think you should update the series to support "many-to-one" bridge where messages from multiple source domains can be forwarded to a single target domain. Please see if you cover the above. Regards, Anup > > --- Patch 1: Domain Registration Notifier --- > > The first patch introduces a lightweight notifier infrastructure in > sbi_domain.c. Subsystems can register a callback via > sbi_domain_register_notifier() which is invoked from sbi_domain_register() > whenever a new domain is registered. This allows subsystems to observe > domain registration events and perform domain-specific setup at > registration time. > > The MPXY bridge driver uses this to bind its channels to the named source > and destination domains as they appear during domain registration, without > requiring polling or deferred initialization. > > --- Patch 2: RPMI REQUEST_FORWARD Service Group Definitions --- > > Adds message protocol definitions for the RPMI REQUEST_FORWARD service > group (SERVICEGROUP_ID: 0x000D) to rpmi_msgprot.h. These definitions > implement the wire format for forwarded RPMI request messages between > domains as specified in the RPMI specification. > > New definitions: > > - enum rpmi_request_forward_service_id: service IDs: > REQFWD_ENABLE_NOTIFICATION (0x01) > REQFWD_RETRIEVE_CURRENT_MESSAGE (0x02) > REQFWD_COMPLETE_CURRENT_MESSAGE (0x03) > > - struct rpmi_request_forward_resp: response for SERVICE_ID 0x02 > carrying status, remaining bytes, returned bytes and a flexible > array member for inline message data. > > - struct rpmi_request_complete_resp: response for SERVICE_ID 0x03 > carrying status and the number of pending forwarded messages. > > --- Patch 3: FDT-based MPXY Bridge Driver --- > > The second patch implements the bridge driver itself. It is probed from > the FDT via the compatible string "opensbi,mpxy-bridge" and configured > with four properties: > > - opensbi,source-domain-name > - opensbi,destination-domain-name > - opensbi,source-channel-id > - opensbi,destination-channel-id > > The bridge registers two MPXY channels — one for the source domain > (sender) and one for the destination domain (receiver). It implements > the RPMI REQUEST_FORWARD (SERVICEGROUP_ID: 0x000D) service group: > > - SERVICE_ID 0x02: REQFWD_RETRIEVE_CURRENT_MESSAGE > - SERVICE_ID 0x03: REQFWD_COMPLETE_CURRENT_MESSAGE > > Message delivery uses per-hart slots so that each hart manages its own > message state independently. Domain context switching is performed via > sbi_domain_context_enter() and sbi_domain_context_exit() to cooperatively > yield execution between the source and destination domains on the shared > hart. > > The driver validates at bind time that the source and destination domains > share identical possible_harts masks, enforcing the requirement that > cooperative context switching is only possible when both domains can run > on the same physical hart. > > --- Device Tree Example --- > > The following DT snippet illustrates a two-domain configuration with a > single shared hart (cpu1). The trusted-domain runs a bare-metal application > at 0x80100000 in S-mode. The untrusted-domain runs Linux with access to > the rest of memory. A bridge node connects them: > > opensbi-domains { > compatible = "opensbi,domain,config"; > > tmem: tmem { > compatible = "opensbi,domain,memregion"; > base = <0x0 0x80100000>; > order = <20>; > }; > > tuart: tuart { > compatible = "opensbi,domain,memregion"; > base = <0x0 0x10011000>; > order = <12>; > mmio; > devices = <&uart1>; > }; > > allmem: allmem { > compatible = "opensbi,domain,memregion"; > base = <0x0 0x0>; > order = <64>; > }; > > tdomain: trusted-domain { > compatible = "opensbi,domain,instance"; > possible-harts = <&cpu1>; > regions = <&tmem 0x3f>, <&tuart 0x3f>; > boot-hart = <&cpu1>; > next-arg1 = <0x0 0x0>; > next-addr = <0x0 0x80100000>; > next-mode = <0x1>; > system-reset-allowed; > }; > > udomain: untrusted-domain { > compatible = "opensbi,domain,instance"; > possible-harts = <&cpu1>; > regions = <&tmem 0x0>, <&tuart 0x0>, <&allmem 0x3f>; > }; > > bridge@0 { > compatible = "opensbi,mpxy-bridge"; > opensbi,source-domain-name = "untrusted-domain"; > opensbi,destination-domain-name = "trusted-domain"; > opensbi,source-channel-id = <1>; > opensbi,destination-channel-id = <2>; > }; > }; > > --- Testing --- > > The series was tested on QEMU SiFive Unleashed (sifive_u) with the > above device tree configuration. The test setup consists of: > > - hart1 shared between trusted-domain and untrusted-domain > - trusted-domain: bare-metal application loaded at 0x80100000 > booting in S-mode via the domain's next-addr > - untrusted-domain: Linux kernel with a kernel thread sending > periodic MPXY messages via FID#5 (SEND_MSG_WITH_RESPONSE) > and FID#6 (SEND_MSG_NO_RESPONSE) on MPXY channel 0x1 > - A separate bare-metal telnet console application polling > MPXY channel 0x2 using REQFWD_RETRIEVE_CURRENT_MESSAGE > and REQFWD_COMPLETE_CURRENT_MESSAGE > > The Linux kernel thread sends a test payload ("MPXY-HELLO seq=N") every > 30 seconds alternating between FID#5 (expecting a response) and FID#6 > (fire-and-forget). The bare-metal application retrieves the forwarded > message via the REQFWD service group and echoes it back as the > COMPLETE response. The Linux driver verifies receipt of the echoed > response. > > Both FID#5 and FID#6 paths were exercised and verified end-to-end > across the domain boundary on a single shared hart without requiring > any dedicated inter-processor communication hardware. > > Changes since v1: > deconfig selects CONFIG_FDT_MPXY_BRIDGE > > Oza Pawandeep (3): > lib: sbi: domain: add domain registration notifier infrastructure > sbi_utils: mailbox: add RPMI REQUEST_FORWARD service group definitions > sbi_utils: mpxy: add FDT-based MPXY bridge driver > > include/sbi/sbi_domain.h | 10 + > include/sbi_utils/mailbox/rpmi_msgprot.h | 20 + > lib/sbi/sbi_domain.c | 59 ++ > lib/utils/mpxy/Kconfig | 4 + > lib/utils/mpxy/fdt_mpxy_bridge.c | 676 +++++++++++++++++++++++ > lib/utils/mpxy/objects.mk | 3 + > platform/generic/configs/defconfig | 2 + > 7 files changed, 774 insertions(+) > create mode 100644 lib/utils/mpxy/fdt_mpxy_bridge.c > > -- > 2.43.0 >
From: Oza Pawandeep <pawandeep.oza@oss.qualcomm.com> This series introduces two new capabilities to OpenSBI: 1. A domain registration notifier mechanism 2. RPMI REQUEST_FORWARD Service Group definations 3. An FDT-configured MPXY bridge driver for inter-domain message passing --- Motivation --- Modern SBI deployments increasingly require cooperation between multiple domains sharing the same physical hart — for example a trusted bare-metal domain and an untrusted domain running on a single core. While OpenSBI's domain context switching infrastructure provides the mechanism for hart multiplexing between domains, there was no existing facility for domains to exchange messages across domain boundaries. This series addresses that gap by implementing a bridge over the MPXY (Message Proxy) extension that uses domain context switching as its transport. --- Patch 1: Domain Registration Notifier --- The first patch introduces a lightweight notifier infrastructure in sbi_domain.c. Subsystems can register a callback via sbi_domain_register_notifier() which is invoked from sbi_domain_register() whenever a new domain is registered. This allows subsystems to observe domain registration events and perform domain-specific setup at registration time. The MPXY bridge driver uses this to bind its channels to the named source and destination domains as they appear during domain registration, without requiring polling or deferred initialization. --- Patch 2: RPMI REQUEST_FORWARD Service Group Definitions --- Adds message protocol definitions for the RPMI REQUEST_FORWARD service group (SERVICEGROUP_ID: 0x000D) to rpmi_msgprot.h. These definitions implement the wire format for forwarded RPMI request messages between domains as specified in the RPMI specification. New definitions: - enum rpmi_request_forward_service_id: service IDs: REQFWD_ENABLE_NOTIFICATION (0x01) REQFWD_RETRIEVE_CURRENT_MESSAGE (0x02) REQFWD_COMPLETE_CURRENT_MESSAGE (0x03) - struct rpmi_request_forward_resp: response for SERVICE_ID 0x02 carrying status, remaining bytes, returned bytes and a flexible array member for inline message data. - struct rpmi_request_complete_resp: response for SERVICE_ID 0x03 carrying status and the number of pending forwarded messages. --- Patch 3: FDT-based MPXY Bridge Driver --- The second patch implements the bridge driver itself. It is probed from the FDT via the compatible string "opensbi,mpxy-bridge" and configured with four properties: - opensbi,source-domain-name - opensbi,destination-domain-name - opensbi,source-channel-id - opensbi,destination-channel-id The bridge registers two MPXY channels — one for the source domain (sender) and one for the destination domain (receiver). It implements the RPMI REQUEST_FORWARD (SERVICEGROUP_ID: 0x000D) service group: - SERVICE_ID 0x02: REQFWD_RETRIEVE_CURRENT_MESSAGE - SERVICE_ID 0x03: REQFWD_COMPLETE_CURRENT_MESSAGE Message delivery uses per-hart slots so that each hart manages its own message state independently. Domain context switching is performed via sbi_domain_context_enter() and sbi_domain_context_exit() to cooperatively yield execution between the source and destination domains on the shared hart. The driver validates at bind time that the source and destination domains share identical possible_harts masks, enforcing the requirement that cooperative context switching is only possible when both domains can run on the same physical hart. --- Device Tree Example --- The following DT snippet illustrates a two-domain configuration with a single shared hart (cpu1). The trusted-domain runs a bare-metal application at 0x80100000 in S-mode. The untrusted-domain runs Linux with access to the rest of memory. A bridge node connects them: opensbi-domains { compatible = "opensbi,domain,config"; tmem: tmem { compatible = "opensbi,domain,memregion"; base = <0x0 0x80100000>; order = <20>; }; tuart: tuart { compatible = "opensbi,domain,memregion"; base = <0x0 0x10011000>; order = <12>; mmio; devices = <&uart1>; }; allmem: allmem { compatible = "opensbi,domain,memregion"; base = <0x0 0x0>; order = <64>; }; tdomain: trusted-domain { compatible = "opensbi,domain,instance"; possible-harts = <&cpu1>; regions = <&tmem 0x3f>, <&tuart 0x3f>; boot-hart = <&cpu1>; next-arg1 = <0x0 0x0>; next-addr = <0x0 0x80100000>; next-mode = <0x1>; system-reset-allowed; }; udomain: untrusted-domain { compatible = "opensbi,domain,instance"; possible-harts = <&cpu1>; regions = <&tmem 0x0>, <&tuart 0x0>, <&allmem 0x3f>; }; bridge@0 { compatible = "opensbi,mpxy-bridge"; opensbi,source-domain-name = "untrusted-domain"; opensbi,destination-domain-name = "trusted-domain"; opensbi,source-channel-id = <1>; opensbi,destination-channel-id = <2>; }; }; --- Testing --- The series was tested on QEMU SiFive Unleashed (sifive_u) with the above device tree configuration. The test setup consists of: - hart1 shared between trusted-domain and untrusted-domain - trusted-domain: bare-metal application loaded at 0x80100000 booting in S-mode via the domain's next-addr - untrusted-domain: Linux kernel with a kernel thread sending periodic MPXY messages via FID#5 (SEND_MSG_WITH_RESPONSE) and FID#6 (SEND_MSG_NO_RESPONSE) on MPXY channel 0x1 - A separate bare-metal telnet console application polling MPXY channel 0x2 using REQFWD_RETRIEVE_CURRENT_MESSAGE and REQFWD_COMPLETE_CURRENT_MESSAGE The Linux kernel thread sends a test payload ("MPXY-HELLO seq=N") every 30 seconds alternating between FID#5 (expecting a response) and FID#6 (fire-and-forget). The bare-metal application retrieves the forwarded message via the REQFWD service group and echoes it back as the COMPLETE response. The Linux driver verifies receipt of the echoed response. Both FID#5 and FID#6 paths were exercised and verified end-to-end across the domain boundary on a single shared hart without requiring any dedicated inter-processor communication hardware. Changes since v1: deconfig selects CONFIG_FDT_MPXY_BRIDGE Oza Pawandeep (3): lib: sbi: domain: add domain registration notifier infrastructure sbi_utils: mailbox: add RPMI REQUEST_FORWARD service group definitions sbi_utils: mpxy: add FDT-based MPXY bridge driver include/sbi/sbi_domain.h | 10 + include/sbi_utils/mailbox/rpmi_msgprot.h | 20 + lib/sbi/sbi_domain.c | 59 ++ lib/utils/mpxy/Kconfig | 4 + lib/utils/mpxy/fdt_mpxy_bridge.c | 676 +++++++++++++++++++++++ lib/utils/mpxy/objects.mk | 3 + platform/generic/configs/defconfig | 2 + 7 files changed, 774 insertions(+) create mode 100644 lib/utils/mpxy/fdt_mpxy_bridge.c