| Message ID | 20210309164030.27361-1-christian.storm@siemens.com |
|---|---|
| State | Rejected |
| Headers | show |
| Series | Add global device ID | expand |
Hi Christian, Dominik, I have some general questions that are quite independent from the implementation: On 09.03.21 17:40, Christian Storm wrote: > From: Dominik Tacke <dominik.tacke@siemens.com> > > Currently, a device ID is used for suricatta modules only. > However, the remote handler can make use of a device ID as > well such that this device ID is used to construct the > remote handler's ZeroMQ socket name, enabling multiple > SWUpdate instances to serve different remotes per different > socket names. I have raised an issue about the remote handler (see https://groups.google.com/g/swupdate/c/5kun17vfu5Y/m/ok5fhnNmCAAJ), Bastian thinks that there is no problems with the licensing. Anyway, this and the fact that the remote handler seems overkilling and it is quite unused, I was thinking about to remove it from SWUpdate. But it looks like there are some users...(you !). Do you really have use case with zeroMQ ? Someone submitted once a patch to substitute zeroMQ with nanoMSG, but there was no follow up. Any good reasons for zeroMQ, apart that it is already part of SWUpdate (it is a good reason, too) ? This patch seems matching your use case and your project, but it seems to me that introduce an artificial connection between the Hawkbit (not suricatta) ID (specified in Hawkbit DDI interface) with a general ID for the device. The "general server" in suricatta uses values from the "identify" section in configuration file, for example. The usage in remote handler is very specific - what about if I have both remote handler and Hawkbit ? It could be true that the ID is different. Should this ID not part of remote handler configuration, that means in sw-description as one of the properties ? Or if not, why is the same as Hakwbit ID ? Note that Hawkbit ID can be retrieved from a "suricatta" section in configuration file to be compatible with the past, or in a more specific "hawkbit" section - because tenant / id are defined in Hawkbit DDI and belong to this interface. > > If no explicit suricatta device ID is given, the global > device ID is used, making the special suricatta device ID > argument optional the global device ID option is given. > > Signed-off-by: Dominik Tacke <dominik.tacke@siemens.com> > Signed-off-by: Christian Storm <christian.storm@siemens.com> > --- > core/swupdate.c | 10 +++++++++- > doc/source/swupdate.rst | 8 ++++++++ > examples/configuration/swupdate.cfg | 3 +++ > handlers/remote_handler.c | 8 +++++--- > include/swupdate.h | 1 + > suricatta/server_hawkbit.c | 12 ++++++++++++ > 6 files changed, 38 insertions(+), 4 deletions(-) > > diff --git a/core/swupdate.c b/core/swupdate.c > index e959568..f823f83 100644 > --- a/core/swupdate.c > +++ b/core/swupdate.c > @@ -84,6 +84,7 @@ static struct option long_options[] = { > {"no-reinstalling", required_argument, NULL, 'R'}, > {"no-transaction-marker", no_argument, NULL, 'M'}, > {"no-state-marker", no_argument, NULL, 'm'}, > + {"device-id", required_argument, NULL, 'D'}, > #ifdef CONFIG_SIGNED_IMAGES > {"key", required_argument, NULL, 'k'}, > {"ca-path", required_argument, NULL, 'k'}, > @@ -156,6 +157,7 @@ static void usage(char *programname) > " -M, --no-transaction-marker : disable setting bootloader transaction marker\n" > " -m, --no-state-marker : disable setting update state in bootloader\n" > " -o, --output <filename> : saves the incoming stream\n" > + " -D, --device-id <device id> : defines this device's ID\n" > " -v, --verbose : be verbose, set maximum loglevel\n" > " --version : print SWUpdate version and exit\n" > #ifdef CONFIG_HW_COMPATIBILITY > @@ -374,6 +376,8 @@ static int read_globals_settings(void *elem, void *data) > /* by convention, errors in a configuration section are ignored */ > (void)parse_image_selector(software_select, sw); > } > + GET_FIELD_STRING(LIBCFG_PARSER, elem, > + "device-id", sw->globals.device_id); > > return 0; > } > @@ -482,7 +486,7 @@ int main(int argc, char **argv) > #endif > memset(main_options, 0, sizeof(main_options)); > memset(image_url, 0, sizeof(image_url)); > - strcpy(main_options, "vhni:e:q:l:Lcf:p:P:o:N:R:Mm"); > + strcpy(main_options, "vhni:e:q:l:Lcf:p:P:o:N:R:MmD:"); > #ifdef CONFIG_MTD > strcat(main_options, "b:"); > #endif > @@ -633,6 +637,10 @@ int main(int argc, char **argv) > optarg, > sizeof(swcfg.globals.publickeyfname)); > break; > + case 'D': > + strlcpy(swcfg.globals.device_id, optarg, > + sizeof(swcfg.globals.device_id)); > + break; > case '1': > swcfg.globals.cert_purpose = parse_cert_purpose(optarg); > break; > diff --git a/doc/source/swupdate.rst b/doc/source/swupdate.rst > index 22d135a..dd1bc6e 100644 > --- a/doc/source/swupdate.rst > +++ b/doc/source/swupdate.rst > @@ -548,6 +548,12 @@ Command line parameters > +-------------+----------+--------------------------------------------+ > | -p <cmd> | string | Execute post-update command. | > +-------------+----------+--------------------------------------------+ > +| -D <id> | string | Define a device ID. This ID will be used | > +| | | for suricatta when no hawkBit ID is given. | > +| | | This device ID will be blended into the | > +| | | remote handler's zmq socket. | > ++-------------+----------+--------------------------------------------+ > + > > Downloader command line parameters > .................................. > @@ -592,6 +598,8 @@ Mandatory arguments are marked with '\*': > | | | e.g., localhost:8080 | > +-------------------------+----------+--------------------------------------------+ > | -i <id> | integer | \* The device ID to communicate to hawkBit.| > +| | | This argument is optional if a device ID | > +| | | argument is given to SWUpdate (-D option) | > +-------------------------+----------+--------------------------------------------+ > | -c <confirm> | integer | Confirm update status to server: 1=AGAIN, | > | | | 2=SUCCESS, 3=FAILED | > diff --git a/examples/configuration/swupdate.cfg b/examples/configuration/swupdate.cfg > index a97f982..36356d1 100644 > --- a/examples/configuration/swupdate.cfg > +++ b/examples/configuration/swupdate.cfg > @@ -40,6 +40,8 @@ > # set expected common name of signer certificate > # select: : string > # select software images set and source (<software>,<mode>) > +# device-id : string > +# define a device ID > globals : > { > > @@ -48,6 +50,7 @@ globals : > syslog = true; > /* public-key-file = "test.pem";*/ > mtd-blacklist = "0 1 2 3 4 5 6"; > + device-id = "Default_ID"; > }; > > # logcolors : set colors for output to stdout / stderr > diff --git a/handlers/remote_handler.c b/handlers/remote_handler.c > index 275859f..bc3b642 100644 > --- a/handlers/remote_handler.c > +++ b/handlers/remote_handler.c > @@ -159,8 +159,10 @@ static int install_remote_image(struct img_type *img, > struct RHmsg RHmessage; > char bufcmd[80]; > > - len = strlen(img->type_data) + strlen(CONFIG_SOCKET_REMOTE_HANDLER_DIRECTORY) + strlen("ipc://") + 4; > + char* device_id = get_swupdate_cfg()->globals.device_id; > > + len = strlen(img->type_data) + strlen(CONFIG_SOCKET_REMOTE_HANDLER_DIRECTORY) > + + strlen(device_id) + strlen("ipc://") + 4; > /* > * Allocate maximum string > */ > @@ -169,8 +171,8 @@ static int install_remote_image(struct img_type *img, > ERROR("Not enough memory"); > return -ENOMEM; > } > - snprintf(connect_string, len, "ipc://%s%s", CONFIG_SOCKET_REMOTE_HANDLER_DIRECTORY, > - img->type_data); > + snprintf(connect_string, len, "ipc://%s%s%s", CONFIG_SOCKET_REMOTE_HANDLER_DIRECTORY, > + device_id, img->type_data); > > ret = zmq_connect(request, connect_string); > if (ret < 0) { > diff --git a/include/swupdate.h b/include/swupdate.h > index e607ed6..bc55970 100644 > --- a/include/swupdate.h > +++ b/include/swupdate.h > @@ -125,6 +125,7 @@ enum { > struct swupdate_global_cfg { > int verbose; > char mtdblacklist[SWUPDATE_GENERAL_STRING_SIZE]; > + char device_id[SWUPDATE_GENERAL_STRING_SIZE]; > int loglevel; > int syslog_enabled; > int dry_run; > diff --git a/suricatta/server_hawkbit.c b/suricatta/server_hawkbit.c > index ed4c800..da133ba 100644 > --- a/suricatta/server_hawkbit.c > +++ b/suricatta/server_hawkbit.c > @@ -1766,6 +1766,18 @@ server_op_res_t server_start(char *fname, int argc, char *argv[]) > } > } > > + /* > + * If no hawkBit device ID has been set, check if the global > + * device ID is set and, if yes, use it as hawkBit device ID. > + */ > + if ((mandatory_argument_count & ID_BIT) != ID_BIT) { > + char* id = get_swupdate_cfg()->globals.device_id; > + if (strnlen(id, SWUPDATE_GENERAL_STRING_SIZE)) { > + SETSTRING(server_hawkbit.device_id, id); > + mandatory_argument_count |= ID_BIT; > + } > + } > + > if (mandatory_argument_count != ALL_MANDATORY_SET) { > fprintf(stderr, "Mandatory arguments missing!\n"); > suricatta_print_help(); > Best regards, Stefano Babic
Hi Stefano, > I have some general questions that are quite independent from the > implementation: Sure, I'll try to answer them :) > > Currently, a device ID is used for suricatta modules only. > > However, the remote handler can make use of a device ID as > > well such that this device ID is used to construct the > > remote handler's ZeroMQ socket name, enabling multiple > > SWUpdate instances to serve different remotes per different > > socket names. > > I have raised an issue about the remote handler (see > https://groups.google.com/g/swupdate/c/5kun17vfu5Y/m/ok5fhnNmCAAJ), Bastian > thinks that there is no problems with the licensing. Anyway, this and the fact > that the remote handler seems overkilling and it is quite unused, I was > thinking about to remove it from SWUpdate. But it looks like there are some > users...(you !). Do you really have use case with zeroMQ ? Not in particular ZeroMQ but a means to attach existing infrastructure: The use case is to have one SWUpdate instance running on a gateway per to be updated device attached to the gateway via some communication channel (e.g., UART). So, SWUpdate is in effect a proxy that may itself interact with, e.g., hawkBit and then applies an update to an attached device via something like the remote handler that orchestrates custom device-specific or even vendor-proprietary update software. As it happens, ZeroMQ is in SWUpdate and is in the device-specific update software, hence we used that. Anyway, as long as there is a means to couple an existing software to SWUpdate in a bi-directional manner (send command+data, receive status and probably push it to, e.g., hawkBit), it's fine. > Someone submitted once a patch to substitute zeroMQ with nanoMSG, but there > was no follow up. Any good reasons for zeroMQ, apart that it is already part > of SWUpdate (it is a good reason, too) ? Nope, see above. > This patch seems matching your use case and your project, but it seems to me > that introduce an artificial connection between the Hawkbit (not suricatta) ID > (specified in Hawkbit DDI interface) with a general ID for the device. The > "general server" in suricatta uses values from the "identify" section in > configuration file, for example. Yes, but both are effectively "device IDs", just named differently and in different locations. So, for the simple case of SWUpdate updating the device it's running on, there's in fact effectively only one global device ID. Consolidating this to one global device ID could make sense? In the case of SWUpdate running as proxy (and not for the device it's running on) it depends on whether we (1) have one SWUpdate instance for all attached devices or (2) have multiple SWUpdate instances, one for each attached device. In case (2), it's the same as above, i.e., a global device ID in this case representing the attached device. In case (1), this is clearly not sufficient as one SWUpdate instance needs to handle multiple device IDs and there's more work to do in this case anyway (e.g., query hawkBit for multiple devices → multiple suricatta processes, concurrent installation to attached devices, ...). > The usage in remote handler is very specific - what about if I have both > remote handler and Hawkbit ? It could be true that the ID is different. Yes, if you give a specific hawkBit device ID, that one has preference and is taken. Only if no specific hawkBit device ID is given, the global device ID is used (for convenience) while it's used for the remote handler unconditionally. > Should this ID not part of remote handler configuration, that means in > sw-description as one of the properties ? Or if not, why is the same as > Hakwbit ID ? The remote handler configuration section could be another option, yes. Putting it in sw-description is not a that good solution as the update artifact is then tied to the device, i.e., you cannot install the very same artifact on two different attached devices although they're identical and such an update artifact could be installed on all (or a subset) of the attached devices. > Note that Hawkbit ID can be retrieved from a "suricatta" section in > configuration file to be compatible with the past, or in a more specific > "hawkbit" section - because tenant / id are defined in Hawkbit DDI and belong > to this interface. Yes, but overridden by command line. The flow with this patch is as follows: (1) read config file (both sections) and |= ID_BIT (2) read specific hawkBit device ID parameter and |= ID_BIT (3) read global device ID and |= ID_BIT (4) complain about missing device ID. Kind regards, Christian
Hi Christian, On 12.03.21 16:44, Christian Storm wrote: > Hi Stefano, > >> I have some general questions that are quite independent from the >> implementation: > > Sure, I'll try to answer them :) > > >>> Currently, a device ID is used for suricatta modules only. >>> However, the remote handler can make use of a device ID as >>> well such that this device ID is used to construct the >>> remote handler's ZeroMQ socket name, enabling multiple >>> SWUpdate instances to serve different remotes per different >>> socket names. >> >> I have raised an issue about the remote handler (see >> https://groups.google.com/g/swupdate/c/5kun17vfu5Y/m/ok5fhnNmCAAJ), Bastian >> thinks that there is no problems with the licensing. Anyway, this and the fact >> that the remote handler seems overkilling and it is quite unused, I was >> thinking about to remove it from SWUpdate. But it looks like there are some >> users...(you !). Do you really have use case with zeroMQ ? > > Not in particular ZeroMQ but a means to attach existing infrastructure: > The use case is to have one SWUpdate instance running on a gateway per > to be updated device attached to the gateway via some communication > channel (e.g., UART). So, SWUpdate is in effect a proxy that may itself > interact with, e.g., hawkBit and then applies an update to an attached > device via something like the remote handler that orchestrates custom > device-specific or even vendor-proprietary update software. This is the exact use case, it allows to send an image to a proprietary installer whose code is not allowed to be GPL. The best technical solution remains to convert it in a SWUpdate handler and link to SWUpdate., so that SWUpdate can control it and we have the whole output in SWUpdate log. With remote handler, we have just the result of ZeroMQ messages. "remote handler" in SWUpdate is really "local" : a connection is just via IPC to another process on the same device. > As it happens, ZeroMQ is in SWUpdate and is in the device-specific > update software, hence we used that. > Anyway, as long as there is a means to couple an existing software to > SWUpdate in a bi-directional manner (send command+data, receive status > and probably push it to, e.g., hawkBit), it's fine. Ok, got it - if there will be no license issue, remote handler based on ZeroMQ can remain part of project. > > >> Someone submitted once a patch to substitute zeroMQ with nanoMSG, but there >> was no follow up. Any good reasons for zeroMQ, apart that it is already part >> of SWUpdate (it is a good reason, too) ? > > Nope, see above. > > >> This patch seems matching your use case and your project, but it seems to me >> that introduce an artificial connection between the Hawkbit (not suricatta) ID >> (specified in Hawkbit DDI interface) with a general ID for the device. The >> "general server" in suricatta uses values from the "identify" section in >> configuration file, for example. > > Yes, but both are effectively "device IDs", just named differently and in > different locations. But we cannot know what the backend consider a unique device ID, and it can be the sum of different fileds (mac address, serial number, etc.). The rule is on the backend. So yes, a general device id can be also added, it is useless in case of update via USB / Webserver. > So, for the simple case of SWUpdate updating the > device it's running on, there's in fact effectively only one global device ID. Mmmhhhh...it seems your use case is different as the one I supposed. > Consolidating this to one global device ID could make sense? I am trying to understand this. > > In the case of SWUpdate running as proxy (and not for the device it's > running on) it depends on whether we > (1) have one SWUpdate instance for all attached devices or If all devices are running SWUpdate and one of them is working as gateway proxy, the solution in SWUpdate is via the swuforward handler. This allows also a "network update" and guarantees that an update is successful if all components are updated. If the other devices are on the same device as the one running SWUpdate (else the remote handler does not work), the whole device itself should be seen as a single device. It is questionable if the remote handler should have multiple sockets, as you propose. I think it is better that there is just one socket, but another way to identify the devices, so the id is put in sw-description (then as part of properties for the handler). I do not bother if the protocol should be changed to support multiple devices as you need. The device id you want to add belong really to the subdevice, because this is what you want to identify with a separate connection (one socket for each device). But the remote handler can send messages where there is a device ID field (and broadcast should be allowed). > (2) have multiple SWUpdate instances, one for each attached device. I do not get the use case, it looks to me you have still several devices. This does not scale very well, and there is no need that whole instances are running at once - and then, there is no reason for multiple instances. > In case (2), it's the same as above, i.e., a global device ID in this > case representing the attached device. If the device id is representing the subdevice, it does not represent the main device. > In case (1), this is clearly not sufficient as one SWUpdate instance > needs to handle multiple device IDs and there's more work to do in this > case anyway (e.g., query hawkBit for multiple devices → multiple > suricatta processes, concurrent installation to attached devices, ...). Which kind of devices are we talking about ? Remote handler is thought for small devices, no linux, with microcontrollers. this leads to smaller image size if compared to the one on the main device. Could you not have a single SWU on Hawkbit for all these devices ? > > >> The usage in remote handler is very specific - what about if I have both >> remote handler and Hawkbit ? It could be true that the ID is different. > > Yes, if you give a specific hawkBit device ID, that one has preference > and is taken. Only if no specific hawkBit device ID is given, the global > device ID is used (for convenience) while it's used for the remote > handler unconditionally. > > >> Should this ID not part of remote handler configuration, that means in >> sw-description as one of the properties ? Or if not, why is the same as >> Hakwbit ID ? > > The remote handler configuration section could be another option, yes. > Putting it in sw-description is not a that good solution as the update > artifact is then tied to the device, i.e., you cannot install the very > same artifact on two different attached devices although they're > identical and such an update artifact could be installed on all (or > a subset) of the attached devices. And something like in swuforwarder where the properties is an array of device IDs ? You can send the same image to all of them. > > >> Note that Hawkbit ID can be retrieved from a "suricatta" section in >> configuration file to be compatible with the past, or in a more specific >> "hawkbit" section - because tenant / id are defined in Hawkbit DDI and belong >> to this interface. > > Yes, but overridden by command line. The flow with this patch is as follows: > (1) read config file (both sections) and |= ID_BIT > (2) read specific hawkBit device ID parameter and |= ID_BIT > (3) read global device ID and |= ID_BIT > (4) complain about missing device ID. Best regards, Stefano
Hi Stefano, > > > > Currently, a device ID is used for suricatta modules only. > > > > However, the remote handler can make use of a device ID as > > > > well such that this device ID is used to construct the > > > > remote handler's ZeroMQ socket name, enabling multiple > > > > SWUpdate instances to serve different remotes per different > > > > socket names. > > > > > > I have raised an issue about the remote handler (see > > > https://groups.google.com/g/swupdate/c/5kun17vfu5Y/m/ok5fhnNmCAAJ), Bastian > > > thinks that there is no problems with the licensing. Anyway, this and the fact > > > that the remote handler seems overkilling and it is quite unused, I was > > > thinking about to remove it from SWUpdate. But it looks like there are some > > > users...(you !). Do you really have use case with zeroMQ ? > > > > Not in particular ZeroMQ but a means to attach existing infrastructure: > > The use case is to have one SWUpdate instance running on a gateway per > > to be updated device attached to the gateway via some communication > > channel (e.g., UART). So, SWUpdate is in effect a proxy that may itself > > interact with, e.g., hawkBit and then applies an update to an attached > > device via something like the remote handler that orchestrates custom > > device-specific or even vendor-proprietary update software. > > This is the exact use case, it allows to send an image to a proprietary > installer whose code is not allowed to be GPL. The best technical solution > remains to convert it in a SWUpdate handler and link to SWUpdate., so that > SWUpdate can control it and we have the whole output in SWUpdate log. With > remote handler, we have just the result of ZeroMQ messages. > > "remote handler" in SWUpdate is really "local" : a connection is just via IPC > to another process on the same device. Yes, exactly, it's a local bidirectional channel to some other software. > > As it happens, ZeroMQ is in SWUpdate and is in the device-specific > > update software, hence we used that. > > Anyway, as long as there is a means to couple an existing software to > > SWUpdate in a bi-directional manner (send command+data, receive status > > and probably push it to, e.g., hawkBit), it's fine. > > Ok, got it - if there will be no license issue, remote handler based on ZeroMQ > can remain part of project. Well, it mustn't be ZeroMQ, e.g., the UDS/pipe handler proposed recently does also work given we have a protocol to speak over the pipe. Then, we can get rid of ZeroMQ but don't lose the remote handler functionality? > > > This patch seems matching your use case and your project, but it seems to me > > > that introduce an artificial connection between the Hawkbit (not suricatta) ID > > > (specified in Hawkbit DDI interface) with a general ID for the device. The > > > "general server" in suricatta uses values from the "identify" section in > > > configuration file, for example. > > > > Yes, but both are effectively "device IDs", just named differently and in > > different locations. > > But we cannot know what the backend consider a unique device ID, and it can be > the sum of different fileds (mac address, serial number, etc.). The rule is on > the backend. So yes, a general device id can be also added, it is useless in > case of update via USB / Webserver. Yes, in those cases it's useless as "you" resolved the device ID by accessing the device directly, true. However, it can still be used for displaying information, e.g., showing the device ID in the webserver. We cannot know, true, but the integrator does know and configures the device ID accordingly and as the backend wants it to be. From an SWUpdate point of view, this could be a random string without semantics. > > So, for the simple case of SWUpdate updating the > > device it's running on, there's in fact effectively only one global device ID. > > Mmmhhhh...it seems your use case is different as the one I supposed. > > > Consolidating this to one global device ID could make sense? > > I am trying to understand this. > > > > > In the case of SWUpdate running as proxy (and not for the device it's > > running on) it depends on whether we > > (1) have one SWUpdate instance for all attached devices or > > If all devices are running SWUpdate and one of them is working as gateway > proxy, the solution in SWUpdate is via the swuforward handler. This allows > also a "network update" and guarantees that an update is successful if all > components are updated. Yes, that's true. > If the other devices are on the same device as the one running SWUpdate (else > the remote handler does not work), Not necessarily, you have to have a piece of software that runs alongside SWUpdate on the same device, true. But this piece of software may very well update other devices, e.g., via UART. > the whole device itself should be seen as a > single device. It is questionable if the remote handler should have multiple > sockets, as you propose. I think it is better that there is just one socket, > but another way to identify the devices, so the id is put in sw-description > (then as part of properties for the handler). I do not bother if the protocol > should be changed to support multiple devices as you need. The device id you > want to add belong really to the subdevice, because this is what you want to > identify with a separate connection (one socket for each device). Yes, that's also an option. But then you cannot update remote devices in parallel as SWUpdate works sequentially, it's an unnecessary limitation. For making SWUpdate "forwarding" update jobs in parallel, you kind of have to "tag" an update as being able to be processed concurrently (i.e., forward) or not (i.e., local firmware update). The same is true if you do local updates though: You can concurrently update, e.g., container images while you cannot concurrently update the device's firmware. > But the remote handler can send messages where there is a device ID field (and > broadcast should be allowed). > > > (2) have multiple SWUpdate instances, one for each attached device. > > I do not get the use case, it looks to me you have still several devices. This > does not scale very well, and there is no need that whole instances are > running at once - and then, there is no reason for multiple instances. Well, you have one SWUpdate on the gateway for each remote device to be updated. As long as SWUpdate is not able to concurrently handle multiple update jobs I guess this is the only way to do this? We do have several (remote) devices attached to the box on which the SWUpdate instances are running on, connected via UART. Each SWUpdate instance is responsible for a particular device. Via the remote handler, it forwards the update to vendor-proprietary software that updates the device and reports back. > > In case (2), it's the same as above, i.e., a global device ID in this > > case representing the attached device. > > If the device id is representing the subdevice, it does not represent the main > device. Exactly. One SWUpdate instance for one device: * One SWUpdate (blocking the others) for the gateway box itself, and * multiple SWUpdate instances responsible for the UART-attached devices. > > In case (1), this is clearly not sufficient as one SWUpdate instance > > needs to handle multiple device IDs and there's more work to do in this > > case anyway (e.g., query hawkBit for multiple devices → multiple > > suricatta processes, concurrent installation to attached devices, ...). > > Which kind of devices are we talking about ? Remote handler is thought for > small devices, no linux, with microcontrollers. this leads to smaller image > size if compared to the one on the main device. Could you not have a single > SWU on Hawkbit for all these devices ? Yes, micro-controllers updated via vendor-proprietary software, which is fed the update artifact via the remote handler. There are update artifacts applicable to all the UART-attached devices on hawkBit but there's also the use case to update particular devices with a special update artifact which gets deployed on a subset of those devices. > > > The usage in remote handler is very specific - what about if I have both > > > remote handler and Hawkbit ? It could be true that the ID is different. > > > > Yes, if you give a specific hawkBit device ID, that one has preference > > and is taken. Only if no specific hawkBit device ID is given, the global > > device ID is used (for convenience) while it's used for the remote > > handler unconditionally. > > > > > > > Should this ID not part of remote handler configuration, that means in > > > sw-description as one of the properties ? Or if not, why is the same as > > > Hakwbit ID ? > > > > The remote handler configuration section could be another option, yes. > > Putting it in sw-description is not a that good solution as the update > > artifact is then tied to the device, i.e., you cannot install the very > > same artifact on two different attached devices although they're > > identical and such an update artifact could be installed on all (or > > a subset) of the attached devices. > > And something like in swuforwarder where the properties is an array of device > IDs ? You can send the same image to all of them. Yes, if they were running SWUpdate or having a non-proprietary update method which they have not. > > > Note that Hawkbit ID can be retrieved from a "suricatta" section in > > > configuration file to be compatible with the past, or in a more specific > > > "hawkbit" section - because tenant / id are defined in Hawkbit DDI and belong > > > to this interface. > > > > Yes, but overridden by command line. The flow with this patch is as follows: > > (1) read config file (both sections) and |= ID_BIT > > (2) read specific hawkBit device ID parameter and |= ID_BIT > > (3) read global device ID and |= ID_BIT > > (4) complain about missing device ID. Kind regards, Christian
Hi Christian, On 15.03.21 11:05, Christian Storm wrote: > Hi Stefano, > >>>>> Currently, a device ID is used for suricatta modules only. >>>>> However, the remote handler can make use of a device ID as >>>>> well such that this device ID is used to construct the >>>>> remote handler's ZeroMQ socket name, enabling multiple >>>>> SWUpdate instances to serve different remotes per different >>>>> socket names. >>>> >>>> I have raised an issue about the remote handler (see >>>> https://groups.google.com/g/swupdate/c/5kun17vfu5Y/m/ok5fhnNmCAAJ), Bastian >>>> thinks that there is no problems with the licensing. Anyway, this and the fact >>>> that the remote handler seems overkilling and it is quite unused, I was >>>> thinking about to remove it from SWUpdate. But it looks like there are some >>>> users...(you !). Do you really have use case with zeroMQ ? >>> >>> Not in particular ZeroMQ but a means to attach existing infrastructure: >>> The use case is to have one SWUpdate instance running on a gateway per >>> to be updated device attached to the gateway via some communication >>> channel (e.g., UART). So, SWUpdate is in effect a proxy that may itself >>> interact with, e.g., hawkBit and then applies an update to an attached >>> device via something like the remote handler that orchestrates custom >>> device-specific or even vendor-proprietary update software. >> >> This is the exact use case, it allows to send an image to a proprietary >> installer whose code is not allowed to be GPL. The best technical solution >> remains to convert it in a SWUpdate handler and link to SWUpdate., so that >> SWUpdate can control it and we have the whole output in SWUpdate log. With >> remote handler, we have just the result of ZeroMQ messages. >> >> "remote handler" in SWUpdate is really "local" : a connection is just via IPC >> to another process on the same device. > > Yes, exactly, it's a local bidirectional channel to some other software. Right. > > >>> As it happens, ZeroMQ is in SWUpdate and is in the device-specific >>> update software, hence we used that. >>> Anyway, as long as there is a means to couple an existing software to >>> SWUpdate in a bi-directional manner (send command+data, receive status >>> and probably push it to, e.g., hawkBit), it's fine. >> >> Ok, got it - if there will be no license issue, remote handler based on ZeroMQ >> can remain part of project. > > Well, it mustn't be ZeroMQ, e.g., the UDS/pipe handler proposed recently > does also work given we have a protocol to speak over the pipe. Then, we > can get rid of ZeroMQ but don't lose the remote handler functionality? But see the discussion, the pipe handler wil ladd a singularity, that means the update will strong depends on external tools that could be compromised. Anyway, you can have the same issue if the partner of the remote handler does the same, and calls "system" to execute a script. > > >>>> This patch seems matching your use case and your project, but it seems to me >>>> that introduce an artificial connection between the Hawkbit (not suricatta) ID >>>> (specified in Hawkbit DDI interface) with a general ID for the device. The >>>> "general server" in suricatta uses values from the "identify" section in >>>> configuration file, for example. >>> >>> Yes, but both are effectively "device IDs", just named differently and in >>> different locations. >> >> But we cannot know what the backend consider a unique device ID, and it can be >> the sum of different fileds (mac address, serial number, etc.). The rule is on >> the backend. So yes, a general device id can be also added, it is useless in >> case of update via USB / Webserver. > > Yes, in those cases it's useless as "you" resolved the device ID by > accessing the device directly, true. However, it can still be used for > displaying information, e.g., showing the device ID in the webserver. > > We cannot know, true, but the integrator does know and configures the > device ID accordingly and as the backend wants it to be. From an > SWUpdate point of view, this could be a random string without semantics. But for such as "free to be defined" paramters, we have already the "identify" section. There is currently no API to get these data from the webserver, but there is for Hawkbit. > > >>> So, for the simple case of SWUpdate updating the >>> device it's running on, there's in fact effectively only one global device ID. >> >> Mmmhhhh...it seems your use case is different as the one I supposed. >> >>> Consolidating this to one global device ID could make sense? >> >> I am trying to understand this. >> >>> >>> In the case of SWUpdate running as proxy (and not for the device it's >>> running on) it depends on whether we >>> (1) have one SWUpdate instance for all attached devices or >> >> If all devices are running SWUpdate and one of them is working as gateway >> proxy, the solution in SWUpdate is via the swuforward handler. This allows >> also a "network update" and guarantees that an update is successful if all >> components are updated. > > Yes, that's true. > > >> If the other devices are on the same device as the one running SWUpdate (else >> the remote handler does not work), > > Not necessarily, you have to have a piece of software that runs alongside > SWUpdate on the same device, true. But this piece of software may very > well update other devices, e.g., via UART. Right - and this was exactly the case I had when I pushed the remote handler. > > >> the whole device itself should be seen as a >> single device. It is questionable if the remote handler should have multiple >> sockets, as you propose. I think it is better that there is just one socket, >> but another way to identify the devices, so the id is put in sw-description >> (then as part of properties for the handler). I do not bother if the protocol >> should be changed to support multiple devices as you need. The device id you >> want to add belong really to the subdevice, because this is what you want to >> identify with a separate connection (one socket for each device). > > Yes, that's also an option. But then you cannot update remote devices in > parallel as SWUpdate works sequentially, it's an unnecessary limitation. It will be a change in the remote-handler only. If you look at the implementation, the SWU forwarder works in parallel and sends the inbound SWU to all devices listed in the properties. > For making SWUpdate "forwarding" update jobs in parallel, you kind of > have to "tag" an update as being able to be processed concurrently > (i.e., forward) or not (i.e., local firmware update). This was not foreseen at the time of the implementation of the remote handler, but it is a know design pattern by update. Each packet should be sent to a "broadcast" or "multicast" address (of course, we have no IP, it is just to give you an idea), and the protocol cannot rely to an ACK after each packet, but it should ask for ASK each single subdevice at the end of the upload. > The same is true > if you do local updates though: You can concurrently update, e.g., container > images while you cannot concurrently update the device's firmware. Not true - again, it depends on the architecture. I have projects where even the firmware on subdevices has an A/B pattern, and you can even update the devices in parallel. > > >> But the remote handler can send messages where there is a device ID field (and >> broadcast should be allowed). >> >>> (2) have multiple SWUpdate instances, one for each attached device. >> >> I do not get the use case, it looks to me you have still several devices. This >> does not scale very well, and there is no need that whole instances are >> running at once - and then, there is no reason for multiple instances. > > Well, you have one SWUpdate on the gateway for each remote device to be > updated. As long as SWUpdate is not able to concurrently handle multiple > update jobs I guess this is the only way to do this? My suggestion is to change the remote handler to let the update in parallel, if this is what you want. From the logical point of view, the device has just one ID and having multiple IDs just as work-around seems to me an artificial construction. > > We do have several (remote) devices attached to the box on which the > SWUpdate instances are running on, connected via UART. > Each SWUpdate instance is responsible for a particular device. Via the > remote handler, it forwards the update to vendor-proprietary software > that updates the device and reports back. But this has nothing to do how the device in its complex is seen outside the box. It is a single device with some subdevices connected via UART to the main processor and they should also be updated. If the remote handler becomes more like the SWU forwarder, you get what you want without any logical complication. > > >>> In case (2), it's the same as above, i.e., a global device ID in this >>> case representing the attached device. >> >> If the device id is representing the subdevice, it does not represent the main >> device. > > Exactly. One SWUpdate instance for one device: > * One SWUpdate (blocking the others) for the gateway box itself, and > * multiple SWUpdate instances responsible for the UART-attached devices. I remain of the opinion this is a very aritificial construction to work-around a current limitation in remote handler, that is a point-to-point connection. And the logical solution for me remains to extend the remote handler (as I said, I have no problem with it even if protocol will change) to allow multiple connections at once. > > >>> In case (1), this is clearly not sufficient as one SWUpdate instance >>> needs to handle multiple device IDs and there's more work to do in this >>> case anyway (e.g., query hawkBit for multiple devices → multiple >>> suricatta processes, concurrent installation to attached devices, ...). >> >> Which kind of devices are we talking about ? Remote handler is thought for >> small devices, no linux, with microcontrollers. this leads to smaller image >> size if compared to the one on the main device. Could you not have a single >> SWU on Hawkbit for all these devices ? > > Yes, micro-controllers updated via vendor-proprietary software, which is > fed the update artifact via the remote handler. > There are update artifacts applicable to all the UART-attached devices > on hawkBit but there's also the use case to update particular devices > with a special update artifact which gets deployed on a subset of those > devices. We need to address the devices with a "broadcast" (all devices) or via a "group" address. I just mention that another option is to push a regulöar handler like ucfw_handler - it updates a microcontroller FW via UART, but of course, source becomes GPLv2 and I guess this is not possible in your project. > > >>>> The usage in remote handler is very specific - what about if I have both >>>> remote handler and Hawkbit ? It could be true that the ID is different. >>> >>> Yes, if you give a specific hawkBit device ID, that one has preference >>> and is taken. Only if no specific hawkBit device ID is given, the global >>> device ID is used (for convenience) while it's used for the remote >>> handler unconditionally. >>> >>> >>>> Should this ID not part of remote handler configuration, that means in >>>> sw-description as one of the properties ? Or if not, why is the same as >>>> Hakwbit ID ? >>> >>> The remote handler configuration section could be another option, yes. >>> Putting it in sw-description is not a that good solution as the update >>> artifact is then tied to the device, i.e., you cannot install the very >>> same artifact on two different attached devices although they're >>> identical and such an update artifact could be installed on all (or >>> a subset) of the attached devices. >> >> And something like in swuforwarder where the properties is an array of device >> IDs ? You can send the same image to all of them. > > Yes, if they were running SWUpdate or having a non-proprietary update > method which they have not. Ok > > >>>> Note that Hawkbit ID can be retrieved from a "suricatta" section in >>>> configuration file to be compatible with the past, or in a more specific >>>> "hawkbit" section - because tenant / id are defined in Hawkbit DDI and belong >>>> to this interface. >>> >>> Yes, but overridden by command line. The flow with this patch is as follows: >>> (1) read config file (both sections) and |= ID_BIT >>> (2) read specific hawkBit device ID parameter and |= ID_BIT >>> (3) read global device ID and |= ID_BIT >>> (4) complain about missing device ID. Best regards, Stefano Babic
Hi Stefano, > > > > As it happens, ZeroMQ is in SWUpdate and is in the device-specific > > > > update software, hence we used that. > > > > Anyway, as long as there is a means to couple an existing software to > > > > SWUpdate in a bi-directional manner (send command+data, receive status > > > > and probably push it to, e.g., hawkBit), it's fine. > > > > > > Ok, got it - if there will be no license issue, remote handler based on ZeroMQ > > > can remain part of project. > > > > Well, it mustn't be ZeroMQ, e.g., the UDS/pipe handler proposed recently > > does also work given we have a protocol to speak over the pipe. Then, we > > can get rid of ZeroMQ but don't lose the remote handler functionality? > > But see the discussion, the pipe handler wil ladd a singularity, that means > the update will strong depends on external tools that could be compromised. > Anyway, you can have the same issue if the partner of the remote handler does > the same, and calls "system" to execute a script. Exactly. You have to trust the secondary software as much as SWUpdate. > > > > > This patch seems matching your use case and your project, but it seems to me > > > > > that introduce an artificial connection between the Hawkbit (not suricatta) ID > > > > > (specified in Hawkbit DDI interface) with a general ID for the device. The > > > > > "general server" in suricatta uses values from the "identify" section in > > > > > configuration file, for example. > > > > > > > > Yes, but both are effectively "device IDs", just named differently and in > > > > different locations. > > > > > > But we cannot know what the backend consider a unique device ID, and it can be > > > the sum of different fileds (mac address, serial number, etc.). The rule is on > > > the backend. So yes, a general device id can be also added, it is useless in > > > case of update via USB / Webserver. > > > > Yes, in those cases it's useless as "you" resolved the device ID by > > accessing the device directly, true. However, it can still be used for > > displaying information, e.g., showing the device ID in the webserver. > > > > We cannot know, true, but the integrator does know and configures the > > device ID accordingly and as the backend wants it to be. From an > > SWUpdate point of view, this could be a random string without semantics. > > But for such as "free to be defined" paramters, we have already the "identify" > section. There is currently no API to get these data from the webserver, but > there is for Hawkbit. Well, you supply hawkBit the device ID via the query GET /TENANT_ID/controller/v1/CONTROLLER_ID and the CONTROLLER_ID is also somewhat "free form". Essentially, it's also a random string as far as SWUpdate is concerned as SWUpdate puts no semantics in this. The thing is, as long as SWUpdate does not try to make sense of a "device ID", it's a random string having no meaning to SWUpdate while certainly for the backend(s). So, they're all the same, random strings for SWUpdate and as such, there may be a global one? > > > > So, for the simple case of SWUpdate updating the > > > > device it's running on, there's in fact effectively only one global device ID. > > > > > > Mmmhhhh...it seems your use case is different as the one I supposed. > > > > > > > Consolidating this to one global device ID could make sense? > > > > > > I am trying to understand this. > > > > > > > > > > > In the case of SWUpdate running as proxy (and not for the device it's > > > > running on) it depends on whether we > > > > (1) have one SWUpdate instance for all attached devices or > > > > > > If all devices are running SWUpdate and one of them is working as gateway > > > proxy, the solution in SWUpdate is via the swuforward handler. This allows > > > also a "network update" and guarantees that an update is successful if all > > > components are updated. > > > > Yes, that's true. > > > > > > > If the other devices are on the same device as the one running SWUpdate (else > > > the remote handler does not work), > > > > Not necessarily, you have to have a piece of software that runs alongside > > SWUpdate on the same device, true. But this piece of software may very > > well update other devices, e.g., via UART. > > Right - and this was exactly the case I had when I pushed the remote handler. I certainly think so :) > > > the whole device itself should be seen as a > > > single device. It is questionable if the remote handler should have multiple > > > sockets, as you propose. I think it is better that there is just one socket, > > > but another way to identify the devices, so the id is put in sw-description > > > (then as part of properties for the handler). I do not bother if the protocol > > > should be changed to support multiple devices as you need. The device id you > > > want to add belong really to the subdevice, because this is what you want to > > > identify with a separate connection (one socket for each device). > > > > Yes, that's also an option. But then you cannot update remote devices in > > parallel as SWUpdate works sequentially, it's an unnecessary limitation. > > It will be a change in the remote-handler only. If you look at the > implementation, the SWU forwarder works in parallel and sends the inbound SWU > to all devices listed in the properties. Yes, via multiple threads. > > For making SWUpdate "forwarding" update jobs in parallel, you kind of > > have to "tag" an update as being able to be processed concurrently > > (i.e., forward) or not (i.e., local firmware update). > > This was not foreseen at the time of the implementation of the remote handler, > but it is a know design pattern by update. Each packet should be sent to a > "broadcast" or "multicast" address (of course, we have no IP, it is just to > give you an idea), and the protocol cannot rely to an ACK after each packet, > but it should ask for ASK each single subdevice at the end of the upload. OK. > > The same is true > > if you do local updates though: You can concurrently update, e.g., container > > images while you cannot concurrently update the device's firmware. > > Not true - again, it depends on the architecture. I have projects where even > the firmware on subdevices has an A/B pattern, and you can even update the > devices in parallel. OK, yes, you're right here. > > > But the remote handler can send messages where there is a device ID field (and > > > broadcast should be allowed). > > > > > > > (2) have multiple SWUpdate instances, one for each attached device. > > > > > > I do not get the use case, it looks to me you have still several devices. This > > > does not scale very well, and there is no need that whole instances are > > > running at once - and then, there is no reason for multiple instances. > > > > Well, you have one SWUpdate on the gateway for each remote device to be > > updated. As long as SWUpdate is not able to concurrently handle multiple > > update jobs I guess this is the only way to do this? > > My suggestion is to change the remote handler to let the update in parallel, > if this is what you want. From the logical point of view, the device has just > one ID and having multiple IDs just as work-around seems to me an artificial > construction. Hm, this is exactly the point I think: Having one device ID and one SWUpdate instance running for the gateway box as well as for all the UART-attached boxes makes this conglomerate kind of a "multifunction device" from, e.g,. hawkBit's perspective. You assign it a firmware in hawkBit and SWUpdate does the right thing: It broadcasts the updates and the UART-attached boxes (respectively, their forwarding "agent") can decide on whether to apply the update or not. If the update is for the gateway box itself, SWUpdate applies this locally. By this, you put the logics "to the field" in terms of SWUpdate or the forwarding agents deciding what to do. Now, if you want to exercise more control on what device (the gateway box itself or the UART-attached boxes) should be updated, then this needs to be encoded, e.g., in the sw-description and parsed and handled accordingly by SWUpdate. This model resembles the "SWU forwarder", quoting from its docs: "The SWU forwarder handler can be used to update other systems where SWUpdate is running. It can be used in case of master / slaves systems, where the master is connected to the network and the "slaves" are hidden to the external world. The master is then the only interface to the world. A general SWU can contain embedded SWU images as single artifacts, and the SWU handler will forward it to the devices listed in the description of the artifact." The main point here is, I guess, this: [...] the "slaves" are hidden to the external world [...] The other model I see is that those are not hidden from the external world but instead exposed: All those devices (the gateway box itself and the UART-attached boxes) are explicitly registered in hawkBit as individual devices. Then, SWUpdate on the gateway box just serves as proxy for the UART-attached boxes while it does apply updates for the gateway box itself locally. The point here is that those UART-attached boxes are not able to run SWUpdate and may need vendor-proprietary forwarding agents. So, all is routed through SWUpdate which ― in case of one global SWUpdate instance ― has to query hawkBit for multiple devices on behalf of them. This way, you put all the logics to the backend (i.e., hawkBit) in terms of deciding which device gets what update. Does that sound somewhat reasonable? > > We do have several (remote) devices attached to the box on which the > > SWUpdate instances are running on, connected via UART. > > Each SWUpdate instance is responsible for a particular device. Via the > > remote handler, it forwards the update to vendor-proprietary software > > that updates the device and reports back. > > But this has nothing to do how the device in its complex is seen outside the > box. It is a single device with some subdevices connected via UART to the main > processor and they should also be updated. If the remote handler becomes more > like the SWU forwarder, you get what you want without any logical > complication. Yes, for the SWU forwarder, you're certainly right. > > > > In case (2), it's the same as above, i.e., a global device ID in this > > > > case representing the attached device. > > > > > > If the device id is representing the subdevice, it does not represent the main > > > device. > > > > Exactly. One SWUpdate instance for one device: > > * One SWUpdate (blocking the others) for the gateway box itself, and > > * multiple SWUpdate instances responsible for the UART-attached devices. > > I remain of the opinion this is a very aritificial construction to work-around > a current limitation in remote handler, that is a point-to-point connection. > And the logical solution for me remains to extend the remote handler (as I > said, I have no problem with it even if protocol will change) to allow > multiple connections at once. OK, in the "SWU forwarder"-like model shadowing devices I do fully agree. > > > > In case (1), this is clearly not sufficient as one SWUpdate instance > > > > needs to handle multiple device IDs and there's more work to do in this > > > > case anyway (e.g., query hawkBit for multiple devices → multiple > > > > suricatta processes, concurrent installation to attached devices, ...). > > > > > > Which kind of devices are we talking about ? Remote handler is thought for > > > small devices, no linux, with microcontrollers. this leads to smaller image > > > size if compared to the one on the main device. Could you not have a single > > > SWU on Hawkbit for all these devices ? > > > > Yes, micro-controllers updated via vendor-proprietary software, which is > > fed the update artifact via the remote handler. > > There are update artifacts applicable to all the UART-attached devices > > on hawkBit but there's also the use case to update particular devices > > with a special update artifact which gets deployed on a subset of those > > devices. > > We need to address the devices with a "broadcast" (all devices) or via a > "group" address. > > I just mention that another option is to push a regulöar handler like > ucfw_handler - it updates a microcontroller FW via UART, but of course, source > becomes GPLv2 and I guess this is not possible in your project. Well, sadly, this is not up to us but another vendor's software, but yes. Many thanks for this discussion and reading all my stuff :) Kind regards, Christian
Hi Christian, On 15.03.21 17:10, Christian Storm wrote: > Hi Stefano, > >>>>> As it happens, ZeroMQ is in SWUpdate and is in the device-specific >>>>> update software, hence we used that. >>>>> Anyway, as long as there is a means to couple an existing software to >>>>> SWUpdate in a bi-directional manner (send command+data, receive status >>>>> and probably push it to, e.g., hawkBit), it's fine. >>>> >>>> Ok, got it - if there will be no license issue, remote handler based on ZeroMQ >>>> can remain part of project. >>> >>> Well, it mustn't be ZeroMQ, e.g., the UDS/pipe handler proposed recently >>> does also work given we have a protocol to speak over the pipe. Then, we >>> can get rid of ZeroMQ but don't lose the remote handler functionality? >> >> But see the discussion, the pipe handler wil ladd a singularity, that means >> the update will strong depends on external tools that could be compromised. >> Anyway, you can have the same issue if the partner of the remote handler does >> the same, and calls "system" to execute a script. > > Exactly. You have to trust the secondary software as much as SWUpdate. > Right. > >>>>>> This patch seems matching your use case and your project, but it seems to me >>>>>> that introduce an artificial connection between the Hawkbit (not suricatta) ID >>>>>> (specified in Hawkbit DDI interface) with a general ID for the device. The >>>>>> "general server" in suricatta uses values from the "identify" section in >>>>>> configuration file, for example. >>>>> >>>>> Yes, but both are effectively "device IDs", just named differently and in >>>>> different locations. >>>> >>>> But we cannot know what the backend consider a unique device ID, and it can be >>>> the sum of different fileds (mac address, serial number, etc.). The rule is on >>>> the backend. So yes, a general device id can be also added, it is useless in >>>> case of update via USB / Webserver. >>> >>> Yes, in those cases it's useless as "you" resolved the device ID by >>> accessing the device directly, true. However, it can still be used for >>> displaying information, e.g., showing the device ID in the webserver. >>> >>> We cannot know, true, but the integrator does know and configures the >>> device ID accordingly and as the backend wants it to be. From an >>> SWUpdate point of view, this could be a random string without semantics. >> >> But for such as "free to be defined" paramters, we have already the "identify" >> section. There is currently no API to get these data from the webserver, but >> there is for Hawkbit. > > Well, you supply hawkBit the device ID via the query > GET /TENANT_ID/controller/v1/CONTROLLER_ID > and the CONTROLLER_ID is also somewhat "free form". Right, at least for the backend. But then, users build their own rules to set the unique-id, wit ha very different list of possible algorithms, using SerialNumbers, random generated salt, MacAddress,..... It is impossible to factorize this because each project is coming with constraints that are outside the technical aspects, and they often include logistic (country where the device was sold, tracking number, etc.). > Essentially, it's also a random string as far as SWUpdate is concerned > as SWUpdate puts no semantics in this. The thing is, as long as SWUpdate > does not try to make sense of a "device ID", it's a random string having > no meaning to SWUpdate while certainly for the backend(s). So, they're > all the same, random strings for SWUpdate and as such, there may be > a global one? Right - but if a global ID can be added, what is really disturbing is the link to the remote handler done via: - len = strlen(img->type_data) + strlen(CONFIG_SOCKET_REMOTE_HANDLER_DIRECTORY) + strlen("ipc://") + 4; + char* device_id = get_swupdate_cfg()->globals.device_id; + len = strlen(img->type_data) + strlen(CONFIG_SOCKET_REMOTE_HANDLER_DIRECTORY) + + strlen(device_id) + strlen("ipc://") + 4; This is an ad hoc change for a specific project and it makes a semantic into SWUpdate. > > >>>>> So, for the simple case of SWUpdate updating the >>>>> device it's running on, there's in fact effectively only one global device ID. >>>> >>>> Mmmhhhh...it seems your use case is different as the one I supposed. >>>> >>>>> Consolidating this to one global device ID could make sense? >>>> >>>> I am trying to understand this. >>>> >>>>> >>>>> In the case of SWUpdate running as proxy (and not for the device it's >>>>> running on) it depends on whether we >>>>> (1) have one SWUpdate instance for all attached devices or >>>> >>>> If all devices are running SWUpdate and one of them is working as gateway >>>> proxy, the solution in SWUpdate is via the swuforward handler. This allows >>>> also a "network update" and guarantees that an update is successful if all >>>> components are updated. >>> >>> Yes, that's true. >>> >>> >>>> If the other devices are on the same device as the one running SWUpdate (else >>>> the remote handler does not work), >>> >>> Not necessarily, you have to have a piece of software that runs alongside >>> SWUpdate on the same device, true. But this piece of software may very >>> well update other devices, e.g., via UART. >> >> Right - and this was exactly the case I had when I pushed the remote handler. > > I certainly think so :) ;-) > > >>>> the whole device itself should be seen as a >>>> single device. It is questionable if the remote handler should have multiple >>>> sockets, as you propose. I think it is better that there is just one socket, >>>> but another way to identify the devices, so the id is put in sw-description >>>> (then as part of properties for the handler). I do not bother if the protocol >>>> should be changed to support multiple devices as you need. The device id you >>>> want to add belong really to the subdevice, because this is what you want to >>>> identify with a separate connection (one socket for each device). >>> >>> Yes, that's also an option. But then you cannot update remote devices in >>> parallel as SWUpdate works sequentially, it's an unnecessary limitation. >> >> It will be a change in the remote-handler only. If you look at the >> implementation, the SWU forwarder works in parallel and sends the inbound SWU >> to all devices listed in the properties. > > Yes, via multiple threads. Exactly. > > >>> For making SWUpdate "forwarding" update jobs in parallel, you kind of >>> have to "tag" an update as being able to be processed concurrently >>> (i.e., forward) or not (i.e., local firmware update). >> >> This was not foreseen at the time of the implementation of the remote handler, >> but it is a know design pattern by update. Each packet should be sent to a >> "broadcast" or "multicast" address (of course, we have no IP, it is just to >> give you an idea), and the protocol cannot rely to an ACK after each packet, >> but it should ask for ASK each single subdevice at the end of the upload. > > OK. > > >>> The same is true >>> if you do local updates though: You can concurrently update, e.g., container >>> images while you cannot concurrently update the device's firmware. >> >> Not true - again, it depends on the architecture. I have projects where even >> the firmware on subdevices has an A/B pattern, and you can even update the >> devices in parallel. > > OK, yes, you're right here. > > >>>> But the remote handler can send messages where there is a device ID field (and >>>> broadcast should be allowed). >>>> >>>>> (2) have multiple SWUpdate instances, one for each attached device. >>>> >>>> I do not get the use case, it looks to me you have still several devices. This >>>> does not scale very well, and there is no need that whole instances are >>>> running at once - and then, there is no reason for multiple instances. >>> >>> Well, you have one SWUpdate on the gateway for each remote device to be >>> updated. As long as SWUpdate is not able to concurrently handle multiple >>> update jobs I guess this is the only way to do this? >> >> My suggestion is to change the remote handler to let the update in parallel, >> if this is what you want. From the logical point of view, the device has just >> one ID and having multiple IDs just as work-around seems to me an artificial >> construction. > > Hm, this is exactly the point I think: Having one device ID and one > SWUpdate instance running for the gateway box as well as for all the > UART-attached boxes makes this conglomerate kind of a "multifunction > device" from, e.g,. hawkBit's perspective. Right - as gateway, it hides how it is built and it exposes just one device. > You assign it a firmware in > hawkBit and SWUpdate does the right thing: It broadcasts the updates and > the UART-attached boxes (respectively, their forwarding "agent") can > decide on whether to apply the update or not. If the update is for the > gateway box itself, SWUpdate applies this locally. > By this, you put the logics "to the field" in terms of SWUpdate or the > forwarding agents deciding what to do. > Now, if you want to exercise more control on what device (the gateway > box itself or the UART-attached boxes) should be updated, then this > needs to be encoded, e.g., in the sw-description and parsed and handled > accordingly by SWUpdate. Right - this should be done in sw-description > This model resembles the "SWU forwarder", quoting from its docs: > > "The SWU forwarder handler can be used to update other systems where SWUpdate > is running. It can be used in case of master / slaves systems, where the master > is connected to the network and the "slaves" are hidden to the external world. > The master is then the only interface to the world. A general SWU can contain > embedded SWU images as single artifacts, and the SWU handler will forward it > to the devices listed in the description of the artifact." > > The main point here is, I guess, this: > > [...] the "slaves" are hidden to the external world [...] > > > > The other model I see is that those are not hidden from the external > world but instead exposed: All those devices (the gateway box itself and > the UART-attached boxes) are explicitly registered in hawkBit as > individual devices. Then, SWUpdate on the gateway box just serves as > proxy for the UART-attached boxes while it does apply updates for the > gateway box itself locally. Yes, this is a second model. What is weird here (but again, it could be plausible in your project) is that if the subdevices are locally connected, I (personal opinion of course) consider a successful update if all of them are updated. But yes, they could be connected via UART but not physically nearby, and they could be very different devices. > The point here is that those UART-attached boxes are not able to run > SWUpdate and may need vendor-proprietary forwarding agents. So, all is > routed through SWUpdate which ― in case of one global SWUpdate instance > ― has to query hawkBit for multiple devices on behalf of them. > This way, you put all the logics to the backend (i.e., hawkBit) in terms > of deciding which device gets what update. Ok > > Does that sound somewhat reasonable? Yes, it is. So your solution for the project is to have multiple SWUpdate's instances running on the "proxy" device, each of them has a an ID for the subdevice. But doing this, you have already to split the instances to have different set of configuration files, sockets, TMPDIR, and so on, else they conflict. Which is then the reason to bind the device-id to the remote handler (the most disturbing thing for me...) ? > > >>> We do have several (remote) devices attached to the box on which the >>> SWUpdate instances are running on, connected via UART. >>> Each SWUpdate instance is responsible for a particular device. Via the >>> remote handler, it forwards the update to vendor-proprietary software >>> that updates the device and reports back. >> >> But this has nothing to do how the device in its complex is seen outside the >> box. It is a single device with some subdevices connected via UART to the main >> processor and they should also be updated. If the remote handler becomes more >> like the SWU forwarder, you get what you want without any logical >> complication. > > Yes, for the SWU forwarder, you're certainly right. > > >>>>> In case (2), it's the same as above, i.e., a global device ID in this >>>>> case representing the attached device. >>>> >>>> If the device id is representing the subdevice, it does not represent the main >>>> device. >>> >>> Exactly. One SWUpdate instance for one device: >>> * One SWUpdate (blocking the others) for the gateway box itself, and >>> * multiple SWUpdate instances responsible for the UART-attached devices. >> >> I remain of the opinion this is a very aritificial construction to work-around >> a current limitation in remote handler, that is a point-to-point connection. >> And the logical solution for me remains to extend the remote handler (as I >> said, I have no problem with it even if protocol will change) to allow >> multiple connections at once. > > OK, in the "SWU forwarder"-like model shadowing devices I do fully agree. > > >>>>> In case (1), this is clearly not sufficient as one SWUpdate instance >>>>> needs to handle multiple device IDs and there's more work to do in this >>>>> case anyway (e.g., query hawkBit for multiple devices → multiple >>>>> suricatta processes, concurrent installation to attached devices, ...). >>>> >>>> Which kind of devices are we talking about ? Remote handler is thought for >>>> small devices, no linux, with microcontrollers. this leads to smaller image >>>> size if compared to the one on the main device. Could you not have a single >>>> SWU on Hawkbit for all these devices ? >>> >>> Yes, micro-controllers updated via vendor-proprietary software, which is >>> fed the update artifact via the remote handler. >>> There are update artifacts applicable to all the UART-attached devices >>> on hawkBit but there's also the use case to update particular devices >>> with a special update artifact which gets deployed on a subset of those >>> devices. >> >> We need to address the devices with a "broadcast" (all devices) or via a >> "group" address. >> >> I just mention that another option is to push a regulöar handler like >> ucfw_handler - it updates a microcontroller FW via UART, but of course, source >> becomes GPLv2 and I guess this is not possible in your project. > > Well, sadly, this is not up to us but another vendor's software, but yes. ok > > > > Many thanks for this discussion and reading all my stuff :) > Best regards, Stefano
Hi Stefano, > > > > > > > This patch seems matching your use case and your project, but it seems to me > > > > > > > that introduce an artificial connection between the Hawkbit (not suricatta) ID > > > > > > > (specified in Hawkbit DDI interface) with a general ID for the device. The > > > > > > > "general server" in suricatta uses values from the "identify" section in > > > > > > > configuration file, for example. > > > > > > > > > > > > Yes, but both are effectively "device IDs", just named differently and in > > > > > > different locations. > > > > > > > > > > But we cannot know what the backend consider a unique device ID, and it can be > > > > > the sum of different fileds (mac address, serial number, etc.). The rule is on > > > > > the backend. So yes, a general device id can be also added, it is useless in > > > > > case of update via USB / Webserver. > > > > > > > > Yes, in those cases it's useless as "you" resolved the device ID by > > > > accessing the device directly, true. However, it can still be used for > > > > displaying information, e.g., showing the device ID in the webserver. > > > > > > > > We cannot know, true, but the integrator does know and configures the > > > > device ID accordingly and as the backend wants it to be. From an > > > > SWUpdate point of view, this could be a random string without semantics. > > > > > > But for such as "free to be defined" paramters, we have already the "identify" > > > section. There is currently no API to get these data from the webserver, but > > > there is for Hawkbit. > > > > Well, you supply hawkBit the device ID via the query > > GET /TENANT_ID/controller/v1/CONTROLLER_ID > > and the CONTROLLER_ID is also somewhat "free form". > > Right, at least for the backend. But then, users build their own rules to set > the unique-id, wit ha very different list of possible algorithms, using > SerialNumbers, random generated salt, MacAddress,..... > > It is impossible to factorize this because each project is coming with > constraints that are outside the technical aspects, and they often include > logistic (country where the device was sold, tracking number, etc.). Exactly, agreed. > > > Essentially, it's also a random string as far as SWUpdate is concerned > > as SWUpdate puts no semantics in this. The thing is, as long as SWUpdate > > does not try to make sense of a "device ID", it's a random string having > > no meaning to SWUpdate while certainly for the backend(s). So, they're > > all the same, random strings for SWUpdate and as such, there may be > > a global one? > > Right - but if a global ID can be added, what is really disturbing is the link > to the remote handler done via: > > - len = strlen(img->type_data) + > strlen(CONFIG_SOCKET_REMOTE_HANDLER_DIRECTORY) + strlen("ipc://") + 4; > + char* device_id = get_swupdate_cfg()->globals.device_id; > > + len = strlen(img->type_data) + > strlen(CONFIG_SOCKET_REMOTE_HANDLER_DIRECTORY) > + + strlen(device_id) + strlen("ipc://") + 4; > > This is an ad hoc change for a specific project and it makes a semantic into > SWUpdate. Well, semantics in so far as it becomes part of the socket name. Either way, this can be easily resolved using the get_tmpdir() prefix mechanism (see below). > > > > > But the remote handler can send messages where there is a device ID field (and > > > > > broadcast should be allowed). > > > > > > > > > > > (2) have multiple SWUpdate instances, one for each attached device. > > > > > > > > > > I do not get the use case, it looks to me you have still several devices. This > > > > > does not scale very well, and there is no need that whole instances are > > > > > running at once - and then, there is no reason for multiple instances. > > > > > > > > Well, you have one SWUpdate on the gateway for each remote device to be > > > > updated. As long as SWUpdate is not able to concurrently handle multiple > > > > update jobs I guess this is the only way to do this? > > > > > > My suggestion is to change the remote handler to let the update in parallel, > > > if this is what you want. From the logical point of view, the device has just > > > one ID and having multiple IDs just as work-around seems to me an artificial > > > construction. > > > > Hm, this is exactly the point I think: Having one device ID and one > > SWUpdate instance running for the gateway box as well as for all the > > UART-attached boxes makes this conglomerate kind of a "multifunction > > device" from, e.g,. hawkBit's perspective. > > Right - as gateway, it hides how it is built and it exposes just one device. > > > You assign it a firmware in > > hawkBit and SWUpdate does the right thing: It broadcasts the updates and > > the UART-attached boxes (respectively, their forwarding "agent") can > > decide on whether to apply the update or not. If the update is for the > > gateway box itself, SWUpdate applies this locally. > > By this, you put the logics "to the field" in terms of SWUpdate or the > > forwarding agents deciding what to do. > > Now, if you want to exercise more control on what device (the gateway > > box itself or the UART-attached boxes) should be updated, then this > > needs to be encoded, e.g., in the sw-description and parsed and handled > > accordingly by SWUpdate. > > Right - this should be done in sw-description > > > This model resembles the "SWU forwarder", quoting from its docs: > > > > "The SWU forwarder handler can be used to update other systems where SWUpdate > > is running. It can be used in case of master / slaves systems, where the master > > is connected to the network and the "slaves" are hidden to the external world. > > The master is then the only interface to the world. A general SWU can contain > > embedded SWU images as single artifacts, and the SWU handler will forward it > > to the devices listed in the description of the artifact." > > > > The main point here is, I guess, this: > > > > [...] the "slaves" are hidden to the external world [...] > > > > > > > > The other model I see is that those are not hidden from the external > > world but instead exposed: All those devices (the gateway box itself and > > the UART-attached boxes) are explicitly registered in hawkBit as > > individual devices. Then, SWUpdate on the gateway box just serves as > > proxy for the UART-attached boxes while it does apply updates for the > > gateway box itself locally. > > Yes, this is a second model. What is weird here (but again, it could be > plausible in your project) is that if the subdevices are locally connected, I > (personal opinion of course) consider a successful update if all of them are > updated. But yes, they could be connected via UART but not physically nearby, > and they could be very different devices. Yes, maybe UART was a bad example, sorry for that! One can also think of a LAN/WAN where the gateway is the only box connected to the outside (i.e., hawkBit) and the boxes inside the network cannot reach hawkBit by themselves. So, SWUpdate running on this gateway box is a proxy for those devices ― and also responsible for updating itself as well (i.e., the gateway box). Nonetheless, fine-grained control which artifacts should be shipped to which devices *as if* they're directly connected to hawkBit should be exercised. To that end, those devices are registered in hawkBit as individual devices, and are not shadowed as one multi-function device "behind" the gateway device on which SWUpdate is running on. To complicate matters, the boxes inside the network cannot digest .swu but need a vendor-specific software to do updates. > > The point here is that those UART-attached boxes are not able to run > > SWUpdate and may need vendor-proprietary forwarding agents. So, all is > > routed through SWUpdate which ― in case of one global SWUpdate instance > > ― has to query hawkBit for multiple devices on behalf of them. > > This way, you put all the logics to the backend (i.e., hawkBit) in terms > > of deciding which device gets what update. > > Ok > > > > > Does that sound somewhat reasonable? > > Yes, it is. > > So your solution for the project is to have multiple SWUpdate's instances > running on the "proxy" device, each of them has a an ID for the subdevice. But > doing this, you have already to split the instances to have different set of > configuration files, sockets, TMPDIR, and so on, else they conflict. Exactly. But I think it would be nice if *one* SWUpdate instance on the gateway box could handle this, meaning: querying hawkBit for itself and on behalf of the other "shadowed" devices and controlling their update process, reporting progress to hawkBit, again on behalf of those "shadowed" devices. What do you think of this? > Which is then the reason to bind the device-id to the remote handler > (the most disturbing thing for me...) ? Well, all SWUpdate instances are placing the socket into the same directory and hence they're distinguished by the name ― which is comprised of the device ID ― to avoid clashes since SOCKET_REMOTE_HANDLER_DIRECTORY is a compile-time rather than a run-time setting.... Granted, maybe we should make this run-time configurable as for the other directories with the get_tmpdir() prefix mechanism? The same is true by the way for SOCKET_NOTIFIER_DIRECTORY used in case of running SWUpdate on FreeBSD... That reminds me of testing current SWUpdate on FreeBSD again ― haven't done that in a while... Kind regards, Christian
diff --git a/core/swupdate.c b/core/swupdate.c index e959568..f823f83 100644 --- a/core/swupdate.c +++ b/core/swupdate.c @@ -84,6 +84,7 @@ static struct option long_options[] = { {"no-reinstalling", required_argument, NULL, 'R'}, {"no-transaction-marker", no_argument, NULL, 'M'}, {"no-state-marker", no_argument, NULL, 'm'}, + {"device-id", required_argument, NULL, 'D'}, #ifdef CONFIG_SIGNED_IMAGES {"key", required_argument, NULL, 'k'}, {"ca-path", required_argument, NULL, 'k'}, @@ -156,6 +157,7 @@ static void usage(char *programname) " -M, --no-transaction-marker : disable setting bootloader transaction marker\n" " -m, --no-state-marker : disable setting update state in bootloader\n" " -o, --output <filename> : saves the incoming stream\n" + " -D, --device-id <device id> : defines this device's ID\n" " -v, --verbose : be verbose, set maximum loglevel\n" " --version : print SWUpdate version and exit\n" #ifdef CONFIG_HW_COMPATIBILITY @@ -374,6 +376,8 @@ static int read_globals_settings(void *elem, void *data) /* by convention, errors in a configuration section are ignored */ (void)parse_image_selector(software_select, sw); } + GET_FIELD_STRING(LIBCFG_PARSER, elem, + "device-id", sw->globals.device_id); return 0; } @@ -482,7 +486,7 @@ int main(int argc, char **argv) #endif memset(main_options, 0, sizeof(main_options)); memset(image_url, 0, sizeof(image_url)); - strcpy(main_options, "vhni:e:q:l:Lcf:p:P:o:N:R:Mm"); + strcpy(main_options, "vhni:e:q:l:Lcf:p:P:o:N:R:MmD:"); #ifdef CONFIG_MTD strcat(main_options, "b:"); #endif @@ -633,6 +637,10 @@ int main(int argc, char **argv) optarg, sizeof(swcfg.globals.publickeyfname)); break; + case 'D': + strlcpy(swcfg.globals.device_id, optarg, + sizeof(swcfg.globals.device_id)); + break; case '1': swcfg.globals.cert_purpose = parse_cert_purpose(optarg); break; diff --git a/doc/source/swupdate.rst b/doc/source/swupdate.rst index 22d135a..dd1bc6e 100644 --- a/doc/source/swupdate.rst +++ b/doc/source/swupdate.rst @@ -548,6 +548,12 @@ Command line parameters +-------------+----------+--------------------------------------------+ | -p <cmd> | string | Execute post-update command. | +-------------+----------+--------------------------------------------+ +| -D <id> | string | Define a device ID. This ID will be used | +| | | for suricatta when no hawkBit ID is given. | +| | | This device ID will be blended into the | +| | | remote handler's zmq socket. | ++-------------+----------+--------------------------------------------+ + Downloader command line parameters .................................. @@ -592,6 +598,8 @@ Mandatory arguments are marked with '\*': | | | e.g., localhost:8080 | +-------------------------+----------+--------------------------------------------+ | -i <id> | integer | \* The device ID to communicate to hawkBit.| +| | | This argument is optional if a device ID | +| | | argument is given to SWUpdate (-D option) | +-------------------------+----------+--------------------------------------------+ | -c <confirm> | integer | Confirm update status to server: 1=AGAIN, | | | | 2=SUCCESS, 3=FAILED | diff --git a/examples/configuration/swupdate.cfg b/examples/configuration/swupdate.cfg index a97f982..36356d1 100644 --- a/examples/configuration/swupdate.cfg +++ b/examples/configuration/swupdate.cfg @@ -40,6 +40,8 @@ # set expected common name of signer certificate # select: : string # select software images set and source (<software>,<mode>) +# device-id : string +# define a device ID globals : { @@ -48,6 +50,7 @@ globals : syslog = true; /* public-key-file = "test.pem";*/ mtd-blacklist = "0 1 2 3 4 5 6"; + device-id = "Default_ID"; }; # logcolors : set colors for output to stdout / stderr diff --git a/handlers/remote_handler.c b/handlers/remote_handler.c index 275859f..bc3b642 100644 --- a/handlers/remote_handler.c +++ b/handlers/remote_handler.c @@ -159,8 +159,10 @@ static int install_remote_image(struct img_type *img, struct RHmsg RHmessage; char bufcmd[80]; - len = strlen(img->type_data) + strlen(CONFIG_SOCKET_REMOTE_HANDLER_DIRECTORY) + strlen("ipc://") + 4; + char* device_id = get_swupdate_cfg()->globals.device_id; + len = strlen(img->type_data) + strlen(CONFIG_SOCKET_REMOTE_HANDLER_DIRECTORY) + + strlen(device_id) + strlen("ipc://") + 4; /* * Allocate maximum string */ @@ -169,8 +171,8 @@ static int install_remote_image(struct img_type *img, ERROR("Not enough memory"); return -ENOMEM; } - snprintf(connect_string, len, "ipc://%s%s", CONFIG_SOCKET_REMOTE_HANDLER_DIRECTORY, - img->type_data); + snprintf(connect_string, len, "ipc://%s%s%s", CONFIG_SOCKET_REMOTE_HANDLER_DIRECTORY, + device_id, img->type_data); ret = zmq_connect(request, connect_string); if (ret < 0) { diff --git a/include/swupdate.h b/include/swupdate.h index e607ed6..bc55970 100644 --- a/include/swupdate.h +++ b/include/swupdate.h @@ -125,6 +125,7 @@ enum { struct swupdate_global_cfg { int verbose; char mtdblacklist[SWUPDATE_GENERAL_STRING_SIZE]; + char device_id[SWUPDATE_GENERAL_STRING_SIZE]; int loglevel; int syslog_enabled; int dry_run; diff --git a/suricatta/server_hawkbit.c b/suricatta/server_hawkbit.c index ed4c800..da133ba 100644 --- a/suricatta/server_hawkbit.c +++ b/suricatta/server_hawkbit.c @@ -1766,6 +1766,18 @@ server_op_res_t server_start(char *fname, int argc, char *argv[]) } } + /* + * If no hawkBit device ID has been set, check if the global + * device ID is set and, if yes, use it as hawkBit device ID. + */ + if ((mandatory_argument_count & ID_BIT) != ID_BIT) { + char* id = get_swupdate_cfg()->globals.device_id; + if (strnlen(id, SWUPDATE_GENERAL_STRING_SIZE)) { + SETSTRING(server_hawkbit.device_id, id); + mandatory_argument_count |= ID_BIT; + } + } + if (mandatory_argument_count != ALL_MANDATORY_SET) { fprintf(stderr, "Mandatory arguments missing!\n"); suricatta_print_help();