| Message ID | 20191107115408.13013-1-zefir.kurtisi@neratec.com |
|---|---|
| State | Accepted |
| Delegated to: | Daniel Golle |
| Headers | show |
| Series | [OpenWrt-Devel] uqmi: add timeout parameter | expand |
Hi Zefir, On 07.11.2019 12:54, Zefir Kurtisi wrote: > Working with Quectel EM12 LTE-module, we observe > regular stalls of the QMI interface which cause > a request issued by uqmi to hang forever. > > Most reproducibly this happens after the device > has been power-cycled and left untouched for a > while (~ 60s+). Most of the time the very first > QMI request fails, since it is not responded by > the module. This is the strace from such a run > (from --get-pin-status): > > open("/dev/cdc-wdm0", O_RDWR|O_EXCL|O_NOCTTY|O_NONBLOCK|O_LARGEFILE) = 6 > fcntl64(6, F_GETFL) = 0x10802 (flags O_RDWR|O_NONBLOCK|O_LARGEFILE) > fcntl64(6, F_SETFL, O_RDWR|O_NONBLOCK|O_LARGEFILE) = 0 > epoll_ctl(3, EPOLL_CTL_ADD, 6, {EPOLLIN|EPOLLRDHUP|EPOLLET, {u32=268567076, u64=1153486808202346496}}) = 0 > write(6, "\1\17\0\0\0\0\0\1\"\0\4\0\1\1\0\2", 16) = 16 > clock_gettime(CLOCK_MONOTONIC, {tv_sec=95, tv_nsec=583444789}) = 0 > clock_gettime(CLOCK_MONOTONIC, {tv_sec=95, tv_nsec=583770264}) = 0 > epoll_pwait(3, > [ hang forever ] > > After killing the blocked uqmi process, the next > request works as expected. > > We don't know whether this is a device FW issue > (we use the latest EM12GPAR01A15M4G) or whether > the device enters some undocumented power-save > mode after idling for some time. Could you share this firmware version, is that a generic Quectel or a customized one? I would like to reproduce and debug the problem but the EM12 I have here has 'EM12GPAR01A_11_M4G'. Also, what platform do you use this modem with?
On 11/16/19 3:59 PM, Piotr Dymacz wrote: > Hi Zefir, > > On 07.11.2019 12:54, Zefir Kurtisi wrote: >> Working with Quectel EM12 LTE-module, we observe >> regular stalls of the QMI interface which cause >> a request issued by uqmi to hang forever. >> >> Most reproducibly this happens after the device >> has been power-cycled and left untouched for a >> while (~ 60s+). Most of the time the very first >> QMI request fails, since it is not responded by >> the module. This is the strace from such a run >> (from --get-pin-status): >> >> open("/dev/cdc-wdm0", O_RDWR|O_EXCL|O_NOCTTY|O_NONBLOCK|O_LARGEFILE) = 6 >> fcntl64(6, F_GETFL) = 0x10802 (flags >> O_RDWR|O_NONBLOCK|O_LARGEFILE) >> fcntl64(6, F_SETFL, O_RDWR|O_NONBLOCK|O_LARGEFILE) = 0 >> epoll_ctl(3, EPOLL_CTL_ADD, 6, {EPOLLIN|EPOLLRDHUP|EPOLLET, {u32=268567076, >> u64=1153486808202346496}}) = 0 >> write(6, "\1\17\0\0\0\0\0\1\"\0\4\0\1\1\0\2", 16) = 16 >> clock_gettime(CLOCK_MONOTONIC, {tv_sec=95, tv_nsec=583444789}) = 0 >> clock_gettime(CLOCK_MONOTONIC, {tv_sec=95, tv_nsec=583770264}) = 0 >> epoll_pwait(3, >> [ hang forever ] >> >> After killing the blocked uqmi process, the next >> request works as expected. >> >> We don't know whether this is a device FW issue >> (we use the latest EM12GPAR01A15M4G) or whether >> the device enters some undocumented power-save >> mode after idling for some time. > > Could you share this firmware version, is that a generic Quectel or a customized > one? I would like to reproduce and debug the problem but the EM12 I have here has > 'EM12GPAR01A_11_M4G'. > > Also, what platform do you use this modem with? > Hi Piotr, we use our own products [1], which are built around a PowerPC (8540) based platform. The FW we received from Codico [2], Quectel's distributor and support proxy for Switzerland. We get preview versions on request, therefore I am not sure if it can be posted publicly. I'll check for restrictions and provide the FW if able. Cheers [1] https://www.neratec.com/products [2] https://www.codico.com/
On 11/17/19 11:23 AM, Zefir Kurtisi wrote: > On 11/16/19 3:59 PM, Piotr Dymacz wrote: >> Hi Zefir, >> >> On 07.11.2019 12:54, Zefir Kurtisi wrote: >>> Working with Quectel EM12 LTE-module, we observe >>> regular stalls of the QMI interface which cause >>> a request issued by uqmi to hang forever. >>> >>> Most reproducibly this happens after the device >>> has been power-cycled and left untouched for a >>> while (~ 60s+). Most of the time the very first >>> QMI request fails, since it is not responded by >>> the module. This is the strace from such a run >>> (from --get-pin-status): >>> >>> open("/dev/cdc-wdm0", O_RDWR|O_EXCL|O_NOCTTY|O_NONBLOCK|O_LARGEFILE) = 6 >>> fcntl64(6, F_GETFL) = 0x10802 (flags >>> O_RDWR|O_NONBLOCK|O_LARGEFILE) >>> fcntl64(6, F_SETFL, O_RDWR|O_NONBLOCK|O_LARGEFILE) = 0 >>> epoll_ctl(3, EPOLL_CTL_ADD, 6, {EPOLLIN|EPOLLRDHUP|EPOLLET, {u32=268567076, >>> u64=1153486808202346496}}) = 0 >>> write(6, "\1\17\0\0\0\0\0\1\"\0\4\0\1\1\0\2", 16) = 16 >>> clock_gettime(CLOCK_MONOTONIC, {tv_sec=95, tv_nsec=583444789}) = 0 >>> clock_gettime(CLOCK_MONOTONIC, {tv_sec=95, tv_nsec=583770264}) = 0 >>> epoll_pwait(3, >>> [ hang forever ] >>> >>> After killing the blocked uqmi process, the next >>> request works as expected. >>> >>> We don't know whether this is a device FW issue >>> (we use the latest EM12GPAR01A15M4G) or whether >>> the device enters some undocumented power-save >>> mode after idling for some time. >> >> Could you share this firmware version, is that a generic Quectel or a customized >> one? I would like to reproduce and debug the problem but the EM12 I have here has >> 'EM12GPAR01A_11_M4G'. >> >> Also, what platform do you use this modem with? >> > > Hi Piotr, > > we use our own products [1], which are built around a PowerPC (8540) based platform. > > The FW we received from Codico [2], Quectel's distributor and support proxy for > Switzerland. We get preview versions on request, therefore I am not sure if it can > be posted publicly. I'll check for restrictions and provide the FW if able. > > > Cheers > > > [1] https://www.neratec.com/products > [2] https://www.codico.com/ > > _______________________________________________ > openwrt-devel mailing list > openwrt-devel@lists.openwrt.org > https://lists.openwrt.org/mailman/listinfo/openwrt-devel > Hi Piotr, Codico/Quectel confirmed that the EM12 FW has no NDA limitations and can be provided to developers. The archive is available for the next 5 days from our company file-server at [1]. Let me know if you need something else, otherwise looking forward whether you can reproduce the observed issue. Cheers [1] https://transfer.neratec.com/5e7a98bb1a51e/EM12GPAR01A15M4G.zip
Hi Zefir, On 19.11.2019 11:34, Zefir Kurtisi wrote: [...] >>>> We don't know whether this is a device FW issue >>>> (we use the latest EM12GPAR01A15M4G) or whether >>>> the device enters some undocumented power-save >>>> mode after idling for some time. >>> >>> Could you share this firmware version, is that a generic Quectel or a customized >>> one? I would like to reproduce and debug the problem but the EM12 I have here has >>> 'EM12GPAR01A_11_M4G'. >>> >>> Also, what platform do you use this modem with? >>> >> >> Hi Piotr, >> >> we use our own products [1], which are built around a PowerPC (8540) based platform. >> >> The FW we received from Codico [2], Quectel's distributor and support proxy for >> Switzerland. We get preview versions on request, therefore I am not sure if it can >> be posted publicly. I'll check for restrictions and provide the FW if able. [...] > Hi Piotr, > > Codico/Quectel confirmed that the EM12 FW has no NDA limitations and can be > provided to developers. The archive is available for the next 5 days from our > company file-server at [1]. Let me know if you need something else, otherwise > looking forward whether you can reproduce the observed issue. > > > Cheers > > > [1] https://transfer.neratec.com/5e7a98bb1a51e/EM12GPAR01A15M4G.zip Thanks! I also contacted my Quectel contact person about that. Will get back to you with my results in next days.
diff --git a/main.c b/main.c index 9b43e5e..aa4634c 100644 --- a/main.c +++ b/main.c @@ -44,6 +44,7 @@ static const struct option uqmi_getopt[] = { { "keep-client-id", required_argument, NULL, 'k' }, { "release-client-id", required_argument, NULL, 'r' }, { "mbim", no_argument, NULL, 'm' }, + { "timeout", required_argument, NULL, 't' }, { NULL, 0, NULL, 0 } }; #undef __uqmi_command @@ -57,6 +58,7 @@ static int usage(const char *progname) " --keep-client-id <name>: Keep Client ID for service <name>\n" " --release-client-id <name>: Release Client ID after exiting\n" " --mbim, -m NAME is an MBIM device with EXT_QMUX support\n" + " --timeout, -t response timeout in msecs\n" "\n" "Services: dms, nas, pds, wds, wms\n" "\n" @@ -103,6 +105,14 @@ static void handle_exit_signal(int signal) uloop_end(); } +static void _request_timeout_handler(struct uloop_timeout *timeout) +{ + fprintf(stderr, "Request timed out\n"); + handle_exit_signal(0); +} + +struct uloop_timeout request_timeout = { .cb = _request_timeout_handler, }; + int main(int argc, char **argv) { static struct qmi_dev dev; @@ -112,7 +122,7 @@ int main(int argc, char **argv) signal(SIGINT, handle_exit_signal); signal(SIGTERM, handle_exit_signal); - while ((ch = getopt_long(argc, argv, "d:k:sm", uqmi_getopt, NULL)) != -1) { + while ((ch = getopt_long(argc, argv, "d:k:smt:", uqmi_getopt, NULL)) != -1) { int cmd_opt = CMD_OPT(ch); if (ch < 0 && cmd_opt >= 0 && cmd_opt < __UQMI_COMMAND_LAST) { @@ -136,6 +146,9 @@ int main(int argc, char **argv) case 'm': dev.is_mbim = true; break; + case 't': + uloop_timeout_set(&request_timeout, atol(optarg)); + break; default: return usage(argv[0]); }
Working with Quectel EM12 LTE-module, we observe regular stalls of the QMI interface which cause a request issued by uqmi to hang forever. Most reproducibly this happens after the device has been power-cycled and left untouched for a while (~ 60s+). Most of the time the very first QMI request fails, since it is not responded by the module. This is the strace from such a run (from --get-pin-status): open("/dev/cdc-wdm0", O_RDWR|O_EXCL|O_NOCTTY|O_NONBLOCK|O_LARGEFILE) = 6 fcntl64(6, F_GETFL) = 0x10802 (flags O_RDWR|O_NONBLOCK|O_LARGEFILE) fcntl64(6, F_SETFL, O_RDWR|O_NONBLOCK|O_LARGEFILE) = 0 epoll_ctl(3, EPOLL_CTL_ADD, 6, {EPOLLIN|EPOLLRDHUP|EPOLLET, {u32=268567076, u64=1153486808202346496}}) = 0 write(6, "\1\17\0\0\0\0\0\1\"\0\4\0\1\1\0\2", 16) = 16 clock_gettime(CLOCK_MONOTONIC, {tv_sec=95, tv_nsec=583444789}) = 0 clock_gettime(CLOCK_MONOTONIC, {tv_sec=95, tv_nsec=583770264}) = 0 epoll_pwait(3, [ hang forever ] After killing the blocked uqmi process, the next request works as expected. We don't know whether this is a device FW issue (we use the latest EM12GPAR01A15M4G) or whether the device enters some undocumented power-save mode after idling for some time. This patch extends uqmi with a timeout option (-t, --timeout <ms>) which if set terminates a request after the given amount of msecs. In our usecase it provides a means of preventing infinitively stuck QMI requests. Since we observe the issue only for the very first request after cold-boot, we use a dummy access early in qmi.sh, e.g. uqmi -d /dev/cdc-wdm0 --get-pin-status -t 3000 >/dev/null 2>&1 This ensures the QMI interface is un-stuck in case it entered the stall-state observed. The change is intentionally not included in this commit, since you don't need it if it works for you. Signed-off-by: Zefir Kurtisi <zefir.kurtisi@neratec.com> --- main.c | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-)