mbox series

[v2,0/12] add ubus support to ltq-[v|a]dsl-app

Message ID 20201215093531.1763475-1-a.heider@gmail.com
Headers show
Series add ubus support to ltq-[v|a]dsl-app | expand

Message

Andre Heider Dec. 15, 2020, 9:35 a.m. UTC
v2:
- drop 0002-ltq-vdsl-app-fix-Wundef-warnings.patch
- use "/dev/dsl_cpe_api" without the "0" suffix for the adsl daemon:
  package/kernel/lantiq/ltq-adsl/patches/100-dsl_compat.patch:+   device_create(dsl_class, NULL, MKDEV(DRV_DSL_CPE_API_DEV_MAJOR, 0), NULL, "dsl_cpe_api");
  package/kernel/lantiq/ltq-vdsl/patches/100-compat.patch:+   device_create(dsl_class, NULL, dsl_devt, NULL, "dsl_cpe_api0");
- use callDSLMetrics() for luci, per jo
- add Tested-by tags

This is to significantly speed up the generation of the metrics.

The motivation comes from the fact that ~2.6s is just way too
ineffcient for interval based metric collectors like prometheus or
collectd.

The luci status page also loads/refreshes alot faster.

$ time /etc/init.d/dsl_control dslstat
real	0m 2.66s
user	0m 0.90s
sys	0m 1.76s

$ time ubus call dsl metrics
real	0m 0.02s
user	0m 0.00s
sys	0m 0.01s

The ltq-adsl-app changes are only compile time tested.

Comments

Andre Heider Jan. 9, 2021, 8:40 a.m. UTC | #1
Hi,

(cc'ed some recent commiters for lantiq)

On 15/12/2020 10:35, Andre Heider wrote:
> v2:
> - drop 0002-ltq-vdsl-app-fix-Wundef-warnings.patch
> - use "/dev/dsl_cpe_api" without the "0" suffix for the adsl daemon:
>    package/kernel/lantiq/ltq-adsl/patches/100-dsl_compat.patch:+   device_create(dsl_class, NULL, MKDEV(DRV_DSL_CPE_API_DEV_MAJOR, 0), NULL, "dsl_cpe_api");
>    package/kernel/lantiq/ltq-vdsl/patches/100-compat.patch:+   device_create(dsl_class, NULL, dsl_devt, NULL, "dsl_cpe_api0");
> - use callDSLMetrics() for luci, per jo
> - add Tested-by tags
> 
> This is to significantly speed up the generation of the metrics.
> 
> The motivation comes from the fact that ~2.6s is just way too
> ineffcient for interval based metric collectors like prometheus or
> collectd.
> 
> The luci status page also loads/refreshes alot faster.
> 
> $ time /etc/init.d/dsl_control dslstat
> real	0m 2.66s
> user	0m 0.90s
> sys	0m 1.76s
> 
> $ time ubus call dsl metrics
> real	0m 0.02s
> user	0m 0.00s
> sys	0m 0.01s
> 
> The ltq-adsl-app changes are only compile time tested.

who do I need to convince to review this? ;)

Thanks,
Andre
Martin Schiller Jan. 13, 2021, 2 p.m. UTC | #2
On 2021-01-09 09:40, Andre Heider wrote:
> Hi,
> 
> (cc'ed some recent commiters for lantiq)
> 
> On 15/12/2020 10:35, Andre Heider wrote:
>> v2:
>> - drop 0002-ltq-vdsl-app-fix-Wundef-warnings.patch
>> - use "/dev/dsl_cpe_api" without the "0" suffix for the adsl daemon:
>>    package/kernel/lantiq/ltq-adsl/patches/100-dsl_compat.patch:+   
>> device_create(dsl_class, NULL, MKDEV(DRV_DSL_CPE_API_DEV_MAJOR, 0), 
>> NULL, "dsl_cpe_api");
>>    package/kernel/lantiq/ltq-vdsl/patches/100-compat.patch:+   
>> device_create(dsl_class, NULL, dsl_devt, NULL, "dsl_cpe_api0");
>> - use callDSLMetrics() for luci, per jo
>> - add Tested-by tags
>> 
>> This is to significantly speed up the generation of the metrics.
>> 
>> The motivation comes from the fact that ~2.6s is just way too
>> ineffcient for interval based metric collectors like prometheus or
>> collectd.
>> 
>> The luci status page also loads/refreshes alot faster.
>> 
>> $ time /etc/init.d/dsl_control dslstat
>> real	0m 2.66s
>> user	0m 0.90s
>> sys	0m 1.76s
>> 
>> $ time ubus call dsl metrics
>> real	0m 0.02s
>> user	0m 0.00s
>> sys	0m 0.01s
>> 
>> The ltq-adsl-app changes are only compile time tested.
> 
> who do I need to convince to review this? ;)

I would also like this great work to be merged.

Martin
Mathias Kresin Jan. 14, 2021, 11:55 a.m. UTC | #3
1/13/21 3:00 PM, Martin Schiller:
> On 2021-01-09 09:40, Andre Heider wrote:
>> Hi,
>>
>> (cc'ed some recent commiters for lantiq)
>>
>> On 15/12/2020 10:35, Andre Heider wrote:
>>> v2:
>>> - drop 0002-ltq-vdsl-app-fix-Wundef-warnings.patch
>>> - use "/dev/dsl_cpe_api" without the "0" suffix for the adsl daemon:
>>>    package/kernel/lantiq/ltq-adsl/patches/100-dsl_compat.patch:+ 
>>> device_create(dsl_class, NULL, MKDEV(DRV_DSL_CPE_API_DEV_MAJOR, 0), 
>>> NULL, "dsl_cpe_api");
>>>    package/kernel/lantiq/ltq-vdsl/patches/100-compat.patch:+ 
>>> device_create(dsl_class, NULL, dsl_devt, NULL, "dsl_cpe_api0");
>>> - use callDSLMetrics() for luci, per jo
>>> - add Tested-by tags
>>>
>>> This is to significantly speed up the generation of the metrics.
>>>
>>> The motivation comes from the fact that ~2.6s is just way too
>>> ineffcient for interval based metric collectors like prometheus or
>>> collectd.
>>>
>>> The luci status page also loads/refreshes alot faster.
>>>
>>> $ time /etc/init.d/dsl_control dslstat
>>> real    0m 2.66s
>>> user    0m 0.90s
>>> sys    0m 1.76s
>>>
>>> $ time ubus call dsl metrics
>>> real    0m 0.02s
>>> user    0m 0.00s
>>> sys    0m 0.01s
>>>
>>> The ltq-adsl-app changes are only compile time tested.
>>
>> who do I need to convince to review this? ;)
> 
> I would also like this great work to be merged.
> 
> Martin

Sending a reviewed-by and/or tested-by (with devices it was tested on) 
might help to push this forward.


best regards
Mathias
Martin Schiller Jan. 14, 2021, 12:34 p.m. UTC | #4
On 2021-01-14 12:55, Mathias Kresin wrote:
> 1/13/21 3:00 PM, Martin Schiller:
>> On 2021-01-09 09:40, Andre Heider wrote:
>>> Hi,
>>> 
>>> (cc'ed some recent commiters for lantiq)
>>> 
>>> On 15/12/2020 10:35, Andre Heider wrote:
>>>> v2:
>>>> - drop 0002-ltq-vdsl-app-fix-Wundef-warnings.patch
>>>> - use "/dev/dsl_cpe_api" without the "0" suffix for the adsl daemon:
>>>>    package/kernel/lantiq/ltq-adsl/patches/100-dsl_compat.patch:+ 
>>>> device_create(dsl_class, NULL, MKDEV(DRV_DSL_CPE_API_DEV_MAJOR, 0), 
>>>> NULL, "dsl_cpe_api");
>>>>    package/kernel/lantiq/ltq-vdsl/patches/100-compat.patch:+ 
>>>> device_create(dsl_class, NULL, dsl_devt, NULL, "dsl_cpe_api0");
>>>> - use callDSLMetrics() for luci, per jo
>>>> - add Tested-by tags
>>>> 
>>>> This is to significantly speed up the generation of the metrics.
>>>> 
>>>> The motivation comes from the fact that ~2.6s is just way too
>>>> ineffcient for interval based metric collectors like prometheus or
>>>> collectd.
>>>> 
>>>> The luci status page also loads/refreshes alot faster.
>>>> 
>>>> $ time /etc/init.d/dsl_control dslstat
>>>> real    0m 2.66s
>>>> user    0m 0.90s
>>>> sys    0m 1.76s
>>>> 
>>>> $ time ubus call dsl metrics
>>>> real    0m 0.02s
>>>> user    0m 0.00s
>>>> sys    0m 0.01s
>>>> 
>>>> The ltq-adsl-app changes are only compile time tested.
>>> 
>>> who do I need to convince to review this? ;)
>> 
>> I would also like this great work to be merged.
>> 
>> Martin
> 
> Sending a reviewed-by and/or tested-by (with devices it was tested on)
> might help to push this forward.

OK.

I've successfully tested the following parts of this patchset (v2) on
our (TDT) xrx200 based systems:

[01/13] ltq-vdsl-app: shutdown upon sigterm
[02/12] ltq-vdsl-app: add ubus support to get metrics
[04/12] luci-mod-status: use the new ubus dsl metrics
[05/12] rpcd-mod-luci: get rid of now unused getDSLStatus ubus rpc
[07/12] ltq-vdsl-app: use ubus to provide metrics
[09/12] ltq-dsl-base: remove usused lantiq_dsl.sh
[10/12] ltq-dsl-base: bump PKG_RELEASE
[11/12] ltq-vdsl-app: bump PKG_RELEASE

I am not able to test the ltq-adsl (Danube etc.) stuff.

Tested-by: Martin Schiller <ms@dev.tdt.de>

It is also important to note that the output of dslstat has changed
completely and lucistat no longer exists. If someone has processed the
output directly, it must be reworked.

Regards,
Martin
Andre Heider Jan. 15, 2021, 5:29 a.m. UTC | #5
Hi,

On 14/01/2021 13:34, Martin Schiller wrote:
> On 2021-01-14 12:55, Mathias Kresin wrote:
>> 1/13/21 3:00 PM, Martin Schiller:
>>> On 2021-01-09 09:40, Andre Heider wrote:
>>>> Hi,
>>>>
>>>> (cc'ed some recent commiters for lantiq)
>>>>
>>>> On 15/12/2020 10:35, Andre Heider wrote:
>>>>> v2:
>>>>> - drop 0002-ltq-vdsl-app-fix-Wundef-warnings.patch
>>>>> - use "/dev/dsl_cpe_api" without the "0" suffix for the adsl daemon:
>>>>>    package/kernel/lantiq/ltq-adsl/patches/100-dsl_compat.patch:+ 
>>>>> device_create(dsl_class, NULL, MKDEV(DRV_DSL_CPE_API_DEV_MAJOR, 0), 
>>>>> NULL, "dsl_cpe_api");
>>>>>    package/kernel/lantiq/ltq-vdsl/patches/100-compat.patch:+ 
>>>>> device_create(dsl_class, NULL, dsl_devt, NULL, "dsl_cpe_api0");
>>>>> - use callDSLMetrics() for luci, per jo
>>>>> - add Tested-by tags
>>>>>
>>>>> This is to significantly speed up the generation of the metrics.
>>>>>
>>>>> The motivation comes from the fact that ~2.6s is just way too
>>>>> ineffcient for interval based metric collectors like prometheus or
>>>>> collectd.
>>>>>
>>>>> The luci status page also loads/refreshes alot faster.
>>>>>
>>>>> $ time /etc/init.d/dsl_control dslstat
>>>>> real    0m 2.66s
>>>>> user    0m 0.90s
>>>>> sys    0m 1.76s
>>>>>
>>>>> $ time ubus call dsl metrics
>>>>> real    0m 0.02s
>>>>> user    0m 0.00s
>>>>> sys    0m 0.01s
>>>>>
>>>>> The ltq-adsl-app changes are only compile time tested.
>>>>
>>>> who do I need to convince to review this? ;)
>>>
>>> I would also like this great work to be merged.
>>>
>>> Martin
>>
>> Sending a reviewed-by and/or tested-by (with devices it was tested on)
>> might help to push this forward.
> 
> OK.
> 
> I've successfully tested the following parts of this patchset (v2) on
> our (TDT) xrx200 based systems:
> 
> [01/13] ltq-vdsl-app: shutdown upon sigterm
> [02/12] ltq-vdsl-app: add ubus support to get metrics
> [04/12] luci-mod-status: use the new ubus dsl metrics
> [05/12] rpcd-mod-luci: get rid of now unused getDSLStatus ubus rpc
> [07/12] ltq-vdsl-app: use ubus to provide metrics
> [09/12] ltq-dsl-base: remove usused lantiq_dsl.sh
> [10/12] ltq-dsl-base: bump PKG_RELEASE
> [11/12] ltq-vdsl-app: bump PKG_RELEASE
> 
> I am not able to test the ltq-adsl (Danube etc.) stuff.
> 
> Tested-by: Martin Schiller <ms@dev.tdt.de>

thanks again, your tags are already part of v2 based on v1 feedback.
Well except on 11/12, but if there isn't another reason to respin this 
series I'll refrain from sending a v3 for just the tag on a version bump ;)

> It is also important to note that the output of dslstat has changed
> completely and lucistat no longer exists. If someone has processed the
> output directly, it must be reworked.

Indeed, which is why I adapted all in-tree users.

I'm only aware of one downstream usage, a collectd plugin using the old 
dslstats. There is a PR on the package repo [0] about it, which is 
already waiting for this set to go in.

[0] https://github.com/openwrt/packages/pull/12175

Thanks,
Andre
Hauke Mehrtens Jan. 25, 2021, 8:22 p.m. UTC | #6
On 12/15/20 10:35 AM, Andre Heider wrote:
> v2:
> - drop 0002-ltq-vdsl-app-fix-Wundef-warnings.patch
> - use "/dev/dsl_cpe_api" without the "0" suffix for the adsl daemon:
>    package/kernel/lantiq/ltq-adsl/patches/100-dsl_compat.patch:+   device_create(dsl_class, NULL, MKDEV(DRV_DSL_CPE_API_DEV_MAJOR, 0), NULL, "dsl_cpe_api");
>    package/kernel/lantiq/ltq-vdsl/patches/100-compat.patch:+   device_create(dsl_class, NULL, dsl_devt, NULL, "dsl_cpe_api0");
> - use callDSLMetrics() for luci, per jo
> - add Tested-by tags
> 
> This is to significantly speed up the generation of the metrics.
> 
> The motivation comes from the fact that ~2.6s is just way too
> ineffcient for interval based metric collectors like prometheus or
> collectd.
> 
> The luci status page also loads/refreshes alot faster.
> 
> $ time /etc/init.d/dsl_control dslstat
> real	0m 2.66s
> user	0m 0.90s
> sys	0m 1.76s
> 
> $ time ubus call dsl metrics
> real	0m 0.02s
> user	0m 0.00s
> sys	0m 0.01s
> 
> The ltq-adsl-app changes are only compile time tested.
> 

Do you have pull requests on github for the luci and package feed changes?

Hauke
Andre Heider Jan. 26, 2021, 6:16 a.m. UTC | #7
On 25/01/2021 21:22, Hauke Mehrtens wrote:
> On 12/15/20 10:35 AM, Andre Heider wrote:
>> v2:
>> - drop 0002-ltq-vdsl-app-fix-Wundef-warnings.patch
>> - use "/dev/dsl_cpe_api" without the "0" suffix for the adsl daemon:
>>    package/kernel/lantiq/ltq-adsl/patches/100-dsl_compat.patch:+   
>> device_create(dsl_class, NULL, MKDEV(DRV_DSL_CPE_API_DEV_MAJOR, 0), 
>> NULL, "dsl_cpe_api");
>>    package/kernel/lantiq/ltq-vdsl/patches/100-compat.patch:+   
>> device_create(dsl_class, NULL, dsl_devt, NULL, "dsl_cpe_api0");
>> - use callDSLMetrics() for luci, per jo
>> - add Tested-by tags
>>
>> This is to significantly speed up the generation of the metrics.
>>
>> The motivation comes from the fact that ~2.6s is just way too
>> ineffcient for interval based metric collectors like prometheus or
>> collectd.
>>
>> The luci status page also loads/refreshes alot faster.
>>
>> $ time /etc/init.d/dsl_control dslstat
>> real    0m 2.66s
>> user    0m 0.90s
>> sys    0m 1.76s
>>
>> $ time ubus call dsl metrics
>> real    0m 0.02s
>> user    0m 0.00s
>> sys    0m 0.01s
>>
>> The ltq-adsl-app changes are only compile time tested.
>>
> 
> Do you have pull requests on github for the luci and package feed changes?

Nope, not yet. I assume you want me to do that, so I'll drop those 
patches here and link to the PRs for v3 instead.

Thanks,
Andre