diff mbox series

[v3] package/gnuradio: fix gnuradio python libraries for cross-compile

Message ID 1689767858-17206-1-git-send-email-gwenj@trabucayre.com
State Accepted
Headers show
Series [v3] package/gnuradio: fix gnuradio python libraries for cross-compile | expand

Commit Message

Gwenhael Goavec-Merou July 19, 2023, 11:57 a.m. UTC 
From: Gwenhael Goavec-Merou <gwenhael.goavec-merou@trabucayre.com>

By default, module libraries have a suffix based on cpython version + host
architecture: this is fine for a native compile when these libraries are used on
the same computer (or similar computers). But when target architecture is not
the same python is unable to find libraries due to the wrong suffix and produces
unclear errors messages:

# python3
Python 3.11.3 (main, Jun 19 2023, 14:15:44) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from gnuradio import blocks  
Traceback (most recent call last):
  File "/home/xxx/buildroot/output/build/gnuradio-3.10.4.0/gr-blocks/python/blocks/__init__.py", line 18, in <module>
ModuleNotFoundError: No module named 'gnuradio.blocks.blocks_python'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/xxx/buildroot/output/build/gnuradio-3.10.4.0/gr-blocks/python/blocks/__init__.py", line 22, in <module>
ModuleNotFoundError: No module named 'gnuradio.blocks.blocks_python'
>>>

By adding _PYTHON_SYSCONFIGDATA_NAME="$(PKG_PYTHON_SYSCONFIGDATA_NAME)" at configure time, sysconfig will
return correct informations (target architecture) instead of host architecture.

Signed-off-by: Gwenhael Goavec-Merou <gwenhael.goavec-merou@trabucayre.com>
---
Changes v2 -> v3:
- replaces PYTHON_MODULE_EXTENSION by _PYTHON_SYSCONFIGDATA_NAME env (arnout)
Changes v1 -> v2:
- replaces patch by PYTHON_MODULE_EXTENSION option (result is the same
  but less "noisy")
---
 package/gnuradio/gnuradio.mk | 4 ++++
 1 file changed, 4 insertions(+)

Comments

Thomas Petazzoni July 21, 2023, 9:12 p.m. UTC | #1 
On Wed, 19 Jul 2023 13:57:38 +0200
Gwenhael Goavec-Merou <gwenj@trabucayre.com> wrote:

> From: Gwenhael Goavec-Merou <gwenhael.goavec-merou@trabucayre.com>
> 
> By default, module libraries have a suffix based on cpython version + host
> architecture: this is fine for a native compile when these libraries are used on
> the same computer (or similar computers). But when target architecture is not
> the same python is unable to find libraries due to the wrong suffix and produces
> unclear errors messages:
> 
> # python3
> Python 3.11.3 (main, Jun 19 2023, 14:15:44) [GCC 11.4.0] on linux
> Type "help", "copyright", "credits" or "license" for more information.
> >>> from gnuradio import blocks    
> Traceback (most recent call last):
>   File "/home/xxx/buildroot/output/build/gnuradio-3.10.4.0/gr-blocks/python/blocks/__init__.py", line 18, in <module>
> ModuleNotFoundError: No module named 'gnuradio.blocks.blocks_python'
> 
> During handling of the above exception, another exception occurred:
> 
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
>   File "/home/xxx/buildroot/output/build/gnuradio-3.10.4.0/gr-blocks/python/blocks/__init__.py", line 22, in <module>
> ModuleNotFoundError: No module named 'gnuradio.blocks.blocks_python'
> >>>  
> 
> By adding _PYTHON_SYSCONFIGDATA_NAME="$(PKG_PYTHON_SYSCONFIGDATA_NAME)" at configure time, sysconfig will
> return correct informations (target architecture) instead of host architecture.
> 
> Signed-off-by: Gwenhael Goavec-Merou <gwenhael.goavec-merou@trabucayre.com>
> ---
> Changes v2 -> v3:
> - replaces PYTHON_MODULE_EXTENSION by _PYTHON_SYSCONFIGDATA_NAME env (arnout)
> Changes v1 -> v2:
> - replaces patch by PYTHON_MODULE_EXTENSION option (result is the same
>   but less "noisy")
> ---
>  package/gnuradio/gnuradio.mk | 4 ++++
>  1 file changed, 4 insertions(+)

Not a show-stopper to merge this patch, but would it be possible to add
a runtime test-case for gnuradio Python modules in support/testing/ ?
I'm sure if you provide enough details to Julien Olivain (in cc), he
will be able to help you add this test.

Thanks!

Thomas
Julien Olivain July 22, 2023, 10:28 a.m. UTC | #2 
Hi Thomas, Gwenhael,

On 21/07/2023 23:12, Thomas Petazzoni wrote:
> On Wed, 19 Jul 2023 13:57:38 +0200
> Gwenhael Goavec-Merou <gwenj@trabucayre.com> wrote:
> 
>> From: Gwenhael Goavec-Merou <gwenhael.goavec-merou@trabucayre.com>
>> 
>> By default, module libraries have a suffix based on cpython version + 
>> host
>> architecture: this is fine for a native compile when these libraries 
>> are used on
>> the same computer (or similar computers). But when target architecture 
>> is not
>> the same python is unable to find libraries due to the wrong suffix 
>> and produces
>> unclear errors messages:
>> 
>> # python3
>> Python 3.11.3 (main, Jun 19 2023, 14:15:44) [GCC 11.4.0] on linux
>> Type "help", "copyright", "credits" or "license" for more information.
>> >>> from gnuradio import blocks
>> Traceback (most recent call last):
>>   File 
>> "/home/xxx/buildroot/output/build/gnuradio-3.10.4.0/gr-blocks/python/blocks/__init__.py", 
>> line 18, in <module>
>> ModuleNotFoundError: No module named 'gnuradio.blocks.blocks_python'
>> 
>> During handling of the above exception, another exception occurred:
>> 
>> Traceback (most recent call last):
>>   File "<stdin>", line 1, in <module>
>>   File 
>> "/home/xxx/buildroot/output/build/gnuradio-3.10.4.0/gr-blocks/python/blocks/__init__.py", 
>> line 22, in <module>
>> ModuleNotFoundError: No module named 'gnuradio.blocks.blocks_python'
>> >>>
>> 
>> By adding 
>> _PYTHON_SYSCONFIGDATA_NAME="$(PKG_PYTHON_SYSCONFIGDATA_NAME)" at 
>> configure time, sysconfig will
>> return correct informations (target architecture) instead of host 
>> architecture.
>> 
>> Signed-off-by: Gwenhael Goavec-Merou 
>> <gwenhael.goavec-merou@trabucayre.com>
>> ---
>> Changes v2 -> v3:
>> - replaces PYTHON_MODULE_EXTENSION by _PYTHON_SYSCONFIGDATA_NAME env 
>> (arnout)
>> Changes v1 -> v2:
>> - replaces patch by PYTHON_MODULE_EXTENSION option (result is the same
>>   but less "noisy")
>> ---
>>  package/gnuradio/gnuradio.mk | 4 ++++
>>  1 file changed, 4 insertions(+)
> 
> Not a show-stopper to merge this patch, but would it be possible to add
> a runtime test-case for gnuradio Python modules in support/testing/ ?
> I'm sure if you provide enough details to Julien Olivain (in cc), he
> will be able to help you add this test.

I'm already comfortable with Gnuradio development, so I'm willing to
write such a test. I'll send a proposal in the coming days...

Thanks for the suggestion!

> 
> Thanks!
> 
> Thomas

Best regards,

Julien.
Thomas Petazzoni July 22, 2023, 9:15 p.m. UTC | #3 
On Wed, 19 Jul 2023 13:57:38 +0200
Gwenhael Goavec-Merou <gwenj@trabucayre.com> wrote:

> From: Gwenhael Goavec-Merou <gwenhael.goavec-merou@trabucayre.com>
> 
> By default, module libraries have a suffix based on cpython version + host
> architecture: this is fine for a native compile when these libraries are used on
> the same computer (or similar computers). But when target architecture is not
> the same python is unable to find libraries due to the wrong suffix and produces
> unclear errors messages:
> 
> # python3
> Python 3.11.3 (main, Jun 19 2023, 14:15:44) [GCC 11.4.0] on linux
> Type "help", "copyright", "credits" or "license" for more information.
> >>> from gnuradio import blocks    
> Traceback (most recent call last):
>   File "/home/xxx/buildroot/output/build/gnuradio-3.10.4.0/gr-blocks/python/blocks/__init__.py", line 18, in <module>
> ModuleNotFoundError: No module named 'gnuradio.blocks.blocks_python'
> 
> During handling of the above exception, another exception occurred:
> 
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
>   File "/home/xxx/buildroot/output/build/gnuradio-3.10.4.0/gr-blocks/python/blocks/__init__.py", line 22, in <module>
> ModuleNotFoundError: No module named 'gnuradio.blocks.blocks_python'
> >>>  
> 
> By adding _PYTHON_SYSCONFIGDATA_NAME="$(PKG_PYTHON_SYSCONFIGDATA_NAME)" at configure time, sysconfig will
> return correct informations (target architecture) instead of host architecture.
> 
> Signed-off-by: Gwenhael Goavec-Merou <gwenhael.goavec-merou@trabucayre.com>
> ---
> Changes v2 -> v3:
> - replaces PYTHON_MODULE_EXTENSION by _PYTHON_SYSCONFIGDATA_NAME env (arnout)
> Changes v1 -> v2:
> - replaces patch by PYTHON_MODULE_EXTENSION option (result is the same
>   but less "noisy")
> ---
>  package/gnuradio/gnuradio.mk | 4 ++++
>  1 file changed, 4 insertions(+)

Applied to master, thanks.

Thomas
Peter Korsgaard Aug. 29, 2023, 4:52 p.m. UTC | #4 
>>>>> "Gwenhael" == Gwenhael Goavec-Merou <gwenj@trabucayre.com> writes:

 > From: Gwenhael Goavec-Merou <gwenhael.goavec-merou@trabucayre.com>
 > By default, module libraries have a suffix based on cpython version + host
 > architecture: this is fine for a native compile when these libraries are used on
 > the same computer (or similar computers). But when target architecture is not
 > the same python is unable to find libraries due to the wrong suffix and produces
 > unclear errors messages:

 > # python3
 > Python 3.11.3 (main, Jun 19 2023, 14:15:44) [GCC 11.4.0] on linux
 > Type "help", "copyright", "credits" or "license" for more information.
 >>>> from gnuradio import blocks  
 > Traceback (most recent call last):
 >   File "/home/xxx/buildroot/output/build/gnuradio-3.10.4.0/gr-blocks/python/blocks/__init__.py", line 18, in <module>
 > ModuleNotFoundError: No module named 'gnuradio.blocks.blocks_python'

 > During handling of the above exception, another exception occurred:

 > Traceback (most recent call last):
 >   File "<stdin>", line 1, in <module>
 >   File "/home/xxx/buildroot/output/build/gnuradio-3.10.4.0/gr-blocks/python/blocks/__init__.py", line 22, in <module>
 > ModuleNotFoundError: No module named 'gnuradio.blocks.blocks_python'
 >>>> 

 > By adding _PYTHON_SYSCONFIGDATA_NAME="$(PKG_PYTHON_SYSCONFIGDATA_NAME)" at configure time, sysconfig will
 > return correct informations (target architecture) instead of host architecture.

 > Signed-off-by: Gwenhael Goavec-Merou <gwenhael.goavec-merou@trabucayre.com>
 > ---
 > Changes v2 -> v3:
 > - replaces PYTHON_MODULE_EXTENSION by _PYTHON_SYSCONFIGDATA_NAME env (arnout)
 > Changes v1 -> v2:
 > - replaces patch by PYTHON_MODULE_EXTENSION option (result is the same
 >   but less "noisy")

Committed to 2023.02.x and 2023.05.x, thanks.
diff mbox series

Patch

diff --git a/package/gnuradio/gnuradio.mk b/package/gnuradio/gnuradio.mk
index 85b359088e..4e82c9e129 100644
--- a/package/gnuradio/gnuradio.mk
+++ b/package/gnuradio/gnuradio.mk
@@ -119,6 +119,10 @@  GNURADIO_CONF_OPTS += -DENABLE_PYTHON=ON
 # mandatory to avoid pybind11 to overwrite variables provided
 # by gnuradio and buildroot
 GNURADIO_CONF_OPTS += -DPYBIND11_PYTHONLIBS_OVERWRITE=OFF
+# mandatory to avoid pybind11 to force libraries extensions
+# with a name based on host architecture
+GNURADIO_CONF_ENV += _PYTHON_SYSCONFIGDATA_NAME="$(PKG_PYTHON_SYSCONFIGDATA_NAME)" \
+	PYTHONPATH=$(PYTHON3_PATH)
 # mandatory to install python modules in site-packages and to use
 # correct path for python libraries
 GNURADIO_CONF_OPTS += -DGR_PYTHON_RELATIVE=ON \