| Message ID | 20220825201809.129034-1-yann.morin.1998@free.fr |
|---|---|
| State | Accepted |
| Headers | show |
| Series | [v2] package/python3: ensure pyc embed rooted paths | expand |
On Thu, 25 Aug 2022 22:18:09 +0200 "Yann E. MORIN" <yann.morin.1998@free.fr> wrote: > To report usable tracebacks, pyc files embed the path of the original py > files, so that users can more easily try and debug the reported issue. > > We generate the pyc files by calling the python3-supplied compileall > script, to scan the directory where python modules are installed. Since > this is done on the build machine, we tell compileall.py to strip away > the TARGET_DIR prefix, as that has no meaning at runtime. > > However, compileall.py forgets [0] to keep a leading / in the front of > the paths, thus generating non-rooted paths., e.g.: > /path/buildroot.ouput/targt/usr/lib/python3.10/argparse.py > gets embedded as: > usr/lib/python3.10/argparse.py > > This is a bit confusing but, as far as we could see, should be mostly be > used for display purposes in tracebacks, and does not seem to impact > actual functionality. > > We fix that by instructing compileall.py that the embedded paths should > be rooted to / which generates proper paths in tracebacks. > > And alternate solution would be to swith gears, and tell compileall.py > exactly the resulting runtime "base" directory, which replaces the > stripping and prefixing; i.e. it's either: > -s $(TARGET_DIR) -p / > or > -d /usr/lib/python$(PYTHON3_MAJOR_VERSION) > > We choose to keep the first solution, because that is semantically what > we really want to do: to strip the leading build-time path, rather than > to force anything. > > Note: the python test-suite was executed with both solutions (in a > pyc-only setup), and the results were exactly the same; so in practice, > -d or -s+-p yield the same results. > > Many thanks go to Vincent for reporting the issue and suggesting the > solutions. > > [0] Not sure whether this is a bug or a feature... > > Reported-by: Vincent Fazio <vfazio@xes-inc.com> > Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr> > > --- > Changes v1 -> v2: > - fix some typoes > - add the [0] ref > --- > package/python3/python3.mk | 1 + > 1 file changed, 1 insertion(+) Sorry, this is the one that I applied to master! Thomas
diff --git a/package/python3/python3.mk b/package/python3/python3.mk index 9af384712a..a9821d9a33 100644 --- a/package/python3/python3.mk +++ b/package/python3/python3.mk @@ -255,6 +255,7 @@ define PYTHON3_CREATE_PYC_FILES $(if $(VERBOSE),,-q) \ $(if $(BR2_PACKAGE_PYTHON3_PYC_ONLY),-b) \ -s $(TARGET_DIR) \ + -p / \ $(TARGET_DIR)/usr/lib/python$(PYTHON3_VERSION_MAJOR) endef
To report usable tracebacks, pyc files embed the path of the original py files, so that users can more easily try and debug the reported issue. We generate the pyc files by calling the python3-supplied compileall script, to scan the directory where python modules are installed. Since this is done on the build machine, we tell compileall.py to strip away the TARGET_DIR prefix, as that has no meaning at runtime. However, compileall.py forgets [0] to keep a leading / in the front of the paths, thus generating non-rooted paths., e.g.: /path/buildroot.ouput/targt/usr/lib/python3.10/argparse.py gets embedded as: usr/lib/python3.10/argparse.py This is a bit confusing but, as far as we could see, should be mostly be used for display purposes in tracebacks, and does not seem to impact actual functionality. We fix that by instructing compileall.py that the embedded paths should be rooted to / which generates proper paths in tracebacks. And alternate solution would be to swith gears, and tell compileall.py exactly the resulting runtime "base" directory, which replaces the stripping and prefixing; i.e. it's either: -s $(TARGET_DIR) -p / or -d /usr/lib/python$(PYTHON3_MAJOR_VERSION) We choose to keep the first solution, because that is semantically what we really want to do: to strip the leading build-time path, rather than to force anything. Note: the python test-suite was executed with both solutions (in a pyc-only setup), and the results were exactly the same; so in practice, -d or -s+-p yield the same results. Many thanks go to Vincent for reporting the issue and suggesting the solutions. [0] Not sure whether this is a bug or a feature... Reported-by: Vincent Fazio <vfazio@xes-inc.com> Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr> --- Changes v1 -> v2: - fix some typoes - add the [0] ref --- package/python3/python3.mk | 1 + 1 file changed, 1 insertion(+)