| Message ID | 20191031145122.13558-1-trini@konsulko.com |
|---|---|
| State | Deferred |
| Delegated to: | Tom Rini |
| Headers | show |
| Series | [U-Boot] test/py: Set cache_dir to /tmp/.pytest_cache | expand |
Hi Tom, On Thu, Oct 31, 2019 at 10:51 PM Tom Rini <trini@konsulko.com> wrote: > > For some time now, pytest has supported setting where a cache directory > that it will use is located (and in turn has logic to re-use this cache > and speed things up when possible). When running the pytest tests on > Azure Pipelines, our source directory is read-only and we now see a Thanks for checking Azure. Though I did not see such warning from pytest in my run below. Which job did you see it? https://dev.azure.com/bmeng/GitHub/_build/results?buildId=134 BTW: It looks the green check mark on https://github.com/u-boot/u-boot/commits/master only contains Travis-CI results. Would you please set up the connection to Azure pipeline as well? > warning about being unable to create the .pytest_cache directory as the > default is in the same place as pytest.ini. Set cache_dir to > /tmp/.pytest_cache so that it will always be placed in a read/write > location. > > Cc: Stephen Warren <swarren@nvidia.com> > Cc: Simon Glass <sjg@chromium.org> > Cc: Bin Meng <bmeng.cn@gmail.com> > Signed-off-by: Tom Rini <trini@konsulko.com> > --- > Looking at > https://docs.pytest.org/en/latest/customize.html#confval-cache_dir we > could use an environment variable but TMP/TMPDIR/etc aren't always going > to be set and that in turn leads to test/py/$TMPDIR/.pytest_cache being > used. I think hard-coding to /tmp here ends up being less problematic > but I am open to suggestions. We could even just ignore the warning, I > only spotted it when seeing why a test had otherwise failed. > --- > test/py/pytest.ini | 1 + > 1 file changed, 1 insertion(+) > > diff --git a/test/py/pytest.ini b/test/py/pytest.ini > index e93d010f1fa2..56c1f0521a54 100644 > --- a/test/py/pytest.ini > +++ b/test/py/pytest.ini > @@ -11,3 +11,4 @@ markers = > notbuildconfigspec: U-Boot: Describes required disabled Kconfig options. > requiredtool: U-Boot: Required host tools for a test. > slow: U-Boot: Specific test will run slowly. > +cache_dir = /tmp/.pytest_cache > -- Regards, Bin
On Thu, Oct 31, 2019 at 11:13:32PM +0800, Bin Meng wrote: > Hi Tom, > > On Thu, Oct 31, 2019 at 10:51 PM Tom Rini <trini@konsulko.com> wrote: > > > > For some time now, pytest has supported setting where a cache directory > > that it will use is located (and in turn has logic to re-use this cache > > and speed things up when possible). When running the pytest tests on > > Azure Pipelines, our source directory is read-only and we now see a > > Thanks for checking Azure. Though I did not see such warning from > pytest in my run below. Which job did you see it? > https://dev.azure.com/bmeng/GitHub/_build/results?buildId=134 I don't see it myself now in a non-failing job and I hit rerun on the one that had failed (since it looked like one of the sometimes fails I see). > BTW: It looks the green check mark on > https://github.com/u-boot/u-boot/commits/master only contains > Travis-CI results. Would you please set up the connection to Azure > pipeline as well? Thanks, I'll put it on my TODO list.
On 10/31/19 8:51 AM, Tom Rini wrote: > For some time now, pytest has supported setting where a cache directory > that it will use is located (and in turn has logic to re-use this cache > and speed things up when possible). When running the pytest tests on > Azure Pipelines, our source directory is read-only and we now see a > warning about being unable to create the .pytest_cache directory as the > default is in the same place as pytest.ini. Set cache_dir to > /tmp/.pytest_cache so that it will always be placed in a read/write > location. > > Cc: Stephen Warren <swarren@nvidia.com> > Cc: Simon Glass <sjg@chromium.org> > Cc: Bin Meng <bmeng.cn@gmail.com> > Signed-off-by: Tom Rini <trini@konsulko.com> > --- > Looking at > https://docs.pytest.org/en/latest/customize.html#confval-cache_dir we > could use an environment variable but TMP/TMPDIR/etc aren't always going > to be set and that in turn leads to test/py/$TMPDIR/.pytest_cache being > used. I think hard-coding to /tmp here ends up being less problematic > but I am open to suggestions. We could even just ignore the warning, I > only spotted it when seeing why a test had otherwise failed. > --- > test/py/pytest.ini | 1 + > 1 file changed, 1 insertion(+) > > diff --git a/test/py/pytest.ini b/test/py/pytest.ini > index e93d010f1fa2..56c1f0521a54 100644 > --- a/test/py/pytest.ini > +++ b/test/py/pytest.ini > @@ -11,3 +11,4 @@ markers = > notbuildconfigspec: U-Boot: Describes required disabled Kconfig options. > requiredtool: U-Boot: Required host tools for a test. > slow: U-Boot: Specific test will run slowly. > +cache_dir = /tmp/.pytest_cache Won't this hard-coding break running parallel invocations; they'll all try to use the same cache directory and conflict with each-other? Various test/py tests create data files (e.g. disk images, files for DFU to download/upload, etc.). That directory must be writable. Can we tell pytest to use (a sub-directory of that) directory as the cache?
On Thu, Oct 31, 2019 at 10:04:18AM -0600, Stephen Warren wrote: > On 10/31/19 8:51 AM, Tom Rini wrote: > > For some time now, pytest has supported setting where a cache directory > > that it will use is located (and in turn has logic to re-use this cache > > and speed things up when possible). When running the pytest tests on > > Azure Pipelines, our source directory is read-only and we now see a > > warning about being unable to create the .pytest_cache directory as the > > default is in the same place as pytest.ini. Set cache_dir to > > /tmp/.pytest_cache so that it will always be placed in a read/write > > location. > > > > Cc: Stephen Warren <swarren@nvidia.com> > > Cc: Simon Glass <sjg@chromium.org> > > Cc: Bin Meng <bmeng.cn@gmail.com> > > Signed-off-by: Tom Rini <trini@konsulko.com> > > --- > > Looking at > > https://docs.pytest.org/en/latest/customize.html#confval-cache_dir we > > could use an environment variable but TMP/TMPDIR/etc aren't always going > > to be set and that in turn leads to test/py/$TMPDIR/.pytest_cache being > > used. I think hard-coding to /tmp here ends up being less problematic > > but I am open to suggestions. We could even just ignore the warning, I > > only spotted it when seeing why a test had otherwise failed. > > --- > > test/py/pytest.ini | 1 + > > 1 file changed, 1 insertion(+) > > > > diff --git a/test/py/pytest.ini b/test/py/pytest.ini > > index e93d010f1fa2..56c1f0521a54 100644 > > --- a/test/py/pytest.ini > > +++ b/test/py/pytest.ini > > @@ -11,3 +11,4 @@ markers = > > notbuildconfigspec: U-Boot: Describes required disabled Kconfig options. > > requiredtool: U-Boot: Required host tools for a test. > > slow: U-Boot: Specific test will run slowly. > > +cache_dir = /tmp/.pytest_cache > > Won't this hard-coding break running parallel invocations; they'll all try > to use the same cache directory and conflict with each-other? > > Various test/py tests create data files (e.g. disk images, files for DFU to > download/upload, etc.). That directory must be writable. Can we tell pytest > to use (a sub-directory of that) directory as the cache? I don't think it will break anything today at least. We don't use pytest's cache_dir directly for creating our disk images, we put those elsewhere. This is strictly used by pytest itself, afaict.
On 10/31/19 10:07 AM, Tom Rini wrote: > On Thu, Oct 31, 2019 at 10:04:18AM -0600, Stephen Warren wrote: >> On 10/31/19 8:51 AM, Tom Rini wrote: >>> For some time now, pytest has supported setting where a cache directory >>> that it will use is located (and in turn has logic to re-use this cache >>> and speed things up when possible). When running the pytest tests on >>> Azure Pipelines, our source directory is read-only and we now see a >>> warning about being unable to create the .pytest_cache directory as the >>> default is in the same place as pytest.ini. Set cache_dir to >>> /tmp/.pytest_cache so that it will always be placed in a read/write >>> location. >>> >>> Cc: Stephen Warren <swarren@nvidia.com> >>> Cc: Simon Glass <sjg@chromium.org> >>> Cc: Bin Meng <bmeng.cn@gmail.com> >>> Signed-off-by: Tom Rini <trini@konsulko.com> >>> --- >>> Looking at >>> https://docs.pytest.org/en/latest/customize.html#confval-cache_dir we >>> could use an environment variable but TMP/TMPDIR/etc aren't always going >>> to be set and that in turn leads to test/py/$TMPDIR/.pytest_cache being >>> used. I think hard-coding to /tmp here ends up being less problematic >>> but I am open to suggestions. We could even just ignore the warning, I >>> only spotted it when seeing why a test had otherwise failed. >>> --- >>> test/py/pytest.ini | 1 + >>> 1 file changed, 1 insertion(+) >>> >>> diff --git a/test/py/pytest.ini b/test/py/pytest.ini >>> index e93d010f1fa2..56c1f0521a54 100644 >>> --- a/test/py/pytest.ini >>> +++ b/test/py/pytest.ini >>> @@ -11,3 +11,4 @@ markers = >>> notbuildconfigspec: U-Boot: Describes required disabled Kconfig options. >>> requiredtool: U-Boot: Required host tools for a test. >>> slow: U-Boot: Specific test will run slowly. >>> +cache_dir = /tmp/.pytest_cache >> >> Won't this hard-coding break running parallel invocations; they'll all try >> to use the same cache directory and conflict with each-other? >> >> Various test/py tests create data files (e.g. disk images, files for DFU to >> download/upload, etc.). That directory must be writable. Can we tell pytest >> to use (a sub-directory of that) directory as the cache? > > I don't think it will break anything today at least. We don't use > pytest's cache_dir directly for creating our disk images, we put those > elsewhere. This is strictly used by pytest itself, afaict. Sure, but does pytest itself have all the required locking in place to support running multiple copies in parallel all set to the same cache directory? I run 3-5 instances of pytest at once on the same system, each managing a different target board.
On Thu, Oct 31, 2019 at 10:12:06AM -0600, Stephen Warren wrote: > On 10/31/19 10:07 AM, Tom Rini wrote: > > On Thu, Oct 31, 2019 at 10:04:18AM -0600, Stephen Warren wrote: > > > On 10/31/19 8:51 AM, Tom Rini wrote: > > > > For some time now, pytest has supported setting where a cache directory > > > > that it will use is located (and in turn has logic to re-use this cache > > > > and speed things up when possible). When running the pytest tests on > > > > Azure Pipelines, our source directory is read-only and we now see a > > > > warning about being unable to create the .pytest_cache directory as the > > > > default is in the same place as pytest.ini. Set cache_dir to > > > > /tmp/.pytest_cache so that it will always be placed in a read/write > > > > location. > > > > > > > > Cc: Stephen Warren <swarren@nvidia.com> > > > > Cc: Simon Glass <sjg@chromium.org> > > > > Cc: Bin Meng <bmeng.cn@gmail.com> > > > > Signed-off-by: Tom Rini <trini@konsulko.com> > > > > --- > > > > Looking at > > > > https://docs.pytest.org/en/latest/customize.html#confval-cache_dir we > > > > could use an environment variable but TMP/TMPDIR/etc aren't always going > > > > to be set and that in turn leads to test/py/$TMPDIR/.pytest_cache being > > > > used. I think hard-coding to /tmp here ends up being less problematic > > > > but I am open to suggestions. We could even just ignore the warning, I > > > > only spotted it when seeing why a test had otherwise failed. > > > > --- > > > > test/py/pytest.ini | 1 + > > > > 1 file changed, 1 insertion(+) > > > > > > > > diff --git a/test/py/pytest.ini b/test/py/pytest.ini > > > > index e93d010f1fa2..56c1f0521a54 100644 > > > > --- a/test/py/pytest.ini > > > > +++ b/test/py/pytest.ini > > > > @@ -11,3 +11,4 @@ markers = > > > > notbuildconfigspec: U-Boot: Describes required disabled Kconfig options. > > > > requiredtool: U-Boot: Required host tools for a test. > > > > slow: U-Boot: Specific test will run slowly. > > > > +cache_dir = /tmp/.pytest_cache > > > > > > Won't this hard-coding break running parallel invocations; they'll all try > > > to use the same cache directory and conflict with each-other? > > > > > > Various test/py tests create data files (e.g. disk images, files for DFU to > > > download/upload, etc.). That directory must be writable. Can we tell pytest > > > to use (a sub-directory of that) directory as the cache? > > > > I don't think it will break anything today at least. We don't use > > pytest's cache_dir directly for creating our disk images, we put those > > elsewhere. This is strictly used by pytest itself, afaict. > > Sure, but does pytest itself have all the required locking in place to > support running multiple copies in parallel all set to the same cache > directory? I run 3-5 instances of pytest at once on the same system, each > managing a different target board. I hope so. Can you throw this into your setup and see if anything breaks? Thanks!
On 10/31/19 10:17 AM, Tom Rini wrote: > On Thu, Oct 31, 2019 at 10:12:06AM -0600, Stephen Warren wrote: >> On 10/31/19 10:07 AM, Tom Rini wrote: >>> On Thu, Oct 31, 2019 at 10:04:18AM -0600, Stephen Warren wrote: >>>> On 10/31/19 8:51 AM, Tom Rini wrote: >>>>> For some time now, pytest has supported setting where a cache directory >>>>> that it will use is located (and in turn has logic to re-use this cache >>>>> and speed things up when possible). When running the pytest tests on >>>>> Azure Pipelines, our source directory is read-only and we now see a >>>>> warning about being unable to create the .pytest_cache directory as the >>>>> default is in the same place as pytest.ini. Set cache_dir to >>>>> /tmp/.pytest_cache so that it will always be placed in a read/write >>>>> location. >>>>> >>>>> Cc: Stephen Warren <swarren@nvidia.com> >>>>> Cc: Simon Glass <sjg@chromium.org> >>>>> Cc: Bin Meng <bmeng.cn@gmail.com> >>>>> Signed-off-by: Tom Rini <trini@konsulko.com> >>>>> --- >>>>> Looking at >>>>> https://docs.pytest.org/en/latest/customize.html#confval-cache_dir we >>>>> could use an environment variable but TMP/TMPDIR/etc aren't always going >>>>> to be set and that in turn leads to test/py/$TMPDIR/.pytest_cache being >>>>> used. I think hard-coding to /tmp here ends up being less problematic >>>>> but I am open to suggestions. We could even just ignore the warning, I >>>>> only spotted it when seeing why a test had otherwise failed. >>>>> --- >>>>> test/py/pytest.ini | 1 + >>>>> 1 file changed, 1 insertion(+) >>>>> >>>>> diff --git a/test/py/pytest.ini b/test/py/pytest.ini >>>>> index e93d010f1fa2..56c1f0521a54 100644 >>>>> --- a/test/py/pytest.ini >>>>> +++ b/test/py/pytest.ini >>>>> @@ -11,3 +11,4 @@ markers = >>>>> notbuildconfigspec: U-Boot: Describes required disabled Kconfig options. >>>>> requiredtool: U-Boot: Required host tools for a test. >>>>> slow: U-Boot: Specific test will run slowly. >>>>> +cache_dir = /tmp/.pytest_cache >>>> >>>> Won't this hard-coding break running parallel invocations; they'll all try >>>> to use the same cache directory and conflict with each-other? >>>> >>>> Various test/py tests create data files (e.g. disk images, files for DFU to >>>> download/upload, etc.). That directory must be writable. Can we tell pytest >>>> to use (a sub-directory of that) directory as the cache? >>> >>> I don't think it will break anything today at least. We don't use >>> pytest's cache_dir directly for creating our disk images, we put those >>> elsewhere. This is strictly used by pytest itself, afaict. >> >> Sure, but does pytest itself have all the required locking in place to >> support running multiple copies in parallel all set to the same cache >> directory? I run 3-5 instances of pytest at once on the same system, each >> managing a different target board. > > I hope so. Can you throw this into your setup and see if anything > breaks? Thanks! That will only tell me whether I happen to notice problems, not whether it's guaranteed there will never be problems. I'll try and ask the pytest folk about the latter. That said, It looks like the cache plugin is intended to run-to-run data passing. We don't currently do that at all (at least, not via the pytest cache plugin); do we plan to? If not, it'd be simpler to just disable the plugin completely (pytest -p no:cacheprovider). Alternatively, I believe option values can be specified on the command-line, and hence can be easily varied per run to avoid conflicts.
diff --git a/test/py/pytest.ini b/test/py/pytest.ini index e93d010f1fa2..56c1f0521a54 100644 --- a/test/py/pytest.ini +++ b/test/py/pytest.ini @@ -11,3 +11,4 @@ markers = notbuildconfigspec: U-Boot: Describes required disabled Kconfig options. requiredtool: U-Boot: Required host tools for a test. slow: U-Boot: Specific test will run slowly. +cache_dir = /tmp/.pytest_cache
For some time now, pytest has supported setting where a cache directory that it will use is located (and in turn has logic to re-use this cache and speed things up when possible). When running the pytest tests on Azure Pipelines, our source directory is read-only and we now see a warning about being unable to create the .pytest_cache directory as the default is in the same place as pytest.ini. Set cache_dir to /tmp/.pytest_cache so that it will always be placed in a read/write location. Cc: Stephen Warren <swarren@nvidia.com> Cc: Simon Glass <sjg@chromium.org> Cc: Bin Meng <bmeng.cn@gmail.com> Signed-off-by: Tom Rini <trini@konsulko.com> --- Looking at https://docs.pytest.org/en/latest/customize.html#confval-cache_dir we could use an environment variable but TMP/TMPDIR/etc aren't always going to be set and that in turn leads to test/py/$TMPDIR/.pytest_cache being used. I think hard-coding to /tmp here ends up being less problematic but I am open to suggestions. We could even just ignore the warning, I only spotted it when seeing why a test had otherwise failed. --- test/py/pytest.ini | 1 + 1 file changed, 1 insertion(+)