| Message ID | 20200603001523.18085-5-jsnow@redhat.com |
|---|---|
| State | New |
| Headers | show |
| Series | python: create installable package | expand |
03.06.2020 03:15, John Snow wrote: > pipenv is a tool used for managing virtual environments with precisely > specified dependencies. It is separate from the dependencies listed in > setup.py, which are (by 'best practices') not supposed to be pinned. > > Note that pipenv is not required to install or use this module; this is > just a convenience for in-tree developing. > > Here, a "blank" pipfile is added with no dependencies, but specifies > Python 3.6 for the virtual environment. > > Pipfile will specify our version minimums, while Pipfile.lock specifies > an exact loudout of packages that were known to operate correctly. This > latter file provides the real value for easy setup of container images > and CI environments. > > Signed-off-by: John Snow <jsnow@redhat.com> > --- > python/Pipfile | 11 +++++++++++ > 1 file changed, 11 insertions(+) > create mode 100644 python/Pipfile > > diff --git a/python/Pipfile b/python/Pipfile > new file mode 100644 > index 00000000000..9534830b5eb > --- /dev/null > +++ b/python/Pipfile > @@ -0,0 +1,11 @@ > +[[source]] > +name = "pypi" > +url = "https://pypi.org/simple" > +verify_ssl = true I didn't find what does [[source]] mean, but seems it a standard header of Pipfile. Not sure, how much sense in it, but: Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> > + > +[dev-packages] > + > +[packages] > + > +[requires] > +python_version = "3.6" >
On 6/5/20 11:37 AM, Vladimir Sementsov-Ogievskiy wrote: > 03.06.2020 03:15, John Snow wrote: >> pipenv is a tool used for managing virtual environments with precisely >> specified dependencies. It is separate from the dependencies listed in >> setup.py, which are (by 'best practices') not supposed to be pinned. >> >> Note that pipenv is not required to install or use this module; this is >> just a convenience for in-tree developing. >> >> Here, a "blank" pipfile is added with no dependencies, but specifies >> Python 3.6 for the virtual environment. >> >> Pipfile will specify our version minimums, while Pipfile.lock specifies >> an exact loudout of packages that were known to operate correctly. This >> latter file provides the real value for easy setup of container images >> and CI environments. >> >> Signed-off-by: John Snow <jsnow@redhat.com> >> --- >> python/Pipfile | 11 +++++++++++ >> 1 file changed, 11 insertions(+) >> create mode 100644 python/Pipfile >> >> diff --git a/python/Pipfile b/python/Pipfile >> new file mode 100644 >> index 00000000000..9534830b5eb >> --- /dev/null >> +++ b/python/Pipfile >> @@ -0,0 +1,11 @@ >> +[[source]] >> +name = "pypi" >> +url = "https://pypi.org/simple" >> +verify_ssl = true > > I didn't find what does [[source]] mean, but seems it a standard header > of Pipfile. > > Not sure, how much sense in it, but: > Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> > Yeah, this is generated by the pipenv tool. I assume this describes which repositories should be probed for downloading packages. --js
03.06.2020 03:15, John Snow wrote: > pipenv is a tool used for managing virtual environments with precisely > specified dependencies. It is separate from the dependencies listed in > setup.py, which are (by 'best practices') not supposed to be pinned. > > Note that pipenv is not required to install or use this module; this is > just a convenience for in-tree developing. > > Here, a "blank" pipfile is added with no dependencies, but specifies > Python 3.6 for the virtual environment. > > Pipfile will specify our version minimums, while Pipfile.lock specifies > an exact loudout of packages that were known to operate correctly. This > latter file provides the real value for easy setup of container images > and CI environments. > > Signed-off-by: John Snow <jsnow@redhat.com> > --- > python/Pipfile | 11 +++++++++++ > 1 file changed, 11 insertions(+) > create mode 100644 python/Pipfile > > diff --git a/python/Pipfile b/python/Pipfile > new file mode 100644 > index 00000000000..9534830b5eb > --- /dev/null > +++ b/python/Pipfile > @@ -0,0 +1,11 @@ > +[[source]] > +name = "pypi" > +url = "https://pypi.org/simple" > +verify_ssl = true > + > +[dev-packages] > + > +[packages] > + > +[requires] > +python_version = "3.6" > Should it be >= or something like this? And, how should I use this all? My failed attempt: [root@kvm python]# pipenv install --python /usr/bin/python3 Virtualenv already exists! Removing existing virtualenv… Creating a virtualenv for this project… Pipfile: /work/src/qemu/john-python-installable/python/Pipfile Using /usr/bin/python3 (3.7.5) to create virtualenv… ⠹ Creating virtual environment...created virtual environment CPython3.7.5.final.0-64 in 112ms creator CPython3Posix(dest=/root/.local/share/virtualenvs/python-4FwBBPCc, clear=False, global=False) seeder FromAppData(download=False, pip=latest, setuptools=latest, wheel=latest, via=copy, app_data_dir=/root/.local/share/virtualenv/seed-app-data/v1.0.1) activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator ✔ Successfully created virtual environment! Virtualenv location: /root/.local/share/virtualenvs/python-4FwBBPCc Warning: Your Pipfile requires python_version 3.6, but you are using 3.7.5 (/root/.local/share/v/p/bin/python). $ pipenv --rm and rebuilding the virtual environment may resolve the issue. $ pipenv check will surely fail. Installing dependencies from Pipfile.lock (44d7bd)… 🐍 ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 0/0 — 00:00:00 To activate this project's virtualenv, run pipenv shell. Alternatively, run a command inside the virtualenv with pipenv run. [root@kvm python]# pipenv shell Warning: Your Pipfile requires python_version 3.6, but you are using 3.7.5 (/root/.local/share/v/p/bin/python). $ pipenv --rm and rebuilding the virtual environment may resolve the issue. $ pipenv check will surely fail. Launching subshell in virtual environment… . /root/.local/share/virtualenvs/python-4FwBBPCc/bin/activate [root@kvm work]# . /root/.local/share/virtualenvs/python-4FwBBPCc/bin/activate (python) [root@kvm work]# python Python 3.7.5 (default, Oct 17 2019, 12:09:47) [GCC 9.2.1 20190827 (Red Hat 9.2.1-1)] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import pylint Traceback (most recent call last): File "<stdin>", line 1, in <module> ModuleNotFoundError: No module named 'pylint' >>> and iotest 297 says: "pylint-3 not found" (honestly, I'm new to using python virtual environment)
On 6/5/20 12:21 PM, Vladimir Sementsov-Ogievskiy wrote: > 03.06.2020 03:15, John Snow wrote: >> pipenv is a tool used for managing virtual environments with precisely >> specified dependencies. It is separate from the dependencies listed in >> setup.py, which are (by 'best practices') not supposed to be pinned. >> >> Note that pipenv is not required to install or use this module; this is >> just a convenience for in-tree developing. >> >> Here, a "blank" pipfile is added with no dependencies, but specifies >> Python 3.6 for the virtual environment. >> >> Pipfile will specify our version minimums, while Pipfile.lock specifies >> an exact loudout of packages that were known to operate correctly. This >> latter file provides the real value for easy setup of container images >> and CI environments. >> >> Signed-off-by: John Snow <jsnow@redhat.com> >> --- >> python/Pipfile | 11 +++++++++++ >> 1 file changed, 11 insertions(+) >> create mode 100644 python/Pipfile >> >> diff --git a/python/Pipfile b/python/Pipfile >> new file mode 100644 >> index 00000000000..9534830b5eb >> --- /dev/null >> +++ b/python/Pipfile >> @@ -0,0 +1,11 @@ >> +[[source]] >> +name = "pypi" >> +url = "https://pypi.org/simple" >> +verify_ssl = true >> + >> +[dev-packages] >> + >> +[packages] >> + >> +[requires] >> +python_version = "3.6" >> > > Should it be >= or something like this? > I think logistically that makes sense, but I'm not sure if the tool supports it. I decided to target the oldest version of Python we support (for non-build infrastructure) to ensure a minimum viability. > And, how should I use this all? > > My failed attempt: > [root@kvm python]# pipenv install --python /usr/bin/python3 > Virtualenv already exists! > Removing existing virtualenv… > Creating a virtualenv for this project… > Pipfile: /work/src/qemu/john-python-installable/python/Pipfile > Using /usr/bin/python3 (3.7.5) to create virtualenv… > ⠹ Creating virtual environment...created virtual environment > CPython3.7.5.final.0-64 in 112ms > creator > CPython3Posix(dest=/root/.local/share/virtualenvs/python-4FwBBPCc, > clear=False, global=False) > seeder FromAppData(download=False, pip=latest, setuptools=latest, > wheel=latest, via=copy, > app_data_dir=/root/.local/share/virtualenv/seed-app-data/v1.0.1) > activators > BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator > > > ✔ Successfully created virtual environment! > Virtualenv location: /root/.local/share/virtualenvs/python-4FwBBPCc > Warning: Your Pipfile requires python_version 3.6, but you are using > 3.7.5 (/root/.local/share/v/p/bin/python). > $ pipenv --rm and rebuilding the virtual environment may resolve the > issue. > $ pipenv check will surely fail. > Installing dependencies from Pipfile.lock (44d7bd)… > 🐍 ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 0/0 — 00:00:00 > To activate this project's virtualenv, run pipenv shell. > Alternatively, run a command inside the virtualenv with pipenv run. > [root@kvm python]# pipenv shell > Warning: Your Pipfile requires python_version 3.6, but you are using > 3.7.5 (/root/.local/share/v/p/bin/python). > $ pipenv --rm and rebuilding the virtual environment may resolve the > issue. > $ pipenv check will surely fail. > Launching subshell in virtual environment… > . /root/.local/share/virtualenvs/python-4FwBBPCc/bin/activate > [root@kvm work]# . > /root/.local/share/virtualenvs/python-4FwBBPCc/bin/activate > (python) [root@kvm work]# python > Python 3.7.5 (default, Oct 17 2019, 12:09:47) > [GCC 9.2.1 20190827 (Red Hat 9.2.1-1)] on linux > Type "help", "copyright", "credits" or "license" for more information. >>>> import pylint > Traceback (most recent call last): > File "<stdin>", line 1, in <module> > ModuleNotFoundError: No module named 'pylint' >>>> > > and iotest 297 says: "pylint-3 not found" > Ah! that's a bug in iotest 297. It's expecting the fedora package there. I'll have to fix that. > (honestly, I'm new to using python virtual environment) > Good questions. I'll document this in the README.rst for this folder, actually... 1. Create a virtual environment > pipenv sync --dev jsnow@probe ~/s/q/python (python-package-refactor)> pipenv sync --dev Creating a virtualenv for this project… Pipfile: /home/jsnow/src/qemu/python/Pipfile Using /usr/bin/python3.6 (3.6.10) to create virtualenv… ⠏ Creating virtual environment...Using base prefix '/usr' New python executable in /home/jsnow/.local/share/virtualenvs/python-QepCANQl/bin/python3.6 Also creating executable in /home/jsnow/.local/share/virtualenvs/python-QepCANQl/bin/python Installing setuptools, pip, wheel...done. Running virtualenv with interpreter /usr/bin/python3.6 ✔ Successfully created virtual environment! Virtualenv location: /home/jsnow/.local/share/virtualenvs/python-QepCANQl Installing dependencies from Pipfile.lock (44d7bd)… 🐍 ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 17/17 — 00:00:07 To activate this project's virtualenv, run pipenv shell. Alternatively, run a command inside the virtualenv with pipenv run. All dependencies are now up-to-date! This command can both create and synchronize the venv's packages with those listed in Pipfile.lock. It may be helpful to know that Pipfile describes which packages, with coarse version requirements. Pipfile.lock describes EXACTLY which package versions to install. Pipenv, therefore, is a way to produce consistent execution environments in which we can run tests that are the same for everybody. When we want to update our packages for this repeatable environment, we can use 'pipenv update' and commit the Pipfile.lock changes back to git. (Note, using --dev here installs the development dependencies. If you omit it, you won't get any packages installed, because there are no runtime dependencies for this package!) 2. Where did it create the venv? jsnow@probe ~/s/q/python (python-package-refactor)> pipenv --venv /home/jsnow/.local/share/virtualenvs/python-QepCANQl 3. Entering the venv jsnow@probe ~/s/q/python (python-package-refactor)> pipenv shell Launching subshell in virtual environment… Welcome to fish, the friendly interactive shell jsnow@probe ~/s/q/python (python-package-refactor)> source /home/jsnow/.local/share/virtualenvs/python-QepCANQl/bin/activate.fish (python) jsnow@probe ~/s/q/python (python-package-refactor)> ^^^^^^ the virtual environment I am in my git branch ^^^^^^^^^^^^^^^^^^^^^^^ This just automates opening a sub-shell and then sourcing the activation file for you. You can do it manually if you'd like. 4. Installing the QEMU package in development mode to the venv So far, pipenv only installed our development requirements. Install the package itself. From inside `pipenv shell`: > pip install -e . Obtaining file:///home/jsnow/src/qemu/python Installing collected packages: qemu Running setup.py develop for qemu Successfully installed qemu Or, if you are outside the venv: > pipenv run pip install -e . 5. Using the qemu package (From inside the venv) (python) jsnow@probe ~/s/q/python (python-package-refactor)> python3 Python 3.6.10 (default, Dec 27 2019, 13:40:13) [GCC 9.2.1 20190827 (Red Hat 9.2.1-1)] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import qemu.core >>> dir(qemu.core) ['QEMUMachine', 'QEMUMonitorProtocol', 'QEMUQtestMachine', 'QEMUQtestProtocol', '__all__', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__path__', '__spec__', 'accel', 'kvm_available', 'list_accel', 'machine', 'qmp', 'qtest', 'tcg_available'] >>>
05.06.2020 20:11, John Snow wrote: > > > On 6/5/20 12:21 PM, Vladimir Sementsov-Ogievskiy wrote: >> 03.06.2020 03:15, John Snow wrote: >>> pipenv is a tool used for managing virtual environments with precisely >>> specified dependencies. It is separate from the dependencies listed in >>> setup.py, which are (by 'best practices') not supposed to be pinned. >>> >>> Note that pipenv is not required to install or use this module; this is >>> just a convenience for in-tree developing. >>> >>> Here, a "blank" pipfile is added with no dependencies, but specifies >>> Python 3.6 for the virtual environment. >>> >>> Pipfile will specify our version minimums, while Pipfile.lock specifies >>> an exact loudout of packages that were known to operate correctly. This >>> latter file provides the real value for easy setup of container images >>> and CI environments. >>> >>> Signed-off-by: John Snow <jsnow@redhat.com> >>> --- >>> python/Pipfile | 11 +++++++++++ >>> 1 file changed, 11 insertions(+) >>> create mode 100644 python/Pipfile >>> >>> diff --git a/python/Pipfile b/python/Pipfile >>> new file mode 100644 >>> index 00000000000..9534830b5eb >>> --- /dev/null >>> +++ b/python/Pipfile >>> @@ -0,0 +1,11 @@ >>> +[[source]] >>> +name = "pypi" >>> +url = "https://pypi.org/simple" >>> +verify_ssl = true >>> + >>> +[dev-packages] >>> + >>> +[packages] >>> + >>> +[requires] >>> +python_version = "3.6" >>> >> >> Should it be >= or something like this? >> > > I think logistically that makes sense, but I'm not sure if the tool > supports it. > > I decided to target the oldest version of Python we support (for > non-build infrastructure) to ensure a minimum viability. > >> And, how should I use this all? >> >> My failed attempt: >> [root@kvm python]# pipenv install --python /usr/bin/python3 >> Virtualenv already exists! >> Removing existing virtualenv… >> Creating a virtualenv for this project… >> Pipfile: /work/src/qemu/john-python-installable/python/Pipfile >> Using /usr/bin/python3 (3.7.5) to create virtualenv… >> ⠹ Creating virtual environment...created virtual environment >> CPython3.7.5.final.0-64 in 112ms >> creator >> CPython3Posix(dest=/root/.local/share/virtualenvs/python-4FwBBPCc, >> clear=False, global=False) >> seeder FromAppData(download=False, pip=latest, setuptools=latest, >> wheel=latest, via=copy, >> app_data_dir=/root/.local/share/virtualenv/seed-app-data/v1.0.1) >> activators >> BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator >> >> >> ✔ Successfully created virtual environment! >> Virtualenv location: /root/.local/share/virtualenvs/python-4FwBBPCc >> Warning: Your Pipfile requires python_version 3.6, but you are using >> 3.7.5 (/root/.local/share/v/p/bin/python). >> $ pipenv --rm and rebuilding the virtual environment may resolve the >> issue. >> $ pipenv check will surely fail. >> Installing dependencies from Pipfile.lock (44d7bd)… >> 🐍 ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 0/0 — 00:00:00 >> To activate this project's virtualenv, run pipenv shell. >> Alternatively, run a command inside the virtualenv with pipenv run. >> [root@kvm python]# pipenv shell >> Warning: Your Pipfile requires python_version 3.6, but you are using >> 3.7.5 (/root/.local/share/v/p/bin/python). >> $ pipenv --rm and rebuilding the virtual environment may resolve the >> issue. >> $ pipenv check will surely fail. >> Launching subshell in virtual environment… >> . /root/.local/share/virtualenvs/python-4FwBBPCc/bin/activate >> [root@kvm work]# . >> /root/.local/share/virtualenvs/python-4FwBBPCc/bin/activate >> (python) [root@kvm work]# python >> Python 3.7.5 (default, Oct 17 2019, 12:09:47) >> [GCC 9.2.1 20190827 (Red Hat 9.2.1-1)] on linux >> Type "help", "copyright", "credits" or "license" for more information. >>>>> import pylint >> Traceback (most recent call last): >> File "<stdin>", line 1, in <module> >> ModuleNotFoundError: No module named 'pylint' >>>>> >> >> and iotest 297 says: "pylint-3 not found" >> > > Ah! that's a bug in iotest 297. It's expecting the fedora package there. > I'll have to fix that. > >> (honestly, I'm new to using python virtual environment) >> > > Good questions. I'll document this in the README.rst for this folder, > actually... > > > > 1. Create a virtual environment > >> pipenv sync --dev > > jsnow@probe ~/s/q/python (python-package-refactor)> pipenv sync --dev > Creating a virtualenv for this project… > Pipfile: /home/jsnow/src/qemu/python/Pipfile > Using /usr/bin/python3.6 (3.6.10) to create virtualenv… > ⠏ Creating virtual environment...Using base prefix '/usr' > New python executable in > /home/jsnow/.local/share/virtualenvs/python-QepCANQl/bin/python3.6 > Also creating executable in > /home/jsnow/.local/share/virtualenvs/python-QepCANQl/bin/python > Installing setuptools, pip, wheel...done. > Running virtualenv with interpreter /usr/bin/python3.6 > > ✔ Successfully created virtual environment! > Virtualenv location: /home/jsnow/.local/share/virtualenvs/python-QepCANQl > Installing dependencies from Pipfile.lock (44d7bd)… > 🐍 ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 17/17 — 00:00:07 > To activate this project's virtualenv, run pipenv shell. > Alternatively, run a command inside the virtualenv with pipenv run. > All dependencies are now up-to-date! > > > This command can both create and synchronize the venv's packages with > those listed in Pipfile.lock. > > It may be helpful to know that Pipfile describes which packages, with > coarse version requirements. Pipfile.lock describes EXACTLY which > package versions to install. > > Pipenv, therefore, is a way to produce consistent execution environments > in which we can run tests that are the same for everybody. > > When we want to update our packages for this repeatable environment, we > can use 'pipenv update' and commit the Pipfile.lock changes back to git. > > (Note, using --dev here installs the development dependencies. If you > omit it, you won't get any packages installed, because there are no > runtime dependencies for this package!) Seems, that was my mistake, now it works, thanks! > > > 2. Where did it create the venv? > > jsnow@probe ~/s/q/python (python-package-refactor)> pipenv --venv > /home/jsnow/.local/share/virtualenvs/python-QepCANQl > > > 3. Entering the venv > > jsnow@probe ~/s/q/python (python-package-refactor)> pipenv shell > Launching subshell in virtual environment… > Welcome to fish, the friendly interactive shell > jsnow@probe ~/s/q/python (python-package-refactor)> source > /home/jsnow/.local/share/virtualenvs/python-QepCANQl/bin/activate.fish > > (python) jsnow@probe ~/s/q/python (python-package-refactor)> > > ^^^^^^ the virtual environment I am in > > my git branch ^^^^^^^^^^^^^^^^^^^^^^^ > > > This just automates opening a sub-shell and then sourcing the activation > file for you. You can do it manually if you'd like. > > > 4. Installing the QEMU package in development mode to the venv > > So far, pipenv only installed our development requirements. Install the > package itself. > > From inside `pipenv shell`: > >> pip install -e . > Obtaining file:///home/jsnow/src/qemu/python > Installing collected packages: qemu > Running setup.py develop for qemu > Successfully installed qemu > > Or, if you are outside the venv: > >> pipenv run pip install -e . > > > 5. Using the qemu package > > (From inside the venv) > > (python) jsnow@probe ~/s/q/python (python-package-refactor)> python3 > Python 3.6.10 (default, Dec 27 2019, 13:40:13) > [GCC 9.2.1 20190827 (Red Hat 9.2.1-1)] on linux > Type "help", "copyright", "credits" or "license" for more information. >>>> import qemu.core >>>> dir(qemu.core) > ['QEMUMachine', 'QEMUMonitorProtocol', 'QEMUQtestMachine', > 'QEMUQtestProtocol', '__all__', '__builtins__', '__cached__', '__doc__', > '__file__', '__loader__', '__name__', '__package__', '__path__', > '__spec__', 'accel', 'kvm_available', 'list_accel', 'machine', 'qmp', > 'qtest', 'tcg_available'] >>>> >
diff --git a/python/Pipfile b/python/Pipfile new file mode 100644 index 00000000000..9534830b5eb --- /dev/null +++ b/python/Pipfile @@ -0,0 +1,11 @@ +[[source]] +name = "pypi" +url = "https://pypi.org/simple" +verify_ssl = true + +[dev-packages] + +[packages] + +[requires] +python_version = "3.6"
pipenv is a tool used for managing virtual environments with precisely specified dependencies. It is separate from the dependencies listed in setup.py, which are (by 'best practices') not supposed to be pinned. Note that pipenv is not required to install or use this module; this is just a convenience for in-tree developing. Here, a "blank" pipfile is added with no dependencies, but specifies Python 3.6 for the virtual environment. Pipfile will specify our version minimums, while Pipfile.lock specifies an exact loudout of packages that were known to operate correctly. This latter file provides the real value for easy setup of container images and CI environments. Signed-off-by: John Snow <jsnow@redhat.com> --- python/Pipfile | 11 +++++++++++ 1 file changed, 11 insertions(+) create mode 100644 python/Pipfile