No method to control the python version used

[epage]

• I am on the latest Poetry version.
• I have searched the issues of this repo and believe that this is not a duplicate.
• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: Windows 10
• Poetry version: 0.12.2

Issue

I found it confusing when my pyproject.toml file says python = "3" and yet a Python 2.7 venv was being created.

I then noticed

The created virtualenv will use the Python executable for which poetry has been installed.
on https://poetry.eustace.io/docs/basic-usage/

I assumed it was using the version of python I ran get-poetry with. I tried again with python3 and still no luck.

It seems its just hard coded in poetry.bat as python and for me to change, I need to change that file.

To me, the ideal thing would be to a way to get arbitrary versions of python, like tox, and allow it to be configured on a per-project basis (like as a dev-dependency).

Workround

Edit poetry.bat to change what python gets invoked.

[xmo-odoo]

To me, the ideal thing would be to a way to get arbitrary versions of python, like tox, and allow it to be configured on a per-project basis (like as a dev-dependency).

And specify the "current python" when running impacted commands e.g. run: tox is good for running automated tests, but if you want to run the project to debug an issue it's not really helpful as that's not its purpose.

Having a way to e.g. poetry run -p 3.5 <script> and have it set up or reuse a previously set up Python 3.5 virtualenv would be super useful. The default should be whatever is specified by the toml file I guess?

[epage]

There are ways to do experimentation in tox but they are a bit confusing and hacky if your problem is user reported and not a tox test failure. Yeah, it seems like it'd make sense to be able to control this on the command line.

Another perspective on this is Rust's toolchain file. Users can pass in a --target flag to specify their toolchain but a file on disk can change what the default toolchain is.

[sdispater]

@epage

The created virtualenv will use the Python executable for which poetry has been installed.
on https://poetry.eustace.io/docs/basic-usage/

This is no longer true (I forgot to update this part of the documentation).

As of version 0.12.0, the poetry script will use the currently activated Python version to create the virtualenv.

Basically, you want to use a tool like pyenv to switch between Python versions, or use a project-specific .python-version so that pyenv knows which Python version you want for your project.

Making Poetry a Python versions manager is not a direction I want to go in since there are tools that do it already.

[epage]

Nice! Looks like that solves the underlying requests which is fine than its different than the suggestion.

So it seems that this is now just down to documentation updates?

[epage]

Drat, doesn't look like pyenv supports Windows.

[sdispater]

I haven't used it but maybe https://docs.python.org/3/using/windows.html#python-launcher-for-windows can help here.

[mmerickel]

I'm experiencing issues here as well when trying to bootstrap a project that requires a different version of python than that which I used to install poetry.

1. poetry should be checking that the running python is compatible with the python= requirements in pyproject.toml and raising an exception if not. This should prevent you from accidentally doing a poetry install using python 2.7 when you have python = 3.6 in your pyproject.toml.

2. poetry really needs to support some sort of --python <PATH> on the commands otherwise poetry's virtualenv creation is semi-worthless afaict. The user interface around this is of course a bit wonky because you don't want the user to have to specify --python $HOME/.pyenv/versions/3.6.6/bin/python for every single command, so poetry needs some memory here. I think pipenv actually does a pretty good job at this by creating a virtualenv named after the hash of the path to the folder. Thus if you find a virtualenv at $cache/$hash_of_path then it should use that one with whatever python that one contains. This means it's remembered the first time you run poetry install --python $home/.pyenv/versions/3.6.6/bin/python. If a different --python is specified there's a open question what to do but probably deleting the old venv and creating a new one.

Another approach would be to let the user create the virtualenv themselves. I know this is supported via the in-project setting but I think the defaults are weird around it right now. Ideally this would be a global opt-out setting where poetry automatically detects the folder by default and uses it. Right now the setting means poetry will always use .venv if turned on instead of letting me select it on a per-project basis.

[sdispater]

@mmerickel

I'm experiencing issues here as well when trying to bootstrap a project that requires a different version of python than that which I used to install poetry.

That's why you should now use the recommended installer which will make Poetry pick up the currently activated Python version. That makes it particularly handy when coupled with pyenv.

poetry should be checking that the running python is compatible with the python= requirements in pyproject.toml and raising an exception if not. This should prevent you from accidentally doing a poetry install using python 2.7 when you have python = 3.6 in your pyproject.toml.

That's planned actually since that's something that has bitten users before.

poetry really needs to support some sort of --python on the commands otherwise poetry's virtualenv creation is semi-worthless afaict.

I have said it before but I have no plan to make Poetry a Python version manager. pyenv exists and does the job really well and if you use the recommended installer pyenv local (or pyenv shell for that matter) does the same thing as a potential --python option.

Right now the setting means poetry will always use .venv if turned on instead of letting me select it on a per-project basis.

Just so you know, I plan on adding support for per-project settings via the config command.

[jaswilli]

I assumed it was using the version of python I ran get-poetry with. I tried again with python3 and still no luck.

Yes, this change just bit me on my build servers (Ubuntu 18.04) where I have always installed poetry via curl -sSL https://raw.githubusercontent.com/sdispater/poetry/master/get-poetry.py | python3 in order to use py3.

Are you saying that going forward that absolutely will not work and I will need to bootstrap pyenv on my build workers in order to use poetry?

[sdispater]

@jaswilli Not necessarily. If your default python points to Python 3, Poetry will pick it up and there is nothing you have to do. However, if your default Python does not point to Python 3 you have to switch to Python 3 for Poetry to use it.

[mmerickel]

I have said it before but I have no plan to make Poetry a Python version manager. pyenv exists and does the job really well and if you use the recommended installer pyenv local (or pyenv shell for that matter) does the same thing as a potential --python option.

@sdispater I 100% agree about avoiding integrating any sort of pyenv support into poetry. The --python flag is an escape hatch to avoid doing that, by allowing the user to specify which python to use without making any assumptions.

That's why you should now use the recommended installer which will make Poetry pick up the currently activated Python version. That makes it particularly handy when coupled with pyenv.

Poetry is shipped on PyPI and as such supports being installed into a virtualenv. It's an official installation method. Decoupling poetry from the version of python it's running on should really be supported.

I don't have a habit of switching the default python in my shell and I consider it a bad practice. My "python" command in my shell always points at the system python. This is why things like poetry run are so great is that it doesn't change that... it just temporarily activates a virtualenv for the duration of the command. I'm hoping that poetry can do a better job supporting that for the poetry install command as well, as it can activate the virtualenv defined and run pip inside of it to install the packages. The actual work that poetry needs to do inside the user's virtualenv is fairly minimal and poetry is already doing it in a subprocess by shelling out to pip, the rest of the logic can be done on whatever python that poetry is running on.

[rnwolf]

These steps work for me.
I am using:

• Ubuntu 18.04,
• with pyenv
  • and pyenv-virtualenv https://github.com/pyenv/pyenv-virtualenv. This is configured to automatically activate/deactivate python on entry/exit to directory.
• poetry installed via pipsi (This is using python 3.6.6)

I want an environment with python 3.7.0

# List all available python versions
pyenv install -l
# Install release
pyenv install 3.7.0
# Create working directory
mkdir myapp-project
cd myapp-project
pyenv virtualenvs
pyenv local 3.7.0

# Now we create a virtual env based on the version of python specified above.
# The following will also install all of the default packages as specified in ~/.pyenv/default-packages
# These default packages may be used to install useful commands inside of the project, such as black for example.
pyenv virtualenv myapp
pyenv local myapp
poetry new --src myapp
cd myapp
echo 'Python in virtual environment should be: '$HOME'/.pyenv/versions/myapp/bin'
sed -i 's/^python\ =\ \".*/python\ =\ \"3.7.0\"/' pyproject.toml
poetry install

As I type this out it occurs to me that I could most probably just add poetry to ~/.pyenv/default-packages that would mean I don't need to install via pipsi.

[butla]

I'm just evaluating poetry today and for my normal usecases being able to easily work with a couple of Python versions without shuffling some global state around is absolutely crucial.

I often worked with people who aren't experts on Python and its wacky (even though it performs very well, in my opinion) way of isolating dependencies and runtime versions with virtualenvs. So I had to explain that, and debug and set it up on people's machines, and on CI servers. But it was doable. Pyenv would complicate that, I'm affraid.

On Ubuntu machines we'd just install whatever Pythons we need from deadsnakes, on Macs that'd be something from Homebrew. With Docker images we just pick the version of python or python-alpine we need, but the command in them is often python3 not python. Then a project would have a tox.ini with a specific version of Python (only one in most cases). Anyone could just pull the project, run tox, activate the virtualenv and work, so that was very nice.

And we could maintain some legacy projects in 2.7, stable stuff in 3.6, and evaluating 3.7.

@sdispater If what I'm talking about is outside of the scope of this project, then maybe we can create some disclaimer in the docs saying what workflows this project isn't for? Cause for me "Python dependency management and packaging made easy" is a bit of a false advertising right now :)

But I'll be sad if I can't use poetry in the future, cause it seems way well thought through and performant than Pipenv, which I use right now in a few projects, while disliking its confusing and non-standard CLI.