pip 9.0.1 failed to install

[Xiangyu-C]

I find only pip 20.2 works to install this package otherwise it will throw no correct version error. My company build environment only specifies pip==9.0.1 (which I don't know why that is). Is there a chance for this to work for lower pip versions? Thanks.

[jonmmease]

Hi @Xiangyu-C,

The issue is that the kaleido package uses the manylinux2014 platform tag which is supported in pip >= 19.3. Using this platform tag tells pip that this package is compatible with Centos 7+/Ubuntu 14.04+.

You might be able to update pip in your build script by adding a call to

$ python -m pip install -U pip

Before packages are installed with pip.

If you know you're running on a compatible operating system, and the only issue is that you have an old version of pip that you can't update, here's a workaround you can use:

• Download the linux wheel from PyPI (https://pypi.org/project/kaleido/#files) or from a GitHub Release (https://github.com/plotly/Kaleido/releases). For version 0.0.3, this is named kaleido-0.0.3-py2.py3-none-manylinux2014_x86_64.whl.
• Rename the manylinux2014 part to manylinux1. e.g. kaleido-0.0.3-py2.py3-none-manylinux1_x86_64.whl.
• pip install from the renamed file.

[Xiangyu-C]

Hi Jon,
Thanks for the response. I wish I had much control over the build envrionment (running Centos 7) but I don't. We don't have the priviledge to run any custom command in the docker build step to pull the packages. Also we can not put custom packages in our aritfactory because everything is automated and scanned from pypi. I will try to negotiate with our DevOps team to see if they will temporarily allow me to run an update on pip if there is nothing that can be done on the kaleido side.

[jonmmease]

I will try to negotiate with our DevOps team to see if they will temporarily allow me to run an update on pip if there is nothing that can be done on the kaleido side.

Hi @Xiangyu-C, I understand the predicament and I know you're not the only one on this situation. I'm not sure the best way forward here. We could publish packages to PyPI under an older platform tag (manylinux1 or manylinux2010), and this would make it possible to install kaleido with older version of pip, but this would falsely advertise the compatibility of the package.

I will try to negotiate with our DevOps team to see if they will temporarily allow me to run an update on pip

FWIW, while it's not super common yet, there are other packages on PyPI that are only available under the manylinux2014 platform tag (e.g. https://pypi.org/project/PyQtWebEngine/#files) so there are other situations were you may run into this limitation as well.

[jonmmease]

After some more thought and discussion, I think I'm comfortable changing the platform tag from manylinux2014 to manylinux1 after all.

Rationale:

• Since the native portion of kaleido is run as a subprocess, and not loaded into the main Python interpreter process, the installation of Kaleido on a pre-manylinux2014 operating system won't introduce binary incompatibility problems with the Python interpreter or other installed libraries.
• The code that is executed by the Python interpreter is pure Python, and is compatible with pre-manylinux2014 operating systems.
• Errors that occur while launching the native kaleido subprocess are already trapped and displayed to the user. Attempting to launch this subprocess on a pre-manylinux2014 operating system will produce a controlled Python exception, and won't crash the Python interpreter. Here is what this looks like today on Centos 7.

[root@17a3f914da92 /]# python3
Python 3.4.10 (default, Oct  4 2019, 19:39:58) 
[GCC 4.4.7 20120313 (Red Hat 4.4.7-23)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import plotly.graph_objects as go
>>> from kaleido.scopes.plotly import PlotlyScope
>>> scope = PlotlyScope()
>>> scope.transform(go.Figure())
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.4/site-packages/kaleido/scopes/plotly.py", line 103, in transform
    figure, format=format, width=width, height=height, scale=scale
  File "/usr/lib/python3.4/site-packages/kaleido/scopes/base.py", line 201, in _perform_transform
    self._ensure_kaleido()
  File "/usr/lib/python3.4/site-packages/kaleido/scopes/base.py", line 120, in _ensure_kaleido
    raise ValueError(message)
ValueError: Failed to start Kaleido subprocess. Error stream:

./bin/kaleido: /lib64/libc.so.6: version `GLIBC_2.14' not found (required by ./bin/kaleido)
./bin/kaleido: /lib64/libc.so.6: version `GLIBC_2.15' not found (required by ./bin/kaleido)
./bin/kaleido: /lib64/libc.so.6: version `GLIBC_2.16' not found (required by ./bin/kaleido)
./bin/kaleido: /lib64/libc.so.6: version `GLIBC_2.14' not found (required by /usr/lib/python3.4/site-packages/kaleido/executable/lib/libnss3.so)
./bin/kaleido: /lib64/libc.so.6: version `GLIBC_2.14' not found (required by /usr/lib/python3.4/site-packages/kaleido/executable/lib/libnssutil3.so)
./bin/kaleido: /lib64/libc.so.6: version `GLIBC_2.14' not found (required by /usr/lib/python3.4/site-packages/kaleido/executable/lib/libnspr4.so)
./bin/kaleido: /lib64/libc.so.6: version `GLIBC_2.15' not found (required by /usr/lib/python3.4/site-packages/kaleido/executable/lib/libnspr4.so)
./bin/kaleido: /lib64/libc.so.6: version `GLIBC_2.17' not found (required by /usr/lib/python3.4/site-packages/kaleido/executable/lib/libnspr4.so)
./bin/kaleido: /lib64/libc.so.6: version `GLIBC_2.14' not found (required by /usr/lib/python3.4/site-packages/kaleido/executable/lib/libexpat.so.1)
./bin/kaleido: /lib64/libc.so.6: version `GLIBC_2.14' not found (required by /usr/lib/python3.4/site-packages/kaleido/executable/lib/libplc4.so)
./bin/kaleido: /lib64/libc.so.6: version `GLIBC_2.14' not found (required by /usr/lib/python3.4/site-packages/kaleido/executable/lib/libplds4.so)

We can work on making this error message more informative, but overall I'm comfortable with this approach.

[jonmmease]

I just added a one-off manylinux1 package for the existing version 0.0.3 (https://pypi.org/project/kaleido/0.0.3/#files), and I tested installing and using it on Centos 7, using the pip version 9 provided by yum install python3-pip.

I also updated the build scripts to do this automatically from here on (a7cf6eb).

Thanks for bringing this up @Xiangyu-C and @WEGFan. I think this should take care of the installation issues you were hitting, but let me know if it doesn't!

[Xiangyu-C]

Hi Jon, thanks very much for this change. I will test and let you know.

Edit: Tested as of today 8/10/20 and it worked in our build environment using pip 9.0.1. Thanks.

[mingwandroid]

You should not abuse ABI tags like this, you should build your software correctly instead.

[jonmmease]

I don't disagree, but I don't know of an alternative that won't prevent users of stock Centos 7 from installing the package. Do you have a suggestion?

This isn't like the (past?) tensorflow situation because installing and importing kaleido will never interfere with other packages on the system/environment because the compiled portion of package it is never loaded into the Python process, so the only user-facing downside that I'm aware of is that it will turn an installation error into a runtime error for Centos 6 and earlier.

[nicolaskruchten]

It's a bit naughty but I think the tradeoff is a good one: users of recent OSes and older pips can install and use it, and users of older OSes think they can install it and get a safe error.

[jonmmease]

and users of older OSes think they can install it and get a safe error.

Also of note, the alternative to the safe runtime error is an install-time pip error that's not very helpful.

Could not find a version that satisfies the requirement kaleido (from versions: none)

We don't do this yet, but we could/should update the runtime error to be helpful and explain the whole compatibility situation.

[michibinder]

Hello :)

I got the above error message "ValueError: Failed to start Kaleido subprocess. Error stream: ...", but running this in my dedicated env

'python -m pip install -U pip'

returns

'Requirement already satisfied: pip in /mnt/lustre01/pf/b/b309199/kernels/plotly-kernel/lib/python3.8/site-packages (21.2.4)'

so I guess with pip 21.2.4 my problem is not the manylinux2014 package, but the operating system of the server I want to use Plotly + kaleido with. I don't have sudo rights. Is there anything I could do to make kaleido work or is just not supported in this case?

Thanks a lot for your help!

Michi