[Packaging] When installed with RPM, run python3.6 if available

[jiasli]

Fix #18222, #15668, #13892, #13612, #13017, #11819

Symptom

When run on fedora:34, Azure CLI fails with:

$ az connectedk8s connect
ImportError: cannot import name '_psutil_linux' from partially initialized module 'psutil' (most likely due to a circular import) (/usr/lib64/az/lib/python3.6/site-packages/psutil/__init__.py)

Cause

The build pipeline uses centos:centos7

azure-cli/scripts/release/rpm/Dockerfile.centos

Line 1 in f2855b8

ARG tag=centos7

to build the RPM package, which has Python 3.6 preinstalled.

Even though azure-cli.spec only specifies python3 (not python3.6):

azure-cli/scripts/release/rpm/azure-cli.spec

Line 13 in 753a11c

%define python_cmd python3

azure-cli/scripts/release/rpm/azure-cli.spec

Line 28 in 753a11c

Requires: %{python_cmd}

due to the preinstalled Python 3.6 on centos:centos7, libpython3.6m.so.1.0()(64bit) is added to the RPM package's requirement:

# dnf repoquery --requires azure-cli-2.26.1
...
libpython3.6m.so.1.0()(64bit)
python3

# dnf repoquery --whatprovides "libpython3.6m.so.1.0()(64bit)"
Last metadata expiration check: 0:05:45 ago on Tue Aug  3 08:55:51 2021.
python3.6-0:3.6.13-2.fc34.x86_64
python3.6-0:3.6.14-1.fc34.x86_64

Therefore, when installing Azure CLI with

dnf install azure-cli

Python 3.6 will be installed.

However, on new Linux distributions like fedora:34, python3 by default points to python3.9.

# command -v python3
/usr/bin/python3

# stat /usr/bin/python3
  File: /usr/bin/python3 -> python3.9

The generated entry script /usr/bin/az in the RPM package is

# cat /usr/bin/az
#!/usr/bin/env bash
bin_dir=`cd "$(dirname "$BASH_SOURCE[0]")"; pwd`
AZ_INSTALLER=RPM PYTHONPATH="$bin_dir"/../lib64/az/lib/python3.6/site-packages python3 -sm azure.cli "$@"

When run on fedora:34, python3.9 is executed with python3.6-compatible so files, leading to failure.

Change

The availability of python3.6 varies on different distributions:

• fedora:34: all python3.6, python3.9 and python3 (links to python3.9) are available
• centos:centos7: only python3 (which is 3.6) is available

This PR dynamically decides which Python to run (python_cmd) by checking if python3.6 is available, which is the same version of Python used to build the RPM. If python3.6 exists, use it, otherwise use python3.

The entry script /usr/bin/az becomes

# cat /usr/bin/az
#!/usr/bin/env bash
bin_dir=`cd "$(dirname "$BASH_SOURCE[0]")"; pwd`
python_cmd=python3
if command -v python3.6 &>/dev/null; then python_cmd=python3.6; fi
AZ_INSTALLER=RPM PYTHONPATH="$bin_dir"/../lib64/az/lib/python3.6/site-packages $python_cmd -sm azure.cli "$@"

Notes

As Python 3.6 will be deprecated by 2021-12-23 (https://www.python.org/downloads/), we need to bump the build image to newer versions so that the RPM is build with newer versions of Python.

[yonzhan]

Packaging