Restore support for newlines in Summary

[kiorky]

setuptools version

>= 59

Python version

3.8

OS

linux

Additional environment information

No response

Description

builds failing with

 File "/code/venv/lib/python3.8/site-packages/setuptools/dist.py", line 151, in single_line
   raise ValueError('Newlines are not allowed')

Expected behavior

Warning or nothing, just escape silently \n for oneline metadata fields

Lot of already released eggs wont ever comply to this policy, but we need to install them at the version they are pinned in our builds for the sake of reproducibility.

How to Reproduce

pip install django-hijack==2.1.10

Output

  ERROR: Command errored out with exit status 1:
   command: venv/bugpip/bin/python -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-10qe4uxx/django-hijack_c7b8d72ff0ad41998610fe76e5822048/setup.py'"'"'; __file__='"'"'/tmp/pip-install-10qe4uxx/django-hijack_c7b8d72ff0ad41998610fe76e5822048/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-awfzhldp
       cwd: /tmp/pip-install-10qe4uxx/django-hijack_c7b8d72ff0ad41998610fe76e5822048/
  Complete output (26 lines):
  running egg_info
  creating /tmp/pip-pip-egg-info-awfzhldp/django_hijack.egg-info
  writing /tmp/pip-pip-egg-info-awfzhldp/django_hijack.egg-info/PKG-INFO
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
    File "/tmp/pip-install-10qe4uxx/django-hijack_c7b8d72ff0ad41998610fe76e5822048/setup.py", line 44, in <module>
      setup(name="django-hijack",
    File "venv/bugpip/lib/python3.8/site-packages/setuptools/__init__.py", line 153, in setup
      return distutils.core.setup(**attrs)
    File "/usr/lib/python3.8/distutils/core.py", line 148, in setup
      dist.run_commands()
    File "/usr/lib/python3.8/distutils/dist.py", line 966, in run_commands
      self.run_command(cmd)
    File "/usr/lib/python3.8/distutils/dist.py", line 985, in run_command
      cmd_obj.run()
    File "venv/bugpip/lib/python3.8/site-packages/setuptools/command/egg_info.py", line 292, in run
      writer(self, ep.name, os.path.join(self.egg_info, ep.name))
    File "venv/bugpip/lib/python3.8/site-packages/setuptools/command/egg_info.py", line 656, in write_pkg_info
      metadata.write_pkg_info(cmd.egg_info)
    File "/usr/lib/python3.8/distutils/dist.py", line 1117, in write_pkg_info
      self.write_pkg_file(pkg_info)
    File "venv/bugpip/lib/python3.8/site-packages/setuptools/dist.py", line 167, in write_pkg_file
      write_field('Summary', single_line(self.get_description()))
    File "venv/bugpip/lib/python3.8/site-packages/setuptools/dist.py", line 151, in single_line
      raise ValueError('Newlines are not allowed')
  ValueError: Newlines are not allowed

Code of Conduct

• I agree to follow the PSF Code of Conduct

[kiorky]

cf #1390 (comment)

[mriedem]

Came here to report the same. Last week setuptools-58.5.3 was OK but now with setuptools-59.1.1 we're failing to install this package because of the newline in the description:

https://github.com/qiskit-community/qiskit-textbook/blob/stable/qiskit-textbook-src/setup.py#L8

Was this a regression from #1390?

[mriedem]

Was this a regression from #1390?

Oh I guess this is intentional in v59.0.0:

https://github.com/pypa/setuptools/blob/main/CHANGES.rst#v5900

Per #2870.

[jaraco]

As mentioned in #1390, this change was introduced intentionally in order to prevent these invalid inputs.

It sounds like this issue only affects older packages installed with the latest Setuptools. These environments are pinning their requirements but not pinning Setuptools. As a result, the maintainers of these environments are asking Setuptools to carry the burden of the legacy introduced by pinning their requirements.

I'd prefer for these projects to advance their dependencies to versions that no longer have the defective usage. But maybe there are other workarounds as well.

Why can't these environments work around the issue by pinning Setuptools as well? Or find another workaround such as installing from wheels (built from older Setuptools)?

For example, I was able to build the affected package thus:

draft $ pip-run -q 'setuptools<59' wheel -- -m pip wheel --no-deps django-hijack==2.1
Collecting django-hijack==2.1
  Using cached django-hijack-2.1.0.tar.gz (16 kB)
  Preparing metadata (setup.py) ... done
Building wheels for collected packages: django-hijack
  Building wheel for django-hijack (setup.py) ... done
  Created wheel for django-hijack: filename=django_hijack-2.1.0-py3-none-any.whl size=24148 sha256=51049e0c5eadb29e31a8c393db68891b61376eeef8311920048a7d3fd97bb838
  Stored in directory: /Users/jaraco/Library/Caches/pip/wheels/8b/57/f0/2cecaf262d2f06211ab69602ac2cd5d64cb093068cb7ee1c65
Successfully built django-hijack
draft $ ls
django_hijack-2.1.0-py3-none-any.whl

I've tried to think of ways Setuptools could support these legacy cases but reject new cases, but I haven't been able to come up with a good approach. I'd really like to avoid leaving Setuptools indefinitely dependent on the legacy of older packages.

[kiorky]

It sounds like this issue only affects older packages installed with the latest Setuptools. These environments are pinning their requirements but not pinning Setuptools. As a result, the maintainers of these environments are asking Setuptools to carry the burden of the legacy introduced by pinning their requirements.

Because every project has it's own pace to release and needs stability and depinning/upgrading those requirements imply extra maintenance and tests time and in case of paid work: costs which are not justified here.
Version pinning is essential to ensure your build is reproducible.
Breaking a build just because a metadata in a dependence, which may not be direct misses a \n which was not even mandatory at the time of the release of that dependence is a total non-sense.

I agree that quality should be enforced on new uploaded releases but installation of older packages should and must be still possible wiithout any change.

Seriously, guys, stripping eternally \n is not a big deal, and if we want to ensure packages now strictly validate, we just should ensure offending dists could not be uploaded.

Another (bad solution i think as it will break integrity of uploaded dists and break custom hash verifications) and also do not work with alternative mirrors where artefacts wont be patched is to patch already uploaded artefacts to make them comply with new code.

[kiorky]

Not to also mention that newer releases of dependences are maybe not compatible with every stack and every stack may not be that easily upgradable, just for an \n.

Seriously, reconsider that non sense breaking change...

[jaraco]

I've created a plugin, setuptools_hacks.bypass_summary_newline that provides a workaround for the issue. Users encountering this issue can install this plugin alongside setuptools 59+ to bypass the error by stripping everything from the first newline. Demo:

~ $ pip-run -q setuptools wheel setuptools_hacks.bypass_summary_newline -- -m pip-run --no-deps --no-binary :all: django-hijack==2.1.10 -- -c pass
Collecting django-hijack==2.1.10
  Using cached django-hijack-2.1.10.tar.gz (20 kB)
  Preparing metadata (setup.py) ... done
Skipping wheel build for django-hijack, due to binaries being disabled for it.
Installing collected packages: django-hijack
    Running setup.py install for django-hijack ... done
Successfully installed django-hijack-2.1.10

This approach allows these select environments dependent on the broken packages to work around the breakage while also retaining the check for new packages. It also clearly pushes the legacy maintenance to the environment maintainers instead of Setuptools itself.

[kiorky]

Sorry but no, its not AT ALL an acceptable solution, please reopen.

I don't want to figure out to push a new plugin in every build pipeline i have.

What we need is build stability and reproducibility.

Futherthemore, no one want to put a "legacy maintenance" on setuptools folks, we just want to have already and running fine build systems not to break for a \n detail.