Exception thrown when output is not configured as utf-8

[DonBraulio]

Describe the bug
Running a program with a rich progressbar inside a docker container, I got the following exception:

  File "/usr/local/lib/python3.6/dist-packages/rich/progress.py", line 150, in track
    sequence, total=total, description=description, update_period=update_period
  File "/usr/local/lib/python3.6/dist-packages/rich/progress.py", line 667, in track
    with self:
  File "/usr/local/lib/python3.6/dist-packages/rich/progress.py", line 627, in __enter__
    self.start()
  File "/usr/local/lib/python3.6/dist-packages/rich/progress.py", line 596, in start
    self.refresh()
  File "/usr/local/lib/python3.6/dist-packages/rich/progress.py", line 812, in refresh
    self.console.print(Control(""))
  File "/usr/local/lib/python3.6/dist-packages/rich/console.py", line 448, in __exit__
    self._exit_buffer()
  File "/usr/local/lib/python3.6/dist-packages/rich/console.py", line 426, in _exit_buffer
    self._check_buffer()
  File "/usr/local/lib/python3.6/dist-packages/rich/console.py", line 993, in _check_buffer
    self.file.write(text)
UnicodeEncodeError: 'ascii' codec can't encode characters in position 16-55: ordinal not in range(128)

The problem can be easily solved in my case by setting the encoding properly:

export PYTHONIOENCODING=utf8

After that, the script runs perfectly fine with the beautiful rich progressbar.

However, setting this env var in some environments might not be possible or have other undesired consequences. It would be great if there was some kind of failback mode, or a WARNING with a suggestion to set this var, instead of failing without further information.

To Reproduce
Use the README progressbar example, let's call this file progress.py:

from rich.progress import track
for step in track(range(100)):
    pass

Run it with ascii encoding vs. utf8 and check the difference:

PYTHONIOENCODING=ascii python progress.py  # Exception thrown
PYTHONIOENCODING=utf8 python progress.py  # Works

Platform

• rich==5.0.0
• Probably works on any python3 installation. Reproduced on Python 3.6.10 on OSX Catalina, and Python 3.6.9 on Ubuntu Linux native and the same version on Linux for tegra (ARM) docker image.

[hedyhli]

I have also gotten this exception before when doing this on a raspberry pi, same error message. But what I did was I used "string".encode('utf-8')

[willmcgugan]

This has come up a few times and I've advised set PYTHONIOENCODING=utf8 which invariably fixes it.

My understanding is that when Python doesn't have a LANG env var set, it will default to ascii encoding. But Rich can't know if the encoding is a result of misconfiguration or intentional. And you'll get the same encoding error with the builtin print.

A fallback is possible for many things, ascii tables, progress bars etc. But there's not much I can do if the user prints CJK or emoji. Given that the problem is a result of a misconfiguration I'm a little reluctant to spend the effort.

A warning is a good idea, and I'll consider that. But I'm also thinking about wrapping that exception, and re-raising it with a helpful message about encodings.

[joaqo]

Hi, I am substituting click for rich in an open source library, and hit this issue when running on docker.

What do you think is the best way to catch this exception so we can direct the user on how to solve it, or even avoid it altogether? I would have complete control over what is printed on the screen or how the code runs, I just want to make the transition from click to rich as easy as possible on users using docker.

[willmcgugan]

Catching the exception will be easy. There is only a single line in Console that actually writes to stdout. I can catch the exception there and re-raise it with some helpful info.

There's not much I can do to avoid it entirely that doesn't potentially break something else.

[willmcgugan]

This is what happens if you get a unicode error now:

UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-75: ordinal not in range(128)
*** You may need to add PYTHONIOENCODING=utf-8 to your environment ***

Hopefully that will save time for developers running Rich code in docker / CI etc.

Still considering other options for this issue...

[joaqo]

That looks awesome, more than good enough for my case. Thanks!

[DonBraulio]

This is great @willmcgugan ! I'll take the chance to thank you for the awesome work on this library 🙌 🙌 🙌
Feel free to close this issue when you want to.