[Performance] Using rich.console.Console() takes multiple seconds

[zachriggle]

Describe the bug
When using rich.console, the startup time of my application increases greatly.

You can see this easily by using hyperfine to benchmark the application.

To Reproduce
First, brew install hyperfine or install Hyperfine in some other way. It takes 80ms for the Python3 interpreter to start, but 1.6s to run rich.console.Console(). This is an unacceptable performance penalty for command-line utilities, and for simple scripts incurs a 20x performance penalty.

Hyperfine Benchmark Results

$ hyperfine "python3 -c 'import sys; sys.exit(0)'"  "python3 -c 'from rich.console import Console; Console()'"
Benchmark #1: python3 -c 'import sys; sys.exit(0)'
  Time (mean ± σ):      83.3 ms ±   3.6 ms    [User: 61.1 ms, System: 19.9 ms]
  Range (min … max):    79.1 ms …  94.3 ms    35 runs

Benchmark #2: python3 -c 'from rich.console import Console; Console()'
  Time (mean ± σ):      1.629 s ±  0.019 s    [User: 992.5 ms, System: 674.1 ms]
  Range (min … max):    1.598 s …  1.660 s    10 runs

Summary
  'python3 -c 'import sys; sys.exit(0)'' ran
   19.57 ± 0.87 times faster than 'python3 -c 'from rich.console import Console; Console()''

Platform

macOS

$ sw_vers
ProductName: macOS
ProductVersion: 11.3
BuildVersion: 20E5172h

Diagnose
I may ask you to cut and paste the output of the following commands. It may save some time if you do it now.

python3 -m rich.diagnose

$ python3 -m rich.diagnose
╭───────────────────────── <class 'rich.console.Console'> ─────────────────────────╮
│ A high level console interface.                                                  │
│                                                                                  │
│ ╭──────────────────────────────────────────────────────────────────────────────╮ │
│ │ <console width=208 ColorSystem.TRUECOLOR>                                    │ │
│ ╰──────────────────────────────────────────────────────────────────────────────╯ │
│                                                                                  │
│     color_system = 'truecolor'                                                   │
│         encoding = 'utf-8'                                                       │
│             file = <_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'> │
│           height = 47                                                            │
│ is_dumb_terminal = False                                                         │
│   is_interactive = True                                                          │
│       is_jupyter = False                                                         │
│      is_terminal = True                                                          │
│   legacy_windows = False                                                         │
│         no_color = False                                                         │
│          options = ConsoleOptions(                                               │
│                        size=ConsoleDimensions(width=208, height=47),             │
│                        legacy_windows=False,                                     │
│                        min_width=1,                                              │
│                        max_width=208,                                            │
│                        is_terminal=True,                                         │
│                        encoding='utf-8',                                         │
│                        justify=None,                                             │
│                        overflow=None,                                            │
│                        no_wrap=False,                                            │
│                        highlight=None,                                           │
│                        height=None                                               │
│                    )                                                             │
│            quiet = False                                                         │
│           record = False                                                         │
│         safe_box = True                                                          │
│             size = ConsoleDimensions(width=208, height=47)                       │
│        soft_wrap = False                                                         │
│           stderr = False                                                         │
│            style = None                                                          │
│         tab_size = 8                                                             │
│            width = 208                                                           │
╰──────────────────────────────────────────────────────────────────────────────────╯

python3 -m rich._windows

platform="Darwin"
WindowsConsoleFeatures(vt=False, truecolor=False)

pip3 freeze | grep rich

$ pip3 freeze | grep rich
rich==9.13.0

Did I help?

If I was able to resolve your problem, consider sponsoring my work on Rich, or buy me a coffee to say thanks.

[willmcgugan]

Console constructor does next to nothing. What you are timing is importing a 20 or so Python files, which takes 70ms or so on my system.

I'd suggest looking in to why it is slow to import things on your system. Try the following switch to see you are importing something you don't expect.

python -X importtime

[zachriggle]

It looks like the bulk of the time is importing pydoc in rich/pager.py.

rich/pager.py
2:import pydoc
20:    _pager = lambda self, content: pydoc.pager(content)
23:        """Use the same pager used by pydoc."""

python3 -X importtime -c 'from rich.console import Console; Console()'

~ ❯❯❯ python3 -X importtime -c 'from rich.console import Console; Console()'
import time: self [us] | cumulative | imported package
import time:       654 |        654 | _frozen_importlib_external
import time:       724 |        724 |   time
import time:       244 |        968 | zipimport
import time:        66 |         66 |     _codecs
import time:       652 |        718 |   codecs
import time:       620 |        620 |   encodings.aliases
import time:       387 |        387 |   encodings.cp437
import time:      1183 |       2907 | encodings
import time:       282 |        282 | encodings.utf_8
import time:       189 |        189 | _signal
import time:       328 |        328 | encodings.latin_1
import time:        56 |         56 |     _abc
import time:       342 |        397 |   abc
import time:       356 |        752 | io
import time:       132 |        132 |       _stat
import time:       274 |        405 |     stat
import time:      1334 |       1334 |     _collections_abc
import time:       407 |        407 |       genericpath
import time:       682 |       1088 |     posixpath
import time:       807 |       3632 |   os
import time:       775 |        775 |   _sitebuiltins
import time:        94 |         94 |     _locale
import time:       325 |        418 |   _bootlocale
import time:       385 |        385 |   types
import time:       381 |        381 |       warnings
import time:       310 |        691 |     importlib
import time:       273 |        273 |       importlib.machinery
import time:       680 |        952 |     importlib.abc
import time:       100 |        100 |           _operator
import time:      2655 |       2755 |         operator
import time:       267 |        267 |         keyword
import time:       659 |        659 |           _heapq
import time:       291 |        950 |         heapq
import time:       122 |        122 |         itertools
import time:       318 |        318 |         reprlib
import time:       660 |        660 |         _collections
import time:      1038 |       6108 |       collections
import time:        66 |         66 |         _functools
import time:       717 |        782 |       functools
import time:       889 |       7778 |     contextlib
import time:       655 |      10074 |   importlib.util
import time:       976 |        976 |         _hashlib
import time:       519 |        519 |         _blake2
import time:       499 |        499 |         _sha3
import time:       359 |       2352 |       hashlib
import time:       912 |        912 |           enum
import time:       106 |        106 |             _sre
import time:       439 |        439 |               sre_constants
import time:       623 |       1061 |             sre_parse
import time:       400 |       1566 |           sre_compile
import time:       284 |        284 |           copyreg
import time:       764 |       3524 |         re
import time:       293 |        293 |               token
import time:      1310 |       1603 |             tokenize
import time:       267 |       1870 |           linecache
import time:       410 |       2279 |         traceback
import time:       304 |        304 |           _weakrefset
import time:       738 |       1042 |         weakref
import time:       408 |        408 |         collections.abc
import time:        46 |         46 |           _string
import time:       906 |        951 |         string
import time:       724 |        724 |         threading
import time:        47 |         47 |         atexit
import time:      2797 |      11769 |       logging
import time:       268 |        268 |         fnmatch
import time:        77 |         77 |         errno
import time:       470 |        470 |         zlib
import time:       467 |        467 |           _compression
import time:       419 |        419 |           _bz2
import time:       383 |       1268 |         bz2
import time:       456 |        456 |           _lzma
import time:       474 |        929 |         lzma
import time:        56 |         56 |         pwd
import time:       396 |        396 |         grp
import time:       802 |       4264 |       shutil
import time:       887 |        887 |           _socket
import time:      1300 |       2187 |         _ssl
import time:       530 |        530 |             math
import time:       457 |        457 |             select
import time:       613 |       1599 |           selectors
import time:      1613 |       3211 |         socket
import time:       391 |        391 |             _struct
import time:       217 |        607 |           struct
import time:       461 |        461 |           binascii
import time:       341 |       1408 |         base64
import time:      3556 |      10361 |       ssl
import time:       387 |        387 |             _bisect
import time:       230 |        616 |           bisect
import time:       371 |        371 |           _sha512
import time:       392 |        392 |           _random
import time:       599 |       1977 |         random
import time:       467 |       2443 |       tempfile
import time:       207 |        207 |       recertifi.version
import time:       256 |        256 |                 nt
import time:       232 |        232 |                 nt
import time:       225 |        225 |                 nt
import time:       222 |        222 |                 nt
import time:       327 |       1260 |               ntpath
import time:       202 |        202 |                 urllib
import time:      1177 |       1379 |               urllib.parse
import time:       790 |       3428 |             pathlib
import time:     11239 |      11239 |             typing
import time:      3969 |      18636 |           importlib.resources
import time:       494 |      19130 |         certifi.core
import time:       265 |      19394 |       certifi
import time:      6663 |      57448 |     recertifi.implementation
import time:      1398 |      58846 |   recertifi
import time:       371 |        371 |   sitecustomize
import time:       243 |        243 |   usercustomize
import time:     10635 |      85375 | site
import time:      1022 |       1022 |   rich
import time:       427 |        427 |         _opcode
import time:       469 |        895 |       opcode
import time:       838 |       1733 |     dis
import time:      1641 |       3373 |   inspect
import time:      2365 |       2365 |   platform
import time:       334 |        334 |           org
import time:        25 |        359 |         org.python
import time:        20 |        379 |       org.python.core
import time:       360 |        738 |     copy
import time:      5827 |       6565 |   dataclasses
import time:       652 |        652 |     _datetime
import time:       809 |       1460 |   datetime
import time:       486 |        486 |     termios
import time:       312 |        798 |   getpass
import time:       819 |        819 |   typing_extensions
import time:       304 |        304 |   rich.errors
import time:      1769 |       1769 |           colorsys
import time:       324 |        324 |               rich.color_triplet
import time:       288 |        611 |             rich.palette
import time:       220 |        830 |           rich._palettes
import time:       244 |        244 |           rich.terminal_theme
import time:      1021 |       1021 |             signal
import time:       309 |        309 |             msvcrt
import time:       481 |        481 |             _posixsubprocess
import time:       571 |       2380 |           subprocess
import time:      6860 |      12081 |         rich.color
import time:       775 |      12856 |       rich.style
import time:       701 |      13556 |     rich.default_styles
import time:     11025 |      11025 |       configparser
import time:       477 |      11501 |     rich.theme
import time:       270 |      25326 |   rich.themes
import time:      2027 |       2027 |     rich._emoji_codes
import time:     20642 |      22668 |   rich._emoji_replace
import time:       368 |        368 |       rich._loop
import time:       213 |        213 |       rich._pick
import time:       447 |        447 |           rich._cell_widths
import time:       423 |        423 |           rich._lru_cache
import time:       342 |       1211 |         rich.cells
import time:       402 |       1613 |       rich._wrap
import time:      1062 |       1062 |             rich.segment
import time:       559 |       1621 |           rich.jupyter
import time:       258 |        258 |               rich.abc
import time:       374 |        632 |             rich.protocol
import time:       663 |       1294 |           rich.measure
import time:       292 |       3206 |         rich.constrain
import time:       377 |       3582 |       rich.align
import time:       417 |        417 |       rich.containers
import time:       246 |        246 |       rich.control
import time:      1433 |       7870 |     rich.text
import time:       481 |       8351 |   rich._log_render
import time:       392 |        392 |   rich.highlighter
import time:       862 |        862 |   rich.markup
import time:       671 |        671 |       pkgutil
import time:       779 |        779 |       sysconfig
import time:       448 |        448 |       _sysconfigdata__darwin_darwin
import time:       326 |        326 |       _osx_support
import time:   3175579 |    3177801 |     pydoc
import time:       319 |    3178119 |   rich.pager
import time:       657 |        657 |     array
import time:      1946 |       2602 |   rich.pretty
import time:       592 |        592 |       rich.box
import time:       381 |        381 |       rich.padding
import time:       350 |       1322 |     rich.panel
import time:       372 |        372 |       rich._ratio
import time:      1928 |       2300 |     rich.table
import time:       299 |       3919 |   rich.scope
import time:       251 |        251 |   rich.screen
import time:       238 |        238 |   rich.styled
import time:      3248 |    3262676 | rich.console

[willmcgugan]

That's interesting. pydoc is relatively quick here.

Since I can't reproduce it. You might want to profile it with scalene, or other profiler. It could be related to your installation of Python, but let me know if there is something I could do on the Rich side to speed up the import.

One option would be to make the import lazy, so pydoc isn't imported if you don't use the pager.

[willmcgugan]

I'll make the import lazy. Couldn't hurt. But let me know if you figure out why it is slow for you.

[zachriggle]

I've tracked the root cause of this to something company, internal. Normal pydoc loads fast, internal pydoc takes 2-3 seconds. Thanks in any case for making the import lazy, lots of tools that I use make use of Rich, and import pydoc by extension, and I cannot reorder $PATH to get the "fast" Python on my workstation, unfortunately. *Zach Riggle*

…

On Thu, Mar 11, 2021 at 3:37 PM Will McGugan ***@***.***> wrote: Closed #1104 <#1104>. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#1104 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAA3IGFCJRJSUJ5L26PHTCLTDEZ3PANCNFSM4Y2FIYDQ> .