[Bug] Python files in home directories open extremely slowly

[tofuya]

[Bug] Python files in home directories open extremely slowly

Reproduction guide

1. Ensure your home directory is recognized as a Projectile project by creating a project file marker under ~
   (e.g., .git or WORKSPACE):

   mkdir ~/.git

2. Start Emacs in daemon mode:

   emacs --daemon

3. Run:

   emacsclient -nw /tmp/test.py

   → File opens in 3–4 seconds.

4. Run:

   emacsclient -nw ~/test.py

   → File takes 3–4 minutes to open.

---

Observed behaviour

• The slowness occurs with both new and existing .py files.
• Opening .py files inside a directory within the home directory takes 3–4 minutes.
• Opening the same file under /tmp/ is noticeably faster (3–4 seconds).
• Reverting Spacemacs to commit 454593c resolves the slowness.
• Removing the Python layer also resolves the slowness.

---

Expected behaviour

• .py files should open quickly, regardless of their directory location.

---

System Info 💻

• OS: darwin
• Emacs: 30.1
• Spacemacs: 0.999.0
• Spacemacs branch: develop (rev. 899032e)
• Graphic display: nil
• Running in daemon: t
• Distribution: spacemacs
• Editing style: vim
• Completion: helm
• Layers:

(python)

• System configuration features: ACL GMP GNUTLS LCMS2 LIBXML2 MODULES NATIVE_COMP NOTIFY KQUEUE PDUMPER SQLITE3 THREADS TREE_SITTER XIM ZLIB

---

Additional notes

• The slowness occurs during the initialization of Python-related functionality.
• Related commit: 6c95fea
• Related issue: #17141
• Related issue: Home directory incorrectly detected as Bazel project when ~/workspace directory exists on macOS bbatsov/projectile#1961

---

Backtrace

Not available (no error, just extreme slowness).

[phenix-factory]

I have the same issue, and it's since this commit 6c95fea

[sunlin7]

It should related to pet-mode, and pet-mode was turn on default in commit 6c95fea#diff-fadfcb4d8a57cc1bcc20910550487f9ab751692c54d6b21c702ca8c9e606aab0L33, and toggle flag was removed also.

As the summary I listed in #17038, the pet-mode still has issues, does not ready as default yet.

I submitted a PR #17143, wait for the maintainer to approve it.

[phenix-factory]

I feel like history is repeating itself here. Can we do something to prevent this?

[kassick]

I can not reproduce here -- maybe because I use direnv only and do not have pixi/pyenv/conda/mamba installed, so (pet-mode) , while active when opening ~/a.py, gives up early. Or because my home is neither a projectile nor project.el project, (pet-project-root) returns nil ...

Just to point that, given python's 999 ways of managing venvs, plus emacs 1001 customizations, hooks, advices, etc.,
bugs like this may hit hard some users while not even showing up to others ...

[phenix-factory]

Just to point that, given python's 999 ways of managing venvs, plus emacs 1001 customizations, hooks, advices, etc.,
bugs like this may hit hard some users while not even showing up to others ...

I understand that, but there was already an issue with pet-mode before.
It's frustrating to have your tool break every month because someone tried to add pet-mode (again).
Now I also know that it's hard and not everyone knows everything, it's just frustration, Emacs could not open a Python file, it just hangs until I kill it…

[sunlin7]

I feel like history is repeating itself here. Can we do something to prevent this?

Actually the Spacemacs CI failure had indicated there're incompatible changes.

[kassick]

@phenix-factory you probably know that already, but since ~/.emacs.d is a git repo, you can always use the reflog to find the commit before the last pull and then reset --hard to that ...

Regressions are indeed frustrating, and every time I see changes to the python layer I also fear that something's gonna break for me (pytest did break with the latest changes), but being able to easily revert at least allows me to work as usual and later investigate ;)

[wyuenho]

Guys, next time when you've found a bug in pet mode when trying to integrate with spacemacs, do you mind opening an issue on my repo or at least tag me to your issues?

I don't use spacemacs and I've only designed pet to work really well for python files inside a project. I wouldn't have known hey there's another issue here that I can fix in a matter of hours unless I come here to look or you let me know.

The problem for this slowness is because by default pet will scan your "project" for config files and tools to automatically configure all the supported python packages for you. If you aren't inside a project, which I really didn't design pet for, as in this case, it's going to scan the entire home directory for config files. There are settings documented to avoid this, but it's up to the user to configure it as designed if this is a common use case.

As it turns out, since how pet is now abstracted by spacemacs and nobody knows how it works, and it's a trivial fix, I'll fix it in pet. I'll do it tonight.

[phenix-factory]

If you aren't inside a project, which I really didn't design pet for, as in this case, it's going to scan the entire home directory for config files.

What is a "project" in this case? How does pet detect a project?

[wyuenho]

What is a "project" in this case? How does pet detect a project?

Anything that projectile or project.el considers a project.

[kassick]

So the issue is that if projectile thinks your home is a project, pet will be very aggressive in finding stuff in a potentially huge directory, right?

If you have a .git in your home, isn't that enought for projectile add it as a known project?

[wyuenho]

Yes. Also please read https://github.com/wyuenho/emacs-pet/blob/main/doc/PERFORMANCE.rst.

If you don't have fd installed and have pet-find-file-from-project-root-recursively at the end of pet-find-file-functions, file searches in any large directories is bound to be somewhat slow.

[wyuenho]

@tofuya @phenix-factory are you home directories under version control or have any kind of project marker files that'll make projectile think your home directory is a project?