Skip to content

gh-143732: allow dict subclasses to be specialized #148128

Open
kumaraditya303 wants to merge 18 commits intopython:mainfrom
kumaraditya303:jit-dict-opt
Open

gh-143732: allow dict subclasses to be specialized #148128
kumaraditya303 wants to merge 18 commits intopython:mainfrom
kumaraditya303:jit-dict-opt

Conversation

@kumaraditya303
Copy link
Copy Markdown
Contributor

@kumaraditya303 kumaraditya303 commented Apr 5, 2026

This is revival of #132383 which allowed specialization for dict subclasses which do not override __getitem__. This PR also allows specialization for dict subclasses which do not override __setitem__.

This is primarily targeted for defaultdict which is the most performance critical subclass of dict.

@kumaraditya303 kumaraditya303 marked this pull request as ready for review April 5, 2026 18:39
@kumaraditya303 kumaraditya303 added the performance Performance or resource usage label Apr 5, 2026
@kumaraditya303
Copy link
Copy Markdown
Contributor Author

@markshannon Can you take a look at this? I'd like get this in time for the beta release.

Copy link
Copy Markdown
Member

@markshannon markshannon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good overall.
The old uops need to be removed and some assertions may no longer be valid.

Also, take care with recorded types. Only strengthen type information on proven or guards, not recorded information.

Comment thread Python/bytecodes.c
Comment thread Python/bytecodes.c
Comment thread Python/bytecodes.c Outdated
Comment thread Python/optimizer_bytecodes.c
Comment thread Python/optimizer_bytecodes.c
Comment thread Python/bytecodes.c Outdated
Comment thread Python/bytecodes.c Outdated
Comment thread Python/bytecodes.c Outdated
PyObject *dict = PyStackRef_AsPyObjectBorrow(dict_st);

assert(PyAnyDict_CheckExact(dict));
assert(PyAnyDict_Check(dict));
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this necessarily true?
It seems unlikely, not impossible, that for some object Py_TYPE(dict)>tp_as_mapping->mp_ass_subscript == _PyDict_StoreSubscript, but !PyDict_Check(dict)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it is possible, that can only happen if someone intentionally sets the slot which would crash at runtime anyways.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would only crash if the object used a different layout.
I could imagine Cython, or pynanobind implementing their own dict for some reason and reusing the existing code.

@markshannon
Copy link
Copy Markdown
Member

GitHub doen't put comments in order, it seems. The "above" in the comments refers to earlier lines, not earlier comments.

Copy link
Copy Markdown
Member

@markshannon markshannon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still think the assert(PyDict_Check(dict)); asserts should be removed.

Also you need watchers for all mortal types, not just mutable ones.

Comment thread Python/optimizer_bytecodes.c Outdated
Comment thread Python/optimizer_bytecodes.c Outdated
@bedevere-app
Copy link
Copy Markdown

bedevere-app Bot commented Apr 30, 2026

When you're done making the requested changes, leave the comment: I have made the requested changes; please review again.

@kumaraditya303
Copy link
Copy Markdown
Contributor Author

I still think the assert(PyDict_Check(dict)); asserts should be removed.

I removed the assertions now and added watchers for all cases.

I have made the requested changes; please review again

@bedevere-app
Copy link
Copy Markdown

bedevere-app Bot commented Apr 30, 2026

Thanks for making the requested changes!

@markshannon: please review the changes made to this pull request.

@bedevere-app bedevere-app Bot requested a review from markshannon April 30, 2026 17:44
@read-the-docs-community
Copy link
Copy Markdown

read-the-docs-community Bot commented Apr 30, 2026

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants