Skip to content
Open
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Tweak msgpack
  • Loading branch information
crusaderky committed Nov 20, 2025
commit 11d63a2b148c445d97da15d9b1d83a181ed07610
1 change: 0 additions & 1 deletion setup.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,6 @@ zip_safe = true
include_package_data = true
python_requires = >=3.9
Copy link
Contributor Author

@crusaderky crusaderky Nov 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed version cap as there is very little that can break anymore with future versions. Future compatibility is now delegated to upstream libraries.

install_requires =
catalogue>=2.0.10,<3
cloudpickle >=3.1.2,<4
msgpack >=1.1,<2
ruamel.yaml >=0.18.16,<1
Expand Down
27 changes: 26 additions & 1 deletion srsly/_msgpack_api.py

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you go into a little more detail than what's in the PR description into what motivated adding _MsgpackExtensions over using the old code?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was wrong. in the opening message. The API

msgpack_encoders.register(name, cb)
msgpack_deecoders.register(name, cb)

was not part of a very old version of msgpack like I thought - it was original in srsly all along.
I had to reimplement it from scratch to make it work with an unpatched msgpack, and to deal with subtle breakages with np.float64 (which is a subclass of builtin float).

Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,22 @@


class _MsgpackExtensions:
"""API for extending msgpack (de)serialization:

srsly.msgpack_encoders.register(name, func)
srsly.msgpack_decoders.register(name, func)

where `name` is a unique ID and `func` is a callable that accepts a single
argument:

- For encoders, the argument is the object to serialize. The callable should
return a new object (typically a dict) or the original object if the callback
does not recognize it.
- For decoders, the argument is the dict to deserialize, as returned by the encoders.
The callable should return a new object or the original dict if the callback
does not recognize it.
"""

__slots__ = ("_ext",)

def __init__(self):
Expand All @@ -25,11 +41,20 @@ def _run(self, obj):
out = func(obj)
if out is not obj:
return out
return obj


class _MsgpackEncoderExtensions(_MsgpackExtensions):
def _run(self, obj):
out = super()._run(obj)
if out is not obj:
return out

# Convert subtypes of base types and tuples to lists.
# Effectively this undoes the strict_types=True option of msgpack.
# This is needed to support np.float64, which is a subclass of builtin float.
# Run this last to allow the user to register their own handlers first.

if isinstance(obj, tuple):
return list(obj)
# Note: bool and memoryview can't be subclassed
Expand All @@ -41,7 +66,7 @@ def _run(self, obj):
return obj


msgpack_encoders = _MsgpackExtensions()
msgpack_encoders = _MsgpackEncoderExtensions()
msgpack_decoders = _MsgpackExtensions()


Expand Down