Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
63ff6c9
fix .dtype property
TomNicholas Jul 15, 2025
e765e13
use zarr data types in create_array_v3_metadata
TomNicholas Jul 15, 2025
c60795a
change expected repr
TomNicholas Jul 15, 2025
b2d9549
fix test for checking dtypes are the same
TomNicholas Jul 15, 2025
a05d9e8
fix icechunk tests
TomNicholas Jul 15, 2025
f172930
fix kerchunk tests
TomNicholas Jul 15, 2025
02dfa56
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 15, 2025
7dfeabe
Merge branch 'develop' into zarr-data-types-refactor-compat2
TomNicholas Jul 15, 2025
b42022f
fix combine test
TomNicholas Jul 15, 2025
7bb6e6f
fix conversion of v3 to v2 metadata
TomNicholas Jul 16, 2025
744fd49
write function to normalize kerchunk references into true json
TomNicholas Jul 16, 2025
35f94b5
use the new function in our tests
TomNicholas Jul 16, 2025
9663d05
remove outdated imports
TomNicholas Jul 16, 2025
e1e54b3
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 16, 2025
915e4c3
Merge branch 'develop' into kerchunk_to_true_json
TomNicholas Jul 16, 2025
ae3f871
Merge branch 'kerchunk_to_true_json' into zarr-data-types-refactor-co…
TomNicholas Jul 16, 2025
8548086
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 16, 2025
01baf83
missed a coersion
TomNicholas Jul 16, 2025
ad0be5c
Merge branch 'kerchunk_to_true_json' of https://github.com/TomNichola…
TomNicholas Jul 16, 2025
de9c8fd
Merge branch 'kerchunk_to_true_json' into zarr-data-types-refactor-co…
TomNicholas Jul 16, 2025
35cfddf
fix dmrpp test
TomNicholas Jul 16, 2025
813ee16
Merge branch 'zarr-data-types-refactor-compat2' of https://github.com…
TomNicholas Jul 16, 2025
f0739c1
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 16, 2025
77493a6
Merge branch 'develop' into zarr-data-types-refactor-compat2
TomNicholas Jul 16, 2025
31941fb
require latest zarr
TomNicholas Jul 16, 2025
64a513b
Merge branch 'zarr-data-types-refactor-compat2' of https://github.com…
TomNicholas Jul 16, 2025
a17884d
update minimum version in pixi
TomNicholas Jul 16, 2025
93f7758
fix array return type and add test
TomNicholas Jul 17, 2025
407f208
change expected group repr dtype str
TomNicholas Jul 17, 2025
09d801d
fix metadata comparison by just using updated version of ArrayV3Metad…
TomNicholas Jul 17, 2025
9117116
don't need to add .nbytes to ManifestArray
TomNicholas Jul 17, 2025
3ebb731
remove rogue print statements
TomNicholas Jul 17, 2025
bf104cb
try fixing zarr data type error in icechunk writer in CI
TomNicholas Jul 17, 2025
b1961b6
remove xfail now that datetime dtypes are supported in zarr
TomNicholas Jul 17, 2025
b0bddf2
add xfail for one test case
TomNicholas Jul 17, 2025
fc1bd34
add note about now supporting big-endian
TomNicholas Jul 17, 2025
925ffec
remove unnecessary conversion of dtype
TomNicholas Jul 17, 2025
0d7ad60
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 17, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fix metadata comparison by just using updated version of ArrayV3Metad…
…ata.to_dict()
  • Loading branch information
TomNicholas committed Jul 17, 2025
commit 09d801d02314dd4b994d156fd2c1f8d094653f25
2 changes: 1 addition & 1 deletion virtualizarr/manifests/array.py
Original file line number Diff line number Diff line change
Expand Up @@ -189,7 +189,7 @@ def __eq__( # type: ignore[override]
if self.shape != other.shape:
raise NotImplementedError("Unsure how to handle broadcasting like this")

if not utils.metadata_identical(self.metadata, other.metadata):
if not self.metadata.to_dict() == other.metadata.to_dict():
Copy link
Member Author

@TomNicholas TomNicholas Jul 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment explains why were are now able to simplify this: zarr-developers/zarr-python#2930 (comment)

return np.full(shape=self.shape, fill_value=False, dtype=np.dtype(bool))
else:
if self.manifest == other.manifest:
Expand Down
18 changes: 0 additions & 18 deletions virtualizarr/manifests/utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -157,24 +157,6 @@ def check_same_shapes(shapes: list[tuple[int, ...]]) -> None:
)


# TODO remove this once https://github.com/zarr-developers/zarr-python/issues/2929 is solved upstream
def metadata_identical(metadata1: ArrayV3Metadata, metadata2: ArrayV3Metadata) -> bool:
"""Checks the metadata of two zarr arrays are identical, including special treatment for NaN fill_values."""
metadata_dict1 = metadata1.to_dict()
metadata_dict2 = metadata2.to_dict()

# fill_value is a special case because numpy NaNs cannot be compared using __eq__, see https://stackoverflow.com/a/10059796
fill_value1 = metadata_dict1.pop("fill_value")
fill_value2 = metadata_dict2.pop("fill_value")
if np.isnan(fill_value1) and np.isnan(fill_value2): # type: ignore[arg-type]
fill_values_equal = fill_value1.dtype == fill_value2.dtype # type: ignore[union-attr]
else:
fill_values_equal = fill_value1 == fill_value2

# everything else in ArrayV3Metadata is a string, Enum, or Dataclass
return fill_values_equal and metadata_dict1 == metadata_dict2


def _remove_element_at_position(t: tuple[int, ...], pos: int) -> tuple[int, ...]:
new_l = list(t)
new_l.pop(pos)
Expand Down
Loading