zarr-developers/zarr-python - CodSpeed

zarr-developers

zarr-python

Blog Docs Changelog

Performance History

Latest Results

perf: build order-coord tuples via .tolist(); document dual representation `morton_order_coords` / `lexicographic_order_coords` built their tuple-of- tuples with a row-by-row `tuple(int(x) for x in row)` comprehension. Using `map(tuple, arr.tolist())` instead does the int conversion in a single C-level call, producing byte-identical native-int tuples ~8-9x faster (~16ms -> ~1.9ms cold build at 32^3). It is a per-shape cached build, so this only speeds the first write to each shard shape, but it is free. Also document in `to_dict_vectorized` why the chunk coordinates are needed in two forms — a numpy array for the vectorized index lookup and hashable tuples for the dict keys — since numpy rows are unhashable and a tuple list can't be used for the vectorized modulo/advanced-indexing. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

d-v-b:perf-sharding-coord-cache

15 hours ago

refactor(indexing): make morton_order_iter the lazy primitive too Per @mkitti: the morton pair was backwards in the same way the lexicographic pair was. Invert it to match — `morton_order_iter` is now the lazy generator primitive and `_morton_order_keys` collects it into a cached tuple, mirroring `lexicographic_order_iter` / `_lexicographic_order_keys`. No behavioral change for the in-tree consumers (all fully consume the sequence) and the Z-order is identical; this keeps the two coordinate- order families symmetric and gives morton the same lazy/early-exit option lexicographic now has. Co-authored-by: Mark Kittisopikul <mkitti@users.noreply.github.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

d-v-b:perf-sharding-coord-cache

1 day ago

chore(deps): bump the actions group across 1 directory with 8 updates (#4014) Bumps the actions group with 8 updates in the / directory: | Package | From | To | | --- | --- | --- | | [prefix-dev/setup-pixi](https://github.com/prefix-dev/setup-pixi) | `0.9.5` | `0.9.6` | | [codecov/codecov-action](https://github.com/codecov/codecov-action) | `6.0.0` | `6.0.1` | | [github/issue-metrics](https://github.com/github/issue-metrics) | `4.2.2` | `4.2.7` | | [j178/prek-action](https://github.com/j178/prek-action) | `2.0.3` | `2.0.4` | | [actions/upload-artifact](https://github.com/actions/upload-artifact) | `7.0.0` | `7.0.1` | | [actions/download-artifact](https://github.com/actions/download-artifact) | `7.0.0` | `8.0.1` | | [pypa/gh-action-pypi-publish](https://github.com/pypa/gh-action-pypi-publish) | `1.13.0` | `1.14.0` | | [zizmorcore/zizmor-action](https://github.com/zizmorcore/zizmor-action) | `0.5.3` | `0.5.6` | Updates `prefix-dev/setup-pixi` from 0.9.5 to 0.9.6 - [Release notes](https://github.com/prefix-dev/setup-pixi/releases) - [Commits](https://github.com/prefix-dev/setup-pixi/compare/1b2de7f3351f171c8b4dfeb558c639cb58ed4ec0...5185adfbffb4bd703da3010310260805d89ebb11) Updates `codecov/codecov-action` from 6.0.0 to 6.0.1 - [Release notes](https://github.com/codecov/codecov-action/releases) - [Changelog](https://github.com/codecov/codecov-action/blob/main/CHANGELOG.md) - [Commits](https://github.com/codecov/codecov-action/compare/57e3a136b779b570ffcdbf80b3bdc90e7fab3de2...e79a6962e0d4c0c17b229090214935d2e33f8354) Updates `github/issue-metrics` from 4.2.2 to 4.2.7 - [Release notes](https://github.com/github/issue-metrics/releases) - [Commits](https://github.com/github/issue-metrics/compare/c9e9838147fd355dace335ba787f01b6641a400a...1e38d5e62363e14db8019ed7d106b9855bdba6cc) Updates `j178/prek-action` from 2.0.3 to 2.0.4 - [Release notes](https://github.com/j178/prek-action/releases) - [Commits](https://github.com/j178/prek-action/compare/6ad80277337ad479fe43bd70701c3f7f8aa74db3...bdca6f102f98e2b4c7029491a53dfd366469e33d) Updates `actions/upload-artifact` from 7.0.0 to 7.0.1 - [Release notes](https://github.com/actions/upload-artifact/releases) - [Commits](https://github.com/actions/upload-artifact/compare/v7...043fb46d1a93c77aae656e7c1c64a875d1fc6a0a) Updates `actions/download-artifact` from 7.0.0 to 8.0.1 - [Release notes](https://github.com/actions/download-artifact/releases) - [Commits](https://github.com/actions/download-artifact/compare/v7...3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c) Updates `pypa/gh-action-pypi-publish` from 1.13.0 to 1.14.0 - [Release notes](https://github.com/pypa/gh-action-pypi-publish/releases) - [Commits](https://github.com/pypa/gh-action-pypi-publish/compare/v1.13.0...cef221092ed1bacb1cc03d23a2d87d1d172e277b) Updates `zizmorcore/zizmor-action` from 0.5.3 to 0.5.6 - [Release notes](https://github.com/zizmorcore/zizmor-action/releases) - [Commits](https://github.com/zizmorcore/zizmor-action/compare/b1d7e1fb5de872772f31590499237e7cce841e8e...5f14fd08f7cf1cb1609c1e344975f152c7ee938d) --- updated-dependencies: - dependency-name: prefix-dev/setup-pixi dependency-version: 0.9.6 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: actions - dependency-name: codecov/codecov-action dependency-version: 6.0.1 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: actions - dependency-name: github/issue-metrics dependency-version: 4.2.7 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: actions - dependency-name: j178/prek-action dependency-version: 2.0.4 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: actions - dependency-name: actions/upload-artifact dependency-version: 7.0.1 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: actions - dependency-name: actions/download-artifact dependency-version: 8.0.1 dependency-type: direct:production update-type: version-update:semver-major dependency-group: actions - dependency-name: pypa/gh-action-pypi-publish dependency-version: 1.14.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: actions - dependency-name: zizmorcore/zizmor-action dependency-version: 0.5.6 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: actions ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com>

main

3 days ago

perf: cache lexicographic chunk coords in sharding codec The subchunk_write_order feature (#3826) regressed sharded write performance: _encode_partial_single rebuilt the full per-shard chunk coordinate grid on every write via `np.array(list(_subchunk_order_iter(..., "lexicographic")))`, and `to_dict_vectorized` rebuilt a tuple key per row with `tuple(coords.ravel())`. For a single-chunk write into a shard with tens of thousands of chunks this roughly doubled write time (~22ms -> ~40ms on test_sharded_morton_write_single_chunk, matching the -44% CodSpeed regression). Add cached `_lexicographic_order` (array) and `_lexicographic_order_keys` (tuples) helpers in indexing.py, mirroring `_morton_order`/`_morton_order_keys`, and pass the cached keys into `to_dict_vectorized` instead of deriving them row-by-row. This restores write throughput to the pre-#3826 baseline while preserving identical chunk ordering (verified equal to np.ndindex across shapes including 0-d and empty). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

d-v-b:perf-sharding-coord-cache

8 days ago

fix: handle NaN fill_value in array metadata equality (#3999) Frozen dataclass __eq__ compared fill_value directly, so two identical metadata objects with a NaN fill value compared unequal (NaN != NaN under IEEE 754). Compare the JSON-serialized form instead, which treats matching NaN and infinite fill values as equal. Fixes #2929 Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

main

10 days ago

docs: build release notes

d-v-b:3.2.2-release

14 days ago

refactor(sharding): store chunks_per_shard explicitly in _ShardIndex (#3975) * refactor(sharding): store chunks_per_shard explicitly in _ShardIndex _ShardIndex previously inferred the chunk grid shape from offsets_and_lengths.shape[:-1]. For 0-D arrays this collapses the array to rank-1, breaking methods that assume rank >= 2 and forcing a numpy compat cast workaround. Store chunks_per_shard as an explicit NamedTuple field instead. This removes the chunks_per_shard property and its cast, and lets several call sites use the field directly instead of reverse-engineering it. Also fix a latent 0-D bug in is_dense, which iterated offsets_and_lengths assuming rank-2. Closes #3974 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test: parametrize over dimensionalities * refactor(sharding): store chunks_per_shard explicitly in _ShardIndex _ShardIndex previously inferred the chunk grid shape from offsets_and_lengths.shape[:-1]. For 0-D arrays this collapses the array to rank-1, breaking methods that assume rank >= 2 and forcing a numpy compat cast workaround. Store chunks_per_shard as an explicit NamedTuple field instead. This removes the chunks_per_shard property and its cast, and lets several call sites use the field directly instead of reverse-engineering it. Also fix a latent 0-D bug in is_dense, which iterated offsets_and_lengths assuming rank-2. Tests for get_chunk_slices_vectorized, _ShardReader.__iter__, and is_dense are parametrized over chunk grid ranks (0-D, 1-D, 2-D) so 0-D is exercised as a normal case rather than a special branch. Closes #3974 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * remove is_dense --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

main

17 days ago

Merge branch 'main' into coalesce-shard-reads

aldenks:coalesce-shard-reads

17 days ago

Latest Branches

+40%

perf: cache lexicographic chunk coords in sharding codec#4012

15 hours ago

2742897

d-v-b:perf-sharding-coord-cache

+8%

docs: 3.2.2 release notes#3992

14 days ago

05021c6

d-v-b:3.2.2-release

+9%

Optimize partial shard reads#3004

17 days ago

f92e501

aldenks:coalesce-shard-reads

© 2026 CodSpeed Technology

Home Terms Privacy Docs