Skip to content

Conversation

@jdx
Copy link
Owner

@jdx jdx commented Dec 4, 2025

Summary

  • Implement resolve_lock_info for core backends (node, bun, go, zig, deno) to fetch checksums during mise lock without downloading full tarballs
  • Add HTTP caching infrastructure with per-URL OnceCell to prevent redundant network requests when fetching checksums for multiple platforms
  • Add shared helper functions (fetch_checksum_from_shasums, fetch_checksum_from_file) to reduce code duplication

Backend Status

Backend Status Checksum Source Notes
core:node ✅ Implemented SHASUMS256.txt from nodejs.org Cached
core:bun ✅ Implemented SHASUMS256.txt from GitHub releases Cached
core:go ✅ Implemented Individual .sha256 files Per-platform
core:deno ✅ Implemented Individual .sha256sum files Per-platform
core:zig ✅ Implemented JSON index with shasum/size Cached
core:java ❌ Not implemented Adoptium has SHA256 checksums Need to parse Adoptium API response
core:python ❌ Not implemented python.org has MD5 checksums MD5 available, SHA256 would require pyenv plugin
core:ruby ❌ Not implemented No upstream checksums Would need to compute from download
core:elixir ❌ Not implemented GitHub releases have no checksums Would need to compute from download
core:erlang ❌ Not implemented erlang.org has no checksums Would need to compute from download
core:rust ❌ Not implemented rustup handles checksums internally N/A - uses rustup
core:swift ❌ Not implemented swift.org has signatures but no checksums Would need GPG verification or compute
aqua ✅ Already implemented Aqua registry checksums Some packages have checksums in registry
github ✅ Already implemented GitHub release checksums files Looks for SHASUMS/checksums files
conda ✅ Already implemented Conda package metadata Built into conda
ubi ❌ Not implemented No upstream checksums ubi downloads don't have checksums
cargo ❌ N/A Cargo handles verification Built into cargo
npm ❌ N/A npm handles verification Built into npm
pipx ❌ N/A pip handles verification Built into pip
gem ❌ N/A gem handles verification Built into gem
asdf/vfox ❌ Plugin-dependent Varies by plugin Would need per-plugin support

HTTP Caching

Added HTTP.get_text_cached() and HTTP_FETCH.json_cached() methods that use a per-URL OnceCell to:

  • Prevent duplicate concurrent requests to the same URL
  • Cache results for the duration of the mise lock operation
  • Share results across multiple platform targets

This addresses the issue where checksums were only being added during installation (via blake3 hash of downloaded files) rather than during mise lock, which caused autofix CI to add checksums on release branches.

Test plan

  • Build succeeds
  • Lint passes
  • Run mise lock with these tools and verify checksums are populated
  • Run CI

🤖 Generated with Claude Code


Note

Add resolve_lock_info to core backends to populate URLs/checksums during mise lock, with per-URL HTTP caching and platform variant support.

  • Locking flow:
    • Implement Backend::resolve_lock_info across core:node, core:bun, core:go, core:deno, core:zig, and core:ruby to return download url and checksum (and size for zig) without installing.
    • cli lock expands backend-declared platform variants and processes each variant in parallel; dry-run reflects variants.
  • Backends:
    • node: Build filename per target; parse checksum from SHASUMS256.txt.
    • bun: Handle baseline/musl variants; derive filename; parse checksum from SHASUMS256.txt; override get_platform_key; expose platform_variants.
    • go: Construct tarball URL; read .sha256 sidecar (honors go_skip_checksum).
    • deno: Construct URL; read .sha256sum sidecar.
    • zig: Read index JSON (cached) for URL/checksum/size; fallback URL for numbered versions.
    • ruby: Parse ruby index for source tarball URL and sha256 (MRI only).
  • HTTP/caching:
    • Add HTTP.get_text_cached and HTTP_FETCH.json_cached using per-URL OnceCell to dedupe concurrent/repeated requests.
  • Shared helpers:
    • Add fetch_checksum_from_shasums and fetch_checksum_from_file for checksum retrieval.
  • Platform/lockfile:
    • Platform parsing supports compound qualifiers (e.g., musl-baseline) and validates new qualifiers.
    • Add Backend::platform_variants hook; default returns base platform.
    • Lockfile PlatformInfo::is_empty; set_platform_info now merges with existing data and skips empty entries.
    • Lock tool selection constrained to current config root.

Written by Cursor Bugbot for commit 733c67f. This will update automatically on new commits. Configure here.

Copilot AI review requested due to automatic review settings December 4, 2025 00:47
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR implements resolve_lock_info for core backends (node, bun, go, deno, zig) to enable checksum fetching during mise lock operations without downloading full tarballs. It also introduces HTTP caching infrastructure to prevent redundant network requests when fetching checksums for multiple platforms.

Key Changes:

  • Added resolve_lock_info implementations for 5 core backends to fetch checksums from upstream sources
  • Introduced HTTP caching with per-URL OnceCell to deduplicate concurrent requests
  • Added shared helper functions to reduce code duplication across backends

Reviewed changes

Copilot reviewed 7 out of 8 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
src/plugins/core/zig.rs Implements resolve_lock_info using cached JSON index with embedded checksums/sizes
src/plugins/core/node.rs Implements resolve_lock_info using cached SHASUMS256.txt file
src/plugins/core/go.rs Implements resolve_lock_info using individual .sha256 files per tarball
src/plugins/core/deno.rs Implements resolve_lock_info using individual .sha256sum files per zip
src/plugins/core/bun.rs Implements resolve_lock_info using cached SHASUMS256.txt from GitHub releases
src/http.rs Adds get_text_cached and json_cached methods with per-URL OnceCell caching
src/backend/static_helpers.rs Adds fetch_checksum_from_shasums and fetch_checksum_from_file helper functions

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Ok(PlatformInfo {
url: Some(format!(
"https://ziglang.org/download/{}/zig-{}-{}-{}.tar.xz",
tv.version, os, arch, tv.version
Copy link

Copilot AI Dec 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The tarball filename format is incorrect - it should be zig-{os}-{arch}-{version}.tar.xz but the code generates zig-{version}-{os}-{arch}.tar.xz. The correct order based on standard Zig release naming is zig-{os}-{arch}-{version}.tar.xz.

Copilot uses AI. Check for mistakes.
// Get or create the OnceCell for this URL
let cell = {
let mut cache = HTTP_CACHE.lock().unwrap();
cache.entry(key).or_default().clone()
Copy link

Copilot AI Dec 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The HashMap uses String keys which requires cloning the URL string. Consider using Arc<str> or a reference-counted key to avoid string clones on cache hits.

Copilot uses AI. Check for mistakes.
/// when locking multiple platforms). Concurrent requests for the same URL will
/// wait for the first fetch to complete.
pub async fn get_text_cached<U: IntoUrl>(&self, url: U) -> Result<String> {
let url = url.into_url().unwrap();
Copy link

Copilot AI Dec 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using unwrap() here will panic on invalid URLs without a helpful error message. Consider using ? operator to propagate the error with proper context, or provide a descriptive error message.

Suggested change
let url = url.into_url().unwrap();
let url = url.into_url().map_err(Into::into)?;

Copilot uses AI. Check for mistakes.
@jdx jdx force-pushed the feat/lock-checksums-core-backends branch 12 times, most recently from 52b498d to 56ce108 Compare December 4, 2025 01:15
@jdx jdx requested a review from Copilot December 4, 2025 01:16
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 9 out of 10 changed files in this pull request and generated 3 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

impl PlatformInfo {
/// Returns true if this PlatformInfo has no meaningful data (for serde skip)
pub fn is_empty(&self) -> bool {
self.checksum.is_none() && self.url.is_none() && self.url_api.is_none()
Copy link

Copilot AI Dec 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The is_empty() method doesn't check the size field. If size is the only populated field, this method will incorrectly return true. Either include size in the check or document why it's intentionally excluded.

Suggested change
self.checksum.is_none() && self.url.is_none() && self.url_api.is_none()
self.checksum.is_none()
&& self.size.is_none()
&& self.url.is_none()
&& self.url_api.is_none()

Copilot uses AI. Check for mistakes.

// Get or create the OnceCell for this URL
let cell = {
let mut cache = HTTP_CACHE.lock().unwrap();
Copy link

Copilot AI Dec 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using unwrap() on a Mutex::lock() can cause panics if the mutex is poisoned. Consider using lock().expect() with a descriptive message or handle the poison error explicitly to improve debugging.

Suggested change
let mut cache = HTTP_CACHE.lock().unwrap();
let mut cache = HTTP_CACHE.lock().expect("Failed to lock HTTP_CACHE mutex (possibly poisoned)");

Copilot uses AI. Check for mistakes.
Comment on lines +391 to +394
Ok(PlatformInfo {
url: Some(format!(
"https://ziglang.org/download/{}/zig-{}-{}-{}.tar.xz",
tv.version, os, arch, tv.version
Copy link

Copilot AI Dec 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] The version appears twice in the URL format string (positions 1 and 4). Consider using a temporary variable to make this clearer and avoid potential inconsistencies if the format string is modified.

Suggested change
Ok(PlatformInfo {
url: Some(format!(
"https://ziglang.org/download/{}/zig-{}-{}-{}.tar.xz",
tv.version, os, arch, tv.version
let version = &tv.version;
Ok(PlatformInfo {
url: Some(format!(
"https://ziglang.org/download/{}/zig-{}-{}-{}.tar.xz",
version, os, arch, version

Copilot uses AI. Check for mistakes.
Implement resolve_lock_info for core backends (node, bun, go, zig, deno)
to fetch checksums during `mise lock` without downloading full tarballs:

- Node: fetches SHASUMS256.txt from nodejs.org mirror
- Bun: fetches SHASUMS256.txt from GitHub releases
- Go: fetches individual .sha256 files per tarball
- Deno: fetches individual .sha256sum files per zip
- Zig: extracts shasum/size from JSON version index

Add HTTP caching infrastructure:
- HTTP.get_text_cached() with per-URL OnceCell for concurrent-safe caching
- HTTP_FETCH.json_cached() for repeated JSON index fetches
- Prevents redundant network requests when fetching checksums for multiple platforms

Add shared helper functions in static_helpers.rs:
- fetch_checksum_from_shasums() for SHASUMS256.txt pattern
- fetch_checksum_from_file() for individual checksum files

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
@jdx jdx force-pushed the feat/lock-checksums-core-backends branch from 56ce108 to b533ed6 Compare December 4, 2025 01:23
Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: Node flavor setting incorrectly applied to all platforms

The build_platform_slug method unconditionally applies the local node.flavor setting (like "glibc") to all target platforms during mise lock. However, flavor variants are platform-specific (e.g., glibc only exists for Linux). The original slug() function correctly uses the current platform's OS and arch, but this new method applies the local flavor to cross-platform targets like macOS, generating invalid URLs like node-v24-darwin-arm64-glibc.tar.gz. The checksum lookup in SHASUMS256.txt will fail to find these non-existent files.

src/plugins/core/node.rs#L668-L680

/// This mirrors the logic from BuildOpts::new() and slug() function
fn build_platform_slug(&self, version: &str, target: &PlatformTarget) -> String {
let settings = Settings::get();
let os = Self::map_os(target.os_name());
let arch = Self::map_arch(target.arch_name());
if let Some(flavor) = &settings.node.flavor {
format!("node-v{version}-{os}-{arch}-{flavor}")
} else {
format!("node-v{version}-{os}-{arch}")
}
}

Fix in Cursor Fix in Web


jdx and others added 3 commits December 3, 2025 19:36
Bun has compile-time variants (baseline, musl) that result in different
download URLs and checksums. This caused checksum mismatches when the
lockfile had checksums for one variant but a different variant was
downloaded at install time (e.g., windows-x64 vs windows-x64-baseline).

Changes:
- Add platform_variants() method to Backend trait for declaring variants
- Update mise lock to expand platforms into all declared variants
- Implement platform_variants for bun (baseline, musl, musl-baseline)
- Override get_platform_key in bun to return correct variant at install
- Add baseline and musl-baseline as valid platform qualifiers

The lockfile now stores checksums for all platform variants, and at
install time the correct variant is looked up based on compile-time
features (AVX2, musl libc).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
The node.flavor setting (like "glibc") was being applied to all platforms
during `mise lock`, but flavor variants only exist for Linux. This caused
invalid URLs like `node-v24-darwin-arm64-glibc.tar.gz` to be generated,
which would fail checksum lookup in SHASUMS256.txt.

Now only apply the flavor setting when the target platform is Linux.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
@jdx
Copy link
Owner Author

jdx commented Dec 4, 2025

bugbot run

@jdx
Copy link
Owner Author

jdx commented Dec 4, 2025

bugbot run

Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: Compound qualifiers cannot be parsed from platform keys

The Platform::parse function splits platform strings by - and only handles 2 or 3 parts. However, this PR adds "musl-baseline" as a valid qualifier (which contains a hyphen). When a Platform with this qualifier calls to_key(), it produces "linux-x64-musl-baseline" (4 parts when split). This cannot be parsed back because Platform::parse rejects strings with more than 3 hyphen-separated parts. This affects the lock command's determine_target_platforms function which silently ignores lockfile platform keys that fail to parse, and prevents users from explicitly specifying compound qualifier platforms via --platform.

src/platform.rs#L14-L33

mise/src/platform.rs

Lines 14 to 33 in 852f661

/// Parse a platform string in the format "os-arch" or "os-arch-qualifier"
pub fn parse(platform_str: &str) -> Result<Self> {
let parts: Vec<&str> = platform_str.split('-').collect();
match parts.len() {
2 => Ok(Platform {
os: parts[0].to_string(),
arch: parts[1].to_string(),
qualifier: None,
}),
3 => Ok(Platform {
os: parts[0].to_string(),
arch: parts[1].to_string(),
qualifier: Some(parts[2].to_string()),
}),
_ => bail!(
"Invalid platform format '{}'. Expected 'os-arch' or 'os-arch-qualifier'",
platform_str
),
}

src/plugins/core/bun.rs#L225-L230

});
variants.push(Platform {
os: platform.os.clone(),
arch: platform.arch.clone(),
qualifier: Some("musl-baseline".to_string()),
});

Fix in Cursor Fix in Web


@github-actions
Copy link

github-actions bot commented Dec 4, 2025

Hyperfine Performance

mise x -- echo

Command Mean [ms] Min [ms] Max [ms] Relative
mise-2025.11.11 x -- echo 19.0 ± 0.4 18.4 21.7 1.00
mise x -- echo 19.0 ± 0.3 18.4 20.2 1.00 ± 0.02

mise env

Command Mean [ms] Min [ms] Max [ms] Relative
mise-2025.11.11 env 18.3 ± 0.3 17.8 19.3 1.00
mise env 18.7 ± 0.5 17.9 22.4 1.02 ± 0.03

mise hook-env

Command Mean [ms] Min [ms] Max [ms] Relative
mise-2025.11.11 hook-env 18.5 ± 0.4 17.9 23.4 1.00
mise hook-env 18.7 ± 0.4 18.0 20.6 1.01 ± 0.03

mise ls

Command Mean [ms] Min [ms] Max [ms] Relative
mise-2025.11.11 ls 15.7 ± 0.3 15.1 16.9 1.00
mise ls 16.0 ± 0.4 15.3 18.1 1.02 ± 0.03

xtasks/test/perf

Command mise-2025.11.11 mise Variance
install (cached) 105ms 106ms +0%
ls (cached) 64ms 64ms +0%
bin-paths (cached) 70ms 70ms +0%
task-ls (cached) 430ms 421ms +2%

jdx and others added 2 commits December 3, 2025 20:10
Two fixes:

1. Platform parsing now handles compound qualifiers like "musl-baseline"
   by joining all parts after os-arch. Previously, "linux-x64-musl-baseline"
   would fail to parse because it has 4 hyphen-separated parts.

2. Bun's AVX2 detection now uses runtime CPUID instead of compile-time
   cfg! checks. This ensures the correct variant is selected based on the
   actual CPU capabilities, not how mise was compiled. musl detection
   remains compile-time since it depends on the binary's libc linkage.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
The node.flavor setting should only apply when locking the current
platform, not all Linux platforms during cross-platform locking.
This matches the behavior in resolve_lockfile_options which checks
is_current_platform before including the flavor option.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
When platform_variants receives a platform that already has a qualifier
(like linux-x64-musl), it should return just that platform instead of
expanding variants. This prevents duplicate HTTP requests and tasks
when a lockfile already contains variant platforms.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
The show_dry_run function now calls backend.platform_variants() to
expand platforms into their variants, matching the behavior of
process_tools. This ensures the dry-run output accurately shows all
platform variants that would be locked (e.g., bun's baseline, musl
variants).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
@jdx jdx enabled auto-merge (squash) December 4, 2025 02:34
@jdx jdx merged commit 8c3ca4c into main Dec 4, 2025
29 checks passed
@jdx jdx deleted the feat/lock-checksums-core-backends branch December 4, 2025 02:44
jdx added a commit that referenced this pull request Dec 4, 2025
…7181)

## Summary

Fix a bug where invalid platform keys (like
`linux-x64-wait-for-gh-rate-limit`) from tool-specific entries in the
lockfile were being propagated as target platforms for all tools during
`mise lock`.

The issue was introduced in #7180 when collecting existing platforms
from lockfiles. `Platform::parse()` accepts any qualifier string (like
`wait-for-gh-rate-limit` from the `wait-for-gh-rate-limit` tool's
lockfile entry), but these should not be used as target platforms for
other tools.

### Root cause

When `mise lock` runs, it collects target platforms from:
1. Common platforms (linux-x64, linux-arm64, macos-x64, macos-arm64,
windows-x64)
2. Current platform
3. **Existing platforms in the lockfile**

Step 3 was reading all `platforms.*` keys from all tools in the lockfile
without validating that they're valid platform qualifiers. So when a
tool like `wait-for-gh-rate-limit` had its own unique platform key
`linux-x64-wait-for-gh-rate-limit`, that was added to the set of target
platforms for ALL tools.

### The fix

Now we call `platform.validate()` after parsing to ensure only
known-valid qualifiers (`gnu`, `musl`, `msvc`, `baseline`,
`musl-baseline`) are included as target platforms.

## Test plan

- [x] Build succeeds
- [x] Lint passes

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> Validate lockfile platform keys and ignore invalid/tool-specific
qualifiers when deriving target platforms for `mise lock`.
> 
> - **Lock CLI**:
> - Update `src/cli/lock.rs` `determine_target_platforms` to include
lockfile platforms only if `Platform::validate()` succeeds, preventing
tool-specific/invalid qualifiers from becoming target platforms.
> 
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
d7ef083. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->

Co-authored-by: Claude <[email protected]>
jdx pushed a commit that referenced this pull request Dec 4, 2025
### 🚀 Features

- **(config)** add support for netrc by @RobotSupervisor in
[#7164](#7164)
- **(lock)** add resolve_lock_info to core backends for checksum
fetching by @jdx in [#7180](#7180)
- **(ruby)** Install ruby from a zip file over HTTPS by @KaanYT in
[#7167](#7167)
- **(tasks)** add `usage` args to Tera context in run scripts by
@iamkroot in [#7041](#7041)

### 🐛 Bug Fixes

- **(lock)** validate platform qualifiers when reading from lockfile by
@jdx in [#7181](#7181)
- **(task)** retry shebang scripts on ETXTBUSY by @iamkroot in
[#7162](#7162)
- **(ui)** remove duplicate 'mise' prefix in verbose footer output by
@jdx in [#7174](#7174)

### 📦️ Dependency Updates

- bump usage-lib to 2.9.0 by @jdx in
[#7177](#7177)

### 📦 Registry

- remove duplicated ubi and github backends from gping by @risu729 in
[#7144](#7144)
- disable bashly test (not working in CI) by @jdx in
[#7173](#7173)
- disable cfn-lint test (failing in CI) by @jdx in
[#7176](#7176)

### Chore

- add fd to mise.toml by @blampe in
[#7178](#7178)

### New Contributors

- @RobotSupervisor made their first contribution in
[#7164](#7164)

## 📦 Aqua Registry Updates

#### New Packages (2)

- [`Kitware/CMake`](https://github.com/Kitware/CMake)
- [`quarto-dev/quarto-cli`](https://github.com/quarto-dev/quarto-cli)

#### Updated Packages (6)

- [`apache/jena`](https://github.com/apache/jena)
- [`apache/spark`](https://github.com/apache/spark)
-
[`danielfoehrKn/kubeswitch`](https://github.com/danielfoehrKn/kubeswitch)
-
[`danielfoehrKn/kubeswitch/switch-sh`](https://github.com/danielfoehrKn/kubeswitch/switch-sh)
- [`evilmartians/lefthook`](https://github.com/evilmartians/lefthook)
- [`updatecli/updatecli`](https://github.com/updatecli/updatecli)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants