Use OpenSSL EVP API for hashing

[randomizedcoder]

Summary

Replace deprecated low-level OpenSSL hash functions (MD5_Init/SHA256_Update/SHA512_Final/etc.) with the EVP digest API (EVP_DigestInit_ex, EVP_DigestUpdate, EVP_DigestFinal_ex).

• The low-level APIs (MD5_*, SHA1_*, SHA256_*, SHA512_*) are deprecated in OpenSSL 3.x
• The EVP API is the recommended interface since OpenSSL 1.1.0
• EVP routes through OpenSSL's provider system (future-proof for hardware acceleration, FIPS modules)
• Nix already requires OpenSSL >= 1.1.1 (checked in meson.build)
• Hash output is identical — EVP is a higher-level wrapper around the same algorithms

Changes

Single file modified: src/libutil/hash.cc

Includes

• Removed: <openssl/md5.h>, <openssl/sha.h>
• Added: <openssl/evp.h>

Hash context union

Replaced four algorithm-specific context structs with a single EVP_MD_CTX * pointer:

// Before
union Hash::Ctx {
    blake3_hasher blake3;
    MD5_CTX md5;
    SHA_CTX sha1;
    SHA256_CTX sha256;
    SHA512_CTX sha512;
};

// After
union Hash::Ctx {
    blake3_hasher blake3;
    EVP_MD_CTX * evp;
};

Added getEVPAlgo() helper to map HashAlgorithm enum to EVP algorithm objects.

Init / Update / Final

Replaced per-algorithm if/else chains with unified EVP calls:

• start(): EVP_MD_CTX_new() + EVP_DigestInit_ex()
• update(): EVP_DigestUpdate()
• finish(): EVP_DigestFinal_ex() + EVP_MD_CTX_free()

BLAKE3 paths are unchanged (it doesn't use OpenSSL).

HashSink lifetime management

Since EVP_MD_CTX * is heap-allocated (unlike the old value-type structs):

• Destructor frees the EVP context if still live
• finish() nulls the pointer after nix::finish() frees it
• currentHash() uses EVP_MD_CTX_copy_ex() to snapshot the digest state (can't memcpy an opaque EVP context)

Test plan

Formatting

• nix develop -c ./maintainers/format.sh — passes clean (clang-format, meson-format, nixfmt, shellcheck)

Build

• Full build compiles cleanly (meson setup build --buildtype=debugoptimized && ninja -C build — 508/508 targets)
• No changes to meson.build, meson.options, or package.nix — libcrypto already provides <openssl/evp.h>

Unit tests (22/22 hash tests pass)

• BLAKE3HashTest.testKnownBLAKE3Hashes{1,2,3} — BLAKE3 known-vector tests
• hashString.testKnownMD5Hashes{1,2} — MD5 known-vector tests
• hashString.testKnownSHA1Hashes{1,2} — SHA-1 known-vector tests
• hashString.testKnownSHA256Hashes{1,2} — SHA-256 known-vector tests
• hashString.testKnownSHA512Hashes{1,2} — SHA-512 known-vector tests
• hashParseExplicitFormatUnprefixed — parse/format round-trip tests
• hashFormat.testRoundTripPrintParse — hash format serialization
• HashJSON and BLAKE3HashJSON — JSON serialization round-trips

Functional tests (125/125 pass, 5 skipped for unrelated reasons)

• nix-functional-tests:main / hash-convert — nix hash convert CLI
• nix-functional-tests:main / hash-path — nix hash path CLI
• nix-functional-tests:main / path-from-hash-part — store path hash resolution
• nix-functional-tests:git-hashing / simple-sha1 — git SHA-1 hashing
• nix-functional-tests:git-hashing / simple-sha256 — git SHA-256 hashing
• nix-functional-tests:git-hashing / fixed — fixed-output derivation hashing
• All other functional tests that depend on hashing (fetchTree, binary-cache, flakes, etc.)

Manual verification

• nix hash file output for MD5, SHA-1, SHA-256, SHA-512 matches coreutils (md5sum, sha1sum, sha256sum, sha512sum) byte-for-byte

🤖 Generated with Claude Code