Skip to content

chore: update the test runner #7404

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
wemeetagain merged 45 commits into from
Feb 27, 2025
Merged

chore: update the test runner #7404

wemeetagain merged 45 commits into from
Feb 27, 2025

Conversation

@nazarhussain
Copy link
Contributor

@nazarhussain nazarhussain commented Jan 27, 2025 •
edited
Loading

Motivation

Update vitest and browser engine.

Description

  • Move root level shared configs to configs directory
  • Update to vitest 3.x
  • Migrate from webdriverio to playwright which is more stable and recommended by Vitest.

NOTE
Reverted the playwright upgrade. See the comments.

lodestar/configs/vitest.config.base.browser.ts

Lines 41 to 43 in d2a7af0

// Recommended provider is `playwright` but it's causing following error on CI
// Error: Failed to connect to the browser session "af5be85a-7f29-4299-b680-b07f0cfc2520" within the timeout.
// TODO: Debug the issue in later versions of playwright and vitest

Steps to test or reproduce

Run all tests

@nazarhussain nazarhussain requested a review from a team as a code owner January 27, 2025 11:28
@nazarhussain nazarhussain marked this pull request as draft January 27, 2025 11:28
@github-actions
Copy link
Contributor

github-actions bot commented Jan 27, 2025 •
edited
Loading

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: a837018 Previous: d1f9f7b Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 941.23 us/op 740.95 us/op 1.27
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 34.305 us/op 30.102 us/op 1.14
BLS verify - blst 801.21 us/op 931.00 us/op 0.86
BLS verifyMultipleSignatures 3 - blst 1.4908 ms/op 1.2823 ms/op 1.16
BLS verifyMultipleSignatures 8 - blst 2.1744 ms/op 2.0341 ms/op 1.07
BLS verifyMultipleSignatures 32 - blst 5.6946 ms/op 4.4062 ms/op 1.29
BLS verifyMultipleSignatures 64 - blst 10.632 ms/op 8.2269 ms/op 1.29
BLS verifyMultipleSignatures 128 - blst 17.306 ms/op 15.660 ms/op 1.11
BLS deserializing 10000 signatures 686.56 ms/op 573.73 ms/op 1.20
BLS deserializing 100000 signatures 6.9511 s/op 5.8944 s/op 1.18
BLS verifyMultipleSignatures - same message - 3 - blst 938.06 us/op 926.67 us/op 1.01
BLS verifyMultipleSignatures - same message - 8 - blst 1.0481 ms/op 1.0630 ms/op 0.99
BLS verifyMultipleSignatures - same message - 32 - blst 1.7826 ms/op 1.6424 ms/op 1.09
BLS verifyMultipleSignatures - same message - 64 - blst 2.5132 ms/op 2.3562 ms/op 1.07
BLS verifyMultipleSignatures - same message - 128 - blst 4.3749 ms/op 3.9599 ms/op 1.10
BLS aggregatePubkeys 32 - blst 19.261 us/op 16.828 us/op 1.14
BLS aggregatePubkeys 128 - blst 69.026 us/op 59.054 us/op 1.17
notSeenSlots=1 numMissedVotes=1 numBadVotes=10 58.191 ms/op 56.438 ms/op 1.03
notSeenSlots=1 numMissedVotes=0 numBadVotes=4 58.287 ms/op 44.557 ms/op 1.31
notSeenSlots=2 numMissedVotes=1 numBadVotes=10 45.603 ms/op 33.416 ms/op 1.36
getSlashingsAndExits - default max 100.53 us/op 50.239 us/op 2.00
getSlashingsAndExits - 2k 326.12 us/op 257.78 us/op 1.27
proposeBlockBody type=full, size=empty 4.8753 ms/op 3.8537 ms/op 1.27
isKnown best case - 1 super set check 203.00 ns/op 399.00 ns/op 0.51
isKnown normal case - 2 super set checks 200.00 ns/op 388.00 ns/op 0.52
isKnown worse case - 16 super set checks 200.00 ns/op 398.00 ns/op 0.50
InMemoryCheckpointStateCache - add get delete 2.4290 us/op 2.4000 us/op 1.01
validate api signedAggregateAndProof - struct 1.4983 ms/op 1.9754 ms/op 0.76
validate gossip signedAggregateAndProof - struct 1.4376 ms/op 1.9284 ms/op 0.75
batch validate gossip attestation - vc 640000 - chunk 32 123.94 us/op 107.30 us/op 1.16
batch validate gossip attestation - vc 640000 - chunk 64 114.27 us/op 92.097 us/op 1.24
batch validate gossip attestation - vc 640000 - chunk 128 118.18 us/op 86.048 us/op 1.37
batch validate gossip attestation - vc 640000 - chunk 256 115.30 us/op 87.985 us/op 1.31
pickEth1Vote - no votes 981.80 us/op 791.51 us/op 1.24
pickEth1Vote - max votes 7.9644 ms/op 4.8509 ms/op 1.64
pickEth1Vote - Eth1Data hashTreeRoot value x2048 14.540 ms/op 10.984 ms/op 1.32
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 18.820 ms/op 15.928 ms/op 1.18
pickEth1Vote - Eth1Data fastSerialize value x2048 455.93 us/op 354.21 us/op 1.29
pickEth1Vote - Eth1Data fastSerialize tree x2048 3.6439 ms/op 2.8601 ms/op 1.27
bytes32 toHexString 403.00 ns/op 532.00 ns/op 0.76
bytes32 Buffer.toString(hex) 265.00 ns/op 424.00 ns/op 0.63
bytes32 Buffer.toString(hex) from Uint8Array 354.00 ns/op 505.00 ns/op 0.70
bytes32 Buffer.toString(hex) + 0x 246.00 ns/op 429.00 ns/op 0.57
Object access 1 prop 0.12500 ns/op 0.32800 ns/op 0.38
Map access 1 prop 0.12300 ns/op 0.31600 ns/op 0.39
Object get x1000 5.9210 ns/op 5.3170 ns/op 1.11
Map get x1000 6.6710 ns/op 5.9950 ns/op 1.11
Object set x1000 33.486 ns/op 18.560 ns/op 1.80
Map set x1000 21.760 ns/op 16.453 ns/op 1.32
Return object 10000 times 0.29790 ns/op 0.29170 ns/op 1.02
Throw Error 10000 times 4.4718 us/op 3.4691 us/op 1.29
toHex 137.37 ns/op 106.18 ns/op 1.29
Buffer.from 131.68 ns/op 98.531 ns/op 1.34
shared Buffer 78.882 ns/op 65.657 ns/op 1.20
fastMsgIdFn sha256 / 200 bytes 2.2230 us/op 1.9220 us/op 1.16
fastMsgIdFn h32 xxhash / 200 bytes 224.00 ns/op 381.00 ns/op 0.59
fastMsgIdFn h64 xxhash / 200 bytes 271.00 ns/op 441.00 ns/op 0.61
fastMsgIdFn sha256 / 1000 bytes 7.3020 us/op 5.9260 us/op 1.23
fastMsgIdFn h32 xxhash / 1000 bytes 341.00 ns/op 497.00 ns/op 0.69
fastMsgIdFn h64 xxhash / 1000 bytes 375.00 ns/op 512.00 ns/op 0.73
fastMsgIdFn sha256 / 10000 bytes 66.364 us/op 47.060 us/op 1.41
fastMsgIdFn h32 xxhash / 10000 bytes 1.8950 us/op 1.8040 us/op 1.05
fastMsgIdFn h64 xxhash / 10000 bytes 1.2630 us/op 1.2750 us/op 0.99
send data - 1000 256B messages 15.457 ms/op 8.5362 ms/op 1.81
send data - 1000 512B messages 19.939 ms/op 12.604 ms/op 1.58
send data - 1000 1024B messages 28.765 ms/op 19.039 ms/op 1.51
send data - 1000 1200B messages 22.480 ms/op 17.564 ms/op 1.28
send data - 1000 2048B messages 22.647 ms/op 15.810 ms/op 1.43
send data - 1000 4096B messages 29.154 ms/op 17.153 ms/op 1.70
send data - 1000 16384B messages 73.846 ms/op 41.962 ms/op 1.76
send data - 1000 65536B messages 248.88 ms/op 244.22 ms/op 1.02
enrSubnets - fastDeserialize 64 bits 1.8130 us/op 974.00 ns/op 1.86
enrSubnets - ssz BitVector 64 bits 341.00 ns/op 492.00 ns/op 0.69
enrSubnets - fastDeserialize 4 bits 135.00 ns/op 318.00 ns/op 0.42
enrSubnets - ssz BitVector 4 bits 417.00 ns/op 486.00 ns/op 0.86
prioritizePeers score -10:0 att 32-0.1 sync 2-0 124.83 us/op 97.350 us/op 1.28
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 149.80 us/op 116.17 us/op 1.29
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 212.81 us/op 181.47 us/op 1.17
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 403.31 us/op 318.77 us/op 1.27
prioritizePeers score 0:0 att 64-1 sync 4-1 485.46 us/op 366.51 us/op 1.32
array of 16000 items push then shift 1.6672 us/op 1.2637 us/op 1.32
LinkedList of 16000 items push then shift 7.8730 ns/op 6.1180 ns/op 1.29
array of 16000 items push then pop 88.580 ns/op 59.559 ns/op 1.49
LinkedList of 16000 items push then pop 7.8440 ns/op 6.0090 ns/op 1.31
array of 24000 items push then shift 2.5159 us/op 1.8639 us/op 1.35
LinkedList of 24000 items push then shift 8.8000 ns/op 6.1530 ns/op 1.43
array of 24000 items push then pop 120.32 ns/op 82.923 ns/op 1.45
LinkedList of 24000 items push then pop 7.7120 ns/op 6.0420 ns/op 1.28
intersect bitArray bitLen 8 7.5360 ns/op 5.2910 ns/op 1.42
intersect array and set length 8 39.727 ns/op 32.442 ns/op 1.22
intersect bitArray bitLen 128 30.878 ns/op 26.088 ns/op 1.18
intersect array and set length 128 647.85 ns/op 538.58 ns/op 1.20
bitArray.getTrueBitIndexes() bitLen 128 1.0890 us/op 1.1080 us/op 0.98
bitArray.getTrueBitIndexes() bitLen 248 1.8850 us/op 1.7520 us/op 1.08
bitArray.getTrueBitIndexes() bitLen 512 3.9030 us/op 3.2050 us/op 1.22
Buffer.concat 32 items 732.00 ns/op 793.00 ns/op 0.92
Uint8Array.set 32 items 1.6060 us/op 1.3650 us/op 1.18
Buffer.copy 2.7090 us/op 2.1490 us/op 1.26
Uint8Array.set - with subarray 2.4400 us/op 1.6220 us/op 1.50
Uint8Array.set - without subarray 1.6350 us/op 1.0760 us/op 1.52
getUint32 - dataview 200.00 ns/op 352.00 ns/op 0.57
getUint32 - manual 130.00 ns/op 285.00 ns/op 0.46
Set add up to 64 items then delete first 2.3184 us/op 1.5850 us/op 1.46
OrderedSet add up to 64 items then delete first 3.7297 us/op 2.4575 us/op 1.52
Set add up to 64 items then delete last 2.6283 us/op 1.8254 us/op 1.44
OrderedSet add up to 64 items then delete last 3.7934 us/op 3.0063 us/op 1.26
Set add up to 64 items then delete middle 2.5009 us/op 2.0154 us/op 1.24
OrderedSet add up to 64 items then delete middle 5.6017 us/op 4.4897 us/op 1.25
Set add up to 128 items then delete first 5.3073 us/op 3.9524 us/op 1.34
OrderedSet add up to 128 items then delete first 8.2368 us/op 6.2042 us/op 1.33
Set add up to 128 items then delete last 5.2955 us/op 3.8270 us/op 1.38
OrderedSet add up to 128 items then delete last 7.8317 us/op 5.8614 us/op 1.34
Set add up to 128 items then delete middle 5.1615 us/op 3.8225 us/op 1.35
OrderedSet add up to 128 items then delete middle 14.851 us/op 11.661 us/op 1.27
Set add up to 256 items then delete first 11.608 us/op 7.7228 us/op 1.50
OrderedSet add up to 256 items then delete first 19.185 us/op 12.165 us/op 1.58
Set add up to 256 items then delete last 10.521 us/op 7.5191 us/op 1.40
OrderedSet add up to 256 items then delete last 15.533 us/op 11.463 us/op 1.35
Set add up to 256 items then delete middle 10.371 us/op 7.4810 us/op 1.39
OrderedSet add up to 256 items then delete middle 46.947 us/op 33.914 us/op 1.38
transfer serialized Status (84 B) 3.8090 us/op 2.0600 us/op 1.85
copy serialized Status (84 B) 1.2550 us/op 1.2730 us/op 0.99
transfer serialized SignedVoluntaryExit (112 B) 2.3340 us/op 2.0690 us/op 1.13
copy serialized SignedVoluntaryExit (112 B) 1.4650 us/op 1.3050 us/op 1.12
transfer serialized ProposerSlashing (416 B) 2.5790 us/op 2.0750 us/op 1.24
copy serialized ProposerSlashing (416 B) 1.4800 us/op 1.3300 us/op 1.11
transfer serialized Attestation (485 B) 2.5350 us/op 2.5580 us/op 0.99
copy serialized Attestation (485 B) 1.4340 us/op 1.3980 us/op 1.03
transfer serialized AttesterSlashing (33232 B) 3.1950 us/op 2.4230 us/op 1.32
copy serialized AttesterSlashing (33232 B) 4.8330 us/op 3.3230 us/op 1.45
transfer serialized Small SignedBeaconBlock (128000 B) 3.6860 us/op 2.5270 us/op 1.46
copy serialized Small SignedBeaconBlock (128000 B) 11.608 us/op 7.9320 us/op 1.46
transfer serialized Avg SignedBeaconBlock (200000 B) 4.1880 us/op 2.6720 us/op 1.57
copy serialized Avg SignedBeaconBlock (200000 B) 17.350 us/op 11.681 us/op 1.49
transfer serialized BlobsSidecar (524380 B) 4.3530 us/op 3.0670 us/op 1.42
copy serialized BlobsSidecar (524380 B) 65.568 us/op 52.088 us/op 1.26
transfer serialized Big SignedBeaconBlock (1000000 B) 4.7790 us/op 3.3020 us/op 1.45
copy serialized Big SignedBeaconBlock (1000000 B) 123.75 us/op 175.96 us/op 0.70
pass gossip attestations to forkchoice per slot 2.8822 ms/op 2.1852 ms/op 1.32
forkChoice updateHead vc 100000 bc 64 eq 0 472.43 us/op 324.91 us/op 1.45
forkChoice updateHead vc 600000 bc 64 eq 0 3.0755 ms/op 2.0124 ms/op 1.53
forkChoice updateHead vc 1000000 bc 64 eq 0 6.4902 ms/op 3.5660 ms/op 1.82
forkChoice updateHead vc 600000 bc 320 eq 0 3.0559 ms/op 2.0177 ms/op 1.51
forkChoice updateHead vc 600000 bc 1200 eq 0 3.0168 ms/op 2.0629 ms/op 1.46
forkChoice updateHead vc 600000 bc 7200 eq 0 3.3779 ms/op 3.0789 ms/op 1.10
forkChoice updateHead vc 600000 bc 64 eq 1000 10.810 ms/op 8.6719 ms/op 1.25
forkChoice updateHead vc 600000 bc 64 eq 10000 10.782 ms/op 8.1821 ms/op 1.32
forkChoice updateHead vc 600000 bc 64 eq 300000 14.726 ms/op 10.420 ms/op 1.41
computeDeltas 500000 validators 300 proto nodes 4.0763 ms/op 3.1519 ms/op 1.29
computeDeltas 500000 validators 1200 proto nodes 4.1405 ms/op 3.2011 ms/op 1.29
computeDeltas 500000 validators 7200 proto nodes 4.2435 ms/op 3.2197 ms/op 1.32
computeDeltas 750000 validators 300 proto nodes 5.9468 ms/op 4.8567 ms/op 1.22
computeDeltas 750000 validators 1200 proto nodes 5.9199 ms/op 4.7473 ms/op 1.25
computeDeltas 750000 validators 7200 proto nodes 6.0222 ms/op 4.7398 ms/op 1.27
computeDeltas 1400000 validators 300 proto nodes 11.115 ms/op 8.9280 ms/op 1.24
computeDeltas 1400000 validators 1200 proto nodes 11.808 ms/op 8.7546 ms/op 1.35
computeDeltas 1400000 validators 7200 proto nodes 13.432 ms/op 8.6767 ms/op 1.55
computeDeltas 2100000 validators 300 proto nodes 18.852 ms/op 13.699 ms/op 1.38
computeDeltas 2100000 validators 1200 proto nodes 17.752 ms/op 13.999 ms/op 1.27
computeDeltas 2100000 validators 7200 proto nodes 18.611 ms/op 13.748 ms/op 1.35
altair processAttestation - 250000 vs - 7PWei normalcase 3.1181 ms/op 1.6347 ms/op 1.91
altair processAttestation - 250000 vs - 7PWei worstcase 4.2029 ms/op 2.4595 ms/op 1.71
altair processAttestation - setStatus - 1/6 committees join 142.55 us/op 97.552 us/op 1.46
altair processAttestation - setStatus - 1/3 committees join 267.37 us/op 181.95 us/op 1.47
altair processAttestation - setStatus - 1/2 committees join 379.99 us/op 283.86 us/op 1.34
altair processAttestation - setStatus - 2/3 committees join 459.13 us/op 364.00 us/op 1.26
altair processAttestation - setStatus - 4/5 committees join 756.66 us/op 502.06 us/op 1.51
altair processAttestation - setStatus - 100% committees join 836.25 us/op 559.64 us/op 1.49
altair processBlock - 250000 vs - 7PWei normalcase 5.5721 ms/op 4.6049 ms/op 1.21
altair processBlock - 250000 vs - 7PWei normalcase hashState 38.833 ms/op 28.680 ms/op 1.35
altair processBlock - 250000 vs - 7PWei worstcase 62.152 ms/op 39.449 ms/op 1.58
altair processBlock - 250000 vs - 7PWei worstcase hashState 105.38 ms/op 100.67 ms/op 1.05
phase0 processBlock - 250000 vs - 7PWei normalcase 2.1458 ms/op 1.5968 ms/op 1.34
phase0 processBlock - 250000 vs - 7PWei worstcase 30.772 ms/op 25.165 ms/op 1.22
altair processEth1Data - 250000 vs - 7PWei normalcase 356.26 us/op 283.87 us/op 1.25
getExpectedWithdrawals 250000 eb:1,eth1:1,we:0,wn:0,smpl:15 7.0540 us/op 4.6720 us/op 1.51
getExpectedWithdrawals 250000 eb:0.95,eth1:0.1,we:0.05,wn:0,smpl:219 47.467 us/op 28.114 us/op 1.69
getExpectedWithdrawals 250000 eb:0.95,eth1:0.3,we:0.05,wn:0,smpl:42 12.257 us/op 7.2840 us/op 1.68
getExpectedWithdrawals 250000 eb:0.95,eth1:0.7,we:0.05,wn:0,smpl:18 6.6670 us/op 4.9100 us/op 1.36
getExpectedWithdrawals 250000 eb:0.1,eth1:0.1,we:0,wn:0,smpl:1020 179.20 us/op 107.87 us/op 1.66
getExpectedWithdrawals 250000 eb:0.03,eth1:0.03,we:0,wn:0,smpl:11777 1.3184 ms/op 708.56 us/op 1.86
getExpectedWithdrawals 250000 eb:0.01,eth1:0.01,we:0,wn:0,smpl:16384 1.6725 ms/op 984.51 us/op 1.70
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,smpl:16384 1.6547 ms/op 976.78 us/op 1.69
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,nocache,smpl:16384 3.7557 ms/op 2.6642 ms/op 1.41
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,smpl:16384 1.7319 ms/op 986.48 us/op 1.76
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,nocache,smpl:16384 3.6817 ms/op 2.7395 ms/op 1.34
Tree 40 250000 create 436.13 ms/op 332.51 ms/op 1.31
Tree 40 250000 get(125000) 145.94 ns/op 114.27 ns/op 1.28
Tree 40 250000 set(125000) 1.4972 us/op 1.2262 us/op 1.22
Tree 40 250000 toArray() 21.353 ms/op 10.089 ms/op 2.12
Tree 40 250000 iterate all - toArray() + loop 20.389 ms/op 9.6569 ms/op 2.11
Tree 40 250000 iterate all - get(i) 56.883 ms/op 41.791 ms/op 1.36
Array 250000 create 2.5482 ms/op 2.4114 ms/op 1.06
Array 250000 clone - spread 835.22 us/op 647.69 us/op 1.29
Array 250000 get(125000) 0.41400 ns/op 0.55300 ns/op 0.75
Array 250000 set(125000) 0.44900 ns/op 0.57200 ns/op 0.78
Array 250000 iterate all - loop 89.799 us/op 73.377 us/op 1.22
phase0 afterProcessEpoch - 250000 vs - 7PWei 53.738 ms/op 39.871 ms/op 1.35
Array.fill - length 1000000 5.7125 ms/op 2.2723 ms/op 2.51
Array push - length 1000000 25.073 ms/op 10.433 ms/op 2.40
Array.get 0.28167 ns/op 0.24772 ns/op 1.14
Uint8Array.get 0.45388 ns/op 0.32679 ns/op 1.39
phase0 beforeProcessEpoch - 250000 vs - 7PWei 24.304 ms/op 17.299 ms/op 1.40
altair processEpoch - mainnet_e81889 311.51 ms/op 229.89 ms/op 1.36
mainnet_e81889 - altair beforeProcessEpoch 20.624 ms/op 15.318 ms/op 1.35
mainnet_e81889 - altair processJustificationAndFinalization 6.8190 us/op 4.5380 us/op 1.50
mainnet_e81889 - altair processInactivityUpdates 5.3258 ms/op 3.1851 ms/op 1.67
mainnet_e81889 - altair processRewardsAndPenalties 50.581 ms/op 36.016 ms/op 1.40
mainnet_e81889 - altair processRegistryUpdates 752.00 ns/op 835.00 ns/op 0.90
mainnet_e81889 - altair processSlashings 179.00 ns/op 390.00 ns/op 0.46
mainnet_e81889 - altair processEth1DataReset 178.00 ns/op 403.00 ns/op 0.44
mainnet_e81889 - altair processEffectiveBalanceUpdates 1.3839 ms/op 1.0195 ms/op 1.36
mainnet_e81889 - altair processSlashingsReset 994.00 ns/op 1.1830 us/op 0.84
mainnet_e81889 - altair processRandaoMixesReset 1.2710 us/op 1.3500 us/op 0.94
mainnet_e81889 - altair processHistoricalRootsUpdate 264.00 ns/op 386.00 ns/op 0.68
mainnet_e81889 - altair processParticipationFlagUpdates 616.00 ns/op 668.00 ns/op 0.92
mainnet_e81889 - altair processSyncCommitteeUpdates 144.00 ns/op 358.00 ns/op 0.40
mainnet_e81889 - altair afterProcessEpoch 57.665 ms/op 41.916 ms/op 1.38
capella processEpoch - mainnet_e217614 930.20 ms/op 951.83 ms/op 0.98
mainnet_e217614 - capella beforeProcessEpoch 69.403 ms/op 61.054 ms/op 1.14
mainnet_e217614 - capella processJustificationAndFinalization 5.3360 us/op 4.5170 us/op 1.18
mainnet_e217614 - capella processInactivityUpdates 16.251 ms/op 13.866 ms/op 1.17
mainnet_e217614 - capella processRewardsAndPenalties 191.68 ms/op 194.38 ms/op 0.99
mainnet_e217614 - capella processRegistryUpdates 6.5410 us/op 5.1350 us/op 1.27
mainnet_e217614 - capella processSlashings 174.00 ns/op 401.00 ns/op 0.43
mainnet_e217614 - capella processEth1DataReset 169.00 ns/op 396.00 ns/op 0.43
mainnet_e217614 - capella processEffectiveBalanceUpdates 11.123 ms/op 13.209 ms/op 0.84
mainnet_e217614 - capella processSlashingsReset 888.00 ns/op 1.1550 us/op 0.77
mainnet_e217614 - capella processRandaoMixesReset 1.5130 us/op 1.3250 us/op 1.14
mainnet_e217614 - capella processHistoricalRootsUpdate 171.00 ns/op 386.00 ns/op 0.44
mainnet_e217614 - capella processParticipationFlagUpdates 567.00 ns/op 685.00 ns/op 0.83
mainnet_e217614 - capella afterProcessEpoch 126.51 ms/op 100.43 ms/op 1.26
phase0 processEpoch - mainnet_e58758 299.53 ms/op 280.77 ms/op 1.07
mainnet_e58758 - phase0 beforeProcessEpoch 92.188 ms/op 70.455 ms/op 1.31
mainnet_e58758 - phase0 processJustificationAndFinalization 6.4320 us/op 4.4140 us/op 1.46
mainnet_e58758 - phase0 processRewardsAndPenalties 40.103 ms/op 43.078 ms/op 0.93
mainnet_e58758 - phase0 processRegistryUpdates 3.1270 us/op 2.6260 us/op 1.19
mainnet_e58758 - phase0 processSlashings 174.00 ns/op 399.00 ns/op 0.44
mainnet_e58758 - phase0 processEth1DataReset 169.00 ns/op 380.00 ns/op 0.44
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 1.0535 ms/op 758.30 us/op 1.39
mainnet_e58758 - phase0 processSlashingsReset 900.00 ns/op 1.1110 us/op 0.81
mainnet_e58758 - phase0 processRandaoMixesReset 1.2650 us/op 1.3870 us/op 0.91
mainnet_e58758 - phase0 processHistoricalRootsUpdate 171.00 ns/op 394.00 ns/op 0.43
mainnet_e58758 - phase0 processParticipationRecordUpdates 854.00 ns/op 1.1540 us/op 0.74
mainnet_e58758 - phase0 afterProcessEpoch 42.196 ms/op 35.027 ms/op 1.20
phase0 processEffectiveBalanceUpdates - 250000 normalcase 2.5175 ms/op 941.43 us/op 2.67
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 1.8033 ms/op 2.1768 ms/op 0.83
altair processInactivityUpdates - 250000 normalcase 22.125 ms/op 18.218 ms/op 1.21
altair processInactivityUpdates - 250000 worstcase 18.767 ms/op 18.964 ms/op 0.99
phase0 processRegistryUpdates - 250000 normalcase 9.6690 us/op 3.3610 us/op 2.88
phase0 processRegistryUpdates - 250000 badcase_full_deposits 319.29 us/op 285.93 us/op 1.12
phase0 processRegistryUpdates - 250000 worstcase 0.5 116.17 ms/op 95.767 ms/op 1.21
altair processRewardsAndPenalties - 250000 normalcase 35.165 ms/op 52.040 ms/op 0.68
altair processRewardsAndPenalties - 250000 worstcase 47.669 ms/op 45.892 ms/op 1.04
phase0 getAttestationDeltas - 250000 normalcase 7.6111 ms/op 4.5851 ms/op 1.66
phase0 getAttestationDeltas - 250000 worstcase 6.3054 ms/op 4.6070 ms/op 1.37
phase0 processSlashings - 250000 worstcase 105.82 us/op 88.184 us/op 1.20
altair processSyncCommitteeUpdates - 250000 131.13 ms/op 90.997 ms/op 1.44
BeaconState.hashTreeRoot - No change 263.00 ns/op 424.00 ns/op 0.62
BeaconState.hashTreeRoot - 1 full validator 74.682 us/op 82.561 us/op 0.90
BeaconState.hashTreeRoot - 32 full validator 1.1738 ms/op 743.50 us/op 1.58
BeaconState.hashTreeRoot - 512 full validator 11.607 ms/op 8.6299 ms/op 1.35
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 102.51 us/op 94.574 us/op 1.08
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 1.8525 ms/op 1.2460 ms/op 1.49
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 21.722 ms/op 24.439 ms/op 0.89
BeaconState.hashTreeRoot - 1 balances 125.31 us/op 75.251 us/op 1.67
BeaconState.hashTreeRoot - 32 balances 798.10 us/op 810.18 us/op 0.99
BeaconState.hashTreeRoot - 512 balances 8.1307 ms/op 8.9999 ms/op 0.90
BeaconState.hashTreeRoot - 250000 balances 190.60 ms/op 161.69 ms/op 1.18
aggregationBits - 2048 els - zipIndexesInBitList 22.901 us/op 18.995 us/op 1.21
byteArrayEquals 32 54.375 ns/op 46.177 ns/op 1.18
Buffer.compare 32 17.550 ns/op 14.702 ns/op 1.19
byteArrayEquals 1024 1.6224 us/op 1.2164 us/op 1.33
Buffer.compare 1024 24.953 ns/op 21.605 ns/op 1.15
byteArrayEquals 16384 26.173 us/op 19.339 us/op 1.35
Buffer.compare 16384 215.76 ns/op 176.88 ns/op 1.22
byteArrayEquals 123687377 199.55 ms/op 148.25 ms/op 1.35
Buffer.compare 123687377 8.5926 ms/op 3.9309 ms/op 2.19
byteArrayEquals 32 - diff last byte 55.291 ns/op 47.335 ns/op 1.17
Buffer.compare 32 - diff last byte 17.756 ns/op 15.887 ns/op 1.12
byteArrayEquals 1024 - diff last byte 1.6738 us/op 1.2584 us/op 1.33
Buffer.compare 1024 - diff last byte 26.507 ns/op 25.049 ns/op 1.06
byteArrayEquals 16384 - diff last byte 28.647 us/op 19.967 us/op 1.43
Buffer.compare 16384 - diff last byte 182.34 ns/op 172.61 ns/op 1.06
byteArrayEquals 123687377 - diff last byte 201.21 ms/op 150.66 ms/op 1.34
Buffer.compare 123687377 - diff last byte 8.4633 ms/op 3.7960 ms/op 2.23
byteArrayEquals 32 - random bytes 5.2560 ns/op 4.8720 ns/op 1.08
Buffer.compare 32 - random bytes 17.488 ns/op 16.980 ns/op 1.03
byteArrayEquals 1024 - random bytes 5.4340 ns/op 4.8570 ns/op 1.12
Buffer.compare 1024 - random bytes 18.700 ns/op 16.601 ns/op 1.13
byteArrayEquals 16384 - random bytes 5.4280 ns/op 4.7870 ns/op 1.13
Buffer.compare 16384 - random bytes 18.493 ns/op 16.966 ns/op 1.09
byteArrayEquals 123687377 - random bytes 6.5700 ns/op 7.5800 ns/op 0.87
Buffer.compare 123687377 - random bytes 18.700 ns/op 20.110 ns/op 0.93
regular array get 100000 times 41.561 us/op 30.452 us/op 1.36
wrappedArray get 100000 times 49.331 us/op 30.447 us/op 1.62
arrayWithProxy get 100000 times 13.758 ms/op 8.9027 ms/op 1.55
ssz.Root.equals 48.206 ns/op 42.008 ns/op 1.15
byteArrayEquals 47.387 ns/op 41.144 ns/op 1.15
Buffer.compare 10.908 ns/op 8.4980 ns/op 1.28
processSlot - 1 slots 12.055 us/op 8.5920 us/op 1.40
processSlot - 32 slots 2.7663 ms/op 1.9488 ms/op 1.42
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 58.968 ms/op 55.735 ms/op 1.06
getCommitteeAssignments - req 1 vs - 250000 vc 2.3659 ms/op 1.8120 ms/op 1.31
getCommitteeAssignments - req 100 vs - 250000 vc 4.5290 ms/op 3.5737 ms/op 1.27
getCommitteeAssignments - req 1000 vs - 250000 vc 4.7991 ms/op 3.7710 ms/op 1.27
findModifiedValidators - 10000 modified validators 817.66 ms/op 718.93 ms/op 1.14
findModifiedValidators - 1000 modified validators 854.68 ms/op 669.57 ms/op 1.28
findModifiedValidators - 100 modified validators 313.59 ms/op 170.02 ms/op 1.84
findModifiedValidators - 10 modified validators 247.89 ms/op 128.80 ms/op 1.92
findModifiedValidators - 1 modified validators 214.12 ms/op 156.53 ms/op 1.37
findModifiedValidators - no difference 209.12 ms/op 136.54 ms/op 1.53
compare ViewDUs 6.9279 s/op 5.9804 s/op 1.16
compare each validator Uint8Array 1.0465 s/op 1.6623 s/op 0.63
compare ViewDU to Uint8Array 1.2779 s/op 724.16 ms/op 1.76
migrate state 1000000 validators, 24 modified, 0 new 927.11 ms/op 845.56 ms/op 1.10
migrate state 1000000 validators, 1700 modified, 1000 new 1.3027 s/op 1.1198 s/op 1.16
migrate state 1000000 validators, 3400 modified, 2000 new 1.4157 s/op 1.1789 s/op 1.20
migrate state 1500000 validators, 24 modified, 0 new 1.0737 s/op 867.92 ms/op 1.24
migrate state 1500000 validators, 1700 modified, 1000 new 1.3509 s/op 1.0612 s/op 1.27
migrate state 1500000 validators, 3400 modified, 2000 new 1.5186 s/op 1.1891 s/op 1.28
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 4.6600 ns/op 5.6900 ns/op 0.82
state getBlockRootAtSlot - 250000 vs - 7PWei 589.70 ns/op 442.33 ns/op 1.33
naive computeProposerIndex 100000 validators 53.024 ms/op 41.385 ms/op 1.28
computeProposerIndex 100000 validators 9.9181 ms/op 6.1564 ms/op 1.61
naiveGetNextSyncCommitteeIndices 1000 validators 8.3777 s/op 5.7172 s/op 1.47
getNextSyncCommitteeIndices 1000 validators 257.84 ms/op 192.31 ms/op 1.34
naiveGetNextSyncCommitteeIndices 10000 validators 8.4992 s/op 5.5743 s/op 1.52
getNextSyncCommitteeIndices 10000 validators 294.90 ms/op 194.74 ms/op 1.51
naiveGetNextSyncCommitteeIndices 100000 validators 8.9926 s/op 7.2194 s/op 1.25
getNextSyncCommitteeIndices 100000 validators 565.14 ms/op 204.71 ms/op 2.76
naive computeShuffledIndex 100000 validators 32.084 s/op 21.875 s/op 1.47
cached computeShuffledIndex 100000 validators 662.23 ms/op 459.63 ms/op 1.44
naive computeShuffledIndex 2000000 validators 509.07 s/op 414.02 s/op 1.23
cached computeShuffledIndex 2000000 validators 45.187 s/op 24.016 s/op 1.88
computeProposers - vc 250000 11.797 ms/op 8.0682 ms/op 1.46
computeEpochShuffling - vc 250000 44.772 ms/op 33.186 ms/op 1.35
getNextSyncCommittee - vc 250000 290.31 ms/op 115.04 ms/op 2.52
computeSigningRoot for AttestationData 27.589 us/op 22.669 us/op 1.22
hash AttestationData serialized data then Buffer.toString(base64) 1.5774 us/op 1.1308 us/op 1.39
toHexString serialized data 1.2451 us/op 878.06 ns/op 1.42
Buffer.toString(base64) 146.75 ns/op 110.62 ns/op 1.33
nodejs block root to RootHex using toHex 137.89 ns/op 105.76 ns/op 1.30
nodejs block root to RootHex using toRootHex 84.775 ns/op 61.694 ns/op 1.37
browser block root to RootHex using the deprecated toHexString 252.14 ns/op 169.02 ns/op 1.49
browser block root to RootHex using toHex 171.83 ns/op 140.64 ns/op 1.22
browser block root to RootHex using toRootHex 157.70 ns/op 133.37 ns/op 1.18

by benchmarkbot/action

@nazarhussain nazarhussain changed the title chore: update the test runner and tests tasks chore: update the test runner and browser engine Jan 28, 2025
@nazarhussain nazarhussain marked this pull request as ready for review January 28, 2025 15:14
@wemeetagain
Copy link
Member

wemeetagain commented Jan 29, 2025

does your vitest-in-process-pool work with vitest 3.x?

@nazarhussain
Copy link
Contributor Author

nazarhussain commented Jan 29, 2025

does your vitest-in-process-pool work with vitest 3.x?

No there is some issue that is causing it to not be compatible. Was looking into it ChainSafe/benchmark#29. Will upgrade it very soon.

@codecov
Copy link

codecov bot commented Jan 29, 2025 •
edited
Loading

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 50.17%. Comparing base (b2ebe49) to head (2e4ae01).
Report is 2 commits behind head on unstable.

Additional details and impacted files 
@@             Coverage Diff              @@
##           unstable    #7404      +/-   ##
============================================
- Coverage     50.45%   50.17%   -0.29%     
============================================
  Files           603      603              
  Lines         40661    40429     -232     
  Branches       2226     2225       -1     
============================================
- Hits          20517    20284     -233     
- Misses        20104    20105       +1     
  Partials         40       40

@nazarhussain nazarhussain changed the title chore: update the test runner and browser engine chore: update the test runner Jan 30, 2025
@wemeetagain
Copy link
Member

wemeetagain commented Jan 30, 2025

does this PR need vitest 3.x? can we maintain 2.x for now?

@nazarhussain
Copy link
Contributor Author

nazarhussain commented Jan 31, 2025

does this PR need vitest 3.x? can we maintain 2.x for now?

The Vitest 3 comes with a lot of changes, but for our interests are:

Vite 6 is considered more performant (specially for frontend stack). I was more interested in workspaces and started this PR because of it, but could not get that fit in our workflow. One issue running tests from individual package folder.

So nothing critical for us, so can hold this PR merge or merge it. As you feel better.

@nazarhussain
Copy link
Contributor Author

nazarhussain commented Feb 27, 2025

We were holding up this PR until vitest-in-process-pool supports Vitest 3, which is now done. So it's safe to merge this PR.

@wemeetagain wemeetagain merged commit 51efa3d into unstable Feb 27, 2025
20 checks passed
@wemeetagain wemeetagain deleted the nh/test-update branch February 27, 2025 15:27
@wemeetagain
Copy link
Member

wemeetagain commented Mar 19, 2025

🎉 This PR is included in v1.28.0 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@wemeetagain wemeetagain wemeetagain approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@nazarhussain @wemeetagain