Beacon committees endpoint returning incorrect data

[mcdee]

I queried /eth/v1/beacon/states/121568/committees on hoodi at the beginning of slot 121568 (which is also the beginning of epoch 3799) and received committees for the prior epoch.

committees.json

lighthouse --version
v7.0.0-beta.5-6d5a2be

This appears to be an intermittent issue; at other times I have queried the committees and received the data for the requested epoch.

[michaelsproul]

Hmm that is really odd.

This seems hard to debug remotely, I'll try to convert one of our Hoodi nodes to an archive node to see if we can repr but that might take a bit.

[michaelsproul]

Can you let us know which BN flags this node is using? Anything funky for --shuffling-cache-size? How about --hierarchy-exponents?

[mcdee]

This is a validating node, so not running as an archive.

Parameters are as per below (xxx masks information but I don't believe any of them should matter).

lighthouse beacon --log-format JSON --logfile-max-size 0 --prune-payloads false --suggested-fee-recipient=xxx --builder=xxx --builder-fallback-disable-checks --historic-state-cache-size 4 --checkpoint-sync-url=https://hoodi-checkpoint-sync.attestant.io/ --network hoodi --execution-endpoint xxx --subscribe-all-subnets --http --http-address 0.0.0.0 --http-port xxx --port xxx --port6 xxx --target-peers 64 --metrics --metrics-address 0.0.0.0 --metrics-port xxx --metrics-allow-origin xxx --disable-enr-auto-update --execution-jwt=xxx --quic-port=xxx

[mcdee]

And here is the code I used to find the issue, in case it's useful: https://github.com/mcdee/committees

[michaelsproul]

Ok, I reckon I know what it is:

• Beacon state HTTP endpoint returns head state if request slot is ahead #4041

I suspect the requests are arriving before we have a block at that slot, so we are returning the head state rather than the head state advanced to the desired slot.

As a workaround, it should be fine if you manually pass the epoch query parameter, although I realise we are massively off spec here.