fix: chunked prefill update#2307

Merged

ryanolson merged 13 commits intoryan/connector-devfrom

ryan/connector-chunked

Aug 7, 2025

Commits on Aug 5, 2025

chunked prefill seems to revisit the get_num_new_matched_tokens more than once; short-circuiting on multiple invocations; will need to reset this state on eviction
ryanolson
committed
still confused about what if get_num_new_matched_tokens is indeed called twice what the behavior of update_state_after_alloc should be after the subsequent calls to get_num_new_matched_tokens
ryanolson
committed
adding counter to track the number of visits to get_num_new_matched_tokens
ryanolson
committed
bumping the logging level to warn when visitor count exceeds 1
ryanolson
committed
updates to get_num_new_matched_tokens behavior; will acquire_local_matches, but will never advance up update the sequence or block positions
ryanolson
committed
adding logging on append_mutable_device_blocks
ryanolson
committed
checkpoint
ryanolson
committed
removing some debug asserts that throw when worker does not call clear_connector_metadata on a no-op pass
ryanolson
committed

Commits on Aug 6, 2025

Commits on Aug 7, 2025