Skip to content

server : separate the notion of position and KV tokens, remove prompt truncation #13576

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
ngxson wants to merge 7 commits into
base: master
Choose a base branch
from
+147 −146
Open

server : separate the notion of position and KV tokens, remove prompt truncation #13576

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
536bea4
server : separate the notion of position and KV tokens, remove prompt…
ngxson May 15, 2025
678d7b1
no more scan loop in n_kv_tokens()
ngxson May 16, 2025
f9cc9f2
fix stats report
ngxson May 16, 2025
de8956a
rm notion of n_kv_tokens
ngxson May 17, 2025
3970042
Merge branch 'master' into xsn/server_separate_pos_tokens
ngxson May 17, 2025
3012326
rm double check for out-of-context
ngxson May 17, 2025
987955f
Apply suggestions from code review
ngxson May 19, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
server : separate the notion of position and KV tokens, remove prompt… 
… truncation
  • Loading branch information
@ngxson
ngxson committed May 15, 2025
commit 536bea4c8206db24cfb2d27d0ab046227ff371b9
Loading
Loading