Skip to content

use llguidance library for constraints (including json schemas) #899

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
EricLBuehler merged 29 commits into from
Dec 2, 2024
Merged

use llguidance library for constraints (including json schemas) #899

Changes from 1 commit
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
7fa4686
import llguidance modules
mmoskal Nov 6, 2024
1dcca91
llg constraint types
mmoskal Nov 6, 2024
eaf52be
integrate llguidance
mmoskal Nov 6, 2024
cc82722
fix handling of stop tokens
mmoskal Nov 6, 2024
fb83b5c
update toktrie
mmoskal Nov 6, 2024
00c36dd
remove submodules
mmoskal Nov 6, 2024
cab564b
fix version conflicts
mmoskal Nov 6, 2024
d1becbd
tok_trie -> tok_env rename
mmoskal Nov 6, 2024
f70b3de
update to latest llguidance
mmoskal Nov 30, 2024
3d22f2b
Merge branch 'master' into llg_cleanup
mmoskal Nov 30, 2024
ca9e346
sync lock
mmoskal Nov 30, 2024
7b3ae50
bump llg (lazy_static fix)
mmoskal Nov 30, 2024
146bc4c
update to latest llguidance, fix conflicts
mmoskal Nov 30, 2024
6019fee
import toktrie via llguidance
mmoskal Nov 30, 2024
dd35965
n=1
mmoskal Dec 1, 2024
3ac55ed
test with llama1b
mmoskal Dec 1, 2024
8ca7514
remove aici folder (no longer used)
mmoskal Dec 1, 2024
7967d42
use more specific type for llg grammars
mmoskal Dec 1, 2024
9a919d8
update python APIs to support json schema and llg
mmoskal Dec 1, 2024
0ad0948
update example to use lark not yacc
mmoskal Dec 1, 2024
816ac8f
rename example
mmoskal Dec 1, 2024
5fc906b
remove testing scripts
mmoskal Dec 1, 2024
5e9cbd2
re-export llguidance for easier LlguidanceGrammar construction
mmoskal Dec 2, 2024
ffcdd2a
fix formatting
mmoskal Dec 2, 2024
fda20fe
fix clippy
mmoskal Dec 2, 2024
b5add20
Merge branch 'master' into llg_cleanup
mmoskal Dec 2, 2024
2c59224
add python samples
mmoskal Dec 2, 2024
976092c
add server samples
mmoskal Dec 2, 2024
ac7c35d
add rust samples
mmoskal Dec 2, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
n=1
  • Loading branch information
@mmoskal
mmoskal committed Dec 1, 2024
commit dd359652076c4e07b2aaa2bfa04955b30d99fdcf
4 changes: 2 additions & 2 deletions scripts/test-infer.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ curl -X POST "${TRT_API_BASE}completions" \
"prompt": "Here is a one line joke: ",
"max_tokens": 100,
"temperature": 0.7,
"n": 5
"n": 1
}' | jq
;;

Expand All @@ -27,7 +27,7 @@ curl -X POST "${TRT_API_BASE}completions" \
"prompt": "Here is a one line joke: ",
"max_tokens": 100,
"temperature": 0.7,
"n": 5
"n": 1
}'
;;

Expand Down