Skip to content

Commit 6e298d8

Browse files
committed
Set kv cache size to f16 by default
1 parent 9c8c2c3 commit 6e298d8

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

llama_cpp/llama.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@ def __init__(
2121
n_ctx: int = 512,
2222
n_parts: int = -1,
2323
seed: int = 1337,
24-
f16_kv: bool = False,
24+
f16_kv: bool = True,
2525
logits_all: bool = False,
2626
vocab_only: bool = False,
2727
use_mmap: bool = True,

0 commit comments

Comments
 (0)