Skip to content

PaliGemma #1636

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
mattdangerw merged 35 commits into from
May 21, 2024
Merged

PaliGemma #1636

Changes from 1 commit
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
cd7e2f8
Add VIT Encoder
divyashreepathihalli Mar 28, 2024
53119bd
Add MHAPooling layer + end to end ViT model
fchollet Mar 29, 2024
aa9bd89
Feature/pg gemma changes
VarunS1997 Apr 2, 2024
962aea7
Misc fixes
divyashreepathihalli Apr 2, 2024
f23974b
update vit model and add a test for verifying output shape.
divyashreepathihalli Apr 3, 2024
53865e0
Feature/pg gemma changes
VarunS1997 Apr 18, 2024
86540c0
Vit model weights conversion
divyashreepathihalli Apr 22, 2024
6160ebe
Update imports
divyashreepathihalli Apr 25, 2024
92e697d
Add vit attention
divyashreepathihalli Apr 27, 2024
0454f1f
add paligemma functional model
divyashreepathihalli May 3, 2024
56a95ce
Paligemma full model checkpoints conversion script
divyashreepathihalli May 4, 2024
9a46b6e
Fix ViT build issue
divyashreepathihalli May 4, 2024
2a2e6b1
Multi modal Refactor for PaliGemma
VarunS1997 May 6, 2024
b2a14d8
Export the public API surface
mattdangerw May 7, 2024
53266b1
update image size arg throughout PaliGemma
divyashreepathihalli May 7, 2024
597daaf
Update convert_paligemma_checkpoints.py
divyashreepathihalli May 7, 2024
87d6c5a
Renames for consistency
mattdangerw May 7, 2024
3ab245c
Update convert_pali_gemma_checkpoints.py
divyashreepathihalli May 7, 2024
671a161
More consistency improvements for PaliGemma
mattdangerw May 7, 2024
ffd5d98
Do the same scaling in our backbone we do for generate
mattdangerw May 7, 2024
b94b6d3
Update conversion and add cli arguments
grasskin May 7, 2024
d262569
Add presets
divyashreepathihalli May 7, 2024
caa7cef
Update pali_gemma_causal_lm_preprocesor.py to default text sequence l…
divyashreepathihalli May 8, 2024
c8f327a
Tokenizer fix
divyashreepathihalli May 8, 2024
bc3811b
Allow fit calls for pali_gemma
mattdangerw May 8, 2024
1afea65
Allow generate on unbatch input
mattdangerw May 8, 2024
cfef757
Remove the score function from pali gemma causal lm
mattdangerw May 8, 2024
0ba0920
Greedy sample by default for pali gemma
mattdangerw May 8, 2024
f4d31dd
Added docstrings for paligemma decoder and backbone
VarunS1997 May 10, 2024
e8bff89
Minor fixes for the vit
mattdangerw May 16, 2024
db0d9d1
Add a response_mask input
mattdangerw May 21, 2024
5267d5a
Update pali_gemma_presets.py path
divyashreepathihalli May 21, 2024
83ee31d
Add a tokenizer docstring for pali gemma
mattdangerw May 21, 2024
edc66e8
Update pali_gemma_presets.py
divyashreepathihalli May 21, 2024
569d89e
More consistent defaults for PaliGemma
mattdangerw May 21, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Export the public API surface
  • Loading branch information
@mattdangerw
mattdangerw committed May 21, 2024
commit b2a14d8f94d1b5ef88572269c2eded1953611784
10 changes: 10 additions & 0 deletions keras_nlp/api/models/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -159,6 +159,16 @@
)
from keras_nlp.src.models.opt.opt_preprocessor import OPTPreprocessor
from keras_nlp.src.models.opt.opt_tokenizer import OPTTokenizer
from keras_nlp.src.models.paligemma.pali_gemma_backbone import PaliGemmaBackbone
from keras_nlp.src.models.paligemma.pali_gemma_causal_lm import (
PaliGemmaCausalLM,
)
from keras_nlp.src.models.paligemma.pali_gemma_causal_lm_preprocesor import (
PaliGemmaCausalLMPreprocessor,
)
from keras_nlp.src.models.paligemma.pali_gemma_tokenizer import (
PaliGemmaTokenizer,
)
from keras_nlp.src.models.phi3.phi3_backbone import Phi3Backbone
from keras_nlp.src.models.phi3.phi3_causal_lm import Phi3CausalLM
from keras_nlp.src.models.phi3.phi3_causal_lm_preprocessor import (
Expand Down