Skip to content

Kaggle Presets #1365

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
mattdangerw merged 27 commits into from
Jan 4, 2024
Merged

Kaggle Presets #1365

Changes from 1 commit
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
62eaf03
Start landing code for Kaggle integration (#1320)
mattdangerw Nov 20, 2023
21fb04c
Switch byte pair tokenizer to save_assets/load_assets (#1322)
mattdangerw Nov 21, 2023
0e3c674
Convert SentencePieceTokenizer and associated models to new assets pa…
nkovela1 Nov 21, 2023
3619a6a
Add tests for Presets workflow, Add Metadata (#1326)
nkovela1 Nov 23, 2023
38806fd
Automatically add the keras framework to kaggle handles (#1331)
mattdangerw Nov 29, 2023
e0d34dc
Fix a failing byte pair tokenizer test (#1336)
mattdangerw Nov 30, 2023
0820d62
Use set comparison for assets (#1335)
mattdangerw Nov 30, 2023
c4b0c3c
Fix whisper tokenizer saving (#1334)
mattdangerw Nov 30, 2023
e3f8d06
Remove special case Bart from_preset (#1333)
mattdangerw Nov 30, 2023
dbb6487
Fix t5 tokenizer presets (#1339)
mattdangerw Nov 30, 2023
6130253
Script to convert presets (#1340)
mattdangerw Nov 30, 2023
814959b
Switch all preset to the new Kaggle format (#1338)
mattdangerw Dec 1, 2023
2aced24
Let kagglehub select latest version (#1342)
mattdangerw Dec 4, 2023
245b7e9
Use the proper title for example (#1346)
Philmod Dec 5, 2023
6ad8a30
Update conversion script (#1347)
mattdangerw Dec 6, 2023
7cc4323
Improve preset error messages (#1349)
mattdangerw Dec 7, 2023
9cc8110
Use subclass checking check_preset_class (#1344)
mattdangerw Dec 7, 2023
4606f32
Add a hacky fix for TF 2.13 and 2.14 weights.h5 loading (#1353)
mattdangerw Dec 7, 2023
9cb5838
Another fix for saving on Keras 2 (#1354)
mattdangerw Dec 7, 2023
039ff45
Switch our preset to there final kaggle location (#1345)
mattdangerw Dec 7, 2023
9cc3f84
Fix rebase issue in bytepair tokenizer (#1366)
nkovela1 Dec 12, 2023
6f7f9a0
Change encoding to utf-8 to fix Kaggle branch test failure for PyTorc…
sampathweb Dec 13, 2023
ddfca77
Fix GPU test issue with Keras 2 (#1368)
nkovela1 Dec 14, 2023
0e43f09
Add in-place modification of file keys for backwards compatibility (#…
nkovela1 Dec 15, 2023
4d84eb1
Add file renaming logic for modification (#1370)
nkovela1 Dec 16, 2023
29a0ae5
Fix task pre-processor in tasks (#1373)
sampathweb Dec 20, 2023
401e569
Backwards compatible fix for functional model saving (#1378)
mattdangerw Jan 4, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Use set comparison for assets (#1335) 
We were relying on the order of `os.listdir` previously, which is
not fixed I believe.

https://github.com/keras-team/keras-nlp/pull/1333/checks?check_run_id=19158555913
  • Loading branch information
@mattdangerw
mattdangerw committed Jan 4, 2024
commit 0820d62ea5db0c8caf861f3eced736db63fc7334
2 changes: 1 addition & 1 deletion keras_nlp/utils/preset_utils_test.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ def test_preset_saving(self, cls, preset_name, tokenizer_type):
"compile_config" not in config_json
) # Test on raw json to include nested keys
config = json.loads(config_json)
self.assertAllEqual(config["assets"], expected_assets)
self.assertEqual(set(config["assets"]), set(expected_assets))
self.assertEqual(config["weights"], "model.weights.h5")

# Try loading the model from preset directory
Expand Down