Skip to content

Data cleaning scripts for dataset release #2440

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merrymercy merged 18 commits into from
Sep 18, 2023
Merged

Data cleaning scripts for dataset release #2440

Changes from 1 commit
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
fcb3a66
exclude some models
merrymercy Sep 6, 2023
e4e8fed
update docs
merrymercy Sep 6, 2023
e604b9e
update identity words
merrymercy Sep 8, 2023
a297dfe
Fix bugs & dates
merrymercy Sep 11, 2023
5db338b
move files
merrymercy Sep 12, 2023
5050adb
update conv release scripts
merrymercy Sep 12, 2023
5099a0e
update compute_stats.py
merrymercy Sep 12, 2023
45d1e1e
update clean scripts
merrymercy Sep 13, 2023
bf72657
improve scripts
merrymercy Sep 13, 2023
4c7edd5
update processing
merrymercy Sep 14, 2023
2708ad6
fix format
merrymercy Sep 14, 2023
1286322
parallelize filtering
merrymercy Sep 14, 2023
1bffba8
Fix pipeline
merrymercy Sep 15, 2023
4521b13
update stats
merrymercy Sep 16, 2023
ac0781d
update
merrymercy Sep 16, 2023
69978f5
update
merrymercy Sep 17, 2023
531925f
Intersect of conv files
merrymercy Sep 17, 2023
62f1674
format
merrymercy Sep 18, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
update
  • Loading branch information
@merrymercy
merrymercy committed Sep 18, 2023
commit ac0781d25f4173ff29ecdf3f1ee3359a0c46f449
2 changes: 1 addition & 1 deletion fastchat/serve/monitor/dataset_release_scripts/lmsys_chat_1m/compute_stats.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@
print(f"#turns: {df['turn'].mean()}")

model_counts = df["model"].value_counts() * scale
print("model counts", model_counts)
#print("model counts", model_counts)
fig = px.bar(x=model_counts.index, y=model_counts)
fig.update_layout(
xaxis_title=None,
Expand Down