Skip to content

Fix ppo jax documentation links #332

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
vwxyzjn merged 20 commits into from
Dec 22, 2022
Merged

Fix ppo jax documentation links #332

Changes from 1 commit
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
1371bcf
ppo jax scan
poringz Dec 3, 2022
9cadde3
Merge remote-tracking branch 'vwxyzjn/master'
poringz Dec 3, 2022
58efd11
reformatting
poringz Dec 3, 2022
c2fa4ed
ppo jax scan: log only last loss metrics
poringz Dec 4, 2022
f89e832
ppo jax scan: reformatting
poringz Dec 4, 2022
b784a80
code clean up
poringz Dec 9, 2022
ce49d33
additional cleanup + reformat + compute gae test
poringz Dec 12, 2022
502de43
reformat gae test file
51616 Dec 20, 2022
d95ebb2
Merge remote-tracking branch 'vwxyzjn/master' into ppo_jax_scan
51616 Dec 20, 2022
e05fd00
documentation for ppo_jax_scan
51616 Dec 20, 2022
3af559c
shorten doc
51616 Dec 20, 2022
5f53fbe
Add docs
vwxyzjn Dec 20, 2022
3e9b834
Merge remote-tracking branch 'origin/ppo_jax_scan' into ppo_jax_scan
51616 Dec 21, 2022
8da70ae
update docs and add the jax gae test in test_envpool
51616 Dec 21, 2022
44b80c2
add png for learning curve vs wall-clock time
51616 Dec 21, 2022
94c9aa3
typo fixed
51616 Dec 21, 2022
5e765d2
change end-to-end test case
vwxyzjn Dec 21, 2022
89ec82a
Merge remote-tracking branch 'vwxyzjn/master' into ppo_jax_scan
51616 Dec 22, 2022
4f782b0
fix doc links
51616 Dec 22, 2022
0b47459
Merge remote-tracking branch 'origin/ppo_jax_scan' into ppo_jax_scan
51616 Dec 22, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
ppo jax scan: reformatting
  • Loading branch information
@poringz
poringz committed Dec 4, 2022
commit f89e8326a676fe0c9290d56dc6d1fdc7116a5959
10 changes: 5 additions & 5 deletions cleanrl/ppo_atari_envpool_xla_jax_scan.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -485,11 +485,11 @@ def rollout(agent_state, episode_stats, next_obs, next_done, key, handle, step_o
"charts/avg_episodic_length", np.mean(jax.device_get(episode_stats.returned_episode_lengths)), global_step
)
writer.add_scalar("charts/learning_rate", agent_state.opt_state[1].hyperparams["learning_rate"].item(), global_step)
writer.add_scalar("losses/value_loss", v_loss[-1,-1].item(), global_step)
writer.add_scalar("losses/policy_loss", pg_loss[-1,-1].item(), global_step)
writer.add_scalar("losses/entropy", entropy_loss[-1,-1].item(), global_step)
writer.add_scalar("losses/approx_kl", approx_kl[-1,-1].item(), global_step)
writer.add_scalar("losses/loss", loss[-1,-1].item(), global_step)
writer.add_scalar("losses/value_loss", v_loss[-1, -1].item(), global_step)
writer.add_scalar("losses/policy_loss", pg_loss[-1, -1].item(), global_step)
writer.add_scalar("losses/entropy", entropy_loss[-1, -1].item(), global_step)
writer.add_scalar("losses/approx_kl", approx_kl[-1, -1].item(), global_step)
writer.add_scalar("losses/loss", loss[-1, -1].item(), global_step)
print("SPS:", int(global_step / (time.time() - start_time)))
writer.add_scalar("charts/SPS", int(global_step / (time.time() - start_time)), global_step)
print("SPS_update:", int(args.num_envs * args.num_steps / (time.time() - update_time_start)))
Expand Down