Use mujoco bindings instead of mujoco_py #2595

rodrigodelazcano · 2022-02-04T16:14:29Z

Changes made:

Create Viewer() class to render window in "human" mode with dm_viewer and glfw
Modified the default viewer_setup() method for all mujoco_environments (only for v3 envs)

rodrigodelazcano · 2022-02-04T16:15:16Z

gym/envs/mujoco/ant_v3.py

    def _get_obs(self):
-        position = self.sim.data.qpos.flat.copy()
-        velocity = self.sim.data.qvel.flat.copy()
+        position = self.sim.position().flat.copy()


This are minor changes. I felt the attributes are more accessible

jkterry1 · 2022-02-04T16:38:31Z

@ikostrikov could you please take a look at this?

ikostrikov · 2022-02-04T17:19:51Z

@jkterry1 @rodrigodelazcano

I ran the tests here:
https://github.com/ikostrikov/gym_dmc

It passes all of them. But I would also suggest running deterministic training to verify that results are identical (or at least similar enough). Right now, my training code is non-deterministic. It will be very useful if you know someone who could test that.

However, the viewer doesn't work for me on v2 versions of the environments. It works on v3, though. Tested on mac with m1 and ubuntu. Here is a script to reproduce the issue:

import gym

env = gym.make('HalfCheetah-v2')

env.reset()

env.render()

Error:

Traceback (most recent call last):
  File "/Users/kostrikov/GitHub/gym_dmc/tmp.py", line 16, in <module>
    env.render()
  File "/Users/kostrikov/miniconda3/envs/py39/lib/python3.9/site-packages/gym/core.py", line 269, in render
    return self.env.render(mode, **kwargs)
  File "/Users/kostrikov/miniconda3/envs/py39/lib/python3.9/site-packages/gym/core.py", line 269, in render
    return self.env.render(mode, **kwargs)
  File "/Users/kostrikov/miniconda3/envs/py39/lib/python3.9/site-packages/gym/envs/mujoco/mujoco_env.py", line 161, in render
    self.viewer_setup()
  File "/Users/kostrikov/miniconda3/envs/py39/lib/python3.9/site-packages/gym/envs/mujoco/half_cheetah.py", line 39, in viewer_setup
    self.viewer.cam.distance = self.model.stat.extent * 0.5
AttributeError: 'WindowViewer' object has no attribute 'cam'

One minor nitpick (feel free to ignore):
position/velocity naming might be confusing.

qpos/qvel in mujoco corresponds to joint position/velocity. At the same time, xpos/xvel corresponds to global cartesian position/velocity.

rodrigodelazcano · 2022-02-04T17:34:56Z

@jkterry1 @rodrigodelazcano

I ran the tests here: https://github.com/ikostrikov/gym_dmc

It passes all of them. But I would also suggest running deterministic training to verify that results are identical (or at least similar enough). Right now, my training code is non-deterministic. It will be very useful if you know someone who could test that.

However, the viewer doesn't work for me on v2 versions of the environments. It works on v3, though. Tested on mac with m1 and ubuntu. Here is a script to reproduce the issue:
import gym

env = gym.make('HalfCheetah-v2')

env.reset()

env.render()
Error:
Traceback (most recent call last):
  File "/Users/kostrikov/GitHub/gym_dmc/tmp.py", line 16, in <module>
    env.render()
  File "/Users/kostrikov/miniconda3/envs/py39/lib/python3.9/site-packages/gym/core.py", line 269, in render
    return self.env.render(mode, **kwargs)
  File "/Users/kostrikov/miniconda3/envs/py39/lib/python3.9/site-packages/gym/core.py", line 269, in render
    return self.env.render(mode, **kwargs)
  File "/Users/kostrikov/miniconda3/envs/py39/lib/python3.9/site-packages/gym/envs/mujoco/mujoco_env.py", line 161, in render
    self.viewer_setup()
  File "/Users/kostrikov/miniconda3/envs/py39/lib/python3.9/site-packages/gym/envs/mujoco/half_cheetah.py", line 39, in viewer_setup
    self.viewer.cam.distance = self.model.stat.extent * 0.5
AttributeError: 'WindowViewer' object has no attribute 'cam'
One minor nitpick (feel free to ignore): position/velocity naming might be confusing.

qpos/qvel in mujoco corresponds to joint position/velocity. At the same time, xpos/xvel corresponds to global cartesian position/velocity.

Hi @ikostrikov. Thank you for your feedback. I actually haven't updated the v2 environments for the new render since I though they will be deprecated. Should I update them? Won't take me long

ikostrikov · 2022-02-04T17:38:12Z

@rodrigodelazcano if it doesn't take too much time, it will be very useful. Many codebases still use v2 by default even though they are deprecated.

rodrigodelazcano · 2022-02-04T17:51:49Z

@rodrigodelazcano if it doesn't take too much time, it will be very useful. Many codebases still use v2 by default even though they are deprecated.

Done :)

ikostrikov · 2022-02-05T18:32:47Z

gym/envs/mujoco/humanoid.py

-        self.viewer.cam.elevation = -20
+        self.viewer.set_free_camera_settings(
+            trackbodyid=2,
+            distance=self.model.stat.extent * 1.0,


Should it be trackbodyid=1 here?

gym/envs/mujoco/humanoidstandup.py

ikostrikov · 2022-02-05T18:33:53Z

gym/envs/mujoco/mujoco_env.py

    )

-DEFAULT_SIZE = 500
+DEFAULT_SIZE = 480


Is the change from 500 to 480 necessary?

gym/envs/mujoco/mujoco_env.py

ikostrikov · 2022-02-05T18:37:04Z

@rodrigodelazcano, thanks again for your work on the viewer! I did another pass on the commits. Please take a look.

Also, could you squash all commits into one after reviewing my comments?

Thanks!

rodrigodelazcano · 2022-02-06T20:02:31Z

Thanks for helping with the review @ikostrikov. I made all the changes, if you can have a final look at it that will be great.

ikostrikov · 2022-02-06T20:24:22Z

@rodrigodelazcano it looks great now! Thanks a lot!

jkterry1 · 2022-02-25T22:29:55Z

@ikostrikov can you please review this PR again rodrigo made a bunch of changes

jkterry1 · 2022-02-26T18:24:10Z

@JamesKCS could you please make sure your issues with ant are properly resolved here too?

JamesKCS · 2022-02-28T23:09:38Z

@JamesKCS could you please make sure your issues with ant are properly resolved here too?

@jkterry1 Thank you for looking into this. I tried to check, but I think I am doing something wrong. I made a new conda environment named test_mujoco_change, activated the environment and ran
~/anaconda3/envs/test_mujoco_change/bin/pip install git+git://github.com/rodrigodelazcano/gym#egg=gym[mujoco] as directed in the other issue,
but the line env = gym.make('Ant-v2') results in the no module named 'mujoco_py' error below (and Ant-v4 gives a similar error). I think I am maybe installing/testing the wrong thing? Thank you again.

/home/james/anaconda3/envs/test_mujoco_change/lib/python3.7/site-packages/gym/envs/registration.py:506: UserWarning: WARN: The environment Ant-v2 is out of date. You should consider upgrading to version `v4` with the environment ID `Ant-v4`.
  f"The environment {path} is out of date. You should consider "
Traceback (most recent call last):
  File "main.py", line 2, in <module>
    env = gym.make('Ant-v2')
  File "/home/james/anaconda3/envs/test_mujoco_change/lib/python3.7/site-packages/gym/envs/registration.py", line 676, in make
    return registry.make(id, **kwargs)
  File "/home/james/anaconda3/envs/test_mujoco_change/lib/python3.7/site-packages/gym/envs/registration.py", line 520, in make
    return spec.make(**kwargs)
  File "/home/james/anaconda3/envs/test_mujoco_change/lib/python3.7/site-packages/gym/envs/registration.py", line 139, in make
    cls = load(self.entry_point)
  File "/home/james/anaconda3/envs/test_mujoco_change/lib/python3.7/site-packages/gym/envs/registration.py", line 55, in load
    mod = importlib.import_module(mod_name)
  File "/home/james/anaconda3/envs/test_mujoco_change/lib/python3.7/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/james/anaconda3/envs/test_mujoco_change/lib/python3.7/site-packages/gym/envs/mujoco/__init__.py", line 15, in <module>
    from gym.envs.mujoco.pusher import PusherEnv
  File "/home/james/anaconda3/envs/test_mujoco_change/lib/python3.7/site-packages/gym/envs/mujoco/pusher.py", line 5, in <module>
    import mujoco_py
ModuleNotFoundError: No module named 'mujoco_py'

ikostrikov · 2022-03-01T20:17:06Z

@jkterry1 I apologize for the slow reply. I'm getting the same error as above.

  File "/home/kostrikov/miniconda3/envs/test_mujoco39/lib/python3.9/site-packages/gym/envs/mujoco/__init__.py", line 15, in <module>
    from gym.envs.mujoco.pusher import PusherEnv
  File "/home/kostrikov/miniconda3/envs/test_mujoco39/lib/python3.9/site-packages/gym/envs/mujoco/pusher.py", line 5, in <module>
    import mujoco_py
ModuleNotFoundError: No module named 'mujoco_py'

rodrigodelazcano · 2022-03-01T20:31:20Z

@JamesKCS , @ikostrikov and @jkterry1, sorry for that. I removed the dependency for mujoco_py since it will be deprecated in the future. In the last commit I recovered it, please try now. Also, @JamesKCS, the contact force issue has only been solved for version 4 environments (Ant-v4)

ikostrikov · 2022-03-01T20:34:21Z

@rodrigodelazcano

PusherEnv does not seem to depend on mujoco_py in any way

gym/gym/envs/mujoco/pusher.py

Line 5 in 78d2b51

import mujoco_py

Is it possible to remove this line (and mujoco_py dependency as well)?

rodrigodelazcano · 2022-03-01T20:37:50Z

Should mujoco_py dependency be removed despite keeping the past versions of mujoco environments?

ikostrikov · 2022-03-01T20:39:46Z

@rodrigodelazcano one potential solution is to have both gym[mujoco] and gym[dm_control], and provide an explanation in the readme + add a warning that gym[mujoco] is deprecated. @jkterry1 what do you think?

jkterry1 · 2022-03-02T16:03:41Z

@rodrigodelazcano I like ikostrikov's proposal, lets do that

jkterry1 · 2022-03-02T16:05:25Z

Instead of saying mujoco-py is deprecated, just say it's unmaintained. We aren't going to/can't drop support for older mujoco-py based versions for the foreseeable future, it's deprecated may not be the best description.

jkterry1 · 2022-03-02T16:06:47Z

Also @rodrigodelazcano , this PR doesn't overwrite the removals in this PR right? #2651

jkterry1 · 2022-03-02T20:03:44Z

Costa ran some basic experiments:

https://wandb.ai/costa-huang/cleanRL/reports/MuJoCo-v2-vs-v4-environments--VmlldzoxNjM1OTAx

The changes appear to be within acceptable deviations for a version bump

vwxyzjn · 2022-03-02T20:08:43Z

Per @jkterry1's suggestion, I started running some benchmarks to see if there are breaking performance changes.

Report here: https://wandb.ai/costa-huang/cleanRL/reports/MuJoCo-v2-vs-v4-environments--VmlldzoxNjM1OTAx

Code here: https://github.com/vwxyzjn/validate-new-gym-mujoco-envs

So... ya they may be different (I personally find mujoco envs shows greater variance with random seeds, and that the means of the curve roughly fall into the min/max range of other curves), c'est la vie" @jkterry1

ikostrikov · 2022-03-02T20:10:25Z

@jkterry1 @vwxyzjn this is unexpected. Can we chat via some messenger?

jkterry1 · 2022-03-02T20:13:39Z

@ikostrikov email me at [email protected], I'll give you my discord information. Basically the people involved in gym are coordinated over discord.

jkterry1 · 2022-04-14T15:35:51Z

For future travelers, after some experimentation between cost and rodrigo, the cause for the performance changes was that we fixed the very long standing bugs with contact forces being zero in ant and walker2d.

* fix impulse sample bug and add adjustable gravity * revert weird sample thing * assertation * add wind * fix wind to use applyforce * black * add tests for wind * fix test bug * run lunar lander tests over 10 runs * reduce wind for heuristic landing * made wind power kwarg * reduce wind power for heuristic landing * update doc * remove wind power from kwarg * fix tests by deleting tests ;) * Add back wind power as kwarg * the black sheep * remove rogue plus

* first commit domain randomize * black * update doc * add some type hints and internalized some functions * we were told, the black bear is innocent; but I should not like to trust myself wit him * Don't need two color conventions * don't multiply twice * hardcore -> domain_randomize & register * remove rogue decorator

… the variable names (openai#2755)

* fix impulse sample bug and add adjustable gravity * revert weird sample thing * assertation * add wind * fix wind to use applyforce * black * add tests for wind * fix test bug * run lunar lander tests over 10 runs * reduce wind for heuristic landing * made wind power kwarg * reduce wind power for heuristic landing * update doc * remove wind power from kwarg * fix tests by deleting tests ;) * Add back wind power as kwarg * the black sheep * remove rogue plus * slight doc fix and variable rename * fix bug with track turn indicators * black

pseudo-rnd-thoughts · 2022-04-17T15:06:49Z

Looking ahead, there are two other large PR (#2671 and #2752) happening that will affect this PR as those updates will have to be replicated over to this PR, i.e. render API and two done API
@rodrigodelazcano Would you prefer to try and merge this PR first then make a subsequent PR to make the changes
Or make the changes in this PR and merge it after #2671 and #2752?

vwxyzjn · 2022-04-17T15:25:32Z

I would prefer to merge this PR and make a release if possible. #2671 and #2752 are much larger refactoring and could break existing code.

@rodrigodelazcano let me know if you have finalized the changes. I would be happy to do another round of benchmarks since contact force setting is changed.

* refactoring play function. Tests for keys to action mapping. * Add mocking pygame events. * partial event processing in class. * pre-commit. * quit pygame after tests. * fix typos in functions names. * Add type hint. * Add test for play function. * remove mockKeyEvent. * remove unused main code. * Adding type hints. * catch custom exception in tests. * Fix magic numbers. * Add test with an actual environment. * fix comment. * Add TODO memo on env.render. * change map with list comprehension. * remove unused imports. * Add type hint. * typo. * docstring.

rodrigodelazcano · 2022-04-18T16:48:10Z

@vwxyzjn the past benchmarks should be enough. I'm trying to fix the CI to allow mujoco envs to be tested

mujoco_bindings mujoco bindings mujoco bindings mujoco bindings dm_control for mujoco envs dm for mj_py mujoco v4 move viewer lint mujoco dependency remove mujoco_py pusherEnv setup dm_control contact forces remove print setup mujoco bindings no rendering rendering viewer_setup dependencies rebase fix dependencies pre-commit fix ping mujoco version fix mujoco test

rodrigodelazcano · 2022-04-19T02:22:41Z

@jkterry1 and @vwxyzjn, just finished updating the CI. I had to skip the rendering test of the old mujoco_py env versions, the past error was deriving from mujoco_py. Please let me know if anymore changes are required

pseudo-rnd-thoughts · 2022-04-19T10:22:34Z

@rodrigodelazcano Does that mean that we don't test the old environment can render or can old environment just not render anymore. I just want to check for backward compatibility

rodrigodelazcano · 2022-04-19T17:06:55Z

@pseudo-rnd-thoughts it only means that the old environments are not being tested for rendering (new v4 envs are), because they were being also skipped on past releases and I not able to find the issue of them not passing test in headless systems .

However, the rendering implementation for old mujoco versions is still the same and should work as usual.

rodrigodelazcano · 2022-04-19T17:22:35Z

This PR has been moved to #2762

rodrigodelazcano commented Feb 4, 2022

View reviewed changes

ikostrikov reviewed Feb 5, 2022

View reviewed changes

rodrigodelazcano force-pushed the master branch from 984ddfc to 04ee74f Compare February 6, 2022 19:59

jkterry1 mentioned this pull request Feb 25, 2022

Contact forces are still zero in Ant-v2 #2593

Closed

rodrigodelazcano force-pushed the master branch 2 times, most recently from 05b0138 to 520a398 Compare February 25, 2022 20:05

rodrigodelazcano force-pushed the master branch from 61ffb3b to e603a8f Compare March 1, 2022 20:28

readd contact forces

75c7fbc

jkterry1 mentioned this pull request Apr 15, 2022

Are Half Cheetah and other Mujoco environments deterministic ? #1193

Closed

younik and others added 4 commits April 15, 2022 11:03

fix openai#2723 and remove unuseful board (openai#2754)

1a3a891

Made documentation for carracing more descriptive, and renamed one of…

da7b8ae

… the variable names (openai#2755)

rodrigodelazcano changed the title ~~Use dm_control instead of mujoco_py~~ Use mujoco bindings instead of mujoco_py Apr 15, 2022

rodrigodelazcano added 8 commits April 18, 2022 22:02

auto gl backend

f7a1f43

new mujoco release dep

bb603ea

update pusher v4

4060922

remove contact forces mujoco

fab4ffb

readd contact forces

45330b4

skip rendering mujoco_py test

8a1b425

Merge branch 'master' of https://github.com/rodrigodelazcano/gym

906789e

rodrigodelazcano mentioned this pull request Apr 19, 2022

Change mujoco_py bindings for mujoco Deepmind bindings #2762

Merged

10 tasks

jkterry1 closed this Apr 19, 2022

Kallinteris-Andreas mentioned this pull request May 3, 2023

mujoco-v5 initial commit Farama-Foundation/Gymnasium-Robotics#104

Closed

34 tasks

Kallinteris-Andreas mentioned this pull request Jun 30, 2023

Add MuJoCo v5 environments Farama-Foundation/Gymnasium#572

Merged

35 tasks

pseudo-rnd-thoughts mentioned this pull request Feb 26, 2024

[Question] Are results on mujoco games v3 and v4 comparable Farama-Foundation/Gymnasium#936

Closed

Use mujoco bindings instead of mujoco_py #2595

Use mujoco bindings instead of mujoco_py #2595

Uh oh!

Conversation

rodrigodelazcano commented Feb 4, 2022

Uh oh!

rodrigodelazcano Feb 4, 2022

Choose a reason for hiding this comment

Uh oh!

jkterry1 commented Feb 4, 2022

Uh oh!

ikostrikov commented Feb 4, 2022

Uh oh!

rodrigodelazcano commented Feb 4, 2022

Uh oh!

ikostrikov commented Feb 4, 2022

Uh oh!

rodrigodelazcano commented Feb 4, 2022

Uh oh!

ikostrikov Feb 5, 2022

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ikostrikov Feb 5, 2022

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ikostrikov commented Feb 5, 2022

Uh oh!

rodrigodelazcano commented Feb 6, 2022

Uh oh!

ikostrikov commented Feb 6, 2022

Uh oh!

jkterry1 commented Feb 25, 2022

Uh oh!

jkterry1 commented Feb 26, 2022

Uh oh!

JamesKCS commented Feb 28, 2022

Uh oh!

ikostrikov commented Mar 1, 2022

Uh oh!

rodrigodelazcano commented Mar 1, 2022

Uh oh!

ikostrikov commented Mar 1, 2022

Uh oh!

rodrigodelazcano commented Mar 1, 2022

Uh oh!

ikostrikov commented Mar 1, 2022

Uh oh!

jkterry1 commented Mar 2, 2022

Uh oh!

jkterry1 commented Mar 2, 2022

Uh oh!

jkterry1 commented Mar 2, 2022

Uh oh!

jkterry1 commented Mar 2, 2022

Uh oh!

vwxyzjn commented Mar 2, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ikostrikov commented Mar 2, 2022

Uh oh!

jkterry1 commented Mar 2, 2022

Uh oh!

jkterry1 commented Apr 14, 2022

Uh oh!

pseudo-rnd-thoughts commented Apr 17, 2022

Uh oh!

vwxyzjn commented Apr 17, 2022

Uh oh!

rodrigodelazcano commented Apr 18, 2022

Uh oh!

rodrigodelazcano commented Apr 19, 2022

Uh oh!

pseudo-rnd-thoughts commented Apr 19, 2022

Uh oh!

rodrigodelazcano commented Apr 19, 2022

Uh oh!

rodrigodelazcano commented Apr 19, 2022

Uh oh!

vwxyzjn commented Mar 2, 2022 •

edited

Loading