add MaMuJoCo-v1 environments

[Kallinteris-Andreas]

Note: this may be merged post gymnasium 1.0.0

Description

This is a small release for MaMuJoCo, mainly updates to Gymnasium/MuJoCo-v5, along with fixing some bugs and cleaning up the code, optimizing the performance of some components, and taking this opportunity to strengthen the unit test cases.

MaMuJoCo-V1

• Now based on Gymnasium/MuJoCo-v5 instead of Gymnasium/MuJoCo-v4 (Add MuJoCo v5 environments Gymnasium#572).
• When factorizatoion=None, the env.gent_action_partitions.dummy_node now contains action_id (it used to be None).
• Added map_local_observations_to_global_state & optimized runtime performance of map_global_state_to_local_observations.
• Added gym_env argument, which can be used to load third-party Gymansium.MujocoEnv environments.

Ant

• Now observes local_categories of cfrc_ext by default (same as Gymnasium/MuJoCo-v5/Ant).
• Renamed global node torso → root.

Humanoid(-Standup)

• No longer observes qfrc_actuator of root & cinert, cvel, qfrc_actuator, cfrc_ext of worldbody (same as Gymnasium/MuJoCo-v5/Humanoid(-Standup)).

Walker2d

• Fixed bug: global nodes are now [root_x, root_z, root_y] (used to be [root_x, root_x, root_z]).

ManySegmentAnt

• frame_skip default set to 5 (same as Gymnasium/Ant).
• option.timestep set to 0.01 (same as Gymnasium/Ant).
• Now uses the same reward function as Gymnasium/Ant.
• Now observes cfrc_ext by default, (same as Gymnasium/MuJoCo-v5).

ManySegmentSwimmer

• Now uses the same option.timestep as Gymansum/Swimmer (0.01).
• Updated model to work with mujoco>=3.0.0.

Implementation details (not relevant for the end user)

• Removed obsk._observation_structure(), it now uses MujocoEnv.observation_structure.
• Cleaned up the generation of local & global categories.
• Removed the ManySegmentAnt and ManySegmentSwimmer classes (instead the xml_file argument is used with Gymnasium/MuJoCo/Ant-v5 and Gymnasium/MuJoCo/Swimmer-v5 frameworks).
• Cleaned up the _generate_local_categories code and removed _generate_global_categories().
• Removed the seed() function.
• Refactored & cleaned up MultiAgentMujocoEnv.__init__().
• More unit tests & extended tests!
• Now relies on gynmansium==1.0.0 API (breaks with gymnasium<1.0.0)
• Jinja2 library requirement is now optional

Note only map_global_state_to_local_observations has been reworked

Other

Type of change

Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

Checklist:

• I have run the pre-commit checks with pre-commit run --all-files (see CONTRIBUTING.md instructions to set it up)
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes

[Kallinteris-Andreas]

@rodrigodelazcano needs a review

[rodrigodelazcano]

Just some minor wording changes in the docs. Otherwise LGTM!

[Kallinteris-Andreas]

@rodrigodelazcano I applied your suggestions, and updated the readme.md