compatibility with Gym 0.26.0

[markub3327]

Hi @erwincoumans,

for fixing the issues #4331 and openai/gym#3073, I must add order_enforce=False flag to all register() at init.py becouse new version of Gym doesn't allow call env.render() before calling env.reset().

Thanks for your time.

[markub3327]

For compatibility with Gym 0.26.0 is needed to change the:
if id in registry.env_specs:
To:
if id in registry.keys():

Thanks.

[markub3327]

For compatibility with Gym 0.26.0 and newer, I must change all step() and reset(), too.

The new form of step() is:

observation, reward, terminated, truncated, info = env.step(action)

The new form of reset() is:

observation, info = env.reset()

The render or renders argument in __init__() in all envs is changed to render_mode.

@erwincoumans Please can you do tests of the correctness?

[erwincoumans]

Thanks for the pull request. Would this change also make PyBullet incompatible with all older versions of Gym? Is there some gym version check possible? Or remove the order enforcing wrapper after make externally?

[markub3327]

There are more changes than the wrapper, because Gym 0.26.0 use new form of env.reset() and env.step().

It will be incompatible with the older Gym..... But checking the Gym version can solve this problem.

Thanks.

[anht-nguyen]

When will these changes for compatibility with Gym 0.26 be merged to the master branch? Thanks!

[araffin]

Fyi, I pushed and released on pypi a subset of pybullet envs compatible with gymnasium: https://github.com/araffin/pybullet_envs_gymnasium
pip install pybullet_envs_gymnasium (I will crosspost this message as it may interest several people)