Question regarding the target dof pos used in Host

[wu-hy]

I noticed that in Host, (current position + scaled action) is used as the target dof position, whereas the standard approach (e.g., in the original Legged Gym) typically uses (default position + scaled action) as the target dof position.

Does the method used in Host offer significant advantages compared to the normal approach? Or was this choice made simply because it demonstrated better performance for this specific task (e.g., get-up) during experimentation?

[TaoHuang13]

The main reason is that this way of computing target dof pos is more compatible with our curriculum on target dof pos (action bound/action rescaler). Note that he action in this paper is normalized within [-a, a] instead of an unbounded one.

[wu-hy]

Thanks for your quick response. The equation of torque computation could be further simplified as $\tau_t = K_p \cdot \beta a_t - K_d \cdot \dot{p}_t$. The goal of the PD controller is to track a target dof position. However, the current method may not guarantee that the dof pos try to converge to the target position during 4 simulation steps (as configured with control.decimation = 4, where each action is executed for 4 steps), since the P-term of the PD controller remains fixed during these steps. Could this lead to unexpected behavior?

[TaoHuang13]

This is correct. However, it would have a negative effect, as the network will overfit to this case. See the ETH legged robot 2018 paper for more details.