FSD Evaluation on SimplerEnv

From Seeing to Doing: Bridging Reasoning and Decision for Robotic Manipulation

[🌐 Website] [📄 Paper] [🤗 Models] [🎯 Datasets] [💬 Demo]

---

This repository is used to evaluate the FSD performance on bridge tasks of SimplerEnv.

💿 Installation

Clone this repo:

git clone --recurse-submodules https://github.com/hilookas/SimplerEnv -b fsd
cd SimplerEnv

Create an anaconda environment:

conda create -n simpler_env python=3.10
conda activate simpler_env

Install SimplerEnv:

# Following the instructions <https://github.com/simpler-env/SimplerEnv#installation>

pip install numpy==1.24.4

pushd ManiSkill2_real2sim
pip install -e .
popd

pip install -e .

Install GraspNet:

# Following the instructions in GSNet/README.md

conda install openblas-devel -c anaconda
pushd MinkowskiEngine
python setup.py install --blas_include_dirs=${CONDA_PREFIX}/include --blas=openblas
popd

pushd GSNet

pushd pointnet2
python setup.py install
popd

pushd knn
python setup.py install
popd

popd

pushd graspnetAPI
pip install .
popd

🏃 Execution

You can run the evaluation using:

bash scripts/fsd_bridge.sh

🙏 Acknowledgments

We sincerely thank the following open-source projects and research works:

• SimplerEnv-SOFAR
• ManiSkill2_real2sim
• graspnetAPI
• graspness_unofficial / graspnet-baseline
• MinkowskiEngine
• pytorch3d

📚 Citation

If you use FSD in your research, please cite our paper:

@misc{yuan2025seeingdoingbridgingreasoning,
      title={From Seeing to Doing: Bridging Reasoning and Decision for Robotic Manipulation}, 
      author={Yifu Yuan and Haiqin Cui and Yibin Chen and Zibin Dong and Fei Ni and Longxin Kou and Jinyi Liu and Pengyi Li and Yan Zheng and Jianye Hao},
      year={2025},
      eprint={2505.08548},
      archivePrefix={arXiv},
      primaryClass={cs.RO},
      url={https://arxiv.org/abs/2505.08548}, 
}