GaussianAvatar: Towards Realistic Human Avatar Modeling from a Single Video via Animatable 3D Gaussians

Liangxiao Hu¹, Hongwen Zhang², Yuxiang Zhang³, Boyao Zhou³, Boning Liu³, Shengping Zhang¹, Liqiang Nie¹,

¹Harbin Institute of Technology ²Beijing Normal University ³Tsinghua University

Projectpage · Paper · Video

📣 Updates

[23/1/2024] Training and inference codes for People Snapshot are released.

Introduction

We present GaussianAvatar, an efficient approach to creating realistic human avatars with dynamic 3D appearances from a single video.

Installation

To deploy and run GaussianAvatar, run the following scripts:

conda env create --file environment.yml
conda activate gs-avatar

Then, compile diff-gaussian-rasterization and simple-knn as in 3DGS repository.

Download models and data

• SMPL/SMPL-X model: register and download SMPL and SMPL-X, and put these files in assets/smpl_files. The folder should have the following structure:

smpl_files
 └── smpl
   ├── SMPL_FEMALE.pkl
   ├── SMPL_MALE.pkl
   └── SMPL_NEUTRAL.pkl
 └── smplx
   ├── SMPLX_FEMALE.npz
   ├── SMPLX_MALE.npz
   └── SMPLX_NEUTRAL.npz

• Data: download the provided data from OneDrive. These data include assets.zip, gs_data.zip and pretrained_models.zip. Please unzip assets.zip to the corresponding folder in the repository and unzip others to gs_data_path and pretrained_models_path.

Run on People Snapshot dataset

We take the subject m4c_processed for example.

Training

python train.py -s $gs_data_path/m4c_processed -m output/m4c_processed --train_stage 1

Evaluation

python eval.py -s $gs_data_path/m4c_processed -m output/m4c_processed --epoch 200

Rendering novel pose

python render_novel_pose.py -s $gs_data_path/m4c_processed -m output/m4c_processed --epoch 200

Todo

• Release the reorganized code and data.
• Provide the scripts for your own video.
• Provide the code for real-time annimation.

Citation

If you find this code useful for your research, please consider citing:

@article{hu2023gaussianavatar,
  title={GaussianAvatar: Towards Realistic Human Avatar Modeling from a Single Video via Animatable 3D Gaussians},
  author={Hu, Liangxiao and Zhang, Hongwen and Zhang, Yuxiang and Zhou, Boyao and Liu, Boning and Zhang, Shengping and Nie, Liqiang},
  journal={arXiv preprint arXiv:2312.02134},
  year={2023}
}

Acknowledgements

This project is built on source codes shared by Gaussian-Splatting, POP, HumanNeRF and InstantAvatar.