RobustHAR: Robust Human Activity Recognition with Smartphone–Smartwatch Fusion

A research framework for Human Activity Recognition (HAR) that studies accelerometer-gyroscope smartphone–smartwatch sensor fusion across diverse deep learning models. The code lives in Code/, with data preparation, supervised and self‑supervised training, and utilities to reproduce figures and results.

Repository Layout

RobustHAR/
├── Code/
│   ├── Loaders/            # Dataset loaders (12 public HAR datasets)
│   ├── Preparers/          # Data preparation pipeline components
│   ├── Training/           # Supervised models (CNN, TCN, LSTM, Transformer, HART, …)
│   ├── Training_SSL/       # Self-supervised (CroSSL) training
│   ├── Models/             # Saved models and metadata
│   ├── final_data/         # Prepared SL datasets (npz)
│   ├── final_data_ssl/     # Prepared SSL datasets (npy)
│   ├── prepare_data.py     # Main data prep entrypoint/config
│   ├── paper_visualizations_barchart.py
│   ├── paper_label_split_table.py
│   └── requirements.txt
├── Other/                  # Supplemental analysis (Data clipping plots, Label distribution analysis,...)
└── README.md               # You are here

Flexible and reusable HAR data preparation pipeline fusing 12 public datasets

• Single-file configuration: All stages are controlled in Code/prepare_data.py (loaders, resampling, fusion, label set, ILP split, scaling, windowing, SSL toggle). Change one place; everything else adapts.
• Minimal rebuilds: The pipeline skips completed stages; use generate_data() to force a full rebuild when you change configuration.
• Easy to repurpose: Works for related multivariate time-series tasks. Add a new loader in Code/Loaders/ and register it in PHONE_LOADERS, WATCH_LOADERS, or BOTH_LOADERS or remove existing ones to adapt to your usecase.
• Programmatic overrides: You can override config values from Python without editing logical code.

import prepare_data as cfg

cfg.MERGE = False          # e.g., train on single-device data only
cfg.FREQUENCY = 50         # resample to 50 Hz
cfg.WINDOW_SIZE = 128      # longer temporal context

data = cfg.generate_data() # rebuild with the new configuration

This design makes the data layer portable across projects: tweak the prepare_data.py config to produce new datasets (SL and SSL) without touching training code.

Quick Start

1. Create environment and install dependencies

# from repo root
cd Code
python -m venv .venv && source .venv/bin/activate  # or: conda create -n robusthar python=3.10
pip install --upgrade pip
pip install -r requirements.txt

2. Prepare data (runs the full 7‑stage pipeline on first use; writes to final_data/ and final_data_ssl/)

# from RobustHAR/Code
python prepare_data.py

Alternatively (as a library):

from prepare_data import get_data
all_data = get_data()  # returns a dict with loaded arrays; creates data if missing

3. Train models

• Supervised (examples):

# Random search (Phase 1)
python Training/CNN/CNN_FULL_random.py
python Training/HART/HART_FULL_random.py

# Grid search (Phase 2)
python Training/CNN/CNN_all_grid.py
python Training/HART/HART_all_grid.py

• Self‑supervised (CroSSL):

python Training_SSL/CroSSL/CNNCroSSL_FULL_random.py
python Training_SSL/CroSSL/CNNCroSSL_all_grid.py

Notes:

• Most models expect a (Multi-)GPU environment (except MLP run on CPU). See Code/Training/ReadMe.md for details.
• Training scripts will trigger data preparation (according to the current config file) automatically if needed.

Reproduce Paper Figures/Tables

# from RobustHAR/Code
python paper_visualizations_barchart.py      # Generates benchmark bar charts
python paper_label_split_table.py            # Generates label split table (and Paper_Figures_generated)

Common Mistakes

• Not enough memory to process all data
• Not providing the correct environment (CPU vs. GPU) for model training
• Data Preparation Pipeline config file prepare_data.py does not align with the input of the DL models when trying to train of non-default prepare_data.py.

Where to Find Detailed Docs

• Data pipeline and configuration: Code/Preparers/Readme.md
• Supervised training framework (models, search strategy, outputs): Code/Training/ReadMe.md
• SSL training (CroSSL): Code/Training_SSL/ReadMe.md
• High‑level project overview: Code/ReadMe.md

Outputs

• Prepared datasets: Code/final_data/ (*_train.npz, *_val.npz, *_test.npz) and Code/final_data_ssl/ (*.npy)
• Trained models and artifacts: Code/Models/{all|phone|watch}/ with .keras/.pt, *.txt metadata, and loss curves (*.png)

Citation

If you use this framework, please cite:

Zubairov, N., Milling, M., Schuller, B., Fleisch, E., & Langer, P. (2025). Enhancing Wearable Human Activity Recognition through Smartphone and Smartwatch Sensor Fusion. Under review.

@article{Zubairov2025HARFusion,
  title   = {Enhancing Wearable Human Activity Recognition through Smartphone and Smartwatch Sensor Fusion},
  author  = {Zubairov, Nikita and Milling, Manuel and Schuller, Bj{"o}rn and Fleisch, Elgar and Langer, Patrick},
  year    = {2025},
  note    = {Under review},
  url     = {https://github.com/ADAMMA-CDHI-ETH-Zurich/RobustHAR}
}