Head and Eye Controlled Cursor Using Electrooculography (EOG) and Inertial Measurement Units (IMU)
A hands-free computer cursor control system using dual-channel EOG for eye event detection and an IMU for head motion tracking. Built as a capstone project demonstrating embedded systems, real-time signal processing, sensor fusion, and machine learning. See docs/data_flow.md for the complete system pipeline.
┌─────────────┐ ┌─────────────┐ ┌──────────────────┐
│ AD8232 x2 │ │ STM32 │ │ PC (Python) │
│ Vertical EOG│────>│ ADC1 (PA0) │ │ Signal Proc. │
│ Horiz. EOG │────>│ ADC2 (PA4) │────>│ State-Space │
└─────────────┘ │ @200Hz │ │ Sensor Fusion │
│ │ └────────┬─────────┘
┌─────────────┐ │ I2C Read │ │
│ MPU9250 │────>│ Raw Gyro │ ▼
│ IMU │ └─────────────┘ ┌──────────────┐
└─────────────┘ │ OS Mouse API │
└──────────────┘
How it works: IMU head motion drives cursor movement (direct proportional in threshold mode, state-space model with velocity decay in statespace mode). Vertical EOG detects blinks (click) and up/down gaze (scroll). Horizontal EOG detects left/right gaze (back/forward). Scroll and navigation require both eye gaze and head motion to agree, preventing false triggers.
| Action | Input | Type |
|---|---|---|
| Cursor Move | IMU Gyro X/Y (proportional or state-space, by mode) | Continuous |
| Left Click | Double Blink (two rapid blinks) | Discrete |
| Right Click | Long Blink (eyes closed >=0.4s) | Discrete |
| Double Click | Double Head Nod (two quick nods, gyro_x) | Discrete |
| Scroll Up/Down | Eye Up/Down + Head Up/Down (eog_v + gx) | Fusion |
| Browser Back/Fwd | Eye Left/Right + Head Left/Right (eog_h + gy) | Fusion |
| Window Switch | Head Roll Flick (gyro_z) | Discrete |
Blink detection uses a 3-state machine analyzing full spike waveforms, not simple thresholds. See docs/detection.md for signal zones, state diagrams, and parameters.
Requires a graphical desktop (Windows / macOS / Linux with X11) for cursor control.
pip install -r requirements.txt
cd python
python -m scripts.generate_demo_data --output ../data/raw # ~10s, deterministic (seed=42)
python -m scripts.train_model --data ../data/raw # ~15s, 100% CV accuracy3 modes × 3 data sources — any combination works (cd python first):
--replay CSV (offline) |
--simulate (no hardware) |
--port COM3 (hardware) |
|
|---|---|---|---|
| threshold | python main.py --replay ../data/raw/demo_replay.csv |
python main.py --simulate |
python main.py --port COM3 |
| statespace | python main.py --replay ../data/raw/demo_replay.csv --mode statespace |
python main.py --simulate --mode statespace |
python main.py --port COM3 --mode statespace |
| ml | python main.py --replay ../data/raw/demo_replay.csv --mode ml |
python main.py --simulate --mode ml |
python main.py --port COM3 --mode ml |
Default mode is
threshold. Hardware port: WindowsCOM3, Linux/dev/ttyUSB0.
Simulator controls: Arrows=move, Space(x2)=left-click, Space(hold)=right-click, N(x2)=double-click, U+Up=scroll-up, D+Down=scroll-down, L+Left=back, R+Right=forward, W=window-switch, Q=quit.
Note: The simulator generates square-wave EOG signals (instant jumps), which differ from the smooth waveforms used to train the SVM. As a result,
--mode mlwith--simulatecannot classify EOG events reliably. Use--replay CSVor real hardware for ML mode.
├── firmware/ # STM32 reference firmware (C)
│ └── Core/Src/
│ ├── main.c # Main loop: dual ADC + I2C + UART @200Hz
│ └── mpu9250.c # MPU9250 I2C driver
│
├── python/ # PC-side application
│ ├── main.py # Entry point with CLI
│ ├── eog_cursor/ # Core library
│ │ ├── config.py # All tunable parameters
│ │ ├── serial_reader.py # STM32 UART data parser (dual-channel)
│ │ ├── signal_processing.py # Low-pass filter, sliding window
│ │ ├── event_detector.py # Blink, gaze, head roll, double nod detectors
│ │ ├── feature_extraction.py # 10 features × 2 channels for SVM classifier
│ │ ├── cursor_control.py # Threshold & state-space controllers
│ │ ├── ml_classifier.py # SVM training and inference (dual-channel)
│ │ ├── simulator.py # Keyboard-based hardware simulator
│ │ └── csv_replay.py # Offline CSV file replay
│ ├── scripts/
│ │ ├── collect_data.py # Labeled data collection from hardware
│ │ ├── generate_demo_data.py# Synthetic dual-channel data generator
│ │ ├── train_model.py # SVM training with cross-validation
│ │ └── visualize.py # Real-time 3-subplot signal visualization
│ ├── tests/ # 53 tests (signal, events, ML, state-space)
│ └── models/ # Trained SVM model + scaler (.gitignored)
│
├── data/raw/ # Generated by scripts/generate_demo_data.py
├── docs/ # Technical deep-dives
│ ├── data_flow.md # System pipeline (firmware + Python, all 9 run configs)
│ ├── detection.md # Blink state machine, signal zones, waveform analysis
│ ├── state_space.md # Matrix derivation, velocity retention analysis, stability proof
│ └── performance.md # Evaluation metrics template (ML + real hardware)
└── requirements.txt
State: [pos_x, vel_x, pos_y, vel_y] — velocity retention (default 0.95, keeps 95% speed per step) controls glide length. See docs/state_space.md for full matrix derivation and stability proof.
Scroll and navigation require both eye gaze and head motion to agree:
| Action | Eye Signal | Head Signal |
|---|---|---|
| Scroll Up | eog_v > 2800 (look up) | gx < -500 (tilt up) |
| Scroll Down | eog_v < 1200 (look down) | gx > 500 (tilt down) |
| Browser Back | eog_h < 1200 (look left) | gy < -500 (turn left) |
| Browser Fwd | eog_h > 2800 (look right) | gy > 500 (turn right) |
| Component | Qty | Purpose | Interface |
|---|---|---|---|
| STM32 MCU (F4/U5/etc.) | 1 | Data acquisition | USB (UART) |
| AD8232 | 2 | EOG analog front-end (V + H) | ADC pins |
| MPU9250 (or MPU6050) | 1 | IMU head tracking | I2C |
| Ag/AgCl electrodes | 5 | EOG signal pickup (2 pairs + 1 ref) | AD8232 input |
Electrode placement: Vertical pair (V+/V-) above and below one eye → eog_v. Horizontal pair (L/R) at outer canthi of both eyes → eog_h. Reference on forehead.
Firmware: Reference code in firmware/, developed with STM32CubeMX + STM32CubeIDE. Generate a CubeMX project for your board, then drop in main.c and mpu9250.c. Data packet format: timestamp,eog_v,eog_h,gyro_x,gyro_y,gyro_z\r\n at 115200 baud. See docs/data_flow.md for details.
All parameters in python/eog_cursor/config.py. Key values:
GYRO_DEADZONE = 500 # Below this = noise
BLINK_THRESHOLD = 3000 # ADC value for blink detection
SS_VELOCITY_RETAIN = 0.95 # Cursor glide (0.8=short, 0.99=long)
CURSOR_SENSITIVITY = 0.01 # Head motion to pixel ratiocd python && python -m pytest tests/ -v53 tests across 3 files:
| File | Key Verifications |
|---|---|
test_event_detector.py — 31 tests |
Double blink detected; single blink ignored; long blink fires on release; long blink max duration rejected; sustained close fires once; cooldown prevents re-trigger; sustained gaze detected; transient gaze rejected; head roll flick triggers window switch; held roll ignored; double head nod triggers double click; single nod ignored |
test_signal_processing.py — 15 tests |
Low-pass preserves DC baseline; high frequency attenuated; sliding window keeps most recent samples; feature vector has correct length; state-space velocity decays to ~0 after 200 iterations |
test_ml_pipeline.py — 7 tests |
Training accuracy >80%; model save/load roundtrip succeeds; predictions are valid labels; streaming classifier produces output; blink features clearly separable from idle |
See docs/performance.md for ML classification accuracy, end-to-end latency, per-action accuracy, and robustness evaluation — all measured with real EOG hardware.
| Decision | Rationale |
|---|---|
| Dual-channel EOG | Enables horizontal gaze for browser back/forward |
| Eye + head fusion | Both must agree — prevents false triggers |
| Processing on PC | Full Python ecosystem, easier debugging |
| State-space model | Physical inertia makes cursor feel natural |
| SVM over deep learning | Small dataset, low latency (<5ms), interpretable |
| Lazy pyautogui import | Enables testing in headless CI |
MIT