Add Sleeptracking

[sillydan1]

Sleeptracking 🛌 💤

This PR adds a sleeptracker with it's own isolated alarm system for wake-up functionality.
This was spurred on by the progress made in #2174 and I took a lot of inspiration from there as well as the approach taken in https://github.com/thiswillbeyourgithub/SleepTk_pinetime_sleep_tracker/blob/main/sleep_tk.py .

I've deliberately tried to keep things as simple as possible and have not implemented extra features such as automatic sleep detection, REM detection etc. I thought it'd be enough to just have the data tracked and a gentle wakeup-alarm - then post-processing can detect REM and sleep quality. I'm no sleep-scientist, I'm a software engineer so I tried to stay within my own domain.

Features

• Tracks your movement (accelerometer data at 2 second poll-rate) very much like how https://github.com/thiswillbeyourgithub/SleepTk_pinetime_sleep_tracker/blob/main/sleep_tk.py does it.
• Tracks your heart rate (every 2 minutes)
• Stores a datapoint every 5 minutes to a logfile
• Wakes you up at the designated time.

Showcase

Not tracking (setting the alarm)

Tracking

Plotting the data

I've created a gnuplot script for plotting the sleep data. Execute with the following command.

Example session (session 3): session-3.csv

gnuplot -e "filename='data/session-1.csv'" -e "title='session 1'" plot.gnuplot

# plot.gnuplot
set terminal png
set title title
set datafile separator ','
system "mkdir -p out"
system "mkdir -p out/data"
set output sprintf("out/%s.png", filename)
set xdata time
set timefmt "%Y-%m-%dT%H:%M:%S"
set xlabel "Time"
set datafile missing "NaN"

# Get the magnitude of the derivative-vector.
# We need 3 sets of delta variables, hence we have 3 functions.
# Return NaN for first point, otherwise delta y or (delta y)/(delta x)
d1(y) = ($0 == 0) ? (y11 = y, NaN) : (y21 = y11, y11 = y, y11-y21)
d2(y) = ($0 == 0) ? (y12 = y, NaN) : (y22 = y12, y12 = y, y12-y22)
d3(y) = ($0 == 0) ? (y13 = y, NaN) : (y23 = y13, y13 = y, y13-y23)
mag(x,y,z) = sqrt((x**2) + (y**2) + (z**2))
scale = 1024  # bma425's default scaling is 1024 (+1g to -1g).
dmag(x,y,z) = mag( \
        abs(d1(x)/scale), \
        abs(d2(y)/scale), \
        abs(d3(z)/scale) \
        )

set ytics nomirror
set y2tics
set ylabel "BPM"
set y2label "g"

# Give some breathing room for the yranges.
set y2range [-0.1:2 < * < 3] extend
set yrange [20:100 < * < 300] extend

plot filename using 1:($2 == 0 ? NaN : $2) smooth csplines t "Heartrate" axes x1y1 lc 7 lw 2, \
     filename using 1:(dmag($3,$4,$5)) w l t "Motion" axes x1y2 lc 8 lw 1

I've been using this for a couple of days (nights) now. Here are my sessions. Note that I am usually a heavy sleeper, but these past 3 days have been quite light due to pollen allergies.

Known Bugs

So far it's been quite good. But during my testing I've noticed some bugs that I can't really pin down and would like so assistance from more experienced maintainers.

• Right after a flash, if you start a sleep tracking session, the system crashes.
  • This also happens after a longer tracking session and you start a new one.
  • I suspect that this has something to do with when saving the sleeptracking settings, but I'm not sure and would like some expert eyes on this.
  • Any assistance on reproducibility could be a great help!
• The logfiles are not being cleared, only appended. This means that the app will eventually fill out the external flash.
  • I did some attempts at alleviating this, but so far to no avail.

Possible Future Improvements

• The gentle alarm is not exactly what is implemented in SleepTK, but I found that it works good enough as an MVP.
• The alarm is a bit too easy to dismiss. Adding a "press 5 times" functionality should be fairly simple though.
• Sleep tracking is not automatically resumed (detection of when you are going to sleep). I found that this would overcomplicate the code, so I left it at manual starting / stopping.
  • This could be done quite simply by setting a bedtime and just start tracking then - but that would not be "smart"
• Adding more data points such as screen-on time and touch-screen interaction would help with figuring out REM / Deep / light sleep.

General feedback

I know this is completely unwarranted, but I just wanted to give some feedback to the project, because I think it's really cool and want to see it improve.

• As an app-developer I think it's a bit annoying and initially confusing that I have to register my app so in so many places (SystemTask, DisplayApp as well as main and CMakeLists).
  • That being said, I like the dependency-injection architecture.
• I think that the Alarm app could benefit from having multiple alarms set at the same time (say max 5 or so).

Please note that this PR description is not written by an AI - I am just a very wordy person.

I had a lot of fun creating this. It was a great opportunity to get more FreeRTOS familiarity and I hope it'll get merged 😄

[github-actions]

Build size and comparison to main:

Section	Size	Difference
text	382580B	3376B
data	944B	0B
bss	22632B	96B

Run in InfiniEmu

[sillydan1]

It looks like the failing pipeline is due to infinisim main.cpp's instantiation of DisplayApp. I'm not sure what the approach here is... Should I open another PR for infinisim, or should I introduce an additional DisplayApp constructor 🤔 ?

[mark9064]

PR for InfiniSim is the way. It doesn't block review of the code here, so no rush in making it

[JustScott]

Just ran across this PR and I love it, the UIs simplicity is awesome!

I'm not a sleep expert either, but I believe you move more during light sleep, so that could also be a metric for tracking the cycles. Something I've wanted is a way to wake my self up during light sleep to make the process of getting out of bed easier, perhaps this could be done by allowing the alarm to go off up to a half hour before the set alarm time if lots of unusual movement is detected, not sure if this would really work or be practical though.

Anyways, I plan on testing out this PR on my pinetime soon, so I'll let you know if I find any bugs you haven't listed above. Great Work!

[sillydan1]

Thanks for your interest @JustScott (as well as any silent lurkers out there 😄 ).
With this recent rebase, I have also collected some usage feedback, this is from personal usage as well as from
friends who I've forced to test this out - please feel free to reply / comment with your own feedback if you have any.

General Pain points:

• 1) It's too easy to forget to start the tracker
• 2) Having to extract the data is super annoying and is "too many steps"
• 3) 5 days of tracking is not enough

Possible Solutions:

• Setting two times, A bedtime and a wakeup time could be an easy fix to point 1) - I might do this.
• Solving 2) can be fixed by either:
  • Do the graphing on the watch
    • This could be very difficult and would also massively complicate the scope of the tracker and there's still
      storage issues
  • Extend the phone apps to extract and graph the data
    • This would require the feature to be merged first - I also have little to no experience with app development - especially swift is very foreign to me.
• Increasing the max sessions to 30 could be an easy fix to point 3) - it does mean that it would use about 1.7MiB.
  • 1.7MiB is almost half of the user storage (4MiB), I've chosen the 14 day middleground, but this is a great topic
    for discussion.

I am actually considering removing the alarm logic altogether, since waking you up is technically not part of the sleep tracking and you can just set an alarm in the Alarm app, or use a normal alarmclock or something. It might just make the app confusing though, since other smartwatches wake you up... So I'm not sure.

[mark9064]

I like the work here. I'm finding it quite hard to review though because we haven't actually figured out what this data is for yet right? I think we would need to have some other software that can actually use the recordings to give some useful information before we could ship it to users, otherwise the app effectively has no functionality for the end user.

I completely get that knowing what to do with the data is difficult though. I had a glance at the literature recently, and it seems that commercial smartwatches still struggle with sleep zone determination. Since those generally have better hardware than the pinetime and more resources to design better software (e.g. comparing against a medical polysomnograph) I think we will struggle to design an accurate sleep zone detector that we could actually ship to users in any capacity.

A sleep quality score might be a more reasonable target? But still I have no clue whether this is realistically achievable or not. I think we could give some metrics like sleep duration (probably just timed from when the user pressed start to the alarm time rather than anything fancy), and maybe average nighttime heart rate and number of minutes with motion detected / percentage of time still? I'm not sure how useful those numbers are to the user, but at least we wouldn't be making any medically dubious claims :)

File space efficiency

For 3): I think a binary format would make more sense here given the resource constraints

Maybe something like so?

Header: start time (4 bytes), version (1 byte) (version is tied to the sampling period also)
Data: x,y,z (6 bytes), HR (1 byte) (or perhaps even some "total motion" rather than each channel)
Means a 12h sleep (upper bound) should be 144 data points, or ~1KB. Space issue resolved?

It's less convenient to parse, but a lot more efficient.

[raj-magesh]

GadgetBridge has some charts for sleep tracking so reporting data in a format it reads might be helpful? Not a dev, so can't really help more, but following that API seems like the best thing to do.

[dhdurgee]

I would like to see this feature added with automatic sleep detection and data exported in the format GadgetBridge expects.

I am migrating to the PineTime from a Fitbit Flex 2 and even it was able to detect sleep, so imagine the processor in my PineTime is more than up to the task.

[sillydan1]

I would like to see this feature added with automatic sleep detection and data exported in the format GadgetBridge expects.

I would love to do this. I just need to wade through GadgetBridge's documentation to figure out what that format is.

imagine the processor in my PineTime is more than up to the task.

From my research, the best results are gained from executing a classifier / neural network-based algorithm on the data, which would probably be too much for the poor little MCU on the PineTime. There are other less accurate stochastic methods of detecting this which would be doable.
Maybe GadgetBridge is able to do sleep detection on the gathered data.

[Storm-Trooper5555]

Hi @sillydan1,

First off great job on this, it seems to work pretty well,

Second sorry just wanted to note that gadget-bridge doesn't really have a preferred format, as it is designed around the vendor having a preexisting output and then you just update create a library in gadget-bridge's code to take in the output of the device,

I think they mention to lookup the implementation for bangle.js for inspiration or guidelines but it does mean you get it to output the data pretty much however you want and then you just have to create a library on the gadget-bridge side to receive it,

That would be written in Kotlin/Java i think so if you aren't really comfortable coding or have no experience with it, then possibly try using the bangle.js implementation and then just copy and refactor the gadget bridge library code for that if you want,

Hope this is helpful

Some links:
Bangle.js sleep-log Readme
Bangle.js GadgetBridge Library
Or for the extremely Hardcore
The Bluetooth GATT communications code reference
(if you use the last one you probably want to use Ctrl+F to find sleep tracking related stuff)