Skip to content

Autotune: calculate CR directly #644

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
scottleibrand merged 12 commits into from
Sep 10, 2017
Merged

Autotune: calculate CR directly #644

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
800710e
calculate CR directly, and use CR*CSF to help calculate ISF
scottleibrand Sep 7, 2017
a3b7c94
improved logging
scottleibrand Sep 7, 2017
ed18a0c
put adjustmentFraction back to 0.5
scottleibrand Sep 7, 2017
c62cc86
clean up autotune-recommends-report
scottleibrand Sep 7, 2017
be15687
new file for counting insulin dosing over a meal period
scottleibrand Sep 7, 2017
d8d2f7f
smooth out mealtime basals in absence of other data
scottleibrand Sep 7, 2017
c97b1f0
smooth slightly more
scottleibrand Sep 7, 2017
28c28a6
remove any trailing / from NIGHTSCOUT_HOST
scottleibrand Sep 7, 2017
080a9f5
make autotune converge even from the most extreme ratios
scottleibrand Sep 8, 2017
1e9ce23
speed up multiday runs by separating out treatments by day
scottleibrand Sep 8, 2017
354b687
replace NUMBER_OF_RUNS with START/END_DAYS_AGO
scottleibrand Sep 8, 2017
085e653
wait for 40s silence before mmtuning to avoid stomping on upto30s waits
scottleibrand Sep 9, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 19 additions & 5 deletions bin/oref0-autotune-prep.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@

var generate = require('oref0/lib/autotune-prep');
function usage ( ) {
console.error('usage: ', process.argv.slice(0, 2), '<pumphistory.json> <profile.json> <glucose.json> [carbhistory.json] [autotune/glucose.json]');
console.error('usage: ', process.argv.slice(0, 2), '<pumphistory.json> <profile.json> <glucose.json> [pumpprofile.json] [carbhistory.json] [autotune/glucose.json]');
}

if (!module.parent) {
Expand All @@ -33,8 +33,9 @@ if (!module.parent) {
}
var profile_input = process.argv.slice(3, 4).pop();
var glucose_input = process.argv.slice(4, 5).pop();
var carb_input = process.argv.slice(5, 6).pop()
var prepped_glucose_input = process.argv.slice(6, 7).pop()
var pumpprofile_input = process.argv.slice(5, 6).pop()
var carb_input = process.argv.slice(6, 7).pop()
var prepped_glucose_input = process.argv.slice(7, 8).pop()

if ( !pumphistory_input || !profile_input || !glucose_input ) {
usage( );
Expand All @@ -51,10 +52,23 @@ if (!module.parent) {
return console.error("Could not parse input data: ", e);
}

var pumpprofile_data = { };
if (typeof pumpprofile_input != 'undefined') {
try {
pumpprofile_data = JSON.parse(fs.readFileSync(pumpprofile_input, 'utf8'));
} catch (e) {
console.error("Warning: could not parse "+pumpprofile_input);
}
}

// disallow impossibly low carbRatios due to bad decoding
if ( typeof(profile_data.carb_ratio) == 'undefined' || profile_data.carb_ratio < 2 ) {
console.log('{ "carbs": 0, "mealCOB": 0, "reason": "carb_ratio ' + profile_data.carb_ratio + ' out of bounds" }');
return console.error("Error: carb_ratio " + profile_data.carb_ratio + " out of bounds");
if ( typeof(pumpprofile_data.carb_ratio) == 'undefined' || pumpprofile_data.carb_ratio < 2 ) {
console.log('{ "carbs": 0, "mealCOB": 0, "reason": "carb_ratios ' + profile_data.carb_ratio + ' and ' + pumpprofile_data.carb_ratio + ' out of bounds" }');
return console.error("Error: carb_ratios " + profile_data.carb_ratio + ' and ' + pumpprofile_data.carb_ratio + " out of bounds");
} else {
profile_data.carb_ratio = pumpprofile_data.carb_ratio;
}
}

try {
Expand Down
6 changes: 3 additions & 3 deletions bin/oref0-autotune-recommends-report.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ csf_new=$(cat $directory/autotune/profile.json | jq '.csf')
carb_ratio_new=$(cat $directory/autotune/profile.json | jq '.carb_ratio')

# Print Header Info
printf "%-${parameter_width}s| %-${data_width}s| %-${data_width}s\n" "Parameter" "Current" "Autotune" >> $report_file
printf "%-${parameter_width}s| %-${data_width}s| %-${data_width}s\n" "Parameter" "Pump" "Autotune" >> $report_file
printf "%s\n" "-------------------------------------" >> $report_file

# Print ISF, CSF and Carb Ratio Recommendations
Expand All @@ -74,10 +74,10 @@ if [ $csf_current != null ]; then
else
printf "%-${parameter_width}s| %-${data_width}s| %-${data_width}.3f\n" "CSF [mg/dL/g]" "n/a" $csf_new >> $report_file
fi
printf "%-${parameter_width}s| %-${data_width}.3f| %-${data_width}.3f\n" "Carb Ratio [g]" $carb_ratio_current $carb_ratio_new >> $report_file
printf "%-${parameter_width}s| %-${data_width}.3f| %-${data_width}.3f\n" "Carb Ratio[g/U]" $carb_ratio_current $carb_ratio_new >> $report_file

# Print Basal Profile Recommendations
printf "%-${parameter_width}s| %-${data_width}s|\n" "Basal Profile [unit/hour]" "-" >> $report_file
printf "%-${parameter_width}s| %-${data_width}s|\n" "Basals [U/hr]" "-" >> $report_file

# Build time_list array of H:M in 30 minute increments to mirror pump basal schedule
time_list=()
Expand Down
129 changes: 62 additions & 67 deletions bin/oref0-autotune.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
#!/bin/bash

# This script sets up an easy test environment for autotune, allowing the user to vary parameters
# like start/end date and number of runs.
# like start/end date.
#
# Required Inputs:
# DIR, (--dir=<OpenAPS Directory>)
Expand All @@ -10,8 +10,6 @@
# Optional Inputs:
# END_DATE, (--end-date=<YYYY-MM-DD>)
# if no end date supplied, assume we want a months worth or until day before current day
# NUMBER_OF_RUNS (--runs=<integer, number of runs desired>)
# if no number of runs designated, then default to 5
# EXPORT_EXCEL (--xlsx=<filenameofexcel>)
# export to excel. Disabled by default
# TERMINAL_LOGGING (--log <true/false(true)>
Expand Down Expand Up @@ -39,7 +37,8 @@ DIR=""
NIGHTSCOUT_HOST=""
START_DATE=""
END_DATE=""
NUMBER_OF_RUNS=1 # Default to a single run if not otherwise specified
START_DAYS_AGO=1 # Default to yesterday if not otherwise specified
END_DAYS_AGO=1 # Default to yesterday if not otherwise specified
EXPORT_EXCEL="" # Default is to not export to Microsoft Excel
TERMINAL_LOGGING=true
RECOMMENDS_REPORT=true
Expand Down Expand Up @@ -96,8 +95,12 @@ case $i in
END_DATE=`date --date="$END_DATE" +%Y-%m-%d`
shift # past argument=value
;;
-r=*|--runs=*)
NUMBER_OF_RUNS="${i#*=}"
-t=*|--start-days-ago=*)
START_DAYS_AGO="${i#*=}"
shift # past argument=value
;;
-d=*|--end-days-ago=*)
END_DAYS_AGO="${i#*=}"
shift # past argument=value
;;
-x=*|--xlsx=*)
Expand All @@ -116,22 +119,25 @@ case $i in
esac
done

# remove any trailing / from NIGHTSCOUT_HOST
NIGHTSCOUT_HOST=$(echo $NIGHTSCOUT_HOST | sed 's/\/$//g')

if [[ -z "$DIR" || -z "$NIGHTSCOUT_HOST" ]]; then
echo "Usage: oref0-autotune <--dir=myopenaps_directory> <--ns-host=https://mynightscout.azurewebsites.net> [--start-date=YYYY-MM-DD] [--end-date=YYYY-MM-DD] [--runs=number_of_runs] [--xlsx=autotune.xlsx] [--log=(true)|false]"
echo "Usage: oref0-autotune <--dir=myopenaps_directory> <--ns-host=https://mynightscout.azurewebsites.net> [--start-days-ago=number_of_days] [--end-days-ago=number_of_days] [--start-date=YYYY-MM-DD] [--end-date=YYYY-MM-DD] [--xlsx=autotune.xlsx] [--log=(true)|false]"
exit 1
fi
if [[ -z "$START_DATE" ]]; then
# Default start date of yesterday
START_DATE=`date --date="1 day ago" +%Y-%m-%d`
START_DATE=`date --date="$START_DAYS_AGO days ago" +%Y-%m-%d`
fi
if [[ -z "$END_DATE" ]]; then
# Default end-date as this morning at midnight in order to not get partial day samples for now
# (ISF/CSF adjustments are still single values across each day)
END_DATE=`date --date="1 day ago" +%Y-%m-%d`
END_DATE=`date --date="$END_DAYS_AGO days ago" +%Y-%m-%d`
fi

if [[ -z "$UNKNOWN_OPTION" ]] ; then # everything is ok
echo "Running oref0-autotune --dir=$DIR --ns-host=$NIGHTSCOUT_HOST --start-date=$START_DATE --runs=$NUMBER_OF_RUNS --end-date=$END_DATE"
echo "Running oref0-autotune --dir=$DIR --ns-host=$NIGHTSCOUT_HOST --start-date=$START_DATE --end-date=$END_DATE"
else
echo "Unknown options. Exiting"
exit 1
Expand All @@ -157,15 +163,6 @@ if [[ $TERMINAL_LOGGING = "true" ]]; then
exec &> >(tee -a autotune.$(date +%Y-%m-%d-%H%M%S).log)
fi

# Pull Nightscout Data
echo "Grabbing NIGHTSCOUT treatments.json for date range..."

# Get Nightscout carb and insulin Treatments
query="find%5Bcreated_at%5D%5B%24gte%5D=`date --date="$START_DATE -4 hours" -Iminutes`&find%5Bcreated_at%5D%5B%24lte%5D=`date --date="$END_DATE +1 days" -Iminutes`"
echo Query: $NIGHTSCOUT_HOST/$query
ns-get host $NIGHTSCOUT_HOST treatments.json $query > ns-treatments.json || die "Couldn't download ns-treatments.json"
ls -la ns-treatments.json || die "No ns-treatments.json downloaded"

# Build date list for autotune iteration
date_list=()
date=$START_DATE;
Expand All @@ -179,76 +176,74 @@ do
fi
done

echo "Grabbing NIGHTSCOUT entries/sgv.json for date range..."
echo "Grabbing NIGHTSCOUT treatments.json and entries/sgv.json for date range..."

# Get Nightscout BG (sgv.json) Entries
for i in "${date_list[@]}"
do
query="find%5Bdate%5D%5B%24gte%5D=`(date -d $i +%s | tr -d '\n'; echo 000)`&find%5Bdate%5D%5B%24lte%5D=`(date --date="$i +1 days" +%s | tr -d '\n'; echo 000)`&count=1000"
echo Query: $NIGHTSCOUT_HOST $query
ns-get host $NIGHTSCOUT_HOST entries/sgv.json $query > ns-entries.$i.json || die "Couldn't download ns-entries.$i.json"
ls -la ns-entries.$i.json || die "No ns-entries.$i.json downloaded"
done

echo "Running $NUMBER_OF_RUNS runs from $START_DATE to $END_DATE"
sleep 2

# Do iterative runs over date range, save autotune.json (prepped data) and input/output
# profile.json
# Loop 1: Run 1 to Number of Runs specified by user or by default (1)
for run_number in $(seq 1 $NUMBER_OF_RUNS)
do
# Loop 2: Iterate through Date Range
for i in "${date_list[@]}"
do
cp profile.json profile.$run_number.$i.json
query="find%5Bdate%5D%5B%24gte%5D=`(date -d $i +%s | tr -d '\n'; echo 000)`&find%5Bdate%5D%5B%24lte%5D=`(date --date="$i +1 days" +%s | tr -d '\n'; echo 000)`&count=1000"
echo Query: $NIGHTSCOUT_HOST $query
ns-get host $NIGHTSCOUT_HOST entries/sgv.json $query > ns-entries.$i.json || die "Couldn't download ns-entries.$i.json"
ls -la ns-entries.$i.json || die "No ns-entries.$i.json downloaded"

# Get Nightscout carb and insulin Treatments
# echo $i $START_DATE;
#query="find%5Bdate%5D%5B%24gte%5D=`(date -d $i +%s | tr -d'\n'; echo 000)`&find%5Bdate%5D%5B%24lte%5D=`(date --date="$i +1 days" +%s | tr -d '\n'; echo 000)`&count=1000"
query="find%5Bcreated_at%5D%5B%24gte%5D=`date --date="$i -5 hours" -Iminutes`&find%5Bcreated_at%5D%5B%24lte%5D=`date --date="$i +1 days" -Iminutes`"
echo Query: $NIGHTSCOUT_HOST/$query
ns-get host $NIGHTSCOUT_HOST treatments.json $query > ns-treatments.$i.json || die "Couldn't download ns-treatments.$i.json"
ls -la ns-treatments.$i.json || die "No ns-treatments.$i.json downloaded"


# Do iterative runs over date range, save autotune.json (prepped data) and input/output profile.json
cp profile.json profile.$i.json
# Autotune Prep (required args, <pumphistory.json> <profile.json> <glucose.json>), output prepped glucose
# data or <autotune/glucose.json> below
echo "oref0-autotune-prep ns-treatments.json profile.json ns-entries.$i.json > autotune.$run_number.$i.json"
oref0-autotune-prep ns-treatments.json profile.json ns-entries.$i.json > autotune.$run_number.$i.json \
|| die "Could not run oref0-autotune-prep ns-treatments.json profile.json ns-entries.$i.json"
echo "oref0-autotune-prep ns-treatments.$i.json profile.json ns-entries.$i.json profile.pump.json > autotune.$i.json"
oref0-autotune-prep ns-treatments.$i.json profile.json ns-entries.$i.json profile.pump.json > autotune.$i.json \
|| die "Could not run oref0-autotune-prep ns-treatments.$i.json profile.json ns-entries.$i.json"

# Autotune (required args, <autotune/glucose.json> <autotune/autotune.json> <settings/profile.json>),
# output autotuned profile or what will be used as <autotune/autotune.json> in the next iteration
echo "oref0-autotune-core autotune.$run_number.$i.json profile.json profile.pump.json > newprofile.$run_number.$i.json"
if ! oref0-autotune-core autotune.$run_number.$i.json profile.json profile.pump.json > newprofile.$run_number.$i.json; then
echo "oref0-autotune-core autotune.$i.json profile.json profile.pump.json > newprofile.$i.json"
if ! oref0-autotune-core autotune.$i.json profile.json profile.pump.json > newprofile.$i.json; then
if cat profile.json | jq --exit-status .carb_ratio==null; then
echo "ERROR: profile.json contains null carb_ratio: using profile.pump.json"
cp profile.pump.json profile.json
exit
else
die "Could not run oref0-autotune-core autotune.$run_number.$i.json profile.json profile.pump.json"
die "Could not run oref0-autotune-core autotune.$i.json profile.json profile.pump.json"
fi
else
# Copy tuned profile produced by autotune to profile.json for use with next day of data
cp newprofile.$run_number.$i.json profile.json
cp newprofile.$i.json profile.json
fi

done # End Date Range Iteration
done # End Number of Runs Loop

if ! [[ -z "$EXPORT_EXCEL" ]]; then
echo Exporting to $EXPORT_EXCEL
oref0-autotune-export-to-xlsx --dir $DIR --output $EXPORT_EXCEL
fi
if ! [[ -z "$EXPORT_EXCEL" ]]; then
echo Exporting to $EXPORT_EXCEL
oref0-autotune-export-to-xlsx --dir $DIR --output $EXPORT_EXCEL
fi

# Create Summary Report of Autotune Recommendations and display in the terminal
if [[ $RECOMMENDS_REPORT == "true" ]]; then
# Set the report file name, so we can let the user know where it is and cat
# it to the screen
report_file=$directory/autotune/autotune_recommendations.log
# Create Summary Report of Autotune Recommendations and display in the terminal
if [[ $RECOMMENDS_REPORT == "true" ]]; then
# Set the report file name, so we can let the user know where it is and cat
# it to the screen
report_file=$directory/autotune/autotune_recommendations.log

echo
echo "Autotune pump profile recommendations:"
echo "---------------------------------------------------------"
echo
echo "Autotune pump profile recommendations:"
echo "---------------------------------------------------------"

# Let the user know where the Autotune Recommendations are logged
echo "Recommendations Log File: $report_file"
echo
# Let the user know where the Autotune Recommendations are logged
echo "Recommendations Log File: $report_file"
echo

# Run the Autotune Recommends Report
oref0-autotune-recommends-report $directory
# Run the Autotune Recommends Report
oref0-autotune-recommends-report $directory

# Go ahead and echo autotune_recommendations.log to the terminal, minus blank lines
cat $report_file | egrep -v "\| *\| *$"
fi
# Go ahead and echo autotune_recommendations.log to the terminal, minus blank lines
cat $report_file | egrep -v "\| *\| *$"
fi

done # End Date Range Iteration
14 changes: 7 additions & 7 deletions bin/oref0-pump-loop.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -330,10 +330,10 @@ function mmtune {
reset_spi_serial.py 2>/dev/null
fi
oref0_init_pump_comms.py
echo -n "Listening for 30s silence before mmtuning: "
echo -n "Listening for 40s silence before mmtuning: "
for i in $(seq 1 800); do
echo -n .
any_pump_comms 30 2>/dev/null | egrep -v subg | egrep No \
any_pump_comms 40 2>/dev/null | egrep -v subg | egrep No \
&& break
done
echo {} > monitor/mmtune.json
Expand All @@ -353,12 +353,12 @@ function maybe_mmtune {
if ( find monitor/ -mmin -15 | egrep -q "pump_loop_completed" ); then
# mmtune ~ 25% of the time
[[ $(( ( RANDOM % 100 ) )) > 75 ]] \
&& echo "Waiting for 30s silence before mmtuning" \
&& wait_for_silence 30 \
&& echo "Waiting for 40s silence before mmtuning" \
&& wait_for_silence 40 \
&& mmtune
else
echo "pump_loop_completed more than 15m old; waiting for 30s silence before mmtuning"
wait_for_silence 30
echo "pump_loop_completed more than 15m old; waiting for 40s silence before mmtuning"
wait_for_silence 40
mmtune
fi
}
Expand All @@ -370,7 +370,7 @@ function any_pump_comms {
# listen for $1 seconds of silence (no other rigs talking to pump) before continuing
function wait_for_silence {
if [ -z $1 ]; then
waitfor=30
waitfor=40
else
waitfor=$1
fi
Expand Down
Loading