July 16, 2017

Maps! Maps! Maps!

My Fitbit is uncomfortable to wear, annoying to charge, and injures people when I play basketball with it on. What’s more, I’m only barely interested in the data from it (steps, flights of stairs climbed, heart rate, sleep) and even if I were more into health tracking, there are serious concerns around accuracy of all consumer-grade trackers .

But I keep wearing my Fitbit because I have a vague dream of using its data as “marginalia” when I scan through my personal timeline. Seeing a sparkline of steps/heartrate/sleep on a zoomed out view of a day might help me orient my recollections or help with a scenario like “find that link I looked at right before bed a few nights ago.”

I haven’t tested out this interface idea yet but I built an importer for my Fitbit data this week. I now have aggregates of steps and heart rates at a granularity of once per minute as well as sleep records.

I can now do silly queries like finding all the places where my heart has beaten over 140bpm in the last few months:


A few frustrating things about the Fitbit ecosystem:

  • Fitbit’s data doesn’t sync to iOS’s HealthKit, unlike most other fitness/health apps in the space. It would be nice if I only had to worry about writing a single HealthKit importer.
  • Timestamps are stored without a timezone (!!!). This would be a hilarious rookie-mistake if it weren’t for the fact people rely on this data to be accurate. It leads to wonderful problems like this: daily step counts (i.e. to hit your 10k step goal) are calculated only when the clock crosses midnight; thus, if you’re flying from Pacific to Eastern timezones and you jump forward to the next day when your plane lands, the daily count will not be triggered and your next day will have two days worth of steps counted at the end of the next night, leading to new personal bests! Please developers, always always always use UTC.

Snapping paths to city grids

Travel/visit data is some of the most important in my Memex system. It acts as an anchor for everything else, giving context around memories or being a pathways to recollection (“find all articles I read while in Tim Hortons”). It’s also useful as “ground data” for problems like trying to figure out which timezone I was in at a given time (ahem, Fitbit.)

I use the Moves app as my tracker but Google Maps also does the same; they both do a great job of passively recording how you move around and where you spend time.

But because keeping a phone’s GPS permanently on is too costly on the battery, the apps sample location only intermittently, leading to GPS trails with lots of details missing. Here’s a snippet of what the raw data looks like coming from Moves — so many ugly diagonal lines slicing the street grid!

![](./assets/2017-07-16/Screenshot 2017-07-15 23.34.50.png)

A few months ago, I tried to use the Google Maps Directions API to try and clean up the lines but it was a lot of time spent trying to make a service do something it wasn’t designed to do.

This week, Mapzen launched a great new API actually designed for this problem: you give it a sequence of coordinates and get back a plausible route that’s snapped to city streets.

Here’s an example from a bike ride where I explored the Wychwood neighbourhood in Toronto. The snapped version is almost an exact match of the route I actually biked.

You have to tell the algorithm which type of mode you’re travelling in but if you specify cycling, the snapped routes are a bit too… legal:

snapping modes

The rightmost image represents a very law-abiding cyclist going the correct direction on one-way streets. Setting the mode to ‘walking’ seems to result in the best paths both for walks and bike rides. (Alternatively, Toronto could be like Montreal and install contraflow bike lanes all over the place so cyclists can get around better.)

It’s a great feeling when new APIs become available to take care of things like this snapping problem or Google’s OCR API mentioned a few newsletters ago.

But oftentimes, APIs regress:

Getting access to Google Hangout chats

I’ve been looking for a good way to get access to my Google Hangouts chat transcripts ever since Google migrated away from Google Talk (⚰️). I can save logs when I use Adium on my desktop computer but this doesn’t work for messages I send and receive on my phone.

This week, I spent some time exploring whether I could get logs through Gmail IMAP access but it turns out only Google Talk records (< 2013) messages are available this way. It doesn’t seem Google is ever going to allow API access to messages and the only path to getting everything is Google Takeout. Sigh.

A Strava importer

On the other hand, I finished a Strava importer this week and I have nothing to report because it was such a pleasant experience. The API worked in an entirely predictable manner, the endpoints and data are comprehensive, and the rate limits are reasonable — an exemplary API!

Here’s a silly graph that shows all combinations of velocities / altitudes while biking in Toronto:

![](./assets/2017-07-16/Screenshot 2017-07-17 01.12.51.png) (The left green bar roughly corresponds with the altitude of Queen St; the middle bar roughly with College St.)

© 2021