Building a Memex by Andrew Louis

Five centuries of logging

Ships are a pretty good way of moving around the planet but it can be an expensive mistake if you miscalculate where you are.

About five hundred years ago, someone figured out that you can estimate speed by tying rope around a piece of wood, throwing the wood overboard, and seeing how much rope unrolled in 28 seconds. If you take measurements every hour or so, you can calculate distance travelled. Combining this data with direction (measured using compasses and astronomy), gives a pretty good estimation of where in the ocean you are.

The piece of wood was called a chip log or log for short. To make the process more efficient, knots were tied at regular intervals and you just had to count how many passed your fingers in the timespan. This is where knots as a measure of speed comes from. Here’s a great video of the technique in action.

(http://prints.rmg.co.uk/art/802507/garthsnaid-1892)

Humans aren’t very good at remembering a lot of things at the same time so navigators started writing down these hourly measurements in a log book. Over time, log books became records of other types of quantitative data (ship direction, supply levels) and qualitative data (what the weather was like, who boarded the ship, and sightings of mermaids (yes, actually)).

(http://www.naval-history.net/OW-US/Patterson/USCGSS_Patterson.htm)

The log book became the backbone of the navigation process. Because longitude was almost impossible to determine until the 18th century, ships would first sail north/south to the latitude of their destination, and then go east/west along the line of longitude. Without logging, ships would have no idea how far they had travelled and could easily run aground on a reef. The collection of techniques were colloquially known as “lead, log, and lookout.”

These days, we associate captain’s logs with bad TV writing but they’re still an important part of maritime operations. All large ships are required to keep a log book and from the perspective of the law, log books are assumed to be completely accurate and free of gaps. One way to make this actually true is by capturing realtime notes in a rough format and later aggregating them into a more polished master log:

Commercial vessels and naval vessels often maintain a draft or “rough” log. The entries from the rough log are later put into the official or “smooth” log. Changes may be made in the “rough” log. However, entries in a vessel’s official log should be made in ink and without erasures. If an entry in the official log must be corrected, it should be stricken with a single line and remain legible. The correction should be initialed by the person required to sign the log for that watch.

I came across this story of a captain who found himself in a storm with a damaged rough log book and no other paper on board. He wanted to keep the ship “legal” and maintain protocol by not writing directly to the ship’s official log which had survived. His solution: writing his rough logs on the walls of his cabin and compiling into the smooth log later. “Never was a rough log rougher,” writes the author.

In last week’s newsletter, I talked about Buckminster Fuller and his Chronofile, a chronological scrapbook that documented the majority of his life. During World War I, he worked for the Navy where he was responsible for cataloguing information about trans-Atlantic crossings of troops. The Chronofile’s inspiration and name came from this nautical culture of organizing information chronologically.

Bucky in the Navy, 1917

It’s not just Bucky. I’ve also been thinking a lot about nautical log books and how they relate to the Memex. As we navigate through the choppy seas of life (please indulge me for a minute), it helps to have a trustworthy log to help guide and orient ourselves, as well as help make sense of our journey when we look back in time. The data in this personal log is created by aggregating rougher chronological data from different sources and using it as a spine for higher-level knowledge that we’re attempting to represent.

Or something like that.

Also, look how beautiful log books can be:

Authorizations/onboarding interface

Today I got so engrossed in nautical history that I left very little time to write about what I actually worked on this week. You better have enjoyed the material!

A quick reminder of what I’m working on: I’m building a self-hosted OSX app of my project that uses Docker (for the API and importers) and Electron to provider a frontend for the data.

This week, I mostly focused on the onboarding experience. In addition to making the oauth flows work even better (see last week’s newsletter), I also added an interface for configuring importers that run off the local OSX filesystem.

I also got the Chrome extension on the official web store which will let new users install this component without any hassle. Here’s the very bare listing which will need screenshots and such soon.

Atlas Recall

Atlas Recall, a “a search engine for your entire digital life”, was a startup with similar goals. They were well-funded ($20M) and had a big talented team but they failed to raise more money. They’re going to shut down and delete all data this week.

Venture capital money was useful for my previous startup but I’m reluctant to put such a personal project on the rocket ship of growth that VCs demand. All sorts of horrible things happen if the rocket ship can’t achieve escape velocity.

The scope of my Memex project is enormous and it’s often been a slog developing; having a VC-funded team would be great! But I’m pretty happy with what’s I’ve built and I’m close to putting out an open-source version that represents the vision. Hopefully I’ve managed to avoid the VC-pressured-pivot-to-sell-user-data-to-Facebook trap!

The Memex does Europe

My partner and I are off to Europe for a few weeks. Right after arriving in London on a red-eye, I’ll head to the Quantified Self meetup where I’m giving a presentation on the project. Then I’ll be in The Netherlands and Italy later where I’ll keep working on the Memex while basking under the Tuscan sun. 😎