Building a Memex by Andrew Louis

The Memex becomes installable

After last week’s depressing experience with javascript, I’m pleased to say I only had to rm -rf node_modules twice this week. I even feel I might have learned a thing or two about javascript and Electron.

A recap of the short-term goal: I want an installable OSX app that I can distribute to people to start beta testing the Memex. It’ll package both the frontend and the API and importer system (running in the background through Docker).

I mainly focused on making Electron talk to Docker this week. Below is a clip of the installation process running. Here’s what it’s doing:

The design is rough but it’s getting pretty close!

What I have to do now:

In general, I’m happy with how this Electron+Docker system will allow me to get the beta out to people without me having any access to their data. However, an outstanding hurdle is figuring out a good way to authorize third-party services like Twitter, Gmail, Github, etc. It’s bad security practice to include the OAuth secret keys inside the app because people who abuse their access tokens will be tied back to my app. I could set up a single hosted OAuth service that the electron app can connect to facilitate authorizations but I could theoretically steal user access tokens this way. What I think I’ll do is do this but open source the OAuth app for people to verify that I’m not caching credentials and also give the paranoid the option to configure OAuth on their own. If anyone has any clever ideas, I’m all ears but I think this is just a fundamental limitation of OAuth which was designed for the era of hosted web apps.

I also worked on the tab system in Electron that lets people run different queries and screens at the same time. Some improvements this week:

Speaking of tabs

Earlier this month, I was talking to Dann about the Memex’s API and how it’s powered by a graph schema implemented on top of Postgres. Dann wrote a great article about implementing a graph database in Javascript and I joked that I could reimplement my API this way.

But first, I told him, I’d have to actually finish reading the article which has been open in a Chrome tab for months. How many months? I decided to actually check and it’s been more than a year (this is basically a graph of browser restarts):

A fun demonstration of using the Memex to quantify shame.

Anticipating the beta

If you’re craving for the ability to do the above (and hopefully the other usecases), the beta should be ready very soon!

Here are a few things you can do now that will ensure you have a lot of data when you get started:

Other weirdos

I love when people send me links of people working on similar projects because it makes me feel less of a weirdo. Here are a few links that have come my way recently:

Please send me your finds!