HUIJZER.XYZ

Updating my notes via email

2024-04-17

Charles Darwin made it a habit to immediately write down anything that conflicted with his own ideas, so his brain would not forget or ignore it. In his own words:

"I had also, during many years, followed a golden rule, namely that whenever published fact, a new observation of thought came across me, which was opposed to my general results, to make a memorandum of it without fail and at once; for I had found by experience that such facts and thoughts were far more apt to escape from the memory than favourable ones."

Based on this, I've also made it a habit to quickly write down new ideas or thoughts. Unlike Darwin, however, I don't carry a notebook with me. Instead, I prefer to store my notes in a Git repository. Unlike a notebook, a Git repository is more fireproof, can more easily be searched and edited, and can scale to much larger sizes. Jeff Huang, for example, wrote that he has a single text file with all his notes from 2008 to 2022. At the time of his writing, the file contained 51,690 handwritten lines of text. He wrote that the file been his "secret weapon".

Similarly, I made a repository with a single README.md file. This file contains a list of all my notes, separated by a heading with the date. Here is a snippet of the file:

# 2024-02-10

Carnegie, Edison, and Rockefeller were not company builders, they were industry builders.

# 2024-02-20

Many companies don’t fail only because they optimize on what they can do already but also because they don’t dare to make changes.
It’s much safer for a CEO to copy other people than make "risky" bets, according to Sandy Munro's experience with car CEOs.

This is quite easy to edit when I'm on my computer, but hard to edit when I only have my phone around. I have to open the file, scroll to the bottom, and then write my note. Optionally, I might need to figure out the day and write a new heading too. As a way to save time, I have been sending myself quick emails with the note instead. This has worked fine for months, but has required me to copy the notes to the repository manually. Since this costs me a few minutes per day, I decided to automate it.

Automating the process

Ideally, I figured, I could send an email to a specific address and have the note automatically added to the repository. Cloudflare Workers seemed like a good fit for this since they are cheap and Cloudflare nowadays has Email Workers (Beta). With Email Workers, you can send and receive emails. All that is needed for receiving emails is a domain.

To setup the worker, I followed the instructions at Cloudflare's documentation. Basically, it involves creating a worker first and setting MX records which point to Cloudflare's email servers. Then, whenever an email comes in, it will be handed to the worker.

To create the worker, I make a new GitHub repository with the following worker.js file:

import PostalMime from 'postal-mime';
import { Octokit } from "@octokit/core";

function authorized_octokit(token) {
  return new Octokit({
    auth: token
  });
}

// From https://developer.mozilla.org/en-US/docs/Glossary/Base64#the_unicode_problem.
function base64ToBytes(base64) {
  const binString = atob(base64);
  return Uint8Array.from(binString, (m) => m.codePointAt(0));
}

function bytesToBase64(bytes) {
  const binString = Array.from(bytes, (byte) =>
    String.fromCodePoint(byte),
  ).join("");
  return btoa(binString);
}

function encodeContent(str) {
  return bytesToBase64(new TextEncoder().encode(str));
}

function decodeContent(base64) {
  return new TextDecoder().decode(base64ToBytes(base64));
}

async function readNotes(octokit) {
  let resp = await octokit.request('GET /repos/{owner}/{repo}/contents/{path}', {
    owner: 'rikhuijzer',
    repo: 'notes',
    path: 'README.md',
    headers: {
      'X-GitHub-Api-Version': '2022-11-28'
    }
  })

  const contentDecoded = decodeContent(resp.data.content);
  return [contentDecoded, resp.data.sha];
}

async function updateNotes(octokit, content, sha, addition) {
  // Check if content contains today in 2024-04-16 format.
  const today = new Date().toISOString().split('T')[0];
  content = content.trim();
  const new_content = content.includes(today) ?
    content + '\n\n' + addition :
    content + '\n\n# ' + today + '\n\n' + addition;

  await octokit.request('PUT /repos/{owner}/{repo}/contents/{path}', {
    owner: 'rikhuijzer',
    repo: 'notes',
    path: 'README.md',
    message: '[bot] Update notes',
    committer: {
      name: 'Rik\'s bot',
      email: '[email protected]'
    },
    content: encodeContent(new_content),
    sha: sha,
    headers: {
      'X-GitHub-Api-Version': '2022-11-28'
    }
  })
}

async function handleEmail(env, message) {
  const parser = new PostalMime()
  const parsedEmail = await parser.parse(message.raw);
  console.log("Mail subject: ", parsedEmail.subject);
  console.log("Text version of Email: ", parsedEmail.text);

  const octokit = authorized_octokit(env.GITHUB_TOKEN);
  const [content, sha] = await readNotes(octokit);
  await updateNotes(octokit, content, sha, parsedEmail.text);
}

export default {
  async email(message, env, ctx) {
    const allowList = [
      "<your-email>@example.com"
    ];
    if (!allowList.includes(message.from)) {
      message.setReject("Address not allowed");
      return;
    }
    await handleEmail(env, message);
  }
}

the following .github/workflows/ci.yml file:

name: ci

on:
  push:
    branches:
      - main
  pull_request:
  workflow_dispatch:

jobs:
  run:
    runs-on: ubuntu-latest
    timeout-minutes: 15
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-node@v4
        with:
          node-version: '20'

      - run: npm install -g [email protected]

      - run: npm install

      - run: wrangler deploy --dry-run --outdir=dist

      - if: github.ref == 'refs/heads/main'
        run: wrangler deploy
        env:
          CLOUDFLARE_ACCOUNT_ID: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }}
          CLOUDFLARE_API_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN }}

the following wrangler.toml file:

name = "update-notes"
main = "worker.js"
compatibility_date = "2023-03-22"

[vars]

and created a package.toml and package-lock.toml by running

$ npm install postal-mime octokit

where the versions are respectively 2.2.5 and 3.2.0 at the time of writing.

Now, if you add the CLOUDFLARE_ACCOUNT_ID and CLOUDFLARE_API_TOKEN secrets to the GitHub repository, you can create and update the worker by pushing updates to the main branch.

Finally, to link the worker to the email, I configured an Email Worker in the Cloudflare dashboard for my domain. Inside the Cloudflare dashboard, I set the Routing rules in Email to Drop for all emails. And added an Email Worker which listens to a Custom address (e.g., [email protected]).

Conclusion

With this setup, it takes less than a minute to update my notes. First, it takes about 20 seconds to open my email app, type "update" and click on "[email protected]" in the to field, and write my notes in the body.

Next, according to Cloudflare's metrics, the worker takes 10ms of CPU time, which means this setup can be run on a free Cloudflare account. In practice, the whole process from sending an email to updating the notes takes about 5 to 30 seconds since email delivery can take a while.

So, all in all, this setup now means that I can add notes to a safe location in less than a minute; even when I'm on the go. Finally, whenever a new observation or thought comes across me, I have a good way to "make a memorandum of it without fail and at once".