Syncing org mode reminders to my bot

This post is about the chatbot I created to managed my reminders and notes. If you missed the first post introducing the bot check out this post.

In this post I am describing how I sync reminders from org-mode on my laptop to my bot.

Design

To sync the reminders I decided on the following approach:

Parse the current file with the org mode parser
Find all the reminders
Put them all in a format that the backend understands into a file
Sync the reminders to the datastore
Notify all the backends using the reminders that they should reload

Implementation

Reminders format

I currently store my reminders in plain text using org mode. A reminder typically look like this:

* Columbia CS@CU Happy Hour
:PROPERTIES:
:REMINDER_DATE: <2017-07-15 Sat 12:00>
:REMINDER_TARGET: laurent
:REMINDER_QUICK_REPLY: yes
:END:

Parsing the org mode file

The following method parses the currently open org mode file and iterates through all the headlines. It keeps only the ones that pass the test laurent/valid-reminder.

  (defun laurent/reminders-current-buffer ()
    "Return the list of org element that represent reminders
     from the current buffer"
    (let ((reminders
          (org-element-map (org-element-parse-buffer) 'headline #'identity)))
      (cl-remove-if-not #'laurent/valid-reminder reminders)))

Filtering valid reminders

So what is a valid reminder? We looked at an example above

* Columbia CS@CU Happy Hour
:PROPERTIES:
:REMINDER_DATE: <2017-07-15 Sat 12:00>
:REMINDER_TARGET: laurent
:REMINDER_QUICK_REPLY: yes
:END:

It should contain a target, and some date specification. It can be an absolute date like here or it can also be repeating:

* Gym Monday
  :PROPERTIES:
  :REMINDER_TARGET: laurent
  :REMINDER_HOUR: 18
  :REMINDER_MINUTE: 0
  :REMINDER_WEEKDAY: 0
  :REMINDER_MESSAGE: It's monday night, workout night! 
  :END:

org-mode's element API exposes functions to access the property of elements, let's use them to express what a valid reminder is:

  (defun laurent/valid-reminder (x)
     "Given an org element returns truthy if it
      is a valid reminder, aka it has a target and
      some time specs supported by the system"
    (and
      (org-element-property :REMINDER_TARGET x)
      (or
        (org-element-property :REMINDER_DATE x)
        (org-element-property :REMINDER_HOUR x)
        (org-element-property :REMINDER_MINUTE x)
        (org-element-property :REMINDER_WEEKDAY x))))

So far we have parsed the current buffer and collected a list of element that are valid reminders. Time to format them to a format that the backend understands: JSON!

Formatting reminders in JSON

Here is how I convert the reminders. I used json-encode to encode the reminder list.

(defun laurent/write-reminders (filename)
  "Write reminders of the current buffer to a file as JSON"
  (let ((reminder-list 
          (mapcar #'laurent/format-reminder (laurent/reminders-current-buffer))))
    (with-temp-buffer
      (insert (json-encode reminder-list))
      (json-pretty-print-buffer)
      (write-file filename))))

As you can notice we don't encode directly laurent/reminders-current-buffer but preprocess each reminder with laurent/format-reminder.

This is because the backend expects all the fields of the reminders to be defined even if they are null. It also expects every entry in the JSON to be properly casted.

That's the responsibility of laurent/format-reminder, to format the reminder to look like what the backend expects:

  (defun laurent/tonum (x)
     "If x is nil return it, otherwise cast x to a number"
     (if (eq x nil)
        nil
       (string-to-number x)))

  (defun laurent/format-reminder (x) 
    "Given an org element representing a reminders 
     make it into a list of cons cell key value pair.
     Keeping only the properties relevant for reminders and
     casting all properties to their expected type"
    (list
      (cons 'title (car (org-element-property :title x)))
      (cons 'target (org-element-property :REMINDER_TARGET x))
      (cons 'hour (laurent/tonum (org-element-property :REMINDER_HOUR x)))
      (cons 'minute (laurent/tonum (org-element-property :REMINDER_MINUTE x)))
      (cons 'day_of_week (laurent/tonum (org-element-property :REMINDER_WEEKDAY x)))
      (cons 'quick_reply (org-element-property :REMINDER_QUICK_REPLY x))
      (cons 'date
            (let ((date (org-element-property :REMINDER_DATE x)))
              (if (eq date nil) nil (substring date 1 -1))))
      (cons 'timezone "America/Los_Angeles")
      (cons 'message  (or (org-element-property :REMINDER_MESSAGE x) (car (org-element-property :title x))))))

Syncing the reminders to the datastore

I wrote a quick python script to copy a file to a redis key. It assumes that redis is running on localhost on the usual redis port.

#!/usr/bin/env python3
import redis
import sys
import time

reminder_file = sys.argv[1]
attempt = 0

with open(reminder_file) as f:
    while attempt < 5:
        print("Attemption connection")
        try:
            attempt += 1
            r = redis.Redis()
            r.ping()
            break
        except Exception as e:
            print(e)
            print("Retrying in 0.5s")
            time.sleep(0.5)
    if attempt >= 5:
        print("Failed to connect")
        sys.exit(1)
    print("Syncing reminders")
    r.set("reminders.json".encode("utf-8"), f.read())
    print("DONE adding the reminders to REDIS")

In order to call it, I first set up port forwarding with the remote redis instance:

  (defun laurent/start-port-fwd (server port)
    "Start port forwarding to a server (ex: \"[email protected]\") and a port 
     like \"4564\""
    (start-process 
      "port-forwarding" 
      "*port-fwd*" 
      "ssh" "-L" (concat port ":localhost:" port) server "-N" ))

All that is left is to tell the backend that the reminders have been updated and put it all together in an interactive function I can call:

(defun laurent/sync-reminders ()
  "Export, sync reminders to the server and reload the reminders on the bot"
  (interactive)
  (let* ((server "[email protected]")
          (redisport "6379")
          (repopath "~/repos/docker_apps")
          (syncprogram (concat repopath "/APPS/backend/add_reminders.py"))
          (tempfile "/tmp/reminders.json")
          (reloadurl "localhost:3000/reload_reminders")
          (reloadcmd (concat "\"for id in \\$(docker ps -q --filter 'name=backend'); do  docker exec -t \\$id  curl -X POST " reloadurl " ;done\""))
          (port-fwd-process (laurent/start-port-fwd server redisport)))
    (message "== Starting Sync ==")
    (message "Exporting reminders")
    (laurent/write-reminders tempfile)
    (message "Copy reminders on server")
    (message (shell-command-to-string (concat syncprogram " " tempfile)))
    (delete-process port-fwd-process) ;; Stop port forwarding
    (message "Reloading reminders")
    (message (shell-command-to-string (concat "ssh " server " " reloadcmd)))
    (message "== DONE with Sync ==")))