Architecting a Better Org Workflow

There's a new carnival in town, and I'm joining in. The theme for this month is "Writing Experience", and boy what an experience it is. About 80% of my daily usage of Emacs is writing, but some parts of my workflow are tedious, or worse yet completely unused, and I've had some quality of life stuff that I've been itching to implement for a while. My configuration is not particularly well formatted either. So in all, I think it's a good time to do a bit of Winter cleaning (Southern Hemisphere rules) and build a better configuration for writing specifically.

Before looking at what my config looks like currently, let's look at what I'm actually using Emacs to write, since a use case is a good place to start from. When I write in Emacs, 99.9% of the time I'm writing in org-mode (the other 0.01% of the time is in markdown-mode with Emacs Everywhere, but I won't cover that here). By my own self-reflection, this is what I mostly use Emacs for in terms of writing (and in no particular order):

Fiction Writing:
- Novels or Short Stories
- Worldbuilding
Technical Writing:
- Essays and Mini-essays
- Notes, and To-Dos
- Knowledge Management
- Literate Programming
Blog Posts
Tabletop RPG Rules

The requirements for these are all relatively distinct, and my flow for some is more streamlined than others. For example, I'm quite happy with my RPG rules writing setup, at least for my current project; and I'm also pretty happy with Weblorg and blog writing/publishing. I feel like my main points of contention are essay writing and personal knowledge management. So let's have a look at what I'm doing now before I start thinking how to do it better.

I initially started writing this with the intention to do up my fiction writing and TTRPG flows, but it really turned into a complete overhaul of my second brain/PKDB setup. Sorry about that!

The Now and Reflection

This is a list of every external package I'm currently using when writing in org:

wc-mode
org-modern
powerthesaurus
eglot with Harper
org-novelist
org-roam
company for use with Harper and Ispell
org-transclusion
yasnippet
weblorg and templatel

I'm not including any packages like ob-racket that enable more language support for babel.

wc-mode is invaluable to me, however I wouldn't mind getting rid of the dependency and trying to implement it myself. It would be good practice with Elisp. For the sake of not reinventing the wheel, let's not do that here.

org-modern is excellent, but my use of it could probably be better. The incompatibility with org-indent bugs me in particular, which probably means I need to have a look at org-modern-indent instead.

powerthesaurus is also invaluable, more-so than wc-mode. However, it would be good to have at least some of this functionality available offline, since I don't always write with an internet connection.

eglot and Harper are excellent, but I want to tweak some of the rules. Unfortunately, Harper doesn't have native support for org so there will always be some incompatibility until then, but I'm willing to put up with that until it gets implemented (if it does).

org-novelist is… good, but I don't make enough use of it to justify it's inclusion. I don't do enough story writing anymore, and the structure is not particularly helpful for worldbuilding or other document types.

org-roam is the most contentious thing here. I feel like my "organisation system", meaning my to-do files, commonplace notebook, and PKDB are where I see the biggest friction currently. Sure my current setup works, but it feels disorganised, and I don't use org-roam as much as I should - probably because the main killer feature, backlinks, just doesn't jive well with how my head works. It also doesn't have any good spaced repetition implementation, which I would like to have.

company. I'm not going to touch this too much; I already did that in my previous blog about Harper, and I'm quite happy in the state that it's in.

org-transclusion. I installed this on a whim because I thought it might be helpful in making org-roam fit better in my head, but I basically haven't touched it since. It might be time for this to go.

yasnippet is great, I use it all the time for things like date insertion, but I wouldn't mind adding some more templates to skeleton some document types.

I don't have much to say about weblorg, you're reading this right now thanks to it. I have two issues here: firstly, citations don't work, and secondly the table of contents links don't work. I will try to fix these, but I'm doubtful that I can. We will see.

Stuff I Want To Have

Big one: I'm crap at writing Bibtex and I use citation generators. The issue is that these are outside Emacs, which causes friction. I think biblio should solve this.

The biggest thing I would absolutely love, is for my PKDB to be easily accessible and editable on my phone. Yes, I know Emacs can run on Android, but it's not exactly a pleasant experience - nor should it be, it's a touch interface not a keyboard interface. I've kinda tried to work around this with a uConsole, but, eh, honestly that workflow has it's own problems. That whole device has it's own usability problems for me that I want to address separately. Back to the main desire here: ideally, I don't want to program my own app in Flutter (although that would be pretty neat, imagine an interactive org-roam-ui) because it's too time consuming currently. I use Orgzly Revived on mobile and I would like my note-taking to be compatible with it; or, in lieu of that, have it be compatible with Logseq (which actually seems like it may be the better solution here).

While we're on the topic of mobile, being able to share some files with my girlfriend would be useful - for example to track a shared grocery list. She is using an iPhone, which might make this more complicated.

I want to start journaling proper. I've been having fun with solo journaling TTRPGs lately and I think it's not a bad idea to integrate that into my daily life, even if it's just a paragraph or so.

If you're not familiar with mini-essays, I recommend watching this video. The takeaway is that I already sort of do this in org-roam, but that's not exactly the most frictionless setup. Zettelkasten is not made for this type of thing in my opinion, but that may just be because of my organisation.

While I'm loving Harper for picking up errors, I also want to be able to inspect the text style of my writing. Currently, I'm doing this with Expresso but having this available within Emacs would be greatly beneficial.

I want some cleaner text selection commands. Particularly, I want mark-word to select the whole word at the cursor and not just to the end of it. I also want something like mark-sentence (forward-sentence is also a bit buggy, so fixing that might be a good idea). Also along these lines, I want to implement some functionality from essay.app, specifically sentence/paragraph reordering and "focused rewriting".

In general I want to change some key-binds to use custom multi-key selection menus through transient, because I find that having such a single entry-point will make it less likely I forget things.

More to the TTRPG side, I want to be able to do dice rolls via the standard syntax (xDy+z) and create roll tables that I can reference remotely easily. Card decks would also be very helpful.

Finally, I'm currently synchronising org files across devices with git. This sucks, and I want to swap to Syncthing as part of this.

OK, that was a lot of words. I'm kind of using this document as a brain dump, so apologies for that in advance. I think though, I have a good idea of what I want to do to improve my experience in Org now. So, it's time to get to work.

Structuring

We need a cool project name for this workflow. Because why not. I'm in a phase of naming things after alchemical/hermetic terms currently (see animus and ouroboros-emacs-themes) so let's stick with that theme - I'll call it "org-logos".

My Emacs configuration depends on straight, and I will be using it here along with use-package. Besides that, I like to keep everything as self-contained as possible. If I'm referencing a package I have installed globally and not as part of this specific workflow, I'll bring this up here.

The best place to start is with configuring org-mode itself, before getting into any additional packages. This comes in 2 parts; one is changing a bunch of variables, and the other is adding an initialisation function which calls a bunch of other functions as a hook. Let's start with the latter.

(defun org-mode-init ()
  "This is called through org-mode-hook, so that I don't need to add-hook 20 million times."
  ;; I want org to display pictures inline.
  (org-display-inline-images)
  ;; I don't want monospace text by default.
(variable-pitch-mode))

(add-hook 'org-mode-hook #'org-mode-init)

Regarding variable-pitch-mode, ouroboros-emacs-themes already supports this across all themes, including monospace formatting for code blocks and verbatims, so I won't go into my configuration for that here.

We'll add more lines to this as we go along. For now this is a good place to start. Onto the variable changes next.

(setq org-log-done 'time ;; I want to keep track of the time tasks are completed.
    org-return-follows-link t ;; I want to be able to follow links easily.
    org-hide-emphasis-markers t ;; This is a big one: hide markup syntax like ** and //.
    org-pretty-entities t ;; These two variables together render latex entities as UTF8, which means that Greek letters, subscript, and superscript are all rendered correctly.
    org-pretty-entities-include-sub-superscripts t
    org-enforce-todo-dependencies t ;; Prevent changing a parent to DONE if the children aren't
    org-enforce-todo-checkbox-dependencies t
    org-agenda-skip-scheduled-if-done t ;; Don't show done stuff in the agenda view
    org-agenda-skip-deadline-if-done t
    org-agenda-skip-timestamp-if-done t
    org-todo-keywords '((sequence "TODO" "WIP" "DELEGATED" "WAITING" "|" "CANCELLED" "DONE")) ;; Some extra keywords in addition to TODO and DONE
    org-refile-targets '((org-agenda-files :maxlevel . 2)) ;; Allow any agenda files to be refile targets
    )

I want to keep all my org agenda files in ~/.org, so I'll be hard-coding that value for this next variable:

(setq org-cite-global-bibliography '("~/.org/global.bib") ;; Global bibliography location)

Agenda and The Exobrain - Organisation

Since I'm already looking at the org-agenda-files variable, this might be a good time to think about the structure of my agenda files and PKDB. Let's abstract a little.

There is an excellent article by Justin Wernick about his exobrain, and another good workshop on Guild of the Rose about the same topic. There are essentially three components that I want:

Task tracking and management: this is what org agenda is currently useful for.
Note taking: this is what most people use org-roam for.
Knowledge refinement: this is what I've been using org-roam for, and where things like mini-essays come in. Flashcards can also fall into this category.

Here's the issue that I've realised while writing this post: Zettelkasten is a great methodology for taking notes, specifically short-hand notes, but it's a terrible methodology for long-form knowledge storage in my opinion. I really want to stress that this is just my personal view - everyone's brains work different - but for example having I've been treating org-roam like a wiki with 1 node for each topic (1 for Python, 1 for SQL, etc). I think that having 1 node for an individual snippet of information, and linking everything with index cards, will be much more useful for synthesis at the refinement stage. That way I can also see backlinks being helpful for me.

OK, so what can I use to cover the knowledge refinement step? Well, I don't think it needs anything complex: in fact I think splitting the org-roam directory into multiple folders is the solution here.

Prettifying

Before we get too stuck in on exobrain optimisation, let's first add some basic user experience enhancements. I like using org-modern, but I also want to fix some minor code block formatting issues that I have with org-indent-mode.

  (use-package org-modern
  :hook ((org-mode . org-modern-mode)
         (org-agenda-finalize . org-modern-agenda)))

(use-package org-modern-indent
  :straight (org-modern-indent :type git :host github :repo "jdtsmith/org-modern-indent")
  :hook ((org-mode . org-modern-indent-mode)))

Also need to set org-startup-indented to t in the big setq block. I also want wc-mode, as stated before.

(use-package wc-mode
  :hook ((org-mode . wc-mode))
  :config (setq wc-modeline-format "WC[%tw]"))

That about covers everything I need or want as far as beautification is concerned.

Generally Useful Stuff

I want to work a bit on the stuff that's going to be generally useful no matter the form of writing.

Let's start with what's been bothering me the most: sentence-based navigation and word/sentence selection. Emacs already has a backward- and forward-sentence function but they don't work for me - they jump to the start and end of the paragraph. The reason for this is very simple and very (in my opinion) stupid: sentence-end-double-space is t by default, which means that a sentence needs to end with two spaces following a period for Emacs to recognise it. Easy enough fix, just setq to nil.

With that fixed, we can have some fun with creating custom marking functions for sentences and words, since those don't behave properly either (by design).

  (defun mark-whole-word ()
  "Marks the whole word underneath the cursor."
  (interactive)
  (forward-word)
  (set-mark-command nil)
  (backward-word))

(defun mark-sentence ()
  "Marks the entire sentence underneath the cursor."
  (interactive)
  (forward-sentence)
  (set-mark-command nil)
  (backward-sentence))

Let's also have some fun with re-ordering sentences and paragraphs. These aren't perfect (for example, paragraph moving can be messy with code blocks) but they're "good enough". I've even made them compatible with C-u.

  (defun move-sentence-right (&optional arg)
  "Moves the whole sentence to the right of the next sentence."
  (interactive "^p")
  (or arg (setq arg 1))
  (while (> arg 0)
    (forward-sentence)
    (backward-sentence)
    (left-char) ;; Capture previous space
    (kill-sentence)
    (forward-sentence)
    (yank)
    (backward-sentence)
    (setq arg (1- arg))))

(defun move-sentence-left (&optional arg)
  "Moves the whole sentence to the left of the previous sentence."
  (interactive "^p")
  (or arg (setq arg 1))
  (while (> arg 0)
    (forward-sentence)
    (backward-sentence)
    (left-char) ;; Capture previous space
    (kill-sentence)
    (backward-sentence)
    (left-char) ;; Capture previous space
    (yank)
    (backward-sentence)
    (setq arg (1- arg))))

(defun move-paragraph-up (&optional arg)
  "Moves the whole paragraph above the previous paragraph."
  (interactive "^p")
  (or arg (setq arg 1))
  (while (> arg 0)
    (forward-paragraph)
    (backward-paragraph)
    (kill-paragraph nil)
    (backward-paragraph)
    (yank)
    (backward-paragraph)
    (setq arg (1- arg))))

(defun move-paragraph-down (&optional arg)
  "Moves the whole paragraph above the previous paragraph."
  (interactive "^p")
  (or arg (setq arg 1))
  (while (> arg 0)
    (forward-paragraph)
    (backward-paragraph)
    (kill-paragraph nil)
    (forward-paragraph)
    (yank)
    (backward-paragraph)
    (setq arg (1- arg))))

If you're not aware, you can already narrow the buffer to a paragraph with narrow-to-defun. That's not enough to give me "focused rewriting" by itself though. Let's implement that next, by copying from this video.

(defun break-out-sentence ()
  "Breaks a sentence out into it's own buffer for editing."
  (interactive)
  (backward-sentence)
  (kill-sentence)
  (let ((buf (generate-new-buffer "*break-out*"))
      (window (split-window-below -10)))
    (set-window-buffer window buf)
    (select-window window))
  (erase-buffer)
  (yank)
  (org-mode))

(defun break-out-choose-sentence ()
  "Chooses a sentence from the break-out buffer."
  (interactive)
  (backward-sentence)
  (kill-sentence)
  (other-window -1)
  (yank)
  (select-window (get-buffer-window "*break-out*"))
  (kill-buffer-and-window))

Powerthesaurus is next. I don't need such a complex configuration for this.

(use-package powerthesaurus
  :bind ("C-x t" . powerthesaurus-transient))

I'm going to use C-c o as my personal command prefix in Org, but Powerthesaurus is generally useful everywhere so I want to keep it on an easily accessible binding. On that note:

  (defun break-out-dwim ()
  "Either break-out or choose sentency depending on buffer name."
  (interactive)
  (if (equal (buffer-name) "*break-out*")
      (break-out-choose-sentence)
    (break-out-sentence)))

(define-key org-mode-map (kbd "C-c o b") #'break-out-dwim)

Let's also bind some keys for the paragraph and sentence manipulation from before, and marking.

(define-key org-mode-map (kbd "C-M-<left>") #'move-sentence-left)
(define-key org-mode-map (kbd "C-M-<right>") #'move-sentence-right)
(define-key org-mode-map (kbd "C-M-<up>") #'move-paragraph-up)
(define-key org-mode-map (kbd "C-M-<down>") #'move-paragraph-down)

(define-key org-mode-map (kbd "C-@") #'mark-whole-word)
(define-key org-mode-map (kbd "M-@") #'mark-sentence)

I also want to put all of these manipulation keys into a transient menu.

(require 'transient)

(transient-define-prefix reorder-transient ()
  "Transient menu for text re-ordering commands."
  ["Move sentence..." ("l" "Left" move-sentence-left)
   ("r" "Right" move-sentence-right)]
  ["Move paragraph..." ("u" "Up" move-paragraph-up)
   ("d" "Down" move-paragraph-down)])

(transient-define-prefix mark-menu-transient ()
  "Transient menu for marking units of text."
  ["Mark" ("w" "Word" mark-whole-word)
   ("s" "Sentence" mark-sentence)
   ("p" "Paragraph" mark-paragraph)
   ("b" "Buffer" mark-whole-buffer)])

(define-key org-mode-map (kbd "C-c o SPC") #'mark-menu-transient)
(define-key org-mode-map (kbd "C-c o r") #'reorder-transient)

Apparently this wouldn't be a post on my blog without glazing Harper, so let's add that in here quickly.

(when (and (not (equal system-type 'windows-nt)) (locate-file "harper-ls" exec-path))
  (with-eval-after-load 'eglot
    (add-to-list 'eglot-server-programs
               '(org-mode . ("harper-ls" "--stdio"))))

  (setq-default eglot-workspace-configuration
              '(:harper-ls (:dialect "Australian")))

  (add-hook 'org-mode-hook 'eglot-ensure))

If you're curious, these are my bindings for using Eglot with Flymake to fix up errors. These are globally useful bindings, so I won't add them into the document here.

(global-set-key (kbd "M-g <left>") 'flymake-goto-prev-error)
(global-set-key (kbd "M-g <right>") 'flymake-goto-next-error)
(global-set-key (kbd "C-x M-f") 'eglot-code-actions)

Also a good idea to set up company with all the right backends.

(defun org-mode-company-hook ()
    "Set company backends for `org-mode' usage."
    (setq-local company-backends
            '((company-capf company-dabbrev company-files company-ispell))))
(add-hook 'org-mode-hook #'org-mode-company-hook)

That seems to be all the generally applicable stuff, except for text style metrics which I'm putting off indefinitely because I'm too stupid/lazy to implement it. Now it's time for the other thing I've been too lazy to touch.

Exobrain 1: Task Management

Task management needs to be logically organised, and after a lot of thinking I've arrived on something that's a mix of GTD, PARA, and Justin Wernick's system.

First we need an inbox/capture step. In my ~/.org directory, I'm going to have a directory structure like so:

.
├── lore
│   ├── raw
│   └── refined
│       ├── journal
│       └── wiki
└── tasks
    ├── maintenance
    └── projects
	└── career

I'll touch on lore in the next two Exobrain sections, tasks is what's important here. Outside of this structure, I have ~/.org/inbox.org, which is where I will dump absolutely everything from new tasks to quick notes. All of this will be refiled to the right place later.

Each file in maintenance is for tracking related tasks, such as house related stuff, and has a short preface to detail the area it refers to, followed by sections for scheduled tasks ("Calendar"), one-off tasks ("TO-DO"), recurring tasks ("Routines"), and finished tasks ("Archive"). I never really want to delete any tasks that build up (although I may modify recurring tasks instead of creating a new task) because you never truly know if you'll need that context in the future.

Projects are similar to maintenance tasks, except each of the above sections is a level 2 heading under a level 1 "Work" section (which also has a TO-DO state). There are also level 1 sections for acceptance criteria and ideas relevant to the project. Career projects are just tasks for my current job - I want to keep these separate from other tasks.

It seems like a pain to manually specify each file here as an agenda file, so let's do it automatically.

(setq org-agenda-files (apply 'append (mapcar
                                 (lambda (dir)
                                   (directory-files-recursively dir org-agenda-file-regexp))
                                 '("~/.org/tasks"))))

I also want to make some functions for creating new maintenance areas and projects.

(defun new-maintenance-file (maintenance-area)
  "Create a new maintenance file for a given `MAINTENANCE-AREA'."
  (interactive "sMaintenance area is: ")
  (find-file (concat "~/.org/tasks/maintenance/" maintenance-area ".org"))
  (insert "\n\n* Calendar\n\n* TO-DO\n\n* Routines\n\n* Archive\n")
  (goto-char (point-min)))

(defun new-project-file (project-name acceptance-criteria)
  "Create a new project file for a given `PROJECT-NAME', with some `ACCEPTANCE-CRITERIA'."
  (find-file (concat "~/.org/tasks/projects/" project-name ".org"))
  (insert (concat "\n\n* Acceptance Criteria\n\n" acceptance-criteria "\n\n* Ideas\n\n* TODO Work\n\n** Calendar\n\n** TO-DO\n\n** Routines\n\n** Archive\n"))
  (goto-char (point-min)))

(defun new-generic-project (project-name acceptance-criteria)
  "Interactive wrapper for `new-project-file', for generic projects."
  (interactive "sProject name is: \nsAcceptance criteria is: ")
  (new-project-file project-name acceptance-criteria))

(defun new-career-project (project-name acceptance-criteria)
  "Interactive wrapper for `new-project-file', for career projects."
  (interactive "sProject name is: \nsAcceptance criteria is: ")
  (new-project-file (concat "career/" project-name) acceptance-criteria))

Now while we do have an inbox file, sometimes I know where something needs to go already and when I capture I would like to be able to specify the target location. This function can do that:

(require 'subr-x)

(defun select-task-area (location headline)
  "Prompt for a file in ~/.org/tasks/maintenance/ to insert the capture."
  (let* ((files (directory-files (concat "~/.org/tasks/" location) t "\\.org$"))
       (choices (mapcar
                 (lambda (x) (string-remove-suffix ".org" x))
                 (mapcar #'file-name-nondirectory files)))
       (selected (completing-read "Choose a file: " choices nil t)))
  (set-buffer (org-capture-target-buffer (concat "~/.org/tasks/" location selected ".org")))
  (org-capture-put-target-region-and-position)
  (widen)
  (goto-char (point-min))
  (setq headline (org-capture-expand-headline headline))
  (re-search-forward (format org-complex-heading-regexp-format
                             (regexp-quote headline))
                     nil t)
  (forward-line 0)))


(defun select-maintenance-area (headline)
  (select-task-area "maintenance/" headline))

(defun select-project (headline)
  (select-task-area "projects/" headline))

(defun select-career-project (headline)
  (select-task-area "projects/career/" headline))

The astute Emacs guru may realise this is absolutely disgusting. It also doesn't work properly, and gives a bunch of cl-assertion-failed errors on C-c C-c. It also widens to the entire org file after insertion. So it's not exactly smooth, but it does the job.

Anyway, we can use this to make capture templates now.

(setq org-capture-templates
      '(("i" "Idea")
      ("ii" "Generic" entry
       (file "~/.org/inbox.org")
       "* %^{TITLE} :idea: \n:Created: %T\n%?"
       :empty-lines 1)
      ("ip" "Project" entry
       (function (lambda () (select-project "Ideas")))
       "* %^{TITLE} \n:Created: %T\n%?"
       :empty-lines 1)
      ("ic" "Career" entry
       (function (lambda () (select-career-project "Ideas")))
       "* %^{TITLE} \n:Created: %T\n%?"
       :empty-lines 1)

      ("t" "Task")
      ("tt" "Generic")
      ("ttt" "Raw" entry
       (file "~/.org/inbox.org")
       "* TODO [%^{priority|#A|#B|#C|#D}] %^{TITLE} :task:\n:Created: %T\n%?"
       :empty-lines 1)
      ("ttm" "Maintenance" entry
       (function (lambda () (select-maintenance-area "TO-DO")))
       "* TODO [%^{priority|#A|#B|#C|#D}] %^{TITLE}\n:Created: %T\n%?"
       :empty-lines 1)
      ("ttp" "Project" entry
       (function (lambda () (select-project "TO-DO")))
       "* TODO [%^{priority|#A|#B|#C|#D}] %^{TITLE}\n:Created: %T\n%?"
       :empty-lines 1)
      ("ttc" "Career" entry
       (function (lambda () (select-career-project "TO-DO")))
       "* TODO [%^{priority|#A|#B|#C|#D}] %^{TITLE}\n:Created: %T\n%?"
       :empty-lines 1)
      ("ts" "Scheduled")
      ("tss" "Raw" entry
       (file "~/.org/inbox.org")
       "* TODO [%^{priority|#A|#B|#C|#D}] %^{TITLE} :scheduled:\n:SCHEDULED: %^T\n%?"
       :empty-lines 1
       :time-prompt t)
      ("tsm" "Maintenance" entry
       (function (lambda () (select-maintenance-area "Calendar")))
       "* TODO [%^{priority|#A|#B|#C|#D}] %^{TITLE} \n:SCHEDULED: %^T\n%?"
       :empty-lines 1
       :time-prompt t)
      ("tsp" "Project" entry
       (function (lambda () (select-project "Calendar")))
       "* TODO [%^{priority|#A|#B|#C|#D}] %^{TITLE} \n:SCHEDULED: %^T\n%?"
       :empty-lines 1
       :time-prompt t)
      ("tsc" "Career" entry
       (function (lambda () (select-career-project "Calendar")))
       "* TODO [%^{priority|#A|#B|#C|#D}] %^{TITLE} \n:SCHEDULED: %^T\n%?"
       :empty-lines 1
       :time-prompt t)

      ("n" "Note")
      ("nn" "Raw" entry
       (file "~/.org/inbox.org")
       "* %^{TITLE} :note: \n:Created: %T\n%?"
       :empty-lines 1)
      ("nm" "Meeting")
      ("nmm" "Raw" entry
       (file "~/.org/inbox.org")
       "* %^{TITLE} :meeting: \n:Created: %T\n%?"
       :clock-in t
       :clock-resume t)
      ("nmc" "Career" entry
       (function (lambda () (select-career-project "Calendar")))
       "* %^{TITLE} \n:Created: %T\n%?"
       :empty-lines 1
       :clock-in t
       :clock-resume t)))

Wow that's a lot of words! It's a bit messy (it could definitely be better by putting all the magic strings into variables) but it gets the job done. One final thing, I want to expose all of these commands under a transient menu, and make some bindings for org-capture and org-agenda.

(defun open-inbox ()
  "Opens `~/.org/inbox.org'"
  (interactive)
  (find-file "~/.org/inbox.org"))

(defun open-maintenance ()
  "Opens `~/.org/tasks/maintenance/'"
  (interactive)
  (find-file "~/.org/tasks/maintenance/"))

(defun open-projects ()
  "Opens `~/.org/tasks/projects/'"
  (interactive)
  (find-file "~/.org/tasks/projects/"))

(defun open-career ()
  "Opens `~/.org/tasks/projects/career/'"
  (interactive)
  (find-file "~/.org/tasks/projects/career/"))

(transient-define-prefix task-management-menu ()
  "Transient menu for task management shortcuts."
  ["Go To" ("i" "Inbox" open-inbox)
   ("m" "Maintenance Directory" open-maintenance)
   ("p" "Projects Directory" open-projects)
   ("c" "Career Directory" open-career)]
  ["Create New" ("M" "Maintenance File" new-maintenance-file)
   ("P" "Project" new-generic-project)
   ("C" "Career Project" new-career-project)])

(define-key global-map (kbd "C-c o t") #'task-management-menu)  
(define-key global-map (kbd "C-c a") #'org-agenda)
(define-key global-map (kbd "C-c a") #'org-capture)

And in terms of task management, that's pretty much it. I'm going to ignore Outlook and Jira integration for now because it's a pain (especially if I want it to be portable), and Orgzly Revived will give me an interface to this on mobile (though without the capture templates obviously).

Exobrain 2: Note Taking

As I said earlier, my goal is to use org-roam for capturing notes and snippets, instead of using it as a wiki (which should be reserved for refinement), chained together by index cards. Both this "web" of raw notes and the journals from the next section should work with Logseq, so we need to do some configuration on that end. I'm following closely this blog, as well as this one. You may remember the previously mentioned structure of ~/.org - all the raw notes I'm capturing with org-roam will be in lore/raw.

(use-package org-roam
  :straight nil
  :ensure t
  :init
  (setq org-roam-v2-ack t)
  :custom
  (org-roam-directory (file-truename "~/.org/lore/raw/"))
  (org-dailies-directory (file-truename "~/.org/refined/journal/"))
  (org-roam-file-exclude-regexp "\\.git/.*\\|logseq/.*$")
  (org-roam-completion-everywhere)
  (org-roam-capture-templates
    '(("d" "default" plain
       "%?"
       :target (file+head "${slug}.org" "#+title: ${title}\n")
       :unnarrowed t)))
  (org-roam-dailies-capture-templates
    '(("d" "default" entry
       "* %?"
       :target (file+head "%<%Y-%m-%d>.org" ;; format matches Logseq
                          "#+title: %<%Y-%m-%d>\n"))))
  :bind (("C-c n f" . org-roam-node-find)
       ("C-c n i" . org-roam-node-insert)
       :map org-roam-dailies-map
       ("Y" . org-roam-dailies-capture-yesterday)
       ("T" . org-roam-dailies-capture-tomorrow))
  :bind-keymap ("C-c n d" . org-roam-dailies-map)
  :config
  (require 'org-roam-dailies) ;; Ensure the keymap is available
  (org-roam-db-autosync-mode))

This configuration sets up all the binds for dailies and nodes. It also sets the filenames for org-roam notes and dailies to be the same as what Logseq is using. I should not that Logseq cannot have multiple pages directories, so it will never be usable for mini-essays, but I don't really care since long writing on a phone is terrible anyway.

I'm going to add a (setq org-attach-id-dir "~/.org/lore/assets") so that Logseq can store and see attachments. Logseq also has some interoperability kinks regarding links, but thankfully this package should sort everything out:

(use-package logseq-org-roam
  :straight (:host github
          :repo "sbougerel/logseq-org-roam"
          :files ("*.el")))

As some notes might go to the inbox, I want a function to refile them to org-roam. There's a good forum thread that has a great example for this functionality.

(defun org-roam-create-note-from-headline ()
  "Create an Org-roam note from the current headline and jump to it.

Normally, insert the headline’s title using the ’#title:’ file-level property
and delete the Org-mode headline. However, if the current headline has a
Org-mode properties drawer already, keep the headline and don’t insert
‘#+title:'. Org-roam can extract the title from both kinds of notes, but using
‘#+title:’ is a bit cleaner for a short note, which Org-roam encourages."
  (interactive)
  (let ((title (nth 4 (org-heading-components)))
        (has-properties (org-get-property-block)))
    (org-cut-subtree)
    (org-roam-find-file title nil nil 'no-confirm)
    (org-paste-subtree)
    (unless has-properties
      (kill-line)
      (while (outline-next-heading)
        (org-promote)))
    (goto-char (point-min))
    (when has-properties
      (kill-line)
      (kill-line))))

For now, I'm going to leave this as it is. I really need to live in this workflow to see how it feels and what changes need to be made. I will make a transient menu for all the org-roam functions however.

(transient-define-prefix org-roam-menu ()
  "Transient menu for task management shortcuts."
  ["Node" ("f" "Capture" org-roam-node-find)
   ("i" "Insert" org-roam-node-insert)
   ("r" "Refile" org-roam-create-node-from-headline)
   ("v" "Visit" org-roam-node-visit)   ]
  ["Dailies" ("t" "Today" org-roam-dailies-capture-today)
   ("y" "Yesterday" org-roam-dailies-capture-yesterday)
   ("n" "Tomorrow" org-roam-dailies-capture-tomorrow)]
  ["Logseq" ("R" "Fix links" logseq-org-roam)])

(define-key global-map (kbd "C-c o n") #'org-roam-menu)

Refinement covers three things in particular:

Journaling
Mini-essay writing and information "aggregation"
"Output", meaning anything that makes use of the information in a broader sense (such as proper essays, or even using my notes for work).

Journaling is already covered by org-roam-dailies and Logseq, so we'll skip it over. Mini essays, and proper essays, are where I can make some improvements. Firstly, I want to update my referencing workflow.

I use BibTeX to store all my sources when I'm working on a bigger writing project. Most of my references will sit in global.bib, unless I'm working on a self-contained project. There's two parts to making my citation workflow a bit nicer: first, I want to have some way of pulling source formatting automatically for resources like arXiv and doi.org, and second I want a transient menu for inserting references in BibTeX.

The first can be solved just by installing biblio, since it already allows pulling sources from doi and arXiv. Simple enough.

(use-package biblio)

And done. Now for the transient menu. I want quick access to doi and arXiv - as well as quick access to some existing entry templates (mainly I'll use this for online resources).

(defun bibtex-online-entry ()
  (interactive)
  (bibtex-entry "Online"))

(defun bibtex-misc-entry ()
  (interactive)
  (bibtex-entry "Misc"))

(defun bibtex-software-entry ()
  (interactive)
  (bibtex-entry "Software"))

(transient-define-prefix bibtex-transient-menu ()
  "Transient menu for task management shortcuts."
  ["Biblio" ("d" "doi.org" doi-insert-bibtex)
   ("x" "arXiv" arxiv-lookup)]
  ["Templates" ("o" "Online Resource" bibtex-online-entry)
   ("m" "Misc" bibtex-misc-entry)
   ("s" "Software" bibtex-software-entry)])

(define-key bibtex-mode-map (kbd "C-c r") #'bibtex-transient-menu)

Now I'm quite happy with my essay writing workflow already, but I wouldn't mind some extra helper functions for mini-essays. I actually want to make use of org-roam for this, so I'm going to slightly modify the configuration. First, the org-roam-directory will be changed to just ~/.org/lore, then I'll update the capture template to push to raw. I'll create another capture template specifically for mini-essays - the goal here is to keep them discoverable within org-roam.

(use-package org-roam
:straight nil
:ensure t
:init
(setq org-roam-v2-ack t)
:custom
(org-roam-directory (file-truename "~/.org/lore/"))
(org-dailies-directory (file-truename "~/.org/lore/refined/journal/"))
(org-roam-file-exclude-regexp "\\.git/.*\\|logseq/.*$")
(org-roam-completion-everywhere)
(org-roam-capture-templates
 '(("c" "Raw" plain
    "%?"
    :target (file+head "raw/${slug}.org" "#+title: ${title}\n")
    :unnarrowed t)
   ("r" "Refined")
   ("rm" "Mini-essay" plain
    "%?"
    :target (file+head "refined/wiki/${slug}.org" "#+title: ${title}\n#+author: Jakub Nowak")
    :unnarrowed t)))
(org-roam-dailies-capture-templates
 '(("d" "default" entry
    "* %?"
    :target (file+head "%<%Y-%m-%d>.org" ;; format matches Logseq
                       "#+title: %<%Y-%m-%d>\n"))))
:bind (("C-c n l" . org-roam-buffer-toggle)
       ("C-c n f" . org-roam-node-find)
       ("C-c n g" . org-roam-graph)
       ("C-c n i" . org-roam-node-insert)
       ("C-c n c" . org-roam-capture)
       :map org-roam-dailies-map
       ("Y" . org-roam-dailies-capture-yesterday)
       ("T" . org-roam-dailies-capture-tomorrow))
:bind-keymap ("C-c n d" . org-roam-dailies-map)
:config
(require 'org-roam-dailies) ;; Ensure the keymap is available
(org-roam-db-autosync-mode))

Now that is essentially done. I may add extra templates for additional refinement stuff - index cards come to mind - but for now this is good enough.

Stuff I Didn't Do

This was rather a lot of work, but I think I'm done with messing with this configuration for now. I did also set up Syncthing over the course of writing this post. There are some things that I left out as they're too annoying to implement for me currently, namely:

Something similar to Expresso
Dice and cards
Jira and Outlook integration with Agenda
A better system for worldbuilding

However I do feel like I addressed my main points of friction with org-roam - I feel comfortable getting more use out of it now that I have it available on mobile. In general it feels nicer to have everything be more organised.

So yeah, that's it. This was a bit of a messy journey that turned into a complete reorganisation of my life, but I hope you enjoyed it and got some useful things out of it as well. You can find the full configuration file here.