Optimizing Project Selection in Emacs

Yesterday [2023-04-10], Spencer Baugh submitted an Emacs patch adding a new user option to project.el, Emacs’s bulit-in project isolation package.

The new user option, project-prompter, determines how Emacs prompts for selecting a project in some project.el commands, such as C-x p p. Previously, these commands would call the function project-prompt-project-dir to do that job. Let’s have a look at that function’s definition:

(defun project-prompt-project-dir ()
  "Prompt the user for a directory that is one of the known project roots.
The project is chosen among projects known from the project list,
see `project-list-file'.
It's also possible to enter an arbitrary directory not in the list."
  (project--ensure-read-project-list)
  (let* ((dir-choice "... (choose a dir)")
         (choices
          ;; XXX: Just using this for the category (for the substring
          ;; completion style).
          (project--file-completion-table
           (append project--list `(,dir-choice))))
         (pr-dir ""))
    (while (equal pr-dir "")
      ;; If the user simply pressed RET, do this again until they don't.
      (setq pr-dir (completing-read "Select project: " choices nil t)))
    (if (equal pr-dir dir-choice)
        (read-directory-name "Select directory: " default-directory nil t)
      pr-dir)))

XXX marks the hack. The call to project--file-completion-table creates a completion table with category project-file, which forces the completion style substring. This is meant to overcome the fact that project-prompt-project-dir uses completing-read with full directory paths as completion candidates–a total pain with Emacs’s default completion styles.

In fact, there are several hacks in this definition that lead to a slightly awkward user experience. The next hack allows choosing an arbitrary directory instead of a known project root directory–to do that project-prompt-project-dir invokes completing-read with a completion table that consists of the known project root directories along with a dummy candidate "... (choose a dir)", which always looks a bit out of place in my completions buffer.

But the way project-prompt-project-dir handles empty minibuffer input is even more baffling–it completely disregards it and prompts you again, in a loop, until you enter something else or quit with C-g. How’s that useful? If the empty input makes no sense–tell me so! Signal an error, do something. Better yet, provide a default selection on empty input. That’s exactly what completing-read’s DEFAULT argument is there for–just use it. Instead this function swallows my keystroke with no feedback. I don’t like that.

Still, these are minor inconveniences. My deeper, conceptual, problem with project-prompt-project-dir is that it prompts me for a directory when really what I want to choose is a project. Of course, in project.el there’s a 1-to-1 correspondence between projects and their root directories, but this behavior prevents a useful abstraction.

With Spencer’s patch, the new project-prompter user option specifies the function responsible for letting us select a project. By default to project-prompt-project-dir so to retain the current behavior for unwary users, while adding a new alternative prompting function called project-prompt-project-name.

The new alternative let’s us select a project by name, rather than by root directory. This is a win in my opinion because it enforces a nice abstraction (projects are distinct from their root directories). It’s also more practical because we can give projects indicative, clearly distinct names even if they reside in directories with generic and similar names.

Unfortunately, project-prompt-project-name inherits most of the problems I described earlier from project-prompt-project-dir by virtue of copy-pasta:

(defun project-prompt-project-name ()
  "Prompt the user for a project, by name, that is one of the known project roots.
The project is chosen among projects known from the project list,
see `project-list-file'.
It's also possible to enter an arbitrary directory not in the list."
  (let* ((dir-choice "... (choose a dir)")
         (choices
          (let (ret)
            (dolist (dir (project-known-project-roots))
              ;; we filter out directories that no longer map to a project,
              ;; since they don't have a clean project-name.
              (if-let (proj (project--find-in-directory dir))
                  (push (cons (project-name proj) proj) ret)))
            ret))
         ;; XXX: Just using this for the category (for the substring
         ;; completion style).
         (table (project--file-completion-table (cons dir-choice choices)))
         (pr-name ""))
    (while (equal pr-name "")
      ;; If the user simply pressed RET, do this again until they don't.
      (setq pr-name (completing-read "Select project: " table nil t)))
    (if (equal pr-name dir-choice)
        (read-directory-name "Select directory: " default-directory nil t)
      (let ((proj (assoc pr-name choices)))
        (if (stringp proj) proj (project-root (cdr proj)))))))

This could use some polish. Spencer put it nicely in response to his patch landing on Emacs master:

I was expecting to need to iterate through some review cycles :)

Ideally, I’d channel my dissatisfaction with the current implementation to crafting a follow up patch to Spencer’s addition. Alas, I’ve maxed out my casual Emacs contributor plan and I’m currently waiting for the FSF to process my copyright assignment papers before I can contribute to Emacs development again. Even if project-prompt-project-name isn’t quite my cap of tea, I can still leverage the new project-prompter user option with a custom project-prompting function of my own. Enter my new init.el resident – esy/read-project-by-name:

(defvar esy/project-name-history nil)

(defvar esy/projects-directory "~/checkouts/")

(defun esy/read-project-by-name ()
  "Read a project name and return its root directory.

If no known project matches the selected name, prompt for a
sub-directory of `esy/projects-directory' using the selected name
as the initial input for completion, and return that directory."
  (let* ((name-dir-alist
          (mapcar (lambda (dir)
                    (cons (project-name (project-current nil dir))
                          dir))
                  (project-known-project-roots)))
         (current (project-current))
         (default (and current (project-name current)))
         (name (completing-read (format-prompt "Project" default)
                                name-dir-alist
                                nil nil nil
                                'esy/project-name-history
                                default)))
    (or (alist-get name name-dir-alist nil nil #'string=)
        (let* ((dir (read-directory-name "Project root directory: "
                                         esy/projects-directory
                                         nil t name))
               (project (project-current nil dir)))
          (when project (project-remember-project project))
          dir))))

Similarly to project-prompt-project-name, this function let’s me select a project by name, instead of having to specify its root directory. Where esy/read-project-by-name differs is in how it handles edge cases, namely:

1. empty minibuffer input, and
2. unknown project names.

On empty input (that is, if I press RET without inserting anything in the minibuffer first), the current project is selected as the default. I use format-prompt to have the minibuffer prompt reflect the default choice.

If I insert an unknown project name, that’s taken to mean that I want to select a new project. In that case esy/read-project-by-name invokes read-directory-name to let me specify the root directory of that new project. Most of my project directories live under ~/checkouts/, so the prompt for the new project’s root directory starts from there. Moreover, the unknown project name that I’ve inserted first is placed in the minibuffer right after ~/checkouts/, so it acts as a hint for further completion operations.

For example, let’s say I have a new Git repository that I’ve just cloned into ~/checkouts/foobar/. If I want to do some project-wide task with it, maybe searching for a regular expression or starting a dedicated shell buffer, I can hit C-x p p and type foobar RET. Now I get the Project root directory: prompt, and the initial input is already ~/checkouts/foobar, so I just press RET again and I’m there.

But what if I can’t remember exactly where I’ve cloned that new repo into, was it foobar or foobaz? Well, no problem. If I do fooba RET at the prompt from C-x p p I get the same prompt for directory as before, except now the initial input is ~/checkouts/fooba. Hitting TAB completes this to ~/checkouts/foobar/ and we’re good to go. Contrast this behavior with how project-prompt-project-dir and project-prompt-project-name handle unknown projects–they both simply say No match in response to C-x p p foobar RET.

Emacs’s project.el got a cool new enhancement. It still isn’t perfect, but it is more extensible than ever, which means I can better tailor it to my needs.