Neon Vagabond Sitemap Index About the Author Cyberpunk Archive Mirror

Category: Emacs

Table of Contents

1. Unique aspects of GNU Emacs

1.1. A general computational platform integrated from inside and outside

To understand what Emacs is, you can look at it in two ways:

  1. The fundamental tool set it provides to users (for scripting and configuration) and package developers. If it was just an editor, that tool set would be limited to a scripting language with access to an API for controlling a pre-made editor interface, and a few minimal ways to connect with or control the outside world. That's because something that's fundamentally a text editor isn't going to be interested in exposing the core command loop, the fundamental building blocks the editor uses for building its user interface, the ability to respond to arbitrary key presses (so non-editor-like UIs can easily be built) and powerful APIs for interacting with the host system. Why would you need that kind of access?
  2. The parts of Emacs that can't be changed from inside it. That would be the C core, representing currently around 25% of the total code base. If Emacs was just a text editor, the unchangeable core would be composed not of general purpose systems that could be used by any kind of text-oriented computing environment, but systems implementing a specific text editor's user interface and functionality.

Neither of these things is actually the case, however. If you look the C core and fundamental tool set provided to package authors and users by Emacs, what you see is:

The rest is all purely in Emacs Lisp – open to be changed, adapted, modified, or overridden completely.

This doesn't look like the core of something that is essentially a text editor. It provides way too much:

  • the general purpose building blocks for creating basically any kind of rich text-oriented interactive interface with its command loop, keymap system, and display system;
  • a standard library that is extremely good at managing processes, ingesting and transforming not only text but also structured data, and doing inter process communication (so good that it even begins to rival Python's standard library);
  • a system that provides a common set of user interface elements (in the form of the echo line, mode line for each buffer, and window management system) for arranging and managing the multiple user interfaces and applications in a coherent and consistent single user interface environment;
  • an environment where all of these interfaces/programs can communicate with each other, call each other's functions, and even inspect each other's user interfaces, thanks to being in the same environment courtesy of the previous point.

This deep integration into a coherent computing environment allows for some truly incredibly impressive automated and integrated workflows, like those of Sacha Chua: see here Audio Brain-Dump Workflow and the way she runs EmacsConf.

This is why, in my humble opinion, the old joke that "Emacs is a great OS" is true. It is a general purpose, Lisp-based single address-space, homogeneous, malleable system designed for ingesting, transforming, and operating on textual and structured data in an interactive text-oriented user interface environment.

Unlike many other homogeneous, malleable computing environments surviving from the Before Times – such as Smalltalk – however, Emacs hasn't become increasingly sealed away in its own little hermetic world. Instead, thanks to its powerful filesystem and process management capabilities it can leverage the world of composable Unix command line tools, much like a shell does, and it can use its powerful IPC and HTTP abilities to talk to non-command-line services as well. It thus regularly builds user interfaces for external tools and services such as Magit, can read and recognize the output formats of common Unix tools like make, ls, diff, grep, and GDB and turn theme into interactive (and even editable) hypertext, often is used to control other applications, and contains powerful capabilities for communicating with and managing its host system (see also). It even has a full cross-platform, native (Lisp-based) POSIX-like shell built into it. Emacs is a Lisp Machine adapted for a world where UNIX won.

Emacs really is a general-purpose computing environment, too. What else could you call an environment that is general enough to express many of the applications one might want outside of anything related to code editing (the ones that are mostly centered around text and images, anyway) within itself, including:

and of course a little-known but powerful text editor?

In a sense, the text editor itself just falls naturally out of the text-oriented computing environment toolbox, and to the degree that things are built on top of that to foster an actually excellent and usable text editor, instead of just a set of components to make a text editor, well, that text editor can simply be considered the "example application" of this powerful computing environment!

Why are people so obsessed with living in this computing environment, though? That's down to the other unique aspects, that all add up to one word: freedom.

1.2. Transparent from the inside

One of the foundational things about Emacs that makes it unique is that every single function and variable that makes Emacs up, barring the C core (which only makes up about 25% of the codebase, and which everyone is always looking to shrink if possible) whether or not it is intended for the user to run as an interactive command, or even for anyone to look at or reuse at all, is transparently there, available, at all times, for the user to inspect and run to see what happens, or recompose into new functionality, or call to script or automate tasks. There is absolutely no limit. There's no small defined set of API functions you can call and variables you can read, no boundaries within the system, either to you as a user, or to your code, or to the other packages you install.

This means that absolutely every piece of code writtenf or Emacs is maximally reusable.

Not only that, but every single one of these functions can not only be run but carries its own documentation and full, original source code (gzipped) along with it so that the user can fully understand what every single thing in the system does and how it does it.

It's also really important to consider what the Emacs computing environment does with this transparency and the fact that documentation can be attached to every symbol in the editor. For one thing, while many programming languages now also have the concept of attaching documentation strings to functions (for example, Python), few allow those docstrings to be rich markup with hypertext capabilities, especially not hypertext capabilities to link to other language symbols and datatypes. Even fewer languages provide their complete introductory, reference, and user manual as on-line (as in, provided with, and accessible at a granular level from within, the software product) marked up hypertext documentation, along with full user manuals for every built in package and library.

It is even rarer for application systems to expose this style of on-line granular (per-functionality) documentation, in addition to user manuals and reference manuals, for their actual user interface and functionality, instead of it just being an internal developer convenience. Emacs is one of the very rare applications where you can quite literally ask it for a detailed description of what any button (command) or key combination (key chord) does, with links to the reference manuals, instead of having to search manually through the user manual hoping for a section that specifically covers the button you're looking for (and it may not, since user manuals are usually much too high level to cover in detail the functionality of each individual button).

Still fewer systems provide so many systems for accessing this documentation: if you press C-h C-h (the help system's help menu), you get this insane list of ways to access the help system:

Commands, Keys and Functions

   m    Show help for current major and minor modes and their commands
   b    Show all key bindings
   M-x describe-key     Show help for key
   c    Show help for key briefly
   w    Show which key runs a specific command

   a    Search for commands (see also M-x apropos)
   d    Search documentation of functions, variables, and other items
   M-x describe-command     Show help for command
   M-x describe-function    Show help for function
   M-x describe-variable    Show help for variable
   o    Show help for function or variable

Manuals

   r    Show Emacs manual
   F    Show Emacs manual section for command
   K    Show Emacs manual section for a key sequence
   i    Show all installed manuals
   R    Show a specific manual
   S    Show description of symbol in pertinent manual

Other Help Commands

   C-e  Extending Emacs with external packages
   p    Search for Emacs packages (see also M-x list-packages)
   P    Describe a specific Emacs package

   t    Start the Emacs tutorial
   C-q  Display the quick help buffer.
   e    Show recent messages (from echo area)
   l    Show last 300 input keystrokes (lossage)
   .    Show local help at point

Miscellaneous

   C-a  About Emacs
   C-f  Emacs FAQ
   C-n  News of recent changes
   C-p  Known problems
   C-d  Debugging Emacs

   g    About the GNU project
   C-c  Emacs copying permission (GNU General Public License)
   C-o  Emacs ordering and distribution information
   C-m  Order printed manuals
   C-t  Emacs TODO
   C-w  Information on absence of warranty

Internationalization and Coding Systems

   I    Describe input method
   C    Describe coding system
   L    Describe language environment
   s    Show current syntax table
   h    Display the HELLO file illustrating various scripts

This isn't even all of them. In what other user interface system can you run a command that means "tell me what internal function what I'm about to do does", then right click on something, navigate through a menu, click on a final command, and see that function, then trivially jump to its definition and inspect it, or get its documentation, as demonstrated here?

Possibly the coolest help commands in the Emacs computing environment are the apropos family of commands – that's right, there's not just the one! Using C-h a apropos to get a list of commands containing the word apropos, we see:

  Type RET on an entry to view its full documentation.

apropos                       <menu-bar> <help-menu> <search-documentation> <find-any-object-by-name>
  Show all meaningful Lisp symbols whose names match PATTERN.
apropos-command               <menu-bar> <help-menu> <search-documentation> <find-commands-by-name>, <help> a, C-h a
  Show commands (interactively callable functions) that match PATTERN.
apropos-documentation         <menu-bar> <help-menu> <search-documentation> <search-documentation-strings>, <help> d, C-h d
  Show symbols whose documentation contains matches for PATTERN.
apropos-follow                M-x ... RET
  Invokes any button at point, otherwise invokes the nearest label
  button.
apropos-function              M-x ... RET
  Show functions that match PATTERN.
apropos-library               M-x ... RET
  List the variables and functions defined by library FILE.
apropos-local-value           M-x ... RET
  Show buffer-local variables whose values match PATTERN.
apropos-local-variable        M-x ... RET
  Show buffer-local variables that match PATTERN.
apropos-user-option           <menu-bar> <help-menu> <search-documentation> <find-options-by-name>
  Show user options that match PATTERN.
apropos-value                 <menu-bar> <help-menu> <search-documentation> <find-option-by-value>
  Show all symbols whose value's printed representation matches
  PATTERN.
apropos-variable              M-x ... RET
  Show variables that match PATTERN.
command-apropos               M-x ... RET
  Show commands (interactively callable functions) that match PATTERN.
customize-apropos             <menu-bar> <options> <customize> <customize-apropos>
  Customize loaded options, faces and groups matching PATTERN.
customize-apropos-faces       <menu-bar> <options> <customize> <customize-apropos-faces>
  Customize all loaded faces matching REGEXP.
customize-apropos-groups      M-x ... RET
  Customize all loaded groups matching REGEXP.
customize-apropos-options     <menu-bar> <options> <customize> <customize-apropos-options>
  Customize all loaded customizable options matching REGEXP.
gnus-group-apropos            M-x ... RET
  List all newsgroups that have names that match a regexp.
gnus-group-description-apropos M-x ... RET
  List all newsgroups that have names or descriptions that match
  REGEXP.
info-apropos                  <menu-bar> <help-menu> <more-manuals> <lookup-subject-in-all-manuals>
  Search indices of all known Info files on your system for STRING.
tags-apropos                  M-x ... RET
  Display list of all tags in tags table REGEXP matches.
xref-find-apropos             <menu-bar> <edit> <goto> <xref-apropos>, C-M-.
  Find all meaningful symbols that match PATTERN.

We've got everything from full-text documentation search to full-text reference manual search, and every kind of specific search you need in between.

This kind of variety and power in help functions is important because a user that needs help finding or understanding something is always going to know a different subset of the information about the thing they want to find, so you need a command for each possible set of information they might already know that can use that information to find the information they don't know. If you only provided help that goes in one direction, or deals with one cross-section of information, than users whose knowledge doesn't happen to match up with those assumptions are going to be left completely lost.

1.3. Malleable at run time

From inside Emacs, while it is running, you can:

  • add new code
  • evaluate expressions
  • redefine functions, variables, or data types
  • create new commands

and all of these changes will take effect immediately, modifying your running system live as you use it, from within it: using Emacs to modify itself. You can also use this functionality to inspect and analyze the state of the running system, as it runs – a feature extended by the Emacs Lisp debugging system, which allows you to have the full power of a debugger as you're using and modifying your running Emacs.

This sort of live, self-referential experience is simply unmatched by most computing environments. There's really nothing like it. It's somewhat similar to the concept of code hot reloading that many web developers may be familiar with, but infinitely more granular, since you're just sending code and new definitions directly to the running system, no need to save whole files or modules, and without the need to totally reload a program and then recreate its state – instead the program just changes without you needing to do anything. Moreover, even hot reloading is not applied, to my knowledge, to systems modifying themselves, instead of just updating other programs.

In fact, the entire way you configure Emacs is built on the back of this run-time malleability: instead of a static configuration file in a dead data format like JSON or TOML that the editor reads and then uses to internally set configuration values and add commands itself, the configuration format of Emacs is Emacs Lisp: when you find a modification that you like through changing Emacs at runtime, and decide that you want it to apply to future Emacs sessions, you just copy those Lisp forms into Emacs's init.el file, which is literally just a file of Lisp code that's run every time Emacs starts up. And eventually, if your customizations eventually become complex, powerful, or interesting enough, you can trivially move them out into a package that anyone else can install, without having to transition from a configuration format to whatever language extensions are written in like other languages. There's no barrier between modifying the editor at runtime, more robustly changing it for yourself, and sharing those modifications.

1.4. Modifications compose robustly

Since redefining things is a built-in language feature, not a meta-level feature of some build system or hack, like "hot reloading" is, the redefinition of a function, variable, or data type can be done without modifying the original code: simply evaluate the new definition, and it will replace the old one in the running state of the system (the "image" in Lisp parlance) without corrupting or modifying the original definition, safely stored in source code form on disk, in any way. Thus, you can have the Lisp code in your init file programmatically perform the redefinition at startup time, and you can have the benefits of persistently modifying some aspect of your system, without having to deal with merge conflicts down the road when new versions of whatever you were modifying come out, or having to worry about losing the original version should you realize there's something wrong with your modification and want to compare or revert it.

If you don't want to completely replace the definition of something, but instead simply want to modify its general behavior in some way, you also have access to the ability to "advise" functions in Emacs Lisp. This functionality allows you to modify the inputs to a function, the outputs from a function, run the function conditionally, run other functions always before or after that function, and any one of a myriad other options, all without modifying or even redefining the original function. Importantly, as well, each of the ways that you can modify the behavior of such a function is represented as a stack of such modification lambdas, thus allowing multiple pieces of code that have absolutely no clue as to the presence of the others to modify a function composably, with a much lower chance of disaster than if they were simply wholesale redefining functions. (Of course, this isn't perfect, but it's an extremely powerful tool).

1.5. Unity of internals and user interactions

  • Users edit text by using a suite of powerful text navigation and manipulation commands on substrates for holding and displaying propertized (semantic+structural metadata infused) text (buffers). This is also how Emacs Lisp internally performs all of its text manipulation and UI building. There's no artificial separation between the two, which means that tools and functionality that help the one can be available to help the other, and vice-versa. See also.
  • Users control the program by pressing keys which are directly mapped to functions that run within the current running process, or through using execute-extended-command. The inputs they are interactively prompted to provide to those functions are just regular arguments, and they can also use key commands to supply such arguments prior to activating the function, as well. "Interactive commands" are literally just Lisp functions, only separate from functions for convenience so that when users are trying to run commands they don't get suggestions for a bunch of functions not usually relevant to interactive uses, and so that those functions can specify how to interactively prompt for values for their arguments. That's it. Users can call non-interactive functions interactively, and non-interactive Lisp can call interactive functions transparently.
  • User interfaces are presented to the user the same way they're represented internally, and the same way user-editable content is: as just propertized text in buffers. The user can always toggle off read-only mode and edit it, or run another application on the text provided as an interface by the previous, or copy the text out to a new buffer for inspection, or do anything else they want with it, just like the program that created the text UI does. This also extends to Lisp other than the Lisp that created the UI, too. Moreover, since buffer manipulation is the core of Emacs Lisp, its absolutely most powerful capability, this means that all of this power is available for all user interfaces. Everything is a buffer, and so everything can have the full power of Emacs brought to bear on it. Thus every UI is scriptable, inspectable, readable, modifiable, by everything within the system, instead of being a flat black box. A buffer is a buffer is a buffer.

This also allows some really cool things, like command-history, which shows the history of commands the user has run as executable Lisp code. Note that this is in part made even possible in the first place because in Lisp, code is just data, so it's easy to return a value that represents the code that was executed without creating a special command language and interpreter thereof, like you would have to do if you had used, for example, JavaScript. A practical application of this is the ability to export a keyboard macro as a Lisp function, for addition to your config and use later, or just out of curiosity, and have the code produced thereby be a first class citizen, operating in exactly the same way "real" Emacs Lisp code should.

1.6. Infinitely malleable

Thanks to the previous points, Emacs can be said to be truly infinitely malleable. Want to replace a core feature, in such a way that everything using that feature immediately and transparently uses the new version? Sure thing, that's trivial – common, in fact. See: Vertico, Corfu, Ido, IComplete, etc. Want to use any part of the existing editor yourself for something else? Sure, you can do that too. Want to change how a core command behaves to fix a bug or improve a feature? Certainly. Want to define new commands? That's so easy it's barely remarked upon.

This is in stark contrast to even editors like Visual Studio Code – which is often touted as being Emacs's equal in terms of customization – but which:

  1. makes a strong distinction between customizing the editor (through flat data and a GUI) and creating an editor extension,
  2. forces you have to make a painful transition from its customization interface to creating a full on extension with a preset project format and a lot of boilerplate and a complex API even for adding a simple command (leading to users asking VSCode developers to reinvent a mini command programming language in JSON to regain the ability to easily write macros),
  3. makes you customize your editor through the pinhole of a powerful, but still rigidly circumscribed extension API, instead of giving you full access to everything (meaning you can't fundamentally change the behavior, layout, UI, or editing functionality of the editor),
  4. due to the previous point, while Visual Studio Code may seem to offer more powerful UI capabilities than Emacs due to being based on a fully featured web browser through Electron, thus allowing you to embed full web applications inside it, these GUI extensions will be unable to use the same UI features and paradigms as Visual Studio Code (such as their own vertical fuzzy searching palettes) since the API doesn't expose the internal tools necessary to do so, and the rest of the editor will be unable to be aware of and thus unable to integrate with or use anything in the UI, thanks in part to extensions not being able to read the editor's user interface (unlike Emacs), and the core of VSCode being unable to be extended to know about the web UI that's been shoved into a tab.

Thus you can see that if you want true malleability in your computing environment, what you want remains – and probably will always remain – Emacs.

Why would you want that infinite malleability? Because it allows you and the software you use to grow together:

Emacs is the real experience in digital gardening. You tend to it for years, decades, as it grows with you to be your own, one-of-a-kind computing environment where you feel comfortable, efficient, and in peace.

It's not as perfect as the commercial flower installations. But they come and go, and your garden endures. As new plants come into fashion, you can get them for your garden too. As they go out of fashion, you can still keep them if you want.

And yes, you can also blog with it too.

Any open, cross- environment where you can change, compose, automate, and adapt any of the vast panoply of tools and interfaces it offers you to fit how you think and how you want to work, instead of only having a few toggles here and there and otherwise being forced to hope you can find a near-enough fit in rigid existing tools or adapt yourself to the workflow of some tool or other, is an environment that is worth investing time into, because it can truly adapt to you and become yours. Such a piece of software is well-worth adopting as personal infrastructure, making you more productive and happy in small ways, letting you move through your tasks with greater ease.

1.7. Application output ("UIs") are composable and open to reflection

Thanks to the fact that interfaces in Emacs are not just:

  1. highly structured trees of structured data, which can be unwieldy and rigid to transform,
  2. nor merely the blank slates of pixel buffers,
  3. nor just text, which is an overly-flexible communication medium where semantic meaning and structure is easily lost, especially when there is no out-of-band information and the text must be structured into a UI that is pleasant for humans to look at,

but a combination of all three (propertized text, where text can be propertized with images and such), interfaces in Emacs are special in several ways:

  1. The same interface can be represented in multiple different ways: https://lists.gnu.org/archive/html/emacs-devel/2020-09/msg00286.html
  2. It is trivial for different programs and pieces of code to inspect, search, modify, manipulate, and reinterpret the raw content (the text) of the UI in whatever way it wants, so that you can essentially take the UI created by one program as its output and input it into another program when you're done using the UI as intended, or have multiple programs doing different things with the same UI content at once if you want to (like using Embark on a Magit line).
  3. Yet at the same time, the semantic meaning of the data represented by the text layout is not lost, nor do ad-hoc structured data formats need to be created to preserve that meaning while remaining human-usable for every program, like in the Unix shell, because a common set of structured data types are understood by all Emacs Lisp code, and subsections of any text in any buffer can be annotated ("propertized") with that data as metadata, to document relations between different textual elements and the semantic meaning of them. Thus, inspection can use just the text level to radically reinterpret or modify UIs, or inspect the properties of that text to respect and integrate with the original intentions of whatever code created the UI without needing access to the internal state of that original code.
  4. Plus, we're not just limited to static, boring old plain text! Since the display system can also read those properties, special properties can be used to go beyond just plain text, including rich text things like font sizes, styles, and faces, as well as images, PDFs, and SVGs – but since all these things can only be introduced as annotations on text, there must be a text alternative to each underneath to be annotated in the first place, thus encouraging the use of text as a substrate and therefore reinforcing all the previous points!

Essentially, propertized text is Emacs's thin waist, allowing extreme composability and flexibility within a common structure and set of primitives.

Thus for example we can have a tool like Embark, that can use advanced patterns to examine the raw text around the cursor when it is activated to determine what kind of "thing" is there and what can be done on it, but which can also inspect the structured metadata attached to that text to more intelligently figure out the "thing" and available actions, so that for instance, when you run Embark on the fuzzy selection UI of Emacs itself (since all UIs in Emacs, even the system level, core ones, are just propertized text), it can read the metadata of each line of the UI to see which are the input and which are the suggestions, and what the suggestions are (buffers, files, etc), and which one the user has selected. It can even export the completion interface as it currently exists to a new buffer for you to preserve as a menu, edit, or manipulate, with all the semantic meaning, button functionality, and even styling preserved. And this can all be done without having to query anything internal, just by doing runtime reflection/introspection on the UI.

1.8. A powerful set of limited primitives

One of the most important things about Emacs is that, as flexible as it is, it is not infinitely flexible.Everything operates on the basis of a common set of interoperable building blocks that are themselves very flexible, but not too flexible – it is important that it is still a fully-unified homogeneous environment:

  • everything displays and operates within a common substrate (buffers) that can only hold one kind of thing (see the next point)
  • everything displays and manipulates a common data format (propertized text)
  • everything uses a common language (Emacs Lisp) so that functions, symbols, variables, macros, libraries, packages, etc can interoperate
  • everything uses a common set of flexible, but not too flexible, data types (hashes, vectors, lists, lambdas, symbols, keywords, propertized text, buffers), allowing further interoperation and ensuring that it is easy for different programs to adapt the data structures produced by others
  • everything has a common concept of the world around it (through the process, file management, network access, database access, and OS interface primitives)
  • everything runs in a common environment (shared memory, shared runtime, etc)

This choice – the choice of a set of primitives that is extremely flexible and powerful, but not infinitely flexible – is I think the core of what makes Emacs so incredible. It means that package or piece of code or data structure is reusable in new situations, because each new situation is never completely beyond their domain of applicability, and everything is maximally composable, because everything speaks a common language in a common context, so it can all interoperate seamlessly.

If any one of these things were too flexible – for instance, if pixel buffers were used instead of buffers of text, or different languages could be used, or the context was broken up – then all that composability, interoperability, and reusability would begin to evaporate.

Think about it this way: an buffer containing a stream of pixels is much more versatile than a buffer containing only propertized text which can have some static images overlaid on top of it. It can display more things, interact in a larger number of ways, etc. However, thanks to that infinite flexibility, there's no common way for different programs to understand that pixel buffer. You can copy it, flip it, crop it, rotate it, display it, and write to it, but you can't inspect it to pull out data or interface elements or semantics, because, well, it could be anything. There could be text in it, but it's just rendered down into individual pixels now, no semantic meaning preserved, and it could be drawn in any one of a nearly infinite number of ways, so good luck trying to reverse-engineer it. There could be images in it, buttons, menu items, but you'd have no way of knowing. Pixel buffers are a black box – once something's written to them, there's no getting it out again.

Meanwhile, something less infinitely flexible, like plain (English, Unicode) text, has more recoverable intrinsic semantics encoded into it: we have characters (letters, symbols, control characters) which are assembled into words and lines, arranged in a relatively predictable way, and probably human-readable. That's why text is so universal. However, even that is too flexible: given a raw stream of text, can you really figure out what the hell's going on? Not really. There certainly could be highly structured semantic data encoded into that text, but you have no guarantee it's in any kind of format you'd recognize, and even if it was, how would you determine that format, when you can't guarantee any kind of common out-of-band means of communication with the code you're talking to?

This is why being able to annotate text with a commonly understood set of data structures is so important: that way, you can attach semantic information, in a format you know other programs (within the world of the common set of primitives, at least) will be able to read and understand. This could be done by centering on an agreed-upon textual header or footer to append this metadata in-band, but in-band annotation comes with its own problems, and this annotation comes with the added hurdle of parsing and serializing it constantly. Providing out of band annotations in native data structures that can be human-readably serialized if necessary, but do not need to be, is superior, in my opinion.

Additionally, it is important that the information primarily visible to humans is still just plain text, and all of the most important content and data is still also just encoded in plain text – that keeps the flexibility and universality of text, the ability to take it elsewhere and to interpret it in a way that ignores the semantics of whatever produced it without having to do extensive and annoying conversions between highly structured data. We just need to be able to, again, imbue that text with meaning.

I also think this homogeneity is extremely important for human reasons, as well: a computing environment with a totally unified context, set of assumptions, data structures, libraries, and language – especially one designed for maximum composability and flexibility – is better for the human beings that use it, because it means that the knowledge we build customizing and building for ourselves can directly, smoothly scale up to the knowledge we need to build things for others or easily being able to understand and contribute to anything anyone else in the community is doing, which can then smoothly scale up to contributing to (most of) the core of the system – there's fewer points at which there's a fundamental break in paradigm, set of concepts, or programming language, where we have to use our limited human patience and time to learn something alien to us.

1.9. Unique user interface paradigm

Another really neat thing about Emacs is the unique user interface paradigm. There are a few aspects to this.

The first few unique aspects, which other users than me have also pointed out, stem naturally from the unity of internals and user interactions:

  1. Literally everything that can be done in the editor, or that the editor can do, is a command that the user can fuzzy search through and run. This is similar to the command palettes of things like Sublime Text, Visual Studio Code, and Obsidian, but infinitely more powerful, because it isn't arbitrarily limited to a set of commands that someone decided should be there – all the functionality of the editor is available.
  2. Everything that the user can do in the interface can be scripted – and not just through recording keypresses and dumbly replaying them, hoping that it works out right, but through the full power of Emacs Lisp to inspect editor state and conditionally do things based on that. This is because all user actions are just code and can in fact be easily recorded and exported as code to build on and add conditionality to.
  3. Every keybinding can be remapped, and new keybinding maps can be swapped in and out or overlaid on top of each other (with sparse keymaps).

There's something even more interesting than all that, in my opinion, though, represented by Magit, Transient menus, and applications like Dired and Proced. In this user interface paradigm, you generally have a consistent, pinned, stateful view of whatever content you're exploring or editing, a main "window" if you will, and then the ability to manipulate, modify, and navigate through it using single letter commands, without even needing modifier keys a lot of the time. These single letter commands can prompt you for input, in fact be trees of related commands, and can, with Transient, even take flags and toggle switches and secondary commands and input.

This means that you can get the higher interaction information throughput of the keyboard (as opposed to the mouse) for communicating what you want to do to the computer, while actually even increasing that throughput relative to something like the shell or key modifier-based interactions, since every command is just a single key press or a few key presses and nothing more, while getting discoverability and live, changing, stateful view of the state of whatever you're exploring or manipulating, like a GUI program, instead of having to follow the "do thing blindly, look at thing, repeat" loop of the shell, while getting the far greater expressiveness of Emacs key commands versus the flat key command hierarchy of GUI programs so that you can express more things with the keyboard.

To be fair, this user interface paradigm does show up in older Windows programs, I think, where you press Alt+<mnemonic letter> to navigate through menus to select commands you want to run on the content in the main window, but in my opinion not in as well-developed of a form.

1.10. A fascinating text editor

Although the rest of the "Emacs is an OS" joke claims that it lacks a good text editor, I think there is certainly an argument to be made that Emacs is a great text editor. It comes with a very rich set of concepts for editing text that puts any word processor and most text editors to shame – characters, words, lines, yes, but also sentences, paragraphs, sections (which you can structurally edit) via outline-minor-mode, and trees of expressions – all placed into a coherent keybinding schema that makes it relatively easy to memorize/remember/guess them, and very easy to look them up. It then also provides a powerful set of operations on each of these concepts, from transposition, to selection, to navigation forward and backward and up and down across them, to jumping to them, as well as an panoply of small utility functionality like zapping to characters, collapsing or expanding white space, and so on.

Yes, each Emacs command tends to encode a specific text object to operate on as well as the action you want to perform on that number of text objects into a single command, so by default you have to memorize MxN (text objects times actions) commands to use Emacs effectively, whereas in modal editors like Vim have an advantage in composability and learnability, because they have a text editing grammar you can use to compose a single mnemonic for the text object you want to effect plus a single mnemonic for what you want to do (plus numbers and adjectives just like Emacs) so you only have M+N commands. Concretely, for instance, if you want commands for deleting words, characters, sentences, and s-expressions, and then also for selecting, moving by, and copying them, in a modal editor there’d be a verb for deleting, selecting, moving past, and copying, and then objects for referring to words, characters, sentences, etcetera, and you can compose just those 8 commands/key bindings into all the actual commands you need, whereas in Emacs you'd have to memorize 16 separate commands. And in a modal editor, every new text object you learn or create automatically gets the benefit of all existing verbs, and vice versa, meaning extending the text editor’s range of concepts or learning new existing ones for yourself has a geometric instead of linear payoff rate, whereas in Emacs you have to define a whole new suite of commands for each thing.

However, this can be worked around: the core insight needed is that Emacs actually does provide all the components necessary out of the box to construct a text editing grammar: selections and region actions! Think about it: the way a text editing grammar works is that instead of specifying a single command that bakes in both the region and the command to be run on the region, you specify what you want to act on and how to act on it separately. Vim does action first and then text object, but that's not inherent to the idea of a text editing grammar – we could also describe what we want the other way around, object and then verb. And hey whaddya know, that’s how regions and their associated commands in Emacs work already: you highlight what you want to act on, and then say what you want to do to it, and Emacs already provides a command to select any text object it knows of, and a command that lets you perform any action it knows how to do on a region instead of a set text object (even supporting specifying multiple arbitrary objects through the use of the mark ring).

Of course, this wouldn’t work if the commands you have to select things were too primitive, like in most non-modal editors, because then the specification of what would be too low level, instead of specifying text objects you’d always be specifying either words or characters or, if you’re lucky, lines. However, Emacs does in fact offer a rich set of text objects baked into commands for selecting or moving by them (as well as doing other things to them, but those aren’t relevant right now).

All you need to apply this insight is to use it as a sort of mental jujutsu practice in existing Emacs – focus on learning and using the text object selection commands and the region action commands, or use the set-mark command and text object movement commands in conjunction with the aforementioned region action commands, if you wanted to also unify movement and text object selection (a good idea). Wax on, wax off! However, this could be made significantly easier, which is why I'm working on a minor mode to do this. It's working out pretty well – I'm daily driving it currently – the only issue is hammering out exactly all the places where such automatic selection after movement of the point by a command isn't wanted.

Moreover, even if it's pure editing language isn't quite as strong as Vim's, Emacs has incredibly powerful large scale text manipulation features like query-replace-*, which is like search and replace, but with many more features than most word processors and text editors could ever hope to achieve, or occur-mode which allows you to collect every occurence of a pattern across a buffer or set of buffers into a new buffer and do manual edits on that buffer, then have the changes written to each respective buffer, or grep which will generate find and grep commands for you, run them, and give you a hyperlinked buffer of the results, and allow you to edit that buffer and write the resulting changes out to all the relevant files transparently. And these are just a few of the powerful capabilities Emacs has!

I also think the fact that Emacs has structural editing commands, in the form of forward-sexp (move forward by expression), backward-sexp (move backward to/by expression), forward-list (move forward to/by an expression group), backward-list (move backward to/by an expression group), down-list (move into an expression group, so that forward- and backward-sexp move by its children instead of siblings), backward-up-list (the reverse of down-list), baked into its core, is incredibly important.

In Vim, despite its powerful and composable editing grammar, you still have to think in terms of a list of lists of literal characters. Therefore, you have to find your way to the next space, comma, or word to jump by sibling expressions; you have to navigate manually by lines and characters to go up or down expression trees; and the commands for doing that all have the specific syntactic elements of the language you're using (and even the specific layout of the piece of code you're editing) baked in. This means that there's a high cognitive overhead to editing in Vim: you have to consider specific syntax and positioning within the text, and the commands will be subtly but importantly different for every langauge you're working in. Hell, the closest thing Vim has to a high level structural command is merely to jump to matching braces with %.

Meanwhile, Emacs views your code natively as a hierarchical tree of expressions, with the syntax it uses to determine where identifiers and expressions begin and end not needing to be provided by you as you edit, but provided using syntax tables provided by the author of the language's major mode. This means that you can cut, copy, paste, navigate up and down and across, and transpose expressions without ever having to worry about the specific way the code is laid out or formatted, or even what the precise syntax is, in many cases. This is just infinitely more powerful and advanced than Vim, in my opinion, the perfect sweet spot between true structural editing – which can be overly restrictive – where structural editing, rather than limiting you, is just an aid to your navigation and editing of code that still remains free and malleable text. In essence, in my opinion, Emacs was ready for the Tree Sitter structural-editing-as-plain-text revolution 40 years before Tree Sitter was even a twinkle in its creator's eye!

More than just being a superior foundational set of concepts for text editing, requiring much less cognitive burden and allowing for more advanced maniupations, the fact that these structural navigation and editing commands are part of Emacs's core encourages, even requires, that authors of major modes for any language give some consideration to trying to provide meaningful and useful functionality for these motion commands. And luckily, thanks to the syntax table abstraction, the way in which Emacs figures out the meaning and structure of your file is abstracted from its commands. Its structural editing commands can gracefully degrade from Lisp to Tree-Sitter to Regular Expressions to simple character-based syntax tables to parenthesis/braces/brackets/etc, depending on what the major mode you're operating in supports, but no matter what the commands remain generally useful and applicable. This means that unlike in NeoVim, where you have to operate with a completely different set of text objects and motions when you have Tree Sitter available compared to when you don't – and the Tree Sitter objects and motions don't integrate will into the grammar – in Emacs you can confidently integrate its structural editing commands into your text editing vocabulary, safe in the knowledge that they will at least be somehwat useful everywhere you go.

Moreover, the structural editing commands provided in Emacs's core are all the building blocks you need to built proper structural editing on top, including things like soft deletion, slurping, barfing, splicing, convoluting, raising, concatenating, and more, all without ever having to rely on any langauge specific code – since all of that semantic and syntactic logic is already encoded in the syntax tables of existing language modes, which they need to do anyway to make font-locking and built-in Emacs movement work – or even rely on any specific parsing strategy.

Another interesting aspect of the Emacs text editing paradigm is that it focuses on being intelligent and automatic, instead of forcing the user to do all the thinking. While Vim (and similar text editor grammars) are excellent because they let you precisely specify exactly the text manipulation you want to do up front, in the editing command language, that's also their major downside: there is a high cognitive overhead to having to think about exactly what you want to do in a given situation, figure out how to express it, and then formulate it into terms Vim can understand. It also simply requires knowing a lot of individual things. Less things than the MxN explosion of first-order Emacs commands, certainly, but here's Emacs's secret weapon: it has second-order commands.

These are called DWIM (do what I mean) commands, and rather then asking you to precisely specify what you want to do, they try to intelligently figure out which of the first order actions you actually want to run. So, while most of the time it does offer the tools to precisely specify what you want to do if you need them – for instance, what to select, what to uppercase, etc – often the only commands you really need to memorize and keep in your pocket are the DWIM commands, which will figure out what you want to do on the fly.

Another example of this philosophy comes from an external package, expand-region: instead of specifying the text object or pair of delimiters you want to expand around, and whether you want to expand around or inside that text object, up front, like in Vim, and then having to replay the entire command over again if you get it wrong or want to adjust, expand region intelligently cycles through expanding your selection to encompass all of the relevant text objects at point, in order of size. This means that to get what you want, all you need to do is keep pressing the same key until you get what you need. It's the classic difference between recognition and recall.

1.11. A long history, yet still keeping up with the modern world

Emacs, in some form or another, has been around since, 1972. fifty two years ago. That's fifty-two years that this thing has been developed. Even GNU Emacs, which came into existence much later, has been developed for exactly 40 years at the time of this writing.

FORTY YEARS.

This age means a lot of things. Some are bad:

  • there is a lot of technical debt that holds it back (single-threaded core, ancient garbage collector, slow and old fashioned Lisp dialect, no promises or async/await for handling asynchronous programming in the standard library, no concept of using a pool Emacs processes as workers, therefore insufficient adoption of asynchronous programming into the package ecosystem, no ability to embed moving pictures or images that's cross platform, difficulty with drawing rounded boxes around text)
  • it uses terminology and keybindings that are unfamiliar to the modern world (kill for cut, kill-ring-save for copy, yank for paste, meta instead of alt, etc)
  • there's a lot of accumulated feature cruft – two entirely separate mail readers and two terminal emulators? – that can be confusing to people

But others are good:

  • it has time to grow and mature, each feature being fully realized and developed
  • it has had time to become incredibly feature rich
  • an absolute wealth of documentation, tutorials, accounts of experiences, and meta-level discussions of its philosophy and benefits have had time to grow and stockpile
  • the community has had time to build an amazing and incredibly complete ecosystem of packages, often with multiple well-implemented, feature-rich, documented, and robust ways to do common things people want like reading mail, doing structural editing, taking notes, writing LaTeX books and papers, managing citations, as well as complete and almost unbeatable environments for programming languages like OCaml, Haskell, Common Lisp, Clojure and so on, as well as an incredibly long tail of generally simple and robust, or at least mostly workable, packages for anything else you could ever possible want thanks to the "scratch my own itch" model of free software development
  • the maintainers have been able to achieve a perfect balance, over time, between moving Emacs forward at a reasonable pace (and in my opinion they've done an incredible job of that in the last several versions), and maintaining stability and backwards compatibility in this tool that so many people have built their entire computational lives around; moreover, these maintainers have been able to develop a sort of institutional wisdom, passing down a clear-eyed understanding of what makes Emacs special and how to preserve that
  • if Emacs has already managed to last for forty years, it is likely to continue to exist for a very long time.

1.12. Rich plain text

Thanks to the ability to propertize text, and its advanced display system, while Emacs is text-based, it is unapologetically rich-text based, in a way where it can not only display that rich text, but actually edit and manipulate it on the same level as plain text, and convert back and forth, in-editor, by the user if need be, in a way other editors can only dream of. Other editors may be able to make plain text look richer, but you won't be able to meaningfully inspect, edit, and manipulate that rich text style, or sometimes even the text when richly displayed like that; others may be able to display rich chrome around plain text, but not display the plain text itself richly. Emacs can do all these things and more. Here's an incomplete list of the things display properties can do with plain text in Emacs:

  • Pixel- or character-wise left, right, and center align
  • Set a constant character- or pixel-wise height or width to crop to
  • Overlay an image, PDF, or SVG (or even a GTK+ widget)
  • Change the font size, face, color, and style (even within a single line)
  • Add hover text
  • Change the mouse cursor when it's over the text
  • Add a box, underline, or overline
  • Add box margin
  • Change the fringe next to that text
  • Raise or lower the text from baseline
  • Turn it into a link or a button
  • Change the background color
  • Turn it into a text box
  • Replace the text

On the subject of the Emacs display system, by the way, how does it work?

Well, for instance, I recently decided to switch dashboard packages to something faster, but it didn't implement image banners by default — so what did I do? The new dashboard package obviously lets you pass in structured data (lists and plists) to decide the structure of your dashboard, where there are strings that it uses to print out the menu items, but how could I extend this to images? Well, it turned out to be extremely simple: I just had to make the headline menu item a string with metadata attached to tell Emacs's display system to display an image on top of it. That's right — images are first-class objects in Emacs, and all you need to do to display them is attach them as metadata to a string. Note, this doesn't produce a new kind of string object that some parts of the language or editor know how to interpret and some don't, and when it is output, you don't lose the text or the metadata — it's just a regular string, imbued directly with rich multimedia metadata, that can be treated like a regular string by your code, while also showing up as an image in the display, from which the original text or metadata can be derived. Here's the first part of the code:

(concat
          "    "
          (propertize "QUAKE EMACS" 'display (create-image "~/.emacs.d/banner-quake.png"))
          "\n"
          "\n"
          (propertize  (format "Started in %s\n" (emacs-init-time)) 'face '(:inherit 'font-lock-comment-face)) ...)

That's right, you can just use the regular string concat function to concatenate together images and text, because images are just text with metadata attached! Here is how it looks:

emacs1.png

And this is how the entire UI works: it's all just structuered data where the leaves are texted annotated with metadata to control its display and interactivity. This metadata principle is how you make text taller, or turn text into buttons and attach handlers to those buttons or draw images or different fonts or icons or display arbitrarily-shaped and sized GUI elements using SVG or embed GTK+ widgets — the entire interface is just structured lists of text with metadata attached that turns that text into rich UI elements in a predictable and deterministic way; and you can get the original text and associated metadata that was used to construct a part of the user interface back out of it at any time, just from the UI itself. Combined with the fact that you can attach arbitrary metadata to that structure text — and in fact that Emacs programs often do so — and also tell the display engine how to interpret new types of metadata, and you have an extremely rich way of building introspectable interfaces! And in the end, you can just use the regular string or list processing functions on all of it, buttons, images, various sizes and fonts of text and all, because images are just text with metadata attached.

This system is even more dynamic and introspectable than HTML is — especially since you can attach arbitrary metadata, so it is common practice to attach the relevant internal UI state as metadata to each UI element you push out to the screen, meaning that any program can reconstruct the state that generated any UI that it is looking at. Not the deep global application state of course, but the stuff conceptually "right behind" the UI anyway. The power of having the end-product user interface your program generates not only be generated from native language data structures and constructs, but also be derivable from that generated UI, so that the native language nature of it all is bidirectional, is incredibly useful, almost mind-bogglingly so to me. HTML/the DOM/JS gets close to this — that's a huge part of why the web stack is so fundamentally powerful and popular — but in my opinion it is a bloated, poorly designed in many respects, version of this, not intended for user interfaces initially; in my opinion, it is also less powerful and interesting, partly because HTML and the DOM aren't very native to JS in actual fact, and partly because with HTML, determining how to interact with and display content comes prior to the content, instead of being attached after.

Yet, the best part is, it isn't just a dumb overlay, or an image broken into little character-size pieces, like in terminals that can "display images": this is a full single image (jpg, png, svg, what have you), rendered as a single image, with the text around it flowed to allow it to fit properly, just like on the DOM (although the reflow algorithm is a lot more predictible imo: the image is a single "logical line", the line is just bigger). The best part is — if you're in an emacs lisp REPL, like eshell, where the data structures returned from the expressions you are evaluated are pretty printed…

emacs2.png

yup that's right, native support for return values that are images! You can even wrap this in a command and use it to cat images. Here's how you'd dynamically extend the built in Emacs Lisp shell's cat command to be able to cat images and Emacs buffers as well as files in 13 lines of code:

(advice-add 'eshell/cat
            :around (lambda (oldfun &rest args)
                        (mapconcat (lambda (arg)
                                       (cond
                                        ((bufferp arg) (concat
                                                        (with-current-buffer arg (buffer-string))
                                                        "\n"))
                                        ((string-match-p (image-file-name-regexp) arg)
                                         (concat
                                          (propertize "image" 'display (create-image arg))
                                          "\n"))
                                        (t (apply oldfun args))))
                                   args)))

And here's me creating and returning a button object, which is then pretty-printed as a button by the REPL (this isn't me inserting the button, I'm just returning the button object and it's displaying it) with a callback that runs when clicked:

emacs3.png

(You could give the buttons outlines and depth and stuff, but I don't bother.)

In a lot of ways, it's like having the stuff iPython Notebook gives you, but as a more powerful, generalized system you can build whole interfaces with instead of it just being a special-case of a REPL that can render return values — it's a whole rendering engine that works based on the metadata of objects passed into it, and you can combine them or add your own.

And all of this is simply how the Lisp Machines worked to produce their entire display.

1.13. The culture

One of the things I love most about Emacs is its community culture. Over the forty years Emacs has been around, the Emacs community has had time to build a rich, unique culture: a mature set of norms and expectations, a deep appreciation for and understanding of the nuances and pros and cons of Emacs and everything that it can do, and an understanding that it may not be for everyone (they've long since abandoned the editor wars).

The two things I especially love about this culture are:

  1. Its deep and pervasive commitment, at least for many packages, to complete documentation of their code, and long, comprehensive, robust, well-written, and erudite user and reference manuals for major packages that have a lot of complex functionality. One of my favorite things to do is sit down and read through an Emacs package README, they're so often quite excellent.
  2. That Emacs's usage is not dominated by software engineering grognards (like Vim) or software development hipsters (like NeoVim) or even software engineers in general (like Visual Studio Code). Instead, its community is almost equally composed of scientists, researchers, data scientists, teachers, and people from the humanities such as philosophers, very many writers and novelists, journalists, social scientists, solo RPG enthusiasts and even musicians. And it is so flexible and responsive to their different needs, preferences, and workflows, that such people can really bring what they use Emacs for to the community, and integrate what they do with Emacs, instead of just silently happening to use it on the sidelines. This leads to a richer, more vibrant, more interesting, and often even more humane culture – and, as someone who approaches software engineering from a more humanities-oriented lens, a culture that is more stimulating and fun for me.

A sort of corollary to these two points, just in general, is that the Emacs community is intensely literate, erudite, well-spoken, and academic, from what I've seen at least. This is just… really pleasing to me. Others may find it off-putting or dry, but for me it just means that almost everything it produces will be a pleasure to read or listen to, engaging, full of interesting ideas and information, and written in an educated, complex style that is enjoyable to read. I don't know if Emacs's heritage in the MIT AI Lab, which was composed of erudite, literate academics and professors, and which spawned off quite literary endeavors such as Infocom, might be influencing this, but it certainly feels like it.

1.14. Programmable, not (just) customizable

Although Emacs Lisp has a lot of capabilities for defining well categorized, grouped, documented, typed sliders and customization variables for packages, and a very advanced and relatively user friendly GUI for manipulating those customizations variables, the power of Emacs is that the predominant way you customize for your needs and workflow it is by programming it. This means that your customizations can be intelligent, conditional, and far more versitile, and it also means that there's an easy on-ramp from simply customizing Emacs to really using it as a full-scale automation tool and computing environment.

Moreover, a tool that is merely customizable has to think of every possible thing someone might want out of it in advance and explicitly program for all of those things. This is both an impossible, Sisyphean task that will always leave someone dissatisfied, and also creates a combinatorial explosion of features and settings to account for and test together, which will inescapably lead to massive, sprawling, buggy codebases. Meanwhile, a programmable tool takes the wise way out: it can focus on making core functionality powerful, composable, accessible, flexible, and versitile, but doesn't have to worry about every behavior or use someone might want out of it: as long as it makes sure to maintain backwards compatability, provide documentation, and make sure modifications can compose robustly, it can easily grow and change to fit everyone while being able to just focus on its core functionality and keeping everything straight. This video says it very eloquently.

More importantly, this programmability, where absolutely anyone can come along and replace, modify, or expand on even the deepest and most fundamental parts of the editor, is part of what's guaranteed Emacs's longevity. Whenever another editor comes out with important functionality, whether that be autocomplete popups, snippets, and fuzzy vertical completion UIs for commands and filenames back in the distant past, to embedded terminals, to side bar file browsers, to language servers, to tree sitter, to multiple cursors, to project management, Emacs was able to (more or less handily) absorb this functionality. Sometimes it wasn't as good as the original, sometimes it was better, but the point is that if there was some piece of functionality that you wanted, you didn't have to switch away from Emacs if you didn't want to, and the rest of the Emacs computing environment would make up for whatever minor deficiencies there were in its implementation of these features. Basically, the more programmable and malleable your editor is, the less likely it will be to suddenly lose popularity and fall into the dustbin of history as people go chasing the next new feature.

2. Emacs as ultimate retrocomputing hobby

Emacs satisfies all of the preconditions for a retrocomputing hobby:

  1. Long (fourty years), interesting, storied history and set of precursors
  2. Maintains many aspects of its heritage, connecting you directly to that heritage (for instance, Emacs Lisp is a clone of MACLISP, the original Lisp Machine language, that exists nowhere else today)
  3. A vibrant and passionate community regularly making things for/with it just for the fun of it, often some of it actually art (and a long history of that being done, given point one).
  4. It can be completely understood inside and out, up and down, left and right, and modified.
  5. It has a lot of unique properties compared to modern computing systems, deeply influenced by its history.

It also has many neat advantages over other forms of the hobby. Namely:

  1. It can be very productively, justifiably, and usefully put to work in your everyday life
  2. A lot of the things that you do with it as a hobby can end up benefitting your everyday life and work productivity
  3. It's completely free!

This makes it the ultimate retrocomputing hobby!

3. Imbuing text with meaning

Some of the most important meanings that you can imbue into text, in my opinion, are:

  1. Symbols to label the intended "thing" that region of text is supposed to represent
  2. Data structures representing the structured concrete syntax tree of the annotated text, to easily maintain an identity between the two and retrieve it
  3. Context or related pieces of data, whether textual or structured
  4. Hypertext links

For a concrete example of the first, the package Marginalia looks for any lines annotated with the file symbol in the minibuffer, and uses those lines as paths to look up file information, which it then adds as further metadata to those lines. This means that anything that provides completions in the minibuffer, independent of Marginalia knowing about the specifics of that code, or that code even needing to know about Marginalia directly, can add useful metadata about the files it is offering to the user to choose from just by encoding the semantics of the text lines it is presenting as suggestions to the user, like so:

(completing-read-default "Note file: "
                         (let ((files (directory-files-recursively quake-org-home-directory ".*.org")))
                             (lambda (str pred flag)
                                 (pcase flag
                                     ('metadata `(metadata (category . file)))
                                     (_ (all-completions str files pred))))))

Imbuing meaning, especially in the form of hypertext links, is a common pattern in Emacs, from grep-mode to compilation-mode but for a powerful example of the benefit of the latter three points, just look at org-mode, which is able to take plain text and give it all of these extremely highly structured and interactive and automated, almost GUI or structural editing style features for manipulation, while maintaining an identity between what the program is seeing and doing, and plain text notation that can fully allow it to recreate all of the information necessary to manipulate and understand the document, is the fact that org mode is able to maintain a full parse tree of the document with references to certain positions in the text, but also annotate the text itself with references to that parse tree, thus meaning there's two way communication – when the user inspects or edits some part of the plain text, it's easy to get the structural and semantic meaning of that part of the text and manipulate it for them to produce new text.

There's also things like the incredible Hyperbole and Embark packages, which have the ability to read and use text already imbued with meaning, but also use plain-text patterns to imbue text that wasn't intentionally imbued with meaning with new meaning, bringing this whole concept full circle and making the entire Emacs environment just that much more powerful.

4. Emacs Lisp really isn't bad

It seems to be common wisdom in the Emacs community that Emacs Lisp is just this terrible programming language that everyone hates, on the level with something like PHP. However, I just don't think that's the case. Emacs Lisp is strongly typed, has an extremely complete and extensive standard library, it has all of the power of a full Lisp including live mutation of a running system, procedural macros, symbolic programming, and homoiconicity, a copy of the Common Lisp standard library (including Common Lisp generic multimethods), and an implementation of Common Lisp's incredibly powerful object system. Yes it's a Lisp-2, but so is Common Lisp, and I personally think that's an inconvenience, but not a fatal flaw.

Honestly, in my opinion, it has both fewer gotchas and is simpler, more straightforward, and more pleasant to program in than many popular modern languages, including things like JavaScript and Python. Python is chockablock full of shittily implemented (the fact that match ing on a variable can assign it, even if it already has a value, or that default argument values are effectively singletons, or any number of other things) and non-composable features (lambdas can't have multiple lines, anyone?). Or what about the insanity that is JavaScript? Even Lua has its share of truly awful design decisions (such as the fact that "arrays" are just tables with consecutive integer keys, so if you accidentally introduce a non-consecutive key into the middle of the consecutive keys of an "array", the iteration primitives will assume that's the end of the array and stop there).

Moreover, Emacs Lisp has a lot of things built into it that make it much more suited to something like Emacs than the alternatives:

  • Things it gets from being a Lisp that other languages lack:
    Macros
    Macros are an extremely important facility to have for something like Emacs, where you want a powerful, general facility for package authors to create domain-specific languages for less-technical users to create the functionality they want (see things like Transient or Easy Menu or Use Package or Hydra, or the macros provided in the Emacs Lisp standard library like with-eval-after-load or with-current-buffer or with-temp-buffer), and want a language that aligns with the Emacs philosophy of being flexible and extensible and programmable itself. Yes, you can do some degree of domain-specific-language creation in other languages, but it's usually done with a bunch of language constructs that essentially take the powerful generality of macros and break them up into separate features, bastardize them, and sprinkle them around the language, and such domain specific languages are usually quite weird, brittle, and composed of lots of little tricks to get them working. Ruby is not an acceptable Lisp just because it has some nice syntax for things.
    Runtime Dynamicism
    the ability to redefine a function, method, or data type, or evaluate an expression, within the context of a running process, and have the whole thing change immediately, without built system hot reloading tricks or incremental compilation or anything of the sort is crucial for something like Emacs.
    Regular Syntax
    Emacs has structural editing commands built into its core. The syntax that gives those structural editing commands the most powerful functionality with the least parser complexity is Lisp, or something similar, hands down.
    Homoiconicity
    The ability to represent actual, executable code, in a format that can be meaningfully structurally manipulated, walked, and modified – that is, linked lists of symbols – as well as just evaluated, as opposed to just as raw strings you have to munge into the right shape and then pray work when you evaluate them, is crucial to functionality like the unity of user interface and programming paradigm. For instance, the ability to capture the actual resultant code the user executes (impossible if you have no representation of code as data) and use it in a data structure that the user can easily view and edit, and then ask Emacs to programmatically manipulate to produce a reusable command that's just like every other function or command reliably, is key to having keyboard macros as powerful as Emacs does. Without such homoiconicity, user actions would have to be represented as new, custom data structures that are then interpreted to be run later by code written in the language the system is actually written in: a mini sub-language that creates distance between the user and the internals.
  • Things that it has uniquely:
    Advice
    Having the ability to advise the behavior of any function in the language at a native level, instead of having to declare advisable functions in a special way or something, as you'd have to do in any language where this was not built into the ground level, is extremely important. See the section on composable modifications.
    Buffers
    Having the concept of buffers built into the language virtual machine, instead of implemented as a library, ensures that buffer manipulation is literally as fast as it can get; moreover it makes it easier for buffer manipulation to form the core language primitive for text manipulation, which benefits user-system homoiconicity and also performance at text manipulation, since buffers are backed by a special data structure designed for fast text manipulation that's much faster than the regular strings other languages use (in this case, gap buffers, but as far as Emacs Lisp is concerned, that's just an implementation detail).
    Buffer-Local Variables
    The ability for some global state variables in Emacs Lisp to be transparently buffer-local (as in, can map to different values depending on which buffer the current code is executing in) is crucial for encapsulation, composability, and generality of Emacs modes and functionality, and it would be almost impossible to achieve ergonomically without that concept being built into the language – which in turn requires buffers themselves to be built into the language.
    Propertized Strings
    The fact that the native, first-class, default string data type in Emacs Lisp is propertized text is deeply important to it being able to ergonomically integrate with Emacs's fundamental paradigm. No other language has this.
    Dynamic Scope
    being able to temporarily redefine a variable or function just for the tree of function calls below a scope is incredibly useful for flexibility, customization, and malleability. Pure lexical scope is simply too brittle by itself!
    Simplicity
    Despite being quite powerful – honestly a good step above most modern scripting languages in power, despite being "merely a domain-specific language", a fact which is what enables such incredible applications to be regularly written in the Emacs computing environment – Emacs Lisp is at its core a very simple and consistent language. I think this is important for learnability.

Now, is this to say that Emacs Lisp can't be improved, or that it's the best language ever? Dear god no, far from it!

  1. Its standard library, while incredibly complete and featureful for the purpose, is full of the same inconsistent naming and argument ordering as Common Lisp's. I think a major naming and argument ordering overhaul could be very helpful to avoid common mistakes (although I don't see how this could happen).
  2. Without cl-lib and eieio, Emacs Lisp's standard library, while rich in other ways, lacks crucial functionality for things such as list manipulation, loop comprehension, conditionals, an object system, and generic multimethods; thus the attempt to discourage the use of those already built in and quite performant libraries out of a dislike for Common Lisp idioms (and concerns over "bloat" which somehow didn't apply to the creation of a second duplicate standard library with seq.el) has led to a proliferation of alternative standard libraries, or at least components of such, such as dash.el and seq.el, which are:

    1. in the former's case, not built in, which leads to larger startup times and slower execution due to dependency bloat all for functions which duplicate functionality already available within Emacs,
    2. in both cases less in line with the Emacs Lisp programming style historically, based on what's been around longer, what has been used the most, and Emacs Lisp's shared heritage with Common Lisp, and
    3. at least in the latter's case, much slower1.

    I think switching to encouraging a more Common Lisp style of programming, including making copious use of cl-lib would solve this problem (the opposite of Stallman's suggestions, ironically, because I don't think the "simpler" and "more elegant" libraries like seq.el provide complete enough functionality in the way the Common Lisp extensions do, so it'd just lead to a further proliferation of libraries).

  3. And of course there's the lamentable fact that Emacs Lisp is a Lisp-2 – I think the weight of the technical arguments falls strongly in favor of Lisp-1s; however, it's far too late to change this aspect of Emacs Lisp, and it's also far from the disaster many think it is.
  4. It's also pretty slow, even with native compilation (although not slower than Python!), which is obviously a problem when you're writing tens of thousands, hundreds of thousands or even millions of lines of complex code in it.

In all, however, I think these issues must be fixed without replacing, removing, superseding, significantly changing, or deprecating Emacs Lisp. Both because of the unique features of Emacs Lisp listed above, and because one of the biggest strengths of Emacs is the 40 years of package development that has occurred for it, all of which was done in Emacs Lisp and depends on that language intimately. Losing incredible, killer packages like paredit, smartparens, org-mode, org-roam, magit, elfeed, SLIME, CIDER, web-mode, AuCTeX, Emacs Speaks Statistics, emacspeak, mu4e, Proof General, citar, and the many other killer applications that have been created for Emacs that regularly convert users of other editors and attract laypeople, would be a complete and utter disaster. The existence of these packages is part of the core appeal of GNU Emacs, what makes it worth engaging with in the first place!

I also think that trying to introduce other languages – even equally domain-specific ones – alongside Emacs Lisp would be a mistake, since it would ruin many of the important properties I've been talking about above, such as the total homogeneity of the Emacs computing environment.

What I want is not the replacement or elimination of Emacs Lisp, nor a rewrite of the Emacs core into Rust, Common Lisp, or RnRS Scheme, then – what I want is the evolution of Emacs Lisp towards a sort of Common Lisp sibling language that is slightly simpler and has Emacs extensions, and then the gradual assimilation of the core into Emacs Lisp as it grows faster. Emacs should be Emacs Lisp. What I want least of all is for Emacs to become a polyglot environment, where impedance mismatches make inspecting, debugging, integrating, interoperating, and dynamically modifying various packages difficult, and where I have to learn a new language for every package I might want to contribute to.

In light of all this, I think the new Guile Emacs has the right idea in this regard: replace the engine under Emacs Lisp with something better – allowing better performance and more expressiveness, and homogeneous interface with similar Lisp languages – while keeping Emacs Lisp the same, but opening the door toward evolution down the road. I also think that the Emacs maintainers have been making solid strides toward this already with the introduction of lexical scoping by default, cl-lib itself, native compilation, and probably in Emacs 31, MPS!

Also, on a personal note, I not only think the "nicer language du jour" – Python, JavaScript, Scheme, or Lua – can't do the job as well as Emacs Lisp does, I also just like them a lot less than basically any Lisp. And I just find Emacs Lisp a complete and utter joy to use, despite its warts, in a way I don't enjoy those languages.

5. Emacs and the UNIX philosophy, part 2

See also: UNIX, Lisp Machines, Emacs, and the Four User Freedoms.

Each of the main tenets of the UNIX philosophy has essentially a kernel of truth to it:

  • we want to break functionality down into individual pieces so that they can be handly used as tools,
  • we want a common data interchange format so that everything is interoperable,
  • and we want to make sure to design everything to be composable.

However, it misunderstands how best to achieve each and every one of these options. Human-readable text is a good medium for data, but it is too flexible to be the data interchange format, by itself, and lacks out-of-band capabilities, as I've said before. Breaking down functionality into entirely separate programs/processes is way, way too hard of a line, causing both severe overhead (forking is expensive), making it more difficult and slower to communicate data in a useful way (no shared memory space), and turning each set of functionality into a tiny little black box where it's difficult to modify or adapt its behavior except through a million esoteric and idiosyncratic flags (see also: programmability vs customizability). Likewise, UNIX's answer to composing programs is shell: a weak, underpowered, insecure language full of gotchas that has to shell out to a million sub-languages with different syntax and semantics to do basic things, and which provides no common substrate of conventions and interoperability between components that are supposed to "compose."

In my opinion, a better solution to each of these examples is available, and exemplified by Emacs, as the last heir to the Lisp Machines:

  • Emacs breaks all functionality down into functions, which can be used handily as tools, but which don't come with the costs of separate processes/programs. It also means that complete, holistic applications can be written for Emacs and still remain composable, since all of their internal functions are exposed to the whole system to use and reuse and script and automate in any way desired. This means that there doesn't have to be an intentional attempt to expose an Emacs program's separate functionalities as sub-commands, or a tension between large integrated programs and small composable "do one thing and do it well" programs. You can have both. (Emacs also has powerful process manipulation capabilities and is not afraid of shelling out, too. Emacs in fact makes an excellent shell.)
  • Emacs provides a data interchange format with all of the benefits of text and none of the downsides. It is flexible, easily reinterpretable, composable, human readable, and simple, but has the capacity to easily encode structured data and the semantics of the data provided out of band and in a way that can be reliably communicated and understood within its environment.
  • Emacs provides a computing environment and language that does functional programming (which is what the UNIX philosophy really is is) better than Bash. It can compose functionality (processes and functions) and transform data in a much more powerful and ergonomic way.

6. Gap Buffers

A lot of people criticize Emacs's use of the gap buffer, and suggest that it go with a more complex data structure at the core like ropes. While I think there can certainly be room for improvement with this core data structure, I don't think ropes are a good solution, and I think this is far from a crippling issue. Let me start with the second point first.

One of the biggest arguments against gap buffers is that they're simply not optimized for multiple cursors. I'm not a big fan of multiple cursors (as opposed to kmacros and occur-mode), but more importantly, gap buffer multiple cursors performance doesn't seem noticeably worse than ropes for most reasonable editing situations on a human scale. Just look at this graph:

cursor_distance.png

I'm eyeballing it here, but it looks like the scaling factor is around 200µs per 6000 bytes. If we say that the threshold of human delay perception for something that isn't extremely reaction-time intensive like gaming is around 50ms, then cursors can be 150,000 bytes, or about 12,500 lines (assuming 120 character lines of code) apart before the delay even becomes noticeable to human perception. With the less naive multiple cursor editing system suggested in that same article, that delay can be cut down by 30%, as well, meaning a 16,250-line separation before the difference even becomes noticeable to the human brain. And you're not going to be using multiple cursors all the time, so a barely-perceptible amount of lag, say 100ms, as you type when using multiple cursors is hardly an issue – which raises that ceiling to 32,500 lines of code apart.

Which raises the question. Who in the hell is using multiple cursors that far apart? The big benefit of multiple cursors is that you can see the edits you're making being made live in all the places you're performing an edit, so that if something goes wrong you can respond to it immediately instead of having to re-record your keyboard macro or what have you, but what font size/monitor combination could possibly show that many lines at once? And if you're skipping around a file viewing only a few cursors at once as you make an edit, wouldn't a keyboard macro be essentially equivalent anyway? More than that, if you've got a file that big and you really want to use multiple cursors for some reason, why not create an *occur* buffer with all the lines you'll be changing, edit that with multiple cursors (likely to be extremely fast) and then write all the edits out at once, which will take less time in total for the gap buffer to do? And that's not just a performance optimization, that's actively better than the alternative, since with an occur-mode buffer there's actually a snowball's chance in hell you'll be able to actually see what you're doing with all those cursors (and so not negate most of the point of multiple cursors)!

Moreover, in my personal opinion, multiple cursors are overrated:

  1. Having to pay attention to every place where I want to make a rote macro-like change at once as I'm making the changes, cycling my eyes between all the lines and trying to keep track of all of them, is a lot more stressful for me than just recording what I want to do, creating an occur mode buffer of all the places I want to do it (or just using isearch to jump between them), and doing the changes one at a time, responding to any accidents or special cases in each line one at a time as they show up, instead of having a big mess appear all at once in several lines. It's not even more work, since to create multiple cursors in widely separated places you'd need to use search anyway.
  2. Additionally, with keyboard macros, there's an easy and natural on-ramp to making whatever you did useful later, through naming the macro, saving it to a register, or even converting it to Elisp code, instead of the operation you did being temporary and ephemeral, limited to the specific instances that you wanted to deal with in one moment.
  3. Speaking of that, keyboard macros have much greater versatility, at least in Emacs, where they can run any keyboard shortcut or Emacs command, not just edit text in a specific place in a buffer. Thus you can use the same facility you use to automate editing text to automate your editor.
  4. Moreover, with Emacs macros at least, it's trivial to structurally edit macros, instead of needing to record them, if something goes wrong, so it's easier to respond to mistakes – not quite as easy as multiple cursors, but I think the other benefits outweigh that.

In addition, it is my opinion that using gap buffers is actually a really savvy development choice. The reason is that they're dirt simple, and more than good enough at human scale, in comparison to other popular data structures which are much, much more complicated to implement and, while they do have benefits at the very high end, tend not to show those benefits most of the time. Moreover, gap buffers have extremely low constant costs, versus other more complex data structures which have a high constant factor, and require a ton of upkeep and maintenance and can get into "bad states": https://coredumped.dev/2023/08/09/text-showdown-gap-buffers-vs-ropes/.

There are some data structures that keep many of the benefits of gap buffers but can alleviate a lot of the problems Emacs has with long lines and non-local editing, however. For instance, there's the Cluffer.

7. Living in the terminal vs living in Emacs

The other environment that many tech-savvy people tend to try to live their whole lives in, that offers a coherent integration between everything and a limited but flexible set of core components, is the terminal/shell. I think I've given a fuckton of evidence on this page (and in my blog post on this) that, as a text-oriented computational environment, Emacs is infinitely superior: its component pieces are fully transparent, scriptable, malleable at run time, far better integrated, can communicate using a better data format, and so on, and Emacs itself is a far superior shell. Here are a few more reasons:

  • While terminals are out here emulating dot matrix printers from 1982 (that have to split images up into character-sized sections and only support 6-bit color, can't be interacted with, and need to be transported as ANSI escape codes in-band) in order to get image support, Emacs can represent clickable, draggable, hoverable, right-clickable, etc arbitrary UIs with SVGs (see the next section), rich text, images, PDFs, clickable buttons, and more.
  • While shells are trying to use ncurses and readline to give you some kind of acceptable autocompletion support, Emacs can give you the full power of your IDE in your shell.
  • TUIs in the terminal all have to manually hard-code keybindings, instead of exposing their functionality as commands that can be arbitrarily remapped by the user at the computational environment level, which also means that while Emacs can guarantee consistent keybindings, users of TUIs just have to hope that everyone agrees on a common standard (and interprets it the same way).
  • TUIs are not usually very scriptable or interoperable under a terminal, so you have to choose between a teletype level interface (shell commands) that offer scriptability and interoperability, or an MS-DOS level interface that offers none of that. Emacs can offer you TUIs that are fully as interoperable and scriptable (even allowing you to take the UI generated by one application and hand it to another, or export it yourself for manipulation and editing, or have multiple interpreting and working together in the same UI).

If you've ever felt the desire to live in the TTY, consider living in a fullscreen Emacs window (or EXWM) instead.

8. SVG Interfaces in Emacs?

One of the really interesting little things that I've seen on the margins of the Emacs world is people making use of the fact that SVGs are a really good medium for drawing arbitrary graphical user interfaces, since they allow drawing arbitrary shapes, colors, gradients, and text, in a way where it is clearly structured and you can assign relative constraints and sizes, and where that structure is coherently maintained instead of being lost when drawn to a pixel image or something, and the fact that Emacs actually allows you to detect arbitrary mouse events with precise pixel locations and supports operating system context menus and menu bars to construct arbitrary user interfaces embedded within Emacs. An example of this is Emacs Easy Draw:

edraw-screenshot.gif

As you can see, this looks and functions like a real interface, no text-mode funny-business or shenanigans.

Another, more minor, example, can be found here, where messages in the Emacs echo area are given full GUI-level graphics and layout capabilities.

I think key to why this seems to work so well is that it's actually really easy to build SVGs in Emacs Lisp, since SVGs can be represented more meaningfully as data structures with semantic information about what they're trying to depict (i.e. what shapes, what text, where the boundaries of them are, how they're translated and rotated), so it's easier to construct them programmatically, and since they're a symbolic tree structure very friendly to Lisp's native data structures, and Emacs Lisp's macro facility makes an SVG DSL quite easy. I think the other reason this is so profitable is that, since SVGs are a limited primitive, it's easy to pull semantic structure and meaning back out of them, when for instance the user clicks on a location within them, or when things need to be changed. If we were trying to use regular pixel buffers, all of this would be a lot more processor intensive and unwieldy.

The possibilities with this are honestly endless. Fully-featured Jupyter notebook style inline widgets and interactive graphs for org-mode, embedding more fully-featured SVG and even bitmap image editors, Possibly even using SVG as the basis for rendering full HTML webpages inside Emacs? Obviously, building on all of this will take an absolutely gigantic, humongous amount of work to actually build the things that use this the way I'm suggesting, but the point is that the potential, the capabilities, are absolutely there.

Of course, there are tradeoffs to this. SVG-based interfaces have the disadvantage relative to something like Xwidgets (which allow you to propertize text with GTK+ widgets) that they aren't interoperable with the host operating system, and we all have to build those widgets, and their interactivity, ourselves, from scratch. However, the advantage relative to Xwidgets is that it's inherently more self-contained and thus more cross-platform, relies on fewer external variables (e.g., relying on GTK and whatever other UI toolkits Xwidgets would need to use to support other platforms to even continue supporting this and not change so much that maintenance is prohibitive, as well as needing to rely on external platforms continuing to allow this), and the entire set of UI widgets, and most of the interactivity and functionality, would be writable in Emacs Lisp, meaning that maintenance and improvement of an SVG-based widget library would be open to far more Emacs users, as well as much easier and quicker to write and experiment with, and in addition also meaning that these widgets would actually be comprehensible and accessible to the rest of Emacs, instead of black boxes.

Another alternative for embedding arbitrary user interfaces within Emacs is turning Emacs into a Wayland compositor. This has many of the downsides of the previous option – interfaces added thusly being black boxes to Emacs Lisp, difficulty of contributing, external dependencies – but has the massive advantage over both SVG interfaces and Xwidgets that it allows importing other interfaces, like Jupyter itself, or Inkscape, or whatever, wholesale into Emacs without the need for extended development. With a little additional elbow grease, this approach could even insure that such imported applications could not only be embedded in Emacs text, managed like Emacs buffers, etc, but be ensured to have consistent keyboard commands. However, ultimately, they'd be even more of a black box to Emacs than Xwidgets would be, making it so that such an approach would probably lead to turning Emacs into a window manager more than actually integrating external things successfully into it. Unless the things that were integrated were written specifically for Emacs, anyway, but then you might as well use SVG. (Unless you want to do 3D rendering or something, which could actually usefully take advantage of Emacs being a Wayland compositor – maybe the ideal is a combo of EWX and SVG)?

Making a full UI toolkit via SVG in Emacs Lisp is honestly an interesting project to look into in the future. It might also be possible to use a headless browser or reasonably complete layout engine to render web page HTML to SVG which Emacs could then display, allowing a faithful 1-to-1 reproduction of HTML pages, just like Xwidgets offers, that also allows Emacs to inspect the content of those pages! Unfortunately, according to a few cursory searches, it looks like most command line HTML to SVG tools are quite dead, and since command-line processes are the easiest to control for Emacs, that does present a problem. However, I maintain this remains a feasible theoretical possibility.

9. Emacs should never become multi-threaded

There's a lot of discussion in Emacs community around how Emacs should become multi-threaded. While I think a lot of these ideas are interesting and could work, I think they're ultimately misguided. Most of the fundamental appeal of Emacs is that, at its core, it is a big, huge ball of mud – and by mud, I mean global mutable state. The fact that it's just this huge ball of state that anything can access and modify at any time, from anywhere, and immediately see the changes live, without hindering the access and modification of that state by any other part of the program, is what allows Emacs to be what it is in the first place. But that kind of global, shared mutable state is just fundamentally incompatible with multithreading, since it leads to race conditions, instability, memory corruption, conflicts, and in general just extremely difficult to track down and resolve bugs. And any solution to that problem would fundamentally chip away at what makes Emacs unique:

  • Implementing zero-shared-memory message-passing, like you can find in languages like Erlang: essentially reduces being able to modify the global state, or the state of other programs, to a limited, rigid, non-transparent API – at that point, you might as well move packages out into plugins and extensions communicating with each other and the editor over RPC like NeoVim does.
  • Implementing exclusive memory access systems like locks and mutexes on buffers and their associated local variables: drastically reduces the usefulness of Emacs's unique (compared to Vim and NeoVim at least) ability to run multiple modes (one major mode plus N minor modes, or even multiple major modes on different regions) on a single buffer, imbuing all kinds of different overlapping meaning and functionality into buffers in a way few other editors can match. Additionally it doesn't solve the problem of dealing with non-buffer-local variables, and introduces the possibility of deadlocks, which are an almost as severe problem as race conditions!

Instead, I think Emacs should lean into what it already does well: shelling out to external processes when parallel execution (for example, super heavy long running computations) is needed. This is already an extremely common pattern in Emacs programming, and one that I think should be enhanced to make as easy and ergonomic as possible, through two methods:

  1. Emacs Lisp's standard library already has extremely powerful primitives for this kind of functionality, but they're ultimately pretty unwieldy and verbose, require a lot of boilerplate and esoteric knowledge about process-management terminology that may seem natural to almost any computer nerd worth their salt but will be alien to the humanities end of the Emacs user base, and just aren't good enough yet. Thus I think integrating things like emacs-aio, which allows you to do async (based on Emacs generators), mutithreaded, and multiprocess programming using a common library interface that's extremely easy and generally works and looks like synchronous code that we're all familiar with, and integrates deeply with Emacs's command loop, is probably the way to go.
  2. Allowing Emacs Lisp programmers to shell out to external Emacs processes to do work more easily by including something like the emacs-async package – but that can allow Emacs instances to communicate data structures (and even closures and callbacks) using raw bytecode instead of needing to serialize and deserialize S-expressions from buffers – in the Emacs Lisp standard library. This gains you the asynchronous programming benefits of multiprocessing and shelling out, without having to lose the unified language, set of data structures, interoperability, and consistency, of the Emacs computing environment.
  3. Creating a pool of Emacs worker processes that can all pull jobs from a central queue (and maybe do work-stealing between each other), for the purposes of making point 2 more efficient.
  4. Making the core of Emacs (the display code and Lisp interpreter) more multithreaded. This shouldn't effect what "user space" Emacs is like at all, but could prevent things like long lines or garbage collection from blocking everything else.

10. Why not to use Emacs

I've talked at extreme length on this page about the reasons why I like GNU Emacs, even to the point of defending certain aspects of it that are usually treated as sub optimal even by extremely dedicated users and maintainers of the software. However, I am no fanatic. Emacs has flaws, some of which I think can be overcome, and some of which I think are the inherent consequence of the tradeoffs it needs to make to be the unique thing that it is.

In fact, there's a reason I phrased the first section on this page so neutrally, as just being about what's unique about Emacs, not "why you should use Emacs" or "why Emacs is the best text editor/OS." And the reason for that is simple: I think the things that make it unique can be massive strengths for some people and complete deal breaking downsides for others. This is why recommending Emacs is one of those things you should really only do to someone who is actively looking for a change, or maybe already a little interested in it (perhaps from looking over your shoulder while you use it). It's like a religion that way: the best way to convert people isn't overt evangelism, but just by not trying to convert them or caring about converting them at all, – only living your life and showing the attractiveness of what you're doing, and letting people who are already interested or looking for a change ask you of their own accord what's up and how they can get in on it. Of course, just like a religion, adherents can often fail at this through enthusiasm to share something that has changed their lives 😉

So, the downsides of Emacs.

  1. Surmountable but still very real issues:
    1. Asynchronous programming has not sufficiently permeated the Emacs ecosystem, and most examples of asynchrony/multiprocessing currently in the ecosystem rely on non-Emacs external tools, which, while advantageous, in my opinion, in the name of reusing work and deep host OS integration, means that you often have to install or have a package download and compile a myriad of little external utilities to use a package that won't occasionally lock up your editor. This can be fixed with my multiprocessing suggestion above.
    2. Emacs Lisp, while not a bad language at all in my opinion, is somewhat old-fashioned, and moreover has an issue with a proliferation of different alternative standard libraries. This can be at least improved with my suggestions for improving Emacs Lisp.
    3. Emacs Lisp is relatively slow, which is a problem for something that's essentially running an entire editor and suite of applications in a single thread (we'll get to threading later). This is being improved about 3x with native compilation, and will probably be further improved (possibly up to a further 10x) if Guile Emacs ever happens.
    4. Emacs's defaults are really often just completely baffling and counterproductive (some examples: the way it handles backup and autosave files, the default completing-read system, the fact that GCMH isn't the default GC strategy, the way canceling an empty interactive search reverts to batch-mode search, the fact that which-key has only recently been added and is off by default, the default JIT lock settings, and much more). Even if respecting IBM CUA isn't really in the cards, there are an infinite number of other things that could be done to make the initial experience of the editor so much better, so many built in packages that could help contribute to a better out of the box experience that just come turned off or misconfigured by default. This could be fixed if only the mailing list politics would allow it.
    5. The Emacs garbage collector is a stop-the-world, non-incremental, non-copying, conservative, single-threaded, low-throughput, non-memory-returning toy compared to modern garbage collector technology. This is being fixed with the scratch/igc branch, which is very likely going to be merged into master.
    6. Interfacing with external programs that speak JSON or JSON-RPC like LSPs can cause the single Emacs thread to choke on large projects just from having to ingest that amount of serialized data. This is being fixed by the new JSON parser.
  2. The insurmountable problems that you simply just have to accept if you want to use Emacs:
    1. It will always be slow-er than something like Helix or NeoVim, because it fundamentally relies on a things like late binding for every possible function, hooks and advice, most of the editor being written in a highly dynamic scripting language, and just doing more in the editor.
    2. It will always be possible to freeze or substantially slow down the editor's response times if you or the authors of the packages you use aren't careful, siduce threading into Emacs without destroying what it is at its corewithout destroying what it is at its core (barring some miracle solution I can't see).
    3. Thanks to its malleability and transparency, it will always be easier to break your Emacs or get strange, inexplicable behavior with it, than any other editor. This isn't as bad as the similar situation with Linux distributions because the core, clean image is always there, an emacs -Q away, but it's still an issue.
    4. If you want to have full control and configuration of your editor, you're going to want to jump into the deep end with vanilla Emacs, but that will always require more configuration and (unfortunately) maintenance than a "just works" editor like VSCode or Helix. Of course, this is very robustly and completely obviated by things like DOOM Emacs, but if you're getting into Emacs, that might not be what you want.
    5. Emacs being a whole separate computing environment on top of your existing operating system:
      • will not be able to perfectly integrate and align with the rest of your OS, which you will almost certainly have to interface with in the form of applications that can't be embedded into Emacs and your window manager, within which it exists, unless you use EXWM;
      • is also simply… another computing environment: if you like the one you already have well enough and don't want any of the additional benefits Emacs can offer then, well, I can't blame you. Although I know I prefer Emacs.

Emacs truly isn't for everyone. If the unique properties of Emacs seem more important to you than their concomitant downsides, then welcome to the club; if they don't, no hard feelings – it's certainly reasonable to have genuinely different priorities!

11. Org is a general-purpose information management hypertext system for Emacs

One of the most common philosophical misunderstandings that I see people have about org-mode is that they assume that it must have a particular intended workflow, like any other literate programming, personal information management, project planning, note-taking, or other similar such system.

This makes sense, since org-mode does represent a cohesive thing, the closest analogue of which, for most people, is some kind of application: after all, it's not just a notation, but a set of interfaces, functionality, and presentations deeply integrated with that notation – possibly integrated to the point of the two almost being completely identical, since anything that isn't literally org-mode.el really isn't "org mode," even if it uses the same syntax.

Despite making sense, however, this misunderstanding causes problems for people trying to figure out how to use the system. It's often unclear to them how to even begin, because they're looking for a specific place to start or way to approach using the system, or expecting to find a tutorial that explains the expected workflow from beginning to end. Instead, all they find is a 600 page manual documenting a dizzying parts manifest of features the vast majority of which they won't even be able to absorb, let alone intuitively understand how these strewn-about component parts in the abstract might possibly be assembled into a concrete, working workflow. This often leads to questions about the "right" way to use it, the "best" way to achieve something, and in searching for answers they'll stumble across a Babelesque bazaar of different responses, most of which don't go into detail but simply recap the more basic, core features org-mode has which every one uses, without mentioning the one or two features that might actually be revolutionary for that specific prospective user's workflow.

The result of this is that most people who use org-mode tend to assume it can only be used in one fairly rigid way – the sort of community-folklore way of using org-mode that's sort of developed as people with that workflow (including the authors) try to explain what org-mode is to others by example, and then those people adopt that workflow as rote because they're used to tools and systems with particular expected workflows – and if their desired workflow doesn't fit with that folklore org workflow, they assume that isn't a way org can be used and move onto some other system. For instance:

  • Most people don't know that org links can be created to refer to, inserted in, and followed from, any file at all, not just org files, making things like Denote and Howm, the main claim to fame of which is allowing you to put their own custom link markup in any other markup file to link between things.
  • Or that you can label org headings with GUIDs and allow any heading in your agenda directory to be a suggestion when completing GUID-based links, as well as setting up a capture template that prompts you for a file name in your agenda directory whenever you capture something new, allowing you to build a Zettelkasten note system where each note is in a separate file but can link to any other heading in any other note as easily as if it was in the same note file, and where GUID locations are stored in a database for easy retrieval, meaning org-roam is largely unnecessary unless you have a truly gigantic number of notes.
  • Nobody seems to realize that with a simple regex modification, headings referred to only by tags or GUIDs, without actual titles at all, are possible in org, making EKG largely pointless.
  • Similarly, nobody seems to realize that you can use the built in org publishing system for publishing blogs without needing to even write any new code, and RSS feeds with a 40 line function, so all those org static blog publishing systems are totally unnecessary.

All of this functionality is possible because, like Emacs, org-mode is less an application with a specific purpose and intended workflow, and more a whole system for doing a related set of tasks. Where Emacs is a system for manipulating rich text, structured data, file systems, and operating system processes interactively in a malleable runtime environment, org-mode is a system for creating and manipulating structured rich text hypertextual information and viewing and exporting it in various ways, interactively. (You can see why org-mode and Emacs are such a natural fit!) It doesn't have "a workflow" because what it offers instead is an extremely powerful general markup language for expressing highly structured hypertext information (both mainline content and metadata), a system for parsing, reading, arranging, and manipulating that structure, and a set of interfaces for that functionality. What you do with that, how you use it, is completely up to you. There are an infinite number of ways to use org-mode, and the difference between any given workflow in org-mode is just a difference of habit, mindset, and a little Emacs Lisp.

What's awesome about this is that tools for thought, systems for managing your thoughts and information, are at their best when they're monolithic, fully integrated into a single system just like your brain is so that you can easily create links and references and relationships between anything that you're thinking about or having to manage, easily able to flow between different styles of workflow and thinking regarding information management depending on what makes sense for the given thing you're grappling with or even how you're feeling that day, or how your workflow changes over time, without ever locking you into one thing. A general purpose information management system that is flexible enough to change with the context and change with you, and handle any kind of information management task you might need, will be infinitely more useful than a bunch of rigid silos, over a longer period of time.

12. On the importance of using what's already in Emacs

One of my biggest pet peeves is people not understanding and making use of all of the powerful functionality that Emacs already provides. (That's why I created Quake Emacs). So many packages have been integrated into Emacs's core, and Emacs itself has been extended with so many incredibly powerful features over the forty years of its evolution, and yet so many of them are poorly documented or marketed, off by default, or badly configured out of the box, that oftentimes people prefer to use huge overhaul packages that complete reinvent the wheel, replicating functionality that could be had with core packages.

What's the actual problem with this besides just wasted potential, though?

In my opinion, it's really two things: one, these big reinvent-the-wheel packages usually require a lot more maintenance than they would if people built on the shoulders of giants, configuring, or improving what's already a part of Emacs, since these packages typically have to reimplement the built in functionality and then go to extra effort to integrate with Emacs on top of it, when all that effort could go into improving the stuff that's already being built and maintained in core Emacs, and two, those big overhaul packages usually don't integrate as well with core Emacs and other packages, possibly also requiring any packages that want to hook into their new more advanced functionality, rather than the functionality that's on by default in Emacs, to explicitly depend on these packages, which makes the Emacs ecosystem a lot more brittle and less modular and composable.

However, there is a new breed of overhaul packages that at least don't fall foul of the second issue, like Vertico and orderless.

13. A creation-first interface

One of the primary wonders of computing is that creation on a computer is essentially completely free, once you have one – the world of information is a post-scarcity one where the only cost is your time, and that's it. Bits can represent anything, from prose to poetry to images to film to interactive games to music, and bits can be created and infinitely duplicated (taking poetic license here, don't come after me, I know they're technically being modified not created) at zero cost, on your own computer or between any other person's computer in the entire world. Even distribution, in the digital age, is nearly free – pick up a $35 computer on eBay or from your nearest surplus store and run Linux on it, every Linux distro comes with a network server – even if getting attention to that distribution isn't.

The other primary wonder of the computer is that it is an infinitely flexible information processing and communication machine – a meta-machine that, if you can describe a behavior, can instantly become a machine-to-do-that-behavior. There's no limit: as long as we understand what we want the computer to do, as long as we can describe the way we want it to process information, or how to link information to behavior, the computer can automate it. There is no fixed purpose to a computer. Thus computers can be a great help in people's everyday lives (as long as they have to communicate or process information regularly), freeing the user from time wasted with repetitive drudgery, giving the more time to think and play and create, or spend time with their friends and family, or hike around in the outdoors.

I feel that this has been lost in the modern era of personal computers, where user interfaces revolve around things like start menus, task bars, desktop folders, and home screens, oriented around launching circumscribed applications for specific tasks (usually content consumption) that present no affordances to be easily wired together or scripted or pass information quickly from one to another for processing, instead of notebook interfaces, shells, or REPLs. Don't get me wrong, there are a huge number of ways in which the fundamental metaphors, design, and capabilities of personal computers and their interfaces have massively improved, and we should preserve those where we can, but my dream operating system would be oriented around a notebook-centric interface, something like what Karl Voit calls an application-less system.

In a document-centric operating system, you should boot up your computer and be immediately presented wixsth something like a notebook interface: a document for creating things – anything you can imagine.

  • You should be able to just go right ahead and start typing if you want to write prose, take structured hierarchical notes or unstructured Zettelkasten notes, register rolodex-style grouped metadata entries, list tasks and subtasks, document upcoming events, or create a presentation. This should be done with simple plain-text markup (for maximum versatility) but still offer deep structural editing, manipulation, and analysis, so that the user isn't stuck doing all the legwork themselves.
  • You should be able to open existing notebooks from previous sessions, or request various views of cross sections and queries on every notebook in your entire system, correlating all the information and content you have created easily and coherently.
  • You should be able to follow hyperlinks inside and between notebooks, and to applications and external websites, and even to activate code or UI elements or run key commands, to correlate together as much information as possible.
  • You should be able to include documents from over the network transparently into your notebook without having to explicitly manage downloading and attaching them.
  • If you want to run code, you should be able to create a code block for any language you have installed on your system and want to use immediately, in line, without having to open a separate editor or IDE program or terminal or REPL, and when you're editing that code you should have access to full syntax highlighting, refactoring support, and autocompletion.
  • Then you should be able to just run that code block with a simple key command and see the output – even if that output is an image or a UI widget, which you should be able to interact with – and then plug that output into further code blocks (even if they're for different languages) or tangle a source code file from your code blocks for contribution of version control.
  • Controlling your system should be actionable through writing whatever code or commands you want to execute in a code block or, if it's small enough, directly inline anywhere in your other text and hitting a key command to execute it, as well as through a fuzzy completion command palette for simpler single commands. You should be able to control other computers you own transparently over the network from this notebook interface.
  • All applications should be accessible through running a command in the palette to open it in a new split or on a new workspace, or running it inline in the existing notebook interface to get an embedded interface, if that makes sense.
  • Every time you open a new window or switch to a new desktop, either an existing notebook or interface, or a fresh new one, should be presented, as makes sense.
  • The environment should offer a means of recognizing patterns in the text you've entered (or semantic metadata or markup on that text) in your notebook and turn those into powerful hyperlinks with a default action and a semantically aware and extensible list of other actions, so that your notebook system can be highly hyperlinked and manipulable with minimal intentional markup.
  • If the user wants to view things like web pages, images, or videos, that should be easily possible either in a new window or inline with the notebook – as they desire – but the notebook should allow the user to toggle "read only mode" off on any of these things, at which point the piece of content will be automatically saved and an editing mode entered where they can annotate, edit, modify, or expand on it.
  • Anywhere in the user interface of any application or in any notebook, it should be possible to store a hypertext link to that location and add it to any other notebook.
  • Anywhere in your interface it should be as easy as possible to open a new or existing notebook for adding-to.
  • Notebooks that contain hyperlinks that run code in the system language – which should be able to launch and control any application, modify or control the system itself, modify the notebook, including changing or adding other hyperlinks or code blocks and running them, as well as opening new documents internal or external – could allow users not just to script things but to trivially construct their own user interfaces for various everyday purposes in an essentially free-form, low-code way, like the beloved classic Macintosh HyperCard system extended to the entire operating system.

Combined with the properties of a homogeneous, open, malleable system with true homology between all user interactions and how interfaces are actually implemented and scripted, I think this interface would maximally empower users: it could incline them toward creation over consumption, specifically creation of their own content, and give them an up-front way to learn how to control, modify, and automate their system, so that they aren't stuck just pulling a lever that's now digital instead of physical.

There is only one system I know of today that allows an experience like this: Emacs. Either through packages like org-mode (& org-babel, org-agenda, org-capture), Embark, org-transclusion, and of course built in functionality like eval-last-sexp, eval-defun, the *scratch* buffer, and TRAMP, or through the less powerful, older, and more obscure, but also much more integrated and versatile GNU Hyperbole.

14. Emacs should shell out

There are two approaches to adding tools and functionality to Emacs.

The first one is to take an existing program with IPC functionality or a command-line interface – usually most profitably the latter, for as much as I think the UNIX philosophy is incomplete and short-sighted, the idea of making sure that as much program functionality as possible can be called from the command line, and having everything communicate through text streams, ensures the composability and scriptability of programs far better than most other paradigms – and write an extensive library of Emacs Lisp functions and defcustom variables wrapping that functionality, providing a way to deeply script, control, and modify the behavior of that program, as well as a set of transient, hydra, or in-buffer user interfaces for that program.

The other is to completely reimplement the functionality of that program in Emacs Lisp from the ground up, using the powerful standard library and platform-agnostic language runtime provided by Emacs itself, providing a self-contained package that functions like a fully independent program installable through the magic of use-package or package-install.

Each of these approaches has benefits and drawbacks.

The first approach's main benefit is that it allows Emacs to take advantage of the already-existing powerful and featurful tools that exist outside of it, while still lending most of the benefits of the Emacs computing environment to those tools. Thus, Emacs can benefit from all of the functionality that those tools have already implemented, and all of the new functionality that they will implement in the future, essentially for free. The maintainers of the Emacs package also get to benefit from sharing the maintenence burden for that functionality with others outside the very small Emacs community, and users of the Emacs package get to benefit from using the same tool that a large community of non-Emacs users also use, such as greater interoperability, more documentation, and more access to help. The main drawback, of course, is that since the tool is not implemented from the ground up in Emacs Lisp, only the functionality and customizability that is exposed through its IPC or command-line API can be accessed through Emacs Lisp. Deeper core functionalities can't be changed, and new functionalities must be layered on top in terms of the actual capabilities of the program itself. In essence, it works like adding more core C code to Emacs, placing an extra boundary on what it can do. The secondary drawback is that having your Emacs computing environment rely on external tools can be a recipe for fragility when you take it with you from place to place, since the programs you've made it rely on may not be as platform independent as Emacs itself is.

The benefits and drawbacks of the second approach are essentially precisely the inverse of those of the first approach: while implementing a tool completely in Emacs Lisp means that it is completely hackable from bottom to top, and much more self contained in the cross-platform runtime Emacs provides, it means that the burden for feature implementation, improvements, and maintenence falls entirely on the package authors, and the burden for documentation and user help falls entirely on the much smaller user community, and interoperability with what other people are using is sacrificed. There's also another drawback to be considered: the fact that, while Emacs Lisp is certainly not the slowest language around – it's a lot faster than Python or Ruby, which are both used to build quite large applications themselves – it's still a lot slower than you might want for very large and complex user-facing applications, especially in a painfully single-threaded environment like Emacs.

As a community, there's no need to choose between these two strategies wholesale, of course; for any given thing you want to do, packages that embody both – such as Gnus (the second strategy) versus Notmuch or Mu4e (the first strategy) – will always exist to choose from, and that's a very good thing! From a user perspective, which of these strategies to choose will depend on one's personal preferences and assessment of uses and needs on a case by case basis.

However, while I don't ever want packages using the first strategy to go away – I use Gnus without anything like offlineimap myself – I do believe that new package authors should strongly prefer the first strategy over the second. While it does compromise the core values of Emacs, the entire history of GNU Emacs beginning with its own core being written in C is one of compromise, and until several issues at the core of Emacs are resolved, that compromise must continue in the name of making sure that Emacs remains pleasant to use as well as powerful.

In this case, compromise must be acceded to because of the fundamental truth that Emacs is a very small (but dedicated) community even within the niche community of text editors, so we simply don't have the resources or manpower to maintain things and establish interoperable standards on our own. To see the truth of this, just consider CEDET, which was an effort to expand Emacs with IDE-like capabilities including project management, language-aware but agnostic source code analysis for refactoring, jumping, and completion, and a lexer and parser generator for language-agnostic structural editing: it quickly died and became abandonware, because no other communities were using or contributing to it and no languages were interested in supporting it just to help the Emacs community out, so all the work fell exclusively on the Emacs community and it failed to boostrap enough interest. Compare that to the success and revitalization Emacs has seen in recent years thanks to the adoption in core of LSP and Tree-Sitter, standards providing the same functionality that were instituted and supported by other editors such as Visual Studio Code and NeoVim, and the greater success of smaller and more focused packages for single IDE features like Treemacs and Projectile. Adopting technologies and programs used by other communities ensures that your package will have a much longer, more reliable, more robust life, which reflects on Emacs's usefulness and longevity! Furthermore, the first strategy only enhances one of the unique features of Emacs, which is that it is essentially a new and different kind of shell, not just a hermetic computing environment (the most similar thing I know of outside of Emacs is Acme, although it suffers from UNIX philosophy nonsense), so it's not a pure loss.

15. Emacs versus Acme

The similarities between Emacs and Acme ( Acme was created as the premier visual text editing environment for the Plan 9 research operating system at Bell Labs) are fascinating case in multiple discovery not unlike both Liebnez and Newton independently discovering calculus: both have the otherwise very unique property of being most commonly used as text editors, but under the hood actually being malleable, programmable, text-oriented computing environments designed not just for reading and editing text but also managing the filesystem and external processes and the interactions and composition between them, all within a unified window management and user interface paradigm.

While this fact is not part of the official marketing of Emacs, it is commonly known among the community. As Irreal, a pillar of the Emacs community, states:

It is […] a Lisp interpreter specialized for dealing with text that has an editor as one of its built-in applications. […] I prefer to think of Emacs as a light-weight Lisp machine suitable for many tasks. […] Regardless of original intentions, Emacs today certainly embodies the notion of being “an extensible environment with text editing as a feature.” But it’s even more. At least for many of us.

Meanwhile, this fact is expliclitly part of the core intenton of Acme. Just look at Rob Pike's original paper on Acme, where it says:

A hybrid of window system, shell, and editor, Acme gives text-oriented applications a clean, expressive, and consistent style of interaction. Traditional window systems support interactive client programs and offer libraries of pre-defined operations such as pop-up menus and buttons to promote a consistent user interface among the clients.

Also not unlike that simultaneous discovery, one of the inventions was clearly superior to the other and went on to have a much longer history, and the other quickly died out and is mostly only of historical interest. Consequently, a lot can be learned from their differing philosophies in how they approached this common idea. I'll briefly take a look.

First, since I don't feel like into detail explaining precisely how Acme works, because I feel like that would distract me even more from the point of this section than I already have been over the last several minutes writing it, I recommend reading this post to familiarize yourself with the basic concepts before moving on.

Now, on to the meat of my critique:

  1. Acme tries to use the mouse for as much as possible. While this is extremely interesting, and some will defend Acme's use of the mouse till their dying day, I think this is an unsuccessful experiment. The use of mouse button chording does expand the bit rate of mouseing input somewhat (by exactly three bits!) but not by enough to make a significant dent in the vast chasm between the information throughput of a keyboard and a mouse, and it doesn't begin to address any of the other problems the mouse has in comparison to the keyboard – most devistatingly of all for a text-focused application being the need to constantly move one hand back and forth between the keyboard and the mouse when switching between editing/navigation and entering content.
  2. Despite provideing a very malleable interface in the form of its tag lines, which you can put any commonly used commands you want access to in, and the fact that its text windows are live text stream devices so it's easy to embed anything that can operate over a TTY in them, and a deeply programmable environment through point (3) below, Rob Pike had very specific user interface design ideas, and decided to skip out on UI programmability and malleability in favor of enforcing them. For instance, he made the intentional choice not to support syntax highlighting or changing the color scheme. Judging by that approach I deem it unlikely Acme would have eventually come to support any of the modern display and fontification features which enrich our experience of Emacs today, either. This may seem like a minor point, but it's rather fundamental: while Emacs seeks to extend its display capabilities as much as possible while maintaining consistency, composability, and reflectability by ensuring a basis in a common set of underlying systems (text propertized with structured data), Acme seeks to maintain consistency, composability, and reflectability of its interfaces through outright hard-capping the capabilities of its interface entirely. This may be part of why Acme died: its inability to adapt to new capabilities and new knowledge (such as the fact that syntax highlighting is actually very beneficial).
  3. Instead of having a language built into it that lets you program it, and using that language to import and control external programs, as Emacs does, Acme instead exposes a simple command language and the event logs, text content, and so on of each of its windows as filesystem devices so that separate processes can read from and write to those devices in order to control it via pure text streams. This means that nothing fundamental about the editor can be modified or changed this way, so the editor must be as minimal as possible, carrying as few features as it can, in order to outsource the rest to more changeable programs. The problem with this is that:
    1. Since these are entirely separate processes communicating over raw text streams with Acme, there is no way for these programs to fundamentally modify any of Acme's core functionality or concepts. Everything must be implemented in terms of Acme's necessarily ecruciatingly limited set of concepts and user interfaces, which means that more complex functionality such as better display capabilties (as mentioned above), changes to how the window layout works such as autocompletion and documentation popups, modifications to how input is processed at a fundamental level like transient menus or hydras or text based UIs like magit, are all nearly impossible.
    2. While a feature can be made available to the user through a shell command that runs a process which can communiate with Acme, since all these processes are launched by the user completely individually, they have no way of knowing about each other, let alone meaningfully sharing information about their current internal state and provided functionality.
    3. Furthermore, if a feature is added to Acme by way of an external process, due to the previous point, other processes still won't be able to know about it or use it in any robust way that isn't extremely brittle, like asking the user to list what commands for Acme they have available manually in a text file somewhere.
    4. As a result, Acme's extensibility and thus longevity is fundamentally hamstrung: imagine trying to replace some core user interface concept in Acme with something better, like Helm or Vertico do with Emacs's built in completion system, or hook into all possible completion interfaces across the whole program to add more annotations useful to the user, as Marginalia does.
  4. Having to use the shell to program Acme exposes the user to all of the flaws of the UNIX environment, full bore, like a firehose. An Acme user will have their fill of a cryptic, underpowered, gotcha-filled shell language that requires you to awkwardly compose together programs that all format their data in completely different ways using dumb text-munging via esoteric commands with even more esoteric arguments. Ever wanted to have to run xargs or figure out the pattern language of find or write AWK scripts or figure out whatever the hell tr does in order to use basic functionality in your editor? Then you'll enjoy Acme. Then there's the added insult to injury that these commands are somewhat unergonomic to run from the editor since, unlike with a Lisp, the borders of any given expression are not clear in a shell language, so the whole shell expression must be selected before it can be run each time.

I think the differences between Acme and Emacs are the perfect case study in the differences between the UNIX philosophy and the Lisp Machine approach to computing systems. And I think it's telling that one survived the death of its host system and has continued to be used for 40 years, and the other did not and has not.

Footnotes:

1

Another example: cl-loop is much faster than dash, seq, or even the built in constructs for higher-order list manipulation like mapcar, because it is a macro that compiles down to low level dolist, dotimes, while, and push calls, all of which are bytecode opcodes.

This work by Novatorine is licensed under CC BY-SA 4.0