Do we really need macros much?

I've kind of fallen out of love with macros to a certain degree.

Now that we don't just have snippets, but also AI, to help deal with boilerplate and refactoring, to a certain degree, yeah — but there's more to it than that.

Ultimately, it's because the vast majority — like 99% — of things you do with macros in Lisp, are just basic textual compression, basically. You get rid of boilerplate, but the resulting DSL is completely isomorphic to the structures and control flow of the underlying code.

You're not implementing fundamentally new language concepts under the hood, so it's really just, on the one hand, an attempt to avoid typing (which when you've got AI, or even good snippets and editor completions and type out 140 words per minute) is really just a non-issue; and on the other hand, you're trying to guarantee code readability by letting the code sort of work at a slightly more abstract layer with a more rich vocabulary for the domain at hand, so you don't lose the forest for the trees.

But in most cases that I've ever seen, the vocabulary that you can provide with structs, interfaces, types, methods, higher order functions, and so on, is essentially just as good and "high level" as what macros can provide, with just a little bit more boilerplate. Most usages of macros I've ever seen are just ways of avoiding defining a few extra types, or wrapping a block in a higher order function, and that's about it.

And the problem with macros is that the cost for it is that you've got to — and I know this is a common criticism, but it is kind of true — learn 36,000 different languages for each different little thing, where each might have slightly different syntax and semantics, or be a kind of counterintuitive and not obvious transform of the underlying code and control flow, while at the same time you have to be aware of that underlying transform, because macros are often very leaky abstractions (due to scoping rules, evaluation time, etc); and if you mess any of that up, it leads to metaprogramming debugging, which is the worst thing imaginable.

You technically can add completely brand new ways of doing control flow or modeling problems using macros; you can introduce pattern matching or even CSP and so on through macros. But that usually introduces so much complexity and macro juice overhead that it's frowned upon severely even in Lisp communities, and for good reason. For instance, all the attempts to implement parametric polymorphism for Lisp types or classes, or monomorphization or JIT compilation for Lisp multimethods: they technically work, but they're a difficult to maintain hack, a leaky abstraction, that just can't reach the level of power and integration of a language that had them from the start.

And this is important to reiterate: the uniquely useful thing that macros can do is so frowned upon for an average project to do, and there's generally a strong push to standardize these things so that everyone recognizes, understand, and uses these things, so that it usually ends up being language authors themselves that provide macros. Even Lispers, at this point, know that you really should be keeping macros to a minimum; preferably using none at all. You even see this with Clojure, where the big Clojure type systems and async IO implementations and CSP implementations and so on are generally provided in the standard library itself. At which point it's not clear to me why you wouldn't want to just implement it in the language in the first place, except for purity reasons.

In general, my heel turn on macros has been because I sort of realized, upon introspection, that in most languages that aren't outrageously underpowered, I've never really felt the need for them over just higher order functions and interfaces and generics. Even in a language like Clojure, with excellent and very easy to use macro facilities, I've never really used them and to the degree I have, they're from libraries where the The same thing could have been implemented relatively as easily, with only marginally more boilerplate — like one or two extra tokens of boilerplate — without them. Even Go can do a decent DSL impression! In fact, in most cases where I've written any substantial amount of Lisp, Scheme, or Clojure, whenever I've created a macro, I later realized I didn't need it, and refactored it into a regular function.

The rest of Lisp's features are similar: extremely cool in theory, but, even after getting a chance to use them, lackluster and generally not useful in practice. Multiple dispatch would be nice, to avoid the visitor pattern, but structurally-typed interfaces (like in Go) satisfy 90% of that need for me; image-based development would also be nice, but has its own pitfalls (the state of your program and the state of the file, in terms of what code is actually running, getting out of sync, which can be a horrifying headache) and in general is easily replaced with simple hot reloading (which is always in sync!) and standalone static binaries to avoid dependency hell.