SVG Interfaces in Emacs?

One of the really interesting little things that I've seen on the margins of the Emacs world is people making use of the fact that SVGs are a really good medium for drawing arbitrary graphical user interfaces, since they allow drawing arbitrary shapes, colors, gradients, and text, in a way where it is clearly structured and you can assign relative constraints and sizes, and where that structure is coherently maintained instead of being lost when drawn to a pixel image or something, and the fact that Emacs actually allows you to detect arbitrary mouse events with precise pixel locations and supports operating system context menus and menu bars to construct arbitrary user interfaces embedded within Emacs. An example of this is Emacs Easy Draw:

As you can see, this looks and functions like a real interface, no text-mode funny-business or shenanigans.

Another, more minor, example, can be found here, where messages in the Emacs echo area are given full GUI-level graphics and layout capabilities.

I think key to why this seems to work so well is that it's actually really easy to build SVGs in Emacs Lisp, since SVGs can be represented more meaningfully as data structures with semantic information about what they're trying to depict (i.e. what shapes, what text, where the boundaries of them are, how they're translated and rotated), so it's easier to construct them programmatically, and since they're a symbolic tree structure very friendly to Lisp's native data structures, and Emacs Lisp's macro facility makes an SVG DSL quite easy. I think the other reason this is so profitable is that, since SVGs are a limited primitive, it's easy to pull semantic structure and meaning back out of them, when for instance the user clicks on a location within them, or when things need to be changed. If we were trying to use regular pixel buffers, all of this would be a lot more processor intensive and unwieldy.

The possibilities with this are honestly endless. Fully-featured Jupyter notebook style inline widgets and interactive graphs for org-mode, embedding more fully-featured SVG and even bitmap image editors, Possibly even using SVG as the basis for rendering full HTML webpages inside Emacs? Obviously, building on all of this will take an absolutely gigantic, humongous amount of work to actually build the things that use this the way I'm suggesting, but the point is that the potential, the capabilities, are absolutely there.

Of course, there are tradeoffs to this. SVG-based interfaces have the disadvantage relative to something like Xwidgets (which allow you to propertize text with GTK+ widgets) that they aren't interoperable with the host operating system, and we all have to build those widgets, and their interactivity, ourselves, from scratch. However, the advantage relative to Xwidgets is that it's inherently more self-contained and thus more cross-platform, relies on fewer external variables (e.g., relying on GTK and whatever other UI toolkits Xwidgets would need to use to support other platforms to even continue supporting this and not change so much that maintenance is prohibitive, as well as needing to rely on external platforms continuing to allow this), and the entire set of UI widgets, and most of the interactivity and functionality, would be writable in Emacs Lisp, meaning that maintenance and improvement of an SVG-based widget library would be open to far more Emacs users, as well as much easier and quicker to write and experiment with, and in addition also meaning that these widgets would actually be comprehensible and accessible to the rest of Emacs, instead of black boxes.

Another alternative for embedding arbitrary user interfaces within Emacs is turning Emacs into a Wayland compositor. This has many of the downsides of the previous option – interfaces added thusly being black boxes to Emacs Lisp, difficulty of contributing, external dependencies – but has the massive advantage over both SVG interfaces and Xwidgets that it allows importing other interfaces, like Jupyter itself, or Inkscape, or whatever, wholesale into Emacs without the need for extended development. With a little additional elbow grease, this approach could even insure that such imported applications could not only be embedded in Emacs text, managed like Emacs buffers, etc, but be ensured to have consistent keyboard commands. However, ultimately, they'd be even more of a black box to Emacs than Xwidgets would be, making it so that such an approach would probably lead to turning Emacs into a window manager more than actually integrating external things successfully into it. Unless the things that were integrated were written specifically for Emacs, anyway, but then you might as well use SVG. (Unless you want to do 3D rendering or something, which could actually usefully take advantage of Emacs being a Wayland compositor – maybe the ideal is a combo of EWX and SVG)?

Making a full UI toolkit via SVG in Emacs Lisp is honestly an interesting project to look into in the future. It might also be possible to use a headless browser or reasonably complete layout engine to render web page HTML to SVG which Emacs could then display, allowing a faithful 1-to-1 reproduction of HTML pages, just like Xwidgets offers, that also allows Emacs to inspect the content of those pages! Unfortunately, according to a few cursory searches, it looks like most command line HTML to SVG tools are quite dead, and since command-line processes are the easiest to control for Emacs, that does present a problem. However, I maintain this remains a feasible theoretical possibility.