Mastering Emacs https://www.masteringemacs.org/feed a blog about mastering the world's best text editor http://www.rssboard.org/rss-specification python-feedgen en-US Thu, 28 Mar 2024 09:12:17 +0000 Combobulate: Interactive Node Editing with Tree-Sitter https://www.masteringemacs.org/article/combobulate-interactive-node-editing-treesitter?utm_source=newsletter&utm_medium=rss Editing code using a concrete syntax tree may seem straightforward, but it's a complex task fraught with challenges. For every command that modifies the code, there's ample room for ambiguity. What if there are multiple legal choices available? How do you create a user experience in Emacs that not only displays the intended changes but also caters to the needs of power users seamlessly? Here I talk about how I've solved that problem in Combobulate. <p class=" text-justify drop-cap">Recently, I talked about the challenges around improving <a href="/article/combobulate-structured-movement-editing-treesitter" class=" article-link">Combobulate</a>, my package that adds structured editing and movement to Emacs, so navigation is <a href="/article/combobulate-intuitive-structured-navigation-treesitter" class=" article-link">intuitive and works the way a human expects</a>. Briefly, instead of relying on elaborate guesswork that breaks easily, there is now a little mini-language that captures how to pluck the right node from the concrete syntax tree provided by tree-sitter, a library that makes it a snap to parse and access said tree of any language for which there is a supported grammar.</p><p class=" text-justify">Intuitive navigation is hard because if you ask a human developer to navigate up, down, left or right in their code, they can do so effortlessly, as there’s little ambiguity around, say, what constitutes the next “line of code”. Good luck asking a computer to make the right choice when it’s got a baker’s dozen of wildly different nodes to contend with. Hence the mini-language.</p><p class=" text-justify">But I’ve talked about <a href="/article/combobulate-intuitive-structured-navigation-treesitter" class=" article-link">that already</a>. You should read that article first, if you haven’t. So today I’d like to talk about a related issue: when you ask Combobulate to do something – say, cloning the node at point, and in effect duplicating a piece of code – and Combobulate can’t decide which node is the right one, because there’s more than one legitimate node to clone at point.</p><p class=" text-justify">The problem space is similar to intuitive navigation, but with a twist. When we navigate we have a limited number of sensible directions we can move (up, down, left, right) and the bindings to go with it – <code>C-M-n</code> to move to the next sibling, <code>C-M-u</code> to move up to a parent, etc. – and when we invoke them we’d expect Emacs to just, you know, go to the right place. You don’t <em>really</em> want annoying minibuffer prompts, banners, billboards, smoke signals, crop circles or much of anything else to petition you and impede your train of thought. You just want to scoot away and not have Emacs stop you to ask for directions.</p><p class=" text-justify">Fair enough. Combobulate goes to a lot of trouble to try and pick the right thing for you in most cases, whether you’re editing or moving. Sometimes, though, you must ask the user to make a choice.</p><p class=" text-justify">And what’s the best way to do that?</p><H2 id="decisions-decisions">Decisions, decisions…</H2><figure> <img src="/static/uploads/combobulate/combobulate-python-indent.gif" alt="" class=" center-block"> <figcaption>The indentation 'carousel' in action. Tapping <code>TAB</code> at the start of a statement and Combobulate will calculate all possible indentation levels and let you choose the one you want interactively.</figcaption> </figure><p class=" text-justify">It’s a quagmire, really, because I’d had this issue multiple times where I really wanted some way of asking the user to make a choice. I had played around with the obvious contenders: some form of <code>completing-read</code> (Emacs’s way of doing <a href="/article/understanding-minibuffer-completion" class=" article-link">minibuffer completion</a>) and a hook to update the buffer as you browse through the candidates. It didn’t really work all that well as it assumes you have a completion system that lets you cruise through the candidates interactively. Not everyone uses that; default Emacs has you <code>TAB</code> endlessly to scratch out a completion like some sort of neolithic farm hand. So that method wouldn’t work well for people who use that type of completion, and it also has the annoying problem of feeling like it was the wrong tool for the job. What if someone uses Helm or any number of other completion systems?</p><p class=" text-justify">I brainstormed some other methods like a <a href="/article/introduction-magit-emacs-mode-git" class=" article-link">Magit</a> transient popup-like thing that you’d pick nodes from, but it also felt cumbersome and I couldn’t figure out how to get transient to do what I wanted it to do.</p><p class=" text-justify">So I did what I always do: I got bored and went off in search of something else to do.</p><p class=" text-justify">Right around that time I’d added a primitive version of “expand region”. It is a simple concept, really: given successive key presses, expand the region to incorporate larger and larger structural elements, starting from point. It’s a nifty way of picking things that ordinary Emacs methods struggle to do well at, though I never cared much for it pre-tree-sitter as I found it too imprecise. Lots of people love it, though, and I figured that it’d be super handy with tree-sitter, as it’s so granular, and I wanted to support a wide range of workflows. So I added a basic version in about 30 minutes, and off it went to Github.</p><figure> <img src="/static/uploads/combobulate/combobulate-expand-region.gif" alt="" class=" center-block"> <figcaption>Combobulate can expand the region with <code>M-h</code>. It highlights the item ahead of the current region, to give you an idea of where you are going. A customizable feature to let you select by number is also enabled here, and Combobulate will show you how much it has selected in the tree view as well.</figcaption> </figure><p class=" text-justify">A few days later I got a polite and <em>totally obvious in hindsight</em> request to make possible to <em>shrink</em> the region. It’s all too easy to overshoot your mark as you’re tapping away and, having done so, you’d have no recourse but to start from scratch. Argh. Terrible UX!</p><p class=" text-justify">So I put on my thinking cap and started wondering how I could crowbar something into <code>combobulate-mark-node-dwim</code> (bound to <code>M-h</code>) so it can toggle the direction and, I… I realized I should just re-use <em>another</em> feature I’d added some months prior: a node / region-based indentation command in Python that lets you interactively browse and select the indentation level you want your node or region to have. (Believe it or not, but Emacs’s crummy Python mode can’t cycle all possible indentations of a region. It’s madness. The crux of the code to build it is <em>right there</em>, too. No TS required at all.)</p><p class=" text-justify">The python indentation command (see above) acts like a carousel: keep moving in one or the other direction and you’ll wind up back where you started. That’s great for a wide range of things, particularly when you’re not sure how many finger taps away something is: is it two or <em>three</em> indentations I want? Tap a bit and find out, as your buffer updates automatically.</p><p class=" text-justify">The indentation command was also a bit ham fisted in some ways, but it was a huge improvement over the ghastly python-mode indentation logic you had before which involved manual tedium and <code>C-x TAB</code> to indent by column.</p><p class=" text-justify">I gambled that if I could file off the rough edges, and build some tooling around it into Combobulate, it could work well for a wide range of tasks. I sat down and spent a fair bit of time hammering out a system of rules and concepts that makes for, I think, a pleasant user experience. Carousel seems like as good a name for it as anything, so that’s what I’ll call it from now on.</p><p class=" text-justify">If you’re already using Combobulate you’ll of course be familiar with the carousel concept already as it’s been in Combobulate for quite a while.</p><H3 id="reading-a-key-vs-minibuffer-prompting">Reading a Key vs Minibuffer Prompting</H3><p class=" text-justify">Let me explain. There’s a philosophical view (and some who disagree with this) that certain user-facing actions in Emacs don’t require a full-blown <em>minibuffer prompt</em> and that <em>reading a key</em> is enough. Which one is the right choice is rarely too contentious, but in a few instances you could lean towards either method, and that’s usually where tempers flash.</p><p class=" text-justify">Reading a key is mechanically simple: you call <code>read-key</code> (or one of its close relatives) and Emacs will patiently wait for a <em>key</em>; any key or key sequence, really. There is no minibuffer history to contend with; recursive minibuffers don’t apply, so you can’t switch out of the minibuffer window, either. You can display a prompt, but the main focus is really about user anticipation: once you know how the command works, and you know it expects a key, you just type it and get on with it. It is about as zero-effort as requesting user input could ever be. It’s used in a wide range of contexts in Emacs today, and it’s perfect when all you need from a user is a key.</p><p class=" text-justify">The alternative is <em>minibuffer prompting</em> which is the all-singing, all-dancing prompt experience you know already.</p><p class=" text-justify">I want Combobulate’s carousel to read a key: the reason is that it means I can capture the key you typed and, if I decide I have no use for it, I can put the key back on the <code>unread-command-events</code>. I could in theory still do that with the minibuffer prompt, but I still have to deal with the fact that it’s a complex system designed for non-trivial user interactions.</p><p class=" text-justify">Reading a key is simpler, and that’s important because it meshes well with the idea that the carousel is there to offer a seamless transition in to, and out of it.</p><H3 id="seamless-transition">Seamless transition</H3><p class=" text-justify">If you want to expand the region with <code>M-h</code>, you probably want to follow that up with another key. Perhaps <code>C-w</code> to kill the region you just expanded. By <em>reading the key</em> I can separate <code>TAB</code> and <code>S-TAB</code> (to cycle to the next and previous choice in the carousel) from irrelevant keys the carousel does not care about. Then, by putting the key the carousel <em>does not</em> care about back into the event loop, you can have Emacs carry out what ever it was going to do as though the carousel wasn’t there at all.</p><p class=" text-justify">End result? You can hit <code>M-h</code> and tap, tap, tap and press any other key that is not recognized by the carousel, and it’ll just execute the key as though you’d never had the carousel at all. No transition; no annoying in-your-face “are you really sure?” prompting; and no thinking required. It behaves the way an experienced Emacs user would expect it to work, and as if you never had the carousel active, even though, there it is, indicating in the echo area that it’s active and awaiting your input.</p><H3 id="maintaining-your-tempo">Maintaining your tempo</H3><p class=" text-justify">I mentioned that <code>TAB</code> and <code>S-TAB</code> cycle nodes. But if you tap, say, <code>M-h</code> for the first time, the carousel interface appears, and you can then repeatedly type <code>M-h</code> again which is the same as pressing <code>TAB</code>. That way you don’t have to move your fingers away from the triggering key and that helps preserve tempo. It makes for a smoother and easier user experience as you don’t have to context shift: oh I hit <code>M-h</code> and now I have to <code>TAB</code> to expand.</p><p class=" text-justify">Because I look up the key in a boring, old <a href="/article/mastering-key-bindings-emacs" class=" article-link">keymap</a> it’s easy for anyone to come along and modify it if they want the carousel to use other keys. It also means I can add additional keys that only apply to specific commands: Combobulate’s expand region functionality lets you cycle between the next and previous candidates with <code>M-h</code> and <code>M-S-h</code>, respectively.</p><p class=" text-justify">The third reason why it’s useful to have the same key is that some operations are incredibly destructive and may leave your buffer in a “broken” state: the syntax is invalid, and tree-sitter may struggle to glue it back together. Having a cohesive view of the buffer at the beginning and operating on it from a clean slate is crucial.</p><H3 id="cohesion">Cohesion</H3><p class=" text-justify">Let’s say you’re deleting code as part of a refactoring operation you’re doing. As you go about doing it, you’re definitely going to leave your code in a broken state at some point. That’s all well and good: you’re a human, and you can fix it.</p><p class=" text-justify">Tree-sitter has error-correction built into its parser, but just because it can mend, and partially recover from, broken syntax, does not mean it leaves the resulting tree in a state where an automated tool like Combobulate can make sensible decisions.</p><p class=" text-justify">That’s particularly true of highly destructive operations like Combobulate’s <em>splicing</em> where you’re eliding text as you try to snip and glue two pieces of code back together: the code you’ve decided to keep, and the code <em>around</em> the code you’re keeping that the splice deleted. Think of HTML where you want to delete the outer tag but keep the things inside it - that’s one part of what you can do with splicing.</p><p class=" text-justify">The problem is, maybe you want to splice two times, but the first splice irrevocably breaks your code in such a way that you’d never be able to make it to the second splice. And how do you delete the “outer” something of any old random piece of code, anyway? Take a look at the figure below to see what I mean.</p><figure> <img src="/static/uploads/combobulate/combobulate-splice-go.gif" alt="" class=" center-block"> <figcaption><code>M-&lt;up&gt;</code> splices up, keeping the siblings, which in this case is one line of code, and then deletes successive parents until it's at the root of the buffer. Note that along the way it leaves the tree in a broken state.</figcaption> </figure><p class=" text-justify">Combobulate can splice nearly anything into something else. But that doesn’t mean it makes <em>syntactic sense</em> to do so. Maybe you do want it in a broken state; perhaps you want to tweak something to make it legal syntax again. Combobulate can’t read your mind so it has to calculate all possible paths.</p><p class=" text-justify">If the splice up command – as it used to do – only went up one level, you could easily break the tree in such a way that you wouldn’t be able to splice again. A broken tree often begets an even more broken tree.</p><p class=" text-justify">To work around <em>that</em>, the carousel virtualizes editing and computes everything on the fly, starting from the clean slate your buffer is (presumably!) in when you first initiate the command.</p><H3 id="virtualized-editing">Virtualized Editing</H3><figure> <img src="/static/uploads/combobulate/combobulate-clone-dwim.gif" alt="" class=" center-block"> <figcaption><code>C-c o c</code> clones a node and, if there are ambiguities, Combobulate will let you cycle through all the choices and interactively preview the change it'll make to your buffer.</figcaption> </figure><p class=" text-justify">This was one of the harder things to build.</p><p class=" text-justify">Tree-sitter generates your tree on-the-fly as you type. Every key press compels tree-sitter to rebuild all or parts of the tree. That is its main benefit, and a lot of work was put into making it fast enough for even the fastest typist to not experience any lag or latency.</p><p class=" text-justify">Unfortunately, if you ask for a node and subsequently edit the buffer, that node is invalidated and marked outdated. If you try to do anything with it – anything at all, even asking it for its type or where it was in the buffer – it throws an error in your face.</p><p class=" text-justify">So you can’t collect the node(s) you want at the start and then modify the buffer in-situ, like the indentation example above shows. The first tap would kill the old nodes (we changed the indentation!), and render that approach useless.</p><p class=" text-justify">Combobulate constructs proxy nodes, and almost every part of Combobulate will accept these proxy nodes in lieu of the real deal. They are slimmed-down versions of the real things, but they reify the most important things we’d want to cling to: the point range in the buffer; the type; the text contained therein; etc.</p><p class=" text-justify">So when you ask Combobulate to present a carousel it actually virtualizes the nodes before any sort of change can take place. It neatly skirts most issues and lets you write code that can in theory modify the buffer without worrying about your nodes expiring when you touch the buffer. Of course, modifying the buffer means you hold on to outdated information, and Combobulate is no oracle, so if you make substantial changes, the proxy nodes might be thoroughly useless.</p><p class=" text-justify">Luckily, that’s usually not a problem because of Combobulate’s refactoring display system.</p><p class=" text-justify">Part of the challenge around the proxy node thing is that most commands do small, localized editing operations: indent some code; expand an envelope; splice some code; clone a node. You get the idea.</p><p class=" text-justify">As you tap through the carousel’s list of valid nodes, it should show you what would happen if the transformation you asked for is carried out on the currently selected node.</p><p class=" text-justify">For this to work well, the carousel works in unison with Combobulate’s refactoring system. The latter sounds fancier than it really is: all it lets you do is describe simple transformations, and visual ones to aid users, to make to a buffer. Add in the carousel’s ability to use <em>undo change groups</em> to revert buffer changes between choices and you can have visible modifications made to your buffer that is properly restored as you cycle through the choices.</p><p class=" text-justify">I think it’s important that, if you ask a command to make changes to your buffer, and if there’s more than one way to make that change, that you can preview all possible options. Combine it with the carousel interface and you’re afforded a fair amount of slop when it comes to point positioning.</p><p class=" text-justify">And if you don’t like the change? You can hit <code>C-g</code>, as with most things in Emacs, to abort the command.</p><p class=" text-justify">So that’s the carousel interface. I think it makes it much easier to visualize what’s going to happen in a non-committal way, and at the same time scroll through the valid node choices available to you.</p> /article/combobulate-interactive-node-editing-treesitter Sat, 23 Mar 2024 14:04:38 +0000 Combobulate: Intuitive, Structured Navigation with Tree-Sitter https://www.masteringemacs.org/article/combobulate-intuitive-structured-navigation-treesitter?utm_source=newsletter&utm_medium=rss Tree-sitter's promise of simplifying code navigation can seem like a dream, but in reality, it introduces its own set of challenges that complicate the process. In this article, I delve into the intricate nature of crafting intuitive navigation within a tree-sitter environment, shedding light on the complexities that can confound our expectations of how code exploration should flow seamlessly. <p class=" text-justify drop-cap">Today, I want to discuss the difficulties of navigating code in a tree-sitter environment. For all the meticulous detail it adds, it also makes intuitive navigation hard. There are too many nodes going in all manner of directions and that’s, well… discombobulating.</p><p class=" text-justify">How to efficiently move around your code using basic concepts such as moving up or down a node, or to the next or previous node, in a way that aligns with our human intuition, is not straightforward.</p><figure> <img src="/static/uploads/combobulate/navigation/tsx-nav-down-and-sibling-slow.gif" alt="" class=" center-block"> <figcaption>An example of Combobulate's refined navigation system in action, with parent-child movement and sibling movement the way a human would expect it to work.</figcaption> </figure><p class=" text-justify">It really does sound simple, but I promise you it is not. <a href="/article/combobulate-structured-movement-editing-treesitter" class=" article-link">Combobulate</a> (<a href="https://github.com/mickeynp/combobulate" class=" article-link">Github</a>), my package that adds structured editing and movement to Emacs, has long had a focus on improving navigation by taking advantage of the precise view of code that <a href="/article/tree-sitter-complications-of-parsing-languages" class=" article-link">tree-sitter</a> offers.</p><p class=" text-justify">So I’d like to take a bit of time out to talk about the problems and solutions and, usefully, the benefits of solving the challenges.</p><p class=" text-justify">Emacs is already full of navigational aids that help you get to where you need to go. To say nothing of the multitude of third-party packages that add novel and interesting ways of getting around also. What tree-sitter offers – and what Combobulate benefits from – is the precision that a Concrete Syntax Tree (“CST”) gives you. If you put in the work, it’ll make navigation (and editing; but this article is about movement) more precise, and make it available in a wider range of circumstances.</p><p class=" text-justify">Lisp in Emacs has great movement and editing already, as you’d expect, but it’s also Emacs’s core programming language, and a syntactically simple language. So if you look a bit further afield, you’ll find other well-written modes like <code>nxml-mode</code>, Emacs’s XML mode, for another example of structured movement done well. It behaves as a seasoned Emacs user would reasonably expect.</p><p class=" text-justify">The reason it’s great in <code>nxml-mode</code> is that it has a standards-conforming parser that truly understands what’s going on. Plus, XML is a straightforward enough language, so it was easy for its author to construct a cohesive narrative for how movement (and, by extension, editing) should work, much as it is in Lisp.</p><p class=" text-justify">What, then, to make of Emacs’s expansive movement commands in modes that don’t have a parser, or some other method that can turn your garden-variety C, Javascript or Python code into a tree-like structure suitable for introspection?</p><p class=" text-justify">How do you bring sensible navigation to complex languages? Programmers can easily tease out structure – it’s their job, after all, to find patterns in things – in code. But to make a machine do the thinking for us is harder.</p><p class=" text-justify">Without a CST, you’re left with what ever you can claw your way to with regular expressions and gnarly imperative code. Your limitations are set by the inefficiencies of the tools at your disposal.</p><p class=" text-justify">By contrast, with a CST, you have richness, depth, and far too much choice.</p><H2 id="combobulates-goals">Combobulate’s Goals</H2><p class=" text-justify">When I set out to create Combobulate I wanted to capture all the things I liked about Paredit, a fancy structured editing tool for Lisp; Emacs’s general movement and editing commands; the breezy, structured movement you get in the likes of <code>nxml-mode</code>; and my own ideas on “Structured Editing” which, well, I’ll get to <em>that</em> in a future post!</p><p class=" text-justify">I also wanted Combobulate to follow a set of rules, design philosophies, or what ever you want to call it. They’re really simple:</p><ol><li>Enhancement. Combobulate should augment Emacs’s rich editing and movement suite and make them work in a wider range of circumstances, and have them behave in a manner that is consistent and familiar to someone with a little bit of practice.</li><li>Ease of use. Combobulate’s actually rather advanced now, but it should be simple to discover and experiment with its features. For every key that Combobulate binds to, there’s an equivalent transient UI (<code>C-c o o</code>) that shows the user what they can do.</li><li>Consistency. Language syntax aside, Combobulate should work consistently between languages, or as much as you can reasonably expect.</li><li><p class=" text-justify">Extensibility. It should, with some experience and practice, be possible to add a new language to Combobulate without having to write endless custom code. All of Combobulate’s languages are procedurally specified.</p><p class=" text-justify">There is no custom code, or magic hacks, for any of Combobulate’s languages, to make them work “in that one edge case”. There’s no “here’s a sneaky function to work around a quirk”, nor will there ever be. If it can’t be specified in Combobulate’s own little DSL, then it either won’t happen at all, or Combobulate must be extended in a pragmatic way so all languages can benefit from it.</p></li><li>Visibility. Having a full-blown CST at your fingertips can yield a mind-boggling array of choices. Combobulate will in most cases ask what you want to do when the context is ambiguous or if you deserve a closer look before it does something. Most commands that edit your code in some way will first preview the change in your buffer, live, so you can decide on the outcome.</li></ol><p class=" text-justify">These rules are important to me, as they help ensure that Combobulate behaves the same across all languages. The bit about not allowing an endless sea of language-specific elisp hatchet jobs is perhaps the most important. That does mean that really advanced features are much slower to appear in Combobulate than they would be if you were to, say, write the best ever structured movement and editing tool for <em>one</em> particular language.</p><p class=" text-justify">Having said that, Combobulate does presently support a decent amount of different languages and formats: JSON, YAML, Python, Javascript+JSX, Typescript+TSX, CSS, and HTML. Adding new ones is not <em>that</em> much of a challenge; in fact, with the recent changes I’ve made and will discuss now, it’s easier than ever.</p><H2 id="getting-around-town">Getting Around Town</H2><p class=" text-justify">The thing at the top of my list to get right in Combobulate was consistent navigation across the board. That has turned out to be really hard to achieve. I’ve spent man-months tweaking, testing, writing and rewriting code to try and do that, before I finally settled on the system that is in use now. What started a year ago as a mini-DSL for one corner of Combobulate is now used everywhere.</p><p class=" text-justify">You can’t just, like, move up, down, left or right in the tree and expect that to yield a useful movement experience. The CST is very granular, which is really good, but also kind of bad. An excess of choice can be worse than the absence of choice. Especially as it pertains to navigation. There are lots of packages out there that crudely expose the rudiments of tree-based navigation – all of it built in to tree-sitter – like go to parent; go to child; to go next or previous sibling. Call it up, down, left or right for a moment. None of this is hard to build and you could slap a primitive movement system together in no time at all, with barely a dozen lines of elisp.</p><p class=" text-justify">But you don’t have just <em>one</em> choice, which would make tree movement fluid and simple, and this a rather boring article.</p><p class=" text-justify">If you could hurl <em>point</em>, like a lead lawn dart, with great abandon, at any place in a tree-sitter-powered buffer, you’d hit a spot that would undoubtedly intersect five, ten, or fifteen nodes easily. Bullseye?</p><p class=" text-justify">Which of those nodes do you choose for a ramshackle, scotch taped movement system?</p><p class=" text-justify">You could opt to let the user choose the node they want to use. That would let someone traverse all different possible directions available around point. It’ll work, but it’s complex and fidgety; not fluid and simple.</p><p class=" text-justify">Plus, I guarantee you that you’d wind up stuck in the unending number of cul-de-sacs and dead ends that these trees have. Even a simple snippet of code can have hundreds of nodes with a mind-boggling number of leaf nodes. Most concrete syntax trees resemble a shrub more than an actual tree.</p><p class=" text-justify">Anyway. I built such a movement tool back in the day: moving around like that is comically awkward and beyond useless in all but the most basic of grammars. Few humans can inuit where they are and much less where they need to go. Here is <a href="https://tree-sitter.github.io/tree-sitter/playground" class=" article-link">the tree-sitter playground</a>. Heavy a small function into it and tell me if you think that tree is easy for a human to navigate without an <em>actual</em> map next to you. Nuh-uh. No, thank you.</p><p class=" text-justify">Which then leads me to the underlying point behind it all. One of the hardest things to build is to come up with a way of picking the nodes you, the human, want to move to given the location of point and your intent. Building the code that can determine the correct <em>sibling</em> node to move to is surely one of the hardest things I’ve built in Combobulate. If you’d told me that when I set out to write Combobulate – and I say that as someone who has long had an interest in language parsing and how to build tooling around it – I would’ve laughed out loud.</p><p class=" text-justify">That’s really what I want to talk about: why it’s hard; how I came up with what I think is a good solution; and how users of Combobulate get to benefit from this cohesive, new system.</p><p class=" text-justify">For all the munificence of a concrete syntax tree, it does complicate matters greatly due to the sheer resolution of the tree. We’ve all wanted this sort of flexibility and now that it’s here, it’s just a mountain of work. Be careful what you wish for, eh?</p><H2 id="step-by-step">Step by Step</H2><p class=" text-justify">But let’s get back to general navigation and how a naive Mickey would go about building it from scratch. Start with sibling navigation.</p><pre class="sourceCode python"><code class="sourceCode python"><span class="kw">def</span> foo(arg_a, arg_b): -!-<span class="kw">if</span> a + b &gt; <span class="dv">0</span>: do_something(a) a, b = do_something_else( a, b, ) <span class="kw">return</span> a + b</code></pre><p class=" text-justify">Let’s say point, <code>-!-</code>, is at the beginning of <code>if</code>. Sibling navigation – in Combobulate it’s bound to <code>C-M-n</code> and <code>C-M-p</code> – should take you between each of the three statements seamlessly. You cannot go to the previous sibling from <code>if</code>, as you’re already at the beginning. If you try to go to the next sibling, your point is placed at the beginning of <code>a, b = do_something_else( ... )</code>; again, and it’s placed at the start of <code>return</code>.</p><p class=" text-justify">Likewise with function parameters for <code>foo</code>. With point at <code>arg_a</code>, it should take you to <code>arg_b</code> when you type <code>C-M-n</code>, and back again with <code>C-M-p</code>.</p><p class=" text-justify">It’s a simple ask. Ask any programmer – even someone who has never laid eyes on Python – where point should move to if you want to go to the next “line of code” and they’d intuitively point out the logical place to go. Sure, you can end up with ambiguities where it’s not so clear, but, I think, in a lot of cases, it <em>is</em>.</p><p class=" text-justify">The trouble is actually programming it.</p><p class=" text-justify">Combobulate started its life with a naive view of how to do these things, and it was only when Emacs 29 and official tree-sitter support came out that I put in a lot of time to making it more robust. Before that Combobulate – and, well, it still is to this day – served more as vehicle for my hobby research into building generalized structured editing and movement.</p><p class=" text-justify">To make sibling navigation work, and to make it more robust – Combobulate is still not perfect, I admit – you must throw away any notion of a “heuristic” that will magically solve this problem. I tried; trust me, I tried.</p><p class=" text-justify">The simplest and most obvious question (and answer) is this: surely tree-sitter can tell you its siblings? Indeed it can! <code>treesit-node-&lt;next/prev&gt;-sibling</code> do just that. They take a node and return its immediate sibling, if there is one.</p><p class=" text-justify">All done, right? Not so. Consider that you may have more than one node to get its siblings <em>from</em>.</p><p class=" text-justify">Take a look at this line of code:</p><pre class="sourceCode python"><code class="sourceCode python"><span class="co"># ... rest of code ...</span> -!-a, b = do_something_else(a, b) <span class="co"># ... rest of code ...</span></code></pre><p class=" text-justify">At point <code>-!-</code> there are four nodes that <em>start</em> at the place point is at. Crack open <a href="/article/evaluating-elisp-emacs" class=" article-link">IELM, Emacs’s builtin repl</a> and check it yourself:</p><pre class="sourceCode commonlisp"><code class="sourceCode commonlisp">ELISP&gt; (combobulate-all-nodes-at-point) (#&lt;treesit-node identifier in <span class="dv">63-64</span>&gt; #&lt;treesit-node pattern_list in <span class="dv">63-67</span>&gt; #&lt;treesit-node assignment in <span class="dv">63-114</span>&gt; #&lt;treesit-node expression_statement in <span class="dv">63-114</span>&gt;)</code></pre><p class=" text-justify">There are four nodes that start at point. The numeric range is the extent they occupy in the buffer. <code>identifier</code> at <code>63-64</code> is a one character long <code>identifier</code> with the node text <code>a</code>. Makes sense when you consider the position of point and the line of code ahead of it.</p><p class=" text-justify">The other three are more specialized node types: <code>pattern_list</code> is <code>a, b</code> indicating a basic form of destructured binding in Python when it’s on the left-hand side of an <code>assignment</code>; the <code>assignment</code> is obvious enough in the context of the previous node; and <code>expression_statement</code> is the whole shebang.</p><p class=" text-justify">So the most obvious and naive attempt is to cycle through the nodes and picking the first node that has a sibling.</p><p class=" text-justify">Unfortunately that’s also the first thing you’re likely to try. But will it work? Probably, I guess. Sometimes.</p><p class=" text-justify">But consider a brace of nodes, each with one or more siblings.</p><p class=" text-justify">Let’s figure out how many siblings our clutch of nodes have:</p><pre class="sourceCode commonlisp"><code class="sourceCode commonlisp">ELISP&gt; (<span class="kw">mapc</span> (<span class="kw">lambda</span> (n) (<span class="kw">princ</span> (<span class="kw">format</span> <span class="st">"Siblings of %s:</span>\n\t<span class="st">Next: %s</span>\n\t<span class="st">Prev: %s</span>\n<span class="st">"</span> n (combobulate-node-next-sibling n) (combobulate-node-prev-sibling n)))) (combobulate-all-nodes-at-point)) Siblings of #&lt;treesit-node identifier in <span class="dv">63-64</span>&gt;: Next: #&lt;treesit-node identifier in <span class="dv">66-67</span>&gt; Prev: <span class="kw">nil</span> Siblings of #&lt;treesit-node pattern_list in <span class="dv">63-67</span>&gt;: Next: #&lt;treesit-node call in <span class="dv">70-114</span>&gt; Prev: <span class="kw">nil</span> Siblings of #&lt;treesit-node assignment in <span class="dv">63-114</span>&gt;: Next: <span class="kw">nil</span> Prev: <span class="kw">nil</span> Siblings of #&lt;treesit-node expression_statement in <span class="dv">63-114</span>&gt;: Next: #&lt;treesit-node return_statement in <span class="dv">119-131</span>&gt; Prev: #&lt;treesit-node if_statement in <span class="dv">21-58</span>&gt;</code></pre><p class=" text-justify">Ambiguities abound. So our original heuristic needs tweaking. That one, pedestrian line of code I showed you before has <em>four</em> nodes that start at point alone. And three out of four them have siblings!</p><p class=" text-justify">The smallest node, <code>identifier</code>, has a sibling of its own: the <code>identifier</code> with the node text <code>b</code>.</p><p class=" text-justify">So if you want sibling navigation to go <em>into</em> the pattern list, you’d have made the right choice if you picked the first node starting from the smallest.</p><p class=" text-justify">In this case, I’d argue that is not what a programmer would expect.</p><p class=" text-justify">They’d be walking along, merrily, only to get pulled into a pattern list with no real recourse to get back out again: navigating to the previous or next sibling according to our rules would limit us to <em>just</em> the <code>a</code> and <code>b</code> identifiers. Why? Because we always pick the smallest node, so at <code>a</code> we’d only ever pick its sibling <code>b</code>; and at <code>b</code>, its only sibling is <code>a</code>. We’re trapped.</p><p class=" text-justify">The only way out is to navigate out via several parent nodes (three, in actual fact) to get back to the expression statement we were cruising along earlier.</p><p class=" text-justify">One tantalizing observation is that, if you look at the sibling nodes available to us, the largest node – the expression statement – point to the other two nodes (<code>if_statement</code> and <code>return_statement</code>) we would logically consider to be siblings.</p><p class=" text-justify">So why not just use the largest node? Let’s try it. <code>combobulate-all-nodes-at-point</code> returns all nodes that start at point ordered from smallest (nearest) to largest.</p><pre class="sourceCode commonlisp"><code class="sourceCode commonlisp">ELISP&gt; (thread-first (combobulate-all-nodes-at-point) (<span class="kw">last</span>) (<span class="kw">car</span>)) #&lt;treesit-node expression_statement in <span class="dv">63-114</span>&gt;</code></pre><p class=" text-justify"><em>(Here I’m using <code>thread-first</code> to cut down on repetition, but it’s not required at all. The output of <code>combobulate-all-nodes-at-point</code> in <code>thread-first</code> is passed to the subsequent form, <code>last</code>, and its output to the one after that, and so on.)</em></p><p class=" text-justify">Okay. Great. Given all the nodes at point; get the last cons cell; then get its first element. That happens to be the expression statement, which is the one we want.</p><p class=" text-justify">Let’s get its previous sibling:</p><pre class="sourceCode commonlisp"><code class="sourceCode commonlisp">ELISP&gt; (thread-first (combobulate-all-nodes-at-point) (<span class="kw">last</span>) (<span class="kw">car</span>) (combobulate-node-prev-sibling)) #&lt;treesit-node if_statement in <span class="dv">21-58</span>&gt;</code></pre><p class=" text-justify">Okay, so our heuristic pans out. Given the largest node at point, its siblings yield the right nodes we want to allow us to move “between lines of code” as most human developers would interpret that phrase.</p><p class=" text-justify">Pop over to the buffer and move point (IELM does not like moving point programmatically, which is really very annoying) to the beginning of the <code>if_statement</code>. On my computer that’s point position <code>21</code> (try it with <code>M-g c</code>.)</p><p class=" text-justify">We’ve tested it works one way. Now let’s head back. From the <code>if_statement</code> let’s move to its next sibling, or the one we just came from, by first applying our heuristic to find ourselves:</p><pre class="sourceCode commonlisp"><code class="sourceCode commonlisp">ELISP&gt; (thread-first (combobulate-all-nodes-at-point) (<span class="kw">last</span>) (<span class="kw">car</span>)) #&lt;treesit-node <span class="kw">block</span> in <span class="dv">21-131</span>&gt;</code></pre><p class=" text-justify">Hmm. Hold on a moment. That’s not our if statement! We need the if statement to pick its sibling. Indeed, <code>block</code> does not have siblings at all:</p><pre class="sourceCode commonlisp"><code class="sourceCode commonlisp">ELISP&gt; (combobulate-node-next-sibling <span class="kw">*</span>) <span class="kw">nil</span></code></pre><p class=" text-justify">So. What went wrong? Most programming languages collect groups of nodes like the ones we think of as “lines of code” into <em>block nodes</em>. Here’s it’s literally called <code>block</code>. In Javascript it’s <code>statement_block</code> or some other name like that. Collections of nodes are held in a ‘meta-node’ that in <em>this</em> case encompasses all the nodes in the <code>def</code> block.</p><p class=" text-justify">You can test it out by asking for the children of <code>block</code>:</p><pre class="sourceCode commonlisp"><code class="sourceCode commonlisp">ELISP&gt; (combobulate-node-children <span class="kw">**</span>) (#&lt;treesit-node if_statement in <span class="dv">21-58</span>&gt; #&lt;treesit-node expression_statement in <span class="dv">63-114</span>&gt; #&lt;treesit-node return_statement in <span class="dv">119-131</span>&gt;)</code></pre><p class=" text-justify">So, the heuristic of using the largest node works just fine… right up until you decide to use it on the first line of code in a block. The <code>block</code> node starts at the exact same point as our putative <code>if_statement</code> node, only it’s much larger:</p><pre class="sourceCode commonlisp"><code class="sourceCode commonlisp">ELISP&gt; (combobulate-all-nodes-at-point) (#&lt;treesit-node if_statement in <span class="dv">20-57</span>&gt; #&lt;treesit-node <span class="kw">block</span> in <span class="dv">20-132</span>&gt;)</code></pre><p class=" text-justify">So our heuristic to use the largest won’t work either. Now you’d have to extend the heuristic to handle <em>exceptions</em>, and special rules, and sneaky hacks, and…, oh my, it’s all getting a bit complicated. And before long you’ll run into other awkward scenarios like this piece of HTML (or JSX, as it’s the same node configuration):</p><pre class="sourceCode html"><code class="sourceCode html"><span class="kw">&lt;foo&gt;</span> -!-<span class="kw">&lt;bar/&gt;</span> <span class="kw">&lt;baz&gt;</span> foo <span class="kw">&lt;/baz&gt;</span> <span class="kw">&lt;/foo&gt;</span></code></pre><p class=" text-justify">Where the previous sibling to <code>&lt;bar/&gt;</code> is… <code>&lt;foo&gt;</code>! Because in HTML the tree structure is <code>(element (start_tag) ... (end_tag))</code>. Which in our particular case would look a bit like this pseudo-tree:</p><pre class="sourceCode commonlisp"><code class="sourceCode commonlisp">(foo_element (start_foo_tag) -!-(bar_self_closing_element ...) (baz_element ...) (end_foo_tag))</code></pre><p class=" text-justify">HTML tags have an <code>element</code> that encompasses the entire cluster, which is sensible, but it also needs two sub-nodes to accurately represent the beginning and end of an element: one at the start, and another at the end. They have to go somewhere, and the only logical and correct place is to be children of <code>element</code>: being the first and last, respectively. And now you’ve got unwanted ‘step’ siblings. Now you’re going to have to deal with this in your sibling navigation code, and that’s yet more exceptions to build into your heuristic.</p><p class=" text-justify">So, yeah, it’s complex. Heuristics don’t work. And if they <em>do</em>, they’re liable to break if the language grammar changes in your favorite tree-sitter plugin (that does happen.) So then you get it working again – yay – and add support for a new language, and that language snaps your carefully-made assumptions in two.</p><p class=" text-justify">You can’t rely on heuristics. You can’t be a lazy developer and think you can code your way out of this with The One Algorithm like I thought you could. Not if you don’t want your heuristic to fail in strange and wonderful ways, like not working on the first statement in block.</p><p class=" text-justify">It took me a lot of rewriting and retries before I declared the approach a total failure. It took me even longer to zoom in on a system that lets me solve the “sibling problem” but also the other, equally awkward, problem of picking the right child node to choose for when you want to go “into” a line of code.</p><p class=" text-justify">Moving between siblings is great, but picking the right child node to move into is just as difficult!</p><H2 id="up-and-down">Up and Down</H2><p class=" text-justify">This problem has plagued Combobulate for a while, and until very recently, it did not use the specification language I invented for sibling navigation in other parts of Combobulate. That resulted in me having to write some rather complex parent-child navigation code that, actually, you know what, worked quite well in most of instances.</p><p class=" text-justify">Except it broke in one of the most awkward places: moving down into HTML/JSX elements. When Combobulate encountered a tag it’d go into its attributes with <code>C-M-d</code> whereas it should, rather more naturally, take you into the element to its first child (be it text or another tag). Now it does the right thing because I can distinguish between being at a node or just inside it.</p><figure> <img src="/static/uploads/combobulate/navigation/down-into-jsx.gif" alt="" class=" center-block"> <figcaption>Regular child navigation where point moves into the JSX elements when at point.</figcaption> </figure><p class=" text-justify">Before I made this change the child navigation code would go into the attribute instead. Useful, generally, but not from where point was. If it did it <em>inside</em> the div element itself it’d make perfect sense, but not right outside it. Thankfully, with the new procedure system for parent-child navigation this issue is now resolved.</p><figure> <img src="/static/uploads/combobulate/navigation/down-into-jsx-attributes.gif" alt="" class=" center-block"> <figcaption>Notice how I move forward one character at the div element so I can move down into the attributes.</figcaption> </figure><p class=" text-justify">To make this work I use a (relatively) simple DSL that looks a bit like this:</p><pre class="sourceCode commonlisp"><code class="sourceCode commonlisp">(:activation-nodes ((:nodes (<span class="st">"jsx_fragment"</span> <span class="st">"jsx_element"</span> <span class="st">"jsx_expression"</span>) <span class="co">;; use `at' because it ensures that point is at the jsx</span> <span class="co">;; element we wish to enter.</span> :position at)) :selector (:choose node :match-children <span class="kw">t</span>))</code></pre><p class=" text-justify">The activation nodes – you can have many of them, each with their own position rule and nodes – instructs Combobulate to look for the node types you see above. Here the point must be ‘at’ the node start of a fragment, element or expression in JSX. This rule will only trigger if that condition is met.</p><p class=" text-justify">All of the fields that take node types do so using a simple rule expansion system that itself has a little mini language captured in <code>combobulate-procedure-expand-rules</code>.</p><p class=" text-justify">Node type production rules are generated by the tree-sitter grammar compiler automatically. Combobulate then has a script to build a <code>combobulate-rules.el</code> file containing all of them in an easier-to-parse format.</p><figure> <img src="/static/uploads/combobulate/combobulate-query-builder.gif" alt="" class=" center-block"> <figcaption>Combobulate's query builder. It has syntax highlighting, code completion, and a handful of utility functions to help you write queries. It uses the grammar and node exports from tree-sitter.</figcaption> </figure><p class=" text-justify">That’s how Combobulate’s interactive <a href="/article/combobulate-editing-searching-new-query-builder" class=" article-link">query builder</a> works. It uses the rules file to provide code completion for node types and fields, and it knows which node types can go inside other nodes.</p><p class=" text-justify">Going back to Python for a moment, we can ask it to list all the node types that belong to the production rule <code>if_statement</code>, but only the ones that are valid inside the <em>condition</em> field of an <code>if_statement</code>:</p><pre class="sourceCode commonlisp"><code class="sourceCode commonlisp">ELISP&gt; (combobulate-procedure-expand-rules '((rule <span class="st">"if_statement"</span> :condition))) (<span class="st">"as_pattern"</span> <span class="st">"comparison_operator"</span> <span class="st">"false"</span> <span class="st">"generator_expression"</span> <span class="st">"list_comprehension"</span> <span class="st">"float"</span> <span class="st">"unary_operator"</span> <span class="st">"await"</span> <span class="st">"set"</span> <span class="st">"dictionary_comprehension"</span> <span class="st">"attribute"</span> <span class="st">"integer"</span> <span class="st">"binary_operator"</span> <span class="st">"parenthesized_expression"</span> <span class="st">"tuple"</span> <span class="st">"ellipsis"</span> <span class="st">"concatenated_string"</span> <span class="st">"true"</span> <span class="st">"none"</span> <span class="st">"string"</span> <span class="st">"set_comprehension"</span> <span class="st">"identifier"</span> <span class="st">"list"</span> <span class="st">"call"</span> <span class="st">"dictionary"</span> <span class="st">"list_splat"</span> <span class="st">"subscript"</span> <span class="st">"conditional_expression"</span> <span class="st">"boolean_operator"</span> <span class="st">"lambda"</span> <span class="st">"named_expression"</span> <span class="st">"not_operator"</span>)</code></pre><p class=" text-justify">You can then further refine your scope with exclusions, regexp matching, and more.</p><p class=" text-justify">Combobulate’s had support for interrogating a language’s production rules for a long time, but they were cumbersome to use with the old procedure system, so now it’s built in.</p><p class=" text-justify">The <code>:selector</code>, if there is one, tells Combobulate what to do with the matching activation node. In the example above it’ll choose the activated node and from that node it’ll get its children. But if you had a <code>:has-parent</code> or <code>:has-ancestor</code> requirement, you could optionally ask the selector use one of those to match from instead of the activation node.</p><p class=" text-justify">The selector can optionally match children, siblings, or use a tree-sitter (or Combobulate) query to winnow down the matches you want. Adding more selectors or activation criteria is easy, if there is ever a requirement.</p><p class=" text-justify">The advantage over a heuristic is manifold, but of course you have to teach Combobulate how to move around. I’m experimenting with generic fallback rules so <em>something</em> always happens if you’re trying to navigate in places Combobulate lacks rules, but I’m not going to release that code unless I feel it adds more value than it takes away by being unintuitive. Combobulate should be predictable and dependable, first and foremost. Luckily, most rules can be generalized across a swathe of node types, hence the need for picking node types by production rule instead of having to list them all out by hand.</p><H3 id="not-quite-sibling-navigation">Not Quite Sibling Navigation</H3><figure> <img src="/static/uploads/combobulate/navigation/json-sibling-nav-value.gif" alt="" class=" center-block"> <figcaption>Sibling navigation on the value side of a JSON object, and then on the key side.</figcaption> </figure><p class=" text-justify">For instance, this is Combobulate’s sibling navigation in action (so you’d move between them with <code>C-M-n</code> and <code>C-M-p</code>), but it’s moving along the <em>values</em> in JSON as opposed to the keys.</p><p class=" text-justify">Doing this with a heuristic is perhaps not <em>impossible</em> but much harder. None of our rules we tried above would work here. To start with, they are not siblings: more like kissing cousins. The structure of an object in JSON looks a bit like this:</p><pre><code>(object (pair key: KEY-NODE value: VALUE-NODE) (pair key: KEY-NODE value: VALUE-NODE) ...)</code></pre><p class=" text-justify">So to move along the value as though they were siblings, Combobulate has to ensure point is at a value node, inside a pair, and then find the sibling of that pair, and go back into <em>it</em> to get <em>its</em> value field node.</p><p class=" text-justify">The rule looks like this.</p><pre class="sourceCode commonlisp"><code class="sourceCode commonlisp">(:activation-nodes ((:nodes ((rule <span class="st">"pair"</span>)) :has-fields <span class="st">"value"</span> :has-ancestor ((irule <span class="st">"pair"</span>)))) :selector (:choose parent :match-query (:query (object (pair (_) (_) @match)<span class="kw">+</span>) :engine combobulate)))</code></pre><p class=" text-justify">You can still walk along the keys in the object, too, of course, which is why the idea of a wondrous heuristic that can do it all is doomed to begin with. You’d need many of them, and you’d have to pick when to use the right one! And then before you know it, you’ll have arrived at where I am now with the procedure system.</p><p class=" text-justify">The procedure system’s been in use for quite a while in Combobulate, but now it’s rewritten and so is most of Combobulate’s navigation system.</p><p class=" text-justify">It has greatly improved not only sibling navigation, but also parent-child navigation and made it much easier to specify things. It’s still not perfect; there are blind spots where I haven’t cottoned on to sibling navigation being useful, or errors where I’m too sloppy with the rules. But those things are fixable! Easy-peasy.</p><p class=" text-justify">As an added bonus, the system that finds and matches procedures is now used to improve the splicing functionality (like <code>M-&lt;up&gt;</code> in Paredit), one of <a href="https://github.com/mickeynp/combobulate" class=" article-link">Combobulate</a>’s “experimental” features that only worked in a few isolated instances. Now it’s ridiculously capable, but that’s for another blog post.</p><p class=" text-justify">Let me know what you think of the new improved movement system.</p> /article/combobulate-intuitive-structured-navigation-treesitter Thu, 29 Feb 2024 17:47:35 +0000 Let's Write a Tree-Sitter Major Mode https://www.masteringemacs.org/article/lets-write-a-treesitter-major-mode?utm_source=newsletter&utm_medium=rss Creating a standard programming major mode presents significant challenges, with the intricate tasks of establishing proper indentation and font highlighting being among the two hardest things to get right. It's painstaking work, and it'll quickly descend into a brawl between the font lock engine and your desire for correctness. Tree-sitter makes writing many major modes a snap: here I demonstrate how to write a working indentation and font lock engine for HTML. <p class=" text-justify drop-cap">Ah, major modes. There are hundreds of them, and most work in the background, rarely surfacing to tell you what they do or why. They power Emacs, and they grant purpose and verisimilitude <a href="/article/why-emacs-has-buffers" class=" article-link">to buffers, one of the most important concepts in Emacs</a>. Without major modes – or indeed the buffers that depend on them – you’d be stuck with a rather dumb and basic editor.</p><p class=" text-justify">Major modes – particularly programming major modes – are often complex because they have to deal with the raw complexity of interfacing with the syntax of a language: font locking, so the text is highlighted; indentation, so you can format your code; navigation, to help you go where you need to; and editing, so you can avail yourself of Emacs’s complex editing facilities. Major modes are often layered: one major mode deriving some of its complexity from another major mode; others, still, depend on a litany of complex interactions with other parts of Emacs to function. To unravel one complex major mode, you’ll spend weeks pulling at strings woven into the cloth that Emacs is cut from.</p><p class=" text-justify">It’s a clever system that holds up well for many of its users: the presentable, user-visible edifice is there for all to behold and is generally manageable, once you know your way around Emacs. Peer beneath the surface – perhaps you want to tweak the font locking or indentation – and you might encounter a fiendishly intricate tangle of regular expressions and elisp code that binds the indentation and font lock system together — if you’re lucky you’ll find a neat braid, and if you’re not… a Gordian knot.</p><p class=" text-justify">That’s hardly the fault of Emacs: it’s just… really complex to get right. The indentation engine for C-like languages (<code>cc-engine.el</code>) is half a meg of dense elisp code: its module description is “core syntax guessing engine for CC mode”, which is a delightfully gnomic summary. It contains 40 years of continued, incremental development and hard-won lessons. It’s a testament to the complexity of some programming languages – <em>cough</em> C and C++ – that it’s even required, and still receiving updates.</p><p class=" text-justify">Tree-sitter is a standard bearer for a new way of doing things, and the topic of this post. I’ve been writing a lot about tree-sitter, and <a href="/article/tree-sitter-complications-of-parsing-languages" class=" article-link">the complications of parsing languages</a>. One of tree-sitter’s benefits is that it eschews the need for error-prone regexp by scanning your code and yielding a detailed <em>concrete syntax tree</em>, much like a real compiler would do.</p><blockquote><p class=" text-justify">Side note: as ambrosial as tree-sitter may be, even it can’t half nelson C into complete submission; achieving tree-like perfection will have to wait if you write tangled preprocessor statements, or regularly write code to the standards required by <a href="https://www.ioccc.org/" class=" article-link">the IOCCC</a>.</p></blockquote><p class=" text-justify">But for all other languages, it’s generally very good — and much, <em>much</em> simpler to work with. As I’ll demonstrate by writing a major mode for HTML with indentation, syntax highlighting, and Imenu support. Curiously, Emacs 29 doesn’t ship with tree-sitter-enabled HTML support, so this should serve as a fine template for all the other languages that build on top of HTML.</p><figure> <img src="/static/uploads/html-ts-mode.svg" alt="" class=" center-block"> <figcaption>HTML major mode powered by tree-sitter</figcaption> </figure><p class=" text-justify">I’m hoping this will demonstrate just how approachable it is to write or amend a TS-enabled major mode. If your favorite language has a TS grammar but lacks an Emacs major mode, you could probably write something that’ll get 80% of the job done in a day or two, even if you’re reasonably new to elisp.</p><p class=" text-justify">Anyway. Let’s get cracking.</p><H2 id="prerequisites">Prerequisites</H2><p class=" text-justify">You’ll need Emacs 29 compiled with tree-sitter, and you’ll need to know how to install TS grammars. My article <a href="/article/how-to-get-started-tree-sitter" class=" article-link">on getting started with tree-sitter</a> will show you how to do all of this. Start with that.</p><p class=" text-justify">Next, you’ll need to install the HTML language grammar. You can find it on <a href="https://github.com/tree-sitter/tree-sitter-html" class=" article-link">on Github</a>. Note that, for reasons that strain credulity, TS grammars are unversioned. There is no way, aside from a commit hash, to tell which version is which. So “use the latest source, Luke” is, unfortunately, the guiding principle here.</p><p class=" text-justify"><strong>NOTE:</strong> They’ve finally realized you can number things and have slowly started doing so. But many are still not tagged and few third-party grammars are either. This document assumes you are using the grammar with the tag <em>v0.20.1</em>. <a href="https://github.com/tree-sitter/tree-sitter-html/releases/tag/v0.20.1" class=" article-link">Direct link to it here</a>. Earlier versions will not work; later ones may not either!</p><p class=" text-justify">Let’s talk about nice-to-haves. You don’t <em>have</em> to do this, but you might find it useful, particularly if you’re following along with another grammar and you are merely using this for reference.</p><p class=" text-justify">Combobulate, my structured and editing package, has code completion, syntax highlighting, and more for tree-sitter’s query language. Query building is essential: you’ll need it for indentation and font locking. Here’s an article where I talk about the feature in more detail: <a href="/article/combobulate-editing-searching-new-query-builder" class=" article-link">Combobulate: Editing and Searching with the new Query Builder</a>.</p><p class=" text-justify">Some knowledge of Emacs’s tree-sitter implementation is helpful, if not required, if you want to really delve into it: <code>(info "(elisp) Parsing Program Source")</code>. The <a href="https://tree-sitter.github.io/tree-sitter/" class=" article-link">official manual</a> is worth reading also. Make sure you read the chapter on query syntax as I will only cover what we need for HTML, which is a simple language with few moving parts.</p><p class=" text-justify">You’ll want to use <code>M-x treesit-explore-mode</code> to visualize the tree hierarchy in a buffer. You can only do this once you’ve created the parser, which is one of the first things we configure in the new major mode.</p><p class=" text-justify">You can find the complete <a href="https://github.com/mickeynp/html-ts-mode" class=" article-link">source on my Github</a>.</p><H2 id="how-tree-sitter-interacts-with-buffers">How Tree-Sitter Interacts with Buffers</H2><p class=" text-justify">It’s important to recognize that a tree-sitter-powered “major mode” need not be a major mode. The new major modes in Emacs 29 configure a bunch of stuff like font locking and so on, but you don’t <em>have</em> to rewrite everything. You can create a tree-sitter parser against any buffer, and keep it at arms length from your existing major modes. Combobulate, for instance, would work reasonably well in non-TS-powered modes (though it does not officially support this presently.)</p><p class=" text-justify">So, you can have your cake and eat it. It also means you don’t <em>necessarily</em> have to use TS for everything: you can often pick and choose. But keep in mind that some major modes use indentation to help with font locking, or font locking to help with indentation. Replace one, and the other may not work!</p><p class=" text-justify">If you just want to experiment with structured movement and editing, then know that you don’t <em>have</em> to create a new major mode. You can build stuff on top of an existing major mode that you like.</p><H2 id="defining-the-major-mode">Defining the Major Mode</H2><p class=" text-justify">One of the main things to remember is that we should leverage what’s already there. There’s already an HTML mode in Emacs, naturally. To find it, use <code>M-x find-function</code> to go to where <code>html-mode</code> is. We can see from its form that it inherits from <code>sgml-mode</code>: <code>(define-derived-mode html-mode sgml-mode ...)</code>. <code>sgml-mode</code>, in turn, derives from <code>text-mode</code>.</p><p class=" text-justify">Now, we <em>could</em> inherit from <code>html-mode</code> and try to leverage what that major mode does already, but given that we want to effectively undo all the work that mode does, I think it makes more sense to just use <code>sgml-mode</code>. SGML does do some things that seem a bit off when you first look at its <code>define-derived-mode</code> declaration.</p><H3 id="a-quick-overview-of-sgml-mode">A quick overview of SGML-mode</H3><p class=" text-justify"><em>You’ll have to crack open <code>sgml-mode.el</code> to follow along, as it’s too much stuff to include here.</em></p><ol type="1"><li><p class=" text-justify">It’s got a bunch of stuff referencing <code>tildify</code>. What’s that? Well, given the insignificance of whitespace in SGML-alike languages, you need a way of representing “hard” spaces: <code>&amp;nbsp;</code> is one such method. So, there’s a bunch of complex code to do all of that in <code>tildify</code>. It doesn’t really interfere with what we want to do, and besides: we’re not interested in converting <code>tildify</code> to tree-sitter either.</p><p class=" text-justify">Also, even if you’re a heavy user of Emacs and HTML, I’m 99% sure that you’ve never heard of tildify!</p></li><li>Aside from that, it does a bunch of work around paragraphs – it’s common for major modes to adjust the paragraph syntax, as it’s meant for prose and not code, to something akin to “paragraphs in code” – and that is what it’s doing here too.</li><li>Adaptive fill is another useful and expansive feature. It’s bound to <code>M-q</code> (<code>fill-paragraph</code>) and many other commands. And all the code here is doing is to try and guide the fill code so it plays well with SGML-like syntax. Whether it works well or not is in the eye of the beholder.</li><li>There’s code to set the comment start, end, and how line breaks are determined. Again, this is something you could reimplement in TS if you wanted, but you’d be leaping ahead of Emacs 29 as there is little official support for TS-powered commenting.</li><li><p class=" text-justify">Skeleton is a code templating tool that is often used for more than just that: before <code>M-x electric-pair</code> became a thing, it was often a stand-in for a wide range of things, such as closing braces or sneakily inserting spaces and newlines in the right places.</p><p class=" text-justify">It’s very similar to its cousin, Tempo, in that they work in much the same way, and both have been neglected for the better part of twenty years.</p></li><li>There’s some font lock stuff – we’ll ditch that for sure.</li><li><code>syntax-propertize-function</code> is part of Emacs’s core machinery – a lot of it written in C – to help Emacs determine boundaries of things like braces and other syntactically important pairs, like quotes for strings. It does more than that; but it’s commonly used for that.</li><li>Imenu takes a regexp (or in TS, a function or regexp) to help you jump to semantically useful HTML element. We’ll be replacing that also.</li><li>Indentation is controlled in part by <code>indent-line-function</code>. We’ll definitely replace that.</li><li><p class=" text-justify">The syntax table is also worth mentioning. It’s a table mapping characters to a purpose. Like defining <code>;</code> is a comment in C; or that <code>"</code> denotes the beginning and end of a string. It’s also responsible for what Emacs consider a word, a symbol, and so on. There’s an existing one called <code>sgml-mode-syntax-table</code>.</p><p class=" text-justify">We <em>could</em> write our own, or we can trust that 30 years of incremental improvements to SGML-mode and friends has resulted in a syntax table that, out of the box at least, probably does what we want.</p></li></ol><p class=" text-justify">So… yeah, there is not a lot to the initialization of the major mode. And as you can see, I’m picking and choosing what I want to replace: if the indentation engine works well, you can probably get away with reusing it, if you’re fortunate enough to have an existing one to work from.</p><H3 id="bare-bones-html-major-mode">Bare bones HTML major mode</H3><p class=" text-justify">With that in mind, we can write our skeleton major mode.</p><pre class="sourceCode commonlisp"><code class="sourceCode commonlisp">(<span class="kw">require</span> 'sgml-mode) <span class="co">;;;###autoload</span> (define-derived-mode html-ts-mode sgml-mode <span class="st">"HTML[ts]"</span> <span class="st">"Major mode for editing HTML with tree-sitter."</span> :syntax-table sgml-mode-syntax-table (setq-local font-lock-defaults <span class="kw">nil</span>) (<span class="kw">when</span> (treesit-ready-p 'html) (treesit-parser-create 'html) (html-ts-setup)))</code></pre><p class=" text-justify">The awkwardly named <code>treesit-ready-p</code> returns non-nil if it is “ready” to deal with the symbol you give it. The symbol being the name of the grammar, which in turn depends on the name of the compiled grammar library you’ve installed.</p><p class=" text-justify">As far as I know, this is the best (and only) way to check for the presence of a valid grammar. There are no user-facing commands to do this. Either way, if you <a href="/article/evaluating-elisp-emacs" class=" article-link">eval the elisp</a> as it’s shown, you should get back <code>t</code>.</p><p class=" text-justify">The next step is creating the parser. To do that, you execute <code>treesit-parser-create</code>. Note that it does indeed return the parser object you asked for, but it <em>also</em> installs it into the current buffer — so the function name is a bit misleading.</p><p class=" text-justify">Effectively, despite creating a parser, we want the side-effect of it installing itself into the buffer. To list parsers belonging to a buffer, consult <code>treesit-parser-list</code>.</p><p class=" text-justify">I recommend you keep the logic of configuring tree-sitter in its own function like I’ve done with <code>html-ts-setup</code>. It just makes debugging and resetting the state easier, so it’s not a hard requirement.</p><pre class="sourceCode commonlisp"><code class="sourceCode commonlisp">(<span class="kw">defun</span><span class="fu"> html-ts-setup </span>() <span class="st">"Setup treesit for html-ts-mode."</span> <span class="co">;; Our tree-sitter setup goes here.</span> <span class="co">;; This handles font locking -- more on that below.</span> (setq-local treesit-font-lock-settings (<span class="kw">apply</span> #'treesit-font-lock-rules html-ts-font-lock-rules))) <span class="co">;; This handles indentation -- again, more on that below.</span> (setq-local treesit-simple-indent-rules html-ts-indent-rules) <span class="co">;; ... everything else we talk about go here also ...</span> <span class="co">;; End with this</span> (treesit-major-mode-setup))</code></pre><p class=" text-justify">The <em>one thing</em> you must remember to do after you’ve configured tree-sitter, is to call <code>treesit-major-mode-setup</code>. Do it after you’ve set up your indentation and font lock rules. It’s an essential step, and if you miss it, or apply it inconsistently during development, your font lock and indentation rules won’t apply properly.</p><p class=" text-justify">Now, let’s write some font lock rules.</p><H2 id="font-locking">Font Locking</H2><p class=" text-justify">One common criticism of trad-style font locking is the lack of granularity. You had a few levels ranging from zero, to mostly zero, to angry fruit salad, as people who disliked full font locking would often call it. And the lack of contextual granularity didn’t help either: <code>M-x customize-apropos-face</code> and type <code>font lock face</code> and you’ll see a lot more than you did in earlier Emacsen. That’s a good thing: now we have the option to highlight function calls and function names differently, for example.</p><p class=" text-justify">To better understand how that’s applied, you need to look at <code>treesit-font-lock-feature-list</code>. It’s a list of lists: the sub-lists contain symbols that are unique to each TS major mode. The symbols map to what they’re going to highlight (like <code>comment</code> for comments, and <code>tag</code> for the HTML tag names), and the lists they’re in correspond to the level of font lock engagement. Set <code>treesit-font-lock-level</code> to a number corresponding to how many of the tiers in the list you want Emacs to render. It’s a nice marriage of old and new: you can reshuffle the list to match what you want, if you’re so inclined, or you can set <code>treesit-font-lock-level</code> to a number, if you prefer a simpler approach to managing this.</p><p class=" text-justify">I’ve opted for this simple feature list. By all means add or remove things to suit your needs.</p><pre class="sourceCode commonlisp"><code class="sourceCode commonlisp">(setq-local treesit-font-lock-feature-list '((comment) (constant tag attribute) (<span class="kw">declaration</span>) (delimiter)))</code></pre><p class=" text-justify">Next up, you need to write the font lock rules. The variable <code>treesit-font-lock-settings</code> is the ultimate variable that controls how font locking works. You probably don’t need to touch it directly. Instead, you’re encouraged to use the function <code>treesit-font-lock-rules</code> to build your rules.</p><p class=" text-justify">The <code>treesit-font-lock-rules</code> function again is a bit peculiar. It takes a variadic number of arguments which must follow a prescribed pattern, like so:</p><pre class="sourceCode commonlisp"><code class="sourceCode commonlisp">(treesit-font-lock-rules :language 'html :override <span class="kw">t</span> :feature 'delimiter '([ <span class="st">"&lt;"</span> <span class="st">"&gt;"</span> <span class="st">"/&gt;"</span> <span class="st">"&lt;/"</span>] @font-lock-bracket-face) :language 'html :override <span class="kw">t</span> :feature 'comment '((comment) @font-lock-comment-face) ... )</code></pre><p class=" text-justify">As you can see, you must specify the <code>:language</code> the rule belongs to. That means you can interleave multiple languages – but more on that in a bit. The <code>:override</code> property is there so you can optionally override a previously-applied font lock rule. Useful as you’ll undoubtedly end up with overlapping rules if your language is complex.</p><p class=" text-justify">The <code>:feature</code> is the name of the feature. You can pick anything you like, but you’ll need to ensure it is somewhere in <code>treesit-font-lock-feature-list</code> also.</p><p class=" text-justify">And finally, you write the query you want to highlight.</p><p class=" text-justify">Top tip. <em>Don’t</em> make the mistake a lot of the TS major mode authors did by cramming all their rules into a function call to <code>treesit-font-lock-rules</code>. That’s what the function asks you to do, but I think it’s a huge faux pas, and a peculiar design choice.</p><p class=" text-justify">If you hardcode the rules as arguments to that function, you (or more likely, a user of your mode) can’t amend the rules post-facto. You can if you put them in a variable first, and then <code>apply</code> them: that way you don’t have to quote everything left, right and center like I did above.</p><p class=" text-justify">Do this instead:</p><pre class="sourceCode commonlisp"><code class="sourceCode commonlisp">(<span class="kw">defvar</span><span class="fu"> html-ts-font-lock-rules</span> '(:language html :override <span class="kw">t</span> :feature delimiter ([<span class="st">"&lt;"</span> <span class="st">"&gt;"</span> <span class="st">"/&gt;"</span> <span class="st">"&lt;/"</span>] @font-lock-bracket-face) :language html :override <span class="kw">t</span> :feature comment ((comment) @font-lock-comment-face))</code></pre><p class=" text-justify">If you need to make it dynamic, you still can, using functions or other methods for building the list dynamically from composable variables. (This is more important than you think if you have a grammar that largely intersects with other ones.)</p><p class=" text-justify">And to apply the rules:</p><pre class="sourceCode commonlisp"><code class="sourceCode commonlisp">(setq-local treesit-font-lock-settings (<span class="kw">apply</span> #'treesit-font-lock-rules html-ts-font-lock-rules)))</code></pre><p class=" text-justify">Much nicer, and someone else can use your rules in their major mode.</p><p class=" text-justify">Regardless of the method used, you can safely append to <code>treesit-font-lock-settings</code> at any point, if you so desire. You can add your own rules (using <code>:override t</code> if you have to) to add custom highlighters to an existing major mode, even.</p><p class=" text-justify">Curiously, there’s no customizable variable for this sort of thing, and so this is the only way. Still, you can do it, and Combobulate’s query builder can do it for you also, if you use it for highlighting.</p><H3 id="resetting-the-font-lock-engine">Resetting the Font Lock Engine</H3><p class=" text-justify">One common problem is cycling the font lock changes. You can try <code>C-x x f</code> to call <code>font-lock-update</code>. That might do it, depending on your workflow. <code>defvar</code> forms don’t re-set when you <code>M-x eval-buffer</code>, so be sure to eval them manually (<code>C-M-x</code> or <code>C-x C-e</code>) if you do it this way.</p><p class=" text-justify">A courtesy call to <code>treesit-major-mode-setup</code> won’t go amiss if you think things aren’t working out for you. Query errors may not surface depending on how you reset things. So try different things; go back to a known state (comment stuff out), and retry.</p><p class=" text-justify">Another way that I like is to cycle the major modes: switch to <code>M-x fundamental-mode</code> and back again.</p><p class=" text-justify">And what if you to do all of it in one go? <a href="/article/keyboard-macros-are-misunderstood" class=" article-link">Record a quick keyboard macro</a>.</p><H3 id="writing-queries">Writing Queries</H3><p class=" text-justify">The cardinal rule is the capturing group – which is what tree-sitter matches against, which needn’t be the whole query, much like regexp capturing groups – must be named after the font lock face you want to use. So use something like <code>@font-lock-comment-face</code> for comments.</p><figure> <img src="/static/uploads/combobulate-query-builder-example.svg" alt="" class=" center-block"> <figcaption>Combobulate's query builder in action. Here I am highlighting just the start tags in gold. You can use the query builder to help you design the queries you want to font lock.</figcaption> </figure><p class=" text-justify">Here’s an example from Combobulate. I’m using Combobulate’s builtin highlighter shortcuts, but you can use <code>@some-font-lock-face</code> instead.</p><blockquote><p class=" text-justify">Combobulate needs to know about the language nodes you want to highlight. To do this for a language that Combobulate does not support (let’s be honest, that’s most likely the case for the language you’re writing!) you will need to:</p><ol><li>Clone Combobulate.</li><li>Edit <code>sources.ini</code> and add a new entry to the grammar and node JSON files.</li><li>Call <code>build-relationships.py</code> either inside the Docker container if you do not have Python 3.11, or outside.</li><li>Re-evaluate the updated <code>combobulate-rules.el</code> file.</li><li>Type <code>M-x combobulate-query-builder</code> in a buffer with your language code in it. You’ll be prompted for the language to activate.</li><li>You’re ready to query.</li></ol></blockquote><p class=" text-justify">If you don’t want to use Combobulate to help you, the builtin method – the only method – is to call <code>treesit-query-capture</code> with a starting node (often the one from <code>treesit-buffer-root-node</code> or <code>treesit-parser-root-node</code>) and the query and then manually inspect the output to see if it’s right. Ugh. It’s messy, and it’s hard work. Trust me, I know. I recommend you <a href="/article/evaluating-elisp-emacs" class=" article-link">learn how to use IELM</a> if you decide to go this route.</p><p class=" text-justify">Also keep in mind that you can give the query machinery (including the font lock rules function) two different styles of queries: strings, which exactly match the syntax that tree-sitter’s official query manual (and engine) expects; <em>or</em>, an s-expression form with a few crucial differences. (Combobulate’s query builder works with strings.)</p><p class=" text-justify">The differences are rather important to know about:</p><ul><li><code>.</code> in the string form (indicating anchoring) becomes <code>:anchor</code> in the s-expression format.</li><li>Predicates, like <code>#match</code> and <code>#eq</code>, become <code>:match</code> and <code>:eq</code>.</li><li>Quantifiers, such as <code>+</code> and <code>*</code>, turn into <code>:+</code> and <code>:*</code>.</li></ul><p class=" text-justify">That’s it.</p><p class=" text-justify">When you write a query, make sure you use at least one capturing group or you’ll get zero matches. It’s a design feature: no capture group, no matches. Confusing, but that’s how it is.</p><p class=" text-justify">So, building on the example query from the screenshot above, the complete version would look a bit like this:</p><pre class="sourceCode commonlisp"><code class="sourceCode commonlisp">(<span class="kw">defvar</span><span class="fu"> html-ts-font-lock-rules</span> '(:language html :override <span class="kw">t</span> :feature tag ((element [(start_tag (tag_name) @font-lock-variable-name-face) (self_closing_tag (tag_name) @font-lock-variable-name-face) (end_tag (tag_name) @font-lock-variable-name-face)]))))</code></pre><p class=" text-justify">Not much to it.</p><p class=" text-justify">Here’s the complete list:</p><pre class="sourceCode commonlisp"><code class="sourceCode commonlisp">(<span class="kw">defvar</span><span class="fu"> html-ts-font-lock-rules</span> '(:language html :feature delimiter ([ <span class="st">"&lt;!"</span> <span class="st">"&lt;"</span> <span class="st">"&gt;"</span> <span class="st">"/&gt;"</span> <span class="st">"&lt;/"</span>] @font-lock-bracket-face) :language html :feature comment ((comment) @font-lock-comment-face) :language html :feature attribute ((attribute (attribute_name) @font-lock-constant-face <span class="st">"="</span> @font-lock-bracket-face (quoted_attribute_value) @font-lock-string-face)) :language html :feature tag ((script_element [(start_tag (tag_name) @font-lock-doc-face) (end_tag (tag_name) @font-lock-doc-face)])) :language html :feature tag ([(start_tag (tag_name) @font-lock-function-call-face) (self_closing_tag (tag_name) @font-lock-function-call-face) (end_tag (tag_name) @font-lock-function-call-face)]) :language html :override <span class="kw">t</span> :feature <span class="kw">declaration</span> ((doctype) @font-lock-keyword-face)))</code></pre><p class=" text-justify">Done right, when you activate <code>M-x html-ts-mode</code> you’ll see everything light up. If not, try fidgeting with <code>treesit-font-lock-level</code> — but beware! It has an edge-trigger to reset font locking in all tree-sitter buffers so changes take effect. You’ll have to set it with <code>setopt</code> or <code>customize-set-variable</code> for the changes to take effect. (Or reset it manually like I showed you above.)</p><p class=" text-justify">But, yeah, well. That’s it. Churn out some queries and stick ’em in a variable. That’s the long and the short of it. I’ve only covered the basics, but you can use <code>#match</code> and friends to match nodes by regexp — convenient, if you want to highlight comments beginning with TODO, or what have you. The sky is the limit here.</p><p class=" text-justify">Now, let’s do indentation.</p><H2 id="indentation">Indentation</H2><p class=" text-justify">If you’ve never written an indentation engine ‘the old-fashioned way,’ then, well, <em>lucky you</em>. Indentation engines are approximately one-third art, one-third science, and one-third misery.</p><p class=" text-justify">Rewriting an existing indentation engine with tree-sitter will likely shed a chunk of weight and complexity. The declarative indentation engine – based on SMIE, the Simple-Minded Indentation Engine, that’s been in Emacs for years – does cut down on the tedium also.</p><p class=" text-justify">The key take-away here is to write <em>one</em> rule at a time. Pick one corner of the buffer and start from there and work your way through, adding one rule as you go. That’s my recommendation, anyway. Enabling <code>treesit--indent-verbose</code> is also helpful here: when you indent, Emacs will tell you the rule that fired.</p><p class=" text-justify">Much like font locking, one variable controls indentation: <code>treesit-simple-indent-rules</code>. You feed it declarative rules that look a bit like this:</p><pre class="sourceCode commonlisp"><code class="sourceCode commonlisp"><span class="co">;; Alist of (LANGUAGE . RULES)</span> `((html <span class="co">;; Rule 1</span> ((parent-is <span class="st">"element"</span>) parent <span class="dv">2</span>) <span class="co">;; Rule 2</span> ((node-is ,(regexp-opt '(<span class="st">"element"</span> <span class="st">"self_closing_tag"</span>))) parent <span class="dv">2</span>) ... ))</code></pre><p class=" text-justify">Such that each RULES entry is of the form <code>(MATCHER ANCHOR OFFSET)</code>. Where <code>(node-is "element")</code> is the MATCHER, in this case a special form defined in <code>treesit-simple-indent-presets</code>. There are many other useful forms, each matching a different node in the tree. Make sure you look, even if you can probably get by with just one or two matchers.</p><p class=" text-justify">The second value is the ANCHOR, <code>parent</code>, indicating that the indentation engine must find the parent of the node point is near or on: here it’s <code>element</code> for the first rule. The OFFSET is how much to indent, and you can use a variable instead of a scalar.</p><p class=" text-justify">With little more than simple declarative rules like this one, you can build an effective indentation engine with a bit of grit and perseverance. You probably don’t need more than 10-15 rules for most languages. As always, you can look at how other major modes in Emacs are doing it for inspiration.</p><p class=" text-justify">You need to read the docstring for <code>treesit-simple-indent-presets</code> and <code>treesit-simple-indent-rules</code>. The manual entries are must-reads also: just punch <code>i</code> from the <code>*Help*</code> window to jump to the pertinent manual node.</p><p class=" text-justify">One way to write rules is with a simple 1-2-3 exercise:</p><ol type="1"><li>Add or amend a rule to <code>treesit-simple-indent-rules</code>. <code>*scratch*</code> or IELM are good ways to do this.</li><li>Reload your major mode, as per above. Now hit <code>TAB</code> at where you want to test your new rule and observe (with <code>treesit--indent-verbose</code> set to <code>t</code>) that it’s firing the right rule <em>and</em> that you’re happy with the indentation.</li><li>Hit <code>RET</code> at the end of the newly-indented line and check if the indentation of the new line is also correct.</li></ol><p class=" text-justify">You can also try <code>M-x indent-region</code> (bound to <code>C-M-\\</code>) and indent the whole region and observe that everything is properly indented. Combine with <code>C-M-% ^\s-+ RET RET</code> to clear out lines beginning with whitespace to test this.</p><p class=" text-justify">I can’t promise this will work flawlessly. It <em>might</em> work, but it comes down to your major mode and your requirements: it would not work in Python or YAML, for example, as whitespace is contextual in those languages.</p><p class=" text-justify">There is no amount of writing endlessly about this that can make up for the fact that this is a case of sitting down and experimenting.</p><p class=" text-justify">I do have some general tips that may help:</p><ol><li><p class=" text-justify">The ANCHOR determines the baseline indentation that you can optionally add or subtract from with OFFSET. Never forget this. You first match <em>something</em> (<code>parent-is</code>, <code>node-is</code>, etc.) and <em>then</em> you find an anchor from <em>that</em> node: perhaps its <code>parent</code>, or something else.</p><p class=" text-justify">The OFFSET is exactly that: a change from the ANCHOR’s position.</p></li><li>You can pass regexps to the node type. You can use this to compress node rules that are identical to one another. Use <code>regexp-opt</code> or <code>rx</code> to do this.</li><li><p class=" text-justify">Some code is <em>self-similar</em>. That is, you can nest the same node type inside the other. Not always, but often. Think of arrays: <code>[1, 2, [3, ...]]</code>.</p><p class=" text-justify">You can define these with two rules: <code>((node-is "element") parent 2)</code>, to indent the current node; and <code>((parent-is "element") parent 2)</code> to catch the parent. That’ll ensure arbitrarily nested nodes indent properly.</p><p class=" text-justify">Bear in mind that you’ll have to decide on the indentation style separately, of course. For HTML it’s simple.</p></li><li>Order matters, so if things don’t work well, and you get snared by an earlier rule than the one you wanted, try moving the rules around. They’re processed first-to-last in the order they’re kept in the list.</li><li><p class=" text-justify">Ending with a <code>no-node</code> rule can be useful as a “catch-all” at the end of your rules list. <code>(no-node parent 0)</code>, for example, simply looks at the parent and maintains its indentation offset.</p><p class=" text-justify">Don’t sleep on this rule. It can carry you through an awful lot of “yeah, just keep the current offset”.</p></li><li>It’s just lisp. So if you want feature switches or toggles, you can splice stuff with backquote (or any number of other ways) to make up the ultimate set of rules.</li><li>You probably want a rule for the root node. In HTML it’s <code>document</code> (NOTE: in earlier versions it was called <code>fragment</code>), and in other languages it might be <code>program</code>, <code>source</code>, <code>document</code>, etc. This is the baseline rule. I’d put it at the top, but it’s not an iron clad rule.</li></ol><p class=" text-justify">Let’s look at how to indent HTML. Well… one way of doing it, anyway!</p><pre class="sourceCode commonlisp"><code class="sourceCode commonlisp">(setq-local treesit-simple-indent-rules `((html ((parent-is <span class="st">"document"</span>) column<span class="dv">-0</span> <span class="dv">0</span>) ((node-is ,(regexp-opt '(<span class="st">"element"</span> <span class="st">"self_closing_tag"</span>))) parent <span class="dv">2</span>) ((node-is <span class="st">"end_tag"</span>) parent <span class="dv">0</span>) ((node-is <span class="st">"/"</span>) parent <span class="dv">0</span>) ((parent-is <span class="st">"element"</span>) parent <span class="dv">2</span>) ((node-is <span class="st">"text"</span>) parent <span class="dv">0</span>) ((node-is <span class="st">"attribute"</span>) prev-sibling <span class="dv">0</span>) ((node-is <span class="st">"&gt;"</span>) parent <span class="dv">0</span>) ((parent-is <span class="st">"start_tag"</span>) prev-sibling <span class="dv">0</span>) (no-node parent <span class="dv">0</span>))))</code></pre><p class=" text-justify">That’s it. There’s more to do in terms of tidying it up (you can merge more nodes, but this is supposed to be readable and instructive) but that is all it takes.</p><p class=" text-justify">Let me explain the rules as they are.</p><ul><li><code>document</code> is the root node, and if it’s the parent of point when we indent then we anchor against <em>its</em> zeroth column (<code>column-0</code>) and add zero to the offset. In other words, the base indentation for anything that is a child of <code>document</code> is 0, because the node itself has an offset of zero also.</li><li><code>element</code> and <code>self_closing_tag</code> are the bread-and-butter of SGML languages. The rule looks at those two nodes at point, checks the parent’s offset and adds two. This gives us nice, simple tree-like indentation so that nested elements are indented properly.</li><li><p class=" text-justify"><code>end_tag</code> needs a bit of explaining. In tree terms, the structure looks a bit like this <code>(element (start_tag) (end_tag))</code>. Meaning, the end and start tags are children of a super-node called <code>element</code>.</p><p class=" text-justify">So when I tell Emacs I want the offset of <code>end_tag</code>’s <code>parent</code>, I’m getting the offset of <code>element</code>. I want it to be zero, because I want the HTML start and end tags to line up:</p><pre><code>&lt;foo&gt; &lt;bar/&gt; Hello, World! &lt;/foo&gt;</code></pre></li><li>You can match against anonymous nodes, as I do with <code>/</code> and <code>&gt;</code>. All I want from them that they respect the offset of their parent so they indent properly also.</li><li>I occasionally need to align things according to the parent of a node. In the little example above, this would correspond to things like <code>text</code>. In this case I want to indent by two where the parent is <code>element</code>. That catches <code>text</code>.</li><li>Attributes are much the same as what we’ve seen before, but with a crucial difference. I want my attributes to look at their sibling to determine their own indentation: in this case with <code>prev-sibling</code> and an offset of 0. Emacs will essentially respect the offset of the preceding attribute when it has to indent the current attribute node.</li><li><code>start_tag</code> gets the same treatment as attributes do: try to hew to the previous sibling’s offset.</li><li>And finally, a catch-all <code>no-node</code> entry that maintains the offset of its parent.</li></ul><p class=" text-justify">I’m sure there are better ways of doing it; there are worse ways, too. Ultimately, you can arrive at something that works well using any number of approaches, which I think is a positive thing indeed.</p><H2 id="imenu">Imenu</H2><p class=" text-justify">Imenu is the last piece of the puzzle for our scrappy HTML tree-sitter major mode. Conceptually, ignoring tree-sitter here, Imenu is nothing more than a manicured list of stuff to show. It’s not hard to write manually, from scratch, either.</p><p class=" text-justify">The tree-sitter “simple imenu” system is anything but, though. It’s a bit awkward to use, as it finds its initial matches with either a regular expression or a custom function that is passed each node in the tree. You cannot use a query directly. If you want to pick the name for the Imenu entry (the node name itself is rarely useful or expressive enough) then you’ll have to write a function to do this also. That, or start tangling with the way the tree-sitter defun finder works, as it’s tightly coupled to that system.</p><p class=" text-justify">Anyway. Here’s a basic example to demonstrate what I am talking about.</p><pre class="sourceCode commonlisp"><code class="sourceCode commonlisp">(<span class="kw">defun</span><span class="fu"> html-ts-imenu-node-p </span>(node) (<span class="kw">and</span> (string-match-p <span class="st">"^h[0-6]$"</span> (treesit-node-text node)) (<span class="kw">equal</span> (treesit-node-type (treesit-node-parent node)) <span class="st">"start_tag"</span>))) (<span class="kw">defun</span><span class="fu"> html-ts-imenu-name-function </span>(node) (<span class="kw">let</span> ((name (treesit-node-text node))) (<span class="kw">if</span> (html-ts-imenu-node-p node) (concat name <span class="st">" / "</span> (thread-first (treesit-node-parent node) (treesit-node-next-sibling) (treesit-node-text))) name))) (setq-local treesit-simple-imenu-settings `((<span class="st">"Heading"</span> html-ts-imenu-node-p <span class="kw">nil</span> html-ts-imenu-name-function)))</code></pre><p class=" text-justify">The gist here is, we need a way to pick the right nodes, which I am doing here with <code>html-ts-imenu-node-p</code>. You’ll also need a way of building the name of the Imenu entry (<code>html-ts-imenu-name-function</code>). Note that I am using a lesser-known feature in Emacs called <code>thread-first</code>. It “threads” the output from <code>(treesit-node-parent node)</code> into the <em>first</em> argument slot of <code>treesit-node-next-sibling</code>, and <em>that</em> output into the first argument of <code>treesit-node-text</code>, which is then returned.</p><p class=" text-justify">Retrieving the element’s <code>raw_text</code> takes a bit of work, but it’s manageable. To see why I need to do this, look at (with <code>treesit-explore-mode</code>) the structure of the node I want to match: the <code>tag_name</code> element is the one we’re matching against to begin with. To get to <code>raw_text</code> I need to get its parent and then its sibling.</p><p class=" text-justify">Finally, I just assign the desired category (“Heading”) and how to match them.</p><p class=" text-justify">Imenu caches its results. Use <code>M-: (imenu-flush-cache)</code> to clear out the Imenu cache between tests. And don’t forget about <a href="/article/which-function-mode" class=" article-link">Which Function Mode</a>, which plugs into the Imenu machinery to show you the current “function” point is in.</p><p class=" text-justify">And that’s that.</p><H2 id="next-steps">Next Steps</H2><p class=" text-justify">With that in place, and knowledge of how to do all of this, tackling <code>treesit-defun-name-function</code> and <code>treesit-defun-type-regexp</code> should be a walk in the park, as it’s similar to Imenu.</p><p class=" text-justify">Other things worth considering is adding font locking and indentation to the <code>style</code> and <code>script</code> tags. One benefit of tree-sitter is that it can seamlessly merge multiple grammars rather easily with <code>treesit-range-rules</code>.</p><p class=" text-justify">Unfortunately, Emacs 29’s support for this is still rather poor and immature. And ideally, you’d want to recycle the CSS and JS rules from their respective TS-enabled major modes. But, sadly, that is not so easy to do, as the rules are not declared directly in a variable. You’d have to come up with a wide range of gnarly tricks to get at them. It’s not hard or impossible, but it’s harder than it should be.</p><p class=" text-justify">And it does not solve the tricky problem of having their respective major modes activate in the right places, either. <a href="/article/polymode-multiple-major-modes-how-to-use-sql-python-in-one-buffer" class=" article-link">Polymode can share major modes in a buffer</a> but it’s not seen much in the way of improvement over the years, and it does not understand tree-sitter.</p><p class=" text-justify">Still, though, a job well done. I spent longer writing the article than I did writing this integration, which I think says it all.</p><p class=" text-justify">You can find the complete file, which should work out of the box in the <a href="https://github.com/mickeynp/html-ts-mode" class=" article-link">html-ts-mode</a> repo on my Github.</p> /article/lets-write-a-treesitter-major-mode Wed, 13 Sep 2023 07:01:06 +0000 What's New in Emacs 29.1? https://www.masteringemacs.org/article/whats-new-in-emacs-29-1?utm_source=newsletter&utm_medium=rss What's new in Emacs 29.1? I go through every change and annotate the ones I feel are interesting or worth knowing more about. <p class=" text-justify drop-cap">Wonderful news! Emacs 29.1 is out now. If you’re wondering what’s new, and why it’s <em>absolutely</em> worth upgrading, keep reading. I’ve annotated the NEWS file – as I’ve been doing for the last thirteen years – with my own thoughts and comments on the deluge of new and interesting features.</p><p class=" text-justify">To celebrate, <a href="/book" class=" article-link">my book on Emacs</a> is also on sale: 29% off.</p><p class=" text-justify">Before I go into the specifics of every minor item, a quick summary of what I think are the major highlights of Emacs 29.1:</p><dl><dt>Official tree-sitter support</dt><dd><p class=" text-justify">Tree-sitter is a third-party library that parses your text (usually code, but also things like Markdown) into a <em>concrete syntax tree</em>. It’s one of the hallmark features in Emacs 29, though it is optional for now.</p><p class=" text-justify">I have written extensively about tree sitter. If you’re new, and wondering why you should care, read my article on <a href="/article/tree-sitter-complications-of-parsing-languages" class=" article-link">tree-sitter and the complications of parsing languages</a>.</p><p class=" text-justify">Briefly, a concrete syntax tree is the distillation of your text or code, and so it’s better than regular expressions and <em>ad hoc</em> snippets of code at extracting meaning from your code. It offers better font locking (syntax highlighting); correct and more precise indentation; and limitless potential to extract syntactically interesting elements from your code. That means you can do <a href="/article/combobulate-structured-movement-editing-treesitter" class=" article-link">structured movement and editing with tree-sitter</a>, as my package Combobulate aims to do, in the vein of paredit and similar tools.</p><p class=" text-justify">Installing tree-sitter is not straight forward, at least until your package manager starts shipping Emacs 29 with tree sitter support. My article on <a href="/article/how-to-get-started-tree-sitter" class=" article-link">how to get started with tree-sitter</a> is a good place to start if you’re using Linux.</p></dd><dt>EGlot, the Language Server Client</dt><dd><p class=" text-justify">EGlot, the leaner complement to the fully-featured LSP-mode, is now built into Emacs. It should work out of the box. Just type <code>M-x eglot</code> in a buffer to get started.</p></dd><dt>Use-package</dt><dd><p class=" text-justify">I wrote about <a href="/article/spotlight-use-package-a-declarative-configuration-tool" class=" article-link">use-package, a declarative configuration tool</a> back when it came out. That was a decade ago, almost. It’s finally in Emacs, which is a good thing indeed.</p><p class=" text-justify">It’s an easier and more expressive way of sharing Emacs configuration snippets, and knowing it’s built in just makes everything much easier.</p></dd><dt>Better long line support</dt><dd><p class=" text-justify">There have been, ah, several bites at this rather bitter cherry over the years.</p><p class=" text-justify">Emacs gums up on very long lines. To say it’s infuriating would be an understatement. I’ve been caught out by this a million times. Sometimes I have to kill Emacs as it’s desperately trying to append to the same, long line of text.</p><p class=" text-justify">It’s still not <em>solved</em>, but it <em>is</em> a lot better. More on that below.</p></dd><dt>Native SQLite Support</dt><dd><p class=" text-justify">SQLite support, in one form or another, has been around for a while. Now you can legitimately compile Emacs with sqlite support, which is a large step forward.</p><p class=" text-justify">As much as we like to delude ourselves into thinking that unstructured orgmode text, or even s-expressions, represent the pinnacle of text editing and the acme of scrappy hackerdom, that mantra does come with a significant penalty to performance. Ask anyone with very large org mode files or someone who wants to fast, random access to clown-sized s-expression trees. I’m sure we’ll see a much larger effort towards representing – yes, indeed, the whole point of Emacs – text in buffers as we know it, but with a fast query engine and efficient storage medium in the backend. The two need not be mutually exclusive.</p></dd><dt>Changing the init directory</dt><dd><p class=" text-justify">You can now instruct Emacs to read its initialization from another directory from the command line. Hugely useful if you’re using and testing multiple Emacs versions at the same time.</p></dd><dt>Pixel Scroll Precision Mode</dt><dd><p class=" text-justify">You can enable <code>M-x pixel-scroll-precision-mode</code> and get smooth scrolling like you do in other programs.</p></dd></dl><H2 id="installation-changes-in-emacs-29.1">Installation Changes in Emacs 29.1</H2><pre><code>Ahead-of-time native compilation can now be requested via configure. Use '--with-native-compilation=aot' to request that all the Lisp files in the Emacs tree should be natively compiled ahead of time. (This is slow on most machines.) This feature existed in Emacs 28.1, but was less easy to request.</code></pre><p class=" text-justify">Native compilation was the marquee feature in Emacs 28. It uses GCC’s libgccjit to add just-in-time compilation for all elisp code, resulting in – usually – greatly improved performance. My article, <a href="/article/speed-up-emacs-libjansson-native-elisp-compilation" class=" article-link">Speed up Emacs with libjansson and native elisp compilation</a>, has all the gory details.</p><p class=" text-justify">This new switch is, as it says, really just a way of front loading the (frustrating to some, but I do not mind) elisp compilation that usually runs in the background in Emacs. You could tell it to do this before also, but it was admittedly harder.</p><p class=" text-justify">The thing is, native compilation is only run once; once it’s done, it won’t run again unless the source file changes.</p><p class=" text-justify">So, if you want to do it all up front while you’re building Emacs from source, you can with this switch.</p><pre><code>Emacs can be built with the tree-sitter parsing library. This library, together with separate grammar libraries for each language, provides incremental parsing capabilities for several popular programming languages and other formatted files. Emacs built with this library offers major modes, described elsewhere in this file, that are based on the tree-sitter's parsers. If you have the tree-sitter library installed, the configure script will automatically include it in the build; use '--without-tree-sitter' at configure time to disable that. Emacs modes based on the tree-sitter library require an additional grammar library for each mode. These grammar libraries provide the tree-sitter library with language-specific lexical analysis and parsing capabilities, and are developed separately from the tree-sitter library itself. If you don't have a grammar library required by some Emacs major mode, and your distro doesn't provide it as an installable package, you can compile and install such a library yourself. Many libraries can be downloaded from the tree-sitter site: https://github.com/tree-sitter Emacs provides a user command, 'treesit-install-language-grammar', that automates the download and build process of a grammar library. It prompts for the language, the URL of the language grammar's VCS repository, and then uses the installed C/C++ compiler to build the library and install it. You can also do this manually. To compile such a library after cloning its Git repository, compile the files "scanner.c" and "parser.c" (sometimes named "scanner.cc" and "parser.cc") in the "src" subdirectory of the library's source tree using the C or C++ compiler, then link these two files into a shared library named "libtree-sitter-LANG.so" ("libtree-sitter-LANG.dll" on MS-Windows, "libtree-sitter-LANG.dylib" on macOS), where LANG is the name of the language supported by the grammar as it is expected by the Emacs major mode (for example, "c" for 'c-ts-mode', "cpp" for 'c++-ts-mode', "python" for 'python-ts-mode', etc.). Then place the shared library you've built in the same directory where you keep the other shared libraries used by Emacs, or in the "tree-sitter" subdirectory of your 'user-emacs-directory', or in a directory mentioned in the variable 'treesit-extra-load-path'. You only need to install language grammar libraries required by the Emacs modes you will use, as Emacs loads these libraries only when the corresponding mode is turned on in some buffer for the first time in an Emacs session. We generally recommend to use the latest versions of grammar libraries available from their sites, as these libraries are in constant development and occasionally add features and fix important bugs to follow the advances in the programming languages they support.</code></pre><p class=" text-justify">As I recommended in the beginning, my article on <a href="/article/how-to-get-started-tree-sitter" class=" article-link">installing tree-sitter</a> is a good place to start if you’re unfamiliar with the finer points of installing tree-sitter. Doubly so for the shared libraries you need for the languages you want tree-sitter to use. I demonstrate how you can both compile from source, but also download pre-compiled binaries. The latter is especially important for OSes like Windows where you may not have the tooling required to do this yourself.</p><p class=" text-justify">It’s… a significant barrier to entry, in my opinion, to require all this work before you can use tree-sitter. The maintainers’ reasons for not shipping the libraries is also reasonable: it’s not in the scope of the GNU Emacs project to lug around a bunch of libraries written by third-party sources.</p><p class=" text-justify">Having said that, the frustration of setting it up is somewhat alleviated by the helpful <code>treesit-install-language-grammar</code> function, as it’ll attempt to guess the target and compiler switches required and automatically check out the tree-sitter libraries from Ghit. But, still, it could be easier.</p><p class=" text-justify">So, yeah, again: read my article where I cover all the gotchas and things you need to know. This sort of stuff is the prize you get for going through with it all:</p><figure> <img src="/static/uploads/combobulate/expand-region.gif" alt="" class=" center-block"> <figcaption>My package, <a href="/article/combobulate-structured-movement-editing-treesitter" class=" article-link">Combobulate</a>, adds advanced editing and movement using tree-sitter. This is Combobulate's expand region feature, bound to <code>M-h</code>.</figcaption> </figure><p class=" text-justify">Thanks to tree-sitter you can have crisp and clear “expand region” functionality that’ll correctly mark the boundaries of each syntactic unit.</p><pre><code>Emacs can be built with built-in support for accessing SQLite databases. This uses the popular sqlite3 library, and can be disabled by using the '--without-sqlite3' option to the 'configure' script.</code></pre><p class=" text-justify">With formal support of SQLite in Emacs, I can imagine that this’ll be one of those options most distro maintainers will enable by default, as SQLite’s widely used in most distros already. That’s good new for everyone: SQLite will find a place in all manner of places in Emacs over time.</p><p class=" text-justify">All we need now is a nice, ergonomic and Lispy query builder layer on top. Any takers?</p><pre><code>Support for the WebP image format. This support is built by default when the libwebp library is available, and includes support for animated WebP images. To disable WebP support, use the '--without-webp' configure flag. Image specifiers can now use ':type webp'.</code></pre><p class=" text-justify">Useful if you do a lot of web development where that is becoming a common file format. I cannot divine how this integrates with the imagemagick configure flag, though. The implication is it is not required at all or that it even works with imagemagick.</p><pre><code>Emacs now installs the ".pdmp" file using a unique fingerprint in the name. The file is typically installed using a file name akin to "...dir/libexec/emacs/29.1/x86_64-pc-linux-gnu/emacs-&lt;fingerprint&gt;.pdmp". If a constant file name is required, the file can be renamed to "emacs.pdmp", and Emacs will find it during startup anyway.</code></pre><p class=" text-justify">This should be of interest to few, except maybe distro maintainers.</p><pre><code>Emacs on X now uses XInput 2 for input events. If your X server has support and you have the XInput 2 development headers installed, Emacs will use the X Input Extension for handling input. If this causes problems, you can configure Emacs with the option '--without-xinput2' to disable this support. '(featurep 'xinput2)' can be used to test for the presence of XInput 2 support from Lisp programs.</code></pre><p class=" text-justify">I reported a couple of bugs against this (which were fixed within the hour!) particularly around mouse wheel scrolling not working well. If you do compile Emacs with this flag, as I have and still do, I’d keep an eye on weird input event regressions. Make sure you report them.</p><pre><code>Emacs can now be optionally built with the Cairo XCB backend. Configure Emacs with the '--with-cairo-xcb' option to use the Cairo XCB backend; the default is not to use it. This backend makes Emacs moderately faster when running over X connections with high latency, but is currently known to crash when Emacs repeatedly closes and opens a display connection to the same terminal; this could happen, for example, if you repeatedly visit files via emacsclient in a single client frame, each time deleting the frame with 'C-x C-c'.</code></pre><p class=" text-justify">Cairo’s an open source 2d rendering library that Emacs added support for some versions ago.</p><pre><code>Emacs now supports being built with pure GTK. To use this option, make sure the GTK 3 (version 3.22.23 or later) and Cairo development files are installed, and configure Emacs with the option '--with-pgtk'. Unlike the default X and GTK build, the resulting Emacs binary will work on any underlying window system supported by GDK, such as Wayland and Broadway. We recommend that you use this configuration only if you are running a window system other than X that's supported by GDK. Running this configuration on X is known to have problems, such as undesirable frame positioning and various issues with keyboard input of sequences such as 'C-;' and 'C-S-u'. Running this on WSL is also known to have problems. Note that, unlike the X build of Emacs, the PGTK build cannot automatically switch to text-mode interface (thus emulating '-nw') if it cannot determine the default display; it will instead complain and ask you to invoke it with the explicit '-nw' option.</code></pre><p class=" text-justify">The warning about not running Emacs built with <code>--with-pgtk</code> under X cannot be understated. You really shouldn’t do that, though you’ll occasionally see references to people insisting that you build Emacs with it, regardless of the window system you’re using. If you’re not using Wayland/Broadway, you probably do not care about this.</p><pre><code>Emacs has been ported to the Haiku operating system. The configuration process should automatically detect and build for Haiku. There is also an optional window-system port to Haiku, which can be enabled by configuring Emacs with the option '--with-be-app', which will require the Haiku Application Kit development headers and a C++ compiler to be present on your system. If Emacs is not built with the option '--with-be-app', the resulting Emacs will only run in text-mode terminals. To enable Cairo support, ensure that the Cairo and FreeType development files are present on your system, and configure Emacs with '--with-be-cairo'. Unlike X, there is no compile-time option to enable or disable double-buffering; it is always enabled. To disable it, change the frame parameter 'inhibit-double-buffering' instead.</code></pre><pre><code>Emacs no longer reduces the size of the Japanese dictionary. Building Emacs includes generation of a Japanese dictionary, which is used by Japanese input methods. Previously, the build included a step of reducing the size of this dictionary's vocabulary. This vocabulary reduction is now optional, by default off. If you need the Emacs build to include the vocabulary reduction, configure Emacs with the option '--with-small-ja-dic'. In an Emacs source tree already configured without that option, you can force the vocabulary reduction by saying make -C leim generate-ja-dic JA_DIC_NO_REDUCTION_OPTION='' after deleting "lisp/leim/ja-dic/ja-dic.el".</code></pre><pre><code>The docstrings of preloaded files are not in "etc/DOC" any more. Instead, they're fetched as needed from the corresponding ".elc" files, as was already the case for all the non-preloaded files.</code></pre><H2 id="startup-changes-in-emacs-29.1">Startup Changes in Emacs 29.1</H2><pre><code>'--batch' and '--script' now adjust the garbage collection levels. These switches now set 'gc-cons-percentage' to 1.0 (up from the default of 0.1). This means that batch processes will typically use more memory than before, but use less time doing garbage collection. Batch jobs that are supposed to run for a long time should adjust the limit back down again.</code></pre><p class=" text-justify">That’s excellent news and it cuts out the middle-man of having to manually <code>let</code>-bind these variables if you were doing complex things in batch mode.</p><pre><code>Emacs can now be used more easily in an executable script. If you start an executable script with #!/usr/bin/emacs -x Emacs will start without reading any init files (like with '--quick'), and then execute the rest of the script file as Emacs Lisp. When it reaches the end of the script, Emacs will exit with an exit code from the value of the final form.</code></pre><p class=" text-justify">I’ll have to experiment and see if I can elide the need for <code>bash</code> in my tool <code>ezf</code>, which is an Emacs-only version of the <code>fzf</code> tool. See: <a href="/article/fuzzy-finding-emacs-instead-of-fzf" class=" article-link">Fuzzy Finding with Emacs Instead of fzf</a>.</p><pre><code>Emacs now supports setting 'user-emacs-directory' via '--init-directory'. Use the '--init-directory' command-line option to set 'user-emacs-directory'.</code></pre><p class=" text-justify">Wonderful news for those of us who have to test our packages using various Emacs versions and user configurations.</p><pre><code>Emacs now has a '--fingerprint' option. This will output a string identifying the current Emacs build, and exit.</code></pre><p class=" text-justify">Useful if you run Emacs in CI, I guess.</p><pre><code>New hook 'after-pdump-load-hook'. This is run at the end of the Emacs startup process, and is meant to be used to reinitialize data structures that would normally be done at load time.</code></pre><p class=" text-justify">This is unlikely to be of use to anyone except elisp grognards doing very particular things.</p><H3 id="native-compilation">Native Compilation</H3><pre><code>New command 'native-compile-prune-cache'. This command deletes old subdirectories of the eln cache (but not the ones for the current Emacs version). Note that subdirectories of the system directory where the "*.eln" files are installed (usually, the last entry in 'native-comp-eln-load-path') are not deleted.</code></pre><p class=" text-justify">If you regularly rebuild Emacs from scratch, it’s probably worth enabling this.</p><pre><code>New function 'startup-redirect-eln-cache'. This function can be called in your init files to change the user-specific directory where Emacs stores the "*.eln" files produced by native compilation of Lisp packages Emacs loads. The default eln cache directory is unchanged: it is the "eln-cache" subdirectory of 'user-emacs-directory'.</code></pre><p class=" text-justify">Files are put in your – usually <code>.emacs.d</code> – directory by default. If you dislike Emacs polluting that directory you can now change it.</p><H2 id="incompatible-changes-in-emacs-29.1">Incompatible changes in Emacs 29.1</H2><pre><code>The image commands have changed key bindings. In previous Emacs versions, the '+', '-' and 'r' keys were bound when point was over an image. In Emacs 29.1, additional commands have been added, and this made it more likely that users would trigger the image commands by mistake. To avoid this, all image commands have been moved to the 'i' prefix keymap, so '+' is now 'i +', '-' is now 'i -', and 'r' is now 'i r'. In addition, these commands are now repeating, so you can rotate an image twice by saying 'i r r', for instance.</code></pre><p class=" text-justify">It’s not clear, but all the key bindings (incl. the new ones) now live under <code>i</code>. See <code>M-x describe-keymap image-slice-map</code>.</p><pre><code>Emacs now picks the correct coding-system for X input methods. Previously, Emacs would use 'locale-coding-system' for input methods, which could in some circumstances be incorrect, especially when the input method chose to fall back to some other coding system. Emacs now automatically detects the coding-system used by input methods, and uses that to decode input in preference to the value of 'locale-coding-system'. This unfortunately means that users who have changed the coding system used to decode X keyboard input must adjust their customizations to 'locale-coding-system' to the variable 'x-input-coding-system' instead.</code></pre><p class=" text-justify">Not sure how many people this is likely to affect, but fixing this is nice. I do feel the variable is a little bit mis-named now, as I would not ordinarily connect locale (be it in the i18n or unix LOCALE sense of the word) to my X windows keyboard input.</p><pre><code>Bookmarks no longer include context for encrypted files. If you're visiting an encrypted file, setting a bookmark no longer includes excerpts from that buffer in the bookmarks file. This is implemented by the new hook 'bookmark-inhibit-context-functions', where packages can register a function which returns non-nil for file names to be excluded from adding such excerpts.</code></pre><p class=" text-justify">Heh. Whoops. It’s easy to see how it could happen due to how bookmarking is implemented in Emacs. The perils of having so many disparate and tangled systems all working together in (mostly) harmony.</p><pre><code>'show-paren-mode' is now disabled in 'special-mode' buffers. In Emacs versions previous to Emacs 28.1, 'show-paren-mode' defaulted off. In Emacs 28.1, the mode was switched on in all buffers. In Emacs 29.1, this was changed to be switched on in all editing-related buffers, but not in buffers that inherit from 'special-mode'. To go back to how things worked in Emacs 28.1, put the following in your init file: (setopt show-paren-predicate t)</code></pre><p class=" text-justify">Unlikely to bother too many people in <em>theory</em> when you read this, but it also affects Help and Info buffers! So if you want it back in those, you clearly have to re-set it for all special buffers.</p><pre><code>Explicitly-set read-only state is preserved when reverting a buffer. If you use the 'C-x C-q' command to change the read-only state of the buffer and then revert it, Emacs would previously use the file permission bits to determine whether the buffer should be read-only after reverting the buffer. Emacs now remembers the decision made in 'C-x C-q'.</code></pre><p class=" text-justify">That’s a handy fix for me if you also use TRAMP or such-like to view files that you only have read permissions to look at. Now you can have auto revert on and yet still apply things like <code>M-x flush-lines</code> without losing the read only flag on revert.</p><pre><code>The Gtk selection face is no longer used for the region. The combination of a Gtk-controlled background and a foreground color controlled by the internal Emacs machinery led to low-contrast faces in common default setups. Emacs now uses the same 'region' face on Gtk and non-Gtk setups.</code></pre><p class=" text-justify">Never spotted this issue myself, though it’s nice to see these little burrs get filed off, one by one.</p><pre><code>'C-h f' and 'C-h x' may now require confirmation when you press 'RET'. If the text in the minibuffer cannot be completed to a single function or command, typing 'RET' will not automatically complete to the shortest candidate, but will instead ask for confirmation. Typing 'TAB' will complete as much as possible, and another 'TAB' will show all the possible completions. This allows you to insist on the functions name even if Help doesn't appear to know about it, by confirming with a second 'RET'.</code></pre><p class=" text-justify">I’m unclear if this is a generic change to the completion system or if it’s limited to just those two commands.</p><pre><code>'w' ('dired-copy-filename-as-kill') has changed behavior. If there are several files marked, file names containing space and quote characters will be quoted "like this".</code></pre><p class=" text-justify">Good. I’ve been caught out by that bug before.</p><pre><code>The 'd' command now more consistently skips dot files. In previous Emacs versions, commands like 'C-u 10 d' would put the "D" mark on the next ten files, no matter whether they were dot files (i.e., "." and "..") or not, while marking the next ten lines with the mouse (in 'transient-mark-mode') and then hitting 'd' would skip dot files. These now work equivalently.</code></pre><p class=" text-justify">Another nice fix. Imagine the panic of seeing <code>..</code> or <code>.</code> marked D after you’ve triggered the delete action…</p><pre><code>'/ a' in "*Packages*" buffer now limits by archive name(s) instead of regexp.</code></pre><p class=" text-justify">I don’t have any strong feelings about this. Is regexp search ever really useful in a repository where you’re looking for things by name? Word stemming and other such techniques would be a more useful feature than regexp. I can’t imagine too many people mind this.</p><pre><code>Setting the goal columns now also affects '&lt;prior&gt;' and '&lt;next&gt;'. Previously, 'C-x C-n' only affected 'next-line' and 'previous-line', but it now also affects 'scroll-up-command' and 'scroll-down-command'.</code></pre><p class=" text-justify">Goal columns change the default column offset point is placed at on a line. Set it to 10, and point is set at the 10th column position when you move up or down a line. All this does is make it so your page up / down keys also respect it.</p><p class=" text-justify">(You can unset a goal column with <code>C-0 C-x C-n</code>.)</p><pre><code>Isearch in "*Help*" and "*info*" now char-folds quote characters by default. This means that you can say 'C-s `foo' (GRAVE ACCENT) if the buffer contains "‘foo" (LEFT SINGLE QUOTATION MARK) and the like. These quotation characters look somewhat similar in some fonts. To switch this off, disable the new 'isearch-fold-quotes-mode' minor mode.</code></pre><p class=" text-justify">This is useful outside those buffers, of course. If you work with prose where the unicode glyphs are common, you can instead search for their elementary ASCII cousins instead and still match. Turn it on – why not?</p><pre><code>Sorting commands no longer necessarily change modification status. In earlier Emacs versions, commands like 'sort-lines' would always change buffer modification status to "modified", whether they changed something in the buffer or not. This has been changed: the buffer is marked as modified only if the sorting ended up actually changing the contents of the buffer.</code></pre><p class=" text-justify">You can still toggle a buffer’s modification manually with <code>M-~</code> though.</p><pre><code>'string-lines' handles trailing newlines differently. It no longer returns an empty final string if the string ends with a newline.</code></pre><pre><code>'TAB' and '&lt;backtab&gt;' are now bound in 'button-map'. This means that if point is on a button, 'TAB' will take you to the next button, even if the mode has bound it to something else. This also means that 'TAB' on a button in an 'outline-minor-mode' heading will move point instead of collapsing the outline.</code></pre><p class=" text-justify">Nice change. Buttons aren’t just <em>buttons</em>, they are often clickable things generally speaking. Hyperlinks often also fall under this category.</p><pre><code>'outline-minor-mode-cycle-map' is now parent of 'outline-minor-mode'. Instead of adding text property 'keymap' with 'outline-minor-mode-cycle' on outline headings in 'outline-minor-mode', the keymap 'outline-minor-mode-cycle' is now active in the whole buffer. But keybindings in 'outline-minor-mode-cycle' still take effect only on outline headings because they are bound with the help of 'outline-minor-mode-cycle--bind' that checks if point is on a heading.</code></pre><p class=" text-justify">Mechanical change that is unlikely to affect day-to-day use if you use outline. I suspect this is mostly done to make it easier to search keymaps or perhaps fix some obscure keyboard events that don’t activate property keymaps.</p><pre><code>'Info-default-directory-list' is no longer populated at Emacs startup. If you have code in your init file that removes directories from 'Info-default-directory-list', this will no longer work.</code></pre><pre><code>'C-k' no longer deletes files in 'ido-mode'. To get the previous action back, put something like the following in your Init file: (require 'ido) (keymap-set ido-file-completion-map "C-k" #'ido-delete-file-at-head)</code></pre><p class=" text-justify">Terrifying default, if you ask me. Good riddance.</p><pre><code>New user option 'term-clear-full-screen-programs'. By default, term.el will now work like most terminals when displaying full-screen programs: When they exit, the output is cleared, leaving what was displayed in the window before the programs started. Set this user option to nil to revert back to the old behavior.</code></pre><p class=" text-justify">As always I recommend you explore alternatives to term. See <a href="/article/running-shells-in-emacs-overview" class=" article-link">Running Shells and Terminal Emulators in Emacs</a>,</p><pre><code>Support for old EIEIO functions is not autoloaded any more. You need an explicit '(require 'eieio-compat)' to use 'defmethod' and 'defgeneric' (which were made obsolete in Emacs 25.1 by 'cl-defmethod' and 'cl-defgeneric'). Similarly you might need to '(require 'eieio-compat)' before loading files that were compiled with an old EIEIO (Emacs&lt;25).</code></pre><p class=" text-justify">Mostly of interest to people maintaining libraries.</p><pre><code>'C-x 8 .' has been moved to 'C-x 8 . .'. This is to open up the 'C-x 8 .' map to bind further characters there.</code></pre><p class=" text-justify">Unlikely to affect a great many people.</p><pre><code>'C-x 8 =' has been moved to 'C-x 8 = ='. You can now use 'C-x 8 =' to insert several characters with macron; for example, 'C-x 8 = a' will insert U+0101 LATIN SMALL LETTER A WITH MACRON. To insert a lone macron, type 'C-x 8 = =' instead of the previous 'C-x ='.</code></pre><p class=" text-justify">It’s just another keymap with a range of characters.</p><H2 id="eshell">Eshell</H2><p class=" text-justify">Eshell is Emacs’s builtin shell written entirely in Elisp. It’s, like, totally awesome. See <a href="/article/complete-guide-mastering-eshell" class=" article-link">Mastering Eshell</a>.</p><pre><code>Eshell's PATH is now derived from 'exec-path'. For consistency with remote connections, Eshell now uses 'exec-path' to determine the execution path on the local or remote system, instead of using the PATH environment variable directly.</code></pre><p class=" text-justify">Should aid with using TRAMP from Eshell.</p><pre><code>'source' and '.' no longer accept the '--help' option. This is for compatibility with the shell versions of these commands, which don't handle options like '--help' in any special way.</code></pre><pre><code>String delimiters in argument predicates/modifiers are more restricted. Previously, some argument predicates/modifiers allowed arbitrary characters as string delimiters. To provide more unified behavior across all predicates/modifiers, the list of allowed delimiters has been restricted to "...", '...', /.../, |...|, (...), [...], &lt;...&gt;, and {...}. See the "(eshell) Argument Predication and Modification" node in the Eshell manual for more details.</code></pre><p class=" text-justify">Seems like a sensible change, though I wonder if there are any instances where this limitation won’t work?</p><pre><code>Eshell pipelines now only pipe stdout by default. To pipe both stdout and stderr, use the '|&amp;' operator instead of '|'.</code></pre><p class=" text-justify">That is in keeping with the behavior of other shells, so that’s good.</p><pre><code>The 'delete-forward-char' command now deletes by grapheme clusters. This command is by default bound to the '&lt;Delete&gt;' function key (a.k.a. '&lt;deletechar&gt;'). When invoked without a prefix argument or with a positive prefix numeric argument, the command will now delete complete grapheme clusters produced by character composition. For example, if point is before an Emoji sequence, pressing '&lt;Delete&gt;' will delete the entire sequence, not just a single character at its beginning.</code></pre><p class=" text-justify">Interesting change, but if you use <code>C-d</code> then know that it calls <code>delete-char</code> and not <code>delete-forward-char</code>. Consider rebinding it if you want this feature.</p><p class=" text-justify">If you regularly work with languages that make use of grapheme clusters – or if you’re a Javascript developer and write purely with Emoji characters – then this is most likely a useful feature for you.</p><pre><code>'load-history' does not treat autoloads specially any more. An autoload definition appears just as a '(defun . NAME)' and the '(t . NAME)' entries are not generated any more.</code></pre><p class=" text-justify">This is of little to no impact to most users.</p><pre><code>The Tamil input methods no longer insert Tamil digits. The input methods 'tamil-itrans' and 'tamil-inscript' no longer insert the Tamil digits, as those digit characters are not used nowadays by speakers of the Tamil language. To get back the previous behavior, use the new 'tamil-itrans-digits' and 'tamil-inscript-digits' input methods instead.</code></pre><pre><code>New variable 'current-time-list' governing default timestamp form. Functions like 'current-time' now yield '(TICKS . HZ)' timestamps if this new variable is nil. The variable defaults to t, which means these functions default to timestamps of the forms '(HI LO US PS)', '(HI LO US)' or '(HI LO)', which are less regular and less efficient. This is part of a long-planned change first documented in Emacs 27. Developers are encouraged to test timestamp-related code with this variable set to nil, as it will default to nil in a future Emacs version and will be removed some time after that.</code></pre><pre><code>Functions that recreate the "*scratch*" buffer now also initialize it. When functions like 'other-buffer' and 'server-execute' recreate "*scratch*", they now also insert 'initial-scratch-message' and set the major mode according to 'initial-major-mode', like at Emacs startup. Previously, these functions ignored 'initial-scratch-message' and left "*scratch*" in 'fundamental-mode'.</code></pre><p class=" text-justify">Heh, you know, whenever I’ve had to recreate that buffer myself I always found I missed the usual scratch buffer blurb.</p><pre><code>Naming of Image-Dired thumbnail files has changed. Names of thumbnail files generated when 'image-dired-thumbnail-storage' is 'image-dired' now always end in ".jpg". This fixes various issues on different platforms, but means that thumbnails generated in Emacs 28 will not be used in Emacs 29, and vice-versa. If disk space is an issue, consider deleting the 'image-dired-dir' directory (usually "~/.emacs.d/image-dired/") after upgrading to Emacs 29.</code></pre><p class=" text-justify">Or you could rename them yourself. See <a href="/article/working-multiple-files-dired" class=" article-link">Working with multiple files in dired</a> and <a href="/article/wdired-editable-dired-buffers" class=" article-link">WDired: Editable Dired Buffers</a>.</p><pre><code>The 'rlogin' method in the URL library is now obsolete. Emacs will now display a warning if you request a URL like "rlogin://foo@example.org".</code></pre><pre><code>Setting 'url-gateway-method' to 'rlogin' is now obsolete. Emacs will now display a warning when setting it to that value. The user options 'url-gateway-rlogin-host', 'url-gateway-rlogin-parameters', and 'url-gateway-rlogin-user-name' are also obsolete.</code></pre><pre><code>The rlogin.el library, and the 'rsh' command are now obsolete. Use something like 'M-x shell RET ssh &lt;host&gt; RET' instead.</code></pre><p class=" text-justify">Rlogin is a series of, putting it mildly, insecure legacy protocols from the disco era of software. They’re long-abandoned and consigned to the history books. So, I mean, yeah, sure, rlogin is awful, but… that horse has long bolted, had millennial kids and is comfortably on its way to retirement. So I think Emacs is a bit late to the party here.</p><pre><code>The user function 'url-irc-function' now takes a SCHEME argument. The user option 'url-irc-function' is now called with a sixth argument corresponding to the scheme portion of the target URL. For example, this would be "ircs" for a URL like "ircs://irc.libera.chat".</code></pre><pre><code>The linum.el library is now obsolete. We recommend using either the built-in 'display-line-numbers-mode', or the 'nlinum' package from GNU ELPA instead. The former has better performance, but the latter is closer to a drop-in replacement. 1. To use 'display-line-numbers-mode', add something like this to your init file: (global-display-line-numbers-mode 1) ;; Alternatively, to use it only in programming modes: (add-hook 'prog-mode-hook #'display-line-numbers-mode) 2. To use 'nlinum', add this to your Init file: (package-install 'nlinum) (global-nlinum-mode 1) ;; Alternatively, to use it only in programming modes: (add-hook 'prog-mode-hook #'nlinum-mode) 3. To continue using the obsolete package 'linum', add this line to your Init file, in addition to any existing customizations: (require 'linum)</code></pre><p class=" text-justify">The “new” (it debuted in Emacs 26) line numbers mode is excellent and far and away the best way to show line numbers in Emacs. I can’t imagine too many are sad to see linum get obsoleted.</p><pre><code>The thumbs.el library is now obsolete. We recommend using the 'image-dired' command instead.</code></pre><pre><code>The autoarg.el library is now marked obsolete. This library provides the 'autoarg-mode' and 'autoarg-kp-mode' minor modes to emulate the behavior of the historical editor Twenex Emacs. We believe it is no longer useful.</code></pre><p class=" text-justify">Interesting historical curiosity that lives on in Emacs, now in an obsolete state. Obsolete in Emacs usually means that it’ll get removed <em>eventually</em> or, indeed, never; it comes down to how many flare-ups there are on the mailing list.</p><p class=" text-justify">Still, Emacs’s many dusty corners remain, to me, a fun time capsule and a look back at how things were. We shouldn’t rush into removing things just because they’re mostly forgotten.</p><pre><code>The quickurl.el library is now obsolete. Use 'abbrev', 'skeleton' or 'tempo' instead.</code></pre><p class=" text-justify">Yet another snippet completion tool. Abbrev is useful for <a href="/article/correcting-typos-misspellings-abbrev" class=" article-link">correcting typos and misspellings</a>, and Skeleton and Tempo are two peas in a pod that offer textual expansion using a simple s-expression-based language.</p><pre><code>The url-about.el library is now obsolete.</code></pre><p class=" text-justify">It seems to generate an HTML page with a list of supported URL schemes. There’s no commentary nor any real version history to accompany it. its actual use is a mystery to me.</p><pre><code>The autoload.el library is now obsolete. It is superseded by the new loaddefs-gen.el library.</code></pre><pre><code>The netrc.el library is now obsolete. Use the 'auth-source-netrc-parse-all' function in auth-source.el instead.</code></pre><pre><code>The url-dired.el library is now obsolete.</code></pre><pre><code>The fast-lock.el and lazy-lock.el libraries have been removed. They have been obsolete since Emacs 22.1. The variable 'font-lock-support-mode' is occasionally useful for debugging purposes. It is now a regular variable (instead of a user option) and can be set to nil to disable Just-in-time Lock mode.</code></pre><p class=" text-justify">Neither are likely to be missed by anyone. Lazy locking and Fast Locking were both attempts to speed up Emacs’s font locking facilities using a number of heuristics and obtuse variables. They’re gone, but the arcana lives on in Emacs’s Just-in-Time font locking engine.</p><pre><code>The 'utf-8-auto' coding-system now produces BOM on encoding. This is actually a bugfix, since this is how 'utf-8-auto' was documented from day one; it just didn't behave according to documentation. It turns out some Lisp programs were using this coding-system on the wrong assumption that the "auto" part means some automagic handling of the end-of-line (EOL) format conversion; those programs will now start to fail, because BOM signature in UTF-8 encoded text is rarely expected. That is the reason we mention this bugfix here. In general, this coding-system should probably never be used for encoding, only for decoding.</code></pre><H2 id="changes-in-emacs-29.1">Changes in Emacs 29.1</H2><pre><code>New user option 'major-mode-remap-alist' to specify favorite major modes. This user option lets you remap the default modes (e.g. 'perl-mode' or 'latex-mode') to your favorite ones (e.g. 'cperl-mode' or 'LaTeX-mode') instead of having to use 'defalias', which can have undesirable side effects. This applies to all modes specified via 'auto-mode-alist', file-local variables, etc.</code></pre><p class=" text-justify"><em>Bit</em> of a hack. Essentially, it’s a “remap” facility for major modes, akin to <code>defalias</code> which would not work here. Use it, as it says above, to swap one major mode for another.</p><p class=" text-justify">However, and it forgets to say this explicitly, its primary use case is for tree-sitter-enabled major modes as they are separate from their non-TS kin; use this variable to rewire them. See <a href="/article/how-to-get-started-tree-sitter" class=" article-link">my article on getting started with tree-sitter</a> for an example of how to use it.</p><p class=" text-justify">Note that this variable only remaps the major mode command – it does <em>not</em> fix your mode hooks or other settings!</p><pre><code>Emacs now supports Unicode Standard version 15.0.</code></pre><p class=" text-justify"><a href="https://www.unicode.org/versions/Unicode15.0.0/" class=" article-link">Unicode 15</a> adds:</p><blockquote><p class=" text-justify">Unicode 15.0 adds 4,489 characters, for a total of 149,186 characters. These additions include 2 new scripts, for a total of 161 scripts, along with 20 new emoji characters, and 4,193 CJK (Chinese, Japanese, and Korean) ideographs.</p></blockquote><pre><code>New user option 'electric-quote-replace-consecutive'. This allows you to disable the default behavior of consecutive single quotes being replaced with a double quote.</code></pre><p class=" text-justify">This is referring to <code>electric-pair[-local]-mode</code>, the built-in pairing tool for characters. You’re <em>probably</em> using it already in newer Emacsen.</p><pre><code>Emacs is now capable of editing files with very long lines. The display of long lines has been optimized, and Emacs should no longer choke when a buffer on display contains long lines. The variable 'long-line-threshold' controls whether and when these display optimizations are in effect. A companion variable 'large-hscroll-threshold' controls when another set of display optimizations are in effect, which are aimed specifically at speeding up display of long lines that are truncated on display. If you still experience slowdowns while editing files with long lines, this may be due to line truncation, or to one of the enabled minor modes, or to the current major mode. Try turning off line truncation with 'C-x x t', or try disabling all known slow minor modes with 'M-x so-long-minor-mode', or try disabling both known slow minor modes and the major mode with 'M-x so-long-mode', or visit the file with 'M-x find-file-literally' instead of the usual 'C-x C-f'. In buffers in which these display optimizations are in effect, the 'fontification-functions', 'pre-command-hook' and 'post-command-hook' hooks are executed on a narrowed portion of the buffer, whose size is controlled by the variables 'long-line-optimizations-region-size' and 'long-line-optimizations-bol-search-limit', as if they were in a 'with-restriction' form. This may, in particular, cause occasional mis-fontifications in these buffers. Modes which are affected by these optimizations and by the fact that the buffer is narrowed, should adapt and either modify their algorithm so as not to expect the entire buffer to be accessible, or, if accessing outside of the narrowed region doesn't hurt performance, use the 'without-restriction' form to temporarily lift the restriction and access portions of the buffer outside of the narrowed region. The new function 'long-line-optimizations-p' returns non-nil when these optimizations are in effect in the current buffer.</code></pre><p class=" text-justify">So long, <code>so-long-mode</code>. Having said that, I’m still getting coughs and sputters on long lines, on occasion. But it’s a marked improvement over older Emacsen. You should follow the advice given above and experiment to find the right settings for you.</p><pre><code>New command to change the font size globally. To increase the font size, type 'C-x C-M-+' or 'C-x C-M-='; to decrease it, type 'C-x C-M--'; to restore the font size, type 'C-x C-M-0'. The final key in these commands may be repeated without the leading 'C-x' and without the modifiers, e.g. 'C-x C-M-+ C-M-+ C-M-+' and 'C-x C-M-+ + +' increase the font size by three steps. When 'mouse-wheel-mode' is enabled, 'C-M-wheel-up' and 'C-M-wheel-down' also increase and decrease the font size globally. Additionally, the user option 'global-text-scale-adjust-resizes-frames' controls whether the frames are resized when the font size is changed.</code></pre><p class=" text-justify">You can already use <code>C-x C-=</code> and friends to adjust a single buffer, so it’s nice to see it extended to all of Emacs. Great for projectors and screen sharing.</p><pre><code>New config variable 'syntax-wholeline-max' to reduce the cost of long lines. This variable is used by some operations (mostly syntax-propertization and font-locking) to treat lines longer than this variable as if they were made up of various smaller lines. This can help reduce the slowdowns seen in buffers made of a single long line, but can also cause misbehavior in the presence of such long lines (though most of that misbehavior should usually be limited to mis-highlighting). You can recover the previous behavior with: (setq syntax-wholeline-max most-positive-fixnum)</code></pre><p class=" text-justify">Definitely worth playing with if you have performance issues. It’s set to 10,000 which I think sounds like a reasonable default. Syntax here refers to a huge range of commands, many of them you probably don’t know of, like <code>parse-partial-sexp</code> and <code>syntax-ppss</code>. Needless to say, everything from font locking to navigation and editing are affected in one way or another.</p><pre><code>New bindings in 'find-function-setup-keys' for 'find-library'. When 'find-function-setup-keys' is enabled, 'C-x L' is now bound to 'find-library', 'C-x 4 L' is now bound to 'find-library-other-window' and 'C-x 5 L' is now bound to 'find-library-other-frame'.</code></pre><p class=" text-justify">Okay, so the wording’s a bit diffuse. When you <a href="/article/evaluating-elisp-emacs" class=" article-link">evaluate</a> <code>(find-function-setup-keys)</code> it’ll bind a bunch of key bindings in the <code>C-x</code>, <code>C-x 4</code> and <code>C-x 5</code> keymaps.</p><p class=" text-justify">I didn’t even <em>know</em> about this feature. Turns out Stallman added it in 1998. Huh. The more you know. So yeah, if execute that function you’ll get some handy key bindings to jump to functions and now libraries.</p><pre><code>New key binding after 'M-x' or 'M-X': 'M-X'. Emacs allows different completion predicates to be used with 'M-x' (i.e., 'execute-extended-command') via the 'read-extended-command-predicate' user option. Emacs also has the 'M-X' (note upper case X) command, which only displays commands especially relevant to the current buffer. Emacs now allows toggling between these modes while the user is inputting a command by hitting 'M-X' while in the minibuffer.</code></pre><p class=" text-justify">Nice change. The <code>M-X</code> (note case) key binding filters the list of commands to the ones deemed “relevant” to your active buffer. It’s a nifty feature and a nice way to discover new things.</p><pre><code>Interactively, 'kill-buffer' will now offer to save the buffer if unsaved.</code></pre><p class=" text-justify">Not much to say here: good.</p><pre><code>New commands 'duplicate-line' and 'duplicate-dwim'. 'duplicate-line' duplicates the current line the specified number of times. 'duplicate-dwim' duplicates the region if it is active. If not, it works like 'duplicate-line'. An active rectangular region is duplicated on its right-hand side. The new user option 'duplicate-line-final-position' specifies where to move point after duplicating a line.</code></pre><p class=" text-justify">Apparently one of the ‘meme arguments’ for why Vi(m) is better, as it has the option to easily do this. Well, good news, Vim users: you can now prostrate yourself before Emacs.</p><p class=" text-justify">Now annoyingly, they didn’t bind them to a key. (Why?). Let’s fix that real quick:</p><p class=" text-justify">:</p><pre><code>(global-set-key (kbd "C-x j") #'duplicate-dwim)</code></pre><p class=" text-justify"><code>C-x j</code> seems free; we’ll wedge it in there.</p><pre><code>Files with the ".eld" extension are now visited in 'lisp-data-mode'.</code></pre><pre><code>'network-lookup-address-info' can now check numeric IP address validity. Specifying 'numeric' as the new optional HINTS argument makes it check if the passed address is a valid IPv4/IPv6 address (without DNS traffic). (network-lookup-address-info "127.1" 'ipv4 'numeric) =&gt; ([127 0 0 1 0])</code></pre><p class=" text-justify">Speaking of network stuff. Did you know that Emacs <a href="/article/network-utilities-emacs" class=" article-link">has a host of wrappers for commandline network utilities</a>?</p><pre><code>New command 'find-sibling-file'. This command jumps to a file considered a "sibling file", which is determined according to the new user option 'find-sibling-rules'.</code></pre><p class=" text-justify">No word on whether this is meant to replace – or indeed, augment – the existing <code>M-x ffap</code> (find file at point) that tries to do much the same. There’s also <code>M-x ff-find-related-file</code> which works well indeed.</p><p class=" text-justify">It’s worth noting that both of these options come with a laundry list of heuristics already, such as finding header files belonging to a source file.</p><p class=" text-justify">The <code>find-sibling-rules</code> variable, meanwhile, is empty.</p><p class=" text-justify">Mmm… there’s a missed opportunity here to merge this stuff.</p><pre><code>New user option 'delete-selection-temporary-region'. When non-nil, 'delete-selection-mode' will only delete the temporary regions (usually set by mouse-dragging or shift-selection).</code></pre><p class=" text-justify">Actually, this is a neat little feature. If you use the mark commands to select stuff, like <code>C-M-SPC</code> or <code>M-@</code>, then you’d delete the text if you typed a character. With this option, it only happens if you select by some other means.</p><pre><code>New user option 'switch-to-prev-buffer-skip-regexp'. This should be a regexp or a list of regexps; buffers whose names match those regexps will be ignored by 'switch-to-prev-buffer' and 'switch-to-next-buffer'.</code></pre><p class=" text-justify">If you use either of these commands (<code>C-x C-&lt;right/left&gt;</code>) to navigate, you’d probably want to add all the flotsam and jetsam you don’t want to switch to.</p><pre><code>New command 'rename-visited-file'. This command renames the file visited by the current buffer by moving it to a new name or location, and also makes the buffer visit this new file.</code></pre><p class=" text-justify">Yep. Emacs has never had the ability to rename a file <em>in-situ</em> from the buffer. Until someone pointed that out to me I thought it did. But then I remember I rename everything in dired — and seemingly so did everyone else who used Emacs up until now when that command was added.</p><p class=" text-justify">A long, over-due command.</p><H2 id="menus">Menus</H2><pre><code>The entries following the buffers in the "Buffers" menu can now be altered. Change the 'menu-bar-buffers-menu-command-entries' variable to alter the entries that follow the buffer list.</code></pre><p class=" text-justify">You can customize Emacs’s menu entries by altering this variable.</p><pre><code>'delete-process' is now a command. When called interactively, it will kill the process running in the current buffer (if any). This can be useful if you have runaway output in the current buffer (from a process or a network connection), and want to stop it.</code></pre><p class=" text-justify">Very useful. You should know about <code>M-x list-processes</code> also.</p><pre><code>New command 'restart-emacs'. This is like 'save-buffers-kill-emacs', but instead of just killing the current Emacs process at the end, it starts a new Emacs process (using the same command line arguments as the running Emacs process). 'kill-emacs' and 'save-buffers-kill-emacs' have also gained new optional arguments to restart instead of just killing the current process.</code></pre><p class=" text-justify">Nice to have.</p><H2 id="drag-and-drop">Drag and Drop</H2><p class=" text-justify">Just as a quick side note: you can drag and drop a bunch of stuff into Emacs and have it do the right thing. Files get opened, for instance, if you drag them into Emacs. If you drag them into message buffer you’ll instead <em>attach</em> them as a file. Great if you do email in Emacs like I do.</p><pre><code>New user option 'mouse-drag-mode-line-buffer'. If non-nil, dragging on the buffer name part of the mode-line will drag the buffer's associated file to other programs. This option is currently only available on X, Haiku and Nextstep (GNUstep or macOS).</code></pre><p class=" text-justify">Neat. No way I’ll ever remember that, but it’s a nice addition.</p><pre><code>New user option 'mouse-drag-and-drop-region-cross-program'. If non-nil, this option allows dragging text in the region from Emacs to another program.</code></pre><p class=" text-justify">That I <em>will</em> remember though.</p><pre><code>New user option 'mouse-drag-and-drop-region-scroll-margin'. If non-nil, this option allows scrolling a window while dragging text around without a scroll wheel.</code></pre><p class=" text-justify">Another one of those features people who move to Emacs come to depend on.</p><pre><code>The value of 'mouse-drag-copy-region' can now be the symbol 'non-empty'. This prevents mouse drag gestures from putting empty strings onto the kill ring.</code></pre><p class=" text-justify">You’d probably want to set this to t.</p><pre><code>New user options 'dnd-indicate-insertion-point' and 'dnd-scroll-margin'. These options allow adjusting point and scrolling a window when dragging items from another program.</code></pre><pre><code>The X Direct Save (XDS) protocol is now supported. This means dropping an image or file link from programs such as Firefox will no longer create a temporary file in a random directory, instead asking you where to save the file first.</code></pre><p class=" text-justify">Never heard of this protocol before, but it reads like a fine improvement over the copy-to-temp approach Emacs employed before.</p><pre><code>New user option 'record-all-keys'. If non-nil, this option will force recording of all input keys, including those typed in response to passwords prompt (this was the previous behavior). The default is nil, which inhibits recording of passwords.</code></pre><p class=" text-justify">Keep in mind Emacs still records most stuff you type. See the lossage for an example: <code>C-h l</code>.</p><pre><code>New function 'command-query'. This function makes its argument command prompt the user for confirmation before executing.</code></pre><pre><code>The 'disabled' property of a command's symbol can now be a list. The first element of the list should be the symbol 'query', which will cause the command disabled this way prompt the user with a y/n or a yes/no question before executing. The new function 'command-query' is a convenient method of making commands disabled in this way.</code></pre><p class=" text-justify">There are a handful of commands marked disabled in Emacs as they’re confusing to beginners. You might’ve run into the prompts when you typed one of them.</p><pre><code>'count-words' will now report buffer totals if given a prefix. Without a prefix, it will only report the word count for the narrowed part of the buffer.</code></pre><pre><code>'count-words' will now report sentence count when used interactively.</code></pre><p class=" text-justify">Cue the interminable arguments over what constitutes a sentence.</p><pre><code>New user option 'set-message-functions'. It allows more flexible control of how echo-area messages are displayed by adding functions to this list. The default value is a list of one element: 'set-minibuffer-message', which displays echo-area messages at the end of the minibuffer text when the minibuffer is active. Other useful functions include 'inhibit-message', which allows specifying, via 'inhibit-message-regexps', the list of messages whose display should be inhibited; and 'set-multi-message' that accumulates recent messages and displays them stacked together.</code></pre><p class=" text-justify">Nifty new features. The echo area is a busy area now, particularly if you’re using a lot of tools, all vying for your attention. Maybe give the multi-message approach a shot if you’re missing messages? I wrote about a similar problem with Eldoc and competing messages. <a href="/article/seamlessly-merge-multiple-documentation-sources-eldoc" class=" article-link">Seamlessly Merge Multiple Documentation Sources with Eldoc</a>.</p><pre><code>New user option 'find-library-include-other-files'. If set to nil, commands like 'find-library' will only include library files in the completion candidates. The default is t, which preserves previous behavior, whereby non-library files could also be included.</code></pre><pre><code>New command 'sqlite-mode-open-file' for examining an sqlite3 file. This uses the new 'sqlite-mode' which allows listing the tables in a DB file, and examining and modifying the columns and the contents of those tables.</code></pre><p class=" text-justify">Nifty!</p><pre><code>'write-file' will now copy some file mode bits. If the current buffer is visiting a file that is executable, the 'C-x C-w' command will now make the new file executable, too.</code></pre><p class=" text-justify">Along the same lines, you should consider the switch to <a href="/article/script-files-executable-automatically" class=" article-link">make script files executable automatically</a>.</p><pre><code>New user option 'process-error-pause-time'. This determines how long to pause Emacs after a process filter/sentinel error has been handled.</code></pre><p class=" text-justify">If you regularly cancel out of compilation buffers or other processes and find the delay annoying, try setting it to 0.</p><pre><code>New faces for font-lock. These faces are primarily meant for use with tree-sitter. They are: 'font-lock-bracket-face', 'font-lock-delimiter-face', 'font-lock-escape-face', 'font-lock-function-call-face', 'font-lock-misc-punctuation-face', 'font-lock-number-face', 'font-lock-operator-face', 'font-lock-property-name-face', 'font-lock-property-use-face', 'font-lock-punctuation-face', 'font-lock-regexp-face', and 'font-lock-variable-use-face'.</code></pre><p class=" text-justify">This is part of the tree-sitter changes. As TS allows for far more precise selection of syntactic constructs, it made sense to add a bunch of faces.</p><p class=" text-justify">You’ll want to customize them to suit your needs.</p><pre><code>New face 'variable-pitch-text'. This face is like 'variable-pitch' (from which it inherits), but is slightly larger, which should help with the visual size differences between the default, non-proportional font and proportional fonts when mixed.</code></pre><pre><code>New face 'mode-line-active'. This inherits from the 'mode-line' face, but is the face actually used on the mode lines (along with 'mode-line-inactive').</code></pre><pre><code>New face attribute pseudo-value 'reset'. This value stands for the value of the corresponding attribute of the 'default' face. It can be used to reset attribute values produced by inheriting from other faces.</code></pre><p class=" text-justify">Useful if you want some, but not all, inheriting faces.</p><pre><code>New X resource "borderThickness". This controls the thickness of the external borders of the menu bars and pop-up menus.</code></pre><pre><code>New X resource "inputStyle". This controls the style of the pre-edit and status areas of X input methods.</code></pre><pre><code>New X resources "highlightForeground" and "highlightBackground". Only in the Lucid build, this controls colors used for highlighted menu item widgets.</code></pre><pre><code>On X, Emacs now tries to synchronize window resize with the window manager. This leads to less flicker and empty areas of a frame being displayed when a frame is being resized. Unfortunately, it does not work on some ancient buggy window managers, so if Emacs appears to freeze, but is still responsive to input, you can turn it off by setting the X resource "synchronizeResize" to "off".</code></pre><pre><code>On X, Emacs can optionally synchronize display with the graphics hardware. When this is enabled by setting the X resource "synchronizeResize" to "extended", frame content "tearing" is drastically reduced. This is only supported on the Motif, Lucid, and no-toolkit builds, and requires an X compositing manager supporting the extended frame synchronization protocol (see https://fishsoup.net/misc/wm-spec-synchronization.html). This behavior can be toggled on and off via the frame parameter 'use-frame-synchronization'.</code></pre><pre><code>New frame parameter 'alpha-background' and X resource "alphaBackground". This controls the opacity of the text background when running on a composited display.</code></pre><p class=" text-justify">Party like it’s 1998. Remember back when everyone took screenshots of their KDE/GNOME/WindowMaker desktops and they all had that obligatory semi-translucent terminal window?</p><pre><code>New frame parameter 'shaded'. With window managers which support this, it controls whether or not a frame's contents will be hidden, leaving only the title bar on display.</code></pre><pre><code>New user option 'x-gtk-use-native-input'. This controls whether or not GTK input methods are used by Emacs, instead of XIM input methods. Defaults to nil.</code></pre><pre><code>New user option 'use-system-tooltips'. This controls whether to use the toolkit tooltips, or Emacs's own native implementation of tooltips as small frames. This option is only meaningful if Emacs was built with GTK+, Nextstep, or Haiku support, and defaults to t, which makes Emacs use the toolkit tooltips. The existing GTK-specific option 'x-gtk-use-system-tooltips' is now an alias of this new option.</code></pre><p class=" text-justify">I long ago disabled tooltips and made them appear in the echo area instead:</p><pre><code>(tooltip-mode nil) (setq tooltip-use-echo-area t) (setq x-gtk-use-system-tooltips nil)</code></pre><pre><code>Non-native tooltips are now supported on Nextstep. This means Emacs built with GNUstep or built on macOS is now able to display different faces and images inside tooltips when the 'use-system-tooltips' user option is nil.</code></pre><pre><code>New minor mode 'pixel-scroll-precision-mode'. When enabled, and if your mouse supports it, you can scroll the display up or down at pixel resolution, according to what your mouse wheel reports. Unlike 'pixel-scroll-mode', this mode scrolls the display pixel-by-pixel, as opposed to only animating line-by-line scrolls.</code></pre><p class=" text-justify">Pixel-perfect scrolling, at least in theory. On my computer it does act funny around large images, though this is mostly a limitation of Emacs’s display engine, more than the pixel scrolling.</p><H2 id="terminal-emacs">Terminal Emacs</H2><pre><code>Emacs will now use 24-bit colors on terminals that support "Tc" capability. This is in addition to previously-supported ways of discovering 24-bit color support: either via the "RGB" or "setf24" capabilities, or if the 'COLORTERM' environment variable is set to the value "truecolor".</code></pre><p class=" text-justify">Emacs has had 24-bit support for years, so this is really only about catching the tail end of termcaps.</p><pre><code>Select active regions with xterm selection support. On terminals with xterm "setSelection" support, the active region may be saved to the X primary selection, following the 'select-active-regions' variable. This support is enabled when 'tty-select-active-regions' is non-nil.</code></pre><p class=" text-justify">Worth setting to t if you prefer running Emacs in a GUI terminal.</p><pre><code>New command to set up display of unsupported characters. The new command 'standard-display-by-replacement-char' produces Lisp code that sets up the 'standard-display-table' to use a replacement character for display of characters that the text-mode terminal doesn't support. This code is intended to be used in your init files. This feature is most useful with the Linux console and similar terminals, where Emacs has a reliable way of determining which characters have glyphs in the font loaded into the terminal's memory.</code></pre><p class=" text-justify">This is the default character to display if Emacs or you terminal cannot. If you have strong views on this, you should contemplate changing it.</p><pre><code>New functions to set terminal output buffer size. The new functions 'tty--set-output-buffer-size' and 'tty--output-buffer-size' allow setting and retrieving the output buffer size of a terminal device. The default buffer size is and has always been BUFSIZ, which is defined in your system's stdio.h. When you set a buffer size with 'tty--set-output-buffer-size', this also prevents Emacs from explicitly flushing the tty output stream, except at the end of display update.</code></pre><p class=" text-justify">I’ve no opinion nor any wisdom to share as to why you’d want to change this. Keen to hear from people who do though.</p><H2 id="ert">ERT</H2><p class=" text-justify">ERT is Emacs’s unit test framework.</p><pre><code>New ERT variables 'ert-batch-print-length' and 'ert-batch-print-level'. These variables will override 'print-length' and 'print-level' when printing Lisp values in ERT batch test results.</code></pre><pre><code>Redefining an ERT test in batch mode now signals an error. Executing 'ert-deftest' with the same name as an existing test causes the previous definition to be discarded, which was probably not intended when this occurs in batch mode. To remedy the error, rename tests so that they all have unique names.</code></pre><pre><code>ERT can generate JUnit test reports. When environment variable 'EMACS_TEST_JUNIT_REPORT' is set, ERT generates a JUnit test report under this file name. This is useful for Emacs integration into CI/CD test environments.</code></pre><pre><code>Unbound test symbols now signal an 'ert-test-unbound' error. This affects the 'ert-select-tests' function and its callers.</code></pre><H2 id="emoji">Emoji</H2><p class=" text-justify">There’s a NEWS category just for Emoji now?</p><pre><code>Emacs now has several new methods for inserting Emoji. The Emoji commands are under the new 'C-x 8 e' prefix.</code></pre><p class=" text-justify">This is now the primary composition key for dealing with emoji.</p><pre><code>New command 'emoji-insert' (bound to 'C-x 8 e e' and 'C-x 8 e i'). This command guides you through various Emoji categories and combinations in a graphical menu system.</code></pre><p class=" text-justify"><code>C-x 8 e e</code> opens a <a href="/article/introduction-magit-emacs-mode-git" class=" article-link">Magit-style</a> popup using the now-builtin transient package. I think this is the first instance of using transient in Emacs?</p><pre><code>New command 'emoji-search' (bound to 'C-x 8 e s'). This command lets you search for and insert an Emoji based on names.</code></pre><p class=" text-justify">Like <code>C-x 8 RET</code>, but the search space limited to just emoji.</p><pre><code>New command 'emoji-list' (bound to 'C-x 8 e l'). This command lists all Emoji (categorized by themes) in a special buffer and lets you choose one of them to insert.</code></pre><p class=" text-justify">Rather useful overview of smileys in a buffer. This is my preferred method for picking out emoji; note that the emoji are hyperlinked will insert the smiley at point.</p><pre><code>New command 'emoji-recent' (bound to 'C-x 8 e r'). This command lets you choose among the Emoji you have recently inserted and insert it.</code></pre><pre><code>New command 'emoji-describe' (bound to 'C-x 8 e d'). This command will tell you the name of the Emoji at point. (It also works for non-Emoji characters.)</code></pre><p class=" text-justify">It prints the unicode descriptor for the character at point. It’s also available in <code>C-u C-x =</code>.</p><pre><code>New commands 'emoji-zoom-increase' and 'emoji-zoom-decrease'. These are bound to 'C-x 8 e +' and 'C-x 8 e -', respectively. They can be used on any character, but are mainly useful for Emoji.</code></pre><pre><code>New command 'emoji-zoom-reset'. This is bound to 'C-x 8 e 0', and undoes any size changes performed by 'emoji-zoom-increase' and 'emoji-zoom-decrease'.</code></pre><pre><code>New input method 'emoji'. This allows you to enter Emoji using short strings, eg ':face_palm:' or ':scream:'.</code></pre><p class=" text-justify">I think I wrote the first input method to insert Emoji. In fact, it was the upcoming emoji code in Emacs 29 that prompted me to blog about it: <a href="/article/inserting-emoji-input-methods" class=" article-link">Inserting Emoji with Input Methods</a>.</p><p class=" text-justify">Input methods are a nifty way of inserting chorded text. It’s a really pleasant way of inserting diacritics. If you’re unfamiliar with input methods, then check out <a href="/article/diacritics-in-emacs" class=" article-link">Olé! Diacritics in Emacs</a>.</p><H2 id="help">Help</H2><pre><code>Variable values displayed by 'C-h v' in "*Help*" are now fontified.</code></pre><p class=" text-justify">Yep. Big fan of this change.</p><pre><code>New user option 'help-clean-buttons'. If non-nil, link buttons in "*Help*" buffers will have any surrounding quotes removed.</code></pre><pre><code>'M-x apropos-variable' output now includes values of variables. Such an apropos buffer is more easily viewed with outlining after enabling 'outline-minor-mode' in 'apropos-mode'.</code></pre><p class=" text-justify">Nifty change indeed. The apropos commands in Emacs should not be slept on.</p><pre><code>New docstring syntax to indicate that symbols shouldn't be links. When displaying docstrings in "*Help*" buffers, strings that are "`like-this'" are made into links (if they point to a bound function/variable). This can lead to false positives when talking about values that are symbols that happen to have the same names as functions/variables. To inhibit this buttonification, use the new "\\+`like-this'" syntax.</code></pre><pre><code>New user option 'help-window-keep-selected'. If non-nil, commands to show the info manual and the source will reuse the same window in which the "*Help*" buffer is shown.</code></pre><p class=" text-justify">If you have strong preferences on where the windows should go, consider enabling this. Alternatively, my article on <a href="/article/demystifying-emacs-window-manager" class=" article-link">Demystifying Emacs’s Window Manager</a> will teach you how to tame window management in Emacs.</p><pre><code>Commands like 'C-h f' have changed how they describe menu bindings. For instance, previously a command might be described as having the following bindings: It is bound to &lt;open&gt;, C-x C-f, &lt;menu-bar&gt; &lt;file&gt; &lt;new-file&gt;. This has been changed to: It is bound to &lt;open&gt; and C-x C-f. It can also be invoked from the menu: File → Visit New File...</code></pre><pre><code>The 'C-h .' command now accepts a prefix argument. 'C-u C-h .' would previously inhibit displaying a warning message if there was no local help at point. This has been changed to call 'button-describe'/'widget-describe' and display button/widget help instead.</code></pre><pre><code>New user option 'help-enable-variable-value-editing'. If enabled, 'e' on a value in "*Help*" will pop you to a new buffer where you can edit the value. This is not enabled by default, because it is easy to make an edit that yields an invalid result.</code></pre><p class=" text-justify">There’s also <code>c</code> in a help window to open its customize window. That is perhaps of more use to most people.</p><pre><code>'C-h b' uses outlining by default. Set 'describe-bindings-outline' to nil to get back the old behavior.</code></pre><p class=" text-justify"><code>C-h b</code> is a great command to know about. It displays all the pertinent bindings in a buffer.</p><pre><code>Jumping to function/variable source now saves mark before moving point. Jumping to source from a "*Help*" buffer moves point when the source buffer is already open. Now, the old point is pushed onto mark ring.</code></pre><p class=" text-justify">The global/local mark rings are worth learning about. They’re useful beacons for moving around in Emacs and they are always precipitated by a user-led action.</p><pre><code>New key bindings in "*Help*" buffers: 'n' and 'p'. These will take you (respectively) to the next and previous "page".</code></pre><p class=" text-justify">Page here meaning the old-fashioned concept in LISP and Emacs where the FORM FEED (<code>^L</code>) character is used to denote a page in code. That’s what all the <code>xxx-page</code> command do; they operate on them.</p><pre><code>'describe-char' now also outputs the name of Emoji sequences.</code></pre><pre><code>New key binding in "*Help*" buffer: 'I'. This will take you to the Emacs Lisp manual entry for the item displayed, if any.</code></pre><p class=" text-justify">There is also <code>i</code> which goes to the manual; this, in turn, tries to find the elisp symbol reference.</p><pre><code>The 'C-h m' ('describe-mode') "*Help*" buffer has been reformatted. It now only includes local minor modes at the start, and the global minor modes are listed after the major mode.</code></pre><p class=" text-justify">I like this change, as this is most likely what you want to look at first, anyway.</p><pre><code>The user option 'help-window-select' now affects apropos commands. The apropos commands will now select the apropos window if 'help-window-select' is non-nil.</code></pre><p class=" text-justify">See above regarding windows.</p><pre><code>'describe-keymap' now considers the symbol at point. If the symbol at point is a keymap, 'describe-keymap' suggests it as the default candidate.</code></pre><p class=" text-justify">Always nice to see contextual completions. I wish more commands did this.</p><pre><code>New command 'help-quick' displays an overview of common commands. The command pops up a buffer at the bottom of the screen with a few helpful commands for various tasks. You can toggle the display using 'C-h C-q'.</code></pre><p class=" text-justify">This is cheat sheet for keyboard shortcuts. My view is still that you should use the menu bar until you’ve memorized the keys you’re ever likely to encounter in a cheat sheet.</p><pre><code>Emacs now comes with Org v9.6. See the file "etc/ORG-NEWS" for user-visible changes in Org.</code></pre><H2 id="outline-mode">Outline Mode</H2><p class=" text-justify">Outline is the progenitor to Org mode and works in much the same way. It is, of course, vastly simpler.</p><pre><code>Support for customizing the default visibility state of headings. Customize the user option 'outline-default-state' to define what headings will be visible initially, after Outline mode is turned on. When the value is a number, the user option 'outline-default-rules' determines the visibility of the subtree starting at the corresponding level. Values are provided to control showing a heading subtree depending on whether the heading matches a regexp, or on whether its subtree has long lines or is itself too long.</code></pre><H3 id="outline-minor-mode">Outline Minor Mode</H3><pre><code>New user option 'outline-minor-mode-use-buttons'. If non-nil, Outline Minor Mode will use buttons to hide/show outlines in addition to the ellipsis. The default is nil, but in 'help-mode' it has the value 'insert' that inserts the buttons directly into the buffer, and you can use 'RET' to cycle outline visibility. When the value is 'in-margins', Outline Minor Mode uses the window margins for buttons that hide/show outlines.</code></pre><pre><code>Buttons and headings now have their own keymaps. 'outline-button-icon-map', 'outline-overlay-button-map', and 'outline-inserted-button-map' are now available as defined keymaps instead of being anonymous keymaps.</code></pre><H2 id="windows">Windows</H2><p class=" text-justify">See my article on <a href="/article/demystifying-emacs-window-manager" class=" article-link">Demystifying Emacs’s Window Manager</a> to truly understand how all these window changes below work, and how you can use them. The subject is well beyond the scope of this blog entry.</p><pre><code>New commands 'split-root-window-below' and 'split-root-window-right'. These commands split the root window in two, and are bound to 'C-x w 2' and 'C-x w 3', respectively. A number of other useful window-related commands are now available with key sequences that start with the 'C-x w' prefix.</code></pre><p class=" text-justify">I am glad they added these. One of the things about Emacs’s window system is that you only ever operate within the part of the tree – Emacs’s window layout is stored in a simple tree structure – that your active window is in. So, splitting against the root window wasn’t possible without elisp.</p><pre><code>New display action 'display-buffer-full-frame'. This action removes other windows from the frame when displaying a buffer on that frame.</code></pre><p class=" text-justify">This is a <code>display-buffer</code> action that ensures a buffer is displayed in a new frame.</p><pre><code>'display-buffer' now can set up the body size of the chosen window. For example, a 'display-buffer-alist' entry of (window-width . (body-columns . 40)) will make the body of the chosen window 40 columns wide. For the height use 'window-height' and 'body-lines', respectively.</code></pre><p class=" text-justify">The body is the size of the window without the margins and all the other bits. You can already set the height/width of a window, but now you can also do it with just the body. Should allow for more fine-grained control.</p><pre><code>'display-buffer' provides more options for using an existing window. The display buffer action functions 'display-buffer-use-some-window' and 'display-buffer-use-least-recent-window' now honor the action alist entry 'window-min-height' as well as the entries listed below to make the display of several buffers in a row more amenable.</code></pre><p class=" text-justify">Excellent news. Placing a window is difficult for Emacs as it has to respect a range of constraints and criteria. And when it fails to do it the way you want or expect it to, it can easily lead to user frustration.</p><pre><code>New buffer display action alist entry 'lru-frames'. This allows specifying which frames 'display-buffer' should consider when using a window that shows another buffer. It is interpreted as per the ALL-FRAMES argument of 'get-lru-window'.</code></pre><p class=" text-justify">LRU meaning Least-Recently-Used.</p><pre><code>New buffer display action alist entry 'lru-time'. 'display-buffer' will ignore windows with a use time higher than this when using a window that shows another buffer.</code></pre><pre><code>New buffer display action alist entry 'bump-use-time'. This has 'display-buffer' bump the use time of any window it returns, making it a less likely candidate for displaying another buffer.</code></pre><p class=" text-justify">Taken together, you can use this to cycle buffers through windows using a combination of lru-time and bump-use-time. Now I’m curious to hear where this was first used.</p><pre><code>New buffer display action alist entry 'window-min-width'. This allows specifying a preferred minimum width of the window used to display a buffer.</code></pre><p class=" text-justify">Useful to help constrain where Emacs can put a buffer (or a window, for that matter) and it should hopefully prevent things from popping up in tiny little windows that won’t fit the buffer’s content.</p><pre><code>You can specify on which window 'scroll-other-window' operates. This is controlled by the new 'other-window-scroll-default' variable, which should be set to a function that returns a window. When this variable is nil, 'next-window' is used.</code></pre><p class=" text-justify">A hidden feature of Emacs is the ability to carry out a limited number of actions on other, nearby windows. <code>C-M-S-v</code> scrolls down the ‘other’ window (useful if you have two or more windows, one you’re editing and the other you’re reading along from.)</p><p class=" text-justify">But now you can seemingly customize how this ‘other’ window is chosen.</p><H2 id="frames">Frames</H2><pre><code>Deleted frames can now be undeleted. The 16 most recently deleted frames can be undeleted with 'C-x 5 u' when 'undelete-frame-mode' is enabled. Without a prefix argument, undelete the most recently deleted frame. With a numerical prefix argument between 1 and 16, where 1 is the most recently deleted frame, undelete the corresponding deleted frame.</code></pre><p class=" text-justify">Neat! I’m sure we’ve all closed a frame by accident.</p><pre><code>The variable 'icon-title-format' can now have the value t. That value means to use 'frame-title-format' for iconified frames. This is useful with some window managers and desktop environments which treat changes in frame's title as requests to raise the frame and/or give it input focus, or if you want the frame's title to be the same no matter if the frame is iconified or not.</code></pre><H2 id="tab-bars-and-tab-lines">Tab Bars and Tab Lines</H2><p class=" text-justify">Tab <em>bars</em> are window configurations; custom set pieces for how you want to arrange your windows. Tab <em>lines</em> are just a tabbed list of buffers.</p><pre><code>New user option 'tab-bar-auto-width' to automatically determine tab width. This option is non-nil by default, which resizes tab-bar tabs so that their width is evenly distributed across the tab bar. A companion option 'tab-bar-auto-width-max' controls the maximum width of a tab before its name on display is truncated.</code></pre><p class=" text-justify">If you want dynamic sizing and dislike truncated names, try it out.</p><pre><code>'C-x t RET' creates a new tab when the provided tab name doesn't exist. It prompts for the name of a tab and switches to it, creating a new tab if no tab exists by that name.</code></pre><p class=" text-justify">Great if you want to navigate by tab name <em>or</em> maybe create a new one if that name does not exist.</p><pre><code>New keymap 'tab-bar-history-mode-map'. By default, it contains 'C-c &lt;left&gt;' and 'C-c &lt;right&gt;' to browse the history of tab window configurations back and forward.</code></pre><p class=" text-justify">This feature is already in older Emacsen; what’s new is the mode map.</p><H2 id="bidi">BIDI</H2><pre><code>Better detection of text suspiciously reordered on display. The function 'bidi-find-overridden-directionality' has been extended to detect reordering effects produced by embeddings and isolates (started by directional formatting control characters such as RLO and LRI). The new command 'highlight-confusing-reorderings' finds and highlights segments of buffer text whose reordering for display is suspicious and could be malicious.</code></pre><p class=" text-justify"><em>Confusables</em> are a group of unicode characters that appear to look like well-known (often Western) characters but are in actual fact from other scripts. This is what spammers and scammers tend to use to make it appear like a domain name or email is actually someone else.</p><H2 id="emacs-server-and-client">Emacs Server and Client</H2><p class=" text-justify">Emacs has a client-server architecture, of course. It’s great! Set <code>EDITOR</code> to <code>emacsclient</code> and then <code>M-x server-start</code> (or any number of other ways of starting an Emacs server!) and now your Emacs will open files in a running instance.</p><pre><code>New command-line option '-r'/'--reuse-frame' for emacsclient. With this command-line option, Emacs reuses an existing graphical client frame if one exists; otherwise it creates a new frame.</code></pre><pre><code>New command-line option '-w N'/'--timeout=N' for emacsclient. With this command-line option, emacsclient will exit if Emacs does not respond within N seconds. The default is to wait forever.</code></pre><pre><code>'server-stop-automatically' can be used to automatically stop the server. The Emacs server will be automatically stopped when certain conditions are met. The conditions are determined by the argument to 'server-stop-automatically', which can be 'empty', 'delete-frame' or 'kill-terminal'.</code></pre><p class=" text-justify">Worthwhile if you have a use case that requires shutting down the server when you are finished. I, however, tend to keep it running forever and ever.</p><H2 id="rcirc">Rcirc</H2><p class=" text-justify">Rcirc is one of <em>two</em> IRC clients in Emacs. The other is ERC.</p><pre><code>New command 'rcirc-when'. This shows the reception time of the message at point (if available).</code></pre><pre><code>New user option 'rcirc-cycle-completion-flag'. Rcirc now uses the default 'completion-at-point' mechanism. The conventional IRC behavior of completing by cycling through the available options can be restored by enabling this option.</code></pre><pre><code>New user option 'rcirc-bridge-bot-alist'. If you are in a channel where a bot is responsible for bridging between networks, you can use this variable to make these messages appear more native. For example, you might set the option to: (setopt rcirc-bridge-bot-alist '(("bridge" . "{\\(.+?\\)}[[:space:]]+"))) for messages like 09:47 &lt;bridge&gt; {john} I am not on IRC to be reformatted into 09:47 &lt;john&gt; I am not on IRC</code></pre><pre><code>New formatting commands. Most IRC clients (including rcirc) support basic formatting using control codes. Under the 'C-c C-f' prefix a few commands have been added to insert these automatically. For example, if a region is active and 'C-c C-f C-b' is invoked, markup is inserted for the region to be highlighted in bold.</code></pre><p class=" text-justify">Remember when ornery graybeard <em>ircII</em> users would shout at mIRC users for using colors in channels? Now you can shout at mIRC users in boldface.</p><H2 id="imenu">Imenu</H2><pre><code>'imenu' is now bound to 'M-g i' globally.</code></pre><p class=" text-justify">I prefer <code>M-i</code>. It’s bound to <code>tab-to-tab-stop</code>, which is beyond useless. Nevertheless, it’s about time they bind one of the most useful navigational aids to a key.</p><pre><code>New function 'imenu-flush-cache'. Use it if you want Imenu to forget the buffer's index alist and recreate it anew next time 'imenu' is invoked.</code></pre><p class=" text-justify">This is of interest to major mode programmers.</p><pre><code>Emacs is now capable of abandoning a window's redisplay that takes too long. This is controlled by the new variable 'max-redisplay-ticks'. If that variable is set to a non-zero value, display of a window will be aborted after that many low-level redisplay operations, thus preventing Emacs from becoming wedged when visiting files with very long lines. The default is zero, which disables the feature: Emacs will wait forever for redisplay to finish. (We believe you won't need this feature, given the ability to display buffers with very long lines.)</code></pre><p class=" text-justify">I hope so, too.</p><H2 id="editing-changes-in-emacs-29.1">Editing Changes in Emacs 29.1</H2><pre><code>'M-SPC' is now bound to 'cycle-spacing'. Formerly it invoked 'just-one-space'. The actions performed by 'cycle-spacing' and their order can now be customized via the user option 'cycle-spacing-actions'.</code></pre><p class=" text-justify">If you didn’t know, this used to set the amount of whitespace around point to just one space. Super-duper command in code and prose: combine with <code>M-^</code> to ‘lift’ the current line to the one above. That’d usually leave excess spacing, which you could then delete with <code>M-SPC</code>.</p><p class=" text-justify">I recommend you review <code>cycle-spacing-actions</code> now that it’s changed.</p><pre><code>'zap-to-char' and 'zap-up-to-char' are case-sensitive for upper-case chars. These commands now behave as case-sensitive for interactive calls when they are invoked with an uppercase character, regardless of the value of 'case-fold-search'.</code></pre><p class=" text-justify">Case folding is mostly applied to the likes of isearch and replace-regexp. With it on (the default), Emacs will ignore casing <em>unless</em> you enter an uppercase letter. At that point Emacs becomes case-sensitive in search and replace. Great feature and really underrated.</p><pre><code>'scroll-other-window' and 'scroll-other-window-down' now respect remapping. These commands (bound to 'C-M-v' and 'C-M-V') used to scroll the other windows without looking at customizations in that other window. These functions now check whether they have been rebound in the buffer shown in that other window, and then call the remapped function instead. In addition, these commands now also respect the 'scroll-error-top-bottom' user option.</code></pre><p class=" text-justify">I’ve never remapped them, so I never knew there were problems when you did.</p><pre><code>Indentation of 'cl-flet' and 'cl-labels' has changed. These forms now indent like this: (cl-flet ((bla (x) (* x x))) (bla 42)) This change also affects 'cl-macrolet', 'cl-flet*' and 'cl-symbol-macrolet'.</code></pre><p class=" text-justify">I have no real opinion on this, but clearly someone did.</p><pre><code>New user option 'translate-upper-case-key-bindings'. Set this option to nil to inhibit the default translation of upper case keys to their lower case variants.</code></pre><p class=" text-justify">If true (default), then Emacs will convert an upper case key binding to a lower case one provided there is no upper case binding. It is, in other words, there to catch mistyped commands. Just leave it as is. It’s a good default.</p><pre><code>New command 'ensure-empty-lines'. This command increases (or decreases) the number of empty lines before point.</code></pre><p class=" text-justify">I can definitely find a user for this in <a href="/article/keyboard-macros-are-misunderstood" class=" article-link">keyboard macros</a>, where it’s hard to assert the presence or absence of a certain number of things without a lot of diligence.</p><pre><code>Improved mouse behavior with auto-scrolling modes. When clicking inside the 'scroll-margin' or 'hscroll-margin' region, point is now moved only when releasing the mouse button. This no longer results in a bogus selection, unless the mouse has also been dragged.</code></pre><pre><code>'kill-ring-max' now defaults to 120.</code></pre><p class=" text-justify">I never noticed the old limit, to be honest. I couldn’t even tell you what it was.</p><pre><code>New user option 'yank-menu-max-items'. Customize this option to limit the number of entries in the menu "Edit → Paste from Kill Menu". The default is 60.</code></pre><pre><code>New user option 'copy-region-blink-predicate'. By default, when copying a region with 'kill-ring-save', Emacs only blinks point and mark when the region is not denoted visually, that is, when either the region is inactive, or the 'region' face is indistinguishable from the 'default' face. Users who would rather enable blinking unconditionally can now set this user option to 'always'. To disable blinking unconditionally, either set this option to 'ignore', or set 'copy-region-blink-delay' to 0.</code></pre><pre><code>Performing a pinch gesture on a touchpad now increases the text scale.</code></pre><p class=" text-justify">Wait until you find out that Emacs has mouse gestures. <code>M-x strokes-help</code>.</p><H2 id="show-paren-mode">Show Paren Mode</H2><p class=" text-justify">Show paren mode highlights the matching delimiters at point.</p><pre><code>New user option 'show-paren-context-when-offscreen'. When non-nil, if the point is in a closing delimiter and the opening delimiter is offscreen, shows some context around the opening delimiter in the echo area. The default is nil. This option can also be set to the symbols 'overlay' or 'child-frame', in which case the context is shown in an overlay or child-frame at the top-left of the current window. The latter option requires a graphical frame. On non-graphical frames, the context is shown in the echo area.</code></pre><p class=" text-justify">This is a <em>great</em> feature, as it does more than it implies here. I recommend you experiment with it on (or set to <code>overlay</code> or <code>child-frame</code>).</p><H2 id="comint">Comint</H2><p class=" text-justify">Comint is the general term for Emacs’s interface that communicates with sub-processes, like bash or python.</p><pre><code>'comint-term-environment' is now aware of connection-local variables. The user option 'comint-terminfo-terminal' and the variable 'system-uses-terminfo' can now be set as connection-local variables to change the terminal used on a remote host.</code></pre><p class=" text-justify">Connection-local variables are variables, much like buffer-local ones, that are specific to a particular network connection.</p><pre><code>New user option 'comint-delete-old-input'. When nil, this prevents comint from deleting the current input when inserting previous input using '&lt;mouse-2&gt;'. The default is t, to preserve previous behavior.</code></pre><pre><code>New minor mode 'comint-fontify-input-mode'. This minor mode is enabled by default in "*shell*" and "*ielm*" buffers. It fontifies input text according to 'shell-mode' or 'emacs-lisp-mode' font-lock rules. Customize the user options 'shell-fontify-input-enable' and 'ielm-fontify-input-enable' to nil if you don't want to enable input fontification by default.</code></pre><p class=" text-justify">Handy and a good default. You’ll get free syntax highlighting. It’s already there for the output, but until now, it never worked with the input.</p><H2 id="mwheel">Mwheel</H2><pre><code>New user options for alternate wheel events. The user options 'mouse-wheel-down-alternate-event' and 'mouse-wheel-up-alternate-event' as well as the variables 'mouse-wheel-left-alternate-event' and 'mouse-wheel-right-alternate-event' have been added to better support systems where two kinds of wheel events can be received.</code></pre><H2 id="internationalization">Internationalization</H2><pre><code>The '&lt;Delete&gt;' function key now allows deleting the entire composed sequence. For the details, see the item about the 'delete-forward-char' command above.</code></pre><p class=" text-justify">This was covered earlier.</p><pre><code>New user option 'composition-break-at-point'. Setting it to a non-nil value temporarily disables automatic composition of character sequences at point, and thus makes it easier to edit such sequences by allowing point to "enter" the composed sequence.</code></pre><pre><code>Support for many old scripts and writing systems. Emacs now supports, and has language-environments and input methods, for several dozens of old scripts that were used in the past for various languages. For each such script Emacs now has font-selection and character composition rules, a language environment, and an input method. The newly-added scripts and the corresponding language environments are: Tai Tham script and the Northern Thai language environment Brahmi script and language environment Kaithi script and language environment Tirhuta script and language environment Sharada script and language environment Siddham script and language environment Syloti Nagri script and language environment Modi script and language environment Baybayin script and Tagalog language environment Hanunoo script and language environment Buhid script and language environment Tagbanwa script and language environment Limbu script and language environment Balinese script and language environment Javanese script and language environment Sundanese script and language environment Batak script and language environment Rejang script and language environment Makasar script and language environment Lontara script and language environment Hanifi Rohingya script and language environment Grantha script and language environment Kharoshthi script and language environment Lepcha script and language environment Meetei Mayek script and language environment Adlam script and language environment Mende Kikakui script and language environment Wancho script and language environment Toto script and language environment Gothic script and language environment Coptic script and language environment Mongolian-traditional script and language environment Mongolian-cyrillic language environment</code></pre><pre><code>The "Oriya" language environment was renamed to "Odia". This is to follow the change in the official name of the script. The 'oriya' input method was also renamed to 'odia'. However, the old name of the language environment and the input method are still supported.</code></pre><pre><code>New Greek translation of the Emacs tutorial. Type 'C-u C-h t' to select it in case your language setup does not do so automatically.</code></pre><pre><code>New Ukrainian translation of the Emacs tutorial.</code></pre><pre><code>New Farsi/Persian translation of the Emacs tutorial.</code></pre><pre><code>New default phonetic input method for the Tamil language environment. The default input method for the Tamil language environment is now "tamil-phonetic" which is a customizable phonetic input method. To change the input method's translation rules, customize the user option 'tamil-translation-rules'.</code></pre><pre><code>New 'tamil99' input method for the Tamil language. This supports the keyboard layout specifically designed for the Tamil language.</code></pre><pre><code>New input method 'slovak-qwerty'. This is a variant of the 'slovak' input method, which corresponds to the QWERTY Slovak keyboards.</code></pre><pre><code>New input method 'cyrillic-chuvash'. This input method is based on the russian-computer input method, and is intended for typing in the Chuvash language written in the Cyrillic script.</code></pre><pre><code>New input method 'cyrillic-mongolian'. This input method is for typing in the Mongolian language using the Cyrillic script. It is the default input method for the new Mongolian-cyrillic language environment, see above. </code></pre><H2 id="changes-in-specialized-modes-and-packages-in-emacs-29.1">Changes in Specialized Modes and Packages in Emacs 29.1</H2><H3 id="ecomplete">Ecomplete</H3><p class=" text-justify">Ecomplete is an address book auto completion tool.</p><pre><code>New commands 'ecomplete-edit' and 'ecomplete-remove'. These allow you to (respectively) edit and bulk-remove entries from the ecomplete database.</code></pre><pre><code>New user option 'ecomplete-auto-select'. If non-nil and there's only one matching option, auto-select that.</code></pre><pre><code>New user option 'ecomplete-filter-regexp'. If non-nil, this user option describes what entries not to add to the database stored on disk.</code></pre><H3 id="auth-source">Auth Source</H3><p class=" text-justify">Auth source is one of Emacs’s secrets stores. It’s used by GNUS and many other programs. I’ve written a detailed article on <a href="/article/keeping-secrets-in-emacs-gnupg-auth-sources" class=" article-link">keeping secrets in Emacs with GnuPG and Auth Sources</a>.</p><pre><code>New user option 'auth-source-pass-extra-query-keywords'. Whether to recognize additional keyword params, like ':max' and ':require', as well as accept lists of query terms paired with applicable keywords. This disables most known behavioral quirks unique to auth-source-pass, such as wildcard subdomain matching.</code></pre><H3 id="dired">Dired</H3><p class=" text-justify">Dired is the Emacs Directory Editor, bound to <code>C-x d</code>.</p><pre><code>'dired-guess-shell-command' moved from dired-x to dired. This means that 'dired-do-shell-command' will now provide smarter defaults without first having to require 'dired-x'. See the node "(emacs) Shell Command Guessing" in the Emacs manual for more details.</code></pre><p class=" text-justify"><code>dired-x</code> was one of those red-headed step children of Dired. It’s always had a long list of really useful features that, somehow, never really made it into the main file, for unknown reasons. That wouldn’t matter if it was loaded by default, but it wasn’t. So you always had two types of Dired users: those who did, and those who did not, know about or use dired-x.</p><p class=" text-justify">Anyway. This is a welcome change. It’ll offer better guesses for what to do with the file at point.</p><pre><code>'dired-clean-up-buffers-too' moved from dired-x to dired. This means that Dired now offers to kill buffers visiting files and dirs when they are deleted in Dired. Before, you had to require 'dired-x' to enable this behavior. To disable this behavior, customize the user option 'dired-clean-up-buffers-too' to nil. The related user option 'dired-clean-confirm-killing-deleted-buffers' (which see) has also been moved to 'dired'.</code></pre><p class=" text-justify">Controversial default, but a lot of people use dired as the focal point of their work.</p><pre><code>'dired-do-relsymlink' moved from dired-x to dired. The corresponding key 'Y' is now bound by default in Dired.</code></pre><pre><code>'dired-do-relsymlink-regexp' moved from dired-x to dired. The corresponding key sequence '% Y' is now bound by default in Dired.</code></pre><p class=" text-justify">Relative symlinks should’ve been part of dired from day one.</p><pre><code>'M-G' is now bound to 'dired-goto-subdir'. Before, that binding was only available if the dired-x package was loaded.</code></pre><p class=" text-justify">Another useful command worth knowing about.</p><pre><code>'dired-info' and 'dired-man' moved from dired-x to dired. The 'dired-info' and 'dired-man' commands have been moved from the dired-x package to dired. They have also been renamed to 'dired-do-info' and 'dired-do-man'; the old command names are obsolete aliases. The keys 'I' ('dired-do-info') and 'N' ('dired-do-man') are now bound in Dired mode by default. The user options 'dired-bind-man' and 'dired-bind-info' no longer have any effect and are obsolete. To get the old behavior back and unbind these keys in Dired mode, add the following to your Init file: (with-eval-after-load 'dired (keymap-set dired-mode-map "N" nil) (keymap-set dired-mode-map "I" nil))</code></pre><p class=" text-justify">These commands will render a groff/troff/lroff ‘man’ file, or a TeXInfo info manual and open it.</p><pre><code>New command 'dired-do-eww'. This command visits the file on the current line with EWW.</code></pre><p class=" text-justify">EWW is Emacs’s Web Wowser.</p><pre><code>'browse-url-of-dired-file' can now call the secondary browser. When invoked with a prefix arg, this will now call 'browse-url-secondary-browser-function' instead of the default browser. 'browse-url-of-dired-file' is bound to 'W' by default in dired mode.</code></pre><p class=" text-justify">Interesting addition. Meanwhile <code>M-x browse-url</code> does not feature this.</p><pre><code>New user option 'dired-omit-lines'. This is used by 'dired-omit-mode', and now allows you to hide based on other things than just the file names.</code></pre><p class=" text-justify">Omit mode hides things you do not wish to see in a dired buffer. Great way to filter out junk.</p><pre><code>New user option 'dired-mouse-drag-files'. If non-nil, dragging file names with the mouse in a Dired buffer will initiate a drag-and-drop session allowing them to be opened in other programs.</code></pre><p class=" text-justify">I use the mouse often in other programs, and drag and drop is in that weird sweet spot for me, speed-wise, where dragging in and out of Emacs is a desirable thing to be able to do.</p><pre><code>New user option 'dired-free-space'. Dired will now, by default, include the free space in the first line instead of having it on a separate line. To get the previous behavior back, say: (setopt dired-free-space 'separate)</code></pre><pre><code>New user option 'dired-make-directory-clickable'. If non-nil (which is the default), hitting 'RET' or 'mouse-1' on the directory components at the directory displayed at the start of the buffer will take you to that directory.</code></pre><pre><code>Search and replace in Dired/Wdired supports more regexps. For example, the regexp ".*" will match only characters that are part of the file name. Also "^.*$" can be used to match at the beginning of the file name and at the end of the file name. This is used only when searching on file names. In Wdired this can be used when the new user option 'wdired-search-replace-filenames' is non-nil (which is the default).</code></pre><p class=" text-justify">Great change. The search scope for regexp in dired and <a href="/article/wdired-editable-dired-buffers" class=" article-link">wdired</a> is now confined to the filename, which is infinitely more useful than the previous behavior.</p><H3 id="elisp">Elisp</H3><pre><code>New command 'elisp-eval-region-or-buffer' (bound to 'C-c C-e'). This command evals the forms in the active region or in the whole buffer.</code></pre><p class=" text-justify">Great addition, as <code>M-x eval-buffer</code> was never bound to anything.</p><pre><code>New commands 'elisp-byte-compile-file' and 'elisp-byte-compile-buffer'. These commands (bound to 'C-c C-f' and 'C-c C-b', respectively) byte-compile the visited file and the current buffer, respectively.</code></pre><p class=" text-justify">Also useful if you rely on the byte compiler to flag common mistakes.</p><H3 id="games">Games</H3><p class=" text-justify">Emacs ships with a range of <a href="/article/fun-games-in-emacs" class=" article-link">fun computer games</a>.</p><pre><code>New user option 'tetris-allow-repetitions'. This controls how randomness is implemented (whether to use pure randomness as before, or to use a bag).</code></pre><H3 id="battery">Battery</H3><p class=" text-justify">The Battery library concerns tooling related to laptop batteries.</p><pre><code>New user option 'battery-update-functions'. This can be used to trigger actions based on the battery status.</code></pre><H3 id="docview">DocView</H3><p class=" text-justify">DocView is Emacs’s image and document converter. It converts all manner of documents to simple images so you can browse them in Emacs.</p><pre><code>doc-view can now generate SVG images when viewing PDF files. If Emacs is built with SVG support, doc-view can generate SVG files when using MuPDF as the converter for PDF files, which generally leads to sharper images (especially when zooming), and allows customization of background and foreground color of the page via the new user options 'doc-view-svg-background' and 'doc-view-svg-foreground'. To activate this behavior, set 'doc-view-mupdf-use-svg' to non-nil if your Emacs has SVG support. Note that, with some versions of MuPDF, SVG generation is known to sometimes produce SVG files that are buggy or can take a long time to render.</code></pre><p class=" text-justify">SVG support is a nice addition, thanks to Emacs’s support for libsvg.</p><H3 id="enriched-mode">Enriched Mode</H3><p class=" text-justify">Enriched mode (“Rich-Text Format”, to others) is a useful little mode if you want basic WYSIWYG-style document editing.</p><pre><code>New command 'enriched-toggle-markup'. This allows you to see the markup in 'enriched-mode' buffers (e.g., the "HELLO" file). Bound to 'M-o m' by default.</code></pre><H3 id="shell-script-mode">Shell Script Mode</H3><pre><code>New user option 'sh-indent-statement-after-and'. This controls how statements like the following are indented: foo &amp;&amp; bar</code></pre><pre><code>New Flymake backend using the ShellCheck program. It is enabled by default, but requires that the external "shellcheck" command is installed.</code></pre><p class=" text-justify">Shellcheck’s amazing for shell script writers of all stripes. Do not write bash or sh scripts without it.</p><H3 id="cc-mode">CC Mode</H3><pre><code>C++ Mode now supports most of the new features in the C++20 Standard.</code></pre><pre><code>In Objective-C Mode, no extra types are recognized by default. The default value of 'objc-font-lock-extra-types' has been changed to nil, since too many identifiers were getting misfontified as types. This may cause some actual types not to get fontified. To get the old behavior back, customize the user option to the value suggested in its doc string.</code></pre><H3 id="cperl-mode">Cperl Mode</H3><pre><code>New user option 'cperl-file-style'. This option determines the indentation style to be used. It can also be used as a file-local variable.</code></pre><H3 id="gud">Gud</H3><p class=" text-justify">GUD is the Grand Unified Debugger. A frontend for a range of debuggers.</p><pre><code>'gud-go' is now bound to 'C-c C-v'. If given a prefix, it will prompt for an argument to use for the run/continue command.</code></pre><pre><code>'perldb' now recognizes '-E'. As of Perl 5.10, 'perl -E 0' behaves like 'perl -e 0' but also activates all optional features of the Perl version in use. 'perldb' now uses this invocation as its default.</code></pre><H3 id="customize">Customize</H3><p class=" text-justify"><code>M-x customize</code> is Emacs’s configuration tool.</p><pre><code>New command 'custom-toggle-hide-all-widgets'. This is bound to 'H' and toggles whether to hide or show the widget contents.</code></pre><H3 id="diff-mode">Diff Mode</H3><pre><code>New user option 'diff-whitespace-style'. Sets the value of the buffer-local variable 'whitespace-style' in 'diff-mode' buffers. By default, this variable is '(face trailing)', which preserves behavior of previous Emacs versions.</code></pre><pre><code>New user option 'diff-add-log-use-relative-names'. If non-nil insert file names in ChangeLog skeletons relative to the VC root directory.</code></pre><H3 id="ispell">Ispell</H3><pre><code>'ispell-region' and 'ispell-buffer' now push the mark. These commands push onto the mark ring the location of the last misspelled word where corrections were offered, so that you can then skip back to that location with 'C-x C-x'.</code></pre><p class=" text-justify">You can pop the mark with <code>C-u C-SPC</code> and return from whence you came.</p><H3 id="dabbrev">Dabbrev</H3><p class=" text-justify">Although dynamic abbrev is powerful, I prefer <a href="/article/text-expansion-hippie-expand" class=" article-link">Hippie Expand</a>.</p><pre><code>New function 'dabbrev-capf' for use on 'completion-at-point-functions'.</code></pre><pre><code>New user option 'dabbrev-ignored-buffer-modes'. Buffers with major modes in this list will be ignored. By default, this includes "binary" buffers like 'archive-mode' and 'image-mode'.</code></pre><H3 id="package">Package</H3><p class=" text-justify">Package is Emacs’s package manager.</p><pre><code>New command 'package-upgrade'. This command allows you to upgrade packages without using 'list-packages'. A package that comes with the Emacs distribution can only be upgraded after you install, once, a newer version from ELPA via the package-menu displayed by 'list-packages'.</code></pre><pre><code>New command 'package-upgrade-all'. This command allows upgrading all packages without any queries. A package that comes with the Emacs distribution will only be upgraded by this command after you install, once, a newer version of that package from ELPA via the package-menu displayed by 'list-packages'.</code></pre><p class=" text-justify">Convenience commands, really. Nice to have.</p><pre><code>New commands 'package-recompile' and 'package-recompile-all'. These commands can be useful if the ".elc" files are out of date (invalid byte code and macros).</code></pre><pre><code>New DWIM action on 'x' in "*Packages*" buffer. If no packages are marked, 'x' will install the package under point if it isn't already, and remove it if it is installed. Customize the new option 'package-menu-use-current-if-no-marks' to the nil value to get back the old behavior of signaling an error in that case.</code></pre><p class=" text-justify">I’d probably turn it off. Seems easy to fat finger and delete something.</p><pre><code>New command 'package-vc-install'. Packages can now be installed directly from source by cloning from their repository.</code></pre><p class=" text-justify">Many of us have been waiting for this feature. It checks out – using VC, Emacs’s generic version control frontend – a repository and installs it as a package.</p><p class=" text-justify">Unfortunately, it does not plug into <code>use-package</code>.</p><pre><code>New command 'package-vc-install-from-checkout'. An existing checkout can now be loaded via package.el, by creating a symbolic link from the usual package directory to the checkout.</code></pre><p class=" text-justify">This is really handy for those of us who develop packages and want to test that it all works.</p><pre><code>New command 'package-vc-checkout'. Used to fetch the source of a package by cloning a repository without activating the package.</code></pre><pre><code>New command 'package-vc-prepare-patch'. This command allows you to send patches to package maintainers, for packages checked out using 'package-vc-install'.</code></pre><p class=" text-justify">Handy – I look forward to seeing if this gets used.</p><pre><code>New command 'package-report-bug'. This command helps you compose an email for sending bug reports to package maintainers, and is bound to 'b' in the "*Packages*" buffer.</code></pre><p class=" text-justify">Also very nice. And a good way of acknowledging the role that repositories play in the package ecosystem in Emacs today.</p><pre><code>New user option 'package-vc-selected-packages'. By customizing this user option you can specify specific packages to install.</code></pre><pre><code>New user option 'package-install-upgrade-built-in'. When enabled, 'package-install' will include in the list of upgradeable packages those built-in packages (like Eglot and use-package, for example) for which a newer version is available in package archives, and will allow installing those newer versions. By default, this is disabled; however, if 'package-install' is invoked with a prefix argument, it will act as if this new option were enabled. In addition, when this option is non-nil, built-in packages for which a new version is available in archives can be upgraded via the package menu produced by 'list-packages'. If you do set this option non-nil, we recommend not to use the 'U' command, but instead to use '/ u' to show the packages which can be upgraded, and then decide which ones of them you actually want to update from the archives. If you customize this option, we recommend you place its non-default setting in your early-init file.</code></pre><p class=" text-justify">The thing this solves is this: how should Emacs handle upgrades to packages it ships with? I recommend you cautiously decide whether to upgrade builtin packages or not.</p><H3 id="emacs-sessions-desktop">Emacs Sessions (Desktop)</H3><p class=" text-justify">Desktop mode saves your Emacs session and restores it when you restart Emacs.</p><pre><code>New user option to load a locked desktop if locking Emacs is not running. The option 'desktop-load-locked-desktop' can now be set to the value 'check-pid', which means to allow loading a locked ".emacs.desktop" file if the Emacs process which locked it is no longer running on the local machine. This allows avoiding questions about locked desktop files when the Emacs session which locked it crashes, or was otherwise interrupted and didn't exit gracefully. See the "(emacs) Saving Emacs Sessions" node in the Emacs manual for more details.</code></pre><H3 id="miscellaneous">Miscellaneous</H3><pre><code>New command 'scratch-buffer'. This command switches to the "*scratch*" buffer. If "*scratch*" doesn't exist, the command creates it first. You can use this command if you inadvertently delete the "*scratch*" buffer.</code></pre><p class=" text-justify">Weirdly useful, this. I like the scratch buffer.</p><H3 id="debugging">Debugging</H3><pre><code>'q' in a "*Backtrace*" buffer no longer clears the buffer. Instead it just buries the buffer and switches the mode from 'debugger-mode' to 'backtrace-mode', since commands like 'e' are no longer available after exiting the recursive edit.</code></pre><p class=" text-justify">A quality of life change more than anything.</p><pre><code>New user option 'debug-allow-recursive-debug'. This user option controls whether the 'e' (in a "*Backtrace*" buffer or while edebugging) and 'C-x C-e' (while edebugging) commands lead to a (further) backtrace. By default, this variable is nil, which is a change in behavior from previous Emacs versions.</code></pre><pre><code>'e' in edebug can now take a prefix arg to pretty-print the results. When invoked with a prefix argument, as in 'C-u e', this command will pop up a new buffer and show the full pretty-printed value there.</code></pre><p class=" text-justify">I like this a lot. Pretty printing output is still a bit poor in Emacs, in my opinion.</p><pre><code>'C-x C-e' now interprets a non-zero prefix arg to pretty-print the results. When invoked with a non-zero prefix argument, as in 'C-u C-x C-e', this command will pop up a new buffer and show the full pretty-printed value there.</code></pre><pre><code>You can now generate a backtrace from Lisp errors in redisplay. To do this, set the new variable 'backtrace-on-redisplay-error' to a non-nil value. The backtrace will be written to a special buffer named "*Redisplay-trace*". This buffer will not be automatically displayed in a window.</code></pre><H3 id="compile">Compile</H3><p class=" text-justify"><code>M-x compile</code> is a generic compilation system with error highlighting, and more. See <a href="/article/compiling-running-scripts-emacs" class=" article-link">Compiling and running scripts in Emacs</a>.</p><pre><code>New user option 'compilation-hidden-output'. This regular expression can be used to make specific parts of compilation output invisible.</code></pre><p class=" text-justify">Can come in handy if you deal with noisy build output</p><pre><code>The 'compilation-auto-jump-to-first-error' user option has been extended. It can now have the additional values 'if-location-known' (which will only jump if the location of the first error is known), and 'first-known' (which will jump to the first known error location).</code></pre><p class=" text-justify">Now <em>that</em> is helpful. It’s worth your time trying that out if you are a regular user of compile, as I am.</p><pre><code>New user option 'compilation-max-output-line-length'. Lines longer than the value of this option will have their ends hidden, with a button to reveal the hidden text. This speeds up operations like grepping on files that have few newlines. The default value is 400; set to nil to disable hiding.</code></pre><p class=" text-justify">This should help stop things like minified files from gumming up everything.</p><H3 id="flymake">Flymake</H3><p class=" text-justify">Flymake is the on-the-fly syntax and linter frontend. There is also <a href="/article/spotlight-flycheck-a-flymake-replacement" class=" article-link">Flycheck, a Flymake replacement</a>.</p><pre><code>New user option 'flymake-mode-line-lighter'.</code></pre><p class=" text-justify">The mode line’s become a dumping ground for every little minor mode or feature, so being able to easily remove the overly verbose “Flymake” string is welcome.</p><p class=" text-justify">I had to resort to all manner of trickery to hide it before. See <a href="/article/hiding-replacing-modeline-strings" class=" article-link">hiding and replacing modeline strings with clean-mode-line</a>.</p><pre><code>New minor mode 'word-wrap-whitespace-mode' for extending 'word-wrap'. This mode switches 'word-wrap' on, and breaks on all the whitespace characters instead of just 'SPC' and 'TAB'.</code></pre><pre><code>New mode, 'emacs-news-mode', for editing the NEWS file. This mode adds some highlighting, makes the 'M-q' command aware of the format of NEWS entries, and has special commands for doing maintenance of the Emacs NEWS files. In addition, this mode turns on 'outline-minor-mode', and thus displays customizable icons (see 'icon-preference') in the margins. To disable these icons, set 'outline-minor-mode-use-buttons' to a nil value.</code></pre><H3 id="kmacro">Kmacro</H3><p class=" text-justify">KMacro is Emacs’s macro recorder or, more accurately, the extensions built in top of the original. You can do an awful lot of cool stuff with them: <a href="/article/keyboard-macros-are-misunderstood" class=" article-link">Keyboard Macros are Misunderstood</a>.</p><pre><code>Kmacros are now OClosures and have a new constructor 'kmacro' which uses the 'key-parse' syntax. It replaces the old 'kmacro-lambda-form' (which is now declared obsolete).</code></pre><p class=" text-justify">This shouldn’t affect any existing, recorded macros.</p><pre><code>savehist.el can now truncate variables that are too long. An element of user option 'savehist-additional-variables' can now be of the form '(VARIABLE . MAX-ELTS)', which means to truncate the VARIABLE's value to at most MAX-ELTS elements (if the value is a list) before saving the value.</code></pre><H3 id="minibuffer-and-completions">Minibuffer and Completions</H3><p class=" text-justify">Minibuffer completion’s a dizzying array of options now. I’ve written a guide on <a href="/article/understanding-minibuffer-completion" class=" article-link">understanding minibuffer completion</a> if you want to know more.</p><pre><code>New commands for navigating completions from the minibuffer. When the minibuffer is the current buffer, typing 'M-&lt;up&gt;' or 'M-&lt;down&gt;' selects a previous/next completion candidate from the "*Completions*" buffer and inserts it to the minibuffer. When the user option 'minibuffer-completion-auto-choose' is nil, 'M-&lt;up&gt;' and 'M-&lt;down&gt;' do the same, but without inserting a completion candidate to the minibuffer, then 'M-RET' can be used to choose the currently active candidate from the "*Completions*" buffer and exit the minibuffer. With a prefix argument, 'C-u M-RET' inserts the currently active candidate to the minibuffer, but doesn't exit the minibuffer. These keys are also available for in-buffer completion, but they don't insert candidates automatically, you need to type 'M-RET' to insert the selected candidate to the buffer.</code></pre><p class=" text-justify">If you use the <em>Completions</em> buffer to help select matches – one of many possible workflows – then you’re likely to find this selection mechanism a useful alternative to the cumbersome methods of picking and choosing from that buffer before.</p><p class=" text-justify">If you use a fancy type-as-you-complete completion framework, then you’re not likely to benefit from this change at all.</p><pre><code>Choosing a completion with a prefix argument doesn't exit the minibuffer. This means that typing 'C-u RET' on a completion candidate in the "*Completions*" buffer inserts the completion into the minibuffer, but doesn't exit the minibuffer.</code></pre><p class=" text-justify">Niche use, but the completion system’s flexible and omnipresent, as you can ‘enter’ the completion mechanism using any number of features in Emacs, such as <code>M-x shell</code>, where the ability to repeatedly insert matches is useful.</p><pre><code>The "*Completions*" buffer can now be automatically selected. To enable this behavior, customize the user option 'completion-auto-select' to t, then pressing 'TAB' will switch to the "*Completions*" buffer when it pops up that buffer. If the value is 'second-tab', then the first 'TAB' will display "*Completions*", and the second one will switch to the "*Completions*" buffer.</code></pre><p class=" text-justify">This feature also plays into the <em>Completions</em> buffer workflow. I’ve never been a huge fan of mucking around with manually selecting stuff, preferring instead to type my way to an answer and then only use the buffer as a popup of remaining choices.</p><pre><code>New user option 'completion-auto-wrap'. When non-nil, the commands 'next-completion' and 'previous-completion' automatically wrap around on reaching the beginning or the end of the "*Completions*" buffer.</code></pre><pre><code>New values for the 'completion-auto-help' user option. There are two new values to control the way the "*Completions*" buffer behaves after pressing a 'TAB' if completion is not unique. The value 'always' updates or shows the "*Completions*" buffer after any attempt to complete. The value 'visual' is like 'always', but only updates the completions if they are already visible. The default value t always hides the completion buffer after some completion is made.</code></pre><p class=" text-justify">All this controls is how or when the buffer is shown, which is again down to the specifics of your completion setup. If you want the buffer to only appear when you tab twice, now you can.</p><pre><code>New commands to complete the minibuffer history. 'minibuffer-complete-history' ('C-x &lt;up&gt;') is like 'minibuffer-complete' but completes on the history items instead of the default completion table. 'minibuffer-complete-defaults' ('C-x &lt;down&gt;') completes on the list of default items.</code></pre><pre><code>User option 'minibuffer-eldef-shorten-default' is now obsolete. Customize the user option 'minibuffer-default-prompt-format' instead.</code></pre><pre><code>New user option 'completions-sort'. This option controls the sorting of the completion candidates in the "*Completions*" buffer. Available styles are no sorting, alphabetical (the default), or a custom sort function.</code></pre><p class=" text-justify">Sort order is weirdly important, as you can sort by frequency, context, etc. I expect this feature will prove enduring and popular going forward.</p><pre><code>New user option 'completions-max-height'. This option limits the height of the "*Completions*" buffer.</code></pre><p class=" text-justify">Instead of toying with <code>display-buffer-alist</code>, you can now set this height directly.</p><pre><code>New user option 'completions-header-format'. This is a string to control the header line to show in the "*Completions*" buffer before the list of completions. If it contains "%s", that is replaced with the number of completions. If nil, the header line is not shown.</code></pre><pre><code>New user option 'completions-highlight-face'. When this user option names a face, the current candidate in the "*Completions*" buffer is highlighted with that face. The nil value disables this highlighting. The default is to highlight using the 'completions-highlight' face.</code></pre><pre><code>You can now define abbrevs for the minibuffer modes. 'minibuffer-mode-abbrev-table' and 'minibuffer-inactive-mode-abbrev-table' are now defined.</code></pre><p class=" text-justify">That’s actually quite helpful. <a href="/article/correcting-typos-misspellings-abbrev" class=" article-link">Abbrev</a> is meant for simple typos and word replacements, and indeed they can happen anywhere, even in the minibuffer.</p><H3 id="isearch-and-replace">Isearch and Replace</H3><pre><code>Changes in how Isearch responds to 'mouse-yank-at-point'. If a user does 'C-s' and then uses '&lt;mouse-2&gt;' ('mouse-yank-primary') outside the echo area, Emacs will, by default, end the Isearch and yank the text at mouse cursor. But if 'mouse-yank-at-point' is non-nil, the text will now be added to the Isearch instead.</code></pre><pre><code>Changes for values 'no' and 'no-ding' of 'isearch-wrap-pause'. Now with these values the search will wrap around not only on repeating with 'C-s C-s', but also after typing a character.</code></pre><p class=" text-justify">Isearch will by default wrap around and start anew if you double-tap <code>C-s</code> at the end of the search.</p><pre><code>New user option 'char-fold-override'. Non-nil means that the default definitions of equivalent characters are overridden.</code></pre><p class=" text-justify">The character folding feature (turning one character into another, for the purposes of ease of searching) is governed by this, new, variable, and <code>char-fold-[include/exclude]</code>.</p><pre><code>New command 'describe-char-fold-equivalences'. It displays character equivalences used by 'char-fold-to-regexp'.</code></pre><p class=" text-justify">This command is an intuitive display of what a character is folded to, if anything.</p><pre><code>New command 'isearch-emoji-by-name'. It is bound to 'C-x 8 e RET' during an incremental search. The command accepts the Unicode name of an Emoji (for example, "smiling face" or "heart with arrow"), like 'C-x 8 e e', with minibuffer completion, and adds the Emoji into the search string.</code></pre><p class=" text-justify">Oddly specific, but… OK.</p><H3 id="gdbmi">GDB/MI</H3><pre><code>New user option 'gdb-debuginfod-enable-setting'. On capable platforms, GDB 10.1 and later can download missing source and debug info files from special-purpose servers, called "debuginfod servers". Use this new option to control whether 'M-x gdb' instructs GDB to download missing files from debuginfod servers when you debug the corresponding programs. The default is to ask you at the beginning of each debugging session whether to download the files for that session.</code></pre><H3 id="glyphless-characters">Glyphless Characters</H3><pre><code>New minor mode 'glyphless-display-mode'. This allows an easy way to toggle seeing all glyphless characters in the current buffer.</code></pre><pre><code>The extra slot of 'glyphless-char-display' can now have cons values. The extra slot of the 'glyphless-char-display' char-table can now have values that are cons cells, specifying separate values for text-mode and GUI terminals.</code></pre><pre><code>"Replacement character" feature for undisplayable characters on TTYs. The 'acronym' method of displaying glyphless characters on text-mode frames treats single-character acronyms specially: they are displayed without the surrounding '[..]' "box", thus in effect treating such "acronyms" as replacement characters.</code></pre><H3 id="registers">Registers</H3><p class=" text-justify">Registers are single-character short-hands that can store a wide range of things, including strings, numbers, points, window configurations, and much more.</p><pre><code>Buffer names can now be stored in registers. For instance, to enable jumping to the "*Messages*" buffer with 'C-x r j m': (set-register ?m '(buffer . "*Messages*"))</code></pre><p class=" text-justify">Neat addition.</p><H3 id="pixel-fill">Pixel Fill</H3><pre><code>This is a new package that deals with filling variable-pitch text.</code></pre><pre><code>New function 'pixel-fill-region'. This fills the region to be no wider than a specified pixel width.</code></pre><H3 id="info">Info</H3><pre><code>Command 'info-apropos' now takes a prefix argument to search for regexps.</code></pre><p class=" text-justify">Great command, and worth knowing about, as it’ll free-form search <em>all</em> known Info manuals. See <a href="/article/full-text-searching-info-mode-apropos" class=" article-link">Full text searching in Info mode with Apropos</a>.</p><pre><code>New command 'Info-goto-node-web' and key binding 'G'. This will take you to the "gnu.org" web server's version of the current info node. This command only works for the Emacs and Emacs Lisp manuals.</code></pre><p class=" text-justify">Convenient shortcut if you want to give someone a link to a manual page.</p><H3 id="shortdoc">Shortdoc</H3><p class=" text-justify">Shortdoc is Emacs’s extensible cheat sheet for elisp. See <a href="/article/emacs-builtin-elisp-cheat-sheet" class=" article-link">Emacs’s Builtin Elisp Cheat Sheet</a>.</p><pre><code>New command 'shortdoc-copy-function-as-kill' bound to 'w'. It copies the name of the function near point into the kill ring.</code></pre><pre><code>'N' and 'P' are now bound to 'shortdoc-{next,previous}-section'. This is in addition to the old keybindings 'C-c C-n' and 'C-c C-p'.</code></pre><H3 id="vc">VC</H3><p class=" text-justify">VC is Emacs’s generic Version Control system that works with all major and minor version control systems. They’re bound to <code>C-x v</code>. I love VC, even though I use Magit with Git for 95% of what I do.</p><p class=" text-justify">It works really well and offers a unified view of all version control systems it supports.</p><pre><code>New command 'vc-pull-and-push'. This commands first does a "pull" command, and if that is successful, does a "push" command afterwards. Currently supported in Git and Bzr.</code></pre><p class=" text-justify">Handy time saver.</p><pre><code>'C-x v b' prefix key is used now for branch commands. 'vc-print-branch-log' is bound to 'C-x v b l', and new commands are 'vc-create-branch' ('C-x v b c') and 'vc-switch-branch' ('C-x v b s'). The VC Directory buffer now uses the prefix 'b' for these branch-related commands.</code></pre><p class=" text-justify">This is a great set of features, and if you use VC, you should definitely learn them.</p><pre><code>New command 'vc-dir-mark-by-regexp' bound to '% m' and '* %'. This command marks files based on a regexp. If given a prefix argument, unmark instead.</code></pre><p class=" text-justify">Much like dired. Note that this is meant to be run in <code>C-x v d</code>, the dired-style VC buffer.</p><pre><code>New command 'C-x v !' ('vc-edit-next-command'). This prefix command requests editing of the next VC shell command before execution. For example, in a Git repository, you can produce a log of more than one branch by typing 'C-x v ! C-x v b l' and then appending additional branch names to the 'git log' command. The intention is that this command can be used to access a wide variety of version control system-specific functionality from VC without complexifying either the VC command set or the backend API.</code></pre><pre><code>'C-x v v' in a diffs buffer allows to commit only some of the changes. This command is intended to allow you to commit only some of the changes you have in your working tree. Begin by creating a buffer with the changes against the last commit, e.g. with 'C-x v D' ('vc-root-diff'). Then edit the diffs to remove the hunks you don't want to commit. Finally, type 'C-x v v' in that diff buffer to commit only part of your changes, those whose hunks were left in the buffer.</code></pre><p class=" text-justify">Partial commits of hunks is a great feature of Magit, and I am happy to see it in VC also.</p><pre><code>'C-x v v' on an unregistered file will now use the most specific backend. Previously, if you had an SVN-covered "~/" directory, and a Git-covered directory in "~/foo/bar", using 'C-x v v' on a new, unregistered file "~/foo/bar/zot" would register it in the SVN repository in "~/" instead of in the Git repository in "~/foo/bar". This makes this command consistent with 'vc-responsible-backend'.</code></pre><pre><code>Log Edit now fontifies long Git commit summary lines. Writing shorter summary lines avoids truncation in contexts in which Git commands display summary lines. See the two new user options 'vc-git-log-edit-summary-target-len' and 'vc-git-log-edit-summary-max-len'.</code></pre><pre><code>New 'log-edit-headers-separator' face. It is used to style the line that separates the 'log-edit' headers from the 'log-edit' summary.</code></pre><pre><code>The function 'vc-read-revision' accepts a new MULTIPLE argument. If non-nil, multiple revisions can be queried. This is done using 'completing-read-multiple'.</code></pre><pre><code>New function 'vc-read-multiple-revisions'. This function invokes 'vc-read-revision' with a non-nil value for MULTIPLE.</code></pre><pre><code>New command 'vc-prepare-patch'. Patches for any version control system can be prepared using VC. The command will query what commits to send and will compose messages for your mail user agent. The behavior of 'vc-prepare-patch' can be modified by the user options 'vc-prepare-patches-separately' and 'vc-default-patch-addressee'.</code></pre><H3 id="message">Message</H3><p class=" text-justify">Message is Emacs’s Message mode intended for e-mails and suchlike.</p><pre><code>New user option 'mml-attach-file-at-the-end'. If non-nil, 'C-c C-a' will put attached files at the end of the message.</code></pre><p class=" text-justify">Don’t forget that you can drag files into a message buffer.</p><pre><code>Message Mode now supports image yanking.</code></pre><pre><code>New user option 'message-server-alist'. This controls automatic insertion of the "X-Message-SMTP-Method" header before sending a message.</code></pre><H3 id="html-mode">HTML Mode</H3><pre><code>HTML Mode now supports "text/html" and "image/*" yanking.</code></pre><p class=" text-justify">Not much to say: great addition. Now I just have to remember that it can do this.</p><H3 id="texinfo-mode">Texinfo Mode</H3><pre><code>'texinfo-mode' now has a specialized 'narrow-to-defun' definition. It narrows to the current node.</code></pre><H3 id="eudc">EUDC</H3><p class=" text-justify">EUDC is a directory client frontend for various protocols, such as LDAP. If you work in a corporate environment, you’re probably using LDAP. You can use EUDC to talk to Microsoft Exchange (or FreeIPA, or …) and pull out contracts and whatnot from it. Powerful stuff.</p><pre><code>Deprecations planned for next release. After Emacs 29.1, some aspects of EUDC will be deprecated. The goal of these deprecations is to simplify EUDC server configuration by making 'eudc-server-hotlist' the only place to add servers. There will not be a need to set the server using the 'eudc-set-server' command. Instead, the 'eudc-server-hotlist' user option should be customized to have an entry for the server. The plan is to obsolete the 'eudc-hotlist' package since Customize is sufficient for changing 'eudc-server-hotlist'. How the 'eudc-server' user option works in this context is to-be-determined; it can't be removed, because that would break compatibility, but it may become synchronized with 'eudc-server-hotlist' so that 'eudc-server' is always equal to '(car eudc-server-hotlist)'. The first entry in 'eudc-server-hotlist' is the first server tried by 'eudc-expand-try-all'. The hotlist simplification will allow 'eudc-query-form' to show a drop down of possible servers, instead of requiring a call to 'eudc-set-server' like it does in this release. The default value of 'eudc-ignore-options-file' will be changed from nil to t.</code></pre><pre><code>New user option 'eudc-ignore-options-file' that defaults to nil. The 'eudc-ignore-options-file' user option can be configured to ignore the 'eudc-options-file' (typically "~/.emacs.d/eudc-options"). Most users should configure this to t and put EUDC configuration in the main Emacs initialization file ("~/.emacs" or "~/.emacs.d/init.el").</code></pre><pre><code>'eudc-expansion-overwrites-query' to 'eudc-expansion-save-query-as-kill'. The user option 'eudc-expansion-overwrites-query' is renamed to 'eudc-expansion-save-query-as-kill' to reflect the actual behavior of the user option. The former is kept as alias.</code></pre><pre><code>New command 'eudc-expand-try-all'. This command can be used in place of 'eudc-expand-inline'. It takes a prefix argument that causes 'eudc-expand-try-all' to return matches from all servers instead of just the matches from the first server to return any. This is useful for example, if one wants to search LDAP for a name that happens to match a contact in one's BBDB.</code></pre><pre><code>New behavior and default for user option 'eudc-inline-expansion-format'. EUDC inline expansion result formatting defaulted to ("%s %s &lt;%s&gt;" firstname name email) Since email address specifications need to comply with RFC 5322 in order to be useful in messages, there was a risk of producing syntax which was standard with RFC 822, but is marked as obsolete syntax by its successor RFC 5322. Also, the first and last name part was never enclosed in double quotes, potentially producing invalid address specifications, which may be rejected by a receiving MTA. Thus, this variable can now additionally be set to nil (the new default), or a function. In both cases, the formatted result will be in compliance with RFC 5322. When set to nil, a default format very similar to the old default will be produced. When set to a function, that function is called, and the returned values are used to populate the phrase and comment parts (see RFC 5322 for definitions). In both cases, the phrase part will be automatically quoted if necessary.</code></pre><pre><code>New function 'eudc-capf-complete' with 'message-mode' integration. EUDC can now contribute email addresses to 'completion-at-point' by adding the new function 'eudc-capf-complete' to 'completion-at-point-functions' in 'message-mode'.</code></pre><p class=" text-justify">The missing link that ties EUDC into Emacs’s general-purpose completion framework called “capf” (short for <code>completion-at-point-functions</code>)</p><pre><code>Additional attributes of query and results in eudcb-macos-contacts.el. The EUDC back-end for the macOS Contacts app now provides a wider set of attributes to use for queries, and delivers more attributes in query results.</code></pre><pre><code>New back-end for ecomplete. A new back-end for ecomplete allows information from that database to be queried by EUDC, too. The attributes present in the EUDC query are used to select the entry type in the ecomplete database.</code></pre><pre><code>New back-end for mailabbrev. A new back-end for mailabbrev allows information from that database to be queried by EUDC, too. Only the attributes 'email', 'name', and 'firstname' are supported.</code></pre><H3 id="ewwshr">EWW/SHR</H3><p class=" text-justify">EWW is Emacs’s Web Wowser. SHR is the renderer that powers it.</p><pre><code>New user option to automatically rename EWW buffers. The 'eww-auto-rename-buffer' user option can be configured to rename rendered web pages by using their title, URL, or a user-defined function which returns a string. For the first two cases, the length of the resulting name is controlled by the user option 'eww-buffer-name-length'. By default, no automatic renaming is performed.</code></pre><pre><code>New user option 'shr-allowed-images'. This complements 'shr-blocked-images', but allows specifying just the allowed images.</code></pre><pre><code>New user option 'shr-use-xwidgets-for-media'. If non-nil (and Emacs has been built with support for xwidgets), display &lt;video&gt; elements with an xwidget. Note that this is experimental; it is known to crash Emacs on some systems, and just doesn't work on other systems. Also see etc/PROBLEMS.</code></pre><p class=" text-justify">If you compile Emacs with xwidget support, then maybe — just maybe – you might be able to watch a video link in your EWW buffer.</p><pre><code>New user option 'eww-url-transformers'. These are used to alter an URL before using it. By default it removes the common "utm_" trackers from URLs.</code></pre><H3 id="find-dired">Find Dired</H3><p class=" text-justify">Who needs <code>find</code> and <code>xargs</code> when you have <a href="/article/dired-shell-commands-find-xargs-replacement" class=" article-link">Dired, the find &amp; xargs replacement</a>?</p><pre><code>New command 'find-dired-with-command'. This enables users to run 'find-dired' with an arbitrary command, enabling running commands previously unsupported and also enabling new commands to be built on top.</code></pre><H3 id="gnus">Gnus</H3><p class=" text-justify">For some, GNUS is a news, mail, and rss reader; for others, it’s a lifestyle.</p><pre><code>Tool bar changes in Gnus/Message. There were previously two styles of tool bars available in Gnus and Message, referred to as 'gnus-summary-tool-bar-retro', 'gnus-group-tool-bar-retro' and 'message-tool-bar-retro', and 'gnus-summary-tool-bar-gnome', 'gnus-group-tool-bar-gnome' and 'message-tool-bar-gnome'. The "retro" tool bars have been removed (as well as the icons used), and the "gnome" tool bars are now the only pre-defined toolbars.</code></pre><pre><code>'gnus-summary-up-thread' and 'gnus-summary-down-thread' bindings removed. The 'gnus-summary-down-thread' binding to 'M-C-d' was shadowed by 'gnus-summary-read-document', and these commands are also available on 'T u' and 'T d' respectively.</code></pre><pre><code>Gnus now uses a variable-pitch font in the headers by default. To get the monospace font back, you can put something like the following in your ".gnus" file: (set-face-attribute 'gnus-header nil :inherit 'unspecified)</code></pre><pre><code>The default value of 'gnus-treat-fold-headers' is now 'head'.</code></pre><pre><code>New face 'gnus-header'. All other 'gnus-header-*' faces inherit from this face now.</code></pre><pre><code>New user option 'gnus-treat-emojize-symbols'. If non-nil, symbols that have an Emoji representation will be displayed as emojis. The default is nil.</code></pre><pre><code>New command 'gnus-article-emojize-symbols'. This is bound to 'W D e' and will display symbols that have Emoji representation as Emoji.</code></pre><pre><code>New mu backend for gnus-search. Configuration is very similar to the notmuch and namazu backends. It supports the unified search syntax.</code></pre><p class=" text-justify">Interesting. <code>mu</code> is a standalone mail indexer, and <code>mu4e</code> a mail reader on top of it. So it’s perhaps possible, then, to use GNUS for mail reading, and outsource storing and indexing?</p><pre><code>'gnus-html-image-cache-ttl' is now a seconds count. Formerly it was a pair of numbers '(A B)' that represented 65536*A + B, to cater to older Emacs implementations that lacked bignums. The older form still works but is undocumented.</code></pre><H3 id="rmail">Rmail</H3><p class=" text-justify">Rmail is yet another way of interacting with email in Emacs.</p><pre><code>Rmail partial summaries can now be applied one on top of the other. You can now narrow the set of messages selected by Rmail summary's criteria (recipients, topic, senders, etc.) by making a summary of the already summarized messages. For example, invoking 'rmail-summary-by-senders', followed by 'rmail-summary-by-topic' will produce a summary where both the senders and the topic are according to your selection. The new user option 'rmail-summary-progressively-narrow' controls whether the stacking of the filters is in effect; customize it to a non-nil value to enable this feature.</code></pre><pre><code>New Rmail summary: by thread. The new command 'rmail-summary-by-thread' produces a summary of messages that belong to a single thread of discussion.</code></pre><H3 id="eieio">EIEIO</H3><p class=" text-justify">EIEIO is Elisp’s CLOS-style object oriented system. It’s also a nursery rhyme.</p><blockquote><p class=" text-justify">Old MacDonald had a farm…</p></blockquote><pre><code>'slot-value' can now be used to access slots of 'cl-defstruct' objects.</code></pre><H3 id="align">Align</H3><p class=" text-justify">Align is Emacs’s text alignment facility.</p><pre><code>Alignment in 'text-mode' has changed. Previously, 'M-x align' didn't do anything, and you had to say 'C-u M-x align' for it to work. This has now been changed. The default regexp for 'C-u M-x align-regexp' has also been changed to be easier for inexperienced users to use.</code></pre><H3 id="help-1">Help</H3><p class=" text-justify">Help is, well, help. It’s the mode you interact with when you ask Emacs for help.</p><pre><code>New mode, 'emacs-news-view-mode', for viewing the NEWS file. This mode is used by the 'C-h N' command, and adds buttons to manual entries and symbol references.</code></pre><pre><code>New user option 'help-link-key-to-documentation'. When this option is non-nil (which is the default), key bindings displayed in the "*Help*" buffer will be linked to the documentation for the command they are bound to. This does not affect listings of key bindings and functions (such as 'C-h b').</code></pre><p class=" text-justify">More interlinking is always a good thing. I’m glad they’re spending more time improving this in recent Emacsen.</p><H3 id="info-look">Info Look</H3><pre><code>info-look specs can now be expanded at run time instead of a load time. The new ':doc-spec-function' element can be used to compute the ':doc-spec' element when the user asks for info on that particular mode (instead of at load time).</code></pre><H3 id="ansi-color">Ansi Color</H3><p class=" text-justify">Ansi color is Emacs’s rather tired and limited ANSI color feature.</p><pre><code>Support for ANSI 256-color and 24-bit colors. 256-color and 24-bit color codes are now handled by ANSI color filters and displayed with the specified color.</code></pre><H3 id="term-mode">Term Mode</H3><p class=" text-justify">Term is Emacs’s terminal emulator. Not to be confused with shells!</p><p class=" text-justify">See <a href="/article/running-shells-in-emacs-overview" class=" article-link">Running Shells and Terminal Emulators in Emacs</a></p><pre><code>New user option 'term-bind-function-keys'. If non-nil, 'term-mode' will pass the function keys on to the underlying shell instead of using the normal Emacs bindings.</code></pre><pre><code>Support for ANSI 256-color and 24-bit colors, italic and other fonts. 'term-mode' can now display 256-color and 24-bit color codes. It can also handle ANSI codes for faint, italic and blinking text, displaying it with new 'term-{faint,italic,slow-blink,fast-blink}' faces.</code></pre><H3 id="project">Project</H3><p class=" text-justify">Project is the latest in a long, <em>long</em> list of project management solutions built into Emacs. This one seems to have a bit more staying power than the others before it (filesets, CEDET’s project management suite EDE, filecache, …) and that is a good thing.</p><p class=" text-justify">Emacs has sorely lacked a proper project manager that people can rally behind.</p><pre><code>'project-find-file' and 'project-or-external-find-file' can include all. The commands 'project-find-file' and 'project-or-external-find-file' now accept a prefix argument, which is interpreted to mean "include all files".</code></pre><pre><code>New command 'project-list-buffers' bound to 'C-x p C-b'. This command displays a list of buffers from the current project.</code></pre><p class=" text-justify">Scoping buffers to the project is good. Shame it’s not using <code>M-x ibuffer</code>.</p><pre><code>'project-kill-buffers' can display the list of buffers to kill. Customize the user option 'project-kill-buffers-display-buffer-list' to enable the display of the buffer list.</code></pre><pre><code>New user option 'project-vc-extra-root-markers'. Use it to add detection of nested projects (inside a VCS repository), or projects outside of VCS repositories. As a consequence, the 'VC project backend' is formally renamed to 'VC-aware project backend'.</code></pre><pre><code>New user option 'project-vc-include-untracked'. If non-nil, files untracked by a VCS are considered to be part of the project by a VC project based on that VCS.</code></pre><H3 id="xref">Xref</H3><p class=" text-justify">Xref is the cross-referencing tool that replaced the rather poor and cumbersome “TAGS” search machinery.</p><pre><code>New command 'xref-go-forward'. It is bound to 'C-M-,' and jumps to the location where you previously invoked 'xref-go-back' ('M-,', also known as 'xref-pop-marker-stack').</code></pre><pre><code>The depth of the Xref marker stack is now infinite. The implementation of the Xref marker stack was changed in a way that allows as many places to be saved on the stack as needed, limited only by the available memory. Therefore, the variables 'find-tag-marker-ring-length' and 'xref-marker-ring-length' are now obsolete and unused; setting them has no effect.</code></pre><p class=" text-justify">Not sure I ever knew, much less reached, the old limit.</p><pre><code>'xref-query-replace-in-results' prompting change. This command no longer prompts for FROM when called without prefix argument. This makes the most common case faster: replacing entire matches.</code></pre><pre><code>New command 'xref-find-references-and-replace' to rename one identifier.</code></pre><p class=" text-justify">Handy, as it saves on typing.</p><pre><code>New variable 'xref-current-item' (renamed from a private version).</code></pre><pre><code>New function 'xref-show-xrefs'.</code></pre><pre><code>'outline-minor-mode' is supported in Xref buffers. You can enable outlining by adding 'outline-minor-mode' to 'xref-after-update-hook'.</code></pre><p class=" text-justify">I didn’t know I wanted this in my life, but… I do! that’s a neat feature. And, just sayin’, if it weren’t for this NEWS entry, nobody – ever – would figure this out.</p><H3 id="file-notifications">File Notifications</H3><p class=" text-justify">Emacs can stay updated on file system changes thanks to a range of OS-specific mechanisms.</p><pre><code>The new command 'file-notify-rm-all-watches' removes all file notifications.</code></pre><H3 id="sql">Sql</H3><pre><code>Sql now supports sending of passwords in-process. To improve security, if an sql product has ':password-in-comint' set to t, a password supplied via the minibuffer will be sent in-process, as opposed to via the command-line.</code></pre><H3 id="image-mode">Image Mode</H3><pre><code>New command 'image-transform-fit-to-window'. This command fits the image to the current window by scaling down or up as necessary. Unlike 'image-transform-fit-both', this can scale the image up as well as down. It is bound to 's w' in Image Mode by default.</code></pre><p class=" text-justify">I do use the Image mode feature in Emacs, but I’ve never had to scale an image up; but it’s good to know it’s there.</p><pre><code>New command 'image-mode-wallpaper-set'. This command sets the desktop background to the current image. It is bound to 'W' in Image Mode by default.</code></pre><p class=" text-justify">I live in Emacs; it may as well be my wallpaper.</p><pre><code>'image-transform-fit-to-{height,width}' are now obsolete. Use the new command 'image-transform-fit-to-window' instead. The keybinding for 'image-transform-fit-to-width' is now 's i'.</code></pre><pre><code>User option 'image-auto-resize' can now be set to 'fit-window'. This works like 'image-transform-fit-to-window'.</code></pre><pre><code>New user option 'image-auto-resize-max-scale-percent'. The new 'fit-window' option will never scale an image more than this much (in percent). It is nil by default, which means no limit.</code></pre><pre><code>New user option 'image-text-based-formats'. This controls whether or not to show a message, when opening certain image formats, explaining how to edit it as text. The default is to show this message for SVG and XPM.</code></pre><pre><code>New command 'image-transform-set-percent'. It allows resizing the image to a percentage of its original size, and is bound to 's p' in Image mode.</code></pre><pre><code>'image-transform-original' renamed to 'image-transform-reset-to-original'. The old name was confusing, and is now an obsolete function alias.</code></pre><pre><code>'image-transform-reset' renamed to 'image-transform-reset-to-initial'. The old name was confusing, and is now an obsolete function alias.</code></pre><H3 id="images">Images</H3><pre><code>New commands 'image-crop' and 'image-cut'. These commands allow interactively cropping/cutting the image at point. The commands are bound to keys 'i c' and 'i x' (respectively) in the local keymap over images. They rely on external programs, by default "convert" from ImageMagick, to do the actual cropping/eliding of the image file.</code></pre><pre><code>New commands 'image-flip-horizontally' and 'image-flip-vertically'. These commands horizontally and vertically flip the image under point, and are bound to 'i h' and 'i v', respectively.</code></pre><pre><code>Users can now add special image conversion functions. This is done via 'image-converter-add-handler'.</code></pre><H3 id="image-dired">Image Dired</H3><p class=" text-justify">Yep, image dired is a thing. It’ll generate thumbnails of images and display them alongside the file.</p><pre><code>'image-dired-image-mode' is now based on 'image-mode'. This avoids converting images in the background, and makes Image-Dired noticeably faster. New keybindings from 'image-mode' are now available in the "*image-dired-display-image*" buffer; press '?' or 'h' in that buffer to see the full list.</code></pre><p class=" text-justify">This is just a consolidation of code.</p><pre><code>Navigation and marking commands now work in image display buffer. The following new bindings have been added: - 'n', 'SPC' =&gt; 'image-dired-display-next' - 'p', 'DEL' =&gt; 'image-dired-display-previous' - 'm' =&gt; 'image-dired-mark-thumb-original-file' - 'd' =&gt; 'image-dired-flag-thumb-original-file' - 'u' =&gt; 'image-dired-unmark-thumb-original-file'</code></pre><p class=" text-justify">They have been sorely missed. I never quite understood why they couldn’t do this before. I ended up writing my own glue code to do this.</p><pre><code>New command 'image-dired-unmark-all-marks'. It removes all marks from all files in the thumbnail and the associated Dired buffer, and is bound to 'U' in the thumbnail and display buffer.</code></pre><pre><code>New command 'image-dired-do-flagged-delete'. It deletes all flagged files, and is bound to 'x' in the thumbnail buffer. It replaces the command 'image-dired-delete-marked', which is now an obsolete alias.</code></pre><pre><code>New command 'image-dired-copy-filename-as-kill'. It copies the name of the marked or current image to the kill ring, and is bound to 'w' in the thumbnail buffer.</code></pre><pre><code>New command 'image-dired-wallpaper-set'. This command sets the desktop background to the image at point in the thumbnail buffer. It is bound to 'W' by default.</code></pre><pre><code>'image-dired-slideshow-start' is now bound to 'S'. It is bound in both the thumbnail and display buffer, and no longer prompts for a timeout; use a numerical prefix (e.g. 'C-u 8 S') to set the timeout.</code></pre><pre><code>New user option 'image-dired-marking-shows-next'. If this option is non-nil (the default), marking, unmarking or flagging an image in either the thumbnail or display buffer shows the next image.</code></pre><pre><code>New face 'image-dired-thumb-flagged'. If 'image-dired-thumb-mark' is non-nil (the default), this face is used for images that are flagged for deletion in the Dired buffer associated with Image-Dired.</code></pre><pre><code>Image information is now shown in the header line of the thumbnail buffer. This replaces the message that most navigation commands in the thumbnail buffer used to show at the bottom of the screen.</code></pre><pre><code>New specifiers for 'image-dired-display-properties-format'. This is used to format the new header line. The new specifiers are: "%d" for the name of the directory that the file is in, "%n" for file's number in the thumbnail buffer, and "%s" for the file size. The default format has been updated to use this. If you prefer the old format, add this to your Init file: (setopt image-dired-display-properties-format "%b: %f (%t): %c")</code></pre><pre><code>New faces for the header line of the thumbnail buffer. These faces correspond to different parts of the header line, as specified in 'image-dired-display-properties-format': - 'image-dired-thumb-header-directory-name' - 'image-dired-thumb-header-file-name' - 'image-dired-thumb-header-file-size' - 'image-dired-thumb-header-image-count'</code></pre><pre><code>PDF support. Image-Dired now displays thumbnails for PDF files. Type 'RET' on a PDF file in the thumbnail buffer to visit the corresponding PDF.</code></pre><pre><code>Support GraphicsMagick command line tools. Support for the GraphicsMagick command line tool ("gm") has been added, and is used when it is available instead of ImageMagick.</code></pre><pre><code>Support Thumbnail Managing Standard v0.9.0 (Dec 2020). This standard allows sharing generated thumbnails across different programs. Version 0.9.0 adds two larger thumbnail sizes: 512x512 and 1024x1024 pixels. See the user option 'image-dired-thumbnail-storage' to use it; it is not enabled by default.</code></pre><pre><code>Reduce dependency on external "exiftool" program. The 'image-dired-copy-with-exif-file-name' command no longer requires an external "exiftool" program to be available. The user options 'image-dired-cmd-read-exif-data-program' and 'image-dired-cmd-read-exif-data-options' are now obsolete.</code></pre><pre><code>Support for bookmark.el. The command 'bookmark-set' (bound to 'C-x r m') is now supported in the thumbnail view, and will create a bookmark that opens the current directory in Image-Dired.</code></pre><p class=" text-justify">Nifty addition to Emacs’s general-purpose bookmarking tool.</p><pre><code>The 'image-dired-slideshow-start' command no longer prompts. It no longer inconveniently prompts for a number of images and a delay: it runs indefinitely, but stops automatically on any command. You can set the delay with a prefix argument, or a negative prefix argument to prompt for a delay. Customize the user option 'image-dired-slideshow-delay' to change the default from 5 seconds.</code></pre><pre><code>'image-dired-show-all-from-dir-max-files' increased to 1000. This user option controls asking for confirmation when starting Image-Dired in a directory with many files. Since Image-Dired creates thumbnails in the background in recent versions, this is not as important as it used to be. You can now also customize this option to nil to disable this confirmation completely.</code></pre><pre><code>'image-dired-thumb-size' increased to 128.</code></pre><pre><code>'image-dired-db-file' renamed to 'image-dired-tags-db-file'.</code></pre><pre><code>'image-dired-display-image-mode' renamed to 'image-dired-image-mode'. The corresponding keymap is now named 'image-dired-image-mode-map'.</code></pre><pre><code>Some commands have been renamed to be shorter. - 'image-dired-display-thumbnail-original-image' has been renamed to 'image-dired-display-this'. - 'image-dired-display-next-thumbnail-original' has been renamed to 'image-dired-display-next'. - 'image-dired-display-previous-thumbnail-original' has been renamed to 'image-dired-display-previous'. The old names are now obsolete aliases.</code></pre><pre><code>'image-dired-thumb-{height,width}' are now obsolete. Customize 'image-dired-thumb-size' instead, which will set both the height and width.</code></pre><pre><code>HTML image gallery generation is now obsolete. The 'image-dired-gallery-generate' command and these user options are now obsolete: 'image-dired-gallery-thumb-image-root-url', 'image-dired-gallery-hidden-tags', 'image-dired-gallery-dir', 'image-dired-gallery-image-root-url'.</code></pre><pre><code>'image-dired-rotate-thumbnail-{left,right}' are now obsolete. Instead, use commands 'image-dired-refresh-thumb' to generate a new thumbnail, or 'image-rotate' to rotate the thumbnail without updating the thumbnail file.</code></pre><pre><code>Some commands and user options are now obsolete. Since 'image-dired-display-image-mode' is now based on 'image-mode', some commands and user options are no longer needed and are now obsolete: 'image-dired-cmd-create-temp-image-options', 'image-dired-cmd-create-temp-image-program', 'image-dired-display-current-image-full', 'image-dired-display-current-image-sized', 'image-dired-display-window-height-correction', 'image-dired-display-window-width-correction', 'image-dired-temp-image-file'.</code></pre><pre><code>New function 'exif-field'. This is a convenience function to extract the field data from 'exif-parse-file' and 'exif-parse-buffer'.</code></pre><H3 id="bookmarks">Bookmarks</H3><p class=" text-justify">Bookmarks is Emacs’s general-purpose bookmark facility. Nearly anything can be bookmarked, recalled and edited. It’s a powerful feature, and greatly underused. Try bookmarking: files, eww buffers, info node buffers, dired, image dired, and more.</p><pre><code>'list-bookmarks' now includes a type column. Types are registered via a 'bookmark-handler-type' symbol property on the jumping function.</code></pre><pre><code>'bookmark-sort-flag' can now be set to 'last-modified'. This will display bookmark list from most recently set to least recently set.</code></pre><pre><code>When editing a bookmark annotation, 'C-c C-k' will now cancel. It is bound to the new command 'bookmark-edit-annotation-cancel'.</code></pre><pre><code>New user option 'bookmark-fringe-mark'. This option controls the bitmap used to indicate bookmarks in the fringe (or nil to disable showing this marker).</code></pre><H3 id="xwidget">Xwidget</H3><p class=" text-justify">Xwidget is an optional, compiled module that – in theory – grants access to ‘widgets’ like Chromium in Emacs. I say <em>in theory</em>, as it can be quite unstable.</p><pre><code>New user option 'xwidget-webkit-buffer-name-format'. This option controls how xwidget-webkit buffers are named.</code></pre><pre><code>New user option 'xwidget-webkit-cookie-file'. This option controls whether the xwidget-webkit buffers save cookies set by web pages, and if so, in which file to save them.</code></pre><pre><code>New minor mode 'xwidget-webkit-edit-mode'. When this mode is enabled, self-inserting characters and other common web browser shortcut keys are redefined to send themselves to the WebKit widget.</code></pre><pre><code>New minor mode 'xwidget-webkit-isearch-mode'. This mode acts similarly to incremental search, and allows searching the contents of a WebKit widget. In xwidget-webkit mode, it is bound to 'C-s' and 'C-r'.</code></pre><p class=" text-justify">Seems useful, if it behaves like isearch does elsewhere.</p><pre><code>New command 'xwidget-webkit-browse-history'. This command displays a buffer containing the page load history of the current WebKit widget, and allows you to navigate it.</code></pre><pre><code>On X, the WebKit inspector is now available inside xwidgets. To access the inspector, right click on the widget and select "Inspect Element".</code></pre><pre><code>"Open in New Window" in a WebKit widget's context menu now works. The newly created buffer will be displayed via 'display-buffer', which can be customized through the usual mechanism of 'display-buffer-alist' and friends.</code></pre><H3 id="tramp">Tramp</H3><p class=" text-justify">Tramp is the remote connection tool in Emacs. It’s excellent, and one of my favorite features.</p><pre><code>New connection methods "docker", "podman" and "kubernetes". They allow accessing containers provided by Docker and similar programs.</code></pre><p class=" text-justify">Great feature additions. Now you can trivially <code>cd</code> into a docker container from <a href="/article/complete-guide-mastering-eshell" class=" article-link">Eshell</a>, from <code>find-file</code>, or anywhere else that accepts a filepath.</p><pre><code>Tramp supports abbreviating remote home directories now. When calling 'abbreviate-file-name' on a Tramp file name, the result will abbreviate the user's home directory, for example by abbreviating "/ssh:user@host:/home/user" to "/ssh:user@host:~".</code></pre><pre><code>New user option 'tramp-use-scp-direct-remote-copying'. When set to non-nil, Tramp does not copy files between two remote hosts via a local copy in its temporary directory, but lets the 'scp' command do this job.</code></pre><pre><code>Proper password prompts for methods "doas", "sudo" and "sudoedit". The password prompts for these methods reflect now the credentials of the user requesting such a connection, and not of the user who is the target. This has always been needed, just the password prompt and the related 'auth-sources' entry were wrong.</code></pre><pre><code>New user option 'tramp-completion-use-cache'. During user and host name completion in the minibuffer, results from Tramp's connection cache are taken into account. This can be disabled by setting the user option 'tramp-completion-use-cache' to nil.</code></pre><H3 id="browse-url">Browse URL</H3><p class=" text-justify">Browse url opens up an URL in your browser.</p><pre><code>New user option 'browse-url-default-scheme'. This user option decides which URL scheme that 'browse-url' and related functions will use by default. For example, you could customize this to "https" to always prefer HTTPS URLs.</code></pre><pre><code>New user option 'browse-url-irc-function'. This option specifies a function for opening "irc://" links. It defaults to the new function 'browse-url-irc'.</code></pre><pre><code>New function 'browse-url-irc'. This multipurpose autoloaded function can be used for opening "irc://" and "ircs://" URLS by any caller that passes a URL string as an initial arg.</code></pre><pre><code>Support for the Netscape web browser has been removed. This support has been obsolete since Emacs 25.1. The final version of the Netscape web browser was released in February, 2008.</code></pre><pre><code>Support for the Galeon web browser has been removed. This support has been obsolete since Emacs 25.1. The final version of the Galeon web browser was released in September, 2008.</code></pre><pre><code>Support for the Mozilla web browser is now obsolete. Note that this historical web browser is different from Mozilla Firefox; it is its predecessor.</code></pre><H3 id="python-mode">Python Mode</H3><pre><code>Project shells and a new user option 'python-shell-dedicated'. When called with a prefix argument, 'run-python' now offers the choice of creating a shell dedicated to the current project. This shell runs in the project root directory and is shared among all project buffers. Without a prefix argument, the kind of shell (buffer-dedicated, project-dedicated or global) is specified by the new 'python-shell-dedicated' user option.</code></pre><p class=" text-justify">A welcome addition. This is the benefit of having a builtin <em>and</em> well-supported project tool in Emacs.</p><H3 id="ruby-mode">Ruby Mode</H3><pre><code>New user option 'ruby-toggle-block-space-before-parameters'.</code></pre><pre><code>Support for endless methods.</code></pre><pre><code>New user options that determine indentation logic. 'ruby-method-params-indent', 'ruby-block-indent', 'ruby-after-operator-indent', 'ruby-method-call-indent', 'ruby-parenless-call-arguments-indent'. See the docstrings for explanations and examples.</code></pre><H3 id="eshell-1">Eshell</H3><p class=" text-justify">Eshell is Emacs’s shell written in elisp.</p><pre><code>New feature to easily bypass Eshell's own pipelining. Prefixing '|', '&lt;' or '&gt;' with an asterisk, i.e. '*|', '*&lt;' or '*&gt;', will cause the whole command to be passed to the operating system shell. This is particularly useful to bypass Eshell's own pipelining support for pipelines which will move a lot of data. See section "Running Shell Pipelines Natively" in the Eshell manual, node "(eshell) Pipelines".</code></pre><p class=" text-justify">Pipelining in eshell was never all that feature complete; it could also be slow. These redirections allow us to circumvent eshell in favor of a shell that implements it correctly – and expediently.</p><pre><code>New module to help supplying absolute file names to remote commands. After enabling the new 'eshell-elecslash' module, typing a forward slash as the first character of a command line argument will automatically insert the Tramp prefix. The automatic insertion applies only when 'default-directory' is remote and the command is a Lisp function. This frees you from having to keep track of whether commands are Lisp function or external when supplying absolute file name arguments. See the "(eshell) Electric forward slash" node in the Eshell manual for details.</code></pre><p class=" text-justify">One annoyance about <code>cd</code>’ing into a remote directory in Tramp was always how easy it was to accidentally <code>cd</code> <em>out</em> of it again. This will go some way towards alleviating that.</p><pre><code>Improved support for redirection operators in Eshell. Eshell now supports a wider variety of redirection operators. For example, you can now redirect both stdout and stderr via '&amp;&gt;' or duplicate one output handle to another via 'NEW-FD&gt;&amp;OLD-FD'. For more information, see the "(eshell) Redirection" node in the Eshell manual.</code></pre><pre><code>New eshell built-in command 'doas'. The privilege-escalation program 'doas' has been added to the existing 'su' and 'sudo' commands from the 'eshell-tramp' module. The external command may still be accessed by using '*doas'.</code></pre><p class=" text-justify"><code>doas</code> is slowly taking over as a smaller – safer – alternative to the sprawling <code>sudo</code> program.</p><pre><code>Double-quoting an Eshell expansion now treats the result as a single string. If an Eshell expansion like '$FOO' is surrounded by double quotes, the result will always be a single string, no matter the type that would otherwise be returned.</code></pre><pre><code>Concatenating Eshell expansions now works more similarly to other shells. When concatenating an Eshell expansion that returns a list, "adjacent" elements of each operand are now concatenated together, e.g. '$(list "a" "b")c' returns '("a" "bc")'. See the "(eshell) Expansion" node in the Eshell manual for more details.</code></pre><pre><code>Eshell subcommands with multiline numeric output return lists of numbers. If every line of the output of an Eshell subcommand like '${COMMAND}' is numeric, the result will be a list of numbers (or a single number if only one line of output). Previously, this only converted numbers when there was a single line of output.</code></pre><pre><code>Built-in Eshell commands now follow Posix/GNU argument syntax conventions. Built-in commands in Eshell now accept command-line options with values passed as a single token, such as '-oVALUE' or '--option=VALUE'. New commands can take advantage of this with the 'eshell-eval-using-options' macro. See "Defining new built-in commands" in the "(eshell) Built-ins" node of the Eshell manual.</code></pre><pre><code>Eshell globs ending with "/" now match only directories. Additionally, globs ending with "**/" or "***/" no longer raise an error, and now expand to all directories recursively (following symlinks in the latter case).</code></pre><pre><code>Lisp forms in Eshell now treat a nil result as a failed exit status. When executing a command that looks like '(lisp form)' and returns nil, Eshell will set the exit status (available in the '$?' variable) to 2. This allows commands like that to be used in conditionals. To change this behavior, customize the new 'eshell-lisp-form-nil-is-failure' user option.</code></pre><H3 id="shell">Shell</H3><p class=" text-justify">Shell mode is Emacs’s wrapper around a common shell, usually bash or the cmd.exe on Windows.</p><pre><code>New user option 'shell-kill-buffer-on-exit'. Enabling this will automatically kill a "*shell*" buffer as soon as the shell session terminates.</code></pre><p class=" text-justify">Nope, not for me. I accidentally kill or terminate my shell, and I don’t want to lose the output of the buffer as a result of that.</p><pre><code>New minor mode 'shell-highlight-undef-mode'. Customize 'shell-highlight-undef-enable' to t if you want to enable this minor mode in "*shell*" buffers. It will highlight undefined commands with a warning face as you type.</code></pre><p class=" text-justify">Seems useful. I’d have to use it for a while to determine how reliable it is.</p><H3 id="calc">Calc</H3><p class=" text-justify"><code>M-x calc</code> is Emacs’s all-powerful symbolic RPN calculator. It’s fantastically advanced, and well worth learning if you do a lot of mathematics.</p><pre><code>New user option 'calc-kill-line-numbering'. Set it to nil to exclude line numbering from kills and copies.</code></pre><p class=" text-justify">I always found that annoying, so I am glad there is now a switch to disable it.</p><H3 id="hierarchy">Hierarchy</H3><p class=" text-justify">Hierarchy is a tree list user interface component.</p><pre><code>Tree Display can delay computation of children. 'hierarchy-add-tree' and 'hierarchy-add-trees' have an optional argument which allows tree-widget display to be activated and computed only when the user expands the node.</code></pre><p class=" text-justify">Being able to thunk the calculations of the child node matches should certainly improve performance (or delay computation until it is needed, anyway) but I do not know of too many packages that use hierarchy.</p><H3 id="proced">Proced</H3><p class=" text-justify">Proced is Emacs’s answer to <code>top</code> and <code>ps</code>. <a href="/article/displaying-interacting-processes-proced" class=" article-link">Displaying and Interacting with processes using Proced</a> is a good place to start if you want to learn more.</p><pre><code>proced.el shows system processes of remote hosts. When 'default-directory' is remote, and 'proced' is invoked with a negative argument like 'C-u - proced', the system processes of that remote host are shown. Alternatively, the user option 'proced-show-remote-processes' can be set to non-nil. 'proced-signal-function' has been marked obsolete.</code></pre><p class=" text-justify">Nifty! You can built a neat little server dashboard with a couple of keyboard macros and some window configurations.</p><pre><code>Proced can now optionally show process details in color. New user option 'proced-enable-color-flag' enables coloring of Proced buffers. This option is disabled by default; customize it to a non-nil value to enable colors.</code></pre><p class=" text-justify">I’ve enabled it, as it does make things easier to read.</p><H3 id="miscellaneous-1">Miscellaneous</H3><pre><code>New user option 'webjump-use-internal-browser'. When non-nil, WebJump will use an internal browser to open web pages, instead of the default external browser.</code></pre><pre><code>New user option 'font-lock-ignore'. This option provides a mechanism to selectively disable font-lock keyword-driven fontifications.</code></pre><p class=" text-justify">I can’t immediately discern if this works with tree-sitter based font locking. Probably not, as it uses <code>font-lock-keywords</code> to drive it.</p><pre><code>New user option 'auto-save-visited-predicate'. This user option is a predicate function which is called by 'auto-save-visited-mode' to decide whether or not to save a buffer. You can use it to automatically save only specific buffers, for example buffers using a particular mode or in some directory.</code></pre><pre><code>New user option 'remote-file-name-inhibit-auto-save-visited'. If this user option is non-nil, 'auto-save-visited-mode' will not auto-save remote buffers. The default is nil.</code></pre><pre><code>New package vtable.el for formatting tabular data. This package allows formatting data using variable-pitch fonts. The resulting tables can display text in variable pitch fonts, text using fonts of different sizes, and images. See the "(vtable) Top" manual for more details.</code></pre><p class=" text-justify">Emacs had long had a package that can tabulate data: <code>tabulated-list.el</code>. This new, and improved version, adds a range of new features.</p><pre><code>New minor mode 'elide-head-mode'. Enabling this minor mode turns on hiding header material, like 'elide-head' does; disabling it shows the header. The commands 'elide-head' and 'elide-head-show' are now obsolete.</code></pre><pre><code>New package ansi-osc.el. Support for OSC ("Operating System Command") escape sequences has been extracted from comint.el in order to provide interpretation of OSC sequences in compilation buffers. Adding the new function 'ansi-osc-compilation-filter' to 'compilation-filter-hook' enables interpretation of OSC escape sequences in compilation buffers. By default, all sequences are filtered out. The list of handlers (already covering OSC 7 and 8) has been extended with a handler for OSC 2, the command to set a window title.</code></pre><p class=" text-justify">Emacs 27 added better directory tracking support for OSC 8. I wrote about <a href="/article/running-shells-in-emacs-overview" class=" article-link">how to set it up for Shell-mode</a>.</p><pre><code>'recentf-mode' now uses abbreviated file names by default. This means that e.g. "/home/foo/bar" is now displayed as "~/bar". Customize the user option 'recentf-filename-handlers' to nil to get back the old behavior.</code></pre><p class=" text-justify">Recent Files is a capable, little feature. It can recall recently used files at will.</p><pre><code>New command 'recentf-open'. This command prompts for a recently opened file in the minibuffer, and visits it.</code></pre><pre><code>'ffap-machine-at-point' no longer pings hosts by default. It will now simply look at a hostname to determine if it is valid, instead of also trying to ping it. Customize the user option 'ffap-machine-p-known' to 'ping' to get the old behavior back.</code></pre><pre><code>The 'run-dig' command is now obsolete; use 'dig' instead.</code></pre><pre><code>Some 'bib-mode' commands and variables have been renamed. To respect Emacs naming conventions, the variable 'unread-bib-file' has been renamed to 'bib-unread-file'. The following commands have also been renamed: 'addbib' to 'bib-add' 'return-key-bib' to 'bib-return-key' 'mark-bib' to 'bib-mark' 'unread-bib' to 'bib-unread'</code></pre><pre><code>'outlineify-sticky' command is renamed to 'allout-outlinify-sticky'. The old name is still available as an obsolete function alias.</code></pre><pre><code>The url-irc library now understands "ircs://" links.</code></pre><pre><code>New command 'world-clock-copy-time-as-kill' for 'world-clock-mode'. It copies the current line into the kill ring.</code></pre><p class=" text-justify">I betcha didn’t know Emacs had a <code>M-x world-clock-mode</code>. It goes well with <code>M-x sunrise-sunset</code> and <code>M-x phases-of-moon</code>.</p><pre><code>'edit-abbrevs' now uses font-locking. The new face 'abbrev-table-name' is used to display the abbrev table name.</code></pre><pre><code>New key binding 'O' in "*Buffer List*". This key is now bound to 'Buffer-menu-view-other-window', which will view this line's buffer in View mode in another window.</code></pre><p class=" text-justify">Nice to have. You should just use <code>M-x ibuffer</code> though.</p><H3 id="scheme-mode">Scheme Mode</H3><pre><code>Auto-detection of Scheme library files. Emacs now automatically enables the Scheme mode when opening R6RS Scheme Library Source (".sls") files and R7RS Scheme Library Definition (".sld") files.</code></pre><pre><code>Imenu members for R6RS and R7RS library members. Imenu now lists the members directly nested in R6RS Scheme libraries ('library') and R7RS libraries ('define-library'). </code></pre><H3 id="new-modes-and-packages-in-emacs-29.1">New Modes and Packages in Emacs 29.1</H3><pre><code>Eglot: Emacs Client for the Language Server Protocol. Emacs now comes with the Eglot package, which enhances various Emacs features, such as completion, documentation, error detection, etc., based on data provided by language servers using the Language Server Protocol (LSP). See the new Info manual "(eglot) Top" for more. Also see "etc/EGLOT-NEWS". If you want to be able to use 'package-install' to upgrade Eglot to newer versions released on GNU ELPA, customize the new option 'package-install-upgrade-built-in' to a non-nil value.</code></pre><p class=" text-justify">You should <em>definitely</em> read the manual for this package if you want to use it. You’ll want to make sure you have the right language server, et al. But, generally speaking, <code>M-x eglot</code> is all you need.</p><pre><code>use-package: Declarative package configuration. use-package is now shipped with Emacs. It provides the 'use-package' macro, which allows you to isolate package configuration in your init file in a way that is declarative, tidy, and performance-oriented. See the new Info manual "(use-package) Top" for more. If you want to be able to use 'package-install' to upgrade use-package to newer versions released on GNU ELPA, customize the new option 'package-install-upgrade-built-in' to a non-nil value.</code></pre><pre><code>New package 'wallpaper'. This package provides the command 'wallpaper-set', which sets the desktop background image. Depending on the system and the desktop, this may require an external program (such as "swaybg", "gm", "display" or "xloadimage"). If so, a suitable command should be detected automatically in most cases. It can also be customized manually if needed, using the new user options 'wallpaper-command' and 'wallpaper-command-args'.</code></pre><p class=" text-justify">A fine complement to <code>M-x zone</code>, Emacs’s screensaver.</p><pre><code>New package 'oclosure'. This allows the creation of OClosures, which are "functions with slots" or "function objects" that expose additional information about themselves. Use the new macros 'oclosure-define' and 'oclosure-lambda' to create OClosures. See the "(elisp) OClosures" node for more information.</code></pre><pre><code>New generic function 'oclosure-interactive-form'. Used by 'interactive-form' when called on an OClosure. This allows specific OClosure types to compute their interactive specs on demand rather than precompute them when created.</code></pre><pre><code>New theme 'leuven-dark'. This is a dark version of the 'leuven' theme.</code></pre><pre><code>New mode 'erts-mode'. This mode is used to edit files geared towards testing actions in Emacs buffers, like indentation and the like. The new ert function 'ert-test-erts-file' is used to parse these files.</code></pre><p class=" text-justify">One of the hardest things about writing tests for a text editor package is having to keep tabs on the point, the text, the window layout, etc. <em>Anything</em> that helps make that easier is a win for everyone.</p><pre><code>New major mode 'js-json-mode'. This is a lightweight variant of 'js-mode' that is used by default when visiting JSON files.</code></pre><pre><code>New major mode 'csharp-mode'. A major mode based on CC Mode for editing programs in the C# language. This mode is auto-enabled for files with the ".cs" extension.</code></pre><pre><code>New major modes based on the tree-sitter library. These new major modes are available if Emacs was built with the tree-sitter library. They provide support for font-locking, indentation, and navigation by defuns based on parsing the buffer text by a tree-sitter parser. Some major modes also offer support for Imenu and 'which-func'. The new modes based on tree-sitter are for now entirely optional, and you must turn them on manually, or load them in your init file, or customize 'auto-mode-alist' to turn them on automatically for certain files. You can also customize 'major-mode-remap-alist' to automatically turn on some tree-sitter based modes for the same files for which a "built-in" mode would be turned on. For example: (add-to-list 'major-mode-remap-alist '(ruby-mode . ruby-ts-mode)) If you try these modes and don't like them, you can go back to the "built-in" modes by restarting Emacs. (If you use desktop.el to save and restore Emacs sessions, make sure no buffer under these modes is recorded in the desktop file, before restarting.) But please tell us why you didn't like the tree-sitter based modes, so that we could try improving them. Each major mode based on tree-sitter needs a language grammar library, usually named "libtree-sitter-LANG.so" ("libtree-sitter-LANG.dll" on MS-Windows), where LANG is the corresponding language name. Emacs looks for these libraries in the following places: . in the directories mentioned in the list 'treesit-extra-load-path' . in the "tree-sitter" subdirectory of your 'user-emacs-directory' (by default, "~/.emacs.d/tree-sitter") . in the standard system directories where other shared libraries are usually installed We recommend to install these libraries in one of the standard system locations (the last place in the above list). If a language grammar library required by a mode is not found in any of the above places, the mode will display a warning when you try to turn it on.</code></pre><p class=" text-justify">This is a good summary of what you need to do to start using tree-sitter. But, as I’ve said before, it can be a bit gnarly if you haven’t done this sort of thing before. My guide to <a href="/article/how-to-get-started-tree-sitter" class=" article-link">tree-sitter</a> is a good place to start.</p><pre><code>New major mode 'typescript-ts-mode'. A major mode based on the tree-sitter library for editing programs in the TypeScript language.</code></pre><pre><code>New major mode 'tsx-ts-mode'. A major mode based on the tree-sitter library for editing programs in the TypeScript language, with support for TSX.</code></pre><p class=" text-justify">Both TSX/TS modes are welcome additions.</p><pre><code>New major mode 'c-ts-mode'. An optional major mode based on the tree-sitter library for editing programs in the C language.</code></pre><pre><code>New major mode 'c++-ts-mode'. An optional major mode based on the tree-sitter library for editing programs in the C++ language.</code></pre><pre><code>New command 'c-or-c++-ts-mode'. A command that automatically guesses the language of a header file, and enables either 'c-ts-mode' or 'c++-ts-mode' accordingly.</code></pre><pre><code>New major mode 'java-ts-mode'. An optional major mode based on the tree-sitter library for editing programs in the Java language.</code></pre><pre><code>New major mode 'python-ts-mode'. An optional major mode based on the tree-sitter library for editing programs in the Python language.</code></pre><pre><code>New major mode 'css-ts-mode'. An optional major mode based on the tree-sitter library for editing CSS (Cascading Style Sheets).</code></pre><pre><code>New major mode 'json-ts-mode'. An optional major mode based on the tree-sitter library for editing programs in the JSON language.</code></pre><pre><code>New major mode 'csharp-ts-mode'. An optional major mode based on the tree-sitter library for editing programs in the C# language.</code></pre><pre><code>New major mode 'bash-ts-mode'. Am optional major mode based on the tree-sitter library for editing Bash shell scripts.</code></pre><pre><code>New major mode 'dockerfile-ts-mode'. A major mode based on the tree-sitter library for editing Dockerfiles.</code></pre><p class=" text-justify">This is Emacs’s first Dockerfile mode. There is no regular mode for it.</p><pre><code>New major mode 'cmake-ts-mode'. A major mode based on the tree-sitter library for editing CMake files.</code></pre><pre><code>New major mode 'toml-ts-mode'. An optional major mode based on the tree-sitter library for editing files written in TOML, a format for writing configuration files.</code></pre><pre><code>New major mode 'go-ts-mode'. A major mode based on the tree-sitter library for editing programs in the Go language.</code></pre><pre><code>New major mode 'go-mod-ts-mode'. A major mode based on the tree-sitter library for editing "go.mod" files.</code></pre><pre><code>New major mode 'yaml-ts-mode'. A major mode based on the tree-sitter library for editing files written in YAML.</code></pre><p class=" text-justify">This mode is a bit lacking in many areas. There is no indentation, for instance. You may want to use the third-party <code>yaml</code> mode instead.</p><pre><code>New major mode 'rust-ts-mode'. A major mode based on the tree-sitter library for editing programs in the Rust language.</code></pre><pre><code>New major mode 'ruby-ts-mode'. An optional major mode based on the tree-sitter library for editing programs in the Ruby language.</code></pre><H3 id="incompatible-lisp-changes-in-emacs-29.1">Incompatible Lisp Changes in Emacs 29.1</H3><pre><code>The implementation of overlays has changed. Emacs now uses an implementation of overlays that is much more efficient than the original one, and should speed up all the operations that involve overlays, especially when there are lots of them in a buffer. As result of this, some minor incompatibilities in behavior could be observed, as described below. Except those minor incompatibilities, no other changes in behavior of overlays should be visible on the Lisp or user level, with the exception of better performance and the order of overlays returned by functions that don't promise any particular order.</code></pre><p class=" text-justify">Overlays are ranges of positions in Emacs that you can apply to a buffer and assign all manner of metadata and contextual information. This is how a wide range of Emacs’s features work: from flymake errors to buttons to many other things. Speedups and efficiency gains here pay dividends everywhere else.</p><pre><code>The function 'overlay-recenter' is now a no-op. This function does nothing, and in particular has no effect on the value returned by 'overlay-lists'. The purpose of 'overlay-recenter' was to allow more efficient lookup of overlays around a certain buffer position; however with the new implementation the lookup of overlays is efficient regardless of their position, and there's no longer any need to "optimize" the lookup, nor any notion of a "center" of the overlays.</code></pre><pre><code>The function 'overlay-lists' returns one unified list of overlays. This function used to return a cons of two lists, one with overlays before the "center" position, the other after that "center". It now returns a list whose 'car' is the list of all the buffer overlays, and whose 'cdr' is always nil.</code></pre><pre><code>'format-prompt' now uses 'substitute-command-keys'. This means that both the prompt and 'minibuffer-default-prompt-format' will have key definitions and single quotes handled specially.</code></pre><pre><code>New function 'substitute-quotes'. This function works like 'substitute-command-keys' but only substitutes quote characters.</code></pre><pre><code>'find-image' now uses 'create-image'. This means that images found through 'find-image' also have auto-scaling applied. (This only makes a difference on HiDPI displays.)</code></pre><pre><code>Changes in how "raw" in-memory XBM images are specified. Some years back Emacs gained the ability to scale images, and you could then specify ':width' and ':height' when using 'create-image' on all image types -- except XBM images, because this format already used the ':width' and ':height' arguments to specify the width/height of the "raw" in-memory format. This meant that if you used these specifications on, for instance, XBM files, Emacs would refuse to display them. This has been changed, and ':width'/':height' now works as with all other image formats, and the way to specify the width/height of the "raw" in-memory format is now by using ':data-width' and ':data-height'.</code></pre><pre><code>"loaddefs.el" generation has been reimplemented. The various "loaddefs.el" files in the Emacs tree (which contain information about autoloads, built-in packages and package prefixes) used to be generated by functions in autoloads.el. These are now generated by loaddefs-gen.el instead. This leads to functionally equivalent "loaddefs.el" files, but they do not use exactly the same syntax, so using 'M-x update-file-autoloads' no longer works. (This didn't work well in most files in the past, either, but it will now signal an error in any file.) In addition, files are scanned in a slightly different way. Previously, ';;;###' specs inside a top-level form (i.e., something like '(when ... ;;;### ...)' would be ignored. They are now parsed as usual.</code></pre><pre><code>Themes have special autoload cookies. All built-in themes are scraped for ';;;###theme-autoload' cookies that are loaded along with the regular auto-loaded code.</code></pre><pre><code>'buffer-modified-p' has been extended. This function was previously documented to return only nil or t. This has been changed to nil/'autosaved'/non-nil. The new 'autosaved' value means that the buffer is modified, but that it hasn't been modified since the time of last auto-save.</code></pre><pre><code>'with-silent-modifications' also restores buffer autosave status. 'with-silent-modifications' is a macro meant to be used by the font locking machinery to allow applying text properties without changing the modification status of the buffer. However, it didn't restore the buffer autosave status, so applying font locking to a modified buffer that had already been auto-saved would trigger another auto-saving. This is no longer the case.</code></pre><pre><code>'prin1' doesn't always escape "." and "?" in symbols any more. Previously, symbols like 'foo.bar' would be printed by 'prin1' as "foo\.bar". This now prints as "foo.bar" instead. The Emacs Lisp reader interprets these strings as referring to the same symbol, so this is virtually always backwards-compatible, but there may theoretically be code out there that expects a specific printed representation. The same is the case with the "?" character: The 'foo?' symbol is now printed as "foo?" instead of "foo\?". If the "." and "?" characters are the first character in the symbol, they will still be escaped, so the '.foo' symbol is still printed as "\.foo" and the '?bar' symbol is still printed as "\?bar".</code></pre><pre><code>Remapping 'mode-line' face no longer works as expected. 'mode-line' is now the parent face of the new 'mode-line-active' face, and remapping parent of basic faces does not work reliably. Instead of remapping 'mode-line', you have to remap 'mode-line-active'.</code></pre><pre><code>'make-process' has been extended to support ptys when ':stderr' is set. Previously, setting ':stderr' to a non-nil value would force the process's connection to use pipes. Now, Emacs will use a pty for stdin and stdout if requested no matter the value of ':stderr'.</code></pre><p class=" text-justify">PTYs (pseudo-terminals) are generally much faster and better to use than pipes when you want to interface with a sub-process.</p><pre><code>User option 'mail-source-ignore-errors' is now obsolete. The whole mechanism for prompting users to continue in case of mail-source errors has been removed, so this option is no longer needed.</code></pre><H3 id="fonts">Fonts</H3><pre><code>Emacs now supports 'medium' fonts. Emacs previously didn't distinguish between the 'regular'/'normal' weight and the 'medium' weight, but it now also supports the (heavier) 'medium' weight. However, this means that if you specify a weight of 'normal' and the font doesn't have this weight, Emacs won't find the font spec. In these cases, replacing ":weight 'normal" with ":weight 'medium" should fix the issue.</code></pre><p class=" text-justify">Better and more sophisticated font rendering is always a positive. Emacs has made so many strides in this area over the years. Don’t forget that <code>M-x customize-face default</code> is still a good way to experiment with fonts.</p><pre><code>Keymap descriptions by Help commands have changed. 'help--describe-command', 'C-h b' and associated functions that output keymap descriptions have changed. In particular, prefix commands are not output at all, and instead of "??" for closures/functions, these functions output "[closure]"/"[lambda]". You can get back the old behavior of including prefix commands by customizing the new option 'describe-bindings-show-prefix-commands' to a non-nil value.</code></pre><pre><code>'downcase' details have changed slightly. In certain locales, changing the case of an ASCII-range character may turn it into a multibyte character, most notably with "I" in Turkish (the lowercase is "ı", 0x0131). Previously, 'downcase' on a unibyte string was buggy, and would mistakenly just return the lower byte of this, 0x31 (the digit "1"). 'downcase' on a unibyte string has now been changed to downcase such characters as if they were ASCII. To get proper locale-dependent downcasing, the string has to be converted to multibyte first. (This goes for the other case-changing functions, too.)</code></pre><pre><code>Functions in 'tramp-foreign-file-name-handler-alist' have changed. Functions to determine which Tramp file name handler to use are now passed a file name in dissected form (via 'tramp-dissect-file-name') instead of in string form.</code></pre><pre><code>'def' indentation changes. In 'emacs-lisp-mode', forms with a symbol with a name that start with "def" have been automatically indented as if they were 'defun'-like forms, for instance: (defzot 1 2 3) This heuristic has now been removed, and all functions/macros that want to be indented this way have to be marked with (declare (indent defun)) or the like. If the function/macro definition itself can't be changed, the indentation can also be adjusted by saying something like: (put 'defzot 'lisp-indent-function 'defun)</code></pre><pre><code>The 'inhibit-changing-match-data' variable is now obsolete. Instead, functions like 'string-match' and 'looking-at' now take an optional INHIBIT-MODIFY argument.</code></pre><pre><code>'gnus-define-keys' is now obsolete. Use 'define-keymap' instead.</code></pre><pre><code>MozRepl has been removed from js.el. MozRepl was removed from Firefox in 2017, so this code doesn't work with recent versions of Firefox.</code></pre><pre><code>The function 'image-dired-get-exif-data' is now obsolete. Use 'exif-parse-file' and 'exif-field' instead.</code></pre><pre><code>'insert-directory' alternatives should not change the free disk space line. This change is now applied in 'dired-insert-directory'.</code></pre><pre><code>'compilation-last-buffer' is (finally) declared obsolete. It has been obsolete since Emacs 22.1, actually.</code></pre><pre><code>Calling 'lsh' now elicits a byte-compiler warning. 'lsh' behaves in somewhat surprising and platform-dependent ways for negative arguments, and is generally slower than 'ash', which should be used instead. This warning can be suppressed by surrounding calls to 'lsh' with the construct '(with-suppressed-warnings ((suspicious lsh)) ...)', but switching to 'ash' is generally much preferable.</code></pre><pre><code>Some functions and variables obsolete since Emacs 24 have been removed: 'Buffer-menu-buffer+size-width', 'Electric-buffer-menu-mode', 'Info-edit-map', 'allout-abbreviate-flattened-numbering', 'allout-exposure-change-hook', 'allout-mode-deactivate-hook', 'allout-structure-added-hook', 'allout-structure-deleted-hook', 'allout-structure-shifted-hook', 'ansi-color-unfontify-region', 'archive-extract-hooks', 'auth-source-forget-user-or-password', 'auth-source-hide-passwords', 'auth-source-user-or-password', 'automatic-hscrolling', 'automount-dir-prefix', 'bibtex-complete', 'bibtex-entry-field-alist', 'buffer-has-markers-at', 'buffer-substring-filters', 'byte-compile-disable-print-circle', 'c-prepare-bug-report-hooks', 'cfengine-mode-abbrevs', 'change-log-acknowledgement', 'chart-map', 'checkdoc-comment-style-hooks', 'comint--unquote&amp;expand-filename', 'comint-dynamic-complete', 'comint-dynamic-complete-as-filename', 'comint-dynamic-simple-complete', 'comint-unquote-filename', 'command-history-map', 'compilation-parse-errors-function', 'completion-annotate-function', 'condition-case-no-debug', 'count-lines-region', 'crisp-mode-modeline-string', 'custom-print-functions', 'cvs-string-prefix-p', 'data-debug-map', 'deferred-action-function', 'deferred-action-list', 'dired-pop-to-buffer', 'dired-shrink-to-fit', 'dired-sort-set-modeline', 'dired-x-submit-report', 'display-buffer-function', 'ediff-choose-window-setup-function-automatically', 'eieio-defgeneric', 'eieio-defmethod', 'emacs-lock-from-exiting', 'erc-complete-word', 'erc-dcc-chat-filter-hook', 'eshell-add-to-window-buffer-names', 'eshell-cmpl-suffix-list', 'eshell-for', 'eshell-remove-from-window-buffer-names', 'eshell-status-in-modeline', 'filesets-cache-fill-content-hooks', 'font-list-limit', 'font-lock-maximum-size', 'font-lock-reference-face', 'gnus-carpal', 'gnus-debug-exclude-variables', 'gnus-debug-files', 'gnus-local-domain', 'gnus-outgoing-message-group', 'gnus-registry-user-format-function-M', 'gnus-secondary-servers', 'gnus-subscribe-newsgroup-hooks', 'gud-inhibit-global-bindings', 'hangul-input-method-inactivate', 'hfy-post-html-hooks', 'image-extension-data', 'image-library-alist', 'inactivate-current-input-method-function', 'inactivate-input-method', 'inhibit-first-line-modes-regexps', 'inhibit-first-line-modes-suffixes', 'input-method-inactivate-hook', 'intdos', 'javascript-generic-mode', 'javascript-generic-mode-hook', 'latex-string-prefix-p', 'macro-declaration-function' (function), 'macro-declaration-function' (variable), 'mail-complete', 'mail-complete-function', 'mail-mailer-swallows-blank-line', 'mail-sent-via', 'make-register', 'makefile-complete', 'menu-bar-kill-ring-save', 'meta-complete-symbol', 'meta-mode-map', 'mh-kill-folder-suppress-prompt-hooks', 'minibuffer-completing-symbol', 'minibuffer-local-filename-must-match-map', 'mode25', 'mode4350', 'mpc-string-prefix-p', 'msb-after-load-hooks', 'nndiary-request-accept-article-hooks', 'nndiary-request-create-group-hooks', 'nndiary-request-update-info-hooks', 'nnimap-split-rule', 'nntp-authinfo-file', 'ns-alternatives-map', 'ns-store-cut-buffer-internal', 'package-menu-view-commentary', 'pascal-last-completions', 'pascal-show-completions', 'pascal-toggle-completions', 'pcomplete-arg-quote-list', 'pcomplete-quote-argument', 'prolog-char-quote-workaround', 'python-buffer', 'python-guess-indent', 'python-indent', 'python-info-ppss-comment-or-string-p', 'python-info-ppss-context', 'python-info-ppss-context-type', 'python-preoutput-result', 'python-proc', 'python-send-receive', 'python-send-string', 'python-use-skeletons', 'quail-inactivate', 'quail-inactivate-hook', 'query-replace-interactive', 'rcirc-activity-hooks', 'rcirc-print-hooks', 'rcirc-receive-message-hooks', 'rcirc-sentinel-hooks', 'read-filename-at-point', 'redraw-modeline', 'reftex-index-map', 'reftex-index-phrases-map', 'reftex-select-bib-map', 'reftex-select-label-map', 'reftex-toc-map', 'register-name-alist', 'register-value', 'report-emacs-bug-info', 'report-emacs-bug-pretest-address', 'rmail-default-dont-reply-to-names', 'rmail-dont-reply-to', 'rmail-dont-reply-to-names', 'robin-inactivate', 'robin-inactivate-hook', 'rst-block-face', 'rst-comment-face', 'rst-definition-face', 'rst-directive-face', 'rst-emphasis1-face', 'rst-emphasis2-face', 'rst-external-face', 'rst-literal-face', 'rst-reference-face', 'semantic-change-hooks', 'semantic-edits-delete-change-hooks', 'semantic-edits-new-change-hooks', 'semantic-edits-reparse-change-hooks', 'semantic-grammar-map', 'semantic-grammar-syntax-table', 'semantic-lex-reset-hooks', 'semanticdb-elisp-sym-function-arglist', 'semanticdb-save-database-hooks', 'set-face-underline-p', 'set-register-value', 'sh-maybe-here-document', 'speedbar-key-map', 'speedbar-syntax-table', 'starttls-any-program-available', 'strokes-modeline-string', 'strokes-report-bug', 'term-default-bg-color', 'term-default-fg-color', 'tex-string-prefix-p', 'timeclock-modeline-display', 'timeclock-modeline-display', 'timeclock-update-modeline', 'toggle-emacs-lock', 'tooltip-use-echo-area', 'turn-on-cwarn-mode', 'turn-on-iimage-mode', 'ucs-input-inactivate', 'ucs-insert', 'url-recreate-url-attributes', 'user-variable-p', 'vc-string-prefix-p', 'vc-toggle-read-only', 'view-return-to-alist', 'view-return-to-alist-update', 'w32-default-color-map' (function), 'which-func-mode' (function), 'window-system-version', 'winner-mode-leave-hook', 'x-cut-buffer-or-selection-value'.</code></pre><pre><code>Some functions and variables obsolete since Emacs 23 have been removed: 'find-emacs-lisp-shadows', 'newsticker-cache-filename', 'process-filter-multibyte-p', 'redisplay-end-trigger-functions', 'set-process-filter-multibyte', 'set-window-redisplay-end-trigger', 'unify-8859-on-decoding-mode', 'unify-8859-on-encoding-mode', 'vc-arch-command', 'window-redisplay-end-trigger', 'x-selection'.</code></pre><pre><code>Some functions and variables obsolete since Emacs 21 or 22 have been removed: 'c-toggle-auto-state', 'find-file-not-found-hooks', 'ls-lisp-dired-ignore-case', 'query-replace-regexp-eval'.</code></pre><pre><code>New generic function 'function-documentation'. It can dynamically generate a raw docstring depending on the type of a function. Used mainly for docstrings of OClosures.</code></pre><pre><code>Base64 encoding no longer tolerates latin-1 input. The functions 'base64-encode-string', 'base64url-encode-string', 'base64-encode-region' and 'base64url-encode-region' no longer accept characters in the range U+0080..U+00FF as substitutes for single bytes in the range 128..255, but signal an error for all multibyte characters. The input must be unibyte encoded text.</code></pre><pre><code>The 'clone-indirect-buffer-hook' is now run by 'make-indirect-buffer'. It was previously only run by 'clone-indirect-buffer' and 'clone-indirect-buffer-other-window'. Since 'make-indirect-buffer' is called by both of these, the hook is now run by all 3 of these functions.</code></pre><pre><code>'?\' at the end of a line now signals an error. Previously, it produced a nonsense value, -1, that was never intended.</code></pre><pre><code>Some libraries obsolete since Emacs 24.1 and 24.3 have been removed: abbrevlist.el, assoc.el, complete.el, cust-print.el, erc-hecomplete.el, mailpost.el, mouse-sel.el, old-emacs-lock.el, patcomp.el, pc-mode.el, pc-select.el, s-region.el, and sregex.el.</code></pre><pre><code>Many seldom-used generalized variables have been made obsolete. Emacs has a number of rather obscure generalized variables defined, that, for instance, allowed you to say things like: (setf (point-min) 4) These never caught on and have been made obsolete. The form above, for instance, is the same as saying (narrow-to-region 4 (point-max)) The following generalized variables have been made obsolete: 'buffer-file-name', 'buffer-local-value', 'buffer-modified-p', 'buffer-name', 'buffer-string', 'buffer-substring', 'current-buffer', 'current-column', 'current-global-map', 'current-input-mode', 'current-local-map', 'current-window-configuration', 'default-file-modes', 'documentation-property', 'eq', 'frame-height', 'frame-width', 'frame-visible-p', 'global-key-binding', 'local-key-binding', 'mark', 'mark-marker', 'marker-position', 'mouse-position', 'point', 'point-marker', 'point-max', 'point-min', 'read-mouse-position', 'screen-height', 'screen-width', 'selected-frame', 'selected-screen', 'selected-window', 'standard-case-table', 'syntax-table', 'visited-file-modtime', 'window-height', 'window-width', and 'x-get-secondary-selection'.</code></pre><pre><code>The 'dotimes' loop variable can no longer be manipulated in the loop body. Previously, the 'dotimes' loop counter could be modified inside the loop body, but only in code using dynamic binding. Now the behavior is the same as when using lexical binding: changes to the loop variable have no effect on subsequent iterations. That is, (dotimes (i 10) (print i) (setq i (+ i 6))) now always prints the numbers 0 .. 9. </code></pre><H3 id="lisp-changes-in-emacs-29.1">Lisp Changes in Emacs 29.1</H3><pre><code>Interpreted closures are "safe for space". As was already the case for byte-compiled closures, instead of capturing the whole current lexical environment, interpreted closures now only capture the part of the environment that they need. The previous behavior could occasionally lead to memory leaks or to problems where a printed closure would not be 'read'able because of an un'read'able value in an unrelated lexical variable.</code></pre><pre><code>New accessor function 'file-attribute-file-identifier'. It returns the list of the inode number and device identifier retrieved by 'file-attributes'. This value can be used to identify a file uniquely. The device identifier can be a single number or (for remote files) a cons of 2 numbers.</code></pre><pre><code>New macro 'while-let'. This is like 'when-let', but repeats until a binding form is nil.</code></pre><p class=" text-justify">Useful addition. I’m liking all the <code>xxx-let</code> bindings they’ve been adding over the years. It cuts down on chaff and it retains readability.</p><pre><code>New function 'make-obsolete-generalized-variable'. This can be used to mark setters used by 'setf' as obsolete, and the byte-compiler will then warn about using them.</code></pre><pre><code>New functions 'pos-eol' and 'pos-bol'. These are like 'line-end-position' and 'line-beginning-position' (respectively), but ignore fields (and are more efficient).</code></pre><pre><code>New function 'compiled-function-p'. This returns non-nil if its argument is either a built-in, or a byte-compiled, or a natively-compiled function object, or a function loaded from a dynamic module.</code></pre><pre><code>'deactivate-mark' can have new value 'dont-save'. This value means that Emacs should deactivate the mark as usual, but without setting the primary selection, if 'select-active-regions' is enabled.</code></pre><pre><code>New 'declare' form 'interactive-args'. This can be used to specify what forms to put into 'command-history' when executing commands interactively.</code></pre><pre><code>The FORM argument of 'time-convert' is mandatory. 'time-convert' can still be called without it, as before, but the compiler now emits a warning about this deprecated usage.</code></pre><pre><code>Emacs now supports user-customizable and themable icons. These can be used for buttons in buffers and the like. See the "(elisp) Icons" and "(emacs) Icons" nodes in the manuals for details.</code></pre><pre><code>New arguments MESSAGE and TIMEOUT of 'set-transient-map'. MESSAGE specifies a message to display after activating the transient map, including a special formatting spec to list available keys. TIMEOUT is the idle time after which to deactivate the transient map. The default timeout value can be defined by the new variable 'set-transient-map-timeout'.</code></pre><pre><code>New forms 'with-restriction' and 'without-restriction'. These forms can be used as enhanced alternatives to the 'save-restriction' form combined with, respectively, 'narrow-to-region' and 'widen'. They also accept an optional label argument, with which labeled narrowings can be created and lifted. See the "(elisp) Narrowing" node for details.</code></pre><H3 id="connection-local-variables">Connection Local Variables</H3><pre><code>Some connection-local variables are now user options. The variables 'connection-local-profile-alist' and 'connection-local-criteria-alist' are now user options, in order to make it more convenient to inspect and modify them.</code></pre><pre><code>New function 'connection-local-update-profile-variables'. This function allows to modify the settings of an existing connection-local profile.</code></pre><pre><code>New macro 'with-connection-local-application-variables'. This macro works like 'with-connection-local-variables', but it allows using another application instead of 'tramp'. This is useful when running code in a buffer where Tramp has already set some connection-local variables.</code></pre><pre><code>New macro 'setq-connection-local'. This allows dynamically setting variable values for a particular connection within the body of 'with-connection-local-{application-}variables'. See the "(elisp) Connection Local Variables" node in the Lisp Reference manual for more information.</code></pre><pre><code>'plist-get', 'plist-put' and 'plist-member' are no longer limited to 'eq'. These function now take an optional comparison PREDICATE argument.</code></pre><pre><code>'read-multiple-choice' can now use long-form answers.</code></pre><pre><code>'M-s c' in 'read-regexp' now toggles case folding.</code></pre><pre><code>'completing-read' now allows a function as its REQUIRE-MATCH argument. This function is called to see whether what the user has typed is a match. This is also available from functions that call 'completing-read', like 'read-file-name'.</code></pre><pre><code>'posn-col-row' can now give position data based on windows. Previously, it reported data only based on the frame.</code></pre><pre><code>'file-expand-wildcards' can now also take a regexp as PATTERN argument.</code></pre><pre><code>vc-mtn (the VC backend for Monotone) has been made obsolete.</code></pre><pre><code>'gui-set-selection' can specify different values for different data types. If DATA is a string, then its text properties are searched for values for each specific data type while the selection is being converted.</code></pre><pre><code>New eldoc function 'elisp-eldoc-var-docstring-with-value'. This function includes the current value of the variable in eldoc display and can be used as a more detailed alternative to 'elisp-eldoc-var-docstring'.</code></pre><pre><code>'save-some-buffers' can now be extended to save other things. Traditionally, 'save-some-buffers' saved buffers, and also saved abbrevs. This has been generalized via the 'save-some-buffers-functions' variable, and packages can now register things to be saved.</code></pre><pre><code>New function 'string-equal-ignore-case'. This compares strings ignoring case differences.</code></pre><pre><code>'symbol-file' can now report natively-compiled ".eln" files. If Emacs was built with native-compilation enabled, Lisp programs can now call 'symbol-file' with the new optional 3rd argument non-nil to request the name of the ".eln" file which defined a given symbol.</code></pre><pre><code>New macro 'with-memoization' provides a very primitive form of memoization.</code></pre><pre><code>'max-char' can now report the maximum codepoint according to Unicode. When called with a new optional argument UNICODE non-nil, 'max-char' will now report the maximum valid codepoint defined by the Unicode Standard.</code></pre><H3 id="seq">Seq</H3><p class=" text-justify">Seq is a library that adds a range of functions that operate on all manner of sequences. It’s filled a gaping void that the likes of <code>dash.el</code> also tries to fill.</p><pre><code>New function 'seq-split'. This returns a list of sub-sequences of the specified sequence.</code></pre><pre><code>New function 'seq-remove-at-position'. This function returns a copy of the specified sequence with the element at a given (zero-based) index removed.</code></pre><pre><code>New function 'seq-positions'. This returns a list of the (zero-based) indices of elements matching a given predicate in the specified sequence.</code></pre><pre><code>New function 'seq-keep'. This is like 'seq-map', but removes all nil results from the returned list.</code></pre><H3 id="themes">Themes</H3><p class=" text-justify">Color themes, that is.</p><pre><code>New hooks 'enable-theme-functions' and 'disable-theme-functions'. These are run after enabling and disabling a theme, respectively.</code></pre><pre><code>Themes can now be made obsolete. Using 'make-obsolete' on a theme is now supported. This will make 'load-theme' issue a warning when loading the theme.</code></pre><pre><code>New hook 'display-monitors-changed-functions'. It is called whenever the configuration of different monitors on a display changes.</code></pre><pre><code>'prin1' and 'prin1-to-string' now take an optional OVERRIDES argument. This argument can be used to override values of print-related settings.</code></pre><pre><code>New minor mode 'header-line-indent-mode'. This is meant to be used by Lisp programs that show a header line which should be kept aligned with the buffer contents when the user switches 'display-line-numbers-mode' on or off, and when the width of line-number display changes. See the "(elisp) Header Lines" node in the Emacs Lisp Reference manual for more information.</code></pre><pre><code>New global minor mode 'lost-selection-mode'. This global minor mode makes Emacs deactivate the mark in all buffers when the primary selection is obtained by another program.</code></pre><pre><code>On X, Emacs will try to preserve selection ownership when a frame is deleted. This means that if you make Emacs the owner of a selection, such as by selecting some text into the clipboard or primary selection, and then delete the current frame, you will still be able to insert the contents of that selection into other programs as long as another frame is open on the same display. This behavior can be disabled by setting the user option 'x-auto-preserve-selections' to nil.</code></pre><pre><code>New predicate 'char-uppercase-p'. This returns non-nil if its argument its an uppercase character.</code></pre><H3 id="byte-compilation">Byte Compilation</H3><pre><code>Byte compilation will now warn about some quoting mistakes in docstrings. When writing code snippets that contains the "'" character (APOSTROPHE), that quote character has to be escaped to avoid Emacs displaying it as "’" (LEFT SINGLE QUOTATION MARK), which would make code examples like (setq foo '(1 2 3)) invalid. Emacs will now warn during byte compilation if it sees something like that, and also warn about when using RIGHT/LEFT SINGLE QUOTATION MARK directly. In both these cases, if these characters should really be present in the docstring, they should be quoted with "\=".</code></pre><pre><code>Byte compilation will now warn about some malformed 'defcustom' types. It is very common to write 'defcustom' types on the form: :type '(choice (const :tag "foo" 'bar)) I.e., double-quoting the 'bar', which is almost never the correct value. The byte compiler will now issue a warning if it encounters these forms.</code></pre><pre><code>'restore-buffer-modified-p' can now alter buffer auto-save state. With a FLAG value of 'autosaved', it will mark the buffer as having been auto-saved since the time of last modification.</code></pre><pre><code>New minor mode 'isearch-fold-quotes-mode'. This sets up 'search-default-mode' so that quote characters are char-folded into each other. It is used, by default, in "*Help*" and "*info*" buffers.</code></pre><p class=" text-justify">We talked about this before.</p><pre><code>New macro 'buffer-local-set-state'. This is a helper macro to be used by minor modes that wish to restore buffer-local variables back to their original states when the mode is switched off.</code></pre><p class=" text-justify">This should help minor mode authors that temporarily alter values restore them to their correct, former values without keeping a copy of the values around.</p><pre><code>New macro 'with-buffer-unmodified-if-unchanged'. If the buffer is marked as unmodified, and code does modifications that, in total, means that the buffer is identical to the buffer before, mark the buffer as unmodified again.</code></pre><pre><code>New function 'malloc-trim'. This function allows returning unused memory back to the operating system, and is mainly meant as a debugging tool. It is currently available only when Emacs was built with glibc as the C library.</code></pre><pre><code>'x-show-tip' no longer hard-codes a timeout default. The new variable 'x-show-tooltip-timeout' allows the user to alter this for packages that don't use 'tooltip-show', but instead call the lower level function directly.</code></pre><pre><code>New function 'current-cpu-time'. It gives access to the CPU time used by the Emacs process, for example for benchmarking purposes.</code></pre><pre><code>New function 'string-edit'. This is meant to be used when the user has to edit a (potentially) long string. It pops up a new buffer where you can edit the string, and the provided callback is called when the user types 'C-c C-c'.</code></pre><p class=" text-justify">Great addition. There’s a wide range of cases in Emacs where that would come in handy.</p><pre><code>New function 'read-string-from-buffer'. This is a modal version of 'string-edit', and can be used as an alternative to 'read-string'.</code></pre><pre><code>The return value of 'clear-message-function' is not ignored anymore. If the function returns 'dont-clear-message', then the message is not cleared, with the assumption that the function cleared it itself.</code></pre><pre><code>The local variables section now supports defining fallback modes. This was previously only available when using a property line (i.e., putting the modes on the first line of a file).</code></pre><pre><code>New function 'flush-standard-output'. This enables display of lines that don't end in a newline from batch-based Emacs scripts.</code></pre><pre><code>New convenience function 'buttonize-region'. This works like 'buttonize', but for a region instead of a string.</code></pre><pre><code>'macroexp-let2*' can omit TEST argument and use single-var bindings.</code></pre><pre><code>New macro-writing macros, 'cl-with-gensyms' and 'cl-once-only'. See the "(cl) Macro-Writing Macros" manual section for descriptions.</code></pre><pre><code>New variable 'last-event-device' and new function 'device-class'. On X Windows, 'last-event-device' specifies the input extension device from which the last input event originated, and 'device-class' can be used to determine the type of an input device.</code></pre><pre><code>Variable 'track-mouse' can have a new value 'drag-source'. This means the same as 'dropping', but modifies the mouse position list in reported motion events if there is no frame underneath the mouse pointer.</code></pre><pre><code>New functions for dragging items from Emacs to other programs. The new functions 'x-begin-drag', 'dnd-begin-file-drag', 'dnd-begin-drag-files', and 'dnd-direct-save' allow dragging contents (such as files and text) from Emacs to other programs.</code></pre><pre><code>New function 'ietf-drums-parse-date-string'. This function parses RFC5322 (and RFC822) date strings, and should be used instead of 'parse-time-string' when parsing data that's standards compliant.</code></pre><pre><code>New macro 'setopt'. This is like 'setq', but is meant to be used for user options instead of plain variables, and uses 'custom-set'/'set-default' to set them.</code></pre><p class=" text-justify">Intended as a drop-in replacement for customizable options, as they can have edge triggers that run when the value changes. That is something <code>setq</code> cannot do, which can in rare cases lead to bugs.</p><pre><code>New utility predicate 'mode-line-window-selected-p'. This is meant to be used from ':eval' mode line constructs to create different mode line looks for selected and unselected windows.</code></pre><pre><code>New variable 'messages-buffer-name'. This variable (defaulting to "*Messages*") allows packages to override where messages are logged.</code></pre><pre><code>New function 'readablep'. This function says whether an object can be written out and then read back by the Emacs Lisp reader.</code></pre><pre><code>New variable 'print-unreadable-function'. This variable allows changing how Emacs prints unreadable objects.</code></pre><pre><code>The user option 'polling-period' now accepts floating point values. This means Emacs can now poll for input during Lisp execution more frequently than once in a second.</code></pre><pre><code>New function 'bidi-string-strip-control-characters'. This utility function is meant for displaying strings when it is essential that there's no bidirectional context. It removes all the bidirectional formatting control characters (such as RLM, LRO, PDF, etc.) from its argument string. The characters it removes are listed in the value of 'bidi-control-characters'.</code></pre><pre><code>The Gnus range functions have been moved to a new library, range.el. All the old names have been made obsolete.</code></pre><pre><code>New function 'function-alias-p'. This predicate says whether an object is a function alias, and if it is, the alias chain is returned.</code></pre><pre><code>New variable 'lisp-directory' holds the directory of Emacs's own Lisp files.</code></pre><pre><code>New facility for handling session state: 'multisession-value'. This can be used as a convenient way to store (simple) application state, and the command 'list-multisession-values' allows users to list (and edit) this data.</code></pre><pre><code>New function 'get-display-property'. This is like 'get-text-property', but works on the 'display' text property.</code></pre><pre><code>New function 'add-display-text-property'. This is like 'put-text-property', but works on the 'display' text property.</code></pre><pre><code>New 'min-width' 'display' property. This allows setting a minimum display width for a region of text.</code></pre><pre><code>New 'cursor-face' text property. This uses 'cursor-face' instead of the default face when cursor is on or near the character and 'cursor-face-highlight-mode' is enabled. The user option 'cursor-face-highlight-nonselected-window' is similar to 'highlight-nonselected-windows', but for this property.</code></pre><pre><code>New event type 'touch-end'. This event is sent whenever the user's finger moves off the mouse wheel on some mice, or when the user's finger moves off the touchpad.</code></pre><pre><code>New event type 'pinch'. This event is sent when a user performs a pinch gesture on a touchpad, which is comprised of placing two fingers on the touchpad and moving them towards or away from each other.</code></pre><pre><code>New hook 'x-pre-popup-menu-hook'. This hook is run before 'x-popup-menu' is about to display a deck-of-cards menu on screen.</code></pre><pre><code>New hook 'post-select-region-hook'. This hook is run immediately after 'select-active-regions'. It causes the region to be set as the primary selection.</code></pre><pre><code>New function 'buffer-match-p'. Check if a buffer satisfies some condition. Some examples for conditions can be regular expressions that match a buffer name, a cons-cell like '(major-mode . shell-mode)' that matches any buffer where 'major-mode' is 'shell-mode' or a combination with a condition like '(and "\\`\\*.+\\*\\'" (major-mode . special-mode))'.</code></pre><pre><code>New function 'match-buffers'. It uses 'buffer-match-p' to gather a list of buffers that match a condition.</code></pre><pre><code>New optional arguments TEXT-FACE and DEFAULT-FACE for 'tooltip-show'. They allow changing the faces used for the tooltip text and frame colors of the resulting tooltip frame from the default 'tooltip' face.</code></pre><H3 id="text-security-and-suspiciousness">Text Security and Suspiciousness</H3><pre><code>New library textsec.el. This library contains a number of checks for whether a string is "suspicious". This usually means that the string contains characters that have glyphs that can be confused with other, more commonly used glyphs, or contains bidirectional (or other) formatting characters that may be used to confuse a user.</code></pre><p class=" text-justify">This is back to what I was saying earlier about <em>confusables</em>.</p><pre><code>New user option 'textsec-check'. If non-nil (which is the default), Emacs packages that are vulnerable to attackers trying to confuse the users will use the textsec library to mark suspicious text. For instance shr/eww will mark suspicious URLs and links, Gnus will mark suspicious From addresses, and Message mode will query the user if the user is sending mail to a suspicious address. If this variable is nil, these checks are disabled.</code></pre><pre><code>New function 'textsec-suspicious-p'. This is the main function Emacs applications should be using to check whether a string is suspicious. It heeds the 'textsec-check' user option.</code></pre><H3 id="keymaps-and-key-definitions">Keymaps and Key Definitions</H3><pre><code>'where-is-internal' can now filter events marked as non key events. If a command maps to a key binding like '[some-event]', and 'some-event' has a symbol plist containing a non-nil 'non-key-event' property, then that binding is ignored by 'where-is-internal'.</code></pre><pre><code>New functions for defining and manipulating keystrokes. These all take the syntax defined by 'key-valid-p', which is basically the same syntax as the one accepted by the 'kbd' macro. None of the older functions have been deprecated or altered, but they are now de-emphasized in the documentation, and we encourage Lisp programs to switch to these new functions. Use 'keymap-set' instead of 'define-key'. Use 'keymap-global-set' instead of 'global-set-key'. Use 'keymap-local-set' instead of 'local-set-key'. Use 'keymap-global-unset' instead of 'global-unset-key'. Use 'keymap-local-unset' instead of 'local-unset-key'. Use 'keymap-substitute' instead of 'substitute-key-definition'. Use 'keymap-set-after' instead of 'define-key-after'. Use 'keymap-lookup' instead of 'lookup-key' and 'key-binding'. Use 'keymap-local-lookup' instead of 'local-key-binding'. Use 'keymap-global-lookup' instead of 'global-key-binding'.</code></pre><p class=" text-justify">I’m glad they’re thinking about unifying the rather inconsistent key binding system in Emacs, but we’re decades away from ever obsoleting the existing ones. They’re just too ingrained (and they work fine.)</p><pre><code>'define-key' now takes an optional REMOVE argument. If non-nil, remove the definition from the keymap. This is subtly different from setting a definition to nil: when the keymap has a parent such a definition will shadow the parent's definition.</code></pre><p class=" text-justify">Weirdly, removing (not marking <code>nil</code> or <code>ignore</code>) was quite hard.</p><pre><code>'read-multiple-choice' now takes an optional SHOW-HELP argument. If non-nil, show the help buffer immediately, before any user input.</code></pre><pre><code>New function 'key-valid-p'. The 'kbd' function is quite permissive, and will try to return something usable even if the syntax of the argument isn't completely correct. The 'key-valid-p' predicate does a stricter check of the syntax.</code></pre><pre><code>New function 'key-parse'. This is like 'kbd', but only returns vectors instead of a mix of vectors and strings.</code></pre><pre><code>New ':type' for 'defcustom' for keys. The new 'key' type can be used for options that should be a valid key according to 'key-valid-p'. The type 'key-sequence' is now obsolete.</code></pre><pre><code>New function 'define-keymap'. This function allows defining a number of keystrokes with one form.</code></pre><pre><code>New macro 'defvar-keymap'. This macro allows defining keymap variables more conveniently.</code></pre><pre><code>'defvar-keymap' can specify 'repeat-mode' behavior for the keymap. Use ':repeat t' to have all bindings be repeatable or for more advanced usage: :repeat (:enter (commands ...) :exit (commands ...))</code></pre><pre><code>'kbd' can now be used in built-in, preloaded libraries. It no longer depends on edmacro.el and cl-lib.el.</code></pre><pre><code>New substitution in docstrings and 'substitute-command-keys'. Use \\`KEYSEQ' to insert a literal key sequence "KEYSEQ" (for example \\`C-k') in a docstring or when calling 'substitute-command-keys', which will use the same face as a command substitution. This should be used only when a key sequence has no corresponding command, for example when it is read directly with 'read-key-sequence'. It must be a valid key sequence according to 'key-valid-p'.</code></pre><pre><code>'lookup-key' is more permissive when searching for extended menu items. In Emacs 28.1, the behavior of 'lookup-key' was changed: when looking for a menu item '[menu-bar Foo-Bar]', first try to find an exact match, then look for the lowercased '[menu-bar foo-bar]'. This has been extended, so that when looking for a menu item with a symbol containing spaces, as in '[menu-bar Foo\ Bar]', first look for an exact match, then the lowercased '[menu-bar foo\ bar]' and finally '[menu-bar foo-bar]'. This further improves backwards-compatibility when converting menus to use 'easy-menu-define'.</code></pre><pre><code>New function 'file-name-split'. This returns a list of all the components of a file name.</code></pre><pre><code>New function 'file-name-parent-directory'. This returns the parent directory of a file name.</code></pre><pre><code>New macro 'with-undo-amalgamate'. It records a particular sequence of operations as a single undo step.</code></pre><pre><code>New command 'yank-media'. This command supports yanking non-plain-text media like images and HTML from other applications into Emacs. It is only supported in modes that have registered support for it, and only on capable platforms.</code></pre><pre><code>New command 'yank-media-types'. This command lets you examine all data in the current selection and the clipboard, and insert it into the buffer.</code></pre><pre><code>New variable 'yank-transform-functions'. This variable allows the user to alter the string to be inserted.</code></pre><pre><code>New command 'yank-in-context'. This command tries to preserve string/comment syntax when yanking.</code></pre><pre><code>New function 'minibuffer-lazy-highlight-setup'. This function allows setting up the minibuffer so that lazy highlighting of its content is applied in the original window.</code></pre><pre><code>New text property 'inhibit-isearch'. If set, 'isearch' will skip these areas, which can be useful (for instance) when covering huge amounts of data (that has no meaningful searchable data, like image data) with a 'display' text property.</code></pre><pre><code>'insert-image' now takes an INHIBIT-ISEARCH optional argument. It marks the image with the 'inhibit-isearch' text property, which inhibits 'isearch' matching the STRING argument.</code></pre><pre><code>New variable 'replace-regexp-function'. Function to call to convert the entered FROM string to an Emacs regexp in 'query-replace' and similar commands. It can be used to implement a different regexp syntax for search/replace.</code></pre><pre><code>'E' in 'query-replace' now edits the replacement with exact case. Previously, this command did the same as 'e'.</code></pre><pre><code>New variables to customize defaults of FROM for 'query-replace*' commands. The new variable 'query-replace-read-from-default' can be set to a function that returns the default value of FROM when 'query-replace' prompts for a string to be replaced. An example of such a function is 'find-tag-default'. The new variable 'query-replace-read-from-regexp-default' can be set to a function (such as 'find-tag-default-as-regexp') that returns the default value of FROM when 'query-replace-regexp' prompts for a regexp whose matches are to be replaced. If these variables are nil (which is the default), 'query-replace' and 'query-replace-regexp' take the default value from the previous FROM-TO pair.</code></pre><H3 id="lisp-pretty-printer-pp">Lisp pretty-printer (‘pp’)</H3><pre><code>New function 'pp-emacs-lisp-code'. 'pp' formats general Lisp sexps. This function does much the same, but applies formatting rules appropriate for Emacs Lisp code. Note that this could currently be quite slow, and is thus appropriate only for relatively small code fragments.</code></pre><pre><code>New user option 'pp-use-max-width'. If non-nil, 'pp' and all 'pp-*' commands that format the results, will attempt to limit the line length when formatting long lists and vectors. This uses 'pp-emacs-lisp-code', and thus could be slow for large lists.</code></pre><pre><code>New function 'file-has-changed-p'. This convenience function is useful when writing code that parses files at run-time, and allows Lisp programs to re-parse files only when they have changed.</code></pre><pre><code>'abbreviate-file-name' now respects magic file name handlers.</code></pre><pre><code>New function 'font-has-char-p'. This can be used to check whether a specific font has a glyph for a character.</code></pre><pre><code>'window-text-pixel-size' now accepts a new argument IGNORE-LINE-AT-END. This controls whether or not the last screen line of the text being measured will be counted for the purpose of calculating the text dimensions.</code></pre><pre><code>'window-text-pixel-size' understands a new meaning of FROM. Specifying a cons as the FROM argument allows to start measuring text from a specified amount of pixels above or below a position.</code></pre><pre><code>'window-body-width' and 'window-body-height' can use remapped faces. Specifying 'remap' as the PIXELWISE argument now checks if the default face was remapped, and if so, uses the remapped face to determine the character width/height.</code></pre><pre><code>'set-window-vscroll' now accepts a new argument PRESERVE-VSCROLL-P. This means the vscroll will not be reset when set on a window that is "frozen" due to a mini-window being resized.</code></pre><H3 id="xdg-support">XDG Support</H3><pre><code>New function 'xdg-state-home'. It returns the new 'XDG_STATE_HOME' environment variable. It should point to a file name that "contains state data that should persist between (application) restarts, but that is not important or portable enough to the user that it should be stored in $XDG_DATA_HOME". (This variable was introduced in the XDG Base Directory Specification version 0.8 released on May 8, 2021.)</code></pre><p class=" text-justify">Let’s hope this is the beginning of well-behaved elisp code (incl. core Emacs!) that’ll put all their junk files there instead of in my <code>.emacs.d</code> directory.</p><pre><code>New function 'xdg-current-desktop'. It returns a list of strings, corresponding to the colon-separated list of names in the 'XDG_CURRENT_DESKTOP' environment variable, which identify the current desktop environment. (This variable was introduced in XDG Desktop Entry Specification version 1.2.)</code></pre><pre><code>New function 'xdg-session-type'. It returns the 'XDG_SESSION_TYPE' environment variable. (This is not part of any official standard; see the man page pam_systemd(8) for more information.)</code></pre><pre><code>New macro 'with-delayed-message'. This macro is like 'progn', but will output the specified message if the body takes longer to execute than the specified timeout.</code></pre><pre><code>New function 'funcall-with-delayed-message'. This function is like 'funcall', but will output the specified message if the function takes longer to execute than the specified timeout.</code></pre><H3 id="locale">Locale</H3><pre><code>New variable 'current-locale-environment'. This holds the value of the previous call to 'set-locale-environment'.</code></pre><pre><code>New macro 'with-locale-environment'. This macro can be used to change the locale temporarily while executing code.</code></pre><H3 id="table">Table</H3><p class=" text-justify">Table creates editable ASCII tables. I find it incredibly buggy.</p><pre><code>New user option 'table-latex-environment'. This allows switching between "table" and "tabular".</code></pre><H3 id="tabulated-list-mode">Tabulated List Mode</H3><pre><code>A column can now be set to an image descriptor. The 'tabulated-list-entries' variable now supports using an image descriptor, which means to insert an image in that column instead of text. See the documentation string of that variable for details.</code></pre><pre><code>':keys' in 'menu-item' can now be a function. If so, it is called whenever the menu is computed, and can be used to calculate the keys dynamically.</code></pre><pre><code>New major mode 'clean-mode'. This is a new major mode meant for debugging. It kills absolutely all local variables and removes overlays and text properties.</code></pre><pre><code>'kill-all-local-variables' can now kill all local variables. If given the new optional KILL-PERMANENT argument, it also kills permanent local variables.</code></pre><pre><code>Third 'mapconcat' argument SEPARATOR is now optional. An explicit nil always meant the empty string, now it can be left out.</code></pre><pre><code>New function 'image-at-point-p'. This function returns t if point is on a valid image, and nil otherwise.</code></pre><pre><code>New function 'buffer-text-pixel-size'. This is similar to 'window-text-pixel-size', but can be used when the buffer isn't displayed.</code></pre><pre><code>New function 'string-pixel-width'. This returns the width of a string in pixels. This can be useful when dealing with variable pitch fonts and glyphs that have widths that aren't integer multiples of the default font.</code></pre><pre><code>New function 'string-glyph-split'. This function splits a string into a list of strings representing separate glyphs. This takes into account combining characters and grapheme clusters, by treating each sequence of characters composed on display as a single unit.</code></pre><H3 id="xwidget-1">Xwidget</H3><pre><code>The function 'make-xwidget' now accepts an optional RELATED argument. This argument is used as another widget for the newly created WebKit widget to share settings and subprocesses with. It must be another WebKit widget.</code></pre><pre><code>New function 'xwidget-perform-lispy-event'. This function allows you to send events to xwidgets. Usually, some equivalent of the event will be sent, but there is no guarantee of what the widget will actually receive. On GTK+, only key and function key events are implemented.</code></pre><pre><code>New function 'xwidget-webkit-load-html'. This function is used to load HTML text into WebKit xwidgets directly, in contrast to creating a temporary file to hold the markup, and passing the URI of the file as an argument to 'xwidget-webkit-goto-uri'.</code></pre><pre><code>New functions for performing searches on WebKit xwidgets. Some new functions, such as 'xwidget-webkit-search', have been added for performing searches on WebKit xwidgets.</code></pre><pre><code>New function 'xwidget-webkit-back-forward-list'. This function returns the history of page-loads in a WebKit xwidget.</code></pre><pre><code>New function 'xwidget-webkit-estimated-load-progress'. This function returns the estimated progress of page loading in a WebKit xwidget.</code></pre><pre><code>New function 'xwidget-webkit-stop-loading'. This function terminates all data transfer during page loads in a WebKit xwidget.</code></pre><pre><code>'load-changed' xwidget events are now more detailed. In particular, they can now have different arguments based on the state of the WebKit widget. 'load-finished' is sent when a load has completed, 'load-started' when a load first starts, 'load-redirected' after a redirect, and 'load-committed' when the WebKit widget first commits to the load.</code></pre><pre><code>New event type 'xwidget-display-event'. These events are sent whenever an xwidget requests that Emacs displays another xwidget. The only arguments to this event are the xwidget that should be displayed, and the xwidget that asked to display it.</code></pre><pre><code>New function 'xwidget-webkit-set-cookie-storage-file'. This function is used to control where and if an xwidget stores cookies set by web pages on disk.</code></pre><pre><code>New variable 'help-buffer-under-preparation'. This variable is bound to t during the preparation of a "*Help*" buffer.</code></pre><pre><code>Timestamps like '(1 . 1000)' now work without warnings being generated. For example, '(time-add nil '(1 . 1000))' no longer warns that the '(1 . 1000)' acts like '(1000 . 1000000)'. This warning, which was a temporary transition aid for Emacs 27, has served its purpose.</code></pre><pre><code>'encode-time' now also accepts a 6-element list with just time and date. '(encode-time (list SECOND MINUTE HOUR DAY MONTH YEAR))' is now short for '(encode-time (list SECOND MINUTE HOUR DAY MONTH YEAR nil -1 nil))'.</code></pre><pre><code>'date-to-time' now accepts arguments that lack month, day, or time. The function now assumes the earliest possible values if its argument lacks month, day, or time. For example, (date-to-time "2021-12-04") now assumes a time of "00:00" instead of signaling an error.</code></pre><pre><code>'format-seconds' now allows suppressing zero-value trailing elements. The new "%x" non-printing control character will suppress zero-value elements that appear after "%x".</code></pre><pre><code>New events for taking advantage of touchscreen devices. The events 'touchscreen-begin', 'touchscreen-update', and 'touchscreen-end' have been added to take better advantage of touch-capable display panels.</code></pre><pre><code>New error symbol 'permission-denied'. This is a subcategory of 'file-error', and is signaled when some file operation fails because the OS doesn't allow Emacs to access a file or a directory.</code></pre><pre><code>Warning about "eager macro-expansion failure" is now an error.</code></pre><pre><code>Previously, the X "reverseVideo" value at startup was heeded for all frames. This meant that if you had a "reverseVideo" resource on the initial display, and then opened up a new frame on a display without any explicit "reverseVideo" setting, it would get heeded there, too. (This included terminal frames.) In Emacs 29, the "reverseVideo" X resource is handled like all the other X resources, and set on a per-frame basis.</code></pre><pre><code>The ':underline' face attribute now accepts a new property. The property ':position' now specifies the position of the underline when used as part of a property list specification for the ':underline' attribute.</code></pre><pre><code>'defalias' records a more precise history of definitions. This is recorded in the 'function-history' symbol property.</code></pre><pre><code>New hook 'save-place-after-find-file-hook'. This is called at the end of 'save-place-find-file-hook'.</code></pre><pre><code>'indian-tml-base-table' no longer translates digits. Use 'indian-tml-base-digits-table' if you want digits translation.</code></pre><pre><code>'indian-tml-itrans-v5-hash' no longer translates digits. Use 'indian-tml-itrans-digits-v5-hash' if you want digits translation.</code></pre><pre><code>'shell-quote-argument' has a new optional argument POSIX. This is useful when quoting shell arguments for a remote shell invocation. Such shells are POSIX conformant by default.</code></pre><pre><code>'make-process' can set connection type independently for input and output. When calling 'make-process', communication via pty can be enabled selectively for just input or output by passing a cons cell for ':connection-type', e.g. '(pipe . pty)'. When examining a process later, you can determine whether a particular stream for a process uses a pty by passing one of 'stdin', 'stdout', or 'stderr' as the second argument to 'process-tty-name'.</code></pre><pre><code>'signal-process' now consults the list 'signal-process-functions'. This is to determine which function has to be called in order to deliver the signal. This allows Tramp to send the signal to remote asynchronous processes. The hitherto existing implementation has been moved to 'internal-default-signal-process'.</code></pre><pre><code>Some system information functions honor remote systems now. 'list-system-processes' returns remote process IDs. 'memory-info' returns memory information of remote systems. 'process-attributes' expects a remote process ID. This happens only when the current buffer's 'default-directory' is remote. In order to preserve the old behavior, bind 'default-directory' to a local directory, like (let ((default-directory temporary-file-directory)) (list-system-processes))</code></pre><pre><code>New functions 'take' and 'ntake'. '(take N LIST)' returns the first N elements of LIST; 'ntake' does the same but works by modifying LIST destructively.</code></pre><pre><code>'string-split' is now an alias for 'split-string'.</code></pre><pre><code>'format-spec' now accepts functions in the replacement. The function is called only when used in the format string. This is useful to avoid side-effects such as prompting, when the value is not actually being used for anything.</code></pre><pre><code>The variable 'max-specpdl-size' has been made obsolete. Now 'max-lisp-eval-depth' alone is used for limiting Lisp recursion and stack usage. 'max-specpdl-size' is still present as a plain variable for compatibility but its limiting powers have been taken away.</code></pre><pre><code>New function 'external-completion-table'. This function returns a completion table designed to ease communication between Emacs's completion facilities and external tools offering completion services, particularly tools whose full working set is too big to transfer to Emacs every time a completion is needed. The table uses new 'external' completion style exclusively and cannot work with regular styles such as 'basic' or 'flex'.</code></pre><pre><code>Magic file name handlers for 'make-directory-internal' are no longer needed. Instead, Emacs uses the already-existing 'make-directory' handlers.</code></pre><pre><code>'(make-directory DIR t)' returns non-nil if DIR already exists. This can let a caller know whether it created DIR. Formerly, 'make-directory's return value was unspecified. </code></pre><H3 id="changes-in-emacs-29.1-on-non-free-operating-systems">Changes in Emacs 29.1 on Non-Free Operating Systems</H3><H3 id="ms-windows">MS-Windows</H3><pre><code>Emacs now supports double-buffering on MS-Windows to reduce display flicker. (This was supported on Free systems since Emacs 26.1.) To disable double-buffering (e.g., if it causes display problems), set the frame parameter 'inhibit-double-buffering' to a non-nil value. You can do that either by adding '(inhibit-double-buffering . t) to 'default-frame-alist', or by modifying the frame parameters of the selected frame by evaluating (modify-frame-parameters nil '((inhibit-double-buffering . t)))</code></pre><pre><code>Emacs now supports system dark mode. On Windows 10 (version 1809 and higher) and Windows 11, Emacs will now follow the system's dark mode: GUI frames use the appropriate light or dark title bar and scroll bars, based on the user's Windows-wide color settings.</code></pre><pre><code>Emacs now uses native image APIs to display some image formats. On Windows 2000 and later, Emacs now defaults to using the native image APIs for displaying the BMP, GIF, JPEG, PNG, and TIFF images. This means Emacs on MS-Windows needs no longer use external image support libraries to display those images. Other image types -- XPM, SVG, and WEBP -- still need support libraries for Emacs to be able to display them. The use of native image APIs is controlled by the variable 'w32-use-native-image-API', whose value now defaults to t on systems where those APIs are available.</code></pre><pre><code>Emacs now supports display of BMP images using native image APIs. When 'w32-use-native-image-API' is non-nil, Emacs on MS-Windows now has built-in support for displaying BMP images.</code></pre><pre><code>GUI Yes/No dialogs now include a "Cancel" button. The "Cancel" button is in addition to "Yes" and "No", and is intended to allow users to quit the dialog, as an equivalent of 'C-g' when Emacs asks a yes/no question via the echo area. This is controlled by the new variable 'w32-yes-no-dialog-show-cancel', by default t. Set it to nil to get back the old behavior of showing a modal dialog with only two buttons: "Yes" and "No".</code></pre><H3 id="cygwin">Cygwin</H3><pre><code>'process-attributes' is now implemented.</code></pre><H3 id="macos">macOS</H3><pre><code>The 'ns-popup-font-panel' command has been removed. Use the general command 'M-x menu-set-font' instead.</code></pre> /article/whats-new-in-emacs-29-1 Sun, 30 Jul 2023 17:32:37 +0000 Combobulate: Editing and Searching with the new Query Builder https://www.masteringemacs.org/article/combobulate-editing-searching-new-query-builder?utm_source=newsletter&utm_medium=rss Combobulate's gained the ability to build tree-sitter queries interactively, complete with highlighting and code completion. With the query builder you can now use it to pass queries to Combobulate's multiple cursors editing facility, or create ad hoc highlighters and Xref searches. <p class=" text-justify drop-cap">Today I’d like to talk about a slew of new features in <a href="/article/combobulate-structured-movement-editing-treesitter" class=" article-link">Combobulate</a>, my tree-sitter-powered package that adds structured movement and editing to a slew of programming languages. You’ll need Emacs 29, and you must <a href="/article/how-to-get-started-tree-sitter" class=" article-link">set up tree-sitter</a> also to use it.</p><p class=" text-justify">One of the main talking points of tree-sitter is the tree structure it generates from your code. Dealing with tree structures when you want to compare and contrast parts of the tree with a pattern is hard, particularly if you want to, say: check that a class has a method, and that method has a variable named “foo,” which, in turn, is assigned a value… and so on.</p><p class=" text-justify">I think you get the picture. So, to navigate these expansive trees by hand, it ships with a functional, though quirky, query language modeled on LISP s-expressions. The query language combines search and hierarchical pattern matching into a pragmatic little language. In Emacs 29, it’s used to font lock your code and control indentation. But it’s valuable to more than just major mode authors. It’s valuable to <em>you</em>, the overworked developer just trying to get through the day; you just haven’t realized that you need it in your life yet.</p><figure> <img src="/static/uploads/combobulate/combobulate-query-builder.gif" alt="" class=" center-block"> <figcaption>Combobulate's query builder. It has syntax highlighting, code completion, and a handful of utility functions to help you write queries.</figcaption> </figure><p class=" text-justify">Now, the thing is, writing queries and testing queries is not fun at all if you want to experiment. Particularly if you have never interfaced with concrete syntax trees, pattern matching, and language grammars before. There’s the double-whammy of learning the query language and its, ah… quirks, and of course, the practicalities of <em>doing</em> something with the queries once you’ve written them.</p><p class=" text-justify">So, one of the things I wanted to write in Combobulate is a user-friendly query builder. Much like Emacs’s venerable <a href="/article/re-builder-interactive-regexp-builder" class=" article-link">re-builder, an interactive regexp builder</a>, I built Combobulate’s query builder to work in much the same way. And just like regular expressions, there’s an art and a science to building them.</p><p class=" text-justify">Tree-sitter queries work like this: you write a query, which looks an awful lot like S-expressions – see the screenshot above for an example – and then you annotate portions of the query with <code>@name</code>, with <code>name</code> being anything you like. That tells tree-sitter to remember the nodes that you’re interested in, provided they match the tree structure you’ve specified. You can be as precise or imprecise as you like: if you want every pair in a dictionary, no problem; if you want just the keys of a dictionary assigned to a variable, then that’s no problem either. Provided your query matches something, it’ll collate all the captured nodes – you can have as many groups as you like – into a list of matches once it’s done scanning your tree.</p><p class=" text-justify">All you have to do now is yoink the matches and do something useful with them. And that’s where things get a little bit harder. Want to highlight stuff? Now you have to write highlighter code. I hope you know your way around font locking or overlay mechanics. Want multiple cursors placed at every node? Yep. More code. Ditto pretty much everything else.</p><p class=" text-justify">And that’s where the other new features in Combobulate enter the fray. I’ve added a handful of utilities that’ll do 80% of what you probably want to do with a query.</p><H2 id="query-building">Query Building</H2><figure> <img src="/static/uploads/combobulate/combobulate-query-builder-popup.svg" alt="" class=" center-block"> <figcaption>Selection of options available inside the query builder.</figcaption> </figure><p class=" text-justify">You can access Combobulate’s query builder either from the popup (<code>C-x o o</code> by default) or by typing <code>C-c o B q</code>. You’ll see a window appear below your current buffer. The query builder is tied to the window you invoked the command in, so keep that in mind.</p><p class=" text-justify">The query builder has its own popup (<code>C-c o</code> by default), and you can, of course, use <code>C-h b</code> also to see a list of valid key bindings.</p><p class=" text-justify">Now, if you haven’t already, I recommend you crack open <a href="https://tree-sitter.github.io/tree-sitter/using-parsers#query-syntax" class=" article-link">the Query syntax manual</a> on tree-sitter’s website. It’s a quick read.</p><p class=" text-justify">Combobulate will proffer completions at point, using your favorite completion framework. It’ll do a reasonable job of being context-aware, so it’ll correctly suggest field names and node names based on the tree hierarchy you’ve described. There’s font locking (syntax highlighting) and basic pretty printing also.</p><p class=" text-justify">You can pick the color to highlight with by naming the group after a font face, but the names tend to be long. To make life a bit easier, Combobulate ships with a large selection of colors that it’ll auto-suggest if you type <code>@</code>. The names are mapped to font faces when the query is executed. You can customize the list, of course, by modifying <code>combobulate-query-match-face-alist</code>. If you use a name Combobulate does not recognize, it’ll pick a random color for you. Some capture group names have contextual meaning elsewhere in Combobulate, but more on that below.</p><H3 id="cookie-cutter-node-queries">Cookie-Cutter Node Queries</H3><p class=" text-justify">Combobulate can pluck the hierarchy of nodes leading to point and insert it into the query builder. You can edit that hierarchy to suit your needs. Use it as a starting point if you’re new to query writing or just want to get on with it. <code>C-c n</code> will copy and match the text of the node at point; <code>C-c h</code> will ask you to pick a parent node from the node at point and then generate the hierarchy from the point node to the parent node you chose. An excellent way to quickly build up a hierarchy of nodes if you want to match function calls inside a method inside a class.</p><p class=" text-justify">The matcher predicate, <code>#match</code>, is a regexp only, unfortunately. That’s useful, yes, except when it’s not. Combobulate will try to escape regexp metacharacters for you, but I do admit it’s rather imperfect.</p><p class=" text-justify">Don’t forget, for browsing the node tree, there’s <code>M-x treesit-explore-mode</code>. Marking text in the buffer will update the explorer to show the marked nodes. I recommend you lean into that feature also.</p><H3 id="highlighting-queries">Highlighting Queries</H3><p class=" text-justify">Because query writing is usually <em>ad hoc</em>, or at the very least, something scoped to a file or a directory, you can install queries as highlighters using a file or directory local variable (<code>C-c f</code> and <code>C-c d</code>, respectively).</p><p class=" text-justify">Combobulate now has two highlighter variables: <code>combobulate-highlight-queries-alist</code>, which is safe for file and directory-local use, and <code>combobulate-highlight-queries-default</code>, an internal variable for language-specific highlighters. Combobulate does come with a couple of default highlighters, just to demonstrate what it can do.</p><p class=" text-justify">Here’s one that catches the rather insidious implicit tuple notation in Python. I’ve been programming in Python for decades, and this one’ll catch me out every now and again because I refactored something from somewhere else and forgot to remove the comma.</p><figure> <img src="/static/uploads/combobulate/combobulate-highlight-implicit-tuple.svg" alt="" class=" center-block"> <figcaption>Default highlight for Python in Combobulate. Here it spots implicit tuple notation due to the trailing comma.</figcaption> </figure><p class=" text-justify">It’s not Combobulate’s job to be a linter or style checker, but it does highlight the utility of on-the-fly querying.</p><p class=" text-justify">Oh, and if you just want to highlight something for a little while, then <code>C-c b</code> will install the query into the current buffer. I find the command helpful when I need to keep an eye on certain things, particularly during gnarly refactoring sessions. The highlighting is a bit more specific and utilitatiran than the usual <a href="/article/highlighting-by-word-line-regexp" class=" article-link">regexp highlighter</a> feature, but when you absolutely have to highlight only functions calls and nothing else, this is the highlighter for you.</p><p class=" text-justify">You can also save your query (<code>C-c C-s</code>) to the <em>query ring</em>, so you can recall the queries later, not just inside the query builder but in other parts of Combobulate, too. If you use <code>savehist-mode</code> (make sure it’s loaded <em>after</em> Combobulate though!), then Combobulate will automatically persist the query ring between sessions.</p><p class=" text-justify">And if this is all just too much, you can use the “DWIM” feature in <code>C-c o h h</code>. It’ll try to guess the thing at point you want to match. It’s far from perfect, but it might be all you need?</p><H3 id="editing-nodes">Editing Nodes</H3><p class=" text-justify">Combobulate’s had support for multiple cursors since day one. It’s a hallmark feature. Correctly selecting the elements you want to edit has always been a challenge with regexp and the bare-bones tooling available to us before the likes of tree-sitter.</p><p class=" text-justify">Combobulate’s existing editing facilities (see <code>C-o o t</code>) work well enough for localized editing: editing matching tags in JSX; editing all the elements in a list; or the arguments of a function. But if you want specialized editing that suits <em>your</em> needs, then you’d have to roll up your sleeves and write some code.</p><p class=" text-justify">Now, though, you can write your own query and annotate the nodes you want to edit with <code>@before</code>, to put the cursor at the start of a node; <code>@after</code>, for the end; or <code>@mark</code>, to mark the node with a region. Press <code>C-c e</code> (in the query builder) to edit them with multiple cursors. Simple as that.</p><p class=" text-justify">It also works <em>outside</em> the query builder provided you have saved your query to the query ring. Type <code>C-c o t q</code> to edit the active query in the ring, and <code>C-c o M-n/p</code> to cycle the active ring query.</p><p class=" text-justify">One caveat is limiting the scope of the search. You can do this by either marking the region you want to act on and then narrowing to that region with Emacs’s narrowing feature: <code>C-x n n</code> to narrow; <code>C-x n w</code> to widen. The other method is writing a specific enough prompt to match just the things you want. (The third, having the query mechanism activate at a specific node instead of the root node of the tree, is currently not supported.)</p><p class=" text-justify">Thanks to the query ring, you can write a handful of useful queries and store them for later use. And if that’s too much of a bother, you can write some elisp code to do this for you:</p><pre class="sourceCode commonlisp"><code class="sourceCode commonlisp">(<span class="kw">defun</span><span class="fu"> mp-edit-stuff </span>() (interactive) (combobulate--mc-place-nodes (combobulate-query-capture (combobulate-root-node) <span class="co">;; Put your query here!</span> <span class="st">"((pair) @before)"</span>)))</code></pre><p class=" text-justify">Now <a href="/article/mastering-key-bindings-emacs" class=" article-link">bind it to a key</a> and you’re set.</p><p class=" text-justify">Now, you might ask, what if you <em>don’t</em> use Multiple Cursors? Perhaps you <a href="/article/keyboard-macros-are-misunderstood" class=" article-link">prefer keyboard macros</a> which do have their own set of unique benefits. Well, read on…</p><H3 id="navigating-query-matches">Navigating Query Matches</H3><p class=" text-justify">Editing queries is but one side of the coin. The obverse is, of course, moving or navigating by query matches. Combobulate’s got basic support for Xref, the cross-referencing feature. It’s been in Emacs for what feels like forever, but nowadays it’s actually useful and it plays a prominent role as a long-sought-after replacement for the antediluvian TAGS search (that never really played well with anything else.)</p><p class=" text-justify">The Xref commands (usually bound to <code>M-?</code>, <code>C-M-?</code>, <code>M-.</code>, etc.) are worth learning about if you have never used them before. Eglot and friends plug into it, to provide seamless reference and definition search and much, much more.</p><p class=" text-justify">Combobulate does <em>not</em> plug into the backend system in Xref. The query thing is far too niche and specialized to want to do that. Instead, you must type <code>C-c x</code> in a query builder buffer, or <code>C-c o x b</code> to trigger Combobulate’s xref feature. It’ll use the query ring if you invoke the xref command outside of the query builder.</p><p class=" text-justify">Xref’s default behavior is to create a buffer with ordered matches that you can navigate with <code>M-g M-n</code> and <code>M-g M-p</code>, much like you can with <a href="/article/searching-buffers-occur-mode" class=" article-link">Occur mode</a>, <a href="/article/dired-shell-commands-find-xargs-replacement" class=" article-link">Grep &amp; find</a>, and <a href="/article/compiling-running-scripts-emacs" class=" article-link">Compilation mode</a>.</p><figure> <img src="/static/uploads/combobulate/combobulate-xref.svg" alt="" class=" center-block"> <figcaption>Build a query and then use Xref to jump to one of the matches. Great for <em>ad hoc</em> searching.</figcaption> </figure><p class=" text-justify">The problem with popping up a buffer is that it assumes you want to consume <em>all</em> the matches, or at least browse the matches alongside the code. So, if you want to quickly skip to a particular entry, the UI is a bit cumbersome.</p><p class=" text-justify">Luckily, you can fix that by customizing <code>xref-show-definitions-function</code> and <code>xref-show-xrefs-function</code>. Set them both to <code>xref-show-definitions-completing-read</code> and you’ll instead see your matches presented in your minibuffer (see picture) where you can pick the one you want. I betcha didn’t know Xref could do that! (Note that I had to set <em>both</em> variables for this feature to work properly. YMMV, but try setting <code>xref-show-definitions-function</code> on its own first.)</p><p class=" text-justify">Because completing read is more likely to prove useful for speedy jumping, there’s a special <code>combobulate-xref-show-xrefs-function</code> variable that defaults to <code>xref-show-definitions-completing-read</code>. Set it to <code>nil</code> to use the default Xref behavior, particularly if you want to use keyboard macros where you want to keep track of where you are between macro playbacks.</p><p class=" text-justify">So what can you use all this for then? Well, why not map out your routes (as I’ve done in the screenshot above) to quickly jump to the router entry in React. Or why not – and I’m sure you have this problem too – collect all your import or include statements with a query so you can jump to the one you want to amend? Sure, your LSP server’s supposed to help you with that, but let’s be honest: it’s often slow and, more often than not, a little bit forgetful or wrong about what you want.</p><p class=" text-justify">And there you have it. The latest crop of features to hit Combobulate which, you know, is still beta software. So expect bugs! You can <a href="https://github.com/mickeynp/combobulate" class=" article-link">find Combobulate on Github</a>.</p> /article/combobulate-editing-searching-new-query-builder Thu, 27 Jul 2023 09:19:30 +0000