Skip to content

Smart Scan: Jump between symbols in a buffer

Oct 31 13
by mickey

A few years ago I wrote Effective Editing I: Movement and in that article I included a bunch of code for a feature I called “Smart Scan.” Back then I didn’t bother putting it on Github so I just left it as source code embedded in the article. I’ve now realized that hundreds have stuck it in their .emacs file but without the benefit of any potential updates, and with no way to actually contribute to the package.

Basically, Smart Scan let’s you jump between symbols in your current buffer that matches the one point is on. It does it unintrusively and without any prompts or other fancy UI gimmicks. Simply put your point on a symbol you want to search for in your buffer and type either M-n or M-p to move forward or backward respectively. Give it a shot. I have moved it Github here. Patches welcome :) Enjoy!

WDired: Editable Dired Buffers

Oct 10 13
by mickey

A long while ago I wrote an article about Working with Multiple files in Dired. In the article I describe how you can work with files across multiple directories — for instance all the matches from a find call — in a dired buffer.

But wouldn’t it be great if you can combine the text editing capabilities of Emacs on the files themselves? Well, if you’re a long-time reader of this blog, you will recognize the rhetorical nature of the question and know instinctively that, yes, it is indeed possible.


WDired — meaning writable dired — has been part of Emacs for a long time now. This feature is another hidden gem of Emacs, only briefly mentioned in a sub node in the dired info manual.

Its mode of operation is simple: if any dired or dired-derived buffer is switched to writable from read-only with the globally-recognized binding C-x C-q, you can edit the dired buffer as though it were an ordinary buffer. If you are in editable dired mode the modeline for the dired buffer will say Editable.

Any change you make to the buffer will remain unchanged until you commit them by typing C-c C-c. To cancel the changes and revert to the original state you can type C-c ESC.

Configuring WDired

By default only the filename is editable, and for most that is good enough. There are, however, a couple of switches that let you change more than just the filename. If you set wdired-allow-to-change-permissions to t you can also edit the permission bits directly, although wdired will ensure you can only enter valid permission bits. If you want a free-form permission field without the system handholding, you can set it to advanced.

Another feature of wdired is its ability to rewrite symlinks. By default wdired-allow-to-redirect-links is set to t, meaning you can by default change the symlinks in Editable mode.

Two useful “safety” variables are wdired-use-interactive-rename which if set to t will prompt for every filename change you have made you when you commit the changes with C-c C-c, and wdired-confirm-overwrite asks if you want to overwrite files if your altered filenames conflict with existing files.

If you are a keen user of Emacs macros you may want to configure wdired-use-dired-vertical-movement as it governs how where the point is put when you navigate up and down the dired list. You can set it to one of three switches: nil, meaning it will not do anything out of the ordinary; sometimes, meaning Emacs will move the point to the beginning of filename if the point is before it; and t, meaning Emacs will always, unequivocally move the point to the beginning of the filename, mirroring the behavior in normal dired mode.

Using WDired

Probably the most useful part of WDired is the ability to edit filenames as though they were ordinary lines of text in a buffer. Most commands work as you would expect, including rectangle functions! Only the filename portion of text killed with the rectangle kill command C-x r k are killed. WDired is fairly intelligent and will only let you edit portions that make sense: you cannot change the file size, for instance, as that portion of the buffer is set to read only.

Replace regexp works well, but don’t expect ^foo to work as the filename is not actually at the beginning of file. Provided you express your regexp so it only affects the filename-portion of the dired buffer, you’re golden. This is a slightly annoying limitation but one worth living with. The functionality afforded by WDired is amazing: regexp replace, [upcase/capitalize]-word; elaborate macros — they all work.

Comint: Writing your own Command Interpreter

Jul 31 13
by mickey

One of the hallmarks of Emacs is its ability to interface with external processes and add to the user experience with the full coterie of Emacs functionality like syntax highlighting, macros, command history, and so on. This functionality has been in Emacs since time immemorial, yet few people make their own command interpreters — also known as comint — in Emacs, fearing that it is either really difficult or not possible.

It’s not surprising people feel this way when you consider how woefully underdocumented this functionality is; the few, scattered references to comint “mode” reveal little, and as a hacker you are forced to use the source, Luke.

The Theory

Before I demonstrate how you use comint, a quick briefing — but rough, as I’m leaving out a few boring steps — on how it works.

At its core, you are spawning a process and either redirecting the stdin, stdout and stderr pipes to Emacs; or you use a pseudo-terminal. The choice between the two is governed by process-connection-type, but it’s unlikely that you would ever want to change that manually.

The fundamental building blocks for interacting with processes are start-process, for kinda-sorta-asynchronous process calls; andcall-process, for synchronous process calls.

One layer above that and we get to comint with its very pared-down, basic interpreter framework. This is what things like M-x shell and the various REPL modes like Python build on. comint has handles all the nitty-gritty stuff like handling input/output; a command history; basic input/output filter hooks; and so on. In other words, it’s the perfect thing to build on if you want something interactive but want more than just what comint has to offer. To use comint is very simple: run-comint takes one argument, PROGRAM, and nothing else; run it with a filepath to your favourite program and watch it fly. For greater configurability, you can use make-comint-in-buffer.

Important caveat about pipe redirection: Oftentimes programs will detect that you are redirecting its pipes to a dumb terminal or file and it disables its shell prompt; this is extremely frustrating as not all programs detect that it is running inside Emacs by looking for the signature environment variables Emacs will set: EMACS and INSIDE_EMACS. If that happens you may get lucky and find a flag you can set to force it to run in “interactive” mode — for example, in Python it’s -i.

One layer above that and we get to things like M-x shell, which I’ve talked about before in Running Shells in Emacs: An Overview.

And finally, you can list all running/open processes by typing M-x list-processes.

Writing a Comint Mode

With that out of the way, let’s write some code. I’ve been playing around with Cassandra, the database, and like all respectable databases it has a fine commandline interface — but no Emacs mode! Oh no!

The most important thing about writing a comint mode is that it’s very easy to get 80% of the way there, but getting those remaining 20 percentage points is the really difficult part! I’m only doing the 80% here!

Let’s write one. To start the Cassandra CLI you run the program cassandra-cli and you’re presented with output similar to this:

If you run cassandra-cli with comint-run — you already have a working, interactive process. It’s barebones and simple, but its defaults are reasonable and it will work well enough. If you want to extend it, you have to write your own wrapper function around make-comint-in-buffer and write a major mode also. So let’s do just that.

The Comint Template

The first thing we need to do is declare some sensible variables so users can change the settings without having to edit the code. The first one is obvious: we need to store a path to cassandra-cli, the program we want to run.

The next variable, cassandra-cli-arguments, holds an (empty) list of commandline arguments.

The third, is an empty and currently disused mode map for storing our custom keybindings. It is inherited from comint-mode-map, so we get the same keys exposed in comint-mode.

Finally, we have cassandra-prompt-regexp, which holds a regular expression that matches the prompt style Cassandra uses. It so happens that by default it sort-of works already, but it pays to be prepared and it’s a good idea to have a regular expression that matches no more than it needs to. Furthermore, as you’re probably going to use this code to make your own comint derivatives, you’ll probably have to change it anyway.

This messy pile of code does some basic housekeeping like re-starting the Cassandra process if you’re already in the buffer, or create the buffer if it does not exist. Annoyingly there is a dearth of macros to do stuff like this in the comint library; a shame, as it would cut down on the boilerplate code you need to write. The main gist of this function is the apply call taking make-comint-in-buffer as the function. Quite honestly, a direct call to make-comint-in-buffer would suffice, but you lose out on niceties like restartable processes and so on; but if you’re writing a comint-derived mode for personal use you may not care about that sort of stuff.

And finally, we have our major mode definition. Observe that we derive from comint-mode. I overwrite the default comint-prompt-regexp with our own, and I force the prompt to be read only also. I add a mode hook and set comint-process-echoes to t to avoid duplicating the input we write on the screen. And finally, I tweak the paragraph settings so you can navigate between each prompt with M-{ and M-}.

And.. that’s more or less it for the template. It’s trivial to tweak it to your own needs and it’s a good place to start.

Let’s add some cool functionality to our cassandra-mode: basic syntax highlighting.

Extending Cassandra Mode

The first thing I want to do is add simple syntax highlighting for the commands you get when you run help;.

We need to think about some simple rules we can come up with that will highlight stuff. This is the hard bit: coming up with a regular expression for non-regular languages is nigh-on impossible to get right; especially not when you’re doing it for a commandline application that spits out all manner of output.

Before we do that though, let’s augment our major mode to support syntax highlighting (which is actually known as font locking in Emacs parlance.)

Add this form to the body of the major mode (next to the existing setq calls) and then add the following form to the top of the file, to hold our font lock rules:

There is one font lock rule: it highlights all matching keywords that I extracted from the help; command.

comint exposes a set of filter function variables that’re triggered and run (in order, it’s a list) when certain conditions are met:

Variable Name Purpose
comint-dynamic-complete-functions List of functions called to perform completion.
comint-input-filter-functions Abnormal hook run before input is sent to the process.
comint-output-filter-functions Functions to call after output is inserted into the buffer.
comint-preoutput-filter-functions List of functions to call before inserting Comint output into the buffer.
comint-redirect-filter-functions List of functions to call before inserting redirected process output.
comint-redirect-original-filter-function The process filter that was in place when redirection is started

Another useful variable is comint-input-sender, which lets you alter the input string mid-stream. Annoyingly its name is inconsistent with the filter functions above.

You can use them to control how input and output is processed and interpreted mid-stream.

And there you go: a simple, comint-enabled Cassandra CLI in Emacs.