Skip to content

What’s New in Emacs 24.3

Mar 11 13
by mickey

Emacs version 24.3 is now released to the public. This release, unlike 24.2, is chock full of goodies. I’ve taken the liberty of annotating things that’re relevant to me — and hopefully you, too, dear reader — but I’ve limited my commentary to things I’m familiar with. Always keen to hear what you think about the changes in the comments.

You can retrieve Emacs 24.3 from one of these locations:

Automatic mirror
Note: As of 11 AM on 2013-03-11 I had issues with some mirrors not yet having the new version available.
http://ftpmirror.gnu.org/emacs/emacs-24.3.tar.xz
http://ftpmirror.gnu.org/emacs/emacs-24.3.tar.gz

What’s New in Emacs 24.3

Installation Changes in Emacs 24.3

The default X toolkit is now Gtk+ version 3.
If you don’t pass --with-x-toolkit to configure, or if you use
--with-x-toolkit=gtk or --with-x-toolkit=yes, configure will try
to build with Gtk+ version 3, and if that fails, try Gtk+ version 2.
You can explicitly require a specific version by passing
--with-x-toolkit=gtk2 or --with-x-toolkit=gtk3 to configure.

Although I always use Emacs in a window manager, and actively recommend that people who can do as well, this change is unlikely to affect me much at all. I don’t use scrollbars, toolbars or menubars.

New configure option --enable-link-time-optimization, to utilize
an appropriate feature provided by GCC since version 4.5.0.


New configure option --without-all to disable most of the optional
features (image support, etc.) that are normally enabled by default.

For the luddites.

New configure option --enable-gcc-warnings (for developing/debugging
Emacs). If building with GCC, this enables compile-time checks that
warn/give errors about possibly-questionable C code. On a recent GNU
system there should be no warnings; on older and on non-GNU systems
the results may be useful to developers.

The configure option --enable-use-lisp-union-type has been
renamed to --enable-check-lisp-object-type, as the resulting
Lisp_Object type no longer uses a union to implement the compile time
check that this option enables.

The configure option --disable-maintainer-mode has been removed,
as it was confusingly-named and rarely useful.

The configure options --program-prefix, --program-suffix, and
--program-transform-name apply to more than just the installed
binaries. Now they also affect the man pages, icons, and the
etc/emacs.desktop file; but not the info pages, since this would break
links between the various manuals.

You can use NO_BIN_LINK=t make install to prevent the installation
overwriting “emacs” in the installation bin/ directory with a link
to “emacs-VERSION”.

Emacs uses libtinfo in preference to libncurses, if available.

On FreeBSD and NetBSD, configure no longer adds /usr/local/lib and
/usr/pkg/lib to the linker search path. You must add them yourself if
you want them.

The standalone scripts rcs-checkin and vcdiff have been removed
(from the bin and libexec directories, respectively). The former is
no longer relevant, the latter is replaced by lisp (in vc-sccs.el).

Startup Changes in Emacs 24.3

Emacs no longer searches for leim-list.el files beneath the standard
lisp/ directory. There should not be any there anyway. If you have
been adding them there, put them somewhere else; e.g., site-lisp.

The --no-site-lisp command line option now works for Nextstep builds.

Changes in Emacs 24.3

Help

C-h f (describe-function) can now perform autoloading.
When this command is called for an autoloaded function whose docstring
contains a key substitution construct, that function’s library is
automatically loaded, so that the documentation can be shown
correctly. To disable this, set help-enable-auto-load to nil.

That’s a potentially useful change, though I cannot offhand recall any instances where I’ve been bitten by this. This is not going to guard against unloaded libraries if you are not looking up autoloaded commands. In other words, if you’re looking up an obscure function or variable and the library isn’t loaded, this won’t help at all.

C-h f now reports previously-autoloaded functions as “autoloaded”,
even after their associated libraries have been loaded (and the
autoloads have been redefined as functions).

I would fill this under “Nice to know”.

ImageMagick

Images displayed via ImageMagick now support transparency and the
:background image specification property.

When available, ImageMagick support is automatically enabled.
It is no longer necessary to call imagemagick-register-types
explicitly to install ImageMagick image types; that function is called
automatically at startup, or when customizing an imagemagick- option.

Most excellent. This should help encourage the use of Imagemagick now that it’s loaded by default.

Setting imagemagick-types-inhibit to t now disables the use of
ImageMagick to view images. (You must call imagemagick-register-types
afterwards if you do not use customize to change this.)

… And this is how you disable it

The new variable imagemagick-enabled-types also affects which
ImageMagick types are treated as images. The function
imagemagick-filter-types returns the list of types that will be
treated as images.

Minibuffer

In minibuffer filename prompts, C-M-f and C-M-b now move to the
next and previous path separator, respectively.

With the caveat, I suppose, that this is not likely to work if you use iswitchb or IDO.

minibuffer-electric-default-mode can shorten “(default …)” to “[...]”
in minibuffer prompts. Just set minibuffer-eldef-shorten-default
non-nil before enabling the mode.

Mode line

New option mode-line-default-help-echo specifies the help text
(shown in a tooltip or in the echo area) for any part of the mode line
that does not have its own specialized help text.

You can now click mouse-3 in the coding system indicator to invoke
set-buffer-file-coding-system.

Server and client

emacsclient now obeys string values for initial-buffer-choice,
if it is told to open a new frame without specifying any file to visit
or expression to evaluate.

New option server-auth-key specifies a shared server key.

Emacs now generates backtraces on fatal errors.
On encountering a fatal error, Emacs now outputs a textual description
of the fatal signal, and a short backtrace on platforms like glibc
that support backtraces.

Other

C-x C-q is now bound to the new minor mode read-only-mode.
This minor mode replaces toggle-read-only, which is now obsolete.

Interesting that they’ve moved it to a new minor mode. I imagine this change was effected to reduce the number of “magic states” a buffer could be in. As a mode, it also means you can customize it further like, say, adding hooks so you can gray out the text if it’s read only.

Most y-or-n prompts now allow you to scroll the selected window.
Typing C-v or M-v at a y-or-n prompt scrolls forward or backward
respectively, without exiting from the prompt.

Very nice and welcome change.

In the Package Menu, newly-available packages are listed as “new”,
and sorted above the other “available” packages by default.

If your Emacs was built from a bzr checkout, the new variable
emacs-bzr-version contains information about the bzr revision used.

New option create-lockfiles specifies usage of lockfiles.
It defaults to t. Changing it to nil inhibits the creation of lock
files (use this with caution).

New option enable-remote-dir-locals, if non-nil, allows directory-local
variables on remote hosts.

This is useful if you use dir-local variables remotely, but there’s an obvious security risk if you enable it.

The entry for PCL-CVS has been removed from the Tools menu.
The PCL-CVS commands are still available via the keyboard.

Using “unibyte: t” in Lisp source files is obsolete.
Use “coding: raw-text” instead.

In the buffer made by M-x report-emacs-bug, the C-c m binding
has been changed to C-c M-i (report-emacs-bug-insert-to-mailer).
The previous binding, introduced in Emacs 24.1, was a mistake, because
C-c LETTER bindings are reserved for user customizations.

Internationalization

New language environment: Persian.

New input method vietnamese-vni.

Nextstep (GNUstep / Mac OS X) port

Support for fullscreen and the frame parameter fullscreen.

A much longed-for feature.

A file dialog is used for open/save operations initiated from the
menu/toolbar.

Editing Changes in Emacs 24.3

Search and Replace

Non-regexp Isearch now performs “lax” space matching.
Each sequence of spaces in the supplied search string may match any
sequence of one or more whitespace characters, as specified by the
variable search-whitespace-regexp. (This variable is also used by a
similar existing feature for regexp Isearch.)

This is great as you can now find matches even though whitespacing may not be spot on.

New Isearch command M-s SPC toggles lax space matching.
This applies to both ordinary and regexp Isearch.

Handy if you don’t want fuzzy space matching.

New option replace-lax-whitespace.
If non-nil, query-replace uses flexible whitespace matching too.
The default is nil.

Global M-s _ starts a symbol (identifier) incremental search,
and M-s _ in Isearch toggles symbol search mode.
M-s c in Isearch toggles search case-sensitivity.

Very useful if you’re coding. It’s equivalent to wrapping an isearch-forward-regexp regexp query in \_< and \_>. Keep in mind that the definition of a symbol varies according to your major mode (and sometimes minor!). It’s usually set to rather sensible defaults in most programming major modes.

Navigation commands

New binding M-g c for goto-char.

The word “char” here is rather misleading: it means the absolute position, as an integer, from the beginning of the buffer.

New binding M-g TAB for move-to-column.

M-g TAB (move-to-column) prompts for a column number if called
interactively with no prefix arg. Previously, it moved to column 1.

New option yank-handled-properties allows processing of text
properties on yanked text, in ways that are more general than just
removing them (as is done by yank-excluded-properties).

Useful if you want to strip out or alter properties on text when it’s yanked. This is unlikely to be of interest to anybody except module developers.

New option delete-trailing-lines specifies whether
M-x delete-trailing-whitespace should delete trailing lines at the end
of the buffer. It defaults to t.

A nice and useful addition, but be wary as it defaults to true.

C-u M-= now counts lines/words/characters in the entire buffer.

C-x 8 RET is now bound to insert-char, which is now a command.
ucs-insert is now an obsolete alias for insert-char.

The z key no longer has a binding in most special modes.
It used to be bound to kill-this-buffer, but z is too easy to
accidentally type.

Rejoice all who have accidentally killed your buffers.

New command C-x r M-w (copy-rectangle-as-kill).
It copies the region-rectangle as the last rectangle kill.

Very nice. This basically makes works exactly like M-w does except it operates on a rect. No more killing then yanking if you want to copy a rect.

Registers

C-x r + is now overloaded to invoke append-to-register.

This small change has rather wide ramifications for those of you who use registers in macros. You can now collect text and append it to the end of a register instead of shunting it to a different buffer. Groundbreaking? Not quite, but very useful indeed.

New option register-separator specifies the register containing
the text to put between collected texts for use with
M-x append-to-register and M-x prepend-to-register.

Combined with the above, this enables you to collate appended texts with a separator symbol.

Changes in Specialized Modes and Packages in Emacs 24.3

Common Lisp emulation (CL)

CL’s main entry is now (require ‘cl-lib).
cl-lib is like the old cl except that it uses the namespace cleanly;
i.e., all its definitions have the “cl-” prefix (and internal definitions
use the “cl–” prefix).

If cl provided a feature under the name foo, then cl-lib
provides it under the name cl-foo instead; with the exceptions of the
few cl definitions that had to use foo* to avoid conflicts with
pre-existing Elisp entities. These have been renamed to cl-foo
rather than cl-foo*.

The old cl is now deprecated and is mainly just a bunch of aliases that
provide the old, non-prefixed names. Some exceptions are listed below:

This is a good change but I doubt we will ever truly be rid of cl, not for a long tie anyway.

cl-flet is not like flet (which is deprecated).
Instead it obeys the behavior of Common-Lisp’s flet.
In particular, in cl-flet function definitions are lexically scoped,
whereas in flet the scoping is dynamic.

cl-labels is slightly different from labels.
The difference is that it relies on the lexical-binding machinery
(as opposed to the lexical-let machinery used previously) to capture
definitions in closures, so such closures will only work if lexical-binding
is in use.

cl-letf is not exactly like letf.
The only difference is in details that relate to some deprecated usage
of symbol-function in place forms.

progv was rewritten to use the let machinery.
A side effect is that variables without corresponding values are bound
to nil rather than being made unbound.

The following methods of extending setf are obsolete
(use features from gv.el instead):
define-modify-macro (use gv-letplace)
defsetf (use gv-define-simple-setter or gv-define-setter)
define-setf-expander (use gv-define-setter or gv-define-expander)
get-setf-method no longer exists (see “Incompatible Lisp Changes”)

Diff mode

Changes are now highlighted using the same color scheme as in
modern VCSes. Deletions are displayed in red (new faces
diff-refine-removed and smerge-refined-removed, and new definition
of diff-removed), insertions in green (new faces diff-refine-added
and smerge-refined-added, and new definition of diff-added).

More standardization. Good.

The variable diff-use-changed-face defines whether to use the
face diff-changed, or diff-removed and diff-added to highlight
changes in context diffs.

The new command diff-delete-trailing-whitespace removes trailing
whitespace introduced by a diff.

Ediff now uses the same color scheme as Diff mode.

Python mode

A new version of python.el, which provides several new features, including:
per-buffer shells, better indentation, Python 3 support, and improved
shell-interaction compatible with iPython (and virtually any other
text based shell).

I blogged about a new python mode a long time ago and it seems it’s made it into trunk. That’s probably good news for most Python users, but as I haven’t yet explored the new python.el mode yet, I will cover this in much greater detail in another post.

Some user options have been replaced/renamed, including (old -> new):

python-indent -> python-indent-offset

python-guess-indent -> python-indent-guess-indent-offset

python-pdbtrack-do-tracking-p -> python-pdbtrack-activate

python-use-skeletons -> python-skeleton-autoinsert

Some user options have been removed, including:

python-indent-string-contents: Strings are never indented.

python-honour-comment-indentation:
Comments are always considered as indentation markers.

python-continuation-offset: Indentation is automatically
calculated in a pep8 compliant way depending on the context.

python-shell-prompt-alist, python-shell-continuation-prompt-alist:
Have no direct mapping as the shell interaction is completely different.

python-python-command, python-jython-command:
Replaced by python-shell-interpreter.

inferior-python-filter-regexp, python-remove-cwd-from-path,
python-pdbtrack-minor-mode-string, python-source-modes:
No longer relevant.

Some commands have been replaced (old -> new):
python-insert-class -> python-skeleton-class
python-insert-def -> python-skeleton-def
python-insert-for -> python-skeleton-for
python-insert-if -> python-skeleton-if
python-insert-try/except -> python-skeleton-try
python-insert-try/finally -> python-skeleton-try
python-insert-while -> python-skeleton-while
python-find-function -> python-nav-jump-to-defun
python-next-statement -> python-nav-forward-sentence
python-previous-statement -> python-nav-backward-sentence
python-beginning-of-defun-function -> python-nav-beginning-of-defun
python-end-of-defun-function -> python-nav-end-of-defun
python-send-buffer -> python-shell-send-buffer
python-send-defun -> python-shell-send-defun
python-send-region -> python-shell-send-region
python-send-region-and-go -> emulate with python-shell-send-region
and python-shell-switch-to-shell
python-send-string -> python-shell-send-string
python-switch-to-python -> python-shell-switch-to-shell
python-describe-symbol -> python-eldoc-at-point

Needless to say, the changes here are likely to break a lot of my custom Python code I’ve written over the years. If you’re relying on customizations to Python or modes that interact with it, I would probably ensure you can fix or live with broken modules until they’re fixed.

D-Bus

New variables dbus-compiled-version and dbus-runtime-version.

The D-Bus object manager interface is implemented.

Variables of type :( u)int32 and :( u)int64 accept floating points,
if their value does not fit into Emacs’s integer range.

The function dbus-call-method is now non-blocking.
It can be interrupted by C-g. dbus-call-method-non-blocking is obsolete.

Signals can also be sent as unicast messages.

The argument list of dbus-register-signal has been extended,
according to the new match rule types of D-Bus.

dbus-init-bus supports private connections.

There is a new function dbus-setenv.

desktop-path no longer includes the “.” directory.
Desktop files are now located in ~/.emacs.d by default.

Dired

dired-do-async-shell-command executes each file sequentially
if the command ends in ; (when operating on multiple files).
Otherwise, it executes the command on each file in parallel.

Very useful and good for people working with commands on many files.

Typing M-n in the minibuffer of dired-do-chmod, dired-do-chgrp,
dired-do-chown, and dired-do-touch yanks the attributes of the
file at point.

Hah! This is superb. I really like this change. Now you’re probably thinking the command, M-n, is arbitrary, but not so: in minibuffer prompts that command would go to the last used element in the minibuffer history; by re-using that command here, it achieves the same effect.

When the region is active, m (dired-mark), u (dired-unmark),
DEL (dired-unmark-backward), and d (dired-flag-file-deletion)
mark/unmark/flag all files in the active region.

Can i hear a hallelujah? this is a most welcome change and one that should’ve been in dired since day one.

The minibuffer default for = (dired-diff) has changed.
it is now the backup file for the file at point, if one exists.
in transient mark mode the default is the file at the active mark.

great idea but for most serious files you’re probably using c-x v =, the vcs diff command.

M-= is no longer bound to dired-backup-diff in Dired buffers.
The global binding for M-=, count-words-region is in effect.

ERC

New module “notifications”, which can send a notification when you
receive a private message or your nickname is mentioned.

ERC will look up server/channel names via auth-source and use any
channel keys found.

New option erc-lurker-hide-list, similar to erc-hide-list, but
only applies to messages sent by lurkers.

reStructuredText mode

Keybindings (see C-c C-h), TAB indentation, filling and auto-filling,
fontification, comment handling, and customization have all been revised
and improved.

That’s good. rST mode always did seem a bit… underdeveloped.

Support for imenu and which-function-mode.

Great!

The reStructuredText syntax is more closely covered.
Sphinx support has been improved.

Sphinx support is very good news for Python docs writers and other fans of Sphinx.

rst-insert-list inserts new list or continues existing lists.

A negative prefix argument always works for rst-adjust.

The window configuration is reset after displaying a TOC.

The constant rst-version describes the rst.el package version.

Ruby mode

Support for percent literals and recognition of regular expressions
in method calls without parentheses with more methods, including Cucumber
steps definitions.

Improved syntax highlighting and indentation.

New command ruby-toggle-block, bound to C-c {.

Some non-standard keybindings/commands have been removed:

ruby-electric-brace; use electric-indent-mode instead.

ruby-mark-defun; use mark-defun.

ruby-beginning-of-defun and ruby-end-of-defun are replaced by
appropriate settings for the variables beginning-of-defun-function
and end-of-defun-function.

These three changes are all part of a larger drive in Emacs to remove major mode-specific functions that’re better served with the built-in Emacs equivalents.

Non-standard keybindings for backward-kill-word, comment-region,
reindent-then-newline-and-indent and newline have been removed.

Shell Script mode

Pairing of parens/quotes uses electric-pair-mode instead of skeleton-pair.

More duplicate cruft removed and standardized.

sh-electric-here-document-mode now controls auto-insertion of here-docs.

sh-use-smie lets you choose a new indentation and navigation code.

VHDL mode

The free software compiler GHDL is supported (and now the default).

Support for the VHDL-AMS packages has been added/updated.

Updated to the 2002 revision of the VHDL standard.

Accepts \r and \f as whitespace.

Apropos

The faces used by Apropos are now directly customizable.
These faces are named apropos-symbol, apropos-keybinding, and so on;
see the apropos Custom group for details.

The old options whose values specified faces to use have been removed
(i.e. apropos-symbol-face, apropos-keybinding-face, etc.).

Buffer Menu

This package has been rewritten to use Tabulated List mode.

Option Buffer-menu-buffer+size-width is now obsolete.
Use Buffer-menu-name-width and Buffer-menu-size-width instead.

Calendar

You can customize the header text that appears above each calendar month.
See the variable calendar-month-header.

New LaTeX calendar style, produced by cal-tex-cursor-week2-summary.

The calendars produced by cal-html include holidays.
Customize cal-html-holidays to change this.

CEDET

The major modes from the parser generators “Bovine” and “Wisent”
are now properly integrated in Emacs. The file suffixes “.by” and “.wy”
are in auto-mode-alist, and the corresponding manuals are included.

EDE

Menu support for the “Configuration” feature. This allows users to
choose the active configuration (such as debug or install) from the menu.

New command ede-set to interactively set project-local variables.

Support for compiling, debugging, and running in “generic” projects.

Autoconf editing support for M4 macros with complex arguments.

Compilation support for the “linux” project type.

“simple” projects have been removed; use “generic” projects instead.

Semantic

Support for parsing #include statements inside a namespace in C/C++.

Improved support for ‘extern “C”‘ declarations in C/C++.

The ability to ignore more common special C/C++ preprocessor symbols,
such as ‘__nonnull’ and ‘__asm’. Add ‘__cplusplus’ macro when parsing C++.
If available, include cdefs.h as an additional source of preprocessor symbols.

Improved C/C++ function pointer parsing.

In Python, support for converting imports to include file names.

Ability to dynamically determine the Python load path.

Support for the Python ‘WITH’ and ‘AT’ keywords.

Improved tooltip completion.

SRecode

The SRecode manual is now included.

Tag generation supports constructor/destructor settings and system
include differentiation.

Addition of ‘Framework’ support: Frameworks are specified when a
particular kind of library (such as Android) is needed in a common language
mode (like Java).

Support for nested templates and let variables override based on priority.

Support for merging tables from multiple related modes, such as
default -> c++ -> arduino.

Compile

Compile has a new option compilation-always-kill.

Customize

custom-reset-button-menu now defaults to t.

Non-option variables are never matched in customize-apropos and
customize-apropos-options (i.e., the prefix argument does nothing for
these commands now).

Term

The variables term-default-fg-color and term-default-bg-color
are now deprecated in favor of the customizable face term.

A pre-customization anachronism removed at last. Make sure you migrate your term color settings!

You can customize how to display ANSI terminal colors and styles
by customizing the corresponding term-color-COLOR,
term-color-underline and term-color-bold faces.

If, like me, you customized ansi-color-names-vector to change the default term colours I suggest you switch to using the faces now. The good news here is you can, should desire to, change more than just the colours for each ANSI Color: there’s nothing stopping you from forcing a different font for certain colours.

Tramp

The syntax has been extended in order to allow ad-hoc proxy definitions.

This is a great change and a most needed one if you want to su to a different user once you’ve connected to a another host and you can’t be bothered figuring out how TRAMP proxies work. Typically I can’t find any information about the new syntax anywhere… I’ll have to dig into the source code and write a post about it later. Despite my best attempts it seems I completely missed the new chapter, here (info "(tramp)Ad-hoc multi-hops").

Remote processes are now also supported on remote MS-Windows hosts.

URL

Structs made by url-generic-parse-url have nil attributes slot.
Previously, this slot stored semicolon-separated attribute-value pairs
appended to some imap URLs, but this is not compatible with RFC 3986.
So now the filename slot stores the entire path and query components,
and the attributes slot is always nil.

New function url-encode-url for encoding a URI string.
The url-retrieve function now uses this to encode its URL argument,
in case that is not properly encoded.

notifications.el supports now version 1.2 of the Notifications API.
The function notifications-get-capabilities returns the supported
server properties.

More misc changes

Flymake uses fringe bitmaps to indicate errors and warnings.
See flymake-fringe-indicator-position, flymake-error-bitmap and
flymake-warning-bitmap.

The FFAP option ffap-url-unwrap-remote can now be a list of strings,
specifying URL types that should be converted to remote file names at
the FFAP prompt. The default is now ‘(“ftp”).

New Ibuffer derived-mode filter, bound to / M.
The old binding for / M (filter by used-mode) is now bound to / m.

New option mouse-avoidance-banish-position specifies where the
banish mouse avoidance setting moves the mouse.

In Perl mode, new option perl-indent-parens-as-block causes non-block
closing brackets to be aligned with the line of the opening bracket.

In Proced mode, new command proced-renice renices marked processes.

New option async-shell-command-buffer specifies the buffer to use
for a new asynchronous shell-command when the default output buffer
*Async Shell Command* is already in use.

Handy if you don’t want your shell output buffer overwritten when you run more than one asynchronous command.

SQL mode has a new option sql-db2-escape-newlines.
If non-nil, newlines sent to the command interpreter will be escaped
by a backslash. The default does not escape the newlines and assumes
that the sql statement will be terminated by a semicolon.

New command tabulated-list-sort, bound to S in Tabulated List mode
(and modes that derive from it), sorts the column at point, or the Nth
column if a numeric prefix argument is given.

Its use should be pretty straight forward but I find that mode a bit buggy. Probably better if you stick to using the one in org mode.

which-func-modes now defaults to t, so Which Function mode, when
enabled, applies to all applicable major modes.

A nice change as Which Function mode worked in more buffers than the which-func-modes gave it credit for.

winner-mode-hook now runs when the mode is disabled, as well as when
it is enabled.

Follow mode no longer works by using advice.
The option follow-intercept-processes has been removed.

javascript-generic-mode is now an obsolete alias for js-mode.

Hooks renamed to avoid obsolete “-hooks” suffix:
semantic-lex-reset-hooks -> semantic-lex-reset-functions
semantic-change-hooks -> semantic-change-functions
semantic-edits-new-change-hooks -> semantic-edits-new-change-functions
semantic-edits-delete-change-hooks -> semantic-edits-delete-change-functions
semantic-edits-reparse-change-hooks -> semantic-edits-reparse-change-functions
semanticdb-save-database-hooks -> semanticdb-save-database-functions
c-prepare-bug-report-hooks -> c-prepare-bug-report-hook
rcirc-sentinel-hooks -> rcirc-sentinel-functions
rcirc-receive-message-hooks -> rcirc-receive-message-functions
rcirc-activity-hooks -> rcirc-activity-functions
rcirc-print-hooks -> rcirc-print-functions
dbus-event-error-hooks -> dbus-event-error-functions
eieio-pre-method-execution-hooks -> eieio-pre-method-execution-functions
checkdoc-style-hooks -> checkdoc-style-functions
checkdoc-comment-style-hooks -> checkdoc-comment-style-functions
archive-extract-hooks -> archive-extract-hook
filesets-cache-fill-content-hooks -> filesets-cache-fill-content-hook
hfy-post-html-hooks -> hfy-post-html-hook
nndiary-request-create-group-hooks -> nndiary-request-create-group-functions
nndiary-request-update-info-hooks -> nndiary-request-update-info-functions
nndiary-request-accept-article-hooks -> nndiary-request-accept-article-functions
gnus-subscribe-newsgroup-hooks -> gnus-subscribe-newsgroup-functions

Obsolete packages

assoc.el
In most cases, assoc+member+push+delq work just as well.
And in any case it’s just a terrible package: ugly semantics, terrible
inefficiency, and not namespace-clean.
*** bruce.el
*** cust-print.el
*** ledit.el
*** mailpost.el
*** mouse-sel.el
*** patcomp.el

Incompatible Lisp Changes in Emacs 24.3

Docstrings starting with * no longer indicate user options.
Only variables defined using defcustom are considered user options.
The function user-variable-p is now an obsolete alias for
custom-variable-p.

The return values of defalias, defun and defmacro have changed,
and are now undefined. For backwards compatibility, defun and
defmacro currently return the name of the newly defined
function/macro, but this should not be relied upon.

random by default now returns a different random sequence in
every Emacs run. Use (random S), where S is a string, to set the
random seed to a value based on S, in order to get a repeatable
sequence in later calls.

If the NEWTEXT arg to replace-match contains a substring “\?”,
that substring is inserted literally even if the LITERAL arg is
non-nil, instead of causing an error to be signaled.

select-window now always makes the window’s buffer current.
It does so even if the window was selected before.

The function x-select-font can return a font spec, instead of a
font name as a string. Whether it returns a font spec or a font name
depends on the graphical library.

face-spec-set no longer sets frame-specific attributes when the
third argument is a frame (that usage was obsolete since Emacs 22.2).

set-buffer-multibyte now signals an error in narrowed buffers.

The CL package’s get-setf-method function no longer exists.
Generalized variables are now part of core Emacs Lisp, and implemented
differently to the way cl.el used to do it. It is not possible to
define a compatible replacement for get-setf-method. See the file
gv.el for internal details of the new implementation.

The arguments of dbus-register-signal are no longer just strings,
but keywords or keyword-string pairs. The old argument list will
still be supported for Emacs 24.x.

Miscellaneous name changes
Some Lisp symbols have been renamed to correct their spelling,
or to be more consistent with standard Emacs terminology.

Renamed functions
hangul-input-method-inactivate -> hangul-input-method-deactivate
inactivate-input-method -> deactivate-input-method
quail-inactivate -> quail-deactivate
robin-inactivate -> robin-deactivate
viper-inactivate-input-method -> viper-deactivate-input-method
viper-inactivate-input-method-action ->
viper-deactivate-input-method-action
ucs-input-inactivate -> ucs-input-deactivate

Renamed hooks
The old hooks are still supported for backward compatibility, but they
are deprecated and will be removed eventually.
input-method-inactivate-hook -> input-method-deactivate-hook
robin-inactivate-hook -> robin-deactivate-hook
quail-inactivate-hook -> quail-deactivate-hook

Renamed variables
follow-deactive-menu -> follow-inactive-menu
inactivate-current-input-method-function ->
deactivate-current-input-method-function

Some obsolete functions, variables, and faces have been removed:
last-input-char, last-command-char, unread-command-char
facemenu-unlisted-faces
rmail-decode-mime-charset
iswitchb-read-buffer
sc-version, sc-submit-bug-report
set-char-table-default
string-to-sequence (use string-to-list or string-to-vector)
compile-internal
modeline
mode-line-inverse-video
follow-mode-off-hook
cvs-commit-buffer-require-final-newline
(use log-edit-require-final-newline instead)
cvs-changelog-full-paragraphs
(use log-edit-changelog-full-paragraphs instead)
cvs-diff-ignore-marks, cvs-diff-buffer-name
vc-ignore-vc-files (use vc-handled-backends instead)
vc-master-templates (use vc-handled-backends instead)
vc-checkout-carefully

Lisp Changes in Emacs 24.3

CL-style generalized variables are now in core Elisp.
setf is autoloaded; push and pop accept generalized variables.
You can define your own generalized variables using gv-define-simple-setter,
gv-define-setter, etc.

Emacs tries to macroexpand interpreted (non-compiled) files during load.
This can significantly speed up execution of non-byte-compiled code,
but can also bump into previously unnoticed cyclic dependencies.
These are generally harmless: they will simply cause the macro calls
to be left for later expansion (as before), but will result in a
warning (“Eager macro-expansion skipped due to cycle”) describing the cycle.
You may wish to restructure your code so this does not happen.

New sampling-based Elisp profiler.
Try M-x profiler-start, do some work, and then call M-x profiler-report.
When finished, use M-x profiler-stop. The sampling rate can be based on
CPU time or memory allocations.

defun also accepts a (declare DECLS) form, like defmacro.
The interpretation of the DECLS is determined by defun-declarations-alist.

New macros setq-local and defvar-local.

Face underlining can now use a wave.

Now you can finally make misspelled words in ispell-minor-mode look like Microsoft Word with squiggly underlines!

read-regexp has a new argument HISTORY; the first argument PROMPT
of read-regexp accepts a string ending with a colon and space, and its
second argument DEFAULTS can be a list of strings accessible via M-n
in the minibuffer ahead of other hard-coded useful regexp-related values.
More commands use read-regexp now to read their regexp arguments.

Very useful change for those of us who use readers and often find ourselves reinventing the history wheel.

Completion

New function completion-table-with-quoting to handle completion
in the presence of quoting, such as file completion in shell buffers.

New function completion-table-subvert to use an existing completion
table, but with a different prefix.

Debugger

New error type and new function user-error.
These do not trigger the debugger.

New option debugger-bury-or-kill, saying what to do with the
debugger buffer when exiting debug.

Set debug-on-message to enter the debugger when a certain
message is displayed in the echo area. This can be useful when trying
to work out which code is doing something.

New var inhibit-debugger, automatically set to prevent accidental
recursive invocations.

Window handling

New command fit-frame-to-buffer adjusts the frame height to
fit the contents.

The command fit-window-to-buffer can adjust the frame height
if the new option fit-frame-to-buffer is non-nil.

New macro with-temp-buffer-window, similar to
with-output-to-temp-buffer.

This is a useful command if you want to generate output with prin1, etc. or print output from an external command in a separate window. I’m personally very happy to see this macro make an appearance.

temp-buffer-resize-mode no longer resizes windows that have been
reused.

New option switch-to-buffer-preserve-window-point to restore a
window’s point when switching buffers.

New display action alist entries window-height and window-width
specify the size of new windows created by display-buffer.

New display action alist entry pop-up-frame-parameters, if
non-nil, specifies frame parameters to give any newly-created frame.

New display action alist entry inhibit-switch-frame, if non-nil,
tells display action functions to avoid changing which frame is
selected.

New display action alist entry previous-window, if non-nil,
specifies window to reuse in display-buffer-in-previous-window.

New display action functions display-buffer-below-selected,
and display-buffer-in-previous-window.

The functions get-lru-window, get-mru-window and get-largest-window
now accept a third argument to avoid choosing the selected window.

Additional values recognized for option window-combination-limit.

The following variables are obsolete, as they can be replaced by
appropriate entries in the display-buffer-alist function introduced
in Emacs 24.1:
dired-shrink-to-fit
display-buffer-reuse-frames
display-buffer-function
special-display-buffer-names
special-display-frame-alist
special-display-function
special-display-regexps

Time

current-time-string no longer requires that its argument’s year
must be in the range 1000..9999. It now works with any year supported
by the underlying C implementation.

current-time now returns extended-format time stamps
(HIGH LOW USEC PSEC), where the new PSEC slot specifies picoseconds.
PSEC is typically a multiple of 1000 on current machines. Other
functions that use this format, such as file-attributes and
format-time-string, have been changed accordingly. Old-format time
stamps are still accepted.

The format of timers in timer-list and timer-idle-list is now
[TRIGGERED-P HI-SECS LO-SECS USECS REPEAT-DELAY FUNCTION ARGS IDLE-DELAY PSECS].
The PSECS slot is new, and uses picosecond resolution. It can be
accessed via the new timer--psecs accessor.

Last-modified time stamps in undo lists now are of the form
(t HI-SECS LO-SECS USECS PSECS) instead of (t HI-SECS . LO-SECS).

(Old MacDonald had a farm) EIEIO

Improved security when handling persistent objects:

eieio-persistent-read now features optional arguments for specifying
the class to load, as well as a flag stating whether subclasses are allowed;
if provided, other classes will be rejected by the reader. For
compatibility with existing code, if the class is omitted only a
warning is issued.

New specialized reader for pulling in classes and signaling errors
without evaluation of suspicious code.

All slots that contain objects must have a :type. Slots with lists
of objects must use a new type predicate for a list of an object type.

Support for find-function and similar utilities, through the addition
of filename support to generated symbols.

Even more misc stuff

Floating point functions now always return special values like NaN,
instead of signaling errors, if given invalid args; e.g., (log -1.0).
Previously, they returned NaNs on some platforms but signaled errors
on others. The affected functions are acos, asin, tan, exp, expt,
log, log10, sqrt, and mod.

New fringe bitmap exclamation-mark.

Seems like flymake’s using it for errors. Neat.

Miscellaneous changes to special forms and macros

defun and defmacro are now macros rather than special forms.

If you ever wanted to know how a function is created in Emacs, now you can look at the lisp code in byte-run.el. Ditto for defmacro.

kbd is now a function rather than a macro.

Miscellaneous new functions

set-temporary-overlay-map sets up a temporary keymap that
takes precedence over most other maps for a short while (normally one key).

autoloadp tests if its argument is an autoloaded object.

autoload-do-load performs the autoloading operation.

buffer-narrowed-p tests if the buffer is narrowed.

file-name-base returns a file name sans directory and extension.

function-get fetches a function property, following aliases.

posnp tests if an object is a posn.

system-users returns the user names on the system.

system-groups returns the group names on the system.

tty-top-frame returns the topmost frame of a text terminal.

The following functions and variables are obsolete:

automount-dir-prefix (use directory-abbrev-alist)
buffer-has-markers-at
macro-declaration-function (use macro-declarations-alist)
window-system-version (provides no useful information)
dired-pop-to-buffer (use dired-mark-pop-up)
query-replace-interactive
font-list-limit (has had no effect since Emacs < 23)

Changes in Emacs 24.3 on Non-Free Operating Systems

Cygwin builds can use the native MS Windows user interface.
Pass --with-w32 to configure. The default remains the X11 interface.

Two new functions are available in Cygwin builds:
cygwin-convert-file-name-from-windows and
cygwin-convert-file-name-to-windows. These functions allow Lisp
code to access the Cygwin file-name mapping machinery to convert
between Cygwin and Windows-native file and directory names.

When invoked with the -nw switch to run on the Windows text-mode terminal,
Emacs now supports mouse-highlight, help-echo (in the echo area), and
mouse-autoselect-window.

On MS Windows Vista and later Emacs now supports symbolic links.

On MS Windows, you can pass --without-libxml2 to configure.bat to omit
support for libxml2, even if its presence is detected.

On Mac OS X, the Nextstep port requires OS X 10.4 or later.

On Mac OS X, configure no longer automatically adds the Fink “/sw”
directories to the search path. You must add them yourself if you want them.

The End!

Phew. That was a lot of changes and I’ve barely explained 10% of them. Let me know if I glossed over some cool new stuff.

Evaluating Lisp Forms in Regular Expressions

Jan 25 13
by mickey

One of the oft-repeated complaints about Emacs is its antiquated regular expression engine: that it cannot compete with Vims’; that it’s GNU-style regex and not PCRE; and that you have to quote everything in triplicates and write your regular expression on carbon copied paper or something. There is some truth to this, but its detractors overlook the features it adds that you won’t find in most other editors or regexp implementations.

I recently did a VimGolf challenge where I abused sort-regexp-fields so I could swap two different words. I decided to do it in the most obtuse way possible so I could demonstrate the flexibility of Emacs’s sort commands, and solve what would be a banal challenge in any editor — swapping exactly two elements, once, in a two-line file — using an “eclectic” feature.

But there’s a better way: a “scalable” way that works with an arbitrary number of elements to swap, and it uses.. regular expressions!

Evaluating Forms

Due to the pervasive nature of Elisp in Emacs, you can invoke elisp from within a call to replace-regexp. That is to say, in the Replace with: portion of the call, you can tell Emacs to evaluate a form and return its output in lieu of (or in combination with) normal strings. The syntax is \,(FORM).

Let’s say you have the following file:

       INITECH.TXT
----------------------------
Peter Gibbons	$15000
Milton Waddams	$12500
Bill Lumbergh	$90000

And you want to give everybody a raise; well, you don’t reaally want to give Lumbergh a raise, but ho hum, right?

The simplest method is to use [query-]replace-regexp with the following parameters.

Replace regexp: \$\([0-9]+\)

Here we search for the salary; we store the dollar amount in a capturing group.

Replace regexp with: $\,(* \#1 2)

And here we replace the salary with the output from an elisp form that multiplies the result from capturing group \1 — but in this case represented as \#1, as we want Emacs to convert the result to an integer first — by two.

And now the output looks like:

       INITECH.TXT
----------------------------
Peter Gibbons	$30000
Milton Waddams	$25000
Bill Lumbergh	$180000

Neat.

Conditional Replace

Let’s refactor some random Java code into CamelCase:

public class application_runner {
    public static void main(String[] args) {
        new Application(create_os_specific_factory());
    }
 
    public static GUIFactory create_os_specific_factory() {
        int sys = read_from_config_file("OS_TYPE");
        if (sys == 0) return new WinFactory();
        else return new OSXFactory();
Replace regexp: _\([[:lower:]]+\)

Replace underscore and all lowercase characters, which we capture for use later.

Replace regexp with: \,(capitalize \1)

Here I use an elisp form to capitalize each match from capturing group \1. The underscore is removed also, so create_os_specific_factory becomes createOsSpecificFactory, and so on.

And now it looks like this:

public class applicationRunner {
    public static void main(String[] args) {
        new Application(createOsSpecificFactory());
    }
 
    public static GUIFactory createOsSpecificFactory() {
        int sys = readFromConfigFile("OS_TYPE");
        if (sys == 0) return new WinFactory();
        else return new OSXFactory();

Unfortunately, the word Os should probably be written like this, OS. Let’s change the form so that certain words are treated differently.

Replace regexp: _\([[:lower:]]+\)

Same as before.

Replace regexp with: \,(cond ((member \1 '("os")) (upcase \1)) (t (capitalize \1)))

This time I’ve use a cond form — basically a case statement — to check that if the capturing group is a member of the list ("os"), we call the function upcase which, you guessed it, uppercases \1; if that condition fails, fall through to the second clause which, because its conditional is t, always returns true and, therefore, we call capitalize on \1. I could’ve used an if statement and saved a bit of typing, but cond is more flexible if you want more than one or two conditionals.

The end result is that we now uppercase OS and leave the rest capitalized.

Swapping Elements

Going back to what I said in the beginning about swapping text. It is perfectly possible, as I’m sure you can imagine now, to swap two items — you don’t even need a cond element for that!

Consider this trivial Python function.

def foobar():
    x = 10
    y = 42
    if x > 10 and y < 40:
        return y + y
    else:
        return x + x

Let’s say we want to swap x and y. That is, of course, a no-op, but there’s nothing stopping you from extending the example here and using it for something more meaningful, such as reversing > and <.

Replace: \(x\)\|y

This is the most simple way of swapping two values. You only need one capturing group, because we only need to compare against one capturing group, hence why y isn’t captured.

Replace with: \,(if \1 "y" "x")

Next, we test for existence; if \1 is not nil — that is to say, we actually have something IN the capturing group, which would only happen if we encounter x — we fall through to the THEN clause ("y") in the if form; if it is nil, we fall through to the ELSE clause ("x").

The result is what you would expect:

def foobar():
    y = 10
    x = 42
    if y > 10 and x < 40:
        return x + x
    else:
        return y + y

The variables are swapped. But let’s say you want to swap more than two elements: you’d need to nest if statements (ugly) or use cond (less ugly, but equally verbose.)

More Cond Magic

Consider the string 1 2 3. To turn it into One Two Three you have two ways of doing it:

One, you can use N capturing groups like so: \(1\)\|...\|\(N\) and in a cond check for the existence of each capturing group to determine if it is the “matched” one. There’s no reason why you couldn’t use just one capturing group and then string match for each item, but it’s swings and roundabouts: you’re either grouping each match you care about in the search bit, or you’re checking for the existence of the elements you care about in a form in the replace bit.

Let’s go with the first option.

Replace: \(1\)\|\(2\)\|\(3\)

Look for, and capture, the three integers.

Replace with: \,(cond (\1 "One") (\2 "Two") (\3 "Three"))

Using cond, if any of the three capturing groups are non-nil, the body of that conditional is returned.

Not unsurprisingly, the result is One Two Three.

If you change the capturing groups to this \(1\|2\)\|3 and if you then change the replacement string to \,(cond (\1 "One") (\2 "Two")), you end up with One One Two. So as you can see, it’s very easy to create rules that merge multiple different strings into one using cond.

More ideas?

I think I’ve amply demonstrated the power of mixing regular expressions and lisp. I know a few more tricks, but I’m always keen to find more, so if you have a novel application I’d love to hear about it.

There’s nothing stopping you from invoking elisp functions that don’t return anything useful; that is, you’re free to call functions just for their side effects, if that is what you want. For instance, if you want to open a list of files in a buffer, use \,(find-file-noselect \1). Another useful application is formatting strings using format. Check out the help file (C-h f format RET) for more information. And lastly, you can use Lisp lists to gather matches for use programmatically later: M-: (setq my-storage-variable '()) then \,(add-to-list 'my-storage-variable \1).

Know any useful regexp hacks? Post a comment!

Fun with Vimgolf 4: Transpositioning text with Tables

Jan 17 13
by mickey

Here’s another vim challenge, and one you might actually encounter frequently in real life.

Transpose the original lines in separate columns, one for each line.

Link to challenge.

Simple, really; a transposition here, some alignment there… but can we do better than the good ole’ brute force approach the Vim guys will invariably use? Can we do it without a call to a shell commands like column and paste? (Psst.. yes we can! It’s Emacs!)

Here’s the original data:

ultricies, vehicula, felis, sed, auctor, aenean, euismod, semper, quam, dapibus
nibh, consequat, consequat, maecenas, sit, amet, mauris, justo, quis, porttitor
curabitur, pharetra, euismod, orci, sit, amet, ullamcorper, mi, tincidunt, et
vitae, lorem, at, mi, feugiat, convallis, ac, eget, dui, fusce
blandit, iaculis, nulla, sit, amet, dolor, nec, est, ornare, volutpat

… Standard comma-, and newline-delimited data, and we must turn it into this:

ultricies  nibh       curabitur    vitae      blandit
vehicula   consequat  pharetra     lorem      iaculis
felis      consequat  euismod      at         nulla
sed        maecenas   orci         mi         sit
auctor     sit        sit          feugiat    amet
aenean     amet       amet         convallis  dolor
euismod    mauris     ullamcorper  ac         nec
semper     justo      mi           eget       est
quam       quis       tincidunt    dui        ornare
dapibus    porttitor  et           fusce      volutpat

Looking into the challenge, I was already thinking “tables” — Emacs tables. Transposition is a common operation in tables (spreadsheets) and mathematics, and Emacs can do both very well indeed.

So here’s what I did, utilizing our old friend org-mode; or rather, one of its subsidiary libraries. You don’t have to be in org-mode for this trick to work. It echoes an earlier VimGolf challenge I did where I used its hierarchical “sort” function to sort an address book: Fun with Vimgolf 1: Alphabetize the Directory.

Command Description
C-x h Mark the whole buffer
M-x org-table-convert-region Converts a region into an org table. The defaults use comma and new line separators by default. Bonus.
M-x org-table-transpose-table-at-point As the name implies this will transpose our table as required by the Vimgolf challenge.
C-x C-x Exchange point and mark.
M-x replace-regexp Replace: ^| \\||
With: RET

Replace the pipe at the beginning of the line and a whitespace, or any pipe.

M-x delete-trailing-whitespace Delete trailing whitespace. Could also be done with a more sophisticated regexp in the penultimate step.

Done right, and it should look like this:

ultricies nibh      curabitur   vitae     blandit
vehicula  consequat pharetra    lorem     iaculis
felis     consequat euismod     at        nulla
sed       maecenas  orci        mi        sit
auctor    sit       sit         feugiat   amet
aenean    amet      amet        convallis dolor
euismod   mauris    ullamcorper ac        nec
semper    justo     mi          eget      est
quam      quis      tincidunt   dui       ornare
dapibus   porttitor et          fusce     volutpat

Six “keystrokes” (for an arbitrarily large definition of “keystroke”…), but done the Emacs way. The best score on VimGolf is currently 31 (characters). Very impressive, but would it work on a 20×20 or with variable-length rows? “Think Abstract,” the developer cried.

The result above should be identical to that of the Vimgolf output, but done without the hyper-specialized and brittle solutions (most?) of the VimGolfers employ.

I didn’t tweak the cell width (whitespacing between each word) for each column; that it aligns perfectly with VimGolf’s output is dumb luck. Maybe they generated the resultant output in Emacs? :)

Nevertheless, the solution is “typical Emacs” and would scale well to very large datasets, and you don’t have to worry about things like unusually long cells; uneven number of rows and columns; etc.

Fun with Vimgolf 3: Swapping Words by Sorting

Jan 14 13
by mickey

Jon over at Irreal’s been busy with VimGolf challenges, and I figured I’d throw in my two pieces of eight.

The “challenge” is a simple. Take this text:

app.config['CHALLENGE_FOLDER'] = SOLUTIONS_FOLDER
app.config['SOLUTIONS_FOLDER'] = CHALLENGE_FOLDER

And turn it into

app.config['CHALLENGE_FOLDER'] = CHALLENGE_FOLDER
app.config['SOLUTIONS_FOLDER'] = SOLUTIONS_FOLDER

As you can see, a simple transposition between two words on a line is all that we need, and Jon’s come up with a solution that solves it in 8 keystrokes. Arguably something like his solution is what I would do were I to do it in real life.

I’m not sure if the Vim guys count typing out strings of text as one atomic operation or if they count each character; in Emacs, arguably, each character is in itself a command as each key stroke will invoke self-insert-command but it’s more fun to think of strings of text as a single keystroke, for simplicity’s sake, and to give ourselves a sporting chance against our Vim nemeses.

So here’s my solution. It only solves this particular challenge and nothing more. It relies on the good ole sort order — that C come before S.

Command Description
C-x h Mark whole buffer
M-x sort-regexp-fields Run sort-regexp-fields. See Sorting Text by Line, Field and Regexp in Emacs for more information
Regexp specifying records to sort: \\([a-z_]+\\)$ We want each record -- that's the part of the line we want Emacs to use for sorting -- to be the last word on each line
Regexp specifying key within record: \\1 The key -- that's the part inside the capturing group from above -- we want to sort by is the entire capturing group.

And we're done. So how does it work? Well, we rely on the side effect that the word CHALLENGE_FOLDER is less than, lexicographically, SOLUTIONS_FOLDER, because C comes before S.

It boils down to this: sort-regexp-fields is pure magic. As my article on the subject talks about at length, you can tell Emacs to only sort by parts of a line -- the part that matches the regular expression -- and using that match, you can then tell Emacs how you want to sort that data. We tell Emacs to sort by the last word on each line and leave the rest untouched. Simple :)

So how many keystrokes is that? Good question. I don't know: it depends on how you count it. Two if you count the commands only; four if you count the commands and the prompts; and many more if you count each character.

As always, these challenges are pointless (though fun!) but they do force you to think on your feet.

Jedi: A completion library for Python

Jan 10 13
by mickey

If you’re using Python with Emacs (using one of several competing, incompatible, and slightly different modes) you are used to a pretty… bare-bones experience: no completion; semi-functional dynamic docstring support; and little in the way of two-way communication between Python and Emacs.

Enter Jedi, a completion library. Yes, Jedi, an editor-agnostic library that publishes auto completion, docstring support, and more. Excellent.

I’ve experimented with Pymacs — an interesting science project that adds “Python-like” support Emacs, so you can avoid interacting with Elisp, except not really — rope, and ropemacs and they were… disappointing. Slow, crash-prone, obtuse and impossible to extend. So I never really used them, and lived without completion or, well, much of anything beyond the REPL and my own handcrafted modifications.

The other alternative is the 600 lbs gorilla, CEDET, and its incomplete Python support, but that’s no good either.

Imagine my surprise, after fidgeting with the dependencies for both Jedi and Jedi.el, the Emacs library for Jedi, that it… works! And it’s good! It’s up-and-coming, I should say, but perfectly usable; it doesn’t get in my way, it’s got some crazy deferreds library it depends on for asynchronous, non-blocking querying of Jedi, but that bit works great — no input lag at all.

It seems to resolve, simplistically (which is good), as many assignments and method calls as one can reasonably expect from a non-evaluating, statically analyzing Python completion library.

Functioning Auto Complete in a Python buffer

The Jedi.el module also Just Works with the excellent auto-complete library, as you can see in the picture above.

Aside from completion, it also offers “find symbol definition at point” (a la TAGS, but not crap) and Jedi.el sensibly binds it to C-. by default. It also has a “related names” functionality, tracking down same-named identifiers in other modules; it uses Anything (now Helm) to display the results, and it is bound to C-c r. And finally, it can show the documentation for the identifier at point (be it a class or function) with C-c d. Useful.

I haven’t used Jedi and Jedi.el long enough to really get to know it, but I’m probably going to extend Jedi.el so it uses eldoc-mode for displaying the function parameters; it’s also a bit rough around the edges, and I may want to tweak certain things to my liking, but overall: huge success!

I highly recommend you give Jedi and Jedi.el a try!

IEdit: Interactive, multi-occurrence editing in your buffer

Oct 2 12
by mickey

Have you ever heard of iedit for Emacs by Victor Ren (Update:Seems the github link I posted earlier was way out of date)? Me neither, until recently, and that’s a terrible shame as it is a cornerstone of my programming workflow now that I’ve learned about it. So what does it do, then? Well, quite simply, you can edit multiple, identical string occurrences at the same time. The twist here is it uses in-buffer editing without disrupting your workflow with prompts, windows or any of that stuff: you plonk your point down on a word you want to change; you run iedit-mode; and now all the occurrences of that word is highlighted on your screen, like isearch, and when you alter a highlighted word, the other highlighted words change also. How cool is that? Modern IDEs have it already — usually hidden away in the “Refactoring” section — and does exactly the same thing, but iedit is a lot dumber as it cannot infer context beyond I want to iedit all occurrences of word point is on.

If you regularly replace variables or words with M-% or C-M-% — well, you can retire that workflow now, as iedit will handle it for you. Sure, go ahead; use the older way if you have complex, partial replacements you want to do, but if you’re renaming a variable in a buffer… Why not use iedit?

Here’s a sample demonstration showing what exactly it is I’m talking about, in brilliant technicolor:

Emacs with IEdit active

Improving iedit

So iedit’s pretty great and all that, but I don’t replace words across a whole buffer very often; sure, I hear you say: “just narrow-to-defun with C-x n d!” Indeed, narrowing’s great, but this blog is all about half-baked, half-inventions and cobbled-together scripts, and this post is no exception!

I prefer a workflow that minimizes the use of commands to do routine tasks — a fairly common goal for most Emacs hackers. The code below aim to do just that: when invoked, it will take the word at point and only iedit occurrences in the local defun (Note: don’t forget that although defun is Lisp-speak, most modes automatically support commands like mark-defun or narrow-to-defun.) If you pass an argument to the function, it will iedit all occurrences in the entire buffer.

The iedit author suggest that you bind iedit-mode — the default command for entering iedit — to C-; and I agree: it’s rarely used and easy to type.

Update: Le Wang pointed out that I was using an older version of iedit; the code has been updated to reflect the API changes.

(require 'iedit)
 
 
(defun iedit-dwim (arg)
  "Starts iedit but uses \\[narrow-to-defun] to limit its scope."
  (interactive "P")
  (if arg
      (iedit-mode)
    (save-excursion
      (save-restriction
        (widen)
        ;; this function determines the scope of `iedit-start'.
        (if iedit-mode
            (iedit-done)
          ;; `current-word' can of course be replaced by other
          ;; functions.
          (narrow-to-defun)
          (iedit-start (current-word) (point-min) (point-max)))))))
 
 
 
(global-set-key (kbd "C-;") 'iedit-dwim)

Hiding and replacing modeline strings with clean-mode-line

Sep 10 12
by mickey

If you are like me, you are probably using one or more minor modes, and most of them add their own lighter to the modeline to tell you that they are running. Common examples include “Abbrev” for Abbrev mode; “yas” for Yasnippet mode and so on. Unfortunately, modeline real estate is expensive, and if you are stuck on a small screen — or if you split your Emacs windows a lot — you will find that it is often truncated, possibly hiding useful information that you actually care about, such as Which Function Mode.

There is no feature in Emacs to selectively hide or compress modeline flotsam, so I have hacked together some code to do the job for you.

For instance, in my Emacs I can collapse the following modeline string (Python Hi AC yas Eldoc Flymake:0/12 Abbrev) into (Py α υ Φ:0/12). I’ve chosen to completely obscure hi-lock-mode, abbrev-mode, and eldoc-mode but “compress” auto-complete-mode, yas/minor-mode, python-mode and flymake-mode.

I used Greek characters (How? C-u C-\ greek RET then C-\ to write Greek characters; see Diacritics in Emacs) because they look cool.

The code works rather well, and I have yet to experience any major issues with it, but do let me know if it breaks something! One important caveat is that a lot of minor modes that reach out and call other functions, like ERC or Flymake, often use functions in lieu of strings in mode-line-format and mode-line-modes, so replacing them with a string might break the minor mode, or cause it to not update properly! Please keep that in mind and experiment in a separate Emacs instance unless you know what you’re doing. Case in point: Flymake. I’ve replaced its own modeline updater function with one of my own with the “Flymake” string replaced; you could probably rejiggle it by advising the original function, then tweaking the modeline string post facto, blah, blah, blah — but I always opt for the simpler solution, which is what I’ve used below!

Here’s the code. Stick it in your emacs file, evaluate it, and you’re done. You can also force a refresh by running M-x clean-mode-line. Flymake will have to be turned off and back on for the changes to take effect in a buffer!

9:17pm update: Fixed a small bug in clean-mode-line !

(defvar mode-line-cleaner-alist
  `((auto-complete-mode . " α")
    (yas/minor-mode . " υ")
    (paredit-mode . " π")
    (eldoc-mode . "")
    (abbrev-mode . "")
    ;; Major modes
    (lisp-interaction-mode . "λ")
    (hi-lock-mode . "")
    (python-mode . "Py")
    (emacs-lisp-mode . "EL")
    (nxhtml-mode . "nx"))
  "Alist for `clean-mode-line'.
 
When you add a new element to the alist, keep in mind that you
must pass the correct minor/major mode symbol and a string you
want to use in the modeline *in lieu of* the original.")
 
 
(defun clean-mode-line ()
  (interactive)
  (loop for cleaner in mode-line-cleaner-alist
        do (let* ((mode (car cleaner))
                 (mode-str (cdr cleaner))
                 (old-mode-str (cdr (assq mode minor-mode-alist))))
             (when old-mode-str
                 (setcar old-mode-str mode-str))
               ;; major mode
             (when (eq mode major-mode)
               (setq mode-name mode-str)))))
 
 
(add-hook 'after-change-major-mode-hook 'clean-mode-line)
 
;;; alias the new `flymake-report-status-slim' to
;;; `flymake-report-status'
(defalias 'flymake-report-status 'flymake-report-status-slim)
(defun flymake-report-status-slim (e-w &optional status)
  "Show \"slim\" flymake status in mode line."
  (when e-w
    (setq flymake-mode-line-e-w e-w))
  (when status
    (setq flymake-mode-line-status status))
  (let* ((mode-line " Φ"))
    (when (> (length flymake-mode-line-e-w) 0)
      (setq mode-line (concat mode-line ":" flymake-mode-line-e-w)))
    (setq mode-line (concat mode-line flymake-mode-line-status))
    (setq flymake-mode-line mode-line)
    (force-mode-line-update)))

Working with Coding Systems and Unicode in Emacs

Aug 9 12
by mickey

Dealing with unicode in Emacs is a daily task for me. Unfortunately, I don’t have the luxury of sticking to just UTF-8 or iso-8859-1; my work involves a lot of fidgeting with a lot of coding systems local to particular regions, so I need a flexible editor that has the right defaults that will cover my most common use-cases. Unsurprisingly, Emacs is more than capable of fulfilling that role.

Emacs has facilities in place for changing the coding system for a variety of things, such as processes, buffers and files. You can also force Emacs to invoke a command with a certain coding system, a concept I will get to in a moment.

The most important change (for me, anyway) is to force Emacs to default to UTF-8. It’s practically a standard, at least in the West, as it is dominant on the Web; has a one-to-one mapping with ASCII; and is flexible enough to represent any unicode character, making it a world-readable format. But enough nattering about that. The biggest issue is convincing Emacs to treat files as UTF-8 by default, when no information in the file explicitly says it is.

I use the following code snippet to enforce UTF-8 as the default coding system for all files, comint processes and buffers. You’re free to replace utf-8 below with your own preferred coding system.

(prefer-coding-system 'utf-8)
(set-default-coding-systems 'utf-8)
(set-terminal-coding-system 'utf-8)
(set-keyboard-coding-system 'utf-8)
;; backwards compatibility as default-buffer-file-coding-system
;; is deprecated in 23.2.
(if (boundp 'buffer-file-coding-system)
    (setq-default buffer-file-coding-system 'utf-8)
  (setq default-buffer-file-coding-system 'utf-8))
 
;; Treat clipboard input as UTF-8 string first; compound text next, etc.
(setq x-select-request-type '(UTF8_STRING COMPOUND_TEXT TEXT STRING))

Once evaluated, Emacs will treat new files, buffers, processes, and so on as though they are UTF-8. Emacs will still use a different coding system if the file has a file-local variable like this -*- coding: euc-tw -*- near the top of the file. (See 48.2.4 Local Variables in Files in the Emacs manual.)

OK, so Emacs will default to UTF-8 for everything. That’s great, but not everything is in UTF-8; how do you deal with cases where it isn’t? How do you make an exception to the proverbial rule? Well, Emacs has got it covered. The command M-x universal-coding-system-argument, bound to the handy C-x RET c, takes as an argument the coding system you want to use, and a command to execute it with. That makes it possible to open files, shells or run Emacs commands as though you were using a different coding system. Very, very useful. This command is a must-have if you have to deal with stuff encoded in strange coding systems.

One problem with the universal coding system argument is that it only cares about Emacs’s settings, not those of your shell or system. That’s a problem, because tools like Python use the environment variable PYTHONIOENCODING to set the coding system for the Python interpreter.

I have written the following code that advises the universal-coding-system-argument function so it also, temporarily for just that command, sets a user-supplied list of environment variables to the coding system.

(defvar universal-coding-system-env-list '("PYTHONIOENCODING")
  "List of environment variables \\[universal-coding-system-argument] should set")
 
(defadvice universal-coding-system-argument (around provide-env-handler activate)
  "Augments \\[universal-coding-system-argument] so it also sets environment variables
 
Naively sets all environment variables specified in
`universal-coding-system-env-list' to the literal string
representation of the argument `coding-system'.
 
No guarantees are made that the environment variables set by this advice support
the same coding systems as Emacs."
  (let ((process-environment (copy-alist process-environment)))
    (dolist (extra-env universal-coding-system-env-list)
      (setenv extra-env (symbol-name (ad-get-arg 0))))
    ad-do-it))

Insert the code into your emacs file and evaluate it, and now Emacs will also set the environment variables listed in universal-coding-system-env-list. One important thing to keep in mind is that Python and Emacs do not share a one-to-one correspondence of coding systems. There will probably be instances where obscure coding systems exist in one and not the other, or that the spelling or punctuation differ; the mapping of such names is left as an exercise to the reader.

Compiling and running scripts in Emacs

May 29 12
by mickey

I’ve talked about running shells and executing shell commands in Emacs before, but that’s mostly used for ad hoc commands or spelunking; what if you want to compile or run your scripts, code or unit tests?

There’s a command for that…

Not surprisingly, Emacs has its own compilation feature, complete with an error message parser that adds syntax highlighting and “go to next/prev error” so you can walk up and down a traceback in Python, or jump to a syntax error, warning or hint in GCC.

There are two ways of using the compilation feature in Emacs: the easiest is invoking M-x compile followed by the compile command you want Emacs to run. So, for Python, you can type M-x compile RET python myfile.py RET and a new *compilation* buffer will appear with the output of the command. Emacs will parse the output and look for certain patterns — stored in the variables compilation-error-regexp-alist[-alist] — and then highlight the output in the compilation buffer with “hyperlinks” that jump to the file and line (and column, if the tool outputs it) where the error occurred.

One annoying thing about compile is its insistence on wanting to save every. single. unsaved buffer before continuing. It’s there to keep you from accidentally compiling a mix of newly saved and old, stale files, which would lead to unexpected behavior, and the only reason it’s still there — and why it insists on saving files unrelated to what you are compiling — is the complete lack of a proper, integrated project management suite in Emacs. But I’ll save that rant for later.

Anyway, if it offends you as much as it offends me, you can add this to your emacs file to shut it up:

;;; Shut up compile saves
(setq compilation-ask-about-save nil)
;;; Don't save *anything*
(setq compilation-save-buffers-predicate '(lambda () nil))

There is nothing stopping you from customizing compilation-save-buffers-predicate to take into account the files or directories so its save mechanism is cleverer, but I personally never bothered.

If compilation is successful — and like most things in the UNIX world, this is governed by the exit code — the modeline and compilation buffer will say so; likewise, if an error occurred, this is also displayed.

You can jump to the next or previous error ([next/previous]-error) with M-g M-n and M-g M-p — they’re bound to more keys, but I think those are the easiest ones to use. Similarly, in the compilation buffer itself (only), you can go to the next/previous file (compilation-[next/previous]-file) with M-g M-} and M-g M-{, respectively, and RET jumps to the location of the error point is on.

There is an undocumented convention in Emacs that commands like dired, grep, and compile can be rerun, reverted or redisplayed by typing g in the buffer.

By default the compilation buffer is just a dumb display and you cannot communicate with the background process. If you pass the universal argument (C-u) you can; the buffer is switched to comint mode, and you can converse with the process as though you were running it in the shell.

Python Debugging with Compile

I work mostly in Python, and quite often I have to debug the script I am writing with pdb, the Python debugger. I use breakpoints a lot, and I have a command bound to F5 that imports pdb and calls pdb.set_trace() — a standard Python debugging idiom. What it also does is highlight the line in bright red so I don’t miss it. And if I call compile from a Python buffer, it checks if there is a breakpoint present in the file: if there is, it switches to the interactive comint mode automagically; if there isn’t, it defaults to the “dumb” display mode. Having it switch automagically is perhaps a slightly pointless flourish, but screw it I’m using Emacs, not vim :)

This is the code I use. Feel free to adapt it for other languages. Send me an e-mail if you do something neat with it.

(require 'python)
(defun python--add-debug-highlight ()
  "Adds a highlighter for use by `python--pdb-breakpoint-string'"
  (highlight-lines-matching-regexp "## DEBUG ##\\s-*$" 'hi-red-b))
 
(add-hook 'python-mode-hook 'python--add-debug-highlight)
 
(defvar python--pdb-breakpoint-string "import pdb; pdb.set_trace() ## DEBUG ##"
  "Python breakpoint string used by `python-insert-breakpoint'")
 
(defun python-insert-breakpoint ()
  "Inserts a python breakpoint using `pdb'"
  (interactive)
  (back-to-indentation)
  ;; this preserves the correct indentation in case the line above
  ;; point is a nested block
  (split-line)
  (insert python--pdb-breakpoint-string))
(define-key python-mode-map (kbd "<f5>") 'python-insert-breakpoint)
 
(defadvice compile (before ad-compile-smart activate)
  "Advises `compile' so it sets the argument COMINT to t
if breakpoints are present in `python-mode' files"
  (when (derived-mode-p major-mode 'python-mode)
    (save-excursion
      (save-match-data
        (goto-char (point-min))
        (if (re-search-forward (concat "^\\s-*" python--pdb-breakpoint-string "$")
                               (point-max) t)
            ;; set COMINT argument to `t'.
            (ad-set-arg 1 t))))))

There’s a Minor Mode for that…

The compile workflow isn’t for everyone; some people want everything in one place: their shell. Well, good news then — you can have your cake and eat it. The minor mode M-x compilation-shell-minor-mode is designed for comint buffers of all sizes and works well with M-x shell and most modes that use comint. You get the same benefits offered by compile without altering your workflow. If you compile or run interpreted stuff in your Emacs shell you’ll feel like a modern-day Prometheus with this minor mode enabled!

Add this to your emacs file and the compilation minor mode will start when shell does.

(add-hook 'shell-mode-hook 'compilation-shell-minor-mode)

Fun with Emacs Calc

Apr 25 12
by mickey

The Challenge

Jon over at Irreal’s Emacs blog posted an interesting solution to a challenge raised by Xah Lee:

How do you convert a string like this 37°26′36.42″N 06°15′14.28″W into a decimal answer like this 37.44345 -6.25396.

First off, I’m not sure Xah’s example answer is entirely correct; my understanding of latitude and longitude is limited to what I can google, and if I type the original degrees, minutes and seconds into this tool by the U.S. FCC it returns 37.44345 6.25396.

Anyway, on with the challenge. Jon’s solution is very interesting, but it got me thinking: surely Emacs has the facility in place to do this already? It’s Emacs, right? Right.

Fun with Emacs Calc

Emacs is equipped with a really, really awesome and spiffy RPN calculator (HP calculator fans, rejoice) capable of manifold things like algebraic and symbolic manipulation; matrix manipulation; conversion routines and much, much more. It’s truly wonderful but really complex, but it does come with a really nice Info manual (type C-h i g (calc) and check it out.) It’s a shame so few people know about its potential, as it’s basically a much simpler version of Mathematica, or even Wolfram Alpha (arguably you’ll have as much trouble telling Calc what you want as you would Wolfram Alpha…)

Anyway, I figured the Emacs calculator would have a facility in place for converting Deg-Min-Sec to decimal form, and sure enough, it does.

To try it out, type C-x * * and the calculator will open. Two windows will appear: the calculator mode and the trail containing a trail — a history — of commands and actions. The first thing we need to do is switch the calculator to “HMS” mode so we can try it out. To do this, type m h in the mode (the left) window and the modeline will change and say something like Calc: 12 Hms. The 12 is the floating point precision.

Next, type in the expression, replacing the unicode symbols above with @ for degrees; ' for minutes; and " for seconds. If you typed it in correctly, it will appear in the calculator window.

All we have to do now is convert it. Calc can convert between a wide range of units and systems, but we only care about decimals. Type c d and Calc will convert it to a decimal number. If you entered 37@26'36.42" you should see 37.44345 appear in its place.

OK, so we know it can do it, but how do we weaponize it? It so happens that Calc comes with a neat, little (though underdocumented) command called calc-eval.

Entering IELM, M-x ielm, we can query the calculator in real time:

ELISP> (calc-eval "1+2")
"3"

Yep. It does work. The hardest part about using it is mapping the “algebraic” notation used above with the indirect, keybinding-based input you use in the RPN calculator. Thankfully, the manual and (often) the trail will tell you the name of the function you are calling.

Let’s digress a little so I can show you how neat this calculator actually is: solving the elementary equation 2x+5=10. In the calculator, type m a to go to algebraic mode; next, type (2x + 5 = 10) — don’t forget the brackets — and it should appear as an equation. Finally, type a S to “solve for” a variable — and when prompted, answer x. The answer will appear in your calculator window. How awesome is that?

Back to the challenge. Calling the “convert to degrees” function is what we need to do, and the answer is hidden in the trail — it’s called deg.

Putting it all together, and we get:

ELISP> (calc-eval "deg(37@ 26' 36.42\")")
"37.44345"

That looks right. But the original challenge said that we had to take a string, like the one given above, and map that. So here’s my solution:

(defun hms-to-dec (hms-str)
  (let ((hms (split-string hms-str "[°′″NW ]" t)))
    (flet ((to-deg ()
                   (string-to-number
                    (calc-eval (format "deg(%s@ %s' %s\")"
                                       (pop hms) (pop hms) (pop hms))))))
      (list (to-deg) (to-deg)))))

Calling it from IELM yields the following answer:

ELISP> (hms-to-dec "37°26′36.42″N 06°15′14.28″W")
(37.44345 6.25396666667)

Looks good to me. Job done, and a fun challenge.