Skip to content

Running Shells in Emacs: An Overview

by mickey on November 1st, 2010

To use Emacs effectively, you must learn to use all that Emacs has to offer. One of Emacs’s strongest selling-points is its shell integration and terminal emulation. If you come from other editors or IDEs you’re probably using an external terminal window (like xterm), a terminal multiplexer (like GNU Screen) or console window (in Windows) and switching back and forth. But there’s a better way…


If you use Emacs as your shell you will have all the functionality that Emacs provides plus you get almost all of the advantages provided by the shell itself, but built into Emacs. This article is only a quick overview of all the shells: I plan on tackling each one, in-depth, later on.

All external, interactive shells in Emacs are derived from something called comint-mode. comint-mode introduces low-level functions for dealing with interactive shells that require user input, a history ring, and so on. Shells like M-x shell and M-x python-shell inherit from comint mode.

Emacs provides you with three shell ways of interacting with a shell (like bash or zsh), and these shells are often called inferior shells. Confusingly, there is a shell (called "shell" or "shell mode") which, by the nature of how it works, is also an inferior shell. Yeah, confusing…

The Inferior Shell ("Shell Mode")

The inferior shell is what you get when you run M-x shell. It is a wrapper around your default shell. It is governed by explicit-shell-file-name, the ESHELL environment variable or shell-file-name, in that order.

NOTE: If you change the default shell, you must create a new variable called explicit-<filename>-args where filename is obviously the filename of the new shell — and the variable istself should contain a list of arguments. If you’re on Windows, make sure you include the extension!

Here’s a quick example of how to get bash working in Windows:

The example above is just a quick-and-dirty example. There’s a better, and more complete, way of getting Cygwin bash to work in Windows.

All shell does is redirect the input and output to Emacs, and that means interactive console programs (like top, ssh, midnight commander, etc.) won’t work properly.

This limitation can be a problem as, traditionally, some UNIX apps behave differently (by suppressing the prompt for instance) if they detect their input and output is redirected. To get around this, some programs will let you force it to run in interactive mode.

If you’re on Windows it is the simplest thing to use out-of-the-box, as, even without interactive support, it is vastly superior to the standard command prompt. In fact, it is my preferred shell as I do a lot of Windows development and need a “native” shell to ensure my stuff works correctly on machines that may not have Cygwin set up.

I replace the redundant “help” binding on f1 with shell:

Terminal Emulator

The Emacs terminal emulator is exactly that; it emulates the VT100-style ANSI escape codes, just like xterm or rxvt. Emacs’s emulator isn’t complete or perfect, so some interactive programs won’t work properly, but most things like top and ssh will. To run the terminal emulator, type M-x term or M-x ansi-term (for full ANSI support). I recommend the latter.

Due to the way terminal emulators work, most of Emacs’s common keybindings are reserved for the subshell itself; to get around this you can switch between line mode and char mode. In line mode the terminal buffer will behave much like shell mode and normal Emacs buffers. In char mode each character is sent directly to the underlying inferior shell without preprocessing by Emacs.

To switch between the two modes type C-c C-j to switch to line mode, and C-c C-k to switch to char mode. To save you from having to switch between modes for one-off commands you can use the alias C-c char which translates into C-x char.

MS Windows: The terminal emulator won’t work on Windows out of the box.

To summarize, Emacs’s Terminal Emulator is a more than adequate emulator and it is flexible and feature-rich enough to run even complex interactive programs like vim!

The Emacs Shell

Emacs comes with its own shell (as in, like bash or zsh) written in entirely in Emacs-Lisp. The shell, named eshell, is a feature-rich replacement for your standard-fare shells like bash with the added bonus of working on any platform Emacs runs on. This is especially useful if you’re on Windows but crave a Linux-like environment. It’s heavily inspired by the likes of bash, and has gone to some lengths to keep the syntax familiar. To invoke it, run M-x eshell.

As the shell is written entirely in elisp, that means you can also extend it with elisp; and the shell has provisions for inlining elisp and using the results as part of your commands. EShell is clever enough to redirect certain commands that Emacs itself offers, such as man, grep and so on. Very useful.

Despite the many advantages Eshell brings to the table, it is not a shoe-in replacement for a terminal emulator running bash.

EShell is a great shell but woefully underdocumented; therefore, I’ve written a guide to mastering Eshell.

Conclusion

Three excellent shell choices, all with their own advantages and disadvantages. If you’re looking for a faithful emulator, then ansi-term is the choice for you; if you are more interested in a dumb terminal that behaves like an Emacs buffer then use shell; if you want something fancier that you can tweak and customize like Emacs itself, eshell may be the right shell for you.

Whatever the case, the shells above should eliminate almost all external terminal emulators for all but the most complex interactive programs out there.

38 Comments
  1. Indeed, EShell is not documented well. Most of all I miss some kind of best practices review. EShell has a funny way of using SSH, but I could not find a way to use GNU Screen. Perhaps, EShell might use something like dtach' or retty’ to imitate the behavior of Screen? Could you, please, cover the questions of terminal multiplexing and remote execution from EShell in your next articles?

    Thank you.

  2. mickey permalink

    Dmitri,

    Thanks for your feedback. I plan on tackling EShell next. It is indeed a complex beast, and there’s a lot of facets I want to cover… I might even have the split the article into two posts.

    -Mickey.

    • Win Smith permalink

      I took eshell for a test drive and soon ran into this
      problem:

      $ /usr/bin/firefox -remote ‘openurl(http://google.com,new-window)’
      Error: Failed to send command: 500 command not parseable

      (The command works as expected under bash/gnome-terminal.)

      Also, sending interrupts to programs didn’t behave
      as expected.

      If you could comment on these problems it would
      be great!

      • mickey permalink

        The quotes have been replaced by smart quotes (try enclosing your commands in pre tags) but when i run it with the smart quotes replaced with single quotes, firefox correctly opens a new window and opens google.com.

        Not sure what your problems were regarding interrupts either: what happened and what did you expect to happen?

        But don’t forget that Eshell is not a terminal emulator: it’s a completely different implementation.

        • Win Smith permalink

          It was single quotes among other possibilities that I tried for escaping the parenthesis. Sorry about the bad formating.

          After some more googling, I’m less sure it’s a problem with with escaping the parenthesis. It might have to do with a difference in the environment, e.g., the value of $MOZ_NO_REMOTE or something. Naturally I tried ‘export MOZ_NO_REMOTE=1′, but to no avail.

          As for the interrupt issue, I can’t reliably reproduce it, but somehow I end up with a process backgrounded but still sending output and no way I can find to foreground it again. (I have the eshell prompt at that point.)

          • mickey permalink

            Hmm. It’s not the parser, as you discovered yourself. I ran this:

            So, listify will parse a commandline parameter string and give it to you as an lisp list, and as you can see it correctly said it’s escaped. The “500 command not parseable” might be the HTTP error code “Internal server error”.

            The way Emacs generally handles inferior processes can be a bit messy: sometimes things just run askew, and EShell (given its complex way of asynchronously launching the processes and filtering their output to EShell) could easily trip up and screw up.

      • Win Smith permalink

        A fix for my problem (running ‘firefox …’ from eshell) turns out to be to invoke emacs23-x from the command line of a terminal or configure the emacs icon to use a controlling terminal.

        I don’t know exactly why this works, but there is a similar situation with Tcl described here.

  3. Oscar permalink

    Woah, just found this blog. Great work! I hope you keep it going like this, it’s really nice :)

    And btw, I didn’t know there was a difference between M-x shell and M-x (ansi-)term, and using ansi-term seems to fix the issues I’ve had with M-x shell. Thank you for that!

    If I could come with a recommendation for a subject to cover in a future blog entry, that would be registers. Once you understand them and why they’re good to know, you can’t stop using them. Especially when doing macros. Or storing window configurations. Etc :)

    • mickey permalink

      Hi Oscar,

      Thanks for the kind words. I have a laundry list of stuff I want to talk about, including registers and bookmarks, but I just have to find the time to write the posts. I’ll see if I can’t write about them in the coming weeks :)

      -Mickey.

  4. david permalink

    Just found this well written blog – I’m looking forward to future posts!

    I’ve a question concerning running programs/scripts in shell or eshell in stead of cmd (Windows). The (Ruby) script asks for input, but the text that asks for the input is not displayed. I read somewhere that this is because the program – not Emacs – buffers the text. How can this buffer be flushed? When running the same script in cmd, there’s no problem. How can the same behavior be realized in Emacs?

    • mickey permalink

      Well, it could be buffering, or it could be that the Ruby script won’t work properly unless it’s running in interactive mode. I’m not sure how you would force the script to flush its own buffer to stdout/err, but you can probably pass the Ruby interpreter a parameter forcing it to run in interactive mode. This is needed with Python and other interactive commandline programs as they, like the problem you experience, won’t show the prompt without.

      So in Python you’d type python -i and it would launch in shell (or eshell).

    • rgiar permalink

      try –inf-ruby-mode which is custom-made for emacs

  5. I recommend turning on color in shell-mode with ansi-color:

    (add-hook ‘shell-mode-hook ‘ansi-color-for-comint-mode-on)
    ; thanks to http://www.emacswiki.org/emacs/AnsiColor

    • mickey permalink

      Good idea. It’s enabled by default in Emacs 23.2, though.

  6. Cool article, thanks! I’ve been using M-x shell but after reading this I took M-x ansi-term for a spin. I wish M-x shell formatted its “ls” output as well as M-x ansi-term does!

  7. John permalink

    I don’t know where I would be today if it weren’t for Emacs shell mode, especially when combined with Gnu Screen.

    I’ll typically start a Screen window with Emacs, and keep that running for months (in some cases years). The buffer ends up containing hundreds of thousands of lines of commands and their results, and becomes a kind of journal of my activity. Searching through the buffer for commands, output, etc is simple. When I want to notate something I know I’ll want to find in the future, I add a comment to the end of the command (for example: bash$ ./runSomeObscureCommand.pl foo foo foo # MELTDOWN COMMAND). Plus this buffer can be saved for future reference.

    Another use has been for trying snips of code when writing Perl scripts. I’ll start a shell buffer, rename it *perl* (so that I can create another *shell* buffer for normal shell use), then run: perl -de 0. Now I can execute small pieces of Perl code, test out things quicker than I can look them up, try out regular expressions, etc. Again, I have a completely searchable log of all my trial-and-error efforts.

  8. Glen permalink

    I’m just a novice, I don’t even know the real shell all that well….which one do you recommend i start with?

    • mickey permalink

      I’d probably use ansi-term as it is the closest approximation to a real terminal emulator.

      • Glen permalink

        ok, great i’ll try it. how do i get it to start an ansi-terminal and execute a couple of commands when I click on emacs. is that something i would place in .emacs?

      • Glen permalink

        I tried putting the following in my .emacs file, which worked

        (ansi-term “/bin/bash”)

        but I want to start in a default directory and run a shell script, so i tried adding the bash option for this,

        (ansi-term “bin/bash –init-file ~/start.sh”

        but this returned an error.

        • Glen permalink

          ok, i figured out how to do this. I created an executable shell script that looks like this:

          #!/bin/bash
          bash –init-file ~/statupscript.sh

          Then I put all the startup commands i want in startupscript.sh, another shell script. don’t know if this is
          ‘propper’, but it works!

  9. I’m still mainly running with a bunch of terminals under GNU Screen running urxvt-unicode (a very lightweight terminal app). I then run a demonised Emacs on-top in full-screen mode (and the occasional in-screen terminal mode display).

    I think the main reason I still do this is because it’s still quicker to Alt-Tab Screen-Esc than C-x-b and search for the terminal I want. I’ve been experimenting with alternative buffer switchers like ido and lusty which do help with code but given the number of buffers my Emacs can have sometime it can look like quite a list.

    The other reason is I’m often running remote screen sessions which is very useful when connections die. Although emacs is pretty stable it does sometimes lock up on me, certainly more often than screen has let me down.

    Perhaps you could also talk about remote terminals (and the interaction between remote tramp and compile-mode) in a future post? I shall certainly try to use in-emacs term-mode more often when I want to do quick things in a shell.

    • mickey permalink

      Alex,

      Have you tried binding your terminal buffers in emacs to different buttons? If you enable Hyper/Super (with xmodmap) you have a full set of keybindings available to you, such as H-0 through to H-9. Here’s a random snippet of code to demonstrate what I’m talking about:

      Alternatively you could give “elscreen” a shot – it brings screen to Emacs (by using Emacs’ window configurations.)

      And I do plan on tackling remote editing/compilation with TRAMP soon-ish. Next article will focus on EShell though!

      -Mickey.

  10. Karthik permalink

    Are you going to do a piece on running multiple *shell* buffers at a time?

    • mickey permalink

      Karthik,

      That can easily be done by either using M-x rename-buffer on the shell buffer or C-u when you invoke one of the shell commands.

    • jaybee permalink

      Have a look at http://www.enigmacurry.com/2008/12/26/emacs-ansi-term-tricks/ which has a neat way of setting things up so the same command either starts a new shell, makes an existing shell buffer active, or renames a shell buffer (prerequisite to starting a new one).

  11. The Sunrise Commander (a dual-pane file manager for GNU Emacs based on Dired) provides convenient integration with term-mode and eshell and makes quite easy to work with multiple terminals in Emacs.

    http://www.emacswiki.org/emacs/Sunrise_Commander

    Cheers,

    José A. Romero L.
    escherdragon at gmail
    “We who cut mere stones must always be envisioning cathedrals.”
    (Quarry worker’s creed)

  12. chuck permalink

    Thanks for turning me on to M-x ansi-term!

    I could use some suggestions, though… right now I usually have both emacs and Terminal open on my Mac, and the Terminal window usually has a couple tabs open, most of which have ssh sessions to different servers. I occasionally will run emacs in one of those ssh sessions to do some quick editing; I’ve noticed that when I do, (1) my alt/option key doesn’t work as Meta, so I have to use Esc instead, and (2) I don’t get any colors, just all green. Often though, I have much more involved editing to do on files on those servers, so I’ll open them in my local emacs using TRAMP. But then I’m usually going back to my Terminal do issue shell commands on the server. It seems like ansi-term could help a lot here, but the first thing I’d be doing in the ansi-term buffer would be sshing to a server; once I do, I’m finding that some things don’t work, for example if I try to run ‘top’ I get the error “‘eterm-color’: unknown terminal type.”

    I’m not even completely sure which would be better to fix — get emacs on the servers set up nicer, or get it working better to run ssh sessions to the servers within M-x ansi-term on my local machine’s emacs.

    • You can fix the “‘eterm-color’: unknown terminal type.” error, by copying “eterm-color” and “eterm-color.ti” from “/usr/share/emacs/23.2/etc/e” into “~/.terminfo/e/” on the remote machine.

  13. Jeff Maner permalink

    Any idea why doskey won’t work in M-x shell? I can call doskey, I can assign doskey macros, but the macros aren’t recognized on invocation. For example, doskey ls=dir $* successfully defines ls as dir, but ls is subsequently not recognized.

    • mickey permalink

      doskey? I thought they deprecated that thing 15 years ago. Assuming we’re talking about the ancient tool for MS-dos that added history and aliasing (as your post would imply) then I’d imagine it is because it hooks — or tries to — some low-level commands in the command console that a simple stdin/out redirection wouldn’t trigger.

      • Jeff Maner permalink

        LOL! Didn’t know it was deprecated 15 years ago, but that’s what I’m talking about. Your response makes good sense. I’ve relied on doskey for years to make cmd.exe understand ls and other commands I prefer to Windows’ native commands. I’m having trouble getting them all working as EShell aliases, so I thought I’d check into using shell instead. Guess it’s back to EShell. Thanks for this post and your great Guide to Mastering EShell!

  14. Hamid permalink

    Hi, Thanks for the article.

    I am using ansi-term, however the tab completion is sadly not working. Is there a way to fix this?

    Thanks,

    H

Trackbacks & Pingbacks

  1. Tweets that mention Running Shells in Emacs: An Overview | Mastering Emacs -- Topsy.com
  2. Jeff Barr’s Blog » Links for Sunday, November 14, 2010
  3. Executing Shell Commands in Emacs | Mastering Emacs
  4. Compiling and running scripts in Emacs | Mastering Emacs
  5. Open Emacs dired buffer from dired | fortune datko

Leave a Reply

Note: XHTML is allowed. Your email address will never be published.

Subscribe to this comment feed via RSS