About Articles Projects Links Apps Feed

Eshell as a main shell

This article was initially posted on Reddit. I’ve revised some arguments by taking the community feedback into account.

Since its inception, Eshell has suffered from an arguably low popularity. See what the community says:

Hell, even the official manual gives a fair warning against its use:

I’m not arguing that those arguments are not valid. (To most extent, they are.)

I want to argue that the issue is not so much that Eshell is not a proper Unix shell (in the sense of POSIX sh and derivatives), but rather perhaps we do not need a Unix shell and all its historical limitations.

In other words: the non-Unix-ness of Eshell is more of a step forward than a hindrance. Let’s review the situation.

“State of the art”

Take this typical scenario: I’ve got a long running process which finally outputs pages of relevant information. Only then I realize that it would have been smart saving that somewhere. Now I want to copy it to the clipboard. How do I do that?

By clumsily mouse-selecting pages of terminal output, assuming my terminal can handle it.

Why can’t I (fuzzy-)search the shell output?

Why can’t I browse the different prompts of my shell session?

Why can’t I copy/paste without a mouse?

Back in the days, VT-like terminals were our main mean of communicating with a machine. Decades went by, our desktop computers can now handle gigabytes of buffering and display in 24-bit colors, and yet we still stick to terminal emulators, that is, programs that emulate the restrictions of those ancient machines.

  • Cannot move the cursor around.
  • Limited colors.
  • Terminal capabilities are neither simple nor portable. See termcap(5).
  • Formatting (e.g. colors) needs ugly escape codes that are not even portable.
  • More hacks than computer science history would have ever hoped for.

Say I run this command:

$ cat file-list.txt

Now I want to see what’s in foo. How do I do that?

  1. Either I mouse-select the file, hoping I’ll not be off-by-one on one side. Then I paste to the prompt.
  2. Or I type the path myself, with a bit of luck completion will bring me there soon enough.
  3. Or I resort to re-call the command from a subcommand and program some output manipulation to extract the filename and then open it with my editor. This is not nearly universal nor even convenient.

All this is terribly backwards and it is high time we moved on.

Terminals vs. shells

It’s important to understand that shells are not (or should not be) semantically bound to terminal emulator restrictions. Shells are a textual interface to the machine. They just need input, evaluation, execution, output. GTK/Qt/Tk/etc. are absolutely capable of handling such a task. Not mentioning Emacs…

So why do shells de facto need a terminal? One thing: curses-like interfaces. Those libraries need terminal capabilities to render. Do we still really need that?

  • It’s most limited in terms of user experience. Mouse selection simply won’t do.
  • It’s not beautiful.
  • It cannot render anything beside text. (Without clumsy hacks at least.)
  • “It works on low-end systems”: there is not much hardware out there that cannot take Wayland or X. Anyways, my case is about desktop computers.
  • It’s not even fast: in fact terminals can be pretty slow at rendering. (Emulators also emulate the baud-rate, etc.)
  • Since it’s character-based, it cannot render fonts of different sizes nor, say, thin lines.
  • “Efficiency on remote systems”: that’s almost a fair statement. But then… TRAMP! ;)

Enter Eshell

Back to my initial example of the long-running process: want to copy that output somewhere? 3 key-strokes with Eshell. And better: could be even 1 if you want to program it your way. The interface is extensible. (Terminals usually aren’t, URxvt and its Perl scripts are not a proud example of extensibility.)

Eshell breaks up with the “terminal-interface” paradigm: the user interface of the Eshell commandline-interface is Emacs itself. The pager is the Emacs buffer. The input is Emacs keystrokes. The extension language is Elisp. Consequences:

  • No need for pagers like less. You won’t ever re-run a long-output command by appending | less to it.
  • Little need for output ad-hoc minipulations (e.g. grep trial-and-error queries--grep is not a search tool): output the result to an Emacs buffer, use some Lisp functions, use Evil ex commands, iedit, swiper, helm-moccur or multiple-cursors…
  • Eshell supports TRAMP! Which means you don’t have to put aside your powerful environment when switching to root or connecting to a remote host: all the power of your Emacs can be used anywhere, the shell included.
  • Find file/URL at point. Or list them all with Ivy/Helm and fuzzy-search them!
  • No need for fzf. “Hey! fzf is awesome!” I know, but even fzf cannot compete with Emacs.
  • At last, done with the clumsy Bash and sh-inspired shell languages! No more nasty word-splitting, no more unexpected expansion. You don’t even need calc or bc when you have Elisp or Emacs’ calc at hand.
  • No need for half-implemented vi-bindings when you can have Evil to edit your command-line. (Some shells like fish can call an editor to edit the commandline. Well, with Eshell it’s one step less.)
  • You can even edit the output directly!
  • You can redirect output to several buffers(!), they don’t even need to be existing files.
  • Eshell has loads of fancy features for globbing, filtering, and editing.
  • Programmable completion with pcomplete, which is arguably easier and yet more powerful than its Bash/Zsh/fish counterparts.

Finally, some Helm niceties:

  • helm-find-files has M-e to switch to Eshell in the currently browsed folder. Bye-bye cd, hello fuzzy-browsing!
  • The previous technique makes it very convenient to go up the hierarchy to some parent folder (C-l by default). Seriously, farewell, annoying little cd ...
  • Still with helm-find-files, M-p will trigger the “cd history”, so you can fuzzy-find the history of your shell paths. Make it persistent with (add-to-list 'desktop-globals-to-save 'helm-ff-history)).
  • Use Helm as a menu for completion. Awesome for completing big file names or packages!
  • helm-find-files has C-c i (and C-u C-c i, etc.) to insert the selected relative/absolute file names at point, that is, in the commandline. This sometimes proves more convenient than file completion.

In terms of extensibility, it’s key to understand that (as of Emacs 26) all prompts are properly identified by Emacs. This opens the door to infinite programmability of the shell interface:

  • Browse all prompts with C-c C-n=/=C-c C-p. (Or simpler bindings…)
  • Copy the output of a command with one keystroke.
  • Collect all the prompts in an Ivy/Helm list.
  • Use hs-minor-mode to fold the outputs.

There is more to come

It’s undeniable however that, as of Emacs 25.2, Eshell has numerous rough edges. Quite a few bug reports have been sent on debbugs and it’s getting better.

Native completion is limited considering very little effort was ever made. The community can help changing that. In the meantime, I’ve implemented a fallback to Bash/fish completion so that I can keep completing just as much as I could with Bash or fish.

Need that curses-based program? Get a (much more) powerful Emacs alternative:

  • git (CLI), gitk, tig, etc. -> magit
  • htop -> symon, proced, helm-top, etc.
  • abook -> org-contacts
  • mutt -> mu4e, gnus, notmuch
  • ncdu -> dired-du
  • cmus/moc -> EMMS
  • newsbeuter -> Elfeed, gnus
  • weechat, irssi -> ERC, etc.
  • rtorrent, transmission-cli -> transmission.el

As opposed to their curses-like counterparts, Emacs alternatives share all those features:

  • Configure/extend in Lisp.
  • (Fuzzy-)search the buffer. Copy anything to the clipboard.
  • No need to change the color theme, it’s already using your Emacs theme. No need for dircolors, lesspipe, Xresources…
  • Emacs bindings (Evil?). All configurable. No more painful context-switch between Emacs-based and vi-based bindings.

In the meantime, there might still be that annoying curses-based program you cannot replace. Eshell allows you to run those programs by transparently switching to a *term* buffer. See eshell-visual-commands.

Curses-based programs are not the only ones that won’t run well on Eshell: REPL are known for working poorly too. REPLs are poorly designed though: from a Unix-philosophy standpoint, only the E is required for every language, while re-implementing the R, P, and L, i.e. the user interface, for each one of them is simply re-inventing the wheel. Instead of that, Emacs has comint-mode, an REPL-lie user interface that can be plugged to any language. With all the powerful features of Emacs that all those specialized REPLs are lacking. For a start, you can write code in Emacs and execute any piece of code from a single keystroke. It’s configurable and extensible in a consistent manner across all languages.

Similarly, “environment setups”, as commonly used by programming language tools such as Python’s virtualenv or Ruby rvm, expect the user to be running a specific shell. This is broken by design. Let me quote u/rpdillon as I could not have said better:

To OP’s point, though, yes, these environments do make assumptions that you’ll be using some kind of bash-compatible shell, which tells you a little bit about how sub-optimal the state-of-the-art is. I suppose the only alternative I can think of is to have the language itself manage many versions of itself so that every developer’s environment wouldn’t have to be customized/extended to negotiate the maze.

The Go programming language for instance does not have such a tool as I know of.

A long-standing critique of Eshell is that it does not support input redirection. This is hardly a major hindrance since you can always use a work-around such as

cat file | command

Another common argument against Eshell is that it cannot run the widely popular Tmux. The latter is used for many things, out of which:

  • Window management, tabs, etc.: you don’t need that if you have Emacs.
  • Leave long-running processes in the background after the user has logged out: this feature is easily provided with nohup or dtach.

All that being said, should Eshell not do the job for you, then maybe give M-x shell a try. It has a similar design: it is a shell emancipated from any terminal; instead of being Lisp based, it runs an existing shell as an interpreter. Bash and zsh are known to work. It has all the advantages an Emacs buffer has over terminal emulators. The main difference then lies in the language: Emacs Lisp vs. Bash and related languages.

An open door to Emacs-Everywhere

Switching to Eshell marked a milestone for me: from then on I dropped all my crufty curses-based programs and switched to much more powerful Emacs alternatives. I now use Emacs everywhere to the point that it even is my window manager. Having a consistent environment well glued together really empowers you.

Ironically, the environment is so powerful that I ended up not using any shell so much anymore, only for a few commands here and there. For which I use Eshell, obviously.

Eshell might not be perfect (yet), a little beta perhaps, but this can only be helped by a growing community: the more people join, the more feedback developers get, the better it will become. Hopefully this pamphlet will help raise your curiosity, if not more esteem for this brilliant project.


Date: 2017-09-04 (Last update: 2018-08-11)

Made with Emacs 27.2 (Org mode 9.4.4)

Creative Commons License