Eshell versus M-x shell

The benefit of using Bash is that it’s the de facto standard for sharing shell commands on the Internet. As such, M-x shell can run any of those shell snippets out there without extra modification.

Eshell constantly requires modifications to the syntax, for instance turning $(...) to ${...} or the input redirection < to a pipe.

You might complain that Bash is a terrible language and Eshell’s Elisp is vastly superior. But even at the language level, M-x shell beats Eshell since it lets the user choose which underlying program to run (with C-u M-x shell). So the same mode can run psql, bash, or rash if you fancy a Racket-powered shell. And Racket is a much more interesting language than Eshell in my opinion :)

virtualenv and other environment managers don’t work well with Eshell. Maybe it would be possible to leverage the direnv Emacs package, but I haven’t looked into it yet.

I’ve heard that nix-env has support for Eshell, so I suppose it would be equally possible to add Eshell support to guix environment.

Until then, M-x shell has better support for environment managers than Eshell.

Eshell only offers poor completion by default. M-x shell can do slightly better if the underlying shell is Bash, but the latter does not always provide great completion.

It’s possible to greatly enhance completion with support from the Fish shell. The fish-completion package supports both M-x shell and Eshell.

You can even display the Fish inline documentation with the helm-fish-completion package.

M-x shell is significantly faster than Eshell at processing long outputs. To the point that performance is rarely an issue (never was for me at least). In Eshell, it is not uncommon to see Emacs grind to a halt because of too long an output.

M-x shell is still an order of magnitude slower than, say, emacs-vterm. The performance issue is significantly reduced when font-lock-mode is off. There may be a few other tricks to boost M-x shell performance.

In Eshell, I implemented a special prompt that reports the duration of the previous command. This is very useful because I don’t have to think beforehand whether I should prefix my command with time. And I typically don’t want to run a slow command twice just to know how much time it took.

This Eshell prompt is available in the eshell-prompt-extras package:

(setq eshell-prompt-function #'epe-theme-multiline-with-status)

Sadly, there is no equivalent to eshell-post-command-hook for M-x shell which makes it impossible to implement using the same (sane) logic.

So I had to resort to a hack. I added the following to my .bashrc

PS1='\e[1;37m[\e[1;32m\w\e[1;37m]\e[m \e[0;34m\D{%F %T}\e[m\n\e[1;37m\$\e[m '

to get a prompt looking like this:

[~/dotfiles] 2020-06-26 16:36:30
$ echo Hi!

With a bit of Elisp, I was able to write a ambrevar/shell-command-duration command that parses the prompt and reports the time a command took. It’s far from perfect but it’s a start.

As mentioned above, M-x shell does not have an equivalent to eshell-post-command-hook which makes it impossible to use it for helm-system-package.

Comint-mode, the major mode behind M-x shell, supports extra fontification by default. For instance, running

$ ip addr

prints the network interfaces in a different colour. Neat, isn’t it?

One of the features I use the most is the “Switch to to Eshell” action from helm-find-files. Until recently, it only supported Eshell.

This is now fixed in Helm 3.6.3 and I just had to add this to my initialization file:

(setq helm-ff-preferred-shell-mode 'shell-mode)

I can now quickly fuzzy-search and switch to the directly I want without ever typing a single cd command.

A very convenient Emacs command is narrow-to-defun (C-x n d): it focuses the buffer on a single function definition and all buffer-global commands are restricted to it. For instance, if I want to replace all occurrences of foo in a given function, without altering the other foo in the rest of the buffer, I can first narrow to the function, then run query-replace over all visible occurrences.

I wanted to do the same with shells. Indeed, it’s very useful to be able to restrict commands to the output of a given command, say, to search an output without hitting matches from other outputs in the same buffer.

I wrote an implementation both for Eshell and M-x shell.

It’s useful to search prompts, maybe to copy a command or to consult its output again.

Helm can fuzzy-search and browse the prompts of all shell (including Eshell) buffers with helm-comint-prompts-all and helm-eshell-prompts-all.

More often than not, we use multiple shells. By default, the shell history is not synchronized between the shells and, worse, it gets overwritten by the M-x shell that was last closed, which means that all previously closed shell histories are gone.

It’s possible to fix this issue with the right shell setup (e.g. see this discussion for Bash) but it’s limited and it’s not general enough since it must be done for all shell sub-programs, when supported at all.

A better approach is to ignore the underlying shell history and use Emacs capabilities. Eshell has the same limitations by default which I had fixed with some custom Elisp, so I just ported it to comint-mode and voilà!

While I was at it, I’ve also applied history filtering such as removing all duplicates (not just when last command matches the last history entry) and other undesirable commands, such as cd ... or commands starting with a space.

(defun ambrevar/ring-delete-first-item-duplicates (ring)
  "Remove duplicates of last command in history.
Return RING.

Unlike `eshell-hist-ignoredups' or `comint-input-ignoredups', it
does not allow duplicates ever.  Surrounding spaces are ignored
when comparing."
  (let ((first (ring-ref ring 0))
	(index 1))
    (while (<= index (1- (ring-length ring)))
      (if (string= (string-trim first)
		   (string-trim (ring-ref ring index)))
	  ;; We don't stop at the first match so that from an existing history
	  ;; it cleans up existing duplicates beyond the first one.
	  (ring-remove ring index)
	(setq index (1+ index))))
    ring))

(defvar ambrevar/shell-history-global-ring nil
  "The history ring shared across shell sessions.")

(defun ambrevar/shell-use-global-history ()
  "Make shell history shared across different sessions."
  (unless ambrevar/shell-history-global-ring
    (when comint-input-ring-file-name
      (comint-read-input-ring))
    (setq ambrevar/shell-history-global-ring (or comint-input-ring (make-ring comint-input-ring-size))))
  (setq comint-input-ring ambrevar/shell-history-global-ring))

(defun ambrevar/shell-history-remove-duplicates ()
  (require 'functions) ; For `ambrevar/ring-delete-first-item-duplicates'.
  (ambrevar/ring-delete-first-item-duplicates comint-input-ring))

(defvar ambrevar/comint-input-history-ignore (concat "^" (regexp-opt '("#" " " "cd ")))
  "`comint-input-history-ignore' can only be customized globally
because `comint-read-input-ring' uses a temp buffer.")

(defun ambrevar/shell-remove-ignored-inputs-from-ring ()
  "Discard last command from history if it matches
`ambrevar/comint-input-history-ignore'."
  (unless (ring-empty-p comint-input-ring)
    (when (string-match ambrevar/comint-input-history-ignore
			(ring-ref comint-input-ring 0))
      (ring-remove comint-input-ring 0))))

(defun ambrevar/shell-sync-input-ring (_)
  (ambrevar/shell-history-remove-duplicates)
  (ambrevar/shell-remove-ignored-inputs-from-ring)
  (comint-write-input-ring))

(defun ambrevar/shell-setup ()
  (setq comint-input-ring-file-name
	(expand-file-name "shell-history" user-emacs-directory))
  (ambrevar/shell-use-global-history)

  ;; Write history on every command, not just on exit.
  (add-hook 'comint-input-filter-functions 'ambrevar/shell-sync-input-ring nil t))

(add-hook 'shell-mode-hook 'ambrevar/shell-setup)

One of the benefits of Eshell is that it integrates shell commands with Emacs. For instance, running grep will display an interactive result in an Emacs buffer.

It’s possible to write an emacsclient wrapper that evaluates the command passed as argument in an Emacs buffer to, so it’s possible to mimic this feature of Eshell rather closely.

Still, this is not as closely integrated to Emacs as it could get. Indeed, Eshell can intertwine its shell language with Elisp. It’s thus able to run any Elisp function.

Maybe a good direction to explore is piper.

I’m happy with M-x shell, the everyday use is much smoother than that of Eshell. Performance being one of the biggest selling point in my experience.

Overall, with the support of few packages such as Helm and helm-fish-completion I get a stellar shell experience. I miss very few features, such as support for “visual” commands, modifiers and predicates which I rarely use.