Common Lisp is a great language. Of course, it cannot be the best at everything, it comes with its load of historical idiosyncrasies and it might not be the language of the future.
But there is one aspect of programming in which Common Lisp keeps it first position after so many years: “interactive programming.” Only few languages come close (maybe Racket is one of them). Interactive programming boosts your productivity so much that I believe it should always be part of the decision process when it comes to choosing a language to work with. Hopefully this advance in the practice of programming will be remembered when we design the language of the future.
I believe that for this reason alone Common Lisp is worth learning (and using).
In the following article, I’m not going to proselytize the language: it’s not perfect and it won’t suit everyone. I simply want to give a depiction more accurate than the common misconceptions about the language, and hopefully help the curious as well as the experienced user enhance their experience of programming.
- You might be deterred from using Common Lisp for the wrong reasons. I’m going to highlight a few of those common “wrong reasons” and if those match yours, then I believe you ought to give it a try.
- Should you want to use Common Lisp, you might be exposed to aged practices or bad tooling. In the following I’ll try to catalogue as decent amount of the most modern advances made in the Common Lisp ecosystem.
I believe that many negative misconceptions about the language stem from myths of forgotten times which bear little relevance today.
Some common myths include:
- Common Lisp does not have compile-time type checking.
- Common Lisp is for imperative, object-oriented programming.
- Common Lisp is too specialized, it’s not for general-purpose development.
- Common Lisp applications are hard to deploy.
Some of these myths subsist today even among Common Lisp developers, simply because the news of the recent developments in the ecosystem haven’t reached them. Indeed, Common Lisp, as an extensible language, has modernized tremendously over the last decades.
Common Lisp is a gigantic programming language and ecosystem. As such, this article is in no way exhaustive! I invite the community to reach out and share their recommendations, corrections and other discussions!
If you enjoy this article and would like to help me keep writing, consider chipping in, every little bit helps to keep me going :)
Getting started with the right tools and learning resources
Common Lisp has earned the reputation of being difficult to set up. Today, some good solutions exist to help newcomers:
- Portacle is an all-in-one bundle that includes SBCL, Emacs, SLIME and more. While the user must learn the basics of Emacs, Portacle is known to be relatively friction-less.
The Guix package manager can set up a Common Lisp development environment in one go:
guix install sbcl git emacs emacs-slime emacs-helm-slime emacs-company emacs-magit emacs-paredit emacs-rainbow-delimiters
guix environment --ad-hocif you don’t want to persist a profile.
If you prefer VI-style bindings:
guix install sbcl git emacs emacs-slime emacs-helm-slime emacs-company emacs-magit emacs-paredit emacs-rainbow-delimiters emacs-evil emacs-evil-collection
If you prefer Lispy over paredit to automate s-expression manipulation (such as automatically balancing parentheses):
guix install sbcl git emacs emacs-slime emacs-helm-slime emacs-company emacs-magit emacs-lispy emacs-rainbow-delimiters
Should you need Common Lisp libraries, you can also install many of them with Guix. For the rest, you’ll have to resort to Quicklisp which is not provided by Guix, so you’ll have to install it manually: https://www.quicklisp.org/beta/
The community-maintained Common Lisp cookbook is probably the best starting point with excellent references to everything from string manipulation to continuous integration.
It’s also a good portal to other resources such as:
- awesome-cl: A curated list of Common Lisp frameworks and libraries.
- Practical Common Lisp: A free online book to learn Common Lisp.
That said, the introductory documentation can be rather overwhelming in my opinion and I’ve invited the community to work on a gentle, modern primer for the complete beginner:
Editor and offline documentation
Many Common Lisp developers use Emacs. You don’t have to, many other editors support Common Lisp as well, such as Atom and SLIMA.
On top of this, there are Helm packages to enhance your SLIME / SLY experience with live, narrowing fuzzy-search and completion for about everything:
Local Common Lisp HyperSpec
The HyperSpec is the Common Lisp reference available here: http://www.lispworks.com/documentation/HyperSpec/Front/index.htm.
While the website look is rather dated, SLY and SLIME will interface with it so that you can browse the specifications directly from Emacs without having to open a browser.
Moreover, the HyperSpec can be installed locally, to be consulted offline. See http://quickdocs.org/clhs/.
SLIME vs. SLY
SLY is a fork of SLIME. SLIME tends to be more conservative while SLY is slightly more featureful.
A few benefits of SLY over SLIME:
- It has more consistent, cleaner code and reuses more of Emacs facilities (like
comint-modefor the REPL).
- It has a better color theme in my opinion.
- It offers better fuzzy completion.
- A feature unique to SLY: stickers allow the user to “stick” non-persistent code to help with debugging by annotating it. See https://github.com/joaotavora/sly#stickers.
- The setup is essentially zero-config with sane defaults, with all the features like mrepl, stickers, etc. enabled by default. Of course you can disable the features you don’t like.
- It compiles with ASDF which makes it easier to package, bundle or deploy.
SLIME does include some features that are only available in SLY as plugins, such as ASDF management. I recommend the following plugins for SLY:
SLIME / SLY tips and tricks
Check out the following not-so-well-known commands, they could dramatically improve your workflow! (Most of them have a SLY equivalent.)
slime-delete-system-fasls(Useful when .fasls are out-of-sync)
C-c M-o: useful when
slime-profile-package, then run the desired functions, then
In particular, note that
slime-who-specializes lists the methods of a given
class, which answers a common complaint coming from people used to languages
from the Algol family: the ability to complete the methods of the
foo class by
ASDF is the de-facto standard build system for Common Lisp.
To use a
FOO system, place it in in
~/common-lisp and call
Better: from the SLIME or SLY REPL, press
,load-system and you’ll be provided
with a list of systems that can be loaded.
Historically ASDF required you to explicitly list the files to compile. This
was cumbersome and error-prone, but thankfully it has been fixed some time ago
package-inferred-system extension which I highly recommend:
- No need to list files.
- No need to list dependencies. With non-inferred-systems, the dependency list can be wrong, e.g. include too many of them or miss some. In that sense inferred-systems are more robust.
- Enforces that every file must be a package.
- If your system has many files, and thus that many packages, it can get quickly cumbersome for the user to load all those packages one by one. Instead, you can create a “meta package” which imports all the other packages of your system, so that the user only has to load this one to access the whole system.
How can we get the list of dependencies since we don’t list them explicitly in the
.asdanymore? Thankfully the snippet below will retrieve them programmatically for us:
(declaim (ftype (function (string) list) package-dependencies)) (defun package-dependencies (pkg-name) (let (depends) (labels ((iter (openlist) (if (null openlist) depends ;; is this a subsystem of foo? (let ((find (search pkg-name (first openlist)))) (if (and find (zerop find)) (iter (append (asdf:system-depends-on (asdf:find-system (first openlist))) (rest openlist))) ;; if not, it's a direct dependency: collect it (progn (pushnew (first openlist) depends :test 'equalp) (iter (rest openlist)))))))) (iter (list pkg-name)))))
(package-dependencies "dbus") ; => ("ironclad cl-xmlspam split-sequence flexi-streams ieee-floats iolib babel trivial-garbage alexandria
Modernizing the language with packages
Myth: Common Lisp is old and does not enjoy the features of modern languages from recent years.
Truth: While the Common Lisp standard is from the 1990s, many modern libraries have continuously updated the language.
One of the biggest selling points of Common Lisp (and Lisp languages in general) is that the language itself is extensible. This means that if new paradigms become fashionable, they can easily be added to the language.
In particular, Common Lisp has the reputation among functional programmers of being very imperative and object-oriented. While these styles were popular in the past, nothing prevents us from programming functionally in Common Lisp.
As with many languages, Common Lisp packages can be
use-d, which means all
symbols will be merged in the current name space, or
import-ed, which means
all symbols will be accessible only via the package prefix.
The benefit of
use is that importing a language-enhancing package gives us a
feel that the enhancements are first class.
Use-ing packages has a few drawbacks however, so be mindful of them. If you
prefer the safe way, stick to package imports.
The drawback however is that if you change your mind and decide to
importthe package instead, you’ll have to add the package prefix to all the related symbols manually.
This issue could be eliminated with sufficient editor support to refactor symbols by their packages. See https://github.com/slime/slime/issues/532 for a discussion.
- It’s harder to see from just a glance which symbols comes from which package.
Use-ing a package without pinning its version (or even commit) is exposes your project to breakage if the package upstream adds a new symbols that would conflict with one of your symbols. As a minimum requirement to
use-ing packages, make sure you pin their exact version. (Thanks to flamingbird for the detailed explanation.)
Local package nicknames
Since you should (almost always)
import packages, you’ll have to prefix their
symbols with the package name. Sometimes this package name can be lengthy to
write: in this case, you should know that you can assign a local nickname to any
(defpackage my-package (:use #:cl) (:local-nicknames (#:a #:alexandria.dev.0)))
So now you can call the much shorter
a:compose from your package!
This approach is preferred over using the
rename-package procedure which is
Recommended language-enhancing packages
The following packages are widespread and useful enough that can be
- Alexandria: various language and functional programming procedures.
- UIOP: Macros and operating-system-level procedures. Included with ASDF in most Common Lisp compilers.
- Trivia: Pattern matching.
The following libraries are also extremely useful, although not to everyone’s taste:
- Trivial-types: More common type definitions.
- Serapeum: a huge collection of utilities (named let, local definitions, hooks, file “human” size, string manipulation, etc.), built as a complement to Alexandria and UIOP. It greatly extends the language.
- Series: Lazy sequences and supporting higher-order functions.
- defclass-std: Saner and more robust defaults for class definitions. In
particular, this prevents accidental typos in the accessor or the initarg, and
it sets a default value (the
initform) automatically by default.
- Sycamore: Functional data structure library
Other language-enhancing libraries:
- Rove: The (only?) testing framework with coverage support.
- ppcre: Advanced string manipulation with unicode support.
- named-readtables: First class name space for readtables.
- generic-cl: Generic function wrapper over various functions in the Common Lisp standard, such as equality predicates and sequence operations. Includes a few lazy sequence utilities.
Myth: Common Lisp is a procedural, object-oriented language. Truth: Many functional programming paradigms are available in Common Lisp.
Scheme developers in particular will be happy to find:
let, local definitions,
flipand more in Serapeum.
- Lazy sequences in Series.
- Functional, immutable data structures in FSet, cl-hamt, Sycamore (thanks to Jach for pointing it out), folio2 (thanks to mikelevins for sharing this one), modf (thanks to dzecniv for sharing).
The availability of the Series package means in particular that you don’t have
to use the (in)famous
LOOP macros if you don’t like them.
Static, parametric and algebraic typing
Myth: Common Lisp does not support static typing.
Truth: ECL, CCL and SBCL at least support compile-time type-checking.
While not exactly static typing since it’s optional, the technical term for it is “gradual typing.”
Note that types defined with a
satisfies predicate are not checked at
compile-time unless at the top-level (this is true for SBCL 2.0 at least).
Classes are a whole different beast and there type checking can currently be done with a separate library: https://github.com/fisxoj/sanity-clause. Also see https://lisp-journey.gitlab.io/blog/how-to-check-slots-types-at-make-instance/.
Serapeum provides helpers for type manipulation: https://github.com/ruricolist/serapeum/blob/master/REFERENCE.md#types. defstar also provides syntactic sugar for type declaration (it also comes with a named let).
Algebraic types are supported out of the box, but notations can be improved with https://github.com/stylewarning/cl-algebraic-data-type.
There is even an ML-style library for Common Lisp: Coalton.
More on the topic:
From the HyperSpec:
(defun equidimensional (a) (or (< (array-rank a) 2) (apply #'= (array-dimensions a)))) ; => EQUIDIMENSIONAL (deftype square-matrix (&optional type size) `(and (array ,type (,size ,size)) (satisfies equidimensional))) ; => SQUARE-MATRIX
Some useful types from the
(deftype pathname-designator () '(or string file-associated-stream pathname)) (deftype function-designator () '(or symbol function)) (deftype string-designator () '(or character symbol string))
(declaim (ftype (function (trivial-types:pathname-designator) pathname) my-basename)) (defun my-basename (path) (uiop:pathname-parent-directory-pathname path)) (compile nil '(lambda () (my-basename 17))) ; in: LAMBDA () ; (MY-BASENAME 17) ; ; note: deleting unreachable code ; ; caught WARNING: ; Constant ; 17 conflicts with its asserted type ; (OR (VECTOR CHARACTER) (VECTOR NIL) BASE-STRING ; (AND STREAM (SATISFIES TRIVIAL-TYPES:FILE-ASSOCIATED-STREAM-P)) PATHNAME). ; See also: ; The SBCL Manual, Node "Handling of Types" ; ; compilation unit finished ; caught 1 WARNING condition ; printed 1 note
Export at definition site
One of the main pain points in Common Lisp development is exporting public symbols.
Traditionally, symbols were exported from a dedicated “package” file that’s loaded first. This is cumbersome and error-prone since the symbols and their exports may run out of sync.
There is an
export function which can be called at definition site. Sadly,
since it’s a function, it won’t be called at compilation time and thus will yield
an error if you attempt to access the symbol from a foreign package at this point.
The trick is to force the evaluation of
export at compilation time with:
(eval-when (:compile-toplevel :load-toplevel :execute) (export FOO))
Since this is rather lengthy, I recommend you use Serapeum’s
Hopefully the syntax will get even more practical once issue 38 is resolved.
Contextual documentation is part of the Common Lisp standard, the so-called “docstrings” which can be placed at the beginning of variable and function definitions.
A detail that is perhaps not so well known is that classes, slots and packages
also support docstrings via the
:documentation keyword. It’s underused in the
community and I believe we should encourage its use.
Another perk of Common Lisp is the ability to declare the types of functions, variables, class slots, etc.
So instead of adding a comment about the accepted types of a function arguments
and the type of the return value, simply
(declaim (ftype (function (number) string) number->string)) (defun number->string (n) ...)
This has the double benefit of documenting the argument types and the return types of your function, as well as enforcing compile-time type-checking.
Same goes with class slots:
(defclass my-compiler () ((name :accessor name :initarg :name :initform "" :type string :documentation "Name of your favourite Common Lisp compiler.") ...))
Common Lisp is a standard and Common Lisp compilers are legions.
Some target specific use cases.
- ECL targets embedded development, with good interop with C.
- ABCL is build on top of the Java virtual machine and thus has good interop with Java.
For general purpose development, the go-to compiler these days is SBCL. It is actively developed and has many benefits:
- Probably the best gradual type checking of all Common Lisp compilers.
- Fast compilation.
- Fast executables.
- Statistical profiler.
- Coverage analysis.
Debugging and stepper
Many Common Lisp compilers offer great support for debbuging.
In particular, the
step macro is part of the standard.
With SBCL, you must first compile the code you want to debug with
(declaim (optimize (speed 0) (space 0) (debug 3)))
Then you can run
(step (foo bar))
to step through the execution of
(foo bar). With SLIME / SLY, the Emacs point
will follow the currently-executed expression!
To increase the level of warning detection, call
(declaim (optimize (speed 0) (space 0) (debug 3)))
before loading a system.
reload-ing a system might raise more warnings.
You can use
,reload-system in SLIME / SLY.
More debugging tips
See this great tutorial in four parts which covers breakpoints, stepping, inspection, class redefinitions and restarts: https://malisper.me/debugging-lisp-part-1-recompilation/.
(Thanks to Jach for sharing.)
Deployment and executable size
The myth: Common Lisp scripts and binaries are hard to distribute; binaries are too big.
The truth: Common Lisp compilers offer a lot of flexibility.
- If portability across compilers is not an issue, it’s trivial to distribute
scripts, e.g. by using the
- It’s even possible to distribute the compiled files (e.g. the .fasls) since they are executable. The exact same compiler must be present on the target machine however.
- Binaries produced by some compilers like SBCL or CCL are rather large for a reason: they include a self-standing complete Lisp image, that is to say more or less the whole compiler plus the required libraries. The benefit: the binary does not depend on the compiler and thus the binary is extremely portable; to top it all, the user can connect a REPL to the stand-alone binary and hack the application while it’s running!
- Some compilers like ECL or CLISP can produce dynamic binaries that link against a runtime library (just like C). Those binaries tend to be very small.
- SBCL can produce compressed executables with no noticeable impact on startup time. With compression, a hello-world binary with all of SBCL fits in some 15 MiB.
Tip: Add this to your
.asd to automatically create compressed binaries with
#+sb-core-compression (defmethod asdf:perform ((o asdf:image-op) (c asdf:system)) (uiop:dump-image (asdf:output-file o c) :executable t :compression t))
Guix vs. Roswell
Roswell is a Common Lisp environment manager that can install various Common Lisp compilers, deploy applications, etc.
While less popular among Common Lisp developers, I believe that Guix advantageously supersedes Roswell:
- Guix does not distribute opaque binaries, it allows us to produce reproducible packages (which are also increasingly more bootstrappable). Thus Guix is much more trustable.
- Guix environments can mix in any other packages, such as libraries or other programming languages. It is thus possible to create environments for complex projects dealing with multiple programming languages at the same time.
- Guix can run environments in containers.
- Guix can automatically “pack” a Common Lisp application with all its dependencies (including non-Lisp ones) as a tarball or a Docker image.
I believe that Common Lisp deserves its reputation of a robust, practical, general purpose programming language. It’s high time we moved on from the old myths and we fixed the few pain points that still remain.
The setup and the learning curve is still a bit steep today, but it does not have to be this way: with the right web sites, documentation and tools, I’m convinced we could boost Common Lisp accessibility and reach out to many more developers out there.
Fashions are changing and many third-party libraries allows the language to keep up with the latest trendy paradigms.
Finally, Common Lisp is a gigantic ecosystem that we never stop exploring. I believe it’s important for the community to be dynamic and communicative about the evolution of the language. So don’t hesitate to reach out to me, share you critics and tell me what you think about this article!