2008-01-11

modernization of emacs lisp

Perm url:
http://xahlee.org/emacs/modernization_of_elisp.html

Modernization of Emacs Lisp

Xah Lee, 2008-01

Over the past decade, before i really got my hands down with Emacs lisp in the last couple of years, i have heard many times of wishes and projects that want to replace elisp with Common Lisp or Scheme lisp. At those times with my ignorance of deep elisp, i wished alone the same line, blithely hoping that one day we'll have a emacs with a better lisp (without me needing to put anything in). (this is the “i want to believe” syndrome typical with Free Software Foundation and OpenSource youngsters) However, in the past 2 years i studied elisp, and chanced to actually spend maybe 30 minutes about this issue of modernizing elisp and looked at some websites about such projects that use Scheme lisp or Common Lisp. I think now that these efforts are not as important as they seem.

Benefits With A New Lisp In Emacs

To create a emacs with Common Lisp or Scheme Lisp as extension language, is not going to work as one might thought. Because, the basis of such wish is that a programer would just learn one language (and a better one at that) and not to have to learn one for X lisp and one for Emacs lisp. This line of thinking is reasonable, but has hidden, unexpected flaws. Because, to create such a extension language in a text-processing system, you necessarily have to need a lot concepts and data types extraneous to the language. Namely, emacs's buffers, point, frame, kill-ring, keymaps, syntax table, fontification data structure, ...etc. So, even if one day we have a emacs with Scheme Lisp as the extension lang, but the deeper fact is that it will actually have perhaps hundreds, of functions inherent in the system that has nothing to do with Scheme the lang or general programing. (and, there will be needs or complexities, to separate what's editing-related function from the language's core functions) So, the final effect is that, to write any program for text processing or editor-extension for the New-Lisp Emac, the programer is really, factually, spending most of his time learning about the working of these functions in this text-processing system, even if he is already a Scheme expert. In short, given a Scheme Emacs (or Common Lisp emacs), it will not make that much of a difference as with current Elisp Emacs, with regards to the effort of writing code for emacs.

There will be, a major practical impact however, if such system come into being and widely used. And that is, the benefits of a unification in general sense of that word (imagine Scheme and Common lisp are unified; or GNU Emacs and Xemacs; South Korea and North Korea, etc.). A more practical benefit is a pre-packaged, coherent, IDE for Common lisp or Scheme lisp development. (imagine Visual Studio with C++ or Visual Basic; Xcode and Objective-C; Java with Eclipse or NetBeans.)

Technical Advantage

The fact a somewhat technically better lisp (CL, Scheme) replacing elisp, has very little practical benefits for a emacs extension language, even when we consider just the technical aspects. It is certainly true, that Common Lisp and Scheme Lisp are technically better than Emacs Lisp, in different ways. For example, elisp uses dynamic scope. Elisp has a datatype that is both character and integer. Elisp does not support exact arithmetics. These defects are not present in CL or Scheme. And, CL supports multi-threads, whereas elisp has single thread. (practically meaning that emacs users must wait for each operation to finish) CL has many tried-and-true libraries. Scheme provide the same language power with a simpler, cleaner language.

However, there are also considerable technical disadvantage. For example, CL is fret with baggage and ugliness, with its voluminous language specification. Scheme does not have a library system (same situation as elisp) untill the R6RS specification in late 2007, and fully conformal implementation are scarce even for R5RS. Both CL and Scheme does not universally support unicode, and while in emacs lisp a infrastructure exist that is already employed to support full unicode for all practical purposes.

When we survey most uses of the extension language in emacs, they are predominantly text processing related tasks, with some other uses such as embedding a text-based client for ftp, irc, mail, web browser, OS command line interface, file manager. The benefits of fixing elisp's weaknesses by CL or Scheme for small-scale programing as these are not major, and is offset by the various technical problems they introduce.

Effort Required

It is a fact that the effort needed to create such as system basically hampered all such projects (although such system with CL and Scheme already exists (and to my knowledge, usable to some extent).). And, the need to advertise such a system to get people to switch from existing system (emacs) is nigh impossible. The 2 facts that the effort needed for such system and the effort needed to cover legacy elisp so people will switch, basically puts a death knell to such projects (or, long, long time down the road, maybe 5, 10 years, if without commercial incentive. (and with such long gestation, in today's technological speeding, i do not think lisp lang itself (in particular, CL, Scheme, or anything with the cons business), are suitable or competitive as a high-level general lang of the 2010s)).

Note: This essay is not meant to discourage the development of a new lisp emacs, especially because they already exist. This essay is meant to put off some common wishful thinking. As noted before, a major benefit of a emacs with Common Lisp or Scheme Lisp is that they provide a unified, packaged, system for development of these languages, and since it uses the same language to extend functionality of the editor, it will transparently increase the number of developers for the editor as well as the language. Here are the home pages of the most prominent on-going efforts:

Hemlock (editor)↗ (Hemlock; Common Lisp)
http://common-lisp.net/project/climacs/ (Climacs; Common Lisp)
http://www-swiss.ai.mit.edu/projects/scheme/documentation/user_8.html (Edwin; Scheme)
(Thanks to Rainer Joswig for pointing out Common Lisp's multi-thread advantage, and pointing out Hemlock.)