2010-05-07

Emacs Line Return And Dos, Unix, Mac, All That ^M ^J

Perm url with updates: http://xahlee.org/emacs/emacs_line_ending_char.html

Emacs Line Return and Windows, Unix, Mac, All That ^M ^J ^L

Xah Lee, 2010-05-07

This page explains line ending conventions Windows, Unix, Mac, and how to change them with emacs.

Here's a short table about different newline convention and how to input them:

NameASCII Codestring notationCaret NotationAbbrevInput method
line feed10\n^JLFCtrl+q Ctrl+j
carriage return13\r^MCRCtrl+q Ctrl+m or Ctrl+q Enter

Following is the newline convention in different operating systems.

Operating SystemNewline ConventionNotes
Unix, Linux, Mac OS X^JMac OS X prefers ^J, but accept the Mac OS Classic's ^M too.
Windows^M^J
Mac OS Classic^M

Why does emacs show ^M in a buffer?

The “^M” is ASCII caret notation for unprintable Carriage return char (ASCII 13). If emacs shows that, it's probably because you have mixed characters of ^M and ^J and emacs cannot interpret them consistently as newlines.

To fix it, call “set-buffer-file-coding-system”, then give one of: “mac”, “dos”, “unix”. Then, save the file. If that does not fix it, you can use find and replace to remove it manually.

How to delete ^M manually?

Call “query-replace” 【Alt+5】, then type the ascii 13 char by 【Ctrl+q Ctrl+m】 for the find string. For replacement string just press Enter for none.

What does 【Ctrl+q】 mean?

The 【Ctrl+q】 is the shortcut for the command “quoted-insert”. It will let you enter the next charater literally. For example, to type a literal tab, press 【Ctrl+q】 then the Tab key.

Some characters do not have a representation on the keyboard. For example, the Enter key is either Line Feed or Carriage Return, depending on the application, but cannot be both at the same time.

To input unprintable ascii chars, you can press 【Ctrl+q】 first, then type a letter indicated by the char caret notation. For example, the Carriage Return has caret notation of “^M”, so, press 【Ctrl+q Ctrl+m】 will type it. Tab is “^I”, so 【Ctrl+q Ctrl+i】 inserts a tab.

For detail about the unprintable ASCII chars, their printable notations, input methods, see: The Confusion of Emacs's Keystroke Representation.

Can i change newline convention from Windows to unix by just deleting ^M?

Not really. When emacs opens a file, it represent all line returns by (ascii 10; line feed; unix convention), doesn't matter what's the actual newline convention in the file. If emacs displays “^M” (ascii 13; carriage return), that's because the file has mixed line endings.

When you save a file, emacs automatically use the right newline when writing the file to disk.

Also, emacs may automatically add a newline to the end of the file when you save it. Which character is added as newline depends on the current file encoding system. So, if you manually change the newline char, emacs may add one that is inconsistent to what you expect. The auto adding newline is controlled by the variables “require-final-newline” and “mode-require-final-newline”.

If you want to convert line ending from different OS, best thing to do is call “set-buffer-file-coding-system”, with a value of “unix”, “mac”, “dos”. (on Mac OS X, use “unix”.)

How to know which newline convention is used by emacs for the current file?

Call “describe-variable” 【Ctrl+h v】, then “buffer-file-coding-system”.

How to quickly find out what ASCII char are those ^M ^J ^L?

Place your cursor on it, then call “describe-char”.

How to change file line endings between Windows/Unix/Mac?

Open the file, then call “set-buffer-file-coding-system” 【Ctrl+x Enter f】. When it prompts you for a coding system, type one of: “mac”, “dos”, “unix”. Then, when you save the file, it'll be saved with the proper encoding for newlines.

To do it batch on a list of files, use the following lisp code:

(defun to-unix-eol (fpath)
  "Change file's line ending to unix convention."
  (let (mybuffer)
    (setq mybuffer (find-file fpath))
    (set-buffer-file-coding-system 'unix) ; or 'mac or 'dos
    (save-buffer)
    (kill-buffer mybuffer)
   )
)

(mapc 'to-unix-eol
 (list
"~/jane/myfile1"
"~/jane/myfile2"
"~/jane/myfile3"
; ...
  )
)

To use the code, first edit the list of files above. Then, select all the code, type 【Alt+x eval-region】. That's it.

If you want the function to work on marked files in dired, then use the following code:

(defun to-unix-eol-on-marked-files ()
  "Change to unix line ending for marked (or next arg) files."
  (interactive)
  (mapc 'to-unix-eol (dired-get-marked-files))
)

Select the code, then call “eval-region”, then 【Alt+x dired】, then press “m” to mark the files you want, then call “to-unix-eol-on-marked-files”.

Thanks to Stefan Monnier for a major tip on this newline issue in emacs.

Was this page useful? If so, please do donate $3, thank you donors!