2010-12-07

Manatee, a haskell environment

Andy Stewart, who wrote tens of elisp packages (see his site), started a new project on Haskell called Manatee. We voice chatted on skype for about 50 min on emacs and haskell and programing.

Manatee is a OS-like environment for Haskell hacking, currently running on top of linux and GTK+.

You can see many screenshots here: Source www.flickr.com. Or a video of screenshots at Source www.youtube.com.

Manatee home page at: http://hackage.haskell.org/package/manatee. Documentation at http://haskell.org/haskellwiki/Manatee.

2010-12-06

Opera browser hotkey idiocy

Discovered, that in Opera, to switch to next/prev tabs in a normal way, you have to press 【Ctrl+F6】 and【Ctrl+Shift+F6】. What a idiocy. (See: Opera Pain; Opera Browser Problems.) Or, in Opera 9.2 or before, press the 1 and 2 keys on the number pad. In Opera 9.5 or later, you have to Enable single-key shortcuts in the preference. (See: http://help.opera.com/Windows/10.63/en/keyboard.html.)

So, in AutoHotkey, i added these to be consistent with all my hotkeys for browsers. (the / and * on number pad to switch to prev/next tabs.)

;; Opera hotkeys
#IfWinActive ahk_class OperaWindowClass

; close window
$Pause::Send ^w

; prev tab
$NumpadDiv::Send ^+{F6}

; next tab
$NumpadMult::Send ^{F6}

See: AutoHotkey Basics and AutoHotkey Example Scripts.

Also, after several years of hearing Opera's stupid “mouse gestures”, today i took 5 min to read up what it is. See: http://help.opera.com/Windows/10.63/en/mouse.html. Rather idiotic.

Unicode in Mathematica

Perm url with updates: http://xahlee.org/math/mathematica_unicode.html

How Mathematica does Unicode?

Xah Lee, 2010-12-05

This page explains some tech detail about how Mathematica uses unicode. This article may not be 100% accurate. I'm putting it up for now after spending a few hours. Am tsill working on it. If you are a Mathematica expert, please comment or correct.

Mathematica supports unicode, but does not use Unicode when saving to file. (See: UNICODE Basics: What's Character Encoding, UTF-8, and All That?)

Mathematica (mma) files uses ascii only. (See: Mathematica Notebook Technology.)

How does it support unicode if it uses only ASCII?

Mathematica's Named characters 「\[Name]」

It has a set of special characters with the syntax 「\[name]」. For example:

Glyph	Syntax
é	\[EAcute]
É	\[CapitalEAcute]
α	\[Alpha]
Δ	\[CapitalDelta]
⊕	\[CirclePlus]
∵	\[Because]
∈	\[Element]
⇔	\[Equivalent]
ℝ	\[DoubleStruckCapitalR]

So, when you type 「\[Alpha]」 in mma, it is displayed as “α”. (All builtin symbols in mma starts with capital letter.)

You can think of them as html's “named character entities”. (See: Character Sets and Encoding in HTML.) There are about 900 named chars. For the complete list, see: Listing of Named Characters.

Many of the named chars are also in unicode, but not all. Similarly, many Math Symbols in Unicode are not in this list. Also, unicode's chinese chars, arabic alphabets etc, are not in mma's named chars.

Map Between Unicode and Named Chars

When you paste a unicode char into mma, there is a map that automatically interprete the unicode as one of the named char.

So, for example, if you paste “α” (GREEK SMALL LETTER ALPHA; “U+x3b1”), it automatically becomes Mathematica's 「\[Alpha]」, and displayed as “α”.

Syntax for Unicode Char 「\:nnnn」

For any unicode that's not one of mma's named char (such as chinese chars), their syntax is this: 「\:nnnn」, where the nnnn is unicode's 4 digit hexidecimal representation of the char. For example, the chinese char “水” (water), unicode hex is “6c34”, in mma is: 「\:6c34」.

The above roughly summarize how mma takes unicode as input.

Some Named Chars has Builtin Meaning

Of the named chars, many has special meaning in mma. For example, 「\[Pi]」 is automatically considered identical to the builtin symbol 「Pi」, which in Mma means mathematical constant. (So, if you type 「N[\[Pi]]」 or 「N[\:03c0]」, they are displayed as 「N[π]」 with meaning of 「N[Pi]」, and if you evaluate it, you get “3.14159”.). Here's some examples of special meaning named chars.

Glyph	Mma named char	Unicode name	Unicode hexidecimal	Default Interpretation
≥	\[GreaterEqual]	GREATER-THAN OR EQUAL TO	2265	GreaterThan 「>=」
π	\[Pi]	GREEK SMALL LETTER PI	03c0	Pi
∞	\[Infinity]	INFINITY	221e	Infinity
∫	\[Integral]	INTEGRAL	222b	Integrate
⋂	\[Intersection]	N-ARY INTERSECTION	22c2	Union
∑	\[Sum]	N-ARY SUMMATION	2211	Sum
√	\[Sqrt]	SQUARE ROOT	221a	Sqrt
⊕	\[CirclePlus]	CIRCLED PLUS	2295	CirclePlus

Note: it appears that it is possible to over-ride the default interpretation of named char to builtin symbol (function, constant), for all or some of the named char. (i haven't investigated on how yet.) See: MakeExpression.

http://reference.wolfram.com/mathematica/tutorial/Operators.html

Alias Shortcut for Named Chars

Some of the named char has one or more aliases for ease of input. For example, to enter α, you can type 【Esc a Esc】 or 【Esc alpha Esc】. Here's some examples:

Glyph	Common Alias
α	a
π	p
∞	inf
≤	<=
°	deg
Δ	D
∈	el
→	->

See: http://reference.wolfram.com/mathematica/tutorial/Introduction-ListingOfNamedCharacters.html.

Characters that are alternatives to standard keyboard operators use these operators as their aliases (e.g. Esc -> Esc for , Esc && Esc for ∧).

Most single-letter aliases stand for Greek letters.

Capital-letter characters have aliases beginning with capital letters.

When there is ambiguity in the assignment of aliases, a space is inserted at the beginning of the alias for the less common character (e.g. Esc -> Esc for \[Rule] and Esc -> Esc for \[RightArrow]).

! is inserted at the beginning of the alias for a Not character.

TeX aliases begin with a backslash \.

SGML aliases begin with an ampersand &.

User-defined aliases conventionally begin with a dot or comma.

See: Special Characters.

List of Named Char with Special Meanings

... work in progress

Inputing Special Chars

You can input a special character by:

Use one of the graphical palettes.
Copy the unicode char somewhere and pasting it in mma.
Type it like this: 「\[Name]」.
Type the Unicode hexadecimal like this: 「\:nnnn」
Type Esc, then the char's alias name, then Esc again.

... work in progress

another anecdote of emacs power

Another anecdote on the power of emacs.

Today, i need to study how Mathematica treats unicode. (See: How Mathematica does Unicode?). In the process, i need to get a file of math unicode symbols. I have a page at Math Symbols in Unicode, but i need just the unicode symbols, not all other content.

So, simply copy the whole file content. Then, put that into a file, say 〔math_symbols_unicode.txt〕. Then, delete all asccii chars. This you can do by calling “query-replace-regexp”. (See: Find and Replace with Emacs) For the replace regex, use 「[[:ascii:]]」. For the replacement string, just press Enter for nothing. Then, press ! to replace all.

Now, i need a space between each symbol. This is a great job for Emacs keyboard macros. Type 【Ctrl+x (】 to start recording. Then type 【→ space】. Then type 【Ctrl+x )】 to end recording. Then, type 【Ctrl+u 999】 (for repeating 999 times the next command), then type 【Alt+x call-last-kbd-macro】. Then, spaces are inserted between each chars.

Thanks to Lew Perin for a correction.

Aaron wrote to note that, to add a space between each char, it's simpler to actually do another query-replace-regexp. Use 「$.$」 for the find regex, and use 「\1 」 for the replacement string.

2010-12-05

Tools to Display Math on Web

Perm url with updates: http://xahlee.org/math/display_math_on_web.html

Tools to Display Math on Web

Xah Lee, 2010-04-13, 2010-12-04

Found 2 tools to write math for the web:

FireMath, a MathML Editor for Firefox

FireMath is a MathML editor as a Firefox plugin. Home at: firemath.info.

MathJax

MathJax. Home at: mathjax.org.

MathJax is an open-source JavaScript display engine for LaTeX and MathML that works in all modern browsers. It was designed with the goal of consolidating the recent advances in web technologies into a single, definitive, math-on-the-web platform supporting the major browsers and operating systems. It requires no setup on the part of the user (no plugins to downlaod or software to install), so the page author can write web documents that include mathematics and be confident that users will be able to view it naturally and easily. One simply includes MathJax and some mathematics in a web page, and MathJax does the rest.

MathJax uses web-based fonts (in those browsers that support it) to produce high-quality typesetting that scales and prints at full resolution (unlike mathematics included as images). MathJax can be used with screen readers, providing accessibility for the visually impaired. With MathJax, mathematics is text-based rather than image-based, and so it is available for search engines, meaning that your equations can be searchable, just like the text of your pages. MathJax allows page authors to write formulas using TeX and LaTeX notation, or MathML, a World Wide Web Constortium standard for representing mathematics in XML format. MathJax will even convert TeX notation into MathML, so that it can be rendered more quickly by those browsers that support MathML natively, or so that you can copy and past it into other programs.

Detexify

Detexify is a tool that lets you draw a math symbol and it shows you the code for LaTeX. The tool is created by Daniel Kirsch. At http://detexify.kirelabs.org/classify.html.

Mathematica

Of course, you can get Mathematica. Student or hobbyist can get it for ~$295. This would be the best option.

Mathematica home page at: wolfram.com.

For a example of its output, in PDF, HTML, MathML/XML, see: Math Typesetting, Mathematica, MathML.

Other

Rant

The technology to display math notation on the web browser, is really dismal. MathML is released in 1998. Now, it is over 10 years, but it is still not widely supported in web browsers. Authors need to invest huge amount of time experimenting with several solutions, and in most cases, requires knowledge of html, css, xml, MathML, javascript, TeX, to various degrees. One early solution is to resort to some tool turn math notations into images. This means, the browser needs to load a huge bunch of images, and the images are ugly, not scalable, cannot be copy and pasted. Images for math notation are still widely practiced today. Another solution in to require readers to download some special plug-in. Another solution is to simply discard the web and use PDF instead.

In the past 10 years, huge amount of web technologies have developed, from blogs to wikis to instant messaging to twitter to interactive road maps to online videos to voice and video chats, but when it comes to math, sadly, the situation is rather stagnant. This is, of course, because relatively very few people need it. Perhaps 0.01% of web users.

What Wikipedia Use?

Wikipedia, which is one of the largest math encyclopedia online, uses images mixed with some html. (See: Help:Displaying a formula) More specifically, it uses a home cooked markup “<math>«body»</math>”, where the “body” is a subset of simple LaTeX code, and on the fly it processed by Texvc to generate images with some html markup.

Was this page useful? If so, please do donate $3, thank you donors!

2010-12-04

Time Mag, Julian Assange, video

Perm url with updates: http://xahlee.org/Periodic_dosage_dir/US_diplomatic_cables_leak.html

Time Mag cover 2010-12-13 Julian Assange

〈TIME's Julian Assange Interview: Full Transcript/Audio〉 (2010-12-01) Source www.time.com

A very interesting video: 〈WikiLeaks's Julian Assange Reviews TIME's Top 10 Leaks〉 Source www.time.com.

2010-12-03

what emacs/lisp tutorial you like to see?

Last poll results:

There's no emacs cookbook or elisp cookbook on the market. If i write a emacs book (published in printed form), are you likely to buy it?

Yes, if it's good. 41 (70%)
No, am not likely to spend money on emacs/elisp book. 14 (24%)
N/A. I don't use emacs... 3 (5%)

Total votes: 58

This week's poll:

For emacs tutorial, what would you like to see more?

1. More howto about GNU Emacs's bundled features and modes.
2. More tutorial about installing/use 3rd party modes and packages.
3. More tutorial on elisp programing for text manipulation.
4. More tutorial on writing modes (working with fonts, buffers, keys, menu, frames/windows, syntax highlight, shell IO...).

In the past i pretty much just wrote what i personally learned or need to do. I'm not sure so far which particular area i'm missing that people wish to see. In the above choices, basically the latter are more difficult, especially number 4.

«1. More howto about GNU Emacs's bundled features and modes.» This is pretty easy, for anyone who use emacs for years.
«2. More tutorial about installing/use 3rd party modes and packages.» This is not so difficult, but the question is which package people like to see? There are so many, and each 3rd party package's quality varies a lot. Some doesn't have any documentation.
«3. More tutorial on elisp programing for text manipulation.» This i've written a lot already, but mostly for processing html. Any thing you need?
«4. More tutorial on writing modes (working with fonts, buffers, keys, menu, frames/windows, syntax highlight, shell IO...).» This is not easy. I haven't wrote much about it. Of what i've written, such as basics of writing a major mode for language, i don't have a complete understanding, such as all the ways to use font-lock for syntax highlight or using overlay.

Result:

For emacs tutorial, what would you like to see more?

6 (24%) More howto about GNU Emacs's bundled features and modes.
3 (12%) More tutorial about installing/use 3rd party modes and packages.
11 (44%) More tutorial on elisp programing for text manipulation.
18 (72%) More tutorial on writing modes (working with fonts, buffers, keys, menu, frames/windows, syntax highlight, shell IO...).

Total Votes: 25

Hilarry Clinton: goto jail!

According one of the US Diplomatic Cables Leak, US Secretary of State Hillary Clinton has ordered espionage on UN officials.

Here's a Gardian article: 〈US diplomats spied on UN leadership〉 (2010-11-28) By Robert Booth and Julian Borger, guardian.co.uk. At: Source www.guardian.co.uk

A classified directive which appears to blur the line between diplomacy and spying was issued to US diplomats under Hillary Clinton's name in July 2009, demanding forensic technical details about the communications systems used by top UN officials, including passwords and personal encryption keys used in private and commercial networks for official communications.

It called for detailed biometric information "on key UN officials, to include undersecretaries, heads of specialised agencies and their chief advisers, top SYG [secretary general] aides, heads of peace operations and political field missions, including force commanders" as well as intelligence on Ban's "management and decision-making style and his influence on the secretariat". A parallel intelligence directive sent to diplomats in the Democratic Republic of the Congo, Uganda, Rwanda and Burundi said biometric data included DNA, fingerprints and iris scans.

Washington also wanted credit card numbers, email addresses, phone, fax and pager numbers and even frequent-flyer account numbers for UN figures and "biographic and biometric information on UN Security Council permanent representatives".

So, the United States, with its justice obsession and all, is now spying on not Iran, China, or some rogue enemy states, but United Nations, a org for world peace that US touts to have helped build. How ironic. And the US want things including passwords, credit card numbers, DNA, iris scan. This is getting serious.

I think UN or something should prosecute Hillary Clinton to the fullest extend of international law.

Emacs Lisp: writing a url-linkify

Perm url with updates: http://xahlee.org/emacs/elisp_html-linkify.html

Emacs Lisp: writing a url-linkify

Xah Lee, 2010-12-03

This page is a little lisp tutorial. A example on writing a function that transform the text under cursor on the fly. If you are not familiar with elisp, see: Emacs Lisp Basics.

Problem

I need to write a elisp command, so that, when pressing a button, the url under cursor, such as:

http://some.example.com/xyz.html

becomes this:

<a class="sorc" href="http://some.example.com/xyz.html"
title="accessed:2010-12-03">Source some.example.com</a>

And pressing another button, the link become this:

<a class="sorcdd" href="#" 
title="accessed:2010-12-03; defunct:2010-12-03; http://some.example.com/xyz.html">Source some.example.com</a>

Detail

In writing blogs, often you need to cite links. The links may be other blogs, news sites, or some random site. Many such url are ephemeral. They exst today, but may be a dead link few months later. Typically, if the url doesn't have a domain, but is hosted blog service site, it is more likely to go bad sooner.

For me, i write many blogs on xahlee.org, so have hundreds of links. When you update your pages years later, you find dead links like 〔http://someRandomBlog.org/importantToday.html〕, and may not remember what that link is about. No author, no title, no idea when that link was active or become dead. Sometimes, link is still good but the domain name owner of the link has changed, so the linked page may become porn site or been bought by domain squatters.

One partial solution is to add access date together with the link.

<p>blab blab news!
See <a href="http://some.example.com/xyz.html">here</a>!
(Accessed on 2010-12-03)</p>

With a access date, at least you know when the link was good. If the link went bad, you or your readers can at least try to see the link thru web archive site such as Internet Archive.

This way, at least you know when the link was good. However, this requires manual insertion of the date. It would be better, if the access date is somehow embedded in the link in some uniform format. HTML4 or even html5 does not have a way to embed access date. So, i decided to use the “title” attribute, like this:

<a class="sorc" href="..." title="accessed:2010-12-03">...</a>

This is not a ideal solution, because the “title” attribute is supposed to be title, not a date stamp. But in practice, i decided it's ok for me to adopt this solution.

I prefer this embedded access date approach because otherwise adding the access date besides the link text is distracting. If you have a paragraph with 3 or more links, each one says “accessed on ...”, that's very annoying.

When later on if i found a link is dead, i can press a button, and emacs will change the link to this format:

<a class="sorcdd" href="#" title="accessed:2010-12-03; defunct:2010-12-03; http://some.example.com/xyz.html">Source some.example.com</a>

Notice that the class value has changed from “sorc” to “sorcdd”. With proper css, the link will be shown as crossed out. Like this: Source some.example.com.

A uniform format to embed accessed date is good. Because, if later on HTML6 or other HTML Microformat has a way to add access date to links, i can easily write a script that change all my thousands of external links to the new format.

Solution

Here's the code:

(defun source-linkify ()
  "Make url at cursor point into a html link.
If there's a text selection, use the text selection as input.

Example: http://example.com/xyz.htm
becomes
<a class=\"sorc\" href=\"http://example.com/xyz.htm\" title=\"accessed:2008-12-25\">Source example.com</a>"
  (interactive)
  (let (url resultLinkStr bds p1 p2 domainName)

    ;; get the boundary of url or text selection
    (if (region-active-p)
        (setq bds (list (region-beginning) (region-end))  )
      (setq bds (bounds-of-thing-at-point 'url))
      )

    ;; set url
    (setq p1 (car bds))
    (setq p2 (cdr bds))
    (setq url (buffer-substring-no-properties p1 p2))

    ;; get the domainName
    (string-match "://\\([^\/]+?\\)/" url)
    (setq domainName  (match-string 1 url))

    (setq url (replace-regexp-in-string "&" "&amp;" url))
    (setq resultLinkStr
          (concat "<a class=\"sorc\" href=\"" url "\""
                  " title=\"accessed:" (format-time-string "%Y-%m-%d")
                  "\""
                  ">" 
                  "Source " domainName
                  "</a>"))

    ;; delete url and insert the link
    (progn (delete-region p1 p2))
    (insert resultLinkStr)))

The code is easy to understand. If you find it difficult, try reading this page Emacs Lisp: Writing a Wrap-URL Function, which has more explanation.

You can assign a hotkey for this command.

The following is the code to turn a link into a dead link format.

(defun defunct ()
  "Make the html link under cursor to a defunct form.
Example:
If cursor is on this line
<a class=\"sorc\" href=\"http://example.com/\" title=\"accessed:2008-12-26\">...</a>
 (and inside the opening tag.)

It becomes:
<a class=\"sorcdd\" href=\"#/\" title=\"accessed:2008-12-26; defunct:2008-12-26; http://example.com\">...</a>"
  (interactive)
  (let (p1 p2 linkStr url)
    (save-excursion

      ;; get the boundary of opening tag
      (search-backward "<a " ) (setq p1 (point) )
      (search-forward "\">") (setq p2 (point) )

      ;; get linkStr
      (setq linkStr (buffer-substring-no-properties p1 p2))

      ;; change the “class” attribute value
      (setq linkStr (replace-regexp-in-string "class=\"sorc\"" "class=\"sorcdd\"" linkStr t t)) 

      ;; (setq linkStr (replace-regexp-in-string "href=\"\\([^\"]+\\)\" +title=\"accessed:\\([^;]+\\)\""
      ;; (concat  "href=\"#\" title=\"accessed:\1; defunct:" (format-time-string "%Y-%m-%d") ";\2\"" )
      ;;  linkStr t t))

      ;; insert defunct date and url etc
      (with-temp-buffer
        (insert linkStr)
        (goto-char 1)
        (search-forward-regexp  "href=\"\\([^\"]+\\)\"")
        (setq url (match-string 1))
        (replace-match "href=\"#\"")
        (search-forward "\"")
        (search-forward "\"")
        (backward-char 1)
        (insert "; defunct:" (format-time-string "%Y-%m-%d") "; " url)
        (setq linkStr (buffer-string))))

    (delete-region p1 p2)
    (insert linkStr)))

Here's the css for the deadlink:

a.sorcdd:link:active, a.sorcdd:link:hover, a.sorcdd:visited:hover, a.sorcdd:visited, a.sorcdd:link
{color:black; cursor:text; text-decoration:line-through}

2010-12-01

Emacs Lisp: Replace String Based On File Name

Perm url with updates: http://xahlee.org/emacs/nav-bar.html

Emacs Lisp: Replace String Based On File Name

Xah Lee, 2006-11-29

This page is emacs lisp lesson on a real-world task. It show how emacs lisp is used in creating HTML navigational bar for a online book. Specifically, we need to insert proper text to each file for a set of files. You should be familiar with Elisp Language Basics.

The Problem

Summary

I want to insert the “Next Chapter”, “Previous Chapter” links to a series of HTML files that are chapters of a book.

This lesson essentially teaches you how to do a regex pattern replace by a function that return text based on the current file's name.

Detail

I have a lot of books in HTML form. Usually, the file names have this pattern: “chap1.html”, “chap2.html”, “chap3.html”... etc.

Each file is a chapter of a book. And in each file, I need to place a navigation bar, so that there's a Next Chapter and Previous Chapter links at the bottom of each page.

Normally, this can be done by writing a short Perl or Python script. The script will open the file, parse the file name so that it knows which chapter this file is. If the current file is “chapter3.html”, then the script will generate a string like this:

<div class="navbar">
<a href="chapter2.html">PREVIOUS</a>|
<a href="index.html">TOP</a>|
<a href="chapter4.html">NEXT</a>
</div>

For a person familiar with scripting languages, the job can be done in about 20 minutes. Basically, your script will traverse a directory and determine which files to process. For each file, your script will parse the file name and generate the navigation bar string appropriate for the file. Then, your script will open the file, insert the nav bar at the appropriate place, then close the file. Your script will need do a backup if you want it to be robust. With that, you'll also have to make sure that the file's owner, group, permissions etc meta data are kept intact.

In the end, some simple script can end up taking twice or trice the time you expected.

However, if you know elisp, you only need to write half of the code, since the file selection, file opening and reading, text decoding, backing up, saving and encoding, etc are all part of the emacs environment. All you really need to write is a elisp function that takes in a file name and returns the navigation bar string. This significantly saves your time. As a added benefit, you get to do this in a interactive, visual process. So, errors are much reduced, and you don't have to worry about your code making a mistake erasing parts of the file or missing some files.

Solution

First, mark the files you want to process in dired. Then, use “dired-do-query-replace-regexp” to do a find and replace operation on a particular string. For example, replace 「<body>」 with 「<body> someNavbarString」. (For a tutorial on using “dired-do-query-replace-regexp”, see: Interactively Find and Replace String Patterns on Multiple Files. )

The trick lies in your replacement string. You want to use a elisp function that returns the appropriate nav bar for the chapter. (so that the Next and Previous links are correct, according to the chapter number of the current file)

In emacs 22, there's a new feature that allows you to put a elisp function as your replacement string. This is done by giving the replacement string this form 「 \,(functionName)」, where functionName is the name of your function. So, if the function that returns the nav bar string is called “ff”, then in your replacement string you can say 「\,(ff)」

Here is the ff function:

(defun ff ()
  "Returns a navigation bar string with Prev and Next links based on the current file name."
 (interactive)
(let (fname navbarStr chapterNum )
  (setq fname (file-name-nondirectory (buffer-file-name)) )
  (setq chapterNum (string-to-number (substring (file-name-sans-extension fname) 4)))
  (setq navbarStr 
        (concat "<div class=\"nav\">★ <a href=\"" "chap"
                (number-to-string (- chapterNum 1))
                ".html" "\">◀</a> <a href=\"index.html\">▲</a> <a href=\"" "chap"
                (number-to-string (+ chapterNum 1))
                ".html" "\">▶</a> Flatland</div>"))
  navbarStr
  ) )

In the above code, the buffer-file-name returns the full path of the file of the current buffer. The file-name-nondirectory truncates the path to just the file name. The line

(setq chapter-num (string-to-number (substring (file-name-sans-extension fname) 4)))

extract the chapter number from the file name.

For a example of a online book with Next/Previous navigation bar, see: Flatland.

Emacs is beautiful!

Second Example

2010-12-01

Today, i need to do similar again. I have a dir with names like this:

x001-1.html
x001-2.html
x002-1.html
x002-2.html
x003-1.html
x003-2.html
...

These are pages for the novel Journey To The West (Monkey King). The first part of the file name is the chapter number. Each chapter has 2 html pages, indicated in the second part of the file name.

In each page, there's a nav bar code like this:

<div class="nav">
<a href="monkey_king.html" title="up">▲</a>
<a href="x002-2.html" title="next">▶</a>
</div>

It allows readers to go to the Table Of Contents page, or go to the next page. But it is missing a nav bar button to go to the previous page. I'd like to fix it, so it should be like this:

<div class="nav">
<a href="x001-2.html" title="previous">◀</a>
<a href="monkey_king.html" title="up">▲</a>
<a href="x002-2.html" title="next">▶</a>
</div>

So, the task is to add this:

<a href="x001-2.html" title="previous">◀</a>

to every page, and the link depends on the current file name.

Solution

Here's what i do to solve it, in the quickest way i know possible with emacs.

First, use find and replace on all files. Replace all occurance of

<div class="nav">
<a href="monkey_king.html" title="up">▲</a>

with

<div class="nav">
<a href="htnshtns" title="previous">◀</a>
<a href="monkey_king.html" title="up">▲</a>

The 「htnshtns」 is just a random string. We use a fixed random string so that we can do multi-file find replace one more time, where the find string will be this fixed string, and the replace string will be generated by a lisp function.

(If you don't know how to do find replace on multiple files, see: Interactively Find and Replace String Patterns on Multiple Files.)

Now, here's the lisp code quickly written:

(defun ff ()
  "..."
  (interactive)
  (let (fname mylist chapterNum pageNum chapterNumNew pageNumNew )

    (setq fname (file-name-nondirectory (buffer-file-name)) )

    (setq mylist (split-string (substring (file-name-sans-extension fname) 1) "-" ) )
    
    (setq chapterNum (string-to-number (nth 0 mylist) ))
    (setq pageNum (string-to-number (nth 1 mylist) ))

    (if (= pageNum 1) 
        (progn
          (setq chapterNumNew (- chapterNum 1))
          (setq pageNumNew 2)
          )
      (progn
        (setq chapterNumNew chapterNum)
        (setq pageNumNew 1)
        )
      )

    (concat "x" (format "%03d" chapterNumNew) "-" (format "%d" pageNumNew) ) ) )

So, with this code, i just call find replace, with find string 「htnshtns」, and replace value of 「\,(ff)」

Emacs is fantastic!

Proliferation of Computing Languages

Perm url with updates: http://xahlee.org/UnixResource_dir/writ/new_langs.html

Proliferation of Computing Languages

Xah Lee, 2008-07, 2008-11, 2010-01-26

There is a proliferation of computer languages today like never before. In this page, i list some of them.

In the following, i try to list some of the langs that are created after 2000, or become very active after 2000.

Lisp family or similar:

Mathematica. Computer algebra system background. Used mostly for math and research in science community.
newLISP. Lisp scripting style. Verdant community of new generation of hobbyist programers.
Arc. Paul Graham squeezing juice out of his celebrity status.
Qi. Common Lisp added with modern functional lang features. Primarily academic and language research.
Clojure. A new lisp dialect on Java platform. Poised as the next industrial lisp, but faces serious competition with other JVM based langs such as Scalar, Groovy.
Scheme, notably PLT Scheme. Used mostly for teaching.
(Dead. Dylan. Apple's re-invention of lisp for industrial programers, active in the 1990s.)

ML Family:

OCaml. Almost all current theorem proofing systems are based on.
Alice. Concurrent, ML derivative. Saarland University, Germany.
F#. Microsoft's offer, based on OCaml.

ML/OCaml derived Proof systems in industrial use:

Modern Functional langs:

Erlang. Functional, concurrent. Mostly used in a telecomunication industry for corcurrency and continuous up-time features.
Haskell Oldish, classic functional lang. Mostly used in academia for teaching and lang research.
Mercury. Logic, functional.
Q. Functional lang, based on term rewriting. Replaced by Pure.
Oz. Concurrent. Multiparadigm. Mostly used in teaching.

Perl Family or derivative:

PHP. Perl derivative for server side web apps. One of the top 5 most popular langs.
Ruby. Perl with rectified syntax and semantics. Somewhat used in industry. User numbers probably less than 5% of Perl or Python.
Perl6. Next generation of perl. In alpha stage.
Sleep. A scripting lang, perl syntax. On Java platform. sleep.dashnine.org/

On Java Virtual Machine:

Scala. A FP+OOP lang on Java platform as a Java alternative.
Groovy. Scritping lang on Java platform.

C derivatives:

ObjectiveC. Strict superset of C. Used as the primary language by Apple for OS X app dev.
C#. Microsoft's answer to Java. Quickly becoming top 10 lang with Microsoft's “.NET” architecture.
D. Clean up of C++.
Go. Google's new lang as improvement of C.

2D graphics related.

Scratch. Derived from SmallTalk + Logo. Primarily for teaching children programing.
ActionScript. Based on Javascript but for interactive 2D graphics. Quickly becomes top 10 lang due to popularity of Adobe Flash.
Processing. 2D graphics on Java platform. Primarily used for art and teaching.

Misc:

JavaScript. Mostly for web browser scripting. One of the top 10 most popular lang. Quickly becoming the standard lang for general purpose scripting. (used by Adobe Flash, Dashboard Widgets, scripting Adobe PDF files, scripting Microsoft Windows, scripting Java)
PowerShell. A modern shell. A scripting lang desgined also for interactive use. Syntax similar to Perl and unix shell tools, but based “.Net”.
Tcl. Scripting, especially for GUI.
Lua. Scripting, popular as a scripting lang in games.
Linden Scripting Language. Used in virtual world Second Life.

Some Random Thoughts

Following are some random comments on comp langs.

Listing Criterion and Popularity

In the above, i tried to not list implementations. (e.g. huge number of Scheme implemented in JVM with fluffs here and there; also e.g. JPython, JRuby, and quite a lot more.) Also, i tried to avoid minor derivatives or variations. Also, i tried to avoid langs that's one-man's fancy with little followings.

In the above, i tried to list only “new” langs that are born or seen with high activity or awareness after 2000. But without this criterion, there are quite a few staples that still have significant user base. e.g. APL, Fortran, Cobol, Forth, Logo (many variants), Pascal (Ada, Modula, Delphi). And others that are today top 10 most popular langs: C++, Visual Basic.

The user base of the langs differ by some magnitude. Some, such as for example PHP, C#, are within the top 10 most popular lang with active users. Some others, are niche but still with sizable user base, such as LSL, Erlang, Mathematica. Others are niche but robust and industrial (counting academia), such as Coq (a proof system), Processing, PLT Scheme, AutoLISP. Few are mostly academic followed with handful of researchers or experimenters, Qi, Arc, Mercury, Q, Concurrent Clean are probably examples.

For those of you developers of Java, Perl, Python for example, it would be fruitful to spend a hour or 2 to look at the Wikipedia articles about these, or their home pages. Wikipedia has several pages that is a listing of comp langs, of which you can read about perhaps over 2 hundreds of langs.

Why The List

I was prompted to have a scan at these new lang because recently i wrote a article titled Fundamental Problems of Lisp, which mentioned my impression of a proliferation of languages (and all sorts of computing tools and applications). Quote:

10 years ago, in the dot com days (~1998), where Java, Javascript, Perl are screaming the rounds. It was my opinion, that lisp will inevitably become popular in the future, simply due to its inherent superior design, simplicity, flexibility, power, whatever its existing problems may be. Now i don't think that'll ever happen as is. Because, due to the tremendous technological advances, in particular in communication (i.e. the internet and its consequences, e.g. Wikipedia, youtube, youporn, social networks sites, blogs, Instant chat, etc) computer languages are proliferating like never before. (e.g. erlang, OCaml, Haskell, PHP, Ruby, c#, f#, perl6, arc, newLISP, Scala, Groovy, Goo, Nice, E, Q, Qz, Mercury, Scratch, Flash, Processing, ..., helped by the abundance of tools, libraries, parsers, existence of infrastructures) New langs, basically will have all the advantages of lisps or lisp's fundamental concepts or principles. I see that, perhaps in the next decade, as communication technologies further hurl us forward, the proliferation of langs will reduce to a trend of consolidation (e.g. fueled by virtual machines such as Microsoft's .NET.).

Creating A Lang Is Easy

In general, creating a lang is relatively easy to do in comparison to equivalent-sized programing tasks in the industry (such as, for example, writing robust signal processing lib, a web server (e.g. video web server), a web app framework, a game engine ...etc.). Computing tasks typically have a goal, where all sorts of complexities and nit-gritty detail arise in the coding process. Creating a lang often is simply based on a individual's creativity that doesn't have much fixed constraints, much as in painting or sculpting. Many langs that have become popular, in fact arose this way. Popularly known examples includes Perl, Python, Ruby, Perl6, Arc. Creating a lang requires the skill of writing a compiler though, which isn't trivial, but today with mega proliferation of tools, even the need for compiler writing skill is reduced. (e.g. Arc, various langs on JVM. (10 years ago, writing a parser is mostly not required due to existing tools such as lex/yacc))

Some lang are created to solve a immediate problem or need. Mathematica, Adobe Flash's ActionScript, Emacs Lisp, LSL would be good examples. Some are created as computer science research byproducts, usually using or resulting a new computing model. Lisp, Prolog, SmallTalk, Haskell, Qi, Concurrent Clean, are of this type.

Some are created by corporations from scratch for one reasons or another. e.g. Java, Javascript, AppleScript, Dylan, C#. The reason is mostly to make money by creating a lang that solves perceived problems or need, as innovation. The problem may or may not actually exist. (C# is a lang created primarily to overrun Java. Java was created first as a lang for embedded devices, then Sun Microsystems pushed it to ride the internet wave to envision “write once run everywhere” and interactivity in web browser. In hindsight, Java's contribution to the science of computer languages is probably just a social one, mainly in popularizing the concept of a virtual machine and automatic memory management (so-called Garbage Collection), and further popularizing OOP after C++.)

Infinite Number Of Syntaxes And Semantics

Looking at some tens of langs, one might think that there might be some unifying factor, some unifying theory or model, that limits the potential creation to a small set of types, classes, models. With influence from Stephen Wolfram book “A New Kind of Science” (see: Notes on A New Kind of Science) , i'd think this is not so. That is to say, different languages are potentially endless, and each can become quite useful or important or with sizable user base. In other words, i think there's no theoretical basis that would govern what languages will be popular due to its technical/mathematical properties. Perhaps another way to phrase this imprecise thought is that, languages will keep proliferating, and even if we don't count langs that created by one-man's fancy, there will still probably be forever birth of languages, and they will all be useful or solve some niche problem, because there is no theoretical or technical reason that sometimes in the future there would be one lang that can be fittingly used to solve all computing problems.

Also, the possibilities of lang's syntax are basically unlimited, even considering the constraint that they be practical and human readable. So, any joe, can potentially create a new syntax. The syntaxes of existing langs, when compared to the number of all potentially possible (human readable) syntaxes, are probably a very small fraction. That is to say, even with so many existing langs today with their wildly differing syntax, we probably are just seeing a few pixels in a computer screen.

Also note here all langs mentioned here are all plain-text linear ones. Spread sheet and visual programing langs would be example of 2D syntax... but i haven't thought about how they can be classified as syntax. (nor do i fully understand the ontology of syntax )

Just some extempore thoughts.

2010-09-10

New programing languages.

Pure (programming language) @ http://code.google.com/p/pure-lang/ (a pattern matching, term-rewriting lang.)
Vala (programming language) @ http://live.gnome.org/Vala (improved C)
Genie (programming language) @ http://live.gnome.org/Genie (like Vala, but python-like syntax)

2010-11-07

Discovered a new programing language. Factor (programming language).

See: Point Free Programing.

what do programers do all day?

What Do Programers Do All Day?

In the past few years, i've been wondering why software engineering is so cumbersome and time consuming. Usually, something that's conceptually trivial, something you think can be done in 10 minutes, ends up hours. In the past few years, i tried to write a account of it whenever this happened to me. See these essays:

(and it is quite time consuming to document this experience. Each essay above usually takes 4 hours to write.)

In all the above cases, they are typically about installing a package, setting a preference, compress a file, get spelling checker to work, setting a environment variable. If you know the technologies well, usually you'd imagine it's just 5 to 10 minutes job. But in each case, i ends up spending hours or half of a day.

In the past week, another 2 such situation happened. One is renaming a file in Subversion (svn) by changing a letter case of a file. See the problem described here: http://groups.google.com/group/ergoemacs.

The other problem is about writing a script to zip up a directory. See: Unix zip Utility Path Problem.

You might have worked in the software industry as a programer for years. Sometimes, you might wonder, “what i've actually been doing in the past few months?”. In a day job situation, half of the time is spent on meeting and communication and other miscellaneous, non-nondescript stuff. Perhaps you actually get to code only for 1/3 of your working time. Of that time, a large part, probably more than half of the time, is probably spent on getting something conceptually trivial to work, among the hacks, spaghetti code, non-existent documentations, problem in tools, etc.

As a programer, i think intuitively we think that programing is actually writing new functions, new libraries, something that doesn't exist before. But i think in reality, more than half the time a working programer spend daily on coding, is actually about fixing stupid problems, working around tools, using knowledge about the environment and past experiences.

2010-12-07

2010-12-06

How Mathematica does Unicode?

Mathematica's Named characters 「\[Name]」

Map Between Unicode and Named Chars

Syntax for Unicode Char 「\:nnnn」

Some Named Chars has Builtin Meaning

Alias Shortcut for Named Chars

List of Named Char with Special Meanings

Inputing Special Chars

2010-12-05

Tools to Display Math on Web

FireMath, a MathML Editor for Firefox

MathJax

Detexify

Mathematica

Other

Rant

What Wikipedia Use?

2010-12-04

2010-12-03

Emacs Lisp: writing a url-linkify

Problem

Detail

Solution

2010-12-01

Emacs Lisp: Replace String Based On File Name

The Problem

Summary

Detail

Solution

Second Example

Solution

Proliferation of Computing Languages

Some Random Thoughts

Listing Criterion and Popularity

Why The List

Creating A Lang Is Easy

Infinite Number Of Syntaxes And Semantics

What Do Programers Do All Day?

Recent Comments

About Me

Subscribe

Follow by Email

FeedBurner FeedCount

Category

Article by Date

Pageviews last month

Popular Posts