The Complexity And Tedium of Software Engineering

Perm url with updates:

The Complexity And Tedium of Software Engineering

Xah Lee, 2009-06-02

This page is a blog of a experience of few examples that illustrates some seemingly trivial task can become quite tedius and complicated in the software industry.

A Complexity with Emacs

Discovered a emacs problem.


  • Seems that emacs 23 will have a problem loading a css-mode written by Stefan Monnier
  • The css-mode.el file does not contain any sort of version number, which makes the problem worse.

Detail: I have a Mac running emacs 22 with OS X, and i have PC running emacs 23 and Windows Vista. When i use the emacs 23 to load css mode, it gives this error: “if: Wrong type argument: integerp, (0 . 8)”.

The problem seems simple in retrospect, but wasn't simple at all when you trying to get things done and things don't work as expected. Here's the story.

Emacs 22 does not have a css mode, emacs 23 does. There's one css mode written by Stefan. I've been using it on the Mac for the past couple of years. The same package is now bundled into emacs 23, which i'm using on PC. However, the code in the 2 files are not identical. I have my emacs setup to load css mode. Since i am migrating to PC, alone with my whole emacs system, and am setting up a Mac and PC and network that'd let me work with either machine harmoniously. On the PC, when i start css mode, it gives error, but not always. You don't know what's wrong. It could be a fuckup in the emacs distro i'm using on PC (which is emacsW32), or it could be emacs 23 problem (emacs 23 is still in beta), or it could be something in my emacs customization that works perfectly well on the Mac but not on the PC with whole new environment. Eventually, i realized that's because sometimes i started plain emacs 23 without loading my own setup, it was using the bundled css mode, so no error, but when i loaded my own emacs setup, it gives error. This seems still simple in retrospect, but wasn't then. I added a version check to my emacs init file(s), so that if emacs is 23, don't load my css mode. The next day, same symptom occurs. Eventually I realized that's because the load path is set so that my own version of css mode comes before the bundled css mode, so that, when i load css, again my old version is loaded. Eventually, the solution is to check if css mode bundled with emacsW32 runs ok on emacs 22 on my Mac; if so, simply use that as the single source for my Mac and PC. When doing this work on checking what versions are the files, i realized that the 2 css mode's files don't contain version number at all. All this takes 5 minutes to read, but was one of the gazillion seemingly trivial issues and problems when setting my Mac/PC networking environment with cygwin and all. This took me one week, and haven't gotten to wholly converting my Mac files to PC. Added the time to google for for the answer, possibly write a report to the emacsers, etc, all in all, i'd say this problem caused me 4 hours.

Here's the emacs version i'm using. On the Mac: “GNU Emacs 22.2.1 (powerpc-apple-darwin8.11.0, Carbon Version 1.6.0) of 2008-04-05 on”. On Windows “GNU Emacs (i386-mingw-nt6.0.6001) of 2009-05-28 on LENNART-69DE564 (patched)”

URL Encoding

Discovered a subtle issue with automating url encoding. In url, if you have the ampersand “&” char, and if this url is to be inside a html doc as a link, then, there's no automated procedure that determines correctly whether the char should be encoded as “%26” or “&”. If the char is used as part of the file name or path, then it should be encoded as “&”, but if it is used as a separator for CGI parameters, then it should be encoded as “%26”.

The rule for Percent encoding the char is that the ampersand is a reserved char, therefore it must be percent encoded if it is used for normal file path names. So, when it is NOT used as part of path names, but used as CGI parameter separaters, with GET method of HTTP request, then it must be left as it is. Now, in html, the ampersand char must be encoded as html entities “&” when adjacent chars are not space (basically). So, it becomes “&”.

In summary, if the url in html link contains “&”, and the char is a CGI param separator, then encode it to “&”, else, encode it as “%26”, and my discovery is that the purpose of the char used in url cannot be syntactically determined with 100% accuracy.

Of course, in practice, all this matters shit. Just use “&” plainly and it all works in 99.999% of situations.

Unicode File Names

Unison and Unicode File Names

Summary: Unison does not support unicode chars.

When using Unison version 2.27, to sync files from Mac to PC, the file name on the mac is “赵薇_flag.jpg”, it is copied to PC using Unison 2.27, and the file name became “èµµè-╪_flag.jpg” when viewed under Windows, but the file name shows up correctly when viewed in EmacsW32.

Mac version: 10.4.11, Windows Vista SP1.

This may indicate that Unison encode file names in utf8, or just ascii. Indeed, it is said on Wikipedia that unicode have problems with non-ascii file names.

When the file is copied from Mac to Windows or Windows to Mac, operating either on Windows or Mac as the local machine, using either OS's file manager, the file name is copied correctly.

Setting up Unison itself is not so trivial. It is trivial in concept, but actually took hours. I have Unison on my Mac installed, and use it few times a year, since about 2006, so i'm familiar with it. On PC, first you have to install cygwin. I know there are Unison binaries for Windows but since i use cygwin, so my first choice is staying with cygwin, since it contains the whole unix environment. Installing cygwin is another story, but once you installed Unison in cygwin, and tried to test sync my Mac and PC, you run into the problem that sshd must be turned on in one of the machines. Namely, sshd should run on the “remote” machine. (setting up a local network among Win and Mac is another complex and tedious story) Then, there's the issue of deciding which machine you want sshd to run or both. On the Mac, i can turn on sshd in a minute. On Windows, i'm not sure. I'm not sure if Windows Vista home edition provide ssh server, and am not sure how to turn it on if so. As far as i know, Windows Vista Home does not come with a ssh client. In the process, also realize that firewall must be turned off for ssh port. So, you spend 30 min or possibly hours (here and there) reading or probing with Windows Firewall control panel and whatnot other admin tools. After a while, i decided it's easier just to turn on sshd on the Mac then Unison from the Windows side to the Mac. At least, have this direction work first, and when that works, i can play with the other direction. After all this done, i tried to Unison, but Unison reports that the Unison version on my Mac and PC is not the same, so it doesn't work. Geeze. The one on my Mac turns out to be Unison 2.13.x, and the one i have in Cygwin is 2.31.x. Now, i figured that with each release of Unison, it probably obsolete some older versions. So, back to digging Unison docs and the web. The simplest solution comes to mind is to update my Unison on my Mac to latest, of which, the unison on fink haven't been updated for a couple of years. I use Fink, and am fairly familiar with using Fink. However, after “fink selfupdate” etc, then “fink desc Unison”, the version there reported seems to be 2.13. Then, searching web on fink home page indicates they have unisone-2.27.57-1007, for my OS 10.4 PowerPC. So, why doesn't it show up in my fink? I remember i had a case last year where fink had obsolete mirror databases, a fault entirely on their end.

After spending maybe some more 30 min, i decided to install Unison from a website, binary download. After that done, i got Unison 2.27.x on the Mac. I tried to sync again, still no go. So, it seems like that the Unison version number must be the same or very close. Checking on Unison website, it looks like the current stable release is 2.27.x, so, my cygwin's 2.31.x is actually a beta. Damn. So, now back to cygwin. Luckily, it appears there are several version of Unison there to be installed, and i easily installed 2.27. Then, finally, test sync is successful. Now, i go back to get my files ready in a state to be synced. (long story there. See: Perl Script for Removing Mac Resource Fork and Switching from Mac/Unix To PC/Windows ) When finally i'm ready to Unison, then there's the Chinese character problem!

Emacs on Windows and Unicode File Names

When using emacsW32, dired has problem dealing with files with chinese char on remote machine. Consequently, when trying to copy, delete, etc, the behavior may be dangerous.

e.g. i have this file 林志玲.jpg on a Mac. I use emacsW32 on Windows to view it thru network. (e.g. the path is // ... ) the file name shows up as “_viu0y~a.jpg” in dired.

“GNU Emacs (i386-mingw-nt6.0.6001) of 2009-05-28 on LENNART-69DE564 (patched)”, Mac version: 10.4.11, Windows Vista SP1.

Popular posts from this blog

11 Years of Writing About Emacs

does md5 creates more randomness?

Google Code shutting down, future of ErgoEmacs