File Aliases Considered A Plague

perm url with updates: http://xahlee.org/UnixResource_dir/writ/hardlink_softlink_alias_junction_plague.html

File Aliases Considered A Plague

Xah Lee, 2009-09-01

Learned today about Windows's Junction. Basically, it's a file aliasing mechanism in NTFS much like unix's hardlink, softlink, and also Mac OS X's HFS+'s alias.

All these i have avoided like a plague in software. They create a lot problems.

I noticed this Windows juncture because i am using rsync thru cygwin to copy files in my Documents folder to my Mac. Rsync keeps telling me permission denied. Here's the error:

xah@xah-PC ~
$ rsync -z -r -v -t --exclude=".DS_Store" --delete --rsh="ssh -l xah" ~/Documents/ xah@169.254.153.147:~/Documents_PC/
Password:
building file list ... rsync: opendir "/cygdrive/c/Users/xah/Documents/My Music" failed: Permission denied (13)
rsync: opendir "/cygdrive/c/Users/xah/Documents/My Pictures" failed: Permission denied (13)
rsync: opendir "/cygdrive/c/Users/xah/Documents/My Videos" failed: Permission denied (13)

In my “Documents” folder, i don't have a folder named “My Music” or such. But of course, apparantly i do. They are hidden. In Explorer, you won't see these files in your Documents dir (unless you have turned on viewing system files in Folder Options).

These hidden files are links to different dirs: “~/My Pictures” and “~/My Videos”. I get perm denied because i haven't made the link destination folders to be shared.

If you use PowerShell “dir -force”, you see:

    Directory: C:\Users\xah\Documents

Mode                LastWriteTime     Length Name
----                -------------     ------ ----
d--hs         5/23/2009   7:15 PM            My Music
d--hs         5/23/2009   7:15 PM            My Pictures
d----         6/18/2009   8:25 PM            My Received Files
d--hs         5/23/2009   7:15 PM            My Videos

Note those “hs” there, probably meaning some Hidden and System attributes in NTFS. If you use cmd.exe's “dir /A”, you see:

08/31/2009  06:27 AM    <DIR>          emacs_stuff
05/23/2009  07:15 PM    <JUNCTION>     My Music [C:\Users\xah\Music]
05/23/2009  07:15 PM    <JUNCTION>     My Pictures [C:\Users\xah\Pictures]
06/18/2009  08:25 PM    <DIR>          My Received Files
05/23/2009  07:15 PM    <JUNCTION>     My Videos [C:\Users\xah\Videos]

At that, the word JUNCTION give me a hook to search on the web, and found the answer.

All these different redirect mechanisms in unix, Mac, and now i know Windows, creates lots of complexities and problems. Problems in security, in copying deleting dirs, in deceptive dir structure. One thing i particularly hate is unix's “hard link”. Though, i guess in some situations they are convenient and probably the best solution.

The rsync problem is solved by adding “--exclude="**/My *" ”. (I was using “--exclude="*/My *" ” to work around the perm denied error, but no go. Annoying, and i lived with it for days. Eventually, i bite the bullet to spend time to resolve it, which lead me to this juncture business. For rsync, it turns out i need two asterisks.)

2009-09-30 Addendum: See also this blog Why doesn't Explorer have an interface for creating hard links? by Raymond Chen, at Source.

Vista VirtualStore

Xah Lee, 2010-01-28

Discovered Windows Vista's “File and Registry Virtualization”, or, “VirtualStore”.

In 2009-09, i'm installing a new version of ErgoEmacs, which me and David Capello is developing. The new version didn't work for me, because, apparantly some previous version of elisp files are still in my program installation dir at “C:/Program Files (x86)/ErgoEmacs”. After reinstalling several times, still the same problem. I was quite frustrated. After hours of looking into the problem, apparantly the problem isn't the new release, but just on my machine. It seems there's some file caching going on, but just couldn't figure out what. At one point, i suspected that my file system is corrupted. This whole incidence has caused me some 20 hours. (See this discussion at ergoemacs forum: Source)

Today, David wrote to me pointing out that the problem could be Vista's file virtualization. Bingo, that was it. Here's what it means.

In Vista, dir path such as “C:\Program Files (x86)\” is not supposed to be accessible for normal users. But a lot old programs still write to that dir for user data. So, when the program tries to do that, and OS detects that the user has no privilege, it will write to “C:\Users\‹your account name›\AppData\Local\VirtualStore\Program Files (x86)\” instead. This means, some application that view directories will get confused or misleading. For example, in emacs dired, it shows me that i have several files in: “c:\Program Files (x86)\ErgoEmacs\bin”, but when you use Windows Explorer or PowerShell or cmd.exe to look, the directory doesn't exist.

The masking of files also is the source of a mysterious problem i have when trying to work with IntelliTypo software (see: Microsoft IntelliType Hacks), where i worked for hours but some expected result mysteriously does not happen.

The file virtualization happens to registry too. For detail, see:

  • Technet: Inside Windows Vista User Account Control (2007-06), by Mark Russinovich. Source
  • Technet: Achieve the Non-Admin Dream with User Account Control (2006-11), by Alex Heaton. Source
  • Blog: Vista Virtual Store or “Where did my files go?” (2008-09-29), by Darrell Source
  • Blog: File and Registry Virtualization – the good, the bad, and the ugly (2005-12-19), by Jerry. Source.

Popular posts from this blog

11 Years of Writing About Emacs

does md5 creates more randomness?

Google Code shutting down, future of ErgoEmacs