2010-09-18

Keyboarding & Macros in Mac and Windows (How to Increase Your Computer Operation Efficiency)

Perm url with updates: http://xahlee.org/mswin/keyboard_macros.html

Keyboarding & Macros in Mac and Windows (How to Increase Your Computer Operation Efficiency)

Xah Lee, 2009-05-26, ..., 2010-09-18

I'm a operation efficiency nerd. When operating a computer, you have to switch apps, click links/checkbox/buttons, invoke menus, open/close window or tab, adjust volume, change songs, switch input focus, switch fields, input/edit text, or perform pointer oriented input such as in a image editing software. (here, we are excluding other forms of input such as voice) In general, all these are inputs, done by using input systems, a combination of software and hardware. Typically, the hardware is just keyboard and mouse (or touchpad or trackball). The software part is the interfaces used in the OS and various apps.

Operation efficiency means you are able to achieve a input task with the least number of units of physical exertion, basically means less keystrokes, less mouse travel, and most importantly, less brain work to carry the task out. (brain work, such as the need to eyeball a process, is actually the most costy.)

Example: Open a URL in Browser

As a concrete example, let's say you have a url in a text editor. Say you want to view that url in a browser. There are many ways to do that. One way, is to use mouse to select the url, right click to select the copy menu, switch or launch a browser (many ways to do this), put focus on the url field, then right click to paste it, then hit Enter to load the url. This method, is rather most typical. Here's the detail of each step.

Selecting Text and Copy

  • Selecting text: Move your mouse and aim the pointer at the beginning of text you want to copy. Press and hold down left mouse button. Move your mouse and aim the pointer at the end of the text. Release the mouse left button.
  • Copy Text: Move the mouse and aim the pointer at the selected text. Press the right mouse button (to bring up the contextual menu). Move the mouse and aim the pointer at the menu item “Copy”. Press the left mouse button.

A much more efficient method for copying text, is to simply hold the Ctrl, then press the c key, then release the Ctrl key. With this method, you don't have the visual/muscle coordination task of aiming the mouse pointer, which is required 2 times with the mouse method. Also, you do not have the visual selection process of choosing a menu, which requires you to scan the menu texts. Copying a text by 【Ctrl+c】 is purely a mechanical process, involves basically just a single step, and for those touch-typists, it's in muscle memory.

A even more efficient method that eliminate the “Selecting Text” step, is that in some editors such as Microsoft Visual Studio, you can press 【Ctrl+c】, and if there's no text selection, the whole line will be copied. So, it saves you the operation of select the line first.

As a example of a less efficient way, is to actually type the url text in the browser field instead of copy and paste. Some people actually do this.

Open a Web Browser Window

Now, here's several ways to open a web browser window.

  • Method 1. Launch browser by Quick Launch: Move mouse and aim the pointer at the browser icon in your Quick Launch bar in your Task Bar. Press left mouse button.
  • Method 2. Launch browser by Start Menu: Press the Start key on the keyboard. Move mouse and aim the pointer a menu item that is your browser.
  • Method 3. Switching to browser: Press and hold down the Alt key. Press Tab key. Press Tab, visually check if the highlighted icon is the app you want. Repeat the previous step until your browser is highlighted. Release the Alt key.
  • Method 4. Keyboard Special Button. Press the keyboard special button named “Web/Home”.

Of the above methods, the “web/home” keyboard button press is the most efficient. This button is available in many of today's keyboards since about early 2000. For exmaple Microsoft Natural Ergonomic 4000. Or, you could've setup one of your function keys to launch/switch browser. (See: AutoHotkey Basics.)

The second most efficient method above is launching by Quick Launch. Clicking a button on a button bar, and is more efficient than going thru menu.

Visit a URL in browser

  • Select the URL Field in Browser by Mouse: Move mouse and aim the pointer at the url field. Press the left mouse button. (in modern browsers, this usually select the whole url in the field. Otherwise, you have to press 【Ctrl+a】 to select the text.)
  • Pasting Text by Mouse: Press the mouse right button. Mov move the mouse and aim the pointer on the contextual menu item “Paste”. Press the Left mouse button.

Now, consider the following method by keyboard:

  • Select the URL Field in Browser by Keyboard: Press and hold down Ctrl. Press l key.
  • Pasting Text by Keyboard: Press and hold down Ctrl. Press v key.

The keyboard is much more efficient, again because it involves less hand/eye coordination, and no brain visual processing of scanning text in menu.

Now, let's consider a example case of extreme efficiency. If your text editor is Emacs, you can skip the steps of selecting url, copy the url, launching the browser, paste the url. All you have to do is place the cursor on the url, then call the command “browse-url”. This you can do by Typing 【Alt+x】, then type “brow”, then Tab to complete the name, then press Enter. This will copy the url, launch or switch to your default browser, and open the page of that url. And if you need to do this often, some people have setup a single key such as F8, so that pressing it does the whole thing.

How to Increase Your Computer Operation Efficiency

Here's some advice for Mac and Windows users for operational efficiency. The following list is roughly ordered from the most important tips first.

Learn Touch Typing

Most programers who don't touch type will start to groan when seeing this. Well, sorry fellas, but this is actually the most important tip for efficiency. Bite the bullet.

There is a question about whether to learn the Dvorak Keyboard Layout. If you don't already touch type, I highly recommend learning touch type with Dvorak. The drawback is that you won't be able to hop to any keyboard and start touch typing, at public libraries, or co-worker's computers, but you can always hunt-n-peck on QWERTY.

Use A Good Keyboard

Use a good keyboard, preferably Microsoft's ergonomic ones. For tips on choosing a good keyboard, see: Computer Keyboards Gallery.

I also recommend a keyboard with many special application launching keys and multimedia keys. The app launching keys are big time savers, and you can customize them to open your frequently used folders too. The multimedia keys such as Play/Pause, Mute, Next Song, are extremely useful if you use your computer to listen to music. Most of today's keyboard have these special keys.

I recommend Microsoft's Ergonomic 4000. It is a split, ergonomic, full-key, full-featured keyboard.

Master OS's shortcut system

Master OS's shortcut system. See:

“Master” means you know all the shortcuts in the OS, and you've examed and experienced and have decided which are better for you.

Learn The OS's Keyboard Config Settings

Learn the OS's ways to config the keyboard. On Windows, it's the keyboard icon under Window's Control Panel. On Mac, it's the Keyboard icon under System Preference. They allow you to tweak global keyboard hotkey settings, to various degrees.

If you have a Microsoft keyboard, i recommend the bundled IntelliType software. (comes in for both Windows and Mac versions) It lets you remap or disable keys or define macros to some extent.

Get A Keyboard Macro Tool

Use macros and other key-remapping or app launching software. On the Mac, i recommend Quicksilver. On Windows, i recommend AutoHotkey. Both are free. (See: AutoHotkey Basics)

Note that there are also very good commercial ones. I've used QuicKeys on the Mac from 1991 to 2002. QuicKeys is now sold by http://startly.com/ (for Mac and Win). On the Mac, there's also Keyboard Maestro from http://www.keyboardmaestro.com/main/ i've used.

Learn A Scripting Lang

On the Mac: Automator and AppleScript, and Bash. See also:

On Windows, learn PowerShell. (See: PowerShell Tutorial) (There's also Visual Basic and cmd.exe, but PowerShell is the new .NET based shell that is more flexible, powerful, and suitable for automation and sys admin.)

Keyboard macros, launchers, shells, scripting langs, all work together as automation tools. For example, you often want a hotkey to invoke some script.

Learn OS's Keybinding API

Learn the OS's keybinding system or API. On the Mac it's the DefaultKeyBinding.dict system. It lets you re-define keys across OS to some extent by a simple config file. See: How To Create Your Own Keybinding In Mac OS X.

I'm not sure what's on Windows. Registry has a lot to do with keybindings at least. There are several tools too, see: How To Create Your Own Keybinding In Microsoft Windows.

Efficiency vs Effort in Learning and Setup

Note that efficient methods usually require you to spend time to learn the key shortcuts, learning the keyboard macro software, or learning scripting languages, and setting up your system.

If you spend only 1 hour a day on computer, apparently learning all the efficient methods is not worth it. How much time you invest in learning and setup these efficient methods depends how often you need to perform the task. Also, how easy it is to learn and setup a new system.

Was this page useful? If so, please do donate $3, thank you donors!

newLISP

Discovered this nice article on newLISP, which gave me my first intro of the lang:

  • 〈How newLISP Took My Breath (And Syntax) Away〉 (2010-01-08). By Greg Slepak. At: Source

See also this comparison: 〈Lisp: Common Lisp, Scheme, Clojure, Emacs Lisp〉 @ http://hyperpolyglot.wikidot.com/lisp.

If you want a quick intro by slide show, see: http://www.nuevatec.com/manual-intro.html.

Baby carrots

Discovered that Baby carrots are basically a mature carrot cut and peeled into small pieces, not really a special plant. Though, these days there are probably genetically modified ones more suitable for baby-cut production.

Munch some, good for your body!

2010-09-17

Art of Armaeater Tokiya

vanilla mania-s

by Armaeater Tokiya Source

perm url: Art of Armaeater Tokiya.

top 15 movies in your head

Meme from Michael Underwood.

The rules: Don't take too long to think about it. Fifteen movies you've watched that will always stick with you. List the first fifteen you can recall in no more than fifteen minutes.

Well, i only got 10 off my head.

Much more of my fav are here Xah's Favorite Movies.

java applet for Voronoi Diagram

A fantastic java applet to draw Voronoi diagram interactively. Very nice.

〈Voro Glide〉@ Source

See also: Great Math Programs.

2010-09-15

Iraq War Photos

Perm url with updates: http://xahlee.org/Periodic_dosage_dir/t2/iraq_pixra.html

Iraq Photos (page 1)

The following photos are copied from http://home.wi.rr.com/davef/iraq.htm on 2004-04-27. The html on the site has title “Some Picture a friend brought back from Iraq”. Otherwise there's no info. The site is gone as of 2004-12.

Iraq Photos Iraq Photos Iraq Photos Iraq Photos Iraq Photos Iraq Photos Iraq Photos Iraq Photos Iraq Photos Iraq Photos Iraq Photos Iraq Photos Iraq Photos Iraq Photos
1, 2, 3, 4, 5, 6

Opera Web Browser Pain

Perm url with updates: http://xahlee.org/js/Opera_browser_problems.html

Opera Pain; Opera Browser Problems

Xah Recommends:
Kindle
Amazon Kindle. Read books under the sun. Review

Xah Lee, 2010-09-14, 2010-10-29

This page documents some usability problems of Opera web browser.

No Drag and Drop

You cannot “drag and drop” images to a folder on desktop to save it. They insist on using different keyboard shortcuts than every browser.

Non-standard Keys

For example, 【Ctrl+Tab】 is for switching tab. But in Opera, it pops up a tab menu instead. Normally, i can quickly switch to different tabs and stop at the one i want, visually looking at each tab for fraction of a second. With Opera, i have to read the titles or small thumbnails.

The 【Ctrl+w】 key lets me close a tab in IE, Firefox, Chrome, Safari, but doesn't work in Opera.

Ctrl+Click】 lets me open a link in a new tab in all browsers. Not so with Opera. It opens the new link in the current tab, same as simple click.

Non-standard Tab Ordering

Also, when closing a tab, instead of showing the tab next to the closed one, Opera tries to be smart and jump to some other tab, something like the one opened before the one you closed. Very confusing. Same thing happens when switching to next/prev tabs using their 1 and 2 key shortcuts on the numerical keypad.

Non-standard Zoom-in Keys

Ctrl++】 does not zoom in.

Does not support Scroll-wheel Side-push

Nudging my mouse's scroll wheel to the sides should scroll window horizontally. It does not work in Opera, but it does in IE, Chrome, Safari, Firefox. While Opera is supposed to suppose some fancy “gestures”. Gesture my ass.

Weird Numerical Keypad Key Behavior

The number keys on the numerical keypad do weird things. For example, 7 seems to zoom out to 20%. 8 zooms to 120%. 5 turns the current tab into a window, inside the main window. 1 and 2 is for switching prev/next tab.

The 1 and 2 key for switching tabs is convenient. You can repeatedly press one of these keys to get to the tab you want. However, if you landed on a empty page (new tab), the key starts to insert text in the url field.

Weird Fancy Features

In 10.5x, it sports some unusual sidebar widgets, like “Unite”, “Notes”, and some Sync. And also a trashcan?! Am i supposed to spend a hour study what these are?

Yeah, supposedly all these can be customized. But every time i tried to look at its preference settings, it's nearly impossible to find or set what i want, because it sports its own fancy structure and UI too.

Non-standard Preference Setting

I tried to use Opera few times every year since 1998. Every time i got frustrated. In the latest version, 10.5x, even the graphical user interface is different. When Apple or Google Chrome sports novel GUI, i didn't have any frustrating experience. I can still do what i want, sometimes more conveniently, and am able to quickly tweak preferences for things i don't like.

Evangelist Håkon Wium Lie Idiot

No wonder Opera's market never grew. In early to mid 2000s, it was plagued by being a ad-ware. But since they removed that in 2005, still nobody uses it. Since about last year, Google Chrome came, and Safari came to Windows. Both already have about twice the users of Opera. And i noticed several times in recent years that the Opera guy Håkon Wium Lie, best known for his CSS work and advocacy, is a Internet Explorer and Microsoft hater. In every opportunity he badmouths IE for this and that. Here's one of his article badmouthing IE. 〈Opera to MS: Get real about interoperability, Mr Gates〉 @ theregister.co.uk.

Opera Pain FAQ

No drag'n'drop, but, images can easily be saved, copied, or used on the desktop by one left and one right click on them.

When i browse porn, i often like to save pics i like. Drag'n'drop is the easiest.

Who needs a keyboard shortcut?

keyboard shortcuts is very convenient especially for power users. e.g. emacs, vi, are all about keys. Even in mouse intensive apps, such as 3D Modeling Blender, or photoshop, gimp, InkScape, when you become a pro you tend to go for the keys. Because, keys are precise and brainless operation, while mousing requires the harder aim and eye-balling decision making, and is slower.

Even though Opera's Ctrl+Tab pop-up scheme is non-standard, but i don't have problem using it.

【Ctrl+Tab】 to switch is convenient. Because i can keep pressing it to switch among tabs quickly, and stop at the tab i want. Usual practical speed is perhaps viewing 6 tabs per second. I normally have about 10 to 20 tabs open in each of Chrome and Firefox. (my 2 daily browser, open at all times) This many because when you do research on a topic, you have a lot pages.

Also, i've got system-wide key macros so that 【Ctrl+Tab】 and 【Ctrl+Shift+Tab】 are single keys (they are the / and * on numpad). Of course, my system works in all browsers but stops dead in just Opera itself.

The conventional hotkey to switch tabs on Windows are【Ctrl+Tab】, 【Ctrl+Shift+Tab】 and or【Ctrl+PageUp】 , 【Ctrl+PageDown】. All browser supports one of these shortcuts, usually both.

In my key macro app (on Windows it's AutoHotkey ), i can easily adjust to one of these. With Opera, it takes some 10 min or more to find out what key is what for each operation i have macro for... the end result has been that i just didn't bother and abandoned opera for the possibility of using it more often.

Ctrl++? does not zoom in, but + and - are all you need.

Thanks. Yesterday while i was frustrated and writing this, noticed the standard zoom in key 【Ctrl++】 doesn't work. However, apparantly 【Ctrl+-】 does zoom out. So naturally one'd think 【Ctrl++】 should work too. So, you can see where this confusion came from. The point here is that the non-standardness adds up, like some might say about emacs. You end up spending days just to learn the tool first, before you can use it fruitfully.

Opera does not support side-scroll by scroll wheel, but who needs it? Opera has a “Fit to Width” button that performs miracles in eliminating horizontal scrolling.

Side scroll is important. There are popular sites that relies solely on side scroll. For example, the social network site Plurk, e.g. http://www.plurk.com/pepelo16.

The plurk site made mouse wheel to scroll horizontally, but there are other sites, that are designed to be horizontally scrolled, either for artistic effect or for practical purposes (e.g. history line, a series of photos, panorama).

All things said, for scroll wheel mouses that has left/right push feature , it seems stupid not to support the functionality of the device. The fact that all other major browsers do support it, indicates it's not some tech hurdle.

One wonders why opera doesn't support it. Is it just sloppiness, or they actually intentionally decided that they didn't want to support it?

What i can't really believe is that they don't support drag'n'drop for saving images. That's the last straw that broke the camel's back, and made me to write my essay. At first as usual i thought the feature is hidden in some preference that i need to turn it on... but not, after searching the web a bit.

The numerical keypad as functional shortcuts keys is part of Opera, i discovered and started to use since maybe 2005 on the Mac.

I recall, few years ago when a new version of Opera came out, they removed those keys, and adopted keys more similar to Firefox. (and i was thinking: “Opera finally realized the weirdness of their keys”) But i like the 1 and 2 keys for tab switching, so i went to the preference and took a while to find a option something like “use previous opera keyset”, so i have 1 and 2 as tab switching keys.

In the latest opera 10.62, i don't know by default these are supported... maybe the reason it doesn't work for u because it was carried over from my preference settings... too lazy to dig now.

Reasons to Support Standard UI

there's a good reason to support standard, or widely adopted practices.

For example, in my case, i have system-wide key macro sets. So, i press one single key to close window or tab in any browser or other app. But when in Opera, things stopped working. So i have to adjust my macros, but then, it is time consuming to find out what's Opera's ways. This seems to be my impression in the past 10 years every year i tried.

Also, in mid 2000s i was on a low-muscled Mac laptop, so i kept a eye on cpu use for any app i use. I recall, Opera even when idle with no page loaded, will suck some 10% of cpu. I recall, all other browsers do much better. I vaguely recall the best was Camino in this regard.

Microsoft Hating

there are few messages in this thread with the common opinion of Microsoft hatred, many are ill informed. Remember, that it is Microsoft that brought the concept of adhering to standard html and css, at the time when Netscape is very busy creating blinks and LiveScript and stuff. In the early days (late 1990s), IE on the mac was the first browser to support css or validation or something (you can easily find it in Wikipedia here Internet Explorer for Mac or other articles about IE, web standard...)

Netscape browser, in my personal experience, in every year since IE started, are worse than IE in EVERY aspect: standard compliance, speed, features, crashiness... There are ample evidence that's still available on the web you can verify. I ranted about it, see: Netscape Crap

IE won the browser war in early 2000s, basically got the whole market. So, they've been lazy, and thus the fact that IE has been worst browser in every aspect since mid 2005. But Microsoft has been putting more effort back to browser in past few years, with IE 8 perhaps being the first better result. (but then the slew of Microsoft haters will badmouth Microsoft doesn't matter what Microsoft do. One big controversial topic is whether IE 8 should simply stick to standard and fuck compatibility to the giga billions of existing web sites. This been blogged to death.)

in recent years, several times i'm ran into Opera's guy Håkon Wium Lie badmouthing IE, and i find his attacks scumbaggy.

For example, in his article〈Opera to MS: Get real about interoperability, Mr Gates〉 @ theregister.co.uk, he attack Microsoft for claiming to play nice yet all their pages still not valid html or something to that effect.

Look at opera home page now:

the above pages are not html valid, out of about 10 pages i just checked now (and their server seems to send wrong css mine type as text/html instead of text/css). Should we now launch a attack on Håkon for being a hypocrite?

Of course, many will say that IE won the first browser war because Microsoft's dubious business tactics, but i don't agree. This article might be of interest: On Microsoft Hatred.

See also: Web Browser Market Share 2010-02.

Was this page useful? If so, please do donate $3, thank you donors!

Using Emacs's DiredPlus Mode

Perm url with updates: http://xahlee.org/emacs/emacs_diredplus_mode.html

Using Emacs's “Dired+” (DiredPlus) Mode

Xah Lee, 2010-09-15

“Dired+” mode, by Drew Adams, is quite nice. Here's some of its features i found very useful.

emacs diredplus mode

“Dired+” mode.

  • More extensive syntax coloring, so you can easily recognize different file types. (after using plain dired for 10 years, the first few days i found the Dired+ colors distracting, but now i prefer it.)
  • Press 【w】 to copy file name. Works with multiple marked files too.
  • 【F】 to open all marked files. (nice!)
  • 【A】 to regex search marked files. (NICE!)
  • 【B】 to byte compile a elisp file.
  • 【L】 to load a elisp file.
  • Use menu “Mark‣Omit Marked” to omit some files. (useful especially if you use “find-dired” but don't want a subset.)

These are in addition to extremely useful features in dired. (See: File Management with Emacs (dired tutorial))

Install

Get it here: http://www.emacswiki.org/emacs/DiredPlus.

To install, just place the file at 〔~/.emacs.d/dired+.el〕, then put this code 「(require 'dired+)」 in your emacs init file. Optionally, call byte-compile-file to make it load and run faster.

Was this page useful? If so, please do donate $3, thank you donors!

2010-09-14

Google Problems and Alternative Search Engines

Perm url with updates: http://xahlee.org/js/alternative_search_engines.html

Google Problems and Alternative Search Engines

Xah Lee, 2010-09-14

There seems to be lots of heat on how Google search engine qualty is going down. For example:

I feel the same. Though, i am not sure that other search engines are better. In this or last year, i've occasionally tried Microsoft's “bing.com”. I don't think it's better. Though, it is true that i am not able to find info i want as compared to say early 2000s. There were quite a few times in recent years that i spend 30 min or hours and unable to find info that i think should be there. Often, Google will show results from a bunch of mass-scale commercial sites that swipe info from other online forums and put on their site, with bad formatting, and with many disruptive, annoying ads.

I think a large problem is SEO, and Google actually encourage it, even to the degree of officially supporting the term and practice (Google has a huge amount of documentation on how to do SEO, e.g. http://googlewebmastercentral.blogspot.com/.)

Since about 2006, when i need some info, my first stop is to find it in Wikipedia, then possibly follow links from there.

I've also noticed quite a few up-coming search engines in this year. Ok, here's a list of alternative search engines.

News search:

Blog search: http://blogsearch.google.com/ (i never find technorati useful.)

Set your browser to one of these. For the next week, when you search, compare results. Let me know what you get.

For some juicy info about search engines, see: Web search engine.

Also, checkout this Wikipedia List of search engines.

2010-09-14

Nathan Staudt wrote to inform me there's http://www.peer-search.net/ that he created. It's actually interesting, and first time i heard of the concept of peer-to-peer search engines. It's based on Yacy, which is a free distributed search engine. De-centralized systems are in general more robust and evolves fast. This could be the future of search engines.

I tried peer-search.net on my name “Xah Lee”, and it's quite interesting. It shows many pages that contains my name, reminiscent of google search results in the early 2000s. In the past, google search is quite useful in searching pages that contain some exact string. For example, if i want to know all pages that i left a comment, or sites that collect online forum posts, or any page that mentions my name, google search in the early days will show them. This exact string can be a computer error message, a product model name, or a phrase in a chat room that you are trying to locate. Quite effective. But today, google search has “smartened up”. It no longer is able to show pages you want where a exact phrase shows up, even if you double quotes your phrase "like this". But, it tries to interpret your search as if a human is asking a question about that phrase. For exmaple, if i search “Xah Lee” now, i'll get pages that contain info about me, my website, my blogs, my google profile, my facebook, my twitter, my yahoo etc pages. All the hundreds of other pages that exists that merely contained the string “Xah Lee” are not returned in the result, even if you went to page 20 or something.

Is this smattering up good or bad? It is probably more useful to majority of people most of the time. But it also made it hard in many situations where you really just need to find pages that contains a particular string exactly, as if searching a file systems or database.

Was this page useful? If so, please do donate $3, thank you donors!

2010-09-13

Nearly Extinct Animals (video)

Perm url with updates: http://xahlee.org/creatures/extinct_animals.html

Nearly Extinct Animals (video)

Xah Lee, 2010-09-13

Extinct and soon to be extinct animal/species

PS i don't support animal rights group, and nor PETA, nor any fervor for Green.

I recommend taking a rational approach. If you can, go back to school and take classes in biology, human history, philosophy, etc. These subjects may seem remote to loving animals, but the extinction of animals isn't just caused by poachers or heartless people, it has to do with the whole society, education, economic system, political system. And even, the protection of animals is just part of human animal problem. There is war, famine, religion, which at large dictate how things change. So, don't just be a bleeding heart and becoming a activist in this or that. Often, the activists are doing the harm.

The song is 〈Whole Again〉 by Atomic Kitten. amazon

Was this page useful? If so, please do donate $3, thank you donors!

Unix Shell Text Processing Tutorial (grep, cat, awk, sort, uniq)

Perm url with updates: http://xahlee.org/UnixResource_dir/unix_shell_text_processing.html

Unix Shell Text Processing Tutorial (grep, cat, awk, sort, uniq)

Xah Lee, 2007-03, 2010-09-13

This page is a basic tutorial on using unix shell's text processing tools. For example, grep, cat, awk, sort, uniq. They are especially useful for processing lines.

How to show only certain lines that contains a text pattern?

Use grep. Example: grep 'html HTTP' myFile will print only lines containing the text “html HTTP”. grep 'html HTTP' *html will apply grep to all files with html suffix. grep -r 'html HTTP' *html will apply grep to all html files in a dir.

Use “-f” to include file name in the result, use “-h” to not print file name. Use “-v” to print lines NOT containing the text. Use “-E” for extended regular expression (similar to Perl's) or use “-P” for perl's regex syntax. Use “-i” to ignore case.

Examples:

  • grep -v 'html HTTP' myFile will print lines not matching the regex.
  • grep -E 'png HTTP|jpg HTTP' myFile will print lines containing either “png HTTP” or “jpg HTTP”.
  • cat myFile | grep 'html HTTP' | awk '{print $12 , $7}' | grep -i -E "livejournal|blogspot" | sort | uniq -c | sort -n

The last example will show lines only containing “html HTTP” in my apache web log “myFile”, then shown only the 12th and 7th columns (which are referral url, and the requested file), then show only lines that contain “livejournal” or “blogspot” text (“-i” for ignore case and “-E” for extended regex pattern), then sort them, then show only unique lines with number of occurance in prepended, then sort that by the numbers.

How to show only certain columns in a text file?

awk '{print $12 , $7}' myFile will print the 12th and 7th column. (columns are separated by spaces by default.) For delimiter other than space, for example the straight double quote, use awk -F\" '{print $12 , $7}' myFile.

Alternative solution is to use the “cut” utility, but it does not accept regex as delimeters. So, if you have column separated by different number of spaces, “cut” is incapable.

How to sort lines in a file?

To sort lines use sort myfile. To sort by considering text as numbers, use sort -n myfile. To reverse order, use “-r”. To sort by comparing a particular column, for example the 2nd column, use sort -k 2 myfile.

Here's a more complex example: sort -k 2,2r -k3,3nr myFile. This will sort by first column first, with reverse order, if tie, sort by 3rd column as numbers and reverse order.

Note, sort does destructive sort by default. For example, if your text file is:

b y
b x

and you use “sort -k 1 myFile”, it will re-order your lines. To make it leave unspecified field as is, use “-s”.

How to show only uniq lines in a file?

sort ~/myfile | uniq. To prepend the line with a count of repeatition, use sort ~/myfile | uniq -c

How to sum up the 2nd column in a file?

awk '{sum += $2} END {print sum}' myfile.

How to show only first few lines of a huge file?

head ~/myfile. If you want to see first n lines, use head -n 100 ~/myfile. If you want to see the bottom of a file, use “tail”.

For complex text processing, you need a full language. See: Perl and Python Tutorial, Emacs Lisp Tutorial.

Processing Multiple Files

How to list only files who's name matches a text pattern?

find ~/myDir -name "*.html" will show just files with “.html” suffix.

How to list only files larger than n bytes?

find ~/myDir -size +900000c will list files in 〔~/myDir〕 larger than 9 Mega bytes.

To list files smaller than a given size, use a minus sign “-” instead of the plus. To list files of exactly a give size, don't use the plus or minus.

How to delete all files who's name matches a text pattern?

find ~/myDir -name "*~" -exec rm {} \; will delete all files files who's name ends with “~”.

Using “find” with “xargs”

How to use “find” on file names that may contain spaces or dash?

find . -print0 | xargs -0 -l -i echo "{}";.

The “-print0” tells “find” to print the file names separeted by a null char. (as opposed to a newline char by “-print”) The “-0” tells xargs to parse input using null char as seperators and take any special char in file name as literal.

The “-l” tells “xargs” to pass just one file name at a time. The “-i” allows you to use “{}” as the file name. The “"{}"” creates quoting around the entire file name, so that “echo” (or another program) will see it as one argument instead of several. (Note: the “-i” must come after “-l”)

Here's a example that uses “find”, “xargs”, and “basename” and ImageMagick's “convert” to convert “bmp” image files to “png”: find . -name "*bmp" -print0 | xargs -0 -l -i basename "{}" ".bmp" | xargs -0 -l -i convert "{}.bmp" "{}.png".

Use GNU Parallel for xargs

Note: a modern replacement for xargs is GNU Parallel. The syntax is almost indentical to xargs, except it runs in parallel. It also doesn't have problems with file names containing quotes or apostrophes.

Man Page

How to get a text output of a man page?

man ls | col -b. The “col -b” formats the man page to plain text (rid of control chars).

How to read a non-compressed man page without the “man” command?

nroff -man n43921.man | col -b

This is convenient when you need to read a man-page file once without adding the dir to your $MANPATH.

How to read a compressed man page without the “man” command?

cat n43921.1 | compress -cd - | nroff -man | col -b

How to read a unformatted man page?

a possible solution: nroff -man ftpshut.8

The “man” command is essentially nroff -e -man file_name | more -s.


Thanks to Ole Tange for telling me about GNU Parallel. (he is the author)

Was this page useful? If so, please do donate $3, thank you donors!

2010-09-12

emacs rename files easily

Perm url with updates: http://xahlee.org/emacs/rename_file_pattern.html

Emacs: Rename Files Interactively by Pattern (regex; wdired)

Xah Lee, 2007-08, 2009-06, ..., 2010-09-12

Suppose you need to rename a bunch of files by a string pattern, for all files in a directory. However, you do not want to rename all of them. You need to look at it in a case-by-case basis. What can you do?

Answer: emacs.

Summary

Type 【Alt+x dired】, then type 【Alt+x wdired-change-to-wdired-mode】. Then, just edit the file names. When done, type 【Ctrl+c Ctrl+c】 to commit the changes.

While in wdired, you can edit the list of file names by emacs's find/replace commands. For example, type 【Alt+5】for “query-replace” or 【Alt+%】 for “query-replace-regexp”.

A older, alternative method, but less convenient, is to call “dired-do-rename-regexp” 【% R】.

Detailed Explanation: Rename by Editing File Names

In emacs 22, there is a new way to rename a bunch of files, by simply editing the list of file names like normal text, then commit the change.

In dired, type 【Alt+x wdired-change-to-wdired-mode】. Then, just edit the file names in dired as if it's a normal text file. When you are done, type 【Ctrl+c Ctrl+c】 to commit the changes.

KeyCommand NamePurpose
Ctrl+x Ctrl+q】 (emacs 23.1)wdired-change-to-wdired-modeStart rename by editing
Ctrl+c Ctrl+cwdired-finish-editCommit change
Ctrl+c Escwdired-abort-changesAbort changes

This feature is also listed under the menu “Immediate‣Edit File Names”. This menu is available only when you are in dired.

Rename By Regex

While in wdired, you can edit the list of file names by emacs's find/replace commands. For example, type 【Alt+5】for “query-replace” or 【Alt+%】 for “query-replace-regexp”.

Warning: Renaming To Existing File Name

There is a bug in emacs 22. (bug#394) If you rename 2 files by exchanging their names, one of the file will get deleted. For example, first file is named A and second is named B, and you rename them so that first is B and second is A. When you do commit, emacs will promp you that A already exsit and if you want to overwrite it. If you answer yes, then file A's content will be gone, and there will NOT be a file named A to be renamed B. In short, you lose a file. This is fixed in emacs 23, but i would be still careful if you rename many files that have existing names. Best to rename them to something else (such as adding a “x” in front of name), commit, and rename again.

dired-do-rename-regexp

A older method to rename files in batch is by the command “dired-do-rename-regexp” 【% R】.

Note: emacs's regex is not based on Perl's regex engine. For a summary and common patterns, see: Emacs Regex.

Was this page useful? If so, please do donate $3, thank you donors!

reddit = porn fodder

this page's content is removed due to possible conflict with Google AdSense. To see the content, goto http://xahlee.org/Periodic_dosage_dir/reddit_porn_fodder.html