Software Dependency Complexity: Fink, Unison
Perm url with updates: http://xahlee.org/comp/software_dependency_complexity.html
Software Dependency Complexity: Fink, Unison
Xah Lee, 2010-06-04
This essay documents a computer industry complexity, in particular, a problem of installing a single software, caused by software dependency.
Today, i got a project to update a website of a university. The first job, is to install the Unison software, which allows 2-way file synchronization.
The remote machine is Mac OS X 10.6 running on a recent server version of mini mac. On the mac, there are 2 primary ways to systematically install a large number of unix software. One is Fink, and another is Macports.
I've been using fink since it started in ~2002. I've casually tried MacPorts once or twice in the past, but didn't have a good experience with it. Over the years i've been using fink, largely i'm a happy user, but i've had some problems so i don't regard it as near perfect. So, i did some reading the web to see if one has really surpassed the other. After a hour, it seems there's no conclusive evidence whether one is much better than the other. Basically, MacPort has more updates, but Fink has more packages and is stable.
So, i decided to use fink. (long story short, i've worked on this server before, and have installed unix tool set thru fink, but recently the professor upgrated the machine from 10.4 server to 10.6 with new hardware, so i removed all fink stuff “/sw/”, necessarily for a fresh start.)
After installing fink, and run thru numerous commands such as selfupdate, and other preparations such as checking Unison versions on the machines i need to work on and available versions on fink and enable unstable branch the latest Unison at fink is in, i started to install Unison.
fink desc unison sudo fink install unison-2.27.57-1008
Then, i got this:
Pick one:  The following package will be installed or updated: unison The following 129 additional packages will be installed: atk1 atk1-shlibs autoconf2.6 automake1.11 avahi avahi-dev cairo cairo-shlibs dbus dbus-glib1.2-dev dbus-glib1.2-shlibs dbus1.3-dev dbus1.3-shlibs docbook-utils enchant1-dev enchant1-shlibs esound esound-bin esound-common esound-shlibs gconf2 gconf2-dev gconf2-shlibs gd2 gd2-shlibs gettext gettext-dev glib2-dev glib2-shlibs gnome-desktop2.11-dev gnome-desktop2.11-shlibs gnome-doc-utils gnome-icon-theme gnome-keyring gnome-keyring-dev gnome-keyring-shlibs gnome-menus gnome-menus2-dev gnome-menus2-shlibs gnome-mime-data gnome-panel-dev gnome-panel-shlibs gnome-python2-py26 gnome-vfs2-unified gnome-vfs2-unified-dev gnome-vfs2-unified-shlibs gnutls26 gnutls26-shlibs gtk+2 gtk+2-dev gtk+2-shlibs gtk-doc gtkglarea2 gtkglarea2-shlibs gtksourceview-1.0-data gtksourceview-dev gtksourceview-shlibs gtkspell2-dev gtkspell2-shlibs gweather-data jadetex lablgtk lablgtk2 libavahi-client3-shlibs libavahi-common3-shlibs libavahi-core6-shlibs libavahi-glib1-dev libavahi-glib1-shlibs libbonobo2 libbonobo2-dev libbonobo2-shlibs libbonoboui2 libbonoboui2-dev libbonoboui2-shlibs libcroco3 libcroco3-shlibs libgettext8-dev libglade2 libglade2-shlibs libgnome2 libgnome2-dev libgnome2-shlibs libgnomecanvas2 libgnomecanvas2-dev libgnomecanvas2-shlibs libgnomecups-dev libgnomecups-shlibs libgnomeprint2.2-dev libgnomeprint2.2-shlibs libgnomeprintui2.2-dev libgnomeprintui2.2-shlibs libgnomeui2-dev libgnomeui2-shlibs libgsf1.114-dev libgsf1.114-shlibs libgweather libgweather-shlibs libidl2 libidl2-shlibs libjpeg8 libpng14 librsvg2 librsvg2-shlibs libsoup2.4.1-ssl libsoup2.4.1-ssl-shlibs libwnck1.22 libwnck1.22-shlibs libxml2-py26 numpy-py26 orbit2 orbit2-dev orbit2-shlibs pango1-xft2-ft219 pango1-xft2-ft219-dev pango1-xft2-ft219-shlibs ptex-base ptex-texmf pycairo-py26 pycairo-py26-dev pygobject2-py26 pygobject2-py26-dev pygobject2-py26-shlibs pygtk2-gtk-py26 pygtk2-gtk-py26-dev pyorbit2-py26 pyorbit2-py26-dev scipy-core-py26 shared-mime-info texlive-base Do you want to continue? [Y/n]
I was a bit surprised. Installing Unison requires 129 additional software? and things like gnome-icon-theme? For a moment i was thinking perhaps i did something wrong. I didn't have time to ponder on it. After some re-checking, i went ahead.
The installation and compilation started about 8:30 pm, and ended at 10:37 pm, with thousands lines of output, that ends with this message:
... New package: dists/unstable/main/binary-darwin-i386/x11/xinitrc_1.5-1_darwin-i386.deb Failed: Fink::SysState: Could not resolve inconsistent dependencies
What the fuck? A web search quickly found this official Fink FAQ at: Source. Quote:
Q6.22: Whenever I try to build from source, Fink keeps waffling between alternate versions of the same library.
A: Often, in a complicated build tree, you may find that some of the packages depend on a particular version of a library, and other depend on a different one (e.g. db47 vs. db44). Consequently, Fink may try to switch to whichever one isn't currently installed in order to satisfy the build dependency for the current package that you're trying to update.
Unfortunately, due to limitations in the build-dependency engine, you may wind up with the dreadedFink::SysState: Could not resolve inconsistent dependencies
message when trying a sufficiently complicated update-all. This generally gives you a command to try to resolve the issue:fink scanpackages sudo apt-get update sudo apt-get install foo=1.23-4
but this may not work for sufficiently complicated updates. You might need to update packages one-by-one, at least for a while.
So, there it is. Installing a single software, with expert know-how, ends up in failure with 4 hours spent. That is the state of the art in software today.
One might think that perhaps it is easier to just compile Unison myself. Note that Unison is written in OCaml. So, to compile Unison you'll need to have the whole OCaml platform. If you are lucky, you might succeed with half a day on that.
Ok, at this point i can maybe try to fix it or try MacPort. So, MacPort it is.
vmm:~ xahlee$ which port /opt/local/bin/port vmm:~ xahlee$ sudo port install unison ---> Computing dependencies for unison ---> Fetching ocaml ---> Attempting to fetch ocaml-3.11.2.tar.bz2 from http://distfiles.macports.org/ocaml ---> Verifying checksum(s) for ocaml ---> Extracting ocaml ---> Configuring ocaml ---> Building ocaml ---> Staging ocaml into destroot ---> Installing ocaml @3.11.2_0 ---> Activating ocaml @3.11.2_0 ---> Cleaning ocaml ---> Fetching unison ---> Attempting to fetch unison-2.32.52.tar.gz from http://distfiles.macports.org/unison ---> Verifying checksum(s) for unison ---> Extracting unison ---> Applying patches to unison ---> Configuring unison ---> Building unison ---> Staging unison into destroot ---> Installing unison @2.32.52_0 ---> Activating unison @2.32.52_0 ---> Cleaning unison vmm:~ xahlee$ which unison /opt/local/bin/unison vmm:~ xahlee$
Success in 10 min.
Note: this is not a story of MacPort vs Fink. They are pretty similar in quality. Also, MacPort by default installs binary, while Fink by default compiles, which i've chosen to not change.
In this essay, i wanted to document the complexity of software. Why is it complex, why so many seemingly simple things are fraught with problems, what's the cause, why is software engineering so complicated.
In this particular case, i think the problem traces to software dependencies. It is really a issue that cannot be practically solved. In order to have a robust dependency system, you have to have a dependency system to start with, that each software can document or communicate as a element in the system. This ideal framework is simply not possible from existing diverse software, that are built from different languages and different operating systems, from different decades. Instead, a dependency system is somewhat cooked up by practical means. That means, there's lots of flaws. In my case, the practical consequences is that, installing a command line file syncing software spuriously requires one to install 128 other software, including a desktop system with images for icons, and cause me 4 hours. LOL.
(For another documentation on the issues of Unison and Cygwin, see: The Complexity And Tedium of Software Engineering)