Incredible Shrinking Music

If it gets any more compact it might just disappear ...

Back when the millennium rolled over from 1 to 2 I was still using a computer with a mere 10 gigabyte hard drive. But newer drives were becoming available with 60 or 80 gigabytes, and 100 was soon to follow. At this point I turned my eyes to my music collection -- a set of shelves holding a few hundred CD's -- and began to wonder ...

Wouldn't it be great if I could copy all those disks to a big hard drive someday? Then I could organize them into folders by composer and album title, even into larger categories like Classical, Jazz, Rock, and Folk, and I'd be able to find what I wanted to hear with a few mouse clicks instead of tilting my head sideways to read the tiny print on the edges of the disk boxes. Instead of loading my 5-disk CD changer I'd be able to have a list of music that would play all day, or all weekend if I wanted to.

Then I did the math. Allowing about 600 megabytes for each CD (because many of them were not the full capacity of 700 megs or 80 minutes) and multiplying by the approximate number of disks on my shelves I got the discouraging result of nearly 400 gigabytes. (I have a LOT of music.) To contain my whole collection I would need an array of at least five 100 meg hard drives. At the going rate of about $200 each, plus a rack of RAID hardware to organize them all, it would have cost over $1400, even with no room to grow.

But of course all I had to do was wait a few years. Three things happened that made my dream system not only doable but easy and cheap.

First was advances in sound compression technology. The huge WAV files on all those CD's were like the pig-sized BMP images that take up so much space because they make no attempt to optimize the way they store their contents. MP3 compression was one answer, of course, and I came to consider it good enough for at least some forms of music (e.g. rock) where the subtlest shades of tone might not be critical. But I wasn't satisfied with that for my classical collection. Having shelled out all those bucks for the best possible sound I wasn't about to run it through a meat grinder before listening to it.

Fortunately FLAC came along -- a "lossless" form of audio compression that can shrink a WAV file to about half its size and then blow it up again with every bit intact. If this format had been used on CD's you could have bought the complete Brandenburg Concertos on a single disk, or all the Beethoven Symphonies and overtures on a 4 disk set instead of 7 or 8. Flac has the further advantage of being open source and free to use. This instantly reduced the amount of storage I would need down to a more manageable 200 gigabytes.

By this time hard drives had grown to 500 gigabytes (the original size of my estimate), with 750 and 1000 -- a full terabyte -- soon to come. Not willing to wait any longer, but still cost conscious, I grabbed a 320 gig drive on sale for $89. Quite a bit better than $1400!

The last ingredient that pulled all the pieces together was software that automated the process of converting the CD's to flac files, and adding them to a searchable database complete with categories that I could edit as I chose. There are many programs to organize music collections, but my favorite is the open source Amarok. Originally for Linux only, and now also available for Mac and Windows, Amarok works well and makes it trivially easy to find, manage, and play your music.

It was a short step from there to acquiring new music in pure digital form. I do this legally through eMusic, a subscription service that lets you budget how fast you want to grow your collection by choosing from a variety of monthly plans and payment options. Depending on your selection you can get your downloads for as little as 43 cents per track, with most full albums (if that word still has any meaning) only a bit above $5.

eMusic convinced me that with a high enough sample rate MP3's were satisfactory even for classical music with its subtle nuances. This launched me into a new acquisitive phase. I started buying more music than I had done at any time since the advent of CD's caused me to replace my LP collection. (Music publishers please take note.) And of course the MP3 albums were even more compact than all those flacs I had pulled off my CD's, so I was able to pack an amazing amount of additional music onto my 320 gig drive and still have space left over.

Digital Peril

What about disaster recovery, you may well ask? With so many hours of effort invested in this project, not to mention the financial investment in new music, what would I do if the hard drive failed?

In the case of the CD's I still had the optical disks for backup. At the beginning I started burning at least the best of the new MP3's to disk for protection, but this seemed to defeat the purpose. It now began to appear as a liability to have to store all the disks. In the end I decided that once I outgrew the old hard drive I would copy all the files to a newer and bigger one, then put the old drive in a drawer as a backup copy. At least that would give me a fallback position so only the files that I added to the new drive would be lost if it failed. (Notice, by the way, that my shelves full of disks had been reduced to a package about 4x6 inches in size.)

I planned to move next to a 500 gig drive, but what happened is that my computer died instead of the hard drive. I ended up moving to a new machine with a full terabyte instead -- a system I would have drooled over ten years ago, but which is now pretty standard. I partitioned the drive into two 500 gig halves, donated one half for my wife and grandchildren to live in under Windows, and moved into my own Linux partition on the other half. After restoring all my files from the old drives, including all the music, I still had almost 200 gigs left over. Room to grow.

The old music drive is tucked into that drawer to give me some peace of mind. Next will be to set up another 1 or 2 terabyte drive in an external enclosure to serve as backup. The thing to keep in mind is that the growth of storage space and the dropping of its prices have not slowed down yet. That old 10 gig drive cost me $160 back in 1998, or around $16 per gigabyte. Now you can buy a terabyte drive for around $90, or about 9 cents per gigabyte. In five years we should be up to 8 or 10 terabyte drives with a cost per gigabyte of about a penny. After that we'll have to start watching the terabyte prices fall until we're up in the petabytes.

The same drive that holds all those music files also contains a few "virtual machines" -- files that contain a simulated computer complete with its own simulated hard drive. They can be opened in their own windows (using VirtualBox), copied, and moved to other hard drives as easily as you move a photo from one folder to another. Each of my virtual machines has more power and storage space than my whole computer back in 2000. The new computer swallows them whole and has space for them to swim around in.

Long ago I got used to the idea that everything I had ever written, if suitably compressed, would fit on a single floppy disk. Now it seems I can look forward to a time when all the files I've ever worked with -- all the images, software, music, etc -- can be saved and copied just as trivially. But by then we may no longer be storing anything ourselves ...

[Coming soon: Life in the clouds.]

Draw Me!

I may not know what I like, but I know art when I see it ...

One of my favorite scenes in the film I, Robot is when the robot, "Sonny," tries to tell the investigating officer about his recurring dream. A picture being worth a thousand words (even to a robot), he proceeds to make a drawing to convey the scene. However he draws it like a robot might, in a series of rapid horizontal scan lines like the ones on a TV tube (remember them?) or an old dot-matrix printer. The result is uncanny -- an artistic rendering generated by a machine.

Like most science fiction this robotic trick has been quickly surpassed by reality, as you can see in this news item about Aikon, a machine that not only can draw, but does so by looking at real objects and imitating the same techniques of seeing and modeling that might be used by a human artist. I suspect that Stephen Hawking, who is himself assisted to communicate by a computer speech synthesizer, welcomes being a subject for such an experiment, as you can see in this example.

Back in the early years of computers (the 1950's) many people felt that in spite of their wizardly ability to perform mathematical calculations, there were other types of thinking that a computer would never be able to do. Alan Turing, whose work in mathematics and logic laid the groundwork for the development of the first computers, disagreed. He could find no difference in principle between what was happening in his mechanical "children" and what takes place in a human brain. Turing was one of the first to predict that computers would not only reach but surpass the abilities of those who invented them.

Oddly, in retrospect, the game of chess was sometimes sited as the kind of thing humans would always be able to do better than machines. As we know now, the opposite is true. Only the top grand masters of the game stand a chance against IBM's Deep Blue supercomputer, and most of us can't defeat a toy that sells at Radio Shack. Chess is trivial to computer analysis, and the amazing thing is that we can do as well as we do while lacking the ability to see as many moves ahead as our automated opponents.

So I for one am not the least surprised to see a portrait of Hawking that could be easily mistaken for the work of a human artist. If you play with Photoshop, or one of its alternatives like the open source GIMP, you may be familiar with the filters that can be applied to a photographic image to make it resemble an oil painting -- even to resemble a particular school of art like Impressionism or Pointillism. What Aikon does is far beyond that, actually reproducing at least some of the creative process to compose original work.

What's that? You say Aikon doesn't feel anything? That what it produces doesn't mean anything, either to itself or those who view it? That to be art it has to convey something about human experience?

Maybe you should have another look at the work of Piet Mondrian, compare it to the Hawking portrait, and ask which of them looks like it was done by a human being.

Or maybe you think that feeling and expression are the kinds of things machines will never be able to do?

The Uncertainty Principle

How can we see it when we're in the middle of it?

Once we were told that a "giant sucking sound" coming from south of the border would represent our jobs leaving the country under NAFTA. Turns out it is really BP, everyone's current favorite oil conglomerate, trying to slurp up some of its product before it disburses itself all over the Gulf of Mexico.

As I posted on Facebook, shouldn't someone be saying I TOLD YOU SO?!?

A catastrophic leak of some kind was pretty easy to predict. In fact, with thousands of wells and platforms and enormous ships plying their lanes through hurricane country, it seems inevitable that sooner or later something would go awry. The actual event -- a specatular explosion and fire followed by an ocean floor leak like a severed artery -- is more sensational than any doomsday scenario likely to have been scripted by advocates for the environment, but the end result will be much the same: untold miles of coastline and thousands of acres of delicate fisheries glurped up with millions of gallons of crude, with an expensive cleanup to follow for years.

As with all historical events, while we're in the middle of it we still don't know how long it will go on, how bad it will get, what the long term effects will be, how many years and billions of dollars it will take to recover. Some predictions show the plume of oil eventually wrapping around much of Florida, with unprecedented effects. If that happens, then just as with Hurricane Katrina it will be another disaster that we share with the people of Louisiana.

Back during the Vietnam War some of my friends encouraged me to stay in college until it was over, using my student status as protection from the military draft. One of the reasons I didn't take their advice was that I couldn't see any end in sight. In the midst of that history unfolding, all I could see was that the war was escalating each year as the number of troops grew, the bombing campaign increased, and the daily body counts kept mounting. Who knew what would happen? The Soviet Union was already supplying the North with weapons; maybe they would get more actively involved. Or maybe the Chinese would suddenly come into the conflict as they had in Korea only fifteen years earlier. 

No one knew how bad it could get or when it would end. Maybe we were just in the early stages of World War III and hadn't figured it out yet. Maybe if we didn't stop the war it would mean the end of everything. In the face of uncertainty I decided, as did many others, to confront the draft, resist the military juggernaut, and add my own drop to the bucket of pacifism in opposing the war. My actions were small in themselves, but collectively we did bring the war to an end, even though it took years.

Maybe we're at that point now. If we don't draw this line in the sand and say that the destruction of our environment is too high a price to pay for oil, and that controling the sources of oil is not an acceptable reason for fighting wars, then who can say where it will end?