The Format Wars

We're having the Miami Book Fair this weekend (report next time), so here's something that affects the future of books, and why you should care how your documents are stored ...

A struggle for world domination has been going on in recent years that will affect your ability to read and write digital documents, whether they are your own or the property of others. 

The most prominent example is the battle between Microsoft's ubiquitous Office formats (consider the chutzpah it takes to trademark the word "office") and the alternative Open Document standard as proposed by a consortium of other companies, individuals, and organizations, and certified by the ISO. There is an old gag among computer hackers that says, "a standard is a good thing, so it's a good thing there are so many of them." The joke of course is that with enough different standards there is no standard at all.

That particular struggle is still unresolved at this point. Microsoft has managed to have their own proposal for an "open" format adopted as a second standard by the ISO, but neglected to fully support it even in their own software, leaving us with a "standard" that no one can use except theoretically. (Joseph Heller must be rolling over in his grave at this example of a Catch 22.)

Meanwhile the opposition -- represented by OpenOffice.org, Sun's StarOffice, and IBM's Lotus Symphony (all free and compatible) -- forges ahead by supporting both Open Documents and the proprietary Microsoft Office formats that people actually use, thus building a bridge across the troubled waters for anyone who wants to take it. In case you can't tell, I count myself in that category.

A footnote to this is that OpenOffice.org and its brethren also support saving documents as Adobe Acrobat PDF (Portable Document Format). If you wonder why Microsoft until recently required third party software to support this widely used feature, you only have to look as far as Microsoft's own answer to PDF -- the XML Paper Specification, or XPS, which they would prefer people to use instead of Acrobat, which is owned by a competitor. However at the moment XPS requires Microsoft Internet Explorer and Microsoft Windows, which kind of defeats the goal of making an electronic document universally readable.

By now you can see where this is going. Governments and big corporations are not the only ones who should be concerned about how their digital documents are archived. The issue of whether and for how long those documents will be readable is something we all have an interest in, whether as small businesses, students, writers, citizens, or anyone who has anything to save on a computer -- which pretty soon will include every last one of us. If your data format dies because your chosen software is driven out of existence by market forces, it is definitely a problem.

We know the nightmare can happen, because it has happened already. Long ago many law offices standardized on WordPerfect because at the time it was the only product that gave them the features they needed for legal documents. But now WordPerfect is struggling to survive in the face of the Microsoft juggernaut on one side and free competitors on the other. If and when it dies, an unknown number of critical legal documents may become unreadable, or garbled by imperfect (no pun intended) conversion programs.

As a writer I have had to migrate my work from the old DOS-based PC-Write to Lotus Ami-Pro, then through two semi-compatible versions of WordPro, and finally two versions of OpenOffice. (The first one was before the new standard.) And this is within a period of 25 years -- an average of just five years per format.

Even those who have exclusively used Microsoft's Word (there's that chutzpah again) will find that they can no longer read their oldest documents unless they have converted them as they went along and new versions came out. Coming up with new formats has proved to be an excellent marketing tool, since it forces all users to buy new software so they can read documents created by others who have bought it already. But clearly what works for the vendor does not work for the consumer.

And Now eBooks

Meanwhile a new front has opened in the arena of electronic publishing. Due to the understandable desire of publishers and authors to protect their work from piracy, an assortment of new standards for Digital Rights Management (or DRM) have sprung up, along with a separate assortment of file formats that work with one or another version of DRM. Unfortunately the result is that hacker joke all over again. (Here's a comparison between the various formats so you can see how bewildering the assortment is.)

Just as with music, when you acquire an ebook you have to choose one of the available formats, and this choice will determine what computers and other devices you will be able to read it with. After that there are many ways to lose access to what you thought you owned. Your format of choice may become obsolete so that it is no longer supported by newer devices. Your computer may die or be stolen, forcing you to plead for a replacement license for a new machine. Or the format that you like on your computer may not work with your phone or PDA or the new reader from Amazon or [insert vendor of your choice].

About the best you can do is to avail yourself of DRM-free documents in a form that is as widely supported as possible. There are many sources for works that are either in the public domain or which have been made available by their authors as freely distributable while remaining under copyright.

In the latter case you are free to read and share them, but not to sell them without arrangement with the author. Cory Doctorow and Bruce Sterling are notable examples of writers who have chosen to "give away" at least some of their work in this way, while continuing to sell traditional books in print form. It is arguably a way to insure wide readership which may in the long run enhance sales.

Free At Last

Alas, even with freely distributable texts there are problems. Since I acquired my Sony Reader last Christmas I've become all too familiar with them.

Project Gutenberg, the first attempt at creating a free library of public domain books, made an early choice to use "plain text" as their format in order to make their books as widely accessible as possible. (Now they are also using HTML and PDF for some books, and even audio -- a whole other can of worms.) While this avoids the pitfalls of those word processors of yore that are no longer with us, it leaves their online books as "plain vanilla" with no ability to use bold or italics or different fonts, and no diagrams or illustrations.

Worse still, their texts have embedded line breaks that make them difficult to adjust for different display sizes. Removing those line breaks can be difficult or impossible depending on what software you use and what your level of expertise is -- and yes, it is maddening that something so simple should be so hard.

In practice this means that when I read a Gutenberg document on my
Sony with its 5x7 page, there
is no combination of font size or orientation that matches where the line
breaks are. It doesn't mean I can't
read it anyway, but it's esthetically unpleasant. (Like this paragraph.)

PDF files can suffer similar problems, because they were not originally meant to have text that would reflow if the page size was changed.

One example of a new solution is the Epub format, created by the International Digital Publishing Forum and supported by Sony and an assortment of other vendors and online distributors like Feedbooks. It is able to handle fonts, page reflow, and even graphics, which makes epub documents a pleasure to read. However, at this point it is yet another new standard -- and the same old joke applies. Come to think of it, they work fine on my Sony Reader, but I didn't have any software that would recognize them on my PC until I downloaded and installed Adobe's Digital Editions.

Hey, wait a minute -- can you still read this? If so, it's because of the universal use of HTML (hypertext markup language) for text on web pages, which is a perfect demonstration of what a true standard can offer us. Let's hope we can someday agree on another one for books and plain old documents.

The Conscientious Objector

On Veterans Day, let's remember that they also serve who do not fight ...

The most courageous man in the history of warfare may be someone you never heard of. I certainly had never heard of him until I discovered the excellent documentary film called simply The Conscientious Objector.

That's right, Desmond T. Doss was a Conscientious Objector during World War II, and the acts of bravery he performed in his service as a combat medic made him the only C.O. ever to be awarded the Medal of Honor.

To really get this story you should visit the website desmonddoss.com, or better yet, see the film, which is available through Netflix. Director Terry Benedict has done a magnificent job of telling the story with quiet emotional power, easily rivaling the best work of Ken Burns. He filmed extensive interviews with Doss just a few years before his death in 2006. He also filmed other friends and survivors who had served with him, and even got a few of them to accompany Doss on a visit to Okinawa, back to the very ground they had struggled to take from the Japanese near the end of the war.

The short version of this tale is that Doss was a Seventh Day Adventist with an aversion to violence that dated back to his childhood, when a dispute between his father and uncle put him in mind of Cain killing Abel -- a story he had never been able to comprehend. "How could a brother do such a thing?" he wondered. From the day he had to hide his father's pistol he swore he would never again touch a weapon. And he never did.

When WWII came along he was offered a deferment on religious grounds, but declined. Instead he volunteered to serve as what he called "a conscientious co-operator." He thought he could be a medic and help to save lives instead of taking them. But though the army accepted him they did not accept his conditions. His commanders expected him to take weapons training and to carry a sidearm once he was in the field, medic or not. Time and again he refused even to touch a gun it if was handed to him.

Almost more problematic was his effort to honor his chosen Sabbath every Saturday, when according to his precepts he was not supposed to work. Sometimes he was allowed and sometimes he wasn't. Always he had to pay, in having to do all the worst jobs on Sundays, and in the derision of his fellow soldiers. Once he was threatened with a court martial, but by then he had an established precedent in his record, a tacit agreement that his beliefs were to be honored, and that he would not have to carry arms.

Everything changed when Doss's unit finally went into action in the South Pacific. He quickly came to be known as the man who would never leave a wounded soldier untended, regardless of the risk to his own safety. This behavior culminated at "Hacksaw Ridge," the highest ground on the island of Okinawa, where horrific fighting went on for day after day.

When an all-out assault left only about one third of his unit able to retreat under their own power, Doss stayed behind with the dead and wounded atop the plateau. One by one he found the survivors, dressed their wounds as best he could, and dragged them single-handed for as much as a hundred yards or more to the edge of the cliff, where he finally lowered them by rope to those waiting down below. He continued this for twelve hours, under constant fire from the enemy. Praying the whole time, he said that he kept repeating, "Just let me get one more, Lord," and when he had done that, "Just one more ..." In the end he had personally rescued 75 of them.

But he wasn't done yet. In subsequent action he was severely wounded himself -- and even then it was in the act of throwing aside a grenade before it exploded, saving more lives. After spending five hours waiting for stretcher bearers to carry him off the field, he gave up his stretcher to a more seriously wounded man. Then he was hit again. This time he made a splint for his own shattered left arm from a rifle butt and crawled to safety, finally saving himself.

I'm getting choked up again just remembering all this. Some of the other guys in his unit, the same ones who had been saved by Doss after earlier ridiculing him, were feeling the same way. A Bronze or Silver Star was not enough for this man. They submitted his name for the Medal of Honor.

Harry Truman, who awarded the medal in a newsreel of the time, is supposed to have said this was a greater honor to him than becoming President. Doss was headline news, a genuine and unprecedented kind of hero. There was even a comic book version of his story, which became the inspiration for director Benedict's making of the film many years later.

Alas, honoring the hero is one thing, and caring for him is another. Doss was 100% disabled as the result of his service. Besides his damaged arm, and legs filled with shrapnel, he lost a lung to tuberculosis contracted in Okinawa. Then the army doctors made things worse by giving him an overdose of antibiotics that left him completely deaf. At the time of the documentary, in 2002, he could only hear somewhat with the aid of a cochlear implant.

For years he declined offers from Hollywood to make movies about him. He preferred to live out his days in peace and quiet on his small farm in the backwoods. Over time he was all but forgotten. But it's not too late to remember him now.

Desmond T. Doss. A man who simply lived his simple faith, and made it count.

Make Us Proud, Mr. President!

The change has been going on ever since the nation was founded. The struggle has been between the idealistic principle expressed in the Declaration of Independence that "all men are created equal" and the compromise that was written into the Constitution which gave representation to states based on all of their free citizens and three-fifths of their slaves.

Like an infectious virus the ideal has worked its way into reality, through the mortal fever of the Civil War, the symbolic victories of people like Jackie Robinson, and the adolescent growing pains of the civil rights movement in the 1960's. It found its most eloquent voice in the person of Martin Luther King, and its greatest tragedy in his loss.

But last night Barack Obama extended King's promise, "that we as a people will get there," beyond the boundaries of race to include all of us, both in this country and the rest of the world.

I'm old enough to remember the years before the civil rights movement began to change the culture of America. When my family arrived in Miami in the 1950's racial segregation was still deeply entrenched, even more so than it was in the New York we had left behind. My parents were embarrassed about the "Colored" signs that everywhere denoted the second class restrooms, waiting rooms, seating areas, and water fountains reserved for those who were second class citizens.

I particularly remember the water fountains. A common arrangement was to have a small, non-chilled fountain attached to the wall next to the refrigerated ones for whites. Sometimes there were cup dispensers labeled "cups for colored." Once in my ignorance I tried to use one, and a passing man told my father that I shouldn't. My father didn't believe in the system, but he was practical, and so told me that I should save the cups for the people who had no choice but to use them. He was right. The next time I passed the same fountain I saw that the dispenser beneath the "cups for colored" sign was empty, as perfect a symbol as I can imagine for the vacant promise of "separate but equal."

I was 16 years old when King delivered his "I have a dream" speech in August of 1963. Home from school for summer vacation, I watched the whole pageant unfold in fuzzy gray images on a small black and white TV. The size of the crowd and the mix of ethnicities in it formed a backdrop as powerful as that of the Lincoln Memorial. I knew I was witnessing history, live as it happened -- a feeling I have seldom had since. There was the moon landing in 1969, the fall of the Berlin Wall in 1989, and last night an acceptance speech in Chicago.

We changed the Constitution in the 1960's so that it was no longer at odds with the noble sentiment in the Declaration. But reality has been slower to come around. It took two years from King's speech before my high school was finally integrated during my senior year. Then the civil rights issue was swamped in the upheaval of the Vietnam War and the larger cultural and generational struggle that is still going on today.

But we've reached another turning point, another day when we can say that things will be different from now on. I've never felt more proud of my country and its citizens. The best thing of all, as revealed in the demographic studies of the electorate, is that we have all done this together, across all the traditional lines that have ever divided us.

"We shall overcome" is not the only part of the old song that has come to pass. It is also true, as it says in the second verse, that "we'll walk hand in hand someday."