Part Four: Open Life

Part Four: Open Life

Part Four

in which Harry Potter is magically translated into German, ants screw up, Google gets competition, and the tangible meets the intangible.

Open Life

Rob McEwen wasn't the son of a gold miner, but his father often told him stories of the gold fever in the nineteenth century. Perhaps these stories played a part in Rob's 1989 decision to leave his job as an investment advisor and become a majority owner in a gold mine in the Red Lake region of Ontario.

Unfortunately, although Rob's gold mine was suffering, it wasn't from gold fever. Mining costs were high and the price of gold on the world market was low. To make matters worse, the miners went on strike. However, Rob didn't lose faith in his mine. Some 18 million ounces of gold had been mined in the Red Lake area and a mine close to Rob's had produced 10 million ounces. So far, only 3 million ounces had come out of Rob's mine, but he remained convinced that the same lode, or vein, must continue into his mine. They only needed to discover where!

In 1999, Rob attended an IT seminar at Massachusetts Institute of Technology (MIT). As it happened, one of the topics was Linux and a new and revolutionary way of making software. Rob got excited. This Open Source thing was just what his gold mine needed!

So in March 2000, Rob went public with a competition called the Goldcorp Challenge. All the data generated over the years of working the mine was published on the competition website, and the task for the entrants was to predict where the company should next drill their mine. The competition was open to everyone and in total the prizes amounted to $575,000.

Rob's idea was unheard of in the mining world. Every gold miner knows never to talk to anyone about their findings - they'd rather take their mining secrets to the grave. To begin with, Rob's own geologists were against his plan. Revealing the mine's data might, for instance, lure buyers into trying to wrest the mine away from Rob and the other owners. For a number of reasons, showing your cards in the mining business wasn't a good idea. Or so people had learned to believe.

Even so, the Goldcorp Challenge proved a success. Naturally, researchers from around the world were interested in this unique opportunity to gain access to the data from a genuine mine. Interested scientists from more than 50 countries visited the site, and files were downloaded more than 1,400 times. Eventually, the competition judges chose a virtual three-dimensional model of the mine created by two Australian companies - Fractal Graphics and Taylor Wall & Associates - as the winner. Using their model the Australians, who had never set foot in Canada, could predict where gold would next be found. Their prize was $105,000.

So what happened to Rob's mine? Did he hit the mother lode of Open Source? So far, Rob's company has mined where indicated by four of the five winning entries, and each one has proved its worth. In 1996, the Red Lake mine produced some 53,000 ounces with a production cost of $360 per ounce. In 2001 - the first post-competition year - they mined 500,000 ounces of gold and as the ore was richer than those they'd previously mined, the production price per ounce of gold was just $59. As the price of gold on the world market was $307 per ounce, the company made a big profit. In contrast, continuing with the old methods and production costs would not have been possible at such a low selling price. Without Rob's Open Source idea and the Goldcorp Challenge, the mine may well have been shut down.1

In this final part of the book I will extend the idea of openness in the world of computers into the world we live in. As Rob's experience demonstrated, openness can be made to work even in the mining of gold. Where else might openness be beneficial? Like the tale of Rob and his golden challenge, many of the stories in this section are from real life. Others are suggestions, parts of the open revolution that have yet to be realized. However, it is important to remember that they are all just examples: the point of Rob's and the other stories is to show that the Open Source way of thinking can be applied anywhere.

Rob's story has nothing to do with Linux or computers, but it does tell us that Open Source need not only be the special privilege of computer programmers. By challenging, through open thinking, the conventions of mean-spirited secrecy we can all be as revolutionary in our own fields as Richard Stallman and Linus Torvalds have been as programmers and Rob McEwen as a gold miner. Through open thinking, you can hit gold - literally! We already have open code, but why settle for that when there's so much more on offer? It's only a matter of altering our ways of thinking, of doing various everyday things and of living our everyday life. And that life can be an Open Life.

hingo Sun, 2006-08-27 19:23

Comments

Anonymous friend (not verified)

Thu, 2010-03-04 21:08

Something is messed up with character encoding. I viewed this site in utf8 and iso-latin, either way it's still bad. Apart from it - very valuable content here :)

Thanks for the nice feedback!

Yes, I think the encoding got messed up when I switched hosting provider a couple years ago. I thought I had fixed it, but I see now most of the book is still messed up. Will have to fix it some day.

Literature

Literature

The "ownership' of software is based on copyright law, and the General Public Licence (GPL) formulated by Richard Stallman is based on a new application of this law. In consequence, one might think this Open Source thinking could be applied to other works that fall under copyright law. And of course that has already been done. First, there was literature.

The invention of the printing press is what prompted the evolution that led to the copyright laws we have today, which makes it a natural starting point. Financing book publishing also resembles the software industry in a number of other ways. The actual creative work is in writing the book, after which any number of copies can be printed - not for free, perhaps, but at a reasonable unit cost. In today's Internet world, it's naturally also possible to distribute a book in digital form, which truly does make copying costs nonexistent.

So, why aren't books published according to the principles of Open Source? Perhaps the reason is simple: people don't think of doing it.

On the other hand, it might also be that Open Source doesn't give quite the same edge in writing a book as it does in computer programming. After all, writing software is usually a very complex process which requires the input, often of hundreds, of people for several years. In this complex process the benefits of openness compared to closed systems is obvious (as is proved by Linux, Apache, etc.).

Writing a book, then, is not so complicated. Actually, your average book can be written by one author, and it doesn't necessarily have to take even a whole year (provided you don't write only at the weekends, in addition to having a day job, which is how I wrote this book). But to collaborate with others to write a book, one would have to be able to divide the workload in some sensible way, and for many books this would be very difficult. If it were fiction, for instance, the writers would have to discuss their plot very carefully, agree on characters and so on, in order for the story to be told with any coherence. In the end, it would be far easier for one person to write the book on their own.

Even so, Open Source books do exist. But rather than being plot-thick detective stories they are more likely to be, for instance, Linux manuals.1

At a very early stage, when the first GNU books were beginning to take off, it was clear that a free program should naturally have a free manual. And behold, writing manuals works fine within the framework of the Open Source process. It's easy enough to divide the parts of a manual for a number of writers to work on. Another benefit is that there's usually a need to update the manual whenever a program changes and evolves. This too makes an Open Source manual better than something not just anybody can change, as most of the old manual can usually be employed in the revised one. It's only necessary to change the parts that relate to what has changed in the new version of the software, to add new chapters for the new features, without having to rewrite the whole book. Such a process would be more complicated with conventional books under current copyright law, which would not allow an out-of-date manual to be updated without permission of its original author/s. Which is why members of the Open Source community always include a text specifically granting this permission to all and sundry every time they release a manual or the code for a program.

And because that is likely to discomfit a lot of publishers, this is a good place to reiterate what has been said about software in earlier chapters. There are bound to be people who feel that publishing a book on the Internet under Open Source terms is a bad idea and will be the ruin of the publishing industry. But this is roughly the same thought that struck traditional software businesses when they first heard of Linux. In actual fact, the situation is the reverse. An Open Source book which is available on the Internet is like free money for a publisher. It's a book that is already written and only needs to be printed for the money to start rolling in. And lots of smart publishers have actually realized that. Apparently, the publisher of most Open Source books is O'Reilly, the same publishing house that once upon a time put Larry Wall, the creator of the programming language Perl, on its payroll and thereby gained its role as purveyor of books to the Open Source movement. But O'Reilly is not the only publisher to discover the Open Source market. Prentice Hall actively seeks Open Source writers for a series which has as its Series Editor the spokesperson of the Open Source community, Bruce Perens.

So there are some Open Source books. But we've slid back into the field of Linux. So is there any other Open Source literature outside Linux manuals?

  • 1Linus Torvalds' grandfather, Ole Torvalds, was actually involved in a detective story collaboration with a number of other Finnish authors writing in Swedish. Each author wrote one chapter, but there was no discussion of plot or characters; the manuscript was simply handed on to the next writer, who had to come up with a way of moving the so-called plot forward. The collaborative book was published in 1950, but it was not a bestseller. The title Den rödgröna skorpionen, which translates as The Red-Green Scorpion, may have had something to do with that, as it's a title that sounds almost as classy in Swedish as it does in English!
hingo Sat, 2006-09-02 06:21

Harry Potter and the magic of Open Source

Harry Potter and the magic of Open Source

In the autumn of 2003, a news item about German Harry Potter fans caught my eye. In June of that year, Harry Potter and the Order of the Phoenix, fifth in the series of books so beloved by children, youth and adults, had been published.

At least, that's when the English-language edition was published. The German edition, having yet to be translated, would have come out some time after it. But a true fan, rather than wait for the German edition, would rush out and buy the English edition, then spell their way through the latest adventure in the original language. On publication of the German edition, it too would be acquired by the fan.

That might be enough for your ordinary run-of-the-mill kind of a fan, but true true fans take their commitment even further. They do the translation themselves. And, naturally, they cooperate with other fans to get it done quickly.

Not long after the publication of Order of the Phoenix in English, more than a thousand German Harry Potter fans were organized on the website www.harry-auf-deutsch.de. The average age of these volunteer translators was sixteen, and many of them had done only two or three years of English at school.

The job was divided into five-page sections. Some volunteers took part by proofreading the texts translated by others, and also by commenting on alternative translations. To translate the vocabulary specific to the magical world created by J.K. Rowling, a Harry Potter English/German dictionary was set up on the site. A lively debate then ensued about the translations offered in the dictionary. For instance, how does one translate the word squib (a person whose parents are wizards, yet he or she is born with no magical powers)?

As you might expect, the project quickly ran into the German publisher's lawyers, because the law prohibits the circulation of unofficial translations of a book over the Internet without the permission of the originating author, usually via their publisher or agent. Fortunately, the Harry-auf-Deutsch website was able to make peace with the German publisher by making the site a closed club. Thus, the final result of the translation effort isn't available to all and sundry, but only for sharing among those who partook in the work.

Harry Potter fans in other countries didn't fare so well. For instance, fans in the Czech Republic also tired of the nine-month wait for the official translation. So, like their fellow fans in Germany, they too produced an unofficial version much sooner. But the site hosting the Czech translation was closed down by the publisher in possession of the Czech rights to Harry Potter. Naturally, this incurred the wrath of the same young book buyers whom the publisher wanted to woo.

Although such Potter translations stretch the limits of copyright law, they do also show the power of Open Source thinking. Where most professional translators spend almost a year translating one novel a group of hundreds or even thousands of volunteer translators could get the job done in a couple of months - or weeks, if less proficient work was acceptable. And the overall quality of the translation needn't really suffer compared to a professional translation. Quite the opposite. If the magic of the Linux phenomenon can be applied to the translation of a novel, it just might be rather exciting to read an Open Source version well before the closed and official published version had come out.

With Harry Potter, J.K. Rowling may have inadvertently inspired a way of translating books that suits the Internet age, but the young wizard has also taken over the number one spot in a more traditional literary sub-culture. For a long time, so called fan fiction literature has offered popular characters in novels and films a life outside their published stories. In a fan-fiction story a well-known character from a series of books or a TV series takes part in fresh adventures written by someone other than the character's creator. Such stories are written as a hobby by loyal fans of the character. Often enough, fan fiction is written in a short story format, but it is fairly common for such fans to write whole unofficial novels. But again, I must stress the words "as a hobby', because selling a story based on someone else's character without the permission of the original creator would be plagiarism and therefore an infringement of their copyright. Actually, it's not exactly legal to distribute the stories over the Internet either even if they are made available free for others to read, but that doesn't seem to bother these fans.

Naturally, the Internet has given the whole fan fiction culture a boost. A popular site for fans of fan fiction, called www.fanfiction.net, publishes new stories on a daily basis. Last time I checked, Alexandre Dumas' three musketeers were fencing their way through an additional 42 stories, Sherlock Holmes solves 347 previously unpublished mysteries, and there are even 91 fan-fiction stories about Homer Simpson. With 8,800 stories, for a long time Star Wars was the undisputed leader but now, nobody can touch Harry Potter - with over a hundred thousand fan-fiction stories, wands have left light sabres far behind. You can read what Harry does during his summer vacations between his term-time activities at Hogwarts in the official stories by J.K. Rowling, or what happens when Harry grows up and goes to university!1

  • 1...where, among other things, Harry begins dating. The R-rated stories are a popular form of fan fiction.
hingo Sat, 2006-09-02 06:28

Wikipedia, an extremely open encyclopedia

Wikipedia, an extremely open encyclopedia

Although most books are best written by one person, there are other kinds of publications that typically have more than one author - sometimes very many. Encyclopedias are one such important category.

Unlike Harry Potter and other novels, encyclopedias are a compilation of independent articles. Many different experts are asked to write articles related to their particular area of expertise. And because the articles for an encyclopedia don't need to be written in any particular order, all participants can be working on it at the same time. So it would seem that an encyclopedia could be compiled and published using Open Source methods. You may not be surprised to learn that there is such an encyclopedia.

Thirty-seven-year-old Jimmy Wales, a fan of Linus Torvalds and Richard Stallman, had a vision of his own for an Open Source project. Since Jimmy knew something about the dynamics of an Open Source project, he also realized that compiling an encyclopedia would fit the ideology of Open Source very well. So, in 1999 he founded the Nupedia project.

However, Nupedia didn't become what Jimmy had planned. Over a period of two years, the project swallowed a significant amount in grants and resulted in the publication of just twelve articles. Twelve articles in two years! So what went wrong?

In theory, the Nupedia project was open to all - anybody could contribute articles for it. In reality, you couldn't just sign up and become a contributor. If you wanted to write an article about something you had to send in an application, which had to be approved at some point, somewhere. All articles were checked and rewritten several times, by other experts - because that is how encyclopedias are usually written. And after two years, only twelve articles had passed this scrutiny! Obviously, that is not how to organize an Open Source project.1

In January 2001, the failed Nupedia was replaced by Wikipedia.2 It used the WikiWikiWeb technology which became widespread in the late nineties. A wiki is a web page that users themselves can read, write, and edit. Originally, wikis were used for group work or on Intranet pages used within a company, but Wikipedia has certainly proved what good use this technology can be put to on the Internet, where everybody has access to the pages.

In a wiki, each page has, for instance, a link at the end that allows users to edit the page immediately. And they don't even need to have a special program to do it - the page they want to edit opens up in the user's own web browser.3 In some cases, no user ID is needed to enable anyone to edit the page, but even where an ID is required it is only necessary to register, and as soon as that is done you are free to edit the page. Trouble-free participation in the editing process is crucial, as all sorts of checking, pre-censoring, and approval of new users is anathema.

Most people react with incredulity when they first hear of wiki technology. Surely it can't work for real! I admit I thought that myself, a long time ago.

If a page can be changed freely by anyone on the public Internet, isn't that asking for trouble? You might expect all sorts of vandals would be rushing in their hordes to make the most of this opportunity.

But despite access and efficiency being central to the wiki philosophy, when it comes to giving people their chance to contribute, this doesn't necessarily mean that no checking is ever done. In addition to writing new articles, the vast army of volunteers involved in the creation of Wikipedia are also involved in checking the changes that are made. On an encyclopedia site it is necessary to make sure the facts are right and that everything gets proofread, which is just as important as writing the articles themselves.

All wiki programs have convenient tools to make checking easier. All the latest changes are found centrally on a page of their own, which makes it easy to look them over. The Diff tool shows all changes, which means you don't have to read the entire page again if you don't want to. And if the changes were done out of malice or you just don't happen to like them, then changing things back is just as easy and accessible to everybody as it was to make the changes in the first place.

Of course there is some vandalism, but attempts to ruin things for others tend to be very short-lived. The vandals are so outnumbered by those who take the creation of the encyclopedia seriously that you rarely see the troubles in Wikipedia before they're gone. But some targets are just too tempting for mischief-makers. Today, you are no longer free to change the front page of the Wikipedia site, because it became a sport for some people to insert large pictures of penises on it.

Ants are a classic example of diligent workers. This is something that parents want to show their children, at some point taking them to an ant hill to gaze in wonder at how big a hill the tiny ants have built. (The lesson being that teamwork makes greater things possible, and that one should work hard… But you'll know all this because you too were taken to watch ants.)

I once read somewhere about a study which showed that about 20 per cent of the ants in an anthill do totally stupid things, such as tear down walls the other ants have just built, or take recently gathered food and stow it where none of them will ever find it, or do other things to sabotage everything the other ants so diligently try to achieve. The theory is that these ants don't do these things out of malice but simply because they're stupid.4

I expect you're thinking what I'm thinking, which is that you know of some human organizations in which the behaviour of at least 20 per cent of the people in them is just as idiotic! And that's why many of our organizations formulate ways of preventing the kind of problems that plague ant colonies. If a number of people do some stupid thing, we make a rule to say it mustn't be done. Then we need a rule that says everybody has to read the rules. Before long, we need managers and inspectors to make sure people read and follow the rules and that nobody does anything stupid, even by mistake. Finally, the organization has a large group of people who spend their time thinking up and writing rules, and enforcing them. And those not involved in doing that are primarily concerned with not breaking the rules, which means they do what they should be doing very carefully.

Critics of Open Source projects claim that their non-existent hierarchy and lack of organization leads to inefficiency. With nobody overseeing the whole, a number of people may inadvertently do the same work and others might do something totally unnecessary, something nobody will ever need. Instinct tells us that to avoid such problems we must increase the number of rules and managers and put more effort into planning.

However, Linux and Wikipedia prove the opposite is true. Rules and planning, that's the most pointless work there is. Instead, everyone should just go ahead, do what needs to be done and spend their energies on actual work. Planning creates Nupedias (and, according to Linus Torvalds, Hurd kernels), whereas the cheerful work that reminds you of an anthill produces the Linuxes and Wikipedias of the world.

This is particularly true when you factor in that not all planners are all that smart. Which means organizations risk having their entire staff doing something really inane, because that's what somebody planned. So, it seems better to have a little overlapping and lack of planning, because at least you have better odds for some of the overlapping activities actually making sense.

According to wiki philosophy most people can and want to do what's right - they actually want to do their best. Inhibiting this natural inclination with a plethora of rules and unnecessary checking procedures is rather more moronic than the limited amount of trouble caused by a small minority of aberrant "ants'. And despite what that 20 per cent of ants do, every summer millions of parents take their children to admire an ant hill, not to criticize it.

Of course, Wikipedia does have rules which writers should follow. The most important rule puts the wiki philosophy in a nutshell: If rules make you nervous and depressed, and not desirous of participating in the wiki, then ignore them entirely and go about your business.5

The two years spent working on Nupedia were essentially a waste of time. Once bitten, twice shy, they say - but apart from the lesson of experience, there wasn't much else to take away from it. At the time of writing, Wikipedia has just turned three. So, how is it doing?

In the early days of Wikipedia, Jimmy Wales had the company of Larry Sanger, who was paid to work on Wikipedia for the first 13 months. This is an interesting detail, if you compare Wikipedia to Linux. Linus Torvalds spent about a year working alone on Linux before other programmers from around the world began little by little to produce an increasing share of the code for Linux. In both cases it took one person to get the process going and give them a shove in the right direction, after which they developed their own momentum and kept going.

If you happened to use Wikipedia in its first year, the subject you looked for often wasn't there, because nobody had yet written an article about it. In accordance with the wiki philosophy, the accidental surfer wasn't then greeted with an error message but instead with an empty field for text and a polite request for the person to write the missing article. Little by little the articles came together, though the quality of the entries varied. Some were written by kids in schools corresponding to junior high, writing in English which was not their mother tongue, while others were penned by learned professors. The voluntary nature of the work naturally influenced the articles. Early on, Wikipedia had a very comprehensive presentation on the headwords "beer' and "Linux'. The article on "Microsoft Windows', on the other hand, presented as obviously important the emulators which enable a user to run Windows software on Linux! You might say that Wikipedia is an encyclopedia that is the way it is because of the people who write it and use it.

Today, Wikipedia has volunteer editors, so-called Wikipedians, in the thousands. In January 2002, i.e. when Wikipedia was a year old, a thousand articles were written or revised each day; in January the next year the number had doubled to 2,000 a day. At the time of writing, the number of articles is approaching 200,000 and in addition to that there are a total of 55 other-language versions of Wikipedia. For instance, there were already a hefty 3,200 articles in the Finnish Wikipedia.6

Interestingly, statistics show that in the summer of 2003, more people visited the Wikipedia website than visited the web version of the venerable Encyclopaedia Britannica. The comparison isn't quite fair, though, as the Britannica's articles are only available to paying customers, but it does confirm that the three-year-old Wikipedia is the most popular Internet encyclopedia. And Jimmy Wales is actually planning to challenge the Britannica on its home turf. In the near future we will see the release of Wikipedia 1.0, a selection of 75,000 articles both in print and available on CD-ROM.7

Whenever there is talk of Wikipedia, people often point out that the king of all dictionaries The Oxford English Dictionary was also an Open Source project of sorts. The first edition, published in 1928, listed more than 400,000 English words in twelve volumes. The editors worked on the first edition for a total of 54 years, allowing the first Chief Editors to have died before it was actually published.8

Apart from the sheer number of words, the reason for the drawn-out process was that much of the OED was written by a large number of eager volunteers who were interested in linguistics. They collected words from newspapers and fiction as well as in markets and the local pubs. The words and their meanings were then mailed to the Chief Editor, whose job it was to coordinate the project. One of the better-known volunteers was J.R.R. Tolkien (1892-1973), who is said to have laboured mostly with words beginning with the letter "W'. (For your information, Tolkien fans, the entries for warm, wasp, water, wick, wallop, waggle and winter were originally by him, so now you can rush off and read some more Tolkien wisdom.) Another volunteer, William Chester Minor, was convicted of murder and continued to send in his contributions from prison. And a Mrs Caland in Holland is said to have remarked that she couldn't stand her husband's never-ending work on "that wretched dictionary'.9 Although the OED in its final published form was not available for free copying, it would seem that the hacker spirit of working has been around for a long time.

  • 1So, how should an Open Source project be organized to make it as successful as possible? See Eric S. Raymond's famous essay "The Cathedral and the Bazaar': https://www.catb.org/~esr/writings/cathedral-bazaar/
  • 2https://wikipedia.org/
  • 3Typically, html code is not used when creating a wiki page, as that would be too difficult for an average web surfer. Instead, the contents of the page are written in a very simple code that resembles ordinary text and which the wiki program then converts into an html page.
  • 4OK, so there ought to be an acknowledgement here for the source of this story, but I've managed to lose it. Surely a little thing like that shouldn't keep a guy from telling a good story!
  • 5https://en.wikipedia.org/wiki/Wikipedia#Policies
  • 6https://fi.wikipedia.org/
  • 7Wired magazine wrote about Jimmy Wales and Wikipedia in November 2003 . The article cites many other examples of using Open Source methods in various non-technical fields of human life. Some of them have been mentioned in this book, some not.
  • 8But please note that The Oxford English Dictionary is a dictionary, whereas Wikipedia and Britannica are encyclopedias. An encyclopedia aims to tell you "everything you ever wanted to know', whereas the aim of a dictionary is to catalogue all the words of a language together with a guide to their pronunciation, etymology and meaning. A parallel project to Wikipedia, called Wiktionary (https://wiktionary.org), aims to produce a multi-language dictionary. At the time of writing, Wiktionary has been around for a year and the English edition has more than 30,000 entries, which is rather less than 10 per cent of the first edition of the OED.
  • 9From The Meaning of Everything, The Story of the Oxford English Dictionary by Simon Winchester, quoted at: https://www.ralphmag.org/CI/oed.html and https://www.askoxford.com/worldofwords/oed/wordsearchers/?view=uk
hingo Sat, 2006-09-02 06:55

dmoz.org, open web page index and MozDex, open search engine

dmoz.org, open web page index and MozDex, open search engine

In addition to Wikipedia, a number of similar projects also deserve mention. Older than Wikipedia and Nupedia is the dmoz.org Open Directory Project, which is an Internet link directory.1 The first and still most popular similar (closed) directory is an icon of the Internet, Yahoo!.2

In 1998 it looked like automated search engines had come to the end of the road. The then most popular search engine, AltaVista, couldn't cope with the explosive increase in web pages, rendering the search results more and more useless. The sheer number of web pages made it hard to find what you were looking for and the situation was made even worse by various advertising companies who had learnt how to abuse the key word directories of web pages to get to the top of the search results lists. The situation resembled the one we now have with e-mails: whatever one did, no matter how proper were the words used in the search, the top of the search results contained pornography and - well, mostly pornography, because Viagra wasn't known then.3

That meant that the future seemed to be more and more dependent on directories made by human effort, such as Yahoo!. This was the niche that Chris Tolles and Rich Skrenta came upon. To them it was clear that gathering the fast-growing Web into any sort of directory cried out for an Open Source approach. So, in June 1998 they founded a project to do just that, and called it GnuHoo - it's a small world, isn't it - or open Yahoo!. Since it wasn't an official Gnu project, they did as Richard Stallman requested and changed the name to NewHoo. Later on, Netscape, which was one of Yahoo!'s competitors, dropped its own directory project and bought NewHoo to be the basis of its own portal and made it the Open Directory. It finally found a home at dmoz.org.

Dmoz, or NewHoo, was a success from the start. In the first month alone it had scored some 31,000 links, which 400 volunteer editors had organized into 3,900 categories. Only a week after that there were 1,200 editors and 40,000 links!4

In the same year, 1998, AltaVista lost its position as the leading automated search engine and was slowly forgotten as the lead was taken over by a newcomer, Google, which thanks to a highly-developed PageRank algorithm could once again make some sense in the order of the search results. That also tipped the balance in the competition between automated search engines and edited ones in favour of the automated ones. However, it is worth noting that Google and many other search engines use dmoz as one source of information in the creation of their own database. So, whenever we use Google, we are in many ways using Open Source. First, the Google servers are based on Linux and Open Source code; and second, the Google search engine uses an Open Source source of information.5

Though the edited directories lost the fight to Google, the Open Source community didn't give up. In April 2004 a new contender joined the search engine competition: MozDex.6

MozDex is a perfectly open search engine. Not only is it based on Open Source code, but the intention is also for it to be open in the presentation of search results. How do we know that the search result Google lists first is really the best? Can we be sure there isn't some Google employee who has fiddled with the database or that they haven't sold the number one spot to whoever pays most? Even though we do trust Google, we cannot be completely certain. MozDex aims to provide search results that offer all users the opportunity to check why the links which come highest on the list really are the most relevant. Next to each search result, there is an (explain) link, which allows you to see on what basis the given search result has scored higher than the other pages in the database.

In the words of the programmer of the MozDex search engine Doug Cutting, "The number of Web search engines is decreasing. Today's oligopoly could soon become a monopoly, with a single company controlling nearly all Web search for its commercial gain. That would not be good for users of the Internet.'

At the time of writing MozDex is still an experiment and its database doesn't yet contain all the web pages on the Internet. But April 9 2004 will remain in history as the day Open Source joined the search engine competition. Google won the first round, but will Open Source make a comeback? That remains to be seen.

hingo Sat, 2006-09-02 07:08

Books for schools, colleges, and cooks, other facts and some fiction

Books for schools, colleges, and cooks, other facts and some fiction

As with Linux manuals and encyclopedias, the content of school and university textbooks tends to be relatively easy to divide into sections, which makes them ideal candidates for being written by a variety of authors using the Open Source method. Schoolbooks are also an enticing field because teachers often produce their own material to use alongside the official textbook for the course they're teaching, particularly at universities. For some university courses a suitable book may not even exist, which makes the students entirely reliant on their lecturer's transparencies. Some courses also cover subjects that simply aren't found altogether in any one book, but rely on a variety of material being brought together from several different books.

What could be better suited to Open Source than that? If the books were free for teachers to use - and by "free' I mean precisely the Open Source kind of freedom, free right to copy anything or change things - a teacher or lecturer could easily combine them to create a book perfectly suited to the curriculum being taught. The most relevant parts of each available book could be brought together, leaving out all unnecessary chapters, and if any chapters do not adequately cover what is necessary for a particular course the teacher could add to or even write their own versions of them. In fact, teachers already often do this, but instead of having a confusing hodgepodge of antiquated books, photocopies and handwritten notes, an Open Source system which allowed copying would enable teachers to compile a properly relevant course book, which could continually evolve and improve, and be updated whenever developments or newly acquired knowledge required new additions to be made.

Open Source course books would also be a good option for students. Towards the end of university studies, it's not impossible for a text book on some elevated technical subject to cost up to €200. An Open Source course book would naturally be cheaper for a publisher to print, just as Linux is cheaper than Windows. In the case of an Open Source course book, students could also elect to print the book themselves, or only the pages they actually needed, which would further bring down the cost of materials - not to mention save trees! In extreme cases a poor student could even read the whole book on the computer screen, for free.

With such compelling arguments for Open Source books, it would be odd if there were not already some on the Internet. But rest assured, they are there to be found. In the past few years the Internet has practically exploded with projects to write Open Source course books. Some projects are only at the idea stage, but others already have material on offer. There are projects that follow a given country's national curriculum, and others that are international, such as for English language teaching. For some reason even the State of California has its own project to produce schoolbooks for junior high school.

It's not easy to get a real overview of this embarrassment of riches for future schoolbooks, but it does seem that the wiki philosophy has once again proved its worth, because one of the most well-developed projects is Wikibooks, which is based on the Wikipedia code.1 At least in its early stages, the project has so far focused on the production of textbooks, although it also includes a number of guides. At the time of writing, Wikibooks is only six months old but there are already dozens of more or less finished books in English, totalling more than a thousand individual chapters; and among the Finnish Wikibooks one book has already reached the complete status. It was a book under the heading Social Sciences entitled Työoikeus (Labour Legislation).2 (Of course, in a wiki nothing is ever completed for ever; the complete status is more like the version number "1.0' of a computer program.)

And while we're on the subject, by the summer of 2003 Wikipedia had spawned quite a number of new projects. In addition to the Wiktionary and other Wikibooks projects already mentioned, there are also: Wikisource, a collection of open texts not originally produced through the wiki method (such as, the Bible, Greek philosophers, the US Declaration of Independence, etc.); Wikitravel, an open travel guide; Wikiquote, a collection of famous quotes3 ; and Wikiversity, a collection of university-level textbooks that uses Wikibooks.

But Wikibooks is by no means the only site for Open Source textbooks; the Internet is simply bursting with high-quality books. One of the most comprehensive directories in this area is The Assayer, which is maintained by Benjamin Crowell.4 The Assayer is not a wiki, it's simply a link directory of open books available elsewhere on the Internet. In addition to the directory itself, the site also offers users a chance to assess and grade the books they've read.

Although in principle The Assayer lists any type of literature, it would seem that most open books are available in the fields we've covered. There are Linux manuals and textbooks. Among the textbooks, mathematics, physics and chemistry are by far the most represented, probably because of their close connection to information technology and therefore to Linux. The next largest group is actually fiction, which is pleasing in itself because it's not something we have to learn for school.

Benjamin Crowell also made his mark as a writer of Open Source course books by writing two series of university textbooks: Light and Matter and Simple Nature. Already Light and Matter - his series on introductory physics - is used as the course books in at least 17 universities. Everything suggests that Open Source literature really has something to give to future generations of students.

And because education really is for life, and not the other way around, it's time to move on to everybody's favourite subject - food. The perfect companion to all the open Linux manuals is Matthew Balmer's The Open Source Cookbook: Fuel for Geeks.5

The book is probably of great use to young nerds who have recently moved into a place of their own and who are about to wake up to the reality of there no longer being a parent around to disturb their long coding sessions with calls to dinner. Because this cookbook really is aimed at nerds, there's a section before the actual recipes that lists what is needed before one starts to cook, which explains the need for basic equipment such as a pot, a microwave oven and a cutting board. After this, there's a run-through of the basic foodstuffs that should be found in every kitchen, even a nerd's; this is the list that takes you beyond coke and frozen pizza. There's also a brief glossary of techniques and terms, which explains the mysteries of having to julienne, knead or marinate.

After "Cooking 101', the recipes are good enough even for a visit from mother. Members of the international hacker community really do share their best with you in the cookbook they have collaborated on at the Slashdot website.6 So, what do you think of a gastronomic adventure that involves deep-fried kangaroo? Naturally, the recipe begins with information on which web stores non-Australians can turn to for kangaroo meat. Oh yes, nerds have the know-how!

Incidentally, Wikibooks also offers a cookbook, so the health of nerds should improve from here on. That is, if they remember to leave their computers every once in a while.7

hingo Sat, 2006-09-02 07:22

Project Gutenberg

Project Gutenberg

One project of great merit definitely deserves to be mentioned here for its archive of free literature. Project Gutenberg has on its server electronic versions of almost all the classics of world literature available for free distribution.1

Unlike in the computer business, which is still very young, a large number of famous works of literature are freely available for copying, publishing, and for distribution over the Internet because they have passed the time limit built into the copyright laws of various countries. Copyright protection lasts a specific number of years following the death of the author or creator of a work, after which it enters the public domain. The protected period of copyright varies from country to country and with the type of work it is, but it the usual period, depending on the country, is usually 50 or 70 years from the death of the creator of the work, or from the date of publication if it was published posthumously.2

Since there aren't many programmers who have died over a hundred years ago, the period of copyright is not something that has affected the development of computer programs, such as Linux.3 But the situation with literature is quite different. Despite the extension of the copyright period, most of the literary classics are already in the public domain, which allows the Gutenberg archives to offer an extensive library of e-books, ranging from the philosophers of antiquity and various translations of the Bible, through Shakespeare's plays and Moby Dick, to Alice in Wonderland and Peter Pan.

Project Gutenberg was begun by Michael Hart as early as 1971, which as e-programs go makes it a really old one. At roughly the same time the UNIX operating system and the C programming language were taking their first tentative steps. It is actually quite amazing that there was even a computer for Michael to use for his project at this early stage. However, the University of Illinois had a computer, and Michael was one of the people to whom the university endowed some computer time.

The other people using the computer at the time mostly did very elementary work on it, such as programming a programming language. That's a bit like making the hammer which will eventually be wielded to make the scaffolding used to build the walls of a house, and only when the house is finished, will others finally get to decorate and furnish it - all quite comparable to cave dwellers discovering fire.

However, Michael Hart didn't want to get involved in such projects, and tried to come up with something else to do with his computer time. After all, at the time this was a resource that would have cost $100 million to buy! You couldn't exactly leave it unused.

Having thought about it for an hour and 47 minutes - so it is said - Michael Hart predicted that the greatest future value of computers would not be their calculating capabilities, but in storing information and in the unlimited distribution and search for information. Not a bad guess! Thus began Project Gutenberg, and the first information stored by Michael was the US Declaration of Independence, which being a public document was common property; that is, in the public domain.

The Declaration of Independence may not qualify for a place in the top ten of Western literature - particularly for non-Americans like myself - but it was more than nationalistic pride that made Michael Hart choose that particular text. There were actually some very practical concerns: it was suitably short, which meant it would fit it on the disks available in 1971. The time for storing the whole of Moby Dick and the Bible came later, as the technology developed.

By 2003, the Gutenberg archives included more than 10,000 e-books, which were published on a 4 gigabyte DVD to celebrate the achievement. The present aim of the project is to reach a million free e-books by the year 2015. Despite its modest beginnings, Project Gutenberg has shown what resources can be found in being free and by using volunteers.4 The project's history also reminds us that the current boom in Open Source thinking isn't really anything new. As Richard Stallman keeps telling those younger than himself, the entire IT business was Open Source in the 1970s. The change to a more closed world came later. The same goes for e-books. Michael Hart was doing it in the 1970s, but it was the twenty-first century before Stephen King tried it.

hingo Sat, 2006-09-02 07:35

Comments

Music

Music

Although the Internet has also revolutionized the music business, most of the big headlines to date have been about Napster, its progeny and the delaying action record companies have been fighting against them. These so-called peer-to-peer technologies are revolutionary in themselves, but there's no denying that their users have crossed over to the wrong side of the law. It is illegal to distribute copyright-protected music over the Internet without permission from the copyright holder.

Even though Napster, which first made it possible for users to swap music files with one another, had to close shop after a court order, it has hardly slowed the sharing of MP3 files. Napster was simply supplanted by a handful of new technologies built in a way that made it impossible to close them down at one central server, as was done with Napster. The record industry, fighting to protect its rights and more importantly its source of revenue, hasn't yet given up what seems a hopeless struggle, but keeps on tossing this or that spanner into the works.

At the time of writing, the record companies have managed to make their already soiled image worse by, among other things, suing the twelve-year-old daughter of a penniless single mother from New York for having used a file-sharing program. Whereas, the companies selling file-sharing programs made themselves even more popular by paying the poor girl's $2,000 settlement with the industry. There was also an Internet collection to get the same $2,000 together. Somehow, it seems the record companies are very much alone in their fight.

Last year a lot of artists added their own voices to the anti-record-label chorus.1 Probably the most well-known of these is Moby, a big name in electronic music who has openly spoken in favour of file-copying fans.2 He says he is genuinely happy that people want to listen to his music and if they can listen to it through file-sharing then that's fine by him. The way Moby sees it, a lot of his fans have actually first heard his music through files downloaded from the Internet, something that makes him happy rather than angry. In addition to Moby, several other musicians are rumoured to be feeding their songs into the file-sharing programs - which is quite contrary to what one would expect, judging from what the record companies say. The group Offspring said in a TV interview that they had even included some MP3 files as bonus material on their latest record but the record company had removed them before it was released.

Formerly of pop-dance duo Wham!, singer George Michael was the first major league pop star to announce that the record he released in the spring of 2004 will be his last.3 The album concluded his deal with his record company and in future he will release all his new songs (of course, he has no rights to the old songs, which are the property of the record company) on his website, where fans can download them for free. There will be no charge for copying the songs, but anyone who wishes to do so can give a donation to one of the charities listed on the site.
The rocking granddads of Offspring are happy to talk big in interviews, lashing out against their greedy record-label masters, but sneaking your own MP3 files into file-sharing programs is not exactly Open Source. They may be angry that a signature on an old recording contract prevents them from putting their own songs onto their own website, and George Michael is certainly leading the way in this, but these are just the first steps. As far as I know, he hasn't agreed to hand out his music for others to profit from or to sell on the same terms as Linus Torvalds did with his Linux code.

But there are some real revolutionaries out there. A small record company called Magnatune states on its front page: We're a record label. But we're not evil.4 The owner John Buckman founded the company after seeing how his wife fared with a standard recording contract she signed with a traditional record company. About a thousand records were sold and his wife, as the recording artist, earned a staggering $45. As a result of the contract, she lost all rights to her own music for the next ten years, which means she cannot put the music she has written on her own home page despite the fact that the record has sold out and cannot be bought or reordered from anywhere. John wanted to found a record company that would offer recording artists a better deal.

The music recorded by Magnatune can be downloaded quite legally either a song at a time or as whole albums from the Magnatune website. In addition to which, the site can be used to create your own personal web radio broadcast by selecting the music you want to hear. (Even realizing such a technically simple and fun idea is practically impossible with traditional copyright practice.) Naturally, one can also order actual CDs from the company, because of course that is what a record company sells. The music available has grown relatively quickly and encompasses a broad range. There are now more than a hundred artists in the Magnatune camp, and the label has already released some 200 albums. It's good to see that different types of music are represented by Magnatune; in addition to the ever-present techno there's a lot of classical music and quite a nice collection of jazz.

Magnatune's music also passes the criteria for openness. Although a separate deal must be made and paid for to use the music commercially, all the music available on the website can be used for any non-commercial purpose according to the rules of Open Source. John Buckman uses the term Open Music to describe the principle. It means anyone can use the music as a background to their own film, or make their own version of one of the songs. Many artists actually make the "source code' of their songs available with a view to this, meaning you can download midi files or given audio tracks. The only condition for reusing songs is that the use must be non-commercial and that the original artists must be acknowledged. And most importantly, the new work must be made available to other people under the same conditions.

This last "ShareAlike' condition brings to mind Richard Stallman's copyleft principle, which most Linux programs use. On the other hand, Open Source computer programmers have seen fit to ease up on the condition of non-commercial use. Because there are some actual costs to the distribution of programs - such as the price of a blank CD and postage - the criteria of non-commercial use didn't seem to further the aims of the GNU movement, which was to spread GNU software as far and wide as possible. History has shown that commercialisation didn't hurt Linux - quite the contrary. The other demands of openness included in Stallman's GPL licence have been enough to safeguard openness and fairness and prevented the creation of monopolies.

Although the conditions for Magnatune's music also include that of non-commercial use, the company's existence and work is a step in the right direction, particularly when compared to the exploitative capitalism of traditional record companies. Perhaps the requirement for non-commercial use is just an evil we have to live with during a transitional stage and something that can be discarded in the future.

The next step has actually been taken by a record company called Opsound.5 The music available on the Opsound website is made freely available in any shape or form, there's only the ShareAlike clause to ensure the material continues to be free and open in the future. Which means the music from Opsound could be used in a film that is sold to a television channel, provided the television channel is given the same right to use the whole of the film in the same way and the channel commits to passing this right on to all its viewers. This model is beginning to look very much like the GPL licence which is used to sell Linux. So, it's looking good!

Statistics show that Opsound has yet to reach the same levels as Magnatune. The number of artists is roughly the same and the music files number around 400, but at the time of writing, the first record is only just being released. The music available through Opsound tends to lean towards machine music, but an interesting specialty is the recordings from various environments that might be best classified as sound effects rather than music. My own favourite is a piece called Pildammsparken, a recording from a Swedish bird lake.

I will end this review of the open music scene with the joyful piece of news I read some time ago in a small local paper of Pietarsaari, the town where I was born. A composer-cum-lyricist called Iiris Lumme had published a song book containing a hundred or so of her own songs, and the joyful thing about the news snippet was that she states categorically in the preface that "all songs may be freely copied and sung; that's why they've been printed here.'

That is indeed why music has been made throughout the ages. But at some point we seemed to have forgotten this, and music became the property of record companies, which meant that artists and listeners have had to ask them for permission to enjoy what they like doing. The example set by Iiris is touching in that this old lady has probably never even heard of Linux and may not have a clue about what it means to share MP3 files over the Internet. To her it was just the natural way to publish her songs. They were written to be sung.

Iiris's openness has also brought her some modest financial success: I, for one, naturally rushed to order a copy of her song book, despite the fact that I'm not much of a singing man. And I suspect there are more people out there who'd like a song book like that - with songs you can sing!

hingo Sat, 2006-09-02 07:47

Comments

The music industry is still having trouble tackling the internet beast. Their efforts seem to continue to be focused on pushing for draconian legislation, randomly suing teenagers for thousands of dollars in damages, and applying new Digital Rights Management technologies. Among these, Sony hit a low bottom by actually employing techniques usually seen in virus software.

So instead of seeing a revolution in Open Music, the last years we have actually more so had a fight whether we will have any music at all on the internet in an accessible format. There is some hope though, as Yahoo was the first to try out publishing music from a big pop star in standard mp3 format. Sadly their crappy online music store did not allow me to actually buy the song since they seemed to have bugs that prevented all Finns (or all Non-Americans probably) from succesffully buying anything. (Not that I'm a fan of Jessica Simpson, but I encourage you too to buy that file, if you can that is.)

John Buckman is still going forward with Magnatune and they are still not evil. Some other companies also sell Creative Commons licensed music. One to note is Fading Ways Music who's founder Neil Leyton is not only active in Creative Commons circles but actively producing records and going on tours himself. I had the pleasure of listening to 2 of his concerts in Helsinki and can affirm that he has a great voice. I also participated in the same panel with him at a seminar on Creative Commons and copyright, and I was surprised at the level of details this Canadian knows about European copyright legislation. So if you see Neil on tour somewhere, listen to him singing but also listen to him talking. (And buy his records!)

Creative Commons and building an open infrastructure

Creative Commons and building an open infrastructure

Although some works akin to Open Source have been created in various cultural activities, there are still some challenges to be faced before the revolution reaches the proportions of Linux. These challenges might be termed problems of logistics. In the production of Open Source programs, such problems have already been solved, but in literature, music and the other arts, fitting solutions are yet to be worked out.

One hurdle still to be overcome in the Open Source revolution in the arts is the default position of all copyright law. It forbids everything. The logic of copyright is that all works of art have a creator, and that person owns all rights to their own work, unless they expressly cede them to someone else or, for instance, expressly give permission for them to be copied. In the past, under US copyright law it was assumed that a person's work could be used freely unless such a work had been expressly marked with the © symbol and thereby protected under copyright law. In other words, the maker had to expressly signal that their work was protected if that is what they wanted, otherwise it was assumed that the work could be freely used and distributed.

And although we are interested in freedom here, in principle the new copyright system is better. It protects creative people from greedy and exploitative people or companies, such as some music publishers, whose lawyers could - if copyright law still depended on use of the © symbol - probably come up with any number of technical violations in how the symbol had been used by an author, artist, or musician, and thereby allow corporations to take over a person's works without paying for the right to do so.

But there is a downside to this. The way things are now, one needs permission for everything. Finding a poem, an article, or a song you love on the Internet doesn't yet give you the right to use it in a work of your own. First, you need to get permission from its copyright owner (usually the author or publisher, or via the author's agent), and that can be difficult because few MP3 files, for instance, include the copyright holder's e-mail address or telephone number. So, a certain song may not get sung at a concert simply because it wasn't possible, without an inordinate amount of effort, to contact the copyright holder and ask for permission. At the same time, the world is full of artists like Iiris Lumme, who would love their songs to be sung, copied, and performed in concerts.

This makes it important for composers, authors and artists around the world to learn to take this difficulty into account in the same way Iiris did in the preface to her song book. Regular Joes like us who want to share our songs and words with others must learn to act within the framework of copyright law.

Computer programmers have learnt to do it when they write programs. The General Public License (GPL) developed by Richard Stallman has become the standard and that licence or a similar one is always mentioned at the beginning of any programming code. In just a few lines it immediately advises anyone reading the code that they are allowed to use it in their own programs. The GPL is recognized by Open Source programmers, and just mentioning it is enough to tell all and sundry that a code is free for anyone to share. Without such a mention the code could not be used - copying and using it would automatically be prohibited by law.

For the Open Source revolution to spread seriously into literature, music and the other arts, all originating artists and songwriters such as Iiris Lumme must learn to include a short GPL-like permission at the beginning of their works. That would immediately signal how the work can be used, and eliminate the need for others to seek permission to use it. If this habit were to become widespread, those who publish writing, photographs, or drawings on their websites could significantly influence the amount of material that can be used freely. All great waterfalls begin with small rivulets, don't they? From such rivulets the entire GPL'd Linux operating system was built.

The GPL has also been used in copyrighting books and music, but since the license itself specifically relates to computers and source code, this is not logical. Which is why similar licences, specific to their purpose, have been developed. A collection of licences under the Creative Commons brand has garnered the most publicity and the greatest number of users.1 Some big names have supported this licensing, among them Eric Eldred, who publishes free literature such as titles from Project Gutenberg, and law professor Lawrence Lessig. (Together they opposed the latest extension of the period of copyright inscribed in copyright law in the US Supreme Court. They lost, but only by a whisker.) Creative Commons has also endeavoured to make its licensing options more widely recognized, so that artists themselves learn to add the (CC) logo to their work when they want to permit copying, just as programmers include the GPL reference in their codes.

On the Creative Commons website people writing for the desk drawer or composing their own songs can easily choose a licence that suits their particular work. The process requires almost no knowledge of law, and finding the right licence takes just a couple of clicks of the mouse. You can allow or prohibit the copying and distribution of your work for commercial purposes, or require that your name be included in the list of contributors to any work in which yours has been used. The notice used by Creative Commons is "Some Rights Reserved', rather than the "All Rights Reserved' conventionally included in copyright notices. If you want to make your work totally free, that is put it in the public domain, you can include a logo that bravely states "No Rights Reserved'.

These days, more websites are displaying the logos of the various licences. Little by little, Creative Commons is becoming the GNU movement of the literary and arts world, and the (CC) logo its GPL licence.

In addition to aspects of the law, another logistical problem stands in the way of using free works of art, and that is how and where to find them. Over the years, web pages with tens of thousands of Open Source programs have grown up to help the Open Source movement. For instance, SourceForge is home to nearly one hundred thousand Open Source projects. Such umbrella indexes do not yet exist for literature, music and the other arts. Creative Commons has created their own search engine called Get Content, which looks for (CC)-licensed works. And those I mentioned earlier, Project Gutenberg and The Assayer, are also good reference sources. There is ongoing development in this area, so in the future finding free works will be even easier than it is today.

One of the difficulties of creating a viable system for finding works is the enormous size of the files. The code for a program is only text written by people, and though it is a huge job to collect tens of thousands of programs on one server, it is possible. Similarly, Project Gutenberg has amassed a lot of literature on its server, as books are merely large text files. Even photographs and other graphic works of art can be stored on a computer. However, music and particularly films take so much space on a hard drive that a wholesale storing of them in an index as well as distributing them through the Internet is technically challenging. Using the so-called broadband connections in use today, one three-minute pop song that has been packed as an MP3 file arrives within minutes, but you'd wait for several hours to receive a feature film packed in Xvid format. In reality, offering thousands of movies to be downloaded off the same server is not possible with today's technology.

Also, it needs to be considered that part of the Open Source philosophy is that it is not enough to supply the user with just the end product - a working computer program or a film to watch - but they should also be given the opportunity to take the work further in the same way as an original creator could do. This means that films couldn't be distributed in tightly packaged formats, but would need to be of at least DV quality. Another challenge is that all available raw materials should be available as well - and in a film that is typically tens or even hundreds of hours of film!

So, Creative Commons has managed to sort out the licensing problems, but we still lack efficient tools to find and distribute free works on the Internet. The traditional www server model may never be able to handle the distribution of several hours of video material. On the other hand, the peer-to-peer technology that has gained ground in the past few years - and which is now unfortunately mostly used for illegal copying of music and films - might just do. But who will create a Creative Commons version of Napster?

Here, it is good to remember that sharing simple computer programs wasn't easy in the 1980s either. The Internet was still in its infancy, few people had an online connection, and even for those who did it was so slow that anything bigger than an ordinary e-mail required a long wait. In those days, GNU programs were rarely distributed via the Internet but were sent on tape through the mail.2 In fact, at the time, the mail order business of selling tapes with programs was an important source of income for Richard Stallman. So, despite the technical limitations of the Internet, there is no reason why films could not already be produced and distributed through the Open Source method.

  • 1https://creativecommons.org/
  • 2Floppy disks were only coming into use in the 1980s and they used many incompatible standards. Also, the storage capacity of one floppy was next to nothing. CDs had yet to catch on.
hingo Sat, 2006-09-02 07:51

Open Source movie night

Open Source movie night

We've already dealt with literature and music. So how about movies? I was about to write that the first Open Source film is yet to be made. But actually, one does already exist. The first and, for now, apparently the only feature film available under the Creative Commons licence is Brian Flemming's Nothing So Strange.1

This is a so-called indie (independent) film, produced on a small budget without support from any of the big Hollywood studios. Made in a fictional documentary style Nothing So Strange is about the investigation of the assassination of the world's richest man, Bill Gates. According to the press release, the plot resembles Oliver Stone's JFK, which similarly investigated the assassination of President Kennedy, the disappearance of evidence, the silence of authorities, and other twists of plot.

What a coincidence that the first Open Source movie is about the murder of Bill Gates, founder of the Microsoft monopoly and sworn enemy of Open Source software. And actually it is a coincidence, because the film wasn't originally planned to be Open Source. In fact, it was produced according to the traditional closed methods of filmmaking. At the time, Brian Flemming may not even have heard of Open Source.

In the end, the film distributors were not interested in the film and simply refused to take it on. Apparently a film that kills off a real and hugely influential person wasn't a plot they wanted to promote - or perhaps the film was just plain bad. I can't judge that, because I've not seen it.

In order to keep the film alive, Brian Flemming released as Open Source the raw material used for the film. Considering the plot, you'd think that some Linux fans would have been interested in buying either a DVD of the actual film or, in a best-case scenario, actually have chosen to edit the film material into versions of their own, creating new endings and new twists to the plot.

Although this did give Brian Flemming's film some publicity, no Open Source movement ever really emerged around the movie. Nobody has made any alternative versions of the film and apparently nobody even ordered a copy of the DVD containing the raw material.

So, even though one Open Source movie does exist, I think I'm justified in saying that the first Open Source movie is yet to be produced. Without in any way diminishing Brian Flemming's valuable contribution to works licensed under Creative Commons, it's obvious that the great strength of the Open Source ideology is not in the publication of the final product, but in the open approach taken throughout the entire production process. Flemming has expressed interest in making a movie as Open Source from beginning to end. So, the cliffhanger now is, who will make history by being first to complete a genuinely Open Source movie?

Meanwhile, it's interesting to wonder what sort of film the first Open Source movie would be. What would distinguish it from the bulk of Hollywood productions that we can watch on television any night of the week?

If we start with the basics, naturally the film's screenplay should in itself be Open Source. It could be based on one of the classics available through Project Gutenberg, but it's more likely to be a story expressly created for the film. Lots of people would be involved in the scriptwriting process, cooperating over the Internet. Perhaps wiki technology could be employed in creating the screenplay?

Arranging the actual shoots, could be problematic for an Open Source film production, but would also provide interesting opportunities. Common to the Open Source method is that anyone, anywhere can get involved in creating or changing any given part of the product. In a film, this could pose a problem with the actors, at least those in the leading roles, who are usually the same people from beginning to end. It would be rather confusing if halfway through the movie the role of a character played by Richard Gere was suddenly taken over by Danny DeVito, only to have DeVito supplanted by the Finnish veteran actor Vesa-Matti Loiri!

To use programmer talk, the actor's person is not modular. You can't share a role among several participants. But in the future, perhaps this too will be overcome. The latter parts of The Matrix trilogy already had long fight sequences entirely done with computer-generated imagery (CGI). As computers evolve, it will be possible to take an accurate, three-dimensional, full-body scan of the actors, after which the film can be made entirely on computers while the actors themselves lie in the hot tub. As this technology becomes widespread, it will be possible for the movement and conversation of the characters to be shot anywhere, and anyone with the right skills will be able to do the animation, just as Open Source programming is done. Even today, that is already possible, if one is satisfied by making an animation feature such as Toy Story, Antz or the Hulk. In fact, such films could be made using Blender, the program I wrote about in the Third Part of this book.

In filmmaking, the Open Source approach would be particularly helpful in creating the special effects and animating the actors, because both these jobs demand an incredible amount of computing power. DreamWorks and the other Hollywood studios actually have so-called animation farms to do this work with thousands of super-fast computers all linked to one network. Their only job is to render the animated scenes, i.e. compute the finished image on the basis of the movements and other instructions given by the animator. Despite the enormous power harnessed to do the job, rendering is nonetheless slow work. It can easily take a whole night to render one scene, although with the simplest scenes one might get time for a longish coffee break.

Computer animation is ideal for the Open Source method of working. It would be perfectly possible to get thousands of volunteer Internet users to give processing capacity on their computers for the making of the next hit movie. There is already a lot of distributed computing being done. Popular topics for which distributed computing is used include the search for electronic messages in the omnipresent cosmic background radiation - in other words, extraterrestrial life.2 In its search for suitable proteins to help find a cure, cancer research makes use of distributed computing.3 And over 700 personal computers are being used in the search for mathematical prime numbers.4 The idea of distributed computing is to make use of the idle processing potential when our computers are switched on but doing nothing. Whenever a computer user is having a coffee, is out to lunch, speaking on the phone, or even reading a web page or writing an e-mail, most of the processing potential of their computer is not being utilised. The websites of the above projects allow willing computer users to install a program that will make use of that idle processing power on their behalf and send the results back to the server, which will then collate the results from all participating computers into a common database.

In 1999, an interesting result was generated by distributed computing when Distributed.net took part in a competition arranged by RSA Security to break an encryption based on the DES algorithm.5 Distributed.net participated with a network of nearly 100,000 computers and broke the encryption in less than 24 hours. Curiously, the computing power of the volunteers of Distributed.net was more than double that of the $250,000 super computer that Electronic Frontier Foundation had built expressly for the purpose.6 The DES Challenge clearly showed how strong the Internet fraternity is, even when it comes to raw computer processing power.7

If somebody wanted to use distributed computing to produce the special effects for a movie - no matter whether it was Open Source or a traditional Hollywood production - I'm quite sure millions would volunteer their computer power. Who wouldn't want to walk tall in a T-shirt saying, for instance: "My computer powered special effects for The Matrix.'

But for the actor problem there is also another solution. What if the film had no main characters? Today, few films have many leading roles and often not many supporting roles either. Obviously, this is because actors are expensive, but an Open Source movie wouldn't have that problem. Volunteers around the world would be keen to get involved in making a film. So why not make a movie with tens or even hundreds of smaller leading roles? The number of extras could run into the thousands - as many as one could get to show up. It's been decades since Hollywood has produced a spectacle with thousands of actors, because it's too expensive. Today, all huge crowd scenes in films are computer generated by cloning a group of perhaps twenty or so into a sea of people thousands strong. But our Open Source movie would have no problems of expense. The more people involved the merrier! Just like with Linux.

So, what we want is a script that no Hollywood studio would agree to shoot. It could have dozens of characters and thousands of extras. The events would be set around the world, in the most exotic places imaginable, and there would be so many of them that no single film studio's travel budget would be able to realize it.

Once the film has been shot, using hundreds of actors and camera crews, it would need editing and post-production work. The available raw material would be shared over the Internet and, if necessary, participating "filmmakers' would send each other material on tapes or DVDs by mail. The filmed material would be edited into vastly different segments in different parts of the world, and slowly evolve into the final film. The music for the film would naturally be distributed under the Creative Commons licence, and for sound effects there's the Opsound archives.

The resulting movie would be a true masterpiece. It would be a monument to the collaboration of hundreds of writers, composers, musicians, cameramen, actors, directors, wardrobe artists, builders, animators, and countless others.

But what would the film be about? What sort of plot would work for such a film, with hundreds of roles and locations? Ironically, one good idea for the storyline is the evolution of Linux and Free Software. After all, Free Software is created by very dissimilar people working in various parts of the world during all hours of the day and night. So what could be better than making a film about these people and their work using the Open Source method? Brian Flemming, if you're reading this, get in touch!

hingo Sat, 2006-09-02 08:03

OpenCores - free hardware

OpenCores - free hardware

Taking the Open Source ideology into literature and other creative areas is an obvious step, but there's yet another interesting question to consider. If software can be produced and distributed using the open model, then how about the hardware? If I can use Linux and other Open Source programs on my computer, when will the computer itself be as open as the programs running on it? In a sense, personal computers are an open architecture in the sense that you can build your own computer by using components that meet certain open standards. But today, the components and their production processes are in no way open.

Offhand, you'd think there's no way the open model could work for computer components - surely, what works for software can't work for hardware. The source code of programs, the text of a book, films and audio files all have one thing in common: information stored in digital form can be copied an unlimited number of times, for free. But computer components are tangible parts of the physical world, and here again we hit the traditional laws of supply and demand. Our empirical knowledge of the world says that one computer is one computer and as such can't simply be distributed by open methods.

But in fact, despite our empirical experience, there are projects working towards the manufacture of Open Source computer components. The most well-known of these, OpenCores.org, actually seems to be doing well and is even mass-producing some products for the computer market!1

So what is this all about?

I don't understand about electronics any more than most people, but to go any further we'll need some knowledge about how integrated circuit cards are made.

Some components of a computer are the micro processor, various RAM memories, a modem or network card for connecting to the Internet, a card to connect the screen and another to bring sound to the speakers. All these various components are attached to the motherboard, which in itself is a brilliant piece of electronics containing several microchips. Consumers assume these bits and pieces all come from a factory somewhere, packed into plastic and cardboard, and are eventually fitted together. To that is added a keyboard, a mouse, speakers, monitor and - Hey, presto! - we have a computer.

But in reality the manufacture of these microchips is a lot like programming - but instead it's called design. Actually, separate programming languages are used for designing microchips. The most popular languages are called VHDL and Verilog.2 These descriptions of integrated circuit cards written in VHDL or Verilog are the really hardcore stuff. To be able to read them, you need to understand both electronics and programming, and all they tend to give a regular Joe like me is a headache. But those who understand this language are the people who make the microchips you and I use every day. And I'm not just talking about computers, because microchips are everywhere: in fridges and cars, in wood pulp factories and elevators.

Having the VHDL language files on a computer allows one to use a more or less automated process to manufacture a so-called gate-level description, then to use these logical gates to create the physical model, or layout, of the integrated circuit. The final stage is to make the mask (a sort of mould) which is used to radiate (or cast) small transistors and wires onto a silicon disk.

In this way the manufacturing line at the microchip factory is relatively automatic for making microchips from semi-conductor materials according to a blueprint. The chips are then packed and sent off to the computer manufacturer, the car plant, fridge factory, or wherever. Naturally, there's an art to building and maintaining a manufacturing line like that, and one essential is that these factories must be cleaner than an surgical operating theatre and be entirely dust free. There's a lot of other magic to the process, but we won't go into all that here. One time-consuming part of the work of designing an integrated circuit is the so-called timing. In this world of physics, one must understand that the electric signal in the circuit only moves with the speed of light - no faster. That's why a logical and accurately designed VHDL file won't generate a microchip that works as it should unless the electrical impulse actually reaches from transistor A to transistor B fast enough but not too fast.

For this general explanation, that is as far as we need to delve into that. We just have to believe that what it comes down to is that the data in a VHDL file is used to make a mould which is then used to make semi-conductor materials become a working microchip.

That means the major single effort in making a microchip goes into designing it. That takes a programmer who can write code in the VHDL language. Then testers are needed to check that the chip works as designed. Naturally, the testing too is done before a single chip is made, by creating a model of the VHDL file inside a computer program to simulate how the chip works. Testing is an extremely important part of the manufacturing of microchips and plays a far larger part in the design work than, for instance, in the writing of software, because it's a lot harder to fix a microchip after it's been made. So, microchips have to be virtually flawless before a manufacturing line is set up to produce them in the thousands.

This simplified explanation doesn't really hold true for all microchips. The fast processors like Pentium, Opteron or Athlon, which are made for new computers almost approach the occult, defying as they seem to do the laws of physics. But for those processors, too, design is the largest part of the job, after which making the fancy microchips is no different from making simpler ones. A great deal of fine-tuning of the production line and other hard work goes into squeezing the last ounce of effect out of these high-speed processors. Again, without going into detail, the challenge is to get as many transistors and wires to fit onto the semi-conductor as possible. And once the production line can be made to manufacture small enough transistors, the next task is to get the electrons that transmit the electrical signal to stick to the right wires. If the wires are too small and too close together, the electrical signals will hop from one wire to the next, everything gets confused, and the processor is useless.

That's the case for the newest and fanciest microprocessors. In contrast, as long as you're not out to set a speed record the manufacture of other processors is - so I'm told - very nearly as easy as I've described above. You take the finished VHDL file to a subcontractor who manufactures microchips, and moments later finished processors start popping out the other end.

OpenCores received a lot of publicity in December 2003, when a genuine integrated circuit was made from its Open Source design.3 Several manufacturers have now made the same chip, but the first, historical, Open Source chip was made by Flextronics Semiconductor, or the same company that makes chips for Nokia, of cell phone fame, and Cisco, which makes most of the routers for the Internet. This certainly proved that Open Source can work even in the components part of the computer business - in hardware as well as in software.

It's not as if the OpenCores chips would not have been realized much earlier as so-called FPGA chips.4 Widely used in electronics because they are flexible, FPGA chips are also popular with OpenCores and others who design chips as a hobby or for teaching purposes. It's a sort of general-purpose microchip. It comes empty from the factory and users can "load' it with whatever features they like, which they've first written in VHDL, for instance. The P in the acronym FPGA means "programmable'. In making prototypes the FPGA chips are invaluable, because they are usually reprogrammable. Which means, once a user has tested their chip they can tweak their own VHDL code and reuse the same chip to try the new version.

To "load' - or program, as the professionals say - an FPGA chip, a separate device is connected to a computer for the empty FPGA circuit to be plugged into. However, some FPGA circuits forget all they had contained every time the computer is turned off, and the chips have to reload each time it's turned on again. Actually, FPGA circuits could be compared to CD burners, which are now an integral part of all new computers. We all used to think CDs could only be made at a CD factory, but then somebody invented the CD burner and now anybody can burn a CD with their home computer. Naturally, a home-burned CD won't be of equal quality or as durable as those made in factories, so there's no point in making multiple copies of CDs at home. But thanks to CD burners any music lover can release their own CD if they want. Similarly, your average electrical engineer can make their own processors using FPGA chips.

And although you won't get the effect of a Pentium or Opteron using these general-purpose chips, a modern FPGA chip can already be used to create a microprocessor corresponding to the ancient 486dx. In plain language that means a computer as powerful as those used in the early nineties could be built by a handy electrical engineer entirely from Open Source components! The equipment necessary to program an FPGA chip isn't even expensive. For those interested, various development kits are available from a number of companies, such as Altera and Xilinx, and prices range from around €80 to several thousand. Of course, a really handy electrical engineer could also build the circuit board that is used to program the chip, in which case the whole process would cost next to nothing.

hingo Sat, 2006-09-02 08:12

Comments

Open Collection - fashion, brands, and fabric

Open Collection - fashion, brands, and fabric

The achievements of the OpenCores project have opened up entirely new opportunities for using the Open Source method. The project has already demonstrated that Open Source thinking can be applied outside the virtual world. We've already dealt with computer programs, literature, music and films, and now OpenCores has taken Open Source thinking into the manufacture of tangible products. Does that mean Open Source can be applied to just about anything? Considering the products sold in stores and all the stuff we live with, it's clear that in the end many of them have very little to do with the possession and treatment of raw materials. Instead, many products have a surprising number of intangible characteristics that are well suited to the Open Source method. Computer programs and the intellectual content of books are intangible, which is why the notion of creating and distributing them through Open Source was so obvious. But a lot of intangible work also goes into the manufacture of other things, which is why almost any product can benefit in some way from Open Source thinking. At the beginning of this book, I used oil as an example of a raw material, a limited natural resource for which production and sales follow the law of supply and demand, a mean-spirited logic. However, Rob McEwen's story showed us that there are advantages to be gained from Open Source thinking, even when digging for gold. If such thinking can be used in gold-mining and in the production of microchips, surely it can also be applied elsewhere - even in drilling for oil.

For instance, let's look at clothing. Could clothes be manufactured using Open Source methods? Why not - particularly as programming was born out of a loom.1 It is clearly time to reunite computer programming and the production of clothes, although this time it is the world of fashion that has something to learn from the hackers.

Clothing is a good example of an everyday product, but that doesn't make clothes mundane or unimportant. If you don't believe me, just ask the next teenager you happen to meet.

When it comes to buying clothes, it's not just about the garments themselves. First of all, there's the brand! Many people would rather buy a very expensive pair of Nike sneakers because the brand has high status on the street, than buy a cheaper unbranded pair, even of equal quality. To maintain the high status of their brand, Nike pays Tiger Woods hundreds of millions of dollars to be their public showcase, yet they pay the Asian workers who make the sneakers just a couple of dollars a day.

Of course clothes - like microchips - are also about design. Today, when people talk about making clothes the sewing itself may never be mentioned, but more likely they'll discuss the coordination of cut, fabric and colour - the design. And if we're talking about sneakers - say, Nike sneakers - the designs are created by engineers like myself, because today's sneakers really are masterworks of engineering. They include so many special features, such as soles with air pockets and ventilation holes for perspiring feet, that just taking a walk has become an equipment sport. Once the design is decided, the actual manufacture of footwear or clothing is a minor detail, and the main focus becomes where it can be done most cheaply. Which is why so many clothes are made in Asia, by people who work for a couple of dollars a day.

But that's not the end of the story. Once the Asians have been good and sewn a big pile of clothes or shoes, these need to be delivered to stores in Europe and the Americas, and that is a matter of logistics. And - apart from the trucks, ships and aeroplanes - logistics are mostly about information technology.

Brands, design, logistics, project control, and so on, are intangible and therefore areas in which Open Source methods are strong. But how is an Open Source clothes brand created? In order to understand that step, we must look to Richard Stallman's General Public Licence (GPL) and the principles behind it. In addition, we need tools that make sharing easier, such as SourceForge or BitKeeper, which Linux programmers use in their work. And then, of course, we need designers, seamstresses, makers, and shopkeepers - all sorts of people to do all sorts of things - just like in Linux. And because it's all about creating a brand, we also need a name. Let's give our clothes the brand name Open Collection.

Open Collection is a clothes brand that lets anybody take part in its creation. Anybody, from children to grandmothers, from students of design or the arts to the best-known designers in the fashion world. Anyone who wants to, can create sewing or knitting patterns, textiles, pictures, logos, and anything else that is part of designing clothes. Their patterns are stored on a www page on the Internet and are available for others to use. Naturally, everything is done with some form of the GPL type of licensing, so that anyone is free to use any of the patterns or textile designs so long as they too contribute their own changes for others to use.

Just as a working computer consists of several smaller programs, a single piece of clothing consists of several separate design elements. One person or company may have developed the fabric; someone else may have created the pattern printed on it; and a third person may have combined the colours in a new way. Yet another person designs the cut of the garment and determines how its seams will look, while a fifth designer decides on the shape of the pockets, and so on. Like many other jobs, making clothes is often a collaborative effort. And collaboration is what Open Source is good at.

Just as there are several Linux distributors, there would surely be many manufacturers of Open Collection clothes. And why not? If you could get fashionable designs free off the Internet and there was a demand for the clothes, then the manufacturers would definitely be interested. And just as Linux is different to Microsoft, so would an Open Collection be different to existing brands of clothes, in that nobody would own it. It would be our common brand. An Open Collection manufacturer in Asia might take the initiative to produce these clothes, but would have to find themselves a European importer or store to sell them. Not having to pay expensive licensing fees to the owner of the brand would in theory allow the manufacturer to pay more to their employees who actually make the clothes. In addition to large-scale clothing factories, there would also be manufacturing on a small-scale. Teenagers could sew their own fashion clothes if they chose, and grandmothers could knit Open Collection sweaters. Some people would make their own clothes to save money, others because they enjoy doing it, but most people would continue to buy their clothes from the stores because they have things they prefer to do, other than sew clothes - just like with Linux. Some people would buy clothes made by the cheapest manufacturer, while others would prefer to buy clothes made in their own country or perhaps handmade clothes - just like with Linux. In schools and colleges, people doing craft- or needlework classes could make their own Open Collection shirts and print them with images of their own choice in their art class. Or they could just print out images they like. And nowadays, special paper for making transfers is available for inkjet printers. Images printed on it can be ironed onto, for instance, a T-shirt. See how much we engineers have done to enable people in the clothing business to take the Open Source revolution to its next stage.

The quality of the Open Collection brand clothes might vary slightly. While uneven quality is usually considered bad, here the diversity would be a feature of the Open Source process used to create the clothes. It is unlikely that any two shirts, each made by a different manufacturer, would ever be completely identical despite their design having originated from the same pattern. One manufacturer might make it in a Mexican cotton fabric while the other one makes it with linen from India. And each homemade shirt from the same design would also be unique. And that is just the way we like it. Open Source is all about the joy of doing, of appreciation for individual makers, and of celebrating versatility!

Something else that differentiates Windows from Linux is that Windows is made by a company, whereas Linux is made by individuals. This is a significant difference which those who have switched to Linux are usually very excited to discover. When a computer user encounters a problem with a program made by a faceless corporation, they must call that corporation's technical support. Usually what they get is an answering machine that tells them to press 1 if you have this problem, press 2 if you have that problem, and press 3 if you want to speak to a human being. If they make the mistake of pressing 3, what they get is a pre-taped message telling them that "unfortunately all our lines are busy at the moment, but please hold and wait for your turn'.

Whereas, if you encounter a problem in an Open Source program you can tell the makers of the program about it directly by e-mail. The e-mail address is always given in the source code of the program, and is even easier to find on the "About' page of the program. The makers are usually very happy to have your feedback and answer you with thanks because you have told them how they can improve their program. If the corporation behind a closed program even answered an e-mail informing them of a flaw, it would probably come from a company lawyer denying the existence of the problem.

So a label on an item of Open Collection clothing would not, for example, say "Made in Hong Kong', but in the spirit of Open Source would read something more like: "These sneakers were sewn by Lu from Shanghai, the design was by Fabrizio from Brazil'. And if they'd also included the seamstress' e-mail address, a buyer happy with their new pair of sneakers could even send Lu a brief note of thanks. At the same time, he or she could ask about Lu's working conditions and make sure that all the employees were getting decent wages and that no child labour was used in the manufacture of the shoes. In these ways, Open Source could even help to address the problems about which the movement against globalization has been voicing such loud concerns in recent years.

Garment manufacture is just one example of the many business activities that could benefit from the adoption of Open Source thinking. However, this book is now written and the as-yet-unwritten innovations of openness will continue to grow out of the ingenuity and imagination of people who see the advantages of Open Source methods. We have seen how Open Source thinking can change the world of gold mining, encyclopedia publishing, and the clothing industry, and I hope such examples will encourage people to develop and innovate in ever more inventive ways. I am convinced that the stories presented in this book are just the beginning, and that there is plenty of room for a new generation of innovators like Richard Stallman and Linus Torvalds.

How about restaurants publishing their secret recipes on the Internet for customers to make at home? The recipes would be freely available for use by other Open Source restaurants who similarly publish their own recipes on the Internet for others to use. Customers could add their own favourites to the collection and, for instance, order their own favourite dish as made by the best chef at a five-star restaurant. The restaurants could have a common menu and a common logo and brand. It would be an Open Source chain of restaurants - perhaps called McOpen.

Yes, yes. There are plenty of ideas to go round, particularly when one gets excited. It's been exciting for me, at least, to relate these stories. I hope they've enthused you, too, because enthusiastic excitement is one of the greatest virtues of the hacker community. I hope their stories will inspire all of us to think and behave more openly - to live an open life.

  • 1As I stated in an earlier footnote the first programmable machine, and therefore the forefather of all computers, is reckoned to be the loom invented by Frenchman Joseph Marie Jacquard in 1801: https://en.wikipedia.org/wiki/Jacquard_loom
hingo Sat, 2006-09-02 08:19