The 451 Group's annual report on the state of the open source business world is out. Already the title: Control and Community suggests they are once again on top of what has been going on this year. Analyzing about 300 open source related businesses they not only "get it right", but were actually able to uncover some facts even I was unaware of and this impressed me a lot. If an analyst can dig up statistics to back up something that I already "intuitively" know in my heart, that is a useful service. But if they can make me go "ah, I didn't know that" on a topic I consider myself quite an expert in, the I'm impressed!

This is an analyst report, available for a price that would be completely unreasonable for a private person. I was pondering whether I should go begging for a free copy to satisfy my curiosity on the topic. But that wasn't necessary, as the next day I was offered a copy by Matthew Aslett himself:

If you have any comments or feedback on the report we would be happy to see it - either privately or publicly - but don't feel obliged to do so. We just wanted to share our research with people who have helped shape our focus on open source-related business strategies over the past two years.

Given such alternatives, it is of course my style to make some public comments right here.

The report dives into a topic that has been controversial for some time, but really got wider acceptance only this year. Without a doubt Oracle's acquisition of Sun controlled open source projects is partially responsible for that awakening, but I also think the time was ripe for such a discussion in many other ways too. It is great to see one of our analyst companies being so close to the pulse of the Open Source community, companies trying to understand the phenomenon of open source will certainly get value for their money by being a 451 Group customer.

The report begins with a rather uncontroversial observation based on a model of 4 stages of evolution of open source:

• Stage 1 – Software developed by communities of individuals
• Stage 2 – Vendors begin to engage with existing open source communities
• Stage 3 – Vendor-dominated open source development and distribution projects
• Stage 4 – Corporate-dominated open source development communities

The research we have carried out for this report, as detailed in Section 3, provides evidence that our observation was correct, and that the industry has entered the fourth stage of commercial open source strategies.

My personal observations agree. But this report does not answer the obvious follow up question: Is that it? Is there a Stage 5 and a Stage 6 and what will they look like? I guess it is too early to tell, but it would be an interesting topic for your report next year.

There is one almost witty - but probably very true - comment about the future beyond Stage 4, originally by who else than Tim O'Reilly:

The predicted open source paradigm shift away from services in support of software and toward software in support of services is becoming reality. The shift is fueled by, and coincides with, increased emphasis on community rather than control, and a renewed emphasis on collaborative development.

(emphasis mine, well said, something to think about)

The report then goes into the inherent tensions and problems in the single vendor dominated open source model compared to the more flourishing community led projects (in all stages 1,2 and 4). This leads to the striking conclusion that:

Ironically, it is the incumbent closed source ISVs that have figured this out, not the so-called open source specialists. Our analysis of 300 vendors showed that 40% of open source specialists are associated with open source projects that use the cathedral development model, compared to just 29% of complementary vendors...

"Ironically"... I suppose this is what has always irked me the last years, while I have been working at these so called "open source specialist" companies. For most of us that engage in open source, the conclusion that collaboratively developed open source is more successful is not breaking news. Instead it was a total shock to me when I finally figured out that my managers at a leading "open source specialist" were completely ignorant of such basic open source dynamics.

Instead, had I worked for Nokia, I would have had the pleasure of working with Ari Jaaksi and the Maemo/Meego credo of "Inclusion, meritocracy, transparency, and upstream first". Four basic principles, it's not like it's rocket science guys! Or at HP under the guidance of long time Debian Project Leader Bdale Garbee. Or Bob Sutor of IBM. Even at Microsoft the guys at the open source lab always had years of solid open source experience completely dwarfing that of any CEO of the average "open source specialist".

There must be something interesting here in the DNA of a large corporation vs VC funded startup - like a general pattern completely unrelated to open source - where when a large corporation decides to engage with something - be it open source or otherwise - they make sure to hire an executive who really has solid competence in that specific area. And by solid competence I mean being a professor, project leader, such things. Startups somehow manage by with something else. Enthusiasm? A good research topic for someone...

There's lots of statistics backing up the evolution into Stage 4. The one that was surprising even to me was the claim that the formation of new open core projects already peaked in 2005-2006 and has since declined. I suppose living in the MySQL world my own perception is also distorted since I still see new companies trying their luck with that model. Anyway, the main graph with this revelation is this one:

Graph from a recent CAOS blog, copyright 451 Group

Even though the statistics seem to back up the conclusions of the report, there are some questions left: Is it possible that some of the companies/projects formed in the last few years, will later change strategy into a more vendor controlled or open core strategy? Also, I think a graph with absolute numbers on Y axis instead of percentages would have been easier to read. The current graph actually has some information loss due to the chosen format. It is possible that the absolute number of single vendor projects was still growing, even if their relative share was declining - this graph hides that information. Also, if the sample is 300 companies, spread over 19 years, it means there is an average 16 companies per year, but surely some years will only have very few, making this graph jump up and down unnecessarily.

Finally, it could be that the dip in single vendor projects is just a reflection of the downturn of the past years, and that such projects will take a larger share again when investors have more money to throw at startups. On the other hand, if they'd listen to the advice of The 451 Group, they'd instead be looking into startups that create products based on larger and more collaborative open source projects, such as we do see in the Eclipse, Apache, Drupal, etc ecosystems.

Whatever the truth is, I fully agree that the current statistics do support the conclusions of the report. Simon Phipps recently blogged about open core as the Open Source bubble that is now over. I initially thought that was perhaps an overly dramatic way of putting things, but looking at the statistics in this report, they actually look a lot like the dot-com bubble. There was a peak of a few years dominated by clueless activity funded by VC money, which now declines, all the while the overall number of new open source projects/companies continues to steadily grow. (In curves about internet adoption you see a similar situation, with a visible decline around 2002, but still dwarfed by the overall exponential growth of the internet in years that follow.)

The report then has some advice for these "specialist vendors" to open up or become irrelevant:

While the single-vendor open source approach is not going to die out, vendors that control open source projects need to transition toward more collaborative development in order to prove that they are more than just another software vendor that happens to release software using an open source license.

At a later stage the report also mentions my previous employer Monty Program and another Sun-fork ForgeRock as examples of single-vendor projects that are trying to be more open in their development than was traditionally the case with such vendors. The statistics at the end actually reveal that the authors have classified up to 25% of single vendor projects as practicing a more open and community oriented "bazaar style" development. It would be interesting to know what other companies than the two mentioned are in this category, and how well they are faring. By now readers of this blog will know that I agree with the categorization of my former employer, on the other hand I'm skeptical whether a large community can actually ever form around one dominating single vendor. (Though OpenJDK may prove to become the exception to the rule, separate blog post on that coming up in a few days.)

So in conclusion, I'll just reiterate that I think this years report from Matt and Jay at The 451 Group is an impressive and comprehensive number crunching exercise that is very timely and aware of the trends and developments in the world of commercial open source - it even uncovers some previously unknown details.

On the other hand, while this report very well answers the question of what is the best way to create a successful open source project with a large developer community, there is no statistical treatment on the topic of what is the best way to actually make money? Hypothetically, it could be that a company can be more valuable to investors by just shutting out the outside community and keeping the development to itself. The side effect would be a weaker product due to less total investment in its development, but does an investor really care as long as he gets his money?

As most companies surveyed must be privately held, such statistics are probably impossible to come buy. Even so, I intend to blog on this topic soon myself, so stay tuned.