Hi Unusually, we actually wrote a text for Monty's keynote speech this year. I'm publishing it below. Part of the speaker notes are just bullet points though, but the text may still be interesting to publish. Also, there is a funny joke about Oracle being a major MariaDB contributor that Monty somehow missed in the speech. The keynote is also available on Youtube. In fact, Sheeri has been nice enough to collect all videos from the conference into a playlist. The slides can be found on O'Reilly conference website as PDF.
First I would like to thank you all for being here. In fact, I feel very good about being back here as this is the first time since 2005 that I give a keynote on the MySQL user conference. After what has been a turbulent year to many of us, it is great that we can all come together to discuss and work face to face again. I would also like to thank O'Reilly Media and Mr Tim O'Reilly. I feel especially in gratitude to them as without them, this conference would not be possible. Tim and O'Reilly Media commitment to MySQL and to open source at large is an example that we all should try to follow. It was even at an O'Reilly conference that the term "open source" was first announced to the wider public. For many years O'Reilly sponsored Larry Wall to develop Perl - the first scripting language with which MySQL has enjoyed a very successful symbiotic relationship. I will come back later in my talk about the importance of supporting the ecosystem we all rely on.
Since the middle of 2008 things has been dominated by the financial crisis. Companies all over the world are focused on cutting costs and managing cash flows. In the previous recession almost 10 years ago, the big winner was Linux and commodity x86 hardware and it continued to win again in this recession. Open source solutions are now also widely used in the middleware layer. The database had remained as the last component in the enterprise IT stack where proprietary vendors extract high margins - the database tax. After spending the last 15 years dominating the World Wide Web, and sneaking into enterprise datacenters through the back door, MySQL was, in part thanks to Sun's backing, now being considered as a strategic alternative by the traditional enterprise CIO. The acquisition of MySQL by its competitor last year of made at many of them to take a small pause in their plans. But the pressure to cut costs is still there. Enterprise datacenters need a low cost and open source database alternative, and it is up to the open source community to deliver one. I will also come back to this topic later in my talk.
MySQL Ecosystem in review: More diversity and renewed commitment When we look at what happened lately in our ecosystem, there was a lot of movement. Many MySQL engineers and consultants have changed jobs during the year, more than ever before. While the executives mostly have left elsewhere, I have been happy to see that most of the technical people have stayed within our ecosystem. This of course includes also the engineers who have chosen to stay as Oracle employees.
The trend of new MySQL forks continued, with MariaDB being one of them. Happily, forks also are looking into joining their efforts, which I believe is important. Times like this provide a good opportunity to think things over, re-create things and see what one can do differently.
Here's a map of the main MySQL forks alive today. Many of the interesting storage engines like Infobright, KickFire and Calpont also end up maintaining their own fork, but that is not included in this picture. Out of all these branches, Drizzle is the only true fork because they really have gone in a different direction and rewritten and refactored a lot of the code and they don't care about backward compatible or ensuring that old MySQL applications will work with Drizzle. The refactoring is a good thing but it makes it very hard to exchange code with the other forks. Then we have OurDelta. OurDelta was started by Arjen Lentz to answer the demand that was created by MySQL's slow 5.0 and 5.1 release cycles and that almost no community patches was not incoopreated into MySQL releases. So OurDelta started to package the community patches into Linux and Solaris installer packages that people could start using. OurDelta is more a packaging effort and patch repository than a development fork, the idea was just to integrate patches that already existed. Percona first created XtraDB as a fork of InnoDB plugin. It contained the famous Google patches for InnoDB multi-core scalability as well as some of their own work. They also had some non-InnoDB patches for MySQL 5.0. All of this recently was released as a new version based on MySQL 5.1 and is now called Percona XtraDB server. Yesterday we awarded Kai Seidler for XAMPP, which is like LAMP for Windows, so it includes MySQL. It is the only other fork together with MariaDB that also includes the PBXT engine. So where is MariaDB in this picture?
MariaDB is the glue that will put everything together. Apart from Drizzle (who we exchange code with), all of the other forks contribute to MariaDB. We want MariaDB to become the consolidating MySQL fork, so that we eventually have less forks, not more. An important goal for MariaDB is to be inclusive and community developed. Actively working with storage engine developers to get them into MariaDB and get MariaDB work better for them. MariaDB 5.1 is based on the latest MySQL 5.1 code, an MariaDB 5.5 will be based on MySQL 5.5 code; We always keep our releases synced with the MySQL code and Oracle is thus a big contributor to MariaDB. MariaDB 5.1 packages are also based on OurDelta's build scripts, so this was very helpful to us since MySQL's build system is not open source. We are also working with Arjen to ensure that the patches he deems to be stable and important are added to MariaDB and Arjen is also actively contributing to MariaDB. In fact, OurDelta 5.1 is MariaDB 5.1, there is no separate OurDelta branch anymore. XtraDB engine from Percona is included in MariaDB too, as well as most of their server level patches. (We are working with Percona to get all the relevant changes into MariaDB so that they can base their future servers on MariaDB). We have included many of the various abandoned patches and storage engines and continue to do so for our next releases. We are also on the internal MySQL lists and ensuring that all relevant patches posted here gets into MariaDB, usually withing hours. Even with all these performance improvements and features, MariaDB is still a backwards compatible drop-in replacement for your old MySQL installations.
MariaDB 5.1 * Community developed. Feature enhanced. Backward compatible.
Including most of the major open source storage engines: PBXT, XtraDB, FederatedX, Maria; Some new ones that didn't make it for 5.1 will be added to 5.2
Thanks to XtraDB we already have in 5.1 most of the performance that MySQL 5.4 beta was is all about. We are using an Open source build system, based on buildbot! We used to say that you can work at MySQL from anywhere. Now you can work ON MySQL from anywhere. We have focuesd a lot on realibitity of the server and this is one of the major reasons it took us a long time to release MariaDB 5.1. We have worked on reducing compiler warnings, removing valgrind errors, extending the test suite and makeing things repeatable, faster execution, shorter code paths, less mutex.
- Accept community patches: 32 key indexes as opposed to 16, Percona patches, Croatian collations
- Monty Program work
- Optimizer fixes for faster complex queries
- Pool of threads with libevent (allows you to have 100,000 connections and less context switches).
- Table elimination
- The above will be more outlined in my MariaDB talk later today
Benchmark
- Comparison of readonly engine performance (Mark Callaghan, Facebook)
- Syntetic benchmark:
- ramdisk: no physical disk io
- HANDLER statement: bypass SQL parsing and optimizer
- Compares "pure engine performance", raw iops
Note: MyISAM is not as performant as it is expected to be for a readonly workload
The partitioned key cache fixes the biggest known MyISAM performance issues
MariaDB 5.2 See slide
Then we have MariaDB 5.3 (Maybe it will be called 5.5) This is where the MariaDB optimizer team has put most of their efforts.
- Backport and extended optimizer enhancements from canceled MySQL 6.0 release (subqueries, etc...)
- Major leap forward for the optimizer (comparable to another big commerical db vendor).
Our optimizer team, who are with us here today, claim that the code is 'almost' ready to be used and they are very excited to finally be able to finish this work that was started years ago on 6.0
- Pluggable replication api
- To enable new, better replication protocols
- Driven by demand from Web 2.0 and cloud users
- In partnership with Codership (Galera), Continuent (Tungsten), please join
- Partly sponsored by Finnish government
- Still lot of patches in commonity; New batch of Google, Facebook, eBay, Percona, OurDelta, etc. patchsets
As those that are following what is happening in the MySQL world, the main thing I intend to focus time on when I left Sun was on the new storage engine, Maria, that I and my team had already worked on for some time and was intended to replace InnoDB. Maria is an extended, crash safe version of MyISAM, the default storage engine for MySQL. Maria is named after my daughter Maria, like MySQL was named after my daugther My. In the future we wanted, and still want, to add full transactional capablities to Maria (the engine). However when Oracle bought Sun and MySQL, we at Monty Program Ab felt it was much more important to provide a guaranteed free alternative of MySQL and we forked MySQL and renamed it MariaDB. Unfortunately it is now confusing many that the database as a whole is called MariaDB, and one of the engines is also called Maria. To fix this problem, we are today announcing the Rename Maria contest. You can stop at our booth in the expo hall to suggest a new name for it, or after the conference you can just visit the askmonty.org website. The new name will be announced in May and the winner will receive a nice Ubuntu NetTop computer from System76. [Note: At this moment the web page for the rename Maria contest seems to not be there. I will add a link here once I know where to link to.]
And we have 2 of these to give away, so if you cannot think of a good name for the Maria engine, you can just enter a raffle at our booth. We will draw a winner at 16 today.
Even if MariaDB is a community project, it is worth mentioning the company I now work for, Monty Program Ab who is the driver of this project. Monty Program Ab today employs most of the top MySQL developers that visitors at this conference should be well familiar with. The team consists of 3 out of 4 architects, the full MySQL optimizer team, most of the original MySQL core developers and a MySQL Fellow, which would be yours truly. The foundation for both our business and the MariaDB project is that we have employed the people who know the MySQL source code better than anyone else in the world. The company is based on the Hacking Business Model, a document I wrote together with Zak Greant some years ago. For example, the company is run for and by the employees, so any profit will be shared among the employees and they are also the ultimate decisionmakers. Another important principle is that everything we do will be open source; All MariaDB related code is on launchpad MariaDB is purely a GPL licensed product, so we don't have any dual licensing business model. As an engineering company, our primary business model is to sell engineering projects to customers who wants us to develop some new feature for MariaDB, or just fix bugs in MariaDB or MySQL. You can also buy support for MariaDB and MySQL, this is something we deliver in many languages together with a network of local support partners.
Our model is focused on building an organic network of companies, rather than having one big company to control everything. We believe we can provide a richer customer experience this way, such as support in many local languages. Incidentally, such an organic ecosystem will also be resistant to hostile takeover attempts. One vehicle for this is the Open Database Alliance, a partner organization for companies that develop open source databases and products or services around them. The aim of fostering such partnerships is to be able to provide a one stop shop for the customer, where any one company can easily offer their customer a full stack of solutions.
To add to our existing MariaDB support offerings, we are today also announcing MariaDB Unlimited Servers Support Subscription Why unlimited; (Because there is no counting of servers, cores....); Fixed price of 36000 dollar! The support covers all stable versions of MariaDB and MySQL (starting from 3.23). It is delivered through our network of local support partners, and by beeing backed by the developers at Monty Program you are guaranteed that no mission critical bug will be too difficult to fix. We've also included an unlimited license for Monyog, the popular monitoring tool from the nice people at Webyog. As an addon option you can also get Zmanda Enterprise Backup. We do of course also offer you traditional 24/7 support subscription for smaller companmies and of course also consulting and training, if needed. Come to our booth so that we can give you an offer
The economics of open source. Sun, under Jonathan Schwartz, not only lead and developed a lot of Open Source projects, they also sponsored projects with machines and money and helped a lot of projects to survive. Sun will leave a big economic hole in the Open Source world that will be hard to fill. Another problem for Open Source is that under an economic down term, even if open source projects are more heavily employed, there is fewer monetary donations and paid for projects for the projects developers. This is something I belive needs to start to change; We need to get away from the notion that open source is there to be used and exploited without having to give anything back. Instead we need to create more ways to get money to those that provide the infrastructure we are all depending so that they can not only survive but actually afford to work full time on these projects. What I have personally done lately to help in this regard are the the following:
- The Open Database alliance was created, both to help customer find all database related services they need from one point but also to fund the projects that the ODBA members are using; This is done by allocation 80 % of the membership fees to open source projects.
- By the way, we are searching for a Director for the ODBA to drive it to the next level; If you are intersted, please contact me!
- A notable part of the MariaDB unlimited support subscription is given out to projects included in MariaDB.
- We have created a bounty system where MariaDB users can combine bounty for features they want to see developed. When the bounty has reached a level that is enough to pay for the development costs, we will allocate someone to do the feature. Before that, anyone in the community can offer to do the feature and if the code is accepted he will receive a major part of the so far collected bounty.
MySQL is an ecosystem, not a company. I really like the theme of this years conference. Perhaps it was chosen a bit as a reaction to people's concerns over Oracle acquiring MySQL, but honestly, this should have been the theme in previous years too! At MySQL AB I always advocated that we should have worked more closely with the community. Now I see that we have ourselves become the community. It is not what I envisioned, but I hope that this will turn out for the best. My dream is that we manage to re-build an ecosystem where: Everything you need to setup and use MySQL is open source. The source, documentation, installers, build system and the basic tools you need to get things going. This doesn't mean that there isn't a place for commercial vendors, but that the basic things everyone needs should be free! With MariaDB we have now re-created a Buildbot based build and QA infrastructure, and we are working on the documentation part. But there is still work to do here. My dream is an ecosystem where... Engineers - both developers and QA - work together but are employed by many different companies. By product companies like Monty Program or Oracle, by consulting and support companies and even by end users themselves. Google and Facebook have certainly set a good example there. My dream is an ecosystem where... All of those engineers contribute code to one main trunk. It's regrettable that this is not the case as it cases a lot of merges back and forth between different forks. As much as we appreciate the patches coming out from end users like the Facebook team, today the practice for them is just to publish their patch and let interested forks pick it up. From my point of view this is not optimal, even with a good patch there is usually many man-hours of work if we then pick up the patch and integrate it into MariaDB. In most open source projects it is the creator of the code who also carries the burden of getting it included into the main upstream code base. In the past this understandable as they upstream wasn't responsive to patches and didn't let community members have commit access to the main trunk but in my dream there is a main trunk where this is not the case anymore. My dream is an ecosystem where... There are multiple competing service companies who offer "enterprise grade" global 24/7 support. Customers can choose among multiple service providers to get support for their open source database. We have a relatively good base to start from. For our part we do our best to help with the Monty Program Support Partner network and Open Database Alliance (ODBA). But the demand from customers for an open source database alternative is still bigger than what the current network of companies is providing. There is room for them to grow into larger global players, and there is room for still more companies to join the game. If you want to be part of this growth, please contact Ethan and talk with him about the ODBA.
- Log in to post comments
- 18662 views