In my previous post I explained why I believe the production of RPM and DEB packages should be more integrated with the rest of your development process. Now it's time to look into how you can put the RPM build scripts inside your main source code repository, and in particular how I did that to produce RPM packages for Drizzle.

A one sentence summary of how RPMs are built is that you take the source tar.gz release of the software, and then you provide a .spec file that includes information on how to build the source, how to package it into one or more rpm files, and how those files should be named and versioned. It is also possible to provide additional source files, or patches to be applied against the official upstream sources - as discussed in the previous post this often happens, and more than I would like to see.

The main catch in what we want to do is the need to provide version numbers in the spec file. They need to correspond to the version of the source tar, otherwise rpmbuild will complain. So if the spec file will be inside the source tar itself, we have a chicken-egg problem: how can I know what version it should have.

Now this is a good opportunity to have autotools do something useful instead of just causing pain to humankind. The spec file is now in lp:~hingo/drizzle/drizzle-integrate-packaging-rpm at support-files/rpm/SPECS/drizzle7.spec. Let's rename it to drizzle7.spec.in and add a few tricks.

Early in a spec file you usually find declarations for the package name and version:


Name: drizzle7
Summary: A Lightweight SQL Database for Cloud and Web
Version: 2011.11.29
Release: 1%{?dist}


The packager will usually then edit the version number and sometimes the Release number (this is the rpm build number made from a specific upstream Version, incremented if you need to correct something in the packages itself).

A well written spec file will then refer to these values via variables. We are in luck, the drizzle7.spec file was mostly well written. I changed one line where BJ used to just copy the URL so it now uses the name and version macros too:


Source0: https://launchpad.net/drizzle/fremont/2011-11-13/+download/%{name}-%{version}.tar.gz


If you know autotools, you can already see the solution. Use autoconf variables:


Name: @PACKAGE@
Summary: A Lightweight SQL Database for Cloud and Web
Version: @PANDORA_RELEASE_VERSION@
Release: 1%{?dist}


Note that the Release number will now always be one. If you need to edit some rpm specific things, you will commit it to the main repository, which will automatically increase the main Version number instead. This feature comes from using the excellent Pandora build system on top of autotools.

There is one more place we need to automate things in. At the end of the .spec file there is a changelog. The last entry says:


* Sun Jan 01 2012 Henrik Ingo - 2011.11.29.2481-2
- Hrmpf... server doesn't actually start:
- --user can only be given in cmd line (ie init script), not in
drizzled.cnf.
- Turns out --user and --daemon don't work well together, so have to use
sudo -u drizzle /usr/sbin/drizzled --daemon instead.
https://bugs.launchpad.net/drizzle/+bug/890910
- [etc...]


A really good RPM guide I read said that you can write pretty much anything in the changelog. Not true. The latest version number has to match the version you are building and the date has to be there too. So I added a generic entry like this:


# Note: Since we are generating RPMs completely automatically, we have
# a generic changelog entry for that. If you actually change the spec
# file or something rpm related, you should add an actual hand written
# entry for that version. You will need to comment out the generic
# entry for that release, and it needs to be uncommented again for
# the next release.

%changelog
* %(date "+%a %b %d %Y") (Automated Drizzle RPM build) - %{version}-%{release}
- New RPMs against newest upstream sources. Please see Drizzle release
notes for details: https://launchpad.net/drizzle/+milestones

If you've paid attention, the use of %{version}-%{release} should have been an obvious trick. I was pleased to see I can even reach the date command via an rpm macro.

We've now created a spec file which will create correct rpm packages from a Drizzle release and we don't even need to touch it. Let's add that to the makefile:


# For convenience we support make rpm directly from bzr repo
rpm: dist
        cp ${PACKAGE_NAME}*.tar.gz support-files/rpm/SOURCES/
        rpmbuild --define="_topdir `pwd`/support-files/rpm/" -ba support-files/rpm/SPECS/${PACKAGE_NAME}.spec
        mv support-files/rpm/SRPMS/* .
        mv support-files/rpm/RPMS/*/* .


The above will first create a source tarball (make dist) and then create rpm's from that one.

I'm using a little known argument to rpmbuild that makes the build happen in the support-files/rpm/BUILD and related directories:


rpmbuild --define="_topdir `pwd`/support-files/rpm/"


The common way to build rpms is to setup the build directory in a central location like /usr/local/rpm/BUILD (et al) and the sysadmin or packager should then collect all of his sources and specs there in order to build rpms. I've never understood why this is a good idea and why I should mix my Drizzle patches with, say, Tuxracer patches. I like to have a specific place for my drizzle work and then I tell rpmbuild to do it's stuff right there. More importantly, the above make rpm command created by one developer (me) doesn't assume that someone else has his rpm build directories setup in any particular location, it will just work within the bzr checkout and be fine.

rpmbuild has quite a nice feature I eventually want to support. If the source tarball includes exactly one *.spec file (with any name, at any path), you can build it directly with


rpmbuild -ta drizzle7-2012.01.30.tar.gz


Currently this doesn't work because it will then expect to find the patches and additional source files that the spec is referring to, but those are still locked inside the tarball, so it fails. (Unless of course you separately put all of them into your rpm build directory, which kind of defeats the purpose of all this.) The next step related to rpm packaging will be to eliminate all those files and push them "upstream" into their right locations elsewhere in the Drizzle sources. (But first we need to get deb packages out too.) Then this simple rpmbuild one liner will work and become the easiest way to create Drizzle rpms.

Oh, almost forgot: So how about continuous integration? Well, now we can just have a Jenkins job that does make rpm and we are pretty much set. Bonus points for also signing the packages and copying them into a yum repository, then we'd have continuous rpm snapshots!