Programming

hingo's picture

Translating reliably between XML and JSON (xml2json)

Last week I was assigned to work on a simple yet interesting problem. MongoDB stores data as JSON. But it turns out we often have customers - especially in the important financial services market segment - where data is in XML. (Yes, SOAP still exists too!) To store that data into MongoDB, we need to transform it into JSON.

(For the impatient reader: here's the Github link!)

Storing XML as a text field

There are various ways one could do this. For example one could simply store the XML document as a whole in a single field, and then extract just a few parts of the XML that are stored as individual JSON keys. One reason to do this is that they can then be indexed and used in queries.

For example:

hingo's picture

Date objects in JSON

Flavio of MongoDB fame complained in a tweet about wanting a date format in JSON. Currently you can only express strings and numbers as the primitive data types in JSON, as well the composite forms of a list: [] and an object: {}. The latter is essentially just a subhierarchy in the object/array that is your full JSON document.

i wonder why there's no "STANDARD" way to (de)serialize times / date times into / from json! They might not be "primitive" but COME ON!!

hingo's picture

HowTo read all diffs by certain user in bzr

In the series of shell one liners, today I wanted to stalk someone, so I came up with this one liner to show all diffs ever created by a specific user in a give bzr repository:

bzr log -n0 | grep -B 1 henrik | grep revno | grep -v merge | \
awk -F'revno: ' '{print $2}' | while read revno; do echo; \
echo '#####################################################################'; \
echo revno: $revno; bzr diff -c$revno; done 

bzr log -n0 prints out the full log history of your repo. It may look something like this:

    ------------------------------------------------------------
    revno: 2441.2.16 [merge]
    committer: Henrik Ingo 
    branch nick: drizzle-json_server-keyvalue
hingo's picture

Making rpm builds a first class citizen: How?

In my previous post I explained why I believe the production of RPM and DEB packages should be more integrated with the rest of your development process. Now it's time to look into how you can put the RPM build scripts inside your main source code repository, and in particular how I did that to produce RPM packages for Drizzle.

hingo's picture

Making rpm builds a first class citizen: Why?

Last weekend I released rpm files for the latest Drizzle Fremont beta (announcement). As part of that work I've also integrated the spec file and other files used by the rpmbuild into the main Drizzle bzr repository (but not yet merged into trunk). In this post I want to explain why I think this is a good thing, and in a follow up post I'll go into what I needed to do to make it work.

(And speaking of stuff you can download, phpMyAdmin 3.5.0-alpha1 now supports Drizzle!)

hingo's picture

Parallelizing MySQL replication slave - a dream come true?

There has been an increase in the discussion about MySQL replication and how to make it happen faster. I don't think Marco Tusa's blog is on Planet MySQL, so let's highlight it first: A dream on MySQL parallel replication. This is a good account of what it looks like out in the field of MySQL databases today - you are increasingly aware of replication, fearing it will be a bottleneck you cannot fix.

hingo's picture

Back to work: 1st week as Senior Performance Architect at Nokia

As some Facebook friends already guessed from my status updates this week, my 9 month paternity leave is now over and I've survived my first week back in work life, waking up at seven in the morning! This is just a personal-life blog post to let everyone know what I'm up to, (For those asking: Ebba is doing fine, she recently started to stand up and even takes steps if I hold her hands.)

During the past months I had many interesting conversations and ideas of what to do next, but in the end Nokia was the company standing out with a very interesting offer. So as of last Tuesday I work at Nokia as Senior Performance Architect in the Mobile Solutions division, better known as the Nokia Ovi web services.

hingo's picture

Overview and archeological exploration of the MepSQL Bakery

This is the final part in a series of posts about the MepSQL build system known as MepSQL Bakery. MepSQL is a (yet another) fork of the MySQL database server, with the server based on the MySQLatFacebook code and the build system based on the MariaDB build system.

In this final post I wish to draw a high level picture of the complete process of building TAR, DEB and (eventually) RPM packages from the source code. There's not much more technical details to add to the previous posts, instead I'm going to make some, shall we say "archeological", observations which imho are interesting given how the build system has evolved when being passed from one project to another. Perhaps more importantly, I will also say a few words about where I think the future direction lies.

Syndicate content