Freedom, Software, Hacking ...

Finally – Google Movies & Music in Sweden

So, we finally got Google Music & Movies services in Sweden! The prices seem a bit on the steep side, apart from maybe the Music service, which is slightly cheaper than Spotify. Will try it out for a month or so to see which one I like better.

For some reason I chose Gravity as my first rental, at 39 Skr. I’ve heard so many bad reviews of it but for better or worse it’ll now be my introduction to Google Movies :–)

Weekend Hack: JSON vs CONF

It was time. I had been putting it off for far too long — learning about JSON and deciding on a new .conf file format for Inadyn. So this weekend I sat myself down to read up on JSON and the multitude of parser libraries for both JSON and traditional .conf file parsers. I was looking for a human readable file format that a user could easily and reliably edit by themselves without it being too error prone or sensitive to mistyping.

I quickly narrowed down my scope of investigation to Jansson for JSON and and Confuse for “standard” UNIX .conf files. Both have very liberal licenses, extensive test suites, very good documentation and are extremely well written! In fact, I will probably use them as prime examples of well maintained Open Source projects in the future :–)

The resulting code is in my toolbox on GitHub.

In my opinion the .conf file is a lot easier to read, write and edit by Joe User, so that’s what I’ll be using for many of my own projects, starting with Inadyn

Just a Programmer

Sometimes people ask me what I do for a living. Usually I tend to pause and think, real hard, becuase the people asking me this aren’t programmers. They use computers, but are mostly limited to a Windows machine, writing in MS Word and browsing the Internet, mostly for Facebook.

I often start off with: “It’s a bit complicated to explain … ”, by which time I’ve lost most of the people in the room listening to me. Sometimes I say: “I’m a software architect.”, because people seem to know what architects do for a living, they draw houses, design stuff and drive SAAB’s. Much like dentists. The prefix “software” however does confuse people.

In reality I’m just a programmer with a little bit more responsibility. A day starts with coming in to the office, or sitting down in my home office, connecting to the company VPN, meanwhile planning loosely the day and my goals. All while checking status of e-mail, our IRC channel and the issue tracker for any recently reported/updated issues in my fields of responsibility. Somewhere around there I have a pretty good idea about what must be done and in what order. So I write it down, in no particular order, on folded A4 sheets I use for TODO lists and start working. About 20 minutes into that, when I’ve just reached The Zone, I get interrupted for an impromptu meeting, or telco, which in turn always leads to another meeting, which in turn runs over and suddenly it’s 16:00 (4 pm) and I have 45 minutes to complete a days work before picking up the kids from school.

Or … you can apply that same crazy to a day when trying to fix a simple bug, finding another horrbile bug and fixing that first, leading to a minor redesign in need of a refactor, which in turn gives me a pain to merge since during the time I tried fixing that minor bug my collegues have done multiple changes to the same files I’ve been working on.

That’s what I do for a living. And yes, I know it’s not sustainable work conditions. I’m working on that, it’s on the TODO list …

Threads vs Events

This is a rant about something I recently found to be a long standing battle line in the world of programming, Lau78. The event vs thread based approach to programming. As rants go I do not aspire to deliver a clear or logical message, what so ever. It’s basically just something I need to get off my chest.

It was not until 2007 I first learned about the event based approach to programming and event libraries like libevent and libev. Up until that point the silver bullet everyone was using was … Threads.

I don’t really know when it all started, maybe it was the Linux revolution, the first NPTL release with GLIBC, Java or Solaris. Nevertheless, from my point of view it was sometime in the mid 90’s during my time at university that the use of threads was starting to become prevalent.

With the rise of the thread based model of programming we now had a hammer, and every problem looked like a nail. I had a gut feeling there was something really wrong with using threads for every conceivable program, but I could not find a way to express it, so I chugged away with my threads, semaphores and condition variables. I convinced myself I was happy like this.

Of course I knew about the event based approach, but it was more or less dismissed as a thing of the past, a while(1) loop to mimic the behavior of PLC’s. So almost every program I wrote, and every program I took over from others, were like Indiana Jones types of mazes full of deadlocks and race conditions.

I thought I did something wrong, and so did many others like me. I spent days and nights trying to understand, refactor, and redesign threaded programs. What I found was a doubt that the thread based model actually didn’t suit every problem, Ous96. There are quite a few domains, however, where thread based models shine. Usually in languages that come with thread support built-in, like Erlang.

Most of the programs I work with today are network daemons. Meaning they are essentially message based applications that spend a lot of time waiting for an event to occur: receiving a data frame, waiting for a timer to expire, a signal to be raised, etc. Of course threads can be used for this, but it is a lot simpler to employ an event based framework instead. Also, they are all written in C for speed and portability between different UNIX systems. For that domain, where I currently make my living, it will be difficult to convince me to ever look at threads again.

New Open Source Releases

Vacation time means catching up on my Open Source projects! :)

Currently I’m shaping up the home pages and this blog to improve the easy access and overview of all the packages I maintain. The following packages have new releases, or can expect new releases soon:

  • Minix Editline v1.14.1
  • SMCRoute v1.99.1 — There’s even a v2.0.0 being planned, with the core of SMCRoute available as a library
  • mrouted minor cleanup an sync with OpenBSD
  • pimd cleanups and bug fixes, needs testing
  • inadyn is in dire need of a release, but needs more testing and fixes

As usual, see my GitHub for the latest commits if you want to try anything out, file an issue report, or if you want to contribute.

Net Install CentOS

I usually run Debian or Ubuntu on my machines. However, having recently found some time to work on my various projects again, I’ve now suddenly found myself in need of a CentOS machine.

The CentOS home page invited me to download an installation ISO, so I went for the small Net Install which started perfectly with my virt-manager in Ubuntu.

All I had to provide was an FTP server and directory:

  • /centos/5.9/os/x86_64
  • Anonymous FTP

That’s it, the graphical installer started and I had to start selecting various packages. Must say it’s a bit confusing since the package naming is not the same in RedHat/CentOS as in Debian.

Oh, and if the installations seems to have gotten stuck, just wait it out. It’ll get there :)

Read the following to get console in virsh working with CentOS guest.

Multicast HowTo

The below setup is done using four Ubuntu 12.04 LTS virtual machines running the linux-virtual kernel package. In the HowTo I mention both pimd and mrouted, since they work out-of-the-box w/o any config changes, but you could just as easily use SMCRoute for the same purpose.

When setting up virtual machines and virtual networking there are several requirements for the host. The most important one, that needs pointing out, is a bug in the IGMP snooping code in the Linux bridging code: the bridge handles the special case 224.0.0.* well, but all unknown multicast streams outside of that segment should also be forwarded as-is to all multicast routers. Since this does not work with the current IGMP snooping code in the Linux kernel bridge code you must disable snooping:

host# echo 0 > /sys/devices/virtual/net/virbr1/bridge/multicast_snooping
host# echo 0 > /sys/devices/virtual/net/virbr2/bridge/multicast_snooping
host# echo 0 > /sys/devices/virtual/net/virbr3/bridge/multicast_snooping

Disabling IGMP snooping on the hosts’ virbr3 is not really necessary, but is done anyway for completeness, and also because I re-use the same setup in other test cases as well.

 R1                       R2                       R3                       R4
+-------+                +-------+                +-------+                +-------+
|eth0   | |eth0   | |eth0   |   |eth0   |
|     .1|----------------|.2   .1|----------------|.2   .1|----------------|.2     |
|   eth1|     virbr1     |   eth1|     virbr2     |   eth1|    virbr3      |       |
+-------+                +-------+                +-------+                +-------+

This setup can be used in two separate use cases, remember there is only one multicast routing socket, so you have to choose one of:

  1. . pimd -c pimd.conf
  2. . mrouted -c mrouted.conf
  3. . smcroute -f smcroute.conf

The default configuration files delivered with pimd and mrouted usually suffice, see their respective manual pages or the comments in each .conf file for help.

When you start mrouted, you’re usually ready to go immediately. But in the case of pimd, wait for routers to peer. Then you can test your setup using ping from R1 to a tcpdump on R4:

R1# ping -I eth1 -t 3
R4# tcpdump -i eth0

As soon as the PIM routers R2 and R3 have peered you should start seeing ICMP traffic reaching R4.

Now, to the actual test case. The first command for R1 adds a route for all multicast packets, that is necessary for all tools where you cannot set the outbound interface for the multicast stream, in our case iperf.

R1# ip route add dev eth1
R1# iperf -u -c -T 3
R4# iperf -s -u -B

The -T option is important since it tells iperf to raise the TTL to 3, the default TTL for multicast is otherwise 1 due to its broadcast like nature.

The desired output from iperf is as follows:

R1# iperf -u -c -T 3
Client connecting to, UDP port 5001
Sending 1470 byte datagrams
Setting multicast TTL to 3
UDP buffer size:  160 KByte (default)
[  3] local port 55731 connected with port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec  1.25 MBytes  1.05 Mbits/sec
[  3] Sent 893 datagrams

R4# iperf -s -u -B
Server listening on UDP port 5001
Binding to local address
Joining multicast group
Receiving 1470 byte datagrams
UDP buffer size:  160 KByte (default)
[  3] local port 5001 connected with port 55731
[ ID] Interval       Transfer     Bandwidth        Jitter   Lost/Total Datagrams
[  3]  0.0-10.0 sec  1.25 MBytes  1.05 Mbits/sec   0.268 ms    0/  893 (0%)

To achieve the same using SMCRoute you need to setup the multicast routing rules manually. The by far easiest way to do this is to update /etc/smcroute.conf and and start/restart smcroute, or send SIGHUP to an already running daemon. The below example makes use of the source-less (*,G) approach, since we in our limited setup have full control over all multicast senders. There is a slight setup cost associated with this: the time it takes the kernel to notify SMCRoute about a new source and before the the actual multicast route is written to the kernel. In most cases this is acceptable.

In smcroute.conf on R2:

mgroup from eth0 group
mroute from eth0 group to eth1

In smcroute.conf on R3:

mgroup from eth0 group
mroute from eth0 group to eth1

Now, start smcroute on each of R2 and R4 and then proceed to start iperf on R4 and R1, as described above. You should get the same result as with mrouted and pimd.

That’s it. Have fun!


  • It doesn’t work? — Check the TTL.
  • It doesn’t work? — Check the TTL!
  • It doesn’t work? — CHECK THE TTL!
  • Why does the TTL in multicast default to 1? — Because multicast is classified as broadcast, which inherently is dangerous. Without proper limitation, like switches with support for IGMP Snooping, multicast IS broadcast.
  • It doesn’t work? — Check your network, maybe a switch between the sender and the receiver doesn’t properly support IGMP Snooping.

File System Pass-Through in KVM/Qemu/libvirt

This post doesn’t cover fully setting up KVM/Qemu with virt-manager and creating virtual machine guests. See the Ubuntu KVM Installtion, VirtManager Guide, the Ubuntu Server Guide on libvirt, or HowtoForge for that.

Instead this blog post details the most relevant steps to get file system pass-through between a Linux host and Qemu guest working. The upstream Qemu docs provide a good starting point, as is the original IBM paper on VirtFS. For users of Ubuntu <= 13.04, watch out for the libvirt bug that I know many people run into, myself included.

First of all, I could never really master the beast called AppArmor in Ubuntu. Once I got the hang of the files to edit, the order to make changes and the syntax of its profile files I think I tried every possible permutation without any success. So I ended up disabling the profile(s) of my VM guests. The UUID in the filename can be found in the details of your VM, or in the process listing on the host: ps fax | grep guestname. Here is an example of how to disable one guest:

aa-complain /etc/apparmor.d/libvirt/libvirt-20b8c6c6-440c-bd76-254e-42fd475e6770

You need to install apparmor-utils to get the aa-complain tool. Where complain basically means ignore any hits from the given profile and just complain in the log. The default is aa-enforce. For more info on AppArmor, see the excellent upstream docs

Now, how to do it. I like virsh, but for most of the time the vmware like virt-manager is a lot more user friendly. In the VM’s Detailed view, click the “Add Hardware” button and select “Filesystem”. This is where the action happens.

  • Type: preset to Passthrough
  • Mode: change to Mapped This is the most important step in this blog, or you will not get read/write support!
  • Source path: select the path on your host that will be shared with this guest. I use /var/lib/libvirt/share but you can use any directory you want
  • Target path: enter magic string that you’ll use in the mount command in the guest. I use share, no slashes or anything. In reality this isn’t a path per se, it’s a tag that the guest sends to the kernel 9p driver via the mount command

Please note that 9P file systems simply pass-through the owner UID/GID and directory permissions from the host to the guest. This can be a bit confusing, but just make sure to use the same for all guests that share the same directory. I chowned it to my account on the host:

host# chown jocke:users /var/lib/libvirt/share

In my guest:/etc/modules I added the following modules, even though the kernel can probably load them itself on demand:


The actual command to get the ball rolling on the guest:

guest# mount -t 9p -o trans=virtio,version=9p2000.L,rw share /mnt

To automatically mount this every time at boot, add the following to your guest:/etc/fstab:

share   /mnt    9p  trans=virtio,version=9p2000.L,rw    0   0

That’s it. Good Luck!