2012-12-12 Diary of a Nerd: LXC inside a virtual machine

Dear Diary,

I’m currently playing around with LXC together with libvirt, which will run in a VM provided my KVM. You might be asking: Why not run LXC on the host itself, and bypass one layer of virtualization?

Whilst I have full control over KVM and LXC on my test setup, I will not have control over KVM (or XEN) in all cases. The specific reason is that we want to move the RaumZeitLabor jail (which currently is running on a 32 bit FreeBSD system), which was generously hosted by UUGRN over the past 3 years, to another hoster which gives us independance. Because we can’t pay much money (and don’t really have much stuff we need in terms of CPU power), we are evaluating a virtual server.

One candidate is vollmar.net – they offer a virtual server (warning: It seems that their english translation has wrong system specs) with 1024MB RAM and 50GB HDD, where the virtualization is done via XEN. Because nested virtualization is only available on Intel machines and is only rarely used so far, I decided to test-drive lxc in a VM.

Because LXC is a container-based virtualization, there’s no need for any virtualization support (you might even run it on your Raspberry Pi), it is no problem to run it inside a virtual machine. With LXC, we can have one container for each logical service, without the risk to have a security leak in one service which potentially affects other services.

Because LXC doesn’t allocate but limits resources, they are shared – this is true for both memory and disk usage (unless you configure it otherwise). That means: If I create a LXC container with a maximum of 128MB RAM and 3GB HDD, and it only uses 32MB/1GB, the remaining 96MB/2GB can be used by other containers. Of course, you could also use separate images and mount them, but that’s not what I’m after.

LXC seems to work without a problem inside the virtual machine, however, I was unable to install a debian installation by using /usr/share/lxc/templates/lxc-debian because the following bunch of errors occur at the end of the installation:

The container itself starts up correctly (more or less), however, when attempting to login, the system immediately returns “Login incorrect” without even prompting for a password. After quite a bit of research, it seems to be that the base system misses quite many tools, even “passwd” is missing (which was not the case, as I found out later – /usr/bin wasn’t simply in the path).

That’s why I decided to put lxc-debian into the thrash and try my luck with debootstrap. After quite some installation time, I found the same symptom: An immediate “Login incorrect”. I wasn’t giving up and created a new user and assigned a password to it; and bingo – I was able to login. Even a “su” to root with password worked.

Further investigation revealed that /dev/pts/0 isn’t in the /etc/securetty file; in order to make that work, we need to put “pts/0” into that file on a single line. Even DHCP networking and ssh logins worked out of the box – will try with lxc-debian another time, even if it throws errors during installation.

So if you have problems with your fresh lxc container, edit your libvirt xml file and replace “/sbin/init” with “/bin/sh” to make container modifications, or edit the files on your host system directly (unless, of course, you’re using disk images).

Further cosmetic changes:

  • Disabled tty2-tty6 in /etc/inittab
  • reboot gives a “Maintenance Password” prompt on reboot – I have seen this in the past, and is probably due to the fact that debian attempts to unmount the / filesystem, but cannot do so.


Add Comment Register

Leave a Comment

NOTE - You can use these HTML tags and attributes:
<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code class="" title="" data-url=""> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong> <pre class="" title="" data-url=""> <span class="" title="" data-url="">