Getting started with IOS-XRd

Published on 2022-10-20.

In my ongoing journey of running networking-related things on other things, I have now reached the world of containers and more specifically running IOS-XR as a container using Docker. My specific use case is lab environment and while it would be a perfectly fine thing to run IOS-XRv (which is a virtual machine) I found XRd to be a bit more pleasant.

The boot time of the VM can be measured in minutes while the container generally booted in less than a minute for me, and while trying things out I preferred the quicker boot time.

Caveats

The container - while good - isn’t perfect. Killing the container will trash the configuration so I had to save it manually (or edit the boot config manually, and start the container). Also anything L2 related will not work (like l2transport on a interface). I also wasn’t able to get LDP running between two devices, not sure why and I didn’t put any time looking into it.

But running ISIS, Segment Routing, BGP and EVPN was possible without any apparent issues.

Short theory

A lot has been written about containers, Docker and docker-compose so I’m not even going to try doing that.

We’re going to use xrd-tools, which is a Cisco repo with some examples and helper scripts. One of these helper scripts is xr-compose which is a wrapper around docker-compose. It takes in a modified version of docker-compose.yml called docker-compose.xr.yml, which specifies what startup config file, what interfaces and more will be used for a specific XRd container.

Server preparations

Before starting our first XRd continer we have to check some settings on the server, and modify as needed.

The base OS is a regular Debian 11.4.0, with nothing special going on. Some preparatory software has been installed:

Docker per the instructions on their website
docker-compose from the github repo, it will be used by xrd-tools
git, python3 and python3-yaml via apt

xrd-tools

Clone the repo and run a helper script to check the server:

user@lab:~$ git clone https://github.com/ios-xr/xrd-tools/
Cloning into 'xrd-tools'...
[...]
user@lab:~$ cd xrd-tools/scripts/
user@lab:~/xrd-tools/scripts$ ./host-check -e xr-compose
[...]
user@lab:~/xrd-tools/scripts$

I only care about the the base checks, xrd-control-plane checks and xr-compose checks.

Cgroups

Update /etc/default/grub:

GRUB_CMDLINE_LINUX_DEFAULT="quiet systemd.unified_cgroup_hierarchy=false"

$ sudo update-grub

Source

Inotify max user instances + Inotify max user watches

We’ll just do what it says:

$ echo fs.inotify.max_user_instances=64000 | sudo tee /etc/sysctl.d/50-xrd.conf
$ echo fs.inotify.max_user_watches=64000 | sudo tee -a /etc/sysctl.d/50-xrd.conf

Bridge iptables

We can’t do exactly what the output says but need some extra steps:

$ echo "br_netfilter" | sudo tee -a /etc/modules
$ echo net.bridge.bridge-nf-call-iptables=0 | sudo tee -a /etc/sysctl.d/50-xrd.conf
$ echo net.bridge.bridge-nf-call-ip6tables=0 | sudo tee -a /etc/sysctl.d/50-xrd.conf

Reboot the server and run the host-check script again. Now there shouldn’t be any issues except a warning about the docker-compose version. It’s alright, nothing to worry about.

xr-compose

xr-compose is another helper script allowing for some extra config when creating a network using docker-compose.

For ease of use I’m just moving it to a directory in my $PATH;

$ sudo mv ~/xrd-tools/scripts/xr-compose /usr/local/bin
$ sudo chown root:root /usr/local/bin/xr-compose

Import the XRd container

The container image is available on the Cisco website (if you have an account). I got the XRd Control Plane version 7.7.1, there’s also a vRouter but I haven’t tested that one.

user@lab:~$ docker load -i xrd-control-plane-container-x64.dockerv1.tgz
a42828b8fe58: Loading layer  1.179GB/1.179GB
Loaded image: localhost/ios-xr:7.7.1
johan@labb:~$ docker images
REPOSITORY         TAG       IMAGE ID       CREATED        SIZE
localhost/ios-xr   7.7.1     dd8d741e50b2   2 months ago   1.15GB

We should be good to go.

Testing

Create a docker-compose.xr.yml file and enter the following:

services:
  xr1:
    image: localhost/ios-xr:7.7.1
    container_name: xr1
    xr_interfaces:
      - Gi0/0/0/0
      - Mg0/RP0/CPU0/0

Launch the environment:

$ xr-compose --launch
INFO - Writing output docker-compose YAML to docker-compose.yml
INFO - Launching docker-compose topology...
[...]
$ docker attach --detach-keys=ctrl-a xr1

Hit the enter key 1-2 times and you should get some output. Enter CTRL-A to leave the console.

If you didn’t encounter any errors, congratulations, everything is working as expected! The final step is to remove the test environment and everything related to it:

$ docker-compose down -v