Run the Antidote network emulator on KVM for better performance

Antidote is the network emulator that runs the labs on the Network Reliability Labs web site. You may install a standalone version of Antidote on your personal computer using the Vagrant virtual environment provisioning tool.

In this post, I show you how to run Antidote on a Linux system with KVM, instead of VirtualBox, on your local PC to achieve better performance — especially on older hardware.

Why use KVM instead of VirtualBox?

Antidote runs emulated network nodes inside a host virtual machine. If these emulated nodes must also run on a hypervisor, as most commercial router images require, then they are running as nested virtual machines inside the host virtual machine. Unless you can pass through your computer’s hardware support for virtualization to the nested virtual machines, they will run slowly.

VirtualBox offers only limited support for nested virtualization. If you are using a Linux system, you can get better performance if you use Libvirt and KVM, which provide native support for nested virtualization.

When to use VirtualBox

If you plan to run Antidote on a Mac or a PC, you should use Antidote’s standard installation with VirtualBox ((Note that HyperV and VMware also support nested virtualization, but they are not open-source tools and are platform-specific.)). Vagrant and VirtualBox are both cross-platform, open-source tools.

The VirtualBox developers plan to improve support for nested virtualization. See the VirtualBox 6.0 changelog and the VirtualBox 6.1 beta release notes for more information. However, the planned VirtualBox nested virtualization features require a 5th generation Core i5 or i7 Broadwell processor, or newer, or an AMD processor. Older hardware, like my Lenovo T420 laptop, which uses a 2nd generation Intel Core i5 Sandy Bridge processor, will not benefit from those improvements.

Check your PC’s virtualization support

Before you start using KVM and Libvirt, verify that your Linux computer has hardware support for virtualization. Enter the following commands in the VM’s terminal.

$ grep -cw vmx /proc/cpuinfo

It should return a value equal to the number of virtual cores on your processor. If it returns 0, then something is wrong. You may need to change your PC’s BIOS or EFI settings.

Check the nested virtualization settings on the host computer:

$ cat /sys/module/kvm_intel/parameters/nested

The output should be “Y”. If it is “N”, run the following commands to enable nested virtualization:

$ sudo modprobe -r kvm_intel
$ sudo modprobe kvm_intel nested=1

To make the changes persistent after a reboot, add a line with the text, options kvm_intel nested=1, to the file /etc/modprobe.d/kvm.conf, as shown below.

$ echo "options kvm_intel nested=1" | sudo tee /etc/modprobe.d/kvm.conf

Install software

On a Linux system running Ubuntu 18.04, install the software the Antidote relies upon. If you are using a different Linux distribution or version, this same procedure should still work. You may need to modify it a bit if you are using different package managers and different software repositories, but you should be able to find the necessary adaptations with a few Internet searches.

Install the KVM hypervisor, an NFS server, Libvirt, and Vagrant. Vagrant does not natively support Libvirt, so you must also install the vagrant-libvirt plugin.

Libvirt

Install libvirt using the following commands:

$ sudo apt update
$ sudo apt -y install libvirt-clients libvirt-daemon-system qemu-kvm
$ newgrp libvirt

Utilities

Install additional utilities, if they are not already installed:

$ sudo apt install -y git curl

Vagrant

Install the latest version of Vagrant. Go to the Vagrant web site and download the Vagrant package, or use the curl command below. Replace the file name in the below example with the latest version. Then, install it.

$ cd ~/Downloads
$ curl -O https://releases.hashicorp.com/vagrant/2.2.6/vagrant_2.2.6_x86_64.deb
$ sudo dpkg -i ~/Downloads/vagrant_2.2.6_x86_64.deb

NFS server

The Antidote project uses NFS to transfer files from the host computer to the Antidote VM. Install an NFS server, if it is not already installed.

$ sudo apt install nfs-kernel-server

Vagrant-libvirt plugin

Enable the source code repositories so you can build the vagrant-libvirt plugin. Edit the file /etc/apt/sources.list and uncomment-out all the lines with deb-src at the start, as shown below:

$ sudo sed -i 's/# deb-src/deb-src/g' /etc/apt/sources.list
$ sudo apt update

Install the build dependencies for the vagrant-libvirt plugin:

$ sudo apt build-dep -y vagrant ruby-libvirt
$ sudo apt install -y qemu libvirt-bin ebtables dnsmasq-base
$ sudo apt install -y libxslt-dev libxml2-dev libvirt-dev zlib1g-dev ruby-dev

Install the plugin. The installation process will build it from the source code, so it may take a few minute to complete:

$ vagrant plugin install vagrant-libvirt

Host updater plugin

Install the other Vagrant plugins. The Antidote Vagrant install guide also requires the vagrant-hostsupdater plugin. Not that you should not install the other plugins mentioned in the Antidote documentation, because those plugins support VirtualBox, which we are not using.

$ vagrant plugin install vagrant-hostsupdater

Login to Vagrantcloud

Finally, create account on vagrantcloud.com. You need access to vagrantcloud.com to download the Vagrant box specified in Antidote’s Vagrantfile.

After you create an account, log in to Vagrant with the following command:

$ vagrant cloud auth login

Enter your Vagrantcloud userid and password.

Download and install Antidote

Antidote requires you to create a lessons directory before you install it. Download the nrelabs-curriculum directory from the NRE Labs GitHub repository.

$ mkdir ~/antidote-local
$ cd ~/antidote-local
$ git clone https://github.com/nre-learning/nrelabs-curriculum

Download the antidote-selfmedicate directory. Unless you specify a new location in the antidote-config.yml configuration file (see below), you must install this directory at the same point in your filesystem as the nrelabs-curriculum directory. That is, each of these should be sub-directories of the same directory.

$ git clone https://github.com/nre-learning/antidote-selfmedicate

Configure Antidote

Antidote has some default values pre-configured the tell it which Vagrant provider to use and how much resources the Minikube VM should consume. The user may modify any of these values by editing the antidote-config.yml file in the antidote-selfmedicate directory.

$ cd antidote-selfmedicate
$ nano antidote-config.yml

Edit the file and enter in values appropriate for your own situation. For my system I chose the following configurations:

vm_config:
  memory: 4096
  cores: 2
  provider: libvirt

Save the file. Below, I describe each line in the file.

Memory

As I mentioned earlier, I am using my 9-year old Lenovo T420 laptop, which only has two hyper-threaded CPU cores and 8 GB memory. It is not powerful enough to run the Antidote VM using default configurations. After some experiments, I found that running an idle Antidote VM with 4 GB of RAM, alongside the other tools I use, results in the general performance of my laptop staying in an acceptable range and, when using Libvirt instead of VirtualBox, seems to support the lessons currently available in the curriculum — just barely.

Cores

I found performance is much better when I respect the KVM Performance Limits for virtual CPU cores, so I set the VM vCPU requirements to 2 CPU, which is appropriate for my laptop.

Provider

Change the provider to libvirt.

Run the Vagrant provisioner

Run Vagrant using the up command. Vagrant will read the Vagrantfile in the antidote-selfmedicate directory and create the environment specified in the file:

$ cd ~/antidote-local/antidote-selfmedicate
$ vagrant up

It may take a long time to get started because it is downloading large files from the NRE Labs repositories.

Testing performance

I compared Antidote’s performance on my Linux PC with both Libvirt and VirtualBox. I ran Antidote with Libvirt on my Ubuntu Linux 18.04 laptop. Then, I reinstalled Ubuntu Linux 18.04 and ran Antidote with VirtualBox.

I ran lessons that launched QEMU/KVM nested virtual machines as network nodes. Each nested VM supported a Juniper VQFX router image. I ran lessons that started one, two, and three VMs. I measured the time it took for the lesson to start. I did not measure how the nodes performed after they started.

The lessons I tested were:

• 1 VM: Junos PyEz lesson 24
• 2 VMs: Device Specific Template Generation lesson 35
• 3 VMs: Junos Terraform lesson 31

Between each test run, I ran the vagrant reload --provision command to restart the Minikube VM and refresh all configurations. I ran each test 7 times, discarded the highest and lowest time, then averaged the remaining five times. I display the test results in the table below.

Start time, in seconds	1 VM	2 VM	3 VM
NRE labs web site	43	38	50
Libvirt	108	139	315
VirtualBox	100	175	560*

I was surprised to see the performance was so close for the lessons with one VM. The Libvirt provider performed a bit better in the two-VM tests. However, the three-VM test showed a large improvement in performance for the Libvirt provider compared to the VirtualBox provider. The VirtualBox three-VMs test failed five out of seven times and the two successful test times were over nine minutes.

I suspect that, because I set the Minikube VM size to 4 GB, which is well below the minimum recommended VM size of 8 GB, the three-VM test is at the very edge of what can be supported in that configuration.

See the chart, below, for another view of the results:

For comparison, I also included timing results from the NRE labs web site. On that site, Antidote is running on bare metal on a modern server, so it is much faster.

Conclusion

Running the Antidote network emulator on a Linux system using KVM and Libvirt, instead of VirtualBox, results in measurably better performance, especially when using an older computer, or when you have limited memory.