Eliminate garbage data in the CORE Network Emulator

While working through some of the previous tutorials about the CORE Network Emulator or IPv6, we noticed some strange broadcast packets in the Wireshark packet analyzer that appeared to have nothing to do with the processes running on the simulated network.

For example, we started a simulation consisting of two nodes connected to the same switch. We started the Wireshark packet analyzer on one of the nodes. No processes are running on the nodes that would generate data traffic on the links between them so we should see no data packets in the packet capture shown above. However, we see many broadcast frames carrying mDNS, NDP, and DCHP requests.

Data passing between simulated nodes in CORE is mixed up with other data packets generated by the host Linux system. Read on to learn how we stop this distracting data.

Linux bridges and broadcast packets

The CORE Network Emulator implements the Switch node type as a Linux bridge running in the host Linux system. If a process running on the host Linux system requests that the Linux kernel service a broadcast message, the Linux kernel automatically forwards that broadcast frame out every interface managed by the Linux system. That includes the interfaces on the Linux bridges that emulate switches in the CORE Network Emulator.

If we have Ethernet Switches set up in the simulated network topology, we will see what appear to be strange broadcast frames passing between ports on the switches. We can see that the source MAC address of each strange broadcast frame is a MAC address of a CORE Switch port — a virtual interface on a Linux bridge.

The 99% solution

There is no complete solution to this problem but we can eliminate most of the problem. One could try to create some complex filtering rules in ebtables but then we might accidentally cause changes to the behavior of the simulated network. The simpler way to eliminate most of the strange broadcast packets is to temporarily (or permanently) stop the processes on the host that generate the broadcast frames.

The primary suspects

The two processes running on the Linux host computer that forward the strange broadcast packets to the Linux bridge interfaces in the CORE simulation are:

1. The avahi-daemon generate MDNS packets
2. The Network Manager application generates IPv4 DHCP packets and IPv6 Neighbor Discovery Protocol packets.

Avahi

Avahi is an Implementation of the DNS Service Discovery and Multicast DNS specifications for Zeroconf Networking ((From About Avahi at avahi.org)). Avahi is the standard implementation of mDNS/DNS-SD on open-source operating systems such as Linux.

Avahi allows programs to publish and discover services and hosts running on a local network. For example, a user can plug their computer into a network and Avahi automatically finds printers to print to, files to look at and people to talk to, as well as advertising the network services running on the machine ((From Wikipedia: Avahi Software)).

We usually do not need mDNS running on a Linux system so we can disable the Avahi daemon without any negative consequences.

NetworkManager

The NetworkManager application makes networking “just work” on a Linux system. It manages Wi-Fi, wired, bridged, VPN, cellular, and Bluetooth connections. NetworkManager allows us to quickly move from one network to another ((From Ubuntu documentation: Network Manager)).

NetworkManager tries to manage every connection on the Linux system. This can cause problems in some situations. For example, we have previously seen how the CORE Network Emulator can temporarily interrupt the host system’s WiFi network connection when a simulation is started because NetworkManager tries to switch to the newly-created virtual Ethernet interfaces because it sees them as “wired connections”.

NetworkManager is more complicated to eliminate. Temporarily stopping it will kill our network connection. If we permanently disable it or remove it, we will have to manage out network connections manually. This would make connecting to WiFi at the local coffee shop a complex task.

The solution is to replace NetworkManager with an alternative that does not interact virtual network connections such as Linux bridges and virtual Ethernet interfaces like NetworkManager does.

A good alternative to NetworkManager is WICD.

Implementing a temporary solution

If we would prefer not to change anything on the system we have two options. We can tolerate the strange broadcast frames in the network simulation and we can use Wireshark filtering rules to show only the traffic we are interested in alalyzing. Or, we can temporarily stop the services that are generating the strange broadcast frames.

To temporarily disable the NetworkManager and Avahai daemon services, enter the commands:

$ sudo service network-manager stop
$ sudo service avahi-daemon stop

This will kill the network connection on the host so we lose connection to our home or office network. We need to execute these commands before we start the CORE simulation because NetworkManager will take down the simulated connections when it stops (and then we’ll need to restart the simulation).

When we restart the computer, these services will start again ((If we do not want to restart the computer, we can start the services again using the commands: sudo service network-manager start, and sudo service avahi-daemon start. But I think restarting ensures proper operation.)).

Implementing the permanent solution

To fix the problem we will first install WICD. Then we will disable Networkmanager and start WICD. Finally, we will permanently disable the Avahi daemon.

Prepare back-out plan

In case we cause unrecoverable problems ((If we chose to remove NetworkManager instead of just disabling the NetworkManager service)), let’s ensure we have an offline copy of the NetworkManager package available on our system so we can re-install NetworkManager without a network connection ((To re-install NetworkManager if we need to, execute the command sudo apt-get install network-manager. Because we will download and cache the NetworkManager package, we will be able to re-install it and restore the computer’s network connection if WICD does not work for us.)).

$ sudo apt-get install -d --reinstall network-manager network-manager-gnome 

We probably will not need this but it never hurts to be prepared.

Install WICD

WICD (pronounced like wicked) is an open-source network manager for Linux that supports only wired and wireless networks (WICD does not support bridged, VPN, cellular, and Bluetooth connections), but it’s a good alternative if NetworkManager causes problems ((From Ubuntu wiki: WICD.)).

To install WICD, enter the following command (or use the Ubuntu Software Center application):

$ sudo apt-get install wicd-gtk

The installer will ask us to choose the users assigned to the WICD group. In this case, we have only one user set up on the system. Press the Space Bar to select the user (an asterisk will appear between the square brackets), then press the Tab key to select the “OK” button, then press Enter.

Disable NetworkManager

NetworkManager does not co-operate with other network manager applications running on the same system. Some would suggest removing NetworkManager from the system ((To remove NetworkManager, enter the command: sudo apt-get remove --purge network-manager-gnome network-manager.)) but I think a less intrusive method is to disable the service so it does not start when the computer starts. This makes it easy to use NetworkManager again, if we need to.

Linux supports multiple ways to disable a service. I think that the easiest method (and one that is also easily reversible) is to add an override file for NetworkManager in the init scripts directory. The service will not start if the properly-configured override file exists in the same directory as the NetworkManager startup script.

The simplest way to do this is:

$ sudo bash -c "echo manual > /etc/init/network-manager.override"

This creates the file network-manager.override in the directory /etc/init. The file contains the text, manual. During startup, the Upstart scripts will find this file and will then start NetworkManager.

Linux tip: we cannot use the sudo echo command directly because it still creates a file with the same privileges as the user and we are trying to create that file in the /etc/init directory, which is owned by root. So we need to execute the echo command in a shell script that runs with root privileges. The bash -c command interprets and executes the remaining text on the line as a shell script.

Disable Avahi

We will use a similar approach when we disable the Avahi daemon so that it will not start when the computer starts. Create a file avahi-daemon.override in the directory /etc/init and enter the text, manual, in the file.

The simplest way is with the command:

$ sudo bash -c "echo manual > /etc/init/avahi-daemon.override"

Restart the system

Restart the computer using the Xubuntu menu or the command:

$ sudo reboot

WICD dbus error

After restarting, we may see an error from WICD that states the WICD daemon could not connect to WICD’s D-Bus interface. This seems to be caused by a bug that has not yet been fixed in Linux.

To resolve this problem, clear the resolv.conf file with the following commands ((From Linux Forums Topic: wicd fix – could not connect to wicd’s D-Bus interface.)):

$ sudo mv -v /etc/resolv.conf /etc/resolv.conf.backup
$ sudo ln -s /run/resolvconf/resolv.conf /etc/resolv.conf
$ sudo rm -v /var/lib/wicd/resolv.conf.orig
$ sudo service wicd start

We should restart the computer again to ensure the problem is fixed.

Verify the fix

Now we should see no more strange broadcast frames arriving at nodes running in a CORE network simulation. We can verify this by starting CORE, creating a small network with two PCs connected to an Ethernet Switch, starting the simulation and then starting Wireshark on one of the simulated PCs.

If we do nothing after that, we should not see any packets arrive at the PC’s interface. Wait five minutes to verify the result. We may see one or two IPv6 multicast messages if the host Linux computer joins or leaves an IPv6 multicast group (we cannot solve that issue but it results in very little traffic and usually generates zero traffic in a typical home or office network).

Conclusion

We implemented some changes to the Linux system configuration that stop the system from periodically forwarding broadcast frames through the virtual bridge interfaces into the CORE network simulation.

 

Notes