Network setup in KVM

Introduction

This post is about setting up networking for KVM guest systems. There are many similar and related posts like libvirt network wiki, Red hat’s KVM network guide, blog post about networking in KVM and also this post.

The following notes are based on my Ubuntu 20.04 laptop.

NAT(default)

The libvirt uses a virtual network switch, which shown as a network interface virbr0 in the host machine. You can check the virbr0 interface by running the ifconfig command. And below is the output on my machine

enp0s31f6: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.1.138.24  netmask 255.255.255.0  broadcast 10.1.138.255
        inet6 fe80::f7e9:67fa:e248:186  prefixlen 64  scopeid 0x20<link>
        ether 98:fa:9b:fe:a5:8e  txqueuelen 1000  (Ethernet)
        RX packets 720054  bytes 1030213234 (1.0 GB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 146312  bytes 29140756 (29.1 MB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
        device interrupt 16  memory 0xd9300000-d9320000  

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 1226  bytes 120921 (120.9 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 1226  bytes 120921 (120.9 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

virbr0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet 192.168.122.1  netmask 255.255.255.0  broadcast 192.168.122.255
        ether 52:54:00:f4:c5:55  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

wlp0s20f3: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        ether 04:ed:33:82:a1:d1  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

So my host machine is running a wired connection through enp0s31f6 interface with IP address 10.1.138.24 while the virbr0 interface got IP address 192.168.122.1. The guest systems connect to it. And by default, it runs at NAT mode, which means the guest machine will got IP address like 192.168.122.xxx.

This means the host and guest systems are not in the same subnet. This makes communication between host and guest systems a little bit tricky. As shown in the [file share guide](/post/filesharingrmd/), the host system uses 192.168.122.xxx(the guest IP) to connect to the SAMBA server on the guest system while the guest system uses 192.168.122.1(the guest network gateway) to connect to the SAMBA server on the host system. Also the guest system is not directly accessible from the real network.

In the guest system, this network will work with either e1000 or the virtio device models.

Bridge

A public bridge can put your host and guest system at the same subnet and make your guest system accessible from the local network.

NOTE: This configuration will NOT work on the wireless interface on the host machine since most of them(wireless protocols) do not support bridging.

NOTE: In my experience, this setting will only work with virtio NIC device model in the guest system settings.

  1. In order to setup a network bridge, you have to first identify your host system’s network management system. There are basically two of them: netplan and network-manager. Run the command networkctl status. If your system is using netplan, the output would be like

     # networkctl status
              State: routable                             
              Address: 192.168.86.242 on enp0s3             
                       fe80::a00:27ff:fe52:69a9 on enp0s3   
              Gateway: 192.168.86.1 (Google, Inc.) on enp0s3
                  DNS: 192.168.86.1                         
       Search Domains: lan                                  
     May 04 15:46:09 demo systemd[1]: Starting Network Service...
     May 04 15:46:09 demo systemd-networkd[625]: Enumeration completed

    If your system is uring network-manager, the output would be like

     # networkctl status
     WARNING: systemd-networkd is not running, output will be incomplete.
    
       State: n/a                                                      
       Address: 10.1.138.24 on enp0s31f6                                 
                192.168.122.1 on virbr0                                  
                fe80::f7e9:67fa:e248:186 on enp0s31f6                    
       Gateway: 10.1.138.250 (New H3C Technologies Co., Ltd) on enp0s31f6

    Also you will find a 01-network-manager-all.yaml at /etc/netplan, saying the renderer is NetworkManager.

  2. Get the current network setting and setup a network bridge. I will use network-manager since that’s running on my machine. For a guide using netplan, one can check this post.

    1. command line tool. Use nmcli con show to display the current connections.

          NAME              UUID      TYPE      DEVICE     
      Wired connection 1  -trimmed-  ethernet  enp0s31f6     
      virbr0              -trimmed-  bridge    virbr0        

      One can also use nmcli device show to find out more details about these network interfaces. To add a new network bridge, use the nmcli tool

       nmcli con add ifname br0 type bridge con-name br0

      This will add a new interface br0 with connection name br0. Then we have to connect it to our physical ethernet connection by running

       nmcli con add type bridge-slave ifname enp0s31f6 master br0

      Now the current connections are

           NAME              UUID      TYPE      DEVICE     
        br0                -trimmed-  bridge    br0
       Wired connection 1  -trimmed-  ethernet  enp0s31f6     
       virbr0              -trimmed-  bridge    virbr0        
    2. graphical tool. One can use the graphical tool for network-manager, the nm-connection-editor to add the bridge. Simply run it from the terminal will bring up its interface.

      1. Choose the + sign for add a new connection. Then select the type Virtual(Bridge) and click Create.
      2. Set the connection name and interface name. I’m using nm-gui-bridge and nm-gui-br respectively.
      3. Click the Add button located to the right of the Bridge connections list. Choose the Ethernet type and set the connection name of the slaved connection(I’m using nm-gui-br-slave1). Add the physical device (in my case it’s enp0s31f6) and click Save.
      4. Click save and back to the connection editor interface. Make sure it self and the slaved connection is shown then you can exit.
  3. (Optional) Other modifications to your bridge. My working place here requires a static IP assigenment without the DHCP service. Therefore before I bring up the br0, I need to manually set up its IP address. One can modify the settings via the nmcli tool.

     nmcli connection modify br0 ipv4.address 10.1.138.24/24
     nmcli connection modify br0 ipv4.method manual
     nmcli connection modify br0 ipv4.gateway xxx.xxx.xxx.xxx

    Here I set br0 according to my working place’s reuqirement. One can see more modifications from the manual.

  4. Bring up the bridge. At this point you need to stop the original ethernet connection and start up the bridge connection. Here I will bring up the bridge connection set up by the command line tool(hence br0) by running

     nmcli con down "Wired connection 1"
     nmcli con up br0

    If you’re connecting to your host machine remotely(especially through this specific interface which you’re about to bringing it down). Then this could be problematic since you will loss connection hence could not bring up the br0 interface. This could be solved by saving these commands in a script and then just execute that script.
    You can check your network status by

     # nmcli connection show 
     NAME                UUID       TYPE      DEVICE    
     br0                 -trimmed-  bridge    br0       
     br0_slave           -trimmed-  ethernet  enp0s31f6 
     virbr0              -trimmed-  bridge    virbr0  

    Note: All these connections should be displayed in a green color indicating they are all active.

  5. Define the network in KVM. Currently this bridge it presented in the system but not visiable to the KVM environment. You can verify this by

     # virsh net-list --all
      Name                 State      Autostart     Persistent
     ----------------------------------------------------------
      default              active     yes           yes

    We need to first prepare a definition file called bridge.xml with contents like this:

     <network>
       <name>br0</name>
       <forward mode="bridge"/>
       <bridge name="br0" />
     </network>

    Then we can use this file to define the network in KVM environment by

     # virsh net-define ./bridge.xml
  6. Activate the network in KVM. Now that it’s defined in the KVM environment, we can start it by

     # virsh net-start br0

    and the current status would be

     # virsh net-list --all
      Name                 State      Autostart     Persistent
     ----------------------------------------------------------
      br0                  active     yes           yes
      default              active     yes           yes

    If you want to auto-start it you can run

     # virsh net-autostart br0

    You can find more setting about this using virsh net-<TAB>, such as net-destroy for stopping a KVM network and net-undefine for undefining a persistent network.

Chao Cheng
Chao Cheng
Statistician

My research interests include applied statistics and machine learning.

Related