Manual networking for KVM

I found the networking configured by libvirt (KVM) did not allow me to firewall the network as I desired.  I use Shorewall for firewalling, and DNSMasq for internal DNS and DHCP.  After a little experimentation, I found that I could configure Ubuntu to create the network.  This allows me to get a reliable firewall configuration with a virtual DMZ.

The virtual hosts are assigned to a bridge, and only have connectivity to other networks as defined in the Shorewall configuration.  A single DNSMasq server provides DSN an DHCP services for all virtual servers, as well as the network the server is connected to.  The network and firewall configuration remains consistent even as servers are cycled up and down.  An additional bridge was created to support virtual servers in the DMZ zone.

Disabling the Existing Network

The command virsh net-list --all will list all the existing networks.   Disable auto-start for each network current marked for auto-start.  To disable the default network use the command virsh net-autostart default --disable.  Replace default with name of the network to be disabled.

You may want to edit the existing networks using the virsh net-edit command.   Record the information being deleted if you want to replicate it into your new configuration.  The example below is the definition for a test network using a bridge named virbr2.

<network>
 <name>test</name>
 <uuid>91ae6378-841c-89d1-cf48-2d66a212e807</uuid>
 <bridge name='virbr2' stp='off' delay='0' />
</network>

You can use the above example to define new networks. Remove the uuid line, and replace the names with appropriate new values.

Defining the Bridges

The existing Ubuntu network scripts can by used to create the bridges when the network starts.  This is done in the /etc/network/interfaces script.  The following example creates a bridged for the default using a bridge name virbr0.

auto virbr0
iface virbr0 inet static
 address 192.0.2.1
 netmask 255.255.255.0
 pre-up brctl addbr virbr0
 post-down brctl delbr virbr0

The address used is an example and will need to be edited appropriately. Additional bridges can be defined using this example as a template.   The address and bridge name need to be unique for each network.  To handle a DMZ I added a bridge named virbr1.

The libvrt.org wiki shows how to build a bridge configuration attached to an interface.   This will create a network outage.  You may want to replicate the eth0 configuration as the br0 configuration.  Copy and update all appropriate configurations before switching from eth0 to br0 and disable automatic start-up for eth0.

Configuring Shorewall

The sample configurations are a good starting point for configuration.  I used the three interface example to handle my DMZ.  The default fw, dmz, loc and net zones are used as is.  An additional virt zone was added to handle virtual serves not in the DMZ. The Shorewall documentation provides documentation for a simpler setup such as might be used on a laptop.

Edit shorewall.conf to ensure IP forwarding is enabled. This is done with the line IP_FORWARDING=Yes. Review this file to see if you want to change anything else.

My loc zone supporting the LAN is routed over eth0 as is the net zone.  A firewall between the Internet and the LAN is used to prevent spoofing of LAN addresses from the Internet. If this were also the Internet firewall, then the loc zone should be on a different Ethernet adapter.

The interfaces file lists each network interface and bridge.  Servers in the dmz zone are expected to have static addresses so DHCP is disabled.  DHCP is enabled on the net interface enabling DNSMasq to provide addresses to the LAN.  It would also be enabled if the net interface got its address via DHCP.   DHCP is optional for the virt zone, but may be required for prepackaged virtual servers.  The routeback option may be required on the bridge zones if servers on the bridge need to talk to each other.

net     eth0            detect          tcpflags,routefilter,nosmurfs,logmartians,blacklist,dhcp
virt    virbr0          detect          tcpflags,routefilter,nosmurfs,logmartians,routeback,dhcp
dmz     virbr1          detect          tcpflags,routefilter,nosmurfs,logmartians,routeback

A single line in the hosts file handles defining the loc zone.  The LAN uses a private address range so it is relatively easy to secure from spoofing from the Internet.  Public IP addresses would be more difficult to secure.

loc     eth0:192.168.10.0/24

The policy file contains an unusable policy allowing full access from loc to net. This should be disabled, or changed to allow access from virt to the net.   Diagnosing problems may be simpler if you define a policy for each local zone to all with logging enabled.   I replicate the all to all policy, and change the origin.

The masq file is deleted.  It is not required in our configuration.  For hosts directly connected to the Internet the provided file is likely sufficient and should be kept.

The routestopped file needs to be edited if used. It is a good idea to have some sort of access when the firewall is stopped. This file is used to define access.

The rules file may need to be tailored significantly.   Many of the rules applying to the loc zone may be applicable to the virt zone.   Consider which protocols each zone need to accept, or connect to.  These must be permitted between the appropriate zones.  Rules may be built that limit access for a particular host or hosts. These are some basic items that should be considered.

  • ICMP must be enabled everywhere.  Pings from the Internet (net zone) are blocked in the default rule set.
  • NTP should not be run on the virtual servers. It should be run on the firewall.
  • DNS should be allowed to the appropriate zone(s).
  • SSH should be allowed between appropriate zones.
  • HTTP and/or HTTPS may need to be allowed between appropriate zones.
  • Services available on or require by the virtual servers should be available from the appropriate zones.
  • DNAT (Destination NAT) may be used to provide access to the DMZ from the Internet (net).
  • Policy rules may be used to provide unlimited access from one zone to other zones. (This makes securing access more difficult.)

Configuring DNSMasq

Installing libvirt will have installed the dnsmasq binaries.  If you haven’t already done so, install the dnsmasq package to install the init scripts.  Install your configuration changes as files in the /etc/dnsmasq.d directory.

The dhcp-range option can be used to assign network-ids to the various interfaces.  This is done by matching the address of the interface to the address range.  Network-ids can then be used to send different DHCP parameters to clients in different address ranges.   The following example provides network-ids matching the Shorewall zones.

dhcp-range=loc,192.168.10.200,192.168.5.250,255.255.255.0,192.168.5.255,120h
dhcp-range=virt,192.168.11.200,192.168.6.250,255.255.255.0,192.168.6.255,120h
dhcp-range=dmz,192.168.32.200,static,255.255.255.0,192.168.32.255,240h

You can then supply zone specific parameters as in the following example defining DNS networks.

dhcp-option=15,example.com # Default
dhcp-option=virt,15,virt.example.com
dhcp-o ption=dmz,15,dmz.example.com

Hosts which provide their name should be given the appropriate static  address as defined in the /etc/hosts file. Static addresses can also be defined in the /etc/ethers file.  Enable this file by adding the read-ethers option to your dnsmasq configuration.   The mac address for the virtual servers can be obtained using the virsh dumpxml command.  If the host has a DHCP lease, its mac address will be recorded in /var/lib/misc/dnsmasq.leases.   The /etc/hosts and /etc/ethers files can be used to document your network.

Review the dnsmasq man page for additional options you may want to set.  The /usr/share/doc/dnsmasq/setup.html file provides good setup information.  The documentation set can be found in  /usr/share/doc/dnsmasq or /usr/share/doc/dnsmasq-base.