QEMU TAP support

(This is part of a larger guide: making a virtual machine.)

You may wish to copy the script file. In the script file's current form/shape, the script file may be quite similar to what will be useful for other virtual machines. However, the changes that are about to be made may be unnecessary for most virtual machines (after the first one is working as desired). To make a conveniently accessible copy of that file, get to a command prompt on the machine that runs the virtual machine software, set some initial variables to identify a script if those variables need to be set, and run the following (on the machine that runs the virtual machine software).

echo ${VMDirBas} ${VMGenNam} ${VMLILNAM}
pwd
cd .
cd ${VMDirBas}/execbin/${VMGenNam}/${VMLILNAM}/sysstart/
cp -i ./exc_${VMLILNAM} ./exc_${VMLILNAM}-before-NIC-cfg
cd ${OLDPWD}
ls -l ${VMDirBas}/execbin/${VMGenNam}/${VMLILNAM}/sysstart/

(The second command was just to help make sure that ${OLDPWD} is not a blank value.)

The next step is to modify the script file. (An original version of the script file may be found in the section on making a Qemu virtual machine configuration.)

echo ${VISUAL}
${VISUAL} ${VMDirBas}/execbin/${VMGenNam}/${VMLILNAM}/sysstart/exc_${VMLILNAM}

Regarding the details on how to modify the script file, there are some various methods that have been documented. Choose from one of the following methods.

Preferred, streamlined process

(This requires that the startup script file is a fairly recent version. People wishing to alter an older version of the startup script file might want to try one of the other sets of steps.)

Find the lines in the startup file that state:


# unset CLIOPTVMFIRSTNIC
# if the previous line is uncommented then the following options will be used
# instead of using any of the earlier options
[ "${CLIOPTVMFIRSTNIC}X" = "X" ] && \
    export CLIOPTVMFIRSTNIC=" -net tap,vlan=4,ifname=tun0\ ,script=${VMDirBas}/execbin/${VMGenNam}/${VMLILNAM}/nicscr/upif0\ ,downscript=${VMDirBas}/execbin/${VMGenNam}/${VMLILNAM}/nicscr/dnif0 "



In this section, remove the hash/comment character from the first line. So, the results should look like:


unset CLIOPTVMFIRSTNIC
# if the previous line is uncommented then the following options will be used
# instead of using any of the earlier options
[ "${CLIOPTVMFIRSTNIC}X" = "X" ] && \
    export CLIOPTVMFIRSTNIC=" -net tap,vlan=4,ifname=tun0\ ,script=${VMDirBas}/execbin/${VMGenNam}/${VMLILNAM}/nicscr/upif0\ ,downscript=${VMDirBas}/execbin/${VMGenNam}/${VMLILNAM}/nicscr/dnif0 "



Note: There are multiple lines that say “unset CLIOPTVMFIRSTNIC”. It is critical that the uncommented line is the one that appears right before the section that says “ -net tap,”. Otherwise, the effect will not be what is described by the rest of this guide.

Also, there must not be a later line in the file that says “unset CLIOPTVMFIRSTNIC” and which is uncommented. (That probably will not be an issue, but is being mentioned because it could conceivably be an issue with some variation of the script.) If any such line is found, comment it out. A quick way to check for such a line is to either use the search functionality in the text editor, or to quit the text editor and run:

grep "unset CLIOPTVMFIRSTNIC" ${VMDirBas}/execbin/${VMGenNam}/${VMLILNAM}/sysstart/exc_${VMLILNAM}

Please proceed to the section: Additional steps to support the new NIC style.

Older instructions/method

Note: There should be no need to follow the steps in this section, if using the provided sample configuration files. However, this is some older documentation that shows a bit of a lengthier process, and these instructions may be closer to the required steps for someone who is using their own Qemu startup scripts instead of the recommended provided sample/example scripts.

This process also may not have been entirely tested, at the time of this writing.

Removing any prior NIC configuration

The goal here is to remove any pre-existing use of the “-net ” parameter so that it doesn't cause any conflicts with the network configuration that is about to be created, and so that there aren't too many unexpected NICs seen on the virtual machine.

For those who are using the sample scripts, try to find lines such as the following:

-net nic,vlan=1,macaddr=52:54:00:12:${VMNUM}:01 \
${CLIOPTVMFIRSTNIC} \

Use the following/upcoming instructions to disable each command line parameter that starts with “-net ” (followed by a parameter that starts with a word, and then generally has a comma and more information which might be additional pieces separated by commas). Also, do disable the parameter that says “${CLIOPTVMFIRSTNIC}” (if it appears) because that ends up getting translated to another “-net ” parameter, and all of the “-net ” parameters need to be removed.

How to disable the lines

One way to disable those lines are to delete them.

A classier way is to “comment out” the unwanted information, which means to turn the unwanted information into “comments” so that they effectively do not have any impact. The benefit to making changes this way is that restoring the previous functionality can be done more easier (by simply reversing the changes later, by removing the characters that cause the information to be a comment).

(A potential disadvantage to this method is that it leaves the screen cluttered with information that is not being actively used. When there is too much of this clutter than the comments can start to become more of a problem, or at least an annoyance, rather than being much of a benefit.)

Commenting out the lines might be a viable option: this is a viable option if using OpenBSD's ksh. This is an option if using OpenBSD's ksh. With this shell, disabling may be done by commenting out the line (by prefacing the line with “$(#” and placing a corresponding ) immediately before the “\” character).

Be warned, though, that this method of disabling lines may not work for some shells that don't check for characters after the start of the comment character. As an idea, perhaps an alternative option is to start the comment with “$(:” (and also placing a corresponding ) immediately before the “\” character).

For people who are using other shells, or who just want to do things as simple as possible right now, remember that an option is to just document and then delete the lines with the unwanted parameters.

In case this wasn't abundantly clear, the goal is not to delete just the word “-net ”, but also delete the information that came right after it. (So, if the parameter says “-net nic,vlan=1,macaddr=52:54:00:12:${VMNUM}:01 ”, then get rid of all of that.

The backslashes at the end of the lines are still needed. (The backslash at the end of the line is telling the computer to ignore the invisible “newline” character, so the computer will treat the next line like it is part of the previous line. That is still needed.) If there are any lines that have only white space (spaces and tabs), and a backslash at the end, then they are unnecessary lines, so they can be removed.

Adding NICs

Add the following lines to the Qemu configuration file, which are based on information from Walkthrough for setting up networking in Qemu.

 -net nic,vlan=4,macaddr=52:54:00:12:04:11 \
 -net tap,vlan=4,ifname=tun0,script=${VMDirBas}/execbin/${VMGenNam}/${VMLILNAM}/nicscr/upif0,downscript=${VMDirBas}/execbin/${VMGenNam}/${VMLILNAM}/nicscr/dnif0 \
 -net nic,vlan=5,macaddr=52:54:00:12:05:01 \
 -net socket,vlan=5,listen=127.0.0.1:43000 \

(Note: the lines that say “ -net tap,vlan=” are fairly long... there is no white space, including any word wrapping, intended to be between the “ -net tap,vlan=” and the backslash that follows that text, except for an optional space immediately before the backslash. The lines could be broken up using escaped newline characters, but then the following text would need to be at the very start of the next line, without any indenting. Though technically possible, those end results might make things look more cumbersome, and so less pleasant to read.)

This example only shows creating two NICs. The example documented at the Walkthrough for setting up networking in Qemu shows creating three NICs. Only one is required, but specifying two might save some time and effort later. If a third one is required, it may be set up later (after confirmation that the first one is working fine).

(Please proceed to the section about: Additional steps to support the new NIC style)

Older partial instructions

This was an earlier idea, which is just being left for reference. It shows an alternative approach. However, it is not likely to be more useful than the alternatives shown.

Just above those lines, insert the following fairly long line:

 -net nic,vlan=4,macaddr=52:54:00:12:04:02 \
 -net tap,vlan=4,ifname=tun0,script=${VMDirBas}/execbin/${VMGenNam}/${VMLILNAM}/nicscr/upif0,downscript=${VMDirBas}/execbin/${VMGenNam}/${VMLILNAM}/nicscr/dnif0 \
 -net nic,vlan=5,macaddr=52:54:00:12:05:01 \
 -net socket,vlan=5,listen=127.0.0.1:43000 \
[#qmusifup]: Additional steps to support the new NIC style

There are a couple of things in the virtual machine startup script file, that may be worth checking.

MAC uniqueness

Make sure that the MAC addresses are customized to this machine. No other machine should be using the same MAC address.

Interface name
Issue number one: Choosing a workable name

The startup script file should now say “ -net tap,” and “,ifname=tun0”. That device name (which is the part after the equal sign: the “tun0”) may need to be customized.

The TAP device (e.g. tun0”) may be something that is good (and even rather required) to customize. Following are some reasons why this might be good to change that:

Platform-specific

Some operating systems may need to use a different device, such as “tap0”. The name of the device here should likely match the name specified on the “ -net tap,” in the virtual machine startup script.

TAP in OpenBSD

In OpenBSD, the TAP support is supported by the tunnel interfaces. The interface name to use will bie something like tun0.

For anyone who is not using all of the sample scripts referenced by this guide, it may be important to know that the interface may need to have link0 passed as a command line parameter to ifconfig (e.g. “ ifconfig tun0 link0 ”.) That is done to enable TAP (meaning to enable operation of handling entire Ethernet frames rather than just IP packets). Those who are using the sample scripts by this guide may not need to worry about that, because that gets handled by the sample NIC startup scripts.

Some less common potential problems: See what TAP devices are available, with the following command:

ls -l /dev/tun[0-9]*

If there are some listed, but not enough, then using something like “ /dev/MAKEDEV tun5 ” may be helpful. (However, that might be a step that was really only needed with older versions of OpenBSD, and might not be needed with recent versions of OpenBSD?) Also, support for TUN/TAP is required to be enabled in the kernel. That should not be an issue with modern “GENERIC” (or similar, like “GENERIC.MP”) kernels, so should not be an issue for most people.

Mac OSX
Unknown, but TunTap for OSX looks potentially interesting.

Users of other platforms are recommended to check for documentation specific to the operating system. There might be notes in the virtual machine software package. The most common device names are probably tap0 or tun0. Most users will probably want to find a pre-developed way to have TUN/TAP supported by the operating system. A Universal TUN/TAP driver might be helpful for the more adventurous/skilled/desperate.

If all else fails, there is probably an alternative approach which won't be quite as nice, but may be functionally suitable. It is to abandon using “ -net tap,”, use an alternative, and have Qemu provide some sort of traffic redirection (using “ -net user,” and “,hostfwd=” or the older -redir variation. Again, this provides a generally inferior experience, and is not recommended, but might be the saving grace that provides salvation in a tight pinch.

Uniqueness might be desirable

Every virtual machine “ -net tap,” NIC should have its own unique tunnel device if traffic separation is desired. If multiple virtual machine NICs (even on different virtual machines) are using the same TUN/TAP device, then the effect will be similar to if multiple NICs are plugged into a single switch/hub. That may be more likely to allow machines to communicate, but could be bad from a security perspective. Also (and this is speculative, but probably right) using the same TUN/TAP interface might be prone to cause problems (like a switching loop that leads to a broadcast storm). All of these may or may not be problems, depending on whether things are done right (from the perspective of proper network design). The *safest* way to do things is generally to have each virtual machine NIC use its own TUN/TAP device (and then handle any resulting traffic routing issues).

[#qmuvlmtc]: Issue number two: Matching is required and good

In the sample script, “,vlan=4” is used. This mismatch does cause problems. This was actually done intentionally, as a design choice meant to cause multi-homed systems to be a bit different than the Qemu VLANs used on simpler setups. However, the downside is simply that this mismatch does need to be taken care of. Later on the file refers to “-vlan=1” in the command line. Those numbers need to match.

In the next file, find the lines that say:

 -net nic,vlan=1,macaddr=52:54:00:12:${VMNUM}:01 \
 ${CLIOPTVMFIRSTNIC} \

Change the “-net nic,vlan=1,” to say “-net nic,vlan=4,”.

Additionally, although this is not strictly required, adjust the MAC address so that the MAC address maches the value used for “,vlan=”. (So, if the command line says “-net nic,vlan=4,”, then make the MAC address end with a 04.) This is nice to do because having the numbers match up can simplify some troubleshooting. (If, later, there are any questions about which NIC visible on the virtual machine matches the command line parameters, recognizing these numbers may speed up the related troubleshooting.)

It is curious that VTUN FAQ uses the phrase “TUN/TAP”. FAQ 1.6: “What is the difference between TUN driver and TAP driver?” states, “TUN works with IP frames. TAP works with Ethernet frames.” Since, IP packets are above Ethernet frames in the OSI Model, a more appropriate term would seem to be TAP/TUN (for the same reason that TCP/IP wasn't named IP/TCP: the slash is treated like the word “over”, just like a division sign in algebra.) TUN/TAP would seem to have been a more appropriate phrasing.

Okay, that likely completes any changes that are needed for the virtual machine startup script. There is some more work to do.

Discussion on example files

(This section does not describe any steps to perform, but is just a mini-discussion about available information.)

The following information will refer to some sample script files that are provided by ][CyberPillar]['s Sample Qemu configuration files.

If it possible that sample configuration files may have come with the Qemu package. (That had been true with OpenBSD's package. That probably was referred to in the package's “install-message” and/or extra documentation files that came with the project: seeing those could be done using commands described by using pkg_* : showing installed software.)

However, those files tended to run some commands that set up bridging. That wasn't necessarily a bad idea, but seems more complex than just simple IP address assignment, and so what is needed to be able to communicate to the physical system. (Bridging is still an option, but is simply just one of multiple approaches that can be taken to implement traffic routing, and that can be done after successful implementation of basic network connectivity to the physical machine.)

[#qmutpscr]: NIC Script files

The example command lines shown above reference some files. Those files need to be created:

upif0

For the upif0 file, see Qemu NIC configuration script files for examples of the contents.

ls -l ~/upif0
lynx -dump http://cyberpillar.com/dirsver/1/mainsite/techns/bhndscen/vrtualiz/createvm/vmnetcfg/vnetspim/vnetwalk/vmncfvr1/upif0 > ~/upif0
mkdir ${VMDirBas}/execbin/${VMGenNam}/${VMLILNAM}/nicscr
mv ~/upif0 ${VMDirBas}/execbin/${VMGenNam}/${VMLILNAM}/nicscr/.
echo ${VISUAL}
${VISUAL} ${VMDirBas}/execbin/${VMGenNam}/${VMLILNAM}/nicscr/upif0

Now, customize the script. (For now, static IP addresses will be used because they are easier to deal with. See: setting current addresses for optional additional coverage on this topic.)

IPv4 address

Locate the line that assigns a value to DESIPv4. Customize that line by placing in the DESIPv4 address.

This will be the address that gets assigned to the tunnel interface on the physical computer. It will need to be in the same subnet as the IPv4 address that will be used by the virtual machine's NIC. For example:

DESIPv4=192.0.2.1

Of course, an address from the actual 192.0.2/24 block should not be used. (Review the documentation to see what IPv4 address was decided upon in an earlier step.) Customize as appropriate.

IPv4 prefix length

Find the line that says:

ifconfig $PASSEDVAR ${DESIPv4} netmask 255.255.255.0

Those who do not yet understand subnetting may wish to just leave that alone. Those who do may wish to customize as desired. For example, the following may minimize the subnet size.

ifconfig $PASSEDVAR ${DESIPv4} netmask 255.255.255.252
IPv6 addressing

Add a line that says something like:

ifconfig $PASSEDVAR inet6 2001:db8::1 prefixlen 64

Of course, an address from the 2001:db8::/32 prefix should not be used. Customize as appropriate.

Tunnel interface
Customizing the name

Customize the name of the TUN/TAP device, if needed. The sample script file contains a line that says:

DEFAULTNIC=tun0

Have the interface name match what is used on the “ -net tun,” command line parameter (right after the “,ifname=”) in the virtual machine startup script file. (This may mean reviewing the contents of a text file that is different than the NIC startup script file.) In some cases, no customization will be needed.

Potential Step: Stop trying to unnecessarily enable support

The sample script file may be a line that says:

ifconfig ${PASSEDVAR} link0

That is a great thing to do, in OpenBSD. However, it might be an OpenBSD-ism that is not quite as appropriate to do in other operating systems. Users of other operating systems should check documentation to determine whether this is desirable. (If in doubt, it may be safer to start experimenting by trying to have that line be commented out.) Using OpenBSD documentation as references: despite the fact that “link0” is discussed by OpenBSD's manual page for ifconfig, and that there seems to be some specific functionality documented by that manual page, what the parameter really does seem to do is to pass some information onto the network card driver. In the case of a tunnel device, OpenBSD's manual page for network tunnel pseudo-devices states, “To enable layer 2 tunneling mode, where the tun interface simulates an Ethernet network interface, the link0 flag needs to be set with ifconfig”. Users of other operating systems may want to check corresponding documentation for similar descriptions.

Description

Optional, but good to do: add a line to the text file (right by where the IP addresses and prefix sizes) that specifies the name of the virtual machine:

ifconfig $PASSEDVAR description "nameOfVirtualMachine"
dnif0

The sample dnif0 file is probably really overly complicated for what is needed for this first example machine. This is true if using the simplicity of static IP addresses. However, a dnif0 file is required.

${VISUAL} ${VMDirBas}/execbin/${VMGenNam}/${VMLILNAM}/nicscr/dnif0

Have the contents say:

#!/bin/sh

Now that the script files have been made, make sure they are usable:

chmod ugo+x ${VMDirBas}/execbin/${VMGenNam}/${VMLILNAM}/nicscr/??if0

In the documentation for the physical machine, make a note that tun0 is now reserved for this machine. No other machines should use tun0. (If there is a desire to repeat this sort of process for another machine, then a different tun interface will need to be reserved for that machine. Also, an adjustment will need to be made to the line in the text files that sets the value for DEFAULTNIC.)

At this time, it is important to clarify that the commands in the script files on the virtual machines are meant for the tunnel interfaces. The tunnel interfaces and the virtual machine must have different IP addresses within the same subnet. For example, if the tunnel device used 2001:db8::5 and 192.0.2.5 then the virtual machine's NIC might use 2001:db8::6 and 192.0.2.6.

Shut down the virtual machine software. This doesn't involve simply rebooting: the entire virtual machine software needs to be shut down in order to have the new command line parameters take effect.

sudo halt -p

Then restart the virtual machine software. (At this point, there should be a clearly documented command on how to start the virtual machine software. If not, review the section about running the virtual machine. Then be sure to document the command that starts up the virtual machine.) The virtual machine software should run the network configuration scripts. When the virtual machine is running again, run ifconfig on the physical machine to make sure that the network configuration scripts had the desired impacts.

[#qmunetst]: Testing the network

After logging into the virtual machine, check its IP configuration (which may have been reset when the machine was restarted). If needed, see: available NICs, manual network addressing: getting the address(es) used by a NIC, manual network addressing: manually setting the address(es) used by a NIC. e.g.:

Required customizations

Like other cases, the network device name may need to be adjusted. (This example shows a device name of em0 which may or may not work on an actual Qemu machine. As needed, review available NICs and customize the device name as needed.)

The IP shown should not be used, because they are reserved for samples/examples/instructions. Customize those addresses and prefix lengths according to the documentation for this virtual machine.

ifconfig
sudo ifconfig em0 192.0.2.2 netmask 255.255.255.0
sudo ifconfig em0 inet6 2001:db8::2 prefixlen 64
sudo ifconfig em0 up
[ "${PAGER}X" = "X" ] && export PAGER=less
echo ${PAGER}
ifconfig -A | ${PAGER}

Initially, try to use ICMP to get communication going between the virtual machine's first NIC and the TAP interface on the physical machine. Both machines should be able to initiate communication. Once that works, see if routing is sufficiently working to allow communication to other NICs on other machines. To check ICMPv6, use ping6. For example:

ping6 -c 3 2001:db8::1

If transmission failed, check some usual problems:

Troubleshooting ideas
IP address assignments

Make sure that each machine has the needed IP address.

Simplifying testing

Make sure that the computer to be reached is the neighboring computer, and not a remote system on a different subnet.

Firewalling
Try temporarily disabling any firewalls if that is safe to do. Re-enable the firewalls after figuring out whether or not that seemed to help.
Isolating the layer

After trying a ping, if “arp -a” does not show the neighboring MAC address then IPv4 (including ICMP/IPv4, and so ping) stands no chance to work and the issue is likely Layer 2, not Layer 3. ping won't succeed if the Layer 2 traffic is not working (which generally means that ping won't succeed won't succeed until “arp -a” is capable of reporting the desired results).

Likewise, after trying a ping6, if “ndp -a” does not show the neighboring MAC address then IPv6 (including ICMPv6/IPv6, and so ping6) stands no chance to work and the issue is likely Layer 2, not Layer 3. ping6 won't succeed if the Layer 2 traffic is not working (which generally means that ping won't succeed won't succeed until “arp -a” is capable of reporting the desired results).

One thing that can cause layer 2 issues is misconfiguration of the virtual machine hardware. (In that case, nothing done from the virtual machine's operating system will likely do any good.) Check the command line parameters, including making sure that any vlan numbers for the same NIC match. (If there is no reference to vlan then a default vlan number may be getting used.) Try having this virtual machine be on a unique vlan from any other virtual machines. Check if the MAC address is showing up in the “arp -a” or “ndp -a” tables for a different IP address. If so, the issue is likely a duplicate MAC address, which probably happens more often with some virtual machine software than when using physical machines.

Handling duplicate packets

If ping says “(DUP!)”, then networking might seem to function somewhat, but really networking is broken quite seriously. Take whatever steps are needed to shut down the virtual machine software. Upon restarting the virtual machine, the issue might not re-occur. If the probelm does re-occur, then there may be a need to shut down shut down all virtual machines that are using the same IP address TCP port number on the “-nic socket,” command line parameters. (Restarting just the machine that uses the “listen” parameter might fix the issue of duplicate packets, but then communication is broken for all the virtual machines using the “connect” parameter to talk to the virtual machine that used the “listen” parameter. So, shutting down all such virtual machines is the recommended fix. Then, of course, restart any that are desired.)

The precise cause of this hasn't been determined by this author (at the time of this writing). Shutting down a virtual machine, and then starting up the virtual machine, seems to make the issue more prone to occur. Perhaps that is the reason that this issue seems to happen a lot more frequently during initial machine setup.

Compared to other types of network connections, this issue might be more prone to happen when using “-nic socket,” with “connect” (and the corresponding “listen”).

The next step

The next step is really not specific to Qemu TAP, so this section is complete. However, very often the next step is to handle some traffic routing, so a quick pointer is being made: walkthrough for making a virtual machine: adding IP routes.