[#vnetspim]:

Specific implementations of virtual machine networking

[#qmnetimp]: Qemu's networking implementation(s)

Note: This documentation is rather full/thorough. For a more focused guide which can probably be completed faster, see setting up networking in Qemu. That section also has some example script files that will help to configure the virtual NICs after they are created, so that the virtual NICs can be effectively useful.

In contrast, this section just tries to document many more details about Qemu's behavior. These details in this section may help for more customized setups or when trying to troubleshoot.

Default MAC address(es)

Qemu uses 52:54:00:12:34:56 and subsequent NICs increase the MAC address from that (although it can be overridden): That is not listed in the IEEE file but the OUI-24 does have the locally administered address bit set to 1 (incidicating it is locally administered). (The following might be based on memory or even speculation, and so may need to be verified: If the first NIC's MAC address is manually specified, the second NIC's address will start with 52:54:00:12:34:5?, ending with a 56 since that would be the first attempted address that isn't already been used. A subsequent NIC would try to use 52:54:00:12:34:57 unless the same Qemu instance had that address specified for a different NIC.)

[#qmnetdoc]: Some available documentation about networking in Qemu

Multiple resources about configuring networking with Qemu have been created. Although this guide may be sufficient, if further information is desired, some more documentation may be useful. Examples of such additional documentation include the Qemu documentation (see the “Network options” in the Invocation section and the Qemu Documentation: Section (3.7) about Network Emulation section), and the Qemu Networking page. If things don't seem to work as expected, there is a troubleshooting section on this page, but keep in mind that the Qemu software may have been updated since this guide was written. Therefore, checking those resources may help if things don't seem to work as expected.

Overview / extended info / Recommendations / Brief overview of some other options

The topic of Qemu VLANs is worth understanding. (Specifically, that topic is worth understanding before using any of the command line options that may reference Qemu VLANs. At least historically, the most common command line options about networking did support Qemu VLANs. So, unless already very familiar with Qemu VLANs, the recommendation is to read the entire section about Qemu VLANs.)

For every NIC which is a part of a Qemu virtual machine, a “-net nic” command line option will be needed. Naturally, for the network interface cards on a virtual machine to be very useful, it will probably be nice if they can communicate with other network cards (which may be virtual network cards or even real, physical network cards). For this to happen, another “-net ” command line parameter, which is not “-net nic”, will also be needed, in addition to having a “-net nic” command line parameter.

Qemu supports a variety of different networking modes to accomplish the task of communicating with network ports on other (virtual or real) devices.

-net tap” is recommended

It is anticipated that “ -net tap” will be desirable sometime. Therefore, if one is looking for a solution that will be usable when interacting with physical machines, and doesn't want to be re-doing some work, and doesn't want to deal with some avoidable limitations/problems, the recommended method will be to interact with a TUN/TAP device.

Qemu's User Documentation: section on Network emulation says, in the subsection called “Using TAP network interfaces”, that using TAP “is the standard way to connect QEMU to a real network.” That's quite a statement considering that User Mode Networking is the default, and that using TAP does require running the virtual machine as a superuser (“root”). It would seem that for such a method, with these higher requirements, to be considered “standard”, it must offer some advantages.

Having every virtual machine use TUN/TAP on every machine may not be required (or even possible). However, successfully implementing communications with a TUN/TAP can have advantages over using alternatives (specifically “ -net user” and “ -net socket”). It will likely be worthwhile to get at least one virtual machine to be successfully communicating with a TUN/TAP device.

Some more details (background knowledge) about this generalized method of networking are available in the section on using a standard-looking virtual NIC on the host machine to communicate with the virtual machine's NIC. The basic method to implement this will be to start using “ -net tap” on the command line. (If there is another method, like “ -net user”, then using the “ -net tap” method may be able to be done instead of using “ -net user”.) To implement this in Qemu, understand Qemu VLANs, and then read the section about using Qemu's “ -net tap” command line parameters.

Hopefully the command line that will be used is stored (in a script file; or a GUI front-ends configuration; or as a lousier minimum, in some sort of command line history). So, to make changes, simply plan to modify that stored command line so that future occurrences (of Qemu running the virtual machine) will use the newer, more preferred command line. If the command line is in a script file, this basically means modifying the script file. (Instructions for editing a text file are available.)

Having endorsed the option of Using Qemu's “ -net tap” command line parameters, such endorsements may lead some people to wonder why there are other options. So, here is a brief overview that describes other options.

In addition to using the “ -net tap” option, Qemu 0.9.1 supported two other types of methods to communicate to other computers: “ -net user” and “ -net socket”.

The “ -net socket” options are only useful for communicating with other Qemu machines. There may be some performance penalty for using those options. Also, if every virtual machine was using the Qemu software and was using only those options, there would be no way to communicate with any physical machines (such as the host machine, and/or machines on the Internet). That may be okay, depending on what a person is trying to accomplish. However, for many virtual machines with networking support, it is not okay to have all virtual machines be unable to communicate with any physical machines, because plans expect that the virtual machine(s) will access one or more physical machines.

If some machines are using “ -net tap” to communicate with physical hardware, “ -net socket” may be able to be used to communicate with other virtual machines. By using protocols that support routing (including IPv6 and IPv4), some machines may have Internet access even the only way to communicate with physical equipment is to do so rather indirectly. Some advantages to using “ -net socket” instead of “ -net tap” is that “ -net socket” won't require using one of the operating system's (possibly limited number of) TUN/TAP devices, and “ -net socket” may (is this true or not?) not require superuser access.

The “-net user” syntax may be simpler and may work for most TCP and UDP communications that are initiated from the machine (with ICMP being a very notable exception, so don't expect pings to work to reach systems outside of Qemu's internal firewall).

Overview: Understanding the role of the Qemu command line options related to NICs
A new command line syntax, and an older command line syntax

At the time of this writing, with Qemu 0.14.1 being released, it appears there are multiple forms of command line parameters.

The original command line parameters, as seen in older versions of Qemu, used -net command line parameters which supported “Qemu VLANs” that each acted similar to an emulated hub. Efforts have been made to implement changes. Qemu's page on the “Google Summer of Code 2010” project “QEMU Machine Protocol”: section from July 14 states an intent to “get rid of vlans”, following up on some removal work a couple days earlier and something achieved (temporarily) earlier as noted by Qemu's page on the “Google Summer of Code 2010” project “QEMU Machine Protocol”: section from June 21. The Qemu Wiki: Network Documentation: section on “creating a network backend” refers to the emulated hub being created by “The obsolete -net syntax”.

However, as of Qemu 0.14, the documentation of the newer “-netdev syntax (introduced in QEMU 0.12.0)”, which is called “the preferred way” (by Qemu Wiki: Network Documentation: section on “creating a network backend”), still is not heavily described by Qemu User Documentation: Network emulation section.

Therefore, relying on official documentation as much as possible, what follows is some documentation about the older syntax which, while perhaps considered obsolete, seems to still be supported at the time of this writing. (It has been believed/guessed that the newer syntax may be rather similar to the older syntax, enough that some manual conversion won't be too difficult to performed.)

[#qemuvlan]: Qemu VLANs

Qemu's VLANs, and/or the documentation about Qemu VLANs, have been known to be a substantial source of confusion. This documentation should help clear that up.

[#qmnvlnoq]: What Qemu VLANs are not: 802.1Q

Qemu Wiki's Networking Guide; Section on QEMU VLANs states that a QEMU VLAN “is not an 802.1q VLAN.” IBM documentation on Qemu networking options says, “A QEMU VLAN is not an 802.1q VLAN.”

This is mentioned because there is quite a bit of networking equipment that support a standard form of VLAN tagging specified by the IEEE standard 802.1Q. Many people may interpret references to Qemu's VLAN as thinking they are compatible with some other VLANs, such as VLANs used by equipment that is using/implementing the 802.1Q standard. With apologies for wasting time of people who haven't heard of 802.1Q VLANs, this warning is given to save a lot of time and/or confusion and/or misconceptions for people who are familiar with that standard.

To put this very simply: Unless a future version of Qemu changes VLAN implementation, do NOT expect that Qemu's VLANs will be compatible with other VLAN implementations (such as a standard 802.1Q implementation).

Therefore, trying to send traffic to be interpreted by a hardware switch (if the switch is implementing the most common VLAN implementation, 802.1Q VLANs) is not likely to work like standard 802.1Q VLAN-tagged traffic works.

[#qmvlnhub]: Qemu VLANs: Like a hub

(For those who aren't as well trained/experienced in computer networking, a hardware “bridge” or “switch” or “hub” is basically like a splitter: they are devices desgined to allow multiple networking devices to be plugged into. Then, usually all the plugged-in devices can communicate with each other (although elaborate advanced customizations can affect that).)

The Qemu User Documentation: Network emulation section of documentation has a subsection called “VLANs” which states, “A VLAN can be symbolised as a virtual connection between several network devices.” A very effective way to think about a Qemu VLAN is to think of it as a switch/bridge/splitter. Qemu Wiki: Network Documentation: section on “creating a network backend” specifically has described this as being “an emulated hub” ... “that forwards traffic from any device connected to it” to all other devices connected to the same emulated hub.

Such a switch/bridge/hub can connect to various devices, which are specified with various different “-net ” command line parameters. Command line parameters will affect what is connected to computer code that acts like a bridge/switch/hub. In at least some simple cases, each individual “-net ” command line parameter may be connecting one device to the VLAN. If trying to visualizing this, think of the command line parameter being a set of directions to plug a single device into the switch. (An individual command line parameter can provide a set of directions that specifies the details for just one device to get plugged into the switch. you may need to have multiple sets of directions that each individually specify just one of the devices that get plugged into the switch. So, for multiple devices to be plugged into the same switch, multiple command line parameters would be used. Multiple devices will typically/always get handled by multiple command line parameters.)

The Qemu command line parameters may allow a VLAN number to be specified. For instance, one set of command line options may use VLAN 0 (the default), while another may specifiy to use VLAN 1 and another may specify VLAN 2. In this case, the distictly separate VLANs are like different switches which are generally not plugged into each other. Generally, this means that the devices plugged into one switch do not directly communicate with devices plugged into another switch.

(Like real switches, it would be possible to have both of those switches connected to the same device. That other device could bridge the traffic, which would be similar to plugging those devices into each other. However, the fact that communication works is because of something that happens beyond the virtual machine software.)

When there are multiple “-net ” command line parameters that connect to the same VLAN, that is like multiple devices that plug into the same switch. It probably is desired to have multiple devices connect to the same switch. By having a NIC on a virtual machine be connected to a switch, and then having a virtual NIC on a host machine be connected to the same switch, these virtual devices can then communicate. This allows the host machine to communicate with the NIC on the virtual machine.

Essentially, the computer code that acts like the bridge/switch/hub can communicate with various types of devices, and that is what allows various network connectivity methods, such as different types of devices, to (possibly indirectly) communicate with each other.

Qemu Wiki: Network Documentation: section on “creating a network backend” says “When creating multiple network devices using the -net syntax, you generally want to specify different vlan ids.” The reason for wanting different identifiers is very much exactly the same reason why, in many cases, a computer with multiple network cards generally doesn't have both network cards being plugged into the same switch. (The basic reasons are: plugging both devices to a single switch may create a switching loop which is probably not what is intended to be created, and sometimes reported problems may occur with local traffic routing when an outgoing packet has a destination of a local NIC that isn't a loopback interface.)

So, as an example: “ -net nic -net tap,ifname=nicup -net socket,vlan=1,listen ” would create a NIC on the virtual machine which is on the same default VLAN, VLAN 0, as the connection to a TAP/TUN interface. It also creates a socket network connection that other machines may use. Such traffic is shared by all machines that are using the socket (because that is how such socket connections work), but the traffic isn't sent to any other devices plugged into VLAN1 because the command line parameters did not specify anything else is connected to VLAN1.

The following is believed to be true: VLANs on different Qemu machines generally do not need to have the same VLAN number. This is mainly because the devices don't inter-communicate. (Qemu VLANs are not 802.1Q VLANs.) Basically, a VLAN number is typically like a number of a network switch that is dedicated to supporting just one machine. VLAN 0 would be the first switch for a virtual machine, and VLAN 1 would be the second switch for the same virtual machine. A different virtual machine may have a VLAN 1. this would be like a switch dedicated to that machine, and it has nothing whatsoever to do with any switches for other machines. (-net socket might be an exception, as that type of VLAN is specifically designed to involve multiple instances (/copies) of the virtual machine software.)

Attention needed: The following note was made, perhaps while learning how this worked. It may or may not be accurate: testing may be in order:
If a VLAN other than zero is specified, the NIC will only communiate to the switch using that VLAN (or perhaps traffic involving VLAN 0 as well?).

[#qemnonet]: Using the “-net user” command line parameters Overview of the command line options: The “ -net none” command line parameter
This command line parameter specifies that the virtual machine should not have a NIC. Not even the one user-mode NIC, that is usually created by default, will be used for this virtual machine instance.
[#qmnetnic]: The “ -net nic” command line parameter

The “ -net nic” command line parameter creates a NIC on the virtual machine.

Connectivity overview

This NIC communicates with some computer code that is part of the virtual machine software. This computer code allows the NIC to communicate to any devices, as long as each device is connected to the same VLAN. It is similar/identical in concept to a computer that communicates to a hub/splitter which allows the computer's NIC to communicate to multiple devices, as long as each device is plugged into the same physical switch/bridge/hub/splitter.

To simplify things, it may be easiest to think of some of the computer networking code as a virtual bridge/switch/hub that is used to connect to (virtual and/or real) networking devices. The “-net nic” command line parameter simply creates a device on the virtual machine. That device acts like a standard network interface card (NIC). Then that NIC is virtually connected to the computer code which is acting like a bridge/switch/hub.

Naturally, this computer code, which is being symbolized by a virtual bridge/switch/hub, should communicate with something else, in addition to the virtual NIC created by this “ -net nic” command line parameter. Until it does, this computer code would be as incredibly pointless as a physical switch/bridge/hub/splitter that has just one thing plugged into it. Communication with something else will be needed to really start being useful. Specifically, this computer code, which may be treated similar to a virtual bridge/switch/hub (at least for small setups), should communicate with something else that is specified by another “ -net ” parameter. (The other “ -net ” parameter should not be a “ -net none” parameter, and probably should be something other than another “ -net nic” parameter.

The other “ -net” commands involve connecting other things, specified by the details of the command line parameter, to the same computer code (or virtual bridge/switch if that is easier to imagine) that is already connected to the virtual machine's NIC that this “ -net nic” command line parameter set up.

Specifying a unique MAC address

The “-net nic” command line parameter supports several options. Particularly when multiple virtual machines may be used, the most important of these options may be ,macaddr=. The default value for each virtual machine is like using a syntax of “ -net nic,macaddr=52:54:00:12:34:56 ” for the first network card on a virtual machine, and if that default value is used, then the next video card will default to using “ -net nic,macaddr=52:54:00:12:34:57 ”. Multiple (virtual) machines can have difficulties communicating when they have the same address.

An option may be to take a group of bits, such as eight-bit octet bytes, or four-bit nibbles, and to assign the second-last group of bits a unique number based on what machine is being used, and the last group of bits may be assigned a unique number for each NIC. That method should work to provide a unique MAC address for each machine as long as there are fewer than 16 NICS in each machine, and fewer than 16 machines (if nibbles are being used).

Matching a “ -net nic” command line parameter to another type of “ -net ” command line parameter (other than “ -net none”)

Understanding the use of Qemu VLANs, particularly how they act similar to a hub, will likely be the best approach. Simply connect all the devices that should communicate directly to each other to the same VLAN.

The examples in Qemu documentation tend to show command line parameters used in an order like: -net nic -net other -net nic -net other. There may be an implication that the second -net other somehow corresponds to the second -net nic. However, what may be more accurate is simply that whatever is described by one command line parameters will likely be able to communicate with whatever is described by another command line parameter if they use the same VLAN. (Of course, logical network settings like different IP addresses could cause such communication to fail, but the command line parameters are like wiring, and having things on the same VLAN is like saying that that connections are wired sufficiently, to be able to communicate with each other without having problems due to the wiring.) It doesn't matter whether the connection is a virtual machine's NIC created with -net nic or another type of connection such as a TUN/TAP device created with -net tap. If both of those command line parameters use the same VLAN, then they should be able to communicate (on a virtual “layer two” level).

[#qmnetusr]: Using the “-net user” command line parameters

A portion of the computer code that handles networking in Qemu is code that is capable of sending and receiving some TCP/IPv4 and UDP/IPv4 packets on the host machine. This shouldn't be surprising: Many network-aware programs on the host machine can send and receive TCP/IPv4 and UDP/IPv4 packets. More details about how the virtual machine software interacts with the host machine's TCP/IPv4 and UDP/IPv4 packets may be availabe in the documentation about having the host machine treat the virtual machine like a standard networking application.

The “ -net user” command line parameter specifies that Qemu should allow this portion of networking code to interact with one of the VLANs. When this happens, other devices connected to the same VLAN can communicate with the portion of the Qemu networking code which is able to send and receive supported TCP/IPv4 and UDP/IPv4 packets of the host machine.

This requires some of the least pre-configuration, and may be a good tool to test that things work. However, there are some disadvantages and this may be undesirable in the long term because of that. One advantage, though, is that the qemu program can be run as user mode. (Specifically, this is meaning that Qemu does not need to be using the “root” account.)

The Qemu program will run a DHCP server that the virtual machine may use to configure the relevant “-net nic” (or perhaps multiple “-net nic”s?). There is a sort of firewall implemented, which prohibits new incoming connections unless support for such connections is specified by command line parameters. As described by the section on “having the host machine treat the virtual machine like a standard networking application”, ICMP may not fully work with outside hosts. However, the local DHCP server on the inside of this virtual firewall does allow ping traffic. It does seem that server, which by default is at 10.0.2.2, will (by default) respond to at least some addresses from within the range of 10.0.2.15 through 16 addresses higher (up through 10.0.0.31). (This can be used to help determine functionality of some local networking software, although it might not help determine connectivity for machines outside the firewall.)

Qemu supports incoming traffic by using corresponding command line parameters. Older versions of Qemu used a completely separate -redir while newer versions may use a ,hostfwd portion to the -net user parameter. (See Qemu User Documentation: section on “Network emulation” and/or Qemu User Documentation: Invocation section for details on the syntax.)

Note this method may be pretty slow (compared to using different types of -net command line parameters).

[#qemnetap]: Using the “-net tap” command line parameters

This command line option enables some computer networking code, which is part of Qemu, to communicate with a TUN/TAP device running on the hsot machine. This implements Using a virtual NIC on the host machine to communicate with the virtual machine's NIC.

This method of networking provides a very, very useful feature: It is with this method of networking that a virtual machine can communicate to a physical NIC. This also has relatively high requirements.

A TUN/TAP interface in the host operating system must be set up, and be available. If needed, see the documentation on see section on having/creating a TUN/TAP device. This may take some effort, especially if the operating system imposes some incompatibilities or limitations, and the operating system may have a limited number of TUN interfaces. Such a limit may not be easily exceeded, so if there is a limit of four TUN devices on the host machine's operating system, that limitation may not allow five virtual machines to each have their own TUN device.

Another requirement for using a TUN device is that the entire Qemu instance may need to run with full administrator rights (running as root), just because this method of networking is being used.

As a quick summary/example, if a command line parameter of “ -net user,vlan=0 ” had been used, try replacing it with a reference to a TUN/TAP device. (Remove the command line parameters that are about using “ -net user”, but keep the “ -net nic command line parameter.)

For example, one may be able to use the following command line parameters (if Qemu is a new enough version to support the ,downscript= syntax):

-net tap,vlan=0,ifname=tap2,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown

Make sure the TUN/TAP device being referenced is correct. For example (using OpenBSD as an example), the section on having/creating a TUN/TAP device may have details like specifying to use a hardware name of tun* instead of tap*.

If the intent is to communicate with a specific NIC, be sure to customize those command line parameters, as needed, to make sure that the correct Qemu VLAN is used. (Namely, the related command line parameters should have matching “vlan=” portions to the command line parameters.)

There may be one (or possibly more than one) script file used. The best approach may vary, but for many who are making the first virtual machine, there may be some benefit to using custom, and rather blank, script files instead of using the default script files. The default script files may execute commands to help set up some processing of traffic (and, more specifically, bridging NICs on the host machine). Instead, using some rather blank script files may be better to learn more about the whole process. For example, instead of referencing the default script filenames as shown in the above example, the following may be preferable. (In this case, the ,ifname= is using an OpenBSD system as an example.)

-net tap,vlan=1,ifname=tun3,script=./nicscr/niconeup,downscript=./nicscr/niconedn

The script files probably should not be completely blank: They may run best if the text file starts with “#! /bin/sh”. If any commands are known to be needed, they may refer to the device by using “$1”. For instance, the following may be a great example script:

#! /bin/sh
echo Now setting up or taking down $1

Make sure that ownerships and/or permissions are set as needed. For example, if all such script files fit the filespec of ./nicscr/* then perhaps run:

chmod u+x ./nicscr/*

Make sure that each script file being referenced when configuration a network connection (whether the script is being used with a “,script=” parameter or a “,downscript=” parameter) has a file successfully existing at the location specified by the command line, and has proper permissions set.

Make sure that when the virtual machine is run, it will have needed privileges. Specifically, this may mean that the virtual machine needs to run as a superuser (the “root” account). For many Unix systems, a way to do this is to locate the beginning of the command line that runs Qemu, and prepend the word “sudo ”.

[#qmnetsck]: Using the “-net socket” command line parameters

This implements having virtual machines communicate directly with each other. There are two possible methods to implement this: using a client/server setup, or using multicast. If using a client/server setup, this instance of Qemu needs to know whether it should be listening (as the server) or connecting (as a client).

This may be a fairly quick way to set up a configuration that involves having multiple virtual machines communicate directly with each other.

Note that if one machine is listening, other machines will not try to re-connect to each other, nor to the same initial address, when that machine goes down. This means if the main machine, which is listening, goes down, then all other machines will completely lose network connectivity until they are restarted (after the main listening machine is restarted).

-net socket,listen

e.g. “ -net socket,vlan=1,listen=127.0.0.1:1234

Qemu listens to traffic on the specified TCP port which comes after the optional host, and the required colon and the required TCP port number (e.g. 1234). This is meant to then be available for other Qemu machines that use -net socket,connect to connect to the same IP address and TCP port number.

[#qmskrqln]: Warning: These sort of client/server socket connections need a stable listener

Note that this machine should be created before the other Qemu machines and should remain active until the other Qemu machines are done running: If this machine stops working (even briefly, if this Qemu process gets restarted) then all other Qemu machines that relied on this socket connection will need to be restarted. (Just re-creating this Qemu process, without stopping all other Qemu programs communicating on this port, may not work and, even worse, may be responsible for some troubles such as packet duplication.)

(It is presumed, though not yet fully tested by the author of this documentation, that this problem might be able to be effectively avoided by using multicast instead of using this sort of client/server model.)

-net socket,connect

e.g. -net socket,vlan=2,connect=127.0.0.1:1234

This Qemu machine makes a connection to a socket at the specified IP address and TCP port number. Once that connection is made, the virtual machine can easily communicate with other network devices that are connected to the same socket. That may include a NIC on the virtual machine of the Qemu program that is using the -net socket,listen parameter, and also may include other Qemu machines that did and/or will also use the -net socket,connect to connect to the same socket.

-net socket,mcast

(This section is largely a placeholder, and is not filled with a lot of tested material. This is likely similar to -net socket,listen and -net socket,connect without the requirement that one machine be started the entire time that the other machines are being used. Be familiar with multicast (and whether the firewall traffic allows multicast traffic, which might be potentially unwanted, in and/or out of the firewall). Some more details may be available by reviewing additional documentation on Qemu networking.

Other options
Newer versions of Qemu have added additional options, such as -net dump and, if VDE support is compiled in, -net vde. However, these are not meant to be the simplest approaches to get basic networking functional. For further details on these options (and possibly others?), see the Qemu Wiki's Networking Guide, Qemu Documentation's section on Network emulation, and the command line parameters documented in the “Network options” section of the Qemu Documentation: section on “Invocation” (details about command line parameters).
More/other info
Troubleshooting the NICs on the virtual machine

These are tips related to Qemu. Of course, the problems could be issues with the guest environment. Troubleshooting such issues is not described in this section, even though they could be the source of some problems.

Checking the command line
If using an elaborate command line in a script file, one option may be to put everything on one very long command line. Another option for a Unix script file may be to split up the command line, by including the escape character before a “new line” sequence (which is actually typically just one character, 0x0a). If this is done, make sure that every portion of the command line, except the last line, does have an escape character before the line's “new line” sequence. Also make sure that some sort of hidden white space (a space character or a tab character) did not accidentally end up being after any of those escape characters. A rather quick way to do that thoroughly might be to prepend the word echo before the rest of the command line.
Troubleshooting when using -net user

Realize that ping and may tracert implementations use ICMP. Also, realize that ICMP and IPv6 and other protocols are not supported by the user-mode NICs which may support only TCP/IPv4 and UDP/IPv4. ( ICMP/IPv4 may not be supported. This is because limitations by operating systems may routinely state that not all software can customize packets to the degree that ICMP-creating software tends to customize packets. Specifically, the virtual machine software may be limited by the operating system. If the operating system does not limit the virtual machine software, the virtual machine software may still be limited by the fact that it is designed to work with other operating systems that do have these enforced limtations.) So commands often used to test connections (using ICMP) may fail, even if connections would work just fine for normal usage involving UDP and TCP traffic.

Also, if using user-mode NICs, realize there is a built in firewall along with some built-in services like a DHCP server. Using a ping command on that DHCP server may work, but trying to ping or tracert beyond that firewall, which is internal to the virtual machine software, may not work so well.

Troubleshooting if using -net tap
The script won't run

e.g., the output may look something like: “could not launch network script”, and may later show other errors.

Note all of the output of the screen; especially any other error messages that may exist. Chances are that the error about having troubles running the script might be some output that was caused by another error, so definitely note what the other error messages are.

Check that the specified script exists. (This involves making sure that the script file exists, as well as checking the command line to ensure that the correct script file was specified). Check that the script file starts with a line that just says “#! /bin/sh”. Check the permissions (e.g. chmod ugo+xr scriptname).

If these things seem right, and if this test would be safe to perform, then see if the NIC configuration script can be run from the command line, using just a reference to the script. (e.g. run “./scriptname”. Note that this is NOT using a period and a space first like “. ./scriptname”). Do this with the same username as the user that will be running the qemu instance. Do this from the same directory that Qemu is running from: Inserting a command to show the current directory (just before running Qemu) may help.

If that doesn't work, try focusing on any other error messages that may be showing up.

Device 'tap' could not be initialized

See if there are any other error messages. For instance, perhaps an earlier line says, “warning: could not open /dev/tun0 (Device busy): no virtual network emulation

In that case, see if anything else may be using the /dev/tun0 device. (Details may be in the section about files that are in use. Although that process may be more challenging when looking for actual files, it may actually be simpler for devices. For instance, just running fstat | grep -i tun0 ” may actually produce some very fast and useful results.) (In one example case, the problem was caused by an earlier copy of dhcpd listening on that interface.) If the device is in use by some software, consider stopping that software, then starting the virtual machine, and then, if desired, starting the software which needed to be manually stopped. If that process fixes things, then consider what caused the situation. For instance, find out if the software was supposed to be stopped when the virtual machine software had quit earlier. Determine if this situation likely to repeat itself when the virtual machine is stopped again.

This is probably caused by the virtual NIC on the host computer getting into some sort of invalid state. The exact cause of this may be unknown: Perhaps the user tried to use some sort of command like “sudo ifconfig tun0 destroy”. Restarting the host computer may help. This might not be as likely to happen as frequently the first time that the virtual machine software is started. Also, if less experimenting with the tunnel interfaces is done, perhaps the issue simply won't re-arise (even if the virtual machine software is quit, and then restarted).

If that doesn't tend to resolve things, consider trying to change the type of virtual NIC to -net user, at least to test to see if that helps qemu to run successfully.

Traffic is not working, even though it did before

Attempt to listen to the traffic. With an OpenBSD guest, it has been seen that running tcpdump on the guest would cause traffic to work while tcpdump was running, and then the traffic stopped working when tcpdump stopped running. It seems this was resolvable by taking these steps:

  • Power off the qemu guest machine.
  • Then (as root, on the host machine), if the interface being used is tun2, run:
    sudo ifconfig tun2 -link0
    sudo ifconfig tun2 down
    sudo ifconfig tun2 destroy
(Other commands were also run, like ifconfig tun2 -link0 down, but they probably didn't help any.) Then run “. /etc/netstart tun2”. (The exact effects of that will vary based on what is in the relevant /etc/hostname.tun? file. A sensible example of what may be in there are any commends needed to create the interface, bring the interface UP, and place the interface in Layer 2 tunneling mode by setting the link0 flag.) There should be no error messages. Then things worked better (after a new Qemu instance started).

Attempting to do this without powering down the Qemu instance did not work.

This would only happen on subsequent executions of the Qemu instance. The first instance of Qemu after each reboot worked fine (which makes some sense, because the tun interface would have been created during the reboot).

tap vs. tun
This may be OpenBSD specific: using some sort of syntax like ifname=tap* may not work. For more information, see having/making a TUN/TAP device.
/dev/tun7 is mentioned in an error message, even though it wasn't specified

Qemu may show that it cannot open /dev/tun7, which is interesting since tun7 wasn't the specified tun interface: This may show up even if the command line explicitly specifies a different interface. This may be more likely to appear if there are multiple NICs on the virtual machine.

This specific error may be OpenBSD specific, as unlike the qemu documentation that says to use ifname=tap#, /usr/local/share/doc/qemu/README.OpenBSD has references to using /dev/tun# (and the command line to use is ifname=tun#). The reason this would be OpenBSD specific doesn't respresent terrible weirdness with OpenBSD: It is simply because OpenBSD's tap functionality is implemented by the driver called tun (with link0 passed to ifconfig).

Make sure the desired TUN interface exists, both in the output from ifconfig and on the file system at /dev/t???? (e.g. /dev/tun* or /dev/tap*, whichever is appropriate for the operating system). If so, attempt to simply re-order the -net options on the command line. This has been known to resolve the issue. Perhaps the best bet is to make sure that between each “-net nic” is another type of “ -net” command. Something else that may help is to mention all the TAP interfaces (made with “ -net tap”) first, before any socket interface (that is made by “ -net socket”).

Traffic is duplicated

e.g., after each packet sent out with the ping command, perhaps two replies are received. Hopefully the ping command will notice and report this error.

This may be caused from the issue related to restarting a machine that uses “ -net socket,listen”. If so, this should be considered to be a bug in the virtual machine software. Fortunately, triggering the bug may be preventable.

Whenever a qemu machine using “ -net socket,listen” is shut down, all machines that use “-net socket,connect” should also be shut down. (This means not just telling the operating system to power off, but to have the virtual machine software quit.) All of those machines should be simultaenously shut down before starting up the machine that uses “ -net socket,listen”.

If this error occurs because that process wasn't followed, try shutting down all of the machines to see if that fixes things. If not, also restart the host computer that runs the virtual machine using “ -net socket,listen”. (If that isn't sufficient, try also restarting all host computers that run any of the virtual machines that connect to the virtual machine that uses “ -net socket,listen”)

The script ran but did not work as expected
Know that the script is run directory from qemu and the environment variables may have different values compared to the shell that ran qemu. (Most likely, the variables will be unset.)
Troubleshooting if using -net socket
Qemu may allow a telnet connection to the port, although doing so may cause troubles. It is recommended to use traffic viewing software (like tcpdump) to see if there are duplicate packets, particularly if one of the hists is using -net socket,listen. If so, one may get by simply by shutting down all of the virtual machines and then starting the virtual machine with -net socket,listen before the others.
Handling firewalling from User-mode networking

Note that in many cases, using such command line options (to modify how user-mode networking is handled) is not recommended, because the recommended route is to use a different implementation of networking (instead of using user-mode networking).

If using Qemu's User-mode networking, the virtual machine software will ignore incoming connections unless the virtual machine software is listening onto the specified UDP/TCP port with command line options. To use the command line options, see Qemu documentation: Command line “Invocation” section (and specifically the sub-sections related to user mode networking). Specifically, for newer versions of Qemu, the syntax to use as part of a “-net user” option is to append “,hostfwd=udp:53-:53”.

Older versions of qemu had this implemented with -redir. However, the public qemu-doc file has since stated that this (and some other options) “are still processed and applied to -net user. Mixing them with the new configuration syntax gives undefined results. Their use for new applications is discouraged as they will be removed from future versions.”

Some other documentation that may be relevant:

[#mkhypvnt]: Hyper-V

Further details about setting up Hyper-V are also available: making a Hyper-V configuration. This section is rather specific to networking.

(Information which had been in this section has now been moved to a section that is more specific to Hyper-V. See: Hyper-V virtual hardware: section about networking.)

Other virtual machine software

Further specific guides are not currently available right here. There are some similar sections that may have some similar details: try seeing walkthroughs for virtual machine networking (perhaps less likely to have info, but if info does exist then it will likely be most useful), Virtual machine networking: specific implementations (which might just refer to resources at the previous hyperlink, and this page), and/or perhaps virtual network configuration.