Updating Base Image

Once a child image is created, any changes made to the parent image will cause the child image to become invalid. So, some care should be taken to make sure that the parent image is in good shape.

If there is a problem with the parent image, there are a few possible approaches:

The third approach may be the safest approach, in the long term. However, it may be a more time consuming approach, and may be undesirable in the short term. So, hopefully the image was good enough that this approach doesn't need to be taken. However, if that route is, indeed, going to be taken, then here are some steps to help make that happen rather successfully and easily.

If you think that the parent image was created successfully, and that there is no need to update the image, then that's probably a good thing. There may be no need to perform yet another update, and so there may be no need to follow these steps. However, just remember that these instructions exist, so that you can refer to these instructions if there is a need to update the files.

First, really consider whether all child images are deletable. If there is any doubt at all, then don't adjust the parent image until a copy is made. If there is a copy made, at least that means that a child image can be re-usable.

verifying virualization variable values

echo VMDirBas=${VMDirBas} VMGenNam=${VMGenNam} VMLILNAM=${VMLILNAM}

Note: VMLILNAM should be set to a virtual machine that uses a parent image. Chances might be fairly high that this variable is pointing to another virtual machine, so make any changes are are needed.

There should be no active/running virtual machines that use any child images. The following is one example of checking for virtual machines that are likely related (using a process from seeing what is running).

ps -auxww | grep -i ${VMGenNam} | grep -v grep
Some notes about using ps

Note that although the above syntax will work okay with OpenBSD, ps syntax differences are significant enough that no single syntax is preferred by all the Unix implementations. Users of other operating systems may need to adjust the example shown. (Common modifications may include taking out the hyphen right after the word ps and/or removing some parameters, such as one or both of the w characters.)

Multiple redirections are shown. The final redirection is discussed in the section about “Having grep exclude itself”.

...

Make sure disk is writable

When verifying virualization variable values, VMLILNAM should be pointing at the parent image.

ls -l ${VMDirBas}/diskimgs/baseimgs/${VMGenNam}/${VMLILNAM}/${VMLILNAM}?.qc2
sudo chmod ug+w ${VMDirBas}/diskimgs/baseimgs/${VMGenNam}/${VMLILNAM}/${VMLILNAM}?.qc2
ls -l ${VMDirBas}/diskimgs/baseimgs/${VMGenNam}/${VMLILNAM}/${VMLILNAM}?.qc2

Starting the machine

Start a copy of a virtual machine that is configured to use the “parent” disk image.

${VMDirBas}/execbin/${VMGenNam}/${VMLILNAM}/sysstart/exc_${VMLILNAM}

(More steps are described by: Starting a created virtual machine.)

Some Networking notes

Some notes:

  • The system may have a IPv6 link-local address, even if other addresses don't end up working.
    • If this guide has been getting followed precisely, and if the NIC is not using unpredictability to mask the address (which is common if IPv6 “privacy” extensions are in use), the IPv6 address(es) may even be rather predictable. For the first NIC, try: fe80::5054:ff:fe12:##01/64 where the ## represents the VMNUM value. The 01 at the end of the address is intended to correspond to the NIC number, so simply increase that number if there is a different NIC to try to communicate with.
    • You could also try connecting to an IPv6 mutlicast address. On another system (such as the “physical” system which is running the “virtual machine” hardware) that is on the same network link, identify the “interface identifier” for the NIC that is connected to the same system. (See: available NICs for information about identifiers.) For example, if the identifier on the “physical” system was tun144 then you may try: “ ping6 ff02::1tun144
    • And check for neighbors (using NDP)?
  • The system may be set to use automatic addressing (such as DHCP/IPv4). If that doesn't work, prepare to use the local display (see: use Qemu VNC server).
    • Technically, using the local display is going to be advisable, because the system will not have any pre-recognized SSH keys. Therefore, using the local display would be needed to show the “SSH key” fingerprint data, to be able to verify those keys when making the SSH keys connection. However, if the system is on a network that is fully trusted, then accepting any automatically-generated SSH keys may be considered to be a rather low security risk.
  • Prepare to make adjustments to the networking. Manually setting current address may involve using a command like “ifconfig 192.0.2.2 prefixlen 24”.
    • It is preferable to not be trying to change the long term network configuration.
  • If another system uses a similar IP address, the TUN/TAP device for that other system might use an IP address from the same subnet (and maybe even the same IP address!) as the TUN/TAP device for the system that is being focused on. This can be a problem, even if the other virtual machine is not running. The safest approach is to check all TUN/TAP devices that might be using the same subnet, and remove all IP addresses that aren't currently needed.
Logging in

Note that if you do log in with SSH from a system that had some old SSH keys, a warning may be given about SSH-key mismatch. Some warnings may try to get the user's attention: don't worry too much about something that ought to be expected. (See Dealing with a(n unauthorized) key change.)

Once logged in, make the changes that caused there to be a desire to update the image.

Perform the following wrap-up steps:

Remove symbolic link

If this file was created earlier, then it is pointing to old data. Even worse, that old pointer will be in the way of creating a new pointer. So get rid of the old one.

sudo rm /var/log/aide/aide-check-parent-created.txt
Delete identity files

base identity files

Handle SSH keys

As part of the “base identity files” section, the SSH keys have been deleted. Make sure the deletion is documented, and remove the reference to the old keys from ~/.ssh/known_hosts file on whatever machine has been primarily used as an SSH client into this machine. (Hopefully this “base image” updating is being done rather early in the process, so there aren't many machines that have a reference to the old SSH data.)

Wrap up

Perform the steps described by:

  • Perform some more wrap-up steps
    • When those instructions show a customizable parameter to aidedbup, it may be nice to specify a customized value.
    • See: base image wrap-up

 

Shut down virtual machine
sudo halt -p
  • (Normally, you could also try to use a device that has the relevant private key...

    ssh -i keyfile -l _endsys -p 22 sysname-or-IPaddress

    However, SSH is probably broken since the “key” files were deleted. If an existing SSH connection is already created, it may still be operational, but new SSH connections may be unavailable to be created at this time.)

Setting variable(s)

For some of these directions to work most smoothly, let's set VMLILNAM to match the name of the parent machine. (The VMLILNAM may have been changed to point to a child image. If so, that should be changed, at least temporarily, so that some of the earlier instructions can be re-used rather directly.)

echo VMParNam=${VMParNam}
[ ! "X${VMParNam}" = "X" ] && export VMLILNAM=${VMParNam}
Compressing and re-childing image

This might be a little bit different than the first time that the image was compressed. These instructions are also designed for the image of the first hard drive. An image of the second hard drive may need to have the files renamed.

echo VMDirBas=${VMDirBas} VMGenNam=${VMGenNam} VMLILNAM=${VMLILNAM}

verifying variables related to virtual machines

Verify what filenames are expected to be used by the virtual machine.

grep -i " -hd" ${VMDirBas}/execbin/${VMGenNam}/${VMLILNAM}/sysstart/exc_${VMLILNAM}

View the files in that location

ls -Fltr ${VMDirBas}/diskimgs/baseimgs/${VMGenNam}/${VMLILNAM}/
Compressing first disk

Compress the first disk image. That may be done by:

  • First, setting this variable...

    export VMCrDsNm=1
  • Then, follow these steps:

    • Preparation: old method

      set these variables...

      export VMDskDir=${VMDirBas}/diskimgs/baseimgs/${VMGenNam}/${VMLILNAM}/.
      export VMDskBeg=${VMDskDir}/${VMLILNAM}${VMCrDsNm}.qc2
      export VMDskEnd=${VMDskDir}/${VMLILNAM}${VMCrDsNm}-parent-compressed.qc2
      Repeaters

      (If you are repeating these instructions after creating a child image, perhaps because you decided to make some further adjustment to the image, then the ${VMDskDir}/${VMLILNAM}${VMCrDsNm}-parent-compressed.qc2 filename might be actively used as a parent image. In that case, set VMDskEnd to a different value, so that it points to a unique filename. Further adjustments can be made after the compression is completed.)

      [ -f ${VMDskEnd} ] && export VMDskEnd=${VMDskEnd}-updated.qc2

      Yeah, that might make a rather ugly filename like “sysname${VMCrDsNm}-compressed.qc2-updated.qc2”. But a good time to rename the file will be during the childing process, so that ugliness will presumably be quite temporary.

      Preparation: new method

      Actually, no, let's do this instead:

      unset VMDskDir VMDskBeg VMDskEnd
      export VMDskSml=${VMLILNAM}${VMCrDsNm}-updated-parent-compressed.qc2
    • ... and then, follow the steps at: virtual disk small

    • Follow-up: new method
      sudo mkdir ${VMDskDir}/old/
      sudo mkdir ${VMDskDir}/old/oldmom/
      qemu-img info --backing-chain ${VMDskBeg}| grep ^"backing file: "
      sudo echo ${VMDskBeg}
      sudo mv -i ${VMDskBeg} ${VMDskDir}/old/oldmom/

      Also, if that image was a base image, move the old parent. (The name of the old parent was reported with the “ qemu-img info --backing-chain... ” command.) Following shows an example:

      sudo mv -i ${VMDskDir}/${VMLILNAM}${VMCrDsNm}-parent-compressed.qc2 ${VMDskDir}/old/oldmom/.
      sudo ls -l ${VMDskDir}/old/oldmom/.
      ls -l ${VMDskDir}/old/oldmom/.
      ls -l ${VMDskDir}/.

      If you've done this multiple times, there may already be old/oldmom/ directory with a previously-saved file. In that case, don't overwrite it. Instead, adjust these instructions to point to a different directory (such as old/oldmom2/).

      NEXT, we rename the file so that other directions won't be affected by the update of this file. (It would also be possible to keep the altered filename, by modifying other directions. For simplicity, a more common filename will be used.)

      sudo mv -i ${VMDskEnd} ${VMDskDir}/${VMLILNAM}${VMCrDsNm}-parent-compressed.qc2

      ...

      export VMDskEnd=${VMDskDir}/${VMLILNAM}${VMCrDsNm}-parent-compressed.qc2
      sudo chmod a-w ${VMDskEnd}
      sudo ls -l ${VMDskDir}/old/oldmom/.
      ls -l ${VMDskDir}/old/oldmom/.
      ls -l ${VMDskDir}/.
      Follow-up: old method
      Repeaters Part Two

      If these directions are being repeated, you might now have some extra files. Cleaning them up after compression, and before childing, makes some good sense. Directly following these instructions may involve overwriting a file (and, therefore, losing a copy of that file). What is probably true is that the older file is really old, and a newer file has been verified to work, and so that is acceptable. If not, move/rename the older file so that it doesn't get overwritten.

      ls -l ${VMDskDir}/

      If these instructions are actually being repeated multiple times, then before proceeding, really think about whether this makes good sense (considering the results of the above directory listing).

      WARNING: This process is theoretical; it hasn't been fully/safely tested quite yet... Review this very carefully. (Once these instructions have been vetted more throughly, they may be a time saver.)

      [ -f ${VMDskDir}/${VMLILNAM}${VMCrDsNm}-parent-compressed.qc2-updated.qc2 ] && sudo mv -i ${VMDskDir}/${VMLILNAM}${VMCrDsNm}-parent-compressed.qc2 ${VMDskDir}/${VMLILNAM}${VMCrDsNm}-parent-compressed-older-copy.qc2
      [ -f ${VMDskDir}/${VMLILNAM}${VMCrDsNm}-parent-compressed.qc2-updated.qc2 ] && sudo mv -i ${VMDskEnd} ${VMDskDir}/${VMLILNAM}${VMCrDsNm}-parent-compressed.qc2
    • Then, that disk image may be marked as having “read only” permissions:
      sudo chmod u-w ${VMDskBeg}
      ls -Fltr ${VMDskDir}
Childing first disk
export VMDskMom=${VMDskEnd}

The image has now been updated.

ls -l ${VMDskDir}/${VMLILNAM}?.qc2

(Make sure it is “read-only”.)

The parent image of the second hard drive is generally pretty simple, so requiring updates to that image is rather rare. Therefore, these instructions don't provide a series of steps for re-compressing that.

The next task is probably going to be to re-make any child images. (If you haven't yet made any child images, consider yourself lucky, and skip right on past the instructions about re-creating child disks.)

Re-creating child disks

This isn't referring to child “disk image” files that are part of a “backing” file chain that combines multiple files from the same “virtual machine”. This is referring to disks that may be used by another “virtual machine”.

(Note: At the time of the writing of this note, this process has not been tested recently. The process might not precisely match some of the other instructions, which may have been updated more recently. Proceeding very carefully is warranted. Testing the results is also recommended.)

To keep these instructions simple, these instructions do not try to keep any pre-existing data for the children. The following process should be performed for each virtual machine that should be using a child image of the newly updated parent.

echo Parent system name is=${VMParNam}

Update that, if needed. (Chances are that the current value might be okay.)

See how many children there are that need to be updated. One way to see this is:

ls -l ${VMDirBas}/execbin/${VMGenNam}/

However, that likely also includes the parent image. If this guide has been followed fully, a “demonstration” “virtual machine” might also be in the list. The following may show a more focused list:

ls -l ${VMDirBas}/execbin/${VMGenNam}/
ls -l ${VMDirBas}/diskimgs/kiddisks/${VMGenNam}/
  • Next, for each child that needs to be updated, follow the remaining portion of this process.
    • Set the VMLILNAM to point to the child. Chances are that the value is currently pointing at the parent, which is now undesirable.
      echo Old value for system name VMLILNAM=${VMLILNAM}
      export VMLILNAM=newMachineName
      echo New value for system name VMLILNAM=${VMLILNAM}
    • Set directory variable:
      export VMDskDir=${VMDirBas}/diskimgs/kiddisks/${VMGenNam}/${VMLILNAM}
    • See the current files:

      ls -l ${VMDskDir}

      It is probably desirable to stop using any pre-existing images.

      sudo mkdir ${VMDirBas}/diskimgs/kiddisks/${VMGenNam}/oldsysbk/
      sudo mv -i ${VMDskDir} ${VMDirBas}/diskimgs/kiddisks/${VMGenNam}/oldsysbk/VMLILNAM-old
    • Then, re-create.

      sudo mkdir ${VMDskDir}

      Then, back up any desirable images. (No precise example is shown how to do this; one method is to just move them out of the way.)

      Remaking the child image

      Next, this guide re-creates the images (so that they will use the updated parent).

      The following instructions will assume that VMDskDir has been set. Since some prior instructions showed this variable being set, that's assumed to be taken care of already.

      Handle the first disk
      New approach
      export VMCrDsNm=1
      export VMDskMom= parent-1.qc2 1
      Older approach

      Identify the disk file to compress. It may be helpful to see some of the most recent files. Look towards the bottom of the output of this command.

      ls -Fltr ${VMDirBas}/diskimgs/baseimgs/${VMGenNam}/${VMParNam}/

      ...

      export VMCrDsNm=1

      This variable (“temporary long name”) represents a portion of some long pathnames, and gets used at least twice, and maybe multiple times:

      export TMPLNGNM=${VMDirBas}/diskimgs/baseimgs/${VMGenNam}/${VMParNam}/${VMParNam}${VMCrDsNm}-parent-compressed

      Here is the first time it gets used:

      ls -l ${TMPLNGNM}*
      Rationale to rename

      In the long term, we want the parent image to have a sensible name. A sensible name might be something like:

      sysname${VMCrDsNm}-parent-compressed.qc2

      An name that is not sensible to need to keep working with is something like:

      sysname${VMCrDsNm}-parent-compressed.qc2-updated.qc2

      An earlier version of these instructions attempted to just set VMDskMom to that longer name, and the instructions worked out fine for handling the base image and all current “child” machines. But then the instructions to create a new child didn't recognize the convoluted name, and ended up using the old disk.

      The best approach is to just get rid of the convoluted name now.

      Rename

      [ -f ${TMPLNGNM}.qc2-updated.qc2 ] && sudo mv -i ${TMPLNGNM}.qc2 ${TMPLNGNM}-old.qc2

      If that destination file pre-exists, then update the destination filename to something unique. (Do this manually... to simplify these instructions, these instructions do not provide details on how to perform this less-commonly-needed action. For script-makers, mktemp might be helpful. OpenBSD manual page for mktemp)

      Continue the renaming process.

      [ -f ${TMPLNGNM}.qc2-updated.qc2 ] && sudo mv -i ${TMPLNGNM}.qc2-updated.qc2 ${TMPLNGNM}.qc2

      See the results from what just happened.

      ls -l ${TMPLNGNM}*

      Okay, now that the file has been renamed, continue on.

      (Un)set some more variables.

      export VMDskMom=${TMPLNGNM}.qc2
      ls -l ${VMDskMom}
      unset TMPLNGNM
      ls -l ${VMDskMom}

      (The last command should successfully show a file; The file should exist.)

      Handle the second disk
      export VMCrDsNm=2
      Newer process
      export VMDskMom=${VMDirBas}/diskimgs/baseimgs/${VMGenNam}/${VMParNam}/${VMParNam}${VMCrDsNm}.qc2
      Older process

      Identify the disk file to compress. It may be helpful to see some of the most recent files. Look towards the bottom of the output of this command.

      ls -Fltr ${VMDirBas}/diskimgs/baseimgs/${VMGenNam}/${VMParNam}/${VMParNam}${VMCrDsNm}*

      ...

      ls -Fltr ${VMDirBas}/diskimgs/baseimgs/${VMGenNam}/${VMParNam}/
      export VMDskMom=${VMDirBas}/diskimgs/baseimgs/${VMGenNam}/${VMParNam}/${VMParNam}${VMCrDsNm}.qc2
      ls -l ${VMDskMom}

      (The last command should successfully show a file; The file should exist.)

    • Then, repeat for all other machines that may have a disk image which is a child of the old image that has been moved.
Removing old data

As a reminder, you may have troubles using SSH until you remove old key data. If that hasn't been taken care of yet, do so now.

echo ${VISUAL}
${VISUAL} ~/.ssh/known_hosts

Find any line that corresponds to the system that has just been updated, or any children that have been re-created. If the data in this file is related to an old key that has been erased, then that data is of no use. Worse yet, it will cause SSH to freak out, and prohibit using the key. Avoid this problem by removing the lines (and saving the updated file).

Finally, proceed with the next desirable step(s). Chances are, that will involve starting up the virtual machine(s) that use the child images.

Childing the base images

(Skip this section)... this section was previously placed right after the section on compression. It was since decided that this was rather redundant with later notes. If the process works smoothly without these apparently-redundant instructions, then these may be removed.

These instructions used to just point at the hyperlink. Investigating this further, it appears that this might only have been meant for handling the child images. If there are no child images, skipping these instructions may be the most sensible route.

If there were machines that used disks which were children of the older version of the “parent” image, it may be good to have those children point to the newer version. Then again, that likely involves needing to repeat any customization on the existing child, which might be too much hassle to be worth making that update. If the old disk image is still around, simply pointing at that might be the best course of action. (Still, newly created machines can point to the new file.)

The following steps may be helpful:

Consider clean-up

At this time, the new image is ready to go.

There may be some extra files in the parent's directory. Frankly, at the time of this writing, the parent's directory may be a bit of a mess due to some extra files there. Moving some of the extra files to an “old subdirectory may be appropriate.

ls -l ${VMDirBas}/diskimgs/kiddisks/${VMGenNam}/${VMLILNAM}/

However, this guide currently has no specific instructions for how to accomplish that.

  • If another system uses a similar IP address, the TUN/TAP device for that other system might use an IP address from the same subnet (and maybe even the same IP address!) as the TUN/TAP device for the system that is being focused on. This can be a problem, even if the other virtual machine is not running. The safest approach is to check all TUN/TAP devices that might be using the same subnet, and remove all IP addresses that aren't currently needed.