Updating/Upgrading OpenBSD w/o interacting with source code

Using officially released versions
It is possible to upgrade the operating system without needing to compile source code. This is done by using a new “release” of the file system installers. Most famous of all OpenBSD releases is the new official released version that gets created by the OpenBSD team every six months. For details, see the section about upgrading to a new officially released version of OpenBSD. For these -release versions, the OpenBSD team also releases an upgrade guide to make such an upgrade as painless as they can. The biggest downside to relying solely on this option is just that a new “-release” is only released twice a year.

For most users, this is the limit of commonly recommended procedures that don't involve using source code at all. There are additional available options that may involve updates being available more frequently. However, those additional options either involve using source code (at least a bit on one machine), or pursuing options that may not be as commonly recommended for widespread use.

[#obupdrel]: Upgrading to a new -release

There is a separate guide with each new release (starting with version 3.5). Each guide starts with a warning that says something like: “Note: Upgrades are only supported from one release to the release immediately following it. Do not skip releases.” By the fourth upgrade guide there was an additional recommendation and an overview of expected results which has been duplicated in the subsequent guides.

The upgrade guide for the latest release may be found from OpenBSD FAQ: “Commonly Encountered Issues” section's hyperlink called “How do I upgrade my system?”. (The hyperlink's destination is unique for each version of OpenBSD, but the latest hyperlink follows a rather predictable naming pattern, and can be found in the FAQ's “Commonly Encountered Issues” section.)

The upgrade guide may also be found by going to the main page at OpenBSD.org and following the hyperlink to the page about the latest release. That page will have a hyperlink called “How to upgrade”, which is named after the page's subsection that contains the hyperlinked anchor. (The anchor is called “#upgrade”).

Be sure to look at the upgrade guide released with each new version! Especially, read the section called “Before upgrading” (at “#before”). There are occasionally some rather major items listed, which are generally fairly painless to deal with if they are known about before the upgrade procedure is started. Some examples of some rather impacting items include:

This website may not have an extensive guide about upgrading to a new release version. If the operating system hasn't been highly customized, it may be rather painless to do a full re-install. For those who are shocked because of such a difficult-sounding proposal when the word “full re-install” are read, please carefully notice the part of the previous sentence that stated the process “may be rather painless”. In some cases, a rather full re-installation may be very simple and provide some benefits. (In other cases, re-designing things to simplify the upgrading process may be a good long-term goal to consider.)

There may be some advantages to performing a full installation. Of particular note is the following quoted paragraph from (version 5.0 of) the upgrade guide on openbsd.org.

“Upgrading is a convenient way to bring your OpenBSD system up to the most recent version. However, the results are not intended to precisely match the results of a wipe-and-reload installation. Old library files in particular are not removed in the upgrade process, as they may be required by older applications that may or may not be upgraded at this time. If you REALLY wish to get rid of all these old files, you are probably better off reinstalling from scratch.”

Compared to updating an existing operating system by using “upgrade” software, re-installing an operating system will be even less likely to result in any issues caused by an upgrade process being imperfect. At least in theory, it also might end up wiping out some potential security problems if a machine had been compromised. (However, in that case, restoring full data and functionality might involve using old data files and end up re-compromising the system.)

Using code released by a third party

Consider whether this is desirable (from a security perspective) before just proceeding with any of these methods...

BinPatchNG

This solution appears to be by a company called m:tier. See: BinPatchNG (overview), BinPatchNG (&ldquoBinpatch Next Generation”), binpatch framework (binary patches for OpenBSD), (news) article at OpenBSD Journal @ Undeadly.org about BinpatchNG

At the time of this writing, this is fairly recent news. Note that patches may be made only for OpenBSD/amd64 and OpenBSD/i386.

Using a release created separate from the official OpenBSD team

One way of applying an update is to use a “release” that wasn't published by the OpenBSD team. The term “release”, as used in this specific guide, basically refers to a collection of files usable by OpenBSD's installation process. Far from being discouraged, the OpenBSD team's OpenBSD FAQ 5: section on Building a Release (section 5.4) actually publishes instructions for easily creating such a release. Once a release has been created (by having a machine interact with source code), a machine does not require any source code interaction to effectively use an already-created release.

Although the OpenBSD team does create releases (executable/binary files) for the “-release” flavor of the operating system, similar official releases for the “-stable” flavor may be something that is not provided by the OpenBSD team. Unless/if that changes, other options may need to be pursued in order to be able to install -stable.

How to obtain possession of a release created separate from the official OpenBSD team
Creating a release

Although the OpenBSD project doesn't release -stable installers themeselves, OpenBSD's page about following -stable says “you may wish to make a release, creating the standard install file sets”. Doing so does involve using source code to update OpenBSD. Making a release of “-stable” will be, in some way, different than making a release based on published errata.

Personally creating a release from trusted hardware and software should result in a release that is as trustworthy as the code used to make the release. Therefore, using an internally-owned, privately-controlled computer that has only trusted hardware and software can result in the creation of a release that should be pretty trustworthy.

However, to summarize quickly, this process is probably not going to be time-efficient for upgrading just one machine just one time. It probably will be time-efficient if upgrading at least two machines (unless, perhaps, all of the machines are being upgraded at the same time). This is because it does take some time to make a release, but this takes less time than compiling. Compiling is needed for one machine, but then that machine may make a release which can save time since other systems can then use the release instead of needing to perform the step of compiling. There could be exceptions to some of these summarized conclusions. For example, if a fast machine is used to make a release, which is then used to apply an upgrade for a slower machine, then the process of making a release on the fast machine may save time, even if only one (slower) machine is being upgraded.

[#unofstbl]: Using community resources: Unofficial releases

Since the official guide shows how to easily make an operating system release, is there any chance that this has been publicly done? Yes, there is such a chance. Although the most recommended way to obtain such a release is to use trusted source code in order to create a release, it may also be possible to use a third party offering.

Naturally, a key security concern is to make sure that no such release is being used unless it certainly comes from an origin that is known to be trustworthy. Although upgrading (e.g. from “-release” to “-stable”) may result in a system with increased security, note that some people may see some irony in using data from an unofficial source in order to be increasing security.

A “ReadMe” file related to an unofficial OpenBSD “-stable” release acknowledges such concerns by noting, “Please note that these files are not provided by the OpenBSD project. If you don't trust them, don't use them.” Alternatives are then provided.

Such good advice (of not using such files) is much easier to recommend than using these sorts of files. The most recommended method would be some of the other methods that involve only using the official files provided by the OpenBSD project. (One option may be to personally create a release. That, and the other methods that are generally recommended for end users, involve using source code. Information on building/creating releases is included in this guide.)

Although information about third party offerings may be provided here, note that this is done to provide details that might be helpful, and to acknowledge efforts that do exist. However, inclusion of these details is not meant to be a recommendation. If these efforts that are taken end up causing security problems then such efforts really deserve no recognition whatsoever.

ftp.su.se / OpenBSDsupport.org
OpenBSDsupport.org, a third party website which is not an official part of the OpenBSD project that releases the operating system at OpenBSD.org, may regularly have information about third party installation files of -stable. (The released files may be those located at ftp://ftp.su.se/pub/mirrors/openbsd_stable/. Despite the domain name including the word “OpenBSD”, the website notes, “The content published here in no way implies that the OpenBSD project or any member of the OpenBSD team sanctions or approves of such use.”
[#obsdsnap]: Using snapshots
Overview

Another option, using code provided directly from the OpenBSD team, might be to use one of the “snapshot” releases that are not good to try to use in some cases. Reading about this process is strongly recommended to reduce the otherwise quite high likelihood of being surprised with an unstable, or outright completely broken, non-working system.

Snapshots are not always being publicly distributed by the OpenBSD team. Very often they are available from OpenBSD's primary FTP site: “snapshots” section at ftp://ftp.openbsd.org/pub/OpenBSD/snapshots/. When they are available, snapshots will likely use some of the source code which is considered to be part of the “-current” flavor of source code. However, OpenBSD FAQ 5.1: Flavors also discusses “Snapshots”, noting that “snapshots often contain experimental code that isn't yet committed to the tree.” This likely means that a snapshot isn't necessarily the same thing as “-current”. The most precise definition of “-current” is probably the source code that is obtained from the CVS trees when no tag (such as the tag specifying the “patch branch” that represents the “-stable” flavor) is specified.

If there is an interest in using snapshots, be forewarned of the caveats (like all packages disappearing from the FTP site). Because this uses code marked as being from the “-current” source tree, and because installing from a snapshot is generally a recommended step in starting to use “-current”, there is some quoted information about “-current” that should be considered applicable to snapshots.

Though there are some considerations (possibly disadvantages) worth making warnings about, an advantage to using such a release is that it is likely to be based on some newer code.

[#obsnaprb]: Considerations, warnings, and potential problems related to using OpenBSD snapshots (and/or “-current”)
Overview: Not for everyone, but not something that can't be used in production

OpenBSD FAQ 5.1: Flavors says,

“In fact, as our hope is to continually improve OpenBSD, the goal is that -current should be more reliable, more secure, and of course, have greater features than -stable. Put bluntly, the "best" version of OpenBSD is -current.”

This sort of quote makes -current seem like it is really the ideal way to go. But then there's some other quotes give some cause to slow down any undue excitement:

Why the hold up if the “-current” is the “best”?

One theory: Many might instantly draw comparisons to Debian which have releases called “stable”, “testing” which is less stable, and then the even less stable variation, “unstable”. Many people recommend that code which isn't marked as “stable” may be used in some test environments, but should not on any important computers where there is a substantially high need for it to be working right. Such important computers are often called “production” computer systems. (This could refer to not just servers: even workstations that perform a task that is essential for smooth operations may be considered to be part of a “production” environment.) Therefore, a natural inclination of people familiar with these guidelines, and familiar with Debian's naming scheme, is to think it wise to avoid using “-current” on a production system.

However, directly countering that theory, right after noting that most “users should be running either -stable or -release”, the next sentence of OpenBSD FAQ 10: section on “Applying patches in OpenBSD” says, “That being said, many people do run -current on production systems, and it is important that people do so”. Remember, theories that apply to Debian do not always apply to OpenBSD. (OpenBSD FAQ 9.1: Tips for users of other Unix-like Operating Systems may list some other things that are different from OpenBSD and many operating systems that are based on a Linux kernel or other similar platforms.)

It is clear that the OpenBSD project welcomes people to try using -current. Still, there are some considerations (possible difficulties and perhaps even disadvantages) worth knowing about before deciding to advance to using -current instead of -stable.

No handholding

OpenBSD FAQ 5.1: Flavors states, “one should not expect hand-holding from the developers.” The same FAQ also describes that people using “-current” code should be able to “diagnose and deal with a problem”. A person should probably be willing to abandon the use of “-current” and switch to “-stable” (by first upgrading to a newer “-release” version, or if that isn't possible, by re-installing an earlier “-release” and upgrading to “-stable”) in the event of any problem. This is not just theory: OpenBSD FAQ 5: section about the operating system's “Flavors” (FAQ 5.1), subsection called “Keeping Things in Sync” says, “You are on your own if you choose any path other than the supported option of reloading your system from scratch, do not expect assistance from the OpenBSD development team.” The FAQ goes on to say, “Yes, this does mean you should think before committing yourself to using” code from “-current”. Being willing (and being fairly able) to use, review, and perhaps even debug source code may also be useful skills.

OpenBSD FAQ 9.1: Tips for users of other Unix-like Operating Systems states, “The tree is occasionally broken, but this is an extraordinary event that will be corrected rapidly, not something that will be permitted to continue.” This statement does suggest that, even if only for limited periods of time, there may some times when the “-current” code, in some possibly-critical fashion or another, just plain won't work. This is even true with the binary installation files: OpenBSD FAQ 5.1: Flavors states, “There is no promise that the snapshots are completely functional, or even install.”

If this is sounding too scary, then don't expect much, if any, support. Whining about “-current”, because one wasn't sufficiently prepared for related difficulties, would simply be viewed as complaining about self-inflicted pain. This might not generate much sympathy from others. (People who are not prepared to deal with such difficulties should really consider using “-stable” instead of “-current”) As just demonstrated, there are multiple quotations scattered around the FAQ to warn people about using “-current” code.

As an example of where whining gets a person, see the last September 10th comment of OpenBSD Journal @Undeadly.org “New Ports of the Week #36 (September 2, 2010).

Code sync requirement (and ramifications of this requirement)

One of the main considerations to keep in mind is the need to use packages that are compiled for the libraries that come with the version of the operating system that is being used. OpenBSD FAQ 9.1: Tips for users of other Unix-like Operating Systems states,

OpenBSD is a complete system, intended to be kept in sync. It is not a "Kernel plus utilities" that can be upgraded separately from each other. Failure to keep your system (kernel, user utilities, and applications) in sync will result in bad things happening.

OpenBSD FAQ 5.1: Flavors has extremely similar statements:

It is important to understand that OpenBSD is an Operating System, intended to be taken as a whole, not a kernel with a bunch of utilities stuck on. You must make sure your kernel, "userland" (the supporting utilities and files) and ports tree are all in sync, or unpleasant things will happen.

These statements may be more significant than people commonly perceive when initially being exposed to such statements. Basically what these statements mean is that upgrading any part of the complete system, such as a single program, may require that the entire operating system, and all other programs, are also upgraded at the same time. That way, the resulting code of all operating system components, program executable binaries, and libraries are all at versions that are designed to work with each other.

This is far less modular than some other operating systems and/or package management systems that allow at least a bit more version mismatching. One reason why OpenBSD is so strict with versions may, at least in part, be addressed in the last sentence of the first paragraph of the “Keeping Things in Sync” section of OpenBSD FAQ 5.1: Flavors. That sentence states, “Sorry, but again, OpenBSD has limited resources available.”

The requirements of using the latest version of the operating system may be increasingly signficant when an operating system version is not considered to be the latest operating system version for very long. The frequency of code being updated, to the point of incompatibility, may routinely be a timeframe no longer than a small number of days. It is quite possible that the new code and the old code, from just days ago, won't mix so well. This heavily restricts what pre-built versions of a program can be run, and obtaining updates for some small amount of code can necessitate upgrades for a much larger amount of code.

OpenBSD FAQ 15.4.1#NoFun: Crazy errors notes that “even if” a computer's “system was -current just a few weeks ago”, that may not be sufficient. Actually, the reality is the breaking point is frequently shorter than weeks, and so frequently things can definitely become incompatible over a period of days. If a package was updated to use a new “-current” release, it is quite possible that the same package may not work with any older release, including previous “-current” code. OpenBSD FAQ 5.1: Flavors: “Keeping Things in Sync” subsection states, “Yes, this does mean you need to upgrade your system if you want to run” such newer code.

This restriction isn't limited to “-current” releases: Mixing a non-static executable from one “-release” version with library code from an older “-release” will probably also have problems. However, this is particularly being mentioned in this section about “-current” (and/or “snapshot” releases) because trying to use code from a “-current” tree of software “ports”, and specifcally trying to get that code to work on “-release” or even “-stable”, must be commonly attempted. The apparent frequency of this problem is clear from the fact that OpenBSD FAQ 15.4.1#NoFun: Crazy errors says “This is one of the most common errors and you will irritate people when you ask for help about why "nothing seems to work!"”

There are additional comments throughout the FAQ mentioning the need to keep things in sync. For instance, the first sentence of OpenBSD FAQ 15.3.2: “Fetching the ports tree” says, “Before continuing, you must read the section about NOT mixing up your OpenBSD system and ports tree.” (It then mentions a need for a decision of which “flavor” is being used.) OpenBSD FAQ: Building the userland says “As all parts of OpenBSD must be kept in sync, all trees you use should be checked out and updated at the same time.”

Disappearing code (especially packages)

While a -release is the latest for approximately six months, -current snapshot installer files may routinely be removed from the site. The nicest of all possible reasons may be that those files are being replaced with a new version, but that isn't the case every time. In some cases, the old ones just stop being distributed on the main OpenBSD FTP site. So, there is a need to obtain all desirable installation files within that short window of time, or deal with having those installation files not being available.

Those who are using binary “snapshots” packages may do well to start by comparing the dates of the packages. Sometimes a -current may be released without releasing packages. An old set of packages may be removed from the site, without warning. A new set of packages may not be uploaded for days, and possibly weeks, after an older set disappeared. Before installing the operating system from snapshot binary files and thinking that the available packages will work, check whether the available “snapshots” packages have file dates that are at least as new as the “snapshots” operating system installation files. Otherwise, the result may be trees that are not in sync, as described by OpenBSD FAQ 15.4.1#NoFun: Crazy errors. If the situation isn't looking appealing, one option may be to wait days (perhaps weeks) to see if the packages do start to exist with newer file dates.

Although installing -current can be as easy as installing from the snapshots area of OpenBSD's FTP site ( ftp://ftp.openbsd.org/pub/OpenBSD/snapshots/ ), be sure to also download any related packages that might be desirable. This is true even if the package might not be of interest for a while. For instance, if there is a package available today, and 2 weeks later it is decided that such a package would be good to install, the needed old version of the package might no longer be on the site where it was originally obtained. Furthermore, updating to a newer snapshot release may not be a possibility, because it could be that no newer release is available: perhaps no newer release was made, or perhaps another new release was made but has subsequently been taken down.

There may or might not be some other desirable options of dealing with the situation when the binary installer file is not being offered anymore. The possibilities may include installing from a previously-saved copy from when it was available, or continuing without using the desirable software, or compiling from ports (although that may not be possible/feasible if the ports file hasn't been saved), or upgrading the operating system. Although upgrading the operating system may not seem undesirable, what may be less pleasant is for the motivating factor being that a specific old version of some minor software, perhaps a single dependency, wasn't downloaded earlier. This could especially be true if the process of installing an operating system isn't repeatable because there isn't a newer, and also working, binary snapshot installation file (either for that specific software, or for any other needed piece of software that must be upgraded so everything is kept in sync).

The best approach may be to simply download any packages that might be used, even if it is not certain when, or even if, they'll be needed. Also, keep in mind that not only do all software packages have to be kept in sync, but also all dependencies of all packages. This means that having the files needed for a program like nano may involve more than just nano. Additional packages, like gettext and ncurses, may be dependencies and so the correct versions of all of those packages will need to be either installed or available to install. If the dependencies aren't going to be immediately installed, at least obtaining all of the packages will be needed. So problems could exist not only by forgetting to grab a package named after some favorite computer program, but also by not having a copy of the dependency. (A simple way to address that might be to just download all packages when all packages are available, but consider the resources that will be used by doing so.)

Proceeding with snapshots

If all these warnings are understood and heeded, and using -current is still desirable, then proceeding with the snapshots may be an option, if such files are available. Those may be found from under OpenBSD snapshots packages. Also grab the OpenBSD snapshots ports file (which is platform-independent source code) just in case that file will, with additional work, be useful later. Be sure to read the prior warnings about code sync requirements and disappearing code, and grab any packages that will be needed.

While OpenBSD “-stable” has a dedicated page, the main text about Snapshots is a section called “Snapshots” which is part of OpenBSD FAQ 5.1: section about OpenBSD “Flavors”. However, since Snapshots use “-current” code, a lot of text in the FAQ's referring to “-current” code does apply to these Snapshot releases.

Obtaining snapshots

Again, keep in mind that recent snapshots may not be being publicly offered by the OpenBSD team. If they are, get all of the following:

Before installing any release using “-current” code, prepare for a situation where things work worse than they did pre-upgrading, such as a computer system that just plain won't boot whatsoever. Realize that isn't just a theoretical problem: sometimes the code is known not to work. That, and other warnings, have been provided earlier in this section about OpenBSD Snapshots. Be prepared to deal with troubles before proceeding. That's generally wise advice for any such change, but is especially true when dealing with development code (like “-current”/Snapshots), which may be a bit less likely (than code bundled for a release) to be very polished.