File Integrity Checking

This section describes “File Integrity Checking”.

Basically, the file integrity checkers will perform tasks (like checking for a file hash, and determining the permissions on a file) and store information in a database. As long as that database is available (without anyone successfully tampering with it), file integrity checking software can use that database to compare a system's prior state to a system's current state. Differences can be reported.

(Although such software can perform file hashing, this software is typically designed to perform such tasks on multiple files. If all you are seeking is to check a single file's “hash” results, information on that is available in another section about file hashing.)

(Unfortunately, this particular guide may not be cleaned up very, very well.)

Integrit:
	/dirname
		Check directory name and all its contents
	$ /dirname
		Check directory name, but not its contents
	= /dirname
		Check directory, and its contents, except not its subdirectories
	!/dirname
		Do not check this.

There may be some references to /var/db/aide_old which might actually have just been a manually created old copy of /var/db/aide (in which case references to aide_old should be removed). Check all references to aide_old and see if any of them are substantial/important at all.:

Status: This page has some details about running some file integrity checks manually. A better idea may be to automatically report variances, and rotate out old database files so that they are (preserved but) not used for future executions. (Rotating old reports may also be desirable.)

Details for smaller amounts of files

For a small number of files (like two or five, rather than hundreds or thousands), there may be ways to compare the files that seem simpler than some of these file integrity checkers. This section simply provides some details to provide some familiarity.

A very similar/related topic is comparing individual files.

Overview: Some good these programs may do

Running these after installing a new system is installed may be a good idea, and so installing one or more of these programs may be a good idea.

Things this software can help with:

  • report intrusions
  • when problems do occur, e.g. a file lost in the filesystem, this may help identify which files were lost.
  • if a folder is deleted, this can show what files used to be in that folder. (The backup program should probably also be able to tell this.) If data got accidentally moved, this may show where it went to.
  • report files that change due to media errors changing bits
Recommendations
Which solution(s) to use

These are simple guidelines; not hard and fast, strong recommendations.

For now, Aide is the recommended tool. A key reason for this is simply that its support for RegEx's may have allowed a configuration file to be easier to make than Integrit. In the future, radmind or Integrit may rank as high as, or higher, than Aide. For now, however, Aide has the advantage of having a rather complete example configuration file with these instructions. (The example configuration file may be rather tuned to a fairly simple OpenBSD system, and so users of other operating systems may benefit from planning to do some configuration tinkering anyway even if Aide is being used.)

Aide and Integrit may be largely similar, with Aide perhaps having some neglible advantages: Better documentation (There was a note made in the Integrit section, Overview subsection, that suggested the documentation was a bit sparse), and defaults of running more checks to be accurate in determining if changes have taken place. (This difference really is not very impacting when heavily configuration customization is occurring anyway.) AIDE also supports RegEx expressions, which Integrit seems prone not to do (based on Integrit's FAQ information about RegEx). Integrit's documentation notes some compelling-sounding reasoning for the software to not use RegEx, but the reality is that AIDE's existing support for RegEx can more easily allow AIDE to dynamically notice any new filesystem entries. These are just some small touches that make AIDE seem slightly more attractive. On the other hand, Integrit has some auxilary tools (Integrit manual: section about auxilary tools) which can be nice.

OpenBSD comes with mtree which has the advantage of convenience, since it is bundled in with the operating system. It may, however, be more limited when it comes to the simple feature of excluding things. Other operating systems may have other software pre-bundled (in theory, though in reality: often other operating systems do not come pre-bundled with other software).

Common files to exclude
Unix filesystem objects to exclude

Here are some notes. At the time of this writing, this is basically made from an OpenBSD system. For other systems, other objects may be more applicable.

How to use this info

This section shows some command lines that will try to output just the filenames/directories to be excluded. e.g.:

ls /

In this example command, files and directories are displayed. They may be displayed in columns.

The first step is going to be to translate the list of filenames to a space-separated list. e.g.:

ls / | xargs echo

The xargs command strips out the unwanted white space, creates a string, and then runs the specified command with the string as a parameter.

... or, may shell prompts may support the following syntax:
echo $(ls /)

... or, if xargs is not given a parameter, it simply prints out the filenames. So the previous two techniques can be somewhat combined, using:

echo $(ls / | xargs)

Then, there will probably be a desire to run a command on each filename. So, do something such as:

for x in $(ls / | xargs) ; do echo $x ; done

Then customize the echo command as desired.

[#pxrgchfl]: (Posix) Items to ignore

Be careful with exclusions: the “manual” for AIDE from the project's web site: “General guidelines for security” section has some specific advice. Keep in mind that if a log file is specified, and an attacker is able to write to that log file, the specified filename won't be scanned and so the attacker's file may not be detected.

File integrity checkers
Reasoning

Other file integrity checkers may routinely update their database between times when this file integrity checker updates its database.

It is possible that the file integrity checker may even report on its own data.

Examples to exclude
mtree files

This varies. There may be no strong standard. If this documentation's examples are being used, /var/db/mtreedat/*.dat may be used.

AIDE data files

/var/db/aide.db.new and /var/db/aide.db are the default filenames. If this documentation is used, also: /var/db/aide/olddbs/aide.db.old* (or just all of /var/db/aide/olddbs/ if that's easier)

Integrit

/var/db/integrit/known.cdb and /var/db/integrit/current.cdb.new

(If this documentation is used, old files go to the /origbak directory used by cpytobak. Also, if the instructions from this guide are used for setting up Integrit, there may be some logs added to a /var/log/integrit directory.)

Device files
known TTY devices

To print out known TTY devices:

echo \# Exclude Known TTY ports that tend to always look changed
for y in $( sudo grep -i tty /etc/ttys | cut -f 1 | xargs ) ; do echo $y ; done
Some other devices that regularly change

Device objects that regularly change These may change even if they represent devices that are not expected to be used (e.g. mouse/keyboard) /dev/wsmouse /dev/wsmouse0 /dev/wskbd /dev/wskbd0 /dev/console /dev/null /dev/arandom /dev/log

Also, when the system reboots, /dev/tty[C-E]cfg (the precise filename may be OS-dependent: /dev/tty[C]cfg in OpenBSD and /dev/tty[E]cfg in NetBSD)

BPF

For systems that use Berkeley Packet Filter, perhaps also /dev/bpf0 or even /dev/bpf? objects.

Presumably these objects won't exist on non-supporting systems, so only bother excluding them if they pre-exist when making the initial configuration.

Raw disks objects

These could be quite large, which would take quite a file for the file integrity checker to perform. Also, the main thing learned is just if a disk changes, which would happen if any data changed. That seems likely, and so not very worth reporting.

Files that change when processes run
/var/run/utmp and /var/run/*.pid and perhaps even all of /var/run/*.pid
Log files

Unfortunately these might be tampered with, but systems can have log files that are likely to change so regularly that there is no need for routine scans to report they changed.

Log files:

/var/www/dev/log

for z in authlog daemon lastlog maillog messages secure wtmp ; do for y in /var/log/${z} /var/log/$z.?.gz ; do echo $y ; done ; done
/var/cron/log /var/empty/dev/log /var/log/sendmail.st
OpenBSD logs

The following may be more OpenBSD-specific:

/var/log/daily.out /var/log/security.out /var/log/daily.out.old /var/log/security.out.old

Files that change on each reboot
Files that change on each reboot /var/run/dmesg.boot /fastboot
Databases routinely changed
/var/db/ntpd.drift /var/db/dhclient.leases.* (for each NIC) /var/spool/mqueue /var/spool/clientmqueue maybe /var/run/dev.db ????? maybe /var/run/ld.so.hints ?????
Frequently changed directories

Perhaps their contents may be worth scanning, but the attributes of these directories may change regularly.

/var/db /var/mail /var/empty (Really?!? Possibly this is likely because of /var/empty/dev/log) /var/log /var/tmp Perhaps /home
Backup files

Backup files (should be routinely updated)

This may depend on what backup software is being used.

/origbak (if using the cpytobak program)

Possibly OpenBSD specific

Are these created from files that are run by OpenBSD's daily scripts?

Some other files that may change regularly /var/backups/*.current /var/backups/*.backup /var/backups/*.backup.md5
[#pxrgcflc]: Making a configuration file to exclude items
Posix items to ignore

Probably not a complete coincidence, there seem to be some strong similarities between the configuration file systems of AIDE, Integrit, and Tripwire. For instance, excluding a single file may be done by using an exclamation point followed by the filesystem object to exclude.

The following is a series of command lines to help build a part of a configuration file that excludes many of the commonly excluded objects.

[#pxrgcfla]: Part one: items to exclude which will be on many Posix systems

Be sure to set the fintckcf variable before trying to use these example lines.

echo \# !!!!!!!!!! Exclusions follow !!!!!!!!!! | sudo -n tee -a ${fintckcf}
echo | sudo -n tee -a ${fintckcf}

echo \# Data from File Integrity Checker\(s\) | sudo -n tee -a ${fintckcf}
echo \# A file integrity checker database\; maybe updated more recently than | sudo -n tee -a ${fintckcf}
echo \# this check | sudo -n tee -a ${fintckcf}
echo !/var/db/integrit/known.cdb | sudo -n tee -a ${fintckcf}
echo !/var/db/integrit/current.cdb.new | sudo -n tee -a ${fintckcf}
echo \# A file integrity checker database\; maybe updated more recently than this check | sudo -n tee -a ${fintckcf}
for y in /var/db/mtreedat/* ; do echo !${y} ; done | sudo -n tee -a ${fintckcf}
echo # Older files from AIDE file integrity checker\; maybe updated more recently | sudo -n tee -a ${fintckcf}
echo !/var/db/aide_old/.\* | sudo -n tee -a ${fintckcf}
echo # than this check | sudo -n tee -a ${fintckcf}
echo \# Aide database file | sudo -n tee -a ${fintckcf}
echo !/var/db/aide.db | sudo -n tee -a ${fintckcf}
echo \# A database file | sudo -n tee -a ${fintckcf}
echo !/var/db/aide.db.new | sudo -n tee -a ${fintckcf}
echo \# Archived older database file\s\) | sudo -n tee -a ${fintckcf}
for y in /var/db/aide/olddbs/aide.db.old* ; do echo !${y} ; done | sudo -n tee -a ${fintckcf}
echo \# Known device filesystem objects | sudo -n tee -a ${fintckcf}
echo \# Exclude Known TTY ports that tend to always look changed | sudo -n tee -a ${fintckcf}
for y in $( sudo grep -i tty /etc/ttys | grep -v ^\# | cut -f 1 | xargs ) ; do echo !/dev/${y} ; done | sudo -n tee -a ${fintckcf}
echo \# Other device objects ports that tend to always look changed | sudo -n tee -a ${fintckcf}
for y in wsmouse wsmouse0 wskbd wskbd0 console null arandom log ; do echo !/dev/$y ; done | sudo -n tee -a ${fintckcf}
for y in /dev/bpf* ; do echo !$y ; done | sudo -n tee -a ${fintckcf}
echo \# Files that regularly change when processes start/end running | sudo -n tee -a ${fintckcf}
echo !/var/run/utmp | sudo -n tee -a ${fintckcf}
for y in /var/run/*.pid ; do echo !/dev/$y ; done | sudo -n tee -a ${fintckcf}
echo !/var/run | sudo -n tee -a ${fintckcf}
echo \# Log Files | sudo -n tee -a ${fintckcf}
for z in authlog daemon lastlog maillog messages secure wtmp ; do for y in /var/log/$z /var/log/$z.?.gz ; do echo !$y ; done ; done | sudo -n tee -a ${fintckcf}
echo \# Log file that may be updated extremely frequently | sudo -n tee -a ${fintckcf}
echo !/var/cron/log | sudo -n tee -a ${fintckcf}
echo \# Mailboxes | sudo -n tee -a ${fintckcf}
echo !/var/mail/.* | sudo -n tee -a ${fintckcf}
echo \# Maybe OpenBSD specific files | sudo -n tee -a ${fintckcf}
for z in daily.out security.out daily.out.old security.out.old ; do for y in /var/log/$z ; do echo !${y} ; done ; done | sudo -n tee -a /etc/aide.conf
echo !/var/log/sendmail.st | sudo -n tee -a ${fintckcf}
echo \# Filesys objects that change on each reboot | sudo -n tee -a ${fintckcf}
echo !/var/run/dmesg.boot | sudo -n tee -a ${fintckcf}
echo !/fastboot | sudo -n tee -a ${fintckcf}
echo !/etc/resolv.conf | sudo -n tee -a ${fintckcf}
echo !/var/cron/tabs/.sock | sudo -n tee -a ${fintckcf}
echo !/var/cron/tabs/.sock | sudo -n tee -a ${fintckcf}
echo !/var/empty/dev/log | sudo -n tee -a ${fintckcf}
echo \# may change each reboot | sudo -n tee -a ${fintckcf}
for y in /dev/tty[C]cfg ; do echo !$y ; done | sudo -n tee -a ${fintckcf}
echo \# Databases routinely changed | sudo -n tee -a ${fintckcf}
echo !/var/db/ntpd.drift | sudo -n tee -a ${fintckcf}
echo !/var/db/dhclient.leases.* | sudo -n tee -a ${fintckcf}
echo !/var/run/dev.db | sudo -n tee -a ${fintckcf}
echo !/var/run/ld.so.hints | sudo -n tee -a ${fintckcf}
echo \# Directories related to mail \(may change when mail is sent/recv\) | sudo -n tee -a ${fintckcf}
echo !/var/spool/mqueue | sudo -n tee -a ${fintckcf}
echo !/var/spool/clientmqueue | sudo -n tee -a ${fintckcf}

[#pxrgcflb]: Part two

The next part of the exclusion list may refer to some directories. These may be handled a bit differently by the various pieces of file integrity checking software.

[#pxrgcfld]: Part three: More files to exclude

Some of these may be more likely to be more specific to the OpenBSD platform...

echo \# Backup files \(should be routinely updated\) | sudo -n tee -a ${fintckcf}
echo !/origbak | sudo -n tee -a ${fintckcf}
echo \# Backup files \(possibly OpenBSD-specific\) | sudo -n tee -a ${fintckcf}
echo !/var/backups/*.current | sudo -n tee -a ${fintckcf}
echo !/var/backups/*.backup | sudo -n tee -a ${fintckcf}
echo !/var/backups/*.backup.md5 | sudo -n tee -a ${fintckcf}
echo \# Disks \(OpenBSD device names\) | sudo -n tee -a ${fintckcf}
for y in /dev/r[swd[0-9][a-z] ; do echo !$y ; done | sudo -n tee -a ${fintckcf}

The remaining part may, once again, be a bit more specific to the implementation being used. So, do check the documentation/examples/recommendations for the specific software being used.

(Info about other operating systems)
...
Some solutions for Unix-like operating systems

This section lists various options. However, when following a tutorial, it is often nice to have recommendations whereever there are options. This recommendations may change as the tutorial expands to cover different options in more depth, but for now the recommendation is to use both “AIDE” and “integrit”.

[#mtree]: Using mtree
Overview

Using mtree in this way may be slick for a few reasons: the program may be something commonly included in the operating system (therefore requiring little installation), is lightweight, and supports using a key so that the data files themselves are not so easily tampered with. (It is still recommended to compare the data files of a questionable machine with copies of those data files that were stored in a trusted manner on other media, and a 23 digit key does not seem like a lot of bits to prevent brute forcing, but still it is nice to know that there is some effort to making the stored hashes protected. If for no other reason, this may be nice simply as an example that could be used for developing a variation that uses a stronger protection method.)

However, it may generate a fair amount of output. Other solutions may allow for configuration files that exclude certain content which is of little concern, such as TTY devices (e.g. /dev/[pt]typ[0-9] (example taken from OpenBSD). The mtree command may be too simplistic to support such advanced procedures. (This state of being too simplistic for this goal may be, at least in part, due to mtree having an initial design/focus on working with source code, rather than trying to check for unauthorized tampering on a large portion of a computer's filesystem volume.)

Initial initialization

If this hasn't been done yet, make a location for mtree data to go.

sudo mkdir /var/db/mtreedat
sudo chown root /var/db/mtreedat
sudo chmod ug+rwx,o-rwx /var/db/mtreedat/
Creating a new database

Then, to create a file with information about a directory called /stuff, make a data file as follows:

sudo mtree -c -K cksum,flags,gid,gname,link,md5digest,mode,rmd160\
digest,sha1digest,size,time,type,uid,uname -s 12345678901234567890123 -p /stuff | sudo tee /var/db/mtreedat/stuffdir.dat

The above shows a line being split, to aide in common word-wrapping when viewing the documentation. The end result should be that “rmd160digest” is a single word, without white space.

The “ -s 12345678901234567890123 ” refers to a key, and may be any randomly selected digits as long as all 23 digits are remembered for future use.

To get details for all of the system's data, specify the root directory for the path.

Note: The above example does not check the error code during this process. The reason is because the output is being sent to a database file, using piping rather than redirection. The piping will destroy the value of the error code. This is worked around in several other examples where the error code may be appended to the standard output, but in this case the standard output's redirection goes to a file that is later used as a data file. Now, one way to work around this may be to use redirection to a file, rather than piping: that should work just fine (as long as there is sufficient permissions to write to the specified output file).

Users who are running capable command shells are encouraged to scan the entire hard drive. For those who can easily copy and paste command lines, the following, more complex example command line may be a nice way to do that.

A more complex command line syntax

The following syntax shows this, although the following example may use some of the more advanced features of OpenBSD's ksh, which are features that may be found in bash and perhaps not some other shells. Note that the following does not output nearly as much to the screen.

{ sudo $SHELL -c "time ( mtree -c -K cksum,flags,gid,gname,link,md5digest,mode,rmd160\
digest,sha1digest,size,time,type,uid,uname -s 12345678901234567890123 -p / > /var/db/mtreedat/slashdir-$( date +%F%H%M%S%Z%a).dat ) " ; echo Err=$? ; } 2>&1 | tee -a /var/log/mtreedat/mtree-mkdat-$(date +%F%H%M%S%Z%a).log &

In that example, the program will appear to exit immediately. It is still running, but is running in the background because of the final ampersand, as discussed by the section about backgrounding tasks in Unix. To verify that it is running, one may run the “ jobs ” command.

To view progress in more detail, one might have success (or maybe not? This may need to be verified...) by running:

Older command:
tail -f /var/db/mtreedat/slashdir.dat
Newer command:
tail -f $( ls -t /var/log/mtreedat/mtree-mkdat-*.log | head -n 1 )

(Don't panic if it takes mtree multiple seconds, though probably a single-digit number, before output is shown.)

The tail command won't quit. A good indication of the above example being completed is if the error code is being shown. If output seems to be completed, press Ctrl-C to quit tail and then, if needed, re-run jobs to see if the backgrounded command is still running. (Upon being ready to show another command prompt, the shell may display a message about a background task that completes, so a manual check may not be needed.) Naturally, there may be other methods of checking whether the mtree is still running (by seeing what is running and piping the list of running programs through “ grep -i mtree ”).

Then run:

cat $( ls -t /var/log/mtreedat/mtree-mkdat-* | head -n 1 )
ls -l /var/db/mtreedat/slashdir.dat
Compare the system's state to a previously created database

To test a directory, remove some of the command line parameters and use the data file as an input file instead of an output file. Be sure to specify the correct database name (e.g. perhaps modify the following example(s) to use use slashdir.dat instead of stuffdir.dat).

Available command(s)

ksh may use the following (customizing the directory name and database file name as appropriate).

{ sudo time mtree -s 12345678901234567890123 -p /mydir < /var/db/mtreedat/stuffdir.dat ; echo Err=$? ; } 2>&1 | tee -a /var/log/mtreedat/mtree-latest-check-$(date +%F%H%M%S%Z%a).txt
Or, perhaps the following will be nicer:
{ sudo time mtree -s 12345678901234567890123 -p /mydir -f /var/db/mtreedat/stuffdir.dat ; echo Err=$? ; } 2>&1 | tee -a /var/log/mtreedat/mtree-latest-check-$(date +%F%H%M%S%Z%a).txt
Not working: progress reports

For those using OpenBSD's ksh or compatable-enough, and who have also installed the pipe viewer (“pv package, the following was attempted to show a progress report:

{ sudo $SHELL -c "time ( pv /var/db/mtreedat/stuffdir.dat | mtree -s 12345678901234567890123 -p /mydir ) " ; echo Err=$? ; } | tee -a /var/log/mtreedat/mtree-latest-check-$(date +%F%H%M%S%Z%a).log &

That didn't work, because the pv command finished pretty quick. Perhaps the mtree command accepted all of the input about later files, even before it got to testing some of the earlier files. As the mtree command's results are designed to show difference, there may be minimal output if the system has a small amount of changes. So, getting a progress report may not be the easiest thing to do.

(Some other shells will also likely be capable of the above. This guide does not currently have an exhaustive list of which shells those may be.)

Check the output for Err=. Read on for details about the result code.

Alternative syntax

Those using a different command shell which might not support the above syntax could settle for the following, if sufficient scrollback is enabled.

sudo time mtree -s 12345678901234567890123 -p /mydir < /var/db/mtreedat/mydir.dat
echo $?

If the error code is 0, there've been no changes. If the error code is 2, there have been changes. If the error code is 1, well, then the program encountered problems.

[#aide]: AIDE (“Advanced Intrusion Detection Environment”)
Overview

Web sites: http://aide.sf.net redirects to Older AIDE web site ( http://www.cs.tut.fi/~rammer/aide.html ) which references people to AIDE projects page at SF ( http://sf.net/projects/aide ) for the development files.

Newer instructions

At the moment of this writing, most of this section is a bit older than a guide at: AIDE guide (Checking files using AIDE)

Instructions

Note: These instructions may be assuming full access to the files: running as a superuser/root may be assumed. On one hand, it would be better if these instructions included sudo references where superuser/root is needed. On the other hand, if the files being tested are just a limited subdirectory, then perhaps superuser access isn't needed. Further testing may be needed to determine this. For now, the recommendation for simplicity, until this is analyzed further, is to just be root.)

Backing up

The following is meant to back up the original /etc/aide.conf as well as any existing AIDE databases (using default filenames) stored in either /etc/ or /var/db. It is quite likely that on a first run (or even subsequent runs with non-customized versions) that /var/db won't have any databases, do don't worry too much if there is an error about those files not existing.

Quick backup guide for files that are about to be changed
ls -td1 /origbak/ver* | head -n 1

(If the output of this command is “ls: /origbak/ver*: No such file or directory” then /origbak/ver1 may be used. If the output had said “/origbak/ver36” then we would be using the next higher number: “/origbak/ver37.”)

This example code will be using /origbak/etc/ver7/.

mkdir -p /origbak/ver7/etc /origbak/ver7/var/db
chmod go-rwx /origbak/ver7/etc /origbak/ver7 /origbak/ver7/var/db /origbak/ver7/var /origbak
cp -p /etc/aide* /var/db/aide.db* /origbak/ver7/etc/.
First run
[#praideck]: Some pre-checks for AIDE

Run:

aide -v

This will show some information including the program's version. One of the details it will give is the configuration file that is used. For example, one of the lines of the output may be something like:

CONFIG_FILE = "/etc/aide.conf"

Check out what configuration files currently exist. e.g.

ls -l /etc/aide*
Customize the config file

As a side note, some scripts have been added to AIDE configuratin scripts (collection 1). Using those may be an alternative approach to following the step-by-step instructions in this section. The advantage to following the step-by-step instructions is a chance of greater understanding of the process. However, those looking for a faster way to do things might benefit from another guide: AIDE Early Steps

[#aidermcf]: Removing existing rules (from configuration)

The behavior of the program will be determined by various lines in the configuration file. Before making a bunch of new rules, it may make sense to see what rules currently exist, and to comment out the rules if they aren't desired. To see any such commands if they exist, the following lines may show some existing rules (if the configuration filename is /etc/aide.conf).

grep -i ^/ /etc/aide.conf
grep -i ^= /etc/aide.conf
grep -i ^! /etc/aide.conf

It may be possible to remove those lines through “ grep -v ...” trickery, but it is probably easiest just to make the needed changes with a nice text editor.

Overview: AIDE configuration file

Then insert the desired lines.

Further details, about the configuration file's capabilities, are available in the AIDE manual.

Specifying filesystem objects

AIDE uses RegExs.

It seems a dollar sign effectively ends a string, so it is at the end of exact names (especially filenames used this in the manual's examples).

The manual notes, “It is generally a good idea to write the most general rules last.” The manual has some extensive details about pattern matching.

Brief overview of some common tasks
Checking a directory
=etc

This will cause etc and all of its files to be checked.

To sparsely check a directory, you may use something like:

DirRule = p+i+u+g
etc DirRule
Excluding

Start with a !

e.g.:

!/var/log/.*

This is specified as a RegEx, so the “.” (the period/dot) means a character. (It is similar to a ? at a command line.) So this will match any filename of at least one character.

Creating an example file

The below example may be used, or one may read further by checking a “manual” for AIDE from the project's web site. (This web manual is different than the man pages.)

The following backs up the original file, if such a backup has not already been made. If such a backup has been made, this will not overwrite the older backup of the original file. (Do this on a working file; ideally before making any changes to the file.)

[ -f /etc/aide.conf.orig ] || cp /etc/aide.conf /etc/aide.conf.orig
[#aideskpa]: Excluding file system objects from AIDE's checks

After that, this next step certainly has lots of room for optional customization: exclude filesystem objects where changes are expected.

The following is a list based on the section describing regularly changed filesystem objects in OpenBSD. (The example code in that section could also work with AIDE. However, it may not take advantage of AIDE's support for regex references. This support could, at least in theory, allow AIDE to dynamically notice newer filesystem objects that did not exist when the configuration file was made, if the newer filesystem object(s) fit a supported regex pattern. (Plus, far more trivially, the other section does not take advantage of AIDE's ability to end a line with a comment, so comments are on the other lines.)

[#aideskpb]: Starting uncustomized

The following line is designed to overwrite any earlier customizations, which can be great if wanting to apply a different set of customizations.

sudo cp /etc/aide.conf.orig /etc/aide.conf
[#aideskpc]: Initial lines to add to aide config

Some additional instructions, which should be followed, have been moved to a sub-page. Follow the steps at: #aideskpc: Initial lines to add to aide config.

[#aideskpd]: Skip many known issues

At this point, follow one of the following steps:

[#aideskpe]: Updating the AIDE configuration

This looks slightly more up to date, as it has a reference to mtreedat and /var/mail. Therefore, this is the recommended method (for the time being), instead of #aideskpf.

echo \# !!!!!!!!!! Exclusions follow !!!!!!!!!! | sudo -n tee -a /etc/aide.conf
echo | sudo -n tee -a /etc/aide.conf

echo \# Data from File Integrity Checker\(s\) | sudo -n tee -a /etc/aide.conf
echo \# Another file integrity checker database\; maybe updated more recently than | sudo -n tee -a /etc/aide.conf
echo \# this check | sudo -n tee -a /etc/aide.conf
echo !/var/db/integrit/known.cdb | sudo -n tee -a /etc/aide.conf
echo !/var/db/integrit/current.cdb.new | sudo -n tee -a /etc/aide.conf
echo !/var/logs/integrit/* | sudo -n tee -a /etc/aide.conf
echo !/var/db/mtreedat/.\* \# Another file integrity checker database\; maybe updated more recently than this check | sudo -n tee -a /etc/aide.conf
echo !/var/db/aide_old/.\* \# Older files from AIDE file integrity checker\; maybe updated more recently | sudo -n tee -a /etc/aide.conf
echo # than this check | sudo -n tee -a /etc/aide.conf
echo !/var/db/aide.db \# A database file | sudo -n tee -a /etc/aide.conf
echo !/var/db/aide.db.new \# A database file | sudo -n tee -a /etc/aide.conf
echo !/var/db/aide/olddbs/aide.db.old* \# Archived older database file\s\) | sudo -n tee -a /etc/aide.conf
echo \# Known device filesystem objects | sudo -n tee -a /etc/aide.conf
echo \# Exclude Known TTY ports that tend to always look changed | sudo -n tee -a /etc/aide.conf

The list of commands to run has been split up, because trying to paste them all in at once seemed to lead to a lockup (easily broken out of by using Ctrl-C). So, it might be a bit safer to try to just paste in a smaller number of commands at a time.

After running the prior commands, run the following commands:

for y in $( sudo grep -i tty /etc/ttys | grep -v ^\# | cut -f 1 | xargs ) ; do echo !/dev/${y} ; done | sudo -n tee -a /etc/aide.conf
echo \# Other device objects ports that tend to always look changed | sudo -n tee -a /etc/aide.conf
for y in wsmouse wsmouse0 wskbd wskbd0 console null arandom log ; do echo !/dev/$y ; done | sudo -n tee -a /etc/aide.conf
echo !/dev/bpf\* | sudo -n tee -a /etc/aide.conf
echo \# Files that regularly change when processes start/end running | sudo -n tee -a /etc/aide.conf
echo !/var/run/utmp | sudo -n tee -a /etc/aide.conf
echo !/var/run/\*.pid | sudo -n tee -a /etc/aide.conf
echo \# Log Files | sudo -n tee -a /etc/aide.conf

After running the prior commands, run the following commands:

for z in authlog daemon lastlog maillog messages secure wtmp ; do for y in /var/log/$z /var/log/$z.?.gz ; do echo !$y ; done ; done | sudo -n tee -a /etc/aide.conf
echo !/var/cron/log \# Log file that may be updated extremely frequently | sudo -n tee -a /etc/aide.conf
echo \# Mailboxes | sudo -n tee -a /etc/aide.conf
echo !/var/mail/.* | sudo -n tee -a /etc/aide.conf
echo \# Maybe OpenBSD specific files | sudo -n tee -a /etc/aide.conf
for z in daily.out security.out daily.out.old security.out.old ; do for y in /var/log/$z ; do echo !${y} ; done ; done | sudo -n tee -a /etc/aide.conf

After running the prior commands, run the following commands:

echo !/var/log/sendmail.st | sudo -n tee -a /etc/aide.conf
echo \# Filesys objects that change on each reboot | sudo -n tee -a /etc/aide.conf
echo !/var/run/dmesg.boot | sudo -n tee -a /etc/aide.conf
echo !/fastboot | sudo -n tee -a /etc/aide.conf
echo !/etc/resolv.conf | sudo -n tee -a /etc/aide.conf
echo !/var/cron/tabs/.sock | sudo -n tee -a /etc/aide.conf
echo !/var/cron/tabs/.sock | sudo -n tee -a /etc/aide.conf
echo !/var/empty/dev/log | sudo -n tee -a /etc/aide.conf
echo !/dev/tty[C]cfg \# may change each reboot | sudo -n tee -a /etc/aide.conf

After running the prior commands, run the following commands:

for y in /dev/tty[C]cfg ; do echo !${y} ; done | sudo -n tee -a /etc/aide.conf
echo \# Databases routinely changed | sudo -n tee -a /etc/aide.conf
echo !/var/db/ntpd.drift | sudo -n tee -a /etc/aide.conf
echo !/var/db/dhclient.leases.* | sudo -n tee -a /etc/aide.conf
echo !/var/run/dev.db | sudo -n tee -a /etc/aide.conf
echo !/var/run/ld.so.hints | sudo -n tee -a /etc/aide.conf
echo \# Directories related to mail \(may change when mail is sent/recv\) | sudo -n tee -a /etc/aide.conf
echo !/var/spool/mqueue | sudo -n tee -a /etc/aide.conf
echo !/var/spool/clientmqueue | sudo -n tee -a /etc/aide.conf
echo /var/mail$ DirNotContentsNorTime | sudo -n tee -a /etc/aide.conf
[#aideskpf]: Alternative method to create this portion of the configuration

perhaps just set fintckcf to /etc/aide.conf and then access “Part one” (which is from the Making a configuration file to exclude items section). However, this seems a bit less detailed than the example configuration that directly made an /etc/aide.conf. That might mean the more general configuration is out of date, or that the AIDE configuration was simply more capable of excluding items that are known to be non-issues. Either way, it is currently recommended to use #aideskpe rather than this.

[#aideskpg]: More lines to add to aide config

Then, there is some more configuration to add:

echo \# Directories with regularly changing contents | sudo -n tee -a /etc/aide.conf
echo /etc$ DirNotContents | sudo -n tee -a /etc/aide.conf

After running the prior commands, run the following commands:

echo /var/db/aide/olddbs$ DirNotContentsNorTime | sudo -n tee -a /etc/aide.conf
echo /var/db$ DirNotContentsNorTime | sudo -n tee -a /etc/aide.conf

After running the prior commands, run the following commands:

echo /var/empty/dev$ DirNotContentsNorTime | sudo -n tee -a /etc/aide.conf
echo /var/empty$ DirNotContentsNorTime | sudo -n tee -a /etc/aide.conf
echo /var/log$ DirNotContentsNorTime | sudo -n tee -a /etc/aide.conf
echo /var/cron$ DirNotContentsNorTime | sudo -n tee -a /etc/aide.conf
echo /var/tmp$ DirNotContentsNorTime | sudo -n tee -a /etc/aide.conf
echo /var/backups$ DirNotContentsNorTime | sudo -n tee -a /etc/aide.conf
echo /var$ DirNotContentsNorTime | sudo -n tee -a /etc/aide.conf
echo /tmp$ DirNotContentsNorTime \# Probably lots here is likely to change without mattering. | sudo -n tee -a /etc/aide.conf
echo /dev$ DirNotContentsNorTime \# /dev/tty?cfg may change each reboot | sudo -n tee -a /etc/aide.conf
echo # Hopefully /tmp is cleared automatically at least each reboot. | sudo -n tee -a /etc/aide.conf
echo \# Directories that may regularly change when system reboots | sudo -n tee -a /etc/aide.conf
echo /var/cron/atjobs$ DirNotContentsNorTime | sudo -n tee -a /etc/aide.conf

After running the prior commands, run the following commands:

echo /var/cron/tabs$ DirNotContentsNorTime | sudo -n tee -a /etc/aide.conf
echo /var/empty/dev$ DirNotContentsNorTime | sudo -n tee -a /etc/aide.conf
echo /var/run$ DirNotContentsNorTime | sudo -n tee -a /etc/aide.conf
echo /var/www/dev$ DirNotContentsNorTime | sudo -n tee -a /etc/aide.conf
echo \# Backup files \(should be routinely updated\) | sudo -n tee -a /etc/aide.conf
echo !/origbak$ | sudo -n tee -a /etc/aide.conf
echo \# Backup files \(possibly OpenBSD-specific\) | sudo -n tee -a /etc/aide.conf
echo !/var/backups/*.current | sudo -n tee -a /etc/aide.conf
echo !/var/backups/*.backup | sudo -n tee -a /etc/aide.conf
echo !/var/backups/*.backup.md5 | sudo -n tee -a /etc/aide.conf
echo \# Disks \(OpenBSD device names\) | sudo -n tee -a /etc/aide.conf

After running the prior commands, run the following commands:

for y in /dev/r[sw]d[0-9][a-z] ; do echo !$y ; done | sudo -n tee -a /etc/aide.conf
echo \# Other/Misc files where changes have regularly been witnessed upon reboot \(possibly OpenBSD-specific\) | sudo -n tee -a /etc/aide.conf
echo !/etc/resolv.conf.save | sudo -n tee -a /etc/aide.conf
echo !/var/www/dev/log | sudo -n tee -a /etc/aide.conf
echo \(some older config text below\)
echo !/dev/[pt]typ[0-9]* \# TTY console devices | sudo -n tee -a /etc/aide.conf
echo !/var/log/.* \# Log files | sudo -n tee -a /etc/aide.conf
echo !/var/db/.* \# databases | sudo -n tee -a /etc/aide.conf
echo !/var/run/.* \# PID files are likely change on each reboot. | sudo -n tee -a /etc/aide.conf
echo \# However, one may want to make sure PIDs don\'t change at unexpected times. | sudo -n tee -a /etc/aide.conf
echo \# Therefore, the above line might not be most desirable. | sudo -n tee -a /etc/aide.conf
echo !/var/spool/.* \# These files are probably likely to change. | sudo -n tee -a /etc/aide.conf
echo !/proc/.* \# If /proc exists and is used, it will likely change | sudo -n tee -a /etc/aide.conf

Check the filesystem volume items that software is regularly modifying and make sure all such exclusions are in the configuration file.

[#aideckra]: What to scan

Be sure to check out the information from the short AIDE configuration: what to scan section. (That was placed in a separate section for simplicity, because that section is referred to by multiple places.)

[#aideckrb]: Additional recommended changes

Wrapping up the configurationf ile changes is: AIDE configuration: Additional recommended changes.

  • [#aideckrc]: (There also used to be a hyperlink to some information in this section. That information started to be referred to by multiple guides, and so the information was moved to simplify hyperlinking in the long term. The information which was at #aideckrc on this page is now at: AIDE configuration: Additional recommended changes.)
[#aideckre]: Output

(This section about customizing output may generally be skipped, but is mentioned here as an item that, if handled, is best to take care of when editing the configuration file. (At the time of this writing, this hasn't been tested by the author of this text. This text is being written based on AIDE's documentation.))

When AIDE is creating a database, the output goes to a location specified in the report_url line of the configuration file. Output may be sent in multiple directions: by default the output is sent to stdout. (The value of “stdout” may be used as a URL, without needing any sort of protocol handler reference like “file:”. So, the term “URL”, as used by this program, doesn't strictly imply a website or a similar syntax.) Multiple destinations for this output may be specified, and then output will go to all of them.

If the output is sent to standard output, then the output may be redirected (e.g. redirected to a file or piped to a program like tee), or a config file may instead specify a location (possibly starting with “file:”) and in that case the output file may simply be viewed later, such as once the program exits, without the need for any redirection. That's great for one-time use. If there is a desire to always output to the same log file every time, then that sort of preference can be accomplished using something like tee but it could also be accomplished by simply specifying multiple report_url lines.

Then, make sure that the configuration file is valid, by running the following:

time aide -D
echo $?

The desired results of the second command would be a single digit: the number zero (“0”). (The output of the first command will be quite a bit more verbose.)

[#aideinlz]: Initializing

If AIDE has been used before, rotate out the existing database files (following the first steps provided in the section about updating database files). This file rotation isn't something that needs to be done on a system that has never run AIDE before.

{ sudo time aide -i -V231 ; echo Err=$? ; } 2>&1 | tee -a /tmp/aideinit.txt

That process may take minutes (perhaps more than 20 on a freshly installed operating system; this could definitely at least a timespan of hours depending on how much data there is.)

Example output:

AIDE, version 0.15.1

### AIDE database at /var/db/aide.db.new initialized.

     1035.84 real       863.12 user       130.48 sys
Err=0

(The last two lines of the example output are based on a provided example command line, and do not come directly from the AIDE program. The example shown adds up to 2,029.44 seconds, which is 33 minutes 49.44 seconds.)

Once the new database is made, it will have a name specified by the “database_out=” line in the configuration file that is being used. (The configuration file used is likely to be the /etc/aide.conf file.)

grep -i database_out /etc/aide.conf
Post initializing
So that the database may be easily checked and/or updated, go ahead and perform a rotation of the AIDE database files as described in this documentation.
[#aidertat]: Rotating the database files
Abbreviated method

Those using default directories may run:

export AIDEDBFL=/var/db
export AIDEOUTF=/var/db

The following steps will perform rotation of old files, if the situation would recommend doing so. The steps will also rotate out the new files, as recommended. To do this, those using OpenBSD's ksh, or another shell that is compatible enough, may run the following:

echo Rotate out old files...
[ -f $AIDEOUTF/aide.db.new ] && [ -f $AIDEOUTF/aide.db ] && ( sudo mkdir -p ${AIDEDBFL}/aide/olddbs && sudo mv ${AIDEDBFL}/aide.db $AIDEDBFL/aide/olddbs/aide.db.old$( date +%F%H%M%S%Z%a) )
echo Rotate out new files...
[ -f $AIDEOUTF/aide.db.new ] && sudo mv $AIDEOUTF/aide.db.new $AIDEDBFL/aide.db
Overview: further details

This goes into more details about what the rotation command does. This also has some details that might help if using a shell which is not quite able to handle some of the syntax of the abbreviated method.

[#aideosvr]: Check/set variables

Run:

echo AIDEDBFL=$AIDEDBFL and AIDEOUTF=$AIDEOUTF

Those variables were custom-made for this documentation, and are not a part of the AIDE package. So, if these directions are being followed for the first time, the values are not likely to be pre-set. Though the AIDE program does not strictly require making environment variables, these directions do. The simple reason for this requirement is just so that subsequent directions can use those environment variables, and that simplifies some of the later instructions. (The logical alternative that wouldn't require these variables may end up showing the same command multiple times, and that ends up unnecessarily complicating things). Using the variables lets the instructions be easier and more simplistic, while being accurate on varying implementations.

So, these these variables should be set before following the directions that use these variables. If either of the environment variables are unset, then the recommendation is to set them, using the following guidelines. (The alternative would be to adjust other portions of the instructions as appropriate, such as modifying the example command lines more.)

Find the location of the database file paths that will be used by the configuration. The aide man page suggests the databases are in “/etc/”, but in an OpenBSD package this had been seen to default to “/var/db/”. To find this out, run something like the following, making sure to appropriately reference the actual file that the CONFIG_FILE is set to:

grep database /etc/aide.conf | grep -v \#

Some example possible output:

database=file:/var/db/aide.db
database_out=file:/var/db/aide.db.new

To use different database files, either edit the configuration file that generally gets used, or create a new configuration file and specify the alternate configuration file on the command line.

The variables should be set to for the locations of these files. So, go ahead and set environment variables, such as the following examples:

export AIDEDBFL=/var/db
export AIDEOUTF=/var/db

The idea is that these variables are set to the locations of the input file (specified by the database= line) and the output file (specific by the database_out= line), which are both specified in the configuration file that is shown on the CONFIG_FILE line when running the “aide -v ” command.

Rotate out old data

It might be worthwhile to remove/delete files that are too old. This guide is more simplistic, and assumes that all old files are desired. That is probably the way to go, for simplicity, at least when starting out.

Check if there is both a newly created database and an older database. To do this, make sure the OS environment variables for AIDE, coming from this documentation, are set, and then run:

ls -lt $AIDEOUTF/aide.db.new $AIDEDBFL/aide.db

See if the output of the command shows two files.

If there is not both a newly created database and also an older database file:

If there is not an old file and/or if there is not a new database, then one may skip the rest of this section about rotating out old data. (Resume these instructions starting at the section about rotating out the new file.)

If there is an old database and a current database

If so, then note that the old file is probably going to be getting replaced by a new file. If there is a desire to keep that database around, which very well may be a good idea, then back it up.

If $AIDEDBFL is /var/db, which is quite possible, then it makes some sense to not pollute the $AIDEDBFL directory too badly with too many entries related to AIDE. So, the first step to backing up the file is to make a location where the backups will be stored.

sudo mkdir -p $AIDEDBFL/aide/olddbs

Then, perform one of the following:

Preferred method

This preferred method requires a sufficient shell. This method is considered to be easier because it involves fewer steps.

sudo mv $AIDEDBFL/aide.db $AIDEDBFL/aide/olddbs/aide.db.old$( date +%F%H%M%S%Z%a)

Note: This creates a filename that includes a reference to the moment of time when the file was rotated (backed up). The timestamp does not necessarily correspond to the time when the contents of the file were last modified. The reason, why the time of backup is being used, is simply because that time is relatively easy (easier?) to use.

Other methods

(Note: run commands as superuser/root as needed to handle permissions issues.)

Manual

To determine a name for the new file, see what already exists.

ls -lt $AIDEDBFL/aide/olddbs/aide.db.old* | head -n 1

That will show the latest old database file. If there is no such file existing, the following may back up the database file:

mv $AIDEDBFL/aide.db.old $AIDEDBFL/aide/olddbs/aide.db.old1

On the other hand, if the command listing the latest file does show evidence that there were two prior files, perhaps the following would be more appropriate:

mv $AIDEDBFL/aide.db.old $AIDEDBFL/aide/olddbs/aide.db.old3
Other ideas

Perhaps there are other slick methods, such as using awk or using a file rotation program (depending on the operating system, that might be newsyslog or logrotate)?

Rotate out the new file

(This is generally a recommended process to go ahead and follow. Do proceed with this, even after performing previous recommended steps like rotating out old data.)

See if there is a “new” database file.

ls -l $AIDEOUTF/aide.db.new*

If so, that file is about to be overwritten. Determine whether that file can be trusted as being authentic, rather than something that was created by somebody tampering with the files. If the “new” file is valid/genuine then, in the future, using that file as the trusted “old” file will make sense to do.

Before renaming the new file (to a filename that will be recognized as being the official “current” file), make sure any previous (already existing) “old” file has been sufficiently backed up and/or moved out of the way, if desired. Otherwise, the contents of the old file may be forever lost. If backing up an old file was just done (or if the database was just initialized, in which case there may not have been an older file), then it should generally be safe to rename this file.

sudo mv $AIDEOUTF/aide.db.new $AIDEDBFL/aide.db
[#aideupdb]: Compare and update AIDE databases
Rotating
Follow the steps for performing a rotation of the AIDE database files, as described in this documentation.
Variables
One key purpose of following the provided instructions for file rotating is that those instructions involve checking that the operating system environment variables called AIDEDBFL and AIDEOUTF are both set appropriately. (They should be set properly if following all of the guidance in, and/or referred to by, the section about rotating the AIDE database files). Even if, for whatever reason, there is a desire not to rotate the files, do get those environment variables set so they are available for the rest of this documentation.
Creating the updated database file
Overview/plan

If there is a file with the name specified for a new database file, it will be overwritten if proceeding. The previous command should have taken care of that, but if that step was skipped, the recommended action is to either rename the existing file that has the same name as the upcoming output file, or change the configuration file.

Once all of the earlier files are in the proper state, go ahead, then, and make a new update file:

The goal is to run “aide -u ”, and optionally (and recommended by this guide) to also use the “ -V231” parameter for some sort of progress reporting, and check the error code. Ideally the results are piped into a file.

Required initial step

The following command is not required for AIDE, but is required for the upcoming example command lines.

sudo mkdir /var/log/aide
sudo chmod g+w /var/log/aide
Performing the database update

Run what is shown by one of the following sections:

Preferred syntax

This can be accomplished with OpenBSD's ksh and similar/compatible shells by using:

{ sudo time aide -u -V231 ; echo Err=$? ; } 2>&1 | sudo -n tee -a $( mktemp /var/log/aide/aide-latest-check-$(date +%F%H%M%S%Z%a)-XXXXXXXXXX${RANDOM}.txt )
Other shells

If the shell does not support that preferred syntax, one could just run the following:

time aide -u -V231 | tee /var/log/aide/aide-latest-check.txt

This does not capture the error code. (Trying to print the exit code after piping the results to tee will just show the exit code of the tee command.)

[#aidertnc]: Return code

Note: When updating the database, a non-zero result may be common. For example, a return code of 5 represents 4+1, where 4 means there are changed files and 1 means there are new files. The meanings behind the values of 4 and 1 are discussed in the “Diagnostics” section of the manual page. (See Aide manual: Heading #6 for an HTML version, or, for non-HTML viewing, see AIDE source code: documentation files for the file that creates the official manual page).

Afterwards, if everything is good, and the new file is the one that should be used for future updates, then it may be a good idea to rotate AIDE's database files. That might help to sufficiently rename the file so that it eventually gets backed up, rather than being overwritten. Rotating the database files will also have the new file all ready to be used as a current database. If all that sounds good, then rotate the data files.

There is probably no compelling reason to not rotate the database: the primary usefulness of the old file is to generate a report which has now been generated. (If the file containing the report has been confirmed to contain readable/useful information, then the ability to view the report will not go away just because the old database exists.) If the new file is actually not desirable to be used as the current database to refer to when the next report is generated, then either delete the new database file or, if it is still possibly desired, copy/move it to another location (so that the new file isn't accientally overwritten if another update occurs).

[#aiderpvw]: Seeing the report
See: Viewing AIDE's report.
Comparing AIDE databases without updating databases

(This is not necessarily a recommended method of doing things. However, this method is a possible alternative, so this method is being documented here.)

Naturally, first make sure that the desired database files will be used. (To do this, check the configuration file that will be used. If the desired files are not the ones specified, either adjust the files, or specify a different configuration file if needed.)

Either update (which will compare files) and then delete the output file which isn't desired, or avoid creating the unneeded output file by using:

time aide -C -V231

In theroy, an advantage to comparing is more tell-tale exit codes (which may ease automation): The “DIAGNOSTICS” section of the man page has this to say: “Normally, the exit status is 0 if no errors occurred. Except when the --check command was requested, in which case the exit status is” given a value that indicates whether any files have been added/removed/changed. That is the theory. In reality, an actual update command has been known to return a value of 5, so that isn't really an advantage.

So, there's really little advantage to not creating an updated database file while comparing. Perhaps it could be a bit faster: if changed files didn't report each method that was changed, then multiple comparison methods wouldn't be needed as soon as once change was discovered.

Further information
[#aidecdsn]: Database and config signing

AIDE provides some support for signing data files. This guide, however, does not really perform that function. (Reading the AIDE manual's section on “Database and config signing”, using these features involves using “ ./configure ”, presumably run from the directory with AIDE's source code. If the key needs to be created at runtime, that causes database/config signing to be far less convenient for those who may use pre-built binary executables (e.g. OpenBSD's packaging system). Although security may seem like a poor thing to trade away for convenience, if someone can modify a configuration file illegitimately than the person might also be able to modify an executable file, and the configuration file may easily be included in the data that gets regularly “file integrity” checked. So, the payoff for the inconvenience seems... rather minor.)

Note that the example config file may define a variable called “All” which doesn't include all possible tests; the man page on aide.conf describes all the possible values including ones that may not be supported by all systems.

[#integrit]: Integrit
This was recently re-tabbed. The entire Integrit section should be de-tabbed by one tab's worth.
Overview

Similar to AIDE. Integrit is for Unix-like operating systems and is GPLv2. The HTML version of the Integrit home page refers to a page that offers an HTML and a PostScript version as “the integrit homepage”.

The man page may be a bit sparse. Locate the file called “README”. For instance, in OpenBSD, “ pkg_info -L integrit ” will show a filename of /usr/local/share/doc/integrit/README. Commands include “ integrit ”, “ i-ls ”, and “ i-viewdb”.

This may seem to go blazingly fast compared to AIDE. A key reason for this may be that configuration files may be different: AIDE may be performing a heavier amount of work, including multiple different hashing methods like SHA-256. In contrast, Integrit (by default) may be doing minimal work, and with a reduced chance of noticing errors. (AIDE could be sped up by making a configuration file that involves AIDE doing less work.)

Instructions
First run
Pre-check(s)
Version: integrit -V
Customizing configuration file(s)

First, it may be interesting to see the example configuration files (which may be more commented)

ls -l /etc/integrit/*

Note: (The following section may not be very complete yet.

Make a configuration file. The Integrit documentation on Config rule prefixes may help. Here is a simple start to creating a configuration file.

export fintckcf=/etc/integrit/mycustom.cnf
echo root=/ | sudo -n tee -a ${fintckcf}
echo known=/var/db/integrit/known.cdb | sudo -n tee -a ${fintckcf}
echo current=/var/db/integrit/current.cdb.new | sudo -n tee -a ${fintckcf}
sudo mkdir -p $( dirname ${fintckcf} ) /var/db/integrit

Then, run the commands from: part one of Posix regularly changed files: configuration building.

Then...

(The following section may be based on an AIDE configuration, and so may need some additional customization before it is really ready for Integrit.)
echo \# Directories with regularly changing contents | sudo -n tee -a ${fintckcf}
echo DirNotContents = p+i+u+g+m+a+c | sudo -n tee -a ${fintckcf}
echo /var/db DirNotContents | sudo -n tee -a ${fintckcf}
echo /var/mail DirNotContents | sudo -n tee -a ${fintckcf}
echo /var/empty DirNotContents | sudo -n tee -a ${fintckcf}
echo /var/log DirNotContents | sudo -n tee -a ${fintckcf}
echo /var/tmp DirNotContents | sudo -n tee -a ${fintckcf}

Then go do part 3... part three of Posix regularly changed files: configuration building.

Then... The following syntax is for directories, and so may need to be reviewed/revised... (Otherwise, skip this section for now.)

echo # Log file that may be updated extremely frequently | sudo -n tee -a ${fintckcf}
echo !/var/cron/log | sudo -n tee -a ${fintckcf}
echo # Log files | sudo -n tee -a ${fintckcf}
echo !/var/log | sudo -n tee -a ${fintckcf}
echo # databases | sudo -n tee -a ${fintckcf}
echo !/var/db | sudo -n tee -a ${fintckcf}
echo # PID files are likely change on each reboot. | sudo -n tee -a ${fintckcf}
echo !/var/run | sudo -n tee -a ${fintckcf}
echo \# However, one may want to make sure PIDs don\'t change at unexpected times. | sudo -n tee -a ${fintckcf}
echo \# Therefore, the above line might not be most desirable. | sudo -n tee -a ${fintckcf}
echo # These files are probably likely to change. | sudo -n tee -a ${fintckcf}
echo !/var/spool/ | sudo -n tee -a ${fintckcf}
echo # If /proc exists and is used, it will likely change | sudo -n tee -a ${fintckcf}
echo !/proc/ | sudo -n tee -a ${fintckcf}
echo # Probably lots here is likely to change without mattering. | sudo -n tee -a ${fintckcf}
echo !/tmp/ | sudo -n tee -a ${fintckcf}

(The following are some lines from an AIDE configuration file. They may be able to be adapted fairly well to work with Integrit as well.)

echo /var/mail$ DirNotContentsNorTime | sudo -n tee -a ${fintckcf}
echo \# Directories with regularly changing contents | sudo -n tee -a ${fintckcf}
echo /etc$ DirNotContents | sudo -n tee -a ${fintckcf}
echo /var/db/aide/olddbs$ DirNotContentsNorTime | sudo -n tee -a ${fintckcf}
echo /var/db$ DirNotContentsNorTime | sudo -n tee -a ${fintckcf}
echo /var/empty/dev$ DirNotContentsNorTime | sudo -n tee -a ${fintckcf}
echo /var/empty$ DirNotContentsNorTime | sudo -n tee -a ${fintckcf}
echo /var/log$ DirNotContentsNorTime | sudo -n tee -a ${fintckcf}
echo /var/cron$ DirNotContentsNorTime | sudo -n tee -a ${fintckcf}
echo /var/tmp$ DirNotContentsNorTime | sudo -n tee -a ${fintckcf}
echo /var/backups$ DirNotContentsNorTime | sudo -n tee -a ${fintckcf}
echo /var$ DirNotContentsNorTime | sudo -n tee -a ${fintckcf}
echo \# Probably lots here is likely to change without mattering. | sudo -n tee -a ${fintckcf}
echo /tmp$ DirNotContentsNorTime | sudo -n tee -a ${fintckcf}
echo \# /dev/tty?cfg may change each reboot | sudo -n tee -a ${fintckcf}
echo \# /dev/tty?cfg may change each reboot | sudo -n tee -a ${fintckcf}
echo # Hopefully /tmp is cleared automatically at least each reboot. | sudo -n tee -a ${fintckcf}
echo \# Directories that may regularly change when system reboots | sudo -n tee -a ${fintckcf}
echo /var/cron/atjobs$ DirNotContentsNorTime | sudo -n tee -a ${fintckcf}
echo /var/cron/tabs$ DirNotContentsNorTime | sudo -n tee -a ${fintckcf}
echo /var/empty/dev$ DirNotContentsNorTime | sudo -n tee -a ${fintckcf}
echo /var/run$ DirNotContentsNorTime | sudo -n tee -a ${fintckcf}
echo /var/www/dev$ DirNotContentsNorTime | sudo -n tee -a ${fintckcf}
echo \# Backup files \(should be routinely updated\) | sudo -n tee -a ${fintckcf}
echo !/origbak$ | sudo -n tee -a ${fintckcf}
echo \# Backup files \(possibly OpenBSD-specific\) | sudo -n tee -a ${fintckcf}
echo !/var/backups/*.current | sudo -n tee -a ${fintckcf}
echo !/var/backups/*.backup | sudo -n tee -a ${fintckcf}
echo !/var/backups/*.backup.md5 | sudo -n tee -a ${fintckcf}
echo \# Disks \(OpenBSD device names\) | sudo -n tee -a ${fintckcf}
for y in /dev/r[sw]d[0-9][a-z] ; do echo !$y ; done | sudo -n tee -a ${fintckcf}
echo \# Other/Misc files where changes have regularly been witnessed upon reboot \(possibly OpenBSD-specific\) | sudo -n tee -a ${fintckcf}
echo !/etc/resolv.conf.save | sudo -n tee -a ${fintckcf}
echo !/var/www/dev/log | sudo -n tee -a ${fintckcf}
echo \(some older config text below\)
echo !/dev/[pt]typ[0-9]* \# TTY console devices | sudo -n tee -a ${fintckcf}
echo !/var/log/.* \# Log files | sudo -n tee -a ${fintckcf}
echo !/var/db/.* \# databases | sudo -n tee -a ${fintckcf}
echo !/var/run/.* \# PID files are likely change on each reboot. | sudo -n tee -a ${fintckcf}
echo \# However, one may want to make sure PIDs don\'t change at unexpected times. | sudo -n tee -a ${fintckcf}
echo \# Therefore, the above line might not be most desirable. | sudo -n tee -a ${fintckcf}
echo !/var/spool/.* \# These files are probably likely to change. | sudo -n tee -a ${fintckcf}
echo !/proc/.* \# If /proc exists and is used, it will likely change | sudo -n tee -a ${fintckcf}


echo \# Starting off with exclusions | sudo -n tee -a ${fintckcf}
echo \# Data from File Integrity Checker\(s\) | sudo -n tee -a ${fintckcf}
echo \# Data files from Integrit\; in case updated more recently than | sudo -n tee -a ${fintckcf}
echo \# this check | sudo -n tee -a ${fintckcf}
echo !/var/db/integrit/known.cdb | sudo -n tee -a ${fintckcf}
echo !/var/db/integrit/current.cdb.new | sudo -n tee -a ${fintckcf}
echo # mtree may be used as another file integrity checker database\; maybe updated more recently than this check | sudo -n tee -a ${fintckcf}
echo !/var/db/mtreedat/.\* | sudo -n tee -a ${fintckcf}
echo # AIDE may be used as another file integrity checker database\; maybe updated more recently | sudo -n tee -a ${fintckcf}
echo # than this check | sudo -n tee -a ${fintckcf}
echo !/var/db/aide_old/.\* | sudo -n tee -a ${fintckcf}
echo !/var/db/aide.db | sudo -n tee -a ${fintckcf}
echo !/var/db/aide.db.new | sudo -n tee -a ${fintckcf}
echo \# Known device filesystem objects | sudo -n tee -a ${fintckcf}
echo \# Exclude Known TTY ports that tend to always look changed | sudo -n tee -a ${fintckcf}
for y in $( sudo grep -i tty /etc/ttys | grep -v ^\# | cut -f 1 | xargs ) ; do echo !/dev/${y} ; done | sudo -n tee -a ${fintckcf}
echo \# Other device objects ports that tend to always look changed | sudo -n tee -a ${fintckcf}
for y in wsmouse wsmouse0 wskbd wskbd0 console null arandom log ; do echo !/dev/$y ; done | sudo -n tee -a ${fintckcf}
echo !/dev/bpf\* | sudo -n tee -a ${fintckcf}
echo \# Files that regularly change when processes start/end running | sudo -n tee -a ${fintckcf}
echo !/var/run/utmp | sudo -n tee -a ${fintckcf}
echo !/var/run/\*.pid | sudo -n tee -a ${fintckcf}
echo \# Log Files | sudo -n tee -a ${fintckcf}
for z in authlog daemon lastlog maillog messages secure wtmp ; do for y in /var/log/$z /var/log/$z.?.gz ; do echo !$y ; done ; done | sudo -n tee -a ${fintckcf}
echo \# Maybe OpenBSD specific files | sudo -n tee -a ${fintckcf}
for z in daily.out security.out daily.out.old security.out.old ; do for y in /var/log/$z ; do echo !${y} ; done ; done | sudo -n tee -a ${fintckcf}
echo !/var/cron/log | sudo -n tee -a ${fintckcf}
echo !/var/empty/dev/log | sudo -n tee -a ${fintckcf}
echo !/var/log/sendmail.st | sudo -n tee -a ${fintckcf}
echo \# Files that change on each reboot | sudo -n tee -a ${fintckcf}
echo !/var/run/dmesg.boot | sudo -n tee -a ${fintckcf}
echo !/fastboot | sudo -n tee -a ${fintckcf}
echo \# Databases routinely changed | sudo -n tee -a ${fintckcf}
echo !/var/db/ntpd.drift | sudo -n tee -a ${fintckcf}
echo !/var/db/dhclient.leases.* | sudo -n tee -a ${fintckcf}
echo !/var/spool/mqueue | sudo -n tee -a ${fintckcf}
echo !/var/spool/clientmqueue | sudo -n tee -a ${fintckcf}
echo !/var/run/dev.db | sudo -n tee -a ${fintckcf}
echo !/var/run/ld.so.hints | sudo -n tee -a ${fintckcf}

Check the filesystem volume items that software is regularly modifying and make sure all such exclusions are in the configuration file.

sudo mkdir -p /var/db/integrit /etc/integrit
echo \# Exclude other file integrity checkers\; Their data may be routinely routed | sudo tee -a /etc/integrit/mycustom.cnf
echo \# more recently than the last rotatation of Integrit\'s data | sudo tee -a /etc/integrit/mycustom.cnf
echo !/var/db/mtreedat | sudo tee -a /etc/integrit/mycustom.cnf
echo !/var/db/aide | sudo tee -a /etc/integrit/mycustom.cnf
echo !/var/db/aide_old | sudo tee -a /etc/integrit/mycustom.cnf
echo !/var/db/aide.db | sudo tee -a /etc/integrit/mycustom.cnf
echo !/var/db/aide.db.new | sudo tee -a /etc/integrit/mycustom.cnf
echo \# Exclude devices that always/regularly look changed | sudo tee -a /etc/integrit/mycustom.cnf
echo root=/ | sudo tee -a /etc/integrit/mycustom.cnf
echo known=/var/db/integrit/known.cdb | sudo tee -a /etc/integrit/mycustom.cnf
echo current=/var/db/integrit/current.cdb.new | sudo tee -a /etc/integrit/mycustom.cnf
echo | sudo tee -a /etc/integrit/mycustom.cnf
echo \# Check lots: checksum, inode, perms, num-of-links, uid, gid, | sudo tee -a /etc/integrit/mycustom.cnf
echo \# and file times \(access time, modification time, time of file info changing\) | sudo tee -a /etc/integrit/mycustom.cnf
echo \# Access time is disabled \(since the letter is capitalized\) as access | sudo tee -a /etc/integrit/mycustom.cnf
echo \# times may routinely change so frequently that reporting such changes | sudo tee -a /etc/integrit/mycustom.cnf
echo \# typically/often creates results that are not useful/easy to manually process.\) | sudo tee -a /etc/integrit/mycustom.cnf
echo / siplugzAmc | sudo tee -a /etc/integrit/mycustom.cnf
echo | sudo tee -a /etc/integrit/mycustom.cnf
echo \# Except, turn off some checks by referencing capital letters | sudo tee -a /etc/integrit/mycustom.cnf
echo =/tmp SLIMC | sudo tee -a /etc/integrit/mycustom.cnf
Initializing

The following makes a database. The configuration file will be read to identify what filename to use for the database.

sudo time integrit -C /etc/integrit/mycustom.cnf -u -v
sudo mv /var/db/integrit/current.cdb.new /var/db/integrit/known.cdb

Also:

sudo mkdir -p /var/log/integrit/

Either or both of the database files may be specified on the command line, overriding what is in the configuration file. To specify the most recent database, which is the newer one, use -N. For the database where things are older (and hopefully from a time when things were known to be good), use -O.

The process may take several seconds.

The database is viewable with “ i-viewdb /var/db/integrit/known.cdb

[#intgrtat]: Rotate integrit's databases file(s)
See what exists

See if both a new database file (possibly named /var/db/integrit/current.cdb.new) and also an older database file /var/db/integrit/known.cdb both exist.

ls -l /var/db/integrit/current.cdb.new /var/db/integrit/known.cdb
Handling the older file

If both a new database file (e.g. (possibly named /var/db/integrit/current.cdb.new) and also an older database file (e.g. possibly named /var/db/integrit/known.cdb) do exist, then then determine whether to back up an old database file should be copied/saved/renamed, or just replaced. If using the approach of copying files as a backup method, perhaps just run:

cpytobak /var/db/integrit/known.cdb
Prepare for a new database

Once the process of handling an old database file (if it did pre-exist) has been completed, proceed with these instructions to handle the most recent database.

If there is currently a database using the filename of a newly-created database, then move the new database so that it has the name of the old database. Do overwrite any existing old database (because the assumption is that the old database has already been backed up/rotated as needed).

sudo mv /var/db/integrit/current.cdb.new /var/db/integrit/known.cdb

Now, the system is prepared to be able to create a new database file.

Compare and update database

First, go through the process of rotating integrit's database file(s). At the end of that rotation process, there should not be a file with the new/output database's name. (There may be a /var/db/integrit/known.cdb but there should not be a /var/db/integrit/current.cdb.new file.)

To use the databases that are at the locations specified in a configuration file, the following syntax will work:

Older command:
sudo time integrit -C /etc/integrit/mycustom.cnf -cuv
Newer command:
{ sudo $SHELL -c "time ( integrit -C /etc/integrit/mycustom.cnf -cuv ) " ; echo Err=$? ; } 2>&1 | sudo -n tee -a /var/log/integrit/integrit-update-$(date +%F%H%M%S%Z%a).log &

Or, here is a way that will stop searching for more log file content once integrit runs. That would seem to be even slicker, although it does violate a fundamental principle of multiuser systems, as it assumes there is no other copies of the tail command running. This is not likely a good assumption for automation. If another user is running the tail command, this will even close down the program that is started by the other user. However, this impelmentation might be okay to do on a server that is still not being used by other users. (Use only at your own descretion/risk!)

{ { sudo $SHELL -c "time ( integrit -C /etc/integrit/mycustom.cnf -cuv ) " ; echo Err=$? ; } 2>&1 | sudo -n tee -a /var/log/integrit/integrit-update-$(date +%F%H%M%S%Z%a).log ; sudo pkill tail ; } &

To watch progress, run:

jobs
tail -f -n +0 $( ls -t /var/log/integrit/integrit-update-*.log | head -n 1 )

Note: This isn't meaning to suggest that progress will be visible before the very end, or perhaps not until the short report exceeds the operating system's pipe buffer. (Especially if the goal is to automate a command after this is done running, it may be easier to not background the task. To accomplish this without backgrounding the task, simply leave off the ampersand (“ &”) at the end of the command line that runs integrit. Then, just watch for when the task shows output. Once that is done, the computer will present another command prompt.) However, at some point some progress will be shown, and running this tail command will cause progress to be displayed at some point (possibly at the very end) without requiring any sort of intervention.

To view the entire report again, simply leave off the -f command line option. (The resulting command, “tail -n +0 ”, is effectively the same as running “cat”.)

Then, if the new update should replace the old one, feel free to back up the older one as appropriate, and replace the older database:

cpytobak /var/db/integrit/known.cdb
sudo mv /var/db/integrit/current.cdb.new /var/db/integrit/known.cdb

The databases may also be specified on the command line possibly with -N or -O).

Comparing a single filesystem object

If a database has been previously created, there are a couple of tools that may be able to check a single filesystem object, without needing to ask integrit to perform a full check. These tools come with Integrit, and are documented by Integrit documentation: Auxiliary Tools. Here is an example of how this may be done:

i-viewdb /var/db/integrit/known.cdb | grep -i /etc/passwd | tee /var/log/integrit/database_info.txt
i-ls /etc/passwd | tee /var/log/integrit/current_file.txt
diff -s /var/log/integrit/database_info.txt /var/log/integrit/current_file.txt
echo $?

Note that the results of the diff command will indicate if there are any differences. If there are differences, the differences may be in the file's contents, or the file's metadata, or perhaps just a difference in the types of data used by the testing software. (The latter refers to the possibility that integrit may not have checked the exact same things that i-ls had.) So, if ther are any differences at all, review what those differences are before getting excited about the differences. (Verifying whether the output is identical, except for possible differences in the types of tests moved, would be a fairly straightforward task that many computer programmers could implement.)

Interpreting the output of Integrit's auxiliary tools may benefit from reviewing the documentation about Integrit's list of switches.

Solution(s) for Microsoft Windows
(Runtimeware) Sentinel
Freeware for Windows. Includes a RegWatch portion. See also GRWU - Runtimeware Updater
radmind

radmind appears to be a more advanced solution which involves a client and a server. The chief advantage to this is that files get stored on a server and those files may be used to repair the files which were changed on a client. radmind exists for various platforms: Unix, Windows 2K and above (probably not including WinME), Mac, Solaris.

It appears that radmind has a bit of a steeper learning curve, and this guide does not currently provide a bunch of further details about using radmind.

Others
Spot the Difference

Spot the Difference

Spot the Difference is mentioned on a site dedicated to OpenBSD. In a nearby URL, Kernel Panic (Italian TLD): Software describes this as being “An (old and unmaintained) file integrity checker”. (The emphasis was in the original text that was quoted.) Wayback Machine @ Archive.org's cache of the “Spot the Difference” home page from May 6, 2006 states, “Current release is 0.2.1.” That was still true on August 19, 2015.

This software is under 100KB but does require Python. Works in Windows and Unix, and stores information in a database which may be MySQL, PostgreSQL, SQLite, or dbm. License info: SourceForge download page for “Spot The Difference” describes the license as a “BSD license”. However, the web pages don't seem to actually have a license that is viewable before downloading.

Tripwire
Tripwire.org will direct users to Open Source Tripwire (a projects area on SourceForge), but not before mentioning some of the commercial offerings from Tripwire.com. http://tripwire.sf.net redirects to SF.net's “Project tripwire” wordpress page.
OSIris
Mentioned by Samhain Labs review, this has source code for Unix and Windows as well as Windows binaries.
FCheck
This was at http://www.geocities.com/fcheck2000/ although since GeoCities went down, that website is likely not at that location anymore.
AFICK (Another File Integrity Checker)
AFICK for Perl.
Samhain
A comparison by Samhain dev: A comparison of several host/file integrity monitoring programs (by the developer of the host/file integrity monitoring program called Samhain) has some dated information (citing AIDE 0.13.1 as not having DB sign/crypt or Conf sign/crypt, but the latest AIDE does support one and/or the other for both DB and Conf).
http://www.l0t3k.org/security/tools/integrity/ lists some others.