Virtual Machine: SMTP Server

Some anti-spam techniques

SPF is recommended. Others are additional available options.

Publishing Sender Policy Framework (“SPF”)

Log into the DNS server.

RFC 7208 section 3.1 states that the RR (DNS “Resource Record”) type should be TXT (for use with the current SPF standard).

At the time of this writing, the latest version of SPF is considered to be version 1, identified by having the TXT RR say “v=spf1 ” (or having the TXT RR end with “v=spf1”, although the first paragraph of RFC 6686 page 4 indicates support for this to be at the start). Although there is such a thing as a TXT RR that starts with “v=spf2”, that is likely related to Sender ID and not common SPF.

Determine the record

Starts with: “v=spf 1”

Add: “ mx” if all the addresses associated with MX records (which can receive E-Mail), for this domain, should also be permitted to send E-Mail

Add: “ a” if all IP addresses for the domain (not including sub-domains) should be permitted to send mail. (This is another option, but is probably unnecessary if “mx” already handles all such desirable IP addresses.) This “a” will actually refer to both IPv6 AAAA records and IPv4 A records.

To also allow another IPv6 subnet, add: “ ip6:2002:cb00:7100::/126

To also allow another IPv4 subnet, add: “ ip4:203.0.113.0/30

Remote systems can be specified, using “ mx:example.com” or “ a:example.com

Do not use “ ptr”, even though the SPF specifications say that software should support it. If many organizations use “ ptr” checking for every received E-Mail, that would put strain on the .arpa nameservers, as noted by the last paragraph of RFC 7208 section 5.5 (which is also the last paragraph of page 24 of that document).

After specifying which servers are authorized, specify what to do with E-Mail coming from unauthorized servers. Using “ -all” indicates that any other host will fail the SPF check because the sender is sending E-Mail that is clearly “not authorized”. Using “ ~all” will specify that any other host is “probably not authorized”. RFC 7208 section 8.5 states that the people in control of the domain “believes the host is not authorized but is not willing to make a strong policy statement. Receiving software SHOULD NOT reject the message based solely on this result, but MAY subject the message to closer scrutiny than normal.”

Sample
v=spf 1 mx ~all
Main domain

Create a DNS record:

TXT record, as described by RFC 6686 Appendix A.
Subdomains
add: *.domain
Effectiveness

After changing the information in the zone configuration files, make sure the DNS server is using the updated information. For Unix, that commonly means telling the DNS server to reload the zone files, possibly by sending a SIGHUP or by restarting the DNS server.

2002:cb00:7100::/126
also do *.domain http://www.spfwizard.net (does not seem to handle IPV6 CIDR?) (Further info to be added...)
Sender ID

As shown by a couple of surveys documented by RFC 6686 page 4, the “spf2.0/” header (used by Sender ID) was getting supported by less than 3% of domains, while the “v=spf1” header (from standard SPF) was getting results of nearly 40-54%.

Microsoft's page for Sender ID, Microsoft's Sender ID wizard

DKIM
... https://en.wikipedia.org/wiki/DomainKeys_Identified_Mail#Patent_encumbrance
DMARC

One thing to know, first of all, is that DMARC involves using both SPF and DKIM.

Implementing DMARC is not currently an official part of this guide, but here are some resources to learn more about the topic:

Getting Started: Running an SMTP server
SMTP server logs
[#smtlgsbj]: Subject logging
Preparation for configuration

Perform: Preparation for configuration.

That sets some variables that are used by some upcoming instructions.

Edit the *.mc file's contents
cpytobak ${SNMCFMC}
echo dnl Add support for logging subject|sudo -n tee -a ${SNMCFMC}
echo LOCAL_CONFIG|sudo -n tee -a ${SNMCFMC}
echo Klog syslog|sudo -n tee -a ${SNMCFMC}
echo HSubject: \$\>+CheckSubject|sudo -n tee -a ${SNMCFMC}
echo|sudo -n tee -a ${SNMCFMC}
echo LOCAL_RULESETS|sudo -n tee -a ${SNMCFMC}
echo SCheckSubject|sudo -n tee -a ${SNMCFMC}
echo R\$\*\\t\$: \$\(log Subject: \$1 \$\) \$1|sudo -n tee -a ${SNMCFMC}
echo ${VISUAL}
sudoedit ${SNMCFMC}

If you'd like to see what that looks like without the escape characters, see: mtx77's answer to Mike B's ServerFault.com question about Sendmail logging subjects (which is where the author of this text originally located this configuration).

Update the *.cf file

Updating the *.cf file.

Test
echo TestMessage| time mail -vs FirstTestSubj $(id -nu)@$(hostname)
sudo tail /var/log/maillog | grep ": Subject:"
Receiving E-Mail from multiple domains
Sendmail
Test

Send mail to a “network name” which is not yet supported by Sendmail. If there are not multiple DNS names handy, that's fine: the IP address can effectively work for this test. For instance:

echo TestMessage| time mail -vs SubjFirstTestv6destIP $(id -nu)@2001:db8:1::25

That sample address was based on: example subnets (used by this guide). Alternatively:

echo TestMessage| time mail -vs SubjFirstTestv4destIP $(id -nu)@198.51.100.25

What is likely to happen is that the Sendmail MSA/MTA will receive the mail, thinking that the mail will then be delivered. However, Sendmail will find that it cannot treat the IP address as a domain name, and that the Sendmail “from localhost [127.0.0.1]” reported that the address “had” a “permanent fatal error” described as “500 Host unknown”. We'll want to make a change so that the Sendmail “from localhost [127.0.0.1]” recognizes the E-Mail address as itself.

cpytobak /etc/mail/virtual-domains
echo 198.51.100.25| sudo -n tee -a /etc/mail/virtual-domains
Steps to fix rejections of domains
Edit the *.mc file's contents
cpytobak ${SNMCFMC}
sudoedit ${SNMCFMC}

Find some lines in the text file. (Referencing the virtusertable filename may be a useful search term.)

dnl Enable support for /etc/mail/virtusertable.
dnl Used to do N -> N address mapping.
dnl
FEATURE(`virtusertable', `hash -o /etc/mail/virtusertable')dnl
dnl
dnl Rewrite (unqualified) outgoing email addresses using the
dnl mapping listed in /etc/mail/genericstable
dnl
FEATURE(genericstable, `hash -o /etc/mail/virtusertable')dnl
dnl

Insert some lines, before the sentence about Rewrite (and before the next line of code). These lines should look like this:

VIRTUSER_DOMAIN_FILE(`/etc/mail/virtual-domains')dnl
dnl

... and so the portion of the text file may look something like this (after being edited). (The text file is shown in three parts to make it easier to notice where the line got inserted.)

dnl Enable support for /etc/mail/virtusertable.
dnl Used to do N -> N address mapping.
dnl
FEATURE(`virtusertable', `hash -o /etc/mail/virtusertable')dnl
dnl
VIRTUSER_DOMAIN_FILE(`/etc/mail/virtual-domains')dnl
dnl
dnl Rewrite (unqualified) outgoing email addresses using the
dnl mapping listed in /etc/mail/genericstable
dnl
FEATURE(genericstable, `hash -o /etc/mail/virtusertable')dnl
dnl

Save the change to the file.

Make another required file

Now that Sendmail's *.cf file refers to this virtual-domains file, it must exist (or else Sendmail will fail the next time someone tries to start it). Even if there is no immediate desire to support other domains, the file must exist. (It may be zero bytes. The requirement is just that the file simply has to exist.)

If you don't want to support any other domains at this time (and are just adding support for later):

sudo touch /etc/mail/virtual-domains

If this isn't done, then mailq will complain when it is used as a test during the routine process of updating the *.cf file.

Update the *.cf file

Updating the *.cf file.

Update virt user table plaintext file

If you do want to support one or more E-Mail domains, add them. (You may do this instead of the prior step, or in addition to the prior step.) Remember, this doesn't strictly need to be DNS names. It could be IP addresses that support would be desirable for.

echo 198.51.100.25| sudo -n tee -a /etc/mail/virtual-domains
echo 2001:db8:1::25| sudo -n tee -a /etc/mail/virtual-domains
Update virt user table database
is there a virtual-domains.db file??
Have Sendmail support config
sudo kill -HUP $(sudo head -1 /var/run/sendmail.pid)
Test
echo TestMessage| time mail -vs SubjNewerTestv6destIP $(id -nu)@2001:db8:1::25

That sample address was based on: example subnets (used by this guide). Alternatively:

echo TestMessage| time mail -vs SubjNewerTestv4destIP $(id -nu)@198.51.100.25
Alternate approaches
Other possible approaches might include using local-host-names or relay-domains, although Kiranjith's Blog on Sendmail virtual domains/users notes “While defining the virtusertable the domain name should not exist in local-host-names.”

This method seems to be rathe compatible with the virtusertable feature.

Permitting a user to specify “From:” address

If a user can run the telnet command, the user may be able to generate all sorts of customizations of message headers. (Details for doing this are shown by SMTP sample.) However, a user might be able to do that using even easier methods, like having an MUA (Mail User Agent) use the “From” address specified by the user (e.g., with the -r parameter to the mail command), and having the E-Mail address be handled by Sendmail. Well, Sendmail can offer some control regarding which users can use that easy approach.

(Details are not currently available here... but may be getting added here after some further verification/testing of...) The Linux Documentation Project: Some Useful sendmail Configurations.

Supporting virtual usernames
Situation
Overview: Understanding what this solves

By default, Sendmail is rather tied to the existence of Unix accounts, except that additional E-Mail addresses may be specified by using the aliases.db file (which is created by using the aliases file and then running sudo newaliases). The format of the aliases.db file limits the E-Mail addresses to what is on the physical machine.

This provides a limit. Any supported username exists on all domains.

For example: If you have some staff members named Alice and Bob, and the staff support two different charitable organizations: CleanWater and AvailableFood. Sending E-Mail to bob@cleanwater.example.org and bob@availablefood.example.org could result in both E-Mails going to the same account, which is okay. (Bob may want the E-Mails separated into separate mailboxes, but there are various ways of accomplishing that including using a client-side filter. So the E-Mail server doesn't necessarily need to do that.) However, what if one computer wishes to support multiple companies (which is commonly done at Internet service providers). If CustomerOne has an accounts named Alice and Bob, and CustomerTwo has accounts named for Carol and Dan, then there are no problems. However, trouble starts to happen if CustomerTwo also hires three more people who individually have the names of Erin, Alice, and Frank. The problem is that an E-Mail address of “alice@(something)” cannot have mail to go different Inboxes, if default behaviors are used. So, non-defaults are needed.

As a nice side effect, E-Mail address will not be tied to the user accounts. So there will be no temptation to be adding a user account just to be able to support an E-Mail address.

Resolving this in Sendmail

This might depend on the feature that uses the virtual-domains file, described elsewhere...

Preparation for configuration

Perform: Preparation for configuration.

That sets some variables that are used by some upcoming instructions.

Update the virtusertable (text config)

As shown in some sample configuration (see comments in the pre-existing file, or look at FreeBSD Sendmail configuration which discusses some files including the /etc/mail/virtusertable file), there are some various ways this file can handle things. In the following example, all E-Mail sent to one domain is sent to a similarly-named mailbox at another domain. In this case, the first domain (where the mail gets sent to) happens to be an IPv4 address, while the second domain (where E-Mail will be sent to) happens to be localhost.

e.g.:

cpytobak /etc/mail/virtusertable
echo| sudo -n tee -a /etc/mail/virtusertable
echo @198.51.100.25\\t\%1@localhost| sudo -n tee -a /etc/mail/virtusertable
echo @2001:db8:1::25\\t\%1@localhost| sudo -n tee -a /etc/mail/virtusertable
echo ${VISUAL}
sudoedit /etc/mail/virtusertable
  • What works:
    • IPv4 addresses
    • domain names
  • What doesn't:
    • IPv6 addresses
      • Surrounding the domain name with square brackets seems to give better errors. Instead of “Colon illegal in host name” “part”, at least the SMTP server recognizes such a domain as a possibly valid address. However, an error message then reports “config error: mail loops” “back to me (MX problem?)

Mik Telecom O'Reilly's CD bookshelves: Sendmail 3rd: FEATURE(virtusertable) notes that instead of an E-Mail address, the second parameter can start with “error:” and then contain some text. Starting with Sendmail 8.10, that text can start with a DSN (“Delivery Service Notification” value, also mentioned at running the SMTP server), so the resulting second parameter might look something like “error:5.7.0:550 CustomMessage”. (To be quite clear, that “5.7.0:550 ” portion is optional.)

Update the virtusertable (hash)
[ -f /etc/mail/virtusertable.db ]&&cpytobak /etc/mail/virtusertable.db
ls -l /etc/mail/virtusertable*
cat /etc/mail/virtusertable | sudo makemap hash /etc/mail/virtusertable # a .db suffix is implied
ls -l /etc/mail/virtusertable*
Have Sendmail support config
sudo kill -HUP $(sudo head -1 /var/run/sendmail.pid)
Test

Testing this involves having the same username at different E-Mail addresses go to different mailboxes. This should be fairly easy to create a test, using multiple domains (like the IPv4 address, and localhost). However, this guide does not provide full directions for this, yet.

Send mail to IP address(es).

echo TestMessage| time mail -vs SubjNewerTestVirtUsernamesV6 $(id -nu)@2001:db8:1::25

That sample address was based on: example subnets (used by this guide). Alternatively:

echo TestMessage| time mail -vs SubjNewerTestVirtUsernamesV4 $(id -nu)@198.51.100.25
Using Milters
Verify the MSA supports Milters
If the MSA is Sendmail
echo /quit | sendmail -bt -d0.1 | grep -i "Compiled with: " | grep -i MILTER

If that turned up zero results, then expand the search a bit...

echo /quit | sendmail -bt -d0.1

... and look to see if the word MILTER shows up anywhere in the “Compiled with: ” section.

The next details will vary quite a bit depending on what software you intend to use. This guide shows an approach. Do not assume that this is “the” (only good) right approach. There are many good ways to successfully approach these sorts of things.

Sendmail+AMaViSd+more
Overview

Currently, this guide uses Sendmail and AMaViSd.

At the time of this writing, there is one SetUID program. It has not yet been determined just how necessary that is, or isn't. This basically means that the /usr/local/share/sendmail/mail.local file needs to be trusted. It might be possible to add some restrictions to reduce how much that program needs to be trusted, but that hasn't (yet) been fully confirmed by the author of this text.

Installation
Improving the setup

After AMaViSd is installed and functioning, further configuration can make it more useful...

Obtain SpamAssassin Score
Installing SpamAssassin

The system's package manager may have already installed SpamAssassin when installing AMaViSd. In that case, the package manager will probably be intelligent enough to not cause problems by a re-attempt.

sudo -i pkg_add -ivv p5-Mail-SpamAssassin
Modify AMaViSd config
Backup and Start Editing
cpytobak /etc/amavisd.conf
echo ${VISUAL}
sudoedit /etc/amavisd.conf
Make sure checks are enabled

Check whether the bypass is commented out. It might have been uncommented before, but now it should be commented out.

@bypass_spam_checks_maps  = (1);  # controls running of anti-spam code

Comment it out:

# @bypass_spam_checks_maps  = (1);  # controls running of anti-spam code
SpamAssassin-specific AMaViSd Config

Change:


# $db_home   = "$MYHOME/db";      # dir for bdb nanny/cache/snmp databases, -D
# $helpers_home = "$MYHOME/var";  # working directory for SpamAssassin, -S
# $lock_file = "$MYHOME/var/amavisd.lock";  # -L
# $pid_file  = "$MYHOME/var/amavisd.lock";  # -P
#NOTE: create directories $MYHOME/tmp, $MYHOME/var, $MYHOME/db manually

to:


# $db_home   = "$MYHOME/db";      # dir for bdb nanny/cache/snmp databases, -D
$helpers_home = "$MYHOME/var";  # working directory for SpamAssassin, -S
# $lock_file = "$MYHOME/var/amavisd.lock";  # -L
$pid_file  = "$MYHOME/var/amavisd.lock";  # -P
#NOTE: create directories $MYHOME/tmp, $MYHOME/var, $MYHOME/db manually

Set some required values. Find:

$sa_tag_level_deflt  = 2.0;  # add spam info headers if at, or above that level
$sa_tag2_level_deflt = 6.2;  # add 'spam detected' headers at that level
$sa_kill_level_deflt = 6.9;  # triggers spam evasive actions (e.g. blocks mail)
$sa_dsn_cutoff_level = 10;   # spam level beyond which a DSN is not sent
$sa_crediblefrom_dsn_cutoff_level = 18; # likewise, but for a likely valid From

This next part is a matter of opinion. If you wish to have SpamAssassin starting to affect what mail is received, then the defaults might be fine. If the goal is just to have SpamAssassin report its findings, without blocking the mail flow, some adjustments may be desirable. Having SpamAssassin simply report is less likely to cause accidental problems from a false positive.

The following levels are recommended, by this guide, for people who are not yet experienced with using SpamAssassin. However, some people will have other preferences, and disagree with these recommendations.

$sa_tag_level_deflt  = 0.0;  # add spam info headers if at, or above that level
$sa_tag2_level_deflt = 6.2;  # add 'spam detected' headers at that level
$sa_kill_level_deflt = 9996.9;  # triggers spam evasive actions (e.g. blocks mail)
$sa_dsn_cutoff_level = 10;   # spam level beyond which a DSN is not sent
$sa_crediblefrom_dsn_cutoff_level = 18; # likewise, but for a likely valid From
Some misc notes

(These are scratch notes, and may be implemented into the guide at a later point. Future versions of this guide might simply remove some of this text. If it seems useful, then save it.)

SpamAssassin may also have its own config:

AWeber Communications info on SpamAssassin Score states, “Content filtering is only one way ISP's analyze messages. An increasing weight is being given to the reputation of the sender and other factors.” https://spamassassin.apache.org/full/3.1.x/doc/Mail_SpamAssassin_Conf.html#scoring_options http://wiki.apache.org/spamassassin/AvoidingFpsForSenders says: “Make sure you have active and monitored abuse and postmaster addresses. Register them with abuse.net.” also, the "Newsletter Aides" section has some tips... http://spamassassin.apache.org/tests_3_3_x.html Specify language(s) per https://en.wikipedia.org/wiki/SpamAssassin
More notes
note: http://www.ijs.si/software/amavisd/ says to adjust login.... The /etc/amavisd.conf indicates that some directories may need to be made manually, like /var/amavisd/var/ but it seems that's already done. grep "^\$MYHOME =" /etc/amavisd.conf sudo mkdir /var/amavisd/var sudo ln -s /usr/bin/mail /bin/mail
Restart AMaViSd

Either send it a SIGHUP, or simply run:

sudo time amavisd reload
echo ${?}
Test

Send yourself a couple of E-Mails. In one of them, have the message contain the Generic Test of Unsolicited Bulk E-Mail string, documented at:

Generic Test of Unsolicited Bulk E-Mail.

Read the resulting E-Mail. (The preferable way to do this is generally to use an MUA (Mail User Agent), and check headers.) Check if the headers show a higher spam score.

Others

Some other tests...

  • Vipul's Razor
  • DCC

are mentioned by an older mail handling guide which also documents many of the same processes as this version of the guide.