Servers getting E-Mail

This is mainly about how to interact with other servers. For information about servers getting E-Mail from users running E-Mail client software, see the section about users receiving E-Mail.

Receiving E-Mail

This section is about setting up a server so that it may receive E-Mail. Most commonly, unencrypted and unauthenticated E-Mails are received using SMTP on TCP port 25. An E-Mail server may require authentication to accept mail. This is generally done with another port, such as receiving unencrypted E-Mail on the submission E-Mail TCP port number 587. (There are other ports that may be in relatively common use.)

Once E-Mail is received, the computer that receives the E-Mail will typically be doing something with the E-Mail. One possible option is to do some pre-processing, such as running additional checks including scanning any attachments for malware. (However, if an E-Mail is going to be rejected, ideally this is communicated to the computer that sent the E-Mail before the E-Mail has been fully accepted by the receiving computer.) The E-Mail needs to be fully processed to determine who needs a copy of the E-Mail. To verify that the destination E-Mail address is valid is a task commonly done before the E-Mail is fully accepted. However, after then, the E-Mail server may perform an action like supporting a distribution “list”, and sending the E-Mail to multiple recipients. (This may end up duplicating the E-Mail, especially if some recipients have their mail forwarded off-site. However, (at least in theory?) there may be “de-duplication” methods so multiple copies of the mail don't need to use multiple copies of disk space.

Basic options on what an E-Mail program may do with the mail may include relaying E-Mail (although publicly-usable relays are typically shunned because they may be abused by spammers), forwarding E-Mail, or storing the E-Mail so that it may then be retrieved when an end user checks it. ...

[#mailwrpr]: mailwrapper

Since many programs in Unix may have been written with an assumption of sendmail being used, this program may help provide some compatibility for any other software to be used. Basically /usr/sbin/mailwrapper figures out what filename was used to run the program (since other executable names may link to the /usr/sbin/mailwrapper file), then looks up details in the /etc/mailer.conf program to see what is really desired, and then runs an appropriate program for the desired task. As a wrapper, the program may not really perform any substantial mail handling itself, but relies on other programs.

See: OpenBSD 2.6 Manual Page for mailwrapper (especially and the end, in the “Bugs” section of the OpenBSD 2.6 Manual Page for mailwrapper), /etc/mailer.conf, and, for example, OpenBSD Manual Page for OpenSMTPD (OpenBSD's smtpd) that shows some modifications to be made.

[#opnsmrcv]: Recieving mail with OpenSMTPD
Software history

OpenBSD 4.6 released a new piece of software, smtpd, which was previously referred to as OpenSMTPD. Prior to the announcement of OpenSMTPd being enabled into OpenBSD's build, the software may have needed to be added to the operating system. (Although Calomel.org's OpenSMTPD guide had posted some old instructions about how to do that. However, before simply accepting all of the details provided by that guide, be aware of concerns about the Calomel.org site and specifically the OpenSMTPD guide on that site: information on Calomel.org shows some people finding that Calomel.org has provided problematic information, especially for the OpenSMTPD guide at that site.)

For details on setting up OpenSMTPD to be receiving E-Mail, see: Receiving E-Mail with OpenSMPTD

See: OpenBSD Manual Page for (OpenSMTPD) smtpd software, OpenBSD Manual Page for smtpctl.

[#sndmlrcv]: Receiving E-Mail via the program called “sendmail”

Yes, the irony is acknowledged: this section of text is about receiving E-Mail with a program that is named sendmail.

Information on this topic has been moved to a seperate section dedicted to the topic of receiving E-Mail with the “sendmail” program.

This section did have some hyperlink anchors before this re-organization. In case anyone has linked to the prior location, here are some redirectors:

Accepting E-Mail

Determining if the system wants to accept the E-Mail (after any sort of anti-spam measures are taken)

...

[#emailmda]: Mail delivery

Once E-Mail has been accepted, the mail is handled by whatever software code is going to be the “Mail Delivery Agent” (“MDA”). The main job of an MDA is to copy mail to the desired “mailbox” so that a user may access the mail.

Note that the role of MDA is typically rather focused on delivery to the users on the local system. E-Mail that needs to be handled by another organization's E-Mail server will likely be handled by role of MTA, which may be different software.

Modern and advanced setups may involve having the MDA perform scans before deliverying the mail, hoping that the scans may identify unwanted E-Mail before it is actually accessible to end users.

To clarify: examples of such unwanted E-Mail may be:

  • dangerous (containing malware)
  • “unsolicited commercial E-Mail” (“UCE”) which is most likely to have no real effect other than to waste resources (including the end user's time, and also including computer resources like network bandwidth and disk space)
  • illegal material that may be likely to lead to legal liability (material likely to infringe upon a valid copyright)
  • other E-Mail that most professionals agree is inappropriate to exist in professional organizations (perhaps E-Mail that contains rather sexual content that possibly fits within all of the previously-mentioned categories).

Many people like such unwanted E-Mails to be identified and categorized, and then for the unwanted E-Mail to not be mixed in with wanted E-Mail. (There are different recommendations on what to do with the unwanted E-Mail, including letting the sender know that it will not be sent, or accepting the E-Mail and then storing it in a seperate location and possibly sending a notification to the recipient that some questionable content was saved.

MDA software for Unix
[#fdmemlda]: fdm (“fetch and deliver mail”)

This program is named from its basic tasks of “fetching” E-Mail, meaning to act as an Mail Retrieval Agent (“MRA”), and to act as a Mail Delivery Agent (“MDA”).

(This section is about using fdm as an MDA. For details about using fdm as an MRA, see fdm as an MRA.)

[#sndmlmda]: Using SendMail as an MDA

(Information has been moved: see: Using SendMail as an MDA.)

../elecmail.htm has a section about “Sending E-Mail”, such as “Dealing with dial-up”.

Mail Transmission (MTA)

To understand this role, see MTA role.

Setup

Very often, the process of setting up an E-Mail server to send mail is covered by the guides that show how to set up a mail server that receives E-Mail. This is likely the case for the guides that are both provided here and are about the following pieces of software:

[#fwdemail]: Forwarding/Relaying E-Mail

Some people may have heard some bad things about relaying E-Mail: the problem is generally when an E-Mail server is a public “open relay” which allows spammers to abuse the service. Private relaying of mail, which is also commonly called “forwarding” E-Mail, is a widespread practice.

The basic concept here is that once an E-Mail message is accepted, the E-Mail message gets delivered to a different E-Mail address than the address that the message specifies. Mail may be forwarded to other local addresses, or even remote addresses, and the practice (properly implemented) typically does not cause problems.

Specifying the desired recipient

Implementing the mail forwarding is one thing, but since implementations may vary, first let's see some common methods of how the recipient is specified.

(This information may currently be rather specific to the Unix platform.)

Aliases

Some patterns of E-Mail addresses are commonly used by various organizations, and these are often stored in a database called aliases.db which might be in the directory /etc/mail/.

Unlike using a .forward file, this can specify E-Mail addresses that don't actually have a real user account. This may also be something that more likely to be implemented (by default, pre-configured for certain common E-Mail addresses). It is also a place that may be fairly centrally managed.

(there is currently little information here on this implementation, other than the following references) OpenBSD manual page for the aliases file (in SendMail's /etc/mail/ directory) states, “This is only the raw data file; the actual aliasing information is placed into a binary format in the file /etc/mail/aliases.db using the program newaliases”. Then, the OpenBSD Manual Page for the newaliases command notes that newaliases command may simply be “identical to” running “sendmail -bi” at the command line.

Using a file to specify forwarding for an individual user

To foward mail from sample@localhost (or any other system name that the E-Mail server is listening to), to remoteaddy@remotesys.example.com:

echo remoteaddy@remotesys.example.com >> ~sample/.forward

This has an advantage of allowing a user to choose a destination for incoming E-Mail, and to even make changes to that destination without needing to bother a system administrator. Such changes also don't involve changing any centralized files that affect any other E-Mail addresses that may be receiving mail.

Procmail 3.22 installation instructions discusses the concept of this file a bit, in section 4. (These instructions do not seem very specific to Procmail. Also, presumably these details might exist in other versions of Procmail documentation. Version 3.22 was just the latest stable version when this quote was being extracted.) One is a recommendation to “be sure to make .forward *world* readable.” (This means to set the “other” permission set to have this file to be “readable”. So, “chmod o+r .forward ”. Whether this is a good idea may be questionable, but remembering the tip might not be a bad idea if having troubles getting it to work.

Procmail 3.22 installation instructions also says “MMDF users should note that they need a .maildelivery file *instead* of the .forward file”. This is likely referring to using Multichannel Memorandum Distribution Facility software which was made in the late 1970s and has little market share. The relevant point, though, is that this sort of functionality may exist with some software by using a different filename than other software. The MDA software will commonly have documentation about such a feature, so look for such documentation (if needed).

Implementation
Aliases

Updating a aliases.db file may be done by updating a text file named aliases (probably in the same location as the aliases.db file) and then running a command called newaliases.

The /usr/bin/newaliases command may be a synlimk to the mailwrapper command, which will act as expected/documented, or may commonly end up running sendmail. Because the sendmail command checks the name of the command that was run from the command line, when sendmail is run using the newaliases command, the effect may be identical to running “ sendmail -bi ”. However, using the newaliases command may be a practice that may be more compatible with systems that may not using something other than sendmail.

Forwarding mail

(This section may need to be re-tested.)

Allowing relaying to specific domains
Overview: the problem

Mail forwarding may involve more than just passing the E-Mail to a remote system: a copy of the mail may or may not end up on a system that forwards mail. If relaying is not set on the machine that is receiving mail, the system that is trying to send mail may end up seeing a message such as the following:


<user@xxxxxxxxxx>: host example.com[192.0.2.1] said:
550 5.7.1 <user@xxxxxxxxxx>... Relaying denied (in reply to RCPT
TO command)
How the solution is found

This approach looks quite similar to what is found by OpenBSD FAQ 10: Section about “Relaying denied” messages (FAQ 10.4) which does also reference some additional resources.

Note: This solution may be using default files from an OpenBSD installation. (It would not seem terribly surprising if other operating systems might come with different default files. However, the same techniques can likely be applied to determine the exact solution for another operating system. The solution may end up being exactly identical to how this works with OpenBSD.)

If sendmail was started with a parameter of “ -C/etc/mail/sendmail.cf ”, then the solution may be pointed to by that configuration file. Specifically, if checking that file, the following text may be seen:

# Hosts for which relaying is permitted ($=R)
FR-o /etc/mail/relay-domains

Later on in the file, the following is seen: # file containing names of hosts for which we receive email
FR-o /etc/mail/local-host-names

Let's check those out:

OpenBSD's version of /etc/mail/local-host-names file (visible via OpenBSD's CVSWeb: /etc/mail/local-host-names, and choosing “text”) will show:

# List additional hostnames that should be considered local (one per line).
# I.e., any hostname for which you wish mail to be accepted (and delivered).
# You do not need to include the system hostname, localhost, the name(s)
# associated with any active network interface or a CNAME that points to one
# of those names.

Note that the term “delivered” refers to storing the E-Mail message in a local mailbox for the user. This will require that a local user account is created (see adding users) for each E-Mail address that uses a domain listed in this file.

If all the mail received for a specific domain will end up getting passed onto remote E-Mail addresses (handled by remote mail servers that are in other domains and which may be run by other organizations), then virtual users may be used and the E-Mail may just be relayed. There are multiple steps for that, one of which is to add any supported domains to another file. Checking out OpenBSD's version of /etc/mail/relay-domains file (visible via OpenBSD's CVSWeb: /etc/mail/relay-domains, and choosing “text”) will show:

# By default, sendmail(8) will not relay for foreign domains. If the mail
# is not destined for (or sent by) a user in the local domain the message
# will be rejected.  Alternately, domains may be listed in /etc/mail/access
# with the RELAY attribute instead of being enumerated here.

On a side note, using the /etc/mail/access file is not needed for this. However, some details about how to use that file, in curious, is seen by MicroBSD Handbook: Sendmail (archived at the Wayback Machine @ Archive.org) (section 13.3).

The solution

Domain name(s) may need to be added to file(s). The specific details, such as exact filenames, may differ a bit based on how a specific domain is supported. However, before getting into those more specific details, here are some more generalized details that apply to multiple implementations:

Add one host per line. Hosts can be specified by IPv4 address octets, so lines could say something like "192.168", "172.16", "10" to refer to, for example, "10.0.0.0 netmask 255.0.0.0" and the other similar addresses similar to IPv4 10/8. Other examples include ".example.com" and "sub.example.com", the former allows anything in example.com and the latter includes sub.example.com and any hosts in that domain.

For local delivery

Part of the comment of OpenBSD's version of /etc/mail/local-host-names file states:

# List additional hostnames that should be considered local (one per line).

Certain domains may be implied, and won't need to be included in this file. However, for any other domains that are receiving mail for local users, make sure the names are entered into this line.

For relaying mail

Part of the comment of OpenBSD's version of /etc/mail/relay-domains file states:

# List of other domains to relay mail for here (one per line).

So, put each domain name on its own line.

Using /etc/hosts

There may not be any particular need to be using the /etc/hosts file. However, if that is desirable, put the system names in this file. (Perhaps computers listed in /etc/hosts don't need to be listed in one of the other files?) However, the /etc/hosts sendmail program needs to know to use this file.

OpenBSD FAQ about sendmail using the /etc/hosts file (FAQ 10.6) provides the method to do this. Make sure the file /etc/mail/service.switch exists. (It probably does not, so plan to create the file.) If the desire is to support the /etc/hosts file and then to use DNS, make the file say:

    hosts       files dns

To cause sendmail to only use the /etc/hosts file and to ignore DNS, just remove the reference to dns from that example.

Once any of these files are updated, be sure to let the sendmail program know to re-read those files, by sending the program a SIGHUP signal. To do this, the following will work in OpenBSD:

kill -HUP $( cat /var/run/sendmail.pid )
Sending E-Mail to users that do not exist locally

(It seems like some old notes got merged into this section; the notes seem to duplicate some of the instructions from supporting virtusertable, which is now described by sendmail virtuser support. This section may need to be re-tested.)

First, before troubleshooting this, it may be worthwhile to make sure the mail works at least for any user that does exist locally. For instance, perhaps sending E-Mail to an Administrator account.

Sending E-Mail to the Administrator account

Many Unix machines will come with the mail command. For instance, try:

which mail

If that file exists, then the following may work:

echo Mail Test | mail -s QuickTest $( whoami $)@localhost

(Additional details on this software are available in the section about the mail command. Other software may be discussed in the section on MUA software).

If E-Mail works for local users, but not remote users, one option would be to make a local user for each E-Mail address. However, creating a user involves modifying the user database, setting a password, creating a home directory, and possibly opening up more security risk than necessary. This seems to be quite a bit of overhead just for an E-Mail address. In fact, this overhead is not necessary.

Start accepting mail at example.com

This will involve using the file /etc/mail/virtusertable. MicroBSD Handbook: Sendmail (section 13.: sendmail Configuration)(archived at the Wayback Machine @ Archive.org) (section 13.3.5) says, "This file is processed in a first match order down the file." In a shown example, documentation notes that “if nothing from” any earlier configuration line “has matched so far,” (then the E-Mail message) “will match the” later example configuration line. A line (the third line) in their sample configuration file doesn't impact E-Mails that do match earlier lines of the configuration file.

As the owner of the file (root), go ahead and edit a text file, specifically /etc/mail/virtusertable.

After modifying this file, the database will need to be updated. See: applying changes to the /etc/mail/virtusertable file.

Additional notes

(These appear to be some old notes related to applying changes to the virtusertable file.)

nano virtusertable
(then run make) (It is believed that the following came from contents of a text file, so the CSS/markup is acting like that.)
example.com
example.net
#disable.example.org
domain.example
192.168
10
172.16
#...
172.31
MicroBSD Handbook: Sendmail (archived at the Wayback Machine @ Archive.org) and http://www.sendmail.org/m4/features.html may have some examples, such as:
@*.baz.org jane@example.net

README file about sendmail Configuration Files says: “virtuser_entire_domain”:

If the virtusertable is enabled and VIRTUSER_DOMAIN or
VIRTUSER_DOMAIN_FILE is used, this feature will cause
addresses to be searched in the map if their domain
parts are subdomains of elements in class {VirtHost}.

VIRTUSER_DOMAIN_FILE(`/etc/mail/virtual-domains')dnl

How it was done:

First, backup the three (sendmail.cf, localhost.cf, submit.cf) /usr/share/sendmail/*.cf files

cd /usr/share/sendmail/cf
mkdir backup
cd...Err... these directions look incomplete
cp ../openbsd-proto.mc attnhigh(incoming instructions here lacked a destination folder ... this command needs customization/fixing! Maybe the current directory?)
nano openbsd-proto.mc

insert
VIRTUSER_DOMAIN_FILE(`/etc/mail/virtual-domains')dnl
example line with surroundings
FEATURE(`virtusertable', `hash -o /etc/mail/virtusertable')dnl

dnl

dnl Rewrite (unqualified) outgoing email addresses using the

dnl mapping listed in /etc/mail/genericstable

dnl

VIRTUSER_DOMAIN_FILE(`/etc/mail/virtual-domains')dnl

FEATURE(genericstable, `hash -o /etc/mail/genericstable')dnl

dnl
Follow-up commands
make
make distribution
sudo mailq
Handling E-Mail
Queue handling
[#emlqsee]: Viewing a queue

Possibly related topic: mailboxes.

There may be multiple ways to do this. The nicest way of seeing how many messages are in the queue is probably to view the queue using the E-Mail server software. An alternate method may be to check where the queue's data is stored: multiple E-Mail server software may have a file on the hard drive for each message.

SendMail (queue viewing)
Viewing the queue using SendMail software

Run the smtpctl program as mailq. If the system is using the mailwrapper command, the easy way to do this is likely to have the /etc/mailer.conf file point to the /usr/sbin/smtpctl executable file. (Another approach that could be done is to use a symlink.)

Note: If running multiple MTAs as described by the amavisd-new package's /usr/local/share/doc/amavisd-new/README.sendmail-dual file, make sure to specify the configuration file. For example:

sudo mailq -C/etc/mail/sendmail.cf

(Point of clarification: Like when telling sendmail about the configuration file, there is no space between the “-C” and the path of the configuration file.)

Accessing files directly

Perhaps at: /var/spool/mqueue

OpenSMTPD
Viewing the queue using OpenSMTPD software

Run the smtpctl program as mailq. If the system is using the mailwrapper command, the easy way to do this is likely to have the /etc/mailer.conf file point to the /usr/sbin/smtpctl executable file. (Another approach that could be done is to use a symlink.)

Microsoft Exchange (queue viewing)
Viewing the queue from within Microsoft Exchange
Graphical approach
Microsoft Exchange 2003
(...)
Microsoft Exchange 2007
Accessing files on hard drive
Microsoft Exchange 2003

The following procedure specifies where the E-Mail is stored, which is probably "\Program Files\Exchsvr\Mailroot\" (or some directory under there, such as “vsi 1\Queue\”). KB 909005 indicates that is where problematic mail may be found in Exchange 2000 Server. For Exchange Server 2003, perhaps also check \InetPub\Mailroot\ folder. (Most commonly all of these would be on C:.)

See: MS KB 822933: How to change the Exchange 2003 SMTP Mailroot folder location or MS KB 318230: How to change the Exchange 2000 SMTP Mailroot folder location (which starts by stating a need to install Windows 2000 Support Tools).

[#emlqflsh]: Flushing the queue

This term may refer to having the software process each message in the queue again (e.g. KB 821901: How to Flush the SMTP Mail Queue in Exchange Server 2003). For this approach, see the section on how to “Process the queue”. Sometimes people may want to just flush all of the E-Mails out of the queue. For this task, see the section about “Removing E-Mails from the queue”.

Process the queue
SendMail (queue processing)

Maybe run the following?

sendmail -q -C/etc/mail/sendmail.cf

?

Microsoft Exchange (queue processing)

KB 821901: Handling the SMTP Mail Queue in Exchange Server 2003 indicates the queue may be “flushed” by pausing the “SMTP virtual server”. After allowing a short time (seconds) to see if the Queue has been substantially reduced in size, don't forget to resume the resume the “SMTP virtual server”!

Learning about messages in the queue
SendMail (queue analyzing)

The files are named after the message ID. For instance, if:

sudo mailq -C/etc/mail/sendmail.cf

shows that the message Q-ID was r7ECFEp2031512 then check /var/spool/mqueue/*r7ECFEp2031512 files. The file starting with “df” contains the message body, while the file starting with “qf” contains information about the queue status.

(In the above example command, referencing the configuration file might be unncessary. However, showing that is meant to clearly communicate the need to specify the configuration file when non-default configuration filenames are being used.)

Note that some preliminary testing indicates that messages might remain in the queue for a brief time after they are sent. Try also running:

sudo cat /var/log/maillog | grep r7ECFEp2031512

(Once again, the r7ECFEp2031512 is the message Q-ID as shown by sudo mailq.)

Check to see if a mailer named “smtp”, or more probably “esmtp”, has noted “stat=Sent”. (The mail may also show a line that indicates a “mailer=relay,” has a “stat=Sent”. This is far less interesting, and does not indicate that the message has effectively left the queue. Perhaps see also: SendMail mailers.)

Removing unwanted E-Mails from the queue

One option might be to use an interface provided by the E-Mail server software. However:

  • That is not the best approach with SendMail, which does not contain a command to flush the queue.
  • Also, that is not the best approach when dealing with very large queues (such as after an attack creating lots of outgoing spam) with at least certain versions of Microsoft Exchange.

Some alternative approaches:

[#emlqmvfl]: Moving queue files out

Locate the messages. (See the section about viewing the queue.) Shut down the E-Mail server. (This does cause some downtime. That downtime might not be a substantially bigger problem than a large queue of unwanted messages.) With multiple pieces of software (at least SendMail and Microsoft Exchange), this causes the E-Mail server to basically stop keeping track of the messages. Then, move the messages out of the queue. (If the messages are files, directions on moving files may be found in the section about moving/renaming files.)

  • Preferably the messages will go to a directory that is not part of the queue, so ../otherdir in Unix or ..\otherdir in DOS and compatible/similar.
  • If the files go to another location on the same mount point (or “drive letter”), this can often be done using a “rename” method which makes simple changes on the filesystem and is much faster than also needing to read and write all of the data. For instance, if /var/spool/ is the mount point, moving data from /var/spool/mqueue-tx/ to a newly created /var/spool/mqueue-tx-large-queue/ will still be on the same mount point.
  • Another option may be to move them to oblivion: this is saying the same thing as “deleting” the files.
  • Another option might be to leave the messages where they are, but tell the E-Mail software to start using a different location on the hard drive. This seems to be the approach discussed by a portion of MS KB 324958.

After the mail server is not going to notice the E-Mails once the mail server is started, then go head and start up the E-Mail server again, so that downtime ends.

Then, now that things are back to a “low pressure” scenario, determine what to do with the messages. For example, hunt for any messages which actually may still be good/wanted. (Are all of the bad messages showing false senders from incorrect domains? Are there messages fromthe correct domains? Do the bad messages seem to have common phrases (such as sexual terms which may be clearly inappropriate for the professional environment being used)? If so, use the error/result/return code from grep in Unix, or find in DOS/Windows, to weed out all of the messages containing some of the clearly unwanted phrases. If that process can seperate out 98% of the messages, then perhaps the remaining 2% of messages will be a managable amount of messages to just be manually reviewed.

Zero out mail queue files

...

This method has the downside of being destructive, so moving queue files out may be a preferred approach. On the plus side: Unverified: There is probably very little, if any, risk to doing this even when the server is running.

SendMail may use two files for each message: one containing the message body and one containing a “control file” that helps to keep track of the status. Using this approach might theoretically have some issues from deleting one file related to the message, but not the other file before SendMail tries to act on the message again.

SendMail processes zero-byte messages very quickly. Try overwriting all of the messages with a zero byte file. (Then process the queue.)

(Note: some preliminary testing seemed to indicate that this didn't work. The files just remained as zero byte files. However, when checking a brief while later, all the files were in fact removed. It is believed they were removed by SendMail. Perhaps a cron job did a better job of getting the queue to be processed?)

Here are some specifics for software:

SendMail (zeroing queue)

Just read through the generalized directions starting at the very beginning of the section about flushing the E-Mail queue.

Microsoft Exchange (zeroing queue)

Some of this information might be meant for Exchange 2003. This might or might not apply as well to other versions.

Using GUI to remove a message

There is a way to do this in the GUI. As I recall, it involves something like viewing the queue, and then double-clicking on a domain, and then pressing Ctrl-A to highlight all of the E-Mails, and then right-click and specify how to handle the E-Mails. See KB 822944: How to Delete Messages from Queues in Exchange Server 2003.

However, that is very unpleasant (especially because the process is a bit slow) if there are hundreds of domains. So, in that case, there are other approaches.

Moving messages out of queue

Perhaps the better method is to move queued messages out because this method allows the messages to be manually reviewed.

This does cause a bit of downtime. (Estimation: Could probably be less than a minute, but could be several minutes depending on how many messages there are and how fast the hardware is.)

Connectors

The other method involves temporarily setting up a new E-Mail connector that doesn't actually connect to the Internet. This can lead to Exchange going through the queue very rapidly. Or, an alternative might be to adjust an existing connector. This is probably doable from within the graphical interface.

MS KB 324958's “Clean up the Exchange Server's SMTP queues” section. Perhaps see: KB 821901 http://srussking.blogspot.com/2006/02/easily-delete-all-mail-in-queue.html http://www.squidworks.net/2010/05/cleaning-up-the-exchange-2003-servers-smtp-queues-after-ndr-attacks/ also discusses Exchange 2000.

Other software

forum post mentions an alternative. (Note that this is currently untested by the author of this text.) Speculation: This probably only works on some versions of Exchange; probably not Exchange 2007 or later. Download the aqadmcli command and use “ aqadmcli delmsg flags=all ” although another post seems to indicate the delmsg flags=all is something to type after typing “ setserver <servername> ”.

Tracking Mail

The typical approach is to check E-Mail logs. (Another approach might be to see the message if it is still in a queue.)

Microsoft Exchange

The precise method of getting this information has been known to change with different Microsoft Exchange versions.

Getting/seeing the log info

TechNet: Configure the Location of Message Tracking Logs states, “By default, the message tracking logs are stored in the C:\Program Files\Microsoft\Exchange Server\TransportRoles\Logs\MessageTracking directory.”

Another resource which may be useful: How to Determine the Database and Log File Locations of the Files You Are Restoring

Newer Exchange versions

Exchange Management Shell is basically PowerShell with some components added on.

TechNet: Search Message Tracking Logs

Here is an example command:

Get-MessageTrackingLog -ResultSize unlimited -Start (Get-Date).AddHours(-48) -Recipients userone@example.org,usertwo@example.net | Out-GridView

That will show any E-Mail going to either of those E-Mail addresses over the last 48 hours.

Resulting Output

Actually, that pops info into a GUI, and seems to limit the amount of useful information that is output. For instance, timestamps may not be shown. So, before showing how to customize the above search, let's first discuss how to adjust the output so it is better:

C7Solutions.com: “Missing the Message Tracking Log Explorer in Exchange 2013? Not anymore...” notes, “to the rescue comes a PowerShell 2.0 feature called Out-GridView.” Though, I tended to favor outputting to HTML, as that seemed to include more information (including timestamps); I would also prefer to edit the file's “<table ” tag to say “<table border ” as that seemed to make things much more readable.

md C:\Temp\
Get-MessageTrackingLog -ResultSize unlimited -Start (Get-Date).AddHours(-48) -Recipients userone@example.org,usertwo@example.net | ConvertTo-Html > C:\Temp\TrackLog.htm
notepad C:\Temp\TrackLog.htm

Then find add the border... Find the “<table” tag and modify it to say “<table border”. Save the result.

View the result:

start C:\Temp\TrackLog.htm

When done, clean up after yourself by deleting the results (if they will never be needed in the future).

del C:\Temp\TrackLog.htm

Options can be used to adjust what results get shown.

Narrowing the date
Start time

You can specify an exact time. Often, a relative time might be easier. Here is a way to specify the last 48 hours:

  • e.g., “ -Start (Get-Date).AddHours(-48) ” for the last two days
End time
(Can also be specified)
Narrowing to user(s)
Sender

Either specify:

-Sender "user@example.com"

Or pipe results using Where-Object (as shown in the “Recipient(s)” section).

Recipient(s)

You can search using Recipients.

Get-MessageTrackingLog -ResultSize unlimited -Start (Get-Date).AddHours(-48) -Recipients userone@example.org,usertwo@example.net,usrthree@example.com | ConvertTo-Html > C:\Temp\TrackLog.htm

Since a message can have multiple recipients, the option is named “-Recipients”. However, it is possible to include just one recipient.

Paul Cunningham's “Searching Message Tracking Logs by Sender or Recipient Email Address” (article on exchangeserverpro.com) stated, “It doesn’t matter whether the recipient was in the To, CC, or BCC of the message, the search will return any match regardless.”

The same article clarifies, “You can specify multiple recipient SMTP addresses simply by separating them with a comma. When you do this the condition is an “or” not an “and”. In other words, any messages with any one of the recipients will be returned in the results, they do not need to be messages sent to all the recipients.”

or, if you want to user a wildcard instead of just an E-Mail address, add this after the search (but before piping to an output method)

| Where-Object {$_.recipients -like "*@gmail.com"}

or...

| Where-Object {$_.recipients -match "gmail"}

The examles showed only a sender or recipient(s), just to keep the exmaples easy. You can use both options.

Specifying subject
-MessageSubject "customizable"
misplaced info
This was about showing users:

It may be helpful to import a module first. (Or this might not be needed.) If this is needed:

impo active-directory

(The impo command is the same as the import-module command, and the module might be abbreviatable as “ac*”.)

Understanding the log info

The logs may use key words, like “DELIVER” or “SEND”. Such words are called the “Event ID” of an action. (These are Exchange Event IDs, and are not meant to be identical to the “event ID” numbers from Windows Event Log Messages.)

If you see a reference to Categorizer, that can refer to third party anti-spam software (like Sophos's PureMessage add-on for Microsoft Exchange).

To understand these words, see TechNet: Message tracking.

Other details about the logs may be seen by: Description of Message Tracking Log Fields.

Using Milter support
Overview

Some MSA/“incoming MTA” software implementations support running an E-Mail through a “milter”. The term “milter” does mean “mail filter”. If the E-Mail is rejected by the “milter”, then the receiving MSA/“incoming MTA” software can inform the sender that the message is rejected. This is preferred over having the MSA/“incoming MTA” accept the message, at which point the sender might disconnect from the SMTP server that receives the message. At that point, the MSA/“incoming MTA” either must successfully deliver the message, or violate specifications/standards, or possibly avoid the violation by informing the sender of the problem. However, informing the sender might not be an approach that the E-Mail standards consider to be a legitimate way to handle an already-accepted E_Mail message. Even worse, trying that approach might not be successful (if any attempt to reply is unsuccessful, such as if the sender does not receive E-Mails from the Internet location (IP address or possibly DNS name) where the E-Mail was sent from. Furthermore, any automated attempt to send an E-Mail to the initial sender may be viewed as completely irresponsible, as that E-Mail may actually contribute to spam. (The primary way this would end up being spam is if the initial sender somehow lied about identity.)

[#sndmmltr]: Using Milters for SendMail
Verifying Milter support

guide recommends checking that the feature is supported. Use:

echo /quit | sendmail -bt -d0.1

... or, if this is easier to remember:

sendmail -bt -d0.1 < /dev/null

The sendmail program will recognize the abnormal standard input, and so will quit after outputting what it needs to.

If you get: “sendmail: unknown option -- d” then make sure you're actually running SendMail. This output can occur if the sendmail executable is actually MailWrapper, and if /etc/mailer.conf is not pointing to the /usr/libexec/sendmail/sendmail file. (However, if that is indeed the apparent problem, note that this probably means that SendMail is not handling mail. This can be easily worked around, and milter support can be checked easily, by just running the correct executable with the full path. However, altering the text file may break another E-Mail solution that is actively being used.)

After the version number, there will (in all probability or certainty) be a line that says “Compiled with:”, and then there will be several “compile option” tags in capital letters. See if one of those “compile option” tags is the word “ MILTER ”. If so, then milter support has been verified.

If Milter support is not part of the version of SendMail that is being used, then take care of that before trying to use Milters. The README.milter file from OpenBSD's amavisd-new package may provide some details on how to get libmilter enabled.

[#sndmcfmc]: How to update SendMail *.mc and *.cf configuration files

First, a quote found on Mark D. Roth's website of “Technical Stuff”:

He who has never hacked sendmail.cf has no soul;
he who has hacked sendmail.cf more than once has no brain.
- Old Hacker Proverb

A page on contrasting .cf files vs. .mc files provides that quote, simply citing it as an “Old Hacker Proverb”. (This was initially posted because it seemed like it just came up randomly out of nowhere. Later, it turns out that this appears to be a modification of another old proverb that predates SendMail, quoted at Quotes Falsely Attributed, John Adams Said It First.)

Overview

The OpenBSD's manual page for “afterboot” discusses some configuration files (which are also discussed by the sections on SendMail configuration files and SendMail available configuration files). Two occurrences where a single paragraph in the manual page takes the time to reference how those files were made. This may be understood more if following some advice to customizing the configuration.

Information from README file about sendmail Configuration Files shows that yet other choices might be available by checking out the /usr/share/sendmail/cf/*.mc files, although first they must be converted to a valid *.cf file.

OpenBSD 4.8's Manual Page titled “afterboot” has stated, “Please see /usr/share/sendmail/README and /usr/share/doc/smm/08.sendmailop/op.me for” information “on generating your own sendmail configuration files.” (Some other notes, which may be older, had stated that OpenBSD's manual page for “afterboot” said, “Please see /usr/share/sendmail/README for information on generating your own sendmail configuration files.”)

viewing the text file locally, or going online to OpenBSD's CVSWeb section of files from /etc/mail/. The /etc/mail/README file states, “You should make changes in the corresponding .mc file and not edit the .cf files directly.”

The /etc/mail/sendmail.cf from OpenBSD states:

DO NOT EDIT THIS FILE!  Only edit the source .mc file.

(Hyperlink added to the quoted text.)

It seems fairly clear, then, that customizations should be applied directly only to the *.mc files. The desired way to change one of the *.cf files is to modify a *.mc file (and then use a proper procedure to create an updated *.mc file.) That is clearly why the previously mentioned (OpenBSD “afterboot”) manual page was making such a point about the origin of each referenced *.cf file.

.cf files vs. .mc files covers the differences intended to exist between *.cf files and *.mc files. Presumably *.cf stands for “configuration” while *.mc presumably stands for “m4 configuration”: presumably the “m” stands for “macro”. (Further details may be found at: Wikipedia's article on m4.)

[#mkmctocf]: How to use a *.mc file and properly create an updated *.cf file

The SendMail documentation for c4 refers to a directory called ${CFDIR}, which should point to /usr/share/sendmail on a typical OpenBSD system. Setting this variable is being done for the purpose of simplifying the remaining examples in this documentation.

export CFDIR=/usr/share/sendmail
Trying an easy way

If the file modified is using the same filename as one of the original configuration files, then perhaps the following may work:

cd ${CFDIR}/cf && sudo make && sudo make distribution
echo Err=$?
Alternate approach (needed if using other filenames)

Otherwise, there is another command that may be tried. However, you should first figure out what the configuration file is. Try to find it with a command such as:

find -iname ${CFDIR} cf.m4

e.g., in OpenBSD the resutls may be ${CFDIR}/m4/cf.m4 but the amavisd-new package's /usr/local/share/doc/amavisd-new/README.sendmail-dual file provides a different location for this configuration file, which is the equivilent to ${CFDIR}/cf/m4/cf.m4

m4 ${CFDIR}/m4/cf.m4 filename.mc > output.cf

e.g.

export CFDIR=/usr/share/sendmail
m4 ${CFDIR}/m4/cf.m4 ${CFDIR}/cf/openbsd-proto.mc | sudo -n tee /etc/mail/output.cf

Note that tee is not using -a, so this overwrites (and does not append).

Testing results

Search the *.cf file for some content related to what was added to the *.mc file file. For example, the following may be an ideal search term to verify the successful addition of support for virtual domains.

grep -i "F{VirtHost}/etc/mail/virtual-domains" /etc/mail/sendmail.cf

For other tasks, the part between the quotation marks should be something different and related to the task that was performed. Specifying filenames, or even entire paths, may be a good bet of something to search for.

After this seems to be updated, try running “ sudo mailq ”. Doing that is known to complain about problems in the resulting *.cf file, so it may be a good quick way to notice some types of errors.

Modifying a Sendmail configuration file to start using Milter support

This includes an overview of the lengthy process of changing a SendMail configuration file. This does not provide substantial details about any specific Milter. (Follow directions for using a specific Milter. However, when those directions reference a SendMail configuration file using a reference to the INPUT_MAIL_FILTER function, then these directions may start to become a bit more useful.

[#cfmkwhmc]: Determining the source *.mc file

The recommended way to have a customized configuration file is to modify a copy of a pre-existing configuration file. This requires determining which pre-existing configuration file to use as a starting point.

The answer might be as clear-cut as reading documentation. As an example, OpenBSD 5.5's manual page for “afterboot” (and earlier versions, before SendMail was removed from OpenBSD for OpenBSD 5.6) documents which files were used for creating the /etc/mail/localhost.cf file and, later in the same paragraph, the file used for creating the /etc/mail/sendmail.cf file.

In OpenBSD, openbsd-proto.mc is the correct filename. The phrase openbsd-proto.mc shows up three times in the /etc/mail/sendmail.cf so another way to determine the desired filename might be:

grep -i \\.mc /etc/mail/sendmail.cf

First, determine the source configuration file that was used to make the configuration file that SendMail is using. If SendMail is actively running, this may be done by seeing what is actively running and checking out what filename is specified by the -C parameter.

As an example, let's say the file being used is the /etc/mail/sendmail.cf file. Use:

grep -i ".mc" /etc/mail/sendmail.cf

On an OpenBSD machine, that should show a filename that keeps re-appearing in comments throughout the file: openbsd-proto.mc. In that case, run:

export CFDIR=/usr/share/sendmail
ls -l ${CFDIR}/cf/openbsd-proto.mc

Or, let's try another method that may work without relying on the comments. (This method does not work for the file just described.) If the configuration file is named /etc/mail/localhost.cf then take the base part of the filename, and run:

export CFDIR=/usr/share/sendmail
ls -l ${CFDIR}/cf/*localhost*
Recognize: there may be more

OpenBSD's /usr/share/sendmail/cf/openbsd-localhost.mc file is only two lines. The first line sets an option. The second line refers to the /usr/share/sendmail/cf/openbsd-proto.mc file that stores all of the other configuration options. We can therefore conclude that editing /usr/share/sendmail/cf/openbsd-proto.mc would also impact the results that occur when using the /usr/share/sendmail/cf/openbsd-localhost.mc file. This is not necessarily a terrible thing, but probably is good to be aware of.

[#sndmltmc]: Applying Milter-related customizations to an m4 configuration file

First, back up the source file. The following commands do this, and then start editing a copy. (The copy should not pre-exist: if it does, then the user will be prompted. In that case, specify “n” (so the file is not overwritten) and customize the name of the copy.)

export CFDIR=/usr/share/sendmail
cpytobak ${CFDIR}/cf/openbsd-proto.mc
sudo cp -i ${CFDIR}/cf/openbsd-proto.mc ${CFDIR}/cf/openbsd-proto-pre-custom-1.mc
echo ${VISUAL}
sudo ${VISUAL} ${CFDIR}/cf/openbsd-proto.mc

Yes, we are backing up a file and then modifying the original filename, not the backup copy. This is because a later process (specifically, the make command) may use the original filename, so the original filename may be the most convenient filename to store the updated file contents.

The first thing to know about this file is that the word “dnl” stands for “delete” these characters and all other characters up “through newline”. Therefore, all lines that start with “dnl ” are comments. (This has been seen documented in one of the comments of a bundled/example file.) The “dnl” does need to be a word, so it does seem that the white space after these letters may be non-optional. (However, this has been seen directly after a right parenthesi, so there is no strict rule that there must be white space directly before the word.) (The # character has been seen in documentation (the README.milter from OpenBSD's amavisd-new package), but that does not appear to be valid with at least one implementation (OpenBSD's SendMail). Also, unexpected blank lines do lead to errors being reported. So, start all comments with “dnl ”.

old guide to using OpenBSD as a content-checking mail server (archived by the Wayback Machine @ Archive.org): “configuring sendmail” section provides some details.

Find a spot in the file to place custom configuration. Since we're going to be adding a line that says “INPUT_MAIL_FILTER”, the most appropriate spot would be next to other lines that say “INPUT_MAIL_FILTER” if there are any such lines. (There probably are not.) Otherwise, try right after the line that says “dnl End of masquerading section.” Otherwise, just find a(n uncommented) line that starts with “define” or “FEATURE”, and which is then followed by several commands, may be a good spot to put custom lines after. (Place the custom lines after the “define” or “FEATURE” line, and before the comments that come afterwards.) (Or, perhaps any line will work as long as the location is not within a multi-line configuration detail like the ifdef as seen in an original file. The exact rules have not been verified at the time of this writing.)

To be a bit less intimidating, we'll start with some actual examples. (Note that this file uses left directional (“backwards”) apostraphe for the left side of a string, and a standard non-directional apostraphe for the right side of a string.)

INPUT_MAIL_FILTER(`smtp-vilter', `S=unix:/var/smtp-vilter/smtp-vilter.sock, F=T')dnl

Note: A lot of the following information was learned by reviewing old guide to using OpenBSD as a content-checking mail server (archived by the Wayback Machine @ Archive.org): “configuring sendmail” section. So, credit is due to Paul Matulis who wrote the guide, according to OpenBSD Journal entry that references a guide to “Installing an OpenBSD mail filter gateway with smtp-vilter, Clam AV, and SpamAssassin”

The first string simply identifies a name for the milter.

The second string is a comma-separated list of options. From the documentation seen, it appears that the first option (to identify a socket) is required, but all others are optional.

The precise options to use may vary greatly due to administrator preference, and perhaps may even vary between different milters. So, this guide does not try to provide all-encompassing recommendations on what values to use for the options, but does try to provide some details on what the options mean. If following an installation guide for a particular Milter, that installation guide will likely provide some recommended values.

Here are some details about some options:

S= option

There are variations of socket types. After the socket type is a colon and then more details about the socket.

The first variation uses a “unix (or local)” socket type, and then specifies the name of a socket (which, perhaps traditionally, or perhaps according to a standard or requirement, or perhaps just due to one example that was seen, it seems that this should look like a filename that ends with “.sock”). This is demonstrated in the above example.

This has been documented as using unix: (as shown in the above example). This has also been documented as unix: (from a README.milter file in OpenBSD's amavisd-new package). Presumably both of these socket types are identical.

Another socket type is inet and does not require a filename. Instead, it requires a port followed by a colon, which is then followed by a hostname. However, IBM documentation on sendmail filter configurations indicates that this should include a port followed by an @ followed by the hostname.

IBM documentation on sendmail filter configurations indicates another socket type: “inet6”.

F= option

The two variations are:

F=T
If the milter is unavailable, then SendMail will “Temporary fail” the connection.
F=T
If the milter is unavailable, then SendMail will “Reject” the connection. (The error is not marked as being “temporary”, hopefully a legitimate remote computer will be less likely to retry.)

These are documented by: Sendmail.org documentation and Claus Aßmann's copy of Sendmail 8.12 Installation and Operation Guide.

T= option

This has a semi-colon separated list of sub-options. For example:

T=C:0;S:10s;R:10m;E:10s

(These might be some very terrible values used for this example.)

The letters just before the colons are references to various lenghts of time before some sort of “timeout”. After the colon is a time (which may be a zero, or a number followed by “s” to indicate the number is a number of seconds, or an umber followed by “m” which indicates that the number is a number of minutes).

Note that multiple mail filters have been known to suggest adding multiple lines to the *.cf file: the INPUT_MAIL_FILTER line might be just one of two lines to add. Read the documentation that comes with any Milter, and see if it provides instructions for a line that says INPUT_MAIL_FILTER and, if so, see if there is another line that should also be getting added at the same time.

Possible resource of additional advice that is more milter-generic (meaning that it might not be quite as specific to any specific milter) : SendMail documentation: Adding New Mail Filters

Using the updated configuration

Run the cf.m4 command to create a *.cf file. Let SendMail know to use the upated configuration: use “/etc/rc.d/sendmail reload”. Or, if using an older operating system where that isn't available, then send it a SIGHUP. e.g., perhaps the following (untested) command:

sudo kill -HUP $(sudo cat /var/run/sendmail.pid | head -1$)
OpenSMTPD

Status update on implementing Milter was closed (about 4 months prior to Auguest 6, 2013). “matter of weeks” ... “before we have a working filter API.” However, the text that was being quoted might have been referring to internal mail filtering, rather than Milter support?

Other/Misc

Some information related to using Microsoft Office 365.