- Proper spelling
There was a time, such as the 1990's, when the preferred spelling of E-Mail was ambiguous. When the term/phrase/word appeared in the middle of a sentence, there were multiple commonly seen options: “E-Mail”, “E-mail”, “e-mail”, “EMail”, “Email”, and “email” could all be regularly found. Within any one piece of writing, consistency was recommended. However, any strong advice to favor one style of writing the word, instead of other styles, was generally considered to be unsubstantiatable. A more widely accepted recommendation was for people to simply use a style that they liked.
The general preference by some formal institutions seem to now favor lowercase letters, and often to drop the hyphen.
The creator/founder of ][Cyber Pillar][ had selected a preference of E-Mail, and so that style is typically used rather consistently throughout the site. (Exceptions might be made when quoting another source.)
- Internet Message Format
The main official standard for E-Mail had been RFC 822: Standard For The Format Of ARPA Internet Text Messages from August 13, 1982 to the release of the newer RFC 2822: Internet Message Format released in April of 2001. It was a period of 18 years and about 8 months before that RFC 822 finally succumbed to having a successor. Even after that time, IETF STD 11: Standard For The Format Of ARPA Internet Text Messages continued to point to RFC 822, though IETF Standards List did note that STD 11 was “Obsoleted by RFC2822”.
- Address Format
The most well-known E-Mail address format involves showing a username, followed by the “at sign” (“@”), and then a domain name which may be looked up via DNS.
- Transporting Internet E-Mail
- [#emailmxa]: Some common “agent” acronyms
Very often, people who start to set up an E-Mail server may be expected to follow some documentation, and the documentation expects that people are already familiar with several acronyms for the various roles of software related to E-Mail. Unfortunately, people often do not already have that familiarity, and also the documentation will very often not provide a clear overview on what these terms mean. As a result, many people have tried to find such an overview and, if unsuccessful, have just tried to muddle through documentation without clearly understanding the different roles.
The documentation using to these acronyms roles may use the “MxA” terminology to refer to various “agent” roles.
With some early E-Mail systems, there were various different pieces of software that performed various functions. With newer E-Mail software, one piece of more complex software will often perform the functionality that had historically been provided by multiple different simpler software programs. Even though some of the more modern software may perform multiple functions, the documentation and setup of some of the software will still refer to the individual roles that had historically been provided by different programs. For this reason, familiarity with some of the simpler roles can be useful, even though different roles may often be provided by a single piece of software.
Some documentation might refer to some of these tasks as being handled by a different role than what is described here. This is generally because the documentation is simply not even mentioning one or more of the roles. Therefore, the tasks that should be handled by the unmentioned role will be documented as being handled by a different role. (As common examples, the role of MRA is often not mentioned, and the tasks performed by the MDA role are sometimes described as if the tasks are being handled by whatever software is performing the role called MTA.) There might even be some confusion by some documentation authors. Sadly, the best advice to be given is this: Just don't get too shocked/confused if some documentation suggests that one role is performing a task that really should be handled by a different role. Despite the inconsistency that can exist, being familiar with the major tasks of a role will still help with understanding some documentation.
To ensure that these definitions really do appear to be technically right, some effort was spent learning the proper intent of each role. This section provides a detailed breakdown that is the most consistent with various E-Mail standards and other documentation that use these terms. The result is that the definitions provided here are as consistent as possible with as much documentation as possible.
Here is a brief summary of these roles. E-Mail administrators are advised to become familiar with the important acronyms related to these roles.
- [#emlmsmxa]: Mail Submission Agent (“MSA”)
A server that receives E-Mail from authorized/local users (such as users on a local network), and then decides how that E-Mail should get processed.
The main job of an MSA is to be receiving E-Mail from authorized users, such as users on the local network. Remote users who have an account on the local network might also send E-Mail to the MSA. When doing so, these remote users typically provide some credentials that authenticate the user, so the local mail server knows that this person is authorized to send E-Mail using this local mail server. (E-Mail which is received from other sources, such as remote E-Mail servers, may actually be handled by a different role, which is the role called MTA.)
Further discussion on MSAs (and MTAs) are covered by the section on E-Mail servers. (Since there is often much in common between the MSA role and part of the MTA role, very often the same software performs both of these roles.)
Once the MSA receives E-Mail, a very common process for the MSA is to determine whether the E-Mail is to a local user or a remote user, and then simply pass the E-Mail onto another role. If the recipient is for a remote user, the MSA just gives the E-Mail to a role called the MDA. Otherwise, the MSA just gives the E-Mail to a role called the MTA.
- [#emlmdmxa]: Mail Delivery Agent (“MDA”)
Receives E-Mail and makes sure that the E-Mail gets to the destination where the E-Mail needs to go. This typically involves placing E-Mail into a local user's mailbox.
The MDA may be a part of an E-Mail server. For example, an ISP may use an MDA to put E-Mails in an end user's mailbox. In this case, the MDA is storing E-Mails, and the MDA may often be referred to as a “message store”.
On a server, if the user has some sort of E-Mail forwarding set up, then the MDA will handle that. (In theory, the MDA could handle that simply by creating a new message and sending the new message to an MSA).
Another type of MDA is for the MDA to be part of an E-Mail client. For example, E-Mail client software may place the E-Mail in the E-Mail client's Inbox, or might sort E-Mail and place E-Mail into a different E-Mail folder. So, after the ISP's MDA has already sorted the E-Mail into an individual user's primary E-Mail box, the user's E-Mail client may use a different MDA to sort the E-Mail again. It is entirely possible for multiple MDAs might handle the same piece of E-Mail. In each of these cases, the MDA is performing the job of making sure the received E-Mail is getting placed to a desired destination.
- Networking may be optional
An MDA may communicate with software performing other roles, such as the role of being an MSA (on an E-Mail server) or the role of an MRA (which may commonly be run on the computer running the E-Mail client software). One possible way to implement these other roles is to have the software write to a temporary file, and then run the MDA software (just like any other program that can be started from the command line). In this type of setup, the MDA can read that temporary file that was created, and deliver the message to a local mailbox. In this type of scenario, the MDA completely performs its primary duty, and can do this task without requiring the use of any network protocols like TCP/IP. (The software performing the other role presumably used the network, but the role of the MDA did not need to use networking for this specific simple example.)
The section about E-Mail delivery has more information about MDAs.
- [#emlmtmxa]: Mail Transfer Agent (“MTA”)
Software which is a Mail Transfer Agent will deliver mail to a remote system. The primary way to identify an MTA is the ability to send E-Mail to a remote system.
Additionally, the MTA may handle incoming E-Mail which comes from a remote MTA. (Incoming E-Mail which comes from an MUA does not typically go to the MTA. Instead, incoming E-Mail which comes from an MUA goes to an MSA.)
Further discussion on MTAs (and MSAs) are covered by the section on E-Mail servers.
- [#emlmrmxa]: Mail Retrieval Agent (“MRA”)
A program on the end user's computer that contacts remote mail servers (using protocols such as IMAP4 or POP3). The E-Mail is then retrieved by the MRA, and may then be handled by an MDA. (At this point, the MDA might be built into the E-Mail client software. At this point, the MDA might simply be storing E-Mail into separate E-Mail folders on the computer that runs the E-Mail client software.)
It seems (from Wikipedia's page on “Email agent (infrastructure)”: section titled “Classification”) that the term “MRA” might refer only to programs that obtain E-Mail from remote systems “but no other client functions.” For instance, if a program performed both the roles of an MRA and an MUA, the program would just be called an MUA.
There may be various terms used for a server that the MRA contacts. Perhaps the MRA might get E-Mail from an MSA/MDA/MTA?
For more information about MRAs, see: Mail Retrieval Agent.
- [#emlmumxa]: Mail User Agent (“MUA”)
This is the software that an end user will use. A traditional MUA has a primary job is to display E-Mail onto a screen, and allow a user to type up E-Mails. The MUA will typically send outgoing E-Mail to an MSA. The exception is if MUA has built-in functionality to be able to act like an MTA. In that case, the MUA effectively acts like an MUA, MSA, and MTA, and the MUA is performing the role of an MTA when it sends E-Mail to a remote MTA.
With the traditional pull-type E-Mail system and software design that involved using separate programs for separate tasks, the MUA gets E-Mail by running the MRA. However, very often MUAs now commonly contain, built-in, the basic functinality of an MRA.
Some Internet standards may refer to client software as being an MUA (even if the Internet standard is really referring to software that performs the role of an MRA).
For more information about MUAs, see: E-Mail programs.
- [#emlmsp]: Mail Service Provider (“MSP”)
Wikipedia's page on “Email agent (infrastructure)” discusses this concept, and has some text about five specific roles (MSA, MTA, MDA, MUA, MRA). These roles may also be discussed by Wikipedia's entry for “Message handling service” (which redirects to the article on “Email”, and more specifically to the section called “Operation overview”).
- [#getemail]: Getting E-Mail
- [#svgeteml]: Servers getting mail
This is mainly about how to interact with other servers. For information about servers getting E-Mail from users running E-Mail client software, see the section about users receiving E-Mail.
- E-Mail storage
E-Mail data must be stored somewhere
- Using local inboxes stored on the E-Mail server
End users typically may access this E-Mail using E-Mail client programs
- [#mailboxs]: Formats of storing E-Mail for one or more end users
Taking the E-Mail and putting it info a mailbox. There may be various formats that E-Mail is stored in.
- [#maildir]: maildir
In this format, E-Mail messages get stored into multiple files. A directory is created for each user. Underneath this directory, subdirectories named new, cur, and tmp exist. MDA software places new messages into the subfolder named tmp and then, once the E-Mail message is fully written to the disk, the E-Mail is moved/renamed so it appears in the new subfolder. Presumably MUA software will move software from new to cur as messages get read. Additional files and/or directories may exist and are documented at the end of the official maildir specification by D. J. Bernstein.
- Support by file systems
When storing E-Mail in the “maildir” format, E-Mail messages may/do have a colon in their name. This is not compatible with common file systems used by Microsoft Windows. (This issue is documented by Wikipedia's article on the maildir format: section about Microsoft Windows compatibility mentions this.)
There are some advantages to this format, such as possibly enabling greater speed and possibly reducing problems with file locking. However, this can result in many small files existing on a file system. That may be unoptimal for some file systems. Wikipedia's article on the maildir format: “Technical issues” section: subsection called “Locking & scaling” notes that filesystems may limit how many files exist in a directory.
For small organizations, most/all data may be accessed using files that do not need to have any active operating system file locks. Wikipedia's article on the maildir format: “Technical issues” section: subsection called “Locking & scaling” describes why large-scale usage with NFS results in this design “scaling poorly.” The article mentions that this format “suffers from the inherent scaling limitations of any one-email-per-file email storage design.”
This is commonly supported by Unix software.
- [#maildirp]: maildir++
- Mentioned by Wikipedia's article on maildir: section about Maildir++.
- MH (“Message Handling”)
- See: Wikipedia's article on “MH Message Handling System”.
- [#mbox]: mbox
A method of storing E-Mail locally, and Unix's traditional method for this until maildir started gaining some popularity. RFC 4155 - The application/mbox Media Type says that the manual page for the mbox format at
qmail.org.should be treated as mostly authoritative” but “that there are many more potential variations that can be encountered in the wild.”
- Microsoft Exchange Store
(Perhaps sometimes called “Microsoft Exchange Information Store”)
- Software that supports this method
- Microsoft Exchange
- Microsoft Exchange 2007
It seems that Exchange Management Tools for Exchange 2007 didn't come out until Exchange Server 2007 SP1. Evidence: Guide to managing Exchange 2007 from an XP machine discusses how to use the software provided by Microsoft Download Center: Download Details for Exchange Server 2007 Management Tools (32-bit). (The guide includes details of other software to install.) Interpreting some text from this guide, the following options seem to be available to administer Public Foldersthis software:
- Using Exchange Management Shell
- Using Exchange Server 2007 Service Pack 1
- This guide seems to acknowledge this as a future option. (The guide was likely published before SP1.) (This text might be able to be updated.)
- Installing Exchange Server 2003 Exchange System Manager on a workstation
- Details are provided at Guide to remotely administering Exchange 2003.
- Other formats
- Mail servers and E-Mail client programs may have their own formats.
- E-Mail archived on a local machine
E-Mail client programs may often have an option to copy or move mail from a server onto a local machine.
- [#usrgteml]: Clients getting mail from servers
- Sending E-Mail
One option should be to run a test by manually communicating with SMTP. Unix systems have traditionally come with an executable file named
and may be able to use something like the following syntax:
- Dealing with dial-up:
(These instructions are not very complete, although the referenced material may be.)
- Consider mail server tutorial page (archive.org'ed version that isn't cut off)
sendmail and re-run with the flags from
command to send E-Mail
- OLD/UNORGANIZED NOTES
- Allowing users to access E-Mail
See: users getting E-Mail
- Misc mail notes (need to be sorted into above)
- Some standard protocols
- [#esmtp]: ESMTP
- See RFC 1869, RFC 2821, 5321. E-Mail Headers and SMTP Commands lists commands added with ESMTP. For the AUTH command described by RFC 2554, there is a newer RFC 4954 updated by 5248.
- [#smtp]: SMTP
- Older text
An older protocol. Is ESMTP a superset of this?
Note (find RFC / etc???) that MAIL TO should have the address surrounded by appropriate inequality mathematical signs. However, with many mail servers, this may be optional. Likewise, many tutorials will leave them off. This is unfortunate because some mail servers will require those signs and so that may lead to confusion by those who are following tutorials that don't mention the signs.
Standard TCP port: 25.
HELO : Should say what system the connection will be appearing to come from (as the mail server sees things, after any sort of NAT on the client side has been performed). A domain name may be used (and is typically used). In some cases, the value after HELO may be highly inaccurate, but in other cases the mail server may perform some checks on the provided value, such as whether reverse DNS of the IP that the client comes from matches the value provided by the HELO command (and the MAIL FROM commands and SPF checks).
Headers: More headers may be entered: Perhaps the most commonly used header with test E-Mails is the subject line. This may be omitted, in which case most mail agents will show the subject line (because they always do, and they will show the subject line) as blank. Such E-Mails may also be more likely to be treated as spam (which is a statement made from pratical experience that has been observed, rather than a statement with lots of strong official standards, like RFCs, pointing to any reasons why).
- Newer text
- Able to use Telnet
SMTP is unencrypted and uses a TCP connection. Many servers allow connections where the client may send data quite slowly. It is pretty heavily text based. All of these factors contribute to a situation that an SMTP connection can often be created using client programs that support Telnet. The first trick is to make sure the client knows what TCP port number to use. (Typically clients may try to use Telnet's default of TCP port 23 or perhaps SSH's TCP port 22. In either of those cases, overriding the default will typically be necessary, as the default TCP port for SMTP is TCP port 25.
Some guides to doing this: SMTP by Telnet guide (is often rendered in small text, and so may need to be resized), MS KB Q153119, TechNet: How to use Telnet to Test SMTP Communication
- Text from the server
Generally referred to as reply codes, these could be success code
messages or error code messages.
- Basic status code
RFC 640 (FTP Reply codes) describes reply codes as ending with a line that starts with a number and a space. Additional lines may exist earlier, with the first line starting with a number and a hyphen (unless the first line is the last line, in which case it needs to start with a number and a space).
(For SMTP, see RFC 821 Appendix E (page 48).)
For ESMTP, Reply (Error/success) codes: RFC 1893.
- Enhanced status codes
- In addition to the basic status codes, there are the enhanced status codes described in RFC 5248.
- [#smtpsamp]: An example
In general, this can be done by accessing an E-Mail server's “submission” port. For example, run the following on the E-Mail server:
If that command does not seem to exist for people using modern versions of Microsoft Windows, see the section on Telnet (especially the reference to using installing Roles and Features in Microsoft Windows Server operating systems, or some other method of Microsoft Windows Software Installation: Software Built-in/Bundled with the operating system. Or, alternatively (and generally preferred, but just a bit more work to set up), use PuTTY or some other SSH client (because SSH clients generally also support Telnet).
If the connection is refused, try connecting the server on the TCP port that is generally opened up for public connections:
Be careful with copying and pasting multiple SMTP commands. Gillis Chehade post about pipelining(/Gillis post about pipelining/Gilles post post about pipelining) states, “It's not polite to
QUITright after '
.' without waiting for
's acknowledgment that it accepted the mail :-)”. He is, in fact, an author (and very well might be the primary author) of OpenSMTPd. With at least early versions of OpenSMTPd, proceeding to the next phase immediately could lead to the SMTP server rejecting an E-Mail stating, “
” (Hyperlink to a section of quoted text. The referenced document, RFC 2920, is IETF STD 60 at the time of this writing.) So, do pause and wait for the server before proceeding to the next command. (This probably won't be a major issue for manual typists, but it can be an issue when using copy and paste.) The issue actually isn't specific to just the end of the message: this “pipelining” may even be an issue earlier in the conversation. Pasting multiple lines of the text of an E-Mail message may be okay, but until the “
500 Pipelining not supported
DATA” command is started, and also after the completion of sending the data, patience is in order. Going a bit slower might prevent errors (in some cases). Going as fast as possible should generally be alright, as long as a response from the server is received before sending the next SMTP command. These slownesses/delays may be unnecessary if the client starts with
EHLOand if the server's response references support of a feature named “
Here is an example of what may be sent to the mail server:
(The inequality signs may not be absolutely required.)
At this point, the server should give a positive acknowledgement. If there is an error message, try repeating the
HELOcommand. The server might actually work better after a second attempt. Once a successful
HELOis acknowledged, proceed with the rest of the E-Mail.
When done sending a message, proceed with:
MAIL FROM: <
RCPT TO: <
The inequality signs (the “
<” and “
>”) surrounding E-Mail addresses should be considered to be required. Some E-Mail servers may not require them. (Perhaps most E-Mail servers might not require the inequality signs.) However, some E-Mail servers have been known to require those inequality signs. (AOL perhaps?) And, the official E-Mail specifications do say that clients are supposed to provide the inequality signs, so those E-Mail servers are not entirely incorrect by enforcing the requirement. (These E-Mail servers are just not seeming to follow Postel's Law as closely as some other E-Mail servers.)
However, know that some of that content (in the sample just shown) is optional. On the lines that specify a person's name before an E-Mail address (which start with “
To:” and “
From:”), the space after the name may be optional. The quotation marks on those lines might also be optional, particularly if the user's name does not include a space. Actually, the names are optional.
Speaking of optional things, the entire envelope (the “
To:” line, the “
From:” line, the “
Subject:” line, the “
Date:” line, and the following blank line) can be omitted. The “envelope” refers certain lines that SMTP recognizes as having special meaning, and which show up at the very beginning of a message. So in the following example envelope, a person can use all, none, or some of the following lines. After starting the message, if an envelope is going to be used, it should be used right away:
Sat 31 Dec 2016 23:59:60 -0700
- Date: RFC 2822 section 3.6.1 says this “is specifically not intended to convey the time that the message is actually transported, but rather the time at which the human or other creator of the message has put the message into its final form, ready for transport.” (So, when a user tells software to send the mail, that is the date being mentioned here.) RFC 5322 section 3.3 (and updated document from RFC 2822 section 3.3) describes some fields on what that date may look like. (RFC 821 page 32 did not seem to support the day of the week.)
If an envelope is being used, then the blank line before the message body is required. Otherwise, the blank line before the message body is often considered to be optional, and quite unnecessary for the E-Mail to be delivered. However, without that blank line, the E-Mail does get successfully delivered but the message body may get treated like a header (viewable when showing full headers). At least the Alpine E-Mail client will expect that blank line for the message body to be treated like the message body. So, do include a blank line.
Then, proceed to type a message. End the message with a line that contains a single period. (If you actually want the E-Mail to contain a line that starts with a period, insert an extra period, as noted by RFC 5321 section 4.5.2.)
This is the body of the E-Mail. If headers are used, they need to come right
after the DATA command. After the headers there should be a blank line, and
that is where the body begins. To end the message, a line consisting of
just a period will likely work to end the DATA portion of the message.
The basic desire is to see an SMTP response 250 after ending the
DATAmessage. If the E-Mail server states “
” followed by some text like “
” or a message ID and the text “
”, then the E-Mail client has successfully sent an E-Mail and an SMTP server has acknowledged receiving the E-Mail. (If this is occurring, and if E-Mail is not being delivered successfully, then the SMTP server's logs should be checked to figure out what happened to the E-Mail.)
Message accepted for delivery
Also, consider using ESMTP. Basically, the only change for that is to start with “
EHLO” instead of “
HELO”. Doing this does two nice things: First, it verifies that the server supports ESMTP as well as SMTP. (An ESMTP-compatible server should also accept SMTP according to the ESMTP standards.) Secondly, an ESMTP server may respond by showing some information such as supported extensions and the maximum size of an E-Mail that may be accepted. If
EHLOis not recognized, which is probably fairly rare and will continue to be rare as time goes on, then the client may try to respond with “
HELO”. (If that fails, consider ending the connection if that hasn't yet been done, and trying to start with the
HELOcommand: The ESMTP standard demands that the computer sending E-Mail performs this behavior, of retrying, in order for its actions to be considered truely ESMTP compliant.)
(There may be other commands to introduce to a server that handles E-Mail, such as using the
LHLOcommand when talking to software that implements Local Mail Transfer Protocol (“LMTP”).)
Notice that mail servers may not accept a connection for various reasons, such as the slow speed of transmission of a manual connection (compared to text that is sent through automated methods) or just because the client is unrecognized and hasn't yet passed grey-listing. These may be anti-spam techniques and sometimes such techniques may violate the E-Mail standards. One technique may involve the server performing a Reverse DNS lookup for the IP address being used by the sending system, and possibly also a DNS request (for an SRV record).
- Some more notes
Technically, the inequality signs in the MAIL FROM and RCPT TO lines are required. In reality, many E-Mail servers treat them as optional, and will accept E-Mail addresses without the preceding less than sign and the subsequent greater-than sign. Unfortunately, some tutorials will even leave off the brackets, which becomes a problem when someone is simply following such a tutorial but is dealing with an E-Mail server that does actually require the inequality signs. A server is absolutely supposed to be allowed, by the E-Mail standards, to require the inequality signs (and actually leaving off the inequality signs is something that the server may not be granted permission by RFC 821 to do). Unfortunately, such a server may simply reject the invalid MAIL FROM command without a lot of clarification on what it expects. Using the inequality signs should never cause problems with a compliant mail server. The requirement for the angle brackets is specified by RFC 821 (page 30): Specifically the inequality signs that are in quotation marks that are outside of the square brackets. (The use of that requirement in the MAIL FROM and RCPT TO commands are dervied from pages 29 and 30.)
this page notes the interpretation without using the SIZE command: “If ommited, mail readers and delivery agents will try to determine the size of a message based on indicators such as them being terminated by a "." on a line by themselves and headers being sent on a line separated from body text by a blank line. But these methods get confused” with advanced scenarios such as embedded attachments. Therefore, the safest method for delivery is likely to use a SIZE command: Doing so will likely allow a period to appear on a single line by itself. Note, however, that the SIZE command is one of the ESMTP commands that is not part of the original RFC 821 SMTP protocol.
220-mtain-md01.r1000.mx.aol.com ESMTP Internet Inbound
220-America Online (AOL) and its affiliated companies do not
220-authorize the use of its proprietary computers and computer
220-networks to accept, transmit, or distribute unsolicited bulk
220-e-mail sent from the internet.
220-AOL may no longer accept connections from IP addresses
220 which no do not have reverse-DNS (PTR records) assigned.
Although AT&T's E-Mail server required the inequality signs, the error message it gave was something that could actually be rather helpful.
220 att.net - Maillennium ESMTP/MULTIBOX alnwmxc01 #243
MAIL FROM: email@example.com
501 need MAIL FROM:<name@domain>
220 mx1.example.org ESMTP spamd IP-based SPAM blocker; Fri Feb 5 07:18:34 2010 HELO example.net 250 Hello, spam sender. Pleased to be wasting your time. MAIL FROM: firstname.lastname@example.org 250 You are about to try to deliver spam. Your time will be spent, for nothing. RCPT TO: email@example.com 250 This is hurting you more than it is hurting me. DATA 451 Temporary failure, please try again later.
RFC 821 identifies commands: HELO MAIL RCPT (Recipient), DATA SEND SOML (Send or mail), SAML (Send and mail) RSET (Reset), VRFY (Verify), EXPN (Expand), HELP, NOOP, QUIT, TURN
D. J. Bernstein's page on SMTP greetings has some suggestions for servers.
- [#postofcp]: POP
- Standard Protocol
- Post Office Protocol is defined by IETF STD 53.
- Caution: E-Mails are often moved by default
- Many E-Mail clients will default to downloading E-Mail and then telling the server to delete the E-Mail, when POP3 is used. (This might not be as true when other protocols are used.) The reason for this seems to just be expected precedent. However, E-Mail programs do usually have an option (a checkbox) to disable this behavior of asking the E-Mail server to delete the E-Mails after they are downloaded. This may be a good and worthwhile thing to check when setting up POP3.
- Current status
POP3 can do the job of checking E-Mail. It is simple enough that E-Mail can be effectively checked using a Telnet application. It might (this is speculation, which may or may not be true) also be easier to use via Telnet, compared to alternatives like IMAP4. However, IMAP4 is likely the superior protocol, allowing for message organization (in E-Mail folders). So, when the user interface being used is a program that has specific support for E-Mail, generally people will probably have a nicer experience by using one of the newer protocols. Although POP3 probably does still have some usage, viewing it as a historical protocol may be the most worthwhile approach. Check out IMAP4.
- Historical versions
If POP3 is historical, there are even older variations. Yes, there's a reason why the version of POP is called POP3.
October 1984's RFC 918: “Post Office Protocol” (which is sometimes referred to as POP1) was marked as obsoleted by RFC 937: “Post Office Protocol - Version 2 in Feburary 1985. POP2 specified a default TCP port 109. The first standard called POP3 was released a few years later in November 1988's RFC 1081: Post Office Protocol - Version 3. As that pre-dated common Internet usage, any modern software is likely to be referring to POP3 (which uses TCP port 110).
Like IMAP, which really has only one common version (IMAP4), POP has only one common version: POP3. POP3 was the most recent version when this started to become popularly supported. Also, there hasn't been a new major version since then. So, any references to POP can generally be safely assumed to be referring to POP3.
Plain ol' standard POP3 may be rather insecure. There may also be enhancements, possibly involving a TCP port number other than 110. (Details may be reviewed at a later time.)
- Microsoft's “Messaging Application Programming Interface” over “Remote Procedure Call” (“MAPI/RPC”)
Some discussion on RPC is at: Tutorial: NFS security: RPC security/port.
- Restricting TCP ports
Social TechNet article: Configure Static RPC Ports on an Exchange 2010 Client Access Server says, “By default, the RPC Client Access service on an Exchange 2010 Client Access server uses the TCP End Point Mapper port (TCP/135) and the dynamic RPC port range (6005-59530) for outgoing connections, every time an Outlook clients establish a connection to Exchange.” Despite mentioning that port range, the web page elsewhere states, “Microsoft recommends you set this to a unique value between 59531 and 60554”.
In a nutshell, restrictions could be added by checking out these (untested?) commands:
REGQUERY HKLM\SYSTEM\CurrentControlSet\services\MSExchangeRPC /v
REGADD HKLM\SYSTEM\CurrentControlSet\services\MSExchangeRPC /v
Port" /t REG_DWORD /d
Also some directions to check out C:\Program Files\Microsoft\Exchange Server\V14\Bin\microsoft.exchange.addressbook.service.exe.config for a setting called
configuration file located in using Notepad, and looking at another registry entry:
REGQUERY HKLM\SYSTEM\CurrentControlSet\services\MSExchangeAB\Parameters /v
REGADD HKLM\SYSTEM\CurrentControlSet\services\MSExchangeAB\Parameters /v
"/t REG_SZ /d
At least that last registry entry seems to be something that may be re-applied after upgrading from Exchange 2010 RTM to SP1 or later.
That seems rather painful just to set a TCP port. So how did most Microsoft Exchange administrators perform such a task? They typically didn't restrict the TCP port. Some may have started to try to accomplish the task, but decided that since Microsoft is making things so absurdly difficult, maybe that approach is just not preferred.
Having such wide ranges led to challenges for any administrator that wanted to restrict allowed traffic to only specific allowed ports. Historically, the common way this was handled was to have users use a VPN so that then they could have access to more TCP ports. The only small hole that Network administrators would need to allow is for the VPN traffic, but once a person was authorized then LAN access to numerous ports could be supported.
Newer versions of Outlook have supported another method (presumably using SSL/TLS?) which allows for mail to be checked without going through as much hassle of connecting to a VPN.
- or is it SMTP AUTH?
??? (Should this go in this section, or by authentication/encryption/etc. section?)
- POP before SMTP
- Checking for new, then sending. This is something the server may like to see, simply a method of authentication. Wikipedia article on POP before SMTP
- Some software
- smtpd was added with OpenBSD 4.6.
- (Being developed for OpenBSD) OpenSMPTD website, tutorial. The old site at http://www.poolp.org/~gilles/smtpd/ seems to no longer be available.
- MTA may be installed by default in Debian? Exists for Cygwin: a basic guide is at http://www.chinese-watercolor.com/LRP/exim/exim-cygwin.html
- Microsoft Exchange Server
The often-used abbreviation “Exchange” is in fact ambiguous because there was also a “Microsoft Exchange Client” software used.
TechNet's Exchange 2003 documentation page on “Internet Protocol Details” shows the exact IMAP4 and POP3 commands supported, as well as statements on the RFCs supported by Exchange's implementation of the IMAP4, POP3, and NNTP protocols, and discusses Exchange's use of WebDAV (and HTTP).
http://www.bacula.org/3.0.x-manuals/en/concepts/concepts/New_Features.html says "Microsoft Exchange organises its storage into Storage Groups with Databases inside them. A default installation of Exchange will have a single Storage Group called 'First Storage Group', with two Databases inside it, "Mailbox Store (SERVER NAME)" and "Public Folder Store (SERVER NAME)", which hold user email and public folders respectively." It also describes the "Recovery Storage Group" as "required as the Standard and Small Business Server versions of Exchange can not ordinarily have more than one Storage Group."
Windows SBS 2003 note: If backing up Exchange with third party software, Q838183 details changing the "Disable Exchange Writer" value within HKLM\SYSTEM\CurrentControlSet\Services\MSExchangeIS\ParametersSystem\ from 1 to zero, and then restarting the MS Exch Info Store service. Doing this may cause two problems (which might be the same): “conflicts with the Backup utility (NTBackup.exe)”, and “the Backup utility cannot back up the Exchange information store and the system state at the same time.” However, this may help with third party backup solutions, such as recommended by Bacula DocuWiki: Application Specific Backups: Exchange Server.
- Lotus Notes
This guide just has a couple of pointers about Lotus Notes.
To send an E-Mail from the command line, Harbinder Singh's answer to salman's SuperUser.com question about sending E-Mail with Lotus Notes notes a need to be in the Lotus Notes directory when running the
command. (An example is shown.) A later comment on the page referred to a page related to the OpenNTF Command Line E-Mail Client (specifically, the Project page for the OpenNTF Command Line E-Mail Client), an open source program specifically designed for interacting with Lotus Notes.
Lotus “used the cc:Mail technology to enhance Lotus Notes” when Lotus acquired cc:Mail (as noted by Wikipedia).
- Wikipedia's page comparing mail servers: section titled “Feature comparison” lists licenses and support for key OS platforms (Unix and similar, Windows, and Mac OS X). Free ones for Windows that may not be mentioned above include: Meldware Mail Server/Meldware Communication Suite (LGPL), Apache James (Apache-licensed), Mercery Mail Transport System (closed source freeware), QPopper (POP3 but not SMTP or IMAP), UW IMAP (POP3/IMAP, not SMTP) and maybe Cyrus IMAP (might be for Windows, is POP3/IMAP, not SMTP).
- Checking E-Mail
Both incoming and outgoing E-Mail may be worth checking. Outgoing E-Mail that is considered abusive is not something that most organizations want to assist spreading, and there may be some reputation consequences such as bad press (perhaps) or being listed on a widely followed “registered E-Mail black list” (“RBL”).
See traffic handling: Preventing unauthorized Internet addresses from sending outgoing mail may help prevent outgoing malmail (e.g. spam). Limit how much mail can be sent to TCP 25. UCE Protect's guide for ISPs suggests checking to see how many signs of offensive behavior occur. Their specific example is if the remote end reports a non-existant user. The arguement is that if a large number of E-Mails is going to invalid users, chances are that it is an E-Mail attack. (A computer may be trying to brute force E-Mails or may be using a mailing list of harvested E-Mails.) For systems with a large amount of traffic, it may make more sense to use a system that looks at the percentage of unwanted E-Mails.
Incoming: Spam, Anti-Virus, etc.)
- Automatic mail handling
After checking for mail, the mail may
have been received (depending on when it was detected as undesirable).
Mail sorting can help prevent delivery. See also the anti-spam section.
- [#exmsimf] Microsoft Exchange's “Intelligent Message Filter” (“IMF”) (a.k.a. Microsoft Exchange's “Intelligent Message Filtering”)
IMF is integrated with Microsoft Exchange: originally, with version 1 of Microsoft Exchange 2003's Intelligent Message Filter (IMFv1), it was a downloadable add-on that Microsoft made available, but the download has since been removed. IMFv2 was included with Microsoft Exchange 2003 Service Pack 2. Microsoft page on Antispam Technologies: section in Intelligent Message Filter says “Based on SmartScreen technology, Exchange Server 2003 IMF provides server-side message filtering, heuristics-based message analysis, and support for per-message spam confidence level (SCL) ratings.” “The Exchange Server 2007 implementation of IMF includes anti-phishing capabilities”, and IMF checks several pieces of data.
TechNet's “Customizing Exchange Server Intelligent Message Filter” says, “By default, Intelligent Message Filter only filters messages and assigns SCL ratings to messages sent through anonymous connections. Messages sent by authenticated users bypass Intelligent Message Filter.” To change this, create/modify a registry DWORD in the registry key called HKLM\Software\Microsoft\Exchange\ContentFilter and use a value name of CheckAuthSessions and set it to 1.
(Untested) command line follows:
REGADD HKLM\Software\Microsoft\Exchange\ContentFilter /v CheckAuthSessions /t REG_DWORD /d 1
http://exchangepedia.com/blog/2006/05/imf-archiving-spam.html says some options are at: Global Settings, Message Delivery properties, Internet Message Filtering tab. (Another location is near SMTP, including choosing whether to scan SMTP, I'm guessing ???)
The archive location may be set in the registry.
- Combating spam
The “Checking E-Mail”
section has details about determining if E-Mail is spam. However, here
are some additional steps, like affecting how the mail server interacts
with the mail submitter.
Comparison of mail servers: Listing support for Antispam Features
may list several features in this section. (If new features become
prominent, perhaps they will appear on that list.)
- Determining if incoming mail is spam
This section isn't necessarily meant to be about forms of mail filtering that prevent mail from being delivered. That would be a separate section known as mail handling. The filters in this section refer to mail being filtered so that problematic mail that doesn't pass the filter will then be tagged/marked/rated/scored so that they can the mail handling routines will then know how to handle the mail. (A piece of mail may go through multiple filters, and some filters may shorten the whole filtering process because it may become known early on how the mail handlers are likely to handle a certain piece of mail.)
- Source filtering
- [#dnsspfus]: Usage of Sender Policy Framework (“SPF”) DNS records
Check whether the E-Mail is claiming to come from an organization that has a published SPF record, and if so, seeing if the sender's IP address is whitelisted from the SPF record. This method generally won't have too many false positives (blaming an innocent sender) because there isn't a lot of guesswork.
This requires using the IP address that connected to the server. Such checking may be performed by the server while the connection is still active (before the server acknowledges receipt of the message), or it could be performed after the connection if the server marks the E-Mail with enough details in the headers such as the RFC 2822-reserved “Received-SPF:” header or possibly other headers like “Received:” and “X-Originating-IP:” (assuming that the receiving E-Mail server doesn't allow forged data to give a false signal). For examples of these headers, see openspf.org site's page on SPF Received Header.
- Microsoft's Sender ID Framework (“SIDF”)
This is very similar to SPF, and the syntax of Sender ID even identifies itself as SPF (so Sender ID is a subset of things like SPF, and may be viewed as Microsoft's extensions of ideas of SPF). To make this quite clear, openspf.org's page on SPF and Sender ID states “SPF and Sender ID are not the same.” It goes on to say that the reference of the “spf2.0/” text in Sender ID “tag name is a historical accident and was assigned by the failed MARID IETF working group. The page goes on to say “Microsoft is aware of the problem and representatives of theirs have stated that they have no plans to fix it.”
Wikipedia's page on Sender ID says “Sender ID is defined primarily in Experimental RFC 4406, but additional parts in RFC 4405, RFC 4407 and RFC 4408.” (Hyperlinks have been added to the quoted original text.) The openspf.org site's page comparing SPF and Sender ID recommends ignoring a recommendation from the RFC 4406 section 3.4 and provides reasons. For more details comparing the two standards, see that Wikipedia's page on Sender ID has some information comparing the Sender ID standard with SPF.
- Blacklists (a.k.a. “Real-time DNS-based Blackhole List” (“RBL”/“DNSBL”), a.k.a. DNS Blocklist)
organizations use a method of sharing information about IP addresses so
that information gathered from spam traps can be used by others, namely
by allowing others to determine whether an IP address is on the
blacklist or not. If an IP address is on the blacklist, the maintainers
of the blacklist may commonly share their evidence with users from that
IP address so that administrators may determine why the site became
blacklisted. For more information on removing a user from a blacklist,
see (section about that???).
- Determining if a site is on a Blacklist
- Often with DNS; there are also sources, such as websites, that may check multiple blacklists that are run by different organizations
- Using the information
- See the Mail score usage section.
- Running/Administering a private blacklist
(Doesn't necessarily need to be public... could be a private blacklist)
e.g. http://spamikaze.org/ used by the “Passive Spam Block List” (“PSBL”) FAQ says “a free software package specifically written to make it easy for mail server admins to run their own DNSBL.” “Passive Spam Block List” (“PSBL”) FAQ
“Passive Spam Block List” (“PSBL”)
“Spam and Open Relay Blocking System” (“SORBS”) (utility project page?) refers to the web page at: Sorbs Project page at SF.net : Anti-relay program - not SORBS.net
- [#malgrlst]: E-Mail Greylisting
Greylisting involves providing, to E-Mail servers that aren't on a long-term or short term white list, an indication that the server should retry sending the E-Mail at a later time. The simple logic is that spammers may be trying to exploit temporary Internet access before they are shut down, and so they won't be able to retry, whereas most E-Mail servers that send E-Mail will regularly have Internet access so they may easily retry.
There may be some issues with Greylisting. One is that legitimate mail may be delayed. For example, if an end user visits a web site and has an initial password sent via E-Mail, the user may need to wait some time before being able to proceed. Also, there is the possibility that the sending server may not retry sending the message. This may be considered a failure of the sending server to follow RFC specifications that dictate that the sender must retry sending the message after receiving an error code such as 451, but not retrying the message is a legitimate behavior when other RFCs are followed.
- Content filtering
- URI DNSBLs
- Wikipedia DNSBL page: section on URL DNSBLs lists three major ones: SURBL (usage policy), URIBL (how to use with SpamAssassin) (about shows lists), imvURI (subscription/evaluation)
- [#scl]: Spam Confidence Level
- What an SCL is
- The level is a number representing how likely the mail seems to be “Unwanted Commercial E-Mail” (“UCE”). Exchange Anti-Spam Stamps (see section on “Spam Confidence Level Stamp) describes this tag, saying “The Content Filter agent uses Microsoft SmartScreen technology to assess the contents of a message and to assign an SCL rating to each message.” (Piratefish 3.0 Tuning (archived by the Wayback Machine @ Archive.org) : section on Spam/Exchange suggests headers are in part of the X-MS-Exchange-Organization-SCL: tag. However, the page also notes that SCL may also show in the Anti-Spam Report, another Anti-Spam Stamp described on the page.
- Viewing/setting the SCL
- Transmitting the SCL
- KB 867633: “if there is a device in between the Exchange servers that does not support EXCH50, the SCL rating will be lost.” Understanding Header Firewall (TechNet's Exchange 2007 Help) (Piratefish 3.0 Tuning (archived by the Wayback Machine @ Archive.org) : section on Spam/Exchange noted a software developer who hadn't implemented this yet (which may be an indication of its difficulty level).
- Viewing the SCL in Outlook
- Info on viewing Exchange Server 2007's Anti-Spam stamps in Outlook 2007. For Outlook 2003, it is more complicated, requiring using a modified SCL.C* file: See Exposing SCL in Outlook, Exposing SCL in OWA (note Karan's note), Outlook, OWA
- Viewing the SCL in an IMF archive
TechNet's “Customizing Exchange Server Intelligent Message Filter” says, as does Microsoft Exchange Server Intelligent Message Filter v2 Operations Guide (download page)'s IMF_SP2.doc, “By default, when Intelligent Message Filter archives a message, it does not archive the SCL rating assigned to the message.” To change this, create/modify a registry DWORD in the registry key called HKEY_LOCAL_MACHINE\Software\Microsoft\Exchange\ContentFilter and use a value name of ArchiveSCL and set it to 1.
From there, it's a matter of viewing the IMF archive.
- Altering SCL
- (Piratefish 3.0 Tuning (archived by the Wayback Machine @ Archive.org) : section on Spam/Exchange (archived by the Wayback Machine @ Archive.org) referred to Exchange 2003 SP2 IMF Keyword Manager.
http://msdn.microsoft.com/en-us/library/ms998865%28EXCHG.65%29.aspx store action threshold http://www.msexchange.org/tutorials/microsoft-exchange-intelligent-message-filter.html http://blogs.technet.com/exchange/archive/2005/12/14/416070.aspx">archived post that says updates will be available on first and third Wednesdays of every month. http://support.microsoft.com/kb/907747 old download page at http://technet.microsoft.com/en-us/exchange/bb288484.aspx has some relevant hyperlinks.
- Bogofilter: OpenBSD guide at Calomel guide for Bogofilter
- [#mssmtscr]: Microsoft SmartScreen
In a nutshell, this is the name for technology which is likely to be updated regularly (as anti-spam efforts adapt to techniques that spammers may use). Therefore, precise details of this technology may vary over time as Microsoft releases updates that get applied on Microsoft's internal servers (such as the servers providing service for HotMail) and other servers that apply Microsoft's released updates (such as the updates for IMF).
The “updated SmartScreenTM Technology” is described by KB 906671: Microsoft Exchange Server 2003 Service Pack 2 release notes's downloadable HTML file as “Microsoft research technology that is used to detect spam messages”. It is used by several Microsoft offerings related to E-Mail: Exchange IMF, Outlook 2003, Outlook 2007, Hotmail, etc. (Microsoft SmartScreen: Spam Filtering lists offerings.) See also: SmartScreen,
Microsoft's Antispam Technologies page: section on “Microsoft SmartScreen”. An April 2004 article on Windows IT Pro about SmartScreen indicates says SmartScreen may evaluate “the contents of each email message and assigns a weighted score to each message according to more than 500,000 characteristics”.
- [#malscusg]: Mail score usage: Automatic processing of E-Mails that have been rated/ranked/scored/marked by filters
Spam in this sense refers to unwanted mail, which may be in one or more of the often similar and overlapping categories of “Unsolicited Commercial E-Mail” (“UCE”), or phishing scams, or viral-infected E-Mails.
- Categorizing/sorting/delivering the E-Mails
Mail may be delivered to the end user's main mailbox, to another specified mailbox (of which a mailbox named “Junk Mail” or “Spam” may be the most commonly implemented names for other mailboxes), or erased. Erasing files is, naturally, far more likely to be irrecoverable than simply redirecting the mail to a separate folder (where it remains, possibly only for a certain number of days before it then becomes erased and not easily recoverable).
- Blocking the E-Mails (“Gateway Blocking”)
- AAAR: Advice against automated rejection
There are two common ways of handling a message: modifying (or initially assigning) a (possibly binary) score to indicate how the message should be handled, or rejecting the mail altogether. Rejecting mail may be ill-advised because, unlike forwarding the mail to a “Junk Mail” folder (where a mistake can be easily recovered by having someone search through that folder), there isn't much of any way for mistakes to be recovered. (Also, recipients and sometimes senders are routinely not informed that such an action took place, and so may be ignorant of the problem when a problem occurs.)
Web page about ISP Imposing Blacklist on E-Mail Customers shows an example of a customer who didn't appreciate the spam protection: “I am concerned now that I can no longer give anyone my” “email address” from a specific provider “with confidence that email sent by that person will actually get to me.” The provider “is censoring my mail, making arbitrary and capricious decisions about who should be able to send mail to me, its paying customer.”
Another example was when an E-Mail provider started combating spam without informing end users, ruining some data harvesting that a user was performing for research purposes.
- Advantages of not rejecting later in the process
If the mail is going to be rejected, doing so during the connection where the mail is transmitted has some advantages. First, the time before the E-Mail is fully received is the only real time that the receiving mail server has a guaranteed opportunity to send a message to the computer that is sending the E-Mail. Attempting to send a message (such as an automatically sent automated E-Mail) later may not work if the sending computer went offline and such an approach is discouraged because of the problem of being more likely to accidentally participating in an attack by sending backscatter and the problem that the sending computer may not be available again.
The message that the receiving E-Mail server communicates to the sender may be some helpful text that the sending computer may choose to relay back to the person who sent the E-Mail, which may be useful to the sender (if the sender mistyped an E-Mail address, for example). The anti-spam method of E-Mail greylisting requires such a message to be sent to the sending computer.
Secondly, blocking the message early on may reduce the resources of the receiving side's infrastructure: Rejected E-Mails won't have to go through a lot of post-receipt spam processing and E-Mails might even be able to be rejected as soon as the sender specified the E-Mail addresses but before the sender had a chance to specify the content of the message.
- Slowing down the mail communication, especially at the first sign of trouble, so that systems sending out large amounts of troublesome E-Mails have to use more resources when many mail servers are slowing communications.
- Causing trouble
An ironic thing is that vigilante justice may lead the justice-provider to become guilty of attacking an offending organization that claims to be an innocent victim. Following such techniques may cause one to be legally liable, so this is not generally recommended. http://www.spamcannibal.org/cannibal.cgi
- Bob Beck's spamd presentation describes how a mail server would stutter. It would significantly slow down the TCP connection of an incoming mail server, down to one character per second. (There's no reason I saw why that speed would be needed: the speed could vary between five characters per second or three characters every five seconds.) This slows down mail delivery until a host is whitelisted. The idea here is that after an initial delay, mail receiving is then sped up. This causes some connection slowness that will be tolerated by real mail servers which will follow standards in order to achive the mail sending they are designed to do. Spammers are likely to give up on the connection attempt, thinking they are tarpitted and moving onto other hosts.
- Reporting spam/phishing/etc.
Reporting spam to the abuse, postmaster, administrator, or root addresses of an ISP may have an effect, or may not. Keep in mind that a lot of spam comes from foreign locations which may not be under the direct jurisdiction of the nation where the recipient's E-Mail address is based out of. However, there are times that international law enforcement agencies may cooperate together and put an end to the activities of some spammers. Sometimes these sorts of actions even results in a noticable decrease of spam on a large scale, although the effect tends to be temporary as some affected spammers find alternate ways to continue their misdeeds. In all likelihood the law enforcement agencies, although they may make press headlines, don't directly contact the submitters of tips to let them know the direct results. Still, if a person is content to report something realizing that there may be no effect and even less of a chance that any effect will be reported back, submission may be an option.
Examples of actions taken against spammers: McColo.com was a web hosting provider shut down by upstream Internet providers which was shut down on November 11, 2008. This resulted in a substantial worldwide decrease in spam, with many estimates in how much spam was reduced being around two thirds, or higher, of all worldwide spam. The decrease was sustained for some time after this action, but eventually spam levels since rose again as spammers adapted. See CBL on McColo. Other examples: Top spammer arrested. Example: FTC announcement of action against spammers that peddled drugs. Example: FTC announcing government agencies working organizations against a lotto scam where the government agencies include the FBI, US Postal Inspection Service, and Canada's “Business Practices and Consumer Protection Authority of B.C.” and “Royal Canadian Mounted Police”
- Internet Crime Complaint Center (a partnership between the United States of America Federal Bureau of Investigation (FBI), National White Collar Crime Center, and the Bureau of Justice Assistance (BJA).
- FTC Spam page now directs people to Spam@uce.gov (announcement that firstname.lastname@example.org will replace email@example.com) A quick search on Google has found that firstname.lastname@example.org may bounce. There's a “Wired” article on FTC reports that describes a bit about what happened to mail at the old address. The FTC may still take complaints: There is an FTC Complaint page that now redirects to the FTC Complaint Assistant site. This site has a “File a Complaint” tab towareds the top fo the page, and also has hyperlinks to Consumer Sentinel/Military (which the Complaint Assistant site describes as being for “members of the United States Armed Forces and their families”) and econsumer.gov for complaints against entities in a nation other than the USA.
There is are a couple of offerings from Vipul Ved Prakash: One is the free “Vipul's Razor” for some platforms (using PERL/make), and the other is the Vipul's company called Cloudmark that makes releases for popular retail operating systems (Microsoft Windows and Mac OS X).
Cloudmark Desktop, which used to be called SpamNet, Cloudmark Desktop is available for several software combinations. For Mac OS X there is support for Thunderbird, for 32-bit Windows Vista there is also support for Windows Mail and Outlook, and for Win98/NT/2K/XP there is support for Outlook Express and Outlook (although NT/98 may be “legacy” (older) release). The software may be downloaded and is available for free for 15 days, after which point the Home Desktop version has an option to pay an annual fee or, for up to two computers, an adware-sponsored (“trialpay”) option. Vipul's Razor FAQ describes Cloudmark's SpamNet as an option for Windows. “SpamNet is an Outlook plugin, whose functionality is equivalent to Razor2 and talks to the same network.”
- Misc info
- Implementations: OpenBSD spamd: See http://www.linux.com/archive/feature/114261 (especially about a permissions issue: after making a file with touch, it says one may need to chown _spamd:_spamd /var/db/spamd ) Content filtering: foreign character sets: http://www.okean.com/antispam/headers.html
- GPG, PGP
- What to do if on an RBL
- Situation Overview
- Handling Real-time Blackhole Lists
To rectify a situation, the first thing is to identify the situation. Chances are that technical support was requested by somebody who was trying to send E-Mail but no longer is able to send E-Mail to one or more addresses. Technical support may notice that there is a reference to an RBL. Perhaps this is figured out by checking the results of a connection to a remote mail server, possibly by reading an E-Mail that came from the sending E-Mail server and which describes the error and includes an error message from an SMTP connection. (The error message from the SMTP connection may include some text that came from the E-Mail server that mail was attempted to be delivered to.) It is this scenario that this guide tries to rectify, focused on helping whoever is trying to send E-Mail.
The outgoing E-Mail is being prevented because the recipient's E-Mail server is rejecting the mail. However, that is probably an act which is intentionally being caused by the E-Mail server of the recipient. In the case where the E-Mail being sent is not E-Mail generally considered to be undesirable and even unethical to send, following some steps is likely to allow the E-Mail to be successfully resent in after a period of time, possibly a matter of hours. Achieving this goal, generally without needing to pay any money to anybody, is the goal of this RBL-removal guide. Benefit to following these steps are that following these steps will often result in a network which many people would find to be more respectable, it reduces the likelihood of a re-occurrance, and it tends to be successful. Other approaches, such as telling the sender to stop bothering to try to send the E-Mail, or trying to convince the remote technical support to reverse their automated actions which probably took some investment to set up in the first place, are actions that may be less likely to achieve some of those benefits, such as improving the network that is sending E-Mail.
- What not to do: higher amounts of unpleasantness
First, don't send spam. Insist that any E-Mail being sent is not spam. If an organization appears to want specific E-Mails to be sent, and those E-Mails are spam, realize that the counter-measures taken by others, including the real-time blackhole lists, may cause failure for some of the efforts to send that spam.
Second, do not be dishonest. This is an excellent tip to follow. Specifically, as it relates to the real-time blackhole lists, do not abuse the reliance of the amount of trust that anti-spam real-time blackhole lists (“RBLs”) do provide. Don't try to fix the impactful pain of not being able to send E-Mail, and then try to see if a re-occurrence can be prevented. Some real-time blackhole lists (“RBLs”) may begin to remove restrictions after an administrator performs some techniques to improve the network and then fills out a form where the administrator provides details of what was done. This may tempt some administrators to try to take a quick route, even if it is dishonest, of claiming that an issue is fixed so that restrictions may begin to be lifted as soon as possible.
Although there may be some pressure to get this resolved quickly, keep a cool head and perform the right actions, which involves identifying a real possibility of what happened and taking actions that actually lead to fixing the vulnerability, and ensuring that is done so that the removal request may be filled out with integrity. The organizations operating real-time blackhole lists (“RBLs”) may take some counter-measures to those who are not successfully remedying what the RBL administrators to consider a problem, especially when claims are being made that things have improved. The inclusion on a real-time blackhole list may be caused by a detected vulnerability (such as an open-relay configuration) or, probably more often, the detected results of a vulnerability (which is namely that the administrators of the real-time blackhole list (“RBL”) received some spam and verified that the listed IP address was used in order to deliver the spam. Spam which is sent automatically is often done at a very high rate, and a false declaration of victory may create a scenario where additional spam is detected between the time when victory was declared and the time that things were actually fixed. If that happens, a re-listing may occur since a problem was detected after a claim that things were fixed. Dealing with a re-listing may be notably worse than dealing with an initial listing.
The main reason that re-listings may be more impactful than earlier listings that cause an entire Internet site from being able to send E-Mails to many locations is that a re-listing may not have the same removal requirements as an earlier listing. For example, that period of time for removal may be a matter of hours or days since the last spam was detected. Additional action may be required, such as a financial payment to the real-time blackhole list (“RBL”). Although the management of the organization who owns the affected system may not be feeling happy about supporting the RBL who they may view as an entity which is not helping them obtain what they want. However, sometimes feelings of desparation or the feeling that success is necessary may lead a party to be willing to make payment anyway. Even still, though, some anti-spam measures like a real-time blackhole list may not be something that a monetary payment to any specific organization would help at all. Perhaps the organization's Internet address will be expected to wait for a large number of days that amount to weeks and perhaps a whole year. If management is making life unpleasant because things are not being resolved within a small number of half hour timeframes, they aren't likely to be overjoyed upon learning that the real-time blackhole lists aren't going to allow delisting to occur for a whole year. However, the administrators of a real-time blackhole lists (“RBL”) may not worry much at all about whether or not a technical support person is going to have a pleasant experience interacting with the management of an organization that is violating the anti-spam policies that the administrators of the real-time blackhole list (“RBL”) are attempting to enforce.
In the case of an offender which has sent many, many spams and/or have had a number of issues over a long time, the timespan may even be considered permanent. For example, an IP address might only be removed from a real-time blackhole list after a manual review indicates that the reverse DNS points to a domain which appears to be registered by an organization which does not seem to have any ties to an organization that was involved with the past abuse. If a company stops using an IP address, and a brand new company starts using that same IP address, and the new company appears to be at least partially owned by somebody who at least partially owned the previous company that was using those IP addreses for abuse, then the restrictions placed against those IP addresses may not be lifted.
An organization that wants to get around that issue might be able to successfully do so by using a new external IP address. However, a real-time blackhole list may respond by blocking a range of IP addresses. Some anti-spam methods may even be designed to try to block all of the IP addresses that are used by entire specific nations. If the spam-sending organization tries to switch Internet Service Providers then they may be able to work around the issue. However, that may be a considerable expense, especially if the company relies on using their current IP addresses and if changing IP addresses involves re-configuring multiple software and/or hardware configurations (such as the configuration files stored in a firewall). This may take a fair amount of effort, and that may involve a fair amount of expense for whatever company is financing such effort.
For example of how requesting a de-listing too early may cause more of a problem than what it solves, an administrator of a network may fill out an electronic form that indicates that an issue has been identified and fixed, and the site may be removed from a real-time blackhole list. E-Mail may begin to work again as the administrator investigates the problem further. If the administrator finds logged information that shows the E-Mail came from a certain workstation, and then the network administrator further finds out that the workstation has been infected with some malware that evaded the anti-malware protection and sent spam, then the administrator may have that system be unplugged from the network until the malware has been sufficiently removed. However, that workstation or a diffrent workstation on the network may have already sent out even one more piece of spam which ended up going to a system that is design to detect (possibly successful) attempts to send spam. Once this is detected, the offender may be re-listed. At least in theory, and probably in practice, it is entirely possible for that trouble-causing spam to be sent in less than five seconds.
There is no strict criteria on exactly what actions are taken based on the number of re-listings and how quickly they have re-occurred. The specifics are up to the maintainers of the real-time blackhole list.
- The non-power of insistance
The administrators of the real-time blackhole lists generally feel pretty safe. They aren't directly blocking incoming E-Mails, but they are simply providing information and it is the receiving E-Mail server that is choosing to implement the information from the real-time blackhole lists. As the administrators of the real-time blackhole lists have found that stubbornness may help to achieve their goals of reduced spam than showing mercies to those who do not abide by the rules that they make up, actions other than cooperation are not likely to yield what are probably the desirable results.
Likewise, insisting on a change by the administrators of the receiving E-Mail server may produce little positive results. With large organizations such as nationwide/continent-wide E-Mail providers, they may (possibly correctly) believe that their techniques are effective in reducing the incoming amount of spam. They may also know that incoming spam uses their electronic resources (such as hard drive space, network bandwidth, and CPU processor time) and may often increase customer dissatisfaction. Therefore, they may believe that the anti-spam measures are causing them more benefit than whatever harm may be threatened by anybody who tries to insist that the recipient changes the way that the recipient's E-Mail server interacts with a server that is on a real-time blackhole list (“RBL”).
Prior to resolution, it is easy for someone to feel like they are being attacked by a real-time blackhole list. That may even be true: The real-time blackhole list is attacking the affected organization's ability to just keep operating with the same vulnerability that led to spam being sent. However, the maintainers of the blacklist do realize that many legitimate organizations may have problems that lead to spam being successfully sent, and they would like to see those problems resolved.
The simple truth is that there is little to no reason why a real-time blackhole list wouldn't have the right to track information that is submitted to that list about what happened with a specific IP address. There is little to no reason why the blackhole list may not have the right to share that information with a party that requests the information. There is little to no reason why the real-time blackhole list would have any legal obligation to change the rating it provides about a specific IP address. And there is little to no reason why the receiving E-Mail server doesn't have the right to request that information. There is also little to no reason why the receiving E-Mail servers are required to accept and deliver E-Mail that it doesn't want to accept and deliver.
Whether or not this is viewed as pleasant by the party which wants to send E-Mail, there is little to no legal (and some would argue ethical) grounds why the real-time blackhole list, vendor of anti-spam software, other reputation provider, or receiving E-Mail server is required to accept E-Mail.
Some may make the claim that a reputation-based system may be cause for a lawsuit regarding libel, and/or that an E-Mail provider who refuses to accept E-Mail is infringing upon some rights for their paying customer to receive E-Mail. Whether or not that is true is something that this guide doesn't try to fully answer: legal professionals including a court judge may be able to offer an opinion about that. However, fixing up an E-Mail system and/or the site's electronic reputation (at least to usable levels) may, in many cases, be far easier and cheaper and quicker for a company to perform than to try to pursue justice through a law-enforcement/judgement system.
- Identifying spam
Some of this may seem obvious, but some of this may not be obvious to large numbers of people. E-Mail which can be easily categorized into some common categories that indicate such E-Mail is undesirable to receive is E-Mail that many people consider to be “spam”. Some people may think that spam specifically refers to unsolicited commercial E-Mail that is sent to somebody who is not likely to be interested in the product. Therefore, targetted E-Mails sent to people within an industry may be E-Mails that don't fit that definition of spam. Other people, however, will define spam more broadly.
Spam may refer to “unsolicited commercial E-Mail” (“UCE”), mail which is both either commercial in nature or which is sent out as part of a mailing list but which also does not have a working unsubscription method, mail that participates in dishonest activity, or mail which is erotic in nature, especially if it contains pornographic imagery in an attachment or which is unsolicited, particularly if it is received by an organization such as a corporation (even if the material was requested by a staff member of the business). An example of something that may be considered dishonest activity is if a hyperlink goes to a location (like a redirection address) but says it goes to another location (like the actual end address). E-Mail that uses most or all words in a different language than most E-Mails received by the receiving E-Mail server, or which uses a font with many characters often used by other languages, may be automatically detected as being likely to be treated as spam. Even if users end up at the same web page (and can be tracked by having them go to the former page), the mismatch in web address information may be an indicator that some automated software would flag as being more likely to be spam.
There are some other ways that software tries to detect spam. Spelling and computer-detected grammer errors are likely to increase an E-Mails likeliness to be viewed as spam, as well as E-Mail that comes from certain Internet addresses or E-Mail that identifies a sender that uses domains that typically send out spam.
Some of the above descriptions may not perfectly match certain definitions, such as legal descriptions used in laws, that are used to refer to spam. However, such material is material that many people would consider to be spam, regardless of whether or not the E-Mail fits a specific definition, whether that is a technical definition or a particular law, such as the USA's CAN-SPAM legislation which may consider some E-Mails (particularly ones that are quite fraudulent) to be illegal. Note that the Internet does extend to areas that have different legislation, such as Australia's anti-spam law which may differ from a law in some portion (including the entirety) of the United States of America, so something like USA's CAN-SPAM law is certainly not the only definition that can be used.
Rather than discuss any further whether a specific type of E-Mail is spam, a recommendation is hereby made to consider making an intelligent judgement call on whether some people are likely to consider the E-Mail to be undesirable. If so, regardless of an opinion about how right or wrong those people might be, simply realize that there may be some resistance to sending such E-Mails.
- IT staff response
- IT staff member frustration
Like many problems, dealing with the problems related to being on a real-time blackhole list (“RBL”) can be quite frustrating when things aren't happening as smoothly as desired. If one is feeling like an innocent person who is being picked on by an apparently powerful organization, remember than the message coming from the real-time blackhole list (“RBL”) is that the organization that sent the E-Mail is not innocent. If one feels like the RBL should not be able to cause such troubles by posting information, or the administrators of the other E-Mail server should not employ the use of information from an RBL, remember that these separate organizations may create choices independant of the desires of those operating the computer that is trying to send E-Mail. Reviewing the “Rights” section of the “Situation overview” may help remind tech staff what rights people actually have, and to feel less violated when rights are not actually being infringed upon. If one feels like the real-time blackhole list (“RBL”) operators seem to have too much power or that the penalties are too harsh, then realize that such feelings are quite common while one is still impacted by the problem. However, many administrators begin to feel less burdened by the whole experience when they find that the whole experience led to them improving the security of a network. They may also realize how much worse the worldwide inconveniences caused by spam would be if administrators were not compelled to make such improvements, and so the RBLs may actually be appreciated in hindsight, by some administrators. Other administrators may continue to believe that the power of the real-time blackhole lists (“RBLs”) is too substantial. However, the best time to consider what can be done about such a problem is after the site is no longer being listed by affected by the real-time blackhole list (“RBL”). Dreams of reallying a whole social movement against RBLs may seem petty when one considers how many people will want to rally behind somebody who the RBLs are claiming has been involved in recent or substantial spamming. Any arguements made against the RBLs will seem to be driven more by enragement over the current stress caused by whatever penalties or other problems are being experienced, rather than genuine long-term care about the issue after one has had a chance to objectively consider the impact on all sides.
As long as there is still a good chance that continued effort will result in further information, such as what happened and what can be done to stop, or at least solidly track, such incidents so that future occurrances can be prevented, or at least responded to rapidly with ease (such as disabling the account of a user who is not obeying policies, possibly also resulting in terminating that person's affiliation with an organization), it is most likely that the most productive approach is going to be to keep trying to gather information.
- Frustration by an organizaton's management
There are some very real reasons why management may easily view even a successful resolution as a negative result from the technical support team which is supposed to respond to, or even proactively prevent ahead of time, such incidents.
How does an IT staff member describe the problem of being on an RBL to a non-technical member of an organization's management? The manager may have a few reasons to start thinking poorly about the IT/tech/computer-support department: There is a problem. The remote end seems to agree with the RBL that the problem is with the local/sending E-Mail server. Even the local tech support is indicating that something may need to be changed on the local side. Furthermore, early estimates suggest that this may take a matter of hours for outgoing E-Mail to be working normally again. This also sounds like it may not be a simple matter, since it will require some research and concentration by the technical support staff to make changes. Clean-up and/or changes in policies may be needed. The whole situation is sounding rather dire.
It may help to have some of the facts and other viewpoints ready. Basically, be ready to acknowledge the facts that were described above, such as the fact this may take some time to resolve. Note that it was a remote end that stopped E-Mail from working, and some technical expertise may be needed to convince the remote end to allow such E-Mail. Encourage the contact that there's a huge probability of the problem being able to be traced, information gathered, and changes made to rectify the situation and to prevent a re-occurrance. These statements may help sound a bit more empowering, so that the management may be more likely to trust you to take care of this.
Here's some more details/strategies/etc. to back up some of those statements: this section needs further review
If the material is spam, educate. Insist on not participating in sending spam. If an organization insists on you helping them to send spam, realize that cooperating may be viewed by many people as being unethical, even if the only other option is to stop helping that organization. Determine whether you agree with that. As a little hint of life-advice: Do not do unethical things.
Reputation (may impact future repuation technologies?)
Explain that RBL lists may change their techniques for detecting spam, and it appears that a review of some security settings may be needed. This isn't necessarily a fault of the IT organization who set things up, but may be a reflection of the changing landscape of how the Internet is set up. A change in policies, such as whether workstations are allowed to communicate directly to remote mail servers, may be required. This sort of change in policies may break some things, such as the E-Mail configurations that some users may be using. If so, technical support will simply have to assist with altering those E-Mail methods so that such E-Mail is relayed through the local E-Mail server so that tracking, logging, and other anti-spam measures may control this. Leaving the situation uncontrolled may cause the organization to be more prone to being re-listed. This may require some time and labor to resolve, including re-configuring how mail flows. This may involve performing some clean-up of an infected system, since malware often sends out spam. Once the cause has been identified, whether that is from E-Mail sent by malware or UCE sent out by a person or even a manager, the actions must discontinue so that problems from re-listings do not re-occur. Because this is enforced by remote systems, this isn't really very negotiable. It is not within the hands of the technical support department that supports the E-Mail server to determine whether remote systems are enforcing rules that they have decided upon. Things tend to work smoothly after the senders cooperate, including fixing problems that may be found.
- Handling the frustrating situation
(This section has been identified as being a bit long, and fairly likely able to be trimmed down a bit.)
Keep things in perspective. Help share the facts and perspective with any management who wants things to be handled. When answers don't come as quickly as desired, remain focused on what information can be determined and then go find out that information. Keep in mind that most incidents can be resolved, although they may take some amount of time like twenty minutes or a couple of hours before things seem resolved. (Then it may take more time, such as minutes or a day or longer for the effects of the blacklisting to stop being so affecting.) There is a possible positive end result (even if not all of the results are positive, like results of delayed E-Mail and unhappy management and a feeling at the end of the day that the day was unpleasantly stressful), and the likely positive end result that will occur is that learning and fixing occurs and the network is improved and a re-occurrance is less likely. Successful handling of a situation that is difficult according to the description of a seemingly knowledgable technical expert is something that may help to increase value of the successful effort, so achieving that success may be worthwhile for the tech staff to pursue. Remember these positive results and let them inspire the tech support staff to keep trying and trying and trying. In theory, there may be an end result that prevents the desired resolution, such as if a real-time blacklist seems like it refuses to delist. However, that isn't the likely scenario in most cases, and in such a case there should be a point of time when it seems that the relevant facts have been determined. Until that time arrives, resist discouragement and keep performing efforts to obtain all of the relevant facts, which is typically a portion of this problem that technical support staff may often find most frustrating. That frustration, however, is likely to pass once a result is obtained, and that result which is usually achievable is to keep trying to get information.
- Identify what blacklists are listing the IP address that the outgoing mailserver connects with
- The reason: Chances are that at least one blacklist may be known about. However, some blacklists will add a site for no real reason other than being on another blacklist. These subsequent lists may also be more likely to remove such affected IP addresses when they notice that the address has been removed from the blacklist that caused the IP address to be added. Therefore, making a record of what lists the site is on first may be good to do before the list grows. This way the older record will identify what sites to list first. Find out the IP address of the outgoing mail server. Run some checks with many of the major lists; the easy way to do this is going to be to use a web site that checks several of the major lists. MAILsweeper tools (mswtools.org) has a list of these in the section with the “Sender IP multi RBL checks” heading. Examples include Anti-Abuse.org and MXToolbox.com. http://spamlinks.net/filter-dnsbl-lookup.htm also lists some sites to use to check for being on a blacklist.
- Start checking if the sites provide information on why the E-Mail server is listed
Trying to check firewall rules to ensure that port 25 is blocked won't be nearly as likely to help resolve the problem if the impactful blacklist is The Spamhaus Project's Policy Black List identifying the sender as a dynamic IP range or if an SPF rule isn't enforced properly. Start by seeing if there is a message from the receiving mail server which may provide a reference to the blacklist that it checked (or might even point out that there's an enirely unrelated reason, such as a remote user's E-Mail box being full).
This is also worth recording while the information is available. The blacklist may remove the information when the IP is off of the blacklist, or perhaps older information becomes unavaialble if and when it is replaced by new information that shows an additional reason why the blacklist found cause to keep the IP address listed.
- Finding the cause
- Check outgoing mail queues
- If spam is being sent, chances are that some messages are being rejected. Some of them may be something that a mail server will attempt to retry. Consider whether to freeze outgoing E-Mail that fit a certian criteria (such as all outgoing E-Mails, if the organization is small and this likely won't be a problem), and make sure that any outgoing spam E-Mails appear to be frozen. (Since the desired end result isn't likely to just leave them frozen forever, and since they are likely dishonest scams that should not be delivered, eventually they should probably be deleted. However, saving some may be helpful for analysis, at least until the cause is understood.)
- Checking server logs
- Try to find out where the E-Mails came from. It may be helpful to know what one of the actually sent E-Mails looked like so that it can be searched for. The content of outgoing E-Mails may be noticed from the outgoing server's retry queue or from some evidence that the blacklist maintainers may have shared. Another option may be to look for errors, such as remote mail servers rejecting mail due to a user not being found.
- Watch traffic
- If the mail server is being asked to deliver a lot of mail from one address, or the firewall notices a lot of attempts to deliver to remote ports 25, the culprit is likely found.
- Perform malware scans on potential machines
- In an environment where rather full access to end user machines is available, perform malware scans.
- Fixing the problem
Naturally, this will depend on how the problem is fixed.
If a machine has malware, ensure that it is cleaned. However, don't stop there. A lot of problems of actual spam being delivered can be prevented by ensuring that systems cannot connect to remote mail ports. Use traffic routing to ensure that only mail servers are the machines that can create (or, possibly, even use) connections to remote systems with TCP port 25. While at it, consider having attempts by other systems to communicate to port 25 have those connections logged and alerted on so that IT staff may realize when this happens: It may be a user needing some assistance but also, perhaps more likely, could mean that a machine is infected with malware.
It also may be worthwhile to implement some other features, such as using SPF DNS records and detecting outgoing spam (by noticing a large amount of error messages from remote servers).
- Consider if ready for de-listing
- Are there more checks to perform? There's a couple of compelling reasons to not de-list early. One is that de-listing before the problem is resolved may result in another delisting occurring shortly later. Another key reason is that some lists may make delistings more difficult. CBL FAQ says “Don't repeatedly ask us to remove an IP without doing anything to fix the problem that caused the listing. We notice people doing this and will refuse to delist the IP if it continues.” SORBS - Listing and Delisting Overview says “If a particular host is relisted more than four times, the listing will be set for a period of one year minimum.” Realizing that repeated de-listing may make it harder to be de-listed in the future. (See section/comments about RBLs not having any requirement to delist... where on this site is that ???)
- Determine de-listing requirements
- may need to be from certain IP??? May need to pay? (If so, the list may not be one that many other organizations rely on, and so it may not be worth the funds. SORBS may be an exception with their “SORBS 'fine'” that requires payment to a charity, although the page has announced they may replace this SORBS 'fine'. Others, however, may not even view paying a charity from a restricted list to be an exception.) Recommended is to first get off the free lists and see if noticable problems are still occuring. Also, before paying, make sure that you aren't getting scammed (such as what is mentioned in the the CBL FAQ that says they “encounter claims that” CBL may “charge a fee for delisting” but such claims are false and typically fraudulent).
- Submit a de-listing request
- Requirement may be made by some to only delist from the offending IP address: This prevents an outsider from delisting and then leading to the affected party from then being penalized for too many delistings without interaction.
- Follow up: submitting more delisting requests and re-running checks to see if any new common public RBLs are listing the IP address
- Done? Maybe not
- There's always the possibility that some private blacklist may still be listing. For example, if CBL noticed something and so did Spamhaus (XBL maybe???) and then AT&T, removing from CBL may help but AT&T may not remove the private blacklist entry.
Have the right perspective. Realize that the administrators of the list are not required to remove a site from the list. Often people affected by the RBLs may feel wronged, but the simple truth (as unpleasant as it may at times seem to be) is that if an RBL wants to require an IT administrator to go through certain steps, the maintainer of the list has a right to determine whatever delisting procedure is desired. They may also take however long they want to remove an address. Keep in mind, too, that the pain probably isn't coming directly from an RBL. The remote mail system may use information from an RBL to determine whether incoming mail is likely to be spam. It could even do so and still allow spam to enter: The problem is not that the RBL provided information, but rather the pain is generally when a remote mail system that uses information from an RBL is deciding to cut off mail delivery (instead of performing some other action, such as assigning the mail a spam score and perhaps allowing it to be delivered to a user's spam box so that the E-Mail can be seen). Regardless, though, if an IP address is on an RBL then that often indicates a problem, and it is probably going to be worthwhile to take care of that problem (and to be removed from the RBL) regardless of what the remote mail system does.
A system may use an RBL to gather information so that the system can use that information to determine determine whether incoming mail is spam, and that isn't necessarily a bad thing. The pai Really, the ones who are inconveniencing the sending IP address are often the ones who implement checking of the RBL and then decide to use that information to refuse mail delivery.
The main blacklists are often reputable because they are often right. If the sending of spam isn't desired, then an undesirable thing probably happened which led to the Blacklist. Finding out what happened will probably be needed so that re-listing doesn't occur. Although there may be some pressure to get delisted as soon as possible, an IT staff may retain credibility of acting responsibly by researching the action that led the RBL adminsitrators to list the site, and to take corrective action to prevent that action from being able to re-occur. Otherwise, delisting may become increasingly difficult (to the point of impossibility).
Before delisting, try to obtain any sort of evidence that the blacklist administrators may provide. This may or may not be available. If it is available, that information may be useful for preventing re-occurrences. If it is available before delisting occurs, the evidence may be less likely to be made available after the delisting occurs.
Once the problem has been remedied, submit a request. It may take some time, from many seconds to days, to be removed. Once remoted, check with other lists to make sure they haven't added the site.
I beleive (???) CBL will say what recent delistings there were.
Cost (financial): Before spending any money, see what the impactful problem is. For example, a site may delist after days but offer to delist sooner with payment. However, if that site isn't being used by other mail servers, the unimpacting blacklist entry may have a low chance of mattering at any time in the near future, and may be even less likely to matter after the blacklisting has been removed.
- Specific Anti-Spam sites
- Composite Blocking List (“CBL”)
If listed on the Spamhaus XBL and the CBL, one may benefit from being removed from the CBL first. As noted in the Spamhaus PBL section, the CBL “is the largest part of” “the Spamhaus XBL subzone.”
- Not Just Another Bogus List (NJABL)
- ... “the SORBS 'fine'”, as mentinoed in http://www.au.sorbs.net/faq/spamdb.shtml#1 and discussed (where??? elsewhere? here?), involves a $50 payment to an approved charitable cause.
See RFC-Ignorant.org and look in the left frame for the “Listing Policy” pages. Review those and ensure full compliance.
- Some others
- The CBL FAQ has a list of some other lists and their status, namely whether the lists are (still) considered to be a good resource.
- (The UCE stands for “Unsolicited Commercial E-Mail”.) This site also references Whitelisted.org which has prominent references to UCEPROTECT on its main page, including being copyrighting by UCEPROTECT-Network. Note removal policy: Waiting up to seven days may be required if payment isn't made. However, the presense on this list may not matter much if remote mail servers aren't blocking based on a presense on UCEProtect, so waiting seven days may not be impactful. UCEProtect (main English page) says “The truth is: Every IP listed will expire 7 days after the LAST abuse is detected, and FREE of charge.” “There is a payment option available for any listee that does not want to wait 7 days but needs to be de-listed immediately.” UCEProtect FAQ Q19 discusses why they charge. Backscatterer.org has stated “powered by UCEPROTECT” on its main page.
- “Passive Spam Block List” (“PSBL”)
- This is documented as an example of an easy removal. However, be sure to check evidence first.
- http://www.comcastblacklist.com/?p=1 indicates that Comcast uses this. AT&T Worldnet refers people to this.
- AT&T Worldnet
- RBL inquiry, Web page about tools and info related to mail policies including AT&T Error Messages. A server response starting with 521 may need to follow the tips at AT&T Worldnet 521 page (or the alternate URL for an AT&T Worldnet 521 page) which basically says to check Spamhaus and Symantec IP Reputation Investigation: IP Removal/Lookup, and then to visit one of the URLs for AT&T Worldnet request for removal blocking (or an alternate URL forAT&T Worldnet request for removal blocking). For further details, consider E-Mailing the E-Mail address email@example.com.
- Microsoft SmartScreen may be similar to a Blacklist. The “Forefront DNSBL”, related to named like “Forefront Online Security for Exchange” (“FOSE”) and “Forefront Server Protection”, is described on an archived post on Forefront DNSBL. (Removal instructions are not yet here ???)
- APEWS / SPEWS
- Anonymous Postmaster Early Warning System (“APEWS”) is meant to be a successor of SPEWS. “first and only public” forum post from APEWS, Wikipedia's article on the now defunct “Spam Prevention Early Warning System” (“SPEWS”). A method used is to look at where IP addresses are assigned to, and comparing that to assignees of other IP addresses related to sites that send spam or web domains of sites that are often promoted by spam. Therefore, spammers may be proactively listed, based on poor reputation, even before any spam is detected from their IP addresses. They do not have a method to send a request for removal from the list (as noted by APEWS FAQ Q41: How does one contact APEWS?).
- No More Funn
- May block a netblock; A request may be made to unblock a single IP address so that it is whitelisted (and removed from the blacklist even if other addresses in the netblock remain blacklisted).
- Another example of a list provider with multiple lists, like a Dynamic User List (DUL). http://www.mail-abuse.com/removereq.html https://securecloud.com/help/en/request_list_removal.html http://www.sdsc.edu/~jeff/spam/Blacklists_Compared.html (weekly comparison, according to APEWS FAQ Q45.) The following are some from the source at http://unixwiz.net/tools/arblcheck.html (there are more/others described above) "relays.ordb.org", "list.dsbl.org", "multihop.dsbl.org", "unconfirmed.dsbl.org", "bl.spamcop.net", "ipwhois.rfc-ignorant.org",
- Some descriptions (in laws) of often unwanted mail
- CAN-SPAM (FTC's “Facts for Business”: page on “The CAN-SPAM ACT: A Compliance Guide for Business”) Australian Spam Act (2003): mentioned on http://www.au.sorbs.net/faq/spamdb.shtml ComLaw Management's offerings of the law's text, Australian Communications and Media Authority (“ACMA”)'s “Spam & e-Security” pages, Google Docs page on Australian Communications and Media Authority (“ACMA”)'s (PDF) document: “Spam Act 2003: A practical guide for business” AT&T Worldnet guidelines on successfully delivering mail refers people to an article whose contents are also visible with Google's (HTML) Cache of TRUSTe's “How Not to Look Like a Phish” (PDF) document
- OLD/UNORGANIZED NOTES
Filtering mailSpamAssassin? Maia mailguard uses SpamAssassin? Greylisting through error messages (of being temporarily down) by the local burdens remote mail servers more, and could conceivably cause legitimate mail to not be sent if the remote system loses Internet access before the mail is sent. Offer a web form that will allow E-Mail input. Allow a web form that will allow an E-Mail address to be white-listed on a temporary, or permanent, basis.
- Other Electronic Mail Formats
- Bulliten Board System E-Mails: QWK files
A small amount of details about
*.qwk files are at TOOGAM's Software Archive: section about E-Mail programs. For some time, the most popular program was called “Silly Little Mail Reader”, which was abbreviated “SLMR”, an abbreviation that was commonly pronounced as “Slimer”.