17.2.1. Accept that One Programmer Is Finite

I wrote sendmail as a part-time, unpaid project. It was intended to be a quick way of making Arpanet mail more accessible to people at U.C. Berkeley. The key was to forward mail between existing networks, all of which were implemented as standalone programs that were unaware that more than one network even existed. Modifying more than a tiny amount of the existing software was infeasible with only one part-time programmer. The design had to minimize the amount of existing code that needed to be modified as well as the amount of new code that needed to be written. This constraint drove most of the rest of the design principles. As it turned out, in most cases they would have been the right thing to do even if there had been a larger team available.

17.2.2. Don’t Redesign User Agents

A Mail User Agent (MUA) is what most end users think of as the “mail system”—it’s the program that they use to read, write, and answer mail. It is quite distinct from the Mail Transfer Agent (MTA), which routes email from the sender to the receiver. At the time sendmail was written, many implementations at least partly combined these two functions, so they were often developed in tandem. Trying to work on both at the same time would have been too much, so Sendmail completely punted on the user interface problem: the only changes to MUAs were to have them invoke sendmail instead of doing their own routing. In particular, there were already several user agents, and people were often quite emotional about how they interacted with mail. Trying to work on both at the same time would have been too much. This separation of the MUA from the MTA is accepted wisdom now, but was far from standard practice at the time.

17.2.3. Don’t Redesign the Local Mail Store

The local mail store (where messages would be saved until the recipient came along to read them) was not formally standardized. Some sites liked to store them in a centralized place, such as /usr/mail, /var/mail, or /var/spool/mail. Other sites liked to store them in the recipient’s home directory (e.g., as a file called .mail). Most sites started each message with a line beginning “From” followed by a space character (an extraordinarily bad decision, but that was the convention at the time), but sites that were Arpanet-focused usually stored messages separated by a line containing four control-A characters. Some sites attempted to lock the mailbox to prevent collisions, but they used different locking conventions (file locking primitives were not yet available). In short, the only reasonable thing to do was treat local mail storage as a black box.

On nearly all sites, the actual mechanism for doing local mailbox storage was embodied in the /bin/mail program. This had a (quite primitive) user interface, routing, and storage built into one program. To incorporate sendmail, the routing portion was pulled out and replaced with a call to sendmail. A -d flag was added to force final delivery, i.e., it prevented /bin/mail from calling sendmail to do the routing. In later years the code used to deliver a message to a physical mailbox was extracted into another program called mail.local. The /bin/mail program exists today only to include a lowest common denominator for scripts to send mail.

17.2.4. Make Sendmail Adapt to the World, Not the Other Way Around

Protocols such as UUCP and BerkNET were already implemented as separate programs that had their own, sometimes quirky, command line structure. In some cases they were being actively developed at the same time as sendmail. It was clear that reimplementing them (for example, to convert them to standard calling conventions) was going to be painful. This led directly to the principle that sendmail should adapt to the rest of the world rather than trying to make the rest of the world adapt to sendmail.

17.2.5. Change as Little as Possible

To the fullest extent possible, during the development of sendmail I didn’t touch anything I didn’t absolutely have to touch. Besides just not having enough time to do it, there was a culture at Berkeley at the time that eschewed most formal code ownership in favor of a policy of “the last person who touched the code is the go-to person for that program” (or more simply, “you touch it, you own it”). Although that sounds chaotic by most modern-day standards, it worked quite well in a world where no one at Berkeley was assigned full time to work on Unix; individuals worked on parts of the system that they were interested in and committed to and didn’t touch the rest of the code base except in dire circumstances.

17.2.6. Think About Reliability Early

The mail system prior to sendmail (including most of the transport systems) wasn’t terribly concerned about reliability. For example, versions of Unix prior to 4.2BSD did not have native file locking, although it could be simulated by creating a temporary file and then linking it to a lock file (if the lock file already existed the link call would fail). However, sometimes different programs writing the same data file wouldn’t agree on how the locking should be done (for example, they might use a different lock file name or even make no attempt to do locking at all), and so it wasn’t that uncommon to lose mail. Sendmail took the approach that losing mail wasn’t an option (possibly a result of my background as a database guy, where losing data is a mortal sin).

17.2.7. What Was Left Out

There were many things that were not done in the early versions. I did not try to re-architect the mail system or build a completely general solution: functionality could be added as the need arose. Very early versions were not even intended to be completely configurable without access to the source code and a compiler (although this changed fairly early on). In general, the modus operandi for sendmail was to get something working quickly and then enhance working code as needed and as the problem was better understood.

17.3.1. Wave 1: delivermail

The first instantiation of sendmail was known as delivermail. It was extremely simple, if not simplistic. Its sole job was to forward mail from one program to another; in particular, it had no SMTP support, and so never made any direct network connections. No queuing was necessary because each network already had its own queue, so the program was really just a crossbar switch. Since delivermail had no direct network protocol support, there was no reason for it to run as a daemon—it would be invoked to route each message as it was submitted, pass it to the appropriate program that would implement the next hop, and terminate. Also, there was no attempt to rewrite headers to match the network to which a message was being delivered. This commonly resulted in messages being forwarded that could not be replied to. The situation was so bad that an entire book was written about addressing mail (called, fittingly, !%@:: A Directory of Electronic Mail Addressing & Networks[AF94]).

All configuration in delivermail was compiled in and was based only on special characters in each address. The characters had precedence. For example, a host configuration might search for an “@” sign and, if one was found, send the entire address to a designated Arpanet relay host. Otherwise, it might search for a colon, and send the message to BerkNET with the designated host and user if it found one, then could check for an exclamation point (“!”) signalling that the message should be forwarded to a designated UUCP relay. Otherwise it would attempt local delivery. This configuration might result in the following:

Note that address delimiters differed in their associativity, resulting in ambiguities that could only be resolved using heuristics. For example, the last example might reasonably be parsed as {Uucp, foo, bar@baz} at another site.

The configuration was compiled in for several reasons: first, with a 16 bit address space and limited memory, parsing a runtime configuration was too expensive. Second, the systems of the time had been so highly customized that recompiling was a good idea, just to make sure you had the local versions of the libraries (shared libraries did not exist with Unix 6th Edition).

Delivermail was distributed with 4.0 and 4.1 BSD and was more successful than expected; Berkeley was far from the only site with hybrid network architectures. It became clear that more work was required.

17.3.2. Wave 2: sendmail 3, 4, and 5

Versions 1 and 2 were distributed under the delivermail name. In March 1981 work began on version 3, which would be distributed under the sendmail name. At this point the 16-bit PDP-11 was still in common use but the 32-bit VAX-11 was becoming popular, so many of the original constraints associated with small address spaces were starting to be relaxed.

The initial goals of sendmail were to convert to runtime configuration, allow message modification to provide compatibility across networks for forwarded mail, and have a richer language on which to make routing decisions. The technique used was essentially textual rewriting of addresses (based on tokens rather than character strings), a mechanism used in some expert systems at the time. There was ad hoc code to extract and save any comment strings (in parentheses) as well as to re-insert them after the programmatic rewriting completed. It was also important to be able to add or augment header fields (e.g., adding a Date header field or including the full name of the sender in the From header if it was known).

SMTP development started in November 1981. The Computer Science Research Group (CSRG) at U.C. Berkeley had gotten the DARPA contract to produce a Unix-based platform to support DARPA funded research, with the intent of making sharing between projects easier. The initial work on the TCP/IP stack was done by that time, although the details of the socket interface were still changing. Basic application protocols such as Telnet and FTP were done, but SMTP had yet to be implemented. In fact, the SMTP protocol wasn’t even finalized at that point; there had been a huge debate about how mail should be sent using a protocol to be creatively named Mail Transfer Protocol (MTP). As the debate raged, MTP got more and more complex until in frustration SMTP (Simple Mail Transfer Protocol) was drafted more-or-less by fiat (but not officially published until August 1982). Officially, I was working on the INGRES Relational Database Management System, but since I knew more about the mail system than anyone else around Berkeley at the time, I got talked into implementing SMTP.

My initial thought was to create a separate SMTP mailer that would have its own queueing and daemon; that subsystem would attach to sendmail to do the routing. However, several features of SMTP made this problematic. For example, the EXPN and VRFY commands required access to the parsing, aliasing, and local address verification modules. Also, at the time I thought it was important that the RCPT command return immediately if the address was unknown, rather than accepting the message and then having to send a delivery failure message later. This turns out to have been a prescient decision. Ironically, later MTAs often got this wrong, exacerbating the spam backscatter problem. These issues drove the decision to include SMTP as part of sendmail itself.

Sendmail 3 was distributed with 4.1a and 4.1c BSD (beta versions), sendmail 4 was distributed with 4.2 BSD, and sendmail 5 was distributed with 4.3 BSD.

17.3.3. Wave 3: The Chaos Years

After I left Berkeley and went to a startup company, my time available to work on sendmail rapidly decreased. But the Internet was starting to seriously explode and sendmail was being used in a variety of new (and larger) environments. Most of the Unix system vendors (Sun, DEC, and IBM in particular) created their own versions of sendmail, all of which were mutually incompatible. There were also attempts to build open source versions, notably IDA sendmail and KJS.

IDA sendmail came from Linköping University. IDA included extensions to make it easier to install and manage in larger environments and a completely new configuration system. One of the major new features was the inclusion of dbm(3) database maps to support highly dynamic sites. These were available using a new syntax in the configuration file and were used for many functions including mapping of addresses to and from external syntax (for example, sending out mail as john_doe@example.com instead of johnd@example.com) and routing.

King James Sendmail (KJS, produced by Paul Vixie) was an attempt to unify all the various versions of sendmail that had sprung up. Unfortunately, it never really got enough traction to have the desired effect. This era was also driven by a plethora of new technologies that were reflected in the mail system. For example, Sun’s creation of diskless clusters added the YP (later NIS) directory services and NFS, the Network File System. In particular, YP had to be visible to sendmail, since aliases were stored in YP rather than in local files.

17.3.4. Wave 4: sendmail 8

After several years, I returned to Berkeley as a staff member. My job was to manage a group installing and supporting shared infrastructure for research around the Computer Science department. For that to succeed, the largely ad hoc environments of individual research groups had to be unified in some rational way. Much like the early days of the Internet, different research groups were running on radically different platforms, some of which were quite old. In general, every research group ran its own systems, and although some of them were well managed, most of them suffered from “deferred maintenance.”

In most cases email was similarly fractured. Each person’s email address was “person@host.berkeley.edu“, where host was the name of the workstation in their office or the shared server they used (the campus didn’t even have internal subdomains) with the exception of a few special people who had @berkeley.edu addresses. The goal was to switch to internal subdomains (so all individual hosts would be in the cs.berkeley.edu subdomain) and have a unified mail system (so each person would have an @cs.berkeley.edu address). This goal was most easily realized by creating a new version of sendmail that could be used throughout the department.

I began by studying many of the variants of sendmail that had become popular. My intent was not to start from a different code base but rather to understand the functionality that others had found useful. Many of those ideas found their way into sendmail 8, often with modifications to merge related ideas or make them more generic. For example, several versions of sendmail had the ability to access external databases such as dbm(3) or NIS; sendmail 8 merged these into one “map” mechanism that could handle multiple types of databases (and even arbitrary non-database transformations). Similarly, the “generics” database (internal to external name mapping) from IDA sendmail was incorporated.

Sendmail 8 also included a new configuration package using the m4(1) macro processor. This was intended to be more declarative than the sendmail 5 configuration package, which had been largely procedural. That is, the sendmail 5 configuration package required the administrator to essentially lay out the entire configuration file by hand, really only using the “include” facility from m4 as shorthand. The sendmail 8 configuration file allowed the administrator to just declare what features, mailers, and so on were required, and m4 laid out the final configuration file.

Much of Section 17.7 discusses the enhancements in sendmail 8.

17.3.5. Wave 5: The Commercial Years

As the Internet grew and the number of sendmail sites expanded, support for the ever larger user base became more problematic. For a while I was able to continue support by setting up a group of volunteers (informally called the “Sendmail Consortium”, a.k.a. sendmail.org) who provided free support via e-mail and newsgroup. But by the late 1990s, the installed base had grown to such an extent that it was nearly impossible to support it on a volunteer basis. Together with a more business-savvy friend I founded Sendmail, Inc.², with the expectation of getting new resources to bear on the code.

Although the commercial product was originally based largely on configuration and management tools, many new features were added to the open-source MTA to support the needs of the commercial world. Notably, the company added support for TLS (connection encryption), SMTP Authentication, site security enhancements such as Denial of Service protection, and most importantly mail filtering plugins (the Milter interface discussed below).

At of this writing the commercial product has expanded to include a large suite of e-mail based applications, nearly all of which are constructed on the extensions added to sendmail during the first few years of the company.

17.3.6. Whatever Happened to sendmail 6 and 7?

Sendmail 6 was essentially the beta for sendmail 8. It was never officially released, but was distributed fairly widely. Sendmail 7 never existed at all; sendmail jumped directly to version 8 because all the other source files for the BSD distribution were bumped to version 8 when 4.4 BSD was released in June 1993.

17.4.1. The Syntax of the Configuration File

The syntax of the configuration file was driven by a couple of issues. First, the entire application had to fit into a 16-bit address space, so the parser had to be small. Second, early configurations were quite short (under one page), so while the syntax was obscure, the file was still comprehensible. However, as time passed, more operational decisions moved out of the C code into the configuration file, and the file started to grow. The configuration file acquired a reputation for being arcane. One particular frustration for many people was the choice of the tab character as an active syntax item. This was a mistake that was copied from other systems of the time, notably make. That particular problem became more acute as window systems (and hence cut-and-paste, which usually did not preserve the tabs) became available.

In retrospect, as the file got larger and 32-bit machines took over, it would have made sense to reconsider the syntax. There was a time when I thought about doing this but decided against it because I didn’t want to break the “large” installed base (which at that point was probably a few hundred machines). In retrospect this was a mistake; I had simply not appreciated how large the install base would grow and how many hours it would save me had I changed the syntax early. Also, when the standards stabilized a fair amount of the generality could have been pushed back into the C code base, thus simplifying the configurations.

Of particular interest was how more functionality got moved into the configuration file. I was developing sendmail at the same time as the SMTP standard was evolving. By moving operational decisions into the configuration file I was able to respond rapidly to design changes—usually in under 24 hours. I believe that this improved the SMTP standard, since it was possible to get operational experience with a proposed design change quite quickly, but only at the cost of making the configuration file difficult to understand.

17.4.2. Rewriting Rules

One of the difficult decisions when writing sendmail was how to do the necessary rewriting to allow forwarding between networks without violating the standards of the receiving network. The transformations required changing metacharacters (for example, BerkNET used colon as a separator, which was not legal in SMTP addresses), rearranging address components, adding or deleting components, etc. For example, the following rewrites would be needed under certain circumstances:

Regular expressions were not a good choice because they didn’t have good support for word boundaries, quoting, etc. It quickly became obvious that it would be nearly impossible to write regular expressions that were accurate, much less intelligible. In particular, regular expressions reserve a number of metacharacters, including “.”, “*”, “+”, “{[}”, and “{]}”, all of which can appear in e-mail addresses. These could have been escaped in configuration files, but I deemed that to be complicated, confusing, and a bit ugly. (This was tried by UPAS from Bell Laboratories, the mailer for Unix Eighth Edition, but it never caught on³.) Instead, a scanning phase was necessary to produce tokens that could then be manipulated much like characters in regular expressions. A single parameter describing “operator characters”, which were themselves both tokens and token separators, was sufficient. Blank spaces separated tokens but were not tokens themselves. The rewriting rules were just pattern match/replace pairs organized into what were essentially subroutines.

Instead of a large number of metacharacters that had to be escaped to lose their “magic” properties (as used in regular expressions), I used a single “escape” character that combined with ordinary characters to represent wildcard patterns (to match an arbitrary word, for example). The traditional Unix approach would be to use backslash, but backslash was already used as a quote character in some address syntaxes. As it turned out, “$” was one of the few characters that had not already been used as a punctuation character in some email syntax.

One of the original bad decisions was, ironically, just a matter of how white space was used. A space character was a separator, just as in most scanned input, and so could have been used freely between tokens in patterns. However, the original configuration files distributed did not include spaces, resulting in patterns that were far harder to understand than necessary. Consider the difference between the following two (semantically identical) patterns:

$+ + $* @ $+ . $={mydomain}
$++$*@$+.$={mydomain}

17.4.3. Using Rewriting for Parsing

Some have suggested that sendmail should have used conventional grammar-based parsing techniques to parse addresses rather than rewriting rules and leave the rewriting rules for address modification. On the surface this would seem to make sense, given that the standards define addresses using a grammar. The main reason for reusing rewriting rules is that in some cases it was necessary to parse header field addresses (e.g., in order to extract the sender envelope from a header when receiving mail from a network that didn’t have a formal envelope). Such addresses aren’t easy to parse using (say) an LALR(1) parser such as YACC and a traditional scanner because of the amount of lookahead required. For example, parsing the address: allman@foo.bar.baz.com <eric@example.com> requires lookahead by either the scanner or the parser; you can’t know that the initial “allman@…” is not an address until you see the “<“. Since LALR(1) parsers only have one token of lookahead this would have had to be done in the scanner, which would have complicated it substantially. Since the rewriting rules already had arbitrary backtracking (i.e., they could look ahead arbitrarily far), they were sufficient.

A secondary reason was that it was relatively easy to make the patterns recognize and fix broken input. Finally, rewriting was more than powerful enough to do the job, and reusing any code was wise.

One unusual point about the rewriting rules: when doing the pattern matching, it is useful for both the input and the pattern to be tokenized. Hence, the same scanner is used for both the input addresses and the patterns themselves. This requires that the scanner be called with different character type tables for differing input.

17.4.4. Embedding SMTP and Queueing in sendmail

An “obvious” way to implement outgoing (client) SMTP would have been to build it as an external mailer, similarly to UUCP, But this would raise a number of other questions. For example, would queueing be done in sendmail or in the SMTP client module? If it was done in sendmail then either separate copies of messages would have to be sent to each recipient (i.e., no “piggybacking”, wherein a single connection can be opened and then multiple RCPT commands can be sent) or a much richer communication back-path would be necessary to convey the necessary per-recipient status than was possible using simple Unix exit codes. If queueing was done in the client module then there was a potential for large amounts of replication; in particular, at the time other networks such as XNS were still possible contenders. Additionally, including the queue into sendmail itself provided a more elegant way of dealing with certain kinds of failures, notably transient problems such as resource exhaustion.

Incoming (server) SMTP involved a different set of decisions. At the time, I felt it was important to implement the VRFY andEXPN SMTP commands faithfully, which required access to the alias mechanism. This would once again require a much richer protocol exchange between the server SMTP module and sendmail than was possible using command lines and exit codes—in fact, a protocol akin to SMTP itself.

I would be much more inclined today to leave queueing in the core sendmail but move both sides of the SMTP implementation into other processes. One reason is to gain security: once the server side has an open instance of port 25 it no longer needs access to root permissions. Modern extensions such as TLS and DKIM signing complicate the client side (since the private keys should not be accessible to unprivileged users), but strictly speaking root access is still not necessary. Although the security issue is still an issue here, if the client SMTP is running as a non-root user who can read the private keys, that user by definition has special privileges, and hence should not be communicating directly with other sites. All of these issues can be finessed with a bit of work.

17.4.5. The Implementation of the Queue

Sendmail followed the conventions of the time for storing queue files. In fact, the format used is extremely similar to the lpr subsystem of the time. Each job had two files, one with the control information and one with the data. The control file was a flat text file with the first character of each line representing the meaning of that line.

When sendmail wanted to process the queue it had to read all of the control files, storing the relevant information in memory, and then sort the list. That worked fine with a relatively small number of messages in the queue, but started to break down at around 10,000 queued messages. Specifically, when the directory got large enough to require indirect blocks in the filesystem, there was a serious performance knee that could reduce performance by as much as an order of magnitude. It was possible to ameliorate this problem by having sendmail understand multiple queue directories, but that was at best a hack.

An alternative implementation might be to store all the control files in one database file. This wasn’t done because when sendmail coding began there was no generally available database package, and when dbm(3) became available it had several flaws, including the inability to reclaim space, a requirement that all keys that hashed together fit on one (512 byte) page, and a lack of locking. Robust database packages didn’t appear for many years.

Another alternative implementation would have been to have a separate daemon that would keep the state of the queue in memory, probably writing a log to allow recovery. Given the relatively low email traffic volumes of the time, the lack of memory on most machines, the relatively high cost of background processes, and the complexity of implementing such a process, this didn’t seem like a good tradeoff at the time.

Another design decision was to store the message header in the queue control file rather than the data file. The rationale was that most headers needed considerable rewriting that varied from destination to destination (and since messages could have more than one destination, they would have to be customized multiple times), and the cost of parsing the headers seemed high, so storing them in a pre-parsed format seemed like a savings. In retrospect this was not a good decision, as was storing the message body in Unix-standard format (with newline endings) rather than in the format in which it was received (which could use newlines, carriage-return/line-feed, bare carriage-return, or line-feed/carriage-return). As the e-mail world evolved and standards were adopted, the need for rewriting diminished, and even seemingly innocuous rewriting has the risk of error.

17.4.6. Accepting and Fixing Bogus Input

Since sendmail was created in a world of multiple protocols and disturbingly few written standards, I decided to clean up malformed messages wherever possible. This matches the “Robustness Principle” (a.k.a. Postel’s Law) articulated in RFC 793⁴. Some of these changes were obvious and even required: when sending a UUCP message to the Arpanet, the UUCP addresses needed to be converted to Arpanet addresses, if only to allow “reply” commands to work correctly, line terminations needed to be converted between the conventions used by various platforms, and so on. Some were less obvious: if a message was received that did not include a From: header field required in the Internet specifications, should you add a From: header field, pass the message on without the From: header field, or reject the message? At the time, my prime consideration was interoperability, so sendmail patched the message, e.g., by adding the From: header field. However, this is claimed to have allowed other broken mail systems to be perpetuated long past the time when they should have been fixed or killed off.

I believe my decision was correct for the time, but is problematic today. A high degree of interoperability was important to let mail flow unimpeded. Had I rejected malformed messages, most messages at the time would have been rejected. Had I passed them through unfixed, recipients would have received messages that they couldn’t reply to and in some cases couldn’t even determine who sent the message—that or the message would have been rejected by another mailer.

Today the standards are written, and for the most part those standards are accurate and complete. It is no longer the case that most messages would be rejected, and yet there is still mail software out there that send out mangled messages. This unnecessarily creates numerous problems for other software on the Internet.

17.4.7. Configuration and the Use of M4

For a period I was both making regular changes to the sendmail configuration files and personally supporting many machines. Since a large amount of the configuration file was the same between different machines, the use of a tool to build the configuration files was desirable. The m4 macro processor was included with Unix. It was designed as a front end for programming languages (notably ratfor). Most importantly, it had “include” capabilities, like “#include” in the C language. The original configuration files used little more than this capability and some minor macro expansions.

IDA sendmail also used m4, but in a dramatically different way. In retrospect I should have probably studied these prototypes in more detail. They contained many clever ideas, in particular the way they handled quoting.

Starting with sendmail 6, the m4 configuration files were completely rewritten to be in a more declarative style and much smaller. This used considerably more of the power of the m4 processor, which was problematic when the introduction of GNU m4 changed some of the semantics in subtle ways.

The original plan was that the m4 configurations would follow the 80/20 rule: they would be simple (hence 20% of the work), and would cover 80% of the cases. This broke down fairly quickly, for two reasons. The minor reason was that it turned out to be relatively easy to handle the vast majority of the cases, at least in the beginning. It became much harder as sendmail and the world evolved, notably with the inclusion of features such as TLS encryption and SMTP Authentication, but those didn’t come until quite a bit later.

The important reason was that it was becoming clear that the raw configuration file was just too difficult for most people to manage. In essence, the .cf (raw) format had become assembly code—editable in principle, but in reality quite opaque. The “source code” was an m4 script stored in the .mc file.

Another important distinction is that the raw format configuration file was really a programming language. It had procedural code (rulesets), subroutine calls, parameter expansion, and loops (but no gotos). The syntax was obscure, but in many ways resembled the sed and awk commands, at least conceptually. The m4 format was declarative: although it was possible to drop into the low-level raw language, in practice these details were hidden from the user.

It isn’t clear that this decision was correct or incorrect. I felt at the time (and still feel) that with complex systems it can be useful to implement what amounts to a Domain Specific Language (DSL) for building certain portions of that system. However, exposing that DSL to end users as a configuration methodology essentially converts all attempts to configure a system into a programming problem. Great power results from this, but at a non-trivial cost.

17.5.1. A Word About Optimizing Internet Scale Systems

In most network-based systems there is a tension between the client and the server. A good strategy for the client may be the wrong thing for the server and vice versa. For example, when possible the server would like to minimize its processing costs by pushing as much as possible back to the client, and of course the client feels the same way but in the opposite direction. For example, a server might want to keep a connection open while doing spam processing since that lowers the cost of rejecting a message (which these days is the common case), but the client wants to move on as quickly as possible. Looking at the entire system, that is, the Internet as a whole, the optimum solution may be to balance these two needs.

There have been cases of MTAs that have used strategies that explicitly favor either the client or the server. They can do this only because they have a relatively small installed base. When your system is used on a significant portion of the Internet you have to design it in order to balance the load between both sides in an attempt to optimize the Internet as a whole. This is complicated by the fact that there will always be MTAs completely skewed in one direction or the other—for example, mass mailing systems only care about optimizing the outgoing side.

When designing a system that incorporates both sides of the connection, it is important to avoid playing favorites. Note that this is in stark contrast to the usual asymmetry of clients and services—for example, web servers and web clients are generally not developed by the same groups.

17.5.2. Milter

One of the most important additions to sendmail was the milter (mail filter) interface. Milter allows for the use of offboard plugins (i.e., they run in a separate process) for mail processing. These were originally designed for anti-spam processing. The milter protocol runs synchronously with the server SMTP protocol. As each new SMTP command is received from the client, sendmail calls the milters with the information from that command. The milter has the opportunity to accept the command or send a rejection, which rejects the phase of the protocol appropriate for the SMTP command. Milters are modeled as callbacks, so as an SMTP command comes in, the appropriate milter subroutine is called. Milters are threaded, with a per-connection context pointer handed in to each routine to allow passing state.

In theory milters could work as loadable modules in the sendmail address space. We declined to do this for three reasons. First, the security issues were too significant: even if sendmail were running as a unique non-root user id, that user would have access to all of the state of other messages. Similarly, it was inevitable that some milter authors would try to access internal sendmail state.

Second, we wanted to create a firewall between sendmail and the milters: if a milter crashed, we wanted it to be clear who was at fault, and for mail to (potentially) continue to flow. Third, it was much easier for a milter author to debug a standalone process than sendmail as a whole.

It quickly became clear that the milter was useful for more than anti-spam processing. In fact, the milter.org⁵ web site lists milters for anti-spam, anti-virus, archiving, content monitoring, logging, traffic shaping, and many other categories, produced by commercial companies and open source projects. The postfix mailer⁶ has added support for milters using the same interface. Milters have proven to be one of sendmail’s great successes.

17.5.3. Release Schedules

There is a popular debate between “release early and often” and “release stable systems” schools of thought. Sendmail has used both of these at various times. During times of considerable change I was sometimes doing more than one release a day. My general philosophy was to make a release after each change. This is similar to providing public access to the source management system tree. I personally prefer doing releases over providing public source trees, at least in part because I use source management in what is now considered an unapproved way: for large changes, I will check in non-functioning snapshots while I am writing the code. If the tree is shared I will use branches for these snapshots, but in any case they are available for the world to see and can create considerable confusion. Also, creating a release means putting a number on it, which makes it easier to track the changes when going through a bug report. Of course, this requires that releases be easy to generate, which is not always true.

As sendmail became used in ever more critical production environments this started to become problematic. It wasn’t always easy for others to tell the difference between changes that I wanted out there for people to test versus changes that were really intended to be used in the wild. Labeling releases as “alpha” or “beta” alleviates but does not fix the problem. The result was that as sendmail matured it moved toward less frequent but larger releases. This became especially acute when sendmail got folded into a commercial company which had customers who wanted both the latest and greatest but also only stable versions, and wouldn’t accept that the two are incompatible.

This tension between open source developer needs and commercial product needs will never go away. There are many advantages to releasing early and often, notably the potentially huge audience of brave (and sometimes foolish) testers who stress the system in ways that you could almost never expect to reproduce in a standard development system. But as a project becomes successful it tends to turn into a product (even if that product is open source and free), and products have different needs than projects.

17.7.1. Configuration Became More Verbose

The original configuration of sendmail was quite terse. For example, the names of options and macros were all single characters. There were three reasons for this. First, it made parsing very simple (important in a 16-bit environment). Second, there weren’t very many options, so it wasn’t hard to come up with mnemonic names. Third, the single character convention was already established with command-line flags.

Similarly, rewriting rulesets were originally numbered instead of named. This was perhaps tolerable with a small number of rulesets, but as their number grew it became important that they have more mnemonic names.

As the environment in which sendmail operated became more complex, and as the 16-bit environment faded away, the need for a richer configuration language became evident. Fortunately, it was possible to make these changes in a backward compatible way. These changes dramatically improved the understandability of the configuration file.

17.7.2. More Connections with Other Subsystems: Greater Integration

When sendmail was written the mail system was largely isolated from the rest of the operating system. There were a few services that required integration, e.g., the /etc/passwd and /etc/hosts files. Service switches had not been invented, directory services were nonexistent, and configuration was small and hand-maintained.

That quickly changed. One of the first additions was DNS. Although the system host lookup abstraction (gethostbyname) worked for looking up IP addresses, email had to use other queries such as MX. Later, IDA sendmail included an external database lookup functionality using dbm(3) files. Sendmail 8 updated that to a general mapping service that allowed other database types, including external databases and internal transformations that could not be done using rewriting (e.g., dequoting an address).

Today, the email system relies on many external services that are, in general, not designed specifically for the exclusive use of email. This has moved sendmail toward more abstractions in the code. It has also made maintaining the mail system more difficult as more “moving parts” are added.

17.7.3. Adaptation to a Hostile World

Sendmail was developed in a world that seems completely foreign by today’s standards. The user population on the early network were mostly researchers who were relatively benign, despite the sometimes vicious academic politics. Sendmail reflected the world in which it was created, putting a lot of emphasis on getting the mail through as reliably as possible, even in the face of user errors.

Today’s world is much more hostile. The vast majority of email is malicious. The goal of an MTA has transitioned from getting the mail through to keeping the bad mail out. Filtering is probably the first priority for any MTA today. This required a number of changes in sendmail.

For example, many rulesets have been added to allow checking of parameters on incoming SMTP commands in order to catch problems as early as possible. It is much cheaper to reject a message when reading the envelope than after you have committed to reading the entire message, and even more expensive after you have accepted the message for delivery. In the early days filtering was generally done by accepting the message, passing it to a filter program, and then sending it to another instance of sendmail if the message passed (the so-called “sandwich” configuration). This is just far too expensive in today’s world.

Similarly, sendmail has gone from being a quite vanilla consumer of TCP/IP connections to being much more sophisticated, doing things like “peeking” at network input to see if the sender is transmitting commands before the previous command has been acknowledged. This breaks down some of the previous abstractions that were designed to make sendmail adaptable to multiple network types. Today, it would involve considerable work to connect sendmail to an XNS or DECnet network, for example, since the knowledge of TCP/IP has been built into so much of the code.

Many configuration features were added to address the hostile world, such as support for access tables, Realtime Blackhole Lists, address harvesting mitigation, denial-of-service protection, and spam filtering. This has dramatically complicated the task of configuring a mail system, but was absolutely necessary to adapt to today’s world.

17.7.4. Incorporation of New Technologies

Many new standards have come along over the years that required significant changes to sendmail. For example, the addition of TLS (encryption) required significant changes through much of the code. SMTP pipelining required peering into the low-level TCP/IP stream to avoid deadlocks. The addition of the submission port (587) required the ability to listen to multiple incoming ports, including having different behaviors depending on the arrival port.

Other pressures were forced by circumstances rather than standards. For example, the addition of the milter interface was a direct response to spam. Although milter was not a published standard, it was a major new technology.

In all cases, these changes enhanced the mail system in some way, be it increased security, better performance, or new functionality. However, they all came with costs, in nearly all cases complicating both the code base and the configuration file.

17.8.1. Things I Would Do Differently

Perhaps my biggest mistake with sendmail was to not recognize early enough how important it was going to be. I had several opportunities to nudge the world in the correct direction but didn’t take them; in fact, in some cases I did damage, e.g., by not making sendmail stricter about bad input when it became appropriate to do so. Similarly, I recognized that the configuration file syntax needed to be improved fairly early on, when there were perhaps a few hundred sendmail instances deployed, but decided not to change things because I didn’t want to cause the installed user base undue pain. In retrospect it would have been better to improve things early and cause temporary pain in order to produce a better long-term result.

17.8.2. Things I Would Do The Same

Of course, some things did work well…