How it compares
SMTPBench is not the right tool for every mail problem, and pretending otherwise would waste your time. Here is what it does that the usual alternatives don't, and, more usefully, when one of them is the better answer.
Accurate as of SMTPBench 1.2.1
Capabilities
Capability categories, not benchmarks. Nothing here is a speed claim in either direction. Throughput depends entirely on the host under test, and a number would be meaningless without naming the hardware on both ends.
| SMTPBench | swaks | smtp-source | JMeter | |
|---|---|---|---|---|
| MX lookup and failover | yes | possible, but you build it | no | no |
| SMTP AUTH | yes | yes | no | yes |
| Concurrency | yes | no | yes | yes |
| Whole-run rate cap | yes | no | no | yes |
| Attachment corpus sampling | yes | possible, but you build it | no | possible, but you build it |
| Per-message tracking headers | yes | possible, but you build it | no | possible, but you build it |
| Machine-readable summary with percentiles | yes | no | no | yes |
| Offline message generation | yes | no | no | no |
| Retry and deferral modeling | yes | no | no | possible, but you build it |
| Install weight | pip, 4 deps | Perl script | needs Postfix | JVM |
~ means possible, but you build it yourself rather than getting it from a flag.
When to use something else
swaks
Better than SMTPBench for a single transaction.
swaks is the right tool for probing one exchange: arbitrary headers, specific auth mechanisms, protocol-level poking at a server that is misbehaving. It is a Perl script with no dependencies worth worrying about, and it has been the standard answer to "test this SMTP server" for two decades. If your question is "what does this server do when I send it exactly this," reach for swaks. SMTPBench answers a different question: what happens across thousands of messages, and can you prove it afterwards.
Postfix smtp-source
Faster, and always will be.
It is C, it ships with Postfix, and for raw messages-per-second at a single host nothing here competes with it. The trade-offs are that it requires Postfix installed on the sending machine, it connects to a host you name rather than resolving MX, and it produces almost no output: you get throughput, not evidence. If you want to know the ceiling of one relay and nothing else, it is a fine answer.
JMeter and k6
Better when SMTP is one step in a larger scenario.
General load platforms win when the mail send is part of a longer flow: sign up over HTTP, wait for the confirmation mail, click the link. They are configurable enough to do most of what SMTPBench does, given enough scripting. The cost is setup weight and that mail-specific behaviour, MX failover in particular, is something you build rather than something you get.
Mailpit, MailHog, smtp4dev, VerifySend
Not competitors. They are the other half.
These catch mail rather than send it. Point SMTPBench at one of them and you have a complete loop: generate realistic traffic, capture it, and inspect what actually arrived. This is a recommended pairing, and a good way to try SMTPBench without sending anything to a real domain. VerifySend is open source too, and like SMTPBench it is maintained by lets.qa. verifysend.com