Mailflow & Anti-Spam Refresher Series – Part 2: Sender Authentication & Spam Filtering

In Part 1 of this blog series — Mail Flow and Anti-Spam Refresher Series – Part 1: Mail Delivery Basics — we explored the nuts and bolts of SMTP delivery. How messages traverse the internet from sender to recipient, and the DNS records that make it all work. But getting mail delivered is only half the battle. The other half is ensuring it’s trusted, secure, and doesn’t end up in the junk folder.

In this post, we’ll look at how platforms filter spam, authenticate email senders, and protect against spoofing and phishing using industry-standard technologies like SPF, DKIM, and DMARC.

These protocols form the backbone of modern email security and are essential for maintaining deliverability and protecting your domain’s reputation.

Note: that this blog doesn’t cover, or look at Anti-Virus detections in messages, this is purely about domain checking and reputation.

SPF — Sender Policy Framework

SPF lets domain owners specify which mail servers are authorized to send email on their behalf. It’s a DNS TXT record that lists permitted IPs or hosts. When a receiving server gets an email claiming to be from your domain, it checks the SPF record to see if the sending server is on the list.

• Pass: The sender is authorized.
• Fail: The sender is not authorized and may be rejected or marked as spam.
• Softfail: Suspicious but not blocked outright.

After adding all your authorized senders to the SPF record, you might want to switch to a hard fail (-all) for extra security. But if DMARC is turned on, hard fail doesn’t add much protection as DMARC already covers you. Also, using hard fail can cause problems with legitimate email forwarding or relaying, since some servers may reject these emails before DMARC or DKIM checks can happen.

Start with soft fail (~all) during deployment and monitor your mail flow via DMARC reports (we’ll discuss DMARC reporting in depth shortly). Move to hard fail (-all) only if you’re confident your SPF record covers all legitimate senders and you don’t rely on email forwarding. This prevents unnecessary disruption while maintaining strong authentication posture.

Be cautious of common misconfigurations like multiple SPF records, exceeding DNS lookup limits (SPF includes can trigger up to 10 lookups; exceeding this causes PERMERROR), or forgetting to update the record when adding new servers.

DKIM — DomainKeys Identified Mail

DKIM adds a cryptographic signature to outgoing emails. The sending server signs selected headers and body content using a private key. The recipient retrieves the public key from DNS to verify the signature.

A valid DKIM signature confirms:

• The message wasn’t altered in transit.
• It was sent by a server authorized by the domain owner.

Microsoft 365 uses strong 2048-bit DKIM keys by default and lets you rotate keys using selectors. When you rotate DKIM keys with Microsoft 365, the process takes four days to finish, and you can’t start another rotation until it’s done.

DMARC – Domain-based Message Authentication, Reporting & Conformance

DMARC ties SPF and DKIM together and adds a policy layer. It tells receivers what to do if an email fails authentication and provides reporting back to the domain owner.

• p=none: Monitor only.
• p=quarantine: Send failing mail to spam.
• p=reject: Block failing mail outright.

DMARC checks more than just valid SPF or DKIM — it looks for “alignment.” For an email to pass DMARC, at least one authentication (SPF or DKIM) must match the domain in the From address. If the domains don’t match, even valid SPF or DKIM results will fail DMARC. That’s why services often require DKIM signing for your custom domain — to ensure alignment and pass DMARC.

When configuring, start with p=none and gradually move to p=quarantine, then p=reject as you gain confidence in your alignment and coverage. DMARC reports help identify spoofing attempts and misconfigured senders.

Beyond the Basics: Modern Enhancements

After setting up SPF, DKIM, and DMARC, it’s important to consider other advanced technologies that impact secure mail delivery:

• ARC (Authenticated Received Chain)
  • Preserves email authentication results across mail forwarding.
  • Helps prevent false DMARC failures when messages are relayed.
• BIMI (Brand Indicators for Message Identification)
  • Allows organizations to display their logo next to authenticated emails.
  • Requires DMARC enforcement and a verified logo certificate.
• MTA-STS (Mail Transfer Agent Strict Transport Security)
  • Enforces encrypted SMTP connections for inbound mail.
  • Helps ensure emails are only delivered securely via TLS.
• DANE (DNS-based Authentication of Named Entities)
  • Uses DNSSEC to publish and validate TLS certificates for SMTP.
  • Protects against certificate spoofing and downgrade attacks.

Troubleshooting Tips

Even with proper authentication, emails can still land in spam due to content issues or sending behavior:

Content and reputation checks: Avoid spammy phrases, misleading headers, or excessive formatting. Ensure your domain’s reputation is clean by checking blacklists and use tools like Microsoft SNDS or Google Postmaster Tools. Use SNDS to monitor your reputation with Microsoft email users (Outlook.com, Hotmail). Google Postmaster Tools provides similar insight for Gmail. Both help you track deliverability but focus on their own networks.

Header analysis: Review message headers for clues about why mail is being junked. In Microsoft 365, the X-Forefront-Antispam-Report header is one to examin. Look for the SCL (Spam Confidence Level, range -1 to 9) and SFV (Spam Filter Verdict) fields. An SCL of 5 or higher indicates spam filtering has flagged the message; SCL -1 means it’s whitelisted. The SFV field tells you the verdict: SPM means spam filtering triggered it. The CAT fields indicate threat categories — CAT:BULK for bulk mail, CAT:PHSH for phishing, and so on. Cross-reference these findings with your DMARC reports to identify whether issues are authentication-related (alignment failures) or content-based (SCL threshold).

Message tracking and diagnostics: Use manual SMTP sessions or message tracking to diagnose delivery failures. In Microsoft 365, message tracking can show the path an email took and any filtering actions applied. This level of detail helps pinpoint whether a message failed at the gateway, transport rules, or user-level filters.

The Power of Email Authentication

Email authentication is no longer optional, they’re essential. Protocols like SPF, DKIM, and DMARC protect your domain, improve deliverability, and help receivers trust your messages. Combined with modern enhancements and good sending practices, they form a robust defense against spoofing, phishing, and spam.

In Part 3, we’ll look at how tools like N‑able’s Mail Assure fit into the SMTP delivery process and help protect you from malicious emails.

 Ben Lee is a Head Nerd at N‑able and has a long history working in the Microsoft space. You can find him on LinkedIn as BenLeeUK or email him at [email protected]