Losing money because your messages aren’t landing in inboxes? Wondering why this is happening and where the problem lies?
In this article, I’ll help you get closer to answering these questions by comparing email deliverability across leading transactional email service providers (bulk and email marketing providers coming soon 🔜) and explaining how much responsibility you have in getting your emails to recipients.
But before we jump in, I encourage you to first review the methodology behind my research to see how the data was gathered and evaluated. Or, if you are eager to read the deliverability results, you can, of course, jump straight into reading those.
Let’s dive in!
Before we start
In 2024, most email service providers (ESPs) are advanced platforms that support feedback loops, one-click unsubscribing, double opt-ins, SPF, DKIM, and additional tools to enhance deliverability and user experience.
However, it’s important to know that ESPs only account for about 30% of your email deliverability. This means that the ESP’s infrastructure and practices are just part of what determines whether your emails reach inboxes.
So, no matter which ESP you choose, you should still:
- Carefully clean your email list
- Build a strong sender reputation
- Complete other essential tasks (which I’ll discuss later)
When you do everything correctly, top ESPs should deliver your emails to inboxes 99.99% of the time. Of course, occasionally, an ESP might have deliverability issues due to downtime or system problems. However, these issues can happen to any platform unexpectedly, and they are usually quickly fixed.
Now, you might be thinking, “So, are deliverability comparisons just a sham then?”. No, they are not; instead, comparisons simply come with inherent limitations and nuances you should critically evaluate when reading them.
With that addressed, let’s move on to my research framework!
Methodology
To ensure a fair and unbiased comparison of email deliverability among various transactional email service providers, I developed a methodology I’ll explain in detail in just a minute. But first, I’ll introduce you to the tool I used to assess my email placement.
GlockApps is an email deliverability testing and monitoring platform that provides detailed insights into how emails perform across various email service providers.
Testing works by simulating the sending of your email campaign to a set of seed email addresses across multiple email providers and spam filters. It then provides insights into where your email lands (Inbox, Tabs, Spam, or not delivered) and highlights factors impacting your email deliverability.
Within GlockApps, you’ll get multiple reports for each test you run:
- Email placement report
- IP analytics report
- Domain analytics report
- Spam score report
- Email providers delivery report
I included the most important information from these reports in the table in the next section of this article.
To run tests in GlockApps, I used my paid account and simply followed the instructions shared by the platform for running a manual email test:
Now, onto the methodology:
- Registered accounts at the same time: With all the tested ESPs, I registered accounts around the same time. I also did the same when adding my domain, which was identical for each provider in order to eliminate any discrepancies related to domain reputation or history.
- Didn’t warm up domains: Since I was using free accounts with strict sending limits to reflect the experience of users without premium features, I skipped the domain warm-up process entirely. Additionally, warming up domains for some ESPs but not others could have created inconsistencies, potentially skewing the results.
- Used the same email template: All emails sent during testing used the same template, containing the same company details, HTML and CSS code, links, and images to ensure that variations in email design or copy did not influence deliverability outcomes.
You can take a look at what the template consisted of below:
Top transactional email service providers deliverability results
The following table includes a quick summary of my findings on the deliverability of popular transactional email service providers.
There, you can see how often messages sent using these providers land in inboxes, the tabs category, spam folder, or go missing, and how different spam filters rate them.
| Email service provider | Email placement results | Spam filter rating | Inbox email delivery with top providers |
| Mailtrap | Inbox: 78.8% Tabs: 4.8% Spam: 14.4% Missing: 2.0% | Google Spam Filter: Not spam; Not phishy Barracuda: Score 0 Spam Assassin: Score: -3.8 | Gmail (4 years+): 67.50% Gmail (6 months+): 67.50% Google Workspace: 100% Outlook: 77.78% Exchange (Office365): 66.67% Hotmail: 100% Yahoo: 55.56% AOL: 75% Zoho: 50% |
| Amazon SES | Inbox: 77.1% Tabs: 1.9% Spam: 20.0% Missing: 1.0% | Google Spam Filter: Not spam; Not phishy Barracuda: Score 0 Spam Assassin: Score: -4.3 | Gmail (4 years+): 22.22% Gmail (6 months+): 87.50% Google Workspace: 33.33% Outlook: 100% Exchange (Office365): 66.67% Hotmail: 100% Yahoo: 44.44% AOL: 75% Zoho: 100% |
| Mailgun | Inbox: 71.4% Tabs: 3.8% Spam: 23.8% Missing: 1.0% | Google Spam Filter: Not spam; Not phishy Barracuda: Score 0 Spam Assassin: Score: -5.3 | Gmail (4 years+): 100% Gmail (6 months+): 100% Google Workspace: 100% Outlook: 66.67% Exchange (Office365): 66.67% Hotmail: 40% Yahoo: 33.33% AOL: 50% Zoho: 0% |
| SendGrid | Inbox: 61.0% Tabs: 1.0% Spam: 17.1% Missing: 20.9% | Google Spam Filter: Not spam; Not phishy Barracuda: Score 0 Spam Assassin: Score: -0.1 | Gmail (4 years+): 75% Gmail (6 months+): 100% Google Workspace: 66.67% Outlook: 0% Exchange (Office365): 33.33% Hotmail: 0% Yahoo: 33.33% AOL: 50% Zoho: 100% |
| Postmark | Inbox: 83.3% Tabs: 1.0% Spam: 14.3% Missing: 0.9% | Google Spam Filter: Not spam; Not phishy Barracuda: Score 0 Spam Assassin: Score: -4.3 | Gmail (4 years+): 100% Gmail (6 months+): 100% Google Workspace: 100% Outlook: 100% Exchange (Office365): 66.67% Hotmail: 80% Yahoo: 77.78% AOL: 25.00% Zoho: 100% |
Explanation of email deliverability results
The results I shared in the previous section are pretty detailed, but at the end of the day, they are still just numbers. To enable you to gain more from reading the results, I’ll now give a few explanations as well as point out what you should keep in mind.
The drawbacks of free accounts
All the deliverability stats I shared in this article are based on the usage of free ESP accounts, which rely on shared IP pools to send emails. In such a setup, your messages can be affected by the overall reputation of a shared IP, potentially hindering their deliverability.
Also, as free accounts are typically intended for testing and exploring a platform’s features, users of free accounts often focus on trying out the features rather than optimizing their sending practices.
This can, in turn, impact the shared IP pools associated with these accounts and cause the free plan deliverability statistics not to accurately reflect what could be achieved with paid plans, where users tend to be more mindful of best practices and benefit from more stable sending environments.
Here is what deliverability stats look like for a user on a paid Mailtrap Email Sending plan using dedicated IP addresses:
Common mailbox provider issues
When I started the email deliverability testing for this article, I approached it with the assumption that deliverability challenges can happen to any ESP with any mailbox provider. However, when sending to certain major mailbox providers, I observed recurring deliverability issues across the board.
Outlook, Hotmail, Yahoo, and AOL posed consistent hurdles for nearly every ESP I tested, with the exception of Amazon SES, which did not show the same level of difficulty reaching Outlook and Hotmail inboxes.
Spam score rating
In the table sharing my email service providers deliverability results, you saw mentions of three popular spam filters – Google Spam Filter, Barracuda, and SpamAssassin.
These filters analyze incoming emails based on a variety of criteria, such as content quality, sender reputation, as well as technical configurations and employ unique rating methodologies to assess the likelihood of an email being spam.
Here is how the rating scale for each popular email filter works:
- Google Spam Filter employs a machine learning-based approach where each email is assigned a probability score between 0 and 1, indicating the likelihood of it being spam.
A score closer to 0 means the email is likely legitimate, while a score nearing 1 suggests it is spam. Google continuously updates its filtering criteria based on user interactions and evolving spam tactics, ensuring that the rating scale adapts to new threats and legitimate sending practices. - Barracuda, a widely used enterprise-level spam filtering solution, utilizes a point-based scoring system.
Each email is evaluated against a set of predefined rules, and points are assigned for various spam indicators such as suspicious links, excessive use of images, poor HTML formatting, and known blacklisted IP addresses.
For example, an email containing multiple spammy keywords might receive +10 points, while a legitimate email with proper authentication might only accumulate +2 points.
Administrators can set threshold levels (e.g., 5 points) to determine whether an email is classified as spam, quarantine, or delivered to the inbox based on its total score. - SpamAssassin, an open-source spam filtering platform, also uses a scoring system where each detected spam characteristic contributes to the overall score of an email.
The scale typically ranges from 0 to 10, with higher scores indicating a greater likelihood of the email being spam. Factors such as the presence of specific keywords, the use of certain MIME types, DNS blacklists, and email header anomalies each add points to the total score.
For instance, using phrases commonly associated with spam might add +3 points, while failing SPF authentication could add +5 points. Emails that exceed a predefined threshold (commonly around 5 points) are flagged as spam, while those below are delivered to the inbox.
Potential inconsistencies in testing results
Even when the same tests are run by different individuals or organizations, email deliverability results can vary significantly. Culprits can be factors like:
- sending environment
- historical sender reputation
- targeted mailbox providers
- geographic regions
Along with that, timing also plays a role, as provider filters and inbox placement odds can fluctuate from day to day or even hour to hour.
Moreover, adjustments made by ESPs — such as changes to IP pools, filtering algorithms, or authentication policies — can cause results to differ over time. Because of this, sender experiences can be very different, highlighting the inherently dynamic nature of email deliverability.
Sending via API vs sending via SMTP
From a technical standpoint, the sending method — be it SMTP or email API — should not have an effect on your delivery rates. Both methods typically rely on the same infrastructure, IP addresses, and reputation metrics within the email service provider’s ecosystem.
So, if you notice a difference in deliverability between SMTP and API sending, this is likely due to another factor, such as using a different IP pool or a different sending configuration.
Still, regardless of this, as long as you maintain consistent sending practices and a strong sender reputation, you can expect comparable deliverability results whether you send via SMTP or an API.
What your ESP controls
By now, we’ve established that ESPs do influence deliverability, but that influence only covers certain aspects, such as enforcing domain authentication, adhering to SMTP sending standards, and maintaining Internet Service Provider-level configurations.
Of course, ESPs also play a critical role in the management of shared IP reputation, making it vital for them to enforce strict anti-spam rules and swiftly address blacklisted senders.
So, overall, while an ESP’s job is to establish an infrastructure, policies, and processes that empower senders to achieve optimal results, it doesn’t make them obliged to guarantee universal deliverability.
What you control
Once you pick an ESP, from that point forward, you as a sender need to, first, have trust that they will do everything I described above and, second, fulfill your role in achieving high deliverability.
Under this role, you’ll need to:
- Complete the technical setup (authenticate a domain and protect all user-facing forms)
- Build and maintain a domain reputation
- Keep a healthy email list (use double opt-ins, email validation and verification, a sunset policy to remove inactive contacts, and so on)
- Adhere to strong content creation practices (create relevant content, be compliant with anti-spam guidelines, and don’t trigger spam filters)
- Create a professional sender identity
- Build and maintain a sender reputation
- Focus on engaged recipients
- Send expected content at expected times
Following the above principles will enable you to achieve good inbox placement regardless of the ESP platform you use, so make sure you don’t skip any of them. Of course, along with this, you should also make testing and auditing your email deliverability a regular task.
To help you with that, I recommend Mailtrap’s email deliverability testing tutorial:
As well as the tutorial on running an email deliverability audit:
Wrapping up
Email deliverability is a complex concept, and understanding how much an ESP contributes to it can be even more complicated. But, hopefully, after reading this article, you know that an ESP has only partial responsibility and that landing in inboxes will take effort from your end as well.
Of course, not all ESPs are the same. Each one has its own pros and cons—features, integrations, ease of use, pricing—you name it. So, when choosing a platform, take a good look at what matters most to you.
Also, instead of asking, “Which platform will handle all my email delivery for me?” try asking, “Which one will support me best in getting great results after I’ve done things like authentication, cleaning my list, and following best practices?”
With that mindset, you’ll be in a much better spot to pick the right ESP for your needs.
Explore more comparison content:
- SMTP Providers Compared
- Choosing the Best Transactional Email Service
- Top 10 Mass Email Service Providers for Businesses of All Sizes
- Free SMTP Servers Compared
- 10 Best Email APIs
- 7 Best SMTP Service for Email Marketing
- Best 6 Email Marketing Platforms Compared]
Or delve into general email deliverability topics: