Skip to Navigation
How Calendly Streamlines Email Testing with Mailtrap
emails sent every month
developers, QAs, and PdMs involved in email testing
using Mailtrap
What’s Calendly
Calendly is arguably the world’s most popular tool for scheduling meetings. It’s used by over 10 million customers worldwide, with companies such as eBay, Dropbox, and Twilio among their clients.
Calendly connects to your calendar and takes charge of your availability. It coordinates appointment scheduling and keeps all parties in sync through timely reminders. What’s more, it gives you a straightforward overview of upcoming meetings and enables seamless team collaboration.
Email is an essential part of any appointment scheduling, and it’s no different for Calendly. Since 2017, they have been using Mailtrap to test all their transactional emails.
HQ
Atlanta, Georgia, USA
Industry
Business Applications
Key Features Used
The Challenge
The Calendly platform sends out around 40 million emails during an average month. These include meeting invitations, reminders, “thank you” notes, account confirmations, password resets, etc. What’s more, customers can customize their email templates, which creates many variables to test before a release.
At any one time, around 15 developers are involved in building and improving email-related features. A similar number of QAs jump in next, running extensive tests of each new build. One mutual inbox for capturing all their test emails wouldn’t work at such a scale. Instead, each tester would need to set up a separate, personal inbox, or ideally, a few of them to begin with.
At the rapid pace that Calendly is growing every year, this would mean an extensive amount of manual work to set things up and organize the daily work of different teams. What’s more, the emails they send are opened in virtually any email client.
As such, Calendly needed a reliable and accessible way of testing the emails inside and out, validating their HTML/CSS rules, identifying spam threats, etc. Essentially, they needed a way to automate their email testing and make it scalable.
Mailtrap streamlines our testing because everything is concentrated in the same area. I jump between different environments, but I still always know where to look for my test emails.
Michele Rawas
QA Engineer at Calendly
The Result
Calendly integrated Mailtrap for all their email testing. Each engineering environment features a dedicated Mailtrap inbox. As engineers iterate on their emails, each subsequent message lands instantly in a respective Mailtrap inbox.
The most common scenario for engineers is to test whether emails are triggered in the first place and subsequently arrive into their Mailtrap inbox. In some cases, they have to dig deeper – i.e., check a message for HTML/CSS errors, test responsiveness, keep an eye on a spam score, etc.
When all tests pass, an engineer updates a PR, and a new review app is sprung up. Automation using Mailtrap API assigns an available Mailtrap inbox from an over 100 QA inbox pool on the Calendly account. A review app is automatically provisioned with the SMTP credentials of the respective inbox.
A QA engineer opens a Jira ticket and finds a connected PR. There, they see the details of a deployed branch and a respective Mailtrap inbox. As they test the workflows, emails such as account confirmations or meeting reminders are delivered into a test mailbox, one by one.
They pay special attention to new emails and those that were updated. In Mailtrap, it’s easy to tell them apart and compare different versions of the same message. QAs complete the acceptance tests, and if everything works as expected, the build proceeds to staging.
For staging, Calendly sets up one more testing mailbox within Mailtrap. Here, QAs test the fixes of the issues they have previously received from support. They simulate the actions of an end-user and monitor if all emails are sent as expected. If everything goes well, the feature is passed for later release on the Calendly platform.
What I like the most about Mailtrap is its simplicity. It’s very easy to set up. We use API to automate tests, and the API experience has been amazing.
Rob Wilson
Engineering Manager at Calendly
Why Mailtrap
When we talked to Calendly about their use case, each of our interviewees emphasized the convenience of having the entire email testing in one place through shared inboxes accessible for any team member. They also highlighted that Mailtrap makes it easy to find any email in question, compare its versions, and navigate around different environments.
They love having the complete testing toolset at their disposal and being able to verify different aspects of an email right from their Mailtrap inbox.
They also appreciate Mailtrap’s simplicity and the support for automation, provisioning review servers, and automating end-to-end scenarios. It is easy to get up and running, and the whole experience is extremely straightforward.
Finally, they enjoy the peace of mind they have knowing that the emails are not actually sent and will never land in customers’ inboxes. They can therefore work more efficiently and test anything in their apps without risks.
Mailtrap has made it so much simpler for me to test because it’s an organized folder of all of the review apps that we have. It makes it so easy for me to find precisely the emails I need.
Raaziq Whyte
QA Engineer at Calendly
