Amazon SES — Handling Bounces and Complaints
Why do we go for Bounce and Complaint Handling while sending emails?
In general while sending emails, we will prepare some recipient addresses as our mailing list, which is valid and our recipients want and expect our mail. But sometimes, some emails which are invalid will bounce, and if valid recipients do not want your mail, they may mark your email as spam in their email client. High bounce and complaint rates put your account at risk of being shut down. So in order to avoid such problem, we’ll handle the bounces and complaints and will remove those emails for not sending any emails further.
The flow of Bounce and Complaint handling in AWS using SNS:
In order to handle bounces and notifications, AWS provides a service called SNS(Simple Notification Service). Using this feature we can handle bounces or complaints by configuring a webhook endpoint or email or using amazon’s SQS service. As Django developers, we’ll try to know how to handle bounce or complaints using webhooks in the current blog post. For this, we’ll use amazon’s boto package as development API.
1. First, we need to create an SNS Topic. An SNS Topic is a communication channel to send messages and subscribe to notifications. It provides an access point for publishers and subscribers to communicate with each other. The following code snippet will create an SNS Topic called ‘bounce-complaint-topic’.
import boto
sns_conn = boto.sns.connect_to_region(region_name, aws_access_key_id="*********", aws_secret_access_key="*********")
topic_obj = sns_conn.create_topic('bounce-complaint-topic')The success response of creating the topic will be as following.
{u'CreateTopicResponse':
{u'ResponseMetadata':{u'RequestId': u'42b46710-degf-52e6-7d86-2ahc8e1c738c'},
u'CreateTopicResult':{u'TopicArn': u'arn:aws:sns:eu-west-1:467741034465:bounce-complaint-topic'}
}
}Here in the above result, the topic-arn is somewhat different to what we had created, it is formed with the combination of region-name, customer account number and topic name.
2. After creating the topic we’ll now create a subscription request. To receive messages published to a topic, we have to subscribe an endpoint to that topic. An endpoint can be a mobile app, web server, email address, or an Amazon SQS queue that can receive notification messages from Amazon SNS. Once you subscribe an endpoint to a topic and the subscription is confirmed, the endpoint will receive all messages published to that topic. As web developers, we will use web server webhook or URL as an endpoint. With the following code snippet, we’ll create a subscription.
bounce_sub = sns_conn.subscribe(topic_arn_name, 'webserver protocol', 'endpoint')
Ex: bounce_sub = sns_conn.subscribe('arn:aws:sns:eu-west-1:467741034465:bounce-complaint-topic', 'http', 'http://demo.com/bounce/complaint/notify/')After successful creation of the subscription request, we’ll receive some JSON data in the post request to the URL that we configured for our notifications to confirm the subscription. We’ll confirm the subscription by just calling the URL in the JSON response with built-in python libraries.
The sample JSON response will be as following.
{
"Type" : "SubscriptionConfirmation",
"MessageId" : "***********************",
"Token" :"*******************************",
"TopicArn" : "arn:aws:sns:eu-west-1:467741034465:bounce-complaint-topic",
"Message" : "You have chosen to subscribe to the topic arn:aws:sns:eu-west-1:467741034465:bounce-complaint-topic.\nTo confirm the subscription, visit the SubscribeURL included in this message.",
"SubscribeURL" : "https://sns.eu-west-1.amazonaws.com/?Action=ConfirmSubscription&TopicArn=arn:aws:sns:eu-west-1:467741034465:bounce-complaint-topic&Token=*******",
"Timestamp" : "2015-08-12T07:05:43.614Z",
"SignatureVersion" : "1",
"Signature" : "**********************",
"SigningCertURL" : "https://sns.eu-west-1.amazonaws.com/************.pem"
}We’ll just open the “SubscribeURL” to confirm the subscription.
js = json.loads(json_body.replace('\n', ''))
if js["Type"] == "SubscriptionConfirmation":
subscribe_url = js["SubscribeURL"]
urllib.urlopen(subscribe_url)3. After confirming the subscription we’ll now set the notifications types that will be published to the specified Amazon SNS topic, which was set to the web server URL.
conn = ses.connect_to_region(region_name, aws_access_key_id="******", aws_secret_access_key="******")
bounce_notif = conn.set_identity_notification_topic("email", "Bounce", "topic_arn")
complaint_notif = conn.set_identity_notification_topic("email", "Complaint", "topic_arn")On setting the notifications, we’ll receive a success JSON response to the web server URL.
Successful Bounce Notification Subscription JSON response:
{
"Type" : "Notification",
"MessageId" : "****************",
"TopicArn" : "aws:sns:eu-west-1:467741034465:bounce-complaint-topic",
"Message" : "{\"notificationType\":\"AmazonSnsSubscriptionSucceeded\",\"message\":\"You have successfully subscribed your Amazon SNS topic 'aws:sns:eu-west-1:467741034465:bounce-complaint-topic' to receive 'Bounce' notifications from Amazon SES for identity 'info@micropyramid.com'.\"}\n",
"Timestamp" : "2015-08-12T07:05:45.919Z",
"SignatureVersion" : "1",
"Signature" : "***********************",
"SigningCertURL" : "https://sns.eu-west-1.amazonaws.com/*****.pem",
"UnsubscribeURL" : "https://sns.eu-west-1.amazonaws.com/?Action=Unsubscribe&SubscriptionArn=aws:sns:eu-west-1:467741034465:bounce-complaint-topic:*********"
}Successful Complaint Notification Subscription JSON response:
{
"Type" : "Notification",
"MessageId" : "**********",
"TopicArn" : "aws:sns:eu-west-1:467741034465:bounce-complaint-topic",
"Message" : "{\"notificationType\":\"AmazonSnsSubscriptionSucceeded\",\"message\":\"You have successfully subscribed your Amazon SNS topic 'aws:sns:eu-west-1:467741034465:bounce-complaint-topic' to receive 'Complaint' notifications from Amazon SES for identity 'info@micropyramid.com'.\"}\n",
"Timestamp" : "2015-08-12T07:05:46.338Z",
"SignatureVersion" : "1",
"Signature" : "***********",
"SigningCertURL" : "https://sns.eu-west-1.amazonaws.com/SimpleNotificationService-d6d679a1d18e95c2f9ffcf11f4f9e198.pem",
"UnsubscribeURL" : "https://sns.eu-west-1.amazonaws.com/?Action=Unsubscribe&SubscriptionArn=aws:sns:eu-west-1:467741034465:bounce-complaint-topic:******"
}To unsubscribe to the notifications we can just open the “UnsubscribeURL” that is coming in the response. By this, the process of setting our web server URL to the amazon SNS is completed.
From now we’ll receive bounce or complaint notifications to the web server URL that we configured.
“Bounce” Notification:
{
"Type" : "Notification",
"MessageId" : "***********",
"TopicArn" : "arn:aws:sns:eu-west-1:********:bounce-complaint-topic",
"Message" : "{\"notificationType\":\"Bounce\",\"bounce\":{\"bounceSubType\":\"General\",\"bounceType\":\"Permanent\",\"reportingMTA\":\"dsn; a6-178.smtp-out.eu-west-1.amazonses.com\",\"bouncedRecipients\":[{\"action\":\"failed\",\"emailAddress\":\"boubounce@simulator.amazonses.com\",\"status\":\"5.1.1\",\"diagnosticCode\":\"smtp; 550 5.1.1 user unknown\"}],\"timestamp\":\"2015-08-12T07:58:38.130Z\",\"feedbackId\":\"0000014f20eb07dc-58484953-f5d7-4809-800f-ccdfd7041fe4-000000\"},\"mail\":{\"timestamp\":\"2015-08-12T07:58:37.000Z\",\"sourceArn\":\"arn:aws:ses:eu-west-1:********:identity/info@micropyramid.com\",\"source\":\"info@micropyramid.com\",\"messageId\":\"0000014f20eb0414-99d976eb-f1c6-47af-94b2-b68be09b931e-000000\",\"destination\":[\"boubounce@simulator.amazonses.com\"],\"sendingAccountId\":\"********\"}}",
"Timestamp" : "2015-08-12T07:58:38.175Z",
"SignatureVersion" : "1",
"Signature" : "*****************",
"SigningCertURL" : "https://sns.eu-west-1.amazonaws.com/SimpleNotificationService-d6d679a1d18e95c2f9ffcf11f4f9e198.pem",
"UnsubscribeURL" : "https://sns.eu-west-1.amazonaws.com/?Action=Unsubscribe&SubscriptionArn=arn:aws:sns:eu-west-1:********:bounce-complaint-topic:*****"}
“Complaint” Notification:
{
"Type" : "Notification",
"MessageId" : "***********",
"TopicArn" : "arn:aws:sns:eu-west-1:********:bounce-complaint-topic",
"Message" : "{\"notificationType\":\"Complaint\",\"complaint\":{\"complainedRecipients\":[{\"emailAddress\":\"complaint@simulator.amazonses.com\"}],\"complaintFeedbackType\":\"abuse\",\"userAgent\":\"Amazon SES Mailbox Simulator\",\"timestamp\":\"2015-08-12T07:58:39.000Z\",\"feedbackId\":\"**************\"},\"mail\":{\"timestamp\":\"2015-08-12T07:58:37.000Z\",\"sendingAccountId\":\"***********\",\"source\":\"info@micropyramid.com\",\"messageId\":\"************\",\"destination\":[\"complaint@simulator.amazonses.com\"],\"sourceArn\":\"arn:aws:ses:eu-west-1:********:identity/info@micropyramid.com\"}}",
"Timestamp" : "2015-08-12T07:58:40.545Z",
"SignatureVersion" : "1",
"Signature" : "****************",
"SigningCertURL" : "https://sns.eu-west-1.amazonaws.com/SimpleNotificationService-*******.pem",
"UnsubscribeURL" : "https://sns.eu-west-1.amazonaws.com/?Action=Unsubscribe&SubscriptionArn=arn:aws:sns:eu-west-1:********:bounce-complaint-topic:****"
}We can get the corresponding bounce/ complaint emails from above JSON responses and stop them to not to send emails from next time.
