Loading content...
HAVE A QUESTION?
Call us : +1-945-209-7691
- Software DevelopmentIT Consulting & DesignAI & Data SolutionsQuality AssuranceTeam & Resource SupportBusiness Support Services
In today’s digital landscape, efficient communication between software systems is crucial. This post explores how Pub/Sub messaging enables scalable, decoupled, and fault-tolerant email systems.
Pub/Sub ensures emails are delivered even during spikes or partial failures.
Decoupling services allows for independent scaling of producers and consumers.
Message queues prevent bottlenecks and improve async processing speeds.
Loading content...
Let's discuss your project and create a custom web application that drives your business forward. Get started with a free consultation today.
“In today’s digital landscape, efficient communication between software systems is crucial for business success.”
This opening sentence establishes the fundamental business problem being addressed. Modern businesses rely on multiple software systems that need to exchange information efficiently. For example:
When these systems can’t communicate efficiently, businesses experience delays, lost information, and missed opportunities. The statement frames the technical discussion that follows in terms of business value.
In this post, we’ll explore how Pub/Sub works for email integration, step-by-step implementation guidance, and best practices to optimize deliverability and performance.
Publish-subscribe is a messaging pattern where senders (publishers) categorize messages into classes without knowledge of which receivers (subscribers) will receive them. Similarly, subscribers express interest in specific message classes and only receive messages that match their interests, without knowledge of which publishers are generating the messages.
This decoupling offers several advantages:
Email remains a critical communication channel for businesses, but integrating email functionality with modern applications presents several challenges:
Pub/Sub architectures address these challenges elegantly by providing a robust foundation for email processing pipelines.
A typical pub/sub-based email integration system consists of:
Gmailfs API with pub/sub notifications provides a powerful way to integrate email functionality into your applications. Lets explore how this works in practice:
Go to APIs & Services > Library and enable the following:
Please note: Google does not allow service accounts to access Gmail unless you are a G Suite/Workspace admin using domain-wide delegation. For personal accounts, we need to use OAuth access.
Now open the Cloud Shell.
First, you’ll need to create a topic in Google Cloud Pub/Sub that will receive notifications from Gmail. Run the following command:
1
gcloud pubsub topics create gmail-notificationsPlease note: gmail-notifications is the topic name — you can choose your own name.
Now, you need to subscribe to the topic.
1
gcloud pubsub subscriptions create email-worker-sub --topic=email-notificationsEnsure the service account has the correct roles and permissions assigned to it.
If you also need to consume Gmail-Watch notifications (the Gmail API can publish a Pub/Sub message every time a mailbox changes), drop the following FastAPI endpoint into the same service that processes your email-events topic. It receives the HTTP push that Cloud Pub/Sub sends to your webhook, decodes the base-64 payload, and prints the change record.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
from fastapi import FastAPI, Request, status
import base64
import json
app = FastAPI()
@app.post("/gmail/webhook", status_code=status.HTTP_200_OK)
async def gmail_webhook(request: Request):
"""
Endpoint for Cloud Pub/Sub push subscriptions that forward Gmail ‘watch’
notifications. Pub/Sub wraps the notification in a JSON envelope:
{
"message": {
"data": "base64-encoded-json",
"messageId": "1234567890",
...
},
"subscription": "projects/-/subscriptions/your-sub"
}
We need to:
1. Decode the 'data' field (base-64).
2. Parse the inner JSON to read Gmail’s historyId / email metadata.
"""
envelope = await request.json()
message = envelope.get("message", {})
encoded = message.get("data")
if not encoded:
# Pub/Sub might occasionally send test messages with no data.
return {"status": "ignored – no data"}
decoded_bytes = base64.b64decode(encoded)
decoded_str = decoded_bytes.decode("utf-8")
gmail_event = json.loads(decoded_str)
# 👉 Replace with your own processing logic
# (e.g., push historyId into another Pub/Sub topic or queue a worker)
print("📬 Gmail push notification:", json.dumps(gmail_event, indent=2))
return {"status": "success"}Gmail watch() expires every 7 days
Set a cron job or schedulercron job or scheduler to refresh it automatically.
You have to set up the webhook URL for push notification as below
Gmail Subscriber
Integrating your email workflows with a Pub/Sub messaging system brings robustness, scalability, and flexibility to your communications infrastructure. By decoupling event production from email delivery, you can handle varying workloads, ensure deliverability, and add new consumers (analytics, archiving) with minimal changes.