Data integration and cost optimization achieved with MongoDB Atlas Triggers
This is Kobayashi (@berlinbytes) from the Service Reliability Group (SRG) of the Media Division.
#SRGThe Service Reliability Group primarily provides comprehensive support for the infrastructure surrounding our media services, focusing on improving existing services, launching new ones, and contributing to open-source software (OSS).
This article explains the mechanism and examples of MongoDB Atlas Triggers, and introduces how to improve development efficiency and reduce costs through serverless data synchronization and operational automation.
About MongoDB Atlas
MongoDB Atlas is a fully managed cloud database service (DBaaS) available on major cloud providers such as AWS, Google Cloud, and Azure.
It's incredibly convenient because it allows you to build a scalable MongoDB environment in just a few clicks while minimizing infrastructure management.
MongoDB Atlas includes a feature called "Triggers" that streamlines application development and operation.
By utilizing Triggers, you can achieve serverless real-time processing triggered by database changes, as well as periodic maintenance tasks.
From here, I'd like to share the results of my research into how Triggers are being used, based on official examples and other sources.
Overview and Mechanism of MongoDB Atlas Triggers
Triggers are a feature that automatically executes predefined server-side logic (Atlas Functions) when a specific event occurs.
This allows you to implement event-driven processing and scheduled tasks without having to provision or manage infrastructure.
There are two main types of triggers:
1. Database Triggers
This is a trigger that fires when changes are detected to a MongoDB collection (document insertion, update, deletion, or replacement).
Technically, it is based on MongoDB's "Change Streams" feature, which captures change events in real time by monitoring the database's operation log (Oplog).
This allows you to build applications that respond instantly to user actions and changes in device state.
2. Scheduled Triggers
This is a trigger that executes a function periodically based on a specified schedule.
The execution frequency can be defined using "CRON expressions," which are widely used in Unix-like systems.
It is suitable for automating tasks that occur over time, such as nightly batch processing, periodic report generation, data cleanup, or updating tokens for external APIs.
How to use the two triggers
These two triggers can be used interchangeably depending on the objective you want to achieve.
- Situations where Database Triggers are suitable Use this when you want to react to "immediate changes" caused by users or IoT devices. For example, notifying the ordering system the moment inventory decreases, sending a welcome email upon user registration, or sending a push notification when a chat message is posted. Choose this option when you need to react immediately to changes.
- Patterns where Scheduled Triggers are suitable: They are suitable for operational tasks that need to be performed at a fixed time, and for cost management. For example, cases such as reducing database specifications outside of business hours to cut costs, aggregating data every morning to update dashboards, or periodically deleting old log data. You would likely choose this option for any time-based processing.
Real-time integration via Database Triggers
Here, we will introduce examples of system integration triggered by data changes and real-time processing, categorized by pattern.
Instant integration with external systems and event-driven systems
Based on data changes in MongoDB Atlas, we invoke external FaaS (Function as a Service) such as Azure Functions and AWS Lambda to synchronize data with other systems.
- Integration with Microsoft Fabric In building a real-time analytics platform, a configuration is used where changes in data on Atlas trigger Azure Functions, immediately writing the data to a data lake such as OneLake. This ensures that the analytics platform always has access to the latest data.
- Synchronization to Azure Synapse Analytics Similarly, it is used for real-time synchronization from Atlas to Azure Synapse. By detecting change events and persisting data, analysis using SQL and Spark becomes immediately possible.
- Catylex (Contract AI) The results of contract document analysis are reflected in each function in an event-driven manner. By automating processes such as data processing, tagging, and integration with external services, triggered by database updates, an event-driven, scalable system is realized.
IoT and real-time control
Database Triggers also appear to be useful for managing the state of physical devices.
- Bosch (Smart Buildings) This was used in a prototype for controlling lighting equipment from an AR app. When the "light status" document in MongoDB is updated, a trigger is fired, and the actual lighting is switched on/off via the API. The database acts as an intermediary between the app and the physical device, synchronizing the status.
- Arcstone (Manufacturing DX) Event data captured from the manufacturing line via Kafka and MQTT is aggregated in Atlas, and notifications and alerts are pushed to applications on the manufacturing floor as triggers. This functionality can be achieved without setting up a separate application server to capture change data.
Improving the user experience (notifications and synchronization)
It also contributes to improving the user experience in mobile apps and other applications.
- MongoDB Official Tutorial (Push Notifications) This example demonstrates how to send push notifications to users via the Firebase Admin SDK when new books are added or removed from a book app. By directly linking data updates with notifications, you can ensure users never miss the latest information.
- Cognigy (Contact Center AI) It detects changes in order status via chat and voice commands and synchronizes them in real time with inventory management and delivery APIs. This ensures that the customer service AI always provides accurate delivery status information.
Automating operations and reducing costs with Scheduled Triggers
Next, we will introduce examples of how Scheduled Triggers, which are executed at regular intervals, are being used to improve operational efficiency and reduce costs.
Cost optimization (automatic scaling/stopping)
In cloud database cost management, adjusting resources during periods of non-use is highly effective.
- Clear (Fintech) A leading Indian fintech company automated the process of scaling down (reducing specifications) its clusters overnight and scaling them up (restoring specifications) in the morning before business hours began, using Scheduled Triggers. This has resulted in cost savings of approximately $7,000 per month across 10 clusters.
- TAG (Financial Infrastructure) Costs are optimized by dynamically scaling up and down the infrastructure based on traffic forecasts. Using Scheduled Triggers, planned resource adjustments can be made without human intervention, even at night.
- Atlas Cluster Automation As an open-source tool released by members of MongoDB, a mechanism for simultaneously stopping (Pause) or resuming (Resume) tagged clusters has been made public. By automatically stopping clusters that do not need to be running at night or on weekends, such as development or testing environments, you can prevent unnecessary charges.
Regular maintenance and aggregation
Regular tasks necessary for stable system operation and data utilization are also automated.
- Autodesk Platform Services We schedule a process to periodically refresh OAuth Refresh Tokens before they expire. This prevents service interruptions due to authorization errors.
- Goodnotes (digital note-taking app) It regularly generates "aggregated collections" for use with the search and autocomplete functions. By periodically running complex aggregation processes in the background, it maintains fast response times when users search.
summary
MongoDB Atlas Triggers is a tool that can simplify application architecture and promote operational automation.
Database Triggers enable real-time interaction and notifications triggered by data changes, allowing you to build event-driven applications.
Scheduled Triggers allow for cost reduction through automated resource adjustments and serverless implementation of regular data maintenance.
By utilizing these features effectively, we can not only reduce development time but also enjoy business benefits such as optimized infrastructure costs and improved user experience.
At our company, we are already benefiting from cost optimization through Scheduled Triggers across multiple projects.
If you haven't already taken advantage of this feature, why not start by trying out automatic shutdown settings for your development environment or implementing a simple notification function?
SRG is looking for new team members.
If you are interested, please contact us here.