Things to consider to improve MongoDB availability: Configuration
#SRG(Service Reliability Group) is a group that mainly provides cross-sectional support for the infrastructure of our media services, improving existing services, launching new ones, and contributing to OSS.
IntroductionOur MongoDB cluster configuration patternStandaloneReplica SetShardingFor high availabilityManagement ToolsMongoDB Cloud ManagerMongoDB AtlassummaryConclusion
Introduction
MongoDBAs you know, is a document-oriented distributed database that falls into the category of NoSQL databases.
At our companyNode.jsDue to its high compatibility with other games, it has been used for a long time, mainly in the gaming industry.
Around 2011, MongoDB began to be used partially around the 1.X series, and by the time the 2nd series was released, it began to be used on a large scale, mainly in the social gaming field.
Currently, due to its schema-less and scalable features, it is not only used in the gaming industry, but is also widely used as a database for handling master data for media services.
When I search the web for information about our company's use of MongoDB, I find relatively old information, but there isn't much recent information.
- Case study on Tapple
- Example from Pigg Party
So in this article, I would like to briefly introduce the current situation.
Our MongoDB clusterConfiguration Pattern
Standalone
It consists of a single machine without replica sets or sharding.
It is mainly used in testing environments with small data sets, for research purposes such as temporary data and log deployment, and in personal development environments.
Replica Set
Although horizontal distribution by sharding is not required, the system would cause disruption to business if it were unavailable.Replica Setis adopted.
This is used for the daily testing environment and some of the system's functions.
Sharding
ShardingIt can support applications with large data sets that require high throughput, distribute both reads and writes horizontally across the cluster, and easily scale as needed.
You can also improve scalability by configuring each shard as an independent replica set.
On the other hand, the number of components to be managed increases, which inevitably leads to increased costs.
This is also managed by MongoDB Cloud Manager or operated by MongoDB Atlas.
For high availability
Most of our production environments are configured with replica sets or sharding.
In the past, we had a large number of physical servers in our datacenter and operated on a large scale with over 30 shards, but as equipment performance has improved and instances now have very high performance, we no longer operate physical servers.
The platform is built on our own private cloud called "Cycloud" and on public clouds such as AWS and Google Cloud.
As can be seen from the current configuration patterns, since MongoDB is the core database of the service, it is necessary to use replica sets in order to achieve a high level of availability. Sharding is also used for services that require high read and write traffic. As the cluster becomes larger and the amount of management required increases, procedures for scaling and version upgrades become more complicated, so management tools are used to reduce the operational burden.
Management Tools
MongoDB Cloud Manager
MongoDB Cloud Manager automates most of the infrastructure management, allowing you to perform upgrades, monitoring, backups, and more from a web-based UI.
MongoDB Atlas
MongoDB Atlas is a DBaaS (Database-as-a-Service) that is hosted on various public clouds as a managed service from MongoDB.
Compared to MongoDB Cloud Manager, it has the ability to scale up and down in response to workload fluctuations, allowing you to optimize performance and costs. In addition, it provides tools such as index suggestions and performance advisors, allowing you to optimize operational management costs.
As such, there are more opportunities to use official management tools, as using automated deployment management services reduces management costs.
Also, if we just leave this alone, it will end up looking like an introduction to MongoDB, so I'd like to add:
There are also cases where services that used MongoDB have migrated to other databases.
In recent years, databases such as the following have appeared that have APIs compatible with MongoDB.
- Amazon Document DB
- Azure Cosmos DB for MongoDB
- Oracle Database API for MongoDB
I would like to write an article introducing these and providing migration examples at some point.
summary
In summary, the MongoDB configuration is as follows:
- Using replica sets
- Introducing Sharding
is important for ensuring availability and can improve fault tolerance and load balancing.
The introduction of management tools also contributes to reducing operational burdens and operational errors.
And while it's important to use the right tool for the right job, the introduction of DBaaS has also been very effective.
Migrating to a database with a compatible API may also be effective depending on your requirements and is worth considering.
Conclusion
This time, we introduced the MongoDB configuration used within our company from the perspective of availability.
In addition, the following points were not mentioned this time:
- Memory size considerations
- Deciding on instance specifications
- Query optimization
- Cost optimization
- About indexes
- Introducing a caching layer
- Surveillance and monitoring
I plan to continue posting more in the future.
SRG is looking for people to work with us.
If you're interested, please contact us here.