![]() Generally requires more storage space (96 bits or higher) to make reasonable uniqueness guarantees.If you use a timestamp as the first component of the ID, the IDs remain time-sortable.Each application thread generates IDs independently, minimizing points of failure and contention for ID generation.Another popular approach is to use UUIDs. For example, MongoDB’s ObjectId, which is 12 bytes long and encodes the timestamp as the first component. This approach leaves ID generation entirely up to your application, and not up to the database at all. Many existing solutions to the ID generation problem exist here are a few we considered: Generate IDs in web application The system should introduce as few new ‘moving parts’ as possible-a large part of how we’ve been able to scale Instagram with very few engineers is by choosing simple, easy-to-understand solutions that we trust.IDs should ideally be 64 bits (for smaller indexes, and better storage in systems like Redis).Generated IDs should be sortable by time (so a list of photo IDs, for example, could be sorted without fetching more information about the photos).The rest of this blog post addresses how we tackled this issue.īefore starting out, we listed out what features were essential in our system: The typical solution that works for a single database-just using a database’s natural auto-incrementing primary key feature-no longer works when data is being inserted into many databases at the same time. ![]() We evaluated a few different NoSQL solutions, but ultimately decided that the solution that best suited our needs would be to shard our data across a set of PostgreSQL servers.īefore writing data into this set of servers, however, we had to solve the issue of how to assign unique identifiers to each piece of data in the database (for example, each photo posted in our system). ![]() Our first question after deciding to shard out our data was whether PostgreSQL should remain our primary data-store, or whether we should switch to something else. Our application servers run Django with PostgreSQL as our back-end database. To make sure all of our important data fits into memory and is available quickly for our users, we’ve begun to shard our data-in other words, place the data in many smaller buckets, each holding a part of the data. With more than 25 photos & 90 likes every second, we store a lot of data here at Instagram. ![]()
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |