Does mongodb auto generate id?
|
This blog article has been re-published with my permission by MongoDB at https://www.mongodb.com/blog/post/generating-globally-unique-identifiers-for-use-with-mongodb Show
MotivationBy default, MongoDB generates a unique ObjectID identifier that is assigned to the _id field in a new document before writing that document to the database. In many cases the default unique identifiers assigned by MongoDB will meet application requirements. However, in some cases an application may need to create custom unique identifiers, such as:
Due to the multi-threaded and distributed nature of modern applications, it is not always a straightforward task to generate unique identifiers that satisfy application requirements. OverviewThis posting covers the following topics:
How to guarantee identifier uniqueness at the database levelAll of the approaches that we propose here generate identifiers that are globally unique for all practical purposes, however depending on the chosen approach there may be remote edge cases where the generated identifier may not be absolutely globally unique. If the risk of a clash of unique identifiers is deemed to be sufficiently remote or the consequences of such a clash are minor, then one may consider writing to the database without additional uniqueness checking. This is a valid approach that is likely to be adopted by many applications. However, if it is absolutely essential to enforce that the generated identifier is globally unique, code may be written defensively to guarantee that the database will never write the same unique identifier twice. This is relatively easy to implement on a non-sharded collection by specifying a unique index on a particular field, which prevents a duplicate value for that field from ever being written to the collection. Note that by default the _id field always obeys the unique index constraint. If a write fails due to a collision on a field that has a unique index, then the application should catch this error and generate a new unique identifier and try to write the document to the database again. If a collection is sharded, there are restrictions on the use of unique indexes. If the restrictions for using a unique index on a sharded collection cannot be satisfied, then a proxy collection may be used to guarantee uniqueness before writing data to the database. Use ObjectID as a unique identifierDescriptionMongoDB database drivers by default generate an ObjectID identifier that is assigned to the _id field of each document. If a document arrives to the database without an _id value, then the database itself will assign an ObjectID to the _id field. In many cases the ObjectID may be used as a unique identifier in an application. ObjectID is a 96-bit number which is composed as follows:
Benefits
Drawbacks
Single counter document solutionDisclaimerThe approach described in this section is generally not recommended due to the potential for the single counter document to become a bottleneck in the application. DescriptionA unique identifier may be required in a monotonically increasing and continuous sequential order, which is similar to the sequence functionality that is implemented by some RDBMSs. This may be achieved by implementing a solution that is based on a centralised counter document that resides in a collection that we have called uniqueIdentifierCounter. This centralised counter document is the only document in the uniqueIdentifierCounter collection, and its COUNT field will track the current unique identifier value. Each time a new unique identifier is needed, findAndModify will be used on the counter document to atomically increment the COUNT field and return the pre-increment (original) document that has a unique COUNT value. The COUNT value can then be used by the application as a unique identifier. For example the application could assign the COUNT value to the _id field of a document that will be written to a given collection. Example implementationA counter document for unique identifier generation could look as follows: {
"_id" : "UNIQUE COUNT DOCUMENT IDENTIFIER",
"COUNT" : 0,
"NOTES" : “Increment COUNT using findAndModify to ensure that the COUNT field will be incremented atomically with the fetch of this document",
}
And the unique identifier-generation document could be atomically requested and incremented as follows. Note that by default the document returned from findAndModify is the pre-modification document: db.uniqueIdentifierCounter.findAndModify({
query: { _id: "UNIQUE COUNT DOCUMENT IDENTIFIER" },
update: {
$inc: { COUNT: 1 },
},
writeConcern: 'majority'
})
Benefits
Drawbacks
Single counter document with range allocationDescriptionThis approach is similar to the previous approach, with the difference being that instead of incrementing the COUNT value by 1, we may wish to increment it by a larger number that will represent a batch of unique identifiers that will be allocated by the database to the application. For example, if the application knows that it needs 1000 new unique identifiers, then the application would use findAndModify() to atomically get the current COUNT and increment the COUNT value by 1000. The document returned from the findAndModify command would contain the starting value for the batch of unique identifiers, and the application would loop over 1000 values from that starting point. Note that with this approach an application may pass in whatever value it wishes for incrementing the COUNT value, and therefore this approach can be made to be flexible. For large batches this increment would be a large number, and for a single identifier this would be set to 1. Example implementationThe following demonstrates the javascript shell commands that would atomically increment the COUNT by 1000 and return the previous (before the increment) counter document: var seq_increment = 1000;
db.uniqueIdentifierCounter.findAndModify({
query: { _id: "UNIQUE COUNT DOCUMENT IDENTIFIER" },
update: {
$inc: {COUNT: seq_increment },
}
writeConcern: 'majority'
})Benefits
Drawbacks
Multiple range allocator counter documentsDescriptionThis approach is similar to the previous approach, but instead of having a single counter document, we could have many counter documents stored in the uniqueIdentifierCounter collection. For example, there may be 1000 counter documents (numbered 0 to 999) each responsible for allocating 1 billion unique numbers that are taken from a specific range that has been allocated to each counter. In this case, counter 499 would be responsible for allocating values from 499000000000 to 499999999999. Note that this particular example results in unique numbers ranging from 0 to 999,999,999,999 which is a 12 digit number. Example implementationBelow we show the format and initial values assigned to the counter documents in the uniqueIdentifierCounter collection: /* The following would be the initial state of the 0th counter document, which is responsible for the range of unique identifiers from 0 to 999,999,999 */
{
"_id" : "COUNTER DOCUMENT NUMBER 0",
"MAX_VALUE": 999999999,
"COUNT" : 0,
"NOTES" : "Increment COUNT using findAndModify to ensure that the COUNT field will be incremented atomically with the fetch of this document",
}
/* The following would be the initial state of 499th counter document, which is responsible for the range of unique identifiers from 499,000,000,000 to 499,999,999,999 */
{
"_id" : "COUNTER DOCUMENT NUMBER 499"),
"MAX_VALUE": 499999999999,
"COUNT" : 499000000000,
"NOTES" : "Increment COUNT using findAndModify to ensure that the COUNT field will be incremented atomically with the fetch of this document",
}
/* Etc… */
With this approach, each time the application needs a new unique number or a range of unique numbers, the application would randomly generate a number between 0 and 999 which it would use to perform a query against the _id attribute in the uniqueIdentifierCounter collection. This would select a particular counter document from which to retrieve the unique identifier(s). This is demonstrated in the following example, in which we randomly select one of the counter documents, and request a batch of 100 unique numbers from that counter: var which_counter_to_query = Math.floor((Math.random()*1000));
var seq_increment = 100;
db.Unique Identifier_counter.findAndModify({
query: { _id: "COUNTER DOCUMENT NUMBER " + which_counter_to_query},
update: {
$inc: {COUNT: seq_increment },
},
writeConcern: 'majority'
})Benefits
Drawbacks
Randomly choose a unique identifier in the application and retry if it is already usedDescriptionThis approach relies on the fact that if a unique index is defined in a collection, that any document written to that collection must have a unique value assigned to that field in order for it to be successfully written. Therefore, we can randomly generate a unique identifier number in the application, assign it to the unique field in a document, and try to write that document to the collection. If the write succeeds, then we know that the value that we assigned to it is unique. If the write fails, then the application must catch the failure and randomly generate a new unique identifier which can then be used to write the document to the collection. If the number of collisions is low, then this can be an efficient way to write documents to the database with each document having a guaranteed unique identifier. Example implementationIf we know that we will have a maximum of one billion records in our database, in order to have a low probability of selecting an identifier that has already been assigned (aka a collision), we could randomly choose a number between 0 and 999,999,999,999 (from zero to one trillion minus one). For this example, the range that we are using to select the random number is 1000 times bigger than the number of documents that we expect to write to this collection, which results in a worst case expected 0.1% chance of a collision. We would then assign the randomly generated number to a field that has a unique index, and write that document to the database. If the write succeeds, then we know that the identifier is unique. If the write fails, then the application must randomly choose another identifier and try again until the write succeeds. Note that there are some restrictions on the use of unique indexes when used in sharded clusters. If the restrictions for using a unique index on a sharded collection cannot be satisfied, then a proxy collection may be used to help generate unique indexes. Benefits
Drawbacks
Generate unique identifiers in the applicationDescriptionRFC 4122 defines a standard for generating 128-bit UUIDs. The RFC specifies different algorithms for generating UUIDs, which are called UUID versions. Standard libraries exist that will generate 128-bit UUIDs at the application level. The BSON specification defines UUID as a valid subtype, and can be found at: http://bsonspec.org/spec.html. Benefits
Drawbacks
Is MongoDB ID auto increment?Although MongoDB does not support auto-increment sequence as a default feature like some relational databases, we can still achieve this functionality using a counter collection. The counter collection will have a single document that tracks the current unique identifier value.
Is ID mandatory in MongoDB?"_id " is always mandatory. Its a primary key for every document in Mongo. They're like identification and unique and makes your life easier especially when you query the database and hey its considered best practice as well.
How object ID is created in MongoDB?MongoDB - ObjectId. The first 4 bytes representing the seconds since the unix epoch.. The next 3 bytes are the machine identifier.. The next 2 bytes consists of process id.. The last 3 bytes are a random counter value.. Does MongoDB Auto Create collection?MongoDB creates collections automatically when you insert some documents. For example: Insert a document named seomount into a collection named SSSIT. The operation will create the collection if the collection does not currently exist.
|
