Saturday, October 21, 2017

Which nosql option relative to stored procedures and large arrays?

By Hường Hana 11:00 AM , , , Leave a Comment

I have a use case for a nosql data store but I don't know which one to use:

Each document in my data store has a key for _id and another key as an array of objects. Each object hash element of this array has a key for _elementid and another for color.

I want my server proxy to send an update request to the data store with a substring used as regex that qualifies all documents whose _id matches the regex. I then want to push an element onto the array of each document of this output. This new element will have the same color for each unshift but the _elementid will be unique for each.

Is there a nosql option out there that offers this kind of stored procedure? Does it have limits on the length of the array?

* EDIT *

(1) DOCUMENT A: 

{     _id : "this_is-an-example_10982029822",     dataList : [         {             _elementid : "999999283902830",             color : "blue",          }, {             _elementid : "99999273682763",             color : "red"         }     ] }  DOCUMENT B:   {     _id : "this_is-an-example_209382093820",     dataList : [         {             _elementid : "99999182681762",             color : "yellow"         }     ] }

(2) EXAMPLE OF UPDATE REQUEST

(let [regex_ready_array   ["this_is-an-example" "fetcher" "finder"]       fetch_query_regex   (str "^" (clojure.string/join "|^" regex_ready_array))       element_template    {                                 :_elementid { (rand-int 1000000000000000) }                                 :color      "green"                           }       updated_sister_objs (mc/bulk-update connection "arrayStore" {:_id {$regex fetch_query_regex }} "unshift" element_template)])

(3) DOCUMENT A: 

{     _id : "this_is-an-example_10982029822",     dataList : [         {             _elementid : "999999146514612",             color : "green",          }, {             _elementid : "999999283902830",             color : "blue",          }, {             _elementid : "99999273682763",             color : "red"         }     ] }  DOCUMENT B:   {     _id : "this_is-an-example_209382093820",     dataList : [         {             _elementid : "9999997298729873",             color : "green",          }, {             _elementid : "9999918262881762",             color : "yellow"         }     ] }

* EDIT 2 *

(1) the dataList array could be large (large enough that MongoDB's 16mb document size limit would present an issue);

(2) the _elementid values to be assigned to the additional dataList elements will be different for each new element and the the store will auto assign these as random number values

(3) a single update request should apply all updates, rather than one update per additional element;

(4) the OP is looking for a compare-and-contrast between several 'nosql solutions' which MongoDB, Cassandra, Redis and CouchDB being suggested as possible candidates.

1 Answers

Answers 1

By Seeing your question. I understand you are using JSONs and Clojure.

Lets see which are good NoSQL for JSONs. Quick overview of populor NoSQL

  1. Apache Cassandra : Data Model in Cassandra is essentially a hybrid between a key-value and a column-oriented (or tabular) database management system. Its data model is a partitioned row store with consistency.

  2. Redis: Redis maps keys to types of values.It has some abstract datatypes other than string like List, Sets, Sorted Sets, Hash Tables, Geospatial data.

  3. Apache CouchDB : CouchDB manages a collection of JSON documents.

  4. MongoDB : CouchDB manages a collection of BSON documents. BSON is Binary JSON http://bsonspec.org/spec.html.

If you are using lots of JSON payload you could use MongoDB or Apache CouchDB. But you want to update JSONs based on REGEX.

Lets check REGEX capability of CouchDB and MongoDB

  • It can be done easily with MAP Reduce in Both CouchDB and MongoDB

    Regex Select: db.student.find( { f_name: { $regex: 'this_is-an-example.*'} } ).pretty();

  • MongoDB: In mongodb we have regex operations. I have tried it and it works fine.

Reference

  1. https://docs.mongodb.com/manual/reference/operator/query/regex/

  2. mongoDB update statement using regex

  3. https://www.w3resource.com/mongodb/mongodb-regex-operators.php

    • CouchDB: I haven't tried CouchDB with Regex but as far I know it is possible. Regex function is available as per CouchDB documentation.

    { "selector": { "afieldname": {"$regex": "^A"} } }

Reference

  1. http://docs.couchdb.org/en/2.0.0/api/database/find.html
  2. Temporary couchdb view of documents with doc_id matching regular expression

You could you either of this MongoDB and CouchDB. Lots of resources are avalible for MongoDB.

If You Enjoyed This, Take 5 Seconds To Share It

0 comments:

Post a Comment