Friday, March 3, 2017

Do bulk inserts/update in MongoDB with PyMongo

Leave a Comment

How do I bulk update/insert in mongoDb with pymongo/pandas. The error I get is batch op errors occurred I reason I get is because I set the "_id", which I want to do. I code runs fine on first run, but on second run it fails. I want to use pandas in workflow. The data does have a datetime object.

The syntax is completely different for upsert = True, with Update. An efficient solution with update would be helpful, where "_id" or "qid" could be set. But, there are python datetime objects!

InSQL   = 'SELECT * from  database2.table2 ' sqlOut  = pd.read_sql(InSQL,cxn) sqlOut['_id'] = "20170101" + ":"+ sqlOut['Var']     dfOut   = sqlOut.to_json(orient='records',date_format='iso' ) try:     db["test"].insert_many(json.loads(dfOut)) except Exception as e:  print e 

0 Answers

If You Enjoyed This, Take 5 Seconds To Share It

0 comments:

Post a Comment