Join us this Friday for AI in Action at the Virtual Happy Hour! This free virtual event is open to all—enroll now on Academy and don’t miss out.

 

Dynamic data store batch saving? Poor performance.

Vote:
 

Maybe I'm reaching here, but I thought I'd try the DDS for storing large amounts of simple POCO:s - and we're talking LARGE amounts, like 100k objects. Doesn't seem to be working out that good, can't even save 10k objects in a reasonable amount of time. Is there a way to batch save objects? Or am I just pushing the limit of the DDS' capacity? I'm leaning towards just serializing everything directly instead..

#53037
Aug 25, 2011 15:27
Vote:
 

Just out of curiosity: Are you doing 10K calls to save objects or do you have an object graph containing that many objects (like an object containing a large list or something)?

I don't know all the inner workings of the DDS but I'm thinking that if you do separate calls it will end up doing separate DB calls, while when saving a large object graph at once the DDS would be "smart" and group objects into fewer inserts/updates?

If you havent tried putting your objects in an object containing a list (as you probably know you can't store a list directly), it would be interesting to know if it affects the performance.

The same question goes for using transactions - could that make performance better (or worse?). Paul describes DDS transactions here: http://world.episerver.com/Blogs/Paul-Smith/Dates1/2011/4/Dynamic-Data-Store--Tips-Tricks-and-Best-Practises/

#53066
Aug 26, 2011 8:13
Vote:
 

Hi Magnus,

Thanks it's a good idea, I'll try it today. I thought about transactions too but if I'm still just saving them one by one I don't think they will have an positive impact on performance (maybe worsen it). But I'll try that as well (I should be using them anyway).

#53067
Aug 26, 2011 8:39
Vote:
 

Unfortunately I rembered incorrectly - I am in fact using an object graph already. 1 object that holds a few properties and a list of 10k+ simple POCO:s. I tried DDS ExecuteTransaction as well, none of these methods gave any increase in performance.

Saving 1 wrapper object that holds a list of 1k objects takes around 11 seconds. 100 objects clocks in at around 1 second.

#53074
Aug 26, 2011 9:59
Vote:
 

FYI I ended up using a wrapper with a byte array for each type containing the serialized objects, then storing the wrapper to the DDS (= lightning fast). Kind of meaningless to use the DDS for this maybe but at least I don't have to write to the db myself.

#53079
Aug 26, 2011 11:18
Vote:
 

Thanks for sharing your results!

#53082
Aug 26, 2011 11:29
This thread is locked and should be used for reference only. Please use the Episerver CMS 7 and earlier versions forum to open new discussions.
* You are NOT allowed to include any hyperlinks in the post because your account hasn't associated to your company. User profile should be updated.