November Happy Hour will be moved to Thursday December 5th.

Best way to save large amount of plugin data

Vote:
 

Hi!

What is the best way to save large amount of data generated by my plugin?

My plugin will generate a lot of data and in my current project we have an extra database with a couple of tables to handle the storage of the plugin data. The plan is to distribute this plugin later on and then we cannot use an extra database. So what to do? Can the DDS handle large amont of data with a relationship database kind of structure?
The database structure is quite simple. One table Project and one ProjectItems. First we store data in the ProjectItems and after that we create Projects by selecting specific ProjectItems.

Can the DDS handle this without performing issues? Lets say 5000 projects and 100000 project items? On occasion we have to change the staus on 100 project items at the same time, can that be done in one call or do we have to update each row and what will happen with the performance?

I have checked out RavenDB, could that be something to use?

Thanks for your help!

/Kristoffer

#160593
Sep 29, 2016 12:10
Vote:
 

DDS will not handle that amount of data with good performace, it is very bad at handling big amount of data and also giving you good performance.

I would say that RavenDB is one way to go, it will give great performace, other ways are using Azure table storage or Azure document DB. All of those have to downside of beeing outside and also costing money for usage, even though Azure table storage are very low price on.

Will you distribute it to internal project of as an official AddOn?

#160594
Sep 29, 2016 13:12
Vote:
 

Hi Henrik!

Thanks for your response. It will not be an official addon, customer will buy the plugin and we will distribute it. It would be nice if a nuget package could give you all you need.

Then I will look into the RavenDB to if I can get it to work nicelly with my plugin. The RavenDB you can get via a Nuget package so if would be nice to set up a constraint to that package in mine and then it will probably work just fine.

/Kristoffer

#160595
Sep 29, 2016 13:22
Vote:
 

In case of that, make it so that the plugin creates everything that needs in RavenDB.

Another version since you will take money for it is for you self to host the db/table storage and only add specific table/db for each customer, then you need to pay for their usuage but that you can control through license.

#160597
Sep 29, 2016 14:21
Vote:
 

Yes, will do. It looks like RavenDB can run embedded in your application looks really nice!

And that is also antoher way to do it, wont work this time but actually it is a good idea if you want to control how much data the user has if you want to charge depending on usage.

#160599
Sep 29, 2016 14:33
Vote:
 

Nice, I did not know that, then that sounds like the best solution!

#160601
Sep 29, 2016 14:36
Vote:
 

So looked into the RavenDB and if you want to distribute it it is quite expensive. $2200 a year per developer.

I ran into this: http://www.litedb.org/. Looks nice this one aswell and it is free and free to distribute. Anything you know about or do you know about any other NoSql to use?

/Kristoffer

#160619
Sep 30, 2016 12:03
Vote:
 
#160632
Sep 30, 2016 14:05
* You are NOT allowed to include any hyperlinks in the post because your account hasn't associated to your company. User profile should be updated.