November Happy Hour will be moved to Thursday December 5th.
AI OnAI Off
Pundit-blogging with massive commenting (hint request) & possible ObjectStore concerns.
Vote:
Hi,
We're looking for an efficient design to the following problem.
The site we develop is going to have a section for "Pundit Columns" that the users will be able to comment at. We were looking for an effifient design for that that would integrate nicely with the rest of the site.
We have been through some propositions and are still looking at how to best solve the problem. The probable load would be around 20 articles with about 150 comments on each of them daily.
1) Use wordpress or a similar third party software and intgrate it back with episerver. At the moment I really do not feel like this could ever bee a good solution, with us having a problem to synchronize the content back and forth.
2) Use the current blogging functionality which uses XForms to store its data. My concern here is that the functionality of it is not wide enough for our scope, (the comments for our solution would have to be easily moderated, filtered for profanity and so on). It will also grow the size of the ObjectStore significantly, possibly affecting performance of other aspects of the site.
The rest of the solutions assume we have one page per article and debates just about storing the coment.
3) Regular pages for the comments, much like the current newsgroups in episerver seems to be defined. The good part would be that we have the whole comment hierarchy we would get for free, comments can be easily replied-to with the thread structure stored as a page structure rooted at the article the comment are for.
I realize that there may be serious performance drawbacks to that in terms of performance penalty.
4) Use regular pages with comments stored as a list of IItem's in the ObjectStore. One ObjectStore object per page. Especially for this I have done some hacking in the ObjectStore as reflectred in my blog :) http://blog.najmanowicz.com/2007/03/30/episervers-objectstore/
Basically the problem with it would most probably be the database block fragmentation with multiple page comments growing simultaneously. Also this approach loads all comments for the selected page in memory, given that a page can have a couple of hundred of comments, this could quickly become a memory problem.
5) Use regular pages with comments stored as records (1 ObjectStore IItem per comment). This would help us deal with the database fragmentation but it will also quickly grow the ObjestStore size and also we have some concerns regarding the speed of searching for them. Doesn't filtering of the IItem objects require each and every object to be serialized prior to checking the condition?
6) Finally, creating our own table and persisting the comments by ourselves, this way gives us the most flexibility but also creates the cost of development as all the functionality would have to be developed by it.
We also suspect that at some point the customer may want us to add the ability to comment on other pages, so the more flexible the solution we could deliver, the better.
I wonder if any of you have faced a similar dilema recently, and perhaps have some thought regardign it. Also, could anyone share with us some impressions or suggestions concerning the ObjectStore?
Thanks in advance for any hint you may throw our way! :)
Cheers
Mar 30, 2007 17:16
This is a very common issue, the need to store "something" related to pages in EPiServer. And as always, the answer depends on the needs.
The most common solution is to use pages for this, which is easy to do, but perhaps the most costly alternative in regards to performance.
You have done a good job explaining the needs here, and I'll try to comment each proposition, seen from my viewpoint.
1) What you get here is a ready to use infrastructure for commenting, with spam counter-measures and more. I don't know the wordpress architecture, and it might or might not be a suitable alternative. Personally, I agree that keeping the solutions in sync might be more trouble than it is worth.
2) One would think that this is the best option, as the example blog template has solved the same problem with XForms. But, this is not how I would have solved it. Remember that the example template developers have some constraints that you do not have. They cannot add new tables to the database, as this would effectively extend the database with non-supported tables (the templates are just examples.) And, you do not want those extra tables there if you do not install the demo site. The import feature cannot create tables. Likewise, the object store has not been documented, the namespaces are not in the SDK, which means they by default is off-limits. The intention has been to document it, and expose it as a supported feature. We're just not there. That - I think - is why the blog template uses XForms to store the comments. The XForms API uses the object store underneath.
3) With the amount of pages you're talking about here, I'd strongly advise against storing the comments as pages. With 48 effective working weeks, 5 days a week, 20 articles with 150 comments each, we're talking 720 000 pages in a year. If we count in sleeping, and off hours, there will be a new comment every 15 second, which will invalidate all the other comments for a page from the page cache. This will simply not scale.
4) I've read you blog entry. Very interesting! I think this is one of the best options, if you can get it to work, and that the performance good enough. I've heard that there were some changes to the object store API in EPiServer CMS (next version) so you might have to change something down the road if you go for this option. I'm not sure how much penalty you'll get from the db fragmentation. You'll get the higher memory usage anyway, as I presume you'll have to render all the comments anyway, or at least get the count of them. Lets say that each comment is 1K which means 150K for all of them for one article. Approx 15 times the size of the PageData itself.
5) I too share your worries about the filtering. I do not know enough about the object store to say for sure, but your testing might reveal any performance problems with this one. Saving each comment individually would only help in keeping the fragmentation down.
6) More work, more flexible, best performance. I'd go for this one.
On paper, the Object Store seems like the obvious option, but the fact that it has not been documented, and not used by many outside EPiServer makes it kind of risky. The number of tables and classes needed for a comment system using its own tables in the same database as EPiServer should be fairly limited, and the complexity low. I'd go for that option.
/s
#15210
Apr 04, 2007 11:25
Thank you for the informative and really insightful response.
We have had a discussion about it here as well and for the time being we will have 2 implementations using Domain Objects persisted in at least 2 technologies for a start - one for ObjectStore and one for nhibernate.
We are hiding them behind an interface to make it an easily switchable and will look at the performance of both.
I've done some additional analysis on the object store (the result of which you can find again on my blog) and developed a simple persistence using the technology already.
Neat and powerful stuff. Not sure how it handles the indexing still therefore that still raises the performance questions.
#15211
Apr 04, 2007 20:20
