Hi Tom,

The latest Episerver A\B test package does allow for block level testing in the same way as page level testing. What process are you going through that you are able to test pages but not blocks?

The interaction between A\B testing and workflow approvals is a sequential approach. Use the workflow approval process on content edits then on the last workflow step the content should be in a ready to publish state if approved, then use the A\B test option in the publish menu to A\B test the approved draft version. This is because, as you mention, the A\B test allows the draft version to go live on the site for a portion of your visitors, so any approval has to occur before running a test. This is also the reason why we check to see if a user has publish rights to the content in order to setup or start an A\B test as a user trusted with publish rights should know when the draft content is ready to be viewed by site visitors.

I am not sure what your last point means, the module does not work with fetching pages/content? Our A\B package hooks into the loaded content event in the CMS to determine if a user should be/are included in an A\B test, any loads of content under test via the API should trigger the A\B test logic as long as the load of said content triggers a LoadedContent CMS event. Could you clarify the behavior you are seeing?

Thanks,

-Jason

Hi Jason,

Thank you so much for your quick response. Really appreciate it! Happy to have a call about this if you'd prefer? However, in the meantime I will come back to you on your points:

Page vs Block
So ultimately we are testing at page level, but, to be able to test at page level we have to publish a new block on an unpublished (draft) page to begin the test.
We cannot start an A/B test if we simply edit an existing block. Let's just say we wanted to change one word on the page. We have to create a new block paste all the content into it and change the specific word. It would be so much easier if we could just edit the existing blocks.

No approval on challenger version
The challenger page can only be in draft and not published. Therefore we are showing a draft page to customers without having the relevant approval. From what i can understand of your comment this can be avoided - can you explain how?

A/B testing not working on Fetching Content

so the A/B package does 'hook' in to loaded content. But when we try and begin a A/B test on a page that fetches content it doesn't work.

Probably best to have a call about this if you get time?

Thanks,
Tom

Hi Tom,

The behavior you describe is correct for Pages vs Blocks since edits to blocks on a page is not considered an edit to the page itself. The page object in question has no updates to it's data since the block that has been edited had it's data changed, but the reference to the block that the page has is still the same. That is the same reason why editing a block on a page does not trigger a publish action on that page, only on the block. Is testing the block you edit not possible?

For content approval, the approval process just has to happen before starting an A\B test on the content. So the last step in the approval process will put the content in a Ready to Publish state, instead of actually publishing, just create and start an A\B test.

I think I understand the last part, you are programmatically fetching data for the page and loading it in? This is the same scenario as the block since we have to A\B test versions of a page object there would be no change in the page object if that code does something different our testing package has no way of determining what the version B should be. Our package can only test edits to the IVersionable content saved within the CMS.

I am not generally available for calls, but if you think the package is doing something in error feel free to open a support ticket. Our support engineers should be able to direct you to the correct resource to handle your issues.

Thanks,

-Jason