Benchmarking Episerver serializable carts

In version 10.2, Episerver introduced a feature called serializable carts. This enables you to store carts as json in a key-value manner to improve performance.

This project aims to benchmark the performance improvements of Episervers new serializable carts system. I will compare:

1. The old cart helper
2. The new order repository
3. The new order repository combined with the serializable carts feature

In order to create good measurements, I will benchmark the same operations performed through the three different cart systems. These operations include:

1. Create an empty cart and persist it
2. Add a line item with a meta field to that cart and persist it
3. Validate the cart and apply campaigns and persist it
4. Apply payment, process it and persist the cart
5. Persist the cart as a purchase order

I find it difficult to benchmark only one of these operations at a time, therefore when I execute a step, I also execute the steps prior to that step — i.e. in order to benchmark step 3, I also run step 1 and 2 before.

The source code is available at GitHub. Honestly, you should just continue reading there.

Definition of success

If operations 1 to 3 (the most common operations) are quicker when using serializable carts, I think we can consider it a good improvement.

To measure the performance, I execute the operations in a Web API and stress the API using Apache Benchmark:

ab.exe -c 50 -n 1000 http://localhost:61651/api/cart?operationsToExecute=1

• -c 50 indicates 50 clients
• -n 1000 indicates the total number of requests

The output of each benchmark will be the time it takes to process 1000 requests.

Results

See the GitHub page for a well-formed table.

Conclusions

The results speaks for themselves. Serializable carts performs really well compared to the other two. The newer order repository (without serializable carts enabled) is just as fast/slow as the cart helper. Just getting rid of cart helper namespaces won’t bring you any performance improvements.

I think that serializable carts are mature enough to be used in production, but there are drawbacks though. The serialized cart is stored as json in the database, so if you want to query carts based on some metadata, that will be tricky. If you don’t need to do that, go with serializable carts!

Contribute

I’m no expert in neither benchmarking nor Episerver Commerce and there can be things to improve. Please create a pull request if you think you can improve these benchmarks.

In order to try this yourself, follow the steps below:

1. Clone repository at GitHub

   git clone https://github.com/AndreasJilvero/BenchmarkingEpiserverCarts.git

2. Create two local SQL server databases and replace the existing connection strings in the web config
3. Run Episerver SQL install script 

   PM> Install-EPiDatabases

4. Run Episerver SQL update script

   PM> Update-EPiDatabases

5. Run Star.Epi.CMS
6. Login as your Windows user
7. Execute all migration steps
8. Use Apache Benchmark (or any other tool) to perform requests

Andreas J

Read more on my personal blog - https://andreas.jilvero.se/blog