Andreas J
Jun 26, 2017
  4841
(3 votes)

Benchmarking Episerver serializable carts

In version 10.2, Episerver introduced a feature called serializable carts. This enables you to store carts as json in a key-value manner to improve performance.

This project aims to benchmark the performance improvements of Episervers new serializable carts system. I will compare:

  1. The old cart helper
  2. The new order repository
  3. The new order repository combined with the serializable carts feature

In order to create good measurements, I will benchmark the same operations performed through the three different cart systems. These operations include:

  1. Create an empty cart and persist it
  2. Add a line item with a meta field to that cart and persist it
  3. Validate the cart and apply campaigns and persist it
  4. Apply payment, process it and persist the cart
  5. Persist the cart as a purchase order

I find it difficult to benchmark only one of these operations at a time, therefore when I execute a step, I also execute the steps prior to that step — i.e. in order to benchmark step 3, I also run step 1 and 2 before.

The source code is available at GitHub. Honestly, you should just continue reading there.

Definition of success

If operations 1 to 3 (the most common operations) are quicker when using serializable carts, I think we can consider it a good improvement.

To measure the performance, I execute the operations in a Web API and stress the API using Apache Benchmark:

ab.exe -c 50 -n 1000 http://localhost:61651/api/cart?operationsToExecute=1
  • -c 50 indicates 50 clients
  • -n 1000 indicates the total number of requests

The output of each benchmark will be the time it takes to process 1000 requests.

Results

See the GitHub page for a well-formed table.

   Cart helper    Order repository    Serializable carts
1    00:00:02.910    00:00:03.349    00:00:01.991
2    00:00:04.972    00:00:05.354    00:00:02.534
3    00:00:10.099    00:00:11.883    00:00:04.306
4    00:00:16.519    00:00:16.692    00:00:04.331
5    00:00:20.217    00:00:19.387    00:00:08.422

Conclusions

The results speaks for themselves. Serializable carts performs really well compared to the other two. The newer order repository (without serializable carts enabled) is just as fast/slow as the cart helper. Just getting rid of cart helper namespaces won’t bring you any performance improvements.

I think that serializable carts are mature enough to be used in production, but there are drawbacks though. The serialized cart is stored as json in the database, so if you want to query carts based on some metadata, that will be tricky. If you don’t need to do that, go with serializable carts!

Contribute

I’m no expert in neither benchmarking nor Episerver Commerce and there can be things to improve. Please create a pull request if you think you can improve these benchmarks.

In order to try this yourself, follow the steps below:

  1. Clone repository at GitHub
    git clone https://github.com/AndreasJilvero/BenchmarkingEpiserverCarts.git
  2. Create two local SQL server databases and replace the existing connection strings in the web config
  3. Run Episerver SQL install script 
    PM> Install-EPiDatabases
  4. Run Episerver SQL update script
    PM> Update-EPiDatabases
  5. Run Star.Epi.CMS
  6. Login as your Windows user
  7. Execute all migration steps
  8. Use Apache Benchmark (or any other tool) to perform requests

Andreas J

Read more on my personal blog - https://andreas.jilvero.se/blog

Jun 26, 2017

Comments

Quan Mai
Quan Mai Jun 26, 2017 12:10 PM

@Andreas: That's very good performance. However the new cart system really shines if you:

  • Have multiple lineitems in cart
  • Delete the cart after the order is created (which are the normal practices)

It would be perfect if you can add those into the tests

Please login to comment.
Latest blogs
Optimizely Forms: You cannot submit this form because an administrator has turned off data storage.

Do not let this error message scare you, the solution is quite simple!

Tomas Hensrud Gulla | Oct 4, 2024 | Syndicated blog

Add your own tools to the Optimizely CMS 12 admin menu

The menus in Optimizely CMS can be extended using a MenuProvider, and using the path parameter you decide what menu you want to add additional menu...

Tomas Hensrud Gulla | Oct 3, 2024 | Syndicated blog

Integrating Optimizely DAM with Your Website

This article is the second in a series about integrating Optimizely DAM with websites. It discusses how to install the necessary package and code t...

Andrew Markham | Sep 28, 2024 | Syndicated blog

Opticon 2024 - highlights

I went to Opticon in Stockholm and here are my brief highlights based on the demos, presentations and roadmaps  Optimizely CMS SaaS will start to...

Daniel Ovaska | Sep 27, 2024