Dynamic Data Store caching – The internals

This post will give you some background in how caching works in Dynamic Data Store (DDS) and give you knowledge of what to expect when it comes to object instances and such when working with DDS.

Problem with caching

The classical problem when it comes to caching in multithreaded environments is how to deal with object instances.

The easiest approach is to just store the object instance somewhere and when the object is requested next time the same instance is returned. However a problem with this approach is if the object is not immutable and one thread changes the state of the instance. Then since the instance is “shared” all other requests for the same object will be affected by the change. Therefore the approach of caching object instances directly should only be done for immutable objects. This is for example how PageData instances work. The PageData instances that are cached are read-only and to get a writable instance you need to call CreateWritableClone which will return a new not shared instance.

For DDS we cache all objects regardless of they are immutable or not (we don’t have that information). To be able to do this we could not use the approach of caching object instances directly. Instead we cache the object data in an intermediate format and each time the object is requested a new instance is created and the object state is set from the intermediate format. The consequence of this is that the DDS cache will for two subsequent calls for the same object return two separate object instances (they will though have same “state”).  

Two levels of caching

An instance of DynamicDataStore contains an IdentityMap which can be seen as a first level cache. This first level cache stores the object in their “real” format and hence two subsequent Load calls to the same store instance will return the same object instance. The algorithm for loading an object from DDS is that first the IdentityMap is checked for the object, if not present there the “shared” cache (described above) is checked and if not found there the database is queried for the object. You can disable the IdentityMap on a store instance be setting property KeepObjectsInContext to false.

Performance

Since the DDS cache does not store fully instantiated object instances which can be directly delivered the “cost” to deliver an object is higher than for example the PageData cache. However the cost is much less than the cost it would be to load the object from database. Below is a simple test where a measurement is taken for how long it takes for DDS to deliver an object ten times. The first result will hit the IdentityMap inside the store instance meaning it will return the same instance for every call to Load. The second result disables the IdentityMap (this could be seen creating new instances of the store for each Load) meaning it will hit the “shared” cache each Load. The third test runs without IdentityMap and “shared” cache meaning it will Load from database each call.

public class TestObject
{
   public Identity Id { get; set; }
   public string Prop1 {get;set;}
   public Guid Prop2{get;set;}
   public DateTime Prop3{get;set;}
}

[TestMethod]
public void Cache_Measurement()
{
  string testStoreName = "testStore";
  try
  {
      DynamicDataStore store = DynamicDataStoreFactory.Instance.CreateStore(testStoreName, typeof(TestObject));
      CacheProvider.Instance = new HttpRuntimeCacheProvider();

      Stopwatch watch = new Stopwatch();
      
      //Note: Save will put object in cache and IdentityMap
      Identity id = store.Save(new TestObject() { Prop1 = "a string", Prop2 = Guid.NewGuid(), Prop3 = DateTime.Now });
      
      //First measure to load from IdentityMap
      watch.Start();
      for (int i = 0; i < 10; i++)
      {
          TestObject test = store.Load<TestObject>(id);
      }
      watch.Stop();
      Debug.WriteLine(String.Format("Load from IdentityMap took {0} ms", watch.ElapsedMilliseconds.ToString()));

      //Now Load from "shared" cache
      store.KeepObjectsInContext = false;
      store.Refresh();
      watch.Start();
      for (int i = 0; i < 10; i++)
      {
          TestObject test = store.Load<TestObject>(id);
      }
      watch.Stop();
      Debug.WriteLine(String.Format("Load from Shared cache took {0} ms", watch.ElapsedMilliseconds.ToString()));

      //Now Load from database
      CacheProvider.Instance = new NullCacheProvider();
      watch.Start();
      for (int i = 0; i < 10; i++)
      {
          TestObject test = store.Load<TestObject>(id);
      }
      watch.Stop();
      Debug.WriteLine(String.Format("Load from database took {0} ms", watch.ElapsedMilliseconds.ToString()));

  }
  finally
  {
      DynamicDataStoreFactory.Instance.DeleteStore(testStoreName, true);
  }
}

Running the above unit test will give the result as:

Load from IdentityMap took 2 ms
Load from Shared cache took 6 ms
Load from database took 53 ms

Immutable objects

We have plans that it should be possible to decorate (e.g. by an interface or attribute) a type as immutable/read-only or “notcachable”. Then objects marked as immutable could be stored in “shared” cache in their final format which would gain performance. Other objects that are either big in size or loaded very rarely could be marked as “notcachable” to reduce the memory consumption for the application. How/when this will be implemented is not yet decided.