IObjectInstanceCache.ReadThrough

I thought I should share a small tip about a feature that we use internally but that perhaps not all are aware of that it exists. It is probably nothing you will use everyday but it can be really handy if you have "expensive" outbound calls to e.g. the database or a Web API. That is the extension method ReadThrough on interface IObjectInstanceCache.

The signature look like (there are some overloads for example to specify CacheEvictionPolicy)

public static T ReadThrough<T>(
	this IObjectInstanceCache cache,
	string key,
	Func<T> readValue,
	ReadStrategy readStrategy
)
where T : class

The generic type parameter T is the expected type of the item to read/cache. The parameter key is the cachekey for the item. readValue parameter is a delegate that will be called to load the item (and putting it in cache) when it does not exist in cache. Now the interesting parameter here is the readStrategy. That is an enum that can have two values Immediate which will call the readValue delegate immediately to load the item in case it is not present in cache. The other readStrategy option is Wait.

A strategy Wait means that when the first request comes in and find that the item queried for is not present in cache then it will put a marker in cache and then call the readValue delegate to load the item. Now if a second request comes in and asks for the same item before the first readValue delegate has finished then the cache will notice that there is a marker for that item (meaning it is being loaded) and then the readValue delegate will not be called for the second request. Instead will it be put to wait for the first request to finish and then it gets the result from the first request.

So by using Wait strategy you can avoid that several threads simultaneously makes the same expensive call. Instead one call is made and the result is then shared. So if you have some code where you expect that several parallell request might hit at the same time then this could be useful. A typical such location could be resources loaded from startpage, menus etc that could be hit simultaneously by several threads when site is restarted (and hence the cache is empty).

May 11, 2018

Comments

May 11, 2018 01:33 PM

Nice.

I've been using the ReadStrategy for a while... but with this pattern:

Model test;

if (this.synchronizedCache.TryGet(cacheKey, ReadStrategy.Wait, out test) == false)
{
    // Fetch...

    this.synchronizedCache.Insert(
        cacheKey,
        test,
        EvictionPolicy);
}

May 11, 2018 02:33 PM

I would prefer that we would not have added Get and TryGet with ReadStrategy parameter (they are there for historical reasons). The problem with them are that in almost all cases when using cache you have the pattern

Check if item is in cache
If not, fetch item
put item to cache

Now Get and TryGet uses the ReadStrategy in the first step but then when doing step 2 and 3 no markers are added to cache and hence several concurrent threads could pass 1 and simultaneously fetch item. So I would say Get/TryGet is usable when you only check if something is in the cache but if it is not in cache you just return. If the pattern is to fetch and add item to cache I would recommend ReadThrough

valdis May 12, 2018 12:03 AM

IObjectInstanceCache.WhyWouldWantToGetAnItem() & IObjectInstanceCache.TryFirstMethodMentioned().

btw, when you write "second request will be put on wait until 1st will finish". how this is implemented? are there any timeouts/cancellation tokens, etc? is thread of 2nd request suspended or there is some threading primitive that is used to resume 2nd request?

May 12, 2018 12:11 AM

To clarify what I meant with regretting that we added Get/TryGet with ReadStrategy parameters. Even though they can be useful in some special cases the naming of the methods indicates that those are the "usual/normal case" (to me at least) even though I think ReadThrough is the thing the majority wants.

So from an API perspective I think we did some mistakes, we should have promoted what we consider the "main" pattern (ReadThrough) more. Either by naming Get/TryGet more inconvenient (even if that might sound strange) or removing them, or changed/renamed ReadThrough to be an overloaded method of Get. For a good API the "natural" choice should be the right choice.

May 12, 2018 09:24 PM

Somewhat simplified the first thread that enters will add something like ManualResetEvent (in practice we have an own type wrapping the event) in the cache with the same cachekey. When other threads enters and find the event type in the cache they will be be set to Wait for the event to fire. Then when the first thread (that is fetching data) is finished it will put the data in cache and then signal the event that the other treads are waiting for. Then the other threads can read the data from cache and return.

We have logic so the event is fired even if the fetching thread fails or timeouts so threads shuold not get locked forever.

Quan Mai May 21, 2018 10:50 AM

Echo-ing what Johan was saying: https://vimvq1987.com/episerver-commerce-performance-optimization-part-2/

Kane Made It May 24, 2018 07:22 AM

Thanks Johan and Quan, it aboslutely comes in handy when I need it :-)

Please login to comment.