Cacher Middleware

The Cacher Middleware is built-in middleware that gives you capability to cache the results of queries. To speed up your application. It’s only available for QueryBus

How to use it

As for any Middleware, you can use it by adding it to the bus instance.

const queryBus = createQueryBus<QueryHandlerRegistry>();
queryBus.useCacherMiddleware({
    adapter,
    cache: 'all',
    defaultTtl: 300
    defaultStaleTtl: 60,
})

Remember built-in middlewares are intent aware, therefore you can customize the behavior per intent using the key intents.

Of course, the key is in the adapter that you will provide to the createCacherMiddleware function. This adapter must respect the CacherAdapter interface.

Here is an example:

const createMemoryCacheAdapter = (): CacherAdapter => {
    const memory = new Map<string, { value: unknown; expiresAt: number }>();
    return {
        get: async (key) => {
            const entry = memory.get(key);
            if (entry && Date.now() < entry.expiresAt) {
                return entry.value;
            }
            memory.delete(key);
            return null;
        },
        set: async (key: string, value, ttl) => {
            const expiresAt = Date.now() + ttl * 1000;
            memory.set(key, { value, expiresAt });
        },
    };
};
const adapter = createMemoryCache();

We got you cover!

Do not copy/paste this boilerplate, if adapter is not passed to the createCacherMiddleware function, it will default to memoryCache and provide you with the basic cacher above.

Explanation

With the Cacher Middleware, if the intent result is already in the cache, it will be returned directly without calling the handler.

Aside of the adapter, you can provide the following parameters to the createCacherMiddleware function (and per intents):

• cache: all or only-cacheable (default: all) - all will cache all results
• defaultTtl: number (default: 3600) - the default time to live in seconds
• defaultStaleTtl: number (default: 60) - the default time to return stale data in seconds

Cache on demand

If you want to control the cacheable state from the handler (or another middleware), you can pass the option cache equals to only-cacheable. (globally or per intent) Then you need to tell the Middleware which queries are cacheable by adding the cacheable stamp to the result.

const handler = async (envelope: Envelope<Query>, deps: Deps) => {
    // your handler code here
    envelope.addStamp<CacheableStamp>('missive:cacheable', { ttl: 1800, staleTtl: 60 });
    return {
        // your result here
    }
};

Added Stamps

The Cacher Middleware is going to add:

• type FromCacheStamp = Stamp<{ age: number; stale: boolean }, 'missive:cache:hit'>;

  When the result is coming from cache.

• type HandledStamp<R> = Stamp<R, 'missive:handled'>;

  With the result from the cache. (this stamp will always be there, added by this middleware or the handler)

Shortcircuiting the processing

As explain the Middleware guide, you can shortcircuit the processing by NOT calling next() in a middleware.

This is exactly what the CacherMiddleware does. It does not call next() which means that all the middlewares after it will be skipped.

You can change this behavior by providing the shortCircuit flag to false.

    queryBus.useCacherMiddleware({
        adapter: memoryStorage,
        shortCircuit: false, // default is true
        defaultTtl: 20,
        intents: {
            ListAllCharacters: { shortCircuit: true }, // this intent will skip all the middlewares after Cacher
        },
    });

Handling won't happen twice!

Shortcircuiting has no impact on the “handling”. As CacherMiddleware is adding the HandledStamp the bus won’t call the handler anyway when the result is coming from the cache. The only impact of this flag is wether or not the next middlewares are called or not.

With great power comes great responsibility. - Missive.js is not opinionated, it’s up to you to decide what is best for your application.

Why do we default to `shortCircuit: true`

We believe this is the expected behavior, and as subsequent middlewares might have side-effects, we prefer to skip them in those scenarios. (but you do you!)

Stale While Revalidating

The Cacher Middleware is using the Stale While Revalidating pattern by redispatching the query to the bus behind the scene to refresh the cache.

It works like this:

1. While using the useCacherMiddleware method on the QueryBus, the createCacherMiddleware is called with the options and with the QueryBus instance.

   It does mean the middleware has access to the bus and can redispatch the intent.

2. When the result is coming from the cache, the middleware will detect if the data is stale or not.

   It does mean that the middleware cache for ttl seconds + staleTtl seconds in the cache store (via the adapter).

3. If the data is stale, the middleware will redispatch the query to the bus in a fresh new Envelope. The query will then go through the whole chain of middleware(s) and handler(s), and the result will be cached again.

   The Cacher Middleware knows to avoid the cache lookup thanks to the ReprocessedStamp added by the Envelope when the query is redispatched.

This pattern achieves the best of both worlds: speed and freshness.

You can disable this behavior by setting the staleTtl to 0.

1 revalidate at a time

The middleware will not revalidate the cache if the query is already in the process of being revalidated.

Going further

Look at the code of the Cacher Middleware