Introduction

● The Guava project is an open-source release of Google's core Java libraries

○ Stuff like collections, primitives support, concurrency libraries, string processing, & cetera

○ These are the libraries that other projects are built on

● The package com.google.common.cache contains our caching libraries

○ Simple, in-memory caching

○ Thread-safe implementation (internally similar to ConcurrentHashMap)

○ No explicit support for distributed caching

 

Types of Caches

● We provide two types of caches

○ LoadingCache: knows how to load entries when a cache miss occurs

■ LoadingCache.get(key) returns the value associated with key, loading it first if necessary

○ Cache: does not automatically load entries

● We're going to focus on the loading case here; it's usually what you want

 

Simple Loading Cache

CacheLoader<String, String> loader = new CacheLoader<String, String>() {

public String load(String key) {

return key.toUpperCase();

}

};

LoadingCache<String, String> cache = CacheBuilder.newBuilder().build(loader);

 

Simple Loading Cache

cache.size(); // returns 0

cache.getUnchecked("simple test");

// cache miss, invokes CacheLoader

// returns "SIMPLE TEST"

cache.size(); // returns 1

cache.getUnchecked("simple test");

// cache hit

// returns "SIMPLE TEST"

 

Concurrency

● Cache instances are internally implemented very similar to ConcurrentHashMap

○ And are thus thread-safe

● But what happens if multiple threads simultaneously request the same key?

● CacheLoader.load will be invoked a single time for each key, regardless of the number of requesting threads

○ The result will be returned to all requesting threads and inserted into the cache using the equivalent of putIfAbsent

 

Checked Exceptions

● What if loading causes a checked exception?

CacheLoader<String, String> checkedLoader =new CacheLoader<String, String>() {

public String load(String key)throws IOException {

return loadFromDisk(key);

}

};

 

Checked Exceptions

LoadingCache<String, String> cache = CacheBuilder.newBuilder().build(checkedLoader);

try {

cache.get(key);

} catch (ExecutionException e) {

// ensure stack trace is for this thread

throw new IOException(e.getCause());

}

 

Weak Keys

● What if the cache keys are transient objects (e.g.requests), which don't belong in the cache if there areno other references elsewhere?

LoadingCache<Request, Metadata> cache =CacheBuilder.newBuilder().weakKeys().build(loader);

● Allow the garbage collector to immediately collect cache keys when other references are gone

● Causes key equality to be determined using ==

● Cost: 3 new references, adding 16 bytes per entry

 

Eviction

● So far the caches we've shown you will grow without bound

● CacheBuilder can automatically evict elements based on various criteria

 

Eviction: Maximum Size

LoadingCache<String, String> cache = CacheBuilder.newBuilder().maximumSize(200).build(loader);

● Elements will be evicted in approximate LRU order

● Costs:

○ Every access now becomes a lightweight write (to record access order)

○ Evictions occur on write operations

○ 2 new references, in a doubly-linked access queue,adding 16 bytes per entry

 

Eviction: Maximum Weight

Weigher<String, String> weighByLength = new Weigher<String, String>() {public int weigh(String key, String value) {return value.length();

}

};

LoadingCache<String, String> cache = CacheBuilder.newBuilder().maximumWeight(2000).weigher(weighByLength).build(loader);

 

● Eviction order is the same as maximumSize

○ In fact they share the same data structure (and cost)

○ However more than one entry may be evicted at a time (making room for a single large entry)

● Weight is only measured once, when an entry is added to the cache

● Weight is only used to determine whether the cache is over capacity; not for selecting what to evict

Eviction: Maximum Weight

 

Cache Stats

● With an automatic eviction policy in play, one starts to wonder about cache performance

○ What ratio of requests are served directly from cache?

○ How much time is spent loading entries?

● These and other questions can be answered with:

LoadingCache<String, String> cache = CacheBuilder.newBuilder().recordStats()

.build(loader);

// cumulative stats since cache creation

CacheStats stats = cache.stats();

 

Cache Stats

CacheStats stats = cache.stats();

stats.hitRate();

stats.missRate();

stats.loadExceptionRate();

stats.averageLoadPenalty();

CacheStats delta = cache.stats()

.minus(stats);

delta.hitCount();

delta.missCount();

delta.loadSuccessCount();

delta.loadExceptionCount();

delta.totalLoadTime();

 

Eviction: Time to Idle

LoadingCache<String, String> cache =CacheBuilder.newBuilder().expireAfterAccess(2, TimeUnit.MINUTES).build(loader);

● Elements will expire after the specified time has elapsed since the most recent access

● Eviction order is the same as maximumSize

○ They share the same data structure (and cost)

○ However cache size will be dynamic instead of static

○ Evictions performed on read or write operations

● Cost: 2 new references, in a doubly-linked write queue,adding 16 bytes per entry

● Tests can advance time with CacheBuilder.ticker

 

Eviction: Time to Live

LoadingCache<String, String> cache =CacheBuilder.newBuilder().expireAfterWrite(2, TimeUnit.MINUTES).build(loader);

● Elements will expire after the specified time has elapsed since the entry's creation or update

● Useful for dropping stale data from the cache

○ Unlike other expiration strategies this is more about data correctness than resource conservation

● Cost: 2 new references, in a doubly-linked write queue,adding 16 bytes per entry