Loading low-level design…

Design Thread-Safe Cache with TTL - Low-Level Design | OneNoughtOne

00:00:00

Design Thread-Safe Cache with TTL

Intermediate

Design a thread-safe, in-memory cache with per-entry TTL (Time To Live) and pluggable eviction policies. The cache supports get, put, delete, contains, size, clear, and keys operations, all protected by reentrant locks for thread safety.

Each entry has an optional TTL; expired entries are lazily removed on access and periodically swept by a background cleanup daemon. When the cache reaches its fixed capacity, new inserts trigger eviction using a pluggable Strategy pattern with three implementations: LRU (OrderedDict / LinkedHashMap for O(1) operations), LFU (frequency-based with LRU tiebreak), and FIFO. The cache also provides a get-or-load (cache-aside) method that atomically loads and caches missing entries, TTL query and update operations, cache statistics (hits, misses, evictions, hit rate), and an Observer-based event listener system.

Core Use Cases

Put / Get / Delete

Store, retrieve, and remove key-value pairs; get returns null for missing or expired keys

TTL-based expiry

Each entry has an optional TTL; expired entries are lazily evicted on access and periodically via background cleanup

Eviction policies (Strategy pattern)

Pluggable eviction: LRU (Least Recently Used), LFU (Least Frequently Used with LRU tiebreak), FIFO (First In First Out)

Thread safety

All operations are protected by locks (RLock / ReentrantReadWriteLock / recursive_mutex) for concurrent access

Cache statistics

Track hits, misses, evictions, expirations, total puts, and compute hit rate

Get-or-Load (cache-aside)

Atomic get-or-load: if key is missing, call a loader function, cache the result, and return it

TTL management

Query remaining TTL for a key; update TTL for an existing key

Background cleanup daemon

Periodic background thread sweeps and removes expired entries

Event listeners (Observer)

Constraints

Cache has a fixed maximum capacity; new inserts trigger eviction when full
Expired entries are removed lazily on access AND periodically by a background sweep
LRU uses OrderedDict / LinkedHashMap for O(1) access and eviction
LFU tracks frequency per key; ties broken by least recent access time
Thread safety must use reentrant locks to support get-or-load patterns
Eviction policy is swappable at runtime via Strategy pattern

Assumptions

In-memory only (no persistence or distributed cache)
Keys are strings; values are generic
No maximum memory limit (capacity is by entry count, not bytes)
Single JVM / process (not distributed)
Background cleanup interval is configurable

In Scope

Core cache operations: get, put, delete, contains, size, clear, keys
Per-entry TTL with lazy and periodic expiration
Pluggable eviction policies: LRU, LFU, FIFO
Thread-safe operations with reentrant locking
Cache statistics (hits, misses, evictions, expirations, hit rate)
Get-or-load (cache-aside) with loader function
TTL query and update (getRemainingTTL, setTTL)
Background cleanup daemon thread
Event listener/observer for cache events

Out of Scope

Distributed caching (Redis-like clustering)
Persistence to disk
Memory-based eviction (by byte size)
Cache warming and preloading
Write-through / write-behind strategies
Serialization / deserialization
Network protocol (TCP/HTTP cache server)

Approach Guide(Click to expand each section)

Before diving into code, clarify the use cases and edge cases. Understanding the problem deeply leads to better class design.

List Core Use Cases

Identify the primary actions users will perform. For a parking lot: park vehicle, exit vehicle, check availability. Each becomes a method.

Identify Actors

Who interacts with the system? Customers, admins, automated systems? Each actor type may need different interfaces.

Clarify Constraints

What are the limits? Max vehicles, supported vehicle types, payment methods. Constraints drive your data structures.

Ask About Edge Cases

What happens on overflow? Concurrent access? Payment failures? Thinking about edge cases reveals hidden complexity.

Follow-up Questions(Questions an interviewer might ask)

💡 Draw UML class diagrams, sequence diagrams, or any design visualizations. Submit from the Code tab.

Loading canvas...

Loading LLD design...

00:00:00

Design Thread-Safe Cache with TTL

Intermediate

Core Use Cases

Put / Get / Delete

Store, retrieve, and remove key-value pairs; get returns null for missing or expired keys

TTL-based expiry

Each entry has an optional TTL; expired entries are lazily evicted on access and periodically via background cleanup

Eviction policies (Strategy pattern)

Pluggable eviction: LRU (Least Recently Used), LFU (Least Frequently Used with LRU tiebreak), FIFO (First In First Out)

Thread safety

All operations are protected by locks (RLock / ReentrantReadWriteLock / recursive_mutex) for concurrent access

Cache statistics

Track hits, misses, evictions, expirations, total puts, and compute hit rate

Get-or-Load (cache-aside)

Atomic get-or-load: if key is missing, call a loader function, cache the result, and return it

TTL management

Query remaining TTL for a key; update TTL for an existing key

Background cleanup daemon

Periodic background thread sweeps and removes expired entries

Event listeners (Observer)

Constraints

Cache has a fixed maximum capacity; new inserts trigger eviction when full
Expired entries are removed lazily on access AND periodically by a background sweep
LRU uses OrderedDict / LinkedHashMap for O(1) access and eviction
LFU tracks frequency per key; ties broken by least recent access time
Thread safety must use reentrant locks to support get-or-load patterns
Eviction policy is swappable at runtime via Strategy pattern

Assumptions

In-memory only (no persistence or distributed cache)
Keys are strings; values are generic
No maximum memory limit (capacity is by entry count, not bytes)
Single JVM / process (not distributed)
Background cleanup interval is configurable

In Scope

Core cache operations: get, put, delete, contains, size, clear, keys
Per-entry TTL with lazy and periodic expiration
Pluggable eviction policies: LRU, LFU, FIFO
Thread-safe operations with reentrant locking
Cache statistics (hits, misses, evictions, expirations, hit rate)
Get-or-load (cache-aside) with loader function
TTL query and update (getRemainingTTL, setTTL)
Background cleanup daemon thread
Event listener/observer for cache events

Out of Scope

Distributed caching (Redis-like clustering)
Persistence to disk
Memory-based eviction (by byte size)
Cache warming and preloading
Write-through / write-behind strategies
Serialization / deserialization
Network protocol (TCP/HTTP cache server)

Approach Guide(Click to expand each section)

Before diving into code, clarify the use cases and edge cases. Understanding the problem deeply leads to better class design.

List Core Use Cases

Identify the primary actions users will perform. For a parking lot: park vehicle, exit vehicle, check availability. Each becomes a method.

Identify Actors

Who interacts with the system? Customers, admins, automated systems? Each actor type may need different interfaces.

Clarify Constraints

What are the limits? Max vehicles, supported vehicle types, payment methods. Constraints drive your data structures.

Ask About Edge Cases

What happens on overflow? Concurrent access? Payment failures? Thinking about edge cases reveals hidden complexity.

Follow-up Questions(Questions an interviewer might ask)

💡 Draw UML class diagrams, sequence diagrams, or any design visualizations. Submit from the Code tab.

Loading canvas...

Design Thread-Safe Cache with TTL

Core Use Cases

Constraints

Assumptions

In Scope

Out of Scope

Approach Guide(Click to expand each section)

Follow-up Questions(Questions an interviewer might ask)

Explore the full editorial.

Sign in to view submissions

Design Thread-Safe Cache with TTL

Core Use Cases

Constraints

Assumptions

In Scope

Out of Scope

Approach Guide(Click to expand each section)

Follow-up Questions(Questions an interviewer might ask)

Explore the full editorial.

Sign in to view submissions

Design Thread-Safe Cache with TTL

Core Use Cases

Constraints

Assumptions

In Scope

Out of Scope

Approach Guide(Click to expand each section)

Gather Requirements~3 min

Identify Classes & Entities~4 min

Define Relationships~3 min

Apply Design Patterns~4 min

Apply SOLID Principles~3 min

Code Organization~2 min

Follow-up Questions(Questions an interviewer might ask)

1How would you prevent cache stampede (thundering herd) on hot keys?

2How would you implement a distributed cache with consistency?

3How would you add memory-based eviction (max memory limit)?

4How would you implement a multi-level cache (L1 in-process + L2 remote)?

5How would you optimize for read-heavy workloads with minimal lock contention?

Key Topics

Asked At

Explore the full editorial.

Sign in to view submissions

Design Thread-Safe Cache with TTL

Core Use Cases

Constraints

Assumptions

In Scope

Out of Scope

Approach Guide(Click to expand each section)

Gather Requirements~3 min

Identify Classes & Entities~4 min

Define Relationships~3 min

Apply Design Patterns~4 min

Apply SOLID Principles~3 min

Code Organization~2 min

Follow-up Questions(Questions an interviewer might ask)

1How would you prevent cache stampede (thundering herd) on hot keys?

2How would you implement a distributed cache with consistency?

3How would you add memory-based eviction (max memory limit)?

4How would you implement a multi-level cache (L1 in-process + L2 remote)?

5How would you optimize for read-heavy workloads with minimal lock contention?

Key Topics

Asked At

Explore the full editorial.

Sign in to view submissions