Redis: In-Memory Data Store for Speed
A deep technical guide to Redis as a high-performance data layer. Covers data structures, Pub/Sub messaging, caching strategies with TTL and eviction, session management, rate limiting, job queues with BullMQ, Lua scripting, pipelining, transactions, Redis Stack, memory optimization, and high availability with Cluster and Sentinel.
Table of Contents
- 1. Data Structures
- 2. Pub/Sub Messaging
- 3. Caching Strategies
- 4. Session Management
- 5. Rate Limiting
- 6. Redis Queues: Bull and BullMQ
- 7. Cluster Mode and Sentinel
- 8. Lua Scripting
- 9. Pipelining and Transactions
- 10. Redis Stack
- 11. Memory Optimization
- 12. Real-World Experience
- 13. Redis 8: Unified Distribution (2025-2026)
1. Data Structures
Redis is not just a key-value store. It provides specialized data structures, each optimized for specific access patterns. Choosing the right structure is the most important Redis design decision.
- Strings: Simplest type. Holds text, numbers, or binary data up to 512MB. Supports atomic increment/decrement (INCR/DECR). Use for counters, simple caches, feature flags
- Hashes: Field-value maps within a single key. Like a mini-object. O(1) per field access. Use for user profiles, session data, configuration. More memory-efficient than separate string keys
- Lists: Ordered sequences. O(1) push/pop at head/tail. Use for queues (LPUSH + RPOP), activity feeds, recent items. LRANGE for pagination. Max 4 billion elements
- Sets: Unordered collections of unique strings. O(1) membership check (SISMEMBER). Set operations: SUNION, SINTER, SDIFF. Use for tags, unique visitors, permissions
- Sorted Sets (ZSETs): Sets with a score for each member. Ordered by score. O(log N) add/remove. ZRANGEBYSCORE for range queries. Use for leaderboards, priority queues, time-series
- Streams: Append-only log with consumer groups. Like Kafka topics but simpler. XADD to write, XREADGROUP to consume. Use for event sourcing, activity logs, inter-service messaging
- Bitmaps: Bit-level operations on strings. SETBIT/GETBIT for individual bits, BITCOUNT for population count, BITOP for AND/OR/XOR across keys. Extremely memory-efficient for boolean state per user/entity (1 million users = 125KB). Use for daily active users, feature flags per user, bloom filter implementations
// ioredis examples for each data structure
import Redis from 'ioredis';
const redis = new Redis(process.env.REDIS_URL);
// Strings - counter
await redis.incr('bookings:today'); // Atomic increment
await redis.set('feature:dark-mode', 'true', 'EX', 86400); // With 24h TTL
// Hashes - user session
await redis.hmset('session:abc123', {
userId: '42', name: 'Jose', role: 'admin', loginAt: Date.now().toString()
});
const role = await redis.hget('session:abc123', 'role'); // O(1) field access
// Lists - recent activity
await redis.lpush('activity:user:42', JSON.stringify({ action: 'booking', id: 789 }));
await redis.ltrim('activity:user:42', 0, 99); // Keep only last 100
const recent = await redis.lrange('activity:user:42', 0, 9); // Get last 10
// Sets - unique daily visitors
await redis.sadd('visitors:2026-03-17', 'user:42', 'user:78');
const count = await redis.scard('visitors:2026-03-17'); // Unique count
// Sorted Sets - leaderboard
await redis.zadd('leaderboard:march', 150, 'user:42', 280, 'user:78');
const top10 = await redis.zrevrange('leaderboard:march', 0, 9, 'WITHSCORES');
// Streams - event log
await redis.xadd('events:bookings', '*', 'type', 'created', 'bookingId', '789');
// Bitmaps - daily active users (user ID as bit offset)
await redis.setbit('dau:2026-03-17', 42, 1); // Mark user 42 as active
await redis.setbit('dau:2026-03-17', 78, 1); // Mark user 78 as active
const isActive = await redis.getbit('dau:2026-03-17', 42); // Check if active
const totalActive = await redis.bitcount('dau:2026-03-17'); // Count active users
2. Pub/Sub Messaging
Redis Pub/Sub enables fire-and-forget messaging between services. Publishers send messages to channels; all subscribers on that channel receive the message in real time. Messages are not persisted -- if no subscriber is listening, the message is lost.
- Channels: Named message streams. SUBSCRIBE/PUBLISH. Pattern subscriptions with PSUBSCRIBE (e.g.,
events:*matches all event channels) - No persistence: Messages are delivered to connected subscribers only. If a subscriber disconnects and reconnects, it misses messages sent during downtime
- No acknowledgment: Publisher does not know if any subscriber received the message. Use Redis Streams if you need guaranteed delivery
- Dedicated connections: A subscribing connection enters "subscribe mode" and cannot execute other commands. Use separate connections for pub/sub and regular commands
- Use cases: Real-time notifications, cache invalidation across instances, WebSocket broadcasting, inter-service event notifications
// Pub/Sub with ioredis - cache invalidation pattern
import Redis from 'ioredis';
// Publisher (on write operations)
const pub = new Redis(process.env.REDIS_URL);
async function updateUser(userId, data) {
await db.updateUser(userId, data);
await pub.publish('cache:invalidate', JSON.stringify({
type: 'user', id: userId
}));
}
// Subscriber (on each API server instance)
const sub = new Redis(process.env.REDIS_URL); // Separate connection!
sub.subscribe('cache:invalidate');
sub.on('message', (channel, message) => {
const { type, id } = JSON.parse(message);
localCache.delete(`${type}:${id}`); // Clear local in-memory cache
});
3. Caching Strategies
Cache Patterns
- Cache-aside (lazy loading): Application checks cache first. On miss, queries database, stores result in cache with TTL. Most common pattern. Simple but can cause cache stampede on popular keys
- Write-through: Application writes to cache and database simultaneously. Cache is always up-to-date. Higher write latency but eliminates stale reads
- Write-behind (write-back): Application writes to cache only. A background process syncs to database asynchronously. Lowest write latency but risk of data loss if Redis crashes before sync
- Read-through: Cache itself is responsible for loading from database on miss. Application always reads from cache. Requires cache-aware data layer
// Cache-aside pattern with stampede protection
async function getUserById(userId: string): Promise<User> {
const cacheKey = `user:${userId}`;
// 1. Check cache
const cached = await redis.get(cacheKey);
if (cached) return JSON.parse(cached);
// 2. Acquire lock to prevent cache stampede
const lockKey = `lock:${cacheKey}`;
const acquired = await redis.set(lockKey, '1', 'EX', 5, 'NX'); // 5s lock
if (!acquired) {
// Another instance is loading this data -- wait and retry
await new Promise(r => setTimeout(r, 100));
return getUserById(userId);
}
try {
// 3. Load from database
const user = await db.findUserById(userId);
// 4. Store in cache with TTL
await redis.set(cacheKey, JSON.stringify(user), 'EX', 3600); // 1 hour TTL
return user;
} finally {
await redis.del(lockKey); // Release lock
}
}
TTL and Eviction Policies
- TTL (Time to Live): Set with EX (seconds) or PX (milliseconds) on SET, or with EXPIRE command. Always set TTL on cache keys to prevent unbounded memory growth
- allkeys-lru: Evicts least recently used keys when memory is full. Best general-purpose policy for cache workloads
- volatile-lru: Only evicts keys with a TTL set. Keys without TTL are never evicted. Use when mixing persistent and cache data in one instance
- allkeys-lfu (Redis 4.0+): Evicts least frequently used keys. Better than LRU for workloads with a few very hot keys and many cold keys
- noeviction: Returns errors when memory is full. Use for data that must never be evicted (queues, sessions). Requires careful memory planning
- maxmemory: Set with
maxmemory 2gbin redis.conf. Always set this in production. Without it, Redis will use all available memory and the OS will OOM-kill it
4. Session Management
Redis is the standard backend for session storage in distributed applications. Storing sessions in Redis (instead of server memory) enables horizontal scaling -- any server can handle any request because session data is shared.
- Hash-based sessions: Store session fields in a Redis hash. HMSET for batch writes, HGET for single field reads. More efficient than serializing the entire session on every request
- TTL for expiration: Set TTL equal to session timeout (e.g., 24 hours). EXPIRE resets on each request to implement sliding expiration
- Session ID security: Generate cryptographically random session IDs (128+ bits). Never use sequential or predictable IDs. Regenerate ID after authentication to prevent session fixation
- express-session + connect-redis: Standard setup for Express.js. Handles serialization, cookie management, and TTL automatically
- JWT vs Redis sessions: JWTs are stateless but cannot be revoked until expiry. Redis sessions can be instantly revoked (delete the key). Use Redis sessions when you need immediate revocation (logout, security incidents)
// Express.js session with Redis (connect-redis)
import session from 'express-session';
import RedisStore from 'connect-redis';
import Redis from 'ioredis';
const redisClient = new Redis(process.env.REDIS_URL);
app.use(session({
store: new RedisStore({
client: redisClient,
prefix: 'sess:',
ttl: 86400 // 24 hours in seconds
}),
secret: process.env.SESSION_SECRET,
resave: false, // Don't save session if unmodified
saveUninitialized: false, // Don't create session until something stored
cookie: {
secure: true, // HTTPS only
httpOnly: true, // Not accessible via JavaScript
maxAge: 86400000, // 24 hours in milliseconds
sameSite: 'lax'
}
}));
// Force logout (immediate session revocation)
async function forceLogout(sessionId: string) {
await redisClient.del(`sess:${sessionId}`);
}
5. Rate Limiting
Redis is ideal for rate limiting because it provides atomic operations and sub-millisecond latency. Rate limiting protects APIs from abuse, prevents resource exhaustion, and ensures fair usage.
- Fixed window: Count requests per time window (e.g., 100 requests per minute). Simple but allows burst at window boundaries (up to 2x rate)
- Sliding window log: Store timestamp of each request in a sorted set. Count entries within the window. Precise but higher memory usage for high-volume endpoints
- Sliding window counter: Combine current and previous window counts weighted by time elapsed. Good balance of precision and memory efficiency
- Token bucket: Bucket holds tokens, each request consumes one. Tokens replenish at a fixed rate. Allows controlled bursts. Best for API rate limiting
// Sliding window rate limiter with Redis
async function checkRateLimit(userId: string, limit: number, windowSec: number): Promise<boolean> {
const key = `ratelimit:${userId}`;
const now = Date.now();
const windowStart = now - (windowSec * 1000);
const member = `${now}:${Math.random()}`; // Unique member ID
// Atomic pipeline: remove old entries, add new, count, set TTL
const pipeline = redis.pipeline();
pipeline.zremrangebyscore(key, 0, windowStart); // Remove expired entries
pipeline.zadd(key, now, member); // Add current request
pipeline.zcard(key); // Count entries in window
pipeline.expire(key, windowSec); // Set TTL for cleanup
const results = await pipeline.exec();
const count = results[2][1] as number;
if (count > limit) {
await redis.zrem(key, member); // Remove the entry we just added
return false; // Rate limited
}
return true; // Allowed
}
// Usage in Express middleware
app.use(async (req, res, next) => {
const allowed = await checkRateLimit(req.ip, 100, 60); // 100 req/min
if (!allowed) {
res.setHeader('Retry-After', '60');
return res.status(429).json({ error: 'Too many requests' });
}
next();
});
6. Redis Queues: Bull and BullMQ
BullMQ is the standard Redis-based job queue for Node.js. It provides reliable job processing with retries, delays, priorities, rate limiting, and concurrency control. Built on top of Redis Streams and Sorted Sets.
- Job lifecycle: waiting -> active -> completed/failed. Failed jobs can be retried with exponential backoff. Completed jobs can be cleaned up after N days
- Delayed jobs: Schedule jobs to run at a specific time or after a delay. Use for sending emails 30 minutes after signup, scheduled reports, retry delays
- Priority queues: Assign priority to jobs (lower number = higher priority). Payment processing jobs before email notifications
- Rate limiting: Limit how many jobs are processed per time window. Prevent overwhelming external APIs (Stripe, email providers)
- Concurrency: Control how many jobs run simultaneously per worker. Set based on CPU/memory requirements of your job type
- Events: Listen for job completion, failure, progress updates. Use for real-time UI updates via WebSocket
- Repeatable jobs: Cron-style scheduling. Run a job every hour, every day at 3 AM, etc. Uses Redis sorted sets for scheduling
// BullMQ producer and worker
import { Queue, Worker } from 'bullmq';
import Redis from 'ioredis';
const connection = new Redis(process.env.REDIS_URL, { maxRetriesPerRequest: null });
// Producer: add jobs to queue
const emailQueue = new Queue('email', { connection });
await emailQueue.add('booking-confirmation', {
userId: 42,
bookingId: 789,
templateId: 'booking_confirmed'
}, {
attempts: 3, // Retry up to 3 times
backoff: { type: 'exponential', delay: 5000 }, // 5s, 10s, 20s
removeOnComplete: { age: 86400 }, // Clean up after 24h
removeOnFail: { age: 604800 }, // Keep failed jobs for 7 days
priority: 2 // Lower = higher priority
});
// Repeatable job (daily report at 3 AM)
await emailQueue.add('daily-report', { type: 'daily' }, {
repeat: { pattern: '0 3 * * *' } // Cron syntax
});
// Worker: process jobs
const worker = new Worker('email', async (job) => {
const { userId, bookingId, templateId } = job.data;
await job.updateProgress(10);
const user = await db.findUser(userId);
const booking = await db.findBooking(bookingId);
await job.updateProgress(50);
await ses.sendTemplatedEmail(user.email, templateId, { user, booking });
await job.updateProgress(100);
return { sent: true, email: user.email };
}, {
connection,
concurrency: 5, // Process 5 jobs simultaneously
limiter: { max: 10, duration: 1000 } // Max 10 jobs per second
});
worker.on('completed', (job, result) => {
console.log(`Job ${job.id} completed: ${result.email}`);
});
worker.on('failed', (job, err) => {
console.error(`Job ${job.id} failed: ${err.message}`);
});
7. Cluster Mode and Sentinel
Redis Sentinel
Sentinel provides high availability for Redis. It monitors master and replica nodes, performs automatic failover when the master is down, and serves as a service discovery mechanism.
- Monitors master health with configurable thresholds (down-after-milliseconds)
- Automatic failover: promotes a replica to master when the current master fails. Requires quorum (majority of sentinels agree)
- Client configuration: Clients connect to Sentinel to discover the current master. ioredis supports Sentinel natively
- Minimum 3 Sentinel instances for quorum. Place them in different availability zones
- Best for: Single-master setups where data fits in one server's memory
Redis Cluster
Redis Cluster distributes data across multiple master nodes using hash slots (16384 slots). Each master handles a portion of the keyspace. It provides horizontal scaling for data that exceeds single-node memory.
- Data sharding: 16384 hash slots distributed across masters. Key's slot = CRC16(key) mod 16384
- Hash tags: Use {prefix} in keys to force related keys to the same slot. Required for multi-key operations (MGET, pipelines, Lua scripts)
- Minimum 3 masters + 3 replicas (6 nodes) for production. Each master has at least one replica for failover
- ASK/MOVED redirections: Client must handle slot migration during resharding. ioredis handles this automatically
- Best for: Large datasets that exceed single-node memory, or when you need write scaling across multiple masters
ioredis Configuration
// ioredis with Sentinel
const redis = new Redis({
sentinels: [
{ host: 'sentinel-1', port: 26379 },
{ host: 'sentinel-2', port: 26379 },
{ host: 'sentinel-3', port: 26379 }
],
name: 'mymaster', // Sentinel master group name
password: process.env.REDIS_PASSWORD,
sentinelPassword: process.env.SENTINEL_PASSWORD,
enableReadyCheck: true,
maxRetriesPerRequest: 3,
retryStrategy(times) {
return Math.min(times * 200, 5000); // Cap retry delay at 5s
}
});
// ioredis with Cluster
const cluster = new Redis.Cluster([
{ host: 'redis-1', port: 6379 },
{ host: 'redis-2', port: 6379 },
{ host: 'redis-3', port: 6379 }
], {
redisOptions: { password: process.env.REDIS_PASSWORD },
scaleReads: 'slave', // Read from replicas for read-heavy workloads
enableReadyCheck: true,
maxRedirections: 16,
retryDelayOnFailover: 300
});
8. Lua Scripting
Redis embeds a Lua interpreter, allowing you to execute scripts atomically on the server. Lua scripts run as a single atomic operation -- no other command can execute between script steps. This eliminates race conditions without distributed locks.
- EVAL / EVALSHA: EVAL sends the script each time. EVALSHA sends only the SHA1 hash after first load with SCRIPT LOAD. Use EVALSHA in production to reduce network bandwidth
- KEYS and ARGV: Pass key names as KEYS[] and arguments as ARGV[]. Redis uses KEYS to determine which cluster node runs the script. Never hardcode key names inside the script
- Atomicity: The entire script executes atomically. No other client can run commands while a Lua script is running. Keep scripts short to avoid blocking other clients
- Use cases: Rate limiting with complex logic, compare-and-swap operations, multi-step transactions that need atomicity, conditional updates, distributed locks (Redlock)
- Limitations: No external I/O (no HTTP calls, no filesystem). Scripts must be deterministic for replication. Maximum execution time controlled by
lua-time-limit(default 5 seconds)
// Lua script via ioredis - atomic rate limiter
const rateLimitScript = `
local key = KEYS[1]
local limit = tonumber(ARGV[1])
local window = tonumber(ARGV[2])
local now = tonumber(ARGV[3])
-- Remove expired entries
redis.call('ZREMRANGEBYSCORE', key, 0, now - window * 1000)
-- Count current entries
local count = redis.call('ZCARD', key)
if count < limit then
-- Add new entry and set TTL
redis.call('ZADD', key, now, now .. ':' .. math.random())
redis.call('EXPIRE', key, window)
return 1 -- Allowed
end
return 0 -- Denied
`;
// Load script once, use SHA for subsequent calls
const sha = await redis.script('LOAD', rateLimitScript);
// Execute with EVALSHA (atomic, no race conditions)
const allowed = await redis.evalsha(sha, 1, `ratelimit:${userId}`, 100, 60, Date.now());
// ioredis defineCommand for reusable Lua commands
redis.defineCommand('rateLimit', {
numberOfKeys: 1,
lua: rateLimitScript
});
const result = await redis.rateLimit(`ratelimit:${userId}`, 100, 60, Date.now());
9. Pipelining and Transactions
Pipelining
Pipelining sends multiple commands to Redis without waiting for individual responses. Redis processes all commands and sends all responses back at once. This eliminates per-command round-trip latency.
- Performance: Sending 1000 commands individually takes 1000 round trips. With pipelining, it takes 1 round trip. 10-100x throughput improvement for bulk operations
- No atomicity: Pipelined commands are not atomic. Other clients can interleave commands between them. Use MULTI/EXEC if you need atomicity
- Memory: Redis buffers all responses in memory. Pipeline thousands, not millions, of commands at once to avoid memory issues
Transactions (MULTI/EXEC)
MULTI/EXEC provides atomic execution of a group of commands. All commands between MULTI and EXEC are queued and executed as a single atomic operation.
- MULTI: Starts a transaction. Subsequent commands are queued, not executed immediately
- EXEC: Executes all queued commands atomically. Returns array of results
- WATCH: Optimistic locking. WATCH a key before MULTI. If the key changes before EXEC, the transaction is aborted (returns null). Retry the operation
- DISCARD: Cancels a transaction and flushes the command queue
// Pipelining with ioredis - bulk operations
const pipeline = redis.pipeline();
for (let i = 0; i < 1000; i++) {
pipeline.set(`key:${i}`, `value:${i}`, 'EX', 3600);
}
const results = await pipeline.exec(); // Single round trip for 1000 commands
// MULTI/EXEC transaction - atomic transfer
async function transfer(from: string, to: string, amount: number) {
const multi = redis.multi();
multi.decrby(`balance:${from}`, amount);
multi.incrby(`balance:${to}`, amount);
const results = await multi.exec(); // Atomic: both succeed or neither does
if (!results) throw new Error('Transaction failed');
return results;
}
// WATCH + MULTI for optimistic locking
async function incrementIfBelow(key: string, max: number): Promise<boolean> {
while (true) {
await redis.watch(key);
const current = Number(await redis.get(key) || 0);
if (current >= max) {
await redis.unwatch();
return false;
}
const result = await redis.multi()
.set(key, current + 1)
.exec();
if (result) return true; // Success
// result is null = key changed, retry
}
}
10. Redis Stack
Redis 8.0 (GA April 2025) merged all Redis Stack modules into core Redis. RediSearch, RedisJSON, RedisTimeSeries, and RedisBloom are now built-in -- no separate module installation needed. This consolidation delivers significantly lower command latency compared to the module-based architecture. The modules below are now native data capabilities:
- RediSearch: Full-text search engine built into Redis. Create secondary indexes on hash or JSON fields. Supports fuzzy matching, stemming, phonetic matching, auto-complete, and aggregation. Eliminates the need for a separate search engine (Elasticsearch) for many use cases
- RedisJSON: Native JSON document storage with JSONPath queries. Store, update, and retrieve nested JSON without serialization overhead. Combine with RediSearch to create searchable JSON document stores
- RedisGraph (End of Life): RedisGraph reached End of Life in February 2025. For graph database needs, consider FalkorDB (community fork) which continues active development with full Cypher support
- RedisTimeSeries: Purpose-built time-series data structure with automatic downsampling, aggregation, and retention policies. Use for metrics, IoT sensor data, stock prices
- RedisBloom: Probabilistic data structures -- Bloom filters, Cuckoo filters, Count-Min Sketch, Top-K. Memory-efficient approximate membership testing and counting
// RediSearch - full-text search on user profiles
// Create index (run once)
await redis.call('FT.CREATE', 'idx:users', 'ON', 'HASH', 'PREFIX', '1', 'user:',
'SCHEMA', 'name', 'TEXT', 'SORTABLE', 'email', 'TEXT', 'city', 'TAG', 'age', 'NUMERIC');
// Add data (just regular HSET)
await redis.hset('user:42', { name: 'Jose Nobile', email: 'jose@example.com', city: 'Medellin', age: '28' });
// Search with full-text and filters
const results = await redis.call('FT.SEARCH', 'idx:users', '@name:Jose @city:{Medellin}', 'LIMIT', '0', '10');
// RedisJSON - nested document operations
await redis.call('JSON.SET', 'product:100', '$', JSON.stringify({
name: 'Premium Plan', price: 49.99, features: ['unlimited', 'priority', 'api']
}));
// Update nested field without reading the whole document
await redis.call('JSON.NUMINCRBY', 'product:100', '$.price', 10);
const features = await redis.call('JSON.GET', 'product:100', '$.features');
10b. Vector Sets
Redis 8.0 introduces Vector Sets, a new data type for high-dimensional vector similarity search. Critical for AI/ML workloads including semantic search, recommendation systems, and Retrieval-Augmented Generation (RAG). Vector Sets store embedding vectors and support approximate nearest-neighbor (ANN) queries with sub-millisecond latency.
- VSADD / VSGET / VSDEL: Add, retrieve, and delete vectors in a set. Each vector is associated with a string element name and optional attributes for filtered search
- VSSEARCH: Find the K nearest neighbors of a query vector. Supports cosine, L2 (Euclidean), and inner product distance metrics. Returns results ranked by similarity score
- Use cases: Semantic search over document embeddings, product recommendations from user behavior vectors, RAG pipelines pairing LLMs with vector retrieval, image similarity, and anomaly detection
10c. Hash Field Expiration
Redis 7.4+ introduced per-field TTL within hashes, and Redis 8.0 added new commands for atomic get-set-expire operations. Previously, TTL could only be set on entire keys. Now individual hash fields can expire independently, enabling fine-grained cache invalidation within a single hash.
- HEXPIRE / HPEXPIRE: Set TTL on individual hash fields (seconds or milliseconds). Fields are automatically deleted when they expire
- HGETEX (Redis 8.0): Get one or more fields and set/update their expiration atomically. Eliminates the race condition between HGET and HEXPIRE
- HSETEX (Redis 8.0): Set one or more fields with expiration in a single atomic operation. Replaces the HSET + HEXPIRE two-step pattern
- HGETDEL (Redis 8.0): Get and delete fields atomically. Useful for one-time-use tokens, job claim patterns, and queue-like access within hashes
10d. Redis Licensing
Redis licensing has changed significantly. Redis 7.4 (March 2024) moved from BSD to dual SSPL/RSALv2 licensing, restricting cloud providers from offering Redis as a managed service. Redis 8.0 (April 2025) added AGPLv3 as a third option, making Redis triple-licensed (SSPL, RSALv2, or AGPLv3). The AGPLv3 option re-opens Redis to the open-source community while maintaining protections against cloud provider free-riding. Choose the license that best fits your deployment model.
11. Memory Optimization
Redis stores everything in memory, so optimizing memory usage directly reduces infrastructure costs. Small data structure choices compound into significant savings at scale.
- Use hashes for small objects: Redis internally uses ziplist encoding for small hashes (up to
hash-max-ziplist-entries 128andhash-max-ziplist-value 64bytes). This is 10x more memory-efficient than separate string keys - Short key names: Key names consume memory. Use
u:42:ninstead ofuser:42:namewhen you have millions of keys. Save the mapping in code constants - Integer encoding: Redis stores integers (up to 10000 by default) in a shared object pool. Store numeric IDs as integers, not strings
- OBJECT ENCODING: Use
OBJECT ENCODING keyto inspect how Redis stores a key. Ziplist, intset, and embstr are compact encodings. Hashtable and skiplist consume more memory - MEMORY USAGE: Use
MEMORY USAGE keyto see exact bytes consumed by a key including overhead. Profile your hot keys - Compression: Compress large values before storing (gzip, snappy). Trade CPU for memory. Especially effective for JSON payloads and serialized objects
- TTL everywhere: Always set TTL on cache keys. Periodically audit keys without TTL using
OBJECT IDLETIMEto find forgotten keys consuming memory
// Memory-efficient hash packing - store 100 users per hash bucket
// Instead of 100 separate keys, pack into hash buckets
function getUserBucket(userId: number) {
const bucket = Math.floor(userId / 100); // Group by 100s
const field = (userId % 100).toString();
return { key: `users:${bucket}`, field };
}
async function setUser(userId: number, data: object) {
const { key, field } = getUserBucket(userId);
await redis.hset(key, field, JSON.stringify(data));
}
async function getUser(userId: number) {
const { key, field } = getUserBucket(userId);
const raw = await redis.hget(key, field);
return raw ? JSON.parse(raw) : null;
}
// Monitor memory usage
const info = await redis.info('memory');
// used_memory, used_memory_peak, mem_fragmentation_ratio
const memUsage = await redis.call('MEMORY', 'USAGE', 'user:42'); // Bytes for one key
mem_fragmentation_ratio in INFO MEMORY. A ratio above 1.5 indicates significant memory fragmentation. Restart Redis or enable active defrag (activedefrag yes) to reclaim wasted memory.12. Real-World Experience
In production, Redis was a critical infrastructure component powering caching, sessions, and job queues across all microservices.
- Session and cache: Redis as the primary session store for all API servers. Cache-aside pattern for user profiles, gym class schedules, and pricing data. TTLs from 5 minutes (schedule data) to 24 hours (static configuration)
- ioredis client: Used ioredis across all Node.js services with Sentinel configuration for automatic failover. Connection pooling with lazyConnect for efficient resource usage during cold starts
- Reservation queue: Built a booking reservation system using Redis sorted sets combined with ZeroMQ for inter-service communication. Sorted sets held pending reservations with expiration timestamps as scores, enabling automatic cleanup of expired holds
- BullMQ job queues: Email notifications, payment processing, report generation, and scheduled tasks all running through BullMQ. Priority queues ensured payment webhooks were processed before marketing emails
- Cache invalidation: Pub/Sub for real-time cache invalidation across API server instances when data was updated. Combined with TTL as a safety net to prevent stale data even if Pub/Sub message is missed
13. Redis 8: Unified Distribution (2025-2026)
Redis 8 GA: Unified Open Source
Redis 8 merges Redis Stack and Redis Community Edition into a single unified distribution called Redis Open Source. All previously modular functionality — JSON, Time Series, probabilistic data types (Bloom filter, cuckoo filter, count-min sketch, top-k, t-digest), and Redis Query Engine — is now built into the core package. Standalone RediSearch, RedisJSON, RedisTimeSeries, and RedisBloom modules are no longer needed. This eliminates module management complexity and simplifies deployment.
Vector Sets and Performance
Redis 8 introduces vector sets (beta), the first new core data type in years, enabling native vector similarity search for AI/ML workloads like RAG pipelines and recommendation engines. Performance improvements deliver over 30 optimizations with up to 87% faster commands and 2x throughput on key operations.
Redis 8.6 GA
Redis 8.6 GA introduces XADD idempotency, allowing clients to retry XADD commands with a deduplication key to prevent duplicate stream entries — critical for exactly-once event sourcing. The new volatile-lrm eviction policy (Least Recently Modified) evicts keys with TTL based on last modification time rather than last access, better suited for write-heavy caching patterns. TLS certificate-based authentication enables mutual TLS without password-based AUTH, strengthening zero-trust network configurations.
Redis 8.8 Preview
Redis 8.8 milestone builds introduce GCRA (Generic Cell Rate Algorithm) rate limiting as a native command, eliminating the need for Lua scripts or Redis modules for token-bucket rate limiting. The new XNACK command provides explicit negative acknowledgement for stream consumer groups, allowing consumers to signal processing failure and trigger automatic redelivery without waiting for the claim timeout.
Licensing: AGPLv3 Option
Redis Open Source (starting with Redis 8) now offers three license options: the Redis Source Available License v2 (RSALv2), Server Side Public License v1 (SSPLv1), and the GNU Affero General Public License v3 (AGPLv3). The addition of AGPLv3 makes Redis fully open source again under a widely recognized OSI-approved license, addressing community concerns from the 2024 license change.