$Redis Deep Dive Part 1: In-Memory Architecture and Event Loop Explained

For the past few weeks, I’ve been working with Redis while building a circuit breaker with some custom requirements. Answering those requirements led me into the Redis documentation and source code. I collected what I learned into a six-part series, and this is part one.

Redis (REmote DIctionary Server) keeps its working dataset in memory and runs core command execution through a single-threaded event loop. Those decisions explain much of its latency and concurrency behavior.

Everything in RAM

Redis stores its dataset in RAM. Disk-backed databases such as PostgreSQL and MySQL also use memory and page caches, but Redis serves working data without fetching it from persistent storage during each command.

The approximate access latencies show why that matters:

These ranges vary by hardware, but main-memory access is much faster than persistent-storage access. Keeping the working dataset in RAM removes storage-device latency from the command path.

Redis can also benefit from CPU cache locality when frequently accessed parts of its data structures remain close to the processor.

This access model suits data that is read or updated frequently. Uber has described an integrated cache serving high read volume, while GitHub has written about a sharded, replicated rate limiter built with Redis. The architectures and performance characteristics differ, so each example needs to be read in its original context.

The same design sets a capacity boundary because the dataset must fit within the memory available to the Redis deployment. RAM also costs more per gigabyte than persistent storage. Durability requires a persistence strategy, and its configuration determines how much recent data can be lost after a failure.

Event Loop

Redis uses a single-threaded event loop for core command execution. The model follows the Reactor Pattern: one thread watches many client connections and handles work when a socket becomes ready.

Imagine one waiter handling many tables. The waiter records a request and leaves slow kitchen work to someone else, remaining free to respond to other tables.

Redis applies a similar idea with I/O multiplexing, using mechanisms such as epoll on Linux or kqueue on BSD to monitor many client sockets at once.

The loop asks the kernel which client connections are ready. It processes an available command before checking for more ready work.

The core loop in Redis source file ae.c is compact:

void aeMain(aeEventLoop *eventLoop) {
    eventLoop->stop = 0;
    while (!eventLoop->stop) {
        aeProcessEvents(eventLoop, AE_ALL_EVENTS| AE_CALL_BEFORE_SLEEP| AE_CALL_AFTER_SLEEP);
    }
}

Redis abstracts these platform-specific APIs in its event library and selects an implementation for the host system at compile time.

File events come from client sockets. When a socket is ready, the loop triggers the relevant read or write handler. Time events cover scheduled work such as key expiration checks and other periodic maintenance.

Why single-threading works

One thread owns command execution, so data mutations do not need locks between command workers. Commands such as INCR and LPUSH execute atomically relative to other commands.

This also avoids coordination and context-switching costs in the command path. Depending on the workload, network or memory bandwidth may become the limiting resource before command execution uses the full CPU core.

The model has limits. One Redis instance cannot spread core command execution across multiple CPU cores, so scaling may require multiple instances. A long command such as KEYS * or a complex Lua script delays every client because Redis does not preempt a command after it starts.

What about Redis 6.0+ I/O Threads?

Starting with version 6.0, Redis introduced optional I/O threads. This feature does not make command execution multi-threaded. Commands still execute on the main thread, while I/O threads can handle socket reads and writes. That leaves more main-thread time for command execution.

For workloads that need multi-core use within one instance, there are alternatives. Some forks of Redis (like KeyDB) use multithreading.

RESP (REdis Serialization Protocol)

RESP is a text-based protocol with length-prefixed strings and arrays. Its grammar is small enough to parse without the complexity of a general document format.

The length prefix tells a parser how much payload to expect without scanning for a terminator. Network reads may still arrive in parts, but the parser knows when the complete value has been received.

Explicit lengths also make bulk strings binary-safe because a payload can contain arbitrary bytes. The small grammar keeps client parser implementations manageable across programming languages.

Redis 6 introduced RESP3, which expands the protocol with additional collection and numeric types as well as a formal Push message. RESP2 remains common in existing clients and deployments.

Request-response model

The standard communication pattern is simple: a client sends a command to the server as a RESP Array of Bulk Strings. The first element of the array is the command name, and subsequent elements are the arguments. The server then replies with a command-specific RESP type.

For example, the command SET mykey “Hello World” would be encoded by the client as:

*3\r\n$3\r\nSET\r\n$5\r\nmykey\r\n$11\r\nHello World\r\n

The server would reply with a Simple String:

+OK\r\n

Pipelining

Pipelining reduces the cost of network round trips. Instead of sending one command and waiting for its reply, a client can send a batch. Redis processes the commands and returns their replies together.

The batch pays one network round trip rather than one round trip per command.

$ printf "SET key1 1\r\nSET key2 2\r\nGET key1\r\nGET key2\r\n" | \
    redis-cli --pipe

Without Pipelining:

Client -> SET key1 val1 -> Server
Client <- OK <- Server (1 RTT)
Client -> SET key2 val2 -> Server
Client <- OK <- Server (1 RTT)

With Pipelining:

Client -> SET key1 val1, SET key2 val2 -> Server
Client <- OK, OK <- Server (1 RTT total)

Throughput depends on the workload and deployment. The Redis benchmark documentation describes how to measure a specific case.

Connection handling and pooling

Clients connect to Redis through TCP, which defaults to port 6379, or through a Unix domain socket when the processes are co-located. The event-driven I/O model allows one instance to maintain many simultaneous connections.

Establishing a TCP connection involves a three-way handshake. Applications that create independent connections frequently can use a pool instead of opening a new connection for every command.

A connection pool keeps connections open for reuse. The appropriate pool size and idle settings depend on the client library and workload.

A blocking command such as BLPOP occupies its connection while it waits. A long Lua script has a wider effect because command execution on the main thread cannot move to other clients until the script finishes. Clients that multiplex requests on one socket need a policy for commands that block the connection.

For workloads with high throughput or large values, network bandwidth may become the limiting factor before command execution. Measure bytes transferred as part of workload benchmarking rather than choosing network capacity from a general rule.

SSL/TLS performance

Redis 6.0 introduced native support for SSL/TLS encryption. TLS adds handshake latency when a connection is established and consumes CPU while encrypting traffic.

A KeyDB benchmark measured a 30-60% throughput reduction with TLS in its test configurations. Treat that result as one data point rather than a general Redis estimate. Benchmark the chosen deployment without weakening the required security boundary.

These mechanisms explain both Redis’s performance and its limits. Part 2 examines the internal data structures that sit behind Redis commands.

Further Reading

• Official Redis Documentation
• Redis Performance Optimization Guide
• Redis Cluster Tutorial
• Redis Persistence Demystified