ClickHouse vs Elasticsearch: When Each Makes Sense for Logs and Analytics

ClickHouse and Elasticsearch both show up in observability stacks, log pipelines, and analytics platforms - often solving adjacent problems, sometimes competing for the same slot. The confusion is understandable: both can ingest high-throughput event streams, both support time-based queries, and both have found homes in production monitoring systems. But they were built around fundamentally different data structures, and that shapes almost every engineering decision that flows from them.

Elasticsearch is a distributed search engine built on Apache Lucene. Its core data structure is an inverted index: for each token in a field, the index stores a posting list of document IDs that contain it. This makes full-text queries, relevance scoring, and fuzzy matching genuinely fast. ClickHouse, by contrast, is a columnar OLAP database. Data is stored in compressed column files, sorted and partitioned by the primary key. Every SELECT reads only the columns referenced in the query, and vectorized execution operates on column chunks at a time rather than row by row. These are not minor implementation details - they determine what each system is good at and where it struggles.

Storage Efficiency

The storage gap between the two systems is large enough to change infrastructure cost calculations at scale. Elasticsearch's inverted index stores not just your raw data but also the full inverted index structure, document source (_source), and doc_values columns for sorting and aggregations (an on-disk columnar structure built at index time). For a typical log dataset, Elasticsearch's total on-disk footprint is commonly 1–3x the size of raw uncompressed data once these structures are included. ClickHouse's columnar compression routinely achieves 10–16x compression on log data, meaning teams migrating from Elasticsearch to ClickHouse commonly report storage reductions of 5 to 10x, with some structured datasets compressing even further due to ClickHouse's columnar layout making codec selection (LZ4, ZSTD, Delta, DoubleDelta) highly effective.

The reason is architectural. Columnar storage groups the same field across all rows into contiguous byte sequences, and similar values - timestamps incrementing by milliseconds, IP addresses in the same /24 subnet, HTTP status codes with low cardinality - compress extremely well under run-length or delta encoding. A column of HTTP status codes for a web server is mostly 200s and 404s; ClickHouse can represent thousands of them in a handful of bytes. Elasticsearch stores each document's fields interleaved in the _source JSON blob and separately tokenizes text fields into the inverted index, producing structural overhead regardless of the actual information density.

This matters not just for disk cost. Smaller data means faster scans. Analytical queries on ClickHouse that aggregate over billions of rows can complete in seconds precisely because the data being read from disk is dense and the I/O is sequential over a single column.

Query Model: SQL vs DSL

ClickHouse uses SQL with extensions for analytical workloads - window functions, array functions, WITH ROLLUP, GROUPING SETS, quantile() and quantileExact() for percentile aggregations, and argMax() / argMin() for finding the row associated with a maximum or minimum value. Most engineers working with time-series data find these expressive enough to write complex queries without fighting the language. Historically, ClickHouse's query planner has been intentionally constrained: it relied on rule-based, linear execution and good table design - primary key sort order, partitioning, materialized views - rather than inferring a plan at runtime. As of 2024, an experimental cost-based optimizer is under active development, but it is not the default; the guidance to design your schema and sort order for your query patterns remains the primary optimization lever.

Elasticsearch's query language is a JSON DSL layered on top of Lucene query semantics. It handles full-text operations - match, multi_match, query_string, span_near for proximity queries, and function_score for custom relevance weighting - with no equivalent in ClickHouse. Aggregations use a nested bucket-and-metric model: a terms aggregation buckets by field value, a date_histogram aggregation buckets by time interval, and metric aggregations like avg, percentiles, and cardinality sit inside those buckets. For structured analytical queries this works, but it becomes verbose quickly. A query that is two lines of SQL becomes 40 lines of JSON in Elasticsearch's aggregation DSL, and the execution model has limits - high-cardinality terms aggregations require setting explicit size caps, and crossing those limits truncates results — signaled via doc_count_error_upper_bound and sum_other_doc_count in the response, but easy to miss in application code.

For log-analytics-style queries where you want to count events by service, compute the 99th percentile latency by endpoint, and filter by a time window, ClickHouse SQL is considerably more ergonomic and the execution is usually faster by an order of magnitude due to columnar scan performance.

Operational Complexity

Running Elasticsearch in production requires sustained attention to a set of JVM-specific concerns that ClickHouse simply does not have. Heap sizing is a recurring pressure point: the JVM heap should be set to no more than 50% of available RAM, and ideally kept at or below 26–30 GB to stay within the compressed ordinary object pointer (compressed oops) threshold (Elasticsearch 8.x automatically caps its auto-configured heap at 31 GB for this reason). Crossing that threshold degrades garbage collection performance and can introduce Stop-the-World pauses long enough to cause a node to drop out of the cluster. Operators tune GC settings, monitor old-gen heap usage, watch for circuit breaker trips, and manage shard allocation to keep heap per node within bounds.

Shard management is its own discipline. Each Elasticsearch shard is a self-contained Lucene index. Too many small shards create overhead on the cluster state and increase per-node JVM heap consumption; too few large shards limit parallelism during search. The standard guidance is 10–30 GB per shard for search-heavy workloads and 30–50 GB for log-analytics write-heavy workloads (the older heuristic of 20 shards per GB of heap was deprecated in Elasticsearch 8.3). Achieving that over a rolling retention window means configuring Index Lifecycle Management (ILM) to roll over indices on size or age, then cycle through hot, warm, cold, frozen, and delete phases. This is meaningful operational surface area. It works when correctly configured, and it is a source of incidents when it is not.

ClickHouse's operational model is simpler in comparison. The MergeTree engine family handles compaction of data parts in the background - small parts written at ingestion time are merged into larger ones asynchronously - and there is no JVM heap to tune. Memory management is handled by ClickHouse itself, and while it has tuning parameters (max memory usage per query, merge settings, part size limits), they are fewer and more predictable than Elasticsearch's operational surface. Replication uses ZooKeeper or ClickHouse Keeper as a coordination layer, but the replication model is straightforward: replicas store identical data and serve identical queries.

Full-Text Search: Where Elasticsearch Still Wins

ClickHouse has added full-text search capabilities over time. The tokenbf_v1 skip index creates a bloom filter of tokens per data granule, allowing the engine to skip granules that cannot possibly contain a given token. The ngrambf_v1 index extends this to n-gram matches, enabling substring search. For keyword-in-log-line queries these help and can deliver meaningful speedups over a full scan. Since version 23.1, ClickHouse also includes an experimental native full-text (inverted) index that maintains a proper term dictionary and posting lists per data part — more like Lucene's structure. As of early 2026 this index is still experimental (requires SET allow_experimental_full_text_index = 1) and has known limitations, so for production workloads the bloom-filter skip indexes remain the stable option. Elasticsearch's Lucene-based inverted index — with exact posting lists, fuzzy matching, relevance scoring, stemming, synonym expansion, and language-aware tokenization — remains considerably more mature and capable for genuine full-text search requirements.

The practical implication is that query patterns drive the choice. If your primary access pattern is "find log lines containing this error string, show me the last 100 results, let me paginate and filter through them" - that is Elasticsearch's native mode. If your primary access pattern is "count error events by service over the last 24 hours, broken down by HTTP status code, compute p95 latency per endpoint" - that belongs in ClickHouse. The overlap case, which comes up often in observability platforms, is wanting both at the same time. Teams handling this usually route structured event data to ClickHouse for aggregation queries and maintain a shorter retention window in Elasticsearch for full-text search over recent events. That architecture requires coordination on schema and ingestion but avoids compromising either tool's core capabilities.

Which System Fits Your Workload

If your workload is predominantly analytical - time-series aggregations, GROUP BY at scale, dashboards over billions of events - ClickHouse is the better fit. The storage efficiency, SQL expressiveness, and query throughput at that workload type are genuinely difficult to match with Elasticsearch. Teams that have replaced Elasticsearch with ClickHouse for log analytics consistently report faster aggregation queries and lower infrastructure costs, at the expense of richer text search capabilities and a less mature ecosystem around relevance tuning.

If your workload requires full-text search, document retrieval by relevance, fuzzy matching, or complex query-time scoring, Elasticsearch does things ClickHouse cannot replicate without substantial custom engineering. Elasticsearch's query language and Lucene's search capabilities remain the right tool for search-as-a-product use cases - e-commerce search, enterprise search, document retrieval, and anything where ranking quality and recall matter to end users.

The nuanced middle case is observability. Modern observability platforms increasingly use ClickHouse as the primary store for structured traces, metrics, and log fields, while keeping a thin Elasticsearch layer for message-body full-text queries - or using ClickHouse's bloom filter indexes as an acceptable approximation when perfect recall is not required. Whether that trade-off holds up depends on how much of your query traffic requires genuine full-text semantics versus structured field filtering. In most production log analytics workloads, the vast majority of queries are structured - filtering by service name, log level, trace ID, or time range - and ClickHouse handles those faster and at lower cost.

Subscribe to the NeverBlink Newsletter

Get early access to new NeverBlink features, insightful blogs & exclusive events , webinars, and workshops.

We use cookies to provide an optimized user experience and understand our traffic. To learn more, read our use of cookies; otherwise, please choose 'Accept Cookies' to continue using our website.