Elasticsearch Benchmarks

Results

Hint These diagrams are just provided as an overview. Please go to the Kibana dashboard if you want to analyze the results in more detail.

Overview

On this page are the results of the Elasticsearch nightly benchmarks based on the main branch. The Apache Software Foundation also provides a similar page for the Lucene nightly benchmarks.

Benchmarks

Benchmarking Methodology

All benchmarks are executed by Rally targetting clusters running the latest Elasticsearch snapshot build from the main branch. The builds use the last commit at or before 18:00 UTC on each day. Benchmark evironments are recreated every day on AWS or GCP VMs. One of the VMs host the benchmark driver (Rally), while others the Elasticsearch nodes. Unless specified differently all VMs are placed in the same region with a preference of a single AZ.

Intentionally these are not scalability benchmarks. Their purpose is to help Elasticsearch developers spot performance regressions.

Benchmarking Environment

All hosts used in the benchmarks have the following shared characteristics:

OS: Ubuntu 22.04 LTS
System configuration: following Elasticsearch documentation
sysfs settings:
- /sys/kernel/mm/transparent_hugepage/enabled = always
- /sys/kernel/mm/transparent_hugepage/defrag = always
JVM: JDK bundled with Elasticsearch

History of changes

2025-09-17:

Migrated benchmarks running on bare-metal machines to cloud VMs. Effectively, benchmarks were moved from Ubuntu 18.04 kernel 5.4.0-65 to Ubuntu 22.04 kernels 6.8.0-1029-aws and 6.5.0-1025-gcp. Applies to benchmarks based on the following tracks: dense_vector, eql, geonames, geopoint, geopointshape, geoshape, http_logs, nested, noaa, nyc_taxis, percolator, pmc, and sql.

2025-07-14:

Replaced data disk on one target machine used by the tracks geonames, geopoint, percolator, nyc_taxis and eql with another identical model (Micron_1100_MTFDDAK512TBN).

2025-06-09:

Replaced data disk on one target machine used by the tracks nested, noaa, pmc, sql and http_logs with another identical model (Micron_1100_MTFDDAK512TBN).

2025-06-05:

Upgraded AWS Ubuntu 22.04 image to the one dated 20250516 with kernel 6.8.0-1029-aws (MGLRU enabled).

2025-02-07:

Replaced data disk on one target machine used by the tracks geonames, geopoint, percolator, nyc_taxis and eql with another identical model (Micron_1100_MTFDDAK512TBN).

2024-09-10:

Upgraded AWS image from Ubuntu 18.04 to Ubuntu 22.04 dated 20240701 with kernel 6.5.0-1022-aws (MGLRU enabled).
Upgraded GCP image from Ubuntu 18.04 to Ubuntu 22.04 dated 20240801 with kernel 6.5.0-1025-gcp (MGLRU enabled).

2024-05-10:

Replaced data disk on one target machine used by the tracks nested, noaa, pmc, sql and http_logs with another identical model (Micron_1100_MTFDDAK512TBN).

2024-03-18:

Replaced data disk on one target machine used by the tracks nested, noaa, pmc, sql and http_logs with another identical model (Micron_1100_MTFDDAK512TBN).

2024-01-10:

Replaced data disk on one target machine used by the tracks geonames, geopoint, percolator, nyc_taxis and eql with another identical model (Micron_1100_MTFDDAK512TBN).

2023-12-26:

Replaced data disk on one target machine used by the track http_logs with another identical model (Micron_1100_MTFDDAK512TBN).

2023-12-20:

Replaced data disk on one target machine used by the tracks nested, noaa, pmc, sql and http_logs with another identical model (Micron_1100_MTFDDAK512TBN).

2023-10-27:

Added a new dashboard for elser-ingest track under Search section.

2023-10-27:

Added a new dashboard for msmarco-passage-ranking track under Search section.

2023-10-23:

Added a new dashboard for wikipedia track under Search section.

2023-05-07:

Replaced data disk on one target machine used by the tracks geonames, geopoint, percolator, nyc_taxis and eql with another identical model (Micron_1100_MTFDDAK512TBN).

2023-04-27:

Replaced data disk on one target machine used by the tracks geonames, geopoint, percolator, nyc_taxis and eql with another identical model (Micron_1100_MTFDDAK512TBN).

2022-11-07:

Added a new dashboard for so_vector track under Search section.

2022-11-01:

Added two new dashboards for so-transform and so-frequent-items challenges under Machine Learning section.

2022-08-28:

Updated both arm64 and x86_64 AWS AMIs for ubuntu-bionic-18.04-{arch}-server from 20201027 (arm64, Ubuntu 18.04.5, kernel 5.4.0-1029-aws) and 20200610 (x86_64, Ubuntu 18.04.4, kernel 5.3.0-1023-aws), respectively, to 20220810 (Ubuntu 18.04.6, kernel 5.4.0-1083-aws).

2022-08-18:

Replaced data disk on one target machine used by the tracks pmc and http_logs with another identical model (Micron_1100_MTFDDAK512TBN).

2021-11-08:

Enabled all the tracks and challenges to be run with security on.

2021-10-03:

Replaced data disk on one target machine used by the tracks nested, noaa, pmc, http_logs with another identical model (Micron_1100_MTFDDAK512TBN).

2021-08-23:

Replaced data disk on one target machine used by the tracks geonames, geopoint, percolator, nyc_taxis with another identical model (Micron_1100_MTFDDAK512TBN).

2021-06-01:

Replaced data disk on one target machine used by the tracks nested, noaa, pmc, http_logs with another identical model (Micron_1100_MTFDDAK512TBN).

2021-04-21:

Added new benchmarking hardware and geopointshape benchmarks.

2021-02-07:

Upgraded all Rally load driver and Elasticsearch target servers to Ubuntu 18.04 Bionic, kernel 5.4.0-65.

2021-01-13:

Replaced data disk on one target machine used by the tracks geonames, geopoint, percolator, nyc_taxis with another identical model (Micron_1100_MTFDDAK512TBN).

2020-05-18:

Replaced data disk on one target machine used by the tracks geonames, geopoint, percolator, nyc_taxis with another identical model (Micron_1100_MTFDDAK512TBN).

2020-04-30:

Run nightly benchmarks with JDK bundled with Elasticsearch.

2020-04-06:

Replaced data disk on all target machines used by the tracks nested, noaa, pmc, http_logs with another identical model (Micron_1100_MTFDDAK512TBN).

2020-02-07:

Performed OS package upgrade for load driver server used for tracks nested, noaa, pmc, http_logs and switched to kernel 4.15.0-76-generic.

2020-02-04:

Performed OS package upgrade for load driver server used for tracks geonames, geopoint, percolator, nyc_taxis and switched to kernel 4.15.0-76-generic.

2019-12-12:

Replaced data disk on first target machine used by the tracks nested, noaa, pmc, http_logs with another identical model (Micron_1100_MTFDDAK512TBN).

2019-04-25:

Switch to new benchmarking hardware and add benchmarks for default distribution

2019-04-03:
- Run nightly benchmarks with OpenJDK 11.0.2 instead of Oracle JDK 1.8.0_131-b11 (relates elastic/elasticsearch#40754)
2018-10-02:
- Replaced data disk (Samsung MZ7LN512HMJP-00000) on first target machine with another model (Micron_M600_MTFDDAK512MBF) resulting in higher performance in some benchmarks.
2018-04-12:
- Set transparent_hugepage enabled and defrag settings to always. This makes it consistent with the settings before the upgrade to 4.13.0.
- Upgrade kernel from version 4.13.0-36 to 4.13.0-38.
2018-02-21: Upgrade kernel from version 4.13.0-32 to 4.13.0-36.
2018-02-06: Upgrade kernel from version 4.8.0-53 to 4.13.0-32.
2017-10-06:
- Change from one load generator and one target machine (connected via 1 GBit switch) to one load generator and three target machines (connected via 10 GBit switch).
- CPU has changed from Intel(R) Core(TM) i7-6700 CPU @ 3.40GHz to Intel(R) Core(TM) i7-7700 CPU @ 3.60GHz
- Disk has changed from Crucial MX200 to Samsung MZ7LN512HMJP-00000
2017-06-29: Upgrade JDK from version 1.8.0_101-b13 to 1.8.0_131-b11
2017-06-29: Upgrade kernel from version 4.4.0-72 to 4.8.0-53
2017-04-13: Upgrade kernel from version 4.4.0-38 to 4.4.0-72