Elasticsearch Benchmarks

Results

Overview

On this page are the results of the Elasticsearch nightly benchmarks based on the main branch as of that point in time. The Apache Software Foundation also provides a similar page for the Lucene nightly benchmarks.

Benchmarks

Benchmarking Methodology

All benchmarks are run by Rally against the Elasticsearch main branch as of that date. The benchmark uses four bare-metal server-class machines. On one we run the benchmark driver (Rally), on the other three the benchmark candidate (one to three Elasticsearch nodes, one per machine). All machines are connected via a dedicated 10 GBit switched network.

The benchmarks are intentionally not scalability benchmarks but rather show the performance characteristics of Elasticsearch ranging from one node to at most three nodes and are mainly intended to help the development team spot performance regressions.

Benchmarking Environment

Dedicated servers

Unless specified otherwise on the benchmark page, all benchmarks are run on bare-metal machines with the following specifications:

  • CPU: Intel(R) Core(TM) i7-7700 CPU @ 3.60GHz
  • RAM: 32 GB
  • SSD: Micron_1100_MTFDDAK512TBN
  • OS for Rally load driver servers:
    • Linux kernel version 5.4.0-65
  • OS for Elasticsearch target servers:
    • Linux kernel version 5.4.0-65
    • /sys/kernel/mm/transparent_hugepage/enabled = always
    • /sys/kernel/mm/transparent_hugepage/defrag = always
    • /sys/devices/system/cpu/intel_pstate/no_turbo = 1
  • JVM: JDK bundled with Elasticsearch.

All machines are connected via a dedicated 10 GBit switched network.

History of changes

  • 2024-03-18:
    • Replaced data disk on one target machine used by the tracks nested, noaa, pmc, sql and http_logs with another identical model (Micron_1100_MTFDDAK512TBN).
  • 2024-01-10:
    • Replaced data disk on one target machine used by the tracks geonames, geopoint, percolator, nyc_taxis and eql with another identical model (Micron_1100_MTFDDAK512TBN).
  • 2023-12-26:
    • Replaced data disk on one target machine used by the track http_logs with another identical model (Micron_1100_MTFDDAK512TBN).
  • 2023-12-20:
    • Replaced data disk on one target machine used by the tracks nested, noaa, pmc, sql and http_logs with another identical model (Micron_1100_MTFDDAK512TBN).
  • 2023-10-27:
    • Added a new dashboard for elser-ingest track under Search section.
  • 2023-10-27:
    • Added a new dashboard for msmarco-passage-ranking track under Search section.
  • 2023-10-23:
    • Added a new dashboard for wikipedia track under Search section.
  • 2023-05-07:
    • Replaced data disk on one target machine used by the tracks geonames, geopoint, percolator, nyc_taxis and eql with another identical model (Micron_1100_MTFDDAK512TBN).
  • 2023-04-27:
    • Replaced data disk on one target machine used by the tracks geonames, geopoint, percolator, nyc_taxis and eql with another identical model (Micron_1100_MTFDDAK512TBN).
  • 2022-11-07:
    • Added a new dashboard for so_vector track under Search section.
  • 2022-11-01:
    • Added two new dashboards for so-transform and so-frequent-items challenges under Machine Learning section.
  • 2022-08-28:
    • Updated both arm64 and x86_64 AWS AMIs for ubuntu-bionic-18.04-{arch}-server from 20201027 (arm64, Ubuntu 18.04.5, kernel 5.4.0-1029-aws) and 20200610 (x86_64, Ubuntu 18.04.4, kernel 5.3.0-1023-aws), respectively, to 20220810 (Ubuntu 18.04.6, kernel 5.4.0-1083-aws).
  • 2022-08-18:
    • Replaced data disk on one target machine used by the tracks pmc and http_logs with another identical model (Micron_1100_MTFDDAK512TBN).
  • 2021-11-08:
    • Enabled all the tracks and challenges to be run with security on.
  • 2021-10-03:
    • Replaced data disk on one target machine used by the tracks nested, noaa, pmc, http_logs with another identical model (Micron_1100_MTFDDAK512TBN).
  • 2021-08-23:
    • Replaced data disk on one target machine used by the tracks geonames, geopoint, percolator, nyc_taxis with another identical model (Micron_1100_MTFDDAK512TBN).
  • 2021-06-01:
    • Replaced data disk on one target machine used by the tracks nested, noaa, pmc, http_logs with another identical model (Micron_1100_MTFDDAK512TBN).
  • 2021-04-21:
    • Added new benchmarking hardware and geopointshape benchmarks.
  • 2021-02-07:
    • Upgraded all Rally load driver and Elasticsearch target servers to Ubuntu 18.04 Bionic, Linux Kernel 5.4.0-65
  • 2021-01-13:
    • Replaced data disk on one target machine used by the tracks geonames, geopoint, percolator, nyc_taxis with another identical model (Micron_1100_MTFDDAK512TBN).
  • 2020-05-18:
    • Replaced data disk on one target machine used by the tracks geonames, geopoint, percolator, nyc_taxis with another identical model (Micron_1100_MTFDDAK512TBN).
  • 2020-04-30:
    • Run nightly benchmarks with JDK bundled with Elasticsearch.
  • 2020-04-06:
    • Replaced data disk on all target machines used by the tracks nested, noaa, pmc, http_logs with another identical model (Micron_1100_MTFDDAK512TBN).
  • 2020-02-07:
    • Performed OS package upgrade for load driver server used for tracks nested, noaa, pmc, http_logs and switched to kernel 4.15.0-76-generic.
  • 2020-02-04:
    • Performed OS package upgrade for load driver server used for tracks geonames, geopoint, percolator, nyc_taxis and switched to kernel 4.15.0-76-generic.
  • 2019-12-12:
    • Replaced data disk on first target machine used by the tracks nested, noaa, pmc, http_logs with another identical model (Micron_1100_MTFDDAK512TBN).
  • 2019-04-25:
    • Switch to new benchmarking hardware and add benchmarks for default distribution
  • 2019-04-03:
  • 2018-10-02:
    • Replaced data disk (Samsung MZ7LN512HMJP-00000) on first target machine with another model (Micron_M600_MTFDDAK512MBF) resulting in higher performance in some benchmarks.
  • 2018-04-12:
    • Set transparent_hugepage enabled and defrag settings to always. This makes it consistent with the settings before the upgrade to 4.13.0.
    • Upgrade kernel from version 4.13.0-36 to 4.13.0-38.
  • 2018-02-21: Upgrade kernel from version 4.13.0-32 to 4.13.0-36.
  • 2018-02-06: Upgrade kernel from version 4.8.0-53 to 4.13.0-32.
  • 2017-10-06:
  • 2017-06-29: Upgrade JDK from version 1.8.0_101-b13 to 1.8.0_131-b11
  • 2017-06-29: Upgrade kernel from version 4.4.0-72 to 4.8.0-53
  • 2017-04-13: Upgrade kernel from version 4.4.0-38 to 4.4.0-72