Home | Benchmarks | Categories | Atom Feed

Updated on Sun 21 June 2020

Summary of the 1.1 Billion Taxi Rides Benchmarks

This table lists the fastest query times (measured in seconds) seen in each of my benchmarks broken down by software and hardware setup.

The dataset used has 1.1 billion records, 51 columns and is 500 GB in size when in uncompressed CSV format. Instructions on producing the dataset can be found in my Billion Taxi Rides in Redshift blog post.

Query 1 Query 2 Query 3 Query 4 Setup
0.005 0.011 0.103 0.188 BrytlytDB 2.1 & 5-node IBM Minsky cluster
0.009 0.027 0.287 0.428 BrytlytDB 2.0 & 2-node p2.16xlarge cluster
0.021 0.053 0.165 0.51 OmniSci & 8 Nvidia Pascal Titan Xs
0.027 0.083 0.163 0.891 OmniSci & 8 Nvidia Tesla K80s
0.028 0.2 0.237 0.578 OmniSci & 4-node g2.8xlarge cluster
0.034 0.061 0.178 0.498 OmniSci & 2-node p2.8xlarge cluster
0.036 0.131 0.439 0.964 OmniSci & 4 Nvidia Titan Xs
0.051 0.146 0.047 0.794 kdb+/q & 4 Intel Xeon Phi 7210 CPUs
0.134 0.349 0.542 3.312 OmniSci & a 16" MacBook Pro
0.241 0.826 1.209 1.781 ClickHouse, 3 x c5d.9xlarge cluster
0.347 1.1 1.389 2.935 Clickhouse on DoubleCloud, s1-c32-m128
0.466 1.094 0.742 1.412 Hydrolix & c5n.9xlarge cluster
0.498 0.234 0.734 1.334 DuckDB 0.10.0 & Intel Core i9-14900K
0.762 2.472 4.131 6.041 BrytlytDB 1.0 & 2-node p2.16xlarge cluster
1.034 3.058 5.354 12.748 ClickHouse, Intel Core i5 4670K
1.56 1.25 2.25 2.97 Redshift, 6-node ds2.8xlarge cluster
2 2 1 3 BigQuery
2.362 3.559 4.019 20.412 Spark 2.4 & 21 x m3.xlarge HDFS cluster
3.54 6.29 7.66 11.92 Presto 0.214 & 21 x m3.xlarge HDFS cluster
4 4 10 21 Presto, 50-node n1-standard-4 cluster
4.88 11 12 15 Presto 0.188 & 21-node m3.xlarge cluster
6.41 6.19 6.09 6.63 Amazon Athena
8.1 18.18 n/a n/a Elasticsearch (heavily tuned)
10.19 8.134 19.624 85.942 Spark 2.1, 11 x m3.xlarge HDFS cluster
11 10 21 31 Presto, 10-node n1-standard-4 cluster
11 14 16 22 Presto 0.188 & single i3.8xlarge w/ HDFS
14.389 32.148 33.448 67.312 Vertica, Intel Core i5 4670K
22 25 27 65 Spark 2.3.0 & single i3.8xlarge w/ HDFS
28 31 33 80 Spark 2.2.1 & 21-node m3.xlarge cluster
34.48 63.3 n/a n/a Elasticsearch (lightly tuned)
35 39 64 81 Presto, 5-node m3.xlarge HDFS cluster
43 45 27 44 Presto, 50-node m3.xlarge cluster w/ S3
152 175 235 368 PostgreSQL 9.5 & cstore_fdw
264 313 620 961 Spark 1.6, 5-node m3.xlarge cluster w/ S3
448 797 1811 3286 SQLite 3, Parquet & HDFS
1103 1198 2278 6446 Spark 2.2, 3-node Raspberry Pi cluster
31193 NR NR NR SQLite 3, Internal File Format

  • NR is short for "Not Run".

Copyright © 2014 - 2024 Mark Litwintschik. This site's template is based off a template by Giulio Fidente.