This table lists the fastest query times (measured in seconds) seen in each of my benchmarks broken down by software and hardware setup.
The dataset used has 1.1 billion records, 51 columns and is 500 GB in size when in uncompressed CSV format. Instructions on producing the dataset can be found in my Billion Taxi Rides in Redshift blog post.
Query 1 |
Query 2 |
Query 3 |
Query 4 |
Setup |
0.005 |
0.011 |
0.103 |
0.188 |
BrytlytDB 2.1 & 5-node IBM Minsky cluster |
0.009 |
0.027 |
0.287 |
0.428 |
BrytlytDB 2.0 & 2-node p2.16xlarge cluster |
0.021 |
0.053 |
0.165 |
0.51 |
OmniSci & 8 Nvidia Pascal Titan Xs |
0.027 |
0.083 |
0.163 |
0.891 |
OmniSci & 8 Nvidia Tesla K80s |
0.028 |
0.2 |
0.237 |
0.578 |
OmniSci & 4-node g2.8xlarge cluster |
0.034 |
0.061 |
0.178 |
0.498 |
OmniSci & 2-node p2.8xlarge cluster |
0.036 |
0.131 |
0.439 |
0.964 |
OmniSci & 4 Nvidia Titan Xs |
0.051 |
0.146 |
0.047 |
0.794 |
kdb+/q & 4 Intel Xeon Phi 7210 CPUs |
0.134 |
0.349 |
0.542 |
3.312 |
OmniSci & a 16" MacBook Pro |
0.241 |
0.826 |
1.209 |
1.781 |
ClickHouse, 3 x c5d.9xlarge cluster |
0.347 |
1.1 |
1.389 |
2.935 |
Clickhouse on DoubleCloud, s1-c32-m128 |
0.466 |
1.094 |
0.742 |
1.412 |
Hydrolix & c5n.9xlarge cluster |
0.498 |
0.234 |
0.734 |
1.334 |
DuckDB 0.10.0 & Intel Core i9-14900K |
0.762 |
2.472 |
4.131 |
6.041 |
BrytlytDB 1.0 & 2-node p2.16xlarge cluster |
1.034 |
3.058 |
5.354 |
12.748 |
ClickHouse, Intel Core i5 4670K |
1.56 |
1.25 |
2.25 |
2.97 |
Redshift, 6-node ds2.8xlarge cluster |
2 |
2 |
1 |
3 |
BigQuery |
2.362 |
3.559 |
4.019 |
20.412 |
Spark 2.4 & 21 x m3.xlarge HDFS cluster |
3.54 |
6.29 |
7.66 |
11.92 |
Presto 0.214 & 21 x m3.xlarge HDFS cluster |
4 |
4 |
10 |
21 |
Presto, 50-node n1-standard-4 cluster |
4.88 |
11 |
12 |
15 |
Presto 0.188 & 21-node m3.xlarge cluster |
6.41 |
6.19 |
6.09 |
6.63 |
Amazon Athena |
8.1 |
18.18 |
n/a |
n/a |
Elasticsearch (heavily tuned) |
10.19 |
8.134 |
19.624 |
85.942 |
Spark 2.1, 11 x m3.xlarge HDFS cluster |
11 |
10 |
21 |
31 |
Presto, 10-node n1-standard-4 cluster |
11 |
14 |
16 |
22 |
Presto 0.188 & single i3.8xlarge w/ HDFS |
14.389 |
32.148 |
33.448 |
67.312 |
Vertica, Intel Core i5 4670K |
22 |
25 |
27 |
65 |
Spark 2.3.0 & single i3.8xlarge w/ HDFS |
28 |
31 |
33 |
80 |
Spark 2.2.1 & 21-node m3.xlarge cluster |
34.48 |
63.3 |
n/a |
n/a |
Elasticsearch (lightly tuned) |
35 |
39 |
64 |
81 |
Presto, 5-node m3.xlarge HDFS cluster |
43 |
45 |
27 |
44 |
Presto, 50-node m3.xlarge cluster w/ S3 |
152 |
175 |
235 |
368 |
PostgreSQL 9.5 & cstore_fdw |
264 |
313 |
620 |
961 |
Spark 1.6, 5-node m3.xlarge cluster w/ S3 |
448 |
797 |
1811 |
3286 |
SQLite 3, Parquet & HDFS |
1103 |
1198 |
2278 |
6446 |
Spark 2.2, 3-node Raspberry Pi cluster |
31193 |
NR |
NR |
NR |
SQLite 3, Internal File Format |
- NR is short for "Not Run".