+{/* ARCADE EMBED START */}
+
+{/* ARCADE EMBED END */}
-#### Community engine size
+#### Small engine size
-The community engine is the default query engine for all queries on Dune. It is a shared cluster, meaning that it is used by all Dune users. This means that the community cluster can be busy at times if many users are running queries at the same time.
+The Free engine is the default query engine for all queries on Dune. It is a shared cluster, meaning that it is used by all Dune users. This means that the community cluster can be busy at times if many users are running queries at the same time.
To avoid long loading times and timeouts, we recommend using the medium or large engine for resource-intensive queries.
-Executions on the community engine are free of charge and will time out after 2 minutes.
+Executions on the free engine are free of charge and will time out after 2 minutes.
+
+{/* ARCADE EMBED START */}
+
+
+
+{/* ARCADE EMBED END */}
#### Medium engine size
-The medium engine is built to handle most queries on Dune. It is cheap, reliable and fast. The medium engine will scale up and down depending on the demand.
-Executions on the medium engine cost 10 credits.
-It has a capacity equivalent to the Free engine, but executions can run for longer, timing out only after 30 minutes.
+Run queries manually or programmatically. Credits consumed based on actual compute resources used. Suitable for most standard queries and dashboards.
+
+{/* ARCADE EMBED START */}
+
+
+
+{/* ARCADE EMBED END */}
#### Large engine size
-The large engine is built to handle the most resource-intensive queries on Dune. It's blazing fast, reliable and can easily deal with large amounts of data. The large engine also scales up and down depending on the demand.
+Run queries with maximum timeout and resources. Credits consumed based on actual compute resources used. Ideal for complex queries and heavy workloads.
-In addition to that, the large engine is also the only engine that can handle queries that requires lots of planning time. This mostly happens when you query a large amount of data, when you use a lot of joins or large aggregate window functions.
-
-Executions on the large engine cost 20 credits.
-It has twice the capacity of the Medium engine, and executions can also run for 30 minutes.
\ No newline at end of file
+{/* ARCADE EMBED START */}
+
+
+
+{/* ARCADE EMBED END */}
diff --git a/query-engine/reserved-keywords.mdx b/query-engine/reserved-keywords.mdx
index ee5fb0f2..c922420c 100644
--- a/query-engine/reserved-keywords.mdx
+++ b/query-engine/reserved-keywords.mdx
@@ -16,6 +16,7 @@ where "from" = 0xc8ebccc5f5689fa8659d83713341e5ad19349448
| `ALTER` | reserved |
| `AND` | reserved |
| `AS` | reserved |
+| `ASOF` | reserved |
| `BETWEEN` | reserved |
| `BY` | reserved |
| `CASE` | reserved |
diff --git a/query-engine/writing-efficient-queries.mdx b/query-engine/writing-efficient-queries.mdx
index 80a5a6e2..07f53ffa 100644
--- a/query-engine/writing-efficient-queries.mdx
+++ b/query-engine/writing-efficient-queries.mdx
@@ -3,230 +3,230 @@ title: Writing Efficient Queries
description: Get the most out of DuneSQL by writing efficient queries.
---
-**Writing efficient queries is essential for getting the most out of Dune. This guide will help you understand how to write efficient queries on DuneSQL.**
+This guide provides practical do's and don'ts specifically tailored to DuneSQL and Dune's architecture, allowing you to optimize credit spend.
-In order to write efficient queries on DuneSQL, it's important to understand the underlying architecture of the system. This guide will help you understand how DuneSQL works under the hood so you can write more efficient queries and get the most out of Dune. Sadly there is no magic bullet that will make all your queries run faster, but understanding the underlying architecture of DuneSQL will help you write more efficient queries and get the most out of the system.
+## Key Principles
-## DuneSQL Architecture
+Dune's query engine is optimized for blockchain data analysis with time-partitioned tables and columnar storage.
+Understanding the [DuneSQL architecture](/query-engine/dunesql-architecture) will help you write queries that execute faster and consume fewer resources.
-DuneSQL is a Trino-based query engine designed for handling data stored in a columnar format. More specifically, we use Parquet files as the underlying storage format. This allows for efficient data access and query processing, as well as fast data loading and data compression. In order to understand how to write efficient queries on DuneSQL, it's important to understand how data is stored and accessed in DuneSQL. Therefore this guide will start with a short introduction to databases and then move on to how DuneSQL works under the hood.
+## ✅ DO's: Query Optimization Best Practices
-### Short Introduction to Databases
+### 1. Leverage Time-Based Partitioning
-Let's start with a short introduction into databases so we can understand what we need to optimize for when writing queries on DuneSQL.
+Dune partitions most tables by `block_date` or by `block_time`
-At their core, databases are sophisticated systems designed to store, retrieve, and manage data. Their primary goal is to provide fast, efficient, and reliable access to vast amounts of structured information. You can think of a database as a collection of tables, where each table is a collection of rows and columns. Conceptually, these tables exist in two ways:
+Check on the data explorer (left-side panel) to see which fields are used as partitions.
+Always include time filters to enable partition pruning.
-- **Logical**: The logical view of a table is the way the data is organized and presented to the user. This is the view you see when you query a table.
-- **Physical**: The physical view of a table is the way the data is stored on disk. This is the view you see when you look at the underlying files that make up the table.
-
-Databases are designed to optimize for the logical view of a table, which is the view that users interact with. However, the physical view of a table is also important, as it determines how the data is stored and accessed. In order to optimize the usability of the logical view of a table, databases use a variety of techniques to optimize the physical view of a table. These techniques include:
-
-- **Data partitioning**: Data partitioning is a technique that divides data into smaller chunks called partitions. This reduces the amount of data that needs to be stored and accessed, which improves performance.
-- **Data indexing**: Data indexing is a technique that creates a data structure called an index. This data structure contains information about the data in a table, which allows the database to quickly find the data it needs.
-- **Data storage layout**: Data storage layout relates to how the data is stored on disk. This includes the file format, how the data is physically stored on disk, and how the data is organized in memory. The right data storage layout can significantly improve performance.
-- **Data compression**: Data compression is a technique that reduces the size of data by removing redundant information. This reduces the amount of data that needs to be stored and accessed, which improves performance.
-- **Data caching**: Data caching is a technique that stores frequently accessed data in memory. This reduces the amount of data that needs to be stored and accessed, which improves performance.
-
-For the most part, these techniques are employed in the background and are not visible to the user. However, understanding how **data partitioning**, **data indexing** and the **data storage layout** work is essential for writing efficient queries on DuneSQL.
-
-Databases employ these techniques to combat their most significant challenge: **the I/O bound nature of data storage**. I/O bound refers to the fact that the speed of data access is limited by the speed of the storage device. Read speed, the time it takes to load data from storage to memory, is an essential constraint of databases.
-
-Every time you query a table, the database needs to read the data from disk into memory. This happens in a unit called a page. Pages are the smallest unit of data that can be read from disk into memory. Since reading pages from disk is slow, databases try to minimize the number of pages that need to be read into memory when querying a table. This is where data partitioning and data indexing come into play. In the next section, we'll take a closer look at how DuneSQL works and how you can write queries that minimize the number of pages that need to be read into memory.
-
-**To quickly summarize:** The goal of a database is to provide fast, efficient, and reliable access to vast amounts of structured information. In the end, we want to access the logical view of a table as quickly as possible. To do this, database administrators use a variety of techniques to optimize the physical view of a table. These techniques include data partitioning, data indexing, data compression, and data caching. The goal of these techniques is to minimize the number of pages that need to be read into memory when querying a table.
-
-### DuneSQL Architecture
-
-Now that we understand how databases work, let's take a look at how DuneSQL works under the hood. Specifically, let's look at how data is stored and accessed in DuneSQL.
-
-Dune stores data in parquet files, which are first and foremost columnar storage files, but also utilize some of the advantages of row-oriented storage. Data in parquet systems is partitioned by rows into multiple parquet files, and within each file, the data is further partitioned into row groups. However, the pages inside of row groups store data in columns rather than rows. As a result, the database appears row-oriented at a higher level, but it reads data from column-oriented pages when accessing specific values. Additionally, each parquet file contains metadata about the data it stores, mainly **min/max** column statistics for each column. This allows the database to efficiently skip entire parquet files or row groups within files while scanning through a table, provided that the column contains values that are not random and can be ordered. **Writing queries that take advantage of this is essential for writing efficient queries on DuneSQL.**
-
-In a very simplified schematic view, our `ethereum.transactions` table is stored in a parquet file that looks like this:
-
-
-
-
-
-
-
-
-
-What's important to understand here is that the traditional index structure of a database is not needed in DuneSQL. Instead, the `min/max` values of each column are used to efficiently skip entire parquet files or row groups within files while scanning through a table, provided that the column contains values that are not random and can be ordered. This is a key difference between DuneSQL and e.g. Snowflake.
-
-If you query a table in DuneSQL, the system will access the data in the following order:
-
-1. **File Level:** At the foundational level, the query engine locates the specific parquet files associated with the table or a portion of the table being queried. It reads the metadata contained in the footer of each parquet file to determine whether it might contain the data needed for the query. It will skip any files that do not contain the data needed for the query.
-
-2. **Row Group Level:** Once the appropriate Parquet file is identified, the engine will access the relevant row groups within the file, based on the query conditions. Again, it will first read the metadata of each row group to determine whether it might contain the data needed for the query. If the row group does not contain the data needed for the query, it will skip the row group and move on to the next one.
-
-3. **Column Chunk Level:** Once the appropriate row group is identified, the system will access the relevant column chunks within the row group, based on the query conditions. The column chunks contain the actual data that is needed for the query. The database will only read the column chunks that contain the data needed for the query. It will not read the logical row - saving time and resources.
-
-4. **Page Level:** Within the column chunk, the data is further segmented into pages. The database reads these pages into memory, benefiting from any compression or encoding optimizations.
-
-During this process the engine starts stream processing from the first chunks of data it has read. It will optimize on the flight: based on the data it has already read, it can resign from reading other data if that data is recognized as irrelevant. This is called dynamic filtering and will usually save a lot of resources.
-
-If the query is too large to be processed in memory, the data will "spill" from memory to disk. This is called "spill to disk" and is a common occurrence in databases. This will negatively impact query performance, as reading data from disk is much slower than reading data from memory. To avoid this issue, it's important to write efficient queries that minimize the amount of data that needs to be read into memory.
-
-### Tips for Writing Efficient Queries on DuneSQL
-
-To write efficient queries on DuneSQL, it's crucial to use filter conditions based on columns that are not random and can be ordered. Columns like `block_time`, `block_date` and `block_number` are the best candidates for this, as they are sequentially ordered and can be used to efficiently skip entire parquet files or row groups within files while scanning through a table.
-
-Let's take a look at an example. Say you want to query the `ethereum.transactions` table for a specific transaction hash. The query would usually look like this:
+**Cross-chain tables:** Large cross-chain tables like `tokens.transfers`, `dex.trades`, and `evms.erc20_evt_transfers` are most often also partitioned by `blockchain` in addition to time. Wherever possible, specify the blockchain filter along with time filters to dramatically reduce the amount of data scanned.
```sql
-SELECT * FROM ethereum.transactions
-WHERE hash = 0xce1f1a2dd0c10fcf9385d14bc92c686c210e4accf00a3fe7ec2b5db7a5499cff
+-- ✅ GOOD: Filters by block_date to enable partition pruning
+SELECT
+ hash,
+ "from",
+ "to",
+ value
+FROM base.transactions
+WHERE block_date >= TIMESTAMP '2025-09-01 00:00:00'
+ AND block_date < TIMESTAMP '2025-10-02 00:00:00'
+ AND "to" = 0x827922686190790b37229fd06084350E74485b72
+
+-- ✅ EXCELLENT: Filters by both blockchain and time for cross-chain tables
+SELECT
+ block_time,
+ tx_hash,
+ "from",
+ "to",
+ amount_usd
+FROM dex.trades
+WHERE blockchain = 'ethereum'
+ AND block_time >= TIMESTAMP '2024-10-01'
+ AND block_time < TIMESTAMP '2024-11-01'
```
-using the steps above the query engine would do the following:
-
-1. **File Level:** The query engine tries to locate the specific parquet files associated with the table or a portion of the table being queried. It reads the metadata contained in the footer of each parquet file to determine whether it might contain the data needed for the query. It will try to skip any files that do not contain the data needed for the query. However, since the query is based on a random hash, the engine will not be able to skip any files. It will have to read all the files associated with the table.
-
-We can stop going through the steps here, as the query engine will have to read almost all the files and row groups associated with the table. This will negatively impact query performance, as having to read all pages from all column chunks in all row groups in all parquet files is very inefficient. There might be some very unlikely edge cases where the engine can skip some files, but in general, the engine will have to read all the files associated with the table. An example of a row group or column chunk that can be skipped for this query is if the hash column for a row group or file miraculously contains only values from `0xd... - 0xz...`. In that case, since our hash is `0xc...`, there is no way that the row group or file contains the hash we are looking for, and the engine can skip it. However, this is very unlikely to happen, and most likely the engine will have to read all the files associated with the table.
+### 2. Select Only Required Columns
-Now let's take a look at a query that uses a column that is not random and can be ordered, such as `block_number`:
+Specify only the columns you need. Dune's columnar storage makes this especially effective.
```sql
-SELECT * FROM ethereum.transactions
-WHERE block_number = 14854616
-AND hash = 0xce1f1a2dd0c10fcf9385d14bc92c686c210e4accf00a3fe7ec2b5db7a5499cff
+-- ✅ GOOD: Selects only necessary columns
+SELECT
+ evt_block_time,
+ "from",
+ "to",
+ value
+FROM erc20_ethereum.evt_Transfer
+WHERE contract_address = 0xa0b86991c6218b36c1d19d4a2e9eb0ce3606eb48
+ AND evt_block_time >= NOW() - INTERVAL '7' DAY
```
-using the steps above the query engine would do the following:
-
-1. **File Level:** The query engine tries to locate the specific parquet files associated with the table or a portion of the table being queried. It reads the metadata contained in the footer of each parquet file to determine whether it might contain the data needed for the query. In this case, the engine will check whether the `block_number` we are searching for is within the range of `min/max` values for the `block_number` column in the footer of each parquet file. For example, if the engine starts reading the first parquet file and sees that the `min/max` values for the `block_number` column are `14854600` and `14854620`, it will know that the file does not contain the data needed for the query. It will skip the file and move on to the next one. This will significantly reduce the amount of data that needs to be read into memory, which will improve query performance.
-
-2. **Row Group Level:** Once the appropriate Parquet file is identified, the engine will access the relevant row groups within the file, based on the query conditions. Again, it will first read the metadata of each row group to determine whether it might contain the data needed for the query. If the row group does not contain the data needed for the query, it will skip the row group and move on to the next one. This will further reduce the amount of data that needs to be read into memory, which will improve query performance.
-
-3. **Column Chunk Level:** Once the appropriate row group is identified, the system will access the relevant column chunks within the row group, based on the query conditions. The column chunks contain the actual data that is needed for the query. The database will only read the column chunks that contain the data needed for the query. It will not read the logical row - saving time and resources.
+### 3. Use Efficient JOIN Strategies
-4. **Page Level:** Within the column chunk, the data is further segmented into pages. The database reads these pages into memory, benefiting from any compression or encoding optimizations.
-
-As you can see, the query engine can skip entire parquet files and row groups within files while scanning through a table, provided that the column contains values that are not random and can be ordered. This is why it's important to use filter conditions based on columns that are not random and can be ordered, such as `block_number`,`block_time` or `block_date`.
-
-The above example is pretty theoretical, as you will most likely not be querying the `ethereum.transactions` table for a specific transaction hash. However, the same principle applies to other tables and queries. For example, if we want to query `ethereum.traces` for all calls to a specific smart contract, we can use `block_number`,`block_time` or `block_date` to skip entire parquet files and row groups within files while scanning through the table.
-
-Usually you would write a query like this:
+Put time filters in the ON clause and join on indexed columns when possible.
```sql
-Select
- *
-FROM ethereum.traces
-WHERE to = 0x510100D5143e011Db24E2aa38abE85d73D5B2177
-```
-
-With our new knowledge, we can deduce that this query will be very inefficient, as the engine will have to read almost all the files and row groups associated with the table. Instead, we can optimize the query by helping the engine to skip files with a `block_number` greater than a specific value. For example, if we know that the smart contract was deployed in block `17580248`, we can write the query like this:
-
-```sql
-Select
- "from"
- ,to
- ,block_time
- ,tx_hash
- ,input
-FROM ethereum.traces
-where block_number > 17580247
-and to = 0x510100D5143e011Db24E2aa38abE85d73D5B2177
+-- ✅ GOOD: Efficient join with time filter in ON clause
+SELECT
+ t.hash,
+ t.block_time,
+ l.topic0,
+ l.data
+FROM ethereum.transactions t
+INNER JOIN ethereum.logs l
+ ON t.hash = l.tx_hash
+ AND t.block_date = l.block_date
+ AND l.block_date >= TIMESTAMP '2024-10-01'
+ AND l.block_date < TIMESTAMP '2024-10-02'
+WHERE t."to" = 0xa0b86991c6218b36c1d19d4a2e9eb0ce3606eb48
+ AND t.block_date >= TIMESTAMP '2024-10-01'
+ AND t.block_date < TIMESTAMP '2024-10-02'
```
-By including the `block_number` column in the query, the engine can quickly narrow down the search to files and row groups that have a `block_number` greater than `17580247`. This significantly reduces the amount of data that needs to be read, thereby improving query performance. Additionally, only querying the columns you absolutely need will also improve query performance.
+### 4. Use CTEs for Complex Logic
-We can also apply the same principle to join tables. For example, if we want to join `ethereum.transactions` with `uniswap_v3_ethereum.Pair_evt_Swap`, we can write the query like this:
+Break complex queries into readable Common Table Expressions.
```sql
-Select
- to,
- "from",
- value,
- block_date,
- sqrt_price_x96,
-FROM
- uniswap_v3_ethereum.Pair_evt_Swap swap
- inner join ethereum.transactions t
- on swap.evt_tx_hash = t.hash
- and swap.evt_block_number = t.block_number
-where
- swap.contract_address = 0x510100D5143e011Db24E2aa38abE85d73D5B2177
+-- ✅ GOOD: Using CTEs for better performance and readability
+WITH daily_volumes AS (
+ SELECT
+ block_date as trade_date,
+ SUM(amount_usd) as daily_volume
+ FROM dex.trades
+ WHERE blockchain = 'ethereum'
+ AND block_date >= TIMESTAMP '2024-09-01'
+ AND block_date < TIMESTAMP '2024-10-01'
+ GROUP BY block_date
+),
+volume_metrics AS (
+ SELECT
+ trade_date,
+ daily_volume,
+ AVG(daily_volume) OVER (
+ ORDER BY trade_date
+ ROWS BETWEEN 6 PRECEDING AND CURRENT ROW
+ ) as rolling_7day_avg
+ FROM daily_volumes
+)
+SELECT * FROM volume_metrics
+ORDER BY trade_date DESC
```
-By including the `block_number` column in the join condition, the engine can quickly narrow down the search in `ethereum.transactions` to files and row groups that contain the `block_number` contained in a specific row in `uniswap_v3_ethereum.Pair_evt_Swap`. This significantly reduces the amount of data that needs to be read, thereby improving query performance.
+### 5. Use LIMIT with Large Result Sets
-It also makes a difference whether we join `ethereum.transactions` to `uniswap_v3_ethereum.Pair_evt_Swap` or the other way around. The following query will be even faster:
+Always use LIMIT when you don't need all results, especially for exploratory queries.
```sql
-Select
- to,
- "from",
- value,
- block_time,
- sqrt_price_x96
-FROM ethereum.transactions t
-inner join uniswap_v3_ethereum.Pair_evt_Swap swap
- on t.hash = swap.evt_tx_hash
- and t.block_number = swap.evt_block_number
-where
- swap.contract_address = 0x510100D5143e011Db24E2aa38abE85d73D5B2177
+-- ✅ GOOD: Using LIMIT for large result sets
+SELECT
+ hash,
+ block_time,
+ gas_used,
+ gas_price
+FROM ethereum.transactions
+WHERE block_time >= TIMESTAMP '2024-10-01'
+ORDER BY gas_price DESC
+LIMIT 1000
```
-
Using `block_date` is often better than using `block_number` or `block_time` as there are less distinct values to check against. Use `block_date` during joins where it is possible.{" "}
+### Use Curated Data Tables
-This join order optimization is related to the size of the tables. **You should always join the smaller table on the bigger table**. In this case, `ethereum.transactions` is much bigger than `uniswap_v3_ethereum.Pair_evt_Swap`, so we should join `uniswap_v3_ethereum.Pair_evt_Swap` with `ethereum.transactions`.
+Leverage Dune's [curated tables](https://docs.dune.com/data-catalog/overview) which are pre-computed and optimized rather than use raw logs and traces.
-Inner joins are generally faster than outer joins. If you can use an inner join instead of an outer join, it will improve query performance.
+### Use Materialized Views
-### Notable Exceptions
+[Materialized views](https://docs.dune.com/query-engine/materialized-views) are a way to store the results of a query in a table that can be queried like any other table.
+This is useful when you have a query that takes a long time to run, as you can re-use the results without having to re-execute your whole query.
-A notable exception to the general rule of using sequentially ordered columns is the Solana dataset `account_activity`, which is ordered by `account_keys` rather than `block_time`. This allows for utilizing the min/max values for `account_keys` when building queries based on raw Solana data.
+## Query Performance Troubleshooting
-## Additional Tips for Writing Efficient Queries on DuneSQL
+- Monitor query execution time in the Dune interface
+- Check the query plan for full table scans (run EXPLAIN ANALYZE YOUR_QUERY)
+- Consider query complexity vs. result value trade-offs
-In addition to leveraging the columnar storage format and using sequentially ordered columns, as discussed in the previous section, here are more tips to help you write efficient queries on DuneSQL:
+## ❌ DON'Ts: Query Anti-Patterns to Avoid
-1. **Limit the columns in the SELECT clause**: Only request the columns you need, as it reduces the amount of data the query engine needs to process.
+### 1. **DON'T** Use `SELECT *` on Large Tables
-2. **Use the LIMIT clause**: If you are only interested in a specific number of rows, use the LIMIT clause to avoid processing more data than necessary.
+Avoid selecting all columns, especially on transaction and log tables.
-3. **Leverage partition pruning**: If your data is partitioned, use partition keys in the WHERE clause to help the query engine prune unnecessary partitions and reduce the amount of data scanned. In Dune almost all tables are partitioned by time and/or block number.
+```sql
+-- ❌ BAD: Select all columns unnecessarily
+SELECT *
+FROM ethereum.transactions
+WHERE block_time >= TIMESTAMP '2024-10-01'
+
+-- ✅ GOOD: Select only needed columns
+SELECT hash, from_address, to_address, value
+FROM ethereum.transactions
+WHERE block_time >= TIMESTAMP '2024-10-01'
+```
-4. **Filter early and use predicate pushdown**: Apply filters as early as possible in the query to reduce the amount of data being processed. This takes advantage of predicate pushdown, which pushes filter conditions down to the storage layer, reducing the amount of data read from storage.
+### 2. **DON'T** Create Unnecessary Subqueries
-5. **Use window functions**: Window functions can be more efficient than self-joins or subqueries for computing aggregations over a set of rows related to the current row.
+Avoid nested subqueries when JOINs or window functions work better.
-6. **Avoid using DISTINCT when possible**: DISTINCT can be computationally expensive, especially on large datasets. Trino has implemented a range of [**approximate aggregate functions**](/query-engine/Functions-and-operators/aggregate#approximate-aggregate-functions) functions that can be used to get a good estimate of the distinct values in a column with a much lower computational cost.
+```sql
+-- ❌ BAD: Inefficient correlated subquery
+SELECT
+ b1.number,
+ b1.gas_used,
+ (
+ SELECT AVG(b2.gas_used)
+ FROM ethereum.blocks b2
+ WHERE b2.time >= TIMESTAMP '2024-10-01'
+ AND b2.number <= b1.number
+ AND b2.number > b1.number - 100
+ ) AS avg_100_blocks
+FROM ethereum.blocks b1
+WHERE b1.time >= TIMESTAMP '2024-10-01';
+
+-- ✅ GOOD: Use window functions instead
+WITH gas_used AS (
+ SELECT
+ number,
+ gas_used,
+ AVG(gas_used) OVER (
+ ORDER BY time
+ ROWS BETWEEN 99 PRECEDING AND CURRENT ROW
+ ) as avg_100_blocks
+ FROM ethereum.blocks
+ WHERE time >= TIMESTAMP '2024-10-01'
+)
+SELECT
+ number,
+ gas_used,
+ avg_100_blocks
+FROM gas_used
+```
-7. **Use `UNION ALL` instead of `UNION`**: If you're combining the results of multiple queries, use `UNION ALL` instead of `UNION` to avoid the overhead of removing duplicate rows.
+### 3. **DON'T** Use ORDER BY Without LIMIT on Large Results
-8. **Careful with CTEs**: CTEs are inlined in each place where they're referenced (they're not precomputed) -- and so using CTEs multiple times, or combining CTEs in CTEs can lead to huge query plans.
+Sorting large result sets is expensive.
-9. **Optimize data types**: Use appropriate data types for your columns, as it can improve query performance by reducing the amount of data processed. For example, `varbinary` operations are faster than `varchar` operations as the data is compressed and encoded. Always use the smallest possible data type for your columns.
+```sql
+-- ❌ BAD: Ordering large result set without limit
+SELECT hash, gas_price
+FROM ethereum.transactions
+WHERE block_time >= TIMESTAMP '2024-10-01'
+ORDER BY gas_price DESC
+
+-- ✅ GOOD: Add LIMIT when ordering
+SELECT hash, gas_price
+FROM ethereum.transactions
+WHERE block_time >= TIMESTAMP '2024-10-01'
+ORDER BY gas_price DESC
+LIMIT 10000
+```
-10. **Only order when necessary**: Ordering results can be computationally expensive. If you don't need ordered results, avoid using `ORDER BY`.
-11. **Always use the actual data while filtering. Do not use functions on the filter columns**: For example, if you want to filter on a date, do not use `date_trunc('day', block_time) > '2022-01-01'`. Instead, use `block_time > '2022-01-01'`. The first example will not be able to use the min/max values of the `block_time` column to skip entire parquet files or row groups within files while scanning through a table, while the second example will. The same goes for other functions, such as `substr`, `lower`, `upper` etc.
+## Summary
-12. **Use the `EXPLAIN` command**: Use the `EXPLAIN` command to understand how the query engine processes your query. This can help you identify potential performance bottlenecks and optimize your queries accordingly.
+Efficient DuneSQL queries on Dune require understanding the platform's time-partitioned architecture. Focus on:
-13. **Check your execution statistics**: Click on "last run x seconds ago" to see the execution statistics for your query. This can help you identify potential performance bottlenecks and optimize your queries accordingly.
-
-
-
\ No newline at end of file
+1. **Time filtering** with time fields for partition pruning
+2. **Column selection** to leverage columnar storage
+3. **Strategic joins** with proper filter placement
+4. **Avoiding anti-patterns** that prevent optimizations
\ No newline at end of file
diff --git a/quickstart.mdx b/quickstart.mdx
index 53ff0992..2ff53f62 100644
--- a/quickstart.mdx
+++ b/quickstart.mdx
@@ -14,6 +14,12 @@ This guide will fast-track your understanding of how to **query**, **visualize**
Dune offers many features, but this guide focuses on the essentials you'll need to get started. For more detailed guides, check out our [Analytics Guidelines](/learning-resources) and [Data Tables](/data-catalog/) sections.
+## How you can work with Dune data
+
+Dune provides multiple ways to access and transform blockchain data, whether you're querying directly in our web app, connecting via API, using SQL transformation tools, or accessing data through our Datashare offering:
+
+
+
### **Prerequisites:**
* **Dune Account:** You'll need a Dune account to follow along. If you don't already have one, [sign up here](https://dune.com/auth/register).
diff --git a/web-app/bug-bounty.mdx b/web-app/bug-bounty.mdx
index 99209d15..32adb907 100644
--- a/web-app/bug-bounty.mdx
+++ b/web-app/bug-bounty.mdx
@@ -11,7 +11,7 @@ Please note, it is essential to provide a video or script reproducing the issue.
We do not accept email submissions of vulnerabilities. All reports must go through the form above to ensure they are tracked and processed appropriately.
-We aim to review and respond to submissions within 14 days. While we understand your eagerness to hear back, emailing us for updates will not speed up the review process. We appreciate your patience and cooperation as we work through each submission thoroughly.
+We aim to review and respond to submissions within a month. While we understand your eagerness to hear back, emailing us for updates will not speed up the review process. We appreciate your patience and cooperation as we work through each submission thoroughly.
Repeated inquiries for updates via email may result in disqualification of the submission to ensure fairness and efficiency for all participants.
We value transparency and will work with you to resolve any legitimate issues found. Your efforts help us maintain the security and trustworthiness of our platform. Thank you for your contribution to our security.
diff --git a/web-app/dashboard-minting.mdx b/web-app/dashboard-minting.mdx
deleted file mode 100644
index 7539457b..00000000
--- a/web-app/dashboard-minting.mdx
+++ /dev/null
@@ -1,38 +0,0 @@
----
-title: Mint Dashboards
-description: Mint Dashboards on Dune to Support Wizards
----
-
-The best way to support wizards is to mint their dashboards on Dune. We've built an integration with Zora to mint any "moment" from a Dashboard. Mints cost 0.000777 ETH (roughly $2).
-
-You can mint by going to any dashboard and clicking the blue "mint" button in the top right.
-
-
-
- **If you're a creator looking to enable minting on your dashboards, you must link your wallet to your Dune account [by going to account settings](https://dune.com/settings/wallets).**
-
-## Minting a Moment
-
-When you click mint, you'll see this modal popup. The key interactions are labeled below:
-
-
-
-You'll have to connect your Ethereum wallet, if you don't have one yet you can install [Metamask](https://chromewebstore.google.com/detail/metamask/nkbihfbeogaeaoehlefnkodbefgpgknn?hl=en) and then fill it with ETH from Coinbase or some other onramp. **Minting occurs on Base, so make sure to [transfer your ETH to Base to mint](https://help.coinbase.com/en/coinbase/other-topics/other/base).**
-
-After you select an amount and click mint, Dune will take a screenshot and create the token on Zora for you to mint. This may take 5-10 seconds, but you will then see a prompt in your wallet that looks like this:
-
-
-
-This is showing you the number of tokens you will receive, and how much ETH you are spending + gas fees. After you confirm, it should only take a second before you see a successful mint screen:
-
-
-
-Congrats - you've now saved an onchain moment in an onchain momento forever! Thank you for supporting the community at Dune.
-
-## Claiming Rewards as a Creator
-
-If your dashboard gets minted, log into [zora.co/manage](https://zora.co/manage) with the wallet that is connected to the dashboard owner's dune account to claim your rewards.
-
-Dune takes the creator referral fee, and the first minter referral is not set/taken - [learn about Zora rewards structure here](https://support.zora.co/en/articles/2509953).
-
-Each time someone mints off a dashboard, a unique token id is created on the Dune [zora collection on Base chain](https://basescan.org/address/0xdc6b3687d5d9ada19faad9e782e3ece41e1da7ba). The ipfs URI contains the details about the Dune dashboard and creator. Anyone can mint more of a specific moment (token id) by going to that tokens page on Zora, but we don't currently support minting other people's moments in the dune UI.
diff --git a/web-app/query-editor/query-scheduler.mdx b/web-app/query-editor/query-scheduler.mdx
index a0bff29e..6f848587 100644
--- a/web-app/query-editor/query-scheduler.mdx
+++ b/web-app/query-editor/query-scheduler.mdx
@@ -7,7 +7,7 @@ description: Learn how to leverage the power of query scheduling for a more reli
Queries on Dune usually execute when a user triggers an automatic or interactive execution. This means that if you have a dashboard that is not frequently viewed, the data displayed on the dashboard may be outdated and execution of the queries will only be triggered once a user views the dashboard. Especially for dashboards that contain resource-intensive queries, this can lead to long loading times for the viewer.
-To keep your dashboard up-to-date and to ensure that your queries are executed reliably and in a timely manner, you can schedule them to run at a specific time and frequency. Scheduled queries can be run on medium and large query engines, which will require credits. Credit costs are the same as any other query execution on Dune, you will pay 10 credits for a medium tier execution and 20 credits for a large tier execution.
+To keep your dashboard up-to-date and to ensure that your queries are executed reliably and in a timely manner, you can schedule them to run at a specific time and frequency. Scheduled queries can be run on medium and large query engines, which will require credits. Credits are consumed based on actual compute resources used for each scheduled query execution.
### How to Schedule a Query
diff --git a/zz_unused/conduct-network-analysis.mdx b/zz_unused/conduct-network-analysis.mdx
index cc60d7cb..ee405103 100644
--- a/zz_unused/conduct-network-analysis.mdx
+++ b/zz_unused/conduct-network-analysis.mdx
@@ -101,7 +101,7 @@ import community as community_louvain
from networkx import community
import matplotlib.pyplot as plt
import plotly.graph_objects as go
-import dotenv, os, json
+import json
from dune_client.types import QueryParameter
from dune_client.client import DuneClient
from dune_client.query import QueryBase
@@ -153,7 +153,7 @@ For the Dune API, obtain an API key:
/>
+ 
+
+
+
+
+ 
+
+ 
+
+ 
+
+ 
+
+
+ 
+
+ 
+
+ 
+
+ 
+
+ 
+
+