How to Create TDE File in Tableau
Slow-loading dashboards can be a source of constant frustration, but Tableau offers a powerful solution with its Data Extract feature. Creating a Tableau Data Extract (TDE) is one of the most effective ways to speed up your dashboards and improve overall performance, especially when working with large or slow data sources. This article will walk you through exactly what a TDE is, why you should use one, and how to create and manage them step-by-step.
What is a Tableau Data Extract (TDE)?
A Tableau Data Extract, or .tde file, is a compressed, local copy of a portion or all of your original dataset. Think of it as a highly optimized snapshot of your data, stored in a special format that Tableau can query very quickly. Instead of sending complex queries to a live database every time you interact with your dashboard, Tableau queries this small, fast, local file instead.
It's important to note that from Tableau version 10.5 onwards, the underlying technology for extracts was upgraded from the TDE engine to the Hyper engine, creating files with a .hyper extension. While the file extension has changed, the core concept and the term "extract" remain the same. Since many people still search for and refer to "TDE files," we'll use that term, but rest assured the principles and steps apply to the newer, even faster .hyper files as well.
The main benefits of using an extract are:
Speed: Querying a local, optimized Hyper file is nearly always faster than querying a remote database over a network. This makes your dashboards, filters, and calculations feel much more responsive.
Reduced Database Load: Since Tableau isn't constantly pinging your main database, you reduce the processing load, which is a big relief for your IT department and other users who rely on that database.
Portability and Offline Access: You can package an extract within a Tableau Workbook (.twbx file) and work on it anywhere, even without an internet connection. This is perfect for analyzing data on the road or during presentations.
Extract vs. Live Connection: Which One Should You Choose?
When you connect to data in Tableau, you have two primary options: create an Extract or maintain a Live connection. Neither is strictly better, the right choice depends entirely on your needs.
When to Use a Live Connection
A live connection queries your database directly. Every time you filter, sort, or interact with an element in your dashboard, Tableau sends a query to the source and waits for the data to come back.
Use a live connection when:
The data changes frequently, and you need second-by-second accuracy (e.g., a stock market dashboard or a real-time operational monitor).
Your underlying database is already extremely fast and optimized for analytics (e.g., Google BigQuery, Snowflake, or Amazon Redshift).
Your company has a strict policy against storing data locally.
When to Use an Extract
An extract takes a snapshot of the data and stores it locally. This is generally the default and recommended option for most analytical scenarios.
Use an extract when:
Your dashboard performance is slow or sluggish.
You are working with large datasets from transactional databases, Excel files, or Google Sheets.
You want to minimize the impact on your production database systems.
You need to work on your dashboard offline or share it with people who don't have direct access to the database.
Your data only needs to be updated periodically (e.g., daily, weekly, or monthly).
How to Create a TDE File in Tableau Desktop: A Step-by-Step Guide
Creating an extract is a straightforward process built directly into Tableau's data connection workflow. Let's walk through it.
Step 1: Connect to Your Data Source
First, open Tableau and connect to your data. This can be anything from an Excel file, a CSV, a Google Sheet, or a database connection like Microsoft SQL Server or PostgreSQL. For this example, we'll connect to an Excel file containing sales data.
Once you select your data source and drag your table(s) onto the canvas, you will land on the Data Source page.
Step 2: Select the "Extract" Radio Button
In the top right corner of the Data Source page, you'll see two options under "Connection": Live and Extract. By default, Tableau might choose one for you, but you can always change it.
Click on the Extract radio button. You'll notice that the options below it become active. This is where the real optimization happens.
Step 3: Edit the Extract Settings (The Most Important Step!)
Simply selecting "Extract" offers a performance boost, but clicking the "Edit" link next to it unlocks the ability to make your extract significantly smaller and faster. This is a critical step for working with massive datasets.
Here are the key settings you can configure:
Add Filters to Reduce Data Volume
This is the most impactful setting. You probably don't need to analyze 10 years of historical data. By adding a filter at the extract level, you can tell Tableau to only pull in the data you need.
Click "Add..." and choose a field to filter on. A common choice is a date field. For example, you could set a filter to only include data from the past two years. This can reduce your data from millions of rows to a much more manageable size without affecting what you need for your current analysis.
Aggregate Data for Visible Dimensions
If you don't need row-level detail, you can pre-aggregate your data. For example, if your dataset contains daily sales figures but your report only ever looks at monthly totals, you can check "Aggregate data for visible dimensions." This will roll up the data to the lowest level of detail shown in your visualization, dramatically reducing the extract size. Be cautious with this, as you can't drill back down to the more granular, non-visible dimensions once the extract is created.
Hide Unused Fields
Your dataset might have 150 columns, but maybe you only use 20 in your dashboard. You can click the "Hide All Unused Fields" button before creating your extract. Tableau will exclude these hidden columns from the extract file, making it smaller and quicker to process.
Step 4: Generate the Extract
Once you are happy with your extract settings, navigate to a new worksheet (e.g., click "Sheet 1" at the bottom). As soon as you try to build a visualization or drag a field onto the view, Tableau will prompt you to save the extract file.
This is the moment the TDE (or .hyper) file is actually created. Tableau will query the original data source, apply your filters and aggregations, and then build the optimized extract file.
Step 5: Save a Copy of Your Extract File
A "Save As" dialog box will appear. Choose a location on your computer, name your file, and click "Save." The file will be saved with a .hyper extension. This process can take a few minutes if the dataset is very large. Once finished, you are now working with the lightning-fast extract instead of the live connection.
How to Refresh Your Data Extract
An extract is a snapshot, meaning it won't reflect the latest changes in your original data source until you refresh it. Refreshing is easy to do and can even be automated.
Full vs. Incremental Refresh
Full Refresh: This option replaces all the data in your extract with the data from the source. It rebuilds the entire extract from scratch. To do this, right-click on your data source in the Data pane, navigate to Extract, and select Refresh.
Incremental Refresh: This is a more efficient method that only adds new rows to the extract. To use this, you need a column that tells Tableau which rows are new, such as a Transaction ID, a sequentially numbered Row ID, or a timestamp. In the "Edit Extract" settings, you can configure the incremental refresh settings by specifying which field should be used to identify new rows.
Scheduling Refreshes on Tableau Server or Cloud
For most business dashboards, the goal is automation. When you publish a workbook that uses an extract to Tableau Server or Tableau Cloud, you have the option to set up a refresh schedule. You can configure it to refresh daily, weekly, or at any interval you choose. This completely automates the process, so your viewers always have access to up-to-date data without anyone needing to manually refresh and republish the workbook.
Final Thoughts
Creating Tableau Data Extracts is a foundational skill for anyone looking to build high-performance dashboards. By moving from a live connection to a selectively filtered and aggregated extract, you can take a slow, cumbersome workbook and transform it into a fast, responsive analytical tool that users will love.
While tools like Tableau offer incredible power, the initial setup - connecting data sources, configuring extracts, building dashboards, and setting up refresh schedules - still involves manual effort. At Graphed, we’ve designed a system that automates all that heavy lifting. You connect your data sources once, and then use natural language to ask for the dashboards and reports you need. Instead of manually creating extracts and refreshing data, you can just ask, "Show me my Shopify sales versus Facebook Ads spend for the last quarter," and get a real-time, interactive dashboard in seconds, with all data connections handled for you automatically.