How to Set Up Incremental Refresh in Power BI

Working with massive datasets in Power BI can feel like trying to sip water from a firehose. Every time you hit the refresh button, you're stuck waiting, sometimes for hours, as it pulls in millions of rows all over again. Incremental refresh is Power BI's built-in solution to this problem, allowing you to update only the newest data instead of reloading everything from scratch. This article will walk you through exactly how to set up incremental refresh, step-by-step, making your reports faster, more reliable, and far more manageable.

What Exactly is Incremental Refresh (And Why Use It?)

Every standard Power BI refresh is a “full refresh.” This means Power BI wipes all the data from your report and re-imports every single row from your data source. When your dataset is small, this is perfectly fine. But when you’re dealing with years of historical sales data or millions of website events, a full refresh becomes painfully slow and inefficient.

Incremental refresh changes the game by partitioning your data into time-based chunks like days, months, or years. Instead of reloading the entire dataset, Power BI only refreshes the most recent partitions that can actually change, leaving the older, static historical data untouched. This provides some major benefits:

• Faster Refreshes: This is the headline benefit. A refresh that once took two hours could now take just a few minutes. Faster refreshes mean you get up-to-date data in front of your stakeholders more quickly.
• Greater Reliability: Long-running refreshes are more likely to fail due to network timeouts or resource limitations on the data source. Because incremental refreshes are shorter, they are far more dependable.
• Reduced Resource Consumption: Your refreshes will use less memory, CPU on the Power BI service, and processing power on your underlying data source (like a SQL server). Your data admins will thank you.
• Support for Larger Datasets: Incremental refresh unlocks the ability to work with datasets containing billions of rows, a scale that would be impossible to manage with a full refresh methodology.

Prerequisites: What You Need First

Before you get started, make sure you have a few key things in place. Attempting to set up incremental refresh without this foundation is a common source of frustration, so it's worth checking upfront.

1. A Premium or Pro License: Incremental refresh is a premium-level feature. You'll need a Power BI Pro, Premium Per User (PPU), or Premium capacity workspace to use it. It is not available on the free Power BI license.
2. A Data Source that Supports Query Folding: This is the most critical technical requirement. Simply put, query folding is Power BI's ability to translate your transformation steps (like filtering dates) into the native language of your data source (like SQL) and push that work back to the source server. Without this, Power BI has to download the entire table first and then filter it, defeating the whole purpose. Common data sources that support query folding include:
3. A Date/Time Column: Your source data table must contain a column with a Date/Time or Date data type. This column is used to partition the data and identify which records are new versus historical.

How to Set Up Incremental Refresh: A Step-by-Step Guide

With the prerequisites out of the way, let’s get into the setup process. We’ll break this down into four main stages, starting in the Power Query Editor and ending in the Power BI Service.

Step 1: Define Your Date/Time Parameters in Power Query

The first step is to create two special, reserved parameters that Power BI will use to define the refresh window. These parameters act as dynamic placeholders for the start and end dates of each data slice.

1. Open your report in Power BI Desktop and click Transform Data to open the Power Query Editor.
2. In the Home ribbon of the Power Query Editor, click on the dropdown for Manage Parameters and select New Parameter.
3. Create the first parameter, named RangeStart. This name is case-sensitive and required, so make sure it's spelled exactly right.
4. Click New to create the second parameter, RangeEnd. Again, the name must be exact.

When you're done, you should see both RangeStart and RangeEnd listed as parameters in your Queries pane.

Step 2: Filter Your Data Table Using the Parameters

Now, you need to apply these new parameters to filter the date column of the table you want to configure for incremental refresh. This step tells Power BI that you only want to load data that falls between RangeStart and RangeEnd.

1. In Power Query, select the table you are setting up for incremental refresh.
2. Find the date column you want to filter on (e.g., OrderDate).
3. Click the filter arrow on the column header, navigate to Date/Time Filters > Custom Filter....
4. The Custom Filter window will appear. Configure it as follows:

It's important to use "is after or equal to" for the start and "is before" for the end. This prevents duplicate rows by ensuring that a record existing exactly on the boundary is counted only once.

1. Click OK.

After applying the filter, you will only see the data from your sample date range in the Power Query preview. Don't worry, this is correct! You are now ready to verify that query folding is still working.

To do this, look at the Applied Steps pane on the right. Right-click the Filtered Rows step you just created. If the View Native Query option is enabled (not greyed out), then query folding is working. If it is greyed out, you need to troubleshoot - incremental refresh will fail.

Now, click Close & Apply in the Home ribbon to save your changes and return to Power BI Desktop.

Step 3: Define the Incremental Refresh Policy

With your parameters created and your filter applied, the last step inside Power BI Desktop is to define the actual refresh policy.

1. In the Desktop main window, go to either the Report or Data view.
2. In the Data pane on the far right, find your table. Right-click on it and select Incremental refresh from the context menu.
3. A dialog box will appear. Here's how to configure the policy settings:
4. Click Apply to save the policy.

Step 4: Publish to Power BI Service and Refresh

The final step is to publish your report to the Power BI Service, where the incremental refresh magic actually happens. The refresh policy does not activate within Power BI Desktop - you will only see your small sample window of data there.

1. Save your PBIX file.
2. Publish it to an eligible workspace (one that is Pro, PPU, or on a Premium capacity).
3. Navigate to app.powerbi.com and go to that workspace.
4. Find the semantic model (dataset) for your newly published report. Click the ellipses (...) next to it and go to Settings.
5. Expand the Data source credentials section and make sure Power BI is authorized to connect to your data source. You may need to edit credentials and sign in.
6. Configure the Scheduled refresh if desired, setting it to run daily or at your preferred cadence.
7. Lastly, trigger the first manual refresh by going back to the workspace and clicking the Refresh now icon next to the semantic model.

A crucial point: The VERY FIRST refresh will be slow. It has to load the entire historical dataset you defined in the archive policy (e.g., 5 years of data). After this initial load is complete, all subsequent refreshes will be fast because they will only update your smaller, more recent data window (e.g., the last 7 days).

Common Problems and Best Practices

Setting up incremental refresh can sometimes be tricky. Here are a few tips to avoid common pitfalls:

• Verify Query Folding at Every Step: This is the #1 reason incremental refresh fails. If you add a transformation after your date filter that breaks folding, the entire process won't work. Check the "View Native Query" option frequently.
• Don't Be Alarmed by Limited Data in Desktop: Remember, Power BI Desktop will only display the small slice of data defined by your static RangeStart and RangeEnd parameter values. The full historical dataset is only loaded in the PBI Service. This is expected.
• The First Refresh Test: Before doing your 5-year historical load, you could set your policy to archive just 3 months of data and run a refresh in the Service to confirm everything works. Once validated, you can update the policy to the full 5 years and refresh again.
• Confirm Data Types: Ensure your source date column and your RangeStart/RangeEnd parameters are all set to the exact same data type (Date/Time). Mixing types like Date and Date/Time can cause the filter to fail.

Final Thoughts

Setting up incremental refresh is an essential skill for any Power BI developer working with more than a trivial amount of data. By moving from full to incremental refreshes, you can create reports that are vastly faster, more resilient, and capable of handling enterprise-level datasets without issue.

While mastering specific features like this is a great way to improve your reports, we know that the real challenge lies in connecting and making sense of data scattered across numerous platforms. To make this easier, we built Graphed. It lets you link your different marketing and sales platforms in one place and then build powerful dashboards just by describing what you need in plain English, automating hours of manual reporting work so you can find insights faster.