If you have used Data Flows in Azure Data Factory even once until few days ago, you must have noticed that as soon as you start creating the data flow, it asks you if you want to create a Mapping Data Flow which is basically used for data transformations OR you would like to create a Wrangling Data Flow that are used for sampling purpose. If you do it now, ADF will no longer ask you this option and by default goes to Mapping Data Flows so where the Wrangling option has gone? Let’s see –
Before we locate this, let’s grab one concept here quickly that Wrangling Data Flows are now being implemented by Power Query. If you are familiar with Power BI or have used Power Pivot in Excel then you would know that this is the language used by them to connect to data source and get the data. Also know as M query or Mashup language. And this is now what Microsoft is promoting in terms of Data Wrangling so with this understanding, let’s find out Wrangling Data Flows OR say Power Query in ADF –
You may find this at multiple places like – under the Data Factory Resources –
Or when you click on “+” symbol to add a new resource –
and a third place, under the activities –
If you look at all these snapshots in this post until now, at each place it is suffixed with “(Preview)” and yes, this is true…it’s just that Microsoft has renamed this feature (Data Wrangling) and put it as a separate item but it continues to be a preview feature. However, considering that after a long time, it has got Microsoft’s attention and focus so we can now hope for its General Availability.
Not just this, the biggest USP of this feature is now its scalability…Yes, with all this new focus, it has been announced as “Data Wrangling As Scale With Power Query“. Let’s explore all the new stuff about this in our next article