Power BI | Importing multiple files
Getting Started
To import multiple files from a folder, the following two steps had to be done:
- create a list of all files in the folder
- for each file: read the file and add it to the result table
When importing files with Power BI, you can do both tasks together or each task separately.
The decision, which way to go, ist done after selection the folder:
You could choose between 4 posibilities. Strictly speaking, you have to possibilities, both with the same to final steps.
In the following dialog, you will see all files an a preview of the content for each file. For excel files, you will see the sheet names and the names of the intelligent tables in the sheets.
Click on OK to start the import.
When Power Query is done with this step, you will see the result:
The previous query Samples is still there, but now with the content of all files.
Additionally, you will see four other elements:
How combining the files is done
Each query consists of a list of steps, which are process one after another. Normaly, each step is using the result (data) of the previous step, performs some modifications and has a result (data) for the next step.
So, each step is modifying the whole data of the previous step. Describing some modifications means either
- do one thing, e.g. add an additional column
or
- do something for each row in the data This means, we need some sort of a loop, like “do xyz for each row in the data
Lets see, how Power Query solves this task.
In the query Samples, exampine the Step Invoke Custom Function1
The Step if performing the M function Table.AddColumn
This functions needs 3 parameter:
- table: which is normaly the name of the prevoius step In our example #”Filtered Hidden Files1″
- newColumnName: the name for the column to be added “Transform File”
- columnGenerator: a function which is called for each row in the input table and creates the new column content each #”Transform File”([Content])
This results in the following procedure:
- for each row of the list of files (output from step #”Filtered Hidden Files1″)
- get the content of the column Content (this will be the parameter for the function call)
- call the function “Transform File”([Content]) to create the column with one parameter: the value of the column ([Content] i
Helper Queries (Required)
This is the required function to create the column content for each file
Helper queries (Optional)
For the resulting query Samples to work, only the function definition is required.
But Power Query add some additional elements, to test the function and show the result
Create a parameter used in the query Transform Sample File and define the curent value Sample File
Define a value for the parameter. Here, the first row in the list of files is used.
Create a query and use an excel workbook as input. The name of the excel file is speficied as a parameter
In this query, the previously create parameter Parameter1 is used as parameter (to much of the word parameter, i know :))
Importing multiple files with different formats
If the selected folder contains files with different format, the result is not what you may be expect:
The list of files contains all files, both csv files and xls files
When combining the files, you can select between the files. So first take a look at an csv file:
The csv file looks as expected:
But the xls files looks strange:
But lets try. Click on ok to combine all files.
But, looking at the resulting query, the data of the xls files still looks strange:
To understand this, take a look into the create transfer function:
The crucial instruction is line 2:
Source = Csv.Document(Parameter3,[Delimiter=",", Columns=10, Encoding=1252, QuoteStyle=QuoteStyle.None]),
The source document (each file in the list of files) is interpreted as csv file.
So, the xls files are also read in as csv files. This leads to the strange result.
You can fix this by adding an additional filter step in the query to select only csv files: