Tag: SSDT

Workaround for Building Azure Data Warehouse using Visual Studio

When creating Azure Data Warehouse, I have found the limitation that the Visual Studio SSDT SQL projects does not support Azure Data Warehouse. It causes much pain to the data warehouse development without support on source control and nice code organisation as those offered by SSDT SQL Projects.

Fortunately, I have found a trick to workaround this limitation. It may not feel like a elegant approach, but I found it did do well in my work.

The trick is to create a middle Azure SQL Database that is supported by Visual Studio SSDT SQL projects. We create the Azure Data Warehouse schema in Visual Studio SSDT but target at and deployed to the middle Azure SQL Database.

m2

Then, we use the Data Warehouse Migration Utility to migrate the schema to the Azure Data Warehouse.

 

m1

One limitation of this tool is that this tool does not support the migration of stored procedures. We have to manually copy the stored procedure script from visual studio SSDT and run against the Azure Data Warehouse.

Advertisements
Setup an Azure Dev VM for Testing Power BI + SQL Server 2016 Integration

Setup an Azure Dev VM for Testing Power BI + SQL Server 2016 Integration

This blog post walks through the steps to setup an Azure dev vm for testing Power BI + SQL Server 2016 integration:

  1. Provision the Azure VM using SQL Server 2016 developer image
  2. Configure the SSAS/SSRS services
  3. Install development software
  4. Install sample databases
  5. Integrate SSRS and Power BI Service
  6. Install and configure Personal Power BI Gateway
  7. (Optional) Install and configure Enterprise Power BI Gateway to support Direct Query and live SSAS connection

1. Provision the Azure VM using SQL Server 2016 developer image

02

Assuming we already have an Azure account, we can log on the Azure portal and select to create a new Virtual Machine using SQL Server 2016 RTM Developer image. On the Basics tab of the “Create virtual machine” wizard, we need to provide the name of the VM we aims to create and the username/password of the local administrator of the VM.

03

Next, we need to select the size of the virtual machine from a broad range of options offered by Azure platform. Microsoft recommends DS3 or above for a production environment, but as we are going to use this VM only for development and testing, we can choose something smaller. I chose DS11_V2 personally for my environment.

04

We then need to configure the settings for Storage, Network, etc., we can use the default settings if there is no specific requirement applied.

05

Lastly, we need to configure the SQL Server settings. I have set the SQL connectivity as “Public (internet)” and enabled SQL Authentication in order to access the SQL server from client computers.

06

After the VM is provisioned, we can log onto the VM and we will find that the SQL Server 2016 database engine along with SSIS, SSAS and SSRS components have all been installed and set to run.

08

2. Configure the SSAS/SSRS services shipped with the image

2.1 (Optional) Change SSAS from Multidimensional model to Tabular model
In the VM we have created from the SQL 2016 developer image, the SSAS is deployed in the Multidimenional mode by default. If we need to work with the SSAS Tabular model, we can change the deployment mode in the SSAS configuration file which is located at C:\Program Files\Microsoft SQL Server\MSAS13.MSSQLSERVER\OLAP\Config\msmdsrv.ini.

09.PNG

Open the file, and change the DeploymentMode from 0 to 2. Before the change takes effect, we need to restart the SSAS services on the SQL Server Configuration manager.

10

After the SSAS services is restarted, we can see the SSAS is running using Tabular model.

11

2.2 Configure Reporting Services
Next, we need to configure the reporting services through the Reporting Services Configuration Manager wizard.

12

The steps to configure SQL Server 2016 reporting services are pretty similar to the configuration of previous versions. We should be able to find tons of guides on the Google. The only configuration specific for SQL Server 2016 is related to the Power BI integration which will be covered later in the blog post. After the Reporting Services is configured, we should be able to access the reports web portal to manage SSRS reports.

13

3. Install development tools

Some dev tools we need to install on the VM, including SSDT, Power BI Desktop and DAX Studio:
The link for SSDT: https://msdn.microsoft.com/en-us/library/mt204009.aspx
The link for Power BI Desktop: https://powerbi.microsoft.com/en-us/get-started/
The link for DAX Studio: http://daxstudio.codeplex.com/

4. Install Databases Sample

A number of sample databases can be deployed in the VM as test data sources to support BI development:

5. Integrate SSRS and Power BI Service

The 2016 version of SQL Server Reporting Services comes with a very cool feature that allows SSRS developers to  pin SSRS charts onto Power BI dashboard. We can take the following steps to enable this feature.

Firstly, we need to enable the SQL Server Agent.

14

Then, we go to the Reporting Services Configuration Manager and register the Report server with the Power BI service.

14a

After the report server is registered with Power BI services, we go to Report Manager, locate the report we want to pin to Power BI dashboard, and check whether stored credentials are used in the report, if not, we need to use stored credentials in the report.

14c

14d

Now, we are ready to pin the SSRS charts to Power BI dashboard.

14e

6. Install and configure Power BI Personal Gateway

As we have our SQL Server engine and sample databases up running in the Azure VM, we can now connect to those databases to build Power BI reports and publish them to the online Power BI services. However, to allow the online Power BI services to refresh the data sources hosted in the Azure VM, we also need to setup the Personal Gateway.

Firstly, we need to download the Personal Gateway from the Power BI web site: https://powerbi.microsoft.com/en-us/gateway/, and install the app on the Azure VM. The installation process is very simple, and we should not meet any problem here. After the Personal Gateway is installed, we will be asked to sign in the Power BI online service with our Power BI account to setup the connection between the Personal Gateway and the Power BI services.

17

Now we can connect our Power BI reports to the data sources hosted in the Azure VM and schedule refresh on the report.

18
21

To test the refresh, I have built a report using the Adventure Works products table which displays the number of products in each product category.

22

I have then made a change to the project table, i.e., added a new product.

23

After refreshed the dataset on Power BI online portal,

24

the report on the online Power BI was updated to reflect the change.

25

7. (Optional) Install and configure Enterprise Power BI Gateway to support DirectQuery and live SSAS connection

Now we have setup the Personal Gateway that empowers Power BI services to refresh/load data from the data sources hosted in the Azure VM. However, there are some important features missing from the Personal Gateway,  i.e., the support of DirectQuery and live SSAS connections. Those features are only available in the Enterprise edition of Power BI Gateway.

16

We can download the Enterprise Gateway from Power BI website and run the installation application on the Azure VM. After the installation process is completed, we need to log on the Power BI services to configure the data source connection and service account/credentials.

29

Now the Enterprise Gateway should start to be working. However, when I was trying to access the datasets on Power BI services, I got the following error:

30

After a little bit further researching on how SSAS live connection works, I’ve learnt that the stored credentials I configured in the Data Source Settings (snapshot above) only setup the initial connection to the SSAS services, and the actual queries are executed based on individual user’s permissions. For the SSAS services to know which user the current query is from, along with the query itself, the EffectiveUserName property (in the format of UPN) of the Power BI user, who started the query through UI interaction, is also passed through Enterprise Gateway to the SSAS. The UPN passed from Power BI needs to match a windows account in the local Active Directory (which is the Power BI user’s mapping account in the on-premise Active Directory). SSAS will then check the Analysis Services roles of the matched windows account and decide whether to execute the query.

To get the SSAS live connection working in our Azure VM environment, the setup need to meet the following requirements:

  1. The Azure VM has to be in an Active Directory Domain
  2. A windows account need to be created in the Active Directory for each Power BI services user.
  3. The UPN of the Power BI user account need to be mapped to the UPN of the windows account
  4. The windows account need to be added into the appropriate Analysis Service role

To meet those requirements, we need take the following steps:

Firstly, we need to promote the Azure VM to be a Domain Controller so that the Azure VM itself will be in the domain.

31.PNG

Then, we need to create a windows account (e.g., linxiaodevlinxiao.ma) with UPN like (linxiao.ma@linxiaodev.local) which is the mapping account for the Power BI user.

41

and added the account into the “Users” role in SSAS.

35

Next, we need navigate to the “Map user names” panel on the Power BI portal and added a rule to map the UPN of a Power BI user account to the UPN of the windows account we have created.

40

Now, all the settings have been done and the datasets and reports on the Power BI services should start to get data through from the live SSAS data source.

38