Category: Python/R

Execute R Scripts from Azure Data Factory (V2) through Azure Batch Service

Introduction One requirement I have been recently working with is to run R scripts for some complex calculations in an ADF (V2) data processing pipeline. My first attempt is to run the R scripts using Azure Data Lake Analytics (ADLA) with R extension. However, two limitations of ADLA R extension stopped me from adopting this … Continue reading Execute R Scripts from Azure Data Factory (V2) through Azure Batch Service →

The Tip for Installing R packages on Azure Batch

Problem In one project I have been recently working with, I need to execute R scripts in Azure Batch. The computer nodes of the Azure Batch pool were provisioned with Data Science Virtual Machines which already include common R packages. However, some packages required for the R scripts, such as tidyr and rAzureBatch, are missing … Continue reading The Tip for Installing R packages on Azure Batch →

Why Bother to Use Pandas “Categorical” Type in Python

When we process data using Pandas library in Python, we normally convert the string type of categorical variables to the Categorical data type offered by the Pandas library. Why do we bother to do that, considering there is actually no difference with the output results no matter you are using the Pandas Categorical type or … Continue reading Why Bother to Use Pandas “Categorical” Type in Python →

R Visual – Create Gartner Magic Quadrant-Like Charts in Power BI using ggplot2

In this blog post, I am going to create a R visual that renders the Gartner magic quadrant-like charts in Power BI using the ggplot2 package. A dummy dataset will be created, including three columns, the "Company" column holding the name of the companies which will be ranked in the quadrant chart, the "ExcutionScore" column … Continue reading R Visual – Create Gartner Magic Quadrant-Like Charts in Power BI using ggplot2 →

R Visual – Build Eurovision Voting Network Chart in Power BI

I have been watching Eurovision competitions for several years. I personally think the voting results from Eurovision competitions can be a very good source for the research of relationships between European countries. In this blog post, I will create a social network R visual using iGraph package and use the visual to analyse the voting … Continue reading R Visual – Build Eurovision Voting Network Chart in Power BI →

R Visual – Building Facet Grid in Power BI

The pbix file created for this blog post can be found in my GitHub here. Introduction Since Power BI started to support R visual, it has become difficult to criticise Power BI's visualisation capability because we can now take full advantage of R's powerful visualisation packages such as ggplot2 to create Power BI reports. Unlike … Continue reading R Visual – Building Facet Grid in Power BI →

Scaffolding Azure Machine Learning Experiments

*please download the source code here Microsoft has released the public preview of their newest data science service, Azure Machine Learning, that contains a collection of components to support the end-to-end machine learning solution. The Azure Machine Learning Workbench and the Azure Machine Learning Experimentation service are the two main components offered to machine learning practitioners … Continue reading Scaffolding Azure Machine Learning Experiments →

R Visual – Building Component Cycle Timeline

One common approach to detect exceptions of a machine is to monitor the correlative status of components in the machine. For example, in normal condition, two or more components should be running at the same time, or some components should be running in sequential order. When the components are not running in the way as … Continue reading R Visual – Building Component Cycle Timeline →

Extracting Features from IoT Sensor Data using R

In my previous blog I introduced the common patterns to extract features from IoT sensor data using Python. Although R is not my primary machine learning language it is becoming ubiquitous in Microsoft's data analytics ecosystem after they acquired Revolution Analytics, the major commercial distributor of R. Considering the increasing popularity of R on Microsoft … Continue reading Extracting Features from IoT Sensor Data using R →