Introduction:

In today’s data-driven era, the exponential growth of data has transformed the way organizations operate and make decisions. The sheer volume, velocity, and variety of data generated require sophisticated tools and solutions to extract meaningful insights. Consequently, big data capabilities tools have emerged as the backbone of this transformative process, enabling businesses to harness the power of data for strategic decision-making, innovation, and gaining a competitive edge.

Moreover, these tools play a pivotal role in processing and analyzing vast datasets that traditional databases and analytics tools struggle to manage. The significance of big data tools lies in their ability to uncover patterns, trends, and correlations within massive datasets, providing valuable insights that drive informed decision-making. This not only enhances operational efficiency but also positions organizations to stay ahead in the ever-evolving landscape of data analytics.

Navigating the Big Data Ecosystem: An Overview

As organizations embrace digital transformation, they encounter diverse data types from various sources, including social media, sensors, and IoT devices. Big data tools are essential for ingesting, processing, and extracting actionable intelligence from this diverse and often unstructured data. Moreover, they empower data scientists, analysts, and decision-makers to derive meaningful conclusions, optimize processes, and identify opportunities for growth.

The need for effective big data solutions is underscored by the demands of real-time analytics, predictive modeling, and the imperative to stay agile in an ever-changing business landscape. Whether it’s streamlining operations, improving customer experiences, or gaining a deeper understanding of market trends, big data tools provide the technological foundation for organizations to turn raw data into strategic assets.

In this blog, we’ll delve into a big data capabilities through comparative analysis of prominent big data tools offered by Azure Synapse, Azure Databricks, and Data Factory exploring their features, capabilities, and how they address the evolving challenges of managing and extracting value from massive datasets in today’s dynamic business environment.

Azure Synapse: The Unified Analytics and Data Integration Hub

Azure Synapse Analytics transcends conventional boundaries by merging enterprise data warehousing and Big Data analytics into one coherent service. Furthermore, its unique blend allows for querying data with serverless, on-demand, or provisioned resources at an unparalleled scale. In addition, this synergy between data warehousing and Big Data analytics is designed to provide a unified experience across data ingestion, preparation, management, and serving.

For organizations seeking immediate business intelligence and machine learning needs, Azure Synapse Analytics is a robust choice. Moreover, it bridges the gap between these two worlds, offering a versatile ecosystem for analytics. Whether you require ad-hoc querying, data preparation, or advanced analytics, Synapse has you covered.

Azure Data Bricks: Empowering Data Science and Machine Learning

A fully managed Apache Spark service that can be used for data engineering, data science, and machine learning. It can be used to process large amounts of data quickly and easily. Azure Databricks is a good choice for businesses that need to perform complex data analysis or machine learning tasks.

It empowers you to process massive amounts of data using Apache Spark, a powerful distributed computing engine. Azure Databricks is perfect for big data processing, machine learning, and interactive data exploration. It enables data scientists and engineers to collaborate efficiently and derive valuable insights from complex datasets.

Azure Data Factory: Orchestrating Data Pipelines at Scale

In the realm of big data, orchestrating seamless data workflows is not just a necessity; it’s a strategic imperative. Azure Data Factory stands tall as a robust and versatile solution, empowering organizations to orchestrate data pipelines at scale with unparalleled efficiency and reliability.

Azure Data Factory serves as the conductor, orchestrating the movement and transformation of data across diverse sources and destinations. Its ability to integrate seamlessly with a multitude of data stores, both on-premises and in the cloud, provides the flexibility needed for modern data architectures.

Face-off: Azure Synapse vs. Azure Databricks vs. Data Factory

In the ever-evolving landscape of big data tools, the face-off between Azure Synapse, Azure Databricks, and Data Factory emerges as a critical juncture for organizations navigating the data-driven future. Let’s dissect the strengths, use cases, and unique features of each contender in this heavyweight bout. For this, I ran the same use case three times on these platforms. Each time I got different results based on their performance and cost, which is clearly shown in the graphs below.

Detail Comparison:

Spark Cluster:

Performance Measurement:

Costing Comparison:

Decision Time: Choosing the Right Tool for the Job

• If Unified Analytics and Scalability are Key: Azure Synapse takes the lead.

• For Data Science and Advanced Analytics: Azure Databricks steals the spotlight.

• When Orchestration and Scalable Pipelines Matter: Azure Data Factory emerges as the champion.

Conclusion:

In the subsequent rounds of big data capabilities and their comparative analysis, we’ll delve deeper into each contender, exploring features, use cases, and real-world applications. The big data landscape is vast, exciting, and constantly evolving. Embrace the journey, experiment fearlessly, and stay curious. Your exploration of these tools is not just an exploration of technology; it’s a journey toward transforming data into a strategic asset that propels your organization to new heights of success. Happy exploring!

The post Big Data Capabilities of Azure Synapse, Databricks, and Data Factory appeared first on Impulz Technologies LLC.

In my previous blog posts, I have discussed how to connect the custom Angular application to the Microsoft Dataverse through a web API. Today, I am going to compare Microsoft Dataverse to a traditional database management system like MySQL and discuss the main motivations behind using one technology over the other.

About Impulz Technologies LLC

Impulz Technologies, is a Silver Certified Microsoft Partner company. We specialize in the implementation, consulting and development of Microsoft Dynamics and Microsoft Power Platform. Impulz Technologies also provide staff augmentation services all across the US. For more information please contact us at 630-540-6302 or email us at info@impulztech.com

The post Microsoft Dataverse is not a database appeared first on Impulz Technologies LLC.

Hi, welcome to the Impulz Technologies LLC blog. In this third and final part of the three parts blog post series, I am going to explain the JS code to interact with the Microsoft Dataverse REST API to fetch the accounts table data.

Before going forward let’s quickly recap what we have discussed so far.

Recap

In the first two parts of the blog post we have covered:

1. How to register an application in Microsoft Azure.
2. How to extract different data elements like tenant Id, application id etc. from the Azure portal. These values are required to successfully execute the code within your environment.
3. Theoretical background of how the Azure Active Directory actually works.

If you have not read the previous blog posts, it is highly recommended to read them first from part-1 and part-2.

What we are going to cover today?

Today, we are going to going to consolidate what we have covered so far and apply those concepts in the actual code. This is the scenario we are going to cover in the code today.

• Try to login the user through a silent login API. That is single sign-on (SSO).
• If SSO failed then show the popup to the user to enter the credentials.
• Upon successful login fetch the data from the Dataverse system table ‘Accounts’.

Assumptions

• You are familiar with Angular and specifically the concepts like components, dependency injection and how it works in Angular, services, routes and http client.
• You already know Angular CLI and some of the basic commands.
• JavaScript and HTML concepts like async programming, local storage etc.
• The user going to sign in has an access the accounts table in the Dataverse environment.

From where to download the code?

The code used in this blog post has been published in the GitHub public repository. You can download or clone the code from this repo https://github.com/RealAhmadH/ng-dataverse

You can setup the code by reading the readme file of the repo.

Let’s now review the code.

src/constants.ts

This file is used to store configuration information of the Azure AD and the Microsoft Dataverse environment. You have to fill in the following in order to successfully connect the Dataverse environment:

• AZURE_APP_ID: this is the application identifier which you have configured on your Azure portal.
• AZURE_TENANT_ID: this is the tenant identifier of the Azure AD.
• DATAVERSE_BASE_URL: this is the URL of yours Dataverse environment. You can get this information from the Microsoft Power Apps admin centre.

Within the same file there exists a configuration JSON object required to connect to the Azure environment using the MSAL package. The only required configuration is the client identifier. Rest of the details are optional.

src/app/services/microsoftad.service.ts

In this file the code exists which actually interacts with the Microsoft Azure AD for Single Sign-on (SSO).

Following is the brief description of the code:

Class constructor

• In the constructor of the class, constant object is been passed as a dependency. Constant object contains important configuration information like configuration JSON object, application id, tenant id etc. required to successfully establish a connection with the Azure AD.

Confidential vs Public Client Application

• In the class constructor we initialized an object PublicClientApplication. Microsoft MSAL supports two types of clients: confidential and public. Confidential clients include secure clients like web server applications whereas public client includes desktop, mobile or browser clients. Since, we are using a browser to connect the Azure, that’s why PublicClientApplication object is used to connect to Azure.

ssoSignIn method

• The class instance method ssoSignIn is defined to try to connect the Azure AD first silently (Microsoft account Single Sign-on typically used in every Microsoft apps). If the single sign-on gets failed then it tries to launch the Microsoft login popup and ask the user to enter the credentials.

Scopes and Microsoft Azure AD

Microsoft also issues the authentication token within a single call in response. Azure AD issues token on based of scopes. It is important to understand that you can merge more than one scope within a single call but all the scopes has to be of the same resource we are trying to access. During the sign-in process, we are actually interacting with Microsoft Graph to allow us the access to the user’s profile. Within the same call we can’t ask to issue the access token for the Dataverse because Dataverse is a different Azure resource or API and it is required to repeat the same process to acquire the access token for the Dataverse scope.

Conluding remarks!

This concludes the three parts blog post series. You can apply the same knowledge and ideally the code to other JavaScript frameworks or even the plain JavaScript appkications. Similarly, authentication and authorization works more or less the same for every Azure resource. Ideally, you can extend the same code to other services like Power BI for example.

About Impulz Technologies LLC

Impulz Technologies, is a Silver Certified Microsoft Partner company. We specialize in the implementation, consulting and development of Microsoft Dynamics and Microsoft Power Platform. Impulz Technologies also provide staff augmentation services all across the US. For more information please contact us at 630-540-6302 or email us at info@impulztech.com

The post Connect the Angular with Microsoft Dataverse – Part 3 of 3 appeared first on Impulz Technologies LLC.

Hi, you are welcome to another informative blog post of Impulz Technologies LLC. This is the second part of the series of blog posts in which I am going to explain how to fetch the data from the Dataverse in a single page application (Angular application in our case).

Quick Recap

In the previous blog post I have explained the following:

1. How to register an application in Microsoft Azure.
2. How to setup API permissions in Microsoft Azure.

You can read the previous blog post from this link.

What you expect from this blog post?

In this second part of the blog posts series, I am going to explain how the Microsoft Azure Active Directory (Azure AD) authentication and authorization works and what are the available library/package options available from the Microsoft.

Accessing the resource like Dataverse web API from Azure, involves two steps – Authentication and Authorization.

Authentication vs Authorization

Authentication is the process of proving that you are who you say you are. This is achieved by verification of the identity of a person or device. Microsoft Azure AD uses OpenID Connect Protocol to authenticate the user or the device identity. Upon successful authentication the identity provider, which is Azure AD, is going to issue the identity token. The format of the identity token is JSON Web Token or simply JWT.

Authorization is the act of granting an authenticated party permission to do something. It specifies what data you’re allowed to access and what you can do with that data. The Microsoft Identity Platform uses OAuth 2.0 Protocol for the authorization. Upon successful authorization completion, Microsoft Azure AD issues the access token. The format of the access token is not standard.

There is a good tutorial exists on YouTube which explains the difference between Authentication and Authorization in the context of Azure as an identity platform.

The good is that Microsoft has developed and published a Microsoft Authentication Library which does the hard-work for you and the app developers don’t need to spend time in understanding the underlying protocols.

App types and Authentication Flows

There can be multiple types of application requiring authentication through Azure AD. Apps can be web applications hosted under a secure server environment, browser based app just like ours, mobile apps, desktop apps etc. Authentication and authorization can be performed using different flows for different application types.

In case of an Angular app, our browser is going to create a secure session with Azure AD and authenticate and authorized the Microsoft identity (corporate or school accounts in our case) to gain access to the Dataverse database.

MSAL-JS

MSAL-JS is a library available on JS package manager such as NPM. Through MSAL-JS we are going to perform the single sign-on of the user and fetch the ‘Accounts’ data from the Dataverse. First you need to install the MSAL-JS. You can install it through NPM by using the following command:

npm i @azure/msal-browser

In the next part and final part of this blog post series, I am going to explain the code layout and various configurations within the code. I am going to publish the final code on GitHub.

About Impulz Technologies LLC

Impulz Technologies, is a Silver Certified Microsoft Partner company. We specialize in the implementation, consulting and development of Microsoft Dynamics and Microsoft Power Platform. Impulz Technologies also provide staff augmentation services all across the US. For more information please contact us at 630-540-6302 or email us at info@impulztech.com

The post Connect the Angular with a Microsoft Dataverse using Web API – Part 2 of 3 appeared first on Impulz Technologies LLC.

Hi, welcome to the Impulz Technologies blog. In this multi-series blog post we are going to discuss how web developers can connect the JavaScript Single Page Application or simply SPA to a Microsoft Dataverse as back-end storage service using the standard REST APIs.

We will be discussing the following scenario:

1. User is going to sign-in or authenticate through Single Sign on (SSO) using the Microsoft organization account.
2. Once authenticated by the Microsoft Identity Management (Azure AD in our case) the web app is going to call the Dataverse Web API to display records from the Accounts table.

In this blog post we will be using the following technology stack:

1. Angular as a SPA framework.
2. Microsoft Authentication Library (MSAL) to authenticate the user’s identity and generate the access token from the Microsoft cloud. You can read more information about MSAL from https://learn.microsoft.com/en-us/azure/active-directory/develop/msal-overview

For simplicity we assume the following:

1. You already have an access to the Azure portal which can be accessed from https://portal.azure.com/.
2. You already have a Dataverse setup and a database has been initialized.
3. You are familiar with Node Package Manager (NPM) CLI. NodeJS and NPM can be downloaded and installed from https://nodejs.org/en/download/.
4. You are familiar with JavaScript and TypeScript.
5. You are also familiar with Angular and its associated concepts like components, templates and dependency injections in particular. You can find more information about Angular by visiting https://angular.io/.

Installation of NodeJS and Angular is also out of scope of this blog post.

The first step which we need to take is the app registration in the Microsoft Azure. Once the app (Angular SPA in our case) is registered within the Azure then the app can participate in Single Sign on (SSO) using the Microsoft accounts. In the section below we are going to discuss step-by-step with screen shots how the app can be registered within Azure.

App Registration in Microsoft Azure

1. In your browser login to the Azure admin portal https://portal.azure.com/. Once signed in successfully it is going to look like below:

2. From the Azure postal click on the App registrations and then click on the New registration as shown below:

3. On the new app registration form enter any name you like. Also, on the same form under the redirect URL, select Single Page Application (SPA) in the drop-down and enter the redirect URL. Since, we will be using and by default Angular works on port 4200 that’s why we have mentioned http://localhost:4200 as a redirect URL. This is the same URL to which Microsoft Azure will be redirected upon the successful user authentication.

And then click on Register button to complete the registration process.

4. Copy the application id and store it somewhere like in a Notepad for later use.

5. Click on API permissions on the left hand menu and then click on the Add permission button as shown below:

6. On the Add permission form search for Microsoft Dynamics CRM in the application list and click on it.

7. On the Dynamics CRM application form, please make sure that Delegated permissions is selected and then select user_impersonation from the permissions list and then click Add permissions button.

This completes the registration and the configuration of the app registration in Microsoft Azure. Now, you have to complete one additional step on Azure portal – make a note of the tenant id and copy it somewhere like Notepad. If you have multiple Azure subscriptions then you need to ensure you have already selected the correct subscription from the list. Then click on the tenant properties and copy the tenant id for later use.

This completes the first part of the two parts blog series. In our next blog post same day next week, we are going to discuss how an Angular app can connect to the Dataverse to consume the data.

About Impulz Technologies LLC

Impulz Technologies, is a Silver Certified Microsoft Partner company. We specialize in the implementation, consulting and development of Microsoft Dynamics and Microsoft Power Platform. Impulz Technologies also provide staff augmentation services all across the US. For more information please contact us at 630-540-6302 or email us at info@impulztech.com

The post Connect the Angular App with a Microsoft Dataverse through Web API – Part I of 3 appeared first on Impulz Technologies LLC.