Microsoft Fabric for Dynamics 365: What You Need to Know
Microsoft Fabric is changing the way organizations operate with their data, centralizing ingestion, transformation, visualization, and insights so organizations can make informed decisions and plan for the future.
In this blog, we will dig into Microsoft Fabric's capabilities, highlighting its core components, integration with Dynamics 365, and operational workflows.
Let’s explore the specifics of Microsoft Fabric and its role in shaping contemporary data analytics.
What is Microsoft Fabric?
Microsoft Fabric is a comprehensive platform that streamlines data analytics workflows. It empowers organizations with a consolidated platform to efficiently manage and analyze data. This unified end-to-end solution acts as a 'One-Stop shop' for data-related tasks, from data ingestion and transformation to visualization using Power BI.
Its cohesive platform enables effortless transitions between key functionalities such as:
- Data Engineering
- Warehousing
- Data science
- Real-time analytics
- Data factory
- Power BI.
Fabric's integration with a specialized 'Data Lake' further enhances its capabilities, storing data in versatile formats like Delta, Parquet, CSV, and JSON. This Data Lake serves as a centralized repository accessible to multiple analytic engines, providing a unified source of data akin to OneDrive.
Understanding Microsoft Fabric
Microsoft's strategy with Microsoft Fabric involves centralizing data into what they term a "OneLake." Each aspect of Fabric, like data engineering or data warehousing, has its own associated Data Lake. This system allows for multiple "Lake Houses" and workspaces within it.
Data can be imported into the one Lake from sources like Dynamics 365 or Dataverse, or linked via shortcuts to external file systems such as Azure ADLs, S3, or Dataverse. These shortcuts act as pointers to data stored elsewhere, eliminating the need for redundant copies. Microsoft says it plans to expand shortcut options in the future. The beauty of this setup is it eliminates the need for separate ETL processes for different tasks like data science or reporting.
All processing occurs within Fabric, whether for machine learning, reporting in Power BI, or other analytics tasks, utilizing tools like Data Factory for ETL and Power BI for visualization. Data is ingested or shortcutted into OneLake, allowing users to process and use it across various analytics and reporting functions.
The Six Components of Microsoft Fabric:
Microsoft Fabric comprises six core components, each offering distinct capabilities and functionalities tailored to streamline data management and analytics tasks.
These six components enhance productivity, collaboration, and innovation within data analytics workflows. This provides organizations with a comprehensive and integrated platform to manage and derive insights from their data effectively.
Let’s take a closer look at each:
1. Data Engineering
Data engineering within Microsoft Fabric provides robust tools and capabilities for managing and processing data efficiently, including:
- Lakehouse: Represents a collection of files, folders, and tables that form a database over a data lake. It utilizing Spark or SQL engines for processing, including enhanced capabilities for asset transactions.
- Fabric Notebook: A multi-language interactive programming tool with rich functions, allowing code and markdown authoring similar to a Jupyter Notebook. It supports running and monitoring Spark jobs, visualizing results, and collaborating with team members.
- Apache Spark Jobs: Enables running parallel jobs within the application, where a job can consist of multiple tasks set up in notebooks.
- Data Pipeline: Used for coordinating data movement and transformation, integrating functionalities from Azure data factory. This tool streamlines data workflows and can be utilized within both data engineering and data integration contexts.
These components highlight the versatility and efficiency of data engineering tasks in providing a cohesive environment for data processing, collaboration, and management.
2. Data Factory
The data factory component within Microsoft Fabric offers powerful features for orchestrating data workflows and transformations. Here are the key highlights:
- Data Flow Gen 2: A low-code interface designed to ingest data from numerous sources and perform transformations efficiently. It also simplifies complex data processing tasks with its intuitive interface.
- Data Pipeline: Enables the coordination of data movement and transformation processes, streamlining data workflows across different stages.
By utilizing data factory, users can manage and orchestrate data workflows, from ingestion to transformation, leveraging low-code capabilities.
3. Data Warehousing
The Microsoft Fabric data warehousing functionality offers comprehensive capabilities akin to a traditional enterprise data warehouse.
Here's a breakdown of the key features:
- Traditional Functionality: Supports full transactional T-SQL capabilities expected from an enterprise data warehouse, allowing for robust data management and querying.
- Pipeline and Data Flow Gen 2: Integrates with data pipelines and data flow generation 2 for efficient data movement and transformations within the warehouse environment.
- SQL Analytics Endpoint: Each warehouse includes a SQL analytics endpoint, enabling querying of Delta table data using T-SQL.
Microsoft Fabric's data warehousing supports mirroring of data sources, including Azure SQL, Snowflake, or Cosmos DB, providing flexibility and scalability for data storage and querying needs.
These functionalities within Microsoft Fabric's data warehousing module empower users to manage and analyze large volumes of data effectively, leveraging familiar SQL capabilities and integration with data pipelines.
4. Real-time Analytics
Within Microsoft Fabric, the real-time analytics component offers a range of functionalities tailored for handling live data streams efficiently.
This includes:
- Event Stream: Provides a centralized platform for capturing, transforming, and routing real-time events to designated destinations within a no-code experience. This streamlining of event management simplifies real-time data processing tasks.
- KQL Database: Houses information in a format optimized for executing KQL queries. This facilitates quick and efficient querying of real-time data.
- KQL Query Sets: Enables the execution and manipulation of KQL queries, allowing users to run queries, view results, and manipulate query outputs directly within Data Explorer and the associated database.
Users can effectively manage and analyze real-time data streams, perform complex queries, and derive actionable insights—all within the integrated environment of Microsoft Fabric's real-time analytics module.
5. Data Science Experience
The data science experience provides powerful tools for conducting experiments and training machine learning models efficiently:
- Experiment Management: The experiment is the primary unit for organizing machine learning runs. Each run corresponds to a single execution of machine learning model code. This allows for systematic organization and tracking of experiments.
- Machine Learning Models: These models are files trained to recognize specific patterns based on the desired machine-learning tasks. Fabric's data science environment supports the development and deployment of machine learning models to achieve various objectives.
By utilizing the data science capabilities within Microsoft Fabric, users can conduct experiments, track model executions, and deploy machine learning solutions.
6. Business Intelligence (BI)
The business intelligence component encompasses robust reporting and dashboarding tools designed to facilitate data analysis and visualization.
Here is a key highlight:
- Power BI Integration: Utilizes Power BI, a well-known reporting and dashboarding tool, to empower users to create insightful reports and interactive dashboards. Regardless of your experience with Power BI, the integrated experience enhances data visualization capabilities and integrates with other Fabric functionalities.
By leveraging the business intelligence capabilities of Microsoft Fabric, users can harness the power of data visualization to gain actionable insights, create compelling reports, and drive informed decision-making.
Creating Workspaces within Microsoft Fabric
In Microsoft Fabric, workspaces function as containers within the OneLake, offering environments for organizing and managing data-related tasks with flexibility and customization. There are two primary approaches to creating workspaces:
Using Predefined Experiences:
Within Fabric's predefined experiences like data science, data factory, and real-time analytics, users can access ready-made environments tailored to specific tasks. These experiences include pre-configured components such as:
- Data pipelines
- Data flow Gen 2
- KQL databases
- Event streams
- Machine learning experiments
- and more.
Users can leverage these experiences directly to initiate and manage data projects efficiently.
Creating Custom Workspaces:
Alternatively, users can create custom workspaces from scratch within Fabric by accessing the "New" option in the workspace interface. This method allows users to select and configure specific components based on their requirements. The options include everything included in the list above plus these additional ones:
- Machine learning experiments
- Notebook creation
- Report generation
- Spark job definitions
- Data warehousing components like lakehouse creation or additional machine learning models.
Interconnecting Dynamics 365 and Microsoft Fabric:
To effectively integrate Dynamics 365 with Microsoft Fabric, you have to use specific methods in the Power Apps Maker Portal to route your data.
There are two approaches when completing this process:
Using PowerApps Maker Portal: Link to Microsoft Fabric
- Access the Power Apps Maker Portal and navigate to the tables section on the left-hand side.
- Click on "Analyze" from the top menu bar and select "Link to Microsoft Fabric."
- This action brings in all tables from Dataverse. It also lets you add specific Dynamics 365 Finance and Operations tables associated with the portal.
- You can choose which Dynamics 365 Finance and Operations tables to include within the Fabric environment, enhancing control over data integration.
IMPORTANT NOTE: Please be aware that this method will import ALL tables from the Dataverse.
Using Azure Synapse Link within Power Apps Maker Portal
- Within the Power Apps Maker Portal, locate and pin the "Azure Synapse Link" option to the left-hand menu if not already visible.
- Click on "Azure Synapse Link" and select the Microsoft Fabric (One Lake) link associated with the desired workspace.
- Navigate to "Manage Tables" by clicking on the ellipsis (...) next to the One Lake link.
- This method allows users to choose, manage, and add tables from both Dataverse and D365 Finance and Operations.
IMPORTANT NOTE: To utilize this method, you must have previously configured an Azure Synapse Link for Dataverse to enable data storage and analytics integration.
Both approaches offer flexibility in integrating Dynamics 365 with Microsoft Fabric.
The first method establishes a direct link from the Power Apps Maker Portal, enabling users to choose specific tables for integration, but downloads all tables from the Dataverse. The second method leverages the Azure synapse link setup, providing more customized control over table selection within the Fabric environment.
After setting up the integration, you can access the designated workspace within Microsoft Fabric to manage and utilize the interconnected data effectively.
Stoneridge Software is Here to Help
For more information or to explore Microsoft Fabric further, please reach out to Stoneridge Software. Whether you're interested in a deeper dive into Fabric's capabilities, need assistance with implementation, or want to understand how Fabric can centralize your data effectively, our team can help.
Under the terms of this license, you are authorized to share and redistribute the content across various mediums, subject to adherence to the specified conditions: you must provide proper attribution to Stoneridge as the original creator in a manner that does not imply their endorsement of your use, the material is to be utilized solely for non-commercial purposes, and alterations, modifications, or derivative works based on the original material are strictly prohibited.
Responsibility rests with the licensee to ensure that their use of the material does not violate any other rights.