Designing for Automation in Microsoft Fabric

In the fast-evolving world of enterprise data platforms, automation is not a luxury - it's a necessity. When working with Microsoft Fabric, especially in scalable solutions that span across multiple environments and logical layers (like ingestion, transformation, and serving), it’s essential to embed automation into your design from day one.

A crucial aspect often overlooked is how Microsoft Fabric uniquely identifies artifacts (items) across workspaces and environments. Understanding the difference between item IDs and logical IDs, and how they’re used, is foundational to building robust, automated solutions that can scale and deploy seamlessly across dev, test, and production environments.

What Are Item IDs?

In Microsoft Fabric, an item ID is a globally unique identifier (GUID) that represents a specific item in a workspace - whether it's a notebook, lakehouse, pipeline, or other Fabric artifact. This ID is visible in the URL when navigating to an item, like in this example:

https://app.fabric.microsoft.com/groups/9bcbb7d4-13f7-4bc2-a261-22be96a809dc/pipelines/82e7a8f1-e593-414d-8fab-c9b34a267772

Here, 82e7a8f1-e593-414d-8fab-c9b34a267772 is the item ID for the pipeline, and 9bcbb7d4-13f7-4bc2-a261-22be96a809dc is the workspace ID.

These IDs are also used in Fabric REST APIs, such as the List Items and Get Item operations. They're fundamental to how Fabric tracks and manages content internally.

What Are Logical IDs?

Logical IDs are different, they're Git-related and exist only for items in source-controlled workspaces.

A logical ID is a unique identifier that links a Fabric item in the workspace to its corresponding file and configuration in a Git branch. Think of it as the “anchor” between what lives in Fabric and what’s committed to source control. This makes logical IDs vital in Git-integrated workflows, especially when names or paths change across branches or environments.

You can find the logical ID in the .platform system file that Fabric automatically generates inside the item’s Git directory:

{
  "$schema": "https://developer.microsoft.com/json-schemas/fabric/gitIntegration/platformProperties/2.0.0/schema.json",
  "metadata": {
    "type": "DataPipeline",
    "displayName": "Controller - Full"
  },
  "config": {
    "version": "2.0",
    "logicalId": "bffcdc62-7e33-83b0-4dc9-0f7957777e88"
  }
}

More details: Source Code Format – Microsoft Learn

For items in non source-controlled workspaces, the logicalId will be a blank GUID:

00000000-0000-0000-0000-000000000000

Why Does This Matter?

When promoting artifacts across environments, from development to test to production, you don’t want your production controller pipelines referencing development notebooks or ingestion pipelines. This is where logical IDs shine - they enable Fabric to resolve internal references based on the Git-tracked logical structure, not hardcoded workspace or item IDs.

Handling Feature Branches and Multi-Layered Workspaces

I'm a strong advocate for using feature-isolated development workspaces and separating your Fabric solution into layers such as Storage, Ingest, Prepare and deployed across at least three environments: dev, test (PPE), and prod. This architecture is standard in mature enterprise solutions.

One common question I hear:

“Which workspace should I refer to when invoking other pipelines or notebooks from my controller pipeline?”

Within the Same Workspace

Let’s say you’re referencing another item (pipeline or notebook) within the same workspace. In that case, always reference the version from your feature workspace, not the main dev workspace.

Here’s what happens:

When editing the pipeline in the Fabric UI, you’ll see this reference:

"typeProperties": {
  "notebookId": "0e23e4cb-caf5-41bd-8161-ad34f69679ce",
  "workspaceId": "9bcbb7d4-13f7-4bc2-a261-22be96a809dc"
}

But once committed to Git, Fabric rewrites it as:

"typeProperties": {
  "notebookId": "f69679ce-ad34-8161-41bd-caf50e23e4cb",
  "workspaceId": "00000000-0000-0000-0000-000000000000"
}

What’s going on? Fabric automatically replaces the item ID with the logical ID, and sets the workspace ID to a blank GUID to indicate an intra-workspace reference. It’s elegant and powerful because it allows seamless deployment without having to manually update references between environments.

Referencing Items Across Workspaces

Now, when you need to reference items across workspaces (e.g., from a controller pipeline in the orchestration layer to an ingestion pipeline in another workspace), Fabric does not resolve logical IDs automatically. You need to provide actual workspace and item IDs - and ideally dynamically.

To automate this, check out one of my previous blog posts:
👉 Automating Fabric: Dynamically Configuring Microsoft Fabric Data Pipelines

You can also use SemPy functions in your Notebooks like:

• resolve_item_id

• resolve_item_name

• resolve_workspace_name

• resolve_workspace_name_and_id

These help dynamically fetch the correct IDs based on the environment context.

Automate with Confidence: The fabric-cicd Python Library

For deploying Git-connected Fabric workspaces, I highly recommend using the fabric-cicd Python library. Purpose-built for this exact use case, it has quickly become the preferred deployment tool among many Fabric professionals.

The library enables code-first CI/CD automation by allowing you to deploy workspaces directly from a Git repository structure. It takes care of critical deployment tasks, such as replacing logical IDs with the actual item IDs of the newly deployed artifacts.

A standout feature is its support for environment-specific configurations using a parameters.yml file. This file lets you define and programmatically override values depending on the target environment. That includes, but isn’t limited to, workspace IDs, item IDs, connection strings, connection IDs, and more.

This makes the fabric-cicd library especially powerful for multi-environment deployments where automation, consistency, and traceability are key.

Don't miss the great introductory blog post by Jacob Knightley:
👉 Introducing the Fabric CICD Deployment Tool

A Note on Variable Libraries

Variable Libraries are a great addition to Fabric and will simplify many deployment scenarios. However, they do not handle dynamic references to other Fabric items like notebooks or pipelines. For that, logical IDs (and tools like fabric-cicd) are still essential.

Final Thoughts

I hope this post has helped clarify the role of logical IDs in Microsoft Fabric and why they’re vital when working with Git-connected workspaces. Designing your solution with automation and environment promotion in mind is key to building scalable, robust, and enterprise-ready data platforms.

Keep an eye out for my upcoming (and very small) post on how to use fabric-cicd to deploy multiple interconnected workspaces (Ingest, Prepare, etc.) while preserving cross-layer references.

Until then - automate everything, and automate it smartly 😉