Passing Pipeline Trigger Time to Data Flows in Azure Data Factory: Use Strings, Not Timestamps!

Passing Pipeline Trigger Time to Data Flows in Azure Data Factory: Use Strings, Not Timestamps!

When working with Azure Data Factory (ADF) and the Dataverse connector, passing the pipeline trigger time into a Data Flow can be trickier than expected.

The Scenario

You want to pass the pipeline’s trigger time—using the @pipeline().TriggerTime system variable—into a Data Flow. This is often needed for auditing, filtering, or other time-based logic.

The catch? You’re using Dataverse, which communicates over the Web API and handles datetime values as strings.

The Common Mistake

In Azure Data Factory, you might instinctively define the Data Flow parameter as a timestamp or date type. But ADF doesn’t have a dedicated datetime type—only date and timestamp. So you choose one of those, thinking it aligns with your goal.

Then you hit an error.
And to make matters worse, the error message doesn’t clearly explain the real issue—it can be vague or misleading, which only adds to the confusion. This tripped me up for a while, as I assumed the problem was elsewhere.
image

The Solution

  1. Define the Data Flow parameter as a string.
    image

  2. In the pipeline, pass the @pipeline().TriggerTime system variable directly into this parameter using a pipeline expression.
    image

This small change ensures compatibility with the Dataverse connector and avoids the cryptic conversion error.

Lesson Learned

Even though it’s tempting to use date or timestamp types when dealing with datetime values in ADF, the correct approach for Dataverse is to use strings. It aligns with how the Web API expects the data and helps you avoid hard-to-decipher errors.

Comments

Post a Comment

Popular posts from this blog

Add User As Local Administrator On Domain Controller

I recently was settting up a new Microsoft SharePoint 2010 machine and had promoted the machine to a domain controller before creating my SharePoint admin accounts.  I needed to add several of my accounts to the local Administrators group.  Unfortunately after you promote a server to a domain controller you can no longer access the GUI for Local Users and Groups.  Instead I had to use the command line to add the users. Open a command promt using the "Run as administrator" function and then run the following command. net localgroup Administrators /add {domain}\{user} Note: do not include the {} brackets.
Read more

How to Create SharePoint Items with Power Automate Desktop

Image
Overview Power Automate Desktop is a great way to automate many of your daily task so you can focus on real work. A prime example of this is getting data from one place to another, especially when those data sources do not have an API such as a legacy desktop application or a file. In this example I demonstrate several ways in which an Excel sheet containing fictitious customer data could be loaded into a SharePoint list. In order to do this I first generated some fake data for my customers using Mockaroo . I then created a new SharePoint list and added some columns to match my spreadsheet. Methods My original plan was to just use the PAD recorder , which I did, but after creating that Flow I decided to find other possible ways to accomplish the task. The list below is not a complete list but should provide some ideas. The flexibility of PAD allows for an even wider range of possibilities for carrying out tasks such as these. At the end of each section I have placed the s...
Read more

Harnessing Host Form Data with PCF Controls in Model-Driven Applications

Image
Introduction This tutorial delves into integrating PowerApps Component Framework (PCF) controls with host form data within Microsoft Power Platform’s model-driven apps. This article will guide you through the necessary scripting to expose and consume formContext and globalContext from a custom table called new_Competitor. Aimed at enhancing both custom and Microsoft Form Component PCF controls, this approach ensures dynamic interactions with the host form data. Disclaimer : It’s important to note that there are various methods to retrieve data within PCF controls, including the use of WebAPI. While WebAPI provides a versatile way to access data across different entities and contexts, the approach described in this tutorial focuses on directly integrating with host form data, which can be particularly beneficial in specific use cases where immediate context is crucial. This method allows for real-time data interactions that are essential for certain scenarios, providing a streamlined...
Read more