Overview

Data Model Pipeline are the core components used by Cosmos to extract data from your data sources, clean the data using transformations, and then place the data in your reporting database. As changes are made to extensions you can then publish these changes and refresh the data in your reporting database, so the changes are available to your users. This article covers how to publish and run pipelines in the Data Model section of Cosmos.

Details

The Publish Pipeline and Run Pipeline actions are in the Manage section of the Data Model.

Publishing a Data Model Pipeline

Clicking on Publish Pipeline will package up everything that has been built into the data model and generate a set of pipelines. This will include all data sets and transformations that have been specified in the Cosmos data model. Until the Publish Pipeline button has been pressed, any changes that have been made in the data model will not be present in the current pipeline runs.

Manually Running a Data Model Pipeline

Clicking on Run Pipeline will manually run an update of the data model pipeline. This way, you do not need to wait for the next schedule refresh of the data. If you clicked Publish Pipeline, meaning changes were made to the data model, a full load of the data will be run. Otherwise, an incremental load of the data will take place.