Data Flow
Data Preparation is the ETL module of the Dattos platform. It collects, cleans, transforms, combines, and organizes information from different systems and external sources to generate reliable results for analysis, BI, and visualizations. All of this is done in a no-code interface designed for business users, replacing traditional spreadsheets with greater agility and security.
This article introduces the flow designer: the areas of the screen, how to build and configure a flow, and the features that help document and validate the result.
How a preparation flow works
A data flow is a set of connected steps, each performing a function on the information. You add, connect, and configure the steps according to the flow’s objective.
Each step can have one or more input and output ports, which are where connections between steps are created. Connections represent transitions between steps and follow a horizontal logic, from left to right.
The designer header
At the top of the screen, the process name and, from left to right, the action buttons appear:
Close: closes the flow editing screen and takes you to the process View/Execution screen, where the configuration information is displayed.
Gallery: opens ready-to-use flow templates, with search and filters by data source and result type.
Identification: opens the identification rules for the files used in the flow for automatic upload.
Reference date: appears in Turbo AI processes with a file source and defines the file column from which the reference date comes. See the “Reference date in the source” section below.
Run: takes you to the process Import and Execution screen, based on the flow created in the designer.
Options: lets you Restore template (recovers a template in Dattos .XDTT format) and Download template (downloads the flow built in the .XDTT standard).
Question mark: opens the System Manual (the “Introduction to the Data Preparation Flow” guide) in a new tab and Start Tutorial, with step-by-step instructions for the main features and a video showing how to configure a flow from scratch.
The toolbar and step categories
The toolbar lists all available steps, grouped by category and identified by color:
Data Source: file and data imports.
Preparation: data cleansing, normalization, and modeling.
Combination: data joining and merging.
Aggregation: data consolidation and grouping.
Output: data and result exports.
Adding, connecting, and removing steps
To add: drag the step from the toolbar to the canvas below.
To connect: move the step close to another one with an available output, and the platform suggests the connection; simply release it. To connect manually, click the output port of one step, drag the pointer to the input port of the other, and release it.
To remove: select the step and click the red x displayed above it, or press the Delete key. Confirm in the window that appears by clicking Confirm.
Organizing the layout
As the flow grows, the platform can reposition the steps to keep the diagram readable. There are two ways to do this:
Organize layout: button with a wand icon in the repositioning bar at the bottom of the canvas. Redraws the entire flow, preserving the order of the branches you built and keeping the entire flow within the visible area.
Organize selection: select two or more steps, right-click the canvas, and choose Organize selection. Only the selected section is redrawn in the place where it was, and the rest of the flow remains unchanged.
The same menu includes Select all (Ctrl+A) and Clear selection.
Configuring a step
To open a step’s configuration area, double-click it after adding it to the flow. The options vary according to the selected step. Details for each one are provided in the Data Preparation step articles.
Tip: steps in the Data Source category open the configuration area automatically when added to the flow.
Automatic reprocessing: the flow is reprocessed whenever a step is added, edited, or deleted, so that the data sample accurately reflects the current result.
Step context menu
Right-click any step in the flow to open the context menu:
Configure Step: accesses the specific settings for adjustments.
Edit Name: customizes the step’s display name for more intuitive identification.
Add Description: inserts information about the step. When a description already exists, the menu offers Edit name and description and Delete description.
Delete Step: removes the step from the flow.
Tip: describing the steps makes the flow easier to understand, saves time, reduces the chance of errors, and provides additional clarity in team communication.
Explanatory notes in the designer
In addition to step descriptions, you can add free-form notes in the designer to document the flow. Right-click any blank area and select Add note. A window opens where you can add useful information and insights at the most relevant points in the flow.
To edit or delete a note, right-click it and use the options displayed.
Validating with the Data Sample
The Data Sample appears in a blue bar at the bottom of the designer. Double-click it to expand it. With a configured and selected step, it displays the first 100 rows from that step in the flow.
The sample window contains:
Output port tabs: available according to the selected step.
General size information: for example, “Displaying a sample of 100 rows out of a total of 300 rows - 20 columns”.
Data sample table.
Search field: locates content within the sample.
Simulation date.
Download: exports the sample as an .xlsx file.
Sample limit: the Data Sample displays a maximum of 100 rows, regardless of the size of the data sources loaded into the flow.
Message History
To understand errors and warnings, open the history by clicking any icon in the lower-right corner of the designer. Messages are classified into three types, which can be filtered in the window:
Error: invalid step configurations.
Warning: data flow synchronization failures.
Message: step configuration editing and synchronization events.
For each record, the table lists:
Type: the message category.
Source: the step that originated the message. When not specified, it is an event affecting the flow as a whole.
Message: the message text.
Template Gallery
The Gallery contains ready-made flow templates that you can use in full or reuse in parts. Click the Gallery button in the upper-right corner to open the selection window, with search and filters by step category and use case (accounting, tax, finance, healthcare, among others).
When you click Use Template, the platform loads the template in the designer. To see more details before choosing, click the desired template area and open the screen with the filter tag, a screenshot of the template in the designer, and its description.
Tip: you can add more than one template to your process and then edit the flow to adapt it to your needs.
Configuring the Identification Rule
The Identification Rule allows the platform to automatically route each file sent through the “File Upload” and “Process Execution” screens to the correct flow. The rule uses the contents of specific columns to perform this routing.
Click the Identification Rule button to open the screen where you configure the rule for each file-type Data Source. Each source displays a status:
Valid: the input step already has an identification rule configured.
Invalid: the input step does not yet have a rule configured.
To create the rule, select the contents of a cell (the intersection of a column and a row) and click Add Rule. You can define one or more rules. When you finish, the source status changes to Valid.
Tip: hold down the Ctrl key to select more than one spreadsheet cell.
Which steps appear: only Data Sources that read files and have a data sample are included in the list. When eligible steps still have no configured rule, a red indicator appears on the Identification Rule button in the designer.
Reference date in the source
In Turbo AI processes, the reference date can come from a column in the file itself. This means that a file with multiple reference periods generates one load for each date in a single execution, each containing only the rows for that date, without requiring you to send the same file once per date.
To configure:
In the designer header, click Reference date. The button appears when the flow has at least one file source.
In the Data Sources list on the left, choose the source. The icon next to each source indicates whether its reference date has already been configured.
For each source output, choose the Date column. Only columns of the Date type are displayed.
Under When the date is not a business day in the calendar, choose Move to the next business day (the default) or Reject the date.
Click Save and Close.
Some behaviors to be aware of:
Output without a selected column: its data is not filtered and is included for all reference dates. This is useful, for example, for a reference table that applies to all dates.
More than one configured source: the dates from all of them apply.
Column that no longer exists: if the configured column disappears from the output or changes type, the window displays a warning, and you must remove the configuration from that output before you can save.
Non-file sources: databases and APIs do not provide the date. When they use the reference date in a query, they receive the date for each load.
Details about what changes when running, including the date limit per execution, are provided in the Run a process article.
Versioning and restoration
Using the Options action in the designer, you can download the flow template (.XDTT) and restore a preparation from a saved file. The complete step-by-step instructions are provided in the Versioning and restoration article.
Related articles
Excel File Step
OFX File Step
Text File Step
Run a process
Versioning and restoration