Text File Step

The Text File step imports various text files, such as TXT, CSV, DAT, EDI, RET, REM, and BDP, to start a Data Preparation flow from the data in that file. It reads both files with delimiter characters (such as CSV) and fixed-width files, where each field is determined by the starting and ending character positions.

The Text File is in the Data Source category of the toolbar. After completing the flow, you upload files through the Process View or File Upload screen of the Dattos platform.

Importing the text file

There are two ways to load the file:

• Import File: opens the file explorer for you to select the file.

• Drag and drop: drag the file directly onto the screen.

Configure the step

In the configuration screen you will find:

• File information: identifies the name of the imported file.

• Change File: replaces the imported file. When importing a new file, the platform offers to keep the existing settings and update only the sample, or update all settings based on the new file.

• Name: identifies the name of the step. By default, it uses the name of the imported sample file, but the field is editable.

In Import Options:

• Import data starting from line: indicate the line number where the import starts bringing in data.

• Contains header?: check this if the file has a row with column names.

In Map Columns, select the pattern for the file's information structure: Single record structure or Multiple record structures.

Single record structure

Select the column delimiter, i.e., how columns are separated in the file. The options are: Fixed width, Comma ( , ), Semicolon ( ; ), Colon ( : ), Equals sign ( = ), Pipe ( | ), and Tabulation (tab).

When choosing Fixed width, enter the end positions of each column in the format 5, 10, 12. This way, the platform understands that the first column goes from position 1 to 5, the second from 6 to 10, and the third from 11 to 12.

Adjust the columns

With the delimiter defined, adjust each output column:

• Column Visibility: uncheck to disregard the column.

• Column Order: shows the order of the columns in the output.

• Column Name: shows all output columns; the names can be renamed.

• Data Type: Alphanumeric, Numeric, or Date. For Date, enter the mask pattern used by the column. The field is editable.

• Sample: displays the content of the columns from the first imported row.

Use Close to exit without saving and return to the Designer, or Save to save the settings.

Advanced settings

• File content encoding pattern: Unicode (UTF-8) or Windows (ANSI).

• Decimal separator to convert text to number: Comma ( , ) or Dot ( . ), according to the separator used in the file. For example, content "100,99" read as a number needs the Comma separator; "100.99" needs the Dot separator. In both cases, the displayed result is "100,99".

• Add data source name as column: adds a column with the step name to the sample.

• Add filename as column: adds a column with the name of the imported file to the sample.

Multiple record structures

When the file combines different column patterns in different rows, select the column delimiter from the same options: Fixed width, Comma ( , ), Semicolon ( ; ), Colon ( : ), Equals sign ( = ), Pipe ( | ), and Tabulation (tab).

When choosing Fixed width:

• Fill in Select record type position with the start and end position separated by a comma, in the format 1, 2, indicating where the record type identifier is.

• For each record type found (represented by tabs), fill in Enter end position of each column in the format 5, 10, 12. Thus, the first column goes from position 1 to 5, the second from 6 to 10, and the third from 11 to 12.

When choosing a character delimiter (comma, semicolon, colon, equals, pipe, or tab):

• Ignore double quotes as field delimiter: when checked, the platform ignores the quotes and does not consider them a field value; when unchecked, the quotes are included as a value.

• Select record type position: enter the column number that identifies the record type. For example, "1" indicates that the value that differentiates the record type is in the first column.

Adjust columns by record type

In Map Columns, configure the columns for each record type as needed, changing the name, data type, or disregarding columns for future steps:

• Select record type position: enter the column number that indicates the position of the record type.

• Select record: choose which record type you want to configure, among those identified in the file.

• Column Visibility: uncheck to disregard the column.

• Column Order: shows the order of the columns in the output.

• Column Name: shows all output columns; the names can be renamed.

• Data Type: Alphanumeric, Numeric, or Date. For Date, enter the mask pattern used by the column.

• Sample: displays the content of the columns from the first imported row.

Output settings

Define the desired outputs by indicating which records are the header and which is the content:

• Data Output Name: enter the desired name for the output.

• Structure - Header: indicate which records are the header. The order matters, as it defines the order of the created columns: the columns follow the header registration order.

• Structure - Content: define only one record as the information content.

• Add New Output: creates an additional data output.

Use Close to exit without saving and return to the Designer, or Save to save the settings.

Advanced settings

• File content encoding pattern: Unicode (UTF-8) or Windows (ANSI).

• Decimal separator to convert text to number: Comma ( , ) or Dot ( . ), according to the separator used in the file, following the same logic as the single structure.

• Add data source name as column: adds a column with the step name to the sample.

• Add filename as column: adds a column with the name of the imported file to the sample.

Related articles

• Configure Data Preparation

• Excel File Step

• OFX File Step