Academy
Data Sources

Database Stage

The Database stage connects the data preparation flow directly to a database and imports the result of an SQL query written by you. It is the most robust option for large volumes and for data that is not available in reports or APIs. Database is found in the Data Source category on the toolbar.

The main databases on the market are supported — SQL Server, Oracle, PostgreSQL, MySQL, SAP HANA, Databricks and Google BigQuery, among others. The connection can be made to the source system's own database or to an intermediate base (staging or BI), according to your company's IT policy.

Before you begin

  • Register the database credential in Settings > Credentials (category Database), using the information provided by your company's database team. Use Test connection to validate access before saving.

  • Minimum permissions: the connection user needs only read access to the queried tables.

Database without internet access: the connection can be made through site-to-site VPN/VPC — a private tunnel between the Dattos environment and your network. Contact Dattos support to assess this setup.

How to configure

  • Connector: select the registered database connection.

  • Sample records: number of records shown in the preview while building the flow.

  • SQL Editor: write the query that extracts the data — the output column names are defined by the query's own aliases. The Available parameters link shows the dynamic variables that can be used in the query, such as the execution reference date.

  • Test: runs the query and displays a preview of the returned data.

  • Save: completes the setup. Connect the stage output to the next stages in the flow.

Usage example

A finance team needs the outstanding accounts receivable invoices, which add up to millions of rows per month — too much volume for an API. In the Database stage, they select the connector for the ERP staging database and write the query with the period filter using the reference date parameter. With each run, the monthly position enters the flow directly from the database, with no intermediate file.

  • Custom integrations via Rest API and SQL connectors

  • Rest API Stage

  • ERP Stage

  • Database Export Stage

Was this helpful?