Data Sources

A data source is a connection to a warehouse, database, or data lake. Measured supports all warehouses supported by GX Core, plus additional connectors.

Supported data sources

| Data source | Type string | Notes | |---|---|---| | Snowflake | snowflake | Key-pair and password auth | | PostgreSQL | postgres | Also works for RDS, Aurora, AlloyDB | | BigQuery | bigquery | Service account JSON auth | | Databricks SQL | databricks | | | Redshift | redshift | | | Microsoft SQL Server | mssql | | | Oracle Database | oracle | Oracle DB / ERP / Fusion | | SAP HANA / Datasphere | hana | | | ClickHouse | clickhouse | | | Trino | trino | | | MySQL | mysql | | | SQLite | sqlite | Local development and testing | | Lakehouse | iceberg / delta | Apache Iceberg & Delta Lake tables |

Adding a data source

Via the UI

  1. Navigate to Settings → Data Sources
  2. Click Add data source
  3. Select the type and enter connection details
  4. Click Test connection to verify
  5. Click Save

Via the API

curl -X POST https://dq.your-company.com/api/v1/datasources \
  -H "Authorization: Bearer $DQ_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "data": {
      "type": "datasource",
      "attributes": {
        "name": "Production Snowflake",
        "type": "snowflake",
        "connection_string": "snowflake://user:pass@account/db/schema"
      }
    }
  }'

Credential security

Connection strings are stored encrypted at rest in your Postgres database using AES-256-GCM. The encryption key is derived from your SECRET_KEY environment variable.

Credentials never leave your network, Measured connects directly to your warehouse at checkpoint run time, without routing through any external service.

Connection pooling

Measured maintains a small connection pool (default: 2 connections per datasource) to avoid overwhelming your warehouse during parallel checkpoint runs. Adjust with the MAX_CONNECTIONS_PER_DATASOURCE environment variable.

Network access

Your Measured instance must have network access to reach your warehouses. For:

  • Cloud warehouses (Snowflake, BigQuery, Redshift): ensure outbound HTTPS (443) is allowed from your Measured host
  • Internal databases (Postgres, MySQL): ensure Measured can reach the database host/port, typically via VPC peering or private networking