Skip to main content

This article provides an overview of the configuration settings for the following data sources in TimeXtender:

General Connection Settings

These settings are consistent across all location types.

Location

Supported location types:

  • Local file or folder
  • Azure Blob Storage or Azure Data Lake Storage (ADLS)
  • AWS S3 Bucket
  • SharePoint or OneDrive
  • Google Cloud Storage
  • SFTP

Path

Specify the path to the file(s) (e.g. CSV files) in the selected storage location. Each data connector supports only one specified path. For local files this is the full file path. For files in Azure Blob Storage or ADLS the file path should be set to filename.filetype (e.g. PostedHourLine.parquet)

Include Sub-folders

Choose whether to include files within sub-directories of the specified path. Enable sub-folders when extracting files in Azure Blob Storage or ADLS, even when the file is located in the main blob contained.

Included File Types

List file extensions (comma-separated) to include in the connection. Files with extensions not listed here will be ignored.

File Aggregation Pattern

Specify patterns to aggregate files as a single table using wildcards (* and ?). Files within the specified pattern are treated as one table.

Note: files must have the same schema to be aggregated without error during a full data load.

Metadata URI

Optionally, specify a separate file for metadata parsing instead of the main data files. If set, the Metadata URI will override the Path setting during synchronization but will be ignored during full data loads.

Note: Only one metadata file can be specified.

Location-Specific Connection Settings

Each storage location type has unique connection settings as described below.

Azure Blob Storage or ADLS Settings

Connect to Azure Blob Storage or ADLS using either a connection string or a service principal. When both are provided, the connection string takes precedence.

  • Azure Blob Connection String: The full connection string for the Azure Blob container. The connection string for the storage account can be found under Access keys in the Azure portal.
  • Azure Tenant ID: The tenant ID of the Azure service principal.
  • Azure Client ID: The client ID for the service principal.
  • Azure Client Secret: The client secret for the service principal.
  • Azure Storage Account Name: Name of the Azure storage account.
  • Azure Blob Container: Name of the blob container in Azure.

AWS S3 Bucket Settings

This connection supports Access Key ID-based authentication, with details available in AWS documentation.

  • AWS Region: The region of the S3 bucket (e.g., eu-west-2).
  • AWS Access Key ID: Access key ID to authenticate with the S3 bucket.
  • AWS Secret Access Key: Secret access key for the S3 bucket.
  • AWS Bucket: The name of the S3 bucket.

SharePoint or OneDrive Settings

These storage types are accessed using the Microsoft Graph API. Only OneDrive for Business accounts are supported (personal OneDrive accounts are not compatible).

  • SharePoint/OneDrive Client ID: Client ID of the service principal.
  • SharePoint/OneDrive Client Secret: Client secret of the service principal.
  • SharePoint/OneDrive Tenant ID: Tenant ID of the service principal.
  • SharePoint/OneDrive Drive ID: Drive ID of the file location. (Refer to the Knowledge Base for instructions on obtaining the Drive ID.)

Google Cloud Storage Settings

Supports GCM authentication via service account keys. For more information, see the Google Cloud IAM Documentation.

  • Google Credential File: Path to the Service Account private key (in JSON or P12 format).
  • Google Storage Bucket Name: The name of the Google Cloud storage bucket.

SFTP Settings

The SFTP connector allows authentication via password or public key file. If both are provided, the public key file is used.

  • SFTP Host: Hostname or IP address of the SFTP server.
  • SFTP Port: The port of the SFTP server.
  • SFTP Username: Username for SFTP authentication.
  • SFTP Password: Password associated with the SFTP username.
  • SFTP Key Path: Path to the public key file for the SFTP user.
Be the first to reply!

Reply