Application

CloudBasix Zero-ETL

Amazon RDS SQL Server, Aurora, PostgreSQL, AzureSQL Database, AzureSQL Managed Instance, Google CloudSQL, Oracle, MariaDB, MongoDB, DocumentDB;  Snowflake, Google BigQuery, other

Release Notes

See Limitations

Version 13.39-13.40, CLOUDBASIX Zero-ETL (aka Intercloud)

1. Applicable to target Snowflake, BigQuery, Aurora/RDS PostgreSQL: introduced an option to reseed target tables without recreating (from both Analyze Report and to reseed all tables on replication task level).

Version 13.38, CLOUDBASIX Zero-ETL (aka Intercloud; separated non-SQL Server focused functionality from the CloudBasix Intercloud (SQL Server edition) product, incremented version to 13.38 in line with latest CloudBasix Intercloud version; product name is now Zero-ETL).

1. Applicable to target Snowflake, source SQL Server: resolved an issue with sourcing from databases containing special characters in the name.


Version 4.17-4.18 (update to 4.0 is a prerequisite)

  1. Applicable to target Snowflake, BigQuery, PostgreSQL: introduced an option to reseed tables - via Analyze report or reseed entire replication task (all tables) - without recreating of  target tables, which is important in scenarios in which the user used to access the target is denied DB owner permission after initial seeding.

Version 4.16 (update to 4.0 is a prerequisite)

  1. Applicable to target BigQuery, Snowflake: resolved additional encoding issues, which can affect replication of certain characters in rare cases of source data encoding scenarios.

Version 4.15 (update to 4.0 is a prerequisite)

  1. Applicable to target BigQuery, Snowflake: resolved encoding issue, which can affect replication of certain characters in rare cases of source data encoding scenarios.

Version 4.14 (update to 4.0 is a prerequisite)

  1. Applicable to target BigQuery: fixed ongoing changes processing intermittent issue which was resulting into reporting in logs ERROR: Unable to process pending changes ERROR: invalid byte sequence for encoding "UTF8": 0x00\n Where: unnamed portal parameter $NN.

Version 4.13 (update to 4.0 is a prerequisite)

  1. Applicable to all targets (Snowflake, BigQuery, PostgreSQL etc.) : added necessary handling for the rare case, where included for replication tables are dropped or renamed. Replication for the rest of the tables will remain intact. Affected tables remain available for explicit exclusion.
  2. Applicable to all targets (Snowflake, BigQuery, PostgreSQL etc) : Update SMTP/SES alerting integration to support latest SES requirements.

Version 4.12 (update to 4.0 is a prerequisite)

  1. Applicable to target BigQuery: introduced BigQuery data set partitioning (Hourly/Daily/Monthly) for CSD Type 2 replication type DWH, which helps to substantially reduce ongoing BigQuery read queries per transaction cost.

Version 4.11 (update to 4.0 is a prerequisite)

  1. Applicable to target Snowflake: resolved a bug in analyze report for target snowflake databases with long list of schemas, which could cause the report to not generate.
  2. Resolved a bug applicable to all targets - when export/upload limit, under Replication Tasks\Tasks is set to a low limit but (with the goal to lower instance CPU utilization), if the number of jobs exceeds the limit substantially, in certain scenarios some export/upload processes could stop running.

Version 4.10 (update to 4.0 is a prerequisite)

  1. Applicable to target BigQuery: resolved an issue with replication failing to track changes for source tables, if schema/column definition contains reserved keywords such as LIMIT).

Version 4.9 (update to 4.0 is a prerequisite)

  1. Applicable to target Snowflake: resolved an issue with replication failing for tables with zero records, in certain scenarios.
  2. Applicable to target Google BigQuery: error alerts fail to be generated for certain errors, such as ERROR:Error occurred processing pending changes:ERROR: invalid byte sequence for encoding “UTF8”: 0x00\n Where: unnamed portal parameter $10|ERROR

Version 4.8 (update to 4.0 is a prerequisite)

  1. Applicable to target Google Cloud (GCP)/BigQuery: resolved a region validation issue in in /api/v1/CreateReplicationTask (validation is delegated to GCP).


Version 4.7 (update to 4.0 is a prerequisite)

  1. This update package installs a self-signed SSL certificate, which enables seamless https calls to the API. Instance restart might be required for SSL certificate to become active.
  2. Improvements in /api/v1/CreateReplicationTask, /api/v1/AlterReplicationTask  & /api/v1/ReplicationTaskStatus


Version 4.4 - 4.6 (update to 4.0 is a prerequisite)

    1. Improvements in /api/v1/CreateUser, /api/v1/AlterUser, /api/v1/ActivateInstance, /api/v1/DeactivateInstance, /api/v1/AnalyzeReplicationTask, /api/v1/ReseedTable, /api/v1/ReseedReplicationTask.


Version 4.3 (update to 4.0 is a prerequisite)

  1. Applies to all target and sources (Amazon RDS SQL Server, Aurora, PostgreSQL, AzureSQL Database, AzureSQL Managed Instance, Google CloudSQL, Oracle, MariaDB, MongoDB, DocumentDB;  Snowflake, Google BigQuery, other): addressed a bug which can be triggered in cases where table field length exceeds 64 characters. Reported error is: ERROR:Error occurred processing pending changes:ERROR: column "columnNameWithLenghtGreaterThan64Characters" specified more than once


Version 4.2 (update to 4.0 is a prerequisite; version 4.2 won't show as available if checking for updates on versions prior to 4.0)

  1. Released Windows 2022 based AMI 4.2 (bundle AMI defaulted to IC, with AIRS3 turned off).
  2. Minor bug fixes and cosmetic changes (export/upload drop down list no longer shows archived jobs; dashboard & analyze report instructions showing, when no jobs exists).


Version 4.1 (update to 4.0 is a prerequisite; version 4.1 won't show as available if checking for updates on versions prior to 4.0)

  1. Fixed a bug which can be triggered during update to 4.0, if updating from a version prior to 3.25 (default for /Data Feeds/Tasks/[] is set to zero, causing data extraction processioning to not run). Fixing the value to a greater than zero, in the UI console, would address this issue without the need to apply 4.1.
  2. This update would uninstall the Tomcat9 service and remove respective folder (not removed during the update to 4.0 with goal to facilitate a rollback in case the update to 4.0 failed due to permission issues).


Version 4.0 (update to 3.25 is a prerequisite; version 4.0 won't show as available if checking for updates on versions prior to 3.25)

  1. Update to Apache Tomcat 10, Microsoft Java 17, and Spring Boot 3.0.1
  2. Improved system-wide memory management, which results into reduced memory utilization and reduces the risk of system stalling in case of instance over-utilization.
  3. Introduced Latency Monitor.
  4. Applicable to target BigQuery only: resolved a bug in the functionality of the target object naming casing (Original, All UPPER Case, All lower case) feature.
  5. Resolved a minor bug in target Export/Upload functionality, which in case of  intermittent connectivity to target, can lead to delayed synchronization of pending changes (Export/Upload processes would not run until further pending changes are generated).


Version 3.25

  1. Improved resource management on the target export/upload side, which results to substantial reduction of CPU utilization.
  2. Introduced parallel data feed tasks control (explicit max value and advanced auto-control).


Version 3.23-3.24

  1. Introduced target table naming case selection.


Version 3.21-3.22

  1. Applicable to target Snowflake: introduced auto-escaping of reserved words in auto generated target schema scripts.


Version 3.20

  1. Applicable to target PostgreSQL: resolved a bug introduced in 3.19, causing Analyze Report to fail for single target schema jobs (Schema Structure = Single (Aggregate Source Tables) ), which were created under 3.18 or earlier.


Version 3.19

  1. Applicable to target Google BigQuery: introduced support for indirect Cloud Function (ETL) calls via Load Balancer (added Cloud Endpoint input box, in addition  to Cloud URL),
  2. Applicable to target PostgreSQL: added target option Schema Structure (Multiple (Group by Source Table Schemas) | Single (Aggregate Source Tables) ), where Single (Aggregate Source Tables) is the default target schema mode in previous versions, Multiple (Group by Source Table Schemas) is a new mode, which was previously introduced for target Snowflake.

Version 3.18

  1. Applicable to target Snowflake only in this version: added target option Schema Structure (Multiple (Group by Source Table Schemas) | Single (Aggregate Source Tables) ), where Single (Aggregate Source Tables) is the default target schema mode in previous versions, Multiple (Group by Source Table Schemas) is a new mode:

2. Applicable to source side, with tracking of changes mechanism CDC, for all targets (Snowflake, BigQuery, PostgreSQL):
added monitoring for disabled CDC on source database level, with option to reseed with auto-enabling of CDC.

Version 3.17

  1. Applicable to target Google BigQuery: Introduced support of multiple BigQuery credentials support, which allows replication jobs to stream data into multiple BigQuery/Cloud Functions environment.
    Important: upon update of existing instances, connectivity to target Google BigQuery environments will be lost. Must upload Google Credential files under BigQiery\Credentials, then under each job's settings, which previously relied on a single google credential uploaded json file, assign the newly created credentials from a drop down list. After that, additional credentials can be uploaded, and selected during initial job setup.
  2. Applicable to all targets (Snowflake, BigQuery, PostgreSQL): added support (seeding and tracking of changes) for SQL Calculated Fields, which applies only to SQL change tracking mechanism Change Tracking (not supported with CDC in this version yet).

Version 3.16

  1. Introduced Google BigQuery Cloud Functions (ETL) integration (event driven triggering of Cloud Functions).
  2. Improved the Notifications/Alerts delivery mechanism, and resolved Notifications issues
    2.1 Improved the Batching mode, and detection of repetitive error alerts, to reduce the number of delivered notifications.
    2.2 Improved structure of notification content, to reduce the possibility of a notification to be detected as spam.

Version 3.15

  1. Applies to schema change tracking when target is PostgreSQL:1.1 Addressed a PostgreSQL data import into varchar data type field bug, which leads to an error if source data contain some special characters (error: value too long for type character varying({size})\n Where: COPY …, line {N}, column ….: “…\n\n”). Due to PostgreSQL import/copy command limitation, the workaround is to alter mapping of source SQL table types varchar to PostgreSQL type text.Important: Upon upgrade to version 3.15, as part of the SQL to PostgreSQL schema change tracking functionality handling, PostgreSQL data types varchar (character varying) to text, but executing: alter table {table} alter column {columnname} type text.1.2 Improved tracking of schema changes mechanism to avoid the need to recreate target tables. Schema changes are handled via executing of alter commands (for target Postgres only in this version).
  2. Fixed a bug which applies to cases in which source tables contain columns which names match reserved SQL words on respective target Snowflake, PostgreSQL, Google BigQuery/CloudSQL databases (LIMIT, STATUS).

Version 3.13 - 3.14

  1. Runtime (source side) replication logging bug fix (failing to enable CT/CDC in certain cases, which is a critical error, in certain cases could be misinterpreted and labeled as a non-critical error, and therefore failing to cause the respective replication process status to be rendered as Failed).

Version 3.12

  1. BigQuery update: minor changes related to target table naming (when option "SQL Schema As Table prefix" is selected; defaulted target naming default to "lowercase").
  2. Added additional job configuration field level help/tool tips in UI console.


Version 2.10 - 3.11

  1. Upgraded to latest Spring Boot framework to address a vulnerability (CVE-2022-22965)
  2. BigQuery enhancement: added optional prepending of target BigQuery tables with corresponding source SQL table schemas, to allow aggregation of tables with overlapping names into a single BigQuery data set.

Version 3.9

  1. Introduced [No Retention] logs option to minimize accumulating of text logs.
  2. Improved parallel multi-threaded replication handling for jobs set with high maximum of allowed parallel processes.
  3. Improved memory management.


Version 3.8

  1. Resolved minor bugs related to the BigQuery connector.


Version 3.7

  1. Added support of Google BigQuery.


Version 3.6

  1. Resolved minor bugs related to the Google CloudSQL and Aurora PostgreSQL connector.


Version 3.5

  1. Added support of Google CloudSQL and Aurora PostgreSQL.


Version 3.2-3.4

  1. Resolved minor bugs related to the Snowflake connector.


Version 3.0

  1. Introduced source RDBMS Schema change tracking and translation to Snowflake.
  2. Decommissioned the AI security extension (code name Defrost.AI for Snowflake)


Version 2.0 (code name Defrost.AI for Snowflake)

1. Introduced AI extension for real-time data security scanning.


Version 1.0

  1. Initial product release (support of Snowflake as a target only)