CLOUDBASIC for RDS SQL Server Read Replicas and Disaster Recovery (DR)

Release Notes

IMPORTANT: When applying an upgrade (without help from CloudBasic support) from a version below 12.0 to version 12.0 or above, wait until the update process logs you out of the console, redirects you to a login screen, and a count down is completed. At this point, you only need to leave off the root address IP/URL, which will lead to the appearance of a login screen with a new look. Then, login.

All upgrades to 12.0 or later: upon login after upgrade, the replication schedules execution service would appear stopped (to allow you to validate the upgrade). Click   in the upper right corner (in version 12.5 and above the execution service panel will appear expanded if the service is stopped), then start the service: 

Version 12.235-12.237

  1. Improved query timeout management during initial seeding (starting with lower timeout and proceeding with with automatic retries with automatic increase of time out, which allows to discover issue reported in the logs sooner).

Version 12.234

  1. Introduced filtering of replication jobs by source and name in dashboard and list of jobs.
  2. Introduced moving of tables between primary and secondary, secondary to secondary jobs, and moving to a new auto-created secondary, without the need to exclude/include, which avoids reseeding from being triggered. 
  3. Improved reseeding navigation handling in the list of replication jobs – reseeding status is indicated in the list, facilitating easier management of multiple job reseeding.
  4. Added support for import of p12 certificates (in UI console) containing multiple individual certificates (in previous versions only one of the individual certificates was imported, and manual import into OS certificates storage was required).

Version 12.229-12.233

  1. Resolved dynamic tables discovery caching bug related to dynamic tables handling.
  2. Resolved a bug related to primary-secondary jobs with dynamic handling in HA cluster, which can result into redundant processing of dynamically discovered tables, and ultimately into increased latency.
  3. Added [Validate Primary Job Exclusions] to secondary jobs, as a mean to resolve redundant tables processing in primary-secondary setups, caused by a previous bug triggered during initial setup of primary-secondary jobs. Newly created primary-secondary jobs would not need that validation/fix applied.

Version 12.227-12.228

  1. Fixed a bug which can get triggered in cases with dynamic tables (frequently dropped and recreated), causing replication processes to run for extended period of time, appearing as stalled, with status Exception no Timeout.

Version 12.225-12.226

  1. Introduced automatic creation of secondary jobs along with creating of primary jobs, with the option to designate secondary to handle dynamic tables discovery.
  2. Resolved a bug in the recently introduced dynamic tables discovery functionality.
  3. Resolved a bug in Temporal Tables schema change tracking, related to the recently introduced dynamic tables discovery.
  4. Improved the initial seeding functionality to tolerate schema changes during the process run.

Version 12.223-12.224

  1. Improved handling of S3 Data Lake jobs in HA Cluster, to avoid possible upload of redundant data when HA Cluster instances communication is interrupted, or when a cluster instance is shut down and booted back up. S3 User/IAM Role download permissions required (system json file).
  2. Resolve a bug which for Parquet output format only, SCD Type 2, can trigger naming of the Operation Type column to not be in line with the rest of the system fields.

Version 12.219-12.222

  1. Minor bug fixes.

Version 12.218

  1. Improved logging in the area of dynamically discovering new tables.

Version 12.216-12.217

  1. For [X] One-time replication w/ automatic seeding without change tracking activation (api/CreateReplication) method of replication, added enabling of Triggers and Constraints (which for other methods is part of Promote-To-Primary) as part of the Finalization process. Prior to this update, constraints were supposed to be enabled manually.
  2. Fixed a bug related to Column Store non-PK Indexes, which can trigger data replication to fail with error reported in logs “Change Tracking is not enabled for Table…. Skipping replication”.


Version 12.213-12.215

  1. Further improved logs reporting for jobs involving dynamic dropping/recreating of tables. Certain operations previously classified as errors, are not reported as warnings (i.e. Warning: Table scripting/replication failed. Table no longer exists).
  2. Introduced system status overwrite (processes which successfully handle exceptions reported in previous process runs can go back and overwrite CompletedWithErrors and Failed statuses to CompletedWithWarnings), as a measure to reduce noise (unnecessary escalating of internal processing exception handling), to prevent generating of unnecessary alerts and indicating of both non-critical and critical errors on dashboard.
  3. Related to (2) above, switched default alerting mode from Real-Time to Batched, with default 300 secs delay (in line with the default Latency (RPO/RTO) monitoring default of 5 mins). Multiple errors are now batched and a single delayed alert generated, with voiding/excluding of queued alerts with overwritten status with the specified period of time (default 300 secs).
  4. For S3 Data lake jobs SCD Type 2: introduced [X] Add Operation Type, with default ON/TRUE which (similarly to the Redshift SCD Type 2 jobs), adds column system operation indicating the type of operation (INSERTS|UPDATE|DELETE).
    In /api/CreateS3Replication & /api/AlertS3Operation the tag is <AddOperationType>.

Version 12.211-12.212

  1. Improved logs reporting for jobs involving dynamic dropping/recreating of tables. Certain operations previously classified as errors, are not reported as warnings (i.e. Warning: Table scripting/replication failed. Table no longer exists).
  2. For S3 and Redshift jobs, added [ ] Alert on Successful table schema changes. It is defaulted to False for newly created jobs. It can be turned on in UI under job settings, or over API via calling of /api/AlterS3Replication or /api/AlterRedshiftReplication. The tag name is <AlertOnSuccessfulTableSchemaChanges> 
  3. Introduced new tags to /api/ReplicationStatus:
           <TimeSinceLastCtEndTimeInSeconds> – time in seconds since the process with ID reported in LastCtPId ended (status can be Failed, Success, CompletedWithErrors, COmpeltedWithWarnings )
        <TimeSinceLastSuccessfulCtEndTimeInSeconds> - time in seconds since the last process with status in (Success, CompletedWithErrors, CompletedWithWarnings ).

Version 12.210

  1. Resolved bugs related to the newly introduced in 12.202 (#1) functionality/handling of replication jobs with frequent dynamic dropping and creating/recreating of tables.

Version 12.209

  1. Introduced Execution Service Job Start Delay management in UI (defaulted value, for all instance sizes, to 400 ms).
  2. Fixed a bug in schema change tracking (which could trigger comparing of constraints for excluded tables and other objects).

Version 12.207-12.208

  1. For S3 jobs, CSV output file format, added Tab as a delimiter available for selection, and converted the file extension drop down to a type-in field (default .csv).
    A new tag <CSVFileExtension> is introduced to support above in /api/CreateS3Replication and /api/AlterS3Replication
  2. For new replication initialization, optimized the initial process of identifying tables with timestamp PKs (previously affecting the process for DBs with long list of tables; time reduced from minutes to millisecond).
  3. Resolved bugs related to the newly introduced in 12.202 (#1) functionality/handling of replication jobs with frequent dynamic dropping and creating/recreating of tables.
  4. Resolved a bug related to setting of table(s) for reseeding (manual request for reseeding in Analyze reports, or as part of including a table to the replication job) – as latency was reduced in previous versions – this bug could have resulted to reseeding request being missed.

Version 12.205-12.206

  1. Fixed bugs and further improved the new functionality introduced in 12.204, items #2.
  2. Improved discovery/handling of tables moved from exclusion to inclusion list (handled as part of data replication processes, outside of general tracking of schema changes), handled in line with the newly introduced handling of newly discovered tables (see 12.204, item #2.).

Version 12.203-12.204

  1. Introduced a new folder naming type DayHourEqualValue(day=yyyymmdd/hour=hh) for S3 jobs SCD Type-2  (applicable to FolderStructure in {OneFolderPerTablePerYearMonthDay, OneFolderPerTablePerYearMonthDayHour} only)
  2. Improved discovery/handling of newly created tables (handled as part of data replication processes, outside of general tracking of schema changes)
  3. Introduced a new replication job option [ ] Drop tables which exist only on replica
  4. Resolved a bug in job settings [Update] -> [Apply General Settings to All Replication Jobs] which could cause Redshift/S3 jobs staging part of the replication to switch from StagingBuffer=ReplicaDb  to TempDb (leading to failure in uploading data to Redshift/S3), and replication type TablesOnly to AllObjects (which is unnecessary overhead).

Version 12.201-12.202

  1. Improved handling of replication jobs with frequent dynamic dropping and creating/recreating of tables. General schema change tracking no longer even have to be turned on, for new tables to be detected, created/recreated on replica and seeded. 
  2. Introduced a new feature (inactive by default): [ ] Drop tables which exist only on replica (will be added to API in 12.203)
  3. Fixed a bug related to replication jobs with source Azure SQL DB (specifically related to DB_ID: “When used with Azure SQL Database, DB_ID may not return the same result as querying database_id from sys.databases. If the caller of DB_ID is comparing the result to other sys views, then sys.databases should be queried instead.“).

Version 12.200 (Mar 25, 2021)

  1. New AMI, containing all OS security updates as of Mar 24, 2021, release. No product functionality changes compared to 12.150.
  2. Windows Service RDSWinSvc.exe has been renamed to CloudBasicService.exe (when upgrading from older versions, the upgrade will handle un-installation/re-installation of the service). If the process fails due to OS/AD permission changes form the original setup,  contact Support.
  3. Applicable to newly launched images (starting from 12.200) – application location has been changed to c:/program files/cloudbasic – if installing 3rd party tools ingesting text logs, need to point the tool to the /Logs folder under the root application folder.

Version 12.145-12.150

  1. Introduced source/replica connection pool system and manual management. Improved replication performance.
  2. Bug fixes in schema replication (related to SET ANSI_NULLS ON & SET QUOTED_IDENTIFIER ON)
  3. Improved Promote DB Replica to Primary functionality (in both UI and API).
  4. Bug fixes related to the newly introduced in 12.144, secondary jobs functionality.
  5. Improved replication handling for Azure SQL and Google CloudSQL for SQL Server.

Version 12.143-12.144

  1. Introduced secondary continuous replication jobs option, allowing excluded tables from primary to be added to secondary jobs with alternative schedules and replication options.

Version 12.140-12.142

  1. Introduced Add New Tables without PKs to Reseeding Schedule option, which allows discovered newly added tables without PKs to be automatically added to an existing reseeding schedule, or to auto-activate a pre-configured schedule.

Version 12.138-12.139

  1. Upgrading from older version to 12.138 or later, would default StagingBuffer to TempDB for all jobs.
  2. In dashboard, paused jobs are not listed under active, then followed by disabled.
  3. Improved UI navigation for Notifications Emailing Filter, and password reset.
  4. Improved the monitoring of system DB process functinality to avoid sending of unnecessary alerts in case of i.e. “SystemDB Transaction Log Full”, if the process recovers and core functionality is not affected.

Version 12.136-12.137

  1. Fixes related to creating Azure SQL Database (and Azure Managed SQL Server Instances) to Amazon RDS SQL Server Read Replicas (to achieve intercloud Reporting, intercloud Disaster Recovery; near no downtime migration from Azure SQL Database and managed Azure SQL Instances to Amazon RDS SQL Server Enterprise and Standard editions).
  2. Added a work around fix, to address a bug in Azure SQL Database (Arithmetic overflow error converting expression to data type int), which gets triggered when sourcing data from Azure SQL Database tables with above ~3TB size (per table).

Version 12.135

  1. Introduced connections SSL certificates management in UI console. Connections level SSL certificates can now be installed without RDP (remote desktop) to the OS.
  2. New product images 12.135 and above will come with the Amazon RDS default (as of Jan-11-2001) SSL certificate rds-ca-2019-root.pem. Update to 12.135 and above will also install rds-ca-2019-root.pem.
  3. Introduced monitoring for execution service critical events, which can trigger execution service forced restart.
  4. Introduced an additional EventLogs type Execution Service Logs.

Version 12.133-12.134

  1. Fixed a bug in schema compare V1/V2 related to drop/rename column operation (introduced in version 12.119  which eliminated the need to run schema compare second time in follow up data replication process to handle a DDL change detected in previous process). After synchronizing drop/rename column, the follow up operation was not being process form the queue of pending changes.   
  2. Enhancements in S3 Data Lakes functionality:
    2.1 If [ ] Add table schema prefix to file name (AddTableSchemaPrefix in API method /api/CreateS3Replication) is not selected/false, a check for tables with same names but different schema prefixes will be performed and prevent the job from being initiated. 
    2.2 Introduced Snappy as a compression type for Parquet files (in addition to GZip). If compression Snappy is selected the file type will be generated as {filename}.snappy.parquet
    2.3 Introduced a new Folder Naming Format option (FolderNamingFormat in API method /api/CreateS3Replication) with possible values Default  (YYYY/DD/MM/HH), KeyEqualValue (year=YYYY/month=MM/day=DD/hour=HH). Applicable to SCD=Type2 and FileFormat=OneFolderPerTablePerYearMonthDay or FileFormat=OneFolderPerTablePerYearMonthDayPerHour.

Version 12.132

  1. Fixed a bug in /api/CreateS3Replication related to <TablesToInclude>All<TablesToInclude> 
  2. Added a new S3 Data Lake partitioning type [One Folder per Table per Year/Month/Day/Hour] (OneFolderPerTablePerYearMonthDayHour in API method /api/CreateS3Replication.
  3. Added 2 new parameters to S3 Data lake options:
    [X] Add table schema prefix to file names (AddTableSchemaPrefix in API method /api/CreateS3Replication). Default value is False. If upgrading from older version to 12.132 and above,  backward compatibility is ensured via defaulting this value for existing jobs to True).
    [ ] Convert data types to string (ConvertDataTypesToString in API method in /api/CreateS3Replication). Default value is False. If upgrading from older version to 12.132 and above, the value will be defaulted to True, to ensure backward compatibility.

Version 12.131

  1. Fixed a bug related to establishing a connection handshake (with exported from SQL Server or RDS SSL certificate, imported into EC2’s certificate storage) when RDS.force_ssl=1 in RDS parameter groups is enabled. 
  2. New product images (12.131 and above) are shipped with the standard RDS SSL certificate (rds-ca-2019.pem) imported into the AMI OS certificate storage. 

Version 12.129-12.130

  1. RDS.force_ssl=1 in RDS parameter groups is now enforced. If error containing “handshake” is reported in connecting to RDS, export RDS SSL certificate, and import it into the certificate storage on the EC2. Follow steps in those documentation resources:
    https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/UsingWithRDS.SSL.html
    https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/SQLServer.Concepts.General.SSL.Using.html


Version 12.127-12.128

  1. Replaced Run Background Analyze Hourly & Email Daily Analyze with scheduling of Scan for Discrepancies with Alerting under /Configuration.
  2. Added automatic handling of Identity field added after initial table creation (replica table is dropped and recreated).
  3. Create Partition Function in DDL V2 is now being suppressed if replica server is not EE. 
  4. Resolved DDL replication issues related to sysdiagrams system table in Schema Compare V2.

Version 12.126

  1. Removed defaulting [ ] Replicate Logins, Users & Roles option to OFF in the replication job settings after new replication is created. 
  2. Fixed bugs in DDL V2 Beta which can get triggered in rare cases:
    2.1 Excluded triggers can get created enabled. 
    2.2 Excluded source tables can gets created (no seeding takes place).
    2.3 Excluded Replica Index and Trigger can get dropped. 
    2.4 Schema Compare Version drop down appears uninitialized in job settings for new jobs. Saving of drop down version can malfunction. 

Version 12.125

  1. Introduced DDL V2 Beta which addresses issues (ie incorrect syntax near ‘<‘ …), which can get triggered in rare cases (with option to switch back to V1 under replication job settings).

Version 12.124

  1. Fixed a bug in clustering which gets triggered in the rare case when cluster instance endpoints are long DNS CNAME values. 

Version 12.123

  1. Fixed a bug in generating self-signed SSL certificate which can get triggered in rare cases (upon initiation of the process, user sees an error screen “System error ….backend pool down…this usually happens when the server was previously removed from AD…”, instead of being redirected to login screen).

Version 12.119-12.122

  1. Enhanced DDL replication process (no longer run schema compare second time in follow up data replication process to handle a DDL change detected in previous process).
  2. Introduced integrated to WS backend SchemaCompare processing (<add key=”SchemaCompareVersion” value=”2″ /> denotes integrated, a value of 1 would switch to the SchemaCompareV2 API based method).
  3. Improved system debugging capabilities (WS config <add key=”ShowExceptionDetails” value=”true” /> would enable logging of exception details)

Version 12.117-12.118

  1. Added seamless support for MEMORY_OPTIMIZED_FILEGROUP.
    1.1 No longer have to use a custom DB create script to create a replica DB with MEMORY_OPTIMIZED_FILEGROUP.
    1.2 If a custom script is used to create a replica DB without MEMORY_OPTIMIZED_FILEGROUP (while the source DB is created with this option), then any objects referencing the MEMORY_OPTIMIZED_FILEGROUP will be created without this option on the replica. 
    1.3 Resolved a bug (by design rule) which complicates (All) new replication setup/ which gets triggered for (All) new replication setup only, which are about to replicate DBs which had been involved in previous jobs (show as archived  but not yet fully expired). 
    1.4 Resolved an issue with initial seeding schema replication which in certain cases can result into failure to create certain tables/views/functions with an error “PK and unique constraint not allowed on same column” .

Version 12.115-12.116

  1. New feature: introduced “Partial Replication Job Completion” (related to system exclusion of tables to prevent replication job disabling, which was introduced in version 12.111-12.112). Now all tables with pending changes will be synchronized regardless of whether one or more tables temporarily fail to replicate due to an intermittent error (i.e. table lock, DDL issue etc). 
  2. Optimized Event Logging of information, related to system exclusion of tables to prevent replication job disabling (introduced in version 12.111-12.112).


Version 12.113-12.114

  1. Turned always on logging for DDL/schema replication details (even if Error Logging Only is on)
  2. Resolved a DDL/schema replication issue in the case of when a table exists on the replica, and it is added to replication, but both tables have different schemas. 
  3. Added “Non-Critical Errors” to dashboard. 

Version 12.111-12.112

  1. New feature: prevent replication job disabling, cause by prolonged data replication failure in one or more tables (if not addressed, i.e. complex ambiguous DDL change which for security reasons, in DR use cases, requires user input), by system exclusion of tables 6 h prior to scheduled job disabling (warning is issued 6 h earlier).  

Version 12.107-12.110

  1. Resolved an issue where Redshift tables were not being created until data is added. 
  2. Improved CompareV2 connection handling.


Version 12.105-12.106

  1. New feature: added archiving of replication jobs, which also allows roll back of archived jobs.

Version 12.103-12.104

  1. Fixed a bug which cause [X] Add New Tables to Exclusion List option setup during initial replication setup to not get saved. 
  2. Fixed a table name parsing issue in DDL CompareV2 which causes some excluded tables to be added to replication jobs.

Version 12.100-12.102

  1. Resolved an issue in DDL CompareV2 which is triggered when older version source server is mixed with newer version replica (sample error: ALTER TABLE [dbo].[table] ADD CONSTRAINT [DF__cb_xxxxt__creat__7BXX5XXA] DEFAULT ([dbo].[table2]()) FOR [create_date_time] >>> >> Column already has a DEFAULT bound to it. Could not create constraint or index)

Version 12.99

  1. Added DDL replication support for public role permission granting (such as grant exec on..)

Version 12.98

  1. Added DDL replication support for fixed roles to user assignment (such as db_owner)


Version 12.96-12.97

  1. Fixed a bug in Table Reseeding Scheduling. Improved reseeding schedule execution handling (now when the schedule is altered/reset, last run date is reset too, to allow easier testing/validation).
  2. Resolved an issue with CompareV2 DDL replication of stored procedures and functions (related to previous improvement in DDL replication handling of databases with long list of users). 
  3. Resolved an issue with CompareV2 DDL (schema compare) functionality which causes intermittent DDL replication failures (error smo..:1997 or other error number) if source database server becomes busy.  

Version 12.94-12.95

  1. Improved CompareV2 handling for databases with long list of users.  

Version 12.93

  1. Fixed a bug in /api/AlterReplication, which was preventing all tables in exclusion list to to be removed when <TablesToExclude></TablesToExclude> blank list is passed as a parameter. 
  2. Added a new /api/AlterReplication tag/parameters <TablesToExcludeDisableCt> and <TablesToExcludeDisableCdc> which allow change tracking and CDC to be explicitly disabled for the tables list passed in tag/parameter <TablesToExclude> for exclusion.

Version 12.91-12.92

  1. Starting with 12.91, if DDL Change Tracking is explicitly turned off, it will not be system-activated even in case of data replication errors caused by DDL changes which are not manually applied to the replica yet. 
  2. Improved CompareV2 handling to not report back errors immediately, instead multiple operation retries would be attempted first. 

Version 12.87-90

  1. Fixes in CompareV2.

Version 12.85-86

  1. Improved replication handling for scenarios in which new tables are added, but DDL Replication is set to non-continuous (to allow all tables with pending changes to finish replicating then to render status as Failed (in addition to SystemActivatedSchemaReplication triggering introduced in previous version), and avoid prolonged stalling due to NoTimeOut process status, if DDL CreateTable fails).
  2. Eliminated generation of an unnecessary notification (along with unnecessary multiple retries logged as warnings and a final one logged as an error; process status, if no other errors, will be rendered as CompletedWithWarnings instead of Failed) in the process in which the schema change is discovered (instead, the follow up process will follow the default processing flow of multiple retries with the final one logged a an error).

Version 12.84

  1. Fixed a bug in self-updater which could get triggered when updating from 12.70 or below version into 12.71 and above (after update login page shows as blank, assistance from Support is required). 
  2. Fixed a bug in DDL Replication of extended properties. 
  3. Improved functionality which handles adding of new tables with non-continual DDL tracking of changes (ReplicaTableMissing event is treated as DataReplicationError,  leading to SystemActivatedSchemaReplication)
  4. Added an option to convert replication jobs from TablesOnly to AllObjects after initial seeding, under Replication Job settings.
  5. Improved handling of scenarios where a PK is added to a non-PK table (reseeding is AutoActivated).

Version 12.83

  1. Added Roles/Logins/Users DDL change tracking to CompareV2. 
  2. Added an option to enable/disable “[X] Replicate Logins, Users & Roles” after the initial seeding, under Replication Job settings. By default Logins, Users and Roles found on replica only (if dropped on source, or added as custom on replica directly) are not dropped. To enable dropping those object on replica if removed on source, enable “[ ] Drop Logins, Users & Roles found on replica only“. Need to add users and roles which are added as custom directly on the replica to the Replica Object Exclusions list. 
  3. Added //API Logs preview, and //Retention Policy management under /Advanced.
  4. Fixed a bug in the update to 12.83 package, which resolves an issue with resetting of custom Replication Job settings to defaults. 

Version 12.80-12.82

  1. Optimized processing of Redshift & S3 Data Lake feeds, which helps with minimizing intermittent errors during instance overload. 
  2. Addressed an issue with S3 (re-processing will be handled transparently for user). 
  3. Added an option to Analyze Report to generate a list of tables which exist only on the replica.

Version 12.78-12.79

  1. Improved logs reporting when encrypted objects are involved. Applied a fix related to ongoing DDL replication exclusion of encrypted objects. 
  2. Applied a fix to /api/AlterReplication, which prevents disabled replication jobs from being reenabled by setting DataReplication=On.
  3. Applied a fix which resets table DDL cache, if explicitly altered via the GuideMe dialog box (applicable to ambiguous DDL changes in DR protection scenarios).
  4. Applied a fix to CompareV2, plus disabled dumping of detailed trace to logs.   

Version 12.77

  1. Fixed a bug related to the +L (EC2 with SQL Server) product edition: an error “object reference ….null” can be thrown in case of instance memory over-utilization which affects loading of DDL cache. 

Version 12.73-12.76

  1. Fixed bugs related to self-updater. 
  2. Fixed bugs related to the +L (EC2 with SQL Server) product edition.  
  3. Fixed bugs related to DDL Tracking v2.     

Version 12.72

  1. Fixed a bug related to cdc.    

Version 12.71

  1. Rolled out an improved DDL Tracking v2.    

Version 12.70

  1. Fixed a bug related to computed columns (introduced in 12.69).  

Version 12.68-12.69

  1. Added a new feature allowing to configure replication jobs to be adding newly discovered tables to the excluded table list (to not be automatically replicated). 

Version 12.67

  1. Fixed a bug which can trigger data sync to continue asynchronously while schema changes are being still synchronized, which can lead to temporary delay of synchronization which appears as data loss (introduced in 12.60 along with improving of data handling). 

Version 12.66

  1. Fixed a bug related to DDL tracking, which can get triggered in rare cases. 

Version 12.65

  1. Fixed a bug related to replicating of assemblies into a non-Amazon-RDS replicas

Version 12.64

  1. Fixed a bug related to creation of a staging database (applicable to Redshift & S3 Data Lake replication jobs) on a replica server which already contains one or more read replica databases, from same source database.  

Version 12.63

  1. Fixed a bug related to scenarios in which an excluded table is added back to a replication job (introduced in recent versions, related to system cache). 

Version 12.60-12.62

  1. Improved replication handling for tables without primary keys, with unique indexes only. 
  2. Resolved a bug, introduced along with introducing the new parallel detection limit optimization, which in certain scenarios lead to increased latency.  
  3. Fixed a bug in the Rebuild Indexes UI navigation panel. 

on 12.58-12.59

  1. Fixed bugs related to Managed SQL Azure replication handling. 
  2. Fixed a bug in Analyze Report.  

Version 12.54-12.57

  1. Improved replication handling in the Azure product version build.

Version 12.51-12.53

  1. Updated /api/CreateReplication & /api/AlterReplication API methods – added support for the new setting “Parallel detection limit” (<ParallelReplicationLimit>) & “Parallel replication limit” (<ParallelDetectionLimit>) (rolled out in 12.50).
    <ParallelTablesLimit> in both of the above API methods has been deprecated in 12.51. Backward compatibility is supported: if <ParallelTablesLimit> is used, the value will be mapped/assigned to <ParallelReplicationLimit>.  
  2. Fixed a bug related to the “Parallel detection limit” feature rolled out in 12.50.  


Version 12.49-12.50

  1. Improved replication handling by allowing granular control over concurrent change detection and data replication (replaced the Replication Job setting parameter “Parallel Table Replication Limit”): 
  2. Added two new API methods, which allow Reseeding of individual tables to be initiated/managed over the API (previously available only in the UI under /Replications/Analyze) :
    2.1 /api/ReseedTable
    2.2 /api/ReseedTableStatus
  3. Improved UI.

Version 12.48

  1. Improved Redshift and Azure SQL Data Warehouse replica handling. Latency for SCD Type 1 replication (exact SQL Server DB read replica) has been lowered substantially. To achieve maximum performance, after upgrade, reseed all tables, which will cause the Redshift schema indexes structure to be rebuild under the new and improved indexes definition.
  2. Upgrade to 12.48 form older versions will update the Redshift driver, which addresses an issue reported by AWS on 10-30-2019 (the older version driver caused Redshift restarts in certain cases). 
  3. Improved S3 Data Lakes streaming functionality for non-real time replication job configurations (addresses an issue with accumulating job start offset).
  4. Improved Azure SQL Data Warehouse replica handling. 
  5. Resolved an issue with schema tracking.
  6. Resolved an issue with Active Directory/Windows logins being attempted to be replicated (after upgrade to 12.38) for non-Active Directory Authentication based replications. 
  7. Fixed minor other known bugs. 
  8. Improved the dashboard to include detailed information about latency resulting from replication failures, plus the dashboard is now an operational console where from the respective replication latency chart, the user can go to Analyze Report, Replication Job Settings; Runtime, Redshift, and S3 Data Lake logs:   

Version 12.45-12.47

  1. Addressed an issue related to self-upgrade in cases where the OS is joined to Active Directory and additional local-admin OS accounts are added after initial EC2 activation. 
  2. Enhanced the upgrade panel to report and log more error details, and list history of updates along with detailed upgrade logs.  

Version 12.40-12.44

  1. Renamed “Replication Schedules” to “Replication Jobs” across the entire application.
  2. Improved UI.
  3. Fixed bugs. 
  4. Added new methods in API (applicable only to initial semi-automatic replication seedings in AwaitingBackupRestore status):
    /api/FinalizeReplication
    /api/FinalizeAllReplications

Version 12.39

  1. Fixed a bug in the self-service upgrade package related to joining the instance to Active Directory.

Version 12.38

  1. Fixed a bug related to upgrading from older version to 12.37 or above, where if there had been a past “an ambiguous schema change”, even if it had been resolved, a false positive  error notification shows in the header notification red bar (which take some time to clear out).
  2. Fixed a bug in Analyze report, which gets triggered if list of included tables is >500 and there are excluded tables. 
  3. Enhanced Analyze report to not calculate row counts for excluded tables (which it was a minor issue in cases with long list of excluded tables), unless the list of excluded tables is explicitly requested to be shown in the report. 
  4. Added support for synchronization of Active Directory and Server level logins/users. Note: Dropped users, similarly to dropped tables,  by design, are not removed on the replica (some considerations related to DR and reporting use cases).
  5. Addressed an issue with replication schedules configured to not run in real-time (i.e. to run every 60 mins) – if there is a schema change reported as “detected but will be fixed during next process run” and the process run is rendered as Failed, a follow up process run will take place overwriting the schedule setup (in order to sync the schema immediately and not wait until the next process run).


Version 12.37

  1. Introduced Active Directory integration with database servers. 
  2. Improved SQL-to-SQL replication’s Semi-Automatic option seeding method to support RDS snapshot-restore  (in addition to the previously supported DB level export-import via S3.
  3. Introduced Group Actions allowing one-click replication and HA cluster setting changes to be made to all replications. 
  4. Improved replication handling to allow up to 100 DB replications to be effectively handled on a single CloudBasic Multi-AR High-Availability cluster
  5. Replaced the previous Real-Time alerting mechanism (send alerts as they occur) with a new Batched method of alerting (aggregation of alerts and sending a single one at a specified delayed period of time – generally in the seconds), which prevents hundreds of alerts to be generated (in case of i.e. connectivity degradation to the source server which will affect all replications). The old Real-Time method is still an available option to be selected. 
  6. New DB limits comply with the latest RDS limits (30 DB replications on a single Large EC2 instance, 100 for XLarge and above). 
  7. Improved the replication handling mechanism to not having to redo successful data copy operations in case of overall process failure. 
  8. Resolved an issue related to maintaining API call logs (system level only; not available in UI) retention. In previous versions the API logs were growing indefinitely in some cases, causing the system database to get full overtime. 
  9. Enhanced API to support new functionality (see API Documentation):
    New api methods:
    9.1. /api/GroupActionAlterReplications
    9.2. /api/StartReplicationsExecution 
    9.3. /api/StopReplicationsExecution 
    9.4. /api/ReplicationsExecutionStatus
    9.5 /api/ActivateActiveDirectory 
    9.6 /api/DeactivateActiveDirectory
    9.7 /api/ActiveDirectoryStatus
    9.8 Added new tags to /api/AlterConfiguration:
    	<AlterConfigurationRequest>
    	   <GeneralSettings>
    	     <MonitorLatency>..
    	     <MonitorLatencyAlertInMins>..
    <MonitorLatencyNoAlertWithinPeriodInMins>..
    	   <NotificationSettings>
    		<AlertingMethod>..
    		<AlertsAggregationPeriodInSecs>..			
    <AlertsExpirationInMins>..
     

Version 12.36

  1. Enhanced the “Index Exclusions” (from schema sync) in replica functionality into “Replica Object Exclusions” (reflected to the applicable API methods as well) – now in addition the ability to exclude indexes, stored procedures, views and functions on the replica DB can be dropped or created in order to optimize the replica DB for reporting. 
  2. Altered the way the system activated schema sync works – now it limits the schema compare to tables only (when system activated) regardless of whether the type of replication is “Tables Only” or not. 
  3. Enhanced /api/getlatency/ – in product versions 12.36 and above, the time elapsed since the last successful data synchronization will be returned along with the average latency.
  4. Further reduced workload on primary for SQL-to-SQL replications (eliminated a previously introduced, in recent versions, system query to constantly retrieve FKs for the entire source DB, related to Redshift and S3 Data Lakes data feeds)

Version 12.28-12.35

  1. Fixed a bug related to temporal tables on column level. 
  2. Fixed a bug related to Runtime Logs filtering. 
  3. Fixed a bug related to creating of multiple read-replicas from an Azure SQL data source
  4. Improved backend and UI handling for databases with large number (10,000+) of tables.  
  5. Introduced auto-turning off (optional/default) of ChangeTracking/CDC on the replica for Semi-Automatic replications.
  6. Fixed a bug related to CDC based replications, when CDC activation is pre-existing on the source. 
  7. Added a new feature – “Index Exclusions” (from schema sync) in replica,  which allows indexes on the replica DB to be dropped and created. 

Version 12.26-12.27

  1. Introduced exclusion of databases in (All) replications.
  2. Fixed a bug related to replications with source Azure SQL DB.
  3. Fix a bug related to replicating sequences

Version 12.24-12.25

  1. Improved warning messages clarity for CloudBasic instances joined to Multi-AZ HA Cluster
  2. Resolved a bug related to custom data types for continuous replications in the default “TempDB” staging buffer type.

    In versions 12.23 and earlier, if you come across the below message, upgrade to version 12.24 or above, or switch the the staging buffer type (under [Advanced] for new replications; under schedule edit main tab for existing replications)  from “TempDB” to “ReplicaDB”. 

    Transfer to Destination DB ERROR:Column, parameter, or variable .. : Cannot find data type dbo.MyCustomType.


Version 12.18-12.23

  1. Improved Redshift handling – introduced auto-generation of Primary Keys and Sort Keys based to boost performance:
See below example of how SQL Server PKs and indexes are translated to Redshift PKs and Sort Keys. 
Also note that the following commands executed against Redshift optimizes indexes (similar to the SQL Server Index Rebuild functionally):

vacuum;
analyze;
The very first query, with Sort Key fields included in where clause and in joins, executes slower.  After that the execution plan improves substantially.  

 

 

A sample SQL Server table from which a Redshift table will be auto-generated:

CREATE TABLE [dbo].[Client](
[ID] [uniqueidentifier] NOT NULL,
[Name] [varchar](256) NOT NULL,
[LoweredName] [varchar](256) NOT NULL,
[Description] [varchar](256) NULL,
[AllowedGrant] [int] NOT NULL,
[CreatedOn] [datetime2](7) NOT NULL,
[ClientSecretHash] [varchar](256) NOT NULL,
CONSTRAINT [PK_Client] PRIMARY KEY NONCLUSTERED 
(
[ID] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY],
UNIQUE NONCLUSTERED 
(
[LoweredName] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY],
UNIQUE NONCLUSTERED 
(
[Name] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
GO

 

 

A sample Redshift table which is auto-generated from above SQL Server sample table:
 
CREATE TABLE IF NOT EXISTS voyage0129a.client
(
id VARCHAR(256) NOT NULL ENCODE RAW
,name VARCHAR(1024) ENCODE RAW
,loweredname VARCHAR(1024) ENCODE RAW
,description VARCHAR(1024) ENCODE lzo
,allowedgrant INTEGER ENCODE lzo
,createdon TIMESTAMP WITHOUT TIME ZONE ENCODE lzo
,clientsecrethash VARCHAR(1024) ENCODE lzo
,PRIMARY KEY (id)
)
DISTSTYLE EVEN
SORTKEY (
id, name, loweredname
);


Version 12.11-12.17 

  1. Introduced S3 Data Lake export to JSON format, and file compression options (Bzip2, Gzip) for all supported file formats (CSV, Parquet, JSON).
  2. Switched default staging buffer from ReplicaDb to TempDb.
  3. Enhanced API to reflect the new features in (2) & (3):

    2.1. /api/CreateReplication & /api/AlterReplication

    • Added new tag <StagingBufferType>ReplicaDb</StagingBufferType> <!– ReplicaDb, TempDb (default) –>
    • Added new tag <StoredProceduresToExclude>storedproc1,storedproc2</StoredProceduresToExclude><!– list of Stored Procedures to be excluded from the existing replication. –> 
    • Added new tag <IndexesToExclude>index1,index2</IndexesToExclude><!– list of Indexes to be excluded from the existing replication. –> 

    2.2. /api/CreateS3Replication & /api/AlterS3Replication

    • Added new tag <FileFormat>CSV</FileFormat><!– JSON, CSV, Parquet –>
    • Added new tag <JsonExportType>Line</JsonExportType> <!– Document (default), Line –>
    • Added new tag <Compression>None</Compression> <!– None (default), Bzip2, Gzip –>
  4. Improved Runtime Logs browsing, and added an additional filtering option Schema Changes: 
  5. Fixed a bug related to temporal tables.


Version 12.8-12.10 

  1. Improved Runtime Logs browsing, and added additional filtering options based on content (Data Replication Only, No Timeout, System Logs): 
  2. Bug fixes.

Version 12.6-12.7

  1. Fixed a bug in semi-automatic updating.

Version 12.5-12.6

  1. Introduced naming of replication schedules.
  2. Introduced Favorites in the main menu.

Version 12.1-12.4

  1. Bug fixes.

Version 12.0

  1. A new major feature – exclusion of table fields.
  2. Introduced a completely new overhauled console UI with improved look and user experience.

Version 11.18-11.19

  1. Fixed S3 Data Lake Parquet format related bug.
  2. Other minor bug fixes.

Version 11.11-11.17

  1. For SDC Type 2 Redshift and Db2 warehouse on Cloud, introduced option “Reseeding with preserving of historical records”.
  2. Introduced support of continuous replication for tables without primary keys (CDC only).
  3. Bug fixes.

Version 11.00-11.10

  1. Added CDC support.
  2. Support of continuous change tracking for tables without primary keys for SCD Type 2 replication to RedShift, S3 Data Lakes and DB2 Warehouse On Cloud.
  3. Added new API methods:
    /api/DeactivateInstance,
    /api/GetReplicationsList
    /api/CreateAndBindSelfSignedCertificate
    /api/UnbindAndDeleteSelfSignedCertificate
  4. Bug fixes

Version 10.20-10-23

  1. (applies to SQL Server to Db2 Warehouse on Cloud data feeds only in this version): Analyze report optimization.
  2. Introduced “[ ] Excluded Replica Row Counts” to Analyze report to make possible to report pending changes in cases where replica server (such as Db2 or Redshift) is slow to respond.

Version 10.19

  1. Resolved an bug related to changing of the default system locale.

Version 10.13-10.18

  1. New Feature (applies to SQL Server to Db2 Warehouse on Cloud data feeds only in this version): added a stage-less replication option (staging SQL Server DB is not mandatory if automatic schema synchronization to Db2 is not a requirement)
  2. New Feature (applies to SQL Server to Db2 Warehouse on Cloud data feeds only in this version): added an option to add “cb_sys_operation_type” indicating the type of the operation (‘I’, ‘U’, ‘D’) to Db2 CSD Type 2 tables.
  3. Minor bug fixes applied to SQL server to SQL Server replication mechanism.
  4. Bug fixes applied to Db2 Warehouse on Cloud data replication mechanism.

Version 10.10-10.12

  1. New feature: added self-service system update. A new menu item “Update” is added. If the instance has outbound access to the internet (outside of the VPN in AWS environment) then the update can be installed with a click of a button. If outbound access is not allowed, then a semi-automated update option is available, where an update file is uploaded in the UI Console (in the browser).
  2. New feature: added HA Cluster Instance Affinity for replications. When Multi-AZ HA Clustering is in place, each DB replication can be assigned to the cluster instance delivering lower latency (when the cluster instances are deployed in different regions, in general the instance running in the primary region delivers lower latency), or for the purpose of manually distributing the workload in the cluster. The other cluster instance becomes a standby, which takes over the DB replication if the designated primary instance is down.
  3. New feature: added an option for schema change tracking to not run continuously, which results into workload on primary to be reduced further.
  4. Resolved HTML formatting issue related to emailed errors/notifications, which occurs when Amazon SES with IAM integration (no login/password) is leveraged for emailing. This version comes with the latest Amazon SES library.

Version 10.8-10.9

  1. New feature (applies to SQL Server to S3 Data Lakes replications only): added support of Parquet files (CSV or Parquet files can be exported and uploaded to S3 Data Lakes) for improved integration with products such a Amazon Glue (ETL processing), Amazon Athena (S3 data lake querying), SAS (reporting against S3 data lakes), Hadoop etc.

Version 10.7

  1. New feature: added database replica reindexing capabilities (manual initiation and scheduling; to both UI and API).
  2. Improved replication algorithm (related to master-detailed tables sequencing).
  3. Fixed a bug in replication schedules partial daly execution.

Version 9.8-10.6

  1. Added REST API (https://cloudbasic.net/documentation/api/)
  2. Added option to generate and auto-assign a self-signed SSL certificate for secure access to UI console, API and for HA cluster communication. Also automated binding of an installed CA SSL certificate.
  3. Added option to email daily discrepancy analyze reports in XML and JSON format.
  4. Fixed known bugs.

Version 9.5-9.7

  1. Advanced\Analyze report has been defaulted to not show excluded tables and tables with no PKs. Filtering options need to be selected to show excluded tables and tables with no PKs, with indication if a no PK table is included in a reseeding schedule.
  2. Added detailed DB Promote To Primary logging.
  3. Known bug: Even if doing only a SQL to SQL replication, removing of Drive D: will result into an error.

Version 9.2-9.4

  1. Added scheduling of reseeding for tables without Primary Keys.
  2. Added cataloging (creating of index.json file per table; compatible with SAS) of uploaded to S3 Data Lake flat files for the upload type “One folder per table with multi-files”.
  3. Added Promote To Primary reporting of logs & enhanced the same process in cases when source database is inaccessible.

Version 9.0-9.1

  1. Introduced support for SQL Server 2016 temporal tables
  2. Added SQL Server SCD Type 1-2 streaming of data to Redshift & S3 Data Lakes.
  3. Changed text file logs extension from .txt to .log (stored in C:\Web\WebWS\Logs\MainLogs for SQL Server to SQL Server replications; Redshift portion of logs is stored in C:\Web\WebWS\Logs\Redshift for Redshift replications; S3 portion of logs for certain S3 Data Lake upload types is stored in C:\Web\WebWS\Logs\S3)

Version 8.6-8.8

  1. Fixed DB Schema replication issues.

Version 8.5

  1. Introduced support of SQL Server 2016 AlwaysEncrypt.

Version 8.4

  1. Introduced support of custom data types in the Redshift feed (core shared with Redshift version).

Version 8.3

    1. Resolved bug in schema replication related to FKs/index dependencies.
  1. Introduced support for partitioned tabes.

Version 8.1-8.2

    1. Introduced encrypted Multi-AZ HA Cluster communication (https://cloudbasic.net/documentation/encrypting-ha-cluster-communication/)
    1. Improved Multi-AZ HA Cluster RT synchronization & replication schedules recovery in cases where one server is down for prolonged period of time exceeding change tracking retention period.
  1. Resolved an SMTP SSL handshake compatibility issue with Postfix. SSL port 465/protocol is no longer supported.

Version 8.0

    1. Introduced Instance virtualization (non-administrative users with limited access to a subset of replications), which along with Multi-AZ HA Clustering (enhanced to be compatible with instance virtualization) is supported for instance type/size M4.XLarge and larger only (https://cloudbasic.net/documentation/compare-instance-types/).
    1. Fixed a bug related to table column NOT NULL to NULL type change schema replication.
  1. Fixed bugs in Multi-AZ HA Cluster RT synchronization.

Version 7.10-7.18

Improved Multi-AZ HA cluster load balancing algorithm (https://cloudbasic.net/documentation/configure-ha-cloudbasic-cluster/). 

Version 7.9

  1. Improved free disk space level monitoring to prevent extensive generation of error/warning logs (as a result of the improved replication performance in 7.7) to affect replication execution. Free disk space alerting and logging into event logs starts at 7GB. At 5 GB system starts auto-reducing of logs retention period by 1 day for every 1 GB of disk space consumed. Replication execution is auto-suspended if free disk space reaches 256MB.  Retention period auto-adjustment stops at 2 days to allow reviewing of the reasons for extensive generation of error/warning logs being generated.

      2. Improved System database free space monitoring. If free space crosses 20% threshold, the logs retention is adjusted to 2 days to initiate freeing up space and shrinking of system database. Replication execution is suspended if free system DB space reaches 256MB.

      3. Improved system error/warning emailing/alerting management. The system administrator now can select to not receive warning alerts and to select how often repetitive alerts to be emailed. Default is 60 mins. 

Version 7.8

1. Moved “Error Logging only” option out of system web.config into UI, on replication level (form system-wide to per replication schedule level). Also introduced detailed|basic runtime logging option. When a new replication schedule is create now logging is defaulted to “Error Logging only”.

Version 7.7

1. Introduced asynchronous schema compare/replication, which results into further lowering the replication latency.

Version 7.3, 7.4, 7.5, 7.6

1. bug fixes

Version 7.2

1. Applied a fix preventing reseeding of tables in rare cases related to previously improved replication handling in 7.1

Version 7.1

1. Improved replication handling in cases of table truncation.

Version 7.0

1. No changes related to this product version. Changes are in the core shared with +R product build.

Version 6.9

1. No changes related to this product version. Changes are in the core shared with +S3 product build.

Version 6.8

1. Added support for AzureSQL as a source.

Version 6.7

1. Added a more comprehensive description of supported use cases in the first option (#1 “Continuous Replication with Activation of Change Tracking”)


Version 6.6

Fixed a bug that causes accidental reseeding of individual tables in certain cases, related to change tracking policy maintenance.


Version 6.5

1. Added option to suppress schema compare in order to minimize replication latency further to support applicable use cases.

Note: schema changes must be coordinated with CloudBasic administrator. Schema Compare should be activated for at least one change tracking process run.

Important: rename of column and drop of column are still considered ambiguous operations, and such schema changes need to be applied manually.


Version 6.4

No applicable to this product version changes. Functionality changes related to the replication engine shared with other product versions.


Version 6.3

1. Avoided reseeding of tables if there is a schema change. Introduced schema compare and automatic schema synchronization without the need to reseed.

Note: rename of column and drop of column are considered ambiguous operations, and such schema changes need to be applied manually to the local replica DB used to produce .dat file.
if a column is renamed or dropped, change tracking processes will start failing. An Indication of the reason will be logged as:

>>ERROR:Ambiguous schema change detected. 
Has a column(s) been renamed or dropped in table(s):{table name} ? 
To resume change tracking, apply the proper schema 
change(s) manually to the replica database.

2. Added option to exclude tables from replication in UI. Entire schema is replicated, but tables excluded form replication are not seeded or tracked for changes.


Version 6.2

No applicable to this product version changes. Functionality changes related to the replication engine shared with other product versions. 

Version 6.1

1. Minor cosmetic changes and bug fixes.


Version 6.0

1. Consolidate replication engines of SCD Type 1 (Read-Replica version) and Type 2 & 3.

— Previous versions release notes have been archived

You need to add a widget, row, or prebuilt layout before you’ll see anything here. 🙂