Published: Oct 1, 2016
This white paper deals with how CloudBasic handles Slow Changing Dimensions (SCD), that is, changes occurring over time to the context data of the data mart; it assumes familiarity with the subject of dimensional modeling in data warehouses, but also briefly describes the different SCD types based on the nature of the changes and business needs for history record keeping.
We can see a dimension data row as having two parts, the first part is data attributes that are static and do not change over time, such as the data that identifies a row uniquely. The second part is the data attributes that change over time, but not too frequently, and that may or not need history tracking.
On the history tracking topic, the specific business need may be to not track the history of those changes at all, fully track those changes or just partially track them by keeping current and previous data values for the dimension; in data warehouse parlance, these history tracking strategies are referred to as SCD Types 1, 2 and 3, respectively.
SCD Type 1: Overwrite
This dimension type updates the contents by replacing the old values; at the row level, new data inserts a new row; and the update operation may have a change detection method to only update columns with changed values or simply replaces the entire data contents, in any case, the data value history is lost.
SCD Type 2: Full Changes History
In this type new content again simply creates a new row; and in order to keep track of content changes, data updates also add a new row, and a versioning technique is used to indicate what the current row is.
A general recommendation for tracking these changes is that at minimum three additional attributes be created: a timestamp that indicates the creation of the row, a second timestamp that indicates the update operation, and finally a current row indicator; however, in practice a data row version number may be sufficient, where the highest version value indicates the current row.
SCD Type 3: Current and Previous Values
As before, with this type new content simply creates a new row; but in this case the concern is to keep only the current and previous content; therefore, there is only need for creating an additional attribute value.
There is an initial step whereby a starting target data mart replica is created from the source data mart; CloudBasic is aware of what the unique identifier for each data row is, which remains static over time (commonly known as primary key), and treats the rest of the data in the row as the slow changing dimension.
SCD Type 2
In order to implement a SCD Type 2, CloudBasic creates a version number for each row; upon insert this value is 1, and subsequent updates increase this value by 1.
SCD Type 3
On the other hand, to implement a SCD Type 3 and keep the partial history of these data attributes, CloudBasic creates additional attributes to store the immediately last known data value.
Applying Data Changes
In SCD Type 2 the content change processing keeps track of all of the updates to the data attributes; that is, the full history available, and a version number is used for this purpose.
1. For updates, CloudBasic locates the corresponding rows, reads their version number, increments this number by 1 and inserts a new row with the changed data.
2. For deletes, the corresponding rows are located, their version number is read, incremented by 1 and a row with empty data values is inserted.
3. For inserts, the version number attribute is added to the new rows, and then inserted.
In SCD Type 3 the content change processing keeps track of the current and the immediately previous value of the data attributes; that is, there is only one level of history available.
4. For updates, CloudBasic locates the corresponding rows and moves their current data values into the previous data values, then copies the updated data into the current data values.
5. For deletes, the corresponding rows are located and their current data values are moved into the previous data values, then the current data values are emptied.
6. For inserts, additional attributes are created to store the immediately last known data value, and then inserted.
Managing Data Structure Changes
Business needs also change over time, and an application data structure may change due to this; CloudBasic is aware of changes made to data structures and handles them accordingly.
SCD Type 2
SCD Type 3
You'll be in good company:
Amazing Product for RDS SQL Server replication
Cloudbasic RDS Always-On/Geo-Replicate for SQL Server provided us with just what we were looking for. It saved us a lot of time during our migration process by taking care of the database schema replication and keeping large amounts of data in sync. Also, the support has been amazing in terms of being available in-spite of the time zone differences. Read full review >>
Great product, fantastic support team!
We’re a small, growing company so our AWS budget is tight and the CloudBasic replication product is the perfect solution for our needs. This product has allowed us to implement a reliable, near real-time replication solution without resorting to expensive SQL Server/RDS replication solutions. The CloudBasic product has proven to be fast, reliable, easy to set up, use and manage.
Before our AWS migration we managed our own physical SQL Server installation and used SQL Server log shipping to sync our online and reporting databases with typical latencies of 15-30 minute range. With the CloudBasic solution our latencies are now in the 2-5 second range… Read full review >>
CloudBasic is an AWS Rockstar!
The CloudBasic RDS Always-On/Geo-Replicate for SQL Server solution is an absolute lifesaver when working with RDS SQL Server! SQL Server on RDS falls short in a lot of aspects when it comes to replication and disaster recovery, and this solution makes the process seamless. We’re able to migrate very large amounts of data and keep the in sync while we move applications around, as well as create usable read-replicas that RDS simply doesn’t provide. Anytime we’ve run into any snags in the process, CloudBasic support has been right there available to help get things going right away. I’m truly impressed with the product and the company behind it! Read full review >>
Great product, simplifies architecture and hassle free
CLOUDBASIC RDS Always-On/Geo-Replicate for SQL Server HA/DR/Redshift/S3 provided us a way to replicate multiple on site MSSQL DBs to other datacenters as well as Amazon RDS\EC2 and also from the cloud into our datacenters. After spending weeks of investigating how we could manage this using SQL’s own replication or writing our own replication jobs we found Cloudbasic. We were up and running within a hour and our DBA team could test replication paths hassle free and have full control over the source and destination DBs. The ability to replicate into SQL Web version saves us $10,000’s every year whilst maintaining the ability to have read\write replicas. The Cloudbasic support has been fantastic, helping us with any questions we have and also releasing new features to meet specific needs, we couldn’t ask for more from a product support group. I would highly recommend you trial this product if you are looking at any form of hybrid MSSQL replication and want the most cost optimised solution that works straight out the box (AMI). Read full review >>
Excellent product and support!
CloudBasic RDS AlwaysOn/Geo-Replicate for SQL Server HA/DR is an excellent product backed by great support. Using the pre-configured AMI we were up and running in minutes. Used CloudBasic to migrate SQL Server databases into RDS, followed by setting up cross-region Multi-AR RDS SQL replications for Disaster Recovery and Multi-AZ High Availability. It works well with RDS Multi-AZ, creates SQL Server read-replicas which can be used for reporting, data-locality and offloading of primary RDS SQL Servers. With help form the responsive CloudBasic support team we were able to migrate our databases with no down time. Setting up DR and HA was straightforward.
I strongly recommend the product and the CloudBasic advisory support team… Read full review >>
I’m really happy I found your product
I’m really happy I found your product as there are not many options for this sort of thing with RDS and SQL Server.
Does what it says on the box, easy to use, amazing support.
I found this tool when I was looking for a way to transfer databases between our development systems and our production systems. These comprised of a mix of in house physical SQL, EC2 SQL and and RDS. Cloudbasic Geo-replicate solved this problem.
The product improvements I asked for were implemented within days allowing me to extend our use of this product into our devops continuous delivery model.
Affordable SQL Server High Availability Solution
As a consultant I look for affordable and easy to maintain solutions for my customers. This SQL Server HA solution features a simple web interface. I did not have to worry about configuring server clusters, log shipping etc. I was able to setup a DB instance replication to a cross-region AlwaysOn RDS in minutes. I was pleasantly surprised that MSSQL Web Edition is also supported. I had to call for support related to DB issues on my end. I was assigned a dedicated support representative who proved to be very helpful. I did an extensive testing to ensure that this product is very reliable. With this product I am planning to scale up my Cloud Disaster Recovery implementation efforts.
Simple, Powerful, with Stellar Support
The software delivers what it promises. Its easy to use. The support is fantastic. We were impressed how fast they reached out to us to want to help. The software is fast, economical and affordable. It doesn’t put any stress on your source and destination databases. It seamlessly does what it does and it does it well. I highly recommend this product to anyone.
Simple cross-region RDS replication solution
We were looking for a way to continuously replicate our RDS based SQL Servers from west to east coast, for disaster recovery and reporting purposes. We started from looking for a SQL Server log shipping and high availability based solution ..then found CloudBasic’s Geo-Replicate server. We launched the server from the market place, entered RDS source and destination information and within minutes were able to stream data. The secondary DBs seeded very fast, and … Read All
Easy setup. Various deployment scenarios.
We were looking to utilize AWS as a fail-over site for our on-premise SQL Server based applications. We turned to the AWS Market place to find solutions to help us speed up implementation. Found this product, and within minutes were able to steam SQL Server data to an RDS SQL Server on AWS. All we had to do is enter source and destination SQL server connection strings. Then selected if we wanted a one time migration of the SQL server or also… Read full review >>
Works very well. Easy to use. Some minor issues need to be addressed.
We intended to use this software to do a one time migration of SQL databases from our co-located datacenter to AWS. Then figured it is very easy to setup continuous replication from East to West coast (AWS in California to AWS in Virginia). We currently use EC2 SQL Server Web edition in California, replicating to a small RDS Web Edition in Virginia… Read full review >>
Simple and inexpensive solution to replicate SQL Servers from West to East coast
Our customer base spans 100+ hospitals, including Tenet Healthcare, Baptist Health of South Florida and Texas Health Resources. We migrated our systems to AWS a year ago. Then shortly after, as part of our business continuity planning, selected the pre-AWS Marketplace version of CLOUDBASIC REPLICATE to … Read full review >>
Powerful Yet Simple Replication Solution
The most important feature we were looking for was to be able to access the end point of the replica for our reporting. This product replicates between SQL servers manages by us. We have to ensure the destination SQL Server is used in read-only mode, but this gives us the flexibility to setup replication in various scenarios: on-prem to AWS, across AWS regions, zone-to-zone. It does seem that SQL Server Web edition is supported in all scenarios as well… Read full review >>
CloudBasic SQL Migrate to RDS Solution
I used CloudBasic to Migrate my SQL2012 to an RDS 2012 instance. Install of CloudBasic was easy and once the connection strings were set on the CloudBasic Server webpage, my database connectivity was verified and the db migrated very fast to the RDS instance. CloudBasic is the easiest and fastest solution to replicate your data to an RDS instance for production and or testing. Thank You for making my job easier!!! Read full review >>
CloudBasic works great! Easy to setup and comes with outstanding support.
CloudBasic has a nice web user interface. There’s a quick setup feature to start replicating your databases in minutes after you setup the AMI from Amazon’s Marketplace. If you have any questions, RJ and their support team will go to great lengths to make sure CloudBasic gives you the ROI that you’re looking for in a database replication solution. Awesome product. Read full review >>