Implementation of Incremental Backup

UDA
Enthusiast

Implementation of Incremental Backup

Here in EIM, the challenge with the backup was backup window was growing steadily. When it reached 11 hour window, it became an issue. With the continuous ETL loads & so many report users it became nearly impossible to have that big backup window. Here, ETL almost pulls the real time data from the legacy systems and puts them into DW.

On investigation, it was found that only one table in the database takes almost 9 hours to get backed up. The size of the table was 6 TB +. Then we planned to implement incremental backup for that big table.

With the help of the ETL, we started loggig the changed data (based on dates) for that table into a lookup table. Then we developed a shell script to look into that lookup table for finding out the changed data and generate an ARC script based on that dates and that ARC script is executed from Netvault.

So this helped us to start the incremental backup. This reduced the backup window from 11 hours to 1 hour 15 mins.

The next challenge was the restorability of the data. It would take a long time to restore the data from the tapes. So it was decided to divide the incremental jobs to 3 major parts -
1> Bi-monthly backup
2> Bi-weekly backup
3> Weekly Backup
4> Daily backup

The made the restoring of the data much more easy and quick.
Tags (1)
2 REPLIES
Enthusiast

Re: Implementation of Incremental Backup

Hi Roy,

Would like to ask more details about the backup strategy you guys have implemented.

Can I contact you separately?

Thanks!

Pat

Enthusiast

Re: Implementation of Incremental Backup

Hi Somnath,

How are you? I wish you will be doing great. I need your help to create the same Incremental Backup process in Teradata. Can I have your email ID or any contact number so that I can discuss this with you in detail I will not take much of your time for sure. You can reply me back on this forum or on my email id Ankit.satyam@gmail.com or ankitmani2004@yahoo.co.in . Waiting for your response. Thanks a lot for all your help and time in advance.

Regards,

Ankit