The Online Backup facility has been developed to enable system managers to perform necessary regular database backups while still allowing users the ability to perform updates on the database.
Using this facility enables the automatic restoration of the database including all updates up to the point of failure – without operator intervention or scripting.
Following the restoration, the database will be left in a consistent state. The functionality of the restore process, jrestore, has been extended to allow for the automatic roll-forward of logsets after a database restore has completed. This extension uses the Transaction Journal configuration (JediLoggerConfig) which was
active at the time of the last backup along with the corresponding Transaction Journal Logfiles.
Online Backup Operational Details
The following diagram and description describe the details of an Online Backup
Prior to the commencement of the Online Backup a determination is made of whether an Online Backup is safe with regard to Transaction Logging. Two tests must be passed: firstly transaction logging must be active. Online Backups depend on the Transaction Journaling system. Secondly, a minimum of three logsets must be
configured. This constraint is to prevent the loss of data contained in the Transaction Logs. If this test is passed then:
A Checkpoint is performed. The database is paused for all updates. This pause will ensure that all transactions which have entered the commit phase (i.e. by the processing of either a TRANSEND or TRANSABORT instruction) will be allowed to complete. This will ensure that all committing transactions will be entirely
contained within the current logset. As all committing transactions are allowed to complete, and new transactions are held off from committing, this ensures that the database at this point of time is in a consistent transactional state.
The Transaction Logs are switched to the next (by use of jlogadmin –lnext – internally by the program).
A Checkpoint marker (record) is entered into the new Transaction Logset as the first entry. This entry will have a type specification of “CHECKPOINT” and will have a textual string “Backup Started”. The use of this checkpoint marker is not entirely essential, but is does act as information for the system administrator.
The database is then resumed – any updating processes will be allowed to continue. All updates to the database will cause an entry in the Transaction Logset to be created.
No updates will be lost during this phase of the backup. The delay in time is dependent on the transaction rate prevalent on the computer at this time. This may or may not be noticeable.
The backup now begins. This backup will be a snapshot of each specified file until the end of the specified set. With online processes being allowed to update the database at the same time that the database is being backed up, it is clear that some (many) files will have been backed up prior to updates to such files - for the duration of the backup. Each update during this time will have caused the Transaction Logset to be updated with such updates. Thus the backup set will comprise the snapshot of all files specified for backup plus the image of the updates contained within the current logset. During the restore process, this Transaction Logset will be used to roll forward the database for those updates which occurred during the backup.
Upon completion of the backup the database will again be paused. All processes which are in the commit phase of transactions will be allowed to continue.
A Checkpoint marker is now entered into the Transaction Logset at the next position. This entry will have a type specification of “CHECKPOINT” and will have a textual string “Backup Ended”. Thus it can be seen that the updates to the database during the backup will be recorded in the Transaction Logset bounded by CHECKPOINT records indicating the start and end of the backup.
Once this marker has been written, the logsets are again switched to the next available logset. Note now that this whole process has used three logsets in a very clean manner. The logset which was current prior to the commencement of the backup will be preserved (possibly for later archiving); the logset which was current
at the time the backup ran is preserved and lastly a new current logset is made available for future updates. It will be seen later the relevance of this latter logset during the restore process.
The database is now resumed, allowing all updating processes to continue. Note that all transactions which enter the commit phase will be contained in the latest current logset in their entirety.
The Transaction Logset which was current during the backup and the Transaction Journal configuration file (jediLoggerConfig) are now archived to the same medium and following the database backup. These two components will be required during the restore process to return the database to a consistent state.
Recovering the database from backup media
Important: No users can be allowed on the system during a system restore.
The functionality of the restore process, jrestore, has been extended to allow for the automatic roll-forward of logsets after a database restore has completed. There are three options which an administrator may use in the recovery of the database :
A database restore from the last backup – no option required. This is the standard restore process and would be used following an off-line backup. If the backup was an online backup and updates were made to the database while the system was being backed up, then the database may (will) be left in an inconsistent state.
A full system recovery (option –W): this will return the database to the state it was at the completion of the backup. The database will be in a consistent state. Functionally, this procedure is: jrestore (all files are recovered); roll-forward of the Transaction Journal files – these files follow the standard backup on the backup
A full system recovery followed by a roll-forward of all transactions which occurred following the last backup to the point of failure (options –W –G). This will leave the database in a consistent state. Only complete transactions will be recovered. It is the responsibility of the database administrator to determine which transactions have not been recovered by this mechanism. These transactions will be those which were
incomplete at the point of failure.
(i) The various operations are defined by the options chosen.
(ii) The Transaction Journal (if present) is saved. If the required recovery is to the point of failure, then this configuration will be used for the roll forward of all transactions since the backup completed.
(iii) The Transaction Journal was saved following the database on the same media. This configuration reflects the state of journaling, including journal files upon completion of the backup. The “current” logset indicated in this configuration is that logset which was in use during the backup.
(iv) If a Transaction Journal configuration file existed prior to the commencement of the recovery process, then this is restored. If no configuration existed at this time (media failure perhaps – full system recovery), then the configuration is that which was in force at the end of the backup.
(v) If a full database recovery to the point of failure is specified, then a roll-forward of all completed transactions takes place using the “current” logset as defined in the configuration.
It is required to return the database to a consistent state following a detected problem
since the last backup finished. In the case where the database and logsets reside on
the same disk set and that disk has become corrupt or unusable, (and the Transaction
Journal is not being copied to offline media), it is not possible to recover any
transactional data entered since the end of the last backup.
The command to use in this instance could be:
jrestore -f /dataset/backuptues -W
This assumes that the database will be recreated by the standard jrestore mechanisms. Following this restore, the Transaction logfiles which were saved during the online backup are restored along with the corresponding
jediLoggerConfig file. A roll-forward command of the form:
jlogdup input set=n terminate=eos output set=database
is used internally to roll-forward the database. The set number specification is retrieved from the loaded jediLoggerConfig file as being the current logset. The current logset this refers to is that configured logset which was current for the duration of the online backup. The location of the backup file must be known by the
administrator (in this case /dataset/backuptues ).
At the completion of this internal command the database is returned to a consistent state, indeed the state of the database at the completion of the backup.
In this example the database and logsets are stored on different disk sets physically. In the event that the database disk is lost, it is possible, not only to restore the database to a consistent state, but also possible to roll-forward all transactions which have occurred since the end of the last backup, using the logset stored on another disk. The important configuration required for this to succeed is that the jediLoggerConfig file cannot exist on the database disk but on the disk where the Transaction Logfiles are stored. This is achieved by the use of the JBCLOGCONFDIR environment variable. This variable defines a directory/folder called “config” somewhere on the logfile-resident disk. The contents of this config directory will be created automatically when logfiles are configured by the jlogadmin facility.
The command to run in this instance is :
jrestore -f /dataset/backuptues -W -G
The use of the “-G” option changes the restore operation thus:
Prior to restoring the jediLoggerConfig file from the backup medium, the jediLoggerConfig file which is currently in use (as specified by the JBCLOGCONFDIR variable), is saved to disk. The operations are then performed
exactly as per example 1 and then the following occurs:
The saved jediLoggerConfig file is restored and is then used in a further embedded jlogdup command. This time the set to use for input is taken from the jediLoggerConfig just restored from disk. This is the set which was active at the start of the jrestore operation (in contrast to the set which was current during the backup).
This roll-forward will update the database with all updates since the end of the backup, in transactions which have committed in their entirety. This ensures that the database will again be left in a consistent state.