When the ExaCS database backup is configured using the OCI Web Console, sometimes you might notice that the autobackups are getting failed even if no changes are made to the databases or any other configuration.
Whenever this happens, you will see a message in the OCI console next to the "failed" message for the particular database backup
Once this happens, the first thing you should check is whether the DBaaS tooling on the particular cluster is updated or not. To update the tooling to latest version, please follow the guide from this post.
Once the tooling is updated, please wait until the next autobackup to confirm if it is successful. You can always take a manual backup, but sometimes even after the manual backup is successful, you will have failures in autobackups.
If the tooling update is not solving the failures, next step is to check the log files on the server.
The log files for database backup will be stored in the directory:
/var/opt/oracle/log/DB_NAME/obkup/
Look of the RMAN log files in the above directory. The name of the log file will be similar to :
obkup_2021-11-01_14:53:25.374093353128.log
In my case, checking the file /var/opt/oracle/log/DB_NAME/obkup/obkup_2021-11-01_14:53:25.374093353128.log showed me the below message:
ORA-19625: error identifying file +RECOC1/DB01_XX1KP/ARCHIVELOG/2021_01_06/thread_1_seq_12441.9290.1061108833
The solution
Solution is pretty simple. Since RMAN was not able to find some archivelog files, I ran a crosscheck command for the database in question:
rman target /
crosscheck archivelog all;
I did not try any manual database backups after running crosscheck. The next autobackups were all successful :)