Backup with rsync questions

[dferguson]

I have a few questions regarding rsync as a backup method. When I used the script example it backed up up my entire /home/openkm/ folder. Not sure that is what I want as that folder contains some other applications, workspaces, etc. So I have edited the rsync command to only backup the tomcat folder.

You will also note that I have commented out the mount for now. I will rsync locally until I sort out all the issues.

Code: Select all

#!/bin/bash
#
## BEGIN CONFIG ##
HOST=$(uname -n)
DATABASE_PASS="*secret*"
OPENKM_DB="okmdb"
OPENKM_HOME="/home/openkm"
TOMCAT_HOME="$OPENKM_HOME/tomcat-7.0.61"
DATABASE_EXP="$OPENKM_HOME/ABTopenkm/db"
BACKUP_DIR="/home/openkm/ABTopenkm"
RSYNC_OPTS="-apzhR --stats --delete --exclude=*~ --delete-excluded"
## END CONFIG ##

# Check root user 
if [ $(id -u) != 0 ]; then echo "You should run this script as root"; exit; fi

# Delete older local database backup 
echo -e "### BEGIN: $(date +"%x %X") ###\n"
rm -rf $DATABASE_EXP
mkdir -p $DATABASE_EXP
 
# Mount disk
#if mount | grep "$BACKUP_DIR type" > /dev/null; then
#  echo "$BACKUP_DIR already mounted";
#else
#  mount "$BACKUP_DIR";
#  
#  if mount | grep "$BACKUP_DIR type" > /dev/null; then
#    echo "$BACKUP_DIR mounted";
#  else
#    echo "$BACKUP_DIR error mounting";
#    exit -1;
#  fi
#fi
 
# Stop Tomcat
/etc/init.d/tomcat stop
 
# Clean logs
echo "Clean Tomcat temporal files."
rm -rf $TOMCAT_HOME/logs/*
rm -rf $TOMCAT_HOME/temp/*
rm -rf $TOMCAT_HOME/work/Catalina/localhost
 
# Backup database
if [ -n "$DATABASE_PASS" ]; then
  echo "* Backuping MySQL data from $OPENKM_DB..."
  mysqldump -h localhost -u root -p$DATABASE_PASS $OPENKM_DB > $DATABASE_EXP/mysql_$OPENKM_DB.sql
  echo "-------------------------------------";
fi
 
# Create backup
rsync $RSYNC_OPTS $TOMCAT_HOME $BACKUP_DIR/
 
# Start Tomcat
/etc/init.d/tomcat start
echo -e "\n### END: $(date +"%x %X") ###"
 
# Umount disk
#sync
#umount "$BACKUP_DIR"

When I opened the backup destination as defined in my config section "/home/openkm/ABTopenkm/", I expected to see the contents of the tomcat folder. Instead I saw /home/openkm/ABTopenkm/home/openkm/tomcat/

what is causing this behavior?

[jllort]

might be one of these options "apzhR" is indicating the backup must be with full path ? I have not reviewed, but I suggest take a look into rsync command like help to identify each parameter for what is it.

[dferguson]

(a) --archive: archive mode; equals -rlptgoD (no -H,-A,-x)
(p) --perms: preserve permissions
(z) --compress: compress file data during transfer
(h) --human-readable: output numbers in human readable format
(R) --relative: use relative path names

[jllort]

Seems the R option is not working

[dferguson]

I'll look into that. For now the redundancy in the path isn't a big deal.

However, I am currently using two methods to backup my application and data.

1st, I am creating a tar of the entire tomcat folder and archiving locally for 5 days and externally for 50 days.

2nd, I am using rsync to do the sql dump and backup of the app as given in the example backup.sh text in the community documents. After this I am creating another tar of the these items and archiving them in the same manner as above. 5 most recent locally and 50 most recent externally.

Not sure which is best. I believe the power of rsync is that it only writes new data, so could be potentially faster. However, I want to preserve a history of the database and app (50 days is probably overkill), so is rsync really better.

[dferguson]

I am trying to restore on a VM to practice recovering from a crash or corruption.

I have taken the entire /tomcat-7.0.61/ and copied it to the VM and edited the server.xml with the appropriate IP address, but when I start up the app, the old okm:root and seemingly the rest of the old OpenKM database that I previously had on this VM. How do I force the loading of the new repo and configuration parameters?

[dferguson]

Ok,

I think I have successfully answered my own questions.

Restoring the backup documentation for 6.3.4 didn't specifically call out how to update the mysql with the dump file.

taking the dump file I was able to follow these steps to rebuild the database on a mirror instance:

1. Stop tomcat on mirror
2. replace mirror /tomcat/ folder with backup from main instance backup of /tomcat/ folder
3. locate the mysql_okmdb.sql dump from the backup
4. at terminal

Code: Select all

mysql -u openkm -p okmdb < /path/to/destination/mysql_okmdb.sql

5. restart mysql
6. start tomcat

After these steps my app loaded with the correct database, metadata, users, etc.

Therefore using the backup.sh with rsync given in the user docs is a better solution because it implements the database dump.

[jllort]

Better always backup the tomcat folder, in case you decide for replace for zip one, remember to change hibernate.hbml=none otherwise will drop entire database.

Finally consider taking a look at this other backup tool https://docs.openkm.com/kcenter/view/ok ... ackup.html what goes better than rdiff-backup ( both are incremental backup tool )

[dferguson]

I have developed an issue where my backups arent happening. The backup cron job gets executed but nothign happens. When I open the log I see...

Stopping TomcatPID file found but no matching process was found. Stop aborted.

How can I solve this?

[jllort]

When the tomcat id is not found - basically catalina.pid file into tomcat folder created automatically when tomcat is started as service - is caused because OpenKM has not been started as a service. Be sure you are starting and stopping OpenKM as service.

[dferguson]

I assume OpenKM starts as a service, because I don't actively have to do anything at boot.

Occasionally my Ubuntu 16.04 crashes/freezes and I have no keyboard/mouse control and monitor is black. Furthermore I can't SSH to the machine either. My only solution is a hard reboot. Sometimes everything returns normally, other times I have to start and stop the tomcat service repeatedly before everything returns to normal. However now with this Rsync option it appears that even though OpenKM runs fine the backup.sh does not execute at the next cron job time (as I described in previpus post).

To resolve this I had to delete the catalina.pid file in the tomcat_home folder and the restart the Tomcat service. After that the backup.sh worked as intended.

Can I ask a question, are catalina, tomcat, and openkm the same? I see reference to each and quiet can't figure out each's role and are they incorrectly refereed to interchangeably?

[jllort]

Seems the application is not stopping correctly. I suggest check it and in case the application is not starting or stopping correctly start a new post with it. Indicate the number of files, hardware resources etc... for a better understanding of your scenario.

[dferguson]

Ok, thanks. I have had reliable operation with both Ubuntu and Openkm since deleting the catalina.pid file that last time. I will monitor in the future.