• Backup with rsync questions

  • We tried to make OpenKM as intuitive as possible, but an advice is always welcome.
We tried to make OpenKM as intuitive as possible, but an advice is always welcome.
Forum rules: Please, before asking something see the documentation wiki or use the search feature of the forum. And remember we don't have a crystal ball or mental readers, so if you post about an issue tell us which OpenKM are you using and also the browser and operating system version. For more info read How to Report Bugs Effectively.
 #45346  by dferguson
 
I have a few questions regarding rsync as a backup method. When I used the script example it backed up up my entire /home/openkm/ folder. Not sure that is what I want as that folder contains some other applications, workspaces, etc. So I have edited the rsync command to only backup the tomcat folder.

You will also note that I have commented out the mount for now. I will rsync locally until I sort out all the issues.
Code: Select all
#!/bin/bash
#
## BEGIN CONFIG ##
HOST=$(uname -n)
DATABASE_PASS="*secret*"
OPENKM_DB="okmdb"
OPENKM_HOME="/home/openkm"
TOMCAT_HOME="$OPENKM_HOME/tomcat-7.0.61"
DATABASE_EXP="$OPENKM_HOME/ABTopenkm/db"
BACKUP_DIR="/home/openkm/ABTopenkm"
RSYNC_OPTS="-apzhR --stats --delete --exclude=*~ --delete-excluded"
## END CONFIG ##

# Check root user 
if [ $(id -u) != 0 ]; then echo "You should run this script as root"; exit; fi

# Delete older local database backup 
echo -e "### BEGIN: $(date +"%x %X") ###\n"
rm -rf $DATABASE_EXP
mkdir -p $DATABASE_EXP
 
# Mount disk
#if mount | grep "$BACKUP_DIR type" > /dev/null; then
#  echo "$BACKUP_DIR already mounted";
#else
#  mount "$BACKUP_DIR";
#  
#  if mount | grep "$BACKUP_DIR type" > /dev/null; then
#    echo "$BACKUP_DIR mounted";
#  else
#    echo "$BACKUP_DIR error mounting";
#    exit -1;
#  fi
#fi
 
# Stop Tomcat
/etc/init.d/tomcat stop
 
# Clean logs
echo "Clean Tomcat temporal files."
rm -rf $TOMCAT_HOME/logs/*
rm -rf $TOMCAT_HOME/temp/*
rm -rf $TOMCAT_HOME/work/Catalina/localhost
 
# Backup database
if [ -n "$DATABASE_PASS" ]; then
  echo "* Backuping MySQL data from $OPENKM_DB..."
  mysqldump -h localhost -u root -p$DATABASE_PASS $OPENKM_DB > $DATABASE_EXP/mysql_$OPENKM_DB.sql
  echo "-------------------------------------";
fi
 
# Create backup
rsync $RSYNC_OPTS $TOMCAT_HOME $BACKUP_DIR/
 
# Start Tomcat
/etc/init.d/tomcat start
echo -e "\n### END: $(date +"%x %X") ###"
 
# Umount disk
#sync
#umount "$BACKUP_DIR"
When I opened the backup destination as defined in my config section "/home/openkm/ABTopenkm/", I expected to see the contents of the tomcat folder. Instead I saw /home/openkm/ABTopenkm/home/openkm/tomcat/

what is causing this behavior?
 #45350  by jllort
 
might be one of these options "apzhR" is indicating the backup must be with full path ? I have not reviewed, but I suggest take a look into rsync command like help to identify each parameter for what is it.
 #45353  by dferguson
 
(a) --archive: archive mode; equals -rlptgoD (no -H,-A,-x)
(p) --perms: preserve permissions
(z) --compress: compress file data during transfer
(h) --human-readable: output numbers in human readable format
(R) --relative: use relative path names
 #45366  by dferguson
 
I'll look into that. For now the redundancy in the path isn't a big deal.

However, I am currently using two methods to backup my application and data.

1st, I am creating a tar of the entire tomcat folder and archiving locally for 5 days and externally for 50 days.

2nd, I am using rsync to do the sql dump and backup of the app as given in the example backup.sh text in the community documents. After this I am creating another tar of the these items and archiving them in the same manner as above. 5 most recent locally and 50 most recent externally.

Not sure which is best. I believe the power of rsync is that it only writes new data, so could be potentially faster. However, I want to preserve a history of the database and app (50 days is probably overkill), so is rsync really better.
 #45369  by dferguson
 
I am trying to restore on a VM to practice recovering from a crash or corruption.

I have taken the entire /tomcat-7.0.61/ and copied it to the VM and edited the server.xml with the appropriate IP address, but when I start up the app, the old okm:root and seemingly the rest of the old OpenKM database that I previously had on this VM. How do I force the loading of the new repo and configuration parameters?
 #45370  by dferguson
 
Ok,

I think I have successfully answered my own questions.

Restoring the backup documentation for 6.3.4 didn't specifically call out how to update the mysql with the dump file.

taking the dump file I was able to follow these steps to rebuild the database on a mirror instance:

1. Stop tomcat on mirror
2. replace mirror /tomcat/ folder with backup from main instance backup of /tomcat/ folder
3. locate the mysql_okmdb.sql dump from the backup
4. at terminal
Code: Select all
mysql -u openkm -p okmdb < /path/to/destination/mysql_okmdb.sql
5. restart mysql
6. start tomcat

After these steps my app loaded with the correct database, metadata, users, etc.

Therefore using the backup.sh with rsync given in the user docs is a better solution because it implements the database dump.
 #45375  by jllort
 
Better always backup the tomcat folder, in case you decide for replace for zip one, remember to change hibernate.hbml=none otherwise will drop entire database.

Finally consider taking a look at this other backup tool https://docs.openkm.com/kcenter/view/ok ... ackup.html what goes better than rdiff-backup ( both are incremental backup tool )
 #45512  by dferguson
 
I have developed an issue where my backups arent happening. The backup cron job gets executed but nothign happens. When I open the log I see...
Stopping TomcatPID file found but no matching process was found. Stop aborted.
How can I solve this?
 #45526  by jllort
 
When the tomcat id is not found - basically catalina.pid file into tomcat folder created automatically when tomcat is started as service - is caused because OpenKM has not been started as a service. Be sure you are starting and stopping OpenKM as service.
 #45538  by dferguson
 
I assume OpenKM starts as a service, because I don't actively have to do anything at boot.

Occasionally my Ubuntu 16.04 crashes/freezes and I have no keyboard/mouse control and monitor is black. Furthermore I can't SSH to the machine either. My only solution is a hard reboot. Sometimes everything returns normally, other times I have to start and stop the tomcat service repeatedly before everything returns to normal. However now with this Rsync option it appears that even though OpenKM runs fine the backup.sh does not execute at the next cron job time (as I described in previpus post).

To resolve this I had to delete the catalina.pid file in the tomcat_home folder and the restart the Tomcat service. After that the backup.sh worked as intended.

Can I ask a question, are catalina, tomcat, and openkm the same? I see reference to each and quiet can't figure out each's role and are they incorrectly refereed to interchangeably?
 #45567  by jllort
 
Seems the application is not stopping correctly. I suggest check it and in case the application is not starting or stopping correctly start a new post with it. Indicate the number of files, hardware resources etc... for a better understanding of your scenario.
 #45599  by dferguson
 
Ok, thanks. I have had reliable operation with both Ubuntu and Openkm since deleting the catalina.pid file that last time. I will monitor in the future.

About Us

OpenKM is part of the management software. A management software is a program that facilitates the accomplishment of administrative tasks. OpenKM is a document management system that allows you to manage business content and workflow in a more efficient way. Document managers guarantee data protection by establishing information security for business content.