• (HSQLDB + Linux) Power outage leads to data loss - how to recover?

  • We tried to make OpenKM as intuitive as possible, but an advice is always welcome.
We tried to make OpenKM as intuitive as possible, but an advice is always welcome.
Forum rules: Please, before asking something see the documentation wiki or use the search feature of the forum. And remember we don't have a crystal ball or mental readers, so if you post about an issue tell us which OpenKM are you using and also the browser and operating system version. For more info read How to Report Bugs Effectively.
 #40390  by insuranceguy
 
Synopsis: how to recover lost files, after a power outage/bad shut down of a misconfigured community OpenKM installation under Linux (Debian).

Scenario: For some strange reason your mission critical OpenKM installation is using the default HSQLDB rather than SQL. Also unknown to you, the system shutdown scripts, triggered by your battery backup UPC, don't give Tomcat & OpenKM enough time to shutdown properly.

So one day you have a power outage. Everything seems fine, until at startup, you notice that seven days of work in OpenKM is missing. (or at least since the last time you properly shutdown Tomcat to do a backup) :shock: :oops: :cry:

(Please let me know if there's a better way to do this. I just figured out this method of recovering my work, and it truly saved me a lot a grief!)

Solution: the default HSQLDB appears to store the database organizing your OpenKM repository in RAM only, until a proper shutdown/restart. At restart, it will appear in OpenKM that your latest documents are missing! But the actual pdf files are still there in the repository "datastore" folder. How to find them all, and copy them to a recovery directory to get re-filed by hand in the OpenKM front-end:

# find [REPO-PATH] -ctime -[days-since-backup + 1] -type f ! -name "*.*" -exec cp --preserve=timestamps '{}' [TEMP-RECOVERY-DIRECTORY] \;

or in my case...

# find /repository -ctime -8 -type f ! -name "*.*" -exec cp --preserve=timestamps '{}' /home/recovery/ \;

This searches your repository for your missing pdf files less than # days old (7 in my case). OpenKM saves them using a UUID with no extension, so with out the ! -name "*.*" option you'll accidentally find a bunch of .swf files as well. The -exec part will copy the found files to your temporary recovery directory, while preserving your timestamps so filing will be easier. Once they're there, you'll want them all to have a .pdf extension:

# cd [TEMP-RECOVERY-DIRECTORY]
# find ./ -exec cp --preserve=timestamps '{}' '{}'.pdf \; [changes them all to .pdf while preserving timestamps]
# find ./ ! -name "*.pdf" -exec rm '{}' \; [delete the old copies without .pdf extensions]

I just wanted to share this with the OpenKM community, in case there's one person out there that ever runs into the same situation, with a do-it-yourself OpenKM setup. In my case, it saved me a week's worth of work. :D

Of course, the best solution is to know what you're doing in the first place, and have a proper setup :roll: ...lesson learned, but it wasn't fatal. Now, time to figure out my shutdown scripts, and get things switched over to SQL! Also, OpenKM rocks!
 #40399  by jllort
 
The files are stored at tomcat/repository/datastore in a non human format, their name is the uuid. Try to recover with http://wiki.openkm.com/index.php/Disaster_recovering

My suggestion is move to MySQL server ( export and import to new configuration ) when you got the files recovered and do backup to external network unit or usb ( read here http://wiki.openkm.com/index.php/Backup ... -_MySQL.29 )

About migrating from hsqldb to mysql, a good option could be export entire repository and import again, but also could try an exiting migration procedure seems the mysql migration toolkit http://downloads.mysql.com/archives/migration/ has been deprecated in favour the mysql workbench http://dev.mysql.com/downloads/workbench/ I've never tested this procedure but seems reasonable. Anyway do a backup before trying it. If you go for hsqldb to mysql well be pleased if you share your feedback, can be useful for other users.

About Us

OpenKM is part of the management software. A management software is a program that facilitates the accomplishment of administrative tasks. OpenKM is a document management system that allows you to manage business content and workflow in a more efficient way. Document managers guarantee data protection by establishing information security for business content.