Page 1 of 1

OpenKM 6.3 Fatal Error after restore from backup

PostPosted:Fri Oct 09, 2015 5:17 pm
by paul.crown
Greetings,

I am still learning about OpenKM. I am the only user on my installation. I previously installed and was using OpenKM fine. I had a hard drive failure. So, I restored my installation from a system backup.

Now my login screen now has a red triangle where the OpenKM logo was before. I can login, but immediately get a system error: Application internal error.

I have stopped the installation. Deleted catalina.log, and restarted the installation. The first error that occurs is
Code: Select all
2015-10-09 11:39:43,912 [localhost-startStop-1] FATAL hsqldb.db.HSQLDB4A2D15DEF1.ENGINE- statement error processing log /mnt/storage/openkm/tomcat/repository/okmdbline: 110
org.hsqldb.HsqlException: integrity constraint violation: unique constraint or index violation; SYS_PK_10279 table: OKM_CRON_TAB
        at org.hsqldb.error.Error.error(Unknown Source)
        at org.hsqldb.Constraint.getException(Unknown Source)
        at org.hsqldb.index.IndexAVLMemory.insert(Unknown Source)
        at org.hsqldb.persist.RowStoreAVL.indexRow(Unknown Source)
        at org.hsqldb.TransactionManager2PL.addInsertAction(Unknown Source)
        at org.hsqldb.Session.addInsertAction(Unknown Source)
        at org.hsqldb.Table.insertNoCheckFromLog(Unknown Source)
        at org.hsqldb.persist.ScriptRunner.runScript(Unknown Source)
        at org.hsqldb.persist.ScriptRunner.runScript(Unknown Source)
        at org.hsqldb.persist.Log.processLog(Unknown Source)
        at org.hsqldb.persist.Log.open(Unknown Source)
        at org.hsqldb.persist.Logger.openPersistence(Unknown Source)
        at org.hsqldb.Database.reopen(Unknown Source)
        at org.hsqldb.Database.open(Unknown Source)
        at org.hsqldb.DatabaseManager.getDatabase(Unknown Source)
        at org.hsqldb.DatabaseManager.newSession(Unknown Source)
        at org.hsqldb.jdbc.JDBCConnection.<init>(Unknown Source)
        at org.hsqldb.jdbc.JDBCDriver.getConnection(Unknown Source)
        at org.hsqldb.jdbc.JDBCDriver.connect(Unknown Source)
        at org.apache.tomcat.dbcp.dbcp.DriverConnectionFactory.createConnection(DriverConnectionFactory.java:38)
        at org.apache.tomcat.dbcp.dbcp.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:582)
        at org.apache.tomcat.dbcp.dbcp.BasicDataSource.validateConnectionFactory(BasicDataSource.java:1556)
        at org.apache.tomcat.dbcp.dbcp.BasicDataSource.createPoolableConnectionFactory(BasicDataSource.java:1545)
        at org.apache.tomcat.dbcp.dbcp.BasicDataSource.createDataSource(BasicDataSource.java:1388)
        at org.apache.tomcat.dbcp.dbcp.BasicDataSource.getConnection(BasicDataSource.java:1044)
        at org.hibernate.connection.DatasourceConnectionProvider.getConnection(DatasourceConnectionProvider.java:92)
        at org.hibernate.cfg.SettingsFactory.buildSettings(SettingsFactory.java:113)
        at org.hibernate.cfg.Configuration.buildSettingsInternal(Configuration.java:2863)
        at org.hibernate.cfg.Configuration.buildSettings(Configuration.java:2859)
        at org.hibernate.cfg.Configuration.buildSessionFactory(Configuration.java:1870)
        at com.openkm.dao.HibernateUtil.getSessionFactory(HibernateUtil.java:180)
        at com.openkm.dao.HibernateUtil.getSessionFactory(HibernateUtil.java:91)
        at com.openkm.servlet.RepositoryStartupServlet.init(RepositoryStartupServlet.java:100)
        at javax.servlet.GenericServlet.init(GenericServlet.java:158)
        at org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1284)
        at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1197)
        at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1087)
        at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:5210)
        at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5493)
        at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
        at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:901)
        at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:877)
        at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:632)
        at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:1073)
        at org.apache.catalina.startup.HostConfig$DeployWar.run(HostConfig.java:1857)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
Can you point me in the right direction as to what I could be looking at further to solve this?

Thank you.

Re: OpenKM 6.3 Fatal Error after restore from backup

PostPosted:Sat Oct 10, 2015 3:59 pm
by jllort
The error seems clear, the database seems corrupted.
Code: Select all
/mnt/storage/openkm/tomcat/repository/okmdbline: 110
org.hsqldb.HsqlException: integrity constraint violation: unique constraint or index violation; SYS_PK_10279 table: OKM_CRON_TAB
Is not good idea use Hypersonic database on production environments, we suggest use MySQL, PostgreSQL or other. When Hypersonic database is corrupted is quite difficult or impossible to recover it.

Anyway is strange error for a corrupted database "SYS_PK_10279 table: OKM_CRON_TAB" how did you do the backup, and when ?

Re: OpenKM 6.3 Fatal Error after restore from backup

PostPosted:Sat Oct 10, 2015 4:18 pm
by paul.crown
I am aware and will use MySQL before I let any other users login.

For backups, I use rsync to copy directories to backup. I backed-up openkm last about a week before the error occurred. To restore, I used rsync to recreate the directories and copy back all of the files.

I had guessed that the database was corrupted. I am unfamiliar with Hypersonic databases. Am I able to use the okmdb.script file to "reindex" or "rebuild"?

Thank you.

Re: OpenKM 6.3 Fatal Error after restore from backup

PostPosted:Sun Oct 11, 2015 6:30 pm
by jllort
Try edit okmdb.script ( for example with geany ) and remove any INSERT to the OKM_CRON_TAB ( I do not understand why are getting this error ). Then try to start again.
You're restoring entire tomcat folder no ( with rsync )?
I suggest restore entire backup on other location ( better other server ) and do not overwrite with rsync existing installation.

Re: OpenKM 6.3 Fatal Error after restore from backup

PostPosted:Sun Oct 11, 2015 9:35 pm
by paul.crown
I removed the six INSERT OKM_CRON_TAB lines. Started OpenKM again. The red triangle remains.

Looking at Catalina.log, the FATALs have all gone, but now I have three ERRORs.
Code: Select all
2015-10-11 16:27:05,992 [localhost-startStop-1] ERROR org.hibernate.util.JDBCExceptionReporter- org.hsqldb.HsqlException: file input/output error
2015-10-11 16:27:05,996 [localhost-startStop-1] ERROR com.openkm.core.Config- ** Error reading configuration table **
2015-10-11 16:27:06,159 [localhost-startStop-1] ERROR org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/OpenKM]- Servlet /OpenKM threw load() exception
java.lang.NullPointerException
	at com.openkm.module.db.stuff.SecurityHelper.checkRead(SecurityHelper.java:96)
	at com.openkm.dao.NodeFolderDAO.findByPk(NodeFolderDAO.java:207)
	at com.openkm.module.db.DbRepositoryModule.create(DbRepositoryModule.java:113)
	at com.openkm.module.db.DbRepositoryModule.initialize(DbRepositoryModule.java:85)
	at com.openkm.servlet.RepositoryStartupServlet.start(RepositoryStartupServlet.java:175)
	at com.openkm.servlet.RepositoryStartupServlet.init(RepositoryStartupServlet.java:127)
	at javax.servlet.GenericServlet.init(GenericServlet.java:158)
	at org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1284)
	at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1197)
	at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1087)
	at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:5210)
	at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5493)
	at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
	at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:901)
	at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:877)
	at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:632)
	at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:1073)
	at org.apache.catalina.startup.HostConfig$DeployWar.run(HostConfig.java:1857)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)

Re: OpenKM 6.3 Fatal Error after restore from backup

PostPosted:Sun Oct 11, 2015 9:44 pm
by jllort
Did you changed the grants after restoring the application ? for example if you are executing from openkm user try before executing setting the grants:
Code: Select all
chown -R openkm:openkm /mnt/storage/openkm/
Also ensure you have enough grants to write and delete ( check for it with the same user you're executing the application ).

Re: OpenKM 6.3 Fatal Error after restore from backup

PostPosted:Mon Oct 12, 2015 5:13 pm
by paul.crown
I just added user openkm; it didn't exist. I did change ownership of all openkm files. They are all set to 777 permissions. I am sure this is not right.

However, the 3 error messages are the same.

Re: OpenKM 6.3 Fatal Error after restore from backup

PostPosted:Wed Oct 14, 2015 5:21 pm
by jllort
which user you use to start the application user openkm, root, another ?
with that user you can write into /mnt/storage/openkm ?

Re: OpenKM 6.3 Fatal Error after restore from backup

PostPosted:Wed Oct 14, 2015 6:41 pm
by paul.crown
I started the application with
Code: Select all
sudo tomcat/bin/startup.sh


/mnt/storage/openkm is currently owned by openkm:openkm. Its current permissions are 777; user openkm can read/write/execute; group openkm can read/write/execute; and all others can read/write/execute.

Yes, currently everyone has permissions to /mnt/storage/openkm. I will change this after I have it working again.

Re: OpenKM 6.3 Fatal Error after restore from backup

PostPosted:Thu Oct 15, 2015 3:23 pm
by jllort
whereis the repository folder and where the tomcat ?
post here the contents of the OpenKM.cfg file

Re: OpenKM 6.3 Fatal Error after restore from backup

PostPosted:Mon Oct 19, 2015 8:54 pm
by paul.crown
whereis the repository folder and where the tomcat ?
/mnt/storage/openkm/tomcat/repository
/mnt/storage/openkm/tomcat
post here the contents of the OpenKM.cfg file
Code: Select all
# OpenKM Hibernate configuration values
hibernate.dialect=org.hibernate.dialect.HSQLDialect
hibernate.hbm2ddl=none

# Initial configuration - Linux
system.imagemagick.convert=/usr/bin/convert
system.openoffice.path=/usr/lib/libreoffice
system.swftools.pdf2swf=/opt/openkm/bin/pdf2swf -f -T 9 -t -s storeallcharacters ${fileIn} -o ${fileOut}

# Initial configuration - Windows
#system.imagemagick.convert=C:/openkm/bin/convert.exe
#system.openoffice.path=C:/Program Files (x86)/OpenOffice 4
#system.swftools.pdf2swf=C:/openkm/bin/pdf2swf.exe -f -T 9 -t -s storeallcharacters ${fileIn} -o ${fileOut}

Re: OpenKM 6.3 Fatal Error after restore from backup

PostPosted:Tue Oct 20, 2015 11:42 am
by jllort
Stop OpenKM.
Delete catalina.log file
Start application until the error
Upload here the entire catalina.log file ( into zip )

Re: OpenKM 6.3 Fatal Error after restore from backup

PostPosted:Wed Oct 21, 2015 8:15 pm
by paul.crown
Stopped openkm.
Deleted /mnt/storage/openkm/tomcat/logs/catalina.log
Started openkm.
Zipped /mnt/storage/openkm/tomcat/logs/catalina.log

Zip file attached.
(7.81 KiB) Downloaded 238 times

Re: OpenKM 6.3 Fatal Error after restore from backup

PostPosted:Thu Oct 22, 2015 5:55 pm
by jllort
The code fails here:
Code: Select all
okmRootUuid = NodeBaseDAO.getInstance().getUuidFromPath("/" + Repository.ROOT);
okmRootNode = NodeFolderDAO.getInstance().findByPk(okmRootUuid);
Seems the value okmRootUuid=null
Seeing the log with more detail I see these errors:
Code: Select all
2015-10-21 15:08:36,211 [localhost-startStop-1] WARN  org.hibernate.util.JDBCExceptionReporter- SQL Error: 0, SQLState: null
2015-10-21 15:08:36,211 [localhost-startStop-1] ERROR org.hibernate.util.JDBCExceptionReporter- org.hsqldb.HsqlException: file input/output error
2015-10-21 15:08:36,215 [localhost-startStop-1] ERROR com.openkm.core.Config- ** Error reading configuration table **
I suspect the okmdb ( hypersonic database ) is corrupted or can not be read or similar it has problem. I suggest google for this error:
"org.hibernate.util.JDBCExceptionReporter- org.hsqldb.HsqlException: file input/output error"

Are you sure about the backup is not corrupted ? otherwise will startup without problems. Recover a HSQL database sometimes is not possible, it's a quick way to check openkm but not for production environments.