Open Source Document Management System | OpenKM

Because information matters
https://forum.openkm.com/

unable to reopen IndexReader when using SDK4J find method

https://forum.openkm.com/viewtopic.php?t=12483

Page 1 of 1

unable to reopen IndexReader when using SDK4J find method

PostPosted:Tue Aug 26, 2014 2:15 pm
by Catscratch
Hi,

I try to find documents by using the SDK.
SDK 1.0
OpenKM 6.3.0 Community
Code: Select all
OKMWebservices okm = OKMWebservicesFactory.newInstance("http://localhost:8080/OpenKM", "okmAdmin", "admin");
QueryParams p = new QueryParams();
p.setDomain(QueryParams.DOCUMENT);
List<QueryResult> list = okm.find(p);
On the last line (find) I get an UnknownException.
Code: Select all
com.openkm.sdk4j.exception.UnknowException:  Unable to reopen IndexReader
	at com.openkm.sdk4j.impl.SearchImpl.find(SearchImpl.java:283)
	at com.openkm.sdk4j.OKMWebservice10.find(OKMWebservice10.java:829)
	at TestOkmSdk.test(TestOkmSdk.java:57)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.lang.reflect.Method.invoke(Unknown Source)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
	at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
	at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:459)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:675)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:382)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:192)
After a fresh restart of OpenKM I get results, but not all documents. Only a few. But after some time I only get the exception above. So it seems so be a problem with the lucene indexer I think. What can I do to prevent this?

Thanks!

Re: unable to reopen IndexReader when using SDK4J find metho

PostPosted:Thu Aug 28, 2014 7:53 am
by jllort
First, this is a quite strange search, because you're not filtering for any field, that is not good idea. If you wan to get all documents you got in repository this is not the way for doing it. If you for example do a query with name "*.doc" you got the same type of error ?

Re: unable to reopen IndexReader when using SDK4J find metho

PostPosted:Thu Aug 28, 2014 8:16 am
by Catscratch
Yes.

I added an additional filter param:
Code: Select all
p.setName("*.png");
That should return only 1 document. Because there is only 1 png.
But the same exception appears.

In the server log there is only a warning:
Code: Select all
2014-08-28 10:12:11,189 [http-apr-0.0.0.0-8080-exec-3] WARN  org.apache.cxf.jaxrs.impl.WebApplicationExceptionMapper- WebApplicationException has been caught : Unable to reopen IndexReader
But on the server side I also got a lot of exceptions after rebooting:
Code: Select all
java.io.FileNotFoundException: /opt/openkm63/repository/index/com.openkm.dao.bean.NodeBase/_e.fnm (Datei oder Verzeichnis nicht gefunden)
	at java.io.RandomAccessFile.open(Native Method)
	at java.io.RandomAccessFile.<init>(RandomAccessFile.java:241)
	at org.apache.lucene.store.SimpleFSDirectory$SimpleFSIndexInput$Descriptor.<init>(SimpleFSDirectory.java:69)
	at org.apache.lucene.store.SimpleFSDirectory$SimpleFSIndexInput.<init>(SimpleFSDirectory.java:90)
	at org.apache.lucene.store.NIOFSDirectory$NIOFSIndexInput.<init>(NIOFSDirectory.java:91)
	at org.apache.lucene.store.NIOFSDirectory.openInput(NIOFSDirectory.java:78)
	at org.apache.lucene.store.FSDirectory.openInput(FSDirectory.java:353)
	at org.apache.lucene.index.FieldInfos.<init>(FieldInfos.java:68)
	at org.apache.lucene.index.IndexWriter.getFieldInfos(IndexWriter.java:1197)
	at org.apache.lucene.index.IndexWriter.getCurrentFieldInfos(IndexWriter.java:1213)
	at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:1149)
	at org.hibernate.search.backend.Workspace.createNewIndexWriter(Workspace.java:202)
	at org.hibernate.search.backend.Workspace.getIndexWriter(Workspace.java:180)
	at org.hibernate.search.backend.impl.lucene.PerDPQueueProcessor.run(PerDPQueueProcessor.java:103)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2014-08-26 20:30:04,139 [Hibernate Search: Directory writer-1] ERROR org.hibernate.search.backend.impl.lucene.PerDPQueueProcessor- Unexpected error in Lucene Backend: 
org.hibernate.search.SearchException: Unable to remove class com.openkm.dao.bean.NodeDocument#8c14d400-cdda-4b85-aee2-44f54e0f432a from index.
	at org.hibernate.search.backend.impl.lucene.works.DeleteWorkDelegate.performWork(DeleteWorkDelegate.java:91)
	at org.hibernate.search.backend.impl.lucene.PerDPQueueProcessor.run(PerDPQueueProcessor.java:106)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NullPointerException
	at org.hibernate.search.backend.impl.lucene.works.DeleteWorkDelegate.performWork(DeleteWorkDelegate.java:87)
	... 6 more
and
Code: Select all
2014-08-26 20:30:04,140 [Hibernate Search: Directory writer-1] ERROR org.hibernate.search.exception.impl.LogErrorHandler- Exception occurred org.hibernate.search.SearchException: Unable to remove class com.openkm.dao.bean.NodeDocument#8c14d400-cdda-4b85-aee2-44f54e0f432a from index.
Primary Failure:
	Entity com.openkm.dao.bean.NodeDocument  Id 8c14d400-cdda-4b85-aee2-44f54e0f432a  Work Type  org.hibernate.search.backend.DeleteLuceneWork
Subsequent failures:
	Entity com.openkm.dao.bean.NodeDocument  Id 8c14d400-cdda-4b85-aee2-44f54e0f432a  Work Type  org.hibernate.search.backend.AddLuceneWork
	Entity com.openkm.dao.bean.NodeDocument  Id 8c14d400-cdda-4b85-aee2-44f54e0f432a  Work Type  org.hibernate.search.backend.AddLuceneWork

org.hibernate.search.SearchException: Unable to remove class com.openkm.dao.bean.NodeDocument#8c14d400-cdda-4b85-aee2-44f54e0f432a from index.
	at org.hibernate.search.backend.impl.lucene.works.DeleteWorkDelegate.performWork(DeleteWorkDelegate.java:91)
	at org.hibernate.search.backend.impl.lucene.PerDPQueueProcessor.run(PerDPQueueProcessor.java:106)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NullPointerException
	at org.hibernate.search.backend.impl.lucene.works.DeleteWorkDelegate.performWork(DeleteWorkDelegate.java:87)
	... 6 more
2014-08-26 20:30:04,140 [Hibernate Search: Directory writer-1] WARN  org.hibernate.search.backend.Workspace- going to force release of the IndexWriter lock
That's why I removed the "index" folder from the "repository" folder to recreate the index. But after some time I run in the same exceptions above.

Re: unable to reopen IndexReader when using SDK4J find metho

PostPosted:Sat Aug 30, 2014 10:23 am
by jllort
Seems you got corruption in lucene, probably the computer or the service has shut down on past and from there you got it. First of all I suggest, always got backup and use MySQL or PostgreSQL database if you use default database HSQL ( embeded ) and being corrupted in 90% of cases is not possible recover ( take it in mind ).

In case of lucene repository corruption simply, stop application. Remove folder opt/openkm63/repository/index/. Start application again. Automatically the lucene index will be rebuild ( depending the number of files etc... you'll got top cpu use during some minutes or hours ).

Re: unable to reopen IndexReader when using SDK4J find metho

PostPosted:Sun Aug 31, 2014 9:26 am
by Catscratch
I tried removing the index folder a few times now. After restarting openkm, lucene starts building the index again, which works for some time. But the on a certain point (after some time) I got the exceptions above.

I don't know why. Maybe to many documents? (about 25000)

Re: unable to reopen IndexReader when using SDK4J find metho

PostPosted:Sun Aug 31, 2014 9:12 pm
by pavila
25K documents are very few documents and Lucene index corruption is quite rare, unless you have a problem with the hard disk. I recommend to try a clean OpenKM installation in another machine, and import here all these documents.

Re: unable to reopen IndexReader when using SDK4J find metho

PostPosted:Mon Sep 01, 2014 6:53 am
by Catscratch
Ok, I'll give it a try. Thanks so far.

Re: unable to reopen IndexReader when using SDK4J find metho

PostPosted:Tue Sep 02, 2014 10:37 am
by jllort
As pavila suggest, could be some hardware problem ( disk sector fails ) what will be the cause of this corruption.
All times are UTC
Page 1 of 1
Powered by phpBB® Forum Software © phpBB Limited