• Slow rebuilding Lucene Indexes

  • OpenKM has many interesting features, but requires some configuration process to show its full potential.
OpenKM has many interesting features, but requires some configuration process to show its full potential.
Forum rules: Please, before asking something see the documentation wiki or use the search feature of the forum. And remember we don't have a crystal ball or mental readers, so if you post about an issue tell us which OpenKM are you using and also the browser and operating system version. For more info read How to Report Bugs Effectively.
 #53043  by Greind
 
Hi,

We're currently having more than 205,000 documents inside the system.
However, there was an unexpected shutdown that causes indexes corrupted.
Thus,
https://docs.openkm.com/kcenter/view/ok ... error.html.

Right now we're rebuilding Lucene indexes. However, there's no progress bar to monitor the progress. - Right now we are comparing the list of indexes and searching pdf files and see the total and compare it to the total documents inside the system.

After around 96 hours of rebuilding progress, we can only get around 12% of documents that has been indexes. Also already have 24GB of RAM and 90% consumed already.

1. Is there a way to monitor the progress?
2. Is there a way to increase the speed of rebuilding the indexes?
3. Is there a way to know what is eating up the RAM?

We're in need of urgent help. Thank you in advance for all of your help.
Without this indexing complete, we're not able to do searching for document names and metadata.
Attachments
Login Error.jpeg
Login Error.jpeg (25.51 KiB) Viewed 92 times
Cannot release connection.PNG
Cannot release connection.PNG (70.52 KiB) Viewed 92 times
 #53057  by jllort
 
It seems poor hardware performance, specially I/O disk. The Lucene search engine goes across the database and update records in the file system ( Lucene search store the index in the disk ) -> which means a lot of I/O transactions. If you have bad disk performance then can get this kind of result. Also could be a problem with the database, but in 99% of cases, it is a disk performance problem.

About Us

OpenKM is part of the management software. A management software is a program that facilitates the accomplishment of administrative tasks. OpenKM is a document management system that allows you to manage business content and workflow in a more efficient way. Document managers guarantee data protection by establishing information security for business content.