Page 1 of 1

Slow rebuilding Lucene Indexes

PostPosted:Mon Nov 22, 2021 4:19 pm
by Greind
Hi,

We're currently having more than 205,000 documents inside the system.
However, there was an unexpected shutdown that causes indexes corrupted.
Thus,
https://docs.openkm.com/kcenter/view/ok ... error.html.

Right now we're rebuilding Lucene indexes. However, there's no progress bar to monitor the progress. - Right now we are comparing the list of indexes and searching pdf files and see the total and compare it to the total documents inside the system.

After around 96 hours of rebuilding progress, we can only get around 12% of documents that has been indexes. Also already have 24GB of RAM and 90% consumed already.

1. Is there a way to monitor the progress?
2. Is there a way to increase the speed of rebuilding the indexes?
3. Is there a way to know what is eating up the RAM?

We're in need of urgent help. Thank you in advance for all of your help.
Without this indexing complete, we're not able to do searching for document names and metadata.

Re: Slow rebuilding Lucene Indexes

PostPosted:Sat Nov 27, 2021 12:21 pm
by jllort
It seems poor hardware performance, specially I/O disk. The Lucene search engine goes across the database and update records in the file system ( Lucene search store the index in the disk ) -> which means a lot of I/O transactions. If you have bad disk performance then can get this kind of result. Also could be a problem with the database, but in 99% of cases, it is a disk performance problem.