Page 1 of 1

OpenKM freeze for no apparent reason

Posted: Mon Aug 12, 2019 2:44 pm
by manueldasilva
Hello
for started, sorry for my english ...
we use version 6.3.7 CE and we have a hang that seems random.
When this lock appears, it is impossible to use openkm for about 1 minute.
I can not find anything in the logs (tomcat, openkm or postgresql).
I disabled all the crons, changed cache properties, optimized the indexes without it changing anything except perhaps speed up the display when everything is fine.

would anyone have an eventual track?

Re: OpenKM freeze for no apparent reason

Posted: Tue Aug 13, 2019 1:58 pm
by manueldasilva
We have advanced in our research.
it is the update of the metadata which seems to pose pb

if I repeat several times

Code: Select all

curl -u Admin: Admin -H "Accept: application / json" -X PUT -H "Content type: application / xml" -d <simplePropertiesGroup> <simplePropertyGroup> <name> okp: grp.test </ name> <value> false </ value> </ simplePropertyGroup> </ simplePropertiesGroup> 'http: // localhost: 8080 / OpenKM / services / rest / propertyGroup / setPropertiesSimple? nodeId = 1234 \ & grpName = okg: grp
I freeze openkm for 1min but still without error.

an idea?

Re: OpenKM freeze for no apparent reason

Posted: Thu Aug 15, 2019 10:28 am
by jllort
How many documents do you have? ->

Code: Select all

SELECT COUNT(*) FROM OKM_NODE_BASE
SELECT COUNT(*) FROM OKM_NODE_DOCUMENT
SELECT COUNT(*) FROM OKM_NODE_FOLDER
SELECT COUNT(*) FROM OKM_NODE_MAIL
What hard disk type are you using? SATA ( 7200, 5400 ) , SSD ?

Re: OpenKM freeze for no apparent reason

Posted: Tue Aug 20, 2019 2:28 pm
by manueldasilva
Hello

Code: Select all

OKM_NODE_BASE -> 1087799 
OKM_NODE_DOCUMENT -> 700149
OKM_NODE_FOLDER -> 387650
OKM_NODE_MAIL -> 0
hard disk
SATA 7200

Re: OpenKM freeze for no apparent reason

Posted: Sat Aug 24, 2019 9:50 am
by jllort
What is your current hardware configuration?

Re: OpenKM freeze for no apparent reason

Posted: Mon Aug 26, 2019 8:07 am
by manueldasilva
hello



hard disk
SATA 7200
400Go

Ram
8 Go

CPU (lspcu result)
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 6
On-line CPU(s) list: 0-5
Thread(s) per core: 1
Core(s) per socket: 1
Socket(s): 6
NUMA node(s): 1
Vendor ID: AuthenticAMD
CPU family: 21
Model: 16
Model name: AMD Opteron(tm) Processor 6386 SE
Stepping: 0
CPU MHz: 2800.000
BogoMIPS: 5600.00
Hypervisor vendor: VMware
Virtualization type: full
L1d cache: 16K
L1i cache: 64K
L2 cache: 2048K
L3 cache: 12288K
NUMA node0 CPU(s): 0-5

Re: OpenKM freeze for no apparent reason

Posted: Wed Aug 28, 2019 8:23 pm
by jllort
You have about 1 milion records in the database this is a considerable amount. Might be several reasons why application freeze.
First, Community edition is not so much optimized as professional edition, that means in a huge scenario de performance will not be the same.

Take a look at the number of keywords:

Code: Select all

SELECT count(*) FROM OKM_NODE_KEYWORD;
I suggest periodically delete the dashboard, after a huge import that might cause a lot of problems ( really in a huge scenario it is better to disable it ):

Code: Select all

DELETE FROM OKM_DASHBOARD_ACTIVITY;
Finally, you can analyze queries and methods execution time ( looking for queries what taking a lot of time and methods ):
Administration > Tools > Hibernate Stats and Profiling Stats

You are using MySQL database ?
How many concurrent users do you have ?
How many documents do you have in the text extraction queue ( Administration > Stats > Text extraction )
What are the values of managed.text.* configuration parameters ( Administration > Configuration parameters )

Re: OpenKM freeze for no apparent reason

Posted: Thu Aug 29, 2019 6:49 am
by manueldasilva
Hello
thank you for the time spent!

Code: Select all

"SELECT count(*) FROM OKM_NODE_KEYWORD;"
Result -> 0
Im' using postgresql with max_connections = 100.

"Query statistics" in "Administration > Tools > Hibernate Stats and Profiling Stats" are empty. I just activate them.

"Administration > Stats > Text extraction" -> Total pending extractions: 202029... it seems to me a lot

here is for managed.text.*

Code: Select all

managed.text.extraction	-> Active	
managed.text.extraction.batch	-> 10	    
managed.text.extraction.concurrent	-> Inactive	
managed.text.extraction.pool.size -> 5	    
managed.text.extraction.pool.threads -> 5	    
managed.text.extraction.pool.timeout -> 1

Re: OpenKM freeze for no apparent reason

Posted: Sat Aug 31, 2019 9:50 am
by jllort
I suspect you have a document locked in the text extraction what never ends and periodically try to extract into an infinite loop, check if the number change on time. About stats, remember after restarting are lost ( and is not a good idea having the stats always enabled, only enable for profiling purposes )

Can try to increase the timeout

Code: Select all

managed.text.extraction.pool.timeout -> 1 minute to 5 minutes
Also, I suggest using fewer cores

Code: Select all

managed.text.extraction.pool.size -> 2
managed.text.extraction.pool.threads -> 2
An increase the number of the batch queue ( because you have a lot of files in the queue )

Code: Select all

managed.text.extraction.batch -> 100
An activate the concurrency in text extraction

Code: Select all

managed.text.extraction.concurrent -> True

Re: OpenKM freeze for no apparent reason

Posted: Thu Nov 28, 2019 2:39 pm
by manueldasilva
Hello

Thanks for your help
Do you have other ideas?
I do not see any significant improvement

Re: OpenKM freeze for no apparent reason

Posted: Sat Nov 30, 2019 9:31 am
by jllort
The number of documents in the text extraction queue is decreasing?
I some file always in the queue what never ends processing ( take screenshots and compare between them in time ).

Re: OpenKM freeze for no apparent reason

Posted: Mon Dec 02, 2019 10:49 am
by manueldasilva
The queue is empty.

Re: OpenKM freeze for no apparent reason

Posted: Thu Dec 05, 2019 5:37 pm
by jllort
I suggest doing VM snapshots and analyze the stack https://docs.openkm.com/kcenter/view/ok ... ation.html ( for it you should have some control of the openkm code )