OpenKM freeze for no apparent reason

[manueldasilva]

Hello
for started, sorry for my english ...
we use version 6.3.7 CE and we have a hang that seems random.
When this lock appears, it is impossible to use openkm for about 1 minute.
I can not find anything in the logs (tomcat, openkm or postgresql).
I disabled all the crons, changed cache properties, optimized the indexes without it changing anything except perhaps speed up the display when everything is fine.

would anyone have an eventual track?

[manueldasilva]

We have advanced in our research.
it is the update of the metadata which seems to pose pb

if I repeat several times

Code: Select all

curl -u Admin: Admin -H "Accept: application / json" -X PUT -H "Content type: application / xml" -d <simplePropertiesGroup> <simplePropertyGroup> <name> okp: grp.test </ name> <value> false </ value> </ simplePropertyGroup> </ simplePropertiesGroup> 'http: // localhost: 8080 / OpenKM / services / rest / propertyGroup / setPropertiesSimple? nodeId = 1234 \ & grpName = okg: grp

I freeze openkm for 1min but still without error.

an idea?

[jllort]

How many documents do you have? ->

Code: Select all

SELECT COUNT(*) FROM OKM_NODE_BASE
SELECT COUNT(*) FROM OKM_NODE_DOCUMENT
SELECT COUNT(*) FROM OKM_NODE_FOLDER
SELECT COUNT(*) FROM OKM_NODE_MAIL

What hard disk type are you using? SATA ( 7200, 5400 ) , SSD ?

[manueldasilva]

Hello

Code: Select all

OKM_NODE_BASE -> 1087799 
OKM_NODE_DOCUMENT -> 700149
OKM_NODE_FOLDER -> 387650
OKM_NODE_MAIL -> 0

hard disk
SATA 7200

[jllort]

What is your current hardware configuration?

[manueldasilva]

hello



hard disk
SATA 7200
400Go

Ram
8 Go

CPU (lspcu result)
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 6
On-line CPU(s) list: 0-5
Thread(s) per core: 1
Core(s) per socket: 1
Socket(s): 6
NUMA node(s): 1
Vendor ID: AuthenticAMD
CPU family: 21
Model: 16
Model name: AMD Opteron(tm) Processor 6386 SE
Stepping: 0
CPU MHz: 2800.000
BogoMIPS: 5600.00
Hypervisor vendor: VMware
Virtualization type: full
L1d cache: 16K
L1i cache: 64K
L2 cache: 2048K
L3 cache: 12288K
NUMA node0 CPU(s): 0-5

[jllort]

You have about 1 milion records in the database this is a considerable amount. Might be several reasons why application freeze.
First, Community edition is not so much optimized as professional edition, that means in a huge scenario de performance will not be the same.

Take a look at the number of keywords:

Code: Select all

SELECT count(*) FROM OKM_NODE_KEYWORD;

I suggest periodically delete the dashboard, after a huge import that might cause a lot of problems ( really in a huge scenario it is better to disable it ):

Code: Select all

DELETE FROM OKM_DASHBOARD_ACTIVITY;

Finally, you can analyze queries and methods execution time ( looking for queries what taking a lot of time and methods ):
Administration > Tools > Hibernate Stats and Profiling Stats

You are using MySQL database ?
How many concurrent users do you have ?
How many documents do you have in the text extraction queue ( Administration > Stats > Text extraction )
What are the values of managed.text.* configuration parameters ( Administration > Configuration parameters )

[manueldasilva]

Hello
thank you for the time spent!

Code: Select all

"SELECT count(*) FROM OKM_NODE_KEYWORD;"
Result -> 0

Im' using postgresql with max_connections = 100.

"Query statistics" in "Administration > Tools > Hibernate Stats and Profiling Stats" are empty. I just activate them.

"Administration > Stats > Text extraction" -> Total pending extractions: 202029... it seems to me a lot

here is for managed.text.*

Code: Select all

managed.text.extraction	-> Active	
managed.text.extraction.batch	-> 10	    
managed.text.extraction.concurrent	-> Inactive	
managed.text.extraction.pool.size -> 5	    
managed.text.extraction.pool.threads -> 5	    
managed.text.extraction.pool.timeout -> 1

[jllort]

I suspect you have a document locked in the text extraction what never ends and periodically try to extract into an infinite loop, check if the number change on time. About stats, remember after restarting are lost ( and is not a good idea having the stats always enabled, only enable for profiling purposes )

Can try to increase the timeout

Code: Select all

managed.text.extraction.pool.timeout -> 1 minute to 5 minutes

Also, I suggest using fewer cores

Code: Select all

managed.text.extraction.pool.size -> 2
managed.text.extraction.pool.threads -> 2

An increase the number of the batch queue ( because you have a lot of files in the queue )

Code: Select all

managed.text.extraction.batch -> 100

An activate the concurrency in text extraction

Code: Select all

managed.text.extraction.concurrent -> True

[manueldasilva]

Hello

Thanks for your help
Do you have other ideas?
I do not see any significant improvement

[jllort]

The number of documents in the text extraction queue is decreasing?
I some file always in the queue what never ends processing ( take screenshots and compare between them in time ).

[manueldasilva]

The queue is empty.

[jllort]

I suggest doing VM snapshots and analyze the stack https://docs.openkm.com/kcenter/view/ok ... ation.html ( for it you should have some control of the openkm code )