OpenKM became very slow

[zeropower]

Hi,
I installed OpenKM 5.1.10 on a very powerful server with 2 Intel Xeon X5570 processors and 12GB memory and very fast 10k HDD, I configured OpenKM with PostgreSQL database 9.1.3 at Ubuntu server 11.10 x64 bit with Java memory as

Code: Select all

JAVA_OPTS="-server -Xms6144m -Xmx6144m -XX:PermSize=3072m -XX:MaxPermSize=3072m -Djava.awt.headless=true -Dfile.encoding=utf-8 -XX:+DisableExplicitGC"

While was copying files through webdav almost of the time was taking the full CPU usage and almost 90% of memory, many times also it got hangs and the system must be restarted to work again, finally when i reach 1.6 TB of data the system became very very slow, it tooks 53 seconds to login using admin user and around 45 seconds to other users, also all the time the CPU usage is 100 %, i have some errors at the log file may it helps

Code: Select all

WARN  [DTDEntityResolver] recognized obsolete hibernate namespace http://hibernate.sourceforge.net/. Use namespace http://www.hibernate.org/dtd/ instead. Refer to Hibernate 3.6 Migration Guide!

WARN  [FileLoader] Supplied path does not exist. Unable to add plugins from there.

WARN  [ExecutionUtils] Unable to read script: /opt/OpenKM/jboss-4.2.3.GA/start.bsh

WARN  [ExecutionUtils] Unable to read jar: /opt/OpenKM/jboss-4.2.3.GA/start.jar

WARN  [config] Unable to process deployment descriptor for context '/iesvc'

WARN  [config] Unable to process deployment descriptor for context '/wstats'

WARN  [config] Unable to process deployment descriptor for context '/zecmd'

ERROR [FlateFilter] Stop reading corrupt stream

WARN  [net.xeoh.plugins.base.impl.classpath.loader.FileLoader] Supplied path does not exist. Unable to add plugins from there.

Has anybody face the same problem and solve it ?

[vkasgpta]

i have some of those error messages too... but my db is very small so im seeing the lag from the start but its still not alot of lag right now..

maybe until a pro replies you could export your repo and user data with profiles, rename old directory of openkm to OLDOpenkm and start a new copy up, importing the old stuff?

If that works fine, put add ins one by one (wait a day or 2 after add in) to see if thats causing any conflict?
Cronjobs may also be the cause..?

[pavila]

OpenKM Community is not designed to handle 1TB repositories. If you need support for bigger document repositories you should use OpenKM Professional. In general, I don't recommend you to use OpenKM Community for repositories bigger than a few GB.

[jllort]

As says pavila OpenKM community version has good performance for repositories between 15-25 GB depending hardware configuration etc.. , We know for upper repositories will starting some perfomance degration, althought it we know comunity users has experiences using on repositories upper 200GB and more.

The open source idea is not only get application benefits is returning to comunity, that's the idea of the comunity, but seems majors open source users forget this. althought openkm team are entusiastic programers that we believe open source is a social sofware concept, we continue observate that everybody identifies open source with free. We know some companies are not able to expend money, that's ok, we are happy if these use our software free and collaborate in anyway to us, for example we are happy if you spread openkm in your social network or simply buying our documentation access - symbolic price - for live. We know for example great companies, oil, banking, phramaceutil, government among others are using openkm in a intensive way, getting a benefits for ourself and do not giving any contribution on comunity than demanding help. The IT people should revise ethics.

Believe me when I said than OpenKM team workers need buying food on supermarket than you ( well I have got a small garden for lettuce etc.. OpenKM gives their company workers some free time concept for this kind of live ), and we can not going there with a shirt with "open source enthusiast" and get food, the money is the currency.

Finally we use other open source libraries etc... when some of these projects contacts with us, we offering openkm professional with support freely, because we must return what we get. For example data studio project gives us 1 free project licence because we are open source project, that's a way to contribute in the comunity ( from here our acknoledgements to all people that has collaborated to us http://www.openkm.com/en/overview/acknowledgements.html )

[zeropower]

I think you are right 100 percent but you must make a demo or trial version for professional edition in case of testing.

From the start, i contacted your sales person called Biel and got a very high price, i think it is more than many of document systems at the market now, , no one can ignore that yours has much more features than others offer but how can i convince my manger for buying any software without trying at first and seeing the results at full load.

Finally, i will try to contribute in language translation and try helping others as i can in this forum as i installed it at many linux and windows versions with different databases and really like the bunch of features it has.

[jllort]

We are working on Trial that will be able at ends next month, you are on truth about it. If the problem is testing application we can find some solution for it, that should not be a problem, comment with sales team assigned to you that you have been talking with me and give him the url of this post.

And obviously, thanks for any contibution.

[Sanjay]

This to pavila and jllort ,

Thanks for making a good software and keeping the enthusiasm alive to build the open source software. But i disagree with your views on the performance of the software. The openkm community version should support at least up to 1 TB of data, That is the reason we make DMS systems. It should handle large repositories. These days 1 TB is hard drives are shipped with standard laptops. many websites provides many GB of data storage online. But by your logic Linux operating system should not work properly. They should not give server versions for free. it should not be scalable. Even JBOSS , Tomcat should not provided work properly.
My view is when we make open source software, it should be scalable. It should work according to its aim. If it is not like this the idea of Open source is lost.

Again i thank you for your efforts to make a open source software. I wish you will take a note of my post.

[jllort]

As you said Sanjay is your point of view. Not all open source projects are comparable.

I do not know which is your expertice working in open source projects ( I am not talking about using that's other scenario ). Honestly during last 8-10 years we have tested several models until we found a susteinable project, we have talked with a lot of possible customers, after it we take some decisions, probably you are not agree with us.

You have confusion with the word is optimized and malfunction ( we have thinking be special optimized for small companies ). Community version can hold a lot of documents, Tera and more ( we know some users are using in this scenarios ), but as comes by default you will not get and excellent performance. Do not worry you have got the source code and database structure, you can optimize it as your need, some users are doing it.

Only a question, on which way you contribute in open source ?

[prasanna]

Hi ,

I am very new to the Java and jackrabbit. I came across your project. I find this post. i want to do some optimization of community version to handle data speedily of size 700 - 800 GB but i am very unaware of how i can do it. can you please help me to do the same. i have already downloaded your project and set it up.

thanks and br
prasanna

[pavila]

Get the source code, learn how it works and try to optimize it.

There is no quick solution for this.