Page 1 of 2

5.1.10 Snapshot Unstable These Days

PostPosted:Thu Apr 12, 2012 3:49 am
by kevinchang
Hi jilort and pavila,

I kept upgrading snapshot daily build to get the latest fixes.
However, snapshot builds become unstable in my environment, users will encounter communication error when checkout document.
I myself even spotted lots of error messages in console. Please check the attachment.
Because my installation cannot keep working normally for even one day, I fallback to build:7460, but the error log still here. Really lots of error log, I need your help badly.

Re: 5.1.10 Snapshot Unstable These Days

PostPosted:Thu Apr 12, 2012 8:27 am
by kevinchang
I finally fallback to release build 5.1.9, and it become stable now.
This afternoon is really bad for me, because the nightly build 5.1.10 crash twice this afternoon.
I would like to provide more information and I can repeat the problem any time.

Re: 5.1.10 Snapshot Unstable These Days

PostPosted:Fri Apr 13, 2012 6:39 am
by pavila
I recommend to stop JBoss, and remove $JBOSS_HOME/server/default/tmp & $JBOSS_HOME/server/default/work when upgrading because sometimes JBoss has problems after several hot deploys.

Re: 5.1.10 Snapshot Unstable These Days

PostPosted:Mon Apr 16, 2012 2:57 am
by kevinchang
Hi pavila

I ran clean.sh before each update, but the unstable problem still exist.

The release build 5.1.9 works stable these days.

Re: 5.1.10 Snapshot Unstable These Days

PostPosted:Tue Apr 17, 2012 11:13 am
by pavila
I have seen a lot this error in you log file:
Code: Select all
java.net.SocketException: Too many open files
And I don't it is related to OpenKM-5.1.10. It would help if you can post the steps to reproduce this error.

Re: 5.1.10 Snapshot Unstable These Days

PostPosted:Wed Apr 18, 2012 7:52 am
by kevinchang
Hi pavila,

I'm using Mysql + LDAP.
It's quite simple to repeat this problem in my environment. Upgrade to a 5.1.10 daily build, user start to login, then click any folder or file will cause the "too many open file" error log.

Re: 5.1.10 Snapshot Unstable These Days

PostPosted:Thu Apr 19, 2012 6:56 am
by pavila
Can you reproduce this error in a OpenKM 5.1.10 clean installation? I can't see this error. There are many concurrent users? Many documents in the repository?

Re: 5.1.10 Snapshot Unstable These Days

PostPosted:Sat Apr 21, 2012 11:23 am
by kevinchang
Hi pavila,

I will try a clean installation next Monday.
I notice the "too many open files" error even just after I restart openkm instance, only I am click a single file.
Yes, there are over 10 thousand files in my repository.
Clean installation means jboss environment with 5.1.0 war package, and the same configuarion,right?

Re: 5.1.10 Snapshot Unstable These Days

PostPosted:Mon Apr 23, 2012 10:44 am
by jllort
I'm not sure if 5.1.10 has minimal change and you should remove all tree hibernate libraries under /server/default/lib this is the only difference that you could find, but I'm not totally sure about it on community version.

Re: 5.1.10 Snapshot Unstable These Days

PostPosted:Tue Apr 24, 2012 9:57 am
by kevinchang
Hi jilort,

Where can I get a jboss package with 5.1.10?

Re: 5.1.10 Snapshot Unstable These Days

PostPosted:Fri Apr 27, 2012 6:34 am
by pavila
Actually you can't. Download the JBoss + OpenKM 5.1.9 bundle and replace the WAR with the last night build from OpenKM 5.1.10

Re: 5.1.10 Snapshot Unstable These Days

PostPosted:Mon Apr 30, 2012 1:29 am
by Alexires
April 29 and April 30 build are halting with error
Code: Select all
[WARNING] *** CHECKSUM FAILED - Checksum failed on download: local = 'b1a8b2056692b57d2ac4ddb23991ad798eb59e8b'; remote = '
I was hoping to resolve some problems by moving to the 5.1.10 build, but it sounds like it is rather unstable. What timeframe are you guys looking at for releasing it globally?

Also, I receive the "Too Many Files Open" error when I attempt to import a folder that has already been imported, if it has a lot of files in it (30,000).

Re: 5.1.10 Snapshot Unstable These Days

PostPosted:Wed May 02, 2012 1:14 pm
by Alexires
I think I might have worked out where that IO Too Many Open Files stems from. Apparently Linux is configured to limit the number of total open files in one session. Increasing this value may prevent IO errors (I haven't tried it yet, I'm not home). Check out this link: http://www.cyberciti.biz/faq/linux-incr ... pen-files/

Re: 5.1.10 Snapshot Unstable These Days

PostPosted:Thu May 03, 2012 2:58 am
by Alexires
I've increased ulimit -Hn to 1000000, however, it still seems that OpenKM chokes on 1024 (the default). I'm currently looking at the run.sh file which might not be reading the ulimit properly, hence setting the java ulimit to 1024 (thus giving IO errors). I'll have a play and see if it works.

EDIT: SOLVED! Changed the value for FD_MAX in run.conf manually from "maximum" to a value just lower than the value given in ulimit -Hn (in ubuntu linux). It seems like OpenKM is using (at a max) 2050ish of the value given in ulimit -Hn (calculated by looking at htop and selecting "java -Dprogram...." then hitting the l key to show open descriptors. Scrolling to the bottom gives about 2050 (excluding mem descriptors)) so I'd imagine a value for FD_MAX of 10000 to be fine.

jllort or pavila, perhaps have a look at the run.sh script and see whether it is getting the correct value for ulimit from the system. I don't know a lot about coding, and what I read was fine, but once I changed it, I managed to import much much more of my 30,000 file directory via Import Repository than I could by leaving run.conf as it is...

Re: 5.1.10 Snapshot Unstable These Days

PostPosted:Wed May 09, 2012 10:05 am
by pavila
Is quite odd you have this error in OpenKM 5.1.10 SNAPHSOT I don't understand why this OpenKM release is taking too mucho file descriptors. Which Ubuntu version are you using? Is 32 or 64 bits?