No ManagedConnections available message

[ilagi]

Hi All,
We use OpenKM for a few months now and the first time ever I saw the next error message when anybody tried to login and I had to restart OpenKM to get around it:

Code: Select all

org.jboss.util.NestedSQLException: No ManagedConnections available within configured blocking timeout ( 30000 [ms] );  nested throwable: (javax.resource.ResourceException: No ManagedConnections available within configured blocking timeout ( 30000 [ms] ))

I would guess this has something to do with JBoss try to open a new connection to MySQL, but doesn't get one.

Did anybody see something like thi before? Do you know where to change this timeout?

[jllort]

You've not got configured poll as is explained in wiki for mysql. After some time mysql closes connection and need some extra parameter than jdbc:mysql://localhost:3306/okm_app?autoReconnect=true, try using too <idle-timeout-minutes>28680</idle-timeout-minutes>

I recommend if you've got accessing to that wiki section documentation http://wiki.openkm.com/index.php/MySQL-OpenKM_5.0, that compare your params with that.

[ilagi]

Thanks for the reply.
Actually we are still using OpenKM 4.1, but most of the MySQL connection parameters are the same.

The idle-timeout-minutes was set to 1, so I changed that to 28680 as you said. jdbc:mysql://localhost:3306/okm_app?autoReconnect=true was already there.

Now I need to wait a few days to see whether the issue re-occurs.

Thanks again

[jllort]

I think it'll be solved ( problem normally at weekends where there's few activity or sometimes during the same day is when happens ). That problem has been happened to us too, and that parameter has solved it. I recommend upgrading some day to 5.0.1 version, because has better performance and solves some minor problems in 4.1 version ( Migration from 4.1 to 5.X needs some extra time, but from version 5.X to upper it'll be more easy, basically we've introduced hibernate and that has supposed a migration procedure that need some extra time ).

[ilagi]

Unfortunately now I see a new error.
When any user logs in there is an error message popping up:
Error when communicating with server (findUserTaskInstances)
Error when communicating with server (findPooledTaskInstances)

On the server side there are messages about [DbPersistenceService] hibernate commit failed and Communications link failure complaining about when the last packet successfully received and sent.

[jllort]

You're on migration scenario ? or still on version 4.1 ?

[ilagi]

No, no migration, just the regular 4.1

[jllort]

put here your openkm-ds.xml, repository.xml, and workspace.xml ( it's under repository/repository ). The last two files only the connection reference, not all file. And we'll take a look. You're using on production environment ? And finally wich mysql version ( linux / windows and number vesion ) are you using and the driver you're using ( that's very important too ).

[ilagi]

Here comes all the details.

openkm-ds.xml

Code: Select all

<?xml version="1.0" encoding="UTF-8"?>
<datasources>
  <!-- OpenKM User Activity -->
  <local-tx-datasource>
    <jndi-name>OKMActivityDS</jndi-name>
    <connection-url>jdbc:mysql://10.65.96.141:3306/okm_activity?autoReconnect=true</connection-url>
    <driver-class>com.mysql.jdbc.Driver</driver-class>
    <user-name>openkm</user-name>
    <password>******</password>
    <min-pool-size>5</min-pool-size>
    <max-pool-size>20</max-pool-size>
    <idle-timeout-minutes>28680</idle-timeout-minutes>
    <track-statements/>
    <prepared-statement-cache-size>32</prepared-statement-cache-size>
    <metadata>
      <type-mapping>MySQL</type-mapping>
    </metadata>
  </local-tx-datasource>
 
  <!-- OpenKM User Auth -->
  <local-tx-datasource>
    <jndi-name>OKMAuthDS</jndi-name>
    <connection-url>jdbc:mysql://10.65.96.141:3306/okm_auth?autoReconnect=true</connection-url>
    <driver-class>com.mysql.jdbc.Driver</driver-class>
    <user-name>openkm</user-name>
    <password>******</password>
    <min-pool-size>5</min-pool-size>
    <max-pool-size>20</max-pool-size>
    <idle-timeout-minutes>28680</idle-timeout-minutes>
    <track-statements/>
    <prepared-statement-cache-size>32</prepared-statement-cache-size>
    <metadata>
      <type-mapping>MySQL</type-mapping>
    </metadata>
  </local-tx-datasource>
 
  <!-- OpenKM Dashboard Stats -->
  <local-tx-datasource>
    <jndi-name>OKMDashboardStatsDS</jndi-name>
    <connection-url>jdbc:mysql://10.65.96.141:3306/okm_dashboard?autoReconnect=true</connection-url>
    <driver-class>com.mysql.jdbc.Driver</driver-class>
    <user-name>openkm</user-name>
    <password>******</password>
    <min-pool-size>5</min-pool-size>
    <max-pool-size>20</max-pool-size>
    <idle-timeout-minutes>28680</idle-timeout-minutes>
    <track-statements/>
    <prepared-statement-cache-size>32</prepared-statement-cache-size>
    <metadata>
      <type-mapping>MySQL</type-mapping>
    </metadata>
  </local-tx-datasource>
 
  <!-- OpenKM Workflow -->
  <local-tx-datasource>
    <jndi-name>OKMWorkflowDS</jndi-name>
    <connection-url>jdbc:mysql://10.65.96.141:3306/okm_workflow?autoReconnect=true</connection-url>
    <driver-class>com.mysql.jdbc.Driver</driver-class>
    <user-name>openkm</user-name>
    <password>******</password>
    <min-pool-size>5</min-pool-size>
    <max-pool-size>20</max-pool-size>
    <idle-timeout-minutes>28680</idle-timeout-minutes>
    <track-statements/>
    <prepared-statement-cache-size>32</prepared-statement-cache-size>
    <metadata>
      <type-mapping>MySQL</type-mapping>
    </metadata>
  </local-tx-datasource>
</datasources>

repository.xml:

Code: Select all

<?xml version="1.0"?>
<!DOCTYPE Repository PUBLIC "-//The Apache Software Foundation//DTD Jackrabbit 1.4//EN"
                            "http://jackrabbit.apache.org/dtd/repository-1.4.dtd">
<Repository>
    <FileSystem class="org.apache.jackrabbit.core.fs.local.LocalFileSystem">
        <param name="path" value="${rep.home}/repository"/>
    </FileSystem>
    <Security appName="OpenKM">
      <AccessManager class="es.git.openkm.core.OKMAccessManager"/>
      <!-- <AccessManager class="org.apache.jackrabbit.core.security.SimpleAccessManager"/> -->
    </Security>
    <Workspaces rootPath="${rep.home}/workspaces" defaultWorkspace="default" />
    <Workspace name="${wsp.name}">
        <FileSystem class="org.apache.jackrabbit.core.fs.local.LocalFileSystem">
            <param name="path" value="${wsp.home}"/>
        </FileSystem>
    <PersistenceManager class="org.apache.jackrabbit.core.persistence.bundle.MySqlPersistenceManager">
      <param name="driver" value="com.mysql.jdbc.Driver"/>
      <param name="url" value="jdbc:mysql://10.65.96.141:3306/okm_repository?autoReconnect=true"/>
      <param name="schema" value="mysql"/>
      <param name="user" value="openkm"/>
      <param name="password" value="******"/>
      <param name="schemaObjectPrefix" value="${wsp.name}_"/>
      <param name="externalBLOBs" value="false"/>
    </PersistenceManager>
        <SearchIndex class="org.apache.jackrabbit.core.query.lucene.SearchIndex">
            <param name="path" value="${wsp.home}/index"/>
            <param name="resultFetchSize" value="100"/>
            <param name="useCompoundFile" value="true"/>
            <param name="minMergeDocs" value="100"/>
            <param name="volatileIdleTime" value="3"/>
            <param name="maxMergeDocs" value="100000"/>
            <param name="mergeFactor" value="10"/>
            <param name="maxFieldLength" value="50000"/>
            <!-- Disable extractor pool -->
            <!-- <param name="extractorPoolSize" value="0"/> -->
            <param name="extractorTimeout" value="1000"/>
            <param name="bufferSize" value="10"/>
            <param name="cacheSize" value="1000"/>
            <param name="forceConsistencyCheck" value="false"/>
            <!-- <param name="consistencyCheck" value="true"/> -->
            <!-- <param name="consistencyFix" value="true"/> -->
            <param name="autoRepair" value="true"/>
            <!-- <param name="analyzer" value="es.git.openkm.analysis.SpanishAnalyzer"/> -->
            <param name="respectDocumentOrder" value="false"/>
            <param name="indexingConfiguration" value="${wsp.home}/../../../indexing_configuration.xml"/>
            <param name="textFilterClasses" value="
org.apache.jackrabbit.extractor.PlainTextExtractor,
org.apache.jackrabbit.extractor.PdfTextExtractor,
org.apache.jackrabbit.extractor.HTMLTextExtractor,
org.apache.jackrabbit.extractor.XMLTextExtractor,
org.apache.jackrabbit.extractor.RTFTextExtractor,
org.apache.jackrabbit.extractor.OpenOfficeTextExtractor,
es.git.openkm.extractor.MsExcelTextExtractor,
es.git.openkm.extractor.MsPowerPointTextExtractor,
es.git.openkm.extractor.MsWordTextExtractor,
es.git.openkm.extractor.MsOffice2007TextExtractor,
es.git.openkm.extractor.ExifTextExtractor,
es.git.openkm.extractor.TiffTextExtractor,
es.git.openkm.extractor.AudioTextExtractor" />
        </SearchIndex>
    </Workspace>
    <Versioning rootPath="${rep.home}/version">
        <FileSystem class="org.apache.jackrabbit.core.fs.local.LocalFileSystem">
            <param name="path" value="${rep.home}/version"/>
        </FileSystem>
    <PersistenceManager class="org.apache.jackrabbit.core.persistence.bundle.MySqlPersistenceManager">
      <param name="driver" value="com.mysql.jdbc.Driver"/>
      <param name="url" value="jdbc:mysql://10.65.96.141:3306/okm_repository?autoReconnect=true"/>
      <param name="schema" value="mysql"/>
      <param name="user" value="openkm"/>
      <param name="password" value="******"/>
      <param name="schemaObjectPrefix" value="version_"/>
      <param name="externalBLOBs" value="false"/>
    </PersistenceManager>
    </Versioning>
    <!-- Also see DatabaseDataStore-->
    <DataStore class="org.apache.jackrabbit.core.data.FileDataStore"/>
</Repository>

workspace.xml:

Code: Select all

<?xml version="1.0" encoding="UTF-8"?>
<Workspace name="default">
        <FileSystem class="org.apache.jackrabbit.core.fs.local.LocalFileSystem">
            <param name="path" value="${wsp.home}"/>
        </FileSystem>
        <PersistenceManager class="org.apache.jackrabbit.core.persistence.bundle.DerbyPersistenceManager">
            <param name="url" value="jdbc:derby:${wsp.home}/db;create=true"/>
            <param name="schemaObjectPrefix" value="${wsp.name}_"/>
        </PersistenceManager>
        <SearchIndex class="org.apache.jackrabbit.core.query.lucene.SearchIndex">
            <param name="path" value="${wsp.home}/index"/>
            <param name="resultFetchSize" value="100"/>
            <param name="useCompoundFile" value="true"/>
            <param name="minMergeDocs" value="100"/>
            <param name="volatileIdleTime" value="3"/>
            <param name="maxMergeDocs" value="100000"/>
            <param name="mergeFactor" value="10"/>
            <param name="maxFieldLength" value="50000"/>
            <!-- Disable extractor pool -->
            <!-- <param name="extractorPoolSize" value="0"/> -->
            <param name="extractorTimeout" value="1000"/>
            <param name="bufferSize" value="10"/>
            <param name="cacheSize" value="1000"/>
            <param name="forceConsistencyCheck" value="false"/>
            <!-- <param name="consistencyCheck" value="true"/> -->
            <!-- <param name="consistencyFix" value="true"/> -->
            <param name="autoRepair" value="true"/>
            <!-- <param name="analyzer" value="es.git.openkm.analysis.SpanishAnalyzer"/> -->
            <param name="respectDocumentOrder" value="false"/>
            <param name="indexingConfiguration" value="${wsp.home}/../../../indexing_configuration.xml"/>
            <param name="textFilterClasses" value=" org.apache.jackrabbit.extractor.PlainTextExtractor, org.apache.jackrabbit.extractor.PdfTextExtractor, org.apache.jackrabbit.extractor.HTMLTextExtractor, org.apache.jackrabbit.extractor.XMLTextExtractor, org.apache.jackrabbit.extractor.RTFTextExtractor, org.apache.jackrabbit.extractor.OpenOfficeTextExtractor, es.git.openkm.extractor.MsExcelTextExtractor, es.git.openkm.extractor.MsPowerPointTextExtractor, es.git.openkm.extractor.MsWordTextExtractor, es.git.openkm.extractor.MsOffice2007TextExtractor, es.git.openkm.extractor.ExifTextExtractor, es.git.openkm.extractor.TiffTextExtractor, es.git.openkm.extractor.AudioTextExtractor"/>
        </SearchIndex>
    </Workspace>

It is a MySQL 5.1.34 64bit on Windows Server 2003 64bit. Mysql driver is mysql-connector-java-5.1.8

Let me know if you need anything else.
Thanks

[jllort]

Better if you use jdbc:mysql://localhost:3306/okm_repository?autoReconnect=true"

Can you post the complete server.log when appearing Error when communicating with server (findUserTaskInstances)
Error when communicating with server (findPooledTaskInstances) ( only the error trace not all server.log )

[ilagi]

Sorry, I cannot change the MySQL connection to localhost, because it's on a different server. We have a dedicated MySQL server sitting next to the OpenKM server, LAN connection between the two.

From the server logs you asked for the part where it refers to the error:

Code: Select all

2011-02-08 23:02:28,136 ERROR [org.apache.catalina.core.ContainerBase.[jboss.web].[localhost].[/OpenKM]] Exception while dispatching incoming RPC call
java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:129)
	at org.apache.coyote.http11.InternalInputBuffer.fill(InternalInputBuffer.java:716)
	at org.apache.coyote.http11.InternalInputBuffer$InputStreamInputBuffer.doRead(InternalInputBuffer.java:746)
	at org.apache.coyote.http11.filters.IdentityInputFilter.doRead(IdentityInputFilter.java:116)
	at org.apache.coyote.http11.InternalInputBuffer.doRead(InternalInputBuffer.java:675)
	at org.apache.coyote.Request.doRead(Request.java:428)
	at org.apache.catalina.connector.InputBuffer.realReadBytes(InputBuffer.java:297)
	at org.apache.tomcat.util.buf.ByteChunk.substract(ByteChunk.java:405)
	at org.apache.catalina.connector.InputBuffer.read(InputBuffer.java:312)
	at org.apache.catalina.connector.CoyoteInputStream.read(CoyoteInputStream.java:193)
	at com.google.gwt.user.server.rpc.RPCServletUtils.readContentAsUtf8(RPCServletUtils.java:146)
	at com.google.gwt.user.server.rpc.RemoteServiceServlet.readContent(RemoteServiceServlet.java:335)
	at com.google.gwt.user.server.rpc.RemoteServiceServlet.doPost(RemoteServiceServlet.java:77)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:710)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
	at org.jboss.web.tomcat.filters.ReplyHeaderFilter.doFilter(ReplyHeaderFilter.java:96)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)
	at org.jboss.web.tomcat.security.SecurityAssociationValve.invoke(SecurityAssociationValve.java:182)
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:432)
	at org.jboss.web.tomcat.security.JaccContextValve.invoke(JaccContextValve.java:84)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
	at org.jboss.web.tomcat.service.jca.CachedConnectionValve.invoke(CachedConnectionValve.java:157)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:262)
	at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:844)
	at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583)
	at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:446)
	at java.lang.Thread.run(Thread.java:619)
2011-02-08 23:02:35,402 DEBUG [org.jboss.resource.connectionmanager.IdleRemover] internalRegisterPool: registering pool with interval 1720800000 old interval: 860400000
2011-02-08 23:02:35,480 INFO  [org.apache.jackrabbit.core.query.lucene.DocNumberCache] size=513/1024, #accesses=5906, #hits=4434, #misses=1472, cacheRatio=76%
2011-02-08 23:02:36,886 INFO  [org.apache.jackrabbit.core.persistence.bundle.util.LRUNodeIdCache] num=329/10240 hits=7849 miss=112151
2011-02-08 23:02:39,496 DEBUG [es.git.openkm.module.direct.DirectWorkflowModule] findUserTaskInstances(1967005684772257868302144000978307)
2011-02-08 23:02:39,527 DEBUG [es.git.openkm.module.direct.DirectWorkflowModule] findPooledTaskInstances(1967005684772257868302144000978307)
2011-02-08 23:02:39,574 DEBUG [es.git.openkm.module.direct.DirectWorkflowModule] findUserTaskInstances: []
2011-02-08 23:02:39,574 DEBUG [es.git.openkm.module.direct.DirectWorkflowModule] findPooledTaskInstances: []

[jllort]

I dont like it in your workspace.xml

Code: Select all

<PersistenceManager class="org.apache.jackrabbit.core.persistence.bundle.DerbyPersistenceManager">
<param name="url" value="jdbc:derby:${wsp.home}/db;create=true"/>
<param name="schemaObjectPrefix" value="${wsp.name}_"/>
</PersistenceManager>

When you changed the dbms from hypersonic to mysql, the folder repository was deleted ?
Do you have documents on that installation ? or is a fresh on ?
The error you see in trace log happens too when user doing login, when happens exactly ?

[pavila]

For avoid "Communications link failure" errors, the "idle-timeout-minutes" (in minutes) parameter from the JBoss datasource configuration should be less than "wait_timeout" (in seconds) parameter from /etc/mysql/my.cnf (In case of Windows, the MySQL configuration file is not the same, of course). In other pages, I found that another solution was to disable apparmour.

[ilagi]

Hi Pavila,
Thanks for the tip. I have changed the idle-timeout-minutes to be less than the wait_time of MySQL, but it didn't solve the problem. The error message is still there.
I have the feeling that this error message is related to the fact that we run a once a day automatic restart of MySQL at a fixed time. Is it possible to hide this debug message from the users?
Thanks

[ilagi]

Hi jllort,
I have kept the workspace.xml as it was, I wasn't the one adding that part in what you quoted.


"When you changed the dbms from hypersonic to mysql, the folder repository was deleted ?"
>> When we changed the dbms from hypersonice to mysql the folder repository was not deleted.

"Do you have documents on that installation ? or is a fresh on ?"
>> Yes, I do have documents. It's not a fresh installation, but a 7 months old, which is being used since.

"The error you see in trace log happens too when user doing login, when happens exactly ?"
>> We have a lot more user logins than the number of times this shows up in the trace log. It's in the trace log 3-4 times a day, but we have 30-40 user logins minimum. I have no idea why is it in the trace log at these exact times.