Open Source Document Management System | OpenKM

Because information matters
https://forum.openkm.com/

Text Extractor giving mysql exception for all documents

https://forum.openkm.com/viewtopic.php?t=19650

Page 1 of 1

Text Extractor giving mysql exception for all documents

PostPosted:Fri Apr 10, 2015 9:09 am
by JavaDev
Whenever Text Extractor is executed it gives the following exceptions for all documents[pdf, doc, txt, excel etc]
Code: Select all
2015-04-09 17:26:30,427 [http-bio-80-exec-60] WARN  org.hibernate.util.JDBCExceptionReporter - SQL Error: 1366, SQLState: HY000
2015-04-09 17:26:30,428 [http-bio-80-exec-60] ERROR org.hibernate.util.JDBCExceptionReporter - Incorrect string value: '\xE2\x89\xA520 ...' for column 'NDC_TEXT' at row 1
2015-04-09 17:26:30,428 [http-bio-80-exec-60] WARN  com.openkm.extractor.TextExtractorWorker - could not update: [com.openkm.dao.bean.NodeDocument#6402a1fa-783b-456d-b486-8d7977759412]
com.openkm.core.DatabaseException: could not update: [com.openkm.dao.bean.NodeDocument#6402a1fa-783b-456d-b486-8d7977759412]
        at com.openkm.dao.NodeDocumentDAO.textExtractorHelper(NodeDocumentDAO.java:1344)
        at com.openkm.extractor.TextExtractorWorker.processSerial(TextExtractorWorker.java:138)
        at com.openkm.extractor.TextExtractorWorker.processQueue(TextExtractorWorker.java:125)
        at com.openkm.extractor.TextExtractorWorker.run(TextExtractorWorker.java:80)
        at sun.reflect.GeneratedMethodAccessor222.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at bsh.Reflect.invokeOnMethod(Unknown Source)
        at bsh.Reflect.invokeObjectMethod(Unknown Source)
        at bsh.BSHPrimarySuffix.doName(Unknown Source)
        at bsh.BSHPrimarySuffix.doSuffix(Unknown Source)
        at bsh.BSHPrimaryExpression.eval(Unknown Source)
        at bsh.BSHPrimaryExpression.eval(Unknown Source)
        at bsh.Interpreter.eval(Unknown Source)
        at bsh.Interpreter.eval(Unknown Source)
        at bsh.Interpreter.eval(Unknown Source)
        at com.openkm.util.ExecutionUtils.runScript(ExecutionUtils.java:112)
        at com.openkm.core.Cron$RunnerBsh.run(Cron.java:103)
        at com.openkm.servlet.admin.CronTabServlet.execute(CronTabServlet.java:227)
        at com.openkm.servlet.admin.CronTabServlet.doGet(CronTabServlet.java:115)
        at com.openkm.servlet.admin.CronTabServlet.service(CronTabServlet.java:71)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:311)
        at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:116)
        at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:83)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
        at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:113)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
        at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:101)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
        at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:113)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
        at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:54)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
        at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:45)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
        at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:182)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
        at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:87)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
        at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:173)
        at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346)
        at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:259)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
        at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:501)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:98)
        at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
        at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1040)
        at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:607)
        at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:315)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
Caused by: org.hibernate.exception.GenericJDBCException: could not update: [com.openkm.dao.bean.NodeDocument#6402a1fa-783b-456d-b486-8d7977759412]
        at org.hibernate.exception.SQLStateConverter.handledNonSpecificException(SQLStateConverter.java:140)
        at org.hibernate.exception.SQLStateConverter.convert(SQLStateConverter.java:128)
        at org.hibernate.exception.JDBCExceptionHelper.convert(JDBCExceptionHelper.java:66)
        at org.hibernate.persister.entity.AbstractEntityPersister.update(AbstractEntityPersister.java:2613)
        at org.hibernate.persister.entity.AbstractEntityPersister.updateOrInsert(AbstractEntityPersister.java:2495)
        at org.hibernate.persister.entity.AbstractEntityPersister.update(AbstractEntityPersister.java:2822)
        at org.hibernate.action.EntityUpdateAction.execute(EntityUpdateAction.java:113)
        at org.hibernate.engine.ActionQueue.execute(ActionQueue.java:273)
        at org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:265)
        at org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:185)
        at org.hibernate.event.def.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:321)
        at org.hibernate.event.def.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:51)
        at org.hibernate.impl.SessionImpl.flush(SessionImpl.java:1216)
        at org.hibernate.impl.SessionImpl.managedFlush(SessionImpl.java:383)
        at org.hibernate.transaction.JDBCTransaction.commit(JDBCTransaction.java:133)
        at com.openkm.dao.HibernateUtil.commit(HibernateUtil.java:325)
        at com.openkm.dao.NodeDocumentDAO.textExtractorHelper(NodeDocumentDAO.java:1340)
        ... 63 more
Caused by: java.sql.SQLException: Incorrect string value: '\xE2\x89\xA520 ...' for column 'NDC_TEXT' at row 1
        at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1075)
        at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3562)
        at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3494)
        at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1960)
        at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2114)
        at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2696)
        at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:2105)
        at com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:2398)
        at com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:2316)
        at com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:2301)
        at org.apache.tomcat.dbcp.dbcp.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:105)
        at org.apache.tomcat.dbcp.dbcp.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:105)
        at org.hibernate.persister.entity.AbstractEntityPersister.update(AbstractEntityPersister.java:2595)
        ... 76 more
This is affecting the search.

The encoding of the OpenKM Database is set to utf8-utf8_bin. Also when we tried deleting one of the documents for which the above exception occurred its entry remained in the table OKM_NODE_BASE even after deletion from the trash.

Re: Text Extractor giving MySQL exception for all documents

PostPosted:Sat Apr 11, 2015 9:24 am
by jllort
It seems a codification problem. Which OpenKM version are you using and what MySQL version.
Does your document have chinese or similar language ... characters ?

Re: Text Extractor giving MySQL exception for all documents

PostPosted:Mon Apr 13, 2015 9:38 am
by JavaDev
The Mysql version is 5.1 and OpenKM version is 6.2.5.
Our application would be supporting UTF-8 with 255 characters. Hence the documents consist of the UTF8 characters.
No Chinese or similar lagnguage characters are supported.

Re: Text Extractor giving MySQL exception for all documents

PostPosted:Fri Apr 17, 2015 9:42 am
by pavila
I suggest upgrading to a night build and try to reproduce the problem. Take a look at http://wiki.openkm.com/index.php/Migrat ... 3_to_6.3.1

Re: Text Extractor giving MySQL exception for all documents

PostPosted:Mon Apr 27, 2015 8:00 am
by JavaDev
We have made few application specific changes to the OpenKM API's and also it is not possible for us to upgrade the version at this point of time.
Is there any other alternative solution for this issue?

Re: Text Extractor giving MySQL exception for all documents

PostPosted:Mon Apr 27, 2015 10:22 am
by pavila
The fix has come source code changes. You would need to apply these changes to your own source code. Take a look at subversion changes.
All times are UTC
Page 1 of 1
Powered by phpBB® Forum Software © phpBB Limited