Page 1 of 1

help with dms2okm-knowledge-tree

PostPosted:Thu Nov 09, 2017 1:31 am
by amer1
I was hoping to convert documents from my old DMS, Knowledge Tree. However, the tool fails. I would like to try and debug it. I only found the compiled class files. The following is an example of the error. Anyone have experience with this tool?
Code: Select all
20:00:34.120 [main] INFO  c.o.Main - Migration tool from Knowledge Tree to OpenKM system
20:00:34.783 [main] INFO  c.o.Main - Exporter configured
Total folders: 92
Total documents: 2004
20:00:34.807 [main] INFO  c.o.u.i.RepositoryExporter - Finding content from root folder in [records/misc/gps] path
Processing folder [Records/misc/gps](1/92)
20:00:34.814 [main] INFO  c.o.u.i.RepositoryExporter - Exporting content in folder [Records/misc/gps]
20:00:34.814 [main] INFO  c.o.u.i.RepositoryExporter - Find for child documents in folder [Records/misc/gps]
Processing document [/home/ham/export/repository/Records/misc/gps/mapsource-2008-nuvi200w.TIF](1/2004)
20:00:34.834 [main] INFO  c.o.u.i.RepositoryExporter - Writing document [/home/ham/export/repository/Records/misc/gps/mapsource-2008-nuvi200w.TIF] info (creating folder and generating .okm file)
Exception in thread "main" java.lang.NullPointerException
	at com.openkm.b.a.a.b.getGrantedUsers(DocumentSrvKnowledgetree.java:34)
	at com.openkm.adapter.MetadataAdapter.getMetadata(MetadataAdapter.java:95)
	at com.openkm.util.impexp.RepositoryExporter.exportDocument(RepositoryExporter.java:187)
	at com.openkm.util.impexp.RepositoryExporter.exportFolderContent(RepositoryExporter.java:151)
	at com.openkm.util.impexp.RepositoryExporter.exportFolder(RepositoryExporter.java:237)
	at com.openkm.util.impexp.RepositoryExporter.exportRepository(RepositoryExporter.java:124)
	at com.openkm.Main.main(Main.java:63)
Thank you in advance for any ideas you may have.

Jeremy

Re: help with dms2okm-knowledge-tree

PostPosted:Sat Nov 11, 2017 11:20 am
by jllort
What version of knowledge tree are you trying to migrate ?
Can you share the command line you are using for it ( hide the password with *** ) ?
Can you check the database connection is right ?

Re: help with dms2okm-knowledge-tree

PostPosted:Mon Nov 13, 2017 1:08 pm
by amer1
What version of knowledge tree are you trying to migrate ?
Version 3.7.0.2 Community Edition
Can you share the command line you are using for it ( hide the password with *** ) ?
java -jar dms2okm-knowledge-tree.jar jdbc:mysql://w2k3std64app:3306/dms export ****** /home/germ/export /home/germ/olddms "records/misc/gps"
Can you check the database connection is right ?
I'm sure it makes the DB connection. It gets folder and document counts and finds the first doc. It also creates folders and some files in /home/germ/export.

Re: help with dms2okm-knowledge-tree

PostPosted:Wed Nov 15, 2017 7:11 am
by jabad
Hello.

Could you please try with version 1.1? It can be executed using the same parameters as version 1.0.

You can download it here: https://we.tl/F5ay0sOklp

Thank you

Re: help with dms2okm-knowledge-tree

PostPosted:Thu Nov 16, 2017 1:59 pm
by amer1
Looks like same error.
Code: Select all
08:55:48.251 [main] INFO  c.o.Main - Migration tool from Knowledge Tree to OpenKM system
08:55:49.006 [main] INFO  c.o.Main - Exporter configured
08:55:49.030 [main] INFO  c.o.u.i.RepositoryExporter - Total folders: 92
08:55:49.033 [main] INFO  c.o.u.i.RepositoryExporter - Total documents: 2004
08:55:49.033 [main] INFO  c.o.u.i.RepositoryExporter - Finding content from root folder in [records/misc/gps] path
08:55:49.040 [main] INFO  c.o.u.i.RepositoryExporter - Processing folder [Records/misc/gps](1/92)
08:55:49.040 [main] INFO  c.o.u.i.RepositoryExporter - Exporting content in folder [Records/misc/gps]
08:55:49.040 [main] INFO  c.o.u.i.RepositoryExporter - Find for child documents in folder [Records/misc/gps]
08:55:49.052 [main] INFO  c.o.u.i.RepositoryExporter - Processing document [/home/germ/export/repository/Records/misc/gps/mapsource-2008-nuvi200w.TIF](1/2004)
08:55:49.052 [main] INFO  c.o.u.i.RepositoryExporter - Writing document [/home/germ/export/repository/Records/misc/gps/mapsource-2008-nuvi200w.TIF] info (creating folder and generating .okm file)
Exception in thread "main" java.lang.NullPointerException
	at com.openkm.b.a.a.b.getGrantedUsers(DocumentSrvKnowledgetree.java:34)
	at com.openkm.adapter.MetadataAdapter.getMetadata(MetadataAdapter.java:97)
	at com.openkm.util.impexp.RepositoryExporter.exportDocument(RepositoryExporter.java:183)
	at com.openkm.util.impexp.RepositoryExporter.exportFolderContent(RepositoryExporter.java:147)
	at com.openkm.util.impexp.RepositoryExporter.exportFolder(RepositoryExporter.java:233)
	at com.openkm.util.impexp.RepositoryExporter.exportRepository(RepositoryExporter.java:114)
	at com.openkm.Main.main(Main.java:63)

Re: help with dms2okm-knowledge-tree

PostPosted:Fri Nov 17, 2017 7:31 am
by jabad
Hello.

Here you are a new version 1.1.1: https://we.tl/0tDCjyUqaf

It contains a couple of jars. Each one contains different solutions to try to resolve this problem.

Please check both of them and tell us the solution.

Thank you

Re: help with dms2okm-knowledge-tree

PostPosted:Sat Nov 18, 2017 3:46 am
by amer1
The "other" version did much better. It got all the way through without error. It is having a little trouble with special characters in the document names. But I can fix those few manually. Odd thing, it is creating a large number of versions in the OKM document history during import. In Knowledgetree, the documents went through a workflow. The document's metadata version was incremented with each step in the workflow. However, the document version never changed. Finally, I was hoping the document transaction history would come over. Thanks for your help. This is a big improvement.

Anyway I can get those source code? I may be able to make some enhancements.

Jeremy

Re: help with dms2okm-knowledge-tree

PostPosted:Mon Nov 20, 2017 2:33 pm
by jllort
Hi

Unfortunately any change in metadata is not a version change in OpenKM community version ( this feature is not implemented until OpenKM version 7 what include the metadata changes as a part of document version history ).

About History we do not migrate older history to OpenKM history, for several reasons, the main is that the UUID and type of log will not have relation with previous application log. I suggest take a look at OKM_ACTITY table and also play with log level

https://docs.openkm.com/kcenter/view/ok ... y-log.html
https://docs.openkm.com/kcenter/view/ok ... logactions

Re: help with dms2okm-knowledge-tree

PostPosted:Tue Nov 21, 2017 12:21 am
by amer1
I gave up on the tool. I just wrote a quick VBS script to get my data over there. It's just a single use script, so not fancy. But if anyone still has a community edition Knowledgetree DMS, it may be handy. So I pasted it here with modified creds and names.
Code: Select all
dim pathfrom: pathfrom = "y:\Documents"
dim pathto: pathto = "\\ubuntu16srv1\shared\export\"
dim sServer: sServer = "knowlesgetreesrv1"
dim sDB: sDB = "dms"
dim sID: sID = "mysqluser1"
dim sPW: sPW = "*******"
Dim conn
Dim rs
Dim rs2
Dim sql

Set fso = CreateObject("Scripting.FileSystemObject")
Set oShell = WScript.CreateObject ("WScript.Shell")
const adUseServer = 2
set conn = CreateObject("ADODB.Connection")
set rs = CreateObject("ADODB.Recordset")
rs.CursorLocation = adUseServer
set rs2 = CreateObject("ADODB.Recordset")
rs2.CursorLocation = adUseServer
conn.connectionstring = "Driver={MySQL ODBC 5.3 UNICODE Driver};Server=" & sServer & ";Database=" & sDB & ";User=" & sID & ";Password=" & sPW & ";Option=3;"
conn.Open

sql = "select status_id,document_id,full_path,filename,storage_path,modified,created from document_content_version, documents where document_id = documents.id and status_id = 1;"

rs.Open sql, conn
i = 0
Do Until rs.EOF
	cr=rs("created").value
	mo=rs("modified").value
	p=rs("full_path").value
	f=rs("filename").value
	e="." & lcase(split(f,".")(len(f) - len(replace(f, ".", ""))) )
	n=split(p,"/")(len(p) - len(replace(p, "/", "")))
	path = replace(replace(p,"/","\"),n,"")
	n=n&e
	m=n&".okm"
	wscript.echo "Write to:", pathto & path & m
	oShell.run "cmd.exe /c md """ & pathto & path & """", 0, true
	set outText = fso.CreateTextFile(pathto & path & m, true)
	outText.writeline "{""name"":""" & n & ""","
	outText.writeline """created"":" & timestamp2okm(cr) & ","
	outText.writeline """lastModified"":" & timestamp2okm(mo) & ","

	outText.writeline """notes"":[" 
	sql2 = "select username, datetime, comment from document_transactions, users where users.id = user_id and comment like 'Workflow%' and document_id = " & rs("document_id").value & ";"
	rs2.Open sql2, conn
	j=0
	Do Until rs2.EOF
		if j > 0 then outText.writeline ","
		outText.write "  {""user"":""" & rs2("username").value & """,""date"":" & timestamp2okm(rs2("datetime").value) & ",""text"":""" & replace(rs2("comment").value,"""","\""") & """}"
		rs2.MoveNext
		j=j+1
	Loop
	rs2.Close
	outText.writeline "],"

	outText.write """keywords"":["
	sql2 = "select tag from document_tags, tag_words where tag_id = id and document_id = " & rs("document_id").value & ";"
	rs2.Open sql2, conn
	j=0
	Do Until rs2.EOF
		if j > 0 then outText.write ","
		outText.write """" & rs2("tag").value & """"
		rs2.MoveNext
		j=j+1
	Loop
	rs2.Close
	outText.writeline "]"

	outText.writeline "}"
	outText.close

	wscript.echo "Copy", pathfrom & "\" & replace(rs("storage_path").value,"/","\"), pathto & path & n
	oShell.run "cmd.exe /c copy """ & pathfrom & "\" & replace(rs("storage_path").value,"/","\") & """ """ & pathto & path & n & """"
	rs.MoveNext
	wscript.echo
	i=i+1
Loop
rs.Close
conn.close

wscript.echo "Documents exported:", i

function timestamp2okm(ts)
	timestamp2okm = "{""year"":" & year(ts) & ",""month"":" & month(ts)-1 & ",""dayOfMonth"":" & day(ts) & ",""hourOfDay"":" & hour(ts) & ",""minute"":" & minute(ts) & ",""second"":" & second(ts) & "}"
end function

Re: help with dms2okm-knowledge-tree

PostPosted:Wed Nov 22, 2017 4:16 pm
by jllort
Thanks for sharing it.

Re: help with dms2okm-knowledge-tree

PostPosted:Wed Sep 19, 2018 2:49 pm
by ColH
Thanks for the VBS script. I've been able to export from knowledgetree, but on the import (openkm v6.3.6) I get "Error: null" after just one or two documents.
I'm unsure whether the exporter is the cause of this - I haven't yet worked out how to explore the error - but is the 1.1.1 version of the java exporter still around somewhere?

Re: help with dms2okm-knowledge-tree

PostPosted:Wed Sep 19, 2018 3:41 pm
by ColH
ColH wrote: Wed Sep 19, 2018 2:49 pm Thanks for the VBS script. I've been able to export from knowledgetree, but on the import (openkm v6.3.6) I get "Error: null" after just one or two documents.
I'm unsure whether the exporter is the cause of this - I haven't yet worked out how to explore the error - but is the 1.1.1 version of the java exporter still around somewhere?
Ah ok - I've got further. Info for others doing the same operation:
First off, I was trying to import into /okm:root/ - it seems you have to create a folder and then import into that.
I was then able to import my subfolders, but no docs appeared. This was resolved by not checking 'History'

Re: help with dms2okm-knowledge-tree

PostPosted:Fri Sep 21, 2018 6:51 pm
by jllort
You must import without history and metadata. If it does not work, please share with us more detailed error message or sample for testing from our side. Some folder with no critical files ( test folder from KnowledgeTree is fine ) -> export and share with us for better understanding what's happening if you do not success on it.