No primary keys on tables

[jllort]

I think lucene indexer problem can be solved if do not take in consideration others problems. If concentrate only in this problem should take in mind can not be indexed contents if rsync has not been completed, the binary sync should be done before database. Indexer has two stage:
1- When new record is added the metadata of record is indexed and document is marked in database to index contents, columns NDC_TEXT_EXTRACTED indicates that a document is on queue. Across hibernate api, any record changed is automatically indexed by lucene ( folder, document, mail or record ). That's I think the major difficult if you want to go with option 1.
2- Index documents in pending queue (NDC_TEXT_EXTRACTED = 'F')
SELECT * FROM OKM_NODE_DOCUMENT where NDC_TEXT_EXTRACTED = 'F';

[Alexander]

Do other problems, except conflicts and lucene, exists on db replication method ?

[jllort]

Should not be other problems that nodes modified at same time in both servers. Only database sync will not be the solution, for example edit two files at same time by two differents users -> this mark database with checkout and user token ( which will be different in each case ), both check-in the document here you got a problem that should be solved by human ( conflict ), or for example one user upload new document and other restores older version.

Althought sync database can be an aproximation has some critical problems. Should be registered by order the changes done in master A and try to apply in master B in same order, and if there's some conflict then allow human to make something. What you want to do is not trivial. I think this kind of software - DMS - which have exclusive locks in some operations and have security restrictions which can be changed in both sides can not be sync only by database.

If your initial idea can be possible will be perfect, but after thinking in more deep I think there're so critical problems with this kind of sync to discard it.

[Alexander]

Short report.
1.
Create linux VM with postgresql and bucardo, set it's postgresql as db for openkm -> postgresql#1
2.
Comment out @Lob in file /src/main/java/com/openkm/dao/bean/NodeDocumentVersion.java

@Column(name = "NDV_CONTENT")
---------> //@Lob
private byte[] content;

to make hibernate reflect content to bytea type, not oid. Bucardo able to replicate bytea, but not able to replicate oid;
compile openkm; run openkm.

3.
Create primary keys on tables without them by stored procedure.

4.
Copy VM, change it's IP -> postgresql#2,
copy openkm, configure it to use postgresql#2 -> openkm#2

5. Configure bucardo master-master replication postgresql#1 <--> postgresql#2

Documents, categories, users, taxonomy, etc. working well.

Problem: after moving item, at another server, appears problem with old Folder in Properties tab while new data already in db.

After all, it seems not difficult to provide "changing stream" by creating new table (tbl1) to store changes with source table name, row data in XML , some other info; then handle save (may be by session.save()? ) and put saved in new table (tbl1); transfer it to other server and, by table name and XML data (and other) apply changes in code by API.

[jllort]

About point 5 some questions:
Replication is on real time or scheduled ?

About the problem: "Problem: after moving item, at another server, appears problem with old Folder in Properties tab while new data already in db. " Be more specific and graphic to be able to understand it. With more detail, otherside I can not image it.

And the solution with new table1 and xml, I do not understood.

[Alexander]

About point 5 some questions:
Replication is on real time or scheduled ?

online asynchronous.

About the problem: "Problem: after moving item, at another server, appears problem with old Folder in Properties tab while new data already in db. " Be more specific and graphic to be able to understand it. With more detail, otherside I can not image it.

I attach screenshots in separate post

And the solution with new table1 and xml, I do not understood.

[/quote]
I attach chema in separate post

[Alexander]

1.png (122.71 KiB) Viewed 6014 times

2.png (133.92 KiB) Viewed 6014 times

3.png (163.2 KiB) Viewed 6014 times

[jllort]

I will try to explain the possible cause of the problem:
In firs screenshot in left you select document and right folder that's cause of different path.

When you sync if user has never loged to openkm the /okm:personal/username has not still been created, this is the problem. The main personal, trash nodes are created on first time user login.

[Alexander]

I had to explain screenshots.

------------------------------------------------screenshot 1--------------------------
1. Login as okmAdmin to openkm#1 (left pane)
2. Login as okmAdmin to openkm#2 (right pane)
3. Upload file 2.txt to openkm#1 to /okm:personal/q1
4. Wait for a while - it appears on openkm#2
-------------------------------------------------------------------------------------------

-----------------------------------------------screenshot 2 --------------------------
5.move 2.txt to okm:root at openkm#1
-------------------------------------------------------------------------------------------
-----------------------------------------------screenshot 3 --------------------------
6.Wait for a while - it appears on openkm#2 at okm:root
7.But, on openkm#2, inspite of it appears at okm:root,
properties field contains /okm:personal/q1; related database record is the same as at openkm#1

It seems, that client (browser side ) cache folder field in memory , and try to use it without reloading from DB.

[jllort]

Browser is a snapshot of what you have in db is not live connection, each time you make a click information is refresh. Like any web page you make a query and you get some results, meanwhile you do not make any query you see the last results, this is the normal way for doing it, otherside you'll get server overload,

Imagine you see folder /okm:root/folder1/folder2/folder3
Then a user remove folder2
when you click in some document or to refresh folder3 you'll get Path not exists exception. This is normal error.

Do not think the aplication with two users connected, try to imagine with a lot of simultaneous connections at same time, thousands, and hope you'll understand is the correct way to solve concurrent problems on UI view.

[Alexander]

It seems the reason is 1-st level Hibernate cache.
Replication changes table [okm_node_base] directly, but if Hibernate already caches related object, inconsistency appears.

[jllort]

You're doing test with community or professional version ?

[Alexander]

You're doing test with community or professional version ?

With community version.

[jllort]

I ask about it because on professional version we have control of cache but in community no. After sync could be done some call to clean cache.