Files from trash openkm

[DanielGalvis]

I have OpenKM installed on a virtualized server in AWS with CentOS 7 as the operating system. It is configured with Tomcat 9.0, and the database is on MariaDB. During production hours, the system experiences frequent crashes. The temporary solution to address the issue has been to stop Tomcat and restart the server. However, these incidents are occurring more frequently. When monitoring the machine with 'top,' it's evident that the MariaDB service is consuming the most resources. The server is using a substantial instance type, so the issue does not seem to be related to machine resource constraints.

We discovered that part of the problem is the mysqld service, which is exceeding resource limits because the 'trash' folder is syncing with the database. We need to find out how to make OpenKM synchronize without including the 'trash' folder.

[jllort]

¿The tomcat stops?
¿is created any hprof file?

* I suggest start purge the trash.

Share a screenshot of the cron jobs you have running and I will suggest to stop someone

[DanielGalvis]

how can i clean the trash in the right way, because we find the problem and its the trash, but now we have to know how to do this task

[DanielGalvis]

We realized that the main problem is the OpenKM trash, as it occupies more than 50% of the application's taxonomy. However, we need to know how to proceed with a query, a task, or something similar to eliminate all this taxonomy since it is affecting us. If one tries to delete it manually from the web, it says that the folder does not exist. Therefore, I would like to know if there is a way to do it directly from SSH and the steps to follow. I am attentive if you require relevant information to carry out this task.

[DanielGalvis]

there is the crontab task

[jllort]

If the clean trash process does not work because it is based on a single transaction then you should create a script to iterate in the trash and purge in several steps.

* Should navigate to the deepest folder in the trash hierarchy and purge recursively in the bottom direction, you will nee d to use the next methods

https://docs.openkm.com/kcenter/view/ok ... etChildren
https://docs.openkm.com/kcenter/view/ok ... html#purge
https://docs.openkm.com/kcenter/view/ok ... etChildren
https://docs.openkm.com/kcenter/view/ok ... html#purge

If you have a lot of nodes in the same folder then you should not use the API methods to get the list, you should use LegacyDAO to get the list and then clean in groups of 100
https://docs.openkm.com/kcenter/view/ok ... cydao.html

Also you should use:
https://docs.openkm.com/kcenter/view/ok ... pting.html
https://docs.openkm.com/kcenter/view/ok ... ogger.html ( to create a log )

[DanielGalvis]

Code: Select all

import com.openkm.api.*;
import com.openkm.core.*;
import com.openkm.bean.*;
import com.openkm.module.db.stuff.*;
 
String token = DbSessionManager.getInstance().getSystemToken();
 
for (Folder trash : OKMFolder.getInstance().getChildren(token, "/okm:trash")) {
    print("Trash: " + trash.getPath() + "<br/>");
 
    for (Folder fld : OKMFolder.getInstance().getChildren(token, trash.getPath())) {
        print("About to delete folder: " + fld.getPath() + "<br/>");
        OKMFolder.getInstance().purge(token, fld.getPath());
    }
 
    for (Document doc : OKMDocument.getInstance().getChildren(token, trash.getPath())) {
        print("About to delete document: " + doc.getPath() + "<br/>");
 
        if (OKMDocument.getInstance().isLocked(token, doc.getPath())) {
            OKMDocument.getInstance().forceUnlock(token, doc.getPath());
        }
 
        OKMDocument.getInstance().purge(token, doc.getPath());
    }
 
    for (Mail mail : OKMMail.getInstance().getChildren(token, trash.getPath())) {
        print("About to delete mail: " + mail.getPath() + "<br/>");
        OKMMail.getInstance().purge(token, mail.getPath());
    }
}

i ran that script but appears this error

Code: Select all

 bsh.TargetError: Sourced file: inline evaluation of: ``import com.openkm.api.*; import com.openkm.core.*; import com.openkm.bean.*; . . . '' : 
Method Invocation purge : at Line: 23 : in file: inline evaluation of: ``import com.openkm.api.*; import com.openkm.core.*; import com.openkm.bean.*; . . . '' :
 .purge ( token , doc .getPath ( ) ) Target exception: com.openkm.core.RepositoryException: 
Cant' delete file (not exists) '/usr/local/credifam/tomcat-9.0.83/repository/datastore/2d/5a/5a/eb/2d5a5aeb-b41d-4c9b-9a20-c5a74f5b6d79' 

with many paths but they are different any idea to execute withou this error

[jllort]

* You must execute the repository checker for the okm:trash -> will be created a file DbRepositoryCheckerXXX.log
* From this check get all the errors -> in linux with the command

Code: Select all

cat  DbRepositoryCheckerXXX.log > grep ERROR > error.txt

In the file you have all the missing files you should have in the repository, you must create them ( empty files with same name ), in the linux you can use the command

Code: Select all

touch /datastore/ac/de/2e/3b/acde2e4b-XXXXXXXXX

When you get all the missing files recovered in the file system then will be able to delete from the script

[DanielGalvis]

it works very well but, now i have many documents locket or in edition in the trash folders and i can't delate or do anything with this documents, any idea, very thanks

[DanielGalvis]

i was thinking how i can do to migrate openkm to another server but without trash repository only taxonomy there's any way to do that?

[jllort]

* You must export all the data with the repository exporter tool and then import with the import feature.
* Another way to do it could be with several SQL scripts -> but I do not like much play to with the database at this level -> at least you should do before an database backup.

1- first search the okm:trash node ( can not delete it )

Code: Select all

select NBS_UUID from OKM_NODE_BASE where NBS_NAME='okm:trash'

in my case the UUID value is a3a883a5-507b-4672-b519-2720dfbb1161

2- delete data

Code: Select all

delete from OKM_NODE_DOCUMENT where NBS_UUID in (select NBS_UUID from OKM_NODE_BASE where NBS_CONTEXT='okm_trash' and NBS_UUID<>'a3a883a5-507b-4672-b519-2720dfbb1161');
delete from OKM_NODE_FOLDER where NBS_UUID in (select NBS_UUID from OKM_NODE_BASE where NBS_CONTEXT='okm_trash' and NBS_UUID<>'a3a883a5-507b-4672-b519-2720dfbb1161');
delete from OKM_NODE_MAIL where NBS_UUID in (select NBS_UUID from OKM_NODE_BASE where NBS_CONTEXT='okm_trash' and NBS_UUID<>'a3a883a5-507b-4672-b519-2720dfbb1161');
delete from OKM_NODE_USER_PERMISSION where NUP_NODE in (select NBS_UUID from OKM_NODE_BASE where NBS_CONTEXT='okm_trash' and NBS_UUID<>'a3a883a5-507b-4672-b519-2720dfbb1161');
delete from OKM_NODE_ROLE_PERMISSION where NRP_NODE in (select NBS_UUID from OKM_NODE_BASE where NBS_CONTEXT='okm_trash' and NBS_UUID<>'a3a883a5-507b-4672-b519-2720dfbb1161');
delete from OKM_NODE_BASE where NBS_CONTEXT='okm_trash' and NBS_UUID<>'a3a883a5-507b-4672-b519-2720dfbb1161';

3- Reindexing the whole repository ( lucene search engine )

At then you will get some dirty in the database ( OKM_NODE_PROPERTY for example ) and some orphan files in the datastore.