Page 1 of 1

fail to permission files after using bulk uploader OpenKM 6.3 CE

PostPosted:Mon Jul 05, 2021 8:50 am
by adrian.sima
Hello,

We are currently using OpenKM 6.3 CE and last week I managed to migrate over 50GB of documents into our OpenKM production repository using built-in OpenKM tool: Bulk Uploader/File Uploader. The files were uploaded using 'Administrator' user.

After setting up permissions for other users ( different Roles or separate user access) I noticed there's no visibility for them. Before I opened up this thread I tried several possibilities to grant at least read permissions for the newly migrated resources:

- created a new user and set up permissions read/write ( including recursive permissions) for our main directory
The result here was that the main directory was visible under taxonomy but the files/subfolders within the main directory were not.
- afterwards drilled into the subfolders and granted same permissions ( read/write, including recursive perm.) but the result wasn't access to files but only to the subfolder. If there were more subfolders these weren't available for the end user, just the one I granted permission earlier.

I also researched on various problems with other users using 'Bulk Uploader' but I didn't find anything similar.

The conclusion is that at the moment only the admin who uploaded the files into the application repository can view/edit/grant permissions and there's no workaround in order to grant permissions to the newly migrated files.

Would really be of help if we could determine if wether this is an issue with permissions at OS Level over the uploaded documents or if it's a bug in the application which isn't letting me to make visible the files after using 'Bulk Uploader' openkm tool.

If you require additional printscreens I can deliver them over the e-mail because we can't replicate this issue into the test environment since I have uploaded the files directly into the production environment.

Thanks in advance,

Kind Regards,
Adrian.

Re: fail to permission files after using bulk uploader OpenKM 6.3 CE

PostPosted:Tue Jul 06, 2021 8:41 am
by adrian.sima
Hello,

I attached a test case from an OpenKM 6.3 CE test environment ilustrating the replication steps for the presented issue.

Kind Regards,
Adrian.

Re: fail to permission files after using bulk uploader OpenKM 6.3 CE

PostPosted:Sat Jul 10, 2021 1:10 pm
by jllort
* You should create a ROLE -> assign to a users
* Go to some folder and then change security adding this ROLE -> ensure you check the checkbox named apply security recursive ( if the security is applied to a lot of nodes may take a lot of time until be finished )
Selección_033.png
Selección_033.png (24.48 KiB) Viewed 5903 times
Another way to propagate security at low level is this script ( after the execution is it required you rebuild lucene indexes, because changes done directly from database are not propagated to the search engine )
:https://docs.openkm.com/kcenter/view/ok ... level.html

Re: fail to permission files after using bulk uploader OpenKM 6.3 CE

PostPosted:Fri Jul 16, 2021 1:58 pm
by adrian.sima
Hello Jilort,

Please check step 8 of the previous attached documentation and we see there ROLE_USER already assigned to the Dummy user.

Also check currently attached document which I assigned another role, granted permissions in the directory where I bulk uploaded the files and then logged in with dummy user and there's the same behavior: no visibility over the files.

*I also included recursive permissions in every tested scenario. Even in production environment where I migrated over 50 GB of data I tried all mentioned possibilities and didn't get any positive result in seeing the files with any other user than 'Administrator' user ( the one which I used for Bulk Uploader activity).
(106.1 KiB) Downloaded 213 times
In reviewed the other documentation to propagate security at low level but at some point there's rebuilding lucene indexes which might affect performance of the application? We have 1.2 T of data and I guess you can't rebuild those indexes just for the migrated data, right?

Do you have any other suggestions?

Kind Regards,
Adrian.

Re: fail to permission files after using bulk uploader OpenKM 6.3 CE

PostPosted:Sun Jul 18, 2021 10:45 am
by jllort
When you talk about bulk uploader -> about what tools are you talking about?
Seems you are doing the initial import, why are not using OpenKM repository import https://docs.openkm.com/kcenter/view/ok ... mport.html ? the files to be uploaded are not accessible directly from the OpenKM server?

Re: fail to permission files after using bulk uploader OpenKM 6.3 CE

PostPosted:Mon Jul 19, 2021 8:45 am
by adrian.sima
The bulk uploader I was talking about is referenced in your OpenKM CE 6.3 documentation, accessible at URL: https://docs.openkm.com/kcenter/view/ok ... oader.html

The files I needed to upload in OpenKM aren't from the current OpenKM repository which in our case is underlying on the application server.

The files I uploaded reside in another server, were uploaded using RDP connection to that server, connected with admin user and used the procedure above.

Kind Regards,
Adrian.

Re: fail to permission files after using bulk uploader OpenKM 6.3 CE

PostPosted:Sun Jul 25, 2021 8:46 am
by jllort
It seems the java tool for bulk upload I suggest do not use it, you can drag & drop directly from OpenKM UI https://www.youtube.com/watch?v=JbFvSNJ ... e=emb_logo

In case of a large upload with a lot of folders and files I suggest transfer files from your computer to the OpenKM server and from there use the OpenKM repository import ( Administration > tools > repository import ). It requires an extra step - copy to the server - but sure will work better and you'll get more control of the whole process.

Re: fail to permission files after using bulk uploader OpenKM 6.3 CE

PostPosted:Mon Jul 26, 2021 7:04 am
by adrian.sima
Hello again,

Can we validate 100 % percent that we cannot do anything in regards of the migration process I already did on production environement?

We're talking about over 50GB of data, time consumed resources and a planification which went smoothly until I couldn't access the files from the system on end-users side.

Since we're also involved in a upgrade process for OpenKM Pro (with your partner in Romania, Softex) I'd like from your side some support or a valid workaround to gain time and use the files which were already uploaded.

There's lot of data to be stored locally, there's also sensitive data and might take time in order to get approvals on your suggested scenario.

Thanks again for support but I'd appreciate a little more research on this java tool on your side.

Kind Regards,
Adrian.

Re: fail to permission files after using bulk uploader OpenKM 6.3 CE

PostPosted:Sat Jul 31, 2021 5:02 pm
by jllort
Ir you are going to upgrade to OpenKM pro do not have the sense to use the CE forum. For this purpose, there's the OpenKM professional support website.

In case of huge import should analyze data to transfer and when analyzed decide what is the best manner for doing it. There's no magic process for doing it because although most scenarios are very similar not in all of them can use the same way to import data. I suggest talking with your sales manager in order to coordinate this matter.

Re: fail to permission files after using bulk uploader OpenKM 6.3 CE

PostPosted:Wed Aug 11, 2021 8:38 am
by adrian.sima
Hello,

At the moment we're using OpenKM CE therefore I must use the forum, this migration process is at the moment at project phase and might take from 6 months to 1 year to complete.

Since this tool was available and since I've done so much effort in using this tool can't you propose a decent solution for our blocked scenario?

It's quite simple:

1. 50 GB of data uploaded using Bulk Uploader
2. Files available just for application admin user ( the same user I've uploaded the files with )
3. Tested all sorts of workarounds and can't seem to make data available for other end users.
4. That's why I created a CE Forum user to get professional instructions on this available function.

Scope of work at the moment is to benefit from some kind of support using CE, would be nice at least to find out if at server level can be some checks in order to make files available.

Kind Regards,
Adrian.

Re: fail to permission files after using bulk uploader OpenKM 6.3 CE

PostPosted:Wed Aug 11, 2021 8:53 pm
by jllort
Do not exist a quick answer for your question, because the problem is that your scenario should be analyzed to understanding the problem you are trying to solve and usually is not a task that can be completed in 5 minutes ( bulk upload is not the problem is the solution you tried to apply to the problem ).

Anyway I suggest do not use bulk upload for this huge import you should:
1- copy files in the server and use import tools ( it creates a log -> you need a log - may be this log is not exactly what you need - of what happens and usually have an acceptable speed )
2- OpenKM cron job to upload files from disk location, network location, or another kind of localtion like remote ftp
3- create a java client or c# client and use the application to upload ( can create your own log and take more control -> for example the number of parallel uploads allowed )
4- use hot folder windows client -> do not upload folders only files in the same folder -> that means maybe is not useful to solve the problem you are trying to solve

When having large uploading process you need always ( may be other too, but if you consider these requisits then can thinking about what option may fit better for these requirements ):
* control what happened -> log
* control errors -> log of errors
* When possible do in the background during nonworking hours