Massive operations in community edition?

We tried to make OpenKM as intuitive as possible, but an advice is always welcome.
Forum rules
Please, before asking something see the documentation wiki or use the search feature of the forum. And remember we don't have a crystal ball or mental readers, so if you post about an issue tell us which OpenKM are you using and also the browser and operating system version. For more info read How to Report Bugs Effectively.
Post Reply
macjaeger
Senior Boarder
Senior Boarder
Posts: 56
Joined: Fri Oct 05, 2012 11:30 am

Massive operations in community edition?

Post by macjaeger » Mon Oct 15, 2012 11:14 pm

In OpenKM 6.2 community edition, the massive operations (e.g. add categories to a large number of files) are disabled. Is this due to developement state, or will community edition not have this feature?

jllort
Moderator
Moderator
Posts: 10767
Joined: Fri Dec 21, 2007 11:23 am
Location: Sineu - ( Illes Balears ) - Spain
Contact:

Re: Massive operations in community edition?

Post by jllort » Tue Oct 16, 2012 8:58 pm

Comunity will not have all features present in professional ( have the classical copy, delete and move ).

macjaeger
Senior Boarder
Senior Boarder
Posts: 56
Joined: Fri Oct 05, 2012 11:30 am

Re: Massive operations in community edition?

Post by macjaeger » Tue Oct 16, 2012 9:02 pm

Sad to hear that, yet thanks for the info.

Bruno
Fresh Boarder
Fresh Boarder
Posts: 10
Joined: Tue May 21, 2013 5:31 pm

Re: Massive operations in community edition?

Post by Bruno » Tue May 21, 2013 6:53 pm

Hello,
this is my first post in the forum as for now I am still evaluating the usability of OpenKM, and found all necessary config-info online up till now. So, sorry if I am not familiar with the process of professional to community features migration.

But has this answer changed with 6.2.3 ? I can't say if I encounter a bug or a community version restriction.

I was expecting "Massive add category" (and other massive functions) to work in community version, especially as this page http://wiki.openkm.com/index.php/Massive_add_category doesn't mention it is a professional version feature.
Plus available changelogs for either version are not very clear about this point :

- Community changelog :
6.2.0 : "massive functions" were added --> I suppose these were only copy/move/delete
6.2.2 : Massive checkout/download/cancel checkout implementation
6.2.3 : no mention of new massive functions, but there were "Misc fixes and improvements" who could include that...

- Professional changelog :
6.2.10 : Massive Property Groups update & Massive export to ZIP
(I don't know when the massive add category and others appeared in earlier professional versions)

I tried massively adding categories in the professionnal demo (http://demo.openkm.com/OpenKM) and it works fine.
But while it seems to be available in the menu in community version, it doesn't work on a "massive" scale :
- if one taxonomy folder is selected (on the taxonomy panel) and no document in the files list (right panel), the "Add category" menu option only adds it to the taxonomy folder itself but not to documents and subfolders.
- if no row on the documents lists from the right panel is selected (no blue row) and I check "Select all", the Add category from menu doesn't add it to all selected files and folders (but only the taxonomy folder selected on the left).
- if only some documents are manually checked, then it turns one row blue and only this one receives the category.

(my config for now is a localhost installation on Windows XP SP3, and I checked this with both OKM community v.6.2.3 and 6.2.4-DEV build 8074, with a MySQL okm database and the repository kept out of the MySQL database)


It seems to me that this "massive add category" (as well as massive add property group and massive add keyword) is an absolutely necessary "core" function of OKM : when users try to transfer an existing collection of documents to openKM, there often are dozens and even hundreds of documents which need identical categories and metadata (author, status...). I think it is really sad if this is only available in professional edition, as it doesn't allow to make realistic and rapid tests with community version.
Without this function nor understanding the future implementation (or not) of these functions in community version, I cannot plan OKM set-up and usage correctly, as dealing with each file individually means A LOT more time to spend before our pre-existing data is finally ready for use (my categories go down to numerous sub-levels, which make so many clicks for each file).

So :

1) is this a bug (it looks like one when an interface doesn't do what it suggests), or is there any luck we'll see these functions available in a next release (a new build, or official 6.2.4, or which later one ?) of the community version like the massive checkout/download were added ?

2) If not a bug, can you explain why in more details (do we only have to wait the corresponding community release version based on professional edition, like 6.2.10 for "Massive Property Groups update", or is it a definitive choice to leave it out of community for good ?), as I really don't understand the limitation ?

3) Or would someone know (and agree to explain) how to unlock/enable the function ?
Maybe an OKM community user has already modified the community code to have it work ? Like the way "ciropom" made a patch last november to implement the "massive checkout/download/cancel checkout" ? http://forum.openkm.com/viewtopic.php?f=31&t=8428

4) If none, is there a possible alternative on this step of categorizing numerous imported files to build a first database ?... like maybe :
- using a trial version of professional edition to massively edit files categories and metadata, to then export it and re-import it in community version ?
- modifying the MySQL database itself through SQL queries ? which tables are needed to SELECT from existing documents ? and can the link between selected files and categories be made this way and work in the OpenKM interface ? Anyone knows how to do this through MySQL Workbench ?


thanks for anyone's well needed help...
And thanks and congrats to all developpers of this fantastic application !

jllort
Moderator
Moderator
Posts: 10767
Joined: Fri Dec 21, 2007 11:23 am
Location: Sineu - ( Illes Balears ) - Spain
Contact:

Re: Massive operations in community edition?

Post by jllort » Wed May 22, 2013 9:48 am

I will try to explain fastly. Only one time in year we make major OpenKM community upgrade then we migrate features from professional to community, that will not happens until ends of the year ( we do it for several internal reasons, take in mind at least we maintaining tree versions at same time ). When community is released we add features provided by community users. For example the last feature which involved a community user has been http://wiki.openkm.com/index.php/OMR_templates, there's no problem on it. Other thing we doing is apply patch and make corrections, but our major objective is get maximum stability during a year. Until end of year we will not doing a major upgrade and then will be the moment to migrate features ( that's our compromise with community edition as daily answring in forum ). If you need some feature etc... as other users, you can get source code and make changes you consider, if are interesting we will add in community version to be shared to everyone. We got a detailed http://wiki.openkm.com/index.php/Developer_Guide which explain how working with openkm source code.

And finally, changes can not be done by mysql should be executed across api to propagate changes into lucene.

Bruno
Fresh Boarder
Fresh Boarder
Posts: 10
Joined: Tue May 21, 2013 5:31 pm

Re: Massive operations in community edition?

Post by Bruno » Wed May 22, 2013 2:03 pm

Thank you, I understand better now.

I doubt I would know how to change code myself (I am not a programmer), so if any user has tips about making a patch for "massive add category" I would be very interested

wgerven
Fresh Boarder
Fresh Boarder
Posts: 13
Joined: Thu Jul 18, 2013 11:19 am

Re: Massive operations in community edition?

Post by wgerven » Thu Jul 25, 2013 8:52 pm

Hi Bruno,

Maybe it's a bit late, but I have a suggestion for you. I was faced with the same problem and as such came across your post when searching the internet.
My tip is not at all "practical", but it offers a way to do "massive add category and/or keywords". In principle you can do this with the community version the way you downloaded it, you (or a friend) do need basic knowledge of python though to do this. :roll:

In this example I will describe how to do it: Suppose you have in your file system a folder with books in pdf, that you wish to add to OpenKM and massively add keywords and/or categories.

1) For a moment, don't upload them yet to OpenKM, but prepare the (if you haven't already) categories you wish to add to them in OpenKM. (You don't have to think about keywords yet.)

2) Create a seperate empty folder /temp in OpenKM.

3) Upload 1 random pdf document to /temp, to which you assign those categories you want to massive add to your folder of books.

4) Go to administration/export tab. Chose here to export the folder /temp to some folder on your local hard disk, and select to also export "metadata".

5) Navigate to this folder, it should contain the pdf and a file named like the pdf +.okm

6) Basically the trick is that you can copy the contents of this .okm file for every book, keeping in mind to change filenames in the content appropriately.

7) That's what you need a python script for to do that "massively". I.e., run trough the pdf filenames, create for them *.okm files, copy in those files the metadata of the file you exported & make the right adjustments within this content.

8 ) Then after running such script, you can go to the administration/import tab, and upload the whole folder with books. Make sure here to tick "Metadata" and to upload the books to /temp. As you don't tick "Restore UUIDs" there will be no file corruption as the file UUIDs will be newly generated by OpenKM.

9) All uploaded files should now have the categories attached to them. Keywords can be done massively by including them in the python script, but didn't have to be considered before the python step as OpenKM doesn't need UUIDs for them.

I agree it's not so neet, but it saves a lot of mouseclicks!! :|

Best regards,
Willem.

Edit: Make also sure that the python script fetches the file size and puts in the *.okm file at "size:".
Last edited by wgerven on Mon Jul 29, 2013 9:15 am, edited 1 time in total.

jllort
Moderator
Moderator
Posts: 10767
Joined: Fri Dec 21, 2007 11:23 am
Location: Sineu - ( Illes Balears ) - Spain
Contact:

Re: Massive operations in community edition?

Post by jllort » Fri Jul 26, 2013 8:52 am

In case you have categories assigned to documents you should restore UUID's, otherside is not necessary. If you're migrating from older openkm to newer is good practice store same UUID's, should not be problem on it. Remember first migrate categories and then the rest of the repository ( sometimes this steps seems is not clear ).

wgerven
Fresh Boarder
Fresh Boarder
Posts: 13
Joined: Thu Jul 18, 2013 11:19 am

Re: Massive operations in community edition?

Post by wgerven » Thu Aug 01, 2013 8:06 am

Hi jllort,

how to
Remember first migrate categories and then the rest of the repository ( sometimes this steps seems is not clear ).
:?:

thanks!

jllort
Moderator
Moderator
Posts: 10767
Joined: Fri Dec 21, 2007 11:23 am
Location: Sineu - ( Illes Balears ) - Spain
Contact:

Re: Massive operations in community edition?

Post by jllort » Fri Aug 02, 2013 6:45 pm

Basically you will export okm:root to a server folder and okm:categories to other server folder. When you restore in new OpenKM server first import okm:categories from server folder and the okm:root, that's the idea.

Post Reply