Page 3 of 7

Re: Automatic import

PostPosted:Tue Mar 22, 2011 9:22 pm
by snowman
Hello,

just for the ones who still follow the orginal question. A friend of mine was able to help me out and put together a -really- simple php script that does the upload job. I want to share it with you:
Code: Select all
<?php

$okmurl="http://localhost:8080/OpenKM/";
$okmid="username";
$okmpw="password";
$impfile = $argv[1];

$OKMAuth = new SoapClient($okmurl."OKMAuth?wsdl");
$OKMDocument = new SoapClient($okmurl."OKMDocument?wsdl");

// open file and encode if necessary
$handle = fopen($impfile,'rb');
$file_content = fread($handle,filesize($impfile));
fclose($handle);
$encoded = base64_encode($file_content);

// Login
$token = $OKMAuth->login($okmid, $okmpw);
$OKMDocument->createSimple($token,"/okm:root/file.to.upload",$file_content);

// Logout
$OKMAuth->logout($token);
?> 
It has no error handling but works ok. Embedded in a cron job it should do.

Re: Automatic import

PostPosted:Wed Mar 23, 2011 5:23 am
by joako
I don't see a response, has someone been able to look into this?

Re: Automatic import

PostPosted:Wed Mar 23, 2011 5:25 am
by joako
Snowman,

so we would run from another script like:
Code: Select all
for file in /scan/*pdf;
do;
/usr/local/bin/import.php $file
mv $ file /importedscan/
done;
I will give it a try! Thanks!

Re: Automatic import

PostPosted:Fri Mar 25, 2011 3:03 am
by joako
I've made some changes in your code and created a bash script to call the php code.


/opt/openkm/import.php:
Code: Select all
<?php

$okmurl="http://docs/OpenKM/";
$okmid="okmAdmin";
$okmpw="admin";
$impfile = $argv[1];
$filename =  $argv[2];
$folder =  $argv[3];

$OKMAuth = new SoapClient($okmurl."OKMAuth?wsdl");
$OKMDocument = new SoapClient($okmurl."OKMDocument?wsdl");

// open file and encode if necessary
$handle = fopen($impfile,'rb');
$file_content = fread($handle,filesize($impfile));
fclose($handle);
$encoded = base64_encode($file_content);

// Login
$token = $OKMAuth->login($okmid, $okmpw);
$OKMDocument->createSimple($token,"/okm:root/$folder/$filename",$file_content);

// Logout
$OKMAuth->logout($token);
?>
import.sh

See the new one below!

Re: Automatic import

PostPosted:Fri Mar 25, 2011 4:59 am
by joako
Here is updated version, greatly improved.

You can call from script like I proposed the other day in crontab, or make some trigger when new files are placed.

You can configure the default directory and specify one that overrides it in the command line:

$ ./import.sh import-1.0.0.sh E-Mail
Fri Mar 25 00:56:33 EDT 2011 JOAKO Import: Imported file [import-1.0.0.sh] to [E-Mail] directory

$ ./import.sh import-1.0.0.sh
Fri Mar 25 00:57:38 EDT 2011 JOAKO Import: Imported file [import-1.0.0.sh] to [NEW] directory

Code: Select all
#!/bin/bash
# JOAKO Import 1.0.0 - Copyright 2011 Andrew Joakimsen
# Distributed under the terms of the GNU General Public
# License Version 2 and no future versions.
#
# Script to import files into OpenKM and report error/success.
# Requires the PHP import script written by snowman, with a
# few small tweaks.

###################################
###                             ###
###  C O N F I G U R A T I O N  ###
###                             ###
###################################


# OpenKM directory where files are imported to
OKM_IMPORT_DIR=NEW

# Filesystem directory (no trailing slash) where we put imported files (it will be created)
STORE_DIR=~/we-test

# Filesystem directory (no trailing slash) where we put files that encountered import errors
ERROR_STORE_DIR=~/error-test

####################################

# First let's see if the runtime condition is sane
if [ -z "$1" ]; then
        echo "JOAKO Import: Missing file name"
        echo "Usage: import.sh [filename] [target-directory]"
        exit 1
fi


# If ARG1 is a real file
if [ -d "$1" ]; then
        echo "ERROR: \"$1\" is a directory!"
        exit 1
fi
if [ ! -f "$1" ]; then
        echo "ERROR: \"$1\" Invalid file name "
        exit 1
else
        INFILE=$1
fi

# Check if OKM_IMPORT_DIR was passed as argument
if [ -z "$2" ]; then
        OKM_DIR=$OKM_IMPORT_DIR
else
        OKM_DIR=$2
fi

# Check if the imported files directory exsists
if [ ! -d $STORE_DIR/$OKM_DIR ]; then
        if [ ! -d ~/$STORE_DIR ]; then
                mkdir $STORE_DIR
        fi
        mkdir $STORE_DIR/$OKM_DIR
fi

# Run the import Process
RESULT=$(php /opt/openkm/import.php $INFILE ${INFILE##*/} $OKM_DIR 2>&1)
RESULTCODE=$?

# Deal with the consequences
function importer-error {
        # Check if the import error directory exsists
        if [ ! -d $ERROR_STORE_DIR/$OKM_DIR ]; then
                if [ ! -d $ERROR_STORE_DIR ]; then
                        mkdir $ERROR_STORE_DIR
                fi
                mkdir $ERROR_STORE_DIR/$OKM_DIR
        fi
        echo "`date` JOAKO Import Encountered an error type $RESULTCODE."
        echo "$RESULT"
        mv $INFILE $ERROR_STORE_DIR/$OKM_DIR &> /dev/null
        exit $?
}

if [ $RESULTCODE -ne 0 ]; then # There was an error
        importer-error
else # We check further
        if [ -n "$RESULT" ]; then # There was an error
                importer-error
        else # No error detected
                echo "`date` JOAKO Import: Imported file [${INFILE##*/}] to [$OKM_DIR] directory"
                mv $INFILE $STORE_DIR/$OKM_DIR &> /dev/null
                exit 0
        fi
fi
One thing Is I can't run this on anything but localhost. The first SOAP request returns with 127.0.0.1 for future requests.

/Edit: Minor fix, all paths should work now:

$ ../../import.sh /Users/joako/error-test/NEW/23423423.txt E-Mail
Fri Mar 25 01:04:13 EDT 2011 JOAKO Import: Imported file [23423423.txt] to [E-Mail] directory

Re: Automatic import

PostPosted:Fri Mar 25, 2011 12:54 pm
by jllort
Hi joako

I'm thinking adding a section on wiki called utilities. Can copy the script there, to sharing with other users ? because this kind of information in forum is transparent to major users.

Re: Automatic import

PostPosted:Sat Mar 26, 2011 6:30 am
by joako
Sure, as you see in the code I explicitly made it GPL v2, but can you give me some days, once you add it in the wiki, I probably won't be able to access it.

I'm going to test a few days running from launchd and maybe cron to see how it works. Right now I set to use launchd "QueueDirectories" and it doesn't seem to be importing. Also I know I said GPL v2 but the disclaimers aren't "100% correct" so I would rather everything be correct and tested it before you store it away in your wiki.

Re: Automatic import

PostPosted:Sat Mar 26, 2011 10:05 am
by jllort
Ok, Tell us when it'll be tested, and you considering is right, and will put on wiki, we've openned a new section on main page call utilities. We'll getting all community utilities there.

Re: Automatic import

PostPosted:Mon Mar 28, 2011 4:41 pm
by pavila
You can also make the php code more robust adding exception handling. See this example:
Code: Select all
<?php
  function format_exception($e) {
    if (isset($e->detail)) {
      $reflectionObject = new ReflectionObject($e->detail);
      $properties = $reflectionObject->getProperties();
      $exceptionName = $properties[0]->name;
    } else {
      $exceptionName = "Exception";
    }
    return $exceptionName.": ".$e->faultstring;
  }
 
  try {
    $token = $OKMAuth->login('okmAdmin', 'admin');
    $document = $OKMDocument->getProperties($token, '/okm:root/hosts.txt');
    print_r($document);
  } catch (Exception $e) {
    echo format_exception($e);
  } finally {
    $OKMAuth->logout($token);
  }
?>

Re: Automatic import

PostPosted:Mon Mar 28, 2011 4:47 pm
by pavila
joako wrote:One thing Is I can't run this on anything but localhost. The first SOAP request returns with 127.0.0.1 for future requests
Don't see why. I have tested the script against the a remote server and work pretty well.

Re: Automatic import

PostPosted:Wed Mar 30, 2011 3:24 am
by joako
pavila wrote:
joako wrote:One thing Is I can't run this on anything but localhost. The first SOAP request returns with 127.0.0.1 for future requests
Don't see why. I have tested the script against the a remote server and work pretty well.
I run it through a proxy.

OpenKM runs & listens on localhost and then I run a proxy on the apache webserver from http://server/OpenKM/ to http://127.0.0.1:8080/OpenKM/continue/the/long/url.

Could this be the issue? The SOAP response is what OpenKM think are the server, but it's wrong because it's being passed to the proxy?

Re: Automatic import

PostPosted:Wed Mar 30, 2011 6:51 pm
by pavila
I think this is due to a JBoss configuration parameter. Don't remember now, but is related to JBoss and webservices. Anyway I thinl the OpenKM + JBoss bundle should have this parameter properly configured.

Re: Automatic import

PostPosted:Sat Apr 02, 2011 2:27 pm
by jllort
There's a configuration parameter with jboss about proxy settings, I don't remember exactly the parameter, in older forum post has been talking about it. If you can not find here ( sure there's ) I suggest make the question on public jboss forum, there sure will anwering you about this jboss configuration parameter.

Re: Automatic import

PostPosted:Sun May 08, 2011 6:50 am
by joako
I really haven't been using this.

But now suddenly my OpenKM won't accept the php code any more. Why is this?
Code: Select all
#  php /opt/openkm/import.php /Volumes/Storage/ScanError/Accounting/SCAN26011400_1.pdf 323test23423rwerwe.pdf NEW
PHP Fatal error:  Uncaught SoapFault exception: [env:Server] java.lang.NullPointerException in /opt/openkm/import.php:21
Stack trace:
#0 /opt/openkm/import.php(21): SoapClient->__call('createSimple', Array)
#1 /opt/openkm/import.php(21): SoapClient->createSimple('747998062802045...', '/okm:root/NEW/3...', '%PDF-1.4?%?????...')
#2 {main}
  thrown in /opt/openkm/import.php on line 21

Re: Automatic import

PostPosted:Sun May 08, 2011 6:59 am
by joako
The "good thing" is when I try to upload from web GUI I get similar errors, create new folder gives:


# php /opt/openkm/import.php /Volumes/Storage/ScanError/Accounting/SCAN26011400_1.pdf 323test23423rwerwe.pdf NEW
PHP Fatal error: Uncaught SoapFault exception: [env:Server] java.lang.NullPointerException in /opt/openkm/import.php:21
Stack trace:
#0 /opt/openkm/import.php(21): SoapClient->__call('createSimple', Array)
#1 /opt/openkm/import.php(21): SoapClient->createSimple('747998062802045...', '/okm:root/NEW/3...', '%PDF-1.4?%?????...')
#2 {main}
thrown in /opt/openkm/import.php on line 21


Some RAM in the server went bad. It crashed 3 times in one day before it got taken out. So something must be corrupt...