Page 1 of 2 12 LastLast
Results 1 to 15 of 17

Thread: Tool: Auto-copy images from USB-device and index-image creation

  1. #1

    Tool: Auto-copy images from USB-device and index-image creation

    Hi everybody,

    I liked the function of Asus' original software to automagically copy the contents of a USB-device (SD-card reader for example) to an incremental directory on the harddisk of my WL-HDD.
    Of course, Oleg's firmware outweighed Asus' original, but still I kind of wanted this feature.
    Through some help of this forum, I managed to come up with a set of scripts that helps me do so.

    Advantages:

    1. It works also with multi-card-readers or multi-partition devices that are not detected properly by the automounter
    2. It copies the contents of the storage device to an incremental directory on the harddisk that is easy to recognize (date-time format)
    3. It also creates an jpeg-image which contains the contents of the folder of the harddisk where the files have been backup-ed to and writes it back to the storage device.

    Why no. 3?
    Imagine you are out in the wild and want to backup your SD-card of your camera because it's full and only brought your WL-HDD because it's so small. You just plug your card-reader in and you see the lights flashing, but how can you be sure that everything went well during the copying process?
    The image that is created and written back to the drive contains the directory output and can (or should) be viewable in your camera for example. When you see that all files have proper sizes and are complete, you can safely format your SD-Card and don't have to worry until the end of your trip to see if everything REALLY went fine

    Enough talk, here are the scripts, they all assume that your internal hard-drive is mounted to /opt:

    Insert the following lines into /tmp/local/sbin/post-boot:

    Code:
    insmod /lib/modules/2.4.20/kernel/drivers/scsi/scsi_mod.o
    insmod /lib/modules/2.4.20/kernel/drivers/scsi/sd_mod.o
    insmod /lib/modules/2.4.20/kernel/drivers/usb/storage/usb-storage.o
    and

    Code:
    echo /opt/utils/hotplug > /proc/sys/kernel/hotplug
    The first three lines are needed only if you turned off the ftp-server in the web-interface. If your system already automagically mounts your storage device to /tmp/harddisk, you can omit them.

    Create the file /opt/utils/hotplug:

    Code:
    #!/bin/sh
    
    # check for need to mount usb-drive
    
    if [ $ACTION = "add" ]; then
      devs=`find /dev -name "part?"`;
      # e.g. /dev/scsi/host0/bus0/target0/lun3/part1
      for dev in $devs; do
        part=`echo $dev | sed -e "s/.*part\(.*\)/\1/"`;
        # e.g. 1 if part1
        dev=`echo $dev | sed -e "s/.*dev\(.*\)\/part.*/\1/"`;
        # e.g. /scsi/host0/bus0/target0/lun3
        disc=`ls -l /dev/discs | grep $dev | sed -e 's/.* disc\(.*\) ->.*/\1/`;
        # e.g. 4 if $dev maps /dev/discs/disc4
    
        # check if already in mount-tab
        result=`mount | grep "/dev/discs/disc$disc/"`;
        if [ $? = 1 ]; then
          echo "Not mounted: $disc";
          mounted=0;
          while [ $mounted -lt 1 ]; do
            dest="/tmp/harddisk$count";
            result=`mount | grep "$dest"`;
            # if directory exists, increase counter
            if [ $? = 0 ]; then
              let count=count+1;
            else
              mounted=1;
            fi
          done
          mkdir -p "/tmp/harddisk$count";
          mount /dev/discs/disc$disc/part$part /tmp/harddisk$count
        else
          echo "disc$disc already mounted!"
        fi
      done
    fi
    
    # unmount "dead" mountpoints
    
    if [ $ACTION = "remove" ]; then
      devs=`mount | grep /part | sed -e "s/\(.*\) on .*/\1/"`;
      # e.g. /dev/discs/disc4/part1
      for dev in $devs; do
        if [ \! -e $dev ]; then
          dir=`mount | grep $dev | sed -e "s/.*on \(.*\) type.*/\1/"`;
          # e.g. /tmp/harddisk
          umount $dir;
        fi
      done
    fi
    
    /sbin/hotplug $*
    This file checks for devices in the system and mounts them to /tmp/harddisk (or /tmp/harddisk1[2, 3, 4, ...] if already in use), if this hasn't already happened. Use this if your storage device doesn't come up mounted automagically.

    For image-backup, create first /tmp/local/sbin/post-mount:

    Code:
    #!/bin/sh
    
    dirs=`ls /tmp | grep harddisk`;
    for dir in $dirs; do
      if [ -e /tmp/$dir/dcim ]; then
        folder=`date +%Y%m%d_%H%M%S`;
        mkdir -p /opt/backup/$folder;
        cp -ar /tmp/$dir/dcim/* /opt/backup/$folder;
        # this php script requires ipkg-packages gdlib, php, php-gd and cyrus-sasl
        ls -lR /opt/backup/$folder | /opt/utils/reportjpg.php;
        # modify here according to your camera
        cp /opt/backup/result.jpg /tmp/$dir/dcim/100nikon/dscn0001.jpg
        umount /tmp/$dir
      fi
    done
    This checks for a directory called "DCIM" (almost all digital cameras use this), and if so, copy all files to /opt/backup/[timestamp], create a report-image in jpg-format and write it back to the camera as dscn0001.jpg (change this according to your camera model). Then unmount the drive.
    Unfortunately, my Nikon camera doesn't display this image, it says "this image contains no data", but in Windows Explorer it can be seen, so I assume my Nikon checks some other file attributes. If someone has a solution, please let me know.

    To create the image with the directory output, I had to resort to a php-script because I can't code in C to use libgd and libjpeg from there. Would be much faster and more convenient, but that's up to someone else
    You need the following ipkg-packages: gdlib, php, php-gd, cyrus-sasl (because of some stupid dependencies).

    Put this code under /opt/utils/reportjpg.php:
    Code:
    #!/opt/bin/php
    <?
    while (!feof(STDIN)) {
      $input = trim(fgets(STDIN));
      $res[] = $input;
    }
    $im=imagecreate(2048,1536);
    imagecolorallocate($im,0,0,0);
    $col=imagecolorallocate($im,255,255,255);
    $y=0; $x=0;
    foreach ($res as &$line) {
      imagestring($im,1,$x,$y,$line,$col);
      $y=$y+12;
      if ($y > 1500) {
        $x=$x+400;
        $y=0;
      }
    }
    imagejpeg($im,"/opt/backup/result.jpg",100);
    imagedestroy($im);
    ?>
    Don't forget to chmod 755 all files and save configuration with
    flashfs save; flashfs commit; flashfs enable;
    Then reboot to activate hotplug changes.

    Hope you enjoy this piece of code and if someone could come up with a more ressourceful way of writing the jpg-report-image, please let me know !

    CU!

    Frederik.

  2. #2
    Hi Frederik,

    great work, thank you.

    I dont have some experience in scripting (worse luck). I would like copy pictures from some different cameras. This cameras are have different folder names after dcim/. For that there is a problem with the automaticly created content picture. Your script copy it to dcim/100nikon. Is it possible to find out the name(s) from the folder(s) in dcim/ which includ the pictures and where the script can copy the content picture?

    Because the WL-HDD USB 1.1 transmission is very slow it would be nice to have an optical indicator whether the copy process is still running. I'm looking for a script or executable or code snippet which let me controll one of the LED from the WL-HDD ( the USB-LED would be fine) but I don't find some one.
    If one of you have a solution please let me know.

    Thanks again (also to Oleg)
    Juergen

  3. #3
    Quote Originally Posted by juergen View Post
    Hi Frederik,
    great work, thank you.
    Thanks.

    Quote Originally Posted by juergen View Post
    Is it possible to find out the name(s) from the folder(s) in dcim/ which includ the pictures and where the script can copy the content picture?
    Unfortunately, not really. My Nikon for example has at least two directories, 100nikon and misc. If I take panorama pictures, they will be put in 100p1nikon (or something like that). Other cameras have probably similar multitudes of folder names, so it would really blow up the script.
    You could do something semi-automatic like defining a variable at the beginning which is set according to the dcim contents.
    I just write it from memory, so no guarantee it'll work:

    Code:
    #!/bin/sh
    
    dirs=`ls /tmp | grep harddisk`;
    for dir in $dirs; do
      if [ -e /tmp/$dir/dcim ]; then
        folder=`date +%Y%m%d_%H%M%S`;
        mkdir -p /opt/backup/$folder;
        if [ -e /tmp/$dir/dcim/100nikon ]; then
          destination = "100nikon";
          filename = "dscn0001.jpg";
        fi
        if [ -e /tmp/$dir/dcim/canon ]; then
          destination = "100canon";
          filename = "p00001.jpg";
        fi
        cp -ar /tmp/$dir/dcim/* /opt/backup/$folder;
        # this php script requires ipkg-packages gdlib, php, php-gd and cyrus-sasl
        ls -lR /opt/backup/$folder | /opt/utils/reportjpg.php;
        # modify here according to your camera
        cp /opt/backup/result.jpg /tmp/$dir/dcim/$destination/$filename
        umount /tmp/$dir
      fi
    done
    Please give feedback here if it works as desired.

    Quote Originally Posted by juergen View Post
    Because the WL-HDD USB 1.1 transmission is very slow it would be nice to have an optical indicator whether the copy process is still running. I'm looking for a script or executable or code snippet which let me controll one of the LED from the WL-HDD ( the USB-LED would be fine) but I don't find some one.
    If one of you have a solution please let me know.
    I'd just use a card-reader with a built-in LED (mine cost less than 5 Euros). If it stops flashing, I can plug it off.

    But you could also do something like that in the script:
    Line 3:
    Turn off Power-LED: echo 1 > /dev/gpio/out
    Last line:
    Turn on Power-LED: echo 0 > /dev/gpio/out

    So you'll know if the Power-LED is back on again, the process is finished.

    CU!

    Frederik.

  4. #4
    Hi Frederik,

    it works fine.

    I change /tmp/local/sbin/post-mount as following:

    Code:
    #!/bin/sh
    #Auto-copy images from USB-device and index-image creation
    #Found in thread http://wl500g.info/showthread.php?t=6826
    dirs=`ls /tmp | grep harddisk`;
    for dir in $dirs; do
      if [ -e /tmp/$dir/dcim ]; then
    
        #Start LED flashing
        /opt/utils/WL-HDDPowerLedFlash.sh&
        folder=`date +%Y%m%d_%H%M%S`;
        mkdir -p /opt/backup/$folder;
        # for my Nikon
        if [ -e /tmp/$dir/dcim/100nikon ]; then
          destination="100nikon";
          filename="dscn0001.jpg";
        fi
        # for my Sony
        if [ -e /tmp/$dir/dcim/101msdcf ]; then
          destination="101msdcf";
          filename="dscn0001.jpg";
        fi
        cp -ar /tmp/$dir/dcim/* /opt/backup/$folder;
        # this php script requires ipkg-packages gdlib, php, php-gd and cyrus-sasl
        ls -lR /opt/backup/$folder | /opt/utils/reportjpg.php;
        # modify here according to your camera
        cp /opt/backup/result.jpg /tmp/$dir/dcim/$destination/$filename
        umount /tmp/$dir
    
        #Stop LED flashing
        killall WL-HDDPowerLedFlash.sh
      fi
    done
    and create a little script /opt/utils/WL-HDDPowerLedFlash.sh for LED flashing:

    Code:
    #!/bin/sh
    
    # Signal-Handle
    cleanup()
    {
     #switch on PWR LED
     echo 0 > /dev/gpio/out;
     exit
    }
    
    # Signal-Handle "cleanup" link with EXIT TERM and INT
    trap cleanup EXIT TERM INT
    
    while :
    do
    
      #switch off PWR LED
      echo 1 > /dev/gpio/out;
      sleep 1
    
      #switch on PWR LED
      echo 0 > /dev/gpio/out;
      sleep 1
    
    done
    This lets the Power-LED flash during the copy process.


    I know that a shell script is very nice but I don't like it:

    destination="100nikon"; are not
    destination = "100nikon";


    Thanks a lot
    Juergen

    Don't forget to chmod 755 all files and save configuration with
    flashfs save; flashfs commit; flashfs enable;
    Last edited by juergen; 31-10-2006 at 01:11.

  5. #5
    Quote Originally Posted by juergen View Post
    create a little script /opt/utils/WL-HDDPowerLedFlash.sh for LED flashing:
    This lets the Power-LED flash during the copy process.
    Nice, but how come you can write a script like that if you -as you say - don't have experience in scripting ?

    In fact it seems quite necessesary to me now to have some optical signal because the process to create the result.jpg takes quite some time and then the USB-Stick's LED stops flashing, so it's not a really safe indicator (although one could wait until it starts flashing again shortly when the jpg is written to the SD-Card and then unplug after flashing finally stops).

    Quote Originally Posted by juergen View Post
    I know that a shell script is very nice but I don't like it:
    destination="100nikon"; are not
    destination = "100nikon";
    Well, as I said, I wrote it from memory without testing it. I also realized it when putting it into action back home. Sorry.

    Frederik.

  6. #6

    USB hotplug -script vs binary

    Interesting mails, but some problems to further test and evaluate this.
    - The ipkg packages as referred do not seem to be available (they are not in ipkg list ). Neither are /opt/utils/....

    - I would like to manually 'start' the copy script by shortly pressing the (only) button on the wl-hdd (more than half a second, less than 3 seconds ...).
    There seems to be an /dev/gpio/in which changes its output in relation to that button.
    But I'm not at all comfortable with writing a script around that button.

    - I see there's a /sbin/hotplug which is in binary format. It has no man page, and no -h or -? option. So I'm absolutely not sure if e.g. you/Oleg decided to support the 'hotplug' in the firmware distribution ??

    Gerard.

  7. #7
    Unfortunately, my Nikon camera doesn't display this image, it says "this image contains no data", but in Windows Explorer it can be seen, so I assume my Nikon checks some other file attributes. If someone has a solution, please let me know.
    I have a Nikon too, with the same problem. After some experimenting, I found that the problem is the jpeg sampling.
    Libjpeg produces 2x2,1x1,1x1 samples by default. The Nikon reads only 2x1,1x1,1x1 pictures. Although libjpeg has a function to change the sampling, gdlib has no way to adjust this !

    So I patched the libjpeg source and changed the default to 2x1....!
    Then compiled libjpeg and exchanged it on my WL-HDD

    And now my Nikon shows the picture in full screen mode!

    Cheers Martin

    You have to edit the file jcparam.c line 437 and line 452 from

    Code:
    SET_COMP(0, 1, 2,2, 0, 0,0);
    to
    SET_COMP(0, 1, 2,1, 0, 0,0);

  8. #8
    Hi Martin,

    Quote Originally Posted by martin700 View Post
    So I patched the libjpeg source and changed the default to 2x1....!
    Then compiled libjpeg and exchanged it on my WL-HDD
    thanks for the info, this really sounds great. Do you think you could give a step-by-step instruction on how to do this directly on the WL-HDD so one doesn't have to do a cross-compiling on another machine?

    Or simply upload the patched version (or a .diff-file) so others can just apply that?

    Would help a lot : )!

    Thanks,

    Frederik.

  9. #9
    Quote Originally Posted by fredlcore View Post
    Hi Martin,



    thanks for the info, this really sounds great. Do you think you could give a step-by-step instruction on how to do this directly on the WL-HDD so one doesn't have to do a cross-compiling on another machine?

    Or simply upload the patched version (or a .diff-file) so others can just apply that?

    Would help a lot : )!

    Thanks,

    Frederik.
    The simplest way to do it, is to exchange the library. Of course mine is compiled for Olegs firmware and I doubt that it works in any other setting. You can download it here and exchange it with yours in :
    /opt/lib/. (A backup of the original would be a good idea)

    cheers Martin
    Attached Files Attached Files

  10. #10
    Quote Originally Posted by martin700 View Post
    The simplest way to do it, is to exchange the library. Of course mine is compiled for Olegs firmware and I doubt that it works in any other setting. You can download it here and exchange it with yours in :
    /opt/lib/. (A backup of the original would be a good idea)
    Thanks for the file, but although I also have Oleg's firmware (1.9.2.7-7f) on my WL-HDD, it gives me the following errors:

    Code:
    PHP Warning:  PHP Startup: Unable to load dynamic library '/opt/lib/php/extensions/gd.so' - File not found in Unknown on line 0
    PHP Fatal error:  Call to undefined function imagecreate() in /opt/utils/reportjpg.php on line 7
    When I copy back the original libjpeg.so.62.0.0, all runs fine...
    Do you use a different hardware and/or firmware?

    Any ideas?

    Frederik.

  11. #11

    Thumbs up Your usefull 'autocopy' script

    Hi and thanks for your script, however, before I use it there are a few things I would like to know...

    Would it be possible to copy ANY and ALL files from a usb disc (be it memory card or USB stick) as I have a memory card for my mobile phone, and it uses different directories to store files, and also I would quite like it to backup from any USB card / stick I insert.

    If this could be done, you would be my saviour and hero

  12. #12
    Join Date
    Sep 2007
    Location
    Hannover
    Posts
    88
    I have a problem getting this up and running
    Can't store the hook....
    Code:
    echo /opt/utils/hotplug > /proc/sys/kernel/hotplug
    So if i execute that line it looks fine.
    But if i take a look at it with vi
    Code:
    vi /proc/sys/kernel/hotplug
    the file is still empty. I can edit it according to the command above - but after the next reboot it's gone.
    And yes - I used flashfs save && flashfs commit && flashfs enable ...


    Any ideas?

    - Tobias -

  13. #13
    Join Date
    Apr 2006
    Location
    Heesch, Netherlands
    Posts
    118
    Quote Originally Posted by tobitobsen View Post
    I have a problem getting this up and running
    Can't store the hook....
    Code:
    echo /opt/utils/hotplug > /proc/sys/kernel/hotplug
    So if i execute that line it looks fine.
    But if i take a look at it with vi
    Code:
    vi /proc/sys/kernel/hotplug
    the file is still empty. I can edit it according to the command above - but after the next reboot it's gone.
    And yes - I used flashfs save && flashfs commit && flashfs enable ...


    Any ideas?

    - Tobias -
    That's why you should add this to post-boot.
    Solar inverter monitoring with Asus wl500gx http://solar.reinieren.net (dutch)

  14. #14
    Join Date
    Sep 2007
    Location
    Hannover
    Posts
    88
    ok - so the router would try to copy the files on reboot and not on plug-in?

    I tried without the hotplug-script since the rooter does some 'automount' itself. But the even if I write the copy script into !!post-mount!! it only runs once until next reboot.

    So no real "plug and copy" for both ways?

  15. #15
    Quote Originally Posted by tobitobsen View Post
    ok - so the router would try to copy the files on reboot and not on plug-in?
    No, you need to put the command
    echo /opt/utils/hotplug > /proc/sys/kernel/hotplug
    in your /tmp/local/sbin/post-boot (as it is written above) and then save the flash with flashfs save; flashfs commit; flashfs enable;
    Then reboot to activate hotplug changes.
    After that, you should be able to see the command in /proc/sys/kernel/hotplug and the script should run when a device is plugged in.

    F.

Page 1 of 2 12 LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •