Bash script to browse directory with a custom output ?

[raf_iso]

Hello,

I need to browse a storage disk with almost 700GB of datas. There is a lot of directories.

I would like to make a script that browse all the structure and store it in a text file each night. Then I could make a little search program and it'll be much more faster than searching with the FIND command. Even store it in a mysql database.

I looked the docs about LS and TREE, but I can't get exactly what I want .

I need something like

[YYYY.MM.DD-HH:MM][SIZE][FULLPATH]

for exemple :

[2006.01.11-09:05][4096][./custom/]
[2005.12.30-18:15][23579][./custom/bruit_graph.png]

With this format it's easy to sort by last modification.

- TREE is great, but sometimes dates aren't well displayed.
- LS is nice too, but I can't find a parameter to display the fullpath for each files on one line like TREE with the recursive option.

Please can you help me ?

Thanks you for your help !

RB

[daverj]

This will do what you want. Mind you it will be slow for a 700GB filesystem, but if you are doing it overnight, you might have enough time.

#!/usr/bin/bash

DIR=/tmp
FILE=out.file

# move most current output to backup file
if [ -e ${DIR}/${FILE} ]; then
mv -f ${DIR}/${FILE}.bak
fi

# iterate over filesystem, extract desired info and format
# output to file
find . | while read line; do
m=`stat --format=%y ${line} | cut -d"." -f1`
s=`stat --format=%s ${line}`
echo "[$m][$s][${line}]" >> ${DIR}/${FILE}
done

[raf_iso]

WOW nice script !

I'm playing with it actually. I have a problem with filename and directory with spaces.

For example when the script found "fichier 1.txt" :

[raf@st-107168 tests]$ ./runme.sh
stat: cannot stat `./racine/fichier': No such file or directory
stat: cannot stat `1.txt': No such file or directory

Do you have something magic to resolve this ? I have also french accents and things like that, it's not very important if these non-us chars become "?"...

Oh.. and if there is something very easy to know the time ellapsed for the script it would be perfect !!! Something like [<timestamp at the end> - <timestamp at the begin>].

Thanks you !!!
RB

[raf_iso]

a friend told me to try

find /your/path | xargs stat -c '[%y][%s][%n]'

and it's also working fine.

[ibbo]

mv "$file" "`echo $file | sed -e 's/\ /_/g'`";

where $file is something along the lines of

for file in '/home/mydir'
do
mv "$file" "`echo $file | sed -e 's/\ /_/g'`";
# more batch processing as laid out above
done



Will replace a whitespace charector in your filename with a "_".
Dont know if that is any good but it comes in handy for me when dealing with images from a MAC.

Also as noted 700GB is pretty big and will take some serious effort from your machine to churn through it all. I recomend trying updatedb and locate as a pair of tools that can index and find your files.

Ibbo

[raf_iso]

daverj, your code is working, but the following code is really easier. Anyway, I can't run it in a directory with long filenames, directores with spaces and stuff like that.

[root@fileserver _array]# find . | xargs stat -c '[%y][%s][%n]'
xargs: unmatched single quote; by default quotes are special to xargs unless you use the -0 option
[root@fileserver _array]#

I tried to play with the xags -0 but I can't run it. May be do you know a tip or something for this problem ?

Thanks again!

[KSLinuxUser]

Have you tried the locate command? If you just want to search by file name it is very handy. I don't know if it supports search by date.

[daverj]

raf_iso, two things:

First, the reason I did the script my particular way is because of the way you indicated how you wanted the output formatted. The standard output for stat --format=%y does not conform to the output you desired.
Second, you can easily modify my script to allow for spaces in filenames. Simply add double quotation marks around the filename variable in the stat command:

#!/usr/bin/bash

DIR=/tmp
FILE=out.file

# move most current output to backup file
if [ -e ${DIR}/${FILE} ]; then
mv -f ${DIR}/${FILE}.bak
fi

# iterate over filesystem, extract desired info and format
# output to file
find . | while read line; do
m=`stat --format=%y "${line}" | cut -d"." -f1`
s=`stat --format=%s "${line}"`
echo "[$m][$s][${line}]" >> ${DIR}/${FILE}
done

hope this helps

[raf_iso]

Okay thanks you !! I'll try it tonight

see ya!