PDA

View Full Version : How to iterate over files in a bash script?


Dunhausen
20th September 2007, 02:01 PM
I have been messing around with ticks, parenthesis, brackets etc. forever, and didn't seem to find anything useful on google... but I seem to be having a problem with the fact that some of my files have spaces in the names. Originally, this script was working for files without spaces int the name, but it appeared to view a space as a deliminator when I was trying to iterate.

Surely, though, people write bash scripts like this in function... in fact, this started from a fully functional script I have for making thumbnails for a series of image files. What is different?

#!/bin/bash

for FILENAME in "`find . -name "*.mp3" -print`"
do

ARTIST="`exiftool -Artist "$FILENAME" -p '$Artist'`"
TITLE="`exiftool -Title "$FILENAME" -p '$Title'`"
ALBUM="`exiftool -Album "$FILENAME" -p '$Album'`"
YEAR="`exiftool -Year "$FILENAME" -p '$Year'`"
DIRECTORY="`exiftool -Directory "$FILENAME" -p '$Directory'`"

ruby /home/pimeson/.kde/share/apps/amarok/scripts/wiki_lyrics/cli/wikilyrics.rb -a $ARTIST -t $TITLE -l $ALBUM -y $YEAR --sites cs,P3,yan,o,rics,iki,Download,Mania,ular,rics,5,et ras > "$DIRECTORY/$TITLE.txt"

done

Right now I test it and get an error like
Unsuccessful stat on filename containing newline at /usr/bin/exiftool line 625.
Unsuccessful stat on filename containing newline at /usr/bin/exiftool line 742.
File not found: ./Styx/Styx - Mr. Roboto.mp3

When I tell it to "echo $FILENAME" I get, for example.
./Styx/Styx - Mr. Roboto.mp3 ./Styx/StyxMrRoboto.mp3

Any ideas? :)

sideways
20th September 2007, 02:29 PM
try using -print0 instead of -print in the find command

sideways
20th September 2007, 03:19 PM
actually, -print0 isn't gonna work here, I was thinking of when you pipe the output to 'xargs -0'.

Maybe run the find command first and then pipe the output to a while loop using the read command, then put your code inside the while loop
eg
find . -name "*.mp3" | while read FILENAME; do echo $FILENAME; done

BostonWatcher
20th September 2007, 03:28 PM
sideways,

This solution does not work with filenames containing spaces, unless you replace echo $FILENAME by echo "$FILENAME"

ausadl
20th September 2007, 03:41 PM
try this:

for file in *.mp3
do

echo "file=$file"

done

BostonWatcher
20th September 2007, 03:43 PM
I say that your script should work with filenames containing spaces. Now,
as weird as it may sound, is there any possibility that this specific filename includes a newline character? Do you have this problem for ALL filenames with spaces, or just for SOME?

It may also be related to some 'locale' or encoding issues, mixing character codes.

RupertPupkin
20th September 2007, 08:15 PM
I say that your script should work with filenames containing spaces. Now,
as weird as it may sound, is there any possibility that this specific filename includes a newline character? Do you have this problem for ALL filenames with spaces, or just for SOME?

It may also be related to some 'locale' or encoding issues, mixing character codes.
No, it has nothing to do with that.

There are two problems with his script. First, enclosing the output of the find command in double quotes will cause the for loop to treat that output as one big concatenated string. That's why when he does "echo $FILENAME" he gets both filenames treated as one, instead of two separate filenames.

The other problem is that the default internal field separator (IFS) is any whitespace (including spaces), so if he removed the double quotes around his find command he'd get the filenames split by spaces, if they contain any. Setting the built-in bash IFS variable to be a newline character will ensure that filenames in the find output will only be split on newlines:
#!/bin/bash
IFS="$(echo -e "\n\r")"
for FILENAME in $(find . -name "*.mp3")
do
ARTIST="`exiftool -Artist "$FILENAME" -p '$Artist'`"
TITLE="`exiftool -Title "$FILENAME" -p '$Title'`"
ALBUM="`exiftool -Album "$FILENAME" -p '$Album'`"
YEAR="`exiftool -Year "$FILENAME" -p '$Year'`"
DIRECTORY="`exiftool -Directory "$FILENAME" -p '$Directory'`"

ruby /home/pimeson/.kde/share/apps/amarok/scripts/wiki_lyrics/cli/wikilyrics.rb -a $ARTIST -t $TITLE -l $ALBUM -y $YEAR --sites cs,P3,yan,o,rics,iki,Download,Mania,ular,rics,5,et ras > "$DIRECTORY/$TITLE.txt"

done

Dunhausen
20th September 2007, 08:35 PM
Thanks a lot guys--with those changes this is working fine now:

#!/bin/bash

find . -name "*.mp3" | while read FILENAME;
do

ARTIST="`exiftool -Artist "$FILENAME" -p '$Artist'`"
TITLE="`exiftool -Title "$FILENAME" -p '$Title'`"
ALBUM="`exiftool -Album "$FILENAME" -p '$Album'`"
YEAR="`exiftool -Year "$FILENAME" -p '$Year'`"
DIRECTORY="`exiftool -Directory "$FILENAME" -p '$Directory'`"

ruby /home/pimeson/.kde/share/apps/amarok/scripts/wiki_lyrics/cli/wikilyrics.rb -a $ARTIST -t $TITLE -l $ALBUM -y $YEAR --sites cs,P3,yan,o,rics,iki,Download,Mania,ular,rics,5,et ras > "$DIRECTORY/$TITLE.txt"

done

BostonWatcher, I agree my script should, to all appearances, have been working, but I tried it with other files too and had the same problem.

Dunhausen
20th September 2007, 08:38 PM
Thanks a lot RupertPupkin--as nice as it is to have things working, it is even nicer to know why the are/weren't. :)

I'll have to remember your solution/explanation.

BostonWatcher
20th September 2007, 08:58 PM
OK, I was mislead by the find construct.

In fact I would not iterate over the output of find in this simple query case. I would rather do

for file in *mp3
do
whatever "$file"
done

which is much simpler, but I would make sure I use "$file" inside the do...done, instead of $file. With this, the filename is treated correctly, and does not require redefining the IFS.

Since your original script had the filename protected with quotes, I was mislead, and did not pay attention to the problem with the find construct that RupertPupkin pointed out.

scm
22nd September 2007, 08:18 PM
It's A Really Daft Idea to put spaces in filenames. ;)

Dunhausen
22nd September 2007, 08:45 PM
It actually makes sense in audio collections in as much as you expect the name of the file to match exactly the name of title of the song, etc. (which makes tagging, looking up files in a CDDB, etc., a lot easier).

Hmm... what about introducing a special character--"space in filenames"--that would look like a space... but not be one, or at least, not be the same unicode value. :p

DownloadLinux
11th July 2011, 07:48 AM
A space might need to be preceded by an escape character "\ " when interacting with the shell.

./Styx/Styx\ -\ Mr\.\ Roboto.mp3 ./Styx/StyxMrRoboto.mp3


or place double quotes around the variable.