<---- template headericclude ----->
How to iterate over files in a bash script?
FedoraForum.org - Fedora Support Forums and Community
Results 1 to 13 of 13
  1. #1
    Dunhausen Guest

    How to iterate over files in a bash script?

    I have been messing around with ticks, parenthesis, brackets etc. forever, and didn't seem to find anything useful on google... but I seem to be having a problem with the fact that some of my files have spaces in the names. Originally, this script was working for files without spaces int the name, but it appeared to view a space as a deliminator when I was trying to iterate.

    Surely, though, people write bash scripts like this in function... in fact, this started from a fully functional script I have for making thumbnails for a series of image files. What is different?

    Code:
    #!/bin/bash
    
    for FILENAME in "`find . -name "*.mp3" -print`"
    do
    
    ARTIST="`exiftool  -Artist "$FILENAME" -p '$Artist'`"
    TITLE="`exiftool  -Title "$FILENAME" -p '$Title'`"
    ALBUM="`exiftool  -Album "$FILENAME" -p '$Album'`"
    YEAR="`exiftool  -Year "$FILENAME"  -p '$Year'`"
    DIRECTORY="`exiftool  -Directory "$FILENAME"  -p '$Directory'`"
    
    ruby /home/pimeson/.kde/share/apps/amarok/scripts/wiki_lyrics/cli/wikilyrics.rb -a $ARTIST -t $TITLE -l $ALBUM -y $YEAR --sites cs,P3,yan,o,rics,iki,Download,Mania,ular,rics,5,etras > "$DIRECTORY/$TITLE.txt"
    
    done
    Right now I test it and get an error like
    Code:
    Unsuccessful stat on filename containing newline at /usr/bin/exiftool line 625.
    Unsuccessful stat on filename containing newline at /usr/bin/exiftool line 742.
    File not found: ./Styx/Styx - Mr. Roboto.mp3
    When I tell it to "echo $FILENAME" I get, for example.
    Code:
    ./Styx/Styx - Mr. Roboto.mp3 ./Styx/StyxMrRoboto.mp3
    Any ideas?

  2. #2
    Join Date
    Oct 2004
    Location
    London, UK
    Posts
    4,991
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    try using -print0 instead of -print in the find command

  3. #3
    Join Date
    Oct 2004
    Location
    London, UK
    Posts
    4,991
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    actually, -print0 isn't gonna work here, I was thinking of when you pipe the output to 'xargs -0'.

    Maybe run the find command first and then pipe the output to a while loop using the read command, then put your code inside the while loop
    eg
    Code:
    find . -name "*.mp3"  | while read FILENAME; do echo $FILENAME; done

  4. #4
    Join Date
    Mar 2005
    Posts
    73
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    sideways,

    This solution does not work with filenames containing spaces, unless you replace
    Code:
    echo $FILENAME
    by
    Code:
    echo "$FILENAME"

  5. #5
    Join Date
    Feb 2006
    Location
    Modbury, Australia
    Posts
    99
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    try this:

    for file in *.mp3
    do

    echo "file=$file"

    done

  6. #6
    Join Date
    Mar 2005
    Posts
    73
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    I say that your script should work with filenames containing spaces. Now,
    as weird as it may sound, is there any possibility that this specific filename includes a newline character? Do you have this problem for ALL filenames with spaces, or just for SOME?

    It may also be related to some 'locale' or encoding issues, mixing character codes.

  7. #7
    Join Date
    Nov 2006
    Location
    Detroit
    Posts
    8,726
    Mentioned
    62 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by BostonWatcher
    I say that your script should work with filenames containing spaces. Now,
    as weird as it may sound, is there any possibility that this specific filename includes a newline character? Do you have this problem for ALL filenames with spaces, or just for SOME?

    It may also be related to some 'locale' or encoding issues, mixing character codes.
    No, it has nothing to do with that.

    There are two problems with his script. First, enclosing the output of the find command in double quotes will cause the for loop to treat that output as one big concatenated string. That's why when he does "echo $FILENAME" he gets both filenames treated as one, instead of two separate filenames.

    The other problem is that the default internal field separator (IFS) is any whitespace (including spaces), so if he removed the double quotes around his find command he'd get the filenames split by spaces, if they contain any. Setting the built-in bash IFS variable to be a newline character will ensure that filenames in the find output will only be split on newlines:
    Code:
    #!/bin/bash
    IFS="$(echo -e "\n\r")"
    for FILENAME in $(find . -name "*.mp3")
    do
       ARTIST="`exiftool  -Artist "$FILENAME" -p '$Artist'`"
       TITLE="`exiftool  -Title "$FILENAME" -p '$Title'`"
       ALBUM="`exiftool  -Album "$FILENAME" -p '$Album'`"
       YEAR="`exiftool  -Year "$FILENAME"  -p '$Year'`"
       DIRECTORY="`exiftool  -Directory "$FILENAME"  -p '$Directory'`"
    
       ruby /home/pimeson/.kde/share/apps/amarok/scripts/wiki_lyrics/cli/wikilyrics.rb -a $ARTIST -t $TITLE -l $ALBUM -y $YEAR --sites cs,P3,yan,o,rics,iki,Download,Mania,ular,rics,5,et  ras > "$DIRECTORY/$TITLE.txt"
    
    done

  8. #8
    Dunhausen Guest
    Thanks a lot guys--with those changes this is working fine now:

    Code:
    #!/bin/bash
    
    find . -name "*.mp3"  | while read FILENAME;
    do
    
    ARTIST="`exiftool  -Artist "$FILENAME" -p '$Artist'`"
    TITLE="`exiftool  -Title "$FILENAME" -p '$Title'`"
    ALBUM="`exiftool  -Album "$FILENAME" -p '$Album'`"
    YEAR="`exiftool  -Year "$FILENAME"  -p '$Year'`"
    DIRECTORY="`exiftool  -Directory "$FILENAME"  -p '$Directory'`"
    
    ruby /home/pimeson/.kde/share/apps/amarok/scripts/wiki_lyrics/cli/wikilyrics.rb -a $ARTIST -t $TITLE -l $ALBUM -y $YEAR --sites cs,P3,yan,o,rics,iki,Download,Mania,ular,rics,5,etras > "$DIRECTORY/$TITLE.txt"
    
    done
    BostonWatcher, I agree my script should, to all appearances, have been working, but I tried it with other files too and had the same problem.

  9. #9
    Dunhausen Guest
    Thanks a lot RupertPupkin--as nice as it is to have things working, it is even nicer to know why the are/weren't.

    I'll have to remember your solution/explanation.

  10. #10
    Join Date
    Mar 2005
    Posts
    73
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    OK, I was mislead by the find construct.

    In fact I would not iterate over the output of find in this simple query case. I would rather do
    Code:
    for file in *mp3
    do 
      whatever "$file" 
    done
    which is much simpler, but I would make sure I use "$file" inside the do...done, instead of $file. With this, the filename is treated correctly, and does not require redefining the IFS.

    Since your original script had the filename protected with quotes, I was mislead, and did not pay attention to the problem with the find construct that RupertPupkin pointed out.

  11. #11
    scm Guest
    It's A Really Daft Idea to put spaces in filenames.

  12. #12
    Dunhausen Guest
    It actually makes sense in audio collections in as much as you expect the name of the file to match exactly the name of title of the song, etc. (which makes tagging, looking up files in a CDDB, etc., a lot easier).

    Hmm... what about introducing a special character--"space in filenames"--that would look like a space... but not be one, or at least, not be the same unicode value.

  13. #13
    Join Date
    Jul 2011
    Posts
    1
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Re: How to iterate over files in a bash script?

    A space might need to be preceded by an escape character "\ " when interacting with the shell.

    ./Styx/Styx\ -\ Mr\.\ Roboto.mp3 ./Styx/StyxMrRoboto.mp3


    or place double quotes around the variable.

Similar Threads

  1. bash script to move files
    By Johnny Fist in forum Using Fedora
    Replies: 3
    Last Post: 9th May 2011, 09:51 AM
  2. Bash Script Help
    By Jamwa in forum Using Fedora
    Replies: 5
    Last Post: 17th June 2008, 05:26 PM
  3. Convert bash script to perl script
    By homey in forum Programming & Packaging
    Replies: 1
    Last Post: 2nd September 2006, 04:24 AM

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
[[template footer(Guest)]]