Bash script - File search output to new file

[ampapa]

I wanted to try and learn a little bit of Bash scripting and have the need to open an exisiting file, loop through it's contents, find matching data, then save the matching data to a new file.

Does someone have anything that I can look that might be useful for this process?

Thanks.

ampapa,

[stevea]

Typically this sort of line selection is performed with a tool like grep
$ grep pattern infile >outfile

Is a regular expression match is not sufficient, then consider using 'awk' or 'sed'.

It might also help if you explain the ENTIRE task you are trying to perform. I suspect that ther ear esome steps after the line selection, and often you must think about the whole picture, rather than the step-by-step sequential procedure.

*IF* you really want to process each line sequentially you could try using the 'read' funciton like

<<
#!/bin/bash
( while read line
do
echo "LINE=$line"
done
) < infile
>>
The parens make a sub-shell which takes 'infile' as it's input stream.
While tests the result of "read line" and loops while input lines exist.
Inside the do ... done you can place as many tests as you want to
quality the string $line. In your case you might want to test line
and if it passes, append it to outfile
if [ ... ] ; then echo $line >>outfile ; fi

[ampapa]

Thanks for the reply.

How do these actions then apply to a file?

cat $file |grep -n 'House 12' > temp/tempA

Does the above only find a single line in the file or will it find 'all' lines with the matching pattern?

sed -e 's/://g' temp/tempA > temp/tempB

This appears to be more of a find replace function? correct?

Assuming I wanted to do a find on the following and save to a new file, 'House 120'?

<option value="15">House 12256</option>
<option value="16">House 1203</option>
<option value="17">House 1254</option>
<option value="18">House 1205</option>
<option value="19">House 126</option>
<option value="20">House 12507</option>
<option value="21">House 1208</option>
<option value="22">House 1209</option>
<option value="23">House 120</option>
<option value="24">House 1211</option>
<option value="25">House 122</option>

Would it look like this?

#!/bin/bash
( while read line
do
if cat $line |grep -n 'House 12' ; then echo $line >>temp/tempA ; fi
done
) < temp/original

[pdb]

What output would you expect from your example above?

[ampapa]

Doesn't the $line mean it will output the entire line to 'temp/tempA'?

I'm really interested in the lines that would match "House 12" only then stripping the other data from the line but I was starting slowly to get a feel for the actual commands... actually "House 120" might be a better search string as it would select a few rows as opposed to all the rows.

thx,

[pdb]

You are correct. This can all be done with just one call to grep.

grep -o 'House 12' temp/original > temp/tempA

The -o option tells it to print only the matching part of the line. Without this it will print the entire line. If you want to show 'House 120' 'House 121' etc, you can just use some wildcards in your regular expression. (for example use 'House 12[:digit:]*'). Check out the man page....

[ampapa]

Thanks.

So, I see this as a final....

#!/bin/bash
( while read line
do
if cat $line |grep -n 'House 120[:digit:]*' ; then echo $line >>temp/tempA ; fi
done
) < temp/original


Would that match 'House 120' and 'House 1209' or only 'House 1209'?

[pdb]

The '*' wildcard will match 0 or more occurrences of the previous item. so 'House 120[:digit:]*' would match any of these:

House 120
House 1209
House 12083
House 120462876364

You get the idea. If you want something different, see the man page. The sky's the limit when it comes to regular expressions.

I still say there's no reason to do it so complicated. A single call to grep will do it.

#! /bin/bash
grep 'House 120[:digit:]*' <temp/original >>temp/tempA

That will allow grep to do internal stuff to do it faster than feeding it in line-by-line.

[ampapa]

I'm used to VB programming and loops... hence the do while...

So, grep will essentially do a search on the file contents and output to a new file in a single call... so sweet, no wonder Linux is growing on me!!!

Thanks again for the input.

[pdb]

man pages are your friend

Linux has some great text manipulation utilities on the command line. There are a lot of pages on the 'net that can help you get started with them. Some important ones: head, tail, grep, sed, sort.

See a good overall reference for linux tools at
http://tldp.org/LDP/GNU-Linux-Tools-...html/book1.htm

This is probably my favorite thing about linux: the ability to put together simple commands to do pretty much anything you want.

[stevea]

If you want to select these lines (above) with a LOT more specificity use a regular expression like this:

grep -n <infile "^<option value=\"[12][0-9]\">House [12][0-9]*</option>$"

Regular expressions are powerful and it *IS* worth learning how to create them, as the same RE syntax is used over a large number of Unix/Linux tools.

[homey]

Maybe something like this...

sed -n '/House 120/p' file.txt >new_file.txt

[stevea]

Or more specifically ....

 sed   -n   "/^<option value=\"[12][0-9]\">House [12][0-9]*<\/option>/p"    file.txt