Scripting help

[boknoy]

Hi, I have a logfile as shown below. What I want is to be able to parse and concatenate all the text starting right after "-Send to card 1 -----------------------------" and concatenate all the space delimited values into a single string for each occurance in the logfile. I'm thinking of using awk but if you can suggest any other bash/sh scripts that will do the job just to help me start out with the script.

Eventually, the output I want would be a textfile like this:

Output file contents

210053A0CA00004D004B4E016A9F0C01C8A0596ECE3F3B5230 74EC36C156F5404488E8E628BFF76113F8FD191BEE22E86FF0 B5DD269E02C51D0E8ACAC0E111854E90F3E0A22C1E1824E79A D40F89A0FFDC3958476F05
210053A0CA00004D004B4E016A08EDD5EF026F2015C02EE8E2 39E889481D1BB6D3C5DB8C8ABE9B61CF7F2D00707CD8A3CE64 16E7BDA3CE68EE1C730A9FF26F32E6C32E63A23550EA2A24E5 5259C93215432F589FA605



Input file contents:

===== Xxxxx YYY on CaID 0x1801, pid 0x0258 ====
SysID 9501, Parity Key select 02, Key type 01
-YYY-------------------------------------------
82 70 58 00 00 00 00 00 00 4B 4E 01 6A 9F 0C 01
C8 A0 59 6E CE 3F 3B 52 30 74 EC 36 C1 56 F5 40
44 88 E8 E6 28 BF F7 61 13 F8 FD 19 1B EE 22 E8
6F F0 B5 DD 26 9E 02 C5 1D 0E 8A CA C0 E1 11 85
4E 90 F3 E0 A2 2C 1E 18 24 E7 9A D4 0F 89 A0 EF
DC 39 58 47 6F AA 82 A4 00 00 00
-Send to card 1 -----------------------------
21 00 53 A0 CA 00 00 4D 00 4B 4E 01 6A 9F 0C 01
C8 A0 59 6E CE 3F 3B 52 30 74 EC 36 C1 56 F5 40
44 88 E8 E6 28 BF F7 61 13 F8 FD 19 1B EE 22 E8
6F F0 B5 DD 26 9E 02 C5 1D 0E 8A CA C0 E1 11 85
4E 90 F3 E0 A2 2C 1E 18 24 E7 9A D4 0F 89 A0 EF
DC 39 58 47 6F 05
Xxxxx YYY response: 12 00 07 80 03 B1 01 58 90 00 EE
===== Xxxxx YYY on CaID 0x1801, pid 0x0258 ====
SysID 9501, Parity Key select 02, Key type 01
-YYY-------------------------------------------
82 70 58 00 00 00 00 00 00 4B 4E 01 6A 08 ED D5
EF 02 6F 20 15 C0 2E E8 E2 39 E8 89 48 1D 1B B6
D3 C5 DB 8C 8A BE 9B 61 CF 7F 2D 00 70 7C D8 A3
CE 64 16 E7 BD A3 CE 68 EE 1C 73 0A 9F F2 6F 32
E6 C3 2E 63 A2 35 50 EA 2A 24 E3 52 59 C9 32 15
43 2F 58 9F A6 72 E7 9D 00 00 00
-Send to card 1 -----------------------------
21 00 53 A0 CA 00 00 4D 00 4B 4E 01 6A 08 ED D5
EF 02 6F 20 15 C0 2E E8 E2 39 E8 89 48 1D 1B B6
D3 C5 DB 8C 8A BE 9B 61 CF 7F 2D 00 70 7C D8 A3
CE 64 16 E7 BD A3 CE 68 EE 1C 73 0A 9F F2 6F 32
E6 C3 2E 63 A2 35 50 EA 2A 24 E3 52 59 C9 32 15
43 2F 58 9F A6 05
Xxxxx YYY response: 12 40 07 80 03 B1 01 5A 90 00 AC
===== Xxxxx YYY on CaID 0x1801, pid 0x0258 ====
SysID 9501, Parity Key select 02, Key type 00
-YYY-------------------------------------------
82 70 58 00 00 00 00 00 00 4B 4E 01 42 98 CA 22
1B F0 A6 70 43 34 53 D6 44 AE CF BC 75 83 67 B3
B8 21 C4 4C A5 D1 B8 A5 42 F0 17 27 0D 50 72 89
C0 05 5C D0 07 CF AC D8 DA 0B AF 06 64 BA 21 6C
7F AF 8B 9C 2E EF 8C AC 4A 9F B3 7F 27 F9 0E 1A
0B C9 8A 87 7B AA 82 A4 00 00 00
-Send to card 1 -----------------------------
21 00 53 A0 CA 00 00 4D 00 4B 4E 01 42 98 CA 22
1B F0 A6 70 43 34 53 D6 44 AE CF BC 75 83 67 B3
B8 21 C4 4C A5 D1 B8 A5 42 F0 17 27 0D 50 72 89
C0 05 5C D0 07 CF AC D8 DA 0B AF 06 64 BA 21 6C
-EMM-------------------------------------------
82 70 58 00 00 00 00 00 00 4B 4E 01 AA 17 1C CE
FD 37 B9 97 66 E0 08 23 BF A2 3D 29 D5 DC 53 2E
C8 B0 DD BE 04 EC 0D 6B 37 0E 2A CA 88 EF 98 B3
9E 03 A2 12 F5 D1 81 40 DE 00 45 EF 5A E6 D8 B6
81 3B 51 11 AC 44 C9 27 FD 94 70 FA CC D7 5E B4
DE 54 2B D0 49 AA 82 A4 00 00 00
-Send to card 1 -----------------------------
21 00 53 A0 CA 00 00 4D 00 4B 4E 01 AA 17 1C CE
FD 37 B9 97 66 E0 08 23 BF A2 3D 29 D5 DC 53 2E
C8 B0 DD BE 04 EC 0D 6B 37 0E 2A CA 88 EF 98 B3
9E 03 A2 12 F5 D1 81 40 DE 00 45 EF 5A E6 D8 B6
81 3B 51 11 AC 44 C9 27 FD 94 70 FA CC D7 5E B4
DE 54 2B D0 49 05
Nagra EMM response: 12 40 07 80 03 B1 01 57 90 00 A1
===== Nagra EMM on CaID 0x1800, pid 0x0258 ====
SysID 4E01, Parity Key select 02, Key type 01
-EMM-------------------------------------------
82 70 58 00 00 00 00 00 00 4B 4E 01 6A 9F 0C 01
C8 A0 59 6E CE 3F 3B 52 30 74 EC 36 C1 56 F5 40
44 88 E8 E6 28 BF F7 61 13 F8 FD 19 1B EE 22 E8
6F F0 B5 DD 26 9E 02 C5 1D 0E 8A CA C0 E1 11 85
4E 90 F3 E0 A2 2C 1E 18 24 E7 9A D4 0F 89 A0 EF
DC 39 58 47 6F AA 82 A4 00 00 00
-Send to card 1 -----------------------------
21 00 53 A0 CA 00 00 4D 00 4B 4E 01 6A 9F 0C 01
C8 A0 59 6E CE 3F 3B 52 30 74 EC 36 C1 56 F5 40
44 88 E8 E6 28 BF F7 61 13 F8 FD 19 1B EE 22 E8
6F F0 B5 DD 26 9E 02 C5 1D 0E 8A CA C0 E1 11 85
4E 90 F3 E0 A2 2C 1E 18 24 E7 9A D4 0F 89 A0 EF
DC 39 58 47 6F 05
Nagra EMM response: 12 00 07 80 03 B1 01 58 90 00 EE
===== Nagra EMM on CaID 0x1800, pid 0x0258 ====
SysID 4E01, Parity Key select 02, Key type 01
-EMM-------------------------------------------
82 70 58 00 00 00 00 00 00 4B 4E 01 6A 08 ED D5
EF 02 6F 20 15 C0 2E E8 E2 39 E8 89 48 1D 1B B6
D3 C5 DB 8C 8A BE 9B 61 CF 7F 2D 00 70 7C D8 A3
CE 64 16 E7 BD A3 CE 68 EE 1C 73 0A 9F F2 6F 32
E6 C3 2E 63 A2 35 50 EA 2A 24 E3 52 59 C9 32 15
43 2F 58 9F A6 72 E7 9D 00 00 00
-Send to card 1 -----------------------------
21 00 53 A0 CA 00 00 4D 00 4B 4E 01 6A 08 ED D5
EF 02 6F 20 15 C0 2E E8 E2 39 E8 89 48 1D 1B B6
D3 C5 DB 8C 8A BE 9B 61 CF 7F 2D 00 70 7C D8 A3
CE 64 16 E7 BD A3 CE 68 EE 1C 73 0A 9F F2 6F 32
E6 C3 2E 63 A2 35 50 EA 2A 24 E3 52 59 C9 32 15
43 2F 58 9F A6 05
Nagra EMM response: 12 40 07 80 03 B1 01 5A 90 00 AC
===== Nagra EMM on CaID 0x1800, pid 0x0258 ====
SysID 4E01, Parity Key select 02, Key type 00
-EMM-------------------------------------------
82 70 58 00 00 00 00 00 00 4B 4E 01 42 98 CA 22
1B F0 A6 70 43 34 53 D6 44 AE CF BC 75 83 67 B3
B8 21 C4 4C A5 D1 B8 A5 42 F0 17 27 0D 50 72 89
C0 05 5C D0 07 CF AC D8 DA 0B AF 06 64 BA 21 6C
7F AF 8B 9C 2E EF 8C AC 4A 9F B3 7F 27 F9 0E 1A
0B C9 8A 87 7B AA 82 A4 00 00 00
-Send to card 1 -----------------------------
21 00 53 A0 CA 00 00 4D 00 4B 4E 01 42 98 CA 22
1B F0 A6 70 43 34 53 D6 44 AE CF BC 75 83 67 B3
B8 21 C4 4C A5 D1 B8 A5 42 F0 17 27 0D 50 72 89
C0 05 5C D0 07 CF AC D8 DA 0B AF 06 64 BA 21 6C




Hope someone can help me jumpstart in creating the script.

Thanks in advance!

[InfRecursion]

Interestingly... I needed a change of scenery from this C++ program I'm working on... so I wrote a solution to your problem in Perl.

Basically, I scan the lines and when I see -Send I turn on a flag, when I see Xxxxx, Nagra or -EMM I turn the flag off. When the flag is on, I accumulate the values until its turned off. When the flag turns off, I do a clean up the spaces and output the value that your looking for. There is a space where each newline appears, to get rid of it, follow the instruction in my code.

Here you go:





open (DATA, "data.txt") || die "[ERROR]: Could not open the data file! Aborted.";

$FLAG = "false";


$temp = "";
while ($line = <DATA>)
{
$x = substr($line,0,5);
#print ("X: $x\n");

if ($x eq "-Send")
{
$FLAG = "true";
$line = <DATA>;
}

if ($x eq "Xxxxx" || $x eq "Nagra" || $x eq "-EMM-")
{
$FLAG = "false";
if (length $temp > 0)
{
$temp =~ s/ //g;
$temp =~ s/\n/ /g;
# For no spaces in the variable, uncomment line below and remove the line above.
# $temp =~ s/\n//g;
print "$temp\n";
$temp = "";
}
}

if ($FLAG eq "true")
{
#print $line;
$temp = $temp . $line;
}

}


close(DATA);


The data.txt is just the example of your input file verbatim. Here is the execution... (the forum may distort the output a bit due to the length).



[jpowers@pandora boknoy]$ perl boknoy.pl

210053A0CA00004D004B4E016A9F0C01 C8A0596ECE3F3B523074EC36C156F540 4488E8E628BFF76113F8FD191BEE22E8 6FF0B5DD269E02C51D0E8ACAC0E11185 4E90F3E0A22C1E1824E79AD40F89A0EF DC3958476F05

210053A0CA00004D004B4E016A08EDD5 EF026F2015C02EE8E239E889481D1BB6 D3C5DB8C8ABE9B61CF7F2D00707CD8A3 CE6416E7BDA3CE68EE1C730A9FF26F32 E6C32E63A23550EA2A24E35259C93215 432F589FA605

210053A0CA00004D004B4E014298CA22 1BF0A670433453D644AECFBC758367B3 B821C44CA5D1B8A542F017270D507289 C0055CD007CFACD8DA0BAF0664BA216C

210053A0CA00004D004B4E01AA171CCE FD37B99766E00823BFA23D29D5DC532E C8B0DDBE04EC0D6B370E2ACA88EF98B3 9E03A212F5D18140DE0045EF5AE6D8B6 813B5111AC44C927FD9470FACCD75EB4 DE542BD04905

210053A0CA00004D004B4E016A9F0C01 C8A0596ECE3F3B523074EC36C156F540 4488E8E628BFF76113F8FD191BEE22E8 6FF0B5DD269E02C51D0E8ACAC0E11185 4E90F3E0A22C1E1824E79AD40F89A0EF DC3958476F05

210053A0CA00004D004B4E016A08EDD5 EF026F2015C02EE8E239E889481D1BB6 D3C5DB8C8ABE9B61CF7F2D00707CD8A3 CE6416E7BDA3CE68EE1C730A9FF26F32 E6C32E63A23550EA2A24E35259C93215 432F589FA605

[boknoy]

Just wanna say thanks! Your script works perfectly and more than what I expected, was in perl. Now, I can work on the log file both in Windows and Linux.

[InfRecursion]

Glad it worked out for you... enjoy. :)