0

I am using a While Read Line state to parse our Openstack Swift accounts. Below is some code from a statement I am using to find all data with a specific creation/modification date.

First I run a command to get a list of all the containers in the account:

swift list -A http://sslabapi/auth/v1.0 -U qa04 -K <Account Password> > containers.txt

This command will output a list as follows:

ffd5eef3-2b75-4d4f-b30c-b9836b32a34c

Below is the command I can run to see the contents of the container:

swift list -l -A http://sslabapi/auth/v1.0 -U qa04 -K <Account Password> ffd5eef3-2b75-4d4f-b30c-b9836b32a34c

1507732 2017-09-27 19:55:57 application/octet-stream ffd5eef3-2b75-4d4f-b30c-b9836b32a34c.FR.02
     1812122 2017-09-27 19:55:57 application/octet-stream ffd5eef3-2b75-4d4f-b30c-b9836b32a34c.FS.02

I wrote the following command to parse all the containers and alert on any object in a container that has a specific date "2017-09-27":

while read line; do echo -e swift list -l -A http://sslabapi/auth/v1.0 -U qa04 -K <Account Password> $line | bash; done < containers.txt | awk '{ if ($2 == "2017-09-27") print $0}'

Is there a way to print the contents of of the $line in the print $0 statement? For example the output would be something like:

ffd5eef3-2b75-4d4f-b30c-b9836b32a34c 1812122 2017-09-27 19:55:57 application/octet-stream ffd5eef3-2b75-4d4f-b30c-b9836b32a34c.FS.02

EDIT:

I am still having issues with this, and I am not sure why at this moment. The file "containers.txt" has a list of containers (which are the root directory for files or objects) that get inserted into my command and works without issue. I cannot seem to get the same GUID to print in AWK statement. This is needed so I am able to find the root directory of the files flagged in the AWK statement. The following two statements produce the exact same output:

while read line; do echo -e swift list -l -A http://sslabapi/auth/v1.0 -U auto02 -K [ACCOUNT PASSWORD] $line | bash; done < containers2.txt | awk -v l="$line" '$2 == "2018-05-08" { print l, $0 }'

while read line; do echo -e swift list -l -A http://sslabapi/auth/v1.0 -U auto02 -K [ACCOUNT PASSWORD] $line | bash; done < containers2.txt | awk  '$2 == "2018-05-08" {print $0 }'

The output will look as follows: 9923 2018-05-08 15:56:34 application/octet-stream 4919d1c8-f5c7-448f-98ad-29d88f1a5a84.BC.01

Ideally I would want it to produce and output with the root container in the output like the following:

4919d1c8-f5c7-448f-98ad-29d88f1a5a84 9923 2018-05-08 15:56:34 application/octet-stream 4919d1c8-f5c7-448f-98ad-29d88f1a5a84.BC.01

1

You need to put the awk inside the while loop, not piped into from outside. The loop variable is out of scope by then (and in any case, it would take several different values as well, so which one would you want to see?).

Something along these lines should be fine:

while read line
do
    swift list -l -A http://sslabapi/auth/v1.0 -U qa04 -K <Account Password> "$line" | awk -v line="$line" '{ if ($2 == "2017-09-27") print line $0}'
done < containers.txt

I took the echo/| bash out because they didn't achieve anything. This does mean that you're launching several awk processes instead of one, but it won't be a problem here.

You could also connect the line and data up within the loop if you preferred:

swift list -l -A http://sslabapi/auth/v1.0 -U qa04 -K <Account Password> "$line" | xargs -L 1 echo "$line"

The xargs -L 1 command will run echo "$line" ABC for every line ABC of output from the swift list command, with the effect of prefixing every line with $line. You could then pipe into your existing awk on the outside, adjusting your test to $3 instead of $2 to compensate.

Which of these works better will depend on exactly what your end goal is and how the data is distributed, but both be equivalent as far as it goes here. I would personally prefer the second version, but it doesn't really matter.

0

try this in awk

   awk -v l=$line '{if ($2 == "2017-09-27") print l" "$0}'

my test sample:

# echo $line
abc
# cat tt.txt 
hi 2017-09-27 test1
hi 2017-09-28 test1
# awk -v l=$line '{ if ($2 == "2017-09-27") print l" "$0}' tt.txt
abc hi 2017-09-27 test1
  • while read line; do echo -e swift list -l -A sslabapi/auth/v1.0 -U qa04 -K <Account Password> $line | bash; done < containers.txt | awk -v l=$line '{ if ($2 == "2017-09-27") print l $0}' – bbelden Feb 4 at 15:07
  • I tried the statement above and it produced the following output: 1507732 2017-09-27 19:55:57 application/octet-stream ffd5eef3-2b75-4d4f-b30c-b9836b32a34c.FR.02 1812122 2017-09-27 19:55:57 application/octet-stream ffd5eef3-2b75-4d4f-b30c-b9836b32a34c.FS.02 So it did not seem to add the root container to the output of the AWK command. Thank you for the reply though. – bbelden Feb 4 at 15:09
  • I apologize for the formatting. The spaces did not appear. But that addition did not seem to print the contents of $line. – bbelden Feb 4 at 15:11
  • I just tried to the updated answer and I am still not seeing the container from the $line outputed in the AWK command. – bbelden Feb 4 at 15:17
  • check my test sample.... should work... couldn't replicat your scenarion since URL is not accesable... – msp9011 Feb 4 at 15:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.