1

I have multiple strings in a file, for each one I need to run a command and export the results. Here is an illustration:

file.txt

string1
string2
string3

I'm running:

while read in; do sigtool --hex-dump "$in"; done < file.txt > test.txt

This provides the following output:

737472696e6731a737472696e6732a737472696e6733

I would like the output to be separated by line like below:

737472696e6731
737472696e6732
737472696e6733

How can I achieve that?

| improve this question | |
  • What happens when you run the code you have given? – jesse_b Dec 22 '17 at 2:33
  • This > 737472696e6731a737472696e6732a737472696e6733, it will print in one line, with a space. – mirsad Dec 22 '17 at 2:35
  • I'm sure this isn't the proper way of doing it but what about: while read in; do echo -e "$(sigtool --hex-dump "$in")\n"; done < file.txt > test.txt? – jesse_b Dec 22 '17 at 2:48
  • Do you have to use sigtool for this? have you considered something like perl -lpe '$_ = unpack("H*",$_)' file.txt instead? – steeldriver Dec 22 '17 at 3:02
  • Note this won't be a standard format at all. Have you considered using xxd, which can do conversions in either direction? – Wildcard Dec 22 '17 at 3:29
4

You want newlines. So output newlines.

while read in; do sigtool --hex-dump "$in"; echo; done < file.txt > test.txt

Note the added echo command.

Mind you, the first place I would check is the sigtool man page to see if there's an option to output trailing newlines.

Okay, I wrote the above without actually getting sigtool (from the clamav package) and testing it. Now I have done so.

The code you wrote doesn't work, at least not with clamav-0.99.2-3.el6.x86_64. sigtool accepts input on its standard input. It has no option to accept and translate command line arguments.

So what you really want is, dropping back to the old favorite non-visual editor ex:

printf '%s\n' 'g/^/.!tr -d \\n | sigtool --hex-dump' %p | ex file.txt > test.txt

Demonstration:

$ cat file.txt 
string1
string2
string3
$ printf '%s\n' 'g/^/.!tr -d \\n | sigtool --hex-dump' %p | ex file.txt > test.txt
$ cat test.txt 
737472696e6731
737472696e6732
737472696e6733
$

Explanation:

Globally (for each line matching regex /^/, which is every line) filter each line (.!) through the shell pipeline tr -d \\n | sigtool --hex-dump, then print (p) the entire buffer (%) to standard output, and don't save changes to file.txt.

Another approach, using Awk:

awk -v p='sigtool --hex-dump' '
  {printf "%s", $0 | p; close(p); printf "\n"}
' file.txt > test.txt
| improve this answer | |
  • I suspect that will still unpack the newlines though (resulting in trailing a bytes visible in the OP's concatenated output) – steeldriver Dec 22 '17 at 3:05
  • @steeldriver, okay, okay. I actually did the research this time and tested it. :P – Wildcard Dec 22 '17 at 3:25
  • @Wildcard, great job! Exactly what I needed. – mirsad Dec 22 '17 at 22:24
1

Alternative brute force:

cat test.txt | sigtool --hex-dump | sed 's/0a/\n/g' > gzout.hex

Just convert the hex newlines back as long as you are dealing with an ASCII text file.

(If you're dealing with non-ASCII you may have 0a appearing in the hex dump without signifying a newline in the input.)

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.