-1

could someone help me? I am trying to create a lot of files, but when I try to write to it with echo it adds a space and single quotes, for example:

' (here is the space)  file.tmp' 

'  file1.tmp'

'  file3.tmp' 

the code is this.


cat limpio.tmp | while read line; do
  srcip=$(echo $line | awk -F"," '{print $1}')
  destip=$(echo $line | awk -F"," '{print $2}')
  port=$(echo $line | awk -F"," '{print $NF}')
  echo "${srcip}" >> "$destip.tmp"
done

The problem comes when I try to iterate and view its content with cat and I can't because of the spaces.

thank you very much

8
  • 2
    Could you show a representative sample of the limpio.tmp file? Also, why don't you just do awk -F, '{ print $1 >>($2 ".tmp") }' limpio.tmp?
    – Kusalananda
    Nov 28, 2021 at 6:28
  • bro it because i want to fill this files later sorry for the new question but i dont undestand you code bro
    – truco
    Nov 28, 2021 at 6:33
  • 2
    Adding information to the comments is no use. Please add it to your question and format it. Make it easy for people who want to help you Nov 28, 2021 at 6:57
  • 2
    So the spaces are part of your data it seems. Consider updating your question with the data. Make sure to use the markup tools provided to format the data as code to make it more obvious what the input file looks like. If the spaces are part of you data, you may want to change your question it "how do I remove the spaces?".
    – Kusalananda
    Nov 28, 2021 at 7:08
  • 2
    Posting an exact example of the input file (showing multiple lines) and an exact example of your required output would make it a lot easier to understand your question. Also, edit and post them in your question (correctly formatted) and not in the comments. Nov 28, 2021 at 8:13

3 Answers 3

3

The echo utility inserts spaces, but only as delimiters between arguments when outputting them:

$ echo 1    2 3       4 'A B     C'
1 2 3 4 A B     C

The spaces between B and C are preserved as these are part of the same argument to echo.

So it isn't that echo is inserting spaces, but rather that your data already contains spaces.

I'm assuming that the single quotes that you see are part of the output from ls on your system (see Why is 'ls' suddenly wrapping items with spaces in single quotes?), or possibly part of the shell's tracing output while it's running with set -x active (see Why does bash add single quotes to unquoted failed pathname expansions in a command before executing it?). That is, it's not actually part of the filenames but only some tool's way of rendering the names.

Assume, for example, that your input looks like

174.18.418.83, 10.404.401.94, 53144
174.18.418.83, 10.404.401.94, 53144
174.18.418.83, 10.404.401.94, 53144

You will notice that there are spaces after the comma delimiters. If you read the 2nd field from the above file using the comma as the delimiter, you will therefore retain the space(s) at the start of the field, and if you use that data as a filename, the filename will start with space.

(The observant reader will notice that the code in the question uses the variable line unquoted, this means that the shell would split the variable's value on the spaces and present echo with separate arguments like 174.18.418.83,, 10.404.401.94,, and 53144. So it is true, in a sense, that it is echo that inserts the spaces that needs to be removed, but it would not do that had there been no spaces/tabs in the data to start with. See also When is double-quoting necessary?)

The solution is to use both the comma and any subsequent blank character as the delimiter with awk:

awk -F ',[[:blank:]]*' '{ print $2 }'

The expression ,[[:blank:]]* matches comma followed by any number of blank characters. A "blank" character is a space or a tab. You may change this to , * ("comma, space, asterisk") if you know you have literal spaces.

Now to avoid calling awk three times for each line in the input file.

The print statement in awk can be "redirected" in pretty much the same way as you may redirect output in the shell, using > followed by a filename.

The filename you seem to want to use is whatever is in the 2nd field, with .tmp appended.

This mean you can replace the whole loop in your question with the following single awk command:

awk -F ',[[:blank:]]*' '{ print $1 > ($2 ".tmp") }' limpio.tmp

The expression ($2 ".tmp") evaluates to the string resulting from adding .tmp to whatever is in the 2nd field.

Using > with print means that the output file will be created or truncated (emptied) on the first print and that only subsequent print to the same name would append new lines. If you were to use >> in place of >, the first print would append to the file rather than truncate the file.

There is no built-in check for sanity here, so the above may clobber important files anywhere on the system where you have write access, depending on what's in the 2nd column of the input.

If you use GNU awk, then this is all you need to do. But if you use some other awk and need to write to a vast number of files, you may need to manually call close() on the files as you have processed them, or you may run out of available file descriptors.

awk -F ',[[:blank:]]*' '{ fn = $2 ".tmp"; print $1 >>fn; close(fn) }' limpio.tmp

Here, I'm saving the output filename in a variable, fn. I then append the output to that file, and then I close the file. This ensures that I won't exhaust the available file descriptors, but it also requires me to always use >> with print and manually clean up (remove) the output files if I need to run the awk command again.

GNU awk keeps track of how many file descriptors have been opened and starts closing them if needed (and re-opens the files for appending if further data needs to be written to a file later).

See also:

1
  • Thanks for the support bros i solve the problem.
    – truco
    Nov 29, 2021 at 14:08
0

This is the code you originally posted:

cat limpio.tmp | while read line; do
  srcip=$(echo $line | awk -F"," '{print $1}')
  destip=$(echo $line | awk -F"," '{print $2}')
  port=$(echo $line | awk -F"," '{print $NF}')
  echo "${srcip}" >> "$destip.tmp"
done

That is unnecessarily complicated. Seeing that you're reading comma delimited fields, you could read them directly into variables:

while IFS=$' \t\n,' read -r srcip destip port junk ;do
  echo "$srcip" >> "${destip}.tmp"
done <limpio.tmp
5
  • This seems to solve the user's issue, given that the input file contains spaces. However, your answer does not try to diagnose the issue, nor does it explain how it is solved by your code. If you just want to add a comma to the default value of IFS, btw, you could use IFS=$IFS, read .... You may also want to use printf rather than echo as echo has a tendency to change data under certain circumstances.
    – Kusalananda
    Nov 28, 2021 at 7:57
  • 1
    @they : The OP seems happy with his echo, and the echo is not really the problem. We are not going to teach him EVERYTHING in just one posting. Adding more complications will only distract him from understanding what we're doing here.
    – Pourko
    Nov 28, 2021 at 8:06
  • 1
    Sure, forget about the printf then. Just describe how your solution solves the issue, and what the issue actually was. Because now it's just "here's another way to do what you're doing (and it solves the problem but I'm not telling you why)".
    – Kusalananda
    Nov 28, 2021 at 8:14
  • Also note that there is a security issue in that the user is writing to files named by input data, and that this may overwrite existing files if the data contains paths of existing files (anywhere on the system where the user may write).
    – Kusalananda
    Nov 28, 2021 at 8:20
  • 1
    I swapped echo for printf because a value starting with - or containing \ may break echo. The echo command is non-standard and works in different ways in different places. printf is consistent Nov 28, 2021 at 23:55
0

The easiest fix probably is to remove the double-quotes in echo "${srcip}".

See yourself:

var=' spacy '
echo X$(echo "$var")X
echo X$(echo $var)X

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .