I am very new with linux shell commands. I need to read a text file which contains several lines as '!Platform_series_id = GSE1145'. I should split each line to use 'GSE1145' after the '=' sign part of it. Finally I want to produce a link such as 'ftp://ftp.ncbi.nlm.nih.gov/geo/series/GSE5nnn/GSE1145/suppl/GSE1145_RAW.tar' The bold parts of the link will be taken from the txt file. The remaining parts are constant. This was the story.

If I come to the point where I got stuck:

When I run the following code

while read p; do
  A="$(cut -d'=' -f2 <<<$p)"
  echo "ftp://ftp.ncbi.nlm.nih.gov/geo/series/GSE5nnn/$A/suppl/$A_RAW.tar"
done < a.txt

It gives an unexpected result

/suppl/.tarcbi.nlm.nih.gov/geo/series/GSE5nnn/ GSE1145 /suppl/.tarcbi.nlm.nih.gov/geo/series/GSE5nnn/ GSE1643

Then I tried simply concatenating the same variable

while read p; do
  A="$(cut -d'=' -f2 <<<$p)"
  echo "$A$A"
done < a.txt

But still the answer is not what I expect. It just behaves as it is written echo $A.

GSE1145 GSE1643

To understand the problem, finally I tried the following code:

A="$(cut -d'=' -f2 <<< '!Platform_series_id = GSE1145')"
echo $A$A

It gives me a correct result

GSE1145 GSE1145

So what is the problem when I read it?

By the way, when I try following code, It works! But why? I think this is not the correct way. If I put spaces as many as the length of the $A,

while read p; do
  A="$(cut -d'=' -f2 <<<$p)"
  echo "        $A$A"
done < a.txt

GSE1145 GSE1145 GSE1643 GSE1643

Thanks for helps.

  • Does your text file have DOS-style (CRLF) line endings by any chance? Regardless, I'd suggest using a text-processing utility such as awk for this kind of task - rather than a shell loop. – steeldriver Nov 23 '17 at 0:05
  • It doesn't give you the correct result... When you cut the string the result has a leading space. It's always there, you just don't see it... Try echo X$A and you'll see the space... Anyway, as suggested, this isn't a job for while..read (which should never be used to process text). – don_crissti Nov 23 '17 at 0:13

I should split each line to use 'GSE1145' after the '=' sign

That's a signal to use awk(1). Its basic idea is to read lines and split them automatically, and let you do what you will. To make awk print your link,

awk -F ' += +' \
    '{ printf "ftp://ftp.ncbi.nlm.nih.gov" \
              "/geo/series/GSE5nnn/%s/suppl/%s_RAW.tar", $2, $2 }' filename

That will print something for each line in the file. awk lets you choose which lines to process how, either by line number or by matching a regular expression.

awk is worth learning. Treat it nicely and it'll be your friend.

|improve this answer|||||
  • Thanks for the answer. I have two questions with your suggestion. I have tried it. The last line of the txt file is "!Platform_series_id = GSE2478". First, it processed the last line only. Why? Second, it printed out: "_RAW.tarSE2478ftp.ncbi.nlm.nih.gov/geo/series/GSE5nnn/GSE2478 ". It looks like a little bit mixed up. – ykpemre Nov 23 '17 at 10:28
  • @ykpemre please see my comment about DOS line endings on your original post - you can check using cat -et a.txtfor example – steeldriver Nov 23 '17 at 11:57
  • @steeldriver I checked. It printed this. M-oM-;M-?!Platform_series_id = GSE1145^M$ !Platform_series_id = GSE1643^M$ !Platform_series_id = GSE2109^M$ I dont know what are these characters "M-oM-;M-?" and "^M". – ykpemre Nov 23 '17 at 13:15
  • @ykpemre the ^M characters at the ends of the lines indeed confirm that it is DOS formatted. The M-oM-;M-? at the beginning is probably a byte order mark – steeldriver Nov 23 '17 at 13:21
  • @ykpemre, you're splitting on the regular expression that is the argument to -F. If the pattern isn't present on the line, it won't split, and $2 is empty. Printing it won't look like much, but you should get a blank line for every such input. – James K. Lowden Nov 23 '17 at 15:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.