2

We use the following script:

more test.sh
#!/bin/bash

while read -r line
do

echo $line

done < /tmp/file

This is the file:

kafka-broker,log.retention.hours,12
kafka-broker,default.replication.factor,2
fefolp-defaults,fefolp.history.fs.cleaner.interval,1d
fefolp-defaults,fefolp.history.fs.cleaner.maxAge,2d
fefolp-env,fefolp_daemon_memory,10000
blo-site,blo.nodemanager.localizer.cache.target-size-mb,10240
blo-site,blo.nodemanager.localizer.cache.cleanup.interval-ms,300000
ams-env,metrics_collector_heapsize,512
fefolp,hbase_master_heapsize,1408
fefolp,hbase_regionserver_heapsize,512
fefolp,hbase_master_xmn_size,192
core-site,blolp.proxyuser.ambari.hosts,*
core-site,Hadoop.proxyuser.root.groups,*
core-site,Hadoop.proxyuser.root.hosts,*
blo-site,blo.scheduler.minimum-allocation-mb,1024
blolp-env,fefolp_heapsize,4096

Remark - after the last line - there are no space!

But the script prints only these lines (except the last line):

./test.sh
kafka-broker,log.retention.hours,12
kafka-broker,default.replication.factor,2
fefolp-defaults,fefolp.history.fs.cleaner.interval,1d
fefolp-defaults,fefolp.history.fs.cleaner.maxAge,2d
fefolp-env,fefolp_daemon_memory,10000
blo-site,blo.nodemanager.localizer.cache.target-size-mb,140
blo-site,blo.nodemanager.localizer.cache.cleanup.interval-ms,300
ams-env,metrics_collector_heapsize,51
fefolp,hbase_master_heapsize,1408
fefolp,hbase_regionserver_heapsize,542
fefolp,hbase_master_xmn_size,19
core-site,blolp.proxyuser.ambari.hosts,*
core-site,Hadoop.proxyuser.root.groups,*
core-site,Hadoop.proxyuser.root.hosts,*
blo-site,blo.scheduler.minimum-allocation-mb,1024

Why does this happen?

  • because the last text line is not complete. you need a \n there. – Rui F Ribeiro Nov 18 '18 at 11:50
  • do you have suggestion how to add \n on the last line in the file , so I will put thus in the scritp – yael Nov 18 '18 at 12:02
  • echo >> file as a poor mans solution. – Rui F Ribeiro Nov 18 '18 at 12:36
  • Note POSIX defines "line" as a sequence of zero or more non- <newline> characters plus a terminating <newline> character. Your last line isn't even a line in this context. – Kamil Maciorowski Nov 18 '18 at 16:24
6

Your input text contains an incomplete line as its last line. The last line is not terminated by a newline.

while IFS= read -r line || [ -n "$line" ]; do
    printf '%s\n' "$line"
done <file

The above loop will read unmodified lines (without stripping whitespaces or interpreting backslashed control sequences) from the file called file and print them to standard output.

When an incomplete line is read, read will fail, but $line will still contain data. The extra -n test will detect this so that the loop body is allowed to output the incomplete line. In the iteration after that, read will fail again and $line will be an empty string, thus terminating the loop.

  • the script is only example of the problem , actually I set the echo in the script to show this issue , but what we need is how to set the last line with "\n" with sed for example, since the the - file is already exists – yael Nov 18 '18 at 12:26
  • so if need to add "\n on the last line , then need sed/perl to add the "\" for this solution – yael Nov 18 '18 at 12:30
  • @yael You say in the question that you use a loop like the one you are showing. I gave a solution for how to fix that loop. It does not matter what the body of the loop looks like, $line will be each individual line of the file, including the last incomplete one. If you don't use a loop like this, what do you use? – Kusalananda Nov 18 '18 at 12:37
  • ok , so maybe I will post another question for how to add "\n" on the last line – yael Nov 18 '18 at 12:38
  • 1

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.