1

I am using the below command on a file to extract few lines based on chr# ( different chromosome numbers). This is just a single file am working on. i have 8 such files and for each file I have to do this for chr(1to 22 and then chrX and chrY) , am not using any loop, I did it invidually , but if you see that I want the header to be intact for each of my output. If I execute individually I get the header in the output but if am running but if I run the script for all the 8 files together which is like 8*24 commands in a script one after another, the output does not have any header. Can you tell me why is this happening?

#!/bin/sh
#
#$ -N DOC_gatk_chr
#$ -cwd
#$ -e err_DOC_gatk_chr.txt
#$ -o out_DOC_gatk_chr.txt
#$ -S /bin/sh
#$ -M vivek.das@ieo.eu
#$ -m bea
#$ -l h_vmem=25G

more S_313_IPS_S7995.coverage.sample_interval_summary | head -n1; more S_313_IPS_S7995.coverage.sample_interval_summary | grep "chr1" > S_313_IPS_S7995.chr1.coverage
more S_313_IPS_S7995.coverage.sample_interval_summary | head -n1; more S_313_IPS_S7995.coverage.sample_interval_summary | grep "chr2" > S_313_IPS_S7995.chr2.coverage
more S_313_IPS_S7995.coverage.sample_interval_summary | head -n1; more S_313_IPS_S7995.coverage.sample_interval_summary | grep "chr3" > S_313_IPS_S7995.chr3.coverage
more S_313_IPS_S7995.coverage.sample_interval_summary | head -n1; more S_313_IPS_S7995.coverage.sample_interval_summary | grep "chr4" > S_313_IPS_S7995.chr4.coverage
more S_313_IPS_S7995.coverage.sample_interval_summary | head -n1; more S_313_IPS_S7995.coverage.sample_interval_summary | grep "chr5" > S_313_IPS_S7995.chr5.coverage
more S_313_IPS_S7995.coverage.sample_interval_summary | head -n1; more S_313_IPS_S7995.coverage.sample_interval_summary | grep "chr6" > S_313_IPS_S7995.chr6.coverage
more S_313_IPS_S7995.coverage.sample_interval_summary | head -n1; more S_313_IPS_S7995.coverage.sample_interval_summary | grep "chr7" > S_313_IPS_S7995.chr7.coverage
more S_313_IPS_S7995.coverage.sample_interval_summary | head -n1; more S_313_IPS_S7995.coverage.sample_interval_summary | grep "chr8" > S_313_IPS_S7995.chr8.coverage
more S_313_IPS_S7995.coverage.sample_interval_summary | head -n1; more S_313_IPS_S7995.coverage.sample_interval_summary | grep "chr9" > S_313_IPS_S7995.chr9.coverage

Am running it as a job with qsub so the structure of the script looks like above. It works if I execute the commands individually but if I run them like this , the header does not get printed in the output file, the ';' is not recognized it seems. I tried to ran it with both qsub filename.sh and sh filename.sh. I found that with sh filename.sh the header gets printed in the console. So definitely the command before';' semi-colon is not being written in the file. How can I get rid of this problem.

Desired output:

Target  total_coverage  average_coverage    IPS_S7995_total_cvg IPS_S7995_mean_cvg  IPS_S7995_granular_Q1   IPS_S7995_granular_median   IPS_S7995_granular_Q3   IPS_S7995_%_above_15
chr2:41460-41683    14271   63.71   14271   63.71   56  67  79  100.0
chr2:45338-46352    123888  122.06  123888  122.06  79  123 147 94.6
chr2:218731-218983  11653   46.06   11653   46.06   36  50  55  100.0
chr2:224825-225012  12319   65.53   12319   65.53   57  68  76  100.0
chr2:229912-230090  20983   117.22  20983   117.22  93  120 147 100.0
chr2:230947-231137  22386   117.20  22386   117.20  100 120 139 100.0
chr2:233074-233258  11710   63.30   11710   63.30   54  66  73  100.0
chr2:234086-234300  22952   106.75  22952   106.75  91  113 126 100.0
chr2:242747-242922  20496   116.45  20496   116.45  93  124 142 100.0
chr2:243469-243671  27074   133.37  27074   133.37  126 138 148 100.0

But output am getting is below without the header

chr2:41460-41683    14271   63.71   14271   63.71   56  67  79  100.0
chr2:45338-46352    123888  122.06  123888  122.06  79  123 147 94.6
chr2:218731-218983  11653   46.06   11653   46.06   36  50  55  100.0
chr2:224825-225012  12319   65.53   12319   65.53   57  68  76  100.0
chr2:229912-230090  20983   117.22  20983   117.22  93  120 147 100.0
chr2:230947-231137  22386   117.20  22386   117.20  100 120 139 100.0
chr2:233074-233258  11710   63.30   11710   63.30   54  66  73  100.0
chr2:234086-234300  22952   106.75  22952   106.75  91  113 126 100.0
chr2:242747-242922  20496   116.45  20496   116.45  93  124 142 100.0
chr2:243469-243671  27074   133.37  27074   133.37  126 138 148 100.0
4
  • 1
    "If I execute individually I get the header" What header? Why don't you show us both the output you get and the output you expect / want? – Hauke Laging Jan 27 '15 at 11:10
  • Also consider simplifying it by removing more and giving the files as arguments directly to head and grep. – peterph Jan 27 '15 at 11:19
  • I try to do a kind of troubleshooting and I found that when I am running the command individually it is working but if I am running with qsub as a job the headers are not getting appended in the output file. – ivivek_ngs Jan 27 '15 at 11:34
  • @HaukeLaging . I have already edited the question. Can you please take a look now and let me know if its clear. – ivivek_ngs Jan 27 '15 at 11:43
1

You need something like this:

{ head -n1 S_313_IPS_S7995.coverage.sample_interval_summary; 
  grep "chr1" S_313_IPS_S7995.coverage.sample_interval_summary; } >S_313_IPS_S7995.chr1.coverage

or

awk 'NR==1 || /chr1/' S_313_IPS_S7995.coverage.sample_interval_summary >S_313_IPS_S7995.chr1.coverage

The problem is that the redirection affects only one command. In order to get the output of head and grep in the redirection they have to be grouped. But awk is probably the better choice here.

3
  • I used the previous one even that one works out. { head -n1; grep "chr1"; } <S_333_IPS_S7997.coverage.sample_interval_summary >S_333_IPS_S7997.chr1.coverage – ivivek_ngs Jan 27 '15 at 13:21
  • @vchris_ngs I was afraid that might not work because head reads more than the first line. But an input redirection is seekable so head resets to the position grep should see. It does not work with cat file | { head .... But it would be safe to use the shell bulitin read: { IFS= read -r line; echo "$line"; grep ...;} <... – Hauke Laging Jan 27 '15 at 13:26
  • yes it does not work with cat but with it did work here. anyway I will keep this in mind will use awk for such actions. Thanks a lot. But yes heads work with first link this case so it worked out properly – ivivek_ngs Jan 27 '15 at 13:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.