2

I have a .vcf.gz file, with the following aspect:

#CHROM  POS     ID      REF     ALT          
chr1    10894   chr1:10894:G:A  G       A         
chr1    10915   chr1:10915:G:A  G       A          
chr1    10930   chr1:10930:G:A  G       A 

I want to modify the CHROM column to remove 'chr' and to replace it with nothing, so I want to get a result like the following:

#CHROM  POS     ID      REF     ALT          
1    10894   chr1:10894:G:A  G       A         
1    10915   chr1:10915:G:A  G       A          
1    10930   chr1:10930:G:A  G       A 

Therefore, I wrote the following command line:

zcat input.vcf.gz | sed 's/^chr//' > output.vcf.gz

and it worked. The problem is that I want to save the output file as a zipped one, with the vcf.gz extension. Even if I wrote 'output.vcf.gz', the output file is not zipped.

How can I modify a zipped file and then save it as a zipped file again?

Many thanks!

7
  • 2
    Note that for this particular file type, you should use bgzip instead of gzip. This is a drop-in replacement for gzip but it supports block compression and this allows it to then be indexed by tabix and be used by standard tools like bcftools.
    – terdon
    Commented Sep 18, 2023 at 7:58
  • 1
    If it's a VCF file with a Tabix index, you will also have to remember to re-build that index (which will require that you use bgzip as terdon pointed out).
    – Kusalananda
    Commented Sep 18, 2023 at 8:10
  • 1
    @terdon huh! bgzip! I learn something new every day! Commented Sep 18, 2023 at 12:00
  • 1
    It's specific to this kind of file, @MarcusMüller, I don't know that it is even used outside the field of bioinformatics.
    – terdon
    Commented Sep 18, 2023 at 12:01
  • 1
    @MarcusMüller this is a biiiig subject and kinda off topic here, but please feel free to ping me in /dev/chat if you're curious about it. This is what I do for a living in real life :)
    – terdon
    Commented Sep 18, 2023 at 12:28

2 Answers 2

12

zcat is really but a convenience function of gzip; to cite the gzip/gunzip/zcat manual page (man zcat):

The zcat command is identical to gunzip -c.

Just as you can use gunzip -c (or zcat) in a piped chain of programs, you can use gzip to compress again:

zcat input.vcf.gz | sed 's/^chr//' | gzip > output.vcf.gz
#                                    ^^^^

or

gunzip -c input.vcf.gz | sed 's/^chr//' | gzip > output.vcf.gz
#^^^^^^^^                                 ^^^^

if you like consistency.

That's it. That's all there is to it.

Oh, taking a bet here: you're doing bioinformatics, and your vcf file is actually a "Variant Call Format" file, and can be rather large. gzip isn't a very fast decompressor, and a rather slow compressor. If you're stuck using the gzip compression file format,

unpigz -c input.vcf.gz | sed 's/^chr//' | pigz > output.vcf.gz
#^^^^^^^^                                 ^^^^

pigz does exactly the same as gzip, but scales to multiple CPU cores. Use it.

If you're not bound to keep these files in a gzip container, but are free to choose a more modern format,

unpigz -c input.vcf.gz | sed 's/^chr//' | zstd   -T0   -8 > output.vcf.zst
# decompress using     |                | ^^^^   ^^^   ^^
# unpigz instead of    |     modify     |  \--\   \-\   \\  compression ratio
# gzip/zcat            |                |      \     \    \ -0=very fast 18=very compressed
#                      |                |       \     \     -8 is much better compressed
#                      |                |        \     \    than gzip --best, but faster
#                      |                |         \     \
#                      |                |          \     \- Use as many threads as CPU cores
#                      |                |           \
#                      |                |            \
#                      |                |             \---- Use zstd instead of gzip
4
  • 2
    @MarcusMüller I would still keep it, but I hear what you're saying. I also, personally, favour gzip -d to gunzip in scripts.
    – Kusalananda
    Commented Sep 18, 2023 at 8:08
  • 1
    @Kusalananda same, but we've already got one mental redirection from zcat to gunzip in the man page, I didn't want to drop more indirection at the poor reader Commented Sep 18, 2023 at 8:53
  • Oh hey, I'm sorry, I didn't mean to imply there was anything wrong here, I was just wondering if there was a specific reason, maybe some edge case, which made using -c safer or something. I had already upvoted.
    – terdon
    Commented Sep 18, 2023 at 11:11
  • @terdon no such implication was assumed :) It was just that I took your question, thought about it, and came to the conclusion that it's not what people would expect in that place :) Commented Sep 18, 2023 at 11:17
3

Simply add gzip in the pipe:

zcat input.vcf.gz | sed 's/^chr//' | gzip > output.vcf.gz

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .