4

I have a huge (ca. 20G) text file which contains millions of passages (a.k.a. paragraphs) with headers. Headers and paragraphs are always one line each, e.g.,

Sunshine
This is a sunny day.
Darkness
A cave is a dark place.

What I try to come up with is a terminal command which goes through the text and adds a '>' in front of every header, i.e., every odd-numbered line (lines 1, 3, 5, …), e.g.,

>Sunshine
This is a sunny day.
>Darkness
A cave is a dark place.

Any ideas?

If this is relevant: the above text was just an example. Most of the headers are MD5s, followed by a DNA sequence ('paragraph'), e.g.,

0002ebd9ca12d6b69dfc3066356fc299
CATTAACCATTGGATACCTTCGGGTATATCCCATCCGTGTCTACATACTCTTGTTGCTTTGGCAGGCCGTGGTCACACACTGTGGGCTATGCCTGCATGTGCCTGCCAGAGGACCA

… which I'm trying to convert to

>0002ebd9ca12d6b69dfc3066356fc299
CATTAACCATTGGATACCTTCGGGTATATCCCATCCGTGTCTACATACTCTTGTTGCTTTGGCAGGCCGTGGTCACACACTGTGGGCTATGCCTGCATGTGCCTGCCAGAGGACCA

4
  • 1
    Are blank lines in your input?
    – cuonglm
    Sep 7, 2014 at 16:04
  • 1
    You say "in front", but the > is added as the 2nd character of your examples. Can you clarify where you want it and also, are the single quotes in your examples part of the text file or just a formatting slip-up in your post? Sep 7, 2014 at 16:14
  • sorry, for the confusion. I only want > in front of every second line beginning with the first. > was removed when I created this post and the text was put in the greyed out box. so I used ' to be able to show >
    – BSP
    Sep 7, 2014 at 16:29
  • no blank lines in my file
    – BSP
    Sep 7, 2014 at 16:44

7 Answers 7

4

To edit every other (a.k.a. every second) line, starting with the first, with GNU sed, do

sed '1~2s/^/>/' your_file

This will write the modified file to the standard output.  I.e., if you type just the above, the modified file will display on the screen.  You can put this into a new file by redirecting the output with a >; e.g.,

sed '1~2s/^/>/' your_file > your_new_file

or, if you want to modify your existing file, use -i:

sed -i '1~2s/^/>/' your_file
1
  • 1
    sed -i on a 20g file is a really terrible idea.
    – mikeserv
    Jun 29, 2015 at 9:49
3

POSIXly:

sed 's/^/>/;n' < file.in > file.out
2

Another POSIX answer:

paste -d'>\n' /dev/null - - <infile

It gets:

>Sunshine
This is a sunny day.
>Darkness
A cave is a dark place.
0
0
sed '1,${s/^/>/g;n;n;n}' filename

Testing

cat filename
'Sunshine

'This is a sunny day.

'Darkness

'A cave is a dark place

'Sunshine

'This is a sunny day.

'Darkness

'A cave is a dark place

'Sunshine

'This is a sunny day.

'Darkness

'A cave is a dark place

After I run the command, I get the output as,

sed '1,${s/^/>/g;n;n;n}' new
>'Sunshine

'This is a sunny day.

>'Darkness

'A cave is a dark place

>'Sunshine

'This is a sunny day.

>'Darkness

'A cave is a dark place

>'Sunshine

'This is a sunny day.

>'Darkness

'A cave is a dark place
2
  • Sorry for the late response. I am on a Mac and had to change to a linux system to check your command. it seems to not work for me. a '>' was added to the first line and then after 7 lines or so
    – BSP
    Sep 7, 2014 at 16:20
  • @Ramesh @BSP , please at first clean your data from blank lines via : egrep -v "^$" yoursourcedata > cleaneddata.txt Sep 7, 2014 at 18:31
0

With awk you could try something like,

awk 'NR % 4 == 1 {sub(/^/,">")} {print}' filename

References

https://stackoverflow.com/questions/2099471/add-a-prefix-string-to-beginning-of-each-line

1
  • it seems to not work for me. a '>' was added to the first line and then after 7 lines or so. as i am new to unix commands i also dont see why..
    – BSP
    Sep 7, 2014 at 16:27
0

With no blank lines between each line and no ' character at the start:

$ awk '{print ((NR%2)? ">":"") $0}' passages.txt

gives:

>Sunshine
This is a sunny day.
>Darkness
A cave is a dark place.

Also, going by your responses to all the answers here, your input file isn't single lines with a Line Feed character at the end (\n). It might be worth checking it's source.

1
  • Thanks for the effort. I tried both commands. My file is not double spaced. The first one adds the '>' only every second header. The second one also every second header but also adds a an additional line break
    – BSP
    Sep 7, 2014 at 16:24
0

You can use Vim in Ex mode:

ex -sc '%s/\v(.*\n){2}/> &/|x' file
  1. % select all lines

  2. s substitute

  3. \v turn on magic

  4. x save and close

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .