3

I have a huge data file.txt looks like :

calls...
12311 34213 13344 12345 34532
23345 24445 22445 12344 12333
34456 22211 12334 12234 23344
23345 24445 22445 12344 12333

which I want to delete first row(which is written calls... there).

I used this command:

sed '1d' input.txt > output.txt

which deletes "calls..." . But the problem is that some other rows gets deleted as well. Does any body has any other suggestion that help me to delete calls... without deletion of the other rows inside? my real data has 117,000 rows and 10,000 column when I use this command then the number of rows gets 68,645 , while it must get 116,999. when I try sed in small data like the example I showed here, the number of rows does not change, while in my real data it does. I am really confused why?

  • What's your sed version? – cuonglm Sep 28 '15 at 7:09
  • 3
    sed 1d is really a right answer for deleting the first line. What you get doesn't make sense. Is it possible that your first line does not end in \n and so a bunch of 48355 lines is considered to be the "first line"? I'd like to see your data (or any other file that exhibits the same problem) – Ángel Sep 28 '15 at 9:21
3

You could use tail

tail -n +2 input.txt > output.txt

will print the lines of the file starting by the second (note the + sign)

  • @ adonis , I updated my post. could you look at it? – zara Sep 28 '15 at 5:26
1

You could also use awk command:

Example 1:

awk '{gsub("^calls\.+", "");print}' input.txt > output.txt

Will delete lines from input.txt and put in output.txt.

Example 2:

awk '!/^calls\.+/{print}' input.txt > output.txt

This will print all the lines except the pattern provided in awk command.

0

Use Pattern Addressing

With GNU sed, you can use pattern addresses to limit your commands. For example:

sed -r '/^calls\.{3}$/d' input.txt

will delete only lines that start with a matching pattern.

0

I haven't done a lot of editing of files that large, but given that sed is intended as a STREAM editor it may be ill-suited for the purpose, as if I understand correctly the entire contents of the file will have to be piped through sed unnecessarily. Assuming it is just the one file you are dealing with and you are not trying to embed this behavior in a complicated script, I would simply copy the input file to output, and then use vi to remove the first line. vi does better than most editors (e.g. gedit) on huge files and this seems to be a case where vi would be the best choice. (And yes, I'm a big fan of vi in the first place ;)

cp input.txt output.txt
vi output.txt
# Within vi, type:
ggddZZ
# gg goes to the first line (if you're not there already); dd deletes
# the line you're on; ZZ saves the file and exits.
# Voila, you're done!
  • 1
    vi still needs to process and rewrite the whole file. Actually, for simply deleting the first line it will be worse than a plain sed, specially if it has to syntax highlight your file. – Ángel Sep 28 '15 at 9:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.