Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I am trying to remove the first and the last characters of everyline in a text file and save the resulting truncated version in a new file. Does anyone have an idea about how to do that efficiently using awk or other linux programs/commands specifically for large files?

input.txt

(s,2,4,5,6)
"s,1,5,5,2"
{z,0,4,5,3}
[y,2,4,5,5]
(y,4,4,5,7)
(r,20,4,5,7)
(e,9,4,5,2)

Expected output.txt

s,2,4,5,6
s,1,5,5,2
z,0,4,5,3
y,2,4,5,5
y,4,4,5,79
r,20,4,5,7
e,9,4,5,2
share|improve this question

4 Answers 4

up vote 8 down vote accepted

Another way just for the heck of it:

rev input | cut -c2- | rev | cut -c2-

share|improve this answer
    
Nice! This is significantly faster than the sed and awk solutions proposed so far. –  Gilles Jan 10 '13 at 23:13
    
I proposed this answer for people afraid of sed/awk/regex syntax, but wouldn't have imagined it's faster, especially for large files, with three pipes and passing the entire contents through each one. Would have thought sed or awk reading a line at a time would be more efficient for large files. –  Drake Clarris Jan 14 '13 at 13:35
1  
Guess that's what 40+ years of optimization of many of these *nix utilities will get you! –  Drake Clarris Jan 14 '13 at 13:36
    
@Gilles, it's faster to GNU sed in utf8 locales for some forms of input, and it depends whether you're considering wall clock time, or CPU time. ssed or the Heirloom toolchest sed can achieve better performance. –  Stéphane Chazelas Jan 18 '13 at 15:09

There are many possibilities, as always

sed 's,.\(.*\).$,\1,g' your_file

Explanation

  • , -- the sed delimiter, can be any other character as well, given it is escaped where-ever needed.
  • . Match a single character
  • \(.*\) - Group the remaining part, and this is stored to be retrieve further.
  • . Match a single character again
  • $ - End of line
  • \1 - output the text matched by group above
  • g replace globally on the line.
share|improve this answer
2  
Why g? there will be only one match per line. –  njsg Jan 10 '13 at 10:07
    
Note that it won't remove anything from lines that have fewer than 2 characters. –  Stéphane Chazelas Jan 18 '13 at 14:46

As per you question delete last and first word from input file as below :

sed 's/.$//; s/^.//' inputfile
share|improve this answer
    
It would be cool if you could benchmark these against the other solution, s/.\(.*\).$/\1/. It might be faster on account of not using backreferences, and the question did mention "large files". –  l0b0 Jan 10 '13 at 13:52
2  
@l0b0 I tested with time yes | head -n 10000000 | COMMAND >/dev/null. I get rev input | cut -c2- | rev | cut -c2- → 0.14s, sed 's,.\(.*\).$,\1,' → 3.38s; awk '{print substr($0,2,length()-2);}' → 3.50s; sed 's/.$//; s/^.//' → 5.09s. –  Gilles Jan 10 '13 at 23:09
    
@Gilles +1 That should be an answer. –  l0b0 Jan 11 '13 at 7:58
1  
@Gilles, that's very short lines. I find that for 30 character wide lines, @RahulPatil's solution is 3 times as fast with GNU sed than @juampa's. Also. sed 's/.\(.*\)./\1/' appears to be faster than sed 's/^.\(.*\).$/\1/' (GNU sed again). Also, the performance is dependant on the locale (interpretation of what a character is) and the sed implementation (in that regard, sed from the heirloom toolchest is considerably faster than GNU sed). –  Stéphane Chazelas Jan 18 '13 at 15:00

You can also do it with awk if you prefer

awk '{print substr($0,2,length()-2);}' input.txt > output.txt
share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.