I want to replace spaces (i.e., " ") or new lines (i.e., carriage return) with underscores in a special case - when they occur between two specific strings.

I have html pages and I want to replace the blank spaces and new lines with underscores when they occur between two specific strings.


lots of text...
page_5.html months ago

This is the password: 6743412 <http://website.com etc...
more text...

I want to to go from above to below:

lots of text...
page_5.html months ago__This_is_the_password:_6743412_<http://website.com etc...
more text...

Basically, I want to do the replacement only between the strings ago and <http

It is repetitive html so if I can get this to work it would be very helpful and easy to extract the modified text later.

Something using sed or awk would be best for me.

  • between strings "ago" and what? btw, newlines aren't carriage-returns (^M or \r), they are linefeeds (^J or \n). – cas May 8 '16 at 3:06

Assuming you mean 'between "months ago" and "http://"', thisperl script does what you want:

#! /usr/bin/perl

use strict;

my $join=0;

while(<>) {
    if (m/ ago$/) { $join=1 };
    if (m/http:\/\//) { $join=0 ; s/[[:blank:]]/_/g; };
    if ($join == 1) {
        s/_(seconds|minutes|hours|days|weeks|months|years|ago_)/ $1/g;

NOTE: I have deliberately used [[:blank::]] rather than \s in the first s/// operation above so that it only changes spaces and tabs on that line, not newlines. With the second substition, I want it to replace all kinds of spaces, including newlines (except for the space immediately preceding months ago and the space between months and ago).


lots of text...
page_5.html months ago___This_is_the_password:_6743412_<http://website.com_etc...
more text...

This could also be written as a one-liner, using perl -p:

perl -p -e 'if (m/ ago/) { $join=1 }; if (m/http:\/\//) { $join=0 ; s/[[:blank:]]/_/g; }; if ($join == 1) {s/\s/_/g}; s/_(seconds|minutes|hours|days|weeks|months|years|ago_)/ $1/' speld.txt
|improve this answer|||||
  • Hi, it actually has to be " ago" as the first string. There is a space before the word ago. Sometimes its months ago, years ago - it varies. – speld_rwong May 8 '16 at 3:45
  • i've updated the script to work with various time units, not just months. – cas May 8 '16 at 3:55

A Python snippet:

#!/usr/bin/env python3
import re
with open('file.txt') as f:
    f = f.read()
    spl = re.split(r'(\sago\n|<http://)', f)
    des = re.sub(r'\s', '_', spl[2])
    print(spl[0] + spl[1].replace('\n', '') + des + ''.join(spl[3:]), end='')

The file is split into parts taking ago\n and <http:// as delimeters using re.split. Then the value at index 2 is the desired portion where the replacements need to be done. Finally the file is printed in desired pattern.

|improve this answer|||||


perl -0777 -pe 's{(?<=\bago\b)(.+)(?=<http)}{ ($repl=$1) =~ s/\s/_/g; $repl }sge' file


  • -0777 slurps the file into a single string
  • the "outer" s/// command finds all the substrings you want to modify
  • the "inner" s/// command replaces all whitespace with underscores.
|improve this answer|||||

use vim, one command from your terminal will do it:

vim -c '%s/ ago\n*/ ago__/g|wq' original.txt

original.txt gets modified, so make sure you back it up first!

|improve this answer|||||

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.