75

I have to grep through some JSON files in which the line lengths exceed a few thousand characters. How can I limit grep to display context up to N characters to the left and right of the match? Any tool other than grep would be fine as well, so long as it available in common Linux packages.

This would be example output, for the imaginary grep switch Ф:

$ grep -r foo *
hello.txt: Once upon a time a big foo came out of the woods.

$ grep -Ф 10 -r foo *
hello.txt: ime a big foo came of t
4

4 Answers 4

54

Try to use this one:

grep -r -E -o ".{0,10}wantedText.{0,10}" *

-E tells, that you want to use extended regex

-o tells, that you want to print only the match

-r grep is looking for result recursively in the folder

REGEX:

{0,10} tells, how many arbitrary characters you want to print

. represents an arbitrary character (a character itself wasn't important here, just their number)

Edit: Oh, I see, that Joseph recommends almost the same solution as I do :D

5
  • Thank you. Even though it is essentially the same solution, it is confidence-inspiring that this is the best method when two people independently recommend it.
    – dotancohen
    Oct 23, 2014 at 7:37
  • You're welcome, Unix community simply must cooperate, that's what we are :-)
    – Eenoku
    Oct 23, 2014 at 7:42
  • 2
    Although they are similar the accepted answer didn't work for me (still produced long lines), but one this did. The trick with N=10 doesn't work with a bash shell.
    – meesern
    May 6, 2015 at 8:58
  • 1
    in cygwin -E is significantly faster than -P.
    – Bob Stein
    Apr 24, 2019 at 14:42
  • 1
    this suggestion works on Mac, the suggestion from Joseph R. unfortunately doesn't, although they are pretty close to each other
    – Timofey
    Jun 12, 2020 at 0:06
51

Piping stdout to cut with the -b flag; you can instruct grep's output to only bytes 1 through 400 per line.

grep "foobar" * | cut -b 1-400
6
  • 11
    So much better than all that regex nonsense!
    – dtmland
    Aug 12, 2020 at 18:42
  • 9
    This only gives the first 400 bytes of the matching line. It does not give N bytes before and after the match. Even worse, suppose 'foobar' appears after the 400th byte, then you wouldn't even see the match in the output!
    – joseph
    Feb 3, 2021 at 19:57
  • 2
    @joseph yes but the only reason I use this command to limit the output of grep, is because sometimes a match is found half way through a 5 gigabyte json file where everything is on a single line, and so grep returns 5000 pages of stdout which is useless to me. I'd rather only see the first 400 bytes, if the highlight isn't there, I know this from a massive file, and most times it's not what I want anyway. Feb 3, 2021 at 20:36
  • doesn't really answer the question.
    – mendota
    Mar 16 at 20:17
  • This is computationally WAY faster than using a modified regex as in the accepted answer -- in my tests, about 10-20X faster than a regex matching 10 chars on either side, and 130–360X faster than a regex matching 255 chars on either side! 255 chars is the maximum number one can match for a single character on some systems, but cut allows more context to be preserved (except for very long lines where cut may truncate away the part with the match). This solution is therefore far more practical in many use cases involving large datasets.
    – DavidArndt
    Jun 1 at 18:52
40

With GNU grep:

N=10; grep -roP ".{0,$N}foo.{0,$N}" .

Explanation:

  • -o => Print only what you matched
  • -P => Use Perl-style regular expressions
  • The regex says match 0 to $N characters followed by foo followed by 0 to $N characters.

If you don't have GNU grep:

find . -type f -exec \
    perl -nle '
        BEGIN{$N=10}
        print if s/^.*?(.{0,$N}foo.{0,$N}).*?$/$ARGV:$1/
    ' {} \;

Explanation:

Since we can no longer rely on grep being GNU grep, we make use of find to search for files recursively (the -r action of GNU grep). For each file found, we execute the Perl snippet.

Perl switches:

  • -n Read the file line by line
  • -l Remove the newline at the end of each line and put it back when printing
  • -e Treat the following string as code

The Perl snippet is doing essentially the same thing as grep. It starts by setting a variable $N to the number of context characters you want. The BEGIN{} means this is executed only once at the start of execution not once for every line in every file.

The statement executed for each line is to print the line if the regex substitution works.

The regex:

  • Match any old thing lazily1 at the start of line (^.*?) followed by .{0,$N} as in the grep case, followed by foofollowed by another .{0,$N} and finally match any old thing lazily till the end of line (.*?$).
  • We substitute this with $ARGV:$1. $ARGV is a magical variable that holds the name of the current file being read. $1 is what the parens matched: the context in this case.
  • The lazy matches at either end are required because a greedy match would eat all characters before foo without failing to match (since .{0,$N} is allowed to match zero times).

1That is, prefer not to match anything unless this would cause the overall match to fail. In short, match as few characters as possible.

5
  • 1
    Very nice, thank you. This has the drawback of highlighting the entire output, not just the searched for text, but that can be worked around by appending | grep foo to the end (however loosing the filename highlighting in the process).
    – dotancohen
    Oct 23, 2014 at 7:36
  • 1
    @dotancohen I guess you can't win them all :)
    – Joseph R.
    Oct 23, 2014 at 7:47
  • w/ GNU grep you can specify match colors/applications based on flags applied via environment variables. so maybe even you could win em all, (no promises - not even sure it would work in this case) but i dont personally see the relevance here... anyway... keep playing.
    – mikeserv
    Oct 23, 2014 at 9:06
  • Nice answer. Just a note, using zsh I am unable to get it to work passing N=10 as in the example. However it does work if I export N=10 prior to running the command. Any idea how to adjust the example to work with zsh? Nov 17, 2015 at 23:37
  • Or perl -lne 'print "$ARGV: $_" for /.{0,10}foo.{0,10}/g' Apr 19, 2016 at 12:21
6

Taken from: http://www.topbug.net/blog/2016/08/18/truncate-long-matching-lines-of-grep-a-solution-that-preserves-color/ and https://stackoverflow.com/a/39029954/1150462

The suggested approach ".{0,10}<original pattern>.{0,10}" is perfectly good except for that the highlighting color is often messed up. I've created a script with a similar output but the color is also preserved:

#!/bin/bash

# Usage:
#   grepl PATTERN [FILE]

# how many characters around the searching keyword should be shown?
context_length=10

# What is the length of the control character for the color before and after the matching string?
# This is mostly determined by the environmental variable GREP_COLORS.
control_length_before=$(($(echo a | grep --color=always a | cut -d a -f '1' | wc -c)-1))
control_length_after=$(($(echo a | grep --color=always a | cut -d a -f '2' | wc -c)-1))

grep -E --color=always "$1" $2 | grep --color=none -oE ".{0,$(($control_length_before + $context_length))}$1.{0,$(($control_length_after + $context_length))}"

Assuming the script is saved as grepl, then grepl pattern file_with_long_lines should display the matching lines but with only 10 characters around the matching string.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .