Questions tagged [regular-expression]

Regular expressions are a means of matching a pattern of characters within a string.

70
votes
1answer
8k views

Why does my regular expression work in X but not in Y?

I wrote a regular expression which works well in a certain program (grep, sed, awk, perl, python, ruby, ksh, bash, zsh, find, emacs, vi, vim, gedit, …). But when I use it in a different program (or on ...
84
votes
8answers
164k views

How to run grep with multiple AND patterns?

I would like to get the multi pattern match with implicit AND between patterns, i.e. equivalent to running several greps in a sequence: grep pattern1 | grep pattern2 | ... So how to convert it to ...
225
votes
14answers
313k views

How can I use sed to replace a multi-line string?

I've noticed that, if I add \n to a pattern for substituting using sed, it does not match. Example: $ cat > alpha.txt This is a test Please do not be alarmed $ sed -i'.original' 's/a test\nPlease ...
27
votes
2answers
11k views

Does (should) LC_COLLATE affect character ranges?

Collation order through LC_COLLATE defines not only the sort order of individual characters, but also the meaning of character ranges. Or does it? Consider the following snippet: unset LANGUAGE ...
22
votes
6answers
17k views

How to select first occurrence between two patterns including them

How can I select first occurrence between two patterns including them. Preferably using sed or awk. I have: text something P1 something content1 content2 something P2 something text something P1 ...
265
votes
8answers
268k views

Can grep output only specified groupings that match?

Say I have a file: # file: 'test.txt' foobar bash 1 bash foobar happy foobar I only want to know what words appear after "foobar", so I can use this regex: "foobar \(\w\+\)" The parenthesis ...
10
votes
2answers
1k views

What is the definition of a regular expression?

I recently got into a friendly argument with Ghoti about what constitutes a regular expression in the comments to my answer to this question. I claimed that the following is a regular expression: `[...
23
votes
9answers
72k views

How can I “grep” patterns across multiple lines?

It seems I am misusing grep/egrep. I was trying to search for strings in multiple line and could not find a match while I know that what I'm looking for should match. Originally I thought that my ...
587
votes
13answers
1.3m views

How do I grep for multiple patterns with pattern having a pipe character?

I want to find all lines in several files that match one of two patterns. I tried to find the patterns I'm looking for by typing grep (foo|bar) *.txt but the shell interprets the | as a pipe and ...
264
votes
5answers
294k views

What is the difference between `grep`, `egrep`, and `fgrep`?

Can anyone tell me the technical differences between grep, egrep, and fgrep and provide suitable examples? When do I need to use grep over egrep and vice versa?
12
votes
3answers
17k views

Splitting text files based on a regular expression

I have a text file which I want to split into 64 unequal parts, according to the 64 hexagrams of the Yi Jing. Since the passage for each hexagram begins with some digit(s), a period, and two newlines, ...
6
votes
1answer
5k views

find words with all vowels in order from a file

I have a large dictionary file with 300,000+ words in it and I'm trying to find all words with the vowels aeiou in that order and have only exactly 5 vowels. My current attempt does not seem to be ...
21
votes
5answers
36k views

In a regular expression, which characters need escaping?

In general, which characters in a regular expression need escaping? For example, the following is not syntactically correct: echo '[]' | grep '[]' grep: Unmatched [ or [^ This, however, is ...
15
votes
2answers
31k views

sed one-liner to delete everything between a pair of brackets?

I am working with some text that is full of stuff between brackets [] that I don't want. Since I can delete the brackets myself, I don't need the one-liner to do that for me, but I do need a one-liner ...
10
votes
2answers
2k views

Why do I need to escape regex characters in sed to be interpreted as regex characters?

It seems e.g. cat sed_data.txt | sed 's/\b[0-9]\{3\}\b/NUMBER/g' that I must escape characters to form a regular expression. In this case I had to escape braces in order to be interpreted as a number ...
5
votes
2answers
3k views

How does storing the regular expression in a shell variable avoid problems with quoting characters that are special to the shell?

From Bash Manual Storing the regular expression in a shell variable is often a useful way to avoid problems with quoting characters that are special to the shell. It is sometimes difficult to ...
7
votes
2answers
11k views

Shell test to find a pattern in a string

I just wanted to ask whether there is any command which would work on common shells (bash, dash, kornshell)? It is supposed to check if the line variable contains any part of the path. if [[ $line =~ ...
5
votes
3answers
2k views

Search and replace strings that are not substrings of other strings

I have a list of replacements like so: search_and -> replace big_boy -> bb little_boy -> lb good_dog -> gd ... I need to make replacements for the above, but at the same time avoid ...
5
votes
1answer
2k views

Renaming files to have lower case extensions with 'rename'

I'm trying to currently rename a large set of files and have been using quite kludgy methods to do so, such as: rename 's:(.*)\.MOV:$1.mov:g' *.MOV rename 's:(.*)\.JPG:$1.jpg:g' *.JPG What I'd ...
115
votes
5answers
124k views

How to use find command to search for multiple extensions

I can get all jpg images by using: find . -name "*.jpg" But how can I add png files to the results as well?
18
votes
2answers
35k views

rsync certain files, excluding the rest, ignoring .svn/ directory, recursively

I'm using rsync to copy some files from a share to another. Recursively, I need to: Delete files at destination that were removed from the origin Only sync .php and .js files Exclude every other ...
42
votes
6answers
13k views

Replacing Multiple blank lines with a single blank line in vim / sed

Question more or less says it all. I'm aware that /^$/d will remove all blank lines, but I can't see how to say 'replace two or more blank lines with a single blank line' Any ideas?
24
votes
6answers
101k views

Checking if an input number is an integer

I'm trying to check if an input is an integer and I've gone over it a hundred times but don't see the error in this. Alas it does not work, it triggers the if statement for all inputs (numbers/letters)...
12
votes
2answers
29k views

find command with regex quantifier e.g. {1,2}

I have been trying to create a find command string that will find all files that end with a number 1-99 but exclude all others. e.g. I want to find myfile1 myfile99 but not myfile456 and not ...
26
votes
2answers
56k views

grep and escaping a dollar sign

I want to know which files have the string $Id$. grep \$Id\$ my_dir/mylist_of_files returns 0 occurrences. I discovered that I have to use grep \$Id$ my_dir/mylist_of_files Then I see that the $...
12
votes
5answers
12k views

Non-greedy match with SED regex (emulate perl's .*?)

I want to use sed to replace anything in a string between the first AB and the first occurrence of AC (inclusive) with XXX. For example, I have this string (this string is for a test only): ...
22
votes
2answers
14k views

Grep word boundaries

Accorging to GNU documentation: ‘\<’ Match the empty string at the beginning of word. ‘\>’ Match the empty string at the end of word. My /etc/fstab looks like this: /dev/sdb1 /media/...
17
votes
3answers
2k views

awk repetition {n} is not working

I am trying to print the lines using the repetition symbol {n} but it is not working. For. e.g. I want to print all lines whose length is 4 char long awk '/^.{4}$/' test_data The above code is not ...
2
votes
1answer
688 views

How to nest global matches with sed?

If I do: sed 's/match/replace/g' I know that sed will substitute replace for every occurrence of match on a line. But what if...? echo "match <please dont match this?>" | sed 's/match/replace/...
2
votes
4answers
3k views

Replace text between brackets

I'm using awk '{ gsub(/BAR|WIBBLE/, "FOO"); print }' to replace text in data like: SOMETHING [BAR, WIBBLE] SOMETHING [BAR] This gives the desired result of: SOMETHING [FOO, FOO] SOMETHING [FOO] ...
2
votes
1answer
4k views

What is the email matching regex in basic regex for grep?

I created a text file and put some email addresses in it. Then I used grep to find them. Indeed it worked: # pattern="^[a-zA-Z0-9]+@[a-zA-Z0-9]+\.[a-z]{2,}" # grep -E $pattern regexfile but only as ...
2
votes
1answer
39 views

How does find -name work?

I can't for the life of me figure out how find with the test -name works. I run find / -name *in and returns a bunch of results: /sbin /sbin/sulogin /dev/stdin to name a few. It's as if it ...
2
votes
3answers
21k views

grep to find words with all vowels [duplicate]

Using grep -E and regular expressions I need to find words containing all of a group of characters in no particular order. For instance a word containing all vowels (aeiou) however the vowels can ...
0
votes
2answers
225 views

Problems with regex in grep

I'm trying to remove malware code from a site using a script, I'm having trouble getting the regex right for the malware part. What I want to do eventually is to use sed in order to find the ...
36
votes
8answers
54k views

Simple sed replacement of tabs mysteriously failing

This ought to be really simple, but for some reason it is not working: sed -i.bak -E 's/\t/ /' file.txt Instead of replacing tab characters, it's replacing t characters. I've tried every variation ...
41
votes
4answers
173k views

grep lines starting with “1” in Ubuntu

I try to search for lines that start with "1" using ls -1 | grep ^1* but it returns lines that do not start with 1. What I am missing here?
35
votes
4answers
14k views

Difference between [0-9], [[:digit:]] and \d

In the Wikipedia article on Regular expressions, it seems that [[:digit:]] = [0-9] = \d. What are the circumstances where they do not equal? What is the difference? After some research, I think ...
42
votes
7answers
70k views

Find files in multiple folder names

I am trying to list all the files from dir1, dir2, dir3 and dir4 which might be anywhere in as a sub directory of my cwd using the find command. I tried the following with no success: find . -type f -...
13
votes
1answer
2k views

Why does this BSD grep result differ from GNU grep?

My computer is running macOS 10.12.3 and I am using the system-installed grep utility with version 2.5.1-FreeBSD. These are the outputs I get when testing various regexes: But if I run these using ...
17
votes
4answers
90k views

Finding text between two specific characters or strings

Say I have lines like this: *[234]* *[23]* *[1453]* where * represents any string (except a string of the form [number]). How can I parse these lines with a command line utility and extract the ...
17
votes
1answer
1k views

How to “grep” for line length *not* in a given range?

NOTE: This question is the complement of this Q&A: How to "grep" for line length in a given range? How can we grep for lines that have less than 8 or more than 63 characters, none that ...
17
votes
3answers
11k views

What is the regex to validate Linux users?

When adding a new user, how is the string validated? I suppose there is a regular expression. What is that regular expression?
10
votes
3answers
9k views

Strange behaviour of “ls -a | grep ^\.”

I wanted to list the content of a pwd and display only file starting with dot. I tried ls -a | grep ^\. but I cannot figure out why the output contains also the files which do not start with dot. For ...
9
votes
1answer
35k views

Using regex inside if clause in bash

Look at this if block: #!/bin/bash str="m.m" if [[ "${str}" =~ "m\.m" ]]; then echo "matched" else echo "not matched" exit 1 fi exit 0 This should print "matched", but it doesn't. ...
8
votes
2answers
23k views

Regular expression problem(s) in Bash: [^negate] doesn't seem to work

When I execute ls /directory | grep '[^term]' in Bash I get a regular listing, as if the grep command is ignored somehow. I tried the same thing with egrep, I tried to use it with double and single ...
5
votes
5answers
23k views

Extracting IP address from a text and store it in a variable

I have a text file named abd shown below. 48878 128.206.6.136 34782 128.206.6.137 12817 23.234.22.106 I want to extract only IP address from the text and store it in a variable and use for other ...
4
votes
3answers
27k views

Bash =~ regex and https://regex101.com/

Using https://regex101.com/ I built a regular expression to return the first occurrence of an IP address in a string. RegExp: (?:\d{1,3}\.)+(?:\d{1,3}) RegExp including delimiters: /(?:\d{1,3}\.)+(...
10
votes
4answers
3k views

Regular expression using \\ vs using \

Why does grep e\\.g\\. <<< "this is an e.g. wow" and grep e\.g\. <<< "this is an e.g. wow" do the same thing? If I add a third slash, it also has the same result. BUT, once I ...
7
votes
1answer
1k views

Why are capital letters included in a range of lower-case letters in an awk regex?

$ echo ABC | awk '$0 ~ /^[a-b]/' ABC $ echo ABC | awk '$0 ~ /^[a-a]/' $ echo ABC | awk '$0 ~ /^a/' $ You see. /[a-b]/ captures A, but /[a-a]/ or /a/ doesn't. Why?
6
votes
1answer
514 views

What is the meaning of this test expression []? [duplicate]

I've seen someone use command: ps -ef | grep [h]ttpd and Output is: apache 25125 31006 0 21:54 ? 00:00:00 /usr/sbin/httpd apache 26869 31006 0 22:04 ? 00:00:00 /usr/sbin/httpd ...