New answers tagged grep
0
Here's a simpler example that illustrates the issue:
$ cat input.txt
line one
line two
line three
$ for word in $(cat input.txt) ; do echo $word ; done
line
one
line
two
line
three
The $(cat input.txt) splits its input on whitespace. (Incidentally, in bash you can replace that with $(<input.txt)).
You can use the read built-in command instead:
$ while ...
2
First, you don't need cat with grep. That is enough:
x="$(grep ConfigPath /scratch/env.properties)"
Second, I believe this is not an assignment you want:
y=$(ConfigPath=/scratch/a/b)
If you want variable y to hold ConfigPath=/scratch/a/b string it should be:
y="ConfigPath=/scratch/a/b"
$(...) is a command substitution in Bash.
Third, you should use ...
2
grep is already a really fast way to go through big files and find words or characters in lines, maybe the -w word-regexp makes it a bit slow. Often it is not the grep itself which is slow, it is mostly the output on the terminal. You can simply test it by direct the output to a file:
grep -w "for-outbound-sports\|2019-05-16" Master.csv > greped_master....
2
Assuming all of your numbers are, as shown in the examples, fixed-point decimal:
Using grep
$ grep -E '(Thispeculiarpattern|Somerandomtext|Herewegoyetagain)\(([1-9]|0\.[89])' file
Thispeculiarpattern(1.00);thatpeculiarpattern(0.90);....
Somerandomtext(0.81); somemorerandomtext(0.79):.................................
Herewegoagain(0.71);Herewegoyetagain(0....
0
using awk:
awk -F"[\":]" '{ print $5 }' infile
0
Tried with below mentioned sed command and it worked fine too
command: sed 's/.*:"//1' filename|sed 's/".*//g'
output
sed 's/.*:"//1' filename |sed 's/".*//g'
30c962de-b448-40ac-ade8-da6a8f49ce88
0
If this is actually a JSON document, then using jq would pull the string out regardless of the formatting of the data:
jq -r '.document[2].part.id' file.json
This assumes that the id key is part of a part object, which is in turn part of a particular element in a document array at the top level of the JSON data structure.
Or, you could just extract all id ...
0
Another dirty trick using AWK on provided input.
data='"id":"30c962de-b448-40ac-ade8-da6a8f49ce88","title":
echo $data | awk -F , '{ print $1}' | awk -F id: '{print $1}' | awk '{print $1}'
"30c962de-b448-40ac-ade8-da6a8f49ce88"
0
Quick and dirty like this?
$ grep -o -P '(?<=")[0-9a-f]{8}-([0-9a-f]{4}-){3}[0-9a-f]{12}(?=")' input
30c962de-b448-40ac-ade8-da6a8f49ce88
Looks for any random "30c962de-b448-40ac-ade8-da6a8f49ce88" values with fixed length and dash - positions, enclosed in quotation marks, and prints it out. If you want to insist on the "id": part too, just add it to ...
1
Pipe your input through
sed 's/][^[]*\[/,/g;s/\([^,]*,\)\{3\}//;s/ *,/,/;s/]$//'
s/][^[]*\[/,/g does most of the job by removing everything between the [] and separating it with comma
s/\([^,]*,\)\{3\}// gets rid of the first fields you don't use
s/ *,/,/ strips the obsolete spaces
s/]$//' finally removes the last ]
Update:
Your comment suggests, that ...
0
Can't make this work with grep, but I can make it work with egrep:
egrep -o '([[:print:]\d])\1\1([[:print:]]|$)'
That will match 3 consecutive characters, followed by either white space, a single printable character, or an end of line character.
Note: The -o prevents the issue of aBBBf matching and will show only BBBf.
2
The package of grep 3.3 is for the edge release, i.e. the development branch of Alpine Linux. Release 3.9, which you’re using, has grep 3.1.
If you really want it on your 3.9 system, you could always rebuild it by downloading the relevant package source and running abuild as appropriate.
2
Question: Is there a way in which, using the grep command, I can store the result of the command as two whole lines?
Yes, and your assignment code was correct:
usbs=$(lsblk -o NAME,TRAN,VENDOR,MODEL | grep usb)
This does exactly as required; in a single variable, (not an array), it stores two lines from lsblk separated by a newline. But a for loop is not ...
0
If the numbers are separated by blanks (or on each line) this regex will print only (-o) the matching number(s) (no leading zeros accepted):
grep -wE '[1-9][0-9]{4,}|(3[6-9]|[4-9][0-9])[0-9]{2}'
If the numbers could start with one or more zeros, then use this:
grep -wE '0*([1-9][0-9]{4,}|(3[6-9]|[4-9][0-9])[0-9]{2})'
Or, using PCRE:
grep -wP '0*([1-9]\d{...
8
To prevent Bash expansion from passing files starting with “-” you can use:
echo [!-]*
Which works portably in most shells, or, specific to ksh, bash, zsh:
echo !(-*)
For example: in a directory with this files
$ echo *
a b c ---corporate-discount.csv d -e --option.txt
Will list only (provided extglob is active):
$ shopt -s extglob
$ echo !(-*)
a b c ...
4
Use $ for end of line :
grep '\.log$'
0
Using grep
for n in $(grep -Po "[0-9]+" test); do if [ $n -ge 3600 ]; then echo $n; fi; done
If your file only contains integers, because it will report 5999 against 0.5999
Or without the if
grep -Po "[1-9][0-9]{4,}|[4-9][0-9]{3}|3[6-9][0-9]{2}" test
edit
Stole the leading digit strategy [1-9] on 5 digits from @Phillippos above. Answers are basically ...
42
First, note that the interpretation of arguments starting with dashes is up to the program being started, grep or other. The shell has no direct way to control it.
Assuming you want to process such files (and not ignore them completely), grep, along with most programs, recognizes -- as indicating the end of options, so
grep -r -e "stuff" -- *
will do what ...
1
Regex is not good comparing numbers !
Better use some scripting language. In your case awk would do a good job:
awk -F '[^0-9]*' '{for(i=1;i<=NF;i++){ if (int($i)>3600) { print; next; } }}' test.txt
Dependend on your input you should adapt this a bit.
My short example would e.g. not work correctly with negative numbers.
1
With grep, there is no indexing, so each time you want to search for something, the command needs to read through all the logs. When you have a non-trivial amount of logs, this can take a long time and a lot of CPU power.
With elasticsearch, the messages are indexed as they arrive, so the actual searching can happen faster and with less CPU power. ...
2
The extended regular expression ([1-9][0-9]{2,}|[4-9][0-9]|3[6-9])[0-9]{2} should do the job if you can be sure there are no negative numbers, floats, thousand limiters and so on around
The expression has three paths, all of which have in common the last part [0-9]{2}, which means two digits.
First path is a 1 to 9 with at least two more digits ([0-9]{2,}) ...
4
This is mostly a dupe of new lines and bash variable although that doesn't cover arrays.
From there, to use a variable containing multiple lines, you need to make parameter expansion split at newline and skip globbing, and depending on your data possibly avoid other mangling:
usbs=$( lsusb ... )
IFS=$'\n' # ksh bash zsh; in other shells you may need to ...
1
This sounds like a homework assignment.
Following all your requirements for cat, grep and tr, matching upper and lower case (HE=He=he=hE), as well as words ending in punctuation marks (he.=he=he,=he!):
for word in $(
cat derp.txt | \
tr '[:upper:]' '[:lower:]' | \
tr -d '[:punct:]' | \
tr -d '/r'
)
do
words[$word]=$(cat derp.txt | grep -c $word)
...
1
You mention the use of grep, but I don't really see the need for it if the goal is to find the most frequent words. More likely, you just want to split the text file into words, then run your | sort | uniq -c | sort -nr | head -10 pipeline.
The solution that immediately comes to mind (there are without a doubt a number of alternative solutions) is to just ...
10
This is a good situation to use readarray/mapfile:
readarray -t usbs < <(lsblk -o NAME,TRAN,VENDOR,MODEL | grep usb)
This will create an array with your output where each line is separated into it's own element.
In your case it would make an array like:
usbs=(
'sdb usb Kingston DataTraveler 2.0'
'sdc usb Kingston DT 101 G2'
)
As is you ...
1
In case it's about json data (python's json package is known to report that error message), you could identify which json string has control characters in them with:
perl -Mcharnames=:full -C -l -0777 -ne '
while (/"(?:\\.|[^"])*"/g) {
my $offset = $-[0];
my $string = $&;
@ctrl = map {charnames::viacode(ord($_))} $string =~ /\p{PosixCntrl}/...
1
To look for a specific character
grep and sed don't support backslash notation for control characters. (sed does use backslash for regexp backreferences.) If you are using bash it can convert a backslash sequence to the actual control character before passing to these (or any) programs:
$ grep $'\t' file
$ sed -n /$'\t'/p file
$ # or change to l (ell) to ...
0
To see what that character is:
less sourcefile
or
od -c sourceFile
for a more verbose view.
7
As it says on the tin. You create a command name which is egrep '2019-05-11|Total' and then try to call it. This is not a egrep followed by a parameter but a complete command name. What you want is more likely:
totalSize=$(echo $s3ls| egrep "$currentDate|Total" | awk -F 'Total Size:' '{print $2}'|sed '/^$/d')
If necessary you can use a variable to hold the ...
5
You could use option -o to only print the matching part in combination with -E (extended regular expression)
and pattern .{0,4} to match up to four characters after your search pattern:
$ echo "foo123456789" | grep -oE 'foo.{0,4}'
foo1234
Or you could pipe the result to head -cN (similar to cut) to print N characters:
$ echo "foo123456789" | grep 'foo' | ...
1
grep itself has no such option. You can achieve your goal with the cut command:
grep "foo" bar | cut -c -4
This will only print the first 4 chars of every grep result.
4
Unix V7 was released in the late 70s. That's the version that introduced the Bourne shell.
However, at that time, function support hadn't been added yet, read didn't have -r, there was no printf command. Case insensitive grep was with grep -y. And of course $(...) is not Bourne.
Unix-like systems have evolved quite a bit since then and diverged. In the ...
0
Your script almost already does what you want. You don't need the grep capture which just complicates the printing:
#!/bin/bash
for j in *_seqs.txt; do
while read line; do
grep "^$line" "$j"
done < "$1"
done
1
You just need to escape the or:
function get_clean_bash_history(){
grep -v '^cd \|^ls ' "$HOME/.bash_history"
}
1
Looks like the -E flag was necessary to use with grep, this appears to work:
grep -v -E '^cd +|^ls +'
but please correct me if this is not sufficient or fails in edge cases
0
This answer assumes that the input file is regular text. Alternatively, if the input file is actually XML or JSON or similar, then tools specific to that format should be used.
Try:
awk 'BEGIN{a[0];a[1];a[2];a[3];a[6];a[7];a[10];a[11]} /x/{n=NR} n && (NR-n) in a'
How it works:
BEGIN{a[0];a[1];a[2];a[3];a[6];a[7];a[10];a[11]}
This initializes ...
0
with awk
dd if=/dev/zero of=tempfile bs=1M count=1024 |awk 'END{print $(NF-1)" "$NF}'
1
If you want to grab the part containing the number and MB/s (or GB/s), you could use
dd if=/dev/zero of=tempfile bs=1M count=1024 2>&1 | grep -o '[0-9.]\+ .B/s$'
You need to redirect stderr to stdout, because dd writes this information to stderr.
1
In sed you would used extended regular expression with option -E:
sed -E '/group-title="TR (ULUSAL|HABER|BELGESEL)"/!d;n;n'
The () around the ULUSAL|HABER|BELGESEL make sure the | only applies for the patterns inside.
A line with neither pattern (the ! behind the pattern) gets deleted. All other lines (those with the patterns) execute the n;n, so the line ...
0
Assuming the file is called inputfile, and the output should be stored in outputfile:
grep --no-group-separator -A 2 -E \
"group-title=\"TR ULUSAL\"|group-title=\"TR HABER\"|group-title=\"TR BELGESEL\"" \
inputfile > outputfile
This works by using extended regular expressions (-E), and after-context (-A).
1
Looks like the --exclude-from=FILE flag is what you're looking for. You'll have to create and populate manually each project-specific FILE with entries like :
onedir/*
twodir/*
For convenience, you can also create a shell alias so that running grep will automatically call the --exclude-from=FILE flag.
3
There are various issues here. First of all, the expression ^[ ]w means: find the start of the line, then exactly one space, then a w. So it's actually working perfectly. If you want it to match one or more spaces, you need to add a qualifier to the [ ] character class:
$ grep '^[ ]\+w' text.txt
whitespace 1
whitespace 2
The + means "one or more". ...
0
Correct command :
Use grep -E '^[ ]{0,}' text.txt
-E, --extended-regexp Interpret PATTERN as an extended regular
expression (ERE, see below).
Reason why its not working:
Not using single quotes around regex, bash will open it and your command will become
grep '^[' ] ]0 text.txt which translates to grep with regex '^[' on files ] , ]0 and text....
1
In BRE, in the greedy quantifiers expression {0,}, the braces need to be escaped to achieve the required regex match and always quote your regex string. Without the quotes, the shell tries to apply its own parsing grammar to the provided arguments and in most cases the arguments get word split making grep only see the ^[ part of the regex.
grep '^[ ]\{0,\}w'...
0
Another way of word boundary :
sed -i 's/\bphp\b/vinu/g' file
0
Use word boundaries in the regular expression if your version of sed supports them, e.g.
sed -i 's/\<php\>/vinu/g'
25
Remove w from the options:
grep --include=\*.rb -rn . -e enforc
-w, --word-regexp
Select only those lines containing matches that form whole words. The test is that the matching substring must either be
at the beginning of the line, or preceded by a non-word constituent
character. Similarly, it must be either at the end of the ...
0
Try this:
sed -ne '/^.\{43\}$/s/\t/, /g;/^.\{44\}$/s/\r/./gp' input.file > new.file
or if you're on mac os:
sed -ne $'/^.\{43\}$/s/\t/, /g;/^.\{44\}$/s/\r/./gp' input.file > new.file
I added a space after the comma to match the expected output. That's why the second match is for 44 rather than 43 characters.
0
This is the solution from the top answer by @NominalAnimal but with the usual grep: ... in warnings (instead of /bin/grep: ...):
#!/bin/bash
exec -a grep /bin/grep --color=auto "$@"
1
steeldriver posted the answer as a comment, so I’m posting it as an answer.
echo "FLAG" | base64 outputs RkxBRwo=,
so yourgrep -nr `echo "FLAG" | base64`command
is doinggrep -nr RkxBRwo=But, based on what you’ve showed,
your test.txt file doesn’t contain RkxBRwo=;
it contains RkxBR (followed by other characters).
If you want to search your files
for the ...
Top 50 recent answers are included
