New answers tagged grep
0
There's probably a better way to do this with something like jq, but I've never mastered that tool. For me, I'd use Python for this. Given a valid JSON file based on the partial example that you provided:
{
"x-amazon-apigateway-integration1": {
"uri": "http://Test123.elb.us-east-1.amazonaws.com:8765/remote",
...
0
please note that grep is searching file content while first argument is search PATTERN and other arguments interpreted as FILES to look into
it becomes more clear to you when using grep -H -o flags or put your grep inside a script and run it with bash -x script to see how shell globs expanded before passed as arguments
3
In regexes, * means "any number of the previous item", not "any number of any characters", like it does in shell patterns. And . means "any single character".
So, to look for "anything, followed by literal .txt", you'd use .*\.txt. Or just \.txt, since usually regex matches search for the match anywhere in the line.
...
1
It is in part because grep uses regular expressions (in fact, that's what the re in the name stands for- it's short for global regular expression print).
The * wildcard in regular expressions is different from the * wildcard in shell globbing.
In regular expressions, * means "zero or more of the previous defined object". However, . is also a ...
0
You can use below awk command
awk 'NR==FNR{a[$1];next}($1 in a){print $0}' file1 file2
0
This might be what you want, using any awk in any shell on every UNIX box:
$ cat tst.awk
/^[0-9]/ { prt() }
{ rec = rec $0 ORS }
END { prt() }
function prt() {
if ( rec ~ regexp ) {
printf "%s", rec
}
rec = ""
}
.
$ awk -v regexp='issue' -f tst.awk file
1 ok device issue Some Action which
has ...
0
grep is acting like it should in default mode. From its man page:
...grep searches for PATTERNS in each FILE. PATTERNS is one or more
patterns separated by newline characters, and grep prints each line
that matches a pattern...
So, it's supposed to turn up lines in the text matching a regex. Lines are demarcated by the newline control code, which explains ...
0
Assuming a "record" in the input file is exactly as provided by the OP:
$ sed '/issue/!d; :a; n; /^[0-9]\{1,\} /d; $!ba' file
1 ok device issue Some Action which
has to be taken which
is split into many lines
under d.
$
1
You need multi line grepping here. For which we will need the PCRE enabled -P
option. Since grep will output Null delimited records in the slurp -z mode we remove those via tr command.
$ < file grep -Pzo '.*\S.*issue.*\n(?:\h+.*\n)+' | tr -d '\0'
1
If you can be sure that the first character is a letter (you did not check for that either):
grep '^[^#].*[0-9]' file.txt
0
One approach to this problem is to escape all the regex characters in file1
and in the next step constrain the string from both ends. Note that the ordering of the steps is imp.
This can be referred to as a hybrid regex approach wherein the actual string to be matched has all it's regex BRE chars escaped away so that it becomes a literal string whilst in the ...
0
3.2.8 is at least 9 years old and many versions behind the current version.
The issue is that the - sets it to one mode but the x is a different mode and its confused about which one you want. I think we fixed this (by making it care less, so strictly less correct but more what users expect) in 3.3.4 released in 2012.
ps xef is strictly correct and will ...
0
Try this command: grep "COW"$ masternospaces.txt
4
Assuming GNU grep (the default on Linux) you could use PCRE mode and negative lookbehinds:
$ grep -niP '(?<!de)bug|(?<!other)stuff|error' dummy.log
3:!bug
6:12345 DEBUG bug occured
7:please report BUG to me
8:the filename is critical_bug.log
9:bug should be fix.
11:throws error
12:a stuff
14:c otherstuff stuff
The options used are:
-n, --line-number
...
1
It probably complains because x appears to be what the manpage calls a "BSD option" and as such, it shouldn't take a dash (i.e., it's x, not -x). Not that -x seems to exist anyway, so who knows how it interprets that. Maybe as a synonym for -x, which is what appears to be in e.g. current FreeBSD ps.
I'm also not exactly sure what that combination ...
1
You might not need x, -e is sufficient to select all processes:
ps -ef
This should work on any version of ps on Linux you’re liable to come across.
Current versions of ps from procps-ng interpret the x option with or without a dash without warning; the older version of ps from procps in CentOS 6 adds a warning (but it still lists all the processes, so your ...
0
Try this,
sed -n '/{/,/}/{//!p}' file
Just have the matching pattern inside the first block '/<StartPattern>/ and the second matching pattern into print until /<endPattern>/ which is separated by comma.
here start pattern is an open brace and end pattern is a close brace. So it will be like /{/,/}/
To exclude matching patterns while printing ...
5
First, you should anchor your regular expression to only match at the beginning of the line (^chr1) to avoid finding lines that contain chr1 but it isn't the first string (this can easily happen with an annotated VCF file, for example). Next, you can use the -w option for (GNU) grep:
-w, --word-regexp
Select only those lines containing ...
4
It looks like there are some spaces or tabs after chr1. So you could search for chr1, which is followed by some whitespace characters. Try this:
grep -v "chr1\s\+"
2
grep -f file1 file2 should search in file2 for all the patterns in file1 and print the matching lines. (With matches anywhere, though, so this might yield false positives if a protein name in some way appears after the first column.) Since you want to match fixed strings, you could also add -F.
1
GNU grep allows perl regex, so we can do this to find lines containing both word1 and word2.
grep -P '(?=.*word1)(?=.*word2)' filename
(see this post) for more details.
2
If you are tied to grep then you could maintain multi-line matches ...
file:
T5F6Z12:
Minion did not return. [Not connected]
T5F6Z11:
Pinion did return. [connected]
T5F6Z10:
Minion did not return. [Not connected]
Using
grep -B 1 "Minion" file | grep ":$"
T5F6Z12:
T5F6Z10:
If you can use sed then there is an answer over here in ...
8
You could use -B1 to print previous line as well and then grab only the first line:
$ grep -B1 'Minion' ip.txt
T5F6Z12:
Minion did not return. [Not connected]
$ grep -B1 'Minion' ip.txt | head -n1
T5F6Z12:
Or, do it with awk:
$ awk '/Minion/{print p} {p=$0}' ip.txt
T5F6Z12:
$ awk '/Minion/{sub(/:$/, "", p); print p} {p=$0}' ip.txt
T5F6Z12
Here ...
0
This is what you are looking for ?
$ find . -name '*EXEC_EC*' -mtime +7 -print
. = Represent your current directory,you can replace that with a directory path .
-mtime - Represents the file modification time and is used to find files older than 7 days.
2
It does not make sense to use grep for filtering if you use find anyway.
find . -mindepth 1 -maxdepth 1 -name '*EXEC_EC*' -mtime +7
By default find traverses the whole subtree (i.e. the contents of all subdirectories). -mindepth 1 -maxdepth 1 prevents that as the example works on the content of a single directory only. -mindepth 1 makes find ignore the ...
-2
You don't need to pipe the output of ls into grep. Grep has a built in option for that.
grep -rle 'EXEC_EC'
-l will only list files
-r recursive means will go through all files starting from you current path
-e after this grep expects your pattern as a string
2
The lines you're looking for start with 20/08/02, not with 2020/08/02. Try %y instead of %Y, this will give the last two digits of the year.
2
With bash:
shopt -s extglob # enable extglob
mylist=( !(*_special.MP4) ) # fill array but without files with suffix _special.MP4
echo "${mylist[@]%.MP4}" # remove suffix .MP4
2
Let's say I have these files that end in .MP4:
.
├── 1.MP4
├── 2.MP4
└── 3_special.MP4
If I understand correctly, you want the names of the non-special files, minus the .MP4 extension.
First of all, I would use find instead of ls because it's easier to control and easier to parse the output. Either way, you'll end up with a list files separated by newline ...
0
Using any awk in any shell on every UNIX box:
$ cat tst.awk
match($0,/package[[:alnum:]_]+/) {
sep = (cnt++ ? " ; " : "")
printf "%s./builder make%sMain", sep, substr($0,RSTART,RLENGTH)
if ( !(cnt % 2) ) {
printf "%s./builder cleancache", sep
}
}
END { print "" }
$ awk -f tst.awk file
...
0
Done and tested by awk
to=`awk 'END{print NR}' p.txt`
awk '{for(i=1;i<=NF;i++){if($i ~ /^package/){print "./builder " "Make"$i"Main"}}}' p.txt| awk -v to="$to" '{for(i=30;i<=to;i++){if(NR==i){$0=$0"\n./builder cleancache"}{i=i+29}}}1'
3
No need for a shell to execute commands. You could do the whole thing with perl as:
perl -ne '
for (/\bpackage\w+\b/g) {
system "./builder", "make${_}Main";
system "./builder", "cleancache" unless ++$n % 30
}' input-file
0
First of all, you really don't need sudo su and you certainly don't need sudo su - root. The su command defaults to root, but the sudo command exists to run commands that need to be run as root. So you simply need sudo command. That said, you should avoid running commands as root unless it is actually needed, and hostname does not require root. With all this ...
6
Use the -B ("before") option of grep:
history | grep -B4 release-it
Similar options include also -A ("after") and -C which stands for "context", it works as -A and -B combined.
0
You can pass the command set as a script to the ssh. For your case for example
{
echo "sudo su -"
echo "hostname"
echo "grep PermitRootLogin /opt/ssh/etc/sshd_config/"
} > /tmp/some-script.sh
and then
ssh your-remote < /tmp/some-script.sh
note the "script creation" is just an example for how your script ...
1
It seems your problem is the one covered in
https://www.gnu.org/software/parallel/man.html#EXAMPLE:-Grepping-n-lines-for-m-regular-expressions
EXAMPLE: Grepping n lines for m regular expressions.
The simplest solution to grep a big file for a lot of regexps is:
grep -f regexps.txt bigfile
Or if the regexps are fixed strings:
grep -F -f regexps.txt bigfile
...
0
You might have heard of GNU parallel. This won't work here...
To take advantage of parallelization it has to be a gigantic file and bash won't do, you'll have to switch to C or other real programming languages.
Your code will have to:
Determine the length L of the file
Fork into X processes
They all have to start reading at the n*X_n bit of the file
They ...
0
I found -x worked for me.
Example
$ grep -inx -d skip 'favicon.ico' *
test.txt:1:favicon.ico
Grep Manual
-x, --line-regexp
Select only those matches that exactly match the whole line. For a regular expression pattern, this is like
parenthesizing the pattern and then surrounding it with ^ and $.
-2
Tested and worked fine:
awk -F "," '{for(i=1;i<=NF;i++){if($i ~ /cardiac/){print $i}}}' file.tsv >>output.tsv
input:
cardiac,praveen,ajay
praveen,abhi,cardiac
ppp,cardiac,san
output:
cardiac
cardiac
cardiac
1
Your requirements aren't clear but to print lines that have 8 in the middle regardless of whether or not they also have 8 at the end, which is what I think you want, would just be:
grep '8.' file
2
$ cat tst.awk
BEGIN {
RS=""; ORS="\n\n"; FS=OFS="\n"
skip["[general]"]
skip["[providertrunk0]"]
add = "allow = alaw"
tgt = "nat = no"
}
!($1 in skip) {
for (i=1; i<NF; i++) {
if ( ($i != add) && ($(i+1) == tgt) ) {
$i = $i OFS add
...
1
It may be easier to use awk or sed when multiple conditions are needed:
awk '!/8$/ && /8/' file
This selects lines for printing based on 2 conditions: that it does not match the anchored regex 8$ and does match 8. The same logic in sed: sed -n '/8$/!{/8/p;}'.
With sed and a slightly different logic:
sed '/8$/d; /8/!d' file
This filters lines by ...
0
Like that:
grep -E '8.+$' <FILE>
2
You can use an approach like:
grep -rIlZe foo . |
xargs -r0 grep -lZe bar |
xargs -r0 grep -LZe baz |
xargs -r0 cat > MyOutputFile
That is, feed the list of files generated by the first grep to xargs -r0 to pass to the next grep which further refines the list.
Note the -L option for the last grep which is like -l except that it reports the files ...
0
Tried with below method
awk '{a[$1]++}END{for(x in a){print x,a[x]}}' ppp| awk '$2 >1{sum=sum+$2}END{print sum}'
output
awk '{a[$1]++}END{for(x in a){print x,a[x]}}' ppp| awk '$2 >1{sum=sum+$2}END{print sum}'
5
Adding Python method
#!/usr/bin/python
m=open('ppp','r')
j=[]
f=[]
for i in m:
if i.strip() not in j:
j.append(i.strip())
e=open(...
2
TL;DR:
jq -r -R '
select(contains(" | ")) |
split(" | ") |
.[0] as $text |
(.[1] | fromjson | to_entries | .[0].value ) as $json_obj_value |
"\($text) | \($json_obj_value)"
' yourlogfile.log
Complete answer
Most people don't realize quite how powerful jq is (though the same can be said about awk).
As Kusalananda ...
8
awk 'seen[$0]++ {count += (seen[$0]==2 ? 2:1)} END {print count+0}' file
If a line has been seen before, increment count by either 2 or 1 (based on whether this line is the first duplicate). At the end print count (+0 so that awk prints 0 instead of an empty string in case count was never incremented).
Another approach:
awk '{count += seen[$0]; seen[$0] = (...
3
With Perl:
perl -lne '
$k += qw(2 1 0)[++$h{$_}<=>2];
END { print $k; }
' file
5
We can compute the number of dups by maintaining a hash counter keyed on the input line.
The running counter $k is incremented in the amounts of 2, 1, and 0 when the key has been seen twice, more than two times, or the very first time.
Note the three -valued ...
6
Since you tagged your question linux you likely have the GNU implementation of uniq, which has a -D option:
-D print all duplicate lines
So
$ sort file | uniq -D | wc -l
5
0
Just copy and paste this peace of code into a new bash script file, save it and do chmod +x <file> then run it in your terminal to list all files that contain "foo" and "bar" and don't contain "rab" strings:
#!/bin/bash
function notcontain {
for FILE in $(find . 2> /dev/null); do
if ! grep "...
Top 50 recent answers are included
