-1
votes
1answer
36 views

Match and combine records in Bash [duplicate]

Consider two csv files: $ cat given.csv 123,John 246,Paul 369,George 987,Ringo $ cat family.csv 246,McCartney 123,Lennon 987,Starr 369,Harrison Note that the lines are not ordered! How might I use ...
2
votes
2answers
24 views

Gather job usage information on cluster

I am running simulations on a linux cluster shared by hundreds of other users. There is a job scheduler installed on the cluster that determines who runs where, and one of the scheduler commands ...
3
votes
2answers
72 views

Extract csv files from path to multiple files in bash

I have a files multiple file path for backup purpose ,now i want to filter it based on ".csv " fromat. I have tried this. filesall=(/home/abc/allfiles/*) files=$((${filesall[@]}|grep -F ".csv")) ...
1
vote
1answer
42 views

Store output from one command and process it for another

I am working on Unix command line. I have two files. I want to cat file1.txt and grep the output in file2.txt File1.txt 123A 223A 143A 153A 183A 123J 123P File2.txt: 123A (TYU) 223A (RUT) 143A ...
1
vote
2answers
61 views

grep works fine in command line but returns nothing in script

I'm trying to write a small script to get me list of files that match certain criteria into a file. #!/bin/sh cd /var/www_data/patch/ grep --include=\*.{php,ini,conf,sh} -ril -P ...
0
votes
1answer
91 views

Grep a string with spaces from a file

#!/bin/bash LIST=/errors_exception.txt cd /test for PATTERN in `cat $LIST` do for FILE in $(ls) do if zcat $FILE | grep -Fxq "$PATTERN"; then echo "$PATTERN found ...
-1
votes
2answers
35 views

Searching out a pattern from Logs

The format of the logs is in this manner: Junk Junk Junk : Junk Junk Junk:IP/Port Junk Junk I want to search using the IP and the output should be only the IP/Port part of each line in that log ...
3
votes
2answers
85 views

Getting the combined count of all occurences of a string under multiple directories?

I have a (hopefully) interesting problem that I could use some advice on. I have a system that's essentially used for storing logs. It has a directory structure like: YYYY/MM/DD/hostname/ There are ...
0
votes
1answer
23 views

For a given directory, how do I concatenate the tail end of recently modified files to a new file?

In case there's a better solution to the problem, my goal is to be able to quickly analyze ALL recent apache activity in /usr/local/apache/domlogs. The usefulness of this is when the server load is ...
0
votes
2answers
36 views

How to list similar lines?

The delimiters could be: " "; "_" INPUT: foo#foo_ehh113#The password of user 111 will expire within the next seven d foo#foo_ehh204#The password of user 111 will expire within the next seven d ...
0
votes
1answer
58 views

how do I recursivley grep (or otherwise search for) hex strings

Long and short I'm trying to track down some files left by evil scum on a server. fgrep -r "STRING" /path/here has been working well for finding general stuff. However, how do I search for hex ...
7
votes
2answers
464 views

Why is `tac file | grep foo' (piping) faster than `grep foo < <(tac file)' (process substitution)?

This question was motivated by "Reverse grepping", about grepping a huge file from bottom up. @chaos said: tac file | grep whatever Or a bit more effective: grep whatever < <(tac file) ...
0
votes
2answers
159 views

Solution to integer expression expected [closed]

So I when I run this code on my mac there are no errors, and it provides me the perfect output. But when I run it on Ubuntu or CentOS i get the following error integer expression expected ...
2
votes
3answers
110 views

bash - extract filenames from html file containing multiple links

I have downloaded an html file autogenerated by a script on a webpage. The file contains multiple links, including links to images I am trying to extract the full names of the images, for example ...
1
vote
1answer
59 views

Why does the same sed regex (after grep) fail when run in a bash script vs bash command line?

Here's the script. It is successful when I run it from the BASH prompt, but not in the script. Any ideas? When I say "fails," I mean the sed regex doesn't match anything, so there is no replaced ...
0
votes
2answers
67 views

How to count number of instance of a word starting at another word

I need to grep the MySQL-error log starting at today's date formatted like: `date +"%y%m%d"` (140710) And look for all instances of the word partitioned. I've tried various things to no avail. ...
3
votes
6answers
173 views

ls content of a directory ignoring symlinks

I have a directory in which I would like to list all the content (files and sub directories) without showing the symbolic links. I am using GNU utilities on Linux. The ls version is 8.13. Example: ...
3
votes
1answer
37 views

Matching several patterns on one line

I often use a program called tty0tty, which creates a null modem using PTYs. Now I want to make it part of an automated process. When I start tty0tty it prints (/dev/pts/1) <=> (/dev/pts/2) ...
5
votes
2answers
123 views

Is there a way to make this one-liner faster?

Context I have a directory of thousands of zip files that are dated in the form YYYYMMDD_hhmmss.zip and each about 300K. Within each zip file is about 400 xml files each about 3K. The problem I ...
6
votes
5answers
236 views

Strategy to extract movies's name from this ununiform dataset?

I am working on a movie database problem to improve regular expressions, this is the problem I'm running into. My dataset looks like this: Movie Name (variable space and tabs) year Movie1(can ...
2
votes
4answers
261 views

Inserting a line in a file only if this line isn't yet part the file

Total newbie to shell scripting, I am searching to create a script to automatically apply some custom configuration to text based configuration files. In the present case, I am searching to add 2 ...
0
votes
2answers
106 views

How can I grep for broken HTML tags?

I want to grep through all my HTML files and see if I have any bad tags there, example <br> , <hr> and so on, that is the I want to see what tags are not closed in the HTML file. for ...
2
votes
5answers
170 views

how can I print the Nth and the Mth line above or below the matching line with grep?

Let's say I have a well-structured text file with following content(no leading line numbers): 1 Mon Jun 9 00:11:47 CST 2014 2 eth0 Link encap:Ethernet HWaddr D4:BE:D9:F5:5C:0E 3 ...
0
votes
1answer
46 views

How to output the lines of a file after find and a grep?

Hello I want to output ERROR messages from logfiles from the current date. First I search for the logs of today with specific prefix: find /home/USER/logfilesError/ -maxdepth 1 -type f -name "xy_*" ...
0
votes
2answers
38 views

How can I search filenames which are starting with xy_* and are created/edited today?

Hello I´m currently trying to search for logfiles which are starting with xy_Number and are created/edited today(not last 24 hrs). I tried: find /home/USER/logfilesError/ -maxdepth 1 -type f ...
6
votes
6answers
416 views

Check if a variable contains only what I want, and nothing else

I'm writing a script to build a software from sources, and there's a --platforms option. I would like to allow the user to select multiple items, but I don't know how to prevent them from making a ...
1
vote
2answers
188 views

How to show specific lines from specific columns of a file

First off, I'm not allowed awk, sed or perl. For my assignment I was given a file of employee list: Marketing Ranjit Singh FULLEagles Dean Johnson Marketing Ken Whillans FULLEagles ...
0
votes
1answer
91 views

Duplicate specific text in text file, using perl or grep

I need to duplicate part of a plain text file as follows: The text file consists of a sequence of "pages", of the form: <page [lots of lines of text] /page> <page [lots of ...
2
votes
3answers
73 views

Make grep work for special filenames

I have a set of txt files whose names may contain space or special characters like #. I have a grep solution grep -L "cannot have" $(grep -l "must have" *.txt) to list all the files who have must ...
1
vote
3answers
685 views

Find files containing one string but not the other

I am in a folder with lots of .txt file, I would like to find all the files which contain stringA, but don't contain stringB (they are not necessarily in the same line). Does anyone know how to do ...
1
vote
0answers
33 views

Filter partial output of some script [duplicate]

I need to analyze the output of some script So, I know their header and their tail of some part of the output script, but not the content. I found an approach [1] But I need something like: $ ...
4
votes
1answer
75 views

Redirect grep out to a file does not work [closed]

I am doing a recursive grep by the following command: grep -r "load" . > tmp.txt However, when I hit enter, the command does not execute and bash is waiting for more inputs.. My OS is Ubuntu ...
0
votes
4answers
427 views

bash event not found trying to match and exclude parenthesis in grep

In a very long line I'll summarize with: (foo),(bar,baz(word,right),(end) I want to print only: (bar,baz(word,right To match the second parenthesis, I exclude the word that follows the ...
6
votes
6answers
4k views

bash find lines starting with string

I have a bunch of files and I want to find which one contains sequential lines starting with a certain string. For example for the following file : Aaaaaaaaaaaa Baaaaaaaaaaa Cxxxxxxxxx Cyyyyyyyyy ...
2
votes
2answers
230 views

filter and count with grep in a single pass?

I am using grep at the end of a set of piped commands to filter some rows out of a file. Afterwards I will want to do some arithmetic based on the number of remaining rows. e.g. ...
2
votes
3answers
57 views

How to list backup files missing corresponding real files?

I accidentally deleted some files spread across my home directory, but I do not know exactly which ones were removed. How can I get a list of all backup files missing their corresponding file? ...
3
votes
1answer
99 views

Is there any drawback to using “grep -E” instead of plain “grep”?

I understand that GNU grep without -E uses basic regex and that grep -E uses extended regex. Is there any harm in using grep -E even for situations that would work with plain grep?
6
votes
2answers
522 views

What's the most efficient way to grep for two completely separate things and assign the values to separate variables?

CentOS 6.x I want to take the output from curl, grep for two completely separate strings, and assign their respective values as variables. What is the most efficient way to do this (without writing ...
3
votes
2answers
168 views

How can I open a file that results from grep?

I often grep a bunch of files to find a line, and then grep returns one result. Rather than copying and pasting the filename into a new command, I'd like to be able to open that one result with an ...
1
vote
1answer
258 views

how do i redirect output from tailf & grep to a file [duplicate]

All I wanted is to grep for specific lines in an ongoing log and re-direct it to some file.. tailf log | grep "some words" Now, I want the above command output to get re-directed to some file in ...
-2
votes
1answer
33 views

Implementation Logic [closed]

I have two files,which is something like this: file1 msgid "New" msgstr "" dsvhuvchvdhcvhc msgid "Old" msgstr "hsvhs" kdnvcjdbvj msgid "Bold" msgstr "" file2 dcbjdsbc msgid "New" msgstr "Alas" ...
5
votes
1answer
215 views

counting multiple patterns in a single pass with grep?

I've written a grep loop to iteratively count DNA trinucleotides within a gzipped DNA fasta file containing DNA sequences e.g. declare -a tri=(AAA AAC AAG AAT CAA .. etc) for i in ${tri[@]} do ...
2
votes
1answer
74 views

using grep --perl-regexp

this is a sample output of ls $ ls -lAF -rw------- 1 martin martin 168 Jan 25 23:26 aaa -rw------- 1 martin martin 168 Jan 25 23:26 bbb -rw------- 1 martin martin 168 Jan 25 23:26 ccc I would ...
5
votes
5answers
171 views

A shell script to write selected fields to a single line

I have a log file trace.log which prints the time stamp, thread name and Transaction method and transaction ID as below. 2014-01-23 15:50:41,724 [catalina-exec-35] INFO TRANSACTION getConnection ...
3
votes
1answer
110 views

Can someone clarify this Bash script

I'm learning something about Bash scripting, and I came across this example: The task is to write a Bash script that reads a file, given with the first argument, pulls out those records that have the ...
24
votes
3answers
561 views

Multicolored Grep

I'm trying to get each grep command to highlight it's results in a different color. I can do it manually with a line like this: ls -l GREP_COLORS='mt=01;32' grep c | GREP_COLORS='mt=01;31' grep o | ...
0
votes
2answers
187 views

Count occurrences of line in multiple files

I have log files that I need to grep and count the number of occurrences of a specific line. Issue is that I must start at a specific line of the first file and only count occurrences after that ...
5
votes
5answers
536 views

Pattern matching across multiple lines

What is a quick and easy way to validate if the pattern "301 domaname.com 200" is present within BASH when spread across multiple lines of output? I was using something like the following: awk ...
22
votes
5answers
2k views

Pipe to multiple files in the shell

I have an application which will produce a large amount of data which I do not wish to store onto the disk. The application mostly outputs data which I do not wish to use, but a set of useful ...
0
votes
0answers
28 views

How to use grep command to find the matching pattern and some more characters? [duplicate]

I have a compressed JS code and I want to find a pattern, but since the file is compresses, it has only a line, and the match pattern is all the file. The output is too much large. grep -r ...