0
votes
2answers
103 views

how to remove all double quotes from csv except specific field

I want to remove all double quotes from my csv but not the fourth field (because the four fields represent PATH of file) Please advice how to implement this by sed or awk or perl one liner , etc ...
3
votes
1answer
33 views

awk + gsub + how to work with all kind of characters

The following awk command target is remove spaces and unnecessary quotes from CSV file and only from single words between separators awk 'gsub(/("[ ]+|[ ...
3
votes
1answer
76 views

Is there a way to pass a variable from an awk statement to a bash function as an argument? [closed]

I am trying to look a for some specific values from (such as 1 or 4 and 2 or 3) a text file using an awk statement in a bash script. If this value is found in the file (within the awk statement), then ...
1
vote
3answers
63 views

Sed + how to set parameters in sed command in order to display lines from text file

I find the following way to print line from file In this example we want to print the first line from the hosts file sed -n '1,1p' /etc/hosts 127.0.0.1 localhost localhost.localdomain localhost4 ...
2
votes
3answers
379 views

How to replace the content of specific column with awk?

Given there is 40 columns in a record. I want to replace 35th column so that the 35th column will be replace with the content of 35th column and a "$" symbol. What came up my mind is like: awk ...
3
votes
2answers
80 views

how to grep complex hierarcy?

I have a file as:- start apple 1 a 2 b 3 c start orange 4 a 5 b start mango 1 a start a/b/c 5 z end 4 b end 6 c end start banana 3 c end 4 d 5 e end I want the output as :- 1 apple/a 2 apple/b 3 ...
1
vote
1answer
41 views

Tile logfile into pieces and search for errors

I want to tile a logfile into pieces and then search in this pieces for errors. If there is an error, I want to put the relevant logfile-piece into another file. Little example: Logfile looks like ...
-1
votes
1answer
58 views

Unix Awk group by

Can anyone shed me light how to group by based on the username and sum the values divide it by number of rows of that specific username. for example below. root 25 oracle 25 batch 30 griduser 20 ...
1
vote
2answers
72 views

Parse,Manipulate Data In linux/Unix

I want to have exactly the same output in this post(http://unix.stackexchange.com/questions/126139/extract-data-in-linux-unix)as above,but I have a new input file,which is below: ABCB11 4 ACE 11 ...
1
vote
2answers
171 views

Extract data in linux/unix

I have input data which I want to parse and extract values using awk/grep/sed: group-2 9 10 text color=black,from=ABCB11,fromid=4,order=2,thickness=3,to=ACE,toid=11,use=1,z=1 group-3 0 1 text ...
2
votes
1answer
51 views

I'm trying to count and print song file names using awk as following

RANK NAME BAND YEAR GENERE DOMESTIC/INTERNATIONAL 206:Reach Out, I'll Be There:The Four Tops:1978:Pop:3/2 207:Bye Bye Love:The Everly Brothers:1950:Classic:3/2 ...
1
vote
1answer
41 views

linux if else fails for some IP's

i am using the following code with if then to mount. It works for some IP's, it does not work for some IP's especially for FAILURE statement. Here is the code is_mounted() { mount | grep -qohw ...
6
votes
4answers
2k views

How to extract logs between two time stamps

I want to extract all logs between two timestamps. Some lines may not have the timestamp, but I want those lines also. In short, I want every line that falls under two time stamps. My log structure ...
2
votes
4answers
245 views

How to multiply a data file with another index data file?

I want your help in multiplying my data file with another index data file. Both data files have the same number of rows. But the data file have more than 1000 columns while the index data has only one ...
1
vote
2answers
91 views

What's the difference between print and head?

I know head gives you the beginning section of a file. But in this case, I'm trying to go through my source control and find every file that has a shebang line. And I'm failing to get results when I ...
1
vote
1answer
360 views

syntax error in conditional expression while executing the shell script?

I am working on a project in which I need to make a url call to one of my server from the bash shell script.. http://hostname.domain.com:8080/beat After hitting the above url, I will be getting the ...
6
votes
3answers
2k views

How to subtract rows (lines) with AWK

I'm trying to figure out how I can use AWK to subtract lines. For example, imagine the input file is: 30 20 The output would be: 10 Now, as a test I am trying to calculate the "Used" memory ...
-3
votes
2answers
75 views

How to get a device details only using awk commands

Friends, I have fdisk -l output in one file called test1.txt. Now I want to get the device name only using awk. How to get a device name? I want output like this: /dev/sda1 /dev/sda2 /dev/sda3 ...
4
votes
3answers
4k views

Adding a Column of values in a tab delimited file

How can I add a Column of values in a file which has a certain number of rows. I have a input file like this: Input file: SPATA17 1 217947738 LYPLAL1 1 219383905 FAM47E 4 77192838 SHROOM3 4 ...
0
votes
4answers
85 views

What are alternative tools for searching files in full content?

I realized that most of the text related tools I use, grep/egrep/awk/sed are line-oriented. I guess that is a simplified approach to search in files but I was wondering if there is another set of ...
1
vote
1answer
221 views

how to subtract a constant value from a data file?

I want to subtract a constant number from my data to get new data file. For example, I want to subtract 10 from my data. data.txt Id a b c d e 1 1 2 3 4 5 2 2 3 4 5 6 ...
1
vote
3answers
210 views

How to calculate the sum of the data that have the same ID in the first column?

I want to sum the values of the rows of my data that have the same ID in the first column. My data looks like data.txt Id a b c d e 1 1 2 3 4 5 1 2 3 4 5 ...
0
votes
2answers
105 views

How to Lookup the values of an index values from a big data file?

I have a data file (data_array.txt in the example) and index file, index.txt, at which I want to extract the data from the data file that have the same ID in the index file and store in to a new file, ...
0
votes
3answers
65 views

linux + awk + file manipulation

I have the following file more file.txt 1 2 3 4 4 2 23 4 5 23 2 3 END_OF_LINE 1 3 7 4 2 23 4 6 7 4 2 1 5 How to print only the fields 4 and 5 and 6 until the line END_OF_LINE , and from ...
0
votes
1answer
37 views

rearranging from colum to row in a csv [duplicate]

I have a CSV file that presently has an output like this 1 2 3 4 5 what I need to do is reconfigure the output as follows 1 2 3 4 5 all the entry data on one line, any help would be greatly ...
0
votes
2answers
172 views

Format of SMTP headers for scanning inbox email

I am looking for a simple procedure to extract email from my inbox into a file for further processing, using e.g. awk. With "extract" I mean use e.g. regex to get emails with a particular subject. My ...
3
votes
4answers
381 views

How to Prefix a column values with an apostrophe ( ' )?

I have a CSV file with multiple columns and 1000's of records, I need to prefix all the values of one of the columns (lets say 2nd column) with an apostrophe ' except in the first line or header line, ...
5
votes
3answers
3k views

Filter a .CSV file based on the 5th column values of a file and print those records into a new file

I have a .CSV file with the below format: "column 1","column 2","column 3","column 4","column 5","column 6","column 7","column 8","column 9","column 10 "12310","42324564756","a simple string with a , ...
1
vote
2answers
90 views

How can I reformat tabular data using AWK?

I have a file contain following output. ora.abc.db 1 ONLINE ONLINE serverA Open 2 ONLINE ONLINE serverB Open ora.xyz.db 1 ...
2
votes
3answers
201 views

How to run awk for some number of lines.?

I want to extract some lines with awk. Is it possible to do the following task: ls -l | awk 'BEGIN FOR(i=122;i<=129;i++) FNR==i' How can I display the details from line numbers 122 to 129?
10
votes
5answers
1k views

Using grep or awk

To capture a particular pattern, awk and grep can be used. Why should we use one over the other. How does this work in the background? Which is faster and why? E.g If I had a log file and I wanted ...
4
votes
2answers
810 views

Inotifywait for large number of files in a directory

What I want to do is, to monitor a directory (not recursive, just one) for new files created and append those files to one single big file as they are being written. The number of files that are ...
3
votes
1answer
467 views

Output of `java -version` not matched by grep or awk

On my Linux machine, it isn't clear to me why if I do the following then I don't get only the version string ("1.5.0_32"). # java -version | grep version | awk '{print $NF}' java version "1.5.0_32" ...
0
votes
1answer
185 views

replacing text and deleting text using awk in file using

I have a script and I want to do some replace and delete operations on it. I used sed to extract values between words SendCommands and end like this sed -n '/SendCommands/,/end/p' ddtt As awk was ...
0
votes
3answers
411 views

Sorting IP address according to the second field in file

in my ksh script I need to add the following task ( OS - linux/solaris ) I have the following file more test.txt /etc/backup/app 172.1.120 172.110.120 /etc/backup/app 172.1.120.12 ...
1
vote
2answers
123 views

Problem using awk

I have a problem using awk. Print, from every file given as a parameter, the number of line that has the length at least 10. Also, print the content of that line(s), except the fist 10 characters. At ...
1
vote
3answers
532 views

Newbie struggles with grep, sed, awk to filter html

I am a beginner linux user, trying to teach myself how to use linux tools on a cygwin install. I decided to make up a project to try to teach myself the basics of shell scripting and simultaneously ...
2
votes
2answers
250 views

changing pattern by removing spaces?

file : hi hello,new york, , ,brazil site ,brazil january,month is feb , , ,indiana jones task: delete all horizontal space (spaces and tab) between any two comas (like , ,) if ...
2
votes
1answer
286 views

shell variable in awk is not being passed to all the lines but just for the very first line of input?

inputfile:is2.txt 10.39.5.41,A1,B1 10.39.5.41,A2,B2 10.39.5.41,A3,B3 10.39.5.41,A4,B4 10.39.5.41,A5,B5 10.39.5.41,A6,B6 script : #!/bin/bash second_column="OOOOOOO" # OOOOOOO will be added to ...
1
vote
1answer
167 views

Regex to Cisco Interface

I would like to know if have some sed, grep or awk regex to parse Cisco interface section and exclude if interface is down (shutdown), like example below. Example: ! interface FastEthernet0/1 ip ...
5
votes
2answers
275 views

what is the meaning of 1 at the end of awk script

I was reading this awk script awk -F"=" '{OFS="=";gsub(",",";",$2)}1' I want to know what is the function of 1 at the end of it
2
votes
3answers
869 views

Merging content of two files with comparing content of column

I have a problem trying to merge content of two files : file 1 : 873 1.1.1.1 1033 99.99.99.10 1108 78.2.204.174 1257 78.2.8.181 file 2: test1 78.2.8.181 p6 test2 99.99.99.10 p9 test7 1.1.1.1 ...
2
votes
4answers
974 views

Replace all but a set of characters in a file with newline

how to delete all characters in file except numbers and "." , each word (numbers/dot) should be in new line in file see example2 the solution can be with sed or awk or ksh syntax remark - ...
2
votes
3answers
123 views

Finding out how many programs I have not installed from a list with grep/awk in archlinux

I want to see how many packages from a list are not installed. I use archlinux, so I use pacman -Q to determine if a package is installed. If it is not installed, it returns a message like: "error: ...
11
votes
1answer
3k views

Making a progressbar with “dialog” from rsync output

I'm looking for a way to filter/redirect rsync output in a manner where it can be fed to the "dialog --gauge" command, so I can get a nice looking progressbar during file sync. Currently I have only ...
4
votes
1answer
364 views

Efficient way of comparing in awk

#!/bin/awk BEGIN { while(getline var < compareTo > 0) { orderIds[var]=var; } } { if(orderIds[$0] == "") { print $0; ...
1
vote
1answer
255 views

Need help processing a text file with awk to conform to CSV flat file format

I have the following problem. I collected data on reaction time from over 100 participants for an experiment I am running. Unfortunately, the separators between fields were not consistent, but after a ...
0
votes
2answers
1k views

How to run a shell script containing an awk command

How to run this script (called count.sh)? awk 'BEGIN{ x=0 ; while (x < 5) { x+=1 ; print x ; } }' I'm trying to run it with sh count.sh but it's giving me an error.