3
votes
1answer
62 views

Is there a way to pass a variable from an awk statement to a bash function as an argument? [closed]

I am trying to look a for some specific values from (such as 1 or 4 and 2 or 3) a text file using an awk statement in a bash script. If this value is found in the file (within the awk statement), then ...
4
votes
3answers
247 views

awk with if statements

I am trying to print from a file using awk, but my output is empty. Here is my code so far accountNum=$1 while read -r LINE || [[ -n $LINE ]] ; do awk -F',' '{ if($1==accountNum) { print ...
2
votes
1answer
44 views

Backslash with pipe in system command from awk

I want to invoke shell command from awk so I am using system() but when I am passing expression in I am getting error at backslash. Here is my expression echo "select * from abc limit 1;"| ...
3
votes
2answers
89 views

Get specific line from command output in awk

I have used pipeline to read specific record from impala shell.Here is what I got [cloudera@localhost ~]$ echo "select * from abc where key > 'a-26052014015400' limit 1;" | impala-shell Starting ...
2
votes
3answers
45 views

Get the last minute dated lines from list

If I have an input file with a list of users and their dates they were last signed in, how can I select with a sort | awk function the last minute lines? The dates are written in a +%H:%M:%S format. ...
2
votes
4answers
128 views

How to initialize an array using awk and bash?

I am trying to store values of the first line of a text file into an array. Here is what I have so far: arr_values=() awk ' NR==1 { for (i=0; i<=NF; i++) ...
1
vote
1answer
45 views

Awk that I am stuck on

I have a project I am working on for work. The script I am building initially takes two arguments: one is an unique Identifier and the other is for a log file (both strings). Now I have the first part ...
0
votes
3answers
29 views

Finding and replacing text inside awk block

I want to find and replace a value inside awk block. Here is a sample input file to my script P1 10,9:11/18013013582 ,10:1 ,167:0 ,487:5/E.164 11,9:15/310410532169026 ,10:60 ,167:0 ,487:4/IMSI ...
1
vote
2answers
78 views

Compare set of values from two files and print the output in third file in awk

Hi friends i have problem with comparing two files.I'll try my best to explain.I have two file f1 and f2 which i have to compare on basis of two sets Parameter(p) and Value(v). Basically f1 contains ...
1
vote
3answers
38 views

Shell command to remove the first string that is enclosed with slashes

I need a shell command(sed or awk is preferred) to remove the string that is enclosed within the slashes. For example If my string is /some/dir/example/ then the output should be dir/example/
2
votes
2answers
111 views

how to parse a file that is changed continuously using awk?

for example if we have a file contains : Hello world Hello earth Hi everybody we can easily parse it to get the first and second columns of the lines contains the word Hello using awk: awk ...
2
votes
3answers
387 views

How to copy or read the return value (exit status) from AWK script into the shell script (bash) to be used in if statement to compare

I want to read the error codes or the exit status of awk script into shell to use in if condition to compare for error handling my shell script looks something like: My script: #!/bin/sh awk -v ...
1
vote
2answers
73 views

Renaming a whole folder content

I'm on OSX 10.9.2 (BASH 3.5) and I wanted to rename a whole folder content names. So I run a script and somehow ended with this: s01eDavid.Letterman.2014.02.27.Stupid.Human.Tricks.x264-SWOLLED.mp4 ...
1
vote
1answer
39 views

Tile logfile into pieces and search for errors

I want to tile a logfile into pieces and then search in this pieces for errors. If there is an error, I want to put the relevant logfile-piece into another file. Little example: Logfile looks like ...
0
votes
2answers
56 views

Method to paste specific column when in terminal

I'd like to paste only the third column of data from within the terminal. I originally have something like: 4 SF0086 SF0086 12 SF0087 SF0087 17 TSF141 TSF141 29 TSF070 TSF070 30 TSF028 TSF028 31 ...
-1
votes
1answer
53 views

Unix Awk group by

Can anyone shed me light how to group by based on the username and sum the values divide it by number of rows of that specific username. for example below. root 25 oracle 25 batch 30 griduser 20 ...
0
votes
2answers
137 views

Compare first column in different files

I want to compare one file with other files in the same directory. file1.txt contains: move 34#123# get 11#278 put 33#29#567#23 file1.txt should compare with files 1.txt and 2.txt. 1.txt ...
3
votes
2answers
392 views

How can I print out all but the last n characters in bash?

I'm working with standard input (cating two files together) and want all but the last 10 characters of the result to be passed through the next command. All I can find through searching is using sed, ...
6
votes
4answers
1k views

How to extract logs between two time stamps

I want to extract all logs between two timestamps. Some lines may not have the timestamp, but I want those lines also. In short, I want every line that falls under two time stamps. My log structure ...
1
vote
2answers
112 views

Union of two files and unique in new file

I have two files with few similar column one, two and third column with small variation and few other columns as follows. file 1: AT1 AT22 24 1 ATAGATA ATTATAT AT2 AT24 22 0 ATAGATA ATTATAT ...
-2
votes
1answer
70 views

concatenate or remove multiple files of a directory

In general for file concatenation with the same extension, we use cat *.txt. But in case we are in a different directory and trying to concatenate another directory's files how to do that ? I have ...
4
votes
2answers
266 views

Merge fields in a file

I have a file with 7 columns, a GFF file having chromosomal regions.I want to collapse the rows where REGION ="exon" to only one row in the file.The row has to be collapsed on the basis of regions ...
4
votes
4answers
2k views

Pass shell variable as a /pattern/ to awk

Having the following in one of my shell functions: function _process () { awk -v l="$line" ' BEGIN {p=0} /'"$1"'/ {p=1} END{ if(p) print l >> "outfile.txt" } ' } , so when called as ...
4
votes
2answers
348 views

Replace string with sequential index

Can someone suggest an elegant way to accomplish this? Input: test instant () test instant () ... test instant () //total 1000 lines output should be: test instant1 () test ...
1
vote
1answer
141 views

How to get the JSON STRING from the given string

I have a string given below. I just need to get the first accountHeader (bolded) JSON string, I tried everything.. It didn't work. Can you help me to get the data? <START ...
2
votes
3answers
4k views

How to merge two files based on the matching of two columns?

I have file1 likes: 0 AFFX-SNP-000541 NA 0 AFFX-SNP-002255 NA 1 rs12103 0.6401 1 rs12103_1247494 0.696 1 rs12142199 0.7672 And a file2: 0 AFFX-SNP-000541 1 0 ...
2
votes
2answers
482 views

Do I need to encapsulate awk variables in quotes in order to sanitize them?

As per an answer on stackoverflow, it's my understanding that encapsulating bash variables in double-quotes is a fairly safe way of sanitizing user input. What about awk variables? For example, if ...
0
votes
2answers
82 views

Merging two different files with conditions

I have two different files as shown below. Content of a.txt: HDR|1|||||||||| DTL|@||||||||||| TLR||||||||||||| HDR|1|||||||||||| ...
1
vote
2answers
271 views

How can I make awk on Busybox treat the entire input with NULL character in between as a single record?

I wrote a base64 encode script for my Busybox system. #!/bin/sh base64encode() { /usr/bin/awk -- ' function asc(char,l_found) { l_found=0; for (i=0;i<=255;i++){ if ...
0
votes
3answers
60 views

linux + awk + file manipulation

I have the following file more file.txt 1 2 3 4 4 2 23 4 5 23 2 3 END_OF_LINE 1 3 7 4 2 23 4 6 7 4 2 1 5 How to print only the fields 4 and 5 and 6 until the line END_OF_LINE , and from ...
2
votes
3answers
379 views

awk print ' char

I want to print: cat file.log | awk '{print nir's $1}' output should be: nir's aaa nir's bbb nir's abc nir's dbc The problem is with the ' in nir's. I also tried: cat file.log | awk '{print ...
2
votes
2answers
107 views

Convert numeric string from file to exit code in shell

I'd like to grep /etc/password for a username and return the last two digits of a user-id number as the return code. This is because I have an application whose main way of interacting with other ...
4
votes
3answers
453 views

using awk to split a line on single spaces not multiples

I'm trying to split a line that I have no control over the format of. If parameter 7 and 8 are missing which is possible they will be replaced by a space so I would end up with, field1 field2 field3 ...
3
votes
2answers
640 views

Replace a pattern in a file with a long string

I'm writing a little shell script to generate a directory listing. To make the output easily customizable, the script just builds a HTML-table and than should replace a specific token in a template ...
0
votes
1answer
888 views

shell script to print rows if there is a value in column 2

1 **2** 3 4 3 1 4 2 1 4 3 2 4 **2** 1 3 How can I print all rows, on which there is a 2 in the second column.
1
vote
2answers
411 views

Way to write new filename on wildcard input?

I have some experience with Inix terminals from science internships I've been part of, mainly using a few utilities like grep, awk, and sed but there is one thing I've been trying to figure out for a ...
1
vote
3answers
230 views

How to add Multiple Searches in AWK command

Im trying to search for the files Created in June,July and August months. I used this method ls -lrth|awk '/[Jun][Jul][Aug]/ {print}' but it ain't working. Please help me correct the syntax.
4
votes
3answers
1k views

Summing up an array inside of awk?

I have the following piece of code: sum1= sum2= declare -a a echo $temp | awk '{split($0,a,","); name=a[1] ; for(i=2;i<=4;i++) sum1+=a[i] ; for(i=5;i<=7;i++) sum2+=a[i] }' This code is ...
2
votes
4answers
595 views

Can awk use field identifiers also for shell strings (variables)?

Well, this apparently is not possible the way I'm trying it. This alternate approach to obtain bar as a resulting string works, though: #!/bin/bash path1=/usr/tmp/foo/bar/baz awk -F/ '{print $5}' ...
1
vote
3answers
143 views

Cat 2 files and avoid duplication of lines

I have the file1 with the following content {"name":"clio5", "value":"13"} {"name":"citroen_c4", "value":"23"} {"name":"citroen_c3", "value":"12"} {"name":"golf4", "value":"16"} {"name":"golf3", ...
1
vote
1answer
253 views

Safely convert unicode strings to printable characters

I have many HTML files containing mixed unicode strings like \303\243 and printable characters like %s. What I'd like to do is converting the unicode strings into printable characters in a safe way. ...
3
votes
4answers
825 views

Replacing pattern after nth match is found on each line?

I have a file containing lines: india;austria;japan;chile china;US;nigeria;mexico;russia I want to replace all the occurences of semicolon on each line with e.g. ;NEW;, but starting from the 2nd ...
1
vote
4answers
82 views

How to find what device a file is on (and use that in a script)?

I want to find out what device my file is on so that I can use it in a script. I can get this far: $ df . Filesystem 512-blocks Used Available Capacity Mounted on /dev/disk0s2 498438976 ...
1
vote
2answers
777 views

Parsing string by awk and get only elements without pipes or semi colons

I would like get this string line AUGUSTYN|Stanisław|3589238 without | and : I tried something like that: cat baza|grep "AUGUSTYN" -n|awk -F '|' '{print $1,$2,$3,$4}'` (baza is my file with ...
3
votes
1answer
168 views

No output from inotifywait | awk

I'm attempting to use part of a one-liner found here: Script to monitor folder for new files? When I try the following procedure I get no output whatsoever and I cannot figure out why. In terminal ...
0
votes
1answer
108 views

Why is this variable not getting passed to awk? [duplicate]

Possible Duplicate: external variable in awk How do I pass this variable below? This doesn't work: fname=testfile.txt lsof | awk '/deleted/&&/$fname/ {print $4}' *----no output* ...
3
votes
1answer
350 views

Split file based on a pattern with leading zeros

I have a book in text format. I would like to split the book into several files where each file contains a single chapter. Therefore I'm using the following command: awk '/Chapter/{i++}{print > ...
7
votes
10answers
1k views

What's a good way to filter a text file to remove empty lines?

I have a .csv file (on a mac) that has a bunch of empty lines, e.g.: "1", "2", "lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum ...
0
votes
2answers
520 views

put rows of numbers into a column with shell script

I have a file with many lines, like 1 jfkdajfd 1 2 3 5 2 fkldfjld 3 fdkfloaf 9 10 4 fldfldkf 5 fdskf;ak 12 1 4 I want to get all the numbers and put them in a column in a file, like 1 ...
1
vote
2answers
314 views

List info about files/directories with spaces

I am trying to use ls to get information about files and directories. My current command fails to properly get the name whenever a file/directory has a space. I am using this to list all ...