0
votes
2answers
28 views

Compare first column in different files

I want to compare one file with other files in the same directory. file1.txt contains: move 34#123# get 11#278 put 33#29#567#23 file1.txt should compare with files 1.txt and 2.txt. 1.txt ...
3
votes
2answers
117 views

How can I print out all but the last n characters in bash?

I'm working with standard input (cating two files together) and want all but the last 10 characters of the result to be passed through the next command. All I can find through searching is using sed, ...
7
votes
5answers
316 views

How to extract logs between two time stamps

I want to extract all logs between two timestamps. Some lines may not have the timestamp, but I want those lines also. In short, I want every line that falls under two time stamps. My log structure ...
1
vote
2answers
74 views

Union of two files and unique in new file

I have two files with few similar column one, two and third column with small variation and few other columns as follows. file 1: AT1 AT22 24 1 ATAGATA ATTATAT AT2 AT24 22 0 ATAGATA ATTATAT ...
-2
votes
1answer
59 views

concatenate or remove multiple files of a directory

In general for file concatenation with the same extension, we use cat *.txt. But in case we are in a different directory and trying to concatenate another directory's files how to do that ? I have ...
4
votes
2answers
227 views

Merge fields in a file

I have a file with 7 columns, a GFF file having chromosomal regions.I want to collapse the rows where REGION ="exon" to only one row in the file.The row has to be collapsed on the basis of regions ...
4
votes
4answers
166 views

Pass shell variable as a /pattern/ to awk

Having the following in one of my shell functions: function _process () { awk -v l="$line" ' BEGIN {p=0} /'"$1"'/ {p=1} END{ if(p) print l >> "outfile.txt" } ' } , so when called as ...
4
votes
2answers
143 views

Replace string with sequential index

Can someone suggest an elegant way to accomplish this? Input: test instant () test instant () ... test instant () //total 1000 lines output should be: test instant1 () test ...
1
vote
1answer
78 views

How to get the JSON STRING from the given string

I have a string given below. I just need to get the first accountHeader (bolded) JSON string, I tried everything.. It didn't work. Can you help me to get the data? <START ...
2
votes
3answers
545 views

How to merge two files based on the matching of two columns?

I have file1 likes: 0 AFFX-SNP-000541 NA 0 AFFX-SNP-002255 NA 1 rs12103 0.6401 1 rs12103_1247494 0.696 1 rs12142199 0.7672 And a file2: 0 AFFX-SNP-000541 1 0 ...
2
votes
2answers
176 views

Do I need to encapsulate awk variables in quotes in order to sanitize them?

As per an answer on stackoverflow, it's my understanding that encapsulating bash variables in double-quotes is a fairly safe way of sanitizing user input. What about awk variables? For example, if ...
0
votes
2answers
63 views

Merging two different files with conditions

I have two different files as shown below. Content of a.txt: HDR|1|||||||||| DTL|@||||||||||| TLR||||||||||||| HDR|1|||||||||||| ...
1
vote
2answers
146 views

How can I make awk on Busybox treat the entire input with NULL character in between as a single record?

I wrote a base64 encode script for my Busybox system. #!/bin/sh base64encode() { /usr/bin/awk -- ' function asc(char,l_found) { l_found=0; for (i=0;i<=255;i++){ if ...
0
votes
3answers
40 views

linux + awk + file manipulation

I have the following file more file.txt 1 2 3 4 4 2 23 4 5 23 2 3 END_OF_LINE 1 3 7 4 2 23 4 6 7 4 2 1 5 How to print only the fields 4 and 5 and 6 until the line END_OF_LINE , and from ...
2
votes
3answers
241 views

awk print ' char

I want to print: cat file.log | awk '{print nir's $1}' output should be: nir's aaa nir's bbb nir's abc nir's dbc The problem is with the ' in nir's. I also tried: cat file.log | awk '{print ...
2
votes
2answers
77 views

Convert numeric string from file to exit code in shell

I'd like to grep /etc/password for a username and return the last two digits of a user-id number as the return code. This is because I have an application whose main way of interacting with other ...
4
votes
3answers
252 views

using awk to split a line on single spaces not multiples

I'm trying to split a line that I have no control over the format of. If parameter 7 and 8 are missing which is possible they will be replaced by a space so I would end up with, field1 field2 field3 ...
3
votes
2answers
312 views

Replace a pattern in a file with a long string

I'm writing a little shell script to generate a directory listing. To make the output easily customizable, the script just builds a HTML-table and than should replace a specific token in a template ...
0
votes
1answer
482 views

shell script to print rows if there is a value in column 2

1 **2** 3 4 3 1 4 2 1 4 3 2 4 **2** 1 3 How can I print all rows, on which there is a 2 in the second column.
1
vote
2answers
216 views

Way to write new filename on wildcard input?

I have some experience with Inix terminals from science internships I've been part of, mainly using a few utilities like grep, awk, and sed but there is one thing I've been trying to figure out for a ...
1
vote
3answers
99 views

How to add Multiple Searches in AWK command

Im trying to search for the files Created in June,July and August months. I used this method ls -lrth|awk '/[Jun][Jul][Aug]/ {print}' but it ain't working. Please help me correct the syntax.
4
votes
3answers
753 views

Summing up an array inside of awk?

I have the following piece of code: sum1= sum2= declare -a a echo $temp | awk '{split($0,a,","); name=a[1] ; for(i=2;i<=4;i++) sum1+=a[i] ; for(i=5;i<=7;i++) sum2+=a[i] }' This code is ...
2
votes
4answers
390 views

Can awk use field identifiers also for shell strings (variables)?

Well, this apparently is not possible the way I'm trying it. This alternate approach to obtain bar as a resulting string works, though: #!/bin/bash path1=/usr/tmp/foo/bar/baz awk -F/ '{print $5}' ...
1
vote
3answers
131 views

Cat 2 files and avoid duplication of lines

I have the file1 with the following content {"name":"clio5", "value":"13"} {"name":"citroen_c4", "value":"23"} {"name":"citroen_c3", "value":"12"} {"name":"golf4", "value":"16"} {"name":"golf3", ...
1
vote
1answer
204 views

Safely convert unicode strings to printable characters

I have many HTML files containing mixed unicode strings like \303\243 and printable characters like %s. What I'd like to do is converting the unicode strings into printable characters in a safe way. ...
3
votes
4answers
577 views

Replacing pattern after nth match is found on each line?

I have a file containing lines: india;austria;japan;chile china;US;nigeria;mexico;russia I want to replace all the occurences of semicolon on each line with e.g. ;NEW;, but starting from the 2nd ...
1
vote
4answers
76 views

How to find what device a file is on (and use that in a script)?

I want to find out what device my file is on so that I can use it in a script. I can get this far: $ df . Filesystem 512-blocks Used Available Capacity Mounted on /dev/disk0s2 498438976 ...
1
vote
2answers
554 views

Parsing string by awk and get only elements without pipes or semi colons

I would like get this string line AUGUSTYN|Stanisław|3589238 without | and : I tried something like that: cat baza|grep "AUGUSTYN" -n|awk -F '|' '{print $1,$2,$3,$4}'` (baza is my file with ...
3
votes
1answer
147 views

No output from inotifywait | awk

I'm attempting to use part of a one-liner found here: Script to monitor folder for new files? When I try the following procedure I get no output whatsoever and I cannot figure out why. In terminal ...
0
votes
1answer
94 views

Why is this variable not getting passed to awk? [duplicate]

Possible Duplicate: external variable in awk How do I pass this variable below? This doesn't work: fname=testfile.txt lsof | awk '/deleted/&&/$fname/ {print $4}' *----no output* ...
3
votes
1answer
316 views

Split file based on a pattern with leading zeros

I have a book in text format. I would like to split the book into several files where each file contains a single chapter. Therefore I'm using the following command: awk '/Chapter/{i++}{print > ...
7
votes
10answers
890 views

What's a good way to filter a text file to remove empty lines?

I have a .csv file (on a mac) that has a bunch of empty lines, e.g.: "1", "2", "lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum ...
0
votes
2answers
444 views

put rows of numbers into a column with shell script

I have a file with many lines, like 1 jfkdajfd 1 2 3 5 2 fkldfjld 3 fdkfloaf 9 10 4 fldfldkf 5 fdskf;ak 12 1 4 I want to get all the numbers and put them in a column in a file, like 1 ...
1
vote
2answers
283 views

List info about files/directories with spaces

I am trying to use ls to get information about files and directories. My current command fails to properly get the name whenever a file/directory has a space. I am using this to list all ...
2
votes
1answer
96 views

Extracting some files from a folder and creating a list of these files

I am just a beginner with Unix and so my skills in Unix commands is limited. I am currently dealing with a folder containing about 1000 files and I have to extract some filenames from this folder and ...
3
votes
1answer
1k views

File size limit exceeded in bash [closed]

I have tried this shell script on a SUSE 10 server, kernel 2.6.16.60, ext3 filesystem. The script has a line like this: cat file | awk '{print $1" "$2" "$3}' | sort -n > result The file's size ...
3
votes
2answers
698 views

What is the best way to find a list of several strings within a large text file

The short, general question is: In Unix/Linux, what is the best way to find a list of several (about 150) strings within a large text file? I am asking this to all Unix/Linux experts as a general ...
1
vote
3answers
1k views

how to sum output of awk or other expression with xargs

Suppose i have the following bash shell script: #!/bin/bash export count=0; for i in `ls ./mydoc` ;do pdfinfo ./mydoc/$i | egrep Pages |awk {'print $2'} |xargs -+ $count ; ...
3
votes
1answer
945 views

Make awk use bash with the system() command

Is there a way to make awk use bash instead of sh when running system commands using the system() call? I want to use some bash-specific features such as [[ ]] and < > string comparison operators: ...
19
votes
11answers
2k views

Remove duplicate $PATH entries with awk command

I am trying to write a bash shell function that will allow me to remove duplicate copies of directories from my path environment. I was told that it is possible to achieve this with a one line ...
0
votes
2answers
1k views

Using awk to process ls output with spaces in filenames/paths

All, I have a script in which I use find command to filter out all png files in a given folder and list them along with their size. I want the output in the following format: someFile.png => 1.2K ...
2
votes
1answer
491 views

remove trailing zeros in awk not working. syntax error

regex = "\\.*0+$"; subst = ""; system("echo "id "| awk '{sub(\\.*0+$," subst"); print}'"); It is giving the following error: awk: cmd. line:1: {sub(\.*0+$,); print} awk: cmd. line:1: ^ ...
7
votes
2answers
2k views

A shell script for joining two files

I want to write a shell script that get two files A and B, and get a result like this: File A: user_a tel_a addr_a user_b tel_b addr_b File B: process_1 user_a process_2 user_a process_3 user_b ...
3
votes
2answers
2k views

How to apply the same awk action to different files?

I am new in awk and I do not know if it is possible to write an awk script that does this: I have hundred of data files that I have to sort. For each one I use the following one-liner: awk ...
1
vote
2answers
255 views

$2 (field reference) in awk BEGIN is not working

In the following snippet, $2 in awk is returning empty. What am I doing wrong? I am trying to find the difference between MAX and MIN. #!/bin/ksh if [ $# -ne 1 ]; then echo "Usage: sh ...
4
votes
1answer
307 views

Efficient way of comparing in awk

#!/bin/awk BEGIN { while(getline var < compareTo > 0) { orderIds[var]=var; } } { if(orderIds[$0] == "") { print $0; ...
2
votes
1answer
1k views

Diff the output of two `awk` commands

I'm trying to compute the difference between the output of two awk commands but my simple attempts at it seem to be failing. Here is what I'm trying: diff $(awk '{print $3}' f1.txt | sort -u) $(awk ...
2
votes
3answers
1k views

Trouble getting awk output in for loop

I'm trying to create a script that will check a website for a word. I have a few to check so I'm trying to input them via another file. The file is called "testurls". In the file I list the keyword ...
3
votes
4answers
455 views

Find a string (like grep -q) within only a section of a file

I want to write some Bash that can verify that a string exists in a configuration file. I can't change the file format, it belongs to a different application. The file is subdivided into groups named ...
4
votes
4answers
1k views

How to echo an escaped string

How can I echo an escaped string that contains $ in Bourne Shell? user@server:~$ cat test.sh #!/bin/sh echo $1 user@server:~$ ./test.sh \$sad\$test $sad$test I want it to return an escaped ...