Tagged Questions
5
votes
2answers
75 views
awk + paste for cleaning up PATH?
I have seen this code in .cshrc init files on a few machines. I went through a few awk tutorials in trying to understand how it works, but I am still unable to decrypt it.
setenv PATH `echo $PATH | ...
2
votes
3answers
37 views
Replace certain strings with value from former line
I have the following text:
Source IP,URL
1.1.1.1,example1.com
Blank,example2.com
2.2.2.2,example3.com
Blank,example4.com
Blank,example5.com
Blank,example6.com
Blank,example7.com
7.7.7.7,example8.com
...
3
votes
3answers
73 views
sed + how to remove character/s that start or ended on each number
How do I remove the . character(s) that start in the beginning of each number or end on each number?
Remark – perl one liner also good alternative for sed.
Example input:
.23.12.44.5.
.233.3.3.3
...
3
votes
3answers
66 views
checking data in columns when a data or some may be missing or present?
I am not sure if this possible.
say i have columns like :
Team Colour Game Rainfall PlayerName
XYZ Blue Cricket Yes Kapil
suppose i need to search ...
1
vote
1answer
41 views
Using awk to parse AIX's lsuser output in stanza format
I'm creating a simple awk script to parse AIX's lsuser -f ALL output and return information in one line. A sample of the output is something like this.
root:
id=0
pgrp=something
...
2
votes
6answers
87 views
How can I separate these two outputs using awk command
Here's my output:
MessageID=3990592283244651750-30192b51.13df831d93a.7eb4;EsbTiming=2013-04-12T01:07:46.099
I want only the time.
How can I separate them?
2
votes
3answers
119 views
How can I use SED or AWK to replace placeholders in a template file with variable content that contains special characters?
CentOS 6.3
I'm trying to get a small script to send an email containing a copy of email headers in the body (for the purpose of internal reporting).
The template file contains the following:
...
0
votes
1answer
94 views
comparing lines with awk vs while read line
I have two files one with 17k lines and another one with 4k lines. I wanted to compare position 115 to position 125 with each line in the second file and if there is a match, write the entire line ...
1
vote
3answers
63 views
awk + print line only if the first field start with string as Linux1
how to print the line in case the first field start with Linux1
for example:
echo Linux1_ver2 12542 kernel-update | awk '{if ($1 ~ Linux1 ) print $0;}'
the target is to print the line , ...
1
vote
2answers
121 views
Unix command for pattern matching
I want to count the number of words and print the matched pattern lines which matches exactly with following pattern:
abc-ERROR:
The input File contains:
# abc-ERROR: xyxxkkfgfr
# def-Error: ...
4
votes
4answers
102 views
How to remove character and space from a string
Hello I have one file with output
Name : ABC
Name : CDE
Name : ZYS
I want to get the result
ABC
CDE
ZYS
How can i get it through SED command.
I have tried. sed s/'Name' but ...
-4
votes
1answer
47 views
How to transform two delimited ASCII files
These are 2 common delimited, text file transformation problems I'd like to find scripts to do (I dont know if awk or perl is best for these tasks).
csh is the default shell on my Solaris10 unix ...
6
votes
7answers
621 views
Slick one-liner to convert a list like “1: 2, 3, 4, 5” to “1.2, 1.3, 1.4, 1.5”
Let's say I have a file that looks something like this:
23: a, b, c, d
24: b, d, f
25: c, g
and I want to get output like this:
23.a
23.b
23.c
23.d
24.b
24.d
24.f
25.c
25.g
Of course it's not ...
3
votes
1answer
42 views
How to edit files in-place while performing operation on the values matching the searched pattern?
Given a list files containing random text and many datetimes in ISO format in it (e.g "2012-07-02T10:47:24+02:00"), how can I find all these datetimes and add 2 hours to each one ?
Ideally a solution ...
0
votes
1answer
156 views
How can I find and filter a specific column in a .csv file? [duplicate]
I have .csv files with the following structure:
cat,dog,mouse,pig,bird,cow,...
21,34,54,566,78,1,...
23,10,12,569,56,4,...
32,20,13,123,56,3,...
34,30,44,322,66,2,...
I want to filter the column ...
1
vote
2answers
65 views
Match ids and print original file
I have two files
Original: (5000 entries)
Chr Position
chr1 879108
chr1 881918
chr1 896874
…
and a file with allele frequencies (2000 entries)
Chr Position MAF
chr1 881918 0.007
...
2
votes
2answers
114 views
Extract Data between comma
I have an output file which contains different numbers seperated by commas. But the issue is I do not know the number of commas.
I would in turn like this data to be grep on another file. For ...
1
vote
2answers
82 views
Print fields from file only if line starts with a specific word
I have the following file:
…
LINK=dummy 172.17.100 => 10.218.11 [PATH=/etc]
FILE=hosts 172.17.100 => 10.218.11 [PATH=/etc]
FILE=network 172.17.100 => 10.218.11 ...
4
votes
4answers
736 views
Text between two tags
I wanna retrieve whatever is between these two tags – <tr> </tr> – from an html doc.
Now I don't have any specific html requirements that would warrant for an html parser. I just plain ...
2
votes
2answers
218 views
System calls, AWK, and Bringing in external inputs
awk '{ TEMPVAR="/usr/bin"; printf("%s", system("ls $TEMPVAR")); }' empty
In this example I'm trying to bring in the variable TEMPVAR into the system call. How would I do this?
What I'm aiming to ...
3
votes
2answers
124 views
AWK : Converting Row Formated Data into Columnar
I have a dataset as such:
Name: Jim Bean
Vice: Dice
ID: AFDSDFDSFDSFASFA
LoginTime: 12343314
Name: Bob Dylon
Vice: Trumpets
ID: AFD232SFDSFASFA
LoginTime: 12343314
...
2
votes
1answer
260 views
How can I use awk to (sometimes) remove the first character of a column?
I run a command and pipe it through awk as below :
mycommand | awk '{print $1,tolower($3),$4}
This gives me the following output:
fred.o: t .abcdefg
fred.o: u .rstuvwxy
fred.o: t .defghi
...
2
votes
2answers
59 views
sort a file based on length of the column/row
I need to sort a file based on the number of chars in the first column.
I have no idea on how to go about this. (On Linux, so sed/awk/sort is available).
An example:
.abs is bla bla 12
.abc is bla ...
1
vote
2answers
268 views
Help inserting a new line of text after matching a line of text (sed)?
I need to insert a hidden HTML input tag into any form tag within a bunch of HTML files. I assume this is possible with sed, but need help forming the command.
My idea is to search for any instance ...
2
votes
2answers
93 views
Awk: expanding first field along the column
I have a task which requires some awk scripting:
Lets say I have a file:
A,under1
,under2
,under3
,under4
,under5
B,new1
,new2
,new3
C,sm1
,sm2
How could I make it look like this using awk?
...
3
votes
4answers
939 views
removing the first and the last character of every line from command line
I am trying to remove the first and the last characters of everyline in a text file and save the resulting truncated version in a new file. Does anyone have an idea about how to do that efficiently ...
9
votes
3answers
363 views
Process last line first using awk
I have a data file that I want to normalize using awk, based on the last datapoint. Therefor, I would like to access the last data point first, to normalize the data, then process normally.
The ...
1
vote
3answers
178 views
Script matching literal pattern over multiple lines?
I have a multi-line string in the variable $PAT. $PAT must be searched for within the file $FILE. If $PAT is in $FILE, it needs to print the file with $PAT removed. If $PAT is not found, then print ...
7
votes
10answers
507 views
What's a good way to filter a text file to remove empty lines?
I have a .csv file (on a mac) that has a bunch of empty lines, e.g.:
"1", "2", "lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum ...
1
vote
2answers
222 views
Deleting specific lines and duplicates from a 11gb wordlist text file
I have an 11gb wordlist file which is already sorted as each word is on its own line.
I need to remove duplicates and lines starting from 077.
I guess I need to run sed and sort -u together but I ...
0
votes
2answers
159 views
Easiest way to find/replace in a file using a list?
I have a file, foo.txt, and a regexp I want to find in that file. Each time I find the regexp, I want to take a line from another file, bar.txt, and substitute it in for the regexp match I found in ...
5
votes
2answers
315 views
Replacing missing value blank space with zero
I have input.txt tab-delimited text file around 30K lines, I would like to check each row (s1..s30K lines) for missing value (i.e blank space) and fill the missing value with zero value.See out.txt
...
1
vote
4answers
336 views
Replace text between brackets
I'm using awk '{ gsub(/BAR|WIBBLE/, "FOO"); print }' to replace text in data like:
SOMETHING [BAR, WIBBLE]
SOMETHING [BAR]
This gives the desired result of:
SOMETHING [FOO, FOO]
SOMETHING [FOO]
...
2
votes
3answers
77 views
Print only lines that are completely numeric
I'd like to filter through a text file and only print the lines where each column is a valid floating point number. For example:
3 6 2 -4.2 21.2
3 x 4.2 21.2
3 2 2.2.2
Only the first line would ...
2
votes
3answers
1k views
Extracting column from comma separated text
I have a long comma-separated delimited file with 20K lines. Here's a sample:
"","id","number1","number2","number3","number4","number5","number6","number7"
...
4
votes
1answer
723 views
Awk to remove line if argument is encountered in a specific column
I need to cycle through an entire file of unknown size and remove any line in which a given word (passed in as argument 1) appears in a specified column. In addition, I need to keep track of how many ...
1
vote
4answers
133 views
How to process these DHCP logs to be human readable?
I have this input:
Sep 23 13:43 192.168.6.200
Sep 23 13:44 192.168.6.166
Sep 23 13:45 192.168.6.200
Sep 23 13:46 192.168.6.166
Sep 23 13:47 192.168.6.200
Sep 23 13:48 192.168.6.166
Sep 23 13:49 ...
4
votes
1answer
689 views
How to find out common elements between two files?
For an example, I have 2 files having following info:
File #1:
12
13
14
15
File #2:
12 1
13 2
14 2
15 6
16 7
17 8
Output File:
1
2
2
6
In the output file, I want only the second column ...
7
votes
5answers
2k views
Remove comma between the quotes only in a comma delimited file
I have a input file delimited with commas (,). There are some fields enclosed in double quotes that are having a comma in them. Here is the sample row
123,"ABC, DEV 23",345,534.202,NAME
I need to ...
4
votes
6answers
248 views
What is the easiest way to check if column A and B values goes both ways with AWK? [closed]
What is the easiest way to check if column A and B values goes both ways?
Output to check:
Mike John
John Mike
Pamela Barbara
Barbara Pamela
Mike Paul
Roger Paul
Desired output
Mike <-> ...
4
votes
2answers
194 views
AWK help for filling up rest of the columns with   in file
I'm trying to get AWK filling up following text file for "empty" columns".
Basic idea of finding longest line with AWK NF and checking which line has no value on that column. Then adding x times N/A ...
5
votes
7answers
250 views
Swapping an unlimited number of columns
I have a file with columns. See below for an example:
a b c ... z
1 2 3 ... 26
I'd like to swap all columns where the 1st becomes the last, the second becomes the one before last...etc..
z y x ...
3
votes
1answer
186 views
grep list of names and information from bigger file
I have two files: one with list of names (500 entries) and other having some more information for each entry in A.txt and extra entries too.
File A.txt (each line is starting with > (fasta format) ...
3
votes
2answers
100 views
Column mismatch and substituting
input.txt (tab-delimted)
TTTTOTTT00000000008 RTTTT899 5.00E-28
TTTTOTTT00000000046 RTTTWRR 3.00E-31
TTTTOTTT00000000051 2.00E-11
TTTTOTTT00000000051 7.00E-12
TTTTOTTT00000000054 ...
2
votes
4answers
2k views
awk-Printing column value without new line and adding comma
input.txt
EN1
EN2
EN3
EN4
EN5
output
EN1,EN2,EN3,EN4,EN5
I have tried awk.But it is not printing with comma
awk 'BEGIN { OFS = ","} { printf $1}' input.txt
I have GNU Awk 4.0.0 version
3
votes
2answers
1k views
AWK-Find maximum value in a row & print with header
I need to read a file, find the maximum value in each row, and print the ID of the row, the column the maximum value came from (sno#), and the value from the associated lc# column. If the maximum ...
2
votes
1answer
187 views
text processing-perl
I am a biologist. I would like help of computer experts to help in text file editing.
Here is what the input file looks like:
##dsfsd2
##sdf-sdf sasg 5.6.3
gi34_ex Gen CDS 161 317 . ...
2
votes
4answers
398 views
shorten long lines in a log file
I keep a logfile of a program's output. The trouble is sometimes errors occur dumping very long lines of 7bit ascii data(encoded binary) I don't care to keep. These lines can be 200KB+ long before ...
3
votes
2answers
159 views
Subtracting one text string from another leaving unique characters, perhaps using awk
Is there any way to subtract one column of data containing text from another column containing text and get third column containing unique charcters, for example using awk eg.
Input
ab a
cd d ...
2
votes
2answers
153 views
How to make a list and count LaTeX keywords in a source file?
I am a beginner, trying to make lists (and eventually count and make statistics) the LaTeX keywords appearing in some documents.
Basically a LaTeX "valid" keyword can be identified by:
The fact ...
