New answers tagged text-processing
1
vote
Finding values of one file within range of another file and selecting the top value
Using any awk and any sort:
$ cat tst.sh
#!/usr/bin/env bash
sort -k4,4n -u -- "$1" |
awk '
NR==FNR {
keys[++numKeys] = $3
vals[numKeys] = $5
next
}
{
for ( k=1; k<=...
- 21.9k
1
vote
Changing part of a field and the delimiter is a know value
Based on your previous question I suspect you have tab-separated input so try this:
$ printf '%s\n' "$array" | awk 'BEGIN{FS=OFS="\t"} ($2==0) && c++{s="-"} {$1=$...
- 21.9k
1
vote
How to replace a field by a shell variable being another field?
$ awk -v v="$renamed_column" 'BEGIN{split(v,r)} {$1=r[NR]} 1' <<<"$array1"
2 0.00000 -1.45191
6 0.81778 -0.63413
10 0.85020 -0.60170
8 1.40260 -0.04931
22 3.25781 1.80590
...
- 21.9k
1
vote
Finding values of one file within range of another file and selecting the top value
Try
awk 'NR==FNR {Line[FNR] = $0 # first file processing
Cat[FNR] = $1
Min[FNR] = $2
Max[FNR] = $3
Low[FNR] = 1E10 ...
- 8,232
1
vote
Get values from pipe delimited file to the normal text file in Linux
Any reason why your grep approach didn't work?
grep -Fwif File.txt Lookup.txt
looks promising...?
- 8,232
0
votes
Changing part of a field and the delimiter is a know value
We can use awk to solve it:
printf '%s\n' "$array" |
awk -F '\t' 'BEGIN{OFS=FS}
($2 ~ /^0\.0+$/ && ++k==2),0 {
$1 = $1 "-"
}1' -
We use the range operator ,
The range ...
- 5,333
1
vote
Changing part of a field and the delimiter is a know value
Based on your example, what you appear to want to do is keep a count of occurrences of the string 0.00000 (or possibly the numerical value 0) in the second column, and append - to the value of the ...
- 72.1k
0
votes
How to replace a field by a shell variable being another field?
According to comments to this answer, the data is tab-delimited, not space-delimited as it appears in the question. The code below has been modified with this in mind.
Using paste and cut instead of ...
- 267k
0
votes
How to replace a field by a shell variable being another field?
If you had the data in the two files array1.txt and renamed_column.txt, you could do something like this:
awk -v replfile=renamed_column.txt '{getline x < replfile; $1 = x; print; }' array1.txt &...
- 115k
0
votes
Get values from pipe delimited file to the normal text file in Linux
Store the strings from File.txt as keys in an associative array in awk. For case-insensitive comparison, fold the keys to, e.g., lower-case.
When the second file, Lookup.txt, is being read, test ...
- 267k
-1
votes
Identifying genes from a list of genes
You can also use grep command in linux,by giving command like egrep -wi "complete genelist (1) with inducing pipe among genes" "file name containg all genes(2)".Here genelist(1) is ...
0
votes
awk print only modified lines
Using Raku (formerly known as Perl_6)
raku -ne 'state $i; if .words[1].match( /^ \d**0..3 $/ ) { print .words~" "; put ++$i + 1100 };'
This question is similar to a recent question, and ...
- 1,098
0
votes
awk replace column value with increment values if length equals
Using Raku (formerly known as Perl6)
raku -ne 'state $i; print .words[0]~" "; put S/^ \d**4 $/{++$i + 12000}/ given .words[1];'
Briefly, raku is called at the command line with the -ne (...
- 1,098
1
vote
How to extract specific words out of a file
Assuming you only want to test the values in the 2nd column, you should be doing this:
awk '$2 ~ /entry/{print $2}' file
- 21.9k
0
votes
How to match all lines sharing same words
Using GNU sed
##> regex for Mac address
h2='[[:xdigit:]]{2}'
mac_addr="$h2(:$h2){5}"
sed -En "
/^(Mamba|),$mac_addr,/G
/^[^,]*,($mac_addr),.*\n\1\n/ba
/^Mamba,.*\n/!d
h
s/^...
- 5,333
1
vote
Accepted
Global search + replace, but only in cases where a test passes
Your description says that you have to test any single replacement and keep it or rollback. So you can do the modification in-place, test and in case of failure restore the backup file, do nothing in ...
- 6,180
0
votes
Accepted
How can I batch rename files by combining three variables and ignore file extensions?
There are several problems in your shell script that lead to the errors you observe:
You say that the first name part is to be CLSMFILE. However, in the variable assignment you state name1=$CLMSFILE, ...
- 18.4k
2
votes
Accepted
How to match all lines sharing same words
my understanding is
get all line with Mamba, remember mac
get all line with those mac
I came with this awk
awk -F, '$1=="Mamba" { m[$2]=NR ; } $2 in m { print ;}'
where
-F, use , as ...
- 29.3k
1
vote
return a list using grep (or alternative)
You could do something like this, in GNU awk:
gawk -F'[[:blank:]]*=[[:blank:]]*' -v key=Exec '
BEGINFILE {value = "0"}
$1 == key {value = $2; nextfile}
ENDFILE {split(value,v,/[[:...
- 72.1k
2
votes
How to extract specific words out of a file
Using GNU grep iwth the -o option to return only the matched string:
grep -o '[^ ]*entry[^ ]*'
That will print only words containing the string entry no matter where they are in file or what comes ...
- 9,483
1
vote
How to extract specific words out of a file
When field separator is space character, then:
awk -v RS=' ' 'index($0, "entry")' infile
- 38.2k
2
votes
Accepted
How to extract specific words out of a file
Using sed
$ sed '/entry/s/[^ ]* *\([^ ]*\).*/\1/' input_file
entry1
entry2
entry3
entry4
myentry5
myentry6
yourentry7
yourentry8
ourentry9
ourentry10
Using awk
awk '/entry/{print $2}' input_file
...
- 1,092
1
vote
Change text using bash
sed would be easiest:
gamemode -s | sed -e 's/gamemode is active/<active icon>/' -e 's/gamemode is inactive/<inactive icon>/'
- 7,117
4
votes
cut /etc/passwd but without colons (:) in the resulting set of fields
My version of cut (cut (GNU coreutils) 8.28) has a --output-delimiter argument:
cut -d: -f 1,3,4 --output-delimiter " " /etc/passwd
result:
root 0 0
daemon 1 1
bin 2 2
sys 3 3
- 141
0
votes
0
votes
awk print only modified lines
awk -v k=1100 '{if(length($2) !=4 && length($2)>0){k=k+1;print $1,$2,k }}' file.txt
output
d1001 100 1101
d1002 10 1102
d1003 1 1103
- 4,698
7
votes
Accepted
cut /etc/passwd but without colons (:) in the resulting set of fields
Depending on what output you want
The simplest way is to translate the delimiter to what you want with tr, sed or awk... For example
cut -d: -f1,3,4 /etc/passwd | tr ':' '\t'
cut -d: -f1,3,4 /etc/...
- 1,708
3
votes
Add new lines based on the columns of a tab delimited file
You can do this using GNU sed making use of its extended regex mode -E to aid in writing somewhat easy-to-read regexes.
sed -E '
s/\t/\n/2;T
s/^([^\t]+\t).*\n/&\1/
P;D
' file
Outputs:
...
- 5,333
4
votes
Add new lines based on the columns of a tab delimited file
With awk, loop for the fields after the first one, for every line, and print a new line for every field.
awk -F'\t' -v OFS='\t' '{for (i=2;i<=NF;i++) print $1,$i}' file
Output:
211845 032
215979 ...
- 6,180
1
vote
awk script replace value with increment with if
I propose this solution:
$ awk -F'|' -v OFS='\t' '$2 ~ /^[0-9]{1,3}$/ { $2 = 1100 +(++c) } { print $1,$2 }' file
d1000 1000
d1001 1101
d1002 1102
d1003 1103
d1004
d1005
- 11.2k
4
votes
Accepted
awk script replace value with increment with if
The error may be obvious with more consistent indentation:
BEGIN { FS="|"; OFS="\t" }
{
n=1100
{
if (length($2)!=4 && length($2)>0) {
$2=++n
}
};
...
- 72.1k
5
votes
Accepted
Why does paste command truncate one of the input files?
There shouldn't be a limit. It's just that at least your first input file has DOS/Windows -style CRLF line endings, where the carriage return (CR) returns the cursor position to the start of the line ...
- 115k
2
votes
Replace matched regex in file content with variables
The problem with your sed command is that variables are not expanded inside single quotes, so you need to enclose the sed script in double quotes instead. To illustrate:
$ var="foo"
$ echo ...
- 215k
5
votes
Accepted
awk print only modified lines
You can print the existing line ($0) and a new field like this, , will use the output separator between the arguments.
awk -v n=1100 'length($2)!=4 {print $0,++n}' file
Output:
d1001 100 1101
d1002 ...
- 6,180
3
votes
Accepted
Replace matched regex in file content with variables
The problem is that in your second sed call, you have enclosed the program in single quotes (' ... '). While this is recommended practice, single quotes prevent the shell from expanding shell ...
- 18.4k
1
vote
awk/sed split a cluster file in to multiple files
using awk+sed combo:
awk -v f="wfile_" '
/^>/ && length==5 {
if (a++) print p, ",", NR-1, f a-1
p=NR+1
}
END {print p, ",$" f a}' < file |
split -l 10
...
- 5,333
1
vote
awk/sed split a cluster file in to multiple files
You can use csplit:
csplit --prefix file_ --elide-empty-files --suppress-matched file '/^>....$/' '{*}'
It creates 4 files, named file_00 to _03 with the content that you need.
- 11.2k
3
votes
Accepted
awk/sed split a cluster file in to multiple files
awk '/^>/ && NF==1 {close(out); out="file_"++n; next} {print > out}' file
Based on your test input, the header, where you want to change the output file, is defined as: the ...
- 6,180
0
votes
change value of one column with condition from another column
$ awk '/R$/{$4=1}1' file
-44.2584 0.2603 42.7879 6 0.1 Precentral_L
49.3816 5.3947 40.4102 1 0.1 Precentral_R
-22.5897 9.5277 54.8691 6 0.1 Frontal-Sup_L
26.0365 32.0674 36.7889 1 0.1 Frontal-Sup_R
...
- 21.9k
1
vote
change value of one column with condition from another column
Using sed
$ sed '/.*_R/s/\(\([^ ]* *\)\{3\}\)[0-9]/\11/' input_file
-44.2584 0.2603 42.7879 6 0.1 Precentral_L
49.3816 5.3947 40.4102 1 0.1 Precentral_R
-22.5897 9.5277 54.8691 6 0.1 Frontal-Sup_L
...
- 1,092
1
vote
change value of one column with condition from another column
$ awk '$6 ~ /_R$/ { $4 = 1 }1' input.txt
-44.2584 0.2603 42.7879 6 0.1 Precentral_L
49.3816 5.3947 40.4102 1 0.1 Precentral_R
-22.5897 9.5277 54.8691 6 0.1 Frontal-Sup_L
26.0365 32.0674 36.7889 1 0....
- 66.6k
2
votes
Swap lines in a file
Using Raku (formerly known as Perl_6)
[Adds extra blank line at end]:
raku -ne 'my ($L1,$L2,$L3,$L4) = (get) xx 4; .put for $L3, $L1, $L2, $_, ($L4 // "");'
#OR
raku -e 'my @a.=push($_) ...
- 1,098
0
votes
How to iterate a CSV file in bash?
Fields in a csv file could span several lines, for this reason and others, it's why I preferred to use xsv when I had to parse csv.
One way to parse a csv file with bash and xsv can be :
csvFile="...
- 11
1
vote
replacement for tr with utf-8 capabilities
Using Raku (formerly known as Perl_6)
raku -e 'my @a.=push: .comb(/<alpha>+/)[*-1] if .chars for lines; \
.put for @a.unique>>.flip.sort( *.fc.trans: "àèéìòù" => &...
- 1,098
0
votes
Swap lines in a file
perl one liner
perl -lne '
print reverse $_, /./ ? (<>.<>, $,.<>) : ();
' file
-n reads files line by line and the line is held inside the $_ variable.
the builtin reverse in a ...
- 5,333
0
votes
Decoding URL encoding (percent encoding)
From my laymen research of the topic, it appears that the implementations of the percent-encoding are susceptible to ambiguity in edge cases, such as character encoding potentially being different ...
- 1,697
3
votes
replacement for tr with utf-8 capabilities
The limitation of the GNU implementation of tr with regards to multibyte characters and some of its alternatives are covered in tr analog for unicode characters?.
Here, you can do everything in awk (...
- 466k
1
vote
all possible matched pairs and associated information
Using any awk in any shell on every Unix box:
$ cat tst.awk
BEGIN { FS=OFS="," }
{ map[NR] = $0 }
END {
n = split(map[1],a)
printf "%s%s%s%s", "ID1", OFS, "...
- 21.9k
1
vote
Swap lines in a file
With sed:
sed '/^Delegate/{N;N;N;s/\(.*\)\(\n.*\n.*\n\)\(.*\)/\3\2\1/;}'
Or, making use of the hold space:
sed -ne '/^Delegate/{h;n;N;G;x;n;G;}' -e p
- 466k
1
vote
Replace digits and then characters pattern with sed
Using Raku (formerly known as Perl_6)
~$ raku -pe 's/ "wmd v0.0.0-" <( \d**14 \- \w**12 )> /99999999999999-aaaaaaaaaaaa/' my-file.txt
#OR
~$ raku -pe 's[ "wmd v0.0.0-" <...
- 1,098
Top 50 recent answers are included