2022 Developer Survey is open! Take survey.

New answers tagged

1 vote

Finding values of one file within range of another file and selecting the top value

Using any awk and any sort: $ cat tst.sh #!/usr/bin/env bash sort -k4,4n -u -- "$1" | awk ' NR==FNR { keys[++numKeys] = $3 vals[numKeys] = $5 next } { for ( k=1; k<=...
user avatar
  • 21.9k
1 vote

Changing part of a field and the delimiter is a know value

Based on your previous question I suspect you have tab-separated input so try this: $ printf '%s\n' "$array" | awk 'BEGIN{FS=OFS="\t"} ($2==0) && c++{s="-"} {$1=$...
user avatar
  • 21.9k
1 vote

How to replace a field by a shell variable being another field?

$ awk -v v="$renamed_column" 'BEGIN{split(v,r)} {$1=r[NR]} 1' <<<"$array1" 2 0.00000 -1.45191 6 0.81778 -0.63413 10 0.85020 -0.60170 8 1.40260 -0.04931 22 3.25781 1.80590 ...
user avatar
  • 21.9k
1 vote

Finding values of one file within range of another file and selecting the top value

Try awk 'NR==FNR {Line[FNR] = $0 # first file processing Cat[FNR] = $1 Min[FNR] = $2 Max[FNR] = $3 Low[FNR] = 1E10 ...
user avatar
  • 8,232
1 vote

Get values from pipe delimited file to the normal text file in Linux

Any reason why your grep approach didn't work? grep -Fwif File.txt Lookup.txt looks promising...?
user avatar
  • 8,232
0 votes

Changing part of a field and the delimiter is a know value

We can use awk to solve it: printf '%s\n' "$array" | awk -F '\t' 'BEGIN{OFS=FS} ($2 ~ /^0\.0+$/ && ++k==2),0 { $1 = $1 "-" }1' - We use the range operator , The range ...
user avatar
  • 5,333
1 vote

Changing part of a field and the delimiter is a know value

Based on your example, what you appear to want to do is keep a count of occurrences of the string 0.00000 (or possibly the numerical value 0) in the second column, and append - to the value of the ...
user avatar
  • 72.1k
0 votes

How to replace a field by a shell variable being another field?

According to comments to this answer, the data is tab-delimited, not space-delimited as it appears in the question. The code below has been modified with this in mind. Using paste and cut instead of ...
user avatar
  • 267k
0 votes

How to replace a field by a shell variable being another field?

If you had the data in the two files array1.txt and renamed_column.txt, you could do something like this: awk -v replfile=renamed_column.txt '{getline x < replfile; $1 = x; print; }' array1.txt &...
user avatar
  • 115k
0 votes

Get values from pipe delimited file to the normal text file in Linux

Store the strings from File.txt as keys in an associative array in awk. For case-insensitive comparison, fold the keys to, e.g., lower-case. When the second file, Lookup.txt, is being read, test ...
user avatar
  • 267k
-1 votes

Identifying genes from a list of genes

You can also use grep command in linux,by giving command like egrep -wi "complete genelist (1) with inducing pipe among genes" "file name containg all genes(2)".Here genelist(1) is ...
user avatar
0 votes

awk print only modified lines

Using Raku (formerly known as Perl_6) raku -ne 'state $i; if .words[1].match( /^ \d**0..3 $/ ) { print .words~" "; put ++$i + 1100 };' This question is similar to a recent question, and ...
user avatar
  • 1,098
0 votes

awk replace column value with increment values if length equals

Using Raku (formerly known as Perl6) raku -ne 'state $i; print .words[0]~" "; put S/^ \d**4 $/{++$i + 12000}/ given .words[1];' Briefly, raku is called at the command line with the -ne (...
user avatar
  • 1,098
1 vote

How to extract specific words out of a file

Assuming you only want to test the values in the 2nd column, you should be doing this: awk '$2 ~ /entry/{print $2}' file
user avatar
  • 21.9k
0 votes

How to match all lines sharing same words

Using GNU sed ##> regex for Mac address h2='[[:xdigit:]]{2}' mac_addr="$h2(:$h2){5}" sed -En " /^(Mamba|),$mac_addr,/G /^[^,]*,($mac_addr),.*\n\1\n/ba /^Mamba,.*\n/!d h s/^...
user avatar
  • 5,333
1 vote
Accepted

Global search + replace, but only in cases where a test passes

Your description says that you have to test any single replacement and keep it or rollback. So you can do the modification in-place, test and in case of failure restore the backup file, do nothing in ...
user avatar
  • 6,180
0 votes
Accepted

How can I batch rename files by combining three variables and ignore file extensions?

There are several problems in your shell script that lead to the errors you observe: You say that the first name part is to be CLSMFILE. However, in the variable assignment you state name1=$CLMSFILE, ...
user avatar
  • 18.4k
2 votes
Accepted

How to match all lines sharing same words

my understanding is get all line with Mamba, remember mac get all line with those mac I came with this awk awk -F, '$1=="Mamba" { m[$2]=NR ; } $2 in m { print ;}' where -F, use , as ...
user avatar
  • 29.3k
1 vote

return a list using grep (or alternative)

You could do something like this, in GNU awk: gawk -F'[[:blank:]]*=[[:blank:]]*' -v key=Exec ' BEGINFILE {value = "0"} $1 == key {value = $2; nextfile} ENDFILE {split(value,v,/[[:...
user avatar
  • 72.1k
2 votes

How to extract specific words out of a file

Using GNU grep iwth the -o option to return only the matched string: grep -o '[^ ]*entry[^ ]*' That will print only words containing the string entry no matter where they are in file or what comes ...
user avatar
  • 9,483
1 vote

How to extract specific words out of a file

When field separator is space character, then: awk -v RS=' ' 'index($0, "entry")' infile
user avatar
  • 38.2k
2 votes
Accepted

How to extract specific words out of a file

Using sed $ sed '/entry/s/[^ ]* *\([^ ]*\).*/\1/' input_file entry1 entry2 entry3 entry4 myentry5 myentry6 yourentry7 yourentry8 ourentry9 ourentry10 Using awk awk '/entry/{print $2}' input_file ...
user avatar
  • 1,092
1 vote

Change text using bash

sed would be easiest: gamemode -s | sed -e 's/gamemode is active/<active icon>/' -e 's/gamemode is inactive/<inactive icon>/'
user avatar
  • 7,117
4 votes

cut /etc/passwd but without colons (:) in the resulting set of fields

My version of cut (cut (GNU coreutils) 8.28) has a --output-delimiter argument: cut -d: -f 1,3,4 --output-delimiter " " /etc/passwd result: root 0 0 daemon 1 1 bin 2 2 sys 3 3
user avatar
  • 141
0 votes

how to ignore negative numbers

awk '$3 ~/^[0-9]/{print $3}' file.txt output 3.5 2.5 2.5
user avatar
0 votes

awk print only modified lines

awk -v k=1100 '{if(length($2) !=4 && length($2)>0){k=k+1;print $1,$2,k }}' file.txt output d1001 100 1101 d1002 10 1102 d1003 1 1103
user avatar
7 votes
Accepted

cut /etc/passwd but without colons (:) in the resulting set of fields

Depending on what output you want The simplest way is to translate the delimiter to what you want with tr, sed or awk... For example cut -d: -f1,3,4 /etc/passwd | tr ':' '\t' cut -d: -f1,3,4 /etc/...
user avatar
  • 1,708
3 votes

Add new lines based on the columns of a tab delimited file

You can do this using GNU sed making use of its extended regex mode -E to aid in writing somewhat easy-to-read regexes. sed -E ' s/\t/\n/2;T s/^([^\t]+\t).*\n/&\1/ P;D ' file Outputs: ...
user avatar
  • 5,333
4 votes

Add new lines based on the columns of a tab delimited file

With awk, loop for the fields after the first one, for every line, and print a new line for every field. awk -F'\t' -v OFS='\t' '{for (i=2;i<=NF;i++) print $1,$i}' file Output: 211845 032 215979 ...
user avatar
  • 6,180
1 vote

awk script replace value with increment with if

I propose this solution: $ awk -F'|' -v OFS='\t' '$2 ~ /^[0-9]{1,3}$/ { $2 = 1100 +(++c) } { print $1,$2 }' file d1000 1000 d1001 1101 d1002 1102 d1003 1103 d1004 d1005
user avatar
4 votes
Accepted

awk script replace value with increment with if

The error may be obvious with more consistent indentation: BEGIN { FS="|"; OFS="\t" } { n=1100 { if (length($2)!=4 && length($2)>0) { $2=++n } }; ...
user avatar
  • 72.1k
5 votes
Accepted

Why does paste command truncate one of the input files?

There shouldn't be a limit. It's just that at least your first input file has DOS/Windows -style CRLF line endings, where the carriage return (CR) returns the cursor position to the start of the line ...
user avatar
  • 115k
2 votes

Replace matched regex in file content with variables

The problem with your sed command is that variables are not expanded inside single quotes, so you need to enclose the sed script in double quotes instead. To illustrate: $ var="foo" $ echo ...
user avatar
  • 215k
5 votes
Accepted

awk print only modified lines

You can print the existing line ($0) and a new field like this, , will use the output separator between the arguments. awk -v n=1100 'length($2)!=4 {print $0,++n}' file Output: d1001 100 1101 d1002 ...
user avatar
  • 6,180
3 votes
Accepted

Replace matched regex in file content with variables

The problem is that in your second sed call, you have enclosed the program in single quotes (' ... '). While this is recommended practice, single quotes prevent the shell from expanding shell ...
user avatar
  • 18.4k
1 vote

awk/sed split a cluster file in to multiple files

using awk+sed combo: awk -v f="wfile_" ' /^>/ && length==5 { if (a++) print p, ",", NR-1, f a-1 p=NR+1 } END {print p, ",$" f a}' < file | split -l 10 ...
user avatar
  • 5,333
1 vote

awk/sed split a cluster file in to multiple files

You can use csplit: csplit --prefix file_ --elide-empty-files --suppress-matched file '/^>....$/' '{*}' It creates 4 files, named file_00 to _03 with the content that you need.
user avatar
3 votes
Accepted

awk/sed split a cluster file in to multiple files

awk '/^>/ && NF==1 {close(out); out="file_"++n; next} {print > out}' file Based on your test input, the header, where you want to change the output file, is defined as: the ...
user avatar
  • 6,180
0 votes

change value of one column with condition from another column

$ awk '/R$/{$4=1}1' file -44.2584 0.2603 42.7879 6 0.1 Precentral_L 49.3816 5.3947 40.4102 1 0.1 Precentral_R -22.5897 9.5277 54.8691 6 0.1 Frontal-Sup_L 26.0365 32.0674 36.7889 1 0.1 Frontal-Sup_R ...
user avatar
  • 21.9k
1 vote

change value of one column with condition from another column

Using sed $ sed '/.*_R/s/\(\([^ ]* *\)\{3\}\)[0-9]/\11/' input_file -44.2584 0.2603 42.7879 6 0.1 Precentral_L 49.3816 5.3947 40.4102 1 0.1 Precentral_R -22.5897 9.5277 54.8691 6 0.1 Frontal-Sup_L ...
user avatar
  • 1,092
1 vote

change value of one column with condition from another column

$ awk '$6 ~ /_R$/ { $4 = 1 }1' input.txt -44.2584 0.2603 42.7879 6 0.1 Precentral_L 49.3816 5.3947 40.4102 1 0.1 Precentral_R -22.5897 9.5277 54.8691 6 0.1 Frontal-Sup_L 26.0365 32.0674 36.7889 1 0....
user avatar
  • 66.6k
2 votes

Swap lines in a file

Using Raku (formerly known as Perl_6) [Adds extra blank line at end]: raku -ne 'my ($L1,$L2,$L3,$L4) = (get) xx 4; .put for $L3, $L1, $L2, $_, ($L4 // "");' #OR raku -e 'my @a.=push($_) ...
user avatar
  • 1,098
0 votes

How to iterate a CSV file in bash?

Fields in a csv file could span several lines, for this reason and others, it's why I preferred to use xsv when I had to parse csv. One way to parse a csv file with bash and xsv can be : csvFile="...
user avatar
  • 11
1 vote

replacement for tr with utf-8 capabilities

Using Raku (formerly known as Perl_6) raku -e 'my @a.=push: .comb(/<alpha>+/)[*-1] if .chars for lines; \ .put for @a.unique>>.flip.sort( *.fc.trans: "àèéìòù" => &...
user avatar
  • 1,098
0 votes

Swap lines in a file

perl one liner perl -lne ' print reverse $_, /./ ? (<>.<>, $,.<>) : (); ' file -n reads files line by line and the line is held inside the $_ variable. the builtin reverse in a ...
user avatar
  • 5,333
0 votes

Decoding URL encoding (percent encoding)

From my laymen research of the topic, it appears that the implementations of the percent-encoding are susceptible to ambiguity in edge cases, such as character encoding potentially being different ...
user avatar
3 votes

replacement for tr with utf-8 capabilities

The limitation of the GNU implementation of tr with regards to multibyte characters and some of its alternatives are covered in tr analog for unicode characters?. Here, you can do everything in awk (...
user avatar
1 vote

all possible matched pairs and associated information

Using any awk in any shell on every Unix box: $ cat tst.awk BEGIN { FS=OFS="," } { map[NR] = $0 } END { n = split(map[1],a) printf "%s%s%s%s", "ID1", OFS, "...
user avatar
  • 21.9k
1 vote

Swap lines in a file

With sed: sed '/^Delegate/{N;N;N;s/\(.*\)\(\n.*\n.*\n\)\(.*\)/\3\2\1/;}' Or, making use of the hold space: sed -ne '/^Delegate/{h;n;N;G;x;n;G;}' -e p
user avatar
1 vote

Replace digits and then characters pattern with sed

Using Raku (formerly known as Perl_6) ~$ raku -pe 's/ "wmd v0.0.0-" <( \d**14 \- \w**12 )> /99999999999999-aaaaaaaaaaaa/' my-file.txt #OR ~$ raku -pe 's[ "wmd v0.0.0-" <...
user avatar
  • 1,098

Top 50 recent answers are included