New answers tagged

0

$ join -t, <( sort file1 ) <( sort file2 ) burnj01,Joe Burns,0001 burnj01,Joe Burns,0002 burnj01,Joe Burns,0010 burnj01,Joe Burns,2000 steves02,Santosh Steve,2048 steves02,Santosh Steve,2049 steves02,Santosh Steve,2091 This sorts the two files and passes the sorted contents to the join utility. The join utility does a relational JOIN operation ...


0

You should specify the field separator FS and OFS is used for output field separator. awk 'BEGIN { FS=OFS="," } NR==FNR { saving[$1]=$2; next } ($1 in saving){ print $1, saving[$1], $2 }' file1 file2


4

grep is the simplest command for this. Assuming you have files in one folder then grep 'Total Energy =' dir/* > results.txt Without more information on the way your files are named / organized, I can't be more precise. Hopefully this will let you solve it Other solutions might use sed or awk, but grep is probably the simplest


0

Throwing another one on the pile... csvprintf It works like printf(1). For example, with this input: FirstName,LastName,IdNumber George,Washington,1 Betsy,Ross,2 And this command line: $ cat input.csv | csvprintf -i '%{IdNumber}04d: %{LastName}s, %{FirstName}s\n' You get this output: 0001: Washington, George 0002: Ross, Betsy Disclaimer: I wrote this


0

awk '{if ($0 ~ /puppies/){x=$0;gsub("puppies","bunnies",$0);gsub("I","you",$0);print $0"\n"x}else{print $0}}' filename output you have bunnies cute I have puppies cute I have two kitties you have three bunnies cute I have three puppies cute I have four kitties Python #!/usr/bin/python import re k=re....


0

Test your crashing file with the file-command in Linux and compare it with non-crashing files. Most probable, there is a mismatch in file-encoding, and maybe some unwanted auto-detection involved. Linux offers UTF-8 on most systems since 20 years, and you shouldn't change that. Windows has been later to the party, but more recent installations should use utf-...


3

Some more approaches: sed ' /^1234/!b :1 s/^\(.\{10,19\}\) /\10/ t1' < file perl -pe 'substr($_, 10, 10) =~ y/ /0/ if /^1234/' < file perl -pe 's{^1234.{6}\K.{10}}{$& =~ y/ /0/r}e' < file gawk -v FIELDWIDTHS='10 10 *' -vOFS= ' /^1234/ {gsub(/ /, "0", $2)}; {print}' < file (With perl, add the -C option or -Mopen=...


3

You were almost there...a slight change in your regex plus encasing the s/// in a do-while loop until there are no more spaces left in string position from 11 to 20. sed -e ' :loop s/^\(1234.\{6\}[0-9]\{0,9\}\) /\10/ t loop ' file 12345678909876456000qwerty 12345678909870000000qwerty 23456789011234567 qwerty


1

if a bash script is valid, you coud use the sed quit command: #!/bin/bash sed -E ' # If we find an username /username/ { # make the sobstitution of the username s/^([[:space:]]*username:)/\1toto/g n # Take the next line with the password # Sobstitute the password s/^([[:space:]]*password:).*$/\1totopw/g ...


1

With gnu awk awk -F '^\\s+|:' ' BEGIN { a["username"] = "toto" a["password"] = "totopsw" } ($2 in a) { sub(/:.*/, ": " a[$2]) delete a[$2] };1 ' file set fields separator to be colon or a run of whitespace initilaize array a with keywords to search for n their corresponding ...


5

Another sed alternative: sed '1,/username/{/username/ s/:.*/:toto/}; 1,/password/{/password/ s/:.*/:totopsw/}' infile 1,/regex/ start from the first line up-to first line that matched with given regex (here it's username string), change the "username"; doing the same for "password" part.


4

If the file is longer, here is an awk-based solution that processes it line-wise: awk '/^[[:space:]]+username/ && !u_chng{sub(/:.+$/,": toto"); u_chng=1} /^[[:space:]]+password/ && !p_chng{sub(/:.+$/,": totospw"); p_chng=1} 1' input.yml This will check each line whether it starts with username or password, ...


4

If your file is small enough to fit into memory, you can try reading the entire thing as a single record. That way, sed will only make the substitution for the first time it sees the pattern on the "line" (record): $ sed -Ez 's/(username:)[^\n]*/\1toto/; s/(password:)[^\n]*/\1totopsw/' file.yaml spring: datasource: url: url username:toto ...


0

A few different approaches: Select whatever is between host " and " is: $ sed -En 's/.*host "([^"]*)" is.*/\1/p' file ADMK05786HTREDIRECT_JA3A KOPYUTRCEPORT_JA3A ADMK05786HTSRM_JA3A ADMK05786HTIAS_JA3A ADMK05786HTOTN_JA3A ADMK05786HTWPM OCAIDEKD_JA3A ADMK05786HTSC_JA3A or, using grep with perl compatible regular expressions (-P) ...


1

you are mixing grep's regex and sed's substitution. use either sed -e '/db\.username/s/"[^"]*"/"foo"/' txt db.username="foo" db.password="toto" where /db\.username/ tell sed to operate on line whith db.username (. is special caracter in sed) s replace "[^"]*" a quote, any number of non quote, a ...


0

grep -o '"[^"]\+"' This will return the info between the double quotes. There are definitely more robust ways to do this but if data is always in that format it will work and even if names change it will work.


3

With perl: perl -Mopen=locale -pe 's/\h*\[\d+(\p{dash}\d+)?\]//g' < your-file Where \p{dash} matches characters in the dash punctuation category (such as -֊־᐀᠆‐‑‒–—―⁓⁻₋−⸗⸚⸺⸻⹀〜〰゠︱︲﹘﹣-). That deletes the horizontal whitespace before but not after those [x]. That means however that on your sample, in vivo [47] . Due becomes in vivo . Due. You could improve ...


1

You need to match [ then one or more digits, then ]. The [ and ] are considered special characters in regular expressions so you have to escape them with a leading backslash A digit can be represented either as [0-9] or as [[:digit:]] You need multiple digits so use + to indicate one or more So sed -E 's/\[[[:digit:]]+\]//g' To match a number range, such ...


1

Try this: # it also removes the extra spaces $ sed -e 's/\[[0-9–]*\]//g;s/[[:blank:]]\+/ /g;s/[[:blank:]]\([\.?!:;,]\+\)/\1/g' file Output: Local delivery of a wide selection of growth factors (e.g. platelet-derived growth factor and bone morphogenetic proteins, for example) from electrospun membranes/scaffolds has demonstrated to enhance cell activity in ...


0

gnu sed with branches: sed ' s/puppies/&/g; tfind n :find h s/puppies/bunnies/g s/^I/You/ G ' gawk : awk '$0~/puppies/ { original=$0 sub(/puppies/, "bunnies") sub(/^I/, "You") print $0, original next } { print } '


0

Using the sed editor : sed -e ' h;/puppies/G s//bunnies/ /\n/s/^I/You/ ' file Alternatively with awk awk '{h=$0} sub(/puppies/, "bunnies") { sub(/^I/, "You") $0 = $0 ORS h };1 ' file


1

With GNU sed: sed 'h; s/puppies/bunnies/g; T; s/I/You/; G' < your-file


1

Using sed, and assuming that the trigger is the substring cute at the end of the line: sed '/cute[[:blank:]]*$/ { h; s/^I/You/; s/[[:alpha:]]* cute/bunnies cute/; G; }' file The [[:blank:]]* in the pattern allows for trailing spaces or tabs at the end of the line (the first line has a trailing space in the question). This would produce the text You have ...


1

Using awk: awk 'function prnt() { print edit ORS hold; hold=edit="" } hold { prnt() } /puppies/ { hold=$0; $1="you"; gsub("puppies", "bunnies"); edit=$0; next } END{ if(hold) prnt() }1' infile


0

I came up with a simple example that calculates the SHA256 hash of each file as they appear in the directory. Harmless and easy to verify if it got that wrong. You could use any other processing. #!/bin/sh inotifywait -m "$@" -e create | while read dir event file; do if [ -f "${dir}${file}" ]; then sha256sum "${dir}${file}...


0

Use a soft link ln -s blabla_3300.dat .next. Tell your script to read .next, then rm .next;ln -s blabla3400.dat .next.


0

Gilles had the solution: # xxd -g4 fsl,mf-mix-wakeup-irq 00000000: 07c00000 00007d00 00000000 01400640 ......}......@.@ Note that various od options are invalid in busybox: # od -tx4 fsl,mf-mix-wakeup-irq od: invalid option -- 't' BusyBox v1.31.0 (2021-02-24 17:49:57 UTC) multi-call binary. Usage: od [-aBbcDdeFfHhIiLlOovXx] [FILE]


2

gawk -F, ' { gsub(/"/,"") } FNR==1{ delete timestamp; start=strftime("%Y %m %d 00 00 00", systime()-86400); for(min=0; min<1440; min++) timestamp[strftime("%F %H:%M", mktime(start)+min*60)] $3="Info"; $5="Output"; hdr=$1 FS $3 FS $5; yday=strftime("%...


5

You need to specify the field separator and ensure that you count the fields right. In your case, the field separator is |. But this also implies an "empty" field at the beginning of each line, hence your "actual" field number is 5: ~$ sort -t'|' -k5 file | 2020-10-20T05:46:28.000Z | | eb82a308-543b-4e1a-8f08-e17d2655e5c7 | 2020-10-20T06:...


0

Using any awk in any shell on every UNIX box: From Ed Morton answer, but replacing each character only once (much faster): $ echo 'aaabefhhhhhthkkd' | awk '{ while ( length() ) { # process all letters. let = substr($0,1,1) # extract the first letter if ( gsub(let,"") == 1 ) { # remove ...


1

The configuration file for the Whisker Menu is saved in your xfce4 panel directory: ~/.config/xfce4/panel/whiskermenu-1.rc The actions defined at the bottom of the file contain the same properties as in the "Search Actions" dialog, i.e. name, pattern, command and a boolean regex flag. $ tail -18 ~/.config/xfce4/panel/whiskermenu-1.rc [action2] ...


2

Alternative awk: awk 'BEGIN{ FS=OFS="\t" } NR==FNR{ mutations[$1] =$2; next } FNR>1 { split($3, muts, "," ); for(x in muts) { if (gsub( ",?"muts[x]",?", "", mutations[$2])>0) delete muts[x] } } FNR==1 { $4="missing_mutation"; $5="...


3

The following awk program, which uses GNU awk for convenience (in form of "true double-index arrays"), should do the trick. It is intended to be called with your two files as argument in the order B.txt A.txt (as you already did in your code example). #!/bin/awk -f BEGIN{FS=OFS="\t"} NR==FNR { n_tmp[$1]=split($2,buff,/,/) for (i=...


-1

Command: awk 'NR <5 ||NR>4 && !/^$/' filename output 1 2 3 4 5 5 6 7 8 9


1

There are a number of problems with your command, fortunately they're all easy to resolve: As Quasimodo mentioned in their comment, ’ is not a valid character for delimiting strings within Bourne Again Shell (aka "BASH", the default shell interpreter on Kali Linux). Replacing that with single quotes (') or double quotes (") will be required. ...


0

This uses GNU awk for multi-dimensional arrays: gawk ' BEGIN { FS = "[\t,]" OFS = "\t" } FILENAME == ARGV[1] { for (i = 2; i <= NF; i++) B[$1][$i] = 1 next } FNR == 1 { print $0, "number", "total_mut" next } !($2 in B) { ...


4

sed '1,3b;/./!d' Branch in range 1,3 or delete if empty.


1

With GNU awk for the \< and \> (Word-boundaries anchors): awk 'BEGIN{ FS=OFS="\t" } NR==FNR{ mutations[$1] =$2; next } { split($3, muts, "," ); for(x in muts) { tmp=mutations[$2]; c+=sub( "\\<"muts[x]"\\>", "", tmp) } } FNR==1 { c="number" } { print $0, (c?c:""); c=0 }...


7

Another awk solution: awk 'NR<=3||NF' file This will print all lines where either the line number is smaller or equal 3, or which have at least one "field", defined by default as at least one non-blank character. This way, you can also exclude lines that contain no "text" but are only "visually" empty, i.e. that still ...


3

You were almost there. Just needed to add the following steps. sed -ne '4,${s~^$~~;t};p' < "${File}" select the range 4,$ try the dummy substitution s/^$// in case the s/// succeeds, skip to next record t else fall back n print for either these in-range nonempty lines or all out-of-range lines };p


2

an awk solution: awk '((NR > 3 && $0!~/^$/) || NR <= 3) { print }' file


1

awk 'BEGIN{ OFS=FS="\t" } NR==FNR{ clade[$1]=$2; next } # save clade, mutation of B.tsv in array FNR==1{ print $0, "number"; next } # print header !($2 in clade){ print; next } # no match -> print record { # else... split($3 "," clade[$2], tmp, ",") ...


8

try: sed '4,$ {/^$/d}' infile start from the 4th line until end of the file, delete the empty lines. The problem with your command is that you replace empty line again with empty string in s~^$~~g (which is same as s~^$~~) and you are not deleting it. Note: also since you use different delimiter other than default slash /, to use this style ~^$~d you need ...


0

The method is to make an array indexed by clade and mutation, from the B file. Then iterate the mutations from the A file. Somewhat tricky to deal with a tab-separated file, especially keeping the number of columns where there is no clade. We define the necessary column numbers for the A file as cClade and cMut, and changed these to match the full data ...


3

Try: awk -F, -v yesterday="$(date -d'-1day' +'%F')" ' BEGIN{ for(min=0; min<1440; min++){ mins = "date +%F\" " "\"%T -d\"" min "minutes" yesterday"\"" mins |getline yday_tmp; close(mins); timestamp["\"" yday_tmp "\""] } ...


0

A brute-force approach, to fill in the gaps from your existing solution: it tests 1440 files, but, hey, it's only once a day, and it's yesterday date. In Bash: (a) Get yesterday's date in both formats: dt=20210103 and 2021-01-03. (b) Run a for loop: for hm in {00..23}{00..59}. (c) For any file ${dt}${hm}00.csv not present: printf the required header and ...


0

Using Emacs, you could first select all text in your buffer. Then invoke M-x downcase-region


1

$ cat tst.awk BEGIN { FS=OFS="\t" } { key = (FNR>1 ? $1 : RS) } NR == FNR { $1 = "" map[key] = $0 next } { print $0 map[key] } $ awk -f tst.awk B.tsv A.tsv Sample ID Internal Control Result Consensus clade substitutions deletions 4686427 Pass Not Detected Not Available 161 no yes ...


3

Use join join -t, -a1 B.csv C.csv The -a1 means left outer join (i.e show lines from file1 that are not in file2) If commas at the end of unpaired lines really matter (join -t, B.csv C.csv ; join -t, -v1 B.csv C.csv | perl -pe "s/$/,/" ) | sort


3

Using awk without dis-ordering lines of the original files, but require to loading the first file into the memory and you would need to care to don't run on big file that cannot fit into your memory. awk 'BEGIN { FS=OFS="," } NR==FNR { hold[$1]=$2; next } { print $0, hold[$1] }' fileC fileB for such a case when there was a key ...


Top 50 recent answers are included