Hot answers tagged

8

We use the awk utility to employ flip flop logic to toggle from on state to off as shown: $ awk -v str="==" ' $0""==str{f=!f;next};f ' testfile Last test Using Posixly sed constructs we can implement the flip flop logic: sed -ne ' /^==$/{ x; # access state info from hold s/^$/0/; # initialize state y/01/10/; #...


5

Using GNU datamash: $ datamash -t ' ' -s -g 1 collapse 2 <file A 1,2,9 ABC 12,6 B 4,3 C 5 Options: -t '_' use the space character as field delimiter -s sort the input before grouping -g 1 group on the first field collapse 2 collapse values of the second field into comma-separated list


5

$ awk '{ object[$1]= (object[$1]==""?"":object[$1] ",") $2 } END { for(obj in object) print obj, object[obj] }' infile A 1,2,9 B 4,3 C 5 ABC 12,6 A bit more efficient (use of the memory; matter for a huge file that cannot fit into the memory), i.e not buffering the file partially into the memory that awk command alone does ...


5

In awk, you could collect the line numbers to an array and read through the file once, printing the lines that are mentioned in the array: #!/bin/sh awk -v lines="$*" 'BEGIN { split(lines, a, "[, ]"); for (i in a) b[a[i]] = 1;} NR in b {print $1, $2}' < data.txt The split() splits the ...


3

I'd use perl: $ perl -0777 -ne 'print $3 while /^((\S)\2+\n)(.*?)^\1/smg' < your-file This is a test Another test Last test Or pcregrep: $ pcregrep -Mo3 '(?s)^((\S)\2+\n)(.*?)\n?^\1' < your-file This is a test Another test Last test If it's only about returning what's between fixed delimiters: $ pcregrep -Mo1 '(?s)^==\n(.*?)\n?^==$' < your-file ...


3

The last line of your script is not identical to the one that works on the command line. There is no reason to add those two backslashes. s/(\[0-9]+\.[0-9]{2}\)[0-9]+/\1/ should be s/([0-9]+\.[0-9]{2})[0-9]+/\1/ (also, you don't need to escape the semi-colon, in your first s/// command, but this is not an issue, as it is just ignored)


3

A bit awkward, but I think it gets the job done. The csvkit Python package contains command line tools for working with CSV files. Among these is csvcut, which works like the standard cut command, but which is CSV aware, which means that it allows for fields that may contain embedded field delimiters and newlines, as long as they are properly quoted. With ...


3

tl;dr $ sed '/^==*$/,//{//!p};d' testfile This is a test Another test Last test On first look, a simple range could print all pairs (no loops needed): $ sed -n '/^=/,//p' testfile === This is a test === ==== Another test ==== == Last test == That prints every line between a line that starts with = and the next repeated regex (\\). That could be improved to ...


3

Seems like your question wasn't clear and you got various types of solution. To get only content surrounded by == with GNU sed (syntax/feature might vary with other implementations): $ sed -n '/^==$/,//{//!p}' testfile Last test sed has a way to select a range of lines by specifying two addresses separated by a comma. To avoid repeating the regexp you can ...


2

Using awk in place of sed: newdata='This is the new data' newdata=$newdata awk -F '\n' -v OFS='\n' -v RS= -v ORS='\n\n' \ '$1 == "# SECTION ONE" { $(NF+1) = ENVIRON["newdata"] }; 1' file or newdata='This is the new data' newdata=$newdata awk ' BEGIN { FS = OFS = "\n"; RS = ""; ORS = "\n\n" } $1 ...


2

Perl perl -MList::MoreUtils=all -F'\t' -lane ' print if all { /^comp-/ } @F[0,1]; ' file sed GNU version $ sed -n '/^comp-.*\tcomp-/p' file


2

awk -F'\t' '$1 ~/^comp-/ && $2 ~/^comp-/' infile same but pass the pattern from a parameter: awk -F'\t' -v pat='comp-' '$1 ~"^" pat && $2 ~"^" pat' infile or compare as string match and still pass from a parameter: awk -F'\t' -v str='comp-' 'index($1, str)==1 && index($2, str)==1' infile see also How do I find ...


2

command: awk '{a[++i]=$0}/==/{for(x=NR-1;x<NR;x++)print a[x]}' filename|sed '/^$/d' output This is a test Another test Last test


2

Perl to the rescue! perl -C -pe 's/[\x{80}-\x{ffffffff}]/0/g' -p reads the input line by line, running the code for each and printing it after processing -C turns on UTF-8


2

Using any awk in any shell on every Unix box: $ awk -F, -v OFS='|' 'NF>7{$5=$5",\n"; sub(/\n./,"")} 1' file Name,Age,Country,ID,Address,Category,DOB John Doe|19|England|3653|Manchester, England|Main Worker|20-05-1995 $5=$5",\n" adds a , and a newline (which can't otherwise be present in the current record since records are ...


2

With awk you can try: awk -F, -v OFS='|' 'NR==1{print $0} NR>1{$5 = $5 FS $6; print $1,$2,$3,$4,$5,$7,$8}' file Name,Age,Country,ID,Address,Category,DOB John Doe|19|England|3653|Manchester, England|Main Worker|20-05-1995


2

With bash, one can do this, which handles an arbitrary number of commas in the address field: # function to join strings with a separator join() { local IFS=$1 shift printf '%s\n' "$*" } # process the file { IFS=, read -ra header join '|' "${header[@]}" f=${#header[@]} # expected num of ...


2

You haven't really explained how we can know which commas to keep and which to change. Based on the single example line you have given us, it might be enough to just replace all , that don't come after a space with a |: $ sed -E 's/,(\S)/\|\1/g' file Name|Age|Country|ID|Address|Category|DOB John Doe|19|England|3653|Manchester, England|Main Worker|20-05-1995 ...


1

On a POSIX compliant system, and in a locale that uses the UTF-8 charset you can do: tr -c '\0-\177' '[0*]' < file However note that the GNU implementation of tr at least is not compliant in that regard as it doesn't support multibyte characters yet. On GNU systems, you can use GNU awk for that though where, in recent versions ranges are based on code ...


1

This solved the issue ssh $q "sed -i '/apache$/a sshd: 192.168.2.2 : allow' /etc/hosts.allow" &>> log.txt


1

$ perl -F, -lne 'if ($#F == 7) {$F[4] .= ",$F[5]"; splice @F,5,1}; print join("|",@F);' input.csv Name|Age|Country|ID|Address|Category|DOB John Doe|19|England|3653|Manchester, England|Main Worker|20-05-1995 This perl one-liner checks to see how many comma-separated fields are in each input line. If there are 8 fields, ...


1

#!/bin/bash perl -le ' for (@ARGV) { # separate command line args into filename(s) and line-number(s) # line-numbers can be space and/or comma separated. if (-e $_) { push @files, $_ } else { push @lines, split /,/}; }; @ARGV = @files; $re = join("|",@lines); while(<>) { print join("\t",(split)[0..1]) if ($. =~ m/^($re)$/)...


1

The following bash script builds a sed script that will modify and display the lines by the numbers provided on the command line: #!/bin/bash sed_script=() for lineno do sed_script+=( -e "$lineno ba" ) done sed "${sed_script[@]}" \ -e 'd' -e ':a' -e 's/[[:blank:]]*[^[:blank:]]*$//' <data.txt or, for /bin/sh: #!/bin/...


1

Put the below lines in a text file and name it as get.sh. then make it executable . #!/bin/sh ## this is GNU sed sed -En " $(printf '%sbp\n' "$@" "d;") :p;s/\S+/&\n/2;P " data.txt Now invoke the script as shown: chmod +x ./get.sh ./get.sh 1 3 5


1

Using GNU sed we can do as follows: sed -e '/# SECTION ONE/,/^$/s/^$/__NEW-DATA__\n/' file Range operator in awk can be used to get the desired output awk ' /# SECTION ONE/,!NF{ if (!NF) print "__NEW-DATA__" }1 ' file This method uses POSIXly sed constructs sed -e ' /# SECTION ONE/!b :a n;/^$/!ba G;s/^/__NEW-DATA__/ ' file In ...


1

I wasn't able to do this with sed because the use of commas as the delimiter as well as the comma in the Name field not being a delimiter made it too complicated so I used awk with the gsub option: awk -F ',' 'NR==1; NR>1{gsub("\.0","",$7);print} ' OFS="," file This is somewhat messy given the circumstances but it sets a ...


1

Adding a fake root node around your broken XML is easy enough, modifying the modified XML would be a breeze with xmlstarlet, and deleting the added root node (and the <?xml version="1.0"?> that xmlstarlet adds) once you've modified it is not too hard either: { echo '<root>'; cat file.xml; echo '</root>'; } | xmlstarlet ed -u '//...


1

Assuming you are on GNU sed version of the sed editor. Try to null the path d data in both fileold and filenew files before diffing. So you could do something along these lines: sed -i ' /<path d=/c\ <path d=/> ' -- "$fileold" "$filenew"; Or if you must need to make sure that the characters in between the quotes are only ...


1

You need to protect the backslashes from the shell on the command line either by using quotes OR double them up. myscript.sh 'my_server\/my_repo\/my_image' ..... Or myscript.sh my_server\\/my_repo\\/my_image


1

GNU sed in extended regex mode -E we use the range operator , to select the right block and within that block use the regex of the s/regex/repl/ to zero in on the right line to make the edits. sed -Ee ' /\[ req_distinguished_name ]/,/^$/s/^(\s*commonName\s*=).*/\1__NEWVALUE__/ ' file


Only top voted, non community-wiki answers of a minimum length are eligible