Tag Info

Hot answers tagged

37

You'd think there'd be a utility for that, but I couldn't find it. However, this Perl one-liner should do the trick: perl -pe 's/\e\[?.*?[\@-~]//g' Example: $ command-that-produces-colored-output | perl -pe 's/\e\[?.*?[\@-~]//g' > outfile Or, if you want a script you can save as stripcolorcodes: #! /usr/bin/perl use strict; use warnings; while ...


19

Remove color codes (special characters) with sed sed -r "s/\x1B\[([0-9]{1,2}(;[0-9]{1,2})?)?[m|K]//g" Or Strip ANSI escape sequences in Python Install colorama python package (pip install colorama). Put into stripcolorcodes: #!/usr/bin/env python import colorama, fileinput, sys; colorama.init(strip=True); for line in fileinput.input(): ...


16

The POSIX 2008 standard has a section describing "Shell and Utilities". Generally, if you stick to that your scripts should be fairly future-proof, except possibly for deprecations, but those hardly happen overnight so you should have plenty of time to update your scripts. In some cases where output format for a single utility varies widely across ...


16

cat ./text | awk ' { if ( length > x ) { x = length; y = $0 } }END{ print y }' UPD: summarizing all the advices in the comments awk 'length > max_length { max_length = length; longest_line = $0 } END { print longest_line }' ./text


14

If you can install the Term::ANSIColor module, this perl script works: #!/usr/bin/env perl use Term::ANSIColor qw(colorstrip); print colorstrip $_ while <>;


13

Switching the color is done through escape sequences embedded in the text. Invariably, programs issue ANSI escape sequences, because that's what virtually all terminals support nowadays. The escape sequence to switch the foreground color to red is \e[31m, where \e designates an escape character (octal 033, hexadecimal 1b, also known as ESC, ^[ and various ...


12

I'll try to answer from my experience. Commands don't really adhere to a formal specification, but they do adhere to a requirement to consume and generate line-oriented text. Yes, of course. Before the GNU utilities became a de facto standard, a lot of vendors would have quirky output, especially with respect to ps and ls. This caused a lot of pain. ...


11

Case 1: awk '!NF {if (++n <= 2) print; next}; {n=0;print}' Case 2: awk '!NF {s = s $0 "\n"; n++; next} {if (n>1) printf "%s", s; n=0; s=""; print} END {if (n>1) printf "%s", s}'


11

This should do (under Linux): ps --ppid 2 -p 2 --deselect kthreadd (PID 2) has PPID 0 (on Linux 2.6+) but ps does not allow to filter for PPID 0; thus this work-around.


11

I'd use awk, but not store the whole content of L.txt in memory and do unnecessary hash look ups ;-). list=L.txt file=F.txt LIST="$list" awk ' function nextline() { if ((getline n < list) <=0) exit } BEGIN{ list = ENVIRON["LIST"] nextline() } NR == n { print nextline() }' < "$file"


10

You can use uniq to collapse multiple instance of blank lines into one blank line, but it will also collapse lines which contain text if they are the same and below each other.


9

First, very brief answers to your questions: Formal standardization of input/output conventions: no Breakage in the past due to changing output: yes Absolutely impossible to break future filters: no How can I protect myself against changes: be conservative When you say "API", you're using a term that (for good or ill) implies too much formality around ...


9

Instead of matching based on the URL, try matching based on contents of the certificate. iptables -t nat -I INPUT --sport 443 -m string \ --string www.facebook.com --algo bm -j REJECT You can also match on the fingerprint but if the destination changes or updates their certificate, it will invalidate your rule.


9

With GNU ls (the version on non-embedded Linux and Cygwin, sometimes also found elsewhere), you can exclude some files when listing a directory. ls -I 'temp_log.*' -lrt With zsh, you can let the shell do the filtering. Pass -d to ls so as to avoid listing the contents of matched directories. setopt extended_glob # put this in your .zshrc ls ...


9

I'd use awk: awk 'NR==FNR {a[$1]; next}; FNR in a' L.txt F.txt Update: I've done performance measures; it seems this version scales even better with very large data sets (as is the case with the stated requirements), since the comparison is very fast and overcompensates the effort necessary to build up the hash table.


9

grep -n | sort | sed | cut ( export LC_ALL=C grep -n '' | sort -t: -nmk1,1 ./L - | sed /:/d\;n | cut -sd: -f2- ) <./F That should work pretty quickly (some timed tests are included below) with input of any size. Some notes on how: export LC_ALL=C Because the point of the following operation is to get the entire file of ./F stacked ...


8

The firewall cannot control which HTTPS URLs the client is trying to access, because the URL is encrypted. The firewall can only control which sites the client is connecting to, using IP addresses, but this doesn't help if the HTTP and HTTPS versions of the site are at the same URL (and even if they aren't, you'd have to maintain a huge list of IP ...


8

awk -F '","' 'BEGIN {OFS=","} { if (toupper($5) == "STRING 1") print }' file1.csv > file2.csv Output "12310","42324564756","a simple string with a , comma","string with or, without commas","string 1","USD","12","70%","08/01/2013","" "23525","74535243123","string , with commas, and - hypens and: semicolans","string with or, without commas","string ...


8

With C omitting meaningful error messages: #include <stdio.h> #include <stdlib.h> int main (int argc, char *argv[]) { FILE *L; FILE *F; unsigned int to_print; unsigned int current = 0; char *line = NULL; size_t len = 0; if ((L = fopen(argv[1], "r")) == NULL) { return 1; } else if ((F = fopen(argv[2], ...


7

You can parse the second part of that filter thusly not ( (src and dest) net localnet ) It's shorthand for not src net localnet and not dest net localnet


6

One of the particularity of those processes is that they are not backed by an executable file, so you could do (in zsh): ps /proc/[0-9]*/exe(^-@:h:t) Or with any POSIX shell: ps -p "$(find -L /proc/[0-9]*/exe ! -type l | cut -d/ -f3 | paste -sd, -)" That is check for processes whose /proc/<pid>/exe is a link to a file. But that means you need to ...


6

Pravin offers some good general points, but doesn't really elaborate on any of them and doesn't address your likely actual problems. First, you need to find out how postfix is receiving those messages and why it's choosing to relay them (the two questions are very likely related). The best way to do it is by looking at the message ID of any one of the ...


6

You can have grep look for control characters, some of which are responsible for making the pretty colors on the terminal. dolongtask | grep '[[:cntrl:]]' For example, this echoes a red "test" into grep, which finds it due to it being surrounded by control characters: $ echo -e '\033[00;31mtest\033[00m' | grep --color=none '[[:cntrl:]]' test <-- ...


5

sed -rn "/.{$(<file expand -t1 |wc -L)}/{p;q}" file This first reads the file inside the command substitution and outputs the length of the longest line, (previously, expand converts tabs to spaces, to overcome the semantics of wc -L -- each tab in the line will add 8 instead of 1 to line length). This length is then used in a sed expression meaning ...


5

Use opencv. There are plenty of examples of processing video and face detection.


5

With -d " ", the field separator is one (and only one) space character. Contrary to the shell word splitting, cut doesn't treat space any different than any other character. So cut -d " " -f2 returns "" in root   19, just like it would return "" for cut -d: -f2 in root:::19. You'd need to either squeeze the blanks to transform any sequence of space into one ...


5

One way to recognize kernel processes is that they don't use any user memory, so the vsz field is 0. This also catches zombies (thanks to Stephane Chazelas for this observation), which can be eliminated based on their status. ps axl | awk '$7 != 0 && $10 !~ "Z"' To list just the PIDs: ps -e -o pid= -o state= -o vsize= | awk '$2 != "Z" && ...


5

use classic regex: grep -i 'spider\|bot' or extended regex (or even perl regex -P): grep -Ei 'spider|bot' or multiple literal patterns (faster than a regular expression): grep -Fi -e 'spider' -e 'bot'


5

Perhaps awk? awk 'BEGIN{a = "foo"}; a !~ /foo|bar/ && $0 !~ /foo|bar/{print a}; {a = $0};END{if(a !~ /foo|bar/){print a}}' InputFile


4

if you need all of them find . -maxdepth 1 -type f | sort | awk '$0 > "./reference"' if you need the first find . -maxdepth 1 -type f | sort | awk '$0 > "./reference"{print;exit}'



Only top voted, non community-wiki answers of a minimum length are eligible