Tag Info

Hot answers tagged

31

You'd think there'd be a utility for that, but I couldn't find it. However, this Perl one-liner should do the trick: perl -pe 's/\e\[?.*?[\@-~]//g' Example: $ command-that-produces-colored-output | perl -pe 's/\e\[?.*?[\@-~]//g' > outfile Or, if you want a script you can save as stripcolorcodes: #! /usr/bin/perl use strict; use warnings; while ...


16

The POSIX 2008 standard has a section describing "Shell and Utilities". Generally, if you stick to that your scripts should be fairly future-proof, except possibly for deprecations, but those hardly happen overnight so you should have plenty of time to update your scripts. In some cases where output format for a single utility varies widely across ...


15

Remove color codes (special characters) with sed sed -r "s/\x1B\[([0-9]{1,2}(;[0-9]{1,2})?)?[m|K]//g" Or Strip ANSI escape sequences in Python Install colorama python package (pip install colorama). Put into stripcolorcodes: #!/usr/bin/env python import colorama, fileinput, sys; colorama.init(strip=True); for line in fileinput.input(): ...


13

cat ./text | awk ' { if ( length > x ) { x = length; y = $0 } }END{ print y }' UPD: summarizing all the advices in the comments awk 'length > max_length { max_length = length; longest_line = $0 } END { print longest_line }' ./text


12

If you can install the Term::ANSIColor module, this perl script works: #!/usr/bin/env perl use Term::ANSIColor qw(colorstrip); print colorstrip $_ while <>;


12

I'll try to answer from my experience. Commands don't really adhere to a formal specification, but they do adhere to a requirement to consume and generate line-oriented text. Yes, of course. Before the GNU utilities became a de facto standard, a lot of vendors would have quirky output, especially with respect to ps and ls. This caused a lot of pain. ...


12

Switching the color is done through escape sequences embedded in the text. Invariably, programs issue ANSI escape sequences, because that's what virtually all terminals support nowadays. The escape sequence to switch the foreground color to red is \e[31m, where \e designates an escape character (octal 033, hexadecimal 1b, also known as ESC, ^[ and various ...


9

First, very brief answers to your questions: Formal standardization of input/output conventions: no Breakage in the past due to changing output: yes Absolutely impossible to break future filters: no How can I protect myself against changes: be conservative When you say "API", you're using a term that (for good or ill) implies too much formality around ...


9

Case 1: awk '!NF {if (++n <= 2) print; next}; {n=0;print}' Case 2: awk '!NF {s = s $0 "\n"; n++; next} {if (n>1) printf "%s", s; n=0; s=""; print} END {if (n>1) printf "%s", s}'


8

This should do (under Linux): ps --ppid 2 -p 2 --deselect kthreadd (PID 2) has PPID 0 (on Linux 2.6+) but ps does not allow to filter for PPID 0; thus this work-around.


7

Instead of matching based on the URL, try matching based on contents of the certificate. iptables -t nat -I INPUT --sport 443 -m string \ --string www.facebook.com --algo bm -j REJECT You can also match on the fingerprint but if the destination changes or updates their certificate, it will invalidate your rule.


7

awk -F '","' 'BEGIN {OFS=","} { if (toupper($5) == "STRING 1") print }' file1.csv > file2.csv Output "12310","42324564756","a simple string with a , comma","string with or, without commas","string 1","USD","12","70%","08/01/2013","" "23525","74535243123","string , with commas, and - hypens and: semicolans","string with or, without commas","string ...


6

You can parse the second part of that filter thusly not ( (src and dest) net localnet ) It's shorthand for not src net localnet and not dest net localnet


6

You can have grep look for control characters, some of which are responsible for making the pretty colors on the terminal. dolongtask | grep '[[:cntrl:]]' For example, this echoes a red "test" into grep, which finds it due to it being surrounded by control characters: $ echo -e '\033[00;31mtest\033[00m' | grep --color=none '[[:cntrl:]]' test <-- ...


5

sed -rn "/.{$(<file expand -t1 |wc -L)}/{p;q}" file This first reads the file inside the command substitution and outputs the length of the longest line, (previously, expand converts tabs to spaces, to overcome the semantics of wc -L -- each tab in the line will add 8 instead of 1 to line length). This length is then used in a sed expression meaning ...


5

The firewall cannot control which HTTPS URLs the client is trying to access, because the URL is encrypted. The firewall can only control which sites the client is connecting to, using IP addresses, but this doesn't help if the HTTP and HTTPS versions of the site are at the same URL (and even if they aren't, you'd have to maintain a huge list of IP ...


5

Use opencv. There are plenty of examples of processing video and face detection.


5

With -d " ", the field separator is one (and only one) space character. Contrary to the shell word splitting, cut doesn't treat space any different than any other character. So cut -d " " -f2 returns "" in root   19, just like it would return "" for cut -d: -f2 in root:::19. You'd need to either squeeze the blanks to transform any sequence of space into one ...


5

You can use uniq to collapse multiple instance of blank lines into one blank line, but it will also collapse lines which contain text if they are the same and below each other.


5

One of the particularity of those processes is that they are not backed by an executable file, so you could do (in zsh): ps /proc/[0-9]*/exe(^-@:h:t) Or with any POSIX shell: ps -p "$(find -L /proc/[0-9]*/exe ! -type l | cut -d/ -f3 | paste -sd, -)" That is check for processes whose /proc/<pid>/exe is a link to a file. But that means you need to ...


5

One way to recognize kernel processes is that they don't use any user memory, so the vsz field is 0. This also catches zombies (thanks to Stephane Chazelas for this observation), which can be eliminated based on their status. ps axl | awk '$7 != 0 && $10 !~ "Z"' To list just the PIDs: ps -e -o pid= -o state= -o vsize= | awk '$2 != "Z" && ...


5

use classic regex: grep -i 'spider\|bot' or extended regex (or even perl regex -P): grep -Ei 'spider|bot' or multiple literal patterns (faster than a regular expression): grep -Fi -e 'spider' -e 'bot'


5

Perhaps awk? awk 'BEGIN{a = "foo"}; a !~ /foo|bar/ && $0 !~ /foo|bar/{print a}; {a = $0};END{if(a !~ /foo|bar/){print a}}' InputFile


4

Only covering 1) of your question. Naturally APIs can always change at the will of their creators, and thusly break dependent software, in any language. That said, the great idea of the Unix tools' I/O "APIs" is that there is practically none (maybe 0x0a as line end). A good script filters data with the Unix tools instead of creating it. That means that ...


4

With GNU ls (the version on non-embedded Linux and Cygwin, sometimes also found elsewhere), you can exclude some files when listing a directory. ls -I 'temp_log.*' -lrt With zsh, you can let the shell do the filtering. Pass -d to ls so as to avoid listing the contents of matched directories. setopt extended_glob # put this in your .zshrc ls ...


4

Try this : grep -v "$(grep -E -B1 "foo|bar" InputFile)" InputFile


3

With sed it is more concise: $ find -maxdepth 1 -type f | sort | sed '/^\.\/reference/,$d' Which means (after sorting) delete the reference line (or greater) and all lines following to the last line. The sed 'd' command is used here with an address range, where '/^./reference/' is the start and '$' is the end of the range. (And '$' as an address means ...


3

eg. script-name "$HOME" "reference" ... find doesn't always output the leading ./, as in the case of find bin, or find /tmp. So if you only want the file base-names, this works. update: Added tolower() to allow for a case insensitive comparison, which produces the alphabetical collation mentioned in the question... #!/bin/bash dir="$1"; ref="$2" ...


3

if you need all of them find . -maxdepth 1 -type f | sort | awk '$0 > "./reference"' if you need the first find . -maxdepth 1 -type f | sort | awk '$0 > "./reference"{print;exit}'


3

OK, if the length of the columns is not known, I'd switch to a more powerful language than bash: #!/usr/bin/perl use warnings; use strict; my $string = shift; open my $FH, '<', '1.txt' or die $!; my $first_line = <$FH>; my ($before, $name) = $first_line =~ /(.* )(NAME *)/; my $column = length $before; $string .= ' ' x (length($name) - length ...



Only top voted, non community-wiki answers of a minimum length are eligible