Filter unprintable "block-like" characters with grep

Question

I have a database controlfile on a Linux system that I want to filter out (for training purposes). However, I am unable to find a proper way to get rid of "block-like" characters:

▒▒▒▒
▒▒▒▒
▒▒▒▒
▒▒▒▒▒
▒▒▒{
▒▒▒▒▒▒9
▒▒▒▒
▒▒▒▒▒

I've tried many ways, but they do not get rid of the block chars:

258     strings o1_mf_d3rrgv0l_.ctl|grep -vE '▒'
259     strings o1_mf_d3rrgv0l_.ctl|grep -vE '@|?|+|(|)|<|>'
260     strings o1_mf_d3rrgv0l_.ctl|grep -vE '@|\?|\+|(|)|<|>'
261     strings o1_mf_d3rrgv0l_.ctl|grep -vE '@|\?|\+|\(|\)|\<|\>'
262     strings o1_mf_d3rrgv0l_.ctl|grep -vE '@|\?|\+|\(|\)|\<|\>|!'
263     strings o1_mf_d3rrgv0l_.ctl|grep -vE '@|\?|\+|\(|\)|\<|\>|!|\^|\%|\`'
264     strings o1_mf_d3rrgv0l_.ctl|grep -vE '@|\?|\+|\(|\)|\<|\>|!|\^|\%|\`|\$'
265     strings o1_mf_d3rrgv0l_.ctl|grep -v '[^[:print:]]'
266     strings o1_mf_d3rrgv0l_.ctl|grep -v '[[:print:]]'
267     strings o1_mf_d3rrgv0l_.ctl|grep  '[[:print:]]'
268     strings o1_mf_d3rrgv0l_.ctl|grep -v  '[[:cntrl:]]'
269     strings o1_mf_d3rrgv0l_.ctl|grep -v '\x{09}'
270     strings o1_mf_d3rrgv0l_.ctl|grep -vP '[^\x00-\x7f]'
271     strings o1_mf_d3rrgv0l_.ctl|tr -dc '\007-\011\012-\015\040-\376'
272     strings -1 o1_mf_d3rrgv0l_.ctl|tr -dc '\007-\011\012-\015\040-\376'
273     strings o1_mf_d3rrgv0l_.ctl|tr -dc '[:print:]\n\r'
274     strings o1_mf_d3rrgv0l_.ctl|grep -vE '@|\?|\+|\(|\)|\<|\>|!|\^|\%|\`|\$'
275     strings o1_mf_d3rrgv0l_.ctl|grep -vE '@|\?|\+|\(|\)|\<|\>|!|\^|\%|\`|\;|\:|\=|\$'
276     strings o1_mf_d3rrgv0l_.ctl|grep -vE '@|\?|\+|\(|\)|\<|\>|!|\^|\%|\`|\;|\:|\=|\$|\"'
277     strings o1_mf_d3rrgv0l_.ctl|grep -vE '@|\?|\+|\(|\)|\<|\>|!|\^|\%|\`|\;|\:|\=|\$|\"|\&|\#'

Answer 1

It may be easier to remove anything except "known good" characters. e.g. to limit output to standard ASCII characters you could use

tr -dc '[^ -~\012\015]'

That will only keep characters between SPACE and ~ (character 126) and the CR/LF characters. Everything else will be removed.

Alternatively you might want to replace them with another character, e.g. a space

tr -c '[^ -~\012\015]' ' '

which will keep any indentation levels

Finally, you might be seeing this because of locale settings; eg if the OS thinks you have UTF8 but the terminal isn't, then you might see this.

So setting LANG=C before running the command might change the output

LANG=C strings o1_mf_d3rrgv0l_.ctl

That'll change what the strings command considers to be a printable character.

Answer 2

After an interesting journey I hope this answer your question, with GNU grep:

Sample file.txt:

▒▒▒▒
▒▒▒▒
▒▒▒▒
foo
bar
@▒^@^@▒▒^@^@^@^@^@^@^@^@^@^@<▒^@^@^@^@@^@^@^@^D~z{|}^@^@^▒^@^@^@^@^@^@^@^@^@^@

---

$ grep -v $(printf %b \\U2592) file.txt
foo
bar