Replace all characters except the first four characters

Question

I want to pipe in a command to sed like so:

md5sum input.txt | sed 's/^\(....\).*/\1/;q'

This works by only outputting the first 4 characters of the checksum. However, I want to output the first 4 characters, but also have an x in the place of every other characters (redacting info). I'm so lost now.

Answer 1

With GNU Sed,

md5sum input.txt | sed 's/./x/5g'

This simply skips substituting the 4 first characters of the string and performs the substitution for all other characters.

A POSIX alternative with Awk (although there is probably something simpler),

md5sum xad | awk '{
  four=substr($0, 1, 4)
  rest=substr($0, 5)
  gsub(/./, "x", rest)
  print four, rest
}' OFS=""

Answer 2

POSIXly (I think), you could use a sed loop to repeatedly replace the first non-x character following the 4-character prefix:

$ md5sum input.txt | sed '
:a
s/^\(....x*\)[^x]/\1x/
ta
'

Replace [^x] with [^x ] if you only want to do the substitution in the first field (the checksum).

Answer 3

With perl if GNU sed isn't available:

md5sum input.txt | perl -pe 's/^.{4}(*SKIP)(*F)|./x/g'

^.{4}(*SKIP)(*F) will prevent replacement of first four characters

|. specifies the alternate pattern that has to be replaced

To change only the checksum:

md5sum ip.txt | perl -pe 's/(^.{4}|\h.*$)(*SKIP)(*F)|./x/g'

If the md5sum output starts with a \ (for ex: if filename has a newline character), then you can use ^\\?.{4} instead of ^.{4} to allow first five characters to be left unmasked.

Answer 4

Another perl variant to replace all but the first 4 bytes of the checksum (and the checksum only) with x:

$ md5sum input.txt | perl -pe 's{.{4}\K\S+}{$& =~ s/./x/gr}e'
d632xxxxxxxxxxxxxxxxxxxxxxxxxxxx  input.txt

Answer 5

The problem with Quasímodo's answer is that it also replaces the filename with x's. OP posted a followup question about that. Here's a sed solution that stops at the space:

md5sum always produces a 32-character output for the hash. Instead of detecting a space, you could look for 32-characters then a space and replace the last 28 characters with an X.

md5sum input.txt | sed 's/^\([a-zA-Z0-9]\{4\}\)[a-zA-Z0-9]\{28\} /\1xxxxxxxxxxxxxxxxxxxxxxxxxxxx /g'
35c9xxxxxxxxxxxxxxxxxxxxxxxxxxxx  input.txt

Breaking down the statement:

's/^\([a-zA-Z0-9]\{4\}\)[a-zA-Z0-9]\{28\} /\1xxxxxxxxxxxxxxxxxxxxxxxxxxxx /g'

's/ A                                     / B                             /g'
we're substituting patterns matching A with B globally

's/   [a-zA-Z0-9]       [a-zA-Z0-9]       /                               /g'
we're looking for two groups of alphanumeric  characters

's/   [a-zA-Z0-9]\{4\}  [a-zA-Z0-9]\{28\} /                               /g'
The first group has exactly four characters
The second group has exactly twenty-eight characters

's/ \([a-zA-Z0-9]\{4\}\)[a-zA-Z0-9]\{28\} /                               /g'
The first group is a "capture group" which we can reference later

's/ \([a-zA-Z0-9]\{4\}\)[a-zA-Z0-9]\{28\} /\1                             /g'
We will print out the first group verbatim in the output

's/ \([a-zA-Z0-9]\{4\}\)[a-zA-Z0-9]\{28\} /\1xxxxxxxxxxxxxxxxxxxxxxxxxxxx /g'
We will print x's followed by a space for the next 28 characters

's/^\([a-zA-Z0-9]\{4\}\)[a-zA-Z0-9]\{28\} /\1xxxxxxxxxxxxxxxxxxxxxxxxxxxx /g'
The statement must appear at the start of a line and have a space at the end.

Answer 6

The MD5 hash is always 32 characters long. The first four characters may be had by cutting with cut -c -4, and the 28 x required to pad out the rest of the hash may be added with printf:

$ md5sum somefile
d68610fdffd770de94818268899d6abb  somefile
$ printf '%sxxxxxxxxxxxxxxxxxxxxxxxxxxxx\n' $(md5sum somefile | cut -c -4)
d686xxxxxxxxxxxxxxxxxxxxxxxxxxxx

You may also use %.4s instead of %s in the printf format string to get rid of the cut:

$ printf '%.4sxxxxxxxxxxxxxxxxxxxxxxxxxxxx\n' "$(md5sum somefile)"
d686xxxxxxxxxxxxxxxxxxxxxxxxxxxx

... although this would only work for formatting the hashes of a single file at a time.

Answer 7

With standard POSIX sed you can do it, although it's more complicated:

md5sum input.txt | sed -E 'h;s/^(.{4}).*$/\1/;x;s/^.{4}(.*)$/\1/;s/./x/g;H;x;s/\n//'

Here is what the sed script does:

h                   copy the pattern buffer into the hold buffer
s/^(.{4}).*$/\1/    keep just the first four characters in the pattern buffer
x                   exchange the pattern buffer with the hold buffer
s/^.{4}(.*)$/\1/    keep all but the first four characters in the pattern buffer
s/./x/g             replace each character with an x
H                   append newline and x's to hold buffer (which has the first four)
x                   exchange again; the pattern buffer has an extra newline, though
s/\n//              remove the newline from the pattern buffer

The pattern buffer now has the first four characters plus x's for all the remaining characters; the cycle ends, and it is printed out.

Note: the -E switch enables extended regex syntax. It isn't strictly necessary here; instead, a backslash could be placed before each open and close parenthesis, but I think it's hard enough to read without extra backslashes in it.