3

I have a problem, I have a file test.txt that has a content like this:

objectClass: CPM1
objectClass: CPM2
objectClass: CPM3
objectClass: CPM4
objectClass: CPZ
objectClass: CP04s
objectClass: CP0A
objectClass: CP11
entryDS: 1
nodeId: 21
createTimestamp: 20211014155438Z
modifyTimestamp: 20220313041228Z
MSISDN: 353834079137
IMSI: 272050103105233
NAM: 0
CDC: 1423
IMEISV:: U1RQJgAkQiE=
AUTHINFO: 0
CSP: 11
SUBSCSPVERS: 20
RSA: 20
SUBSRSAVERS: 19
APNID1: 4

Every time the text IMEISV:: is found I want to replace in that file the value after IMEISV:: (in this case U1RQJgAkQiE=) with the result of the following function:

base64 -d | od -t x8 -An | rev

If I do in unix an echo would be like this:

echo U1RQJgAkQiE= | base64 -d | od -t x8 -An | rev

The result would be 3545056200422412

So at the end in the file, the value U1RQJgAkQiE= should be 3545056200422412. Like this:

objectClass: CPM1
objectClass: CPM2
objectClass: CPM3
objectClass: CPM4
objectClass: CPZ
objectClass: CP04s
objectClass: CP0A
objectClass: CP11
entryDS: 1
nodeId: 21
createTimestamp: 20211014155438Z
modifyTimestamp: 20220313041228Z
MSISDN: 353834079137
IMSI: 272050103105233
NAM: 0
CDC: 1423
IMEISV:: 3545056200422412
AUTHINFO: 0
CSP: 11
SUBSCSPVERS: 20
RSA: 20
SUBSRSAVERS: 19
APNID1: 4

Can someone please let me know how to do it with sed/perl...? I'm really a newbie with unix

Improve this question

4 Answers 4

Reset to default
3

How about

unpack("h*", decode_base64($_))

where decode_base64 is provided by the perl MIME module, ex.:

$ perl -MMIME::Base64 -pe 's/IMEISV:: \K(.*)$/unpack("h*", decode_base64($1))/e' test.txt
objectClass: CPM1
objectClass: CPM2
objectClass: CPM3
objectClass: CPM4
objectClass: CPZ
objectClass: CP04s
objectClass: CP0A
objectClass: CP11
entryDS: 1
nodeId: 21
createTimestamp: 20211014155438Z
modifyTimestamp: 20220313041228Z
MSISDN: 353834079137
IMSI: 272050103105233
NAM: 0
CDC: 1423
IMEISV:: 3545056200422412
AUTHINFO: 0
CSP: 11
SUBSCSPVERS: 20
RSA: 20
SUBSRSAVERS: 19
APNID1: 4
Improve this answer
1
  • Thanks a lot steeldriver, this option is a lot faster than using a sed when working with a huge file. And works like a charm too
    – Sergio Jimenez
    Mar 24 at 13:59
3

sed 's/\(IMEISV::\)\(.*\)/echo -n \1" ";echo \2\| base64 -d \| od -t x8 -An\|rev /ge ' input.txt

The trick is the e switch. Output:

objectClass: CPM1
objectClass: CPM2
objectClass: CPM3
objectClass: CPM4
objectClass: CPZ
objectClass: CP04s
objectClass: CP0A
objectClass: CP11
entryDS: 1
nodeId: 21
createTimestamp: 20211014155438Z
modifyTimestamp: 20220313041228Z
MSISDN: 353834079137
IMSI: 272050103105233
NAM: 0
CDC: 1423
IMEISV:: 3545056200422412 
AUTHINFO: 0
CSP: 11
SUBSCSPVERS: 20
RSA: 20
SUBSRSAVERS: 19
APNID1: 4
Improve this answer
2
  • 1
    Thanks a lot. I was lost trying to add the echo to the sed command and of course I didn't even knew about the e option. I suppose that's to use the output of echo as input of the function.
    – Sergio Jimenez
    Mar 23 at 16:46
  • I tried solution from steeldriver because although using this sed works, it takes a really long time when we talk about a 7GB ldif file. But thanks!!!!!
    – Sergio Jimenez
    Mar 24 at 13:58
2

One idea using sed would be to execute your command above within the sed replacement using printf to print to first matched group without a new line.

$ sed "s/\(IMEISV::\)\(.*\)/printf '\1 ' ; echo \2 | base64 -d | od -t x8 -An | rev/e" input_file
objectClass: CPM1
objectClass: CPM2
objectClass: CPM3
objectClass: CPM4
objectClass: CPZ
objectClass: CP04s
objectClass: CP0A
objectClass: CP11
entryDS: 1
nodeId: 21
createTimestamp: 20211014155438Z
modifyTimestamp: 20220313041228Z
MSISDN: 353834079137
IMSI: 272050103105233
NAM: 0
CDC: 1423
IMEISV:: 3545056200422412
AUTHINFO: 0
CSP: 11
SUBSCSPVERS: 20
RSA: 20
SUBSRSAVERS: 19
APNID1: 4
Improve this answer
1 
#------------------
#  base64 in sed
#------------------
set -u

#> format string printf
fmt='%s%s\n'

#> bit, sextet, & octet regex
bit='[01]'
sextet="${bit}{6}"
 octet="${bit}{8}"; byte=$octet

#> base64 charset
b64='/[:alnum:]+'
declare -a b64_chars=({A..Z} {a..z} {0..9} + /)

#> user-defined helper function(s)
oneLine() {
  # collapse stdin into one line
  paste -sd'\0' -
}

esc_rhs() {
  # make stdin pkuggable
  # on the rhs of a s///
  sed -e '
    s:[\/&]:\\&:g
    $!s:$:\\:
  ' -
}

dec2bin() {
  # create an array of size 2^$1
  # Usage: dec2bin 6
  # creates array d2b as:
  # underscore for clarity only
  # $d2b[0] => "000_000"
  # $d2b[1] => "000_001"
  #        ...
  # $d2b[63] => "111_111"

  eval "d2b=($(yes '{0,1}' | sed "$1q" | oneLine))"
}

#> build the encoding lookup table
encode_tbl=$(printf '%s\n' "$(
dec2bin 6
i=0
for c in "${b64_chars[@]}"
do
  printf "$fmt" "$c" "${d2b[$i]}"
  (( i++ ))
done
)" | esc_rhs)

#> build the decoding lookup table
decode_tbl=$(printf '%s\n' "$(
dec2bin 8
for dec in {0..255}
do
  hex=$(printf '%02x' "$dec")
  bin=${d2b[$dec]}
  printf "$fmt" "$bin" "$hex"
done
)" | esc_rhs)

#######> main()
sed -E "
  s/^IMEISV::\s*//;t base64
  b
  :base64
  s/[^${b64}]//g
  /^\$/d
  H;z;x;G

  s/\$/$encode_tbl/
  :encode
  /\n\n/!{
    s/((\n)[${b64}])(.*\1(${sextet}))/\4\2\3/
    b encode
  }

  s/\n.*//;:pad
  /^(${octet})+\$/!{
    s/\$/0/;b pad
  }
  # chunk it in 8-bit portions
  s/(${octet})/& /g
  G;s/\$/${decode_tbl}/;t decode
  :decode
    s/(${octet}) (.*\n\1([[:xdigit:]]{2}))/\3\2/
  t decode

  s/\n.*//
  s/([[:xdigit:]])([[:xdigit:]])/\2\1/g
  s/00?\$//
  s/^/IMEISV:: /
" file
Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.