18

Is it possible to use bash function inside AWK somehow?

Example file (string, int, int, int)

Mike 247808 247809 247810

Trying to convert values from decimal to hexadecimal.

Function defined either in .bashrc or in shell script.

awk '{print $1 ; d2h($2)}' file

awk: calling undefined function d2h input record number 1, file file source line number 1

19

Try to use system() function:

awk '{printf("%s ",$1); system("d2h " $2)}' file

In your case system will call d2h 247808 and then append output of this command to printf output:

Mike 3C800

EDIT:

As system uses sh instead of bash I can't find a way to access .bashrc. But you can still use functions from your current bash script:

#!/bin/bash
d2h() {
    # do some cool conversion here
    echo "$1" # or just output the first parameter
}
export -f d2h
awk '{printf("%s ",$1); system("bash -c '\''d2h "$2"'\''")}' file

EDIT 2:

I don't know why, but this is not working on my Ubuntu 16.04. This is strange, because it used to work on Ubuntu 14.04.

  • This could work if d2h were an executable, but not if it's a "function defined either in .bashrc or in shell script". – Michael Homer Sep 22 '14 at 8:33
  • @MichaelHomer yes, you are right. Thanks to your comment I realised I have answered the question nobody asked. But I have found a way to use functions from the current script (and possibly from other scripts via source) but I can'f figure out why it doesn't work with .bashrc. – akarilimano Sep 22 '14 at 10:10
  • 1
    That's also a command injection vulnerability as the content of $2 ends up being interpreted as shell code. – Stéphane Chazelas Jan 15 '18 at 12:24
4

You can call bash from awk and use its output. That is obviously dangerous from a performance perspective if it happens too often. Quoting the man page:

command | getline [var]

Run command piping the output either into $0 or var,

command would be a bash script that contains the function definition and executes the function.

  • Hauke Laging, very cool! – JJoao Jan 15 '18 at 14:28
  • I've used this many times, and it works well. – Dave Rix Jan 29 at 15:39
3

Try doing this :

awk '{print $1 ; printf "%x\n", $2}' file

AFAIK, you can't use a bash function in awk but only a script. You can use an awk function if needed.

3

Using a user defined bash function inside awk

Disclaimer: I realize this is not what the OP is trying to do, but Google will lead others like me to this answer.

Situation

You have a bash script that is organized with functions (because you do not hate yourself or [most] coworkers) and at least 1 of those functions needs to call another from within awk.

Solution

Script

#!/bin/env bash

# The main function - it's a sound pattern even in BASH
main(){
    # In the awk command I do some tricky things with single quotes. Count carefully...
    # The first $0 is outside the single quotes so it is the name of the current bash script.
    # The second $0 is inside the single quotes so it is awk's current line of input.
    awk '{printf("%s. ", ++c); system("'$0' --do"); print $0}'<<-PRETEND_THIS_IS_AN_INPUT_STREAM
        and
        and
        well
    PRETEND_THIS_IS_AN_INPUT_STREAM
}

# functionized to keep things DRY
doit(){
    echo -n "doin' it "
}


# check for a command switch and call different functionality if it is found
if [[ $# -eq 1 && $1 == "--do" ]];
then
        doit
else
        main
fi

Output

$ ./example.sh
1. doin' it and
2. doin' it and
3. doin' it well
  • I represent Queens, she was raised out in Brooklyn – Bruno Bronosky May 26 '17 at 20:00
2

awk can run awk functions. For it to run bash functions, you'd need awk to execute a bash shell, that bash to interpret the definition of that function, and call that function, with the value extracted by awk passed as arguments.

Not trivial.

bash supports exporting functions so it's available in subsequent invocations of bash, so that's one way to pass the definition of the function to the bash invoked by awk:

export -f d2h

The only ways for awk to execute a command (bash here) are with its system("cmd"), or print... | "cmd" or "cmd" | getline. In all cases, awk runs a shell to interpret that cmd, but it will be sh, not bash. So you need to construct a command line for sh that is a bash invocation that interprets a bash command line to invoke the function, so you need to be careful with quoting:

export -f d2h
<file awk -v q="'" '
  function shquote(s) {
    gsub(q, q "\\" q q, s)
    return q s q
  }
  {print $1; system("bash -c '\''d2h \"$1\"'\'' bash " shquote($2))}'

That does mean running one sh and one bash for each line, so is going to be quite inefficient. That would end up being even significantly more inefficient than having bash do the reading and splitting with a while read loop:

(unset IFS; while read -r a b rest; do
  printf '%s\n' "$a"
  d2h "$b"
 done < file)
0

This will give you good chances.

cat ../logs/em2.log.1 |grep -i 192.168.21.15 |awk '{system("date"); print $1}'

system function enables you to parse bash command within awk stream.

0

Just a quick hack do demonstrate @HaukeLaging command|getline :

1) let input be:

Dear friends
my name is `id -nu` and
today is `date "+%Y-%m-%d"`.

Following shell syntax, in the input,

`command`

is used to denote inline commands to be replaced by the result of his execution.

2) We can expand inline shell command by:

#!/usr/bin/gawk -f

BEGIN  { FS ="`";        }
NF==3  { $2 | getline $2 }
       { print           }

3) Usage (after the usual chmod ):

$ expand-inline input
Dear friends
my name is jjoao and
today is 2018-01-15.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.