16

I want to write the following bash function in a way that it can accept its input from either an argument or a pipe:

b64decode() {
    echo "$1" | base64 --decode; echo
}

Desired usage:

$ b64decode "QWxhZGRpbjpvcGVuIHNlc2FtZQo="
$ b64decode < file.txt
$ b64decode <<< "QWxhZGRpbjpvcGVuIHNlc2FtZQo="
$ echo "QWxhZGRpbjpvcGVuIHNlc2FtZQo=" | b64decode
  • 3
    seems like a pointless function when base64 and bash can do all that anyway. why write a function just to avoid using the -d or --decode option? if you really must have something called b64decode then alias b64decode='base64 --decode'. b64d would be shorter, though and save even more typing. – cas Aug 5 '16 at 13:47
  • 6
    You are right, this was just used as an example, though. – tyrondis Aug 5 '16 at 22:45
22

See Stéphane Chazelas's answer for a better solution.


You can use /dev/stdin to read from standard input

b64decode()
{
    if (( $# == 0 )) ; then
        base64 --decode < /dev/stdin
        echo
    else
        base64 --decode <<< "$1"
        echo
    fi
}
  • $# == 0 checks if number of command line arguments is zero
  • base64 --decode <<< "$1" one can also use herestring instead of using echo and piping to base64
| improve this answer | |
  • 1
    just a clean syntax imo, echo and pipe might be faster.. see unix.stackexchange.com/questions/59007/… and it is herestring, I made a mistake – Sundeep Aug 5 '16 at 4:49
  • 2
    You could pipe the input through tr -d "\n" to remove lines breaks. – Julie Pelletier Aug 5 '16 at 5:51
  • 3
    does base64 support multiple input lines? of course it does, it would be pretty useless if it didn't. see for yourself: ls -l /usr/bin/ | base64 | base64 -d – cas Aug 5 '16 at 13:43
  • 1
    btw, +1. yours is a good answer to a question that only makes any sense if it's just a standin for much more complicated function. – cas Aug 5 '16 at 13:53
  • 5
    You don't actually need the < /dev/stdin; without a file, base64 will simply read from the standard input it inherits from its parent, which is /dev/stdin. – chepner Aug 6 '16 at 0:35
5

Here it should just be:

b64decode() {
  if [ "$#" -gt 0 ]; then
    # concatenated arguments fed via a pipe.
    printf %s "$@" | base64 --decode
  else
    base64 --decode  # read from stdin
  fi
  ret=$?
  echo # add one newline character
  return "$ret" # return with base64's exit status to report decoding
                # errors if any.
}

In any case, do not use base64 --decode < /dev/stdin. At best (on most systems) < /dev/stdin does nothing, it just does the equivalent of dup2(0,0) (duplicating fd 0 onto fd 0, a no-op).

But on Linux or Cygwin, < /dev/stdin does not work properly here. On those systems, opening /dev/stdin is not like duplicating stdin, it reopens from scratch and independently the same file as is currently opened on stdin.

So if previously stdin was pointing in the middle of some regular file, after < /dev/stdin, you'll end up with stdin now pointing at the start of that file. If stdin was the writing end of a pipe (which it shouldn't under normal circumstances), you'll end up with it being the reading end. If it was a socket, then it will fail as sockets cannot be opened. Same if you don't have permission to open that file for reading (for instance because the permissions changed or the file was originally opened on stdin using different credentials).

And after base64 --decode < /dev/stdin returns, the current position of stdin within the file (for seekable file input) will have been left untouched, not left at the end (or wherever base64 stopped reading) since base64's stdin was on a different open file description.

| improve this answer | |
3

Sundeep's answer works for base64 because that utility does not support multiple lines. A more general fix for the more general case

is something like

my_function() {
    if (( ${#} == 0 )) ; then
        while read -r line ; do
            target_utility "${line}"
        done
    else
        target_utility "${@}"
    fi
}
| improve this answer | |
  • (1) My version of base64 supports multi-line input (and so does cas’s).  (2) In what sense is a utility that takes command-line arguments “the more general case” of a utility that reads standard input? If you want to answer a different question, you are free to post that question and answer it.  (3) You don’t need those curly braces, but you should quote "$#". – G-Man Says 'Reinstate Monica' Sep 2 at 0:43
  • 1
    @G-Man, you should quote $# in [ "$#" -eq 0 ], you may quote it in [[ $# -eq 0 ]], but you must not quote them in if (( $# == 0 )). There's a missing IFS= before read though (and probably some --s depending on target_utility). – Stéphane Chazelas Sep 2 at 14:35
  • Also unclear why the answer is criticized for providing a "more general case" since the OP said "You are right, this [base64] was just used as an example, though. – tyrondis Aug 5 '16 at 22:45". – kbulgrien Sep 2 at 22:43
  • @StéphaneChazelas: This is so confusing!  IMHO, it was a bad design decision to make quoting optional in [[]] — it tells people that quoting isn’t so important, and so they try to memorize the rules for when quotes are required.  Forbidding quotes in $(()) is even worse.  As you said, “one side effect of omitting quotes … is that it can send a wrong message to beginners: that it may be all right not to quote variables.” … (Cont’d) – G-Man Says 'Reinstate Monica' Sep 3 at 3:34
  • 1
    @G-Man, hmmm. Quick tests here seem to concur with you (though a=$(("42")) now works in 4.4+), I'll have to double check on the machine I ran tests the other day to understand what I did and why I said that. – Stéphane Chazelas Sep 5 at 7:00
3

People are talking about how base64 --decode might really be a placeholder for a more complicated command — perhaps a pipeline or a compound command — but nobody does anything about it (*).  Well, here’s a non-recursive solution in which the main commands appear only once:

b64decode() {
        if [ "$#" -ne 0 ]
        then
                printf '%s\n' "$1"
        else
                cat
        fi | base64 --decode
        echo
} 

The logic is fairly simple:

  • If there is an argument, write it to stdout.
    • (printf is safer than echo.)
    • The question doesn’t say what to do if there are two or more arguments, so I’m ignoring that case.
  • If not, run a cat.
    • The lonesome cat isn’t useless.  UUOCs are typically characterized by having exactly one filename argument; this one has none.  It connects the input to the function (which is the input of the if statement) to the output of the if statement (which is the input to the base64 –decode statement), much like the cat in this slightly similar answer.

But some people are allergic to cats, so here’s a feline-free version:

b64decode() {
    (
        if [ "$#" -ne 0 ]
        then
                exec <<< "$1"
        fi
        base64 --decode
        echo
    )
}
  • This uses a here-string, so it works in bash, and I guess it will work in ksh (ksh93 or higher), mksh and zsh, but it won’t work in other POSIX-compliant shells.
  • The logic is similar to the first version — if there is an argument, redirect stdin to a here-string of the argument, so base64 will see it as stdin.
  • Otherwise, do nothing (leave stdin alone).
  • If the user says b64decode < file.txt, that (obviously) sets the stdin for the b64decode to be the file.txt file.  But, if we do any form of exec <, that sets the stdin for the entire shell, globally.  So, after base64 has read the here-string, the shell will get an end-of-file (EOF) and exit.  So, we run the entire function in a subshell.  Of course this means that the processing code cannot do anything that changes the shell’s environment.

_______________
(*) Except for kbulgrien, who handled it with recursive code, which strikes me as inappropriate.

| improve this answer | |
2

Both Sundeep's and TomRoche's answers were illuminating and appreciated, but in a different situation target_utility "${@}" could represent more complex code. Duplicating non-trivial code in both the if and in the else could create unnecessary issues. Though the OP's issue may not present a problem ideally resolved through use of recursion, other reader's problems may benefit from using it, or, from considering use of an wrapper function:

my_function() {
    if (( ${#} == 0 )) ; then
        while read -r __my_function ; do
            my_function "${__my_function}"
        done
    else
        target_utility "${@}"
    fi
}

This answer is one of various possible solutions for a "Bash function that accepts input from parameter or pipe" since the OP indicated [in a comment] that base64 was not the actual problem domain. As such, it makes no attempt to assure input is able to be directly read from a file.

Per various comments, there is concern this and similar other templates may not properly handle lines with leading white space; that IFS = may be required before the read. In fact, presence of any character the shell is sensitive to may require special handling before it hits this function. As-is, this code functions properly where leading white space is known to not occur, and where data sensitivities are pre-handled by the emitter.

This was not just a textbook example; it is used in an actual, production script. target_utility is a simple moniker for the actual code fragment. This answer does not presume to be a universal template without need for additional developer sagacity.

(It seems reasonable to assume users are often expected to occasionally adapt answers rather than expect to use them verbatim.)

| improve this answer | |
  • (1) It seems unfortunate to use recursive code for a function that isn’t inherently recursive. (2) It seems unfortunate to use recursive code and not state in the text portion of your answer that you are using recursive code. (3) It seems unfortunate that you have followed TomRoche’s lead and written a loop, when the question does not appear to call for one. – G-Man Says 'Reinstate Monica' Sep 2 at 0:43
  • Thanks for pointing out the recursion and correcting the oversight. The answer credits others while solving a problem that wasn't optimally solved by other (helpful) answers at the time. I shared to help others. It is up to a reader to select answers and review for appropriateness to their needs. This almost looks like an attack when all that was required was an alternative answer standing on its own merits or demerits. Was "It is unfortunate" read it as an offensive statement to re-use in triplicate? I've edited it out; it was not intended as an attack as much as an observation. – kbulgrien Sep 2 at 14:08
  • (1) Yes, my wording was snarky and belligerent.  I apologize.  (2) I still have an issue with the (IMO, unsupported) interpretation that b64decode is supposed to invoke target_utility (base64 --decode) with arguments, and that b64decode < file.txt is supposed to read file.txt one line at a time and invoke target_utility N times.  (3) As @Stéphane Chazelas pointed out in a comment on TomRoche’s answer, read should be IFS= read (to support input lines that begin with space or tab).  (4) Thank you for cleaning up Mohsen Banan’s answer. – G-Man Says 'Reinstate Monica' Sep 3 at 3:31
  • The title of the question is what triggered the process of finding this Q/A for material that aided development of the above to solve a real life problem described by the title. The OP declared that base64 decode was not the "real" problem; pedantic constraint of answers to a particular "example" seems less helpful. When this question and its answers were key to helping solve real problems, alternate answers can be gifts to the community in recognition of the fact that many more people will use this Q/A to solve problems. Since the answer is on-topic per the title, I feel it is "game on". – kbulgrien Sep 3 at 17:30
  • More boiler-plate added to the answer, but the value of what really seems to be nitpicking seems questionable even if there is also value in a judicious hunt for theoretical ideals. In the end, this site is about helping community. All the associated answers and comments are available for a visitor to peruse and consider. It is decidedly less valuable to the community when all answers are identical. If one had to deal with the drain of such critique for every answer, perhaps site usage would go down. People have lives to lead and the OP is apparently already happy. IMO, it is time to move on. – kbulgrien Sep 3 at 17:48
-6

Here is an example in the BSIP dialect of bash.

function vis_userAcctsReport {
G_funcEntry
function describeF {  G_funcEntryShow; cat  << _EOF_
Report on a user account, inputs can come from args or from stdin
Examples:
      ${G_myName} -i userAcctsReport bisos
      echo bisos bystar | ${G_myName} -i userAcctsReport
_EOF_
}
local inputsList="$@"
local thisFunc=${G_thisFunc}

function processEach {
EH_assert [[ $# -eq 1 ]]
local userAcctName=$1
if ! vis_userAcctsExist ${userAcctName} ; then
    EH_problem "${userAcctName} Does Not Exist -- ${thisFunc} Processing Skipped"
    lpReturn 101
fi

ANT_raw "--- ${userAcctName}: passwd, group, id, sudoers ---"

lpDo getent passwd ${userAcctName}
lpDo getent group ${userAcctName}

lpDo sudo -u ${userAcctName} id
lpDo sudo grep ${userAcctName} /etc/sudoers
}

if [ $# -gt 0 ] ; then
local each=""
for each in ${inputsList} ; do
    lpDo processEach ${each}
done
else
local eachLine=""
while read -r -t 1 eachLine ; do
    if [ ! -z "${eachLine}" ] ; then
    local each=""
    for each in ${eachLine} ; do
        lpDo processEach ${each}
    done
    fi
done
fi

lpReturn
}

Disclosure: This answer relates to "BSIF (ByStar Shell-based/bash-based Integration Facilities) Namespace: A name base for a collection of bash libraries and bash commands for autonomous services intgration [sic]." The answering party is the author/maintainer of the bsip package as represented at https://pypi.org/project/bsif/

| improve this answer | |
  • What does "BSIP" mean? – Jeff Schaller Aug 25 at 0:55
  • There are several un-defined commands in your function; hopefully there's a "BSIP" library of some kind that defines them? – Jeff Schaller Aug 25 at 0:57
  • (1) No indication of what BSIP is. (Google doesn’t know what it is, and neither does Jeff.) (2) A lot of code to handle a fairly simple problem, with no explanation. (3) It’s not clear that this even answers the question. At best, it might be a wrong answer. – Scott Aug 25 at 1:31
  • It is a long-standing grammar rule that suggests acronyms not be used without first elaborating what they stand for. Refusal to expand one on a well-meaning query seems... well... difficult. – kbulgrien Sep 2 at 14:35
  • pypi.org/project/bsif "BSIF (ByStar Shell-based/bash-based Integration Facilities) Namespace: A name base for a collection of bash libraries and bash commands for autonomous services intgration [sic]. Support For support, ideas, suggestions, criticism, comments and questions; please contact the author/maintainer Mohsen Banan at: " <redacted /> ... "Installation pip install bsip" – kbulgrien Sep 2 at 14:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.