Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I had no idea that Bash supported so many advanced features! I started working on a simple network program to try and use some of these things I found. My particular program is a basic redis client written completely in Bash. Currently, I can do something like the following:

redis_command -c "ping" redis{001..100}:{6379..6400}

It will send the "ping" command to all of the instances (about 2100) and return the results. I'm currently sending the "ping" command serially, but I thought I could speed up my program by doing all of the initial socket creation in parallel. I tested my program with/without the optimization, and realized that my optimization actually slowed things down. Here's how the optimization looks:

declare -A SOCKETS
function get_socket {
  # args: <hostname> <port>
  # returns: the socket number, or failure return code

  # verify args
  if [[ -z "${1}" ]]; then
    return 1
  fi
  if [[ -z "${2}" ]]; then
    return 1
  fi

  local HOSTPORT="${1}:${2}"
  if [[ -z ${SOCKETS[${HOSTPORT}]} ]]; then
    exec {fd}<>/dev/tcp/${1}/${2}
    SOCKETS[${HOSTPORT}]=${fd}
    RES=${fd}
  else
    RES="${SOCKETS[${HOSTPORT}]}"
  fi
}

if [[ ${PRECONNECT} -eq 1 ]]; then
  echo -n "Pre-connecting to ${#HOSTS[@]} instances... " >&2
  for HOST in ${HOSTS[@]}; do
    get_socket "${HOST%%:*}" "${HOST##*:}" &
    PIDS+=($!)
  done

  # make sure all of our async connections finished before starting
  # TODO, check for errors
  for PID in ${PIDS[@]}; do
    wait ${PID}
  done
  echo "done" >&2
fi

Normally, the get_socket() function on its own works great. If I ever need to send a command to a particular server, I call get_socket() ahead of time, and pass the FD to another function that actually sends the command and parses the response. In the pre-connect scenario, I'm calling get_socket in the background via &. Since get_socket()& runs in a separate process, the FD created isn't available/accessible in the calling process, so my optimization is useless.

Is there any way to send sockets in bash, or some other way that I can parallelize my connects without using a separate process?

share|improve this question

1 Answer 1

Not with /dev/tcp/x/y directly, but you could use pipes to communicate with processes that are started in the background to connect the TCP sockets and transfer data.

Something like

coproc hostA {
  exec {fd}<> /dev/tcp/127.0.0.1/80
  cat <&$fd & cat >&$fd
}

# same for hostB...

echo -e 'HEAD / HTTP/1.0\r\n\r' >&${hostA[1]}
cat <&${hostA[0]}

As you end up running two cat commands anyway, you might as well do without /dev/tcp (which is not always enabled) and use for instance socat:

coproc hostA { socat - tcp:localhost:80; }

Or, you could remove the dependency on bash by using named pipes:

mkfifo hostA.in hostA.out
socat - tcp:localhost:80 < hostA.in > hostA.out &
share|improve this answer
    
One of my goals with this project is to not fork out any external/3rd party apps, so the socat option is not appealing to me, but I really like your use of coprocs to work around the issue I was seeing. I'll give that a shot! –  Michael Spiegle Feb 15 '13 at 23:17

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.