37

Given a command that changes its behaviour when its output is going to a terminal (e.g. produce coloured output), how can that output be redirected in a pipeline while preserving the changed behaviour? There must be a utility for that, which I am not aware of.

Some commands, like grep --color=always, have option flags to force the behaviour, but the question is how to work around programs that rely solely on testing their output file descriptor.

If it matters, my shell is bash on Linux.

  • You have to know how the command tests its output file descriptor and somehow return results consistent with the results that would be seen from an output terminal. – Andrew Henle Dec 16 '15 at 15:11
20

You might get what you need by using unbuffer.

unbufferis a tcl / expect script. Look at the source if you want. Also note the CAVEATS section in man.

Also note that it does not execute aliases such as:

alias ls='ls --color=auto'

unless one add a trick as noted by Stéphane Chazelas:

If you do a alias unbuffer='unbuffer ' (note the trailing space), then aliases will be expanded after unbuffer.

  • Note on aliases acknowledged – it will also not work with any other shell constructs like builtins and functions. – Amir Dec 16 '15 at 15:37
  • 3
    If you do a alias unbuffer='unbuffer ' (note the trailing space), then aliases will be expanded after unbuffer. – Stéphane Chazelas May 18 '16 at 16:52
  • unbuffer it is! sudo apt install expect -- That was unclear. – loxaxs Mar 17 '18 at 11:35
22

A history of toolsets

You are not the first person to want such a tool. People have been wanting such tools for 30 years. And they've existed for almost that long, too.

The earliest tool for this sort of thing was Daniel J. Bernstein's "pty" package, described by Rich Salz as a "Ginsu knife", which Bernstein wrote back at the turn of the 1990s in order to cheat at nethack (sic!). Version 4 of the "pty" package was published in 1992 to comp.sources.unix (volume 25 issues 127 to 135). It's still locatable on the World Wide Web. Paul Vixie described it at the time:

What can I say? It slices, it dices, it washes dishes, it walks the dog. It "just works", meaning that if you follow the directions you'll get a working package without any pulling of hair or gnashing of teeth or other standard porting activities.

Bernstein later updated this, somewhen on or before 1999-04-07, with a "ptyget" package, which he announced:

I've put together a new pseudo-tty allocator, ptyget. An alpha version is at ftp://koobera.math.uic.edu/pub/software/ptyget-0.50.tar.gz. There's a ptyget mailing list; to join, send an empty message to djb-ptyget-requ...@koobera.math.uic.edu. I designed ptyget's interface from scratch. It's much more modular than pty; the basic pty interface has now been split into three pieces:

  • ptyget: a tiny, low-level program — the only setuid program in the package — that allocates a new pseudo-tty and passes it to the program of your choice
  • ptyspawn: another small program that runs a child process under a pseudo-tty, waiting for it to exit and watching for stops
  • ptyio: another, only slightly larger, program that moves data back and forth

The old Ginsu knife pty is now spelled ptybandage, which is a synonym for ptyget ptyio -t ptyspawn; pty -d, for attaching network programs to pseudo-ttys, is now spelled ptyrun, which is a synonym for ptyget ptyio ptyspawn; and nobuf is a synonym for ptyget ptyio -r ptyspawn -23x. I've split off the session management features into a separate package.

That separate package was the "sess" package.

"ptyget" is, incidentally, notable for exemplifying a very early version of, and one of the few published instances of, Berstein's own never-published "redo" build system. dependon is a clear precursor to redo-ifchange.

Usage

ptybandage

ptybandage is what people usually want in a login session. Its primary use case is making programs that are sensitive to whether their standard inputs, outputs, or errors are connected to terminals operate that way even though they are in fact in shell pipelines, or have their standard file descriptors redirected to file.

It takes a command to run (which has to be a proper external command, of course) and runs it in such a way that it thinks that its standard input, output, and error are attached to a terminal, connecting those through to ptybandage's original standard input, output, and error.

It deals with the nuances of running under job control shells, ensuring that a terminal STOP character not only stops ptybandage but also stops the program running attached to the inner terminal.

ptyrun

ptyrun is what people usually want in TCP network servers. Its primary use case is remote execution environments that have not themselves set up terminals, running programs that don't operate as desired when there's no terminal.

It doesn't expect to be running under a job control shell, and if the command being run receives a stop signal it is simply restarted.

Available toolsets

Dru Nelson publishes both "pty" version 4 and "ptyget".

Paul Jarc publishes a fixed version of ptyget, that attempts to deal with the operating-system-specific pseudo-terminal device ioctls in the original that operating systems actually no longer provide.

The nosh source package comes with workalike ptybandange and ptyrun scripts, which use Laurent Bercot's execline tool and the nosh package's own pseudo-terminal management commands. As of nosh version 1.23 these are available pre-packaged in the nosh-terminal-extras package. (Earlier versions only supplied them to people who built from source.)

A few example uses

Jurjgen Oskam using ptybandage on AIX to feed input from a here document to a program that explicity opens and read its controlling terminal for a password prompt:

$ ptybandage dsmadmc <<EOF >uit.txt
joskam
password
query session
query process
quit
EOF

Andy Bradford using ptyrun on OpenBSD under daemontools and ucspi-tcp to make the bgplgsh interactive router control program accessible via the network whilst making it think that it is talking to a terminal:

#!/bin/sh
exec 2>&1
exec envuidgid rviews tcpserver -vDRHl0 0 23 ptyrun /usr/bin/bgplgsh

Further reading

  • Thanks for the lesson in history, but the tools you suggest are not available in a modern Linux distribution, hence not useful. – Amir Dec 17 '15 at 8:53
  • 4
    There's a bunch of hyperlinks and an entire section of the answer devoted to where the tools most definitely are available. – JdeBP Dec 17 '15 at 13:15
  • 1
    What's your opinion on expect? – CMCDragonkai Jan 23 at 3:48
19

You can use socat to start your process with a pty connected, and get socat to connect the other end of the pty to a file. Which AFAIU is exactly what you asked:

socat EXEC:"my-command",pty GOPEN:mylog.log

This method will cause isatty called by my-command to return true and a process that relies only on that will be fooled to output control codes. Note that some processes (notably grep) also check the value of the TERM environment variable, so you might need to set it to something reasonable, like "xterm"

12

How about using script(1)?

For example:

script -q -c 'ls -G' out_file

Will save the ls output to out_file with the color codes preserved.

  • This does not work here. How colours are preserved? Is there a tool that I should use to output out_file with its colours? – Kira Dec 16 '15 at 14:40
  • 1
    @Kira for viewing files with ANSI colour escape sequences, I use less -R. In this case, though, I wanted the output to continue in the pipeline, which did end up in my terminal eventually. Using cat for illustration, it was something like script -q -c 'ls -G' /dev/null | cat, which suppresses the typescript file entirely, leaving only the program’s output. – Amir Dec 16 '15 at 15:05
  • To avoid creating a file, simply use a dash (-) as script output file, for exemple: script -q -c 'ls -G' - – Franklin Piat Nov 25 '16 at 20:38
12

There is also a nice solution posted here on Super User by KarlC:

Compile a small shared library:

echo "int isatty(int fd) { return 1; }" | gcc -O2 -fpic -shared -ldl -o isatty.so -xc -

Then tell your command to load this isatty(3) override dynamically:

LD_PRELOAD=./isatty.so mycommand

This might not work for every command out there, may even break some in unexpected ways, but would probably work in most cases.

  • 2
    For MacOS users, you can get the same behavior by using DYLD_INSERT_LIBRARIES=./isatty.so DYLD_FORCE_FLAT_NAMESPACE=y mycommand – Christopher Shroba Mar 13 '17 at 14:42
0

Based on @Amir's answer, here is a script which generates and then includes the library at runtime:

#!/bin/bash
set -euo pipefail

function clean_up {
  trap - EXIT # Restore default handler to avoid recursion
  [[ -e "${isatty_so:-}" ]] && rm "$isatty_so"
}
# shellcheck disable=2154 ## err is referenced but not assigned
trap 'err=$?; clean_up; exit $err' EXIT HUP INT TERM

isatty_so=$(mktemp --tmpdir "$(basename "$0")".XXXXX.isatty.so)
echo "int isatty(int fd) { return 1; }" \
  | gcc -O2 -fpic -shared -ldl -o "$isatty_so" -xc -
# Allow user to SH=/bin/zsh faketty mycommand
"${SH:-$SHELL}" -c 'eval $@' - LD_PRELOAD="$isatty_so" "$@"

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.