Forward pg_dump to 2 different psql, one log file but not the console

Question

I am trying to push the pg_dump to 2 psql and to a log file but I don't want to add noise to my terminal.

Following the numerous post of the tee tool, I reached this command which is indeed working except that the console is still flooded.

pg_dump -a -t table-country -p 5432 -U usr db-maint2  2>&1 | tee >(psql -p 5432 -U postgres db-project) >(psql -p 5432 -U postgres db-gp-projet) >"initDB_$(date +%Y%m%d_%H%M).log"

I have read around but I don't find a better solution.

Moreover, the stdout from pg_dump is displayed in the console after my command but the logs from both psql are sent to the terminal only and not the log file.

EDIT : ok I found how to output the psql to the logfile :

pg_dump -a -t table-country -p 5432 -U usr db-maint2  2>&1 | tee >(psql -p 5432 -U postgres db-project >"initDB_$(date +%Y%m%d_%H%M).log") >(psql -p 5432 -U postgres db-gp-projet >"initDB_$(date +%Y%m%d_%H%M).log") >"initDB_$(date +%Y%m%d_%H%M).log"

Actually, that logs only one of the psql command, even adding -a handle with tee create only one psql output in the file.log.

Edit nth : No more ideas.

Edit n^th + 1 : ofc, my brain finally woke up.

Answer 1

pg_dump -a -t table-country -p 5432 -U usr db-maint2  2>&1 | tee -a >"${LOGFILE}" >(psql -p 5432 -U postgres db-project >>"${LOGFILE}") >(psql -p 5432 -U postgres db-gp-projet >>"${LOGFILE}") 

Why ?

Because, from left to right :

1. pg_dump goes to stdout
2. 2>&1 dumps error stream to stdout
3. | pipes the stdout & errout
4. to tee -a, which will keep the pipe content and forward it (with append option at least for first forward) to several commands
5. >"${LOGFILE}" FIRST I want to log the pg_dump stdout or errout. Note the syntax for this : no () around the file name. I use a variable in my script.
6. >(psql -p 5432 -U postgres db-project >>"${LOGFILE}") THEN my first COPY as a psql command (with wrapping parenthesis for a command)
7. >>"${LOGFILE}" inside previous line, I use this to append (double >) to my file the stdout of psql (I don't get the errout here, I could have.
8. >(psql -p 5432 -U postgres db-gp-projet >>"${LOGFILE}") THEN the second psql command with a second appending of stdout to the logfile.
9. No output to console, everything is kept by tee, it seems (don't know why to be honest).

Hope it helps some people to understand tee and pipe use cases.

Edit : my final command is

pg_dump -a -t table-country -p 5432 -U usr db-maint2  2>&1 | tee -a > /dev/null >(psql -p 5432 -U postgres db-project >>"${LOGFILE}") >(psql -p 5432 -U postgres db-gp-projet >>"${LOGFILE}")

It drops the standard output rather than polluting the logfile. It would be best to keep the error stream into the logfile though with

pg_dump -a -t table-country -p 5432 -U usr db-maint2  2>"${LOGFILE}" | tee -a > /dev/null >(psql -p 5432 -U postgres db-project >>"${LOGFILE}") >(psql -p 5432 -U postgres db-gp-projet >>"${LOGFILE}")

but it's not tested yet.

Answer 2

Changing the redirection in two ways should fix this problem. 1) Redirect both output & error streams for each command. 2) Change the redirection operator to append instead of overwrite the log file.

The final command should then look like this:

pg_dump -a -t table-country -p 5432 -U usr db-maint2  2>&1 | tee >(psql -p 5432 -U postgres db-project &>> "initDB_$(date +%Y%m%d_%H%M).log") >(psql -p 5432 -U postgres db-gp-projet &>> "initDB_$(date +%Y%m%d_%H%M).log") &>> "initDB_$(date +%Y%m%d_%H%M).log"

The output of the pg_dump command is delivered to the tee command, which is specified in this form:

tee file1 file2 &>> logfile

The file1 and file2 arguments just happen to be processes in this case. tee will write the output from the previous command to file1, file2 and then to standard output. The standard output is caught and redirected to the log file (in append mode).

The output written to file1 is delivered to the psql command, which sets up its own redirection for standard output and error. The same thing happens to the second psql command.

Because the name of the log file is derived using the date command, and has only minute-level accuracy, appending is required to prevent possible overwriting of the log file. Alternatively, the output could be delivered to different log files as well.