Merge two audio sources with x11grab, possible?

Question

Currently I'm running with the following setup.

ffmpeg \
 -i http://pub1.di.fm:80/di_vocaltrance \
 -f x11grab -s 1920x1080 -r 25 -i :0.0 \
 -c:v libx264 -preset fast -pix_fmt yuv420p -s 1280x800 \
 -c:a libmp3lame -ab 96k -ar 22050 \
 -threads 0 \
 -f flv "rtmp://...."

What I want to bake in:

 -f alsa -ac 2 -i hw:0 -acodec pcm_s32le -aq 1 -ab 128k\

Is this possible? And perhaps even mix in "what I hear" (aka playback) so to speak?

I've tried something along the lines of (but not fully understanding -map)

 -i http://pub1.di.fm:80/di_vocaltrance -map 0:1 \
 -f x11grab -s 1920x1080 -r 25 -i :0.0 -map 0:0 \

But that didn't really go so well.

I'm so close now, this gives me sound channels, but everything sounds like I'm inside a small tin-can of sorts.

#!/bin/bash
KEY=`cat key.db`

ffmpeg \
 -i http://pub1.di.fm:80/di_vocaltrance \
 -f alsa -ac 2 -i hw:0 -acodec mp3 -aq 1 -ab 128k \
 -f x11grab -s 1920x1080 -r 25 -i :0.0 \
 -c:v libx264 -preset fast -pix_fmt yuv420p -s 1280x800 \
 -c:a mp3 -ab 96k -ar 22050 \
 -threads 0 \
 -map 0:0 -map 1:0 -map 2:0 \
 -f flv "rtmp://live.twitch.tv/app/$KEY"

It only appears when I mix the two sounds together. Trying to get JACK in here as well, but without any actual results (I thought I'd pass everything via a mixer board to unload ffmpeg from the mixing).

Running with:

 -f alsa -ac 2 -i jack \
 ....

Gives this however:

[alsa @ 0x1399280] cannot set sample format 0x10000 2 (Invalid argument)

jack: Input/output error

Answer 1

You should be able to use the amix filter to mix together your audio.

Example:

ffmpeg \
  -i http://pub1.di.fm:80/di_vocaltrance \ # internet radio
  -f alsa    -i hw:0 \                     # microphone
  -f x11grab -i :0.0 \                     # computer desktop
  -f v4l2    -i /dev/video0 \              # webcam or
  # -f iec61883 -i auto \                  # firewire camcorder
  -filter_complex '[0:a:0][1:a:0]amix=inputs=2;[3:v:0]scale=200:-1[webcam],[2:v:0][webcam]overlay=main_w-overlay_w-10:main_h-overlay_h-10' \
  <output>

I added a webcam overlay also, to remove it, take off the webcam line, the semi-colon and the definitions after the semi-colon.

Be ware that you might have to define -maps if you add any input that has multiple streams to remove the extraneous ones.

Explanation:

-filter_complex

defines filters with multiple inputs or outputs

[0:a:0]

Input 0's audio stream number 0. (internet radio)

[1:a:0]

Input 1's audio stream number 0. (microphone)

amix=inputs=2

Mix together two audio streams, if no input streams are defined, it autoselects two, so in this instance, you should be able to leave out the above two definitions.

;

Start a new filter chain

[3:v:0]

Input 3's video stream number 0. (webcam)

scale=200:-1

Scale the video to 200x???, -1 keeps aspect ratio, so a 600x480 video becomes 200x160.

[webcam]

Call the scaled output "webcam".

,

Continue to the next filter in this chain.

[2:v:0]

Input 2's video stream number 0. (desktop)

[webcam]

The scaled webcam stream.

overlay=main_w-overlay_w-10:main_h-overlay_h-10

Overlay the second input over the first input, placing the second input 10 pixels from right, and 10 pixels from the bottom.

---

Also, you can set the sound card capture source to mix to get what the computer outputs, and you can often unmute the microphone to get a microphone monitor loopback, but to get it to work properly you have to use jack or pulse. I do not know why jack did not work for you, so I can not help you with that, sorry.