This is a series of articles. Follow the link here to get an overview over all articles.
Introduction
A lot of people were asking me how to add more than one audio stream. This article covers the answer for this question.
What’s stored in a file of a stream
Until now we have produced two output stream variants (stream_0 and stream_1). Each file within a variant contains a video and an audio stream. The master.m3u8 file tells the player when to play which stream variant (because it has different video resolutions).
Now imagine that we have 10 audio languages, then we would have still 2 stream variants but each file contains one video stream and 10 audio streams. This is not great for the users and our servers, because they download a file with 10 audio streams and only one of them is played. This uses unnecessarily much bandwidth.
How to make it better
The solution is very simple: Audio and video is separated into different stream variants. So for our example above we would have one stream variant for audio and two stream variants for the different video resolutions.
This makes it much easier to add later more audio streams. The master.m3u8 file contains still the metadata for the player.
./ffmpeg -listen 1 -i rtmp://martin-riedl.de/stream01 \ -filter_complex "[v:0]split=2[vtemp001][vout002];[vtemp001]scale=w=960:h=540[vout001]" \ -preset veryfast -g 25 -sc_threshold 0 \ -map "[vout001]" -c:v:0 libx264 -b:v:0 2000k -maxrate:v:0 2200k -bufsize:v:0 3000k \ -map "[vout002]" -c:v:1 libx264 -b:v:1 6000k -maxrate:v:1 6600k -bufsize:v:1 8000k \ -map a:0 -c:a aac -b:a 128k -ac 2 \ -f hls -hls_time 4 -hls_playlist_type event -hls_flags independent_segments \ -master_pl_name master.m3u8 \ -hls_segment_filename stream_%v/data%06d.ts \ -use_localtime_mkdir 1 \ -var_stream_map "a:0,agroup:audio128 v:0,agroup:audio128 v:1,agroup:audio128" stream_%v.m3u8
Our command needs now only one audio stream so we remove the second “-map a:0”. The other change we make is how the stream variants are created. We have now 3 blocks (separated by a space) within the stream-map. First the audio, the second one the lower video resolution and the third with the high video resolution. The “agroup” groups audio and video streams together. You can use any name you want (I prefer here audio128; 128kbit audio).
Adding a second audio language
Now it’s easy to add a second or even more audio languages.
./ffmpeg -listen 1 -i rtmp://martin-riedl.de/stream01 \ -filter_complex "[v:0]split=2[vtemp001][vout002];[vtemp001]scale=w=960:h=540[vout001]" \ -preset veryfast -g 25 -sc_threshold 0 \ -map "[vout001]" -c:v:0 libx264 -b:v:0 2000k -maxrate:v:0 2200k -bufsize:v:0 3000k \ -map "[vout002]" -c:v:1 libx264 -b:v:1 6000k -maxrate:v:1 6600k -bufsize:v:1 8000k \ -map a:0 -map a:1 -c:a aac -b:a 128k -ac 2 \ -f hls -hls_time 4 -hls_playlist_type event -hls_flags independent_segments \ -master_pl_name master.m3u8 \ -hls_segment_filename stream_%v/data%06d.ts \ -use_localtime_mkdir 1 \ -var_stream_map "a:0,agroup:audio128,language:GER a:1,agroup:audio128,language:ENG v:0,agroup:audio128 v:1,agroup:audio128" stream_%v.m3u8
We add a new “-map a:1” to pick also the second audio input. Adding the second audio to the stream-map is important too. In my example above the “language” metadata is added so that the player can choose between both audio variants.
Can you make tutorials on FFmpeg HLS steamer via CDN ?
Can you explain me more what you exactly want to do?
First off, thanks for this great series! It has greatly accelerated my project in getting HLS working. Everything you’ve shown so far has worked for me but for some reason I’m getting errors when trying to use the language:ENG argument. I get the following error message:
[hls @ 0x55a5d275c200] Invalid keyval language:ENG
My full var_stream_map looks like this:
-var_stream_map “a:0,agroup:test,language:ENG a:1,agroup:test,language:GER v:0,agroup:test v:1,agroup:test v:2,agroup:test v:3,agroup:test v:4,agroup:test v:5,agroup:test”
Do you have any ideas why this isn’t working?
You might have an old version of FFmpeg. The language attribute has been added 2 years ago in the following commit: https://github.com/FFmpeg/FFmpeg/commit/d02289c386ecf1c07f2441674c550008cb869d50
On a first look it seems that you need FFmpeg 4.0 or newer. Try to run the command on the latest version.
You are correct, sir. I was on 4.1.4 which is greater than 4.0 but wasn’t working for some reason. Updated to 4.3.1 and it works perfectly. Thank you!
Hi! To begin with, I want to thank you for this amazing tutorial. It is very useful and clear.
I have a question that I haven’t been able to sort it out, and maybe you have an idea or solution about it.
Is there any way to name the hls_segment_name custom according to the maps settings, and include them in the master file? I mean, by resolution for example.
1080p_%03d.ts
720p_%03d.ts
360p_%03d.ts
I am able to name hls_segment_name by resolution, but they doesn’t appear in the master file..
Don’t know if I explained properly. Sorry about that.
And many thanks in advance!!!
You can define a custom name in the “-var_stream_map” argument. By default, the %v variable is a counter. Simply add a name for each entry in the var_stream_map and this name is used instead of the number:
-var_stream_map “a:0,agroup:audio128,language:GER v:0,agroup:audio128,name=720p v:1,agroup:audio128,name=1080p” stream_%v.m3u8
this will create files with name “stream_720p.m3u8” instead of “stream_1.m3u8”.
The same for the “hls_segment_filename” name.
Hello, Can we produce multiple video tracks as HLS?
Hi, I can tell this is already a great tutorial.
Can you also share, if you know how to add multiple subtitles (different languages) to this conversion sample
I have been searching this from so many days finally found this. Thank you!
Hi martin, I was wondering if you’ve tried using ffmpeg’s delete_segments option.
My use case is something like this: I have multiple people in the same house who want to watch the twitch stream, but we don’t have enough bandwidth to all watch a 1080p60 stream, so I’m using streamlink to download the stream, and stream it using hls on the LAN via ffmpeg.
My command is this: streamlink –twitch-low-latency https://www.twitch.tv/PGL_Dota2 best -O | ffmpeg -i pipe:0 -c copy -f hls -hls_time 2 -hls_list_size 4 -hls_flags delete_segments+independent_segments -hls_playlist_type event ./stream.m3u8
But the directory keeps growing in size instead of being confined to 5 or so segments. What am I doing wrong?
Simply remove the parameter hls_playlist_type event
Can you tell me how can I implement the same with ffmpeg-rtmp-module and view the multitrack audio on vlc or any hls player. Coz the code above is generating 4 streams and I’m lost as in how to implement this with rtmp-streaming-server which I made using this tutorial
https://www.digitalocean.com/community/tutorials/how-to-set-up-a-video-streaming-server-using-nginx-rtmp-on-ubuntu-22-04
The documentation above is made to be used directly in FFmpeg. There is no need to use ffmpeg-rtmp-module.
So using the command above I’m able to generate streams in my linux server but “-use_localtime_mkdir” flag is throwing an error
“Unrecognized option ‘use_localtime_mkdir’.
Error splitting the argument list: Option not found”
So if I’m able to remove that error can you tell me how can I access that stream using http or another method as I’m trying to make a pseudo-live streaming server which can support multi language.
By the way thanks for the blog
Hello , i’m using nginx-rtmp-module for a live service, the service for now working with one audio output, how can i adapt the command that you have mentionned with my old one:
ffmpeg -i rtmp://localhost:1935/live/$name -async 1 -vsync -1
-c:v libx264 -c:a aac -b:v 128k -b:a 32k -map a:0 -c:a aac -b:a 128k -ac 2 -tune zerolatency -preset veryfast -g 30 -crf 23 -f flv -flvflags no_duration_filesize rtmp://localhost:1935/hls/$name_144
-c:v libx264 -c:a aac -b:v 256k -b:a 64k -map a:0 -c:a aac -b:a 128k -ac 2 -tune zerolatency -preset veryfast -g 30 -crf 23 -f flv -flvflags no_duration_filesize rtmp://localhost:1935/hls/$name_240
-c:v libx264 -c:a aac -b:v 768k -b:a 96k -map a:0 -c:a aac -b:a 128k -ac 2 -tune zerolatency -preset veryfast -g 30 -crf 23 -f flv -flvflags no_duration_filesize rtmp://localhost:1935/hls/$name_360
-c:v libx264 -c:a aac -b:v 1024k -b:a 128k -map a:0 -c:a aac -b:a 128k -ac 2 -tune zerolatency -preset veryfast -g 30 -crf 23 -f flv -flvflags no_duration_filesize rtmp://localhost:1935/hls/$name_480
-c:v libx264 -c:a aac -b:v 2048k -b:a 128k -map a:0 -c:a aac -b:a 128k -ac 2 -tune zerolatency -preset veryfast -g 30 -crf 23 -f flv -flvflags no_duration_filesize rtmp://localhost:1935/hls/$name_720
-c:v libx264 -c:a aac -b:v 3096k -b:a 128k -map a:0 -c:a aac -b:a 128k -ac 2 -tune zerolatency -preset veryfast -g 30 -crf 23 -f flv -flvflags no_duration_filesize rtmp://localhost:1935/hls/$name_1080;
Don’t worry about the input , it will be an udp with multiple audio-track , I did try your solution but I didn’t get the correct result , I need the ngnix-rtmp-module since it provide me with a record and it’s simple to use , any help ?
I recommend you to read all articles to understand how HLS with FFmpeg works. It you do so, you no longer need nginx-rtmp-module.
From what i’m seeing, there are multiple issues with this tutorial according to HLS spec.
– #EXT-X-MEDIA:TYPE=AUDIO is missing, most web players need this for track selection.
– #EXT-X-STREAM-INF is missing an AUDIO component.
Here’s a HLS playlist example that is valid:
#EXTM3U
#EXT-X-VERSION:3
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID=”group_groupname”,NAME=”audio_0″,DEFAULT=YES,URI=”stream-1.m3u8″
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID=”group_groupname”,NAME=”audio_0″,DEFAULT=YES,URI=”stream-2.m3u8″
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID=”group_groupname”,NAME=”audio_0″,DEFAULT=YES,URI=”stream-3.m3u8″
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID=”group_groupname”,NAME=”audio_0″,DEFAULT=YES,URI=”stream-4.m3u8″
#EXT-X-STREAM-INF:BANDWIDTH=174900,RESOLUTION=696×572,AUDIO=”group_groupname”
stream-0.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=174900,CODECS=”mp4a.40.2″,AUDIO=”group_groupname”
stream-1.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=174900,CODECS=”mp4a.40.2″,AUDIO=”group_groupname”
stream-2.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=174900,CODECS=”mp4a.40.2″,AUDIO=”group_groupname”
stream-3.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=174900,CODECS=”mp4a.40.2″,AUDIO=”group_groupname”
stream-4.m3u8
Hi! How can I use -var_stream_map option for mapping data stream, below example do not work
-vap_stream_map “v:0,a:0,d:0 v:1,a:1,d:1 v:2,a:2,d:2”
Any ideas what can I do? I would be really grateful)