The goal is to Direct Play all media. This means the container, video, audio and subtitles are all compatible with the client. If the media is incompatible for any reason, Jellyfin will use FFmpeg to convert the media. Direct Stream will occur if the audio, container or subtitles happen to not be supported. If the video codec is unsupported, this will result in video transcoding. Subtitles can be tricky because they can cause Direct Stream (subtitles are remuxed) or video transcoding (burning in subtitles) to occur. This is the most intensive CPU component of transcoding. Decoding is less intensive than encoding.
|MPEG-4 Part 2/SP||❌||❌||❌||❌||❌||✅||✅|
|MPEG-4 Part 2/ASP||❌||❌||❌||❌||❌||✅||✅|
1HEVC support is potentially available by offloading to the operating system, but this has not been tested.
2Android playback is currently broken. Client reports that HEVC is supported and attempts to Direct Stream.
|H.262||MPEG-4 SP/ASP||H.264||MP4 Container2||H.265|
|MPEG-2 Video||DivX||MPEG-4 AVC||HEVC|
If the audio codec is unsupported or incompatible (such as playing a 5.1 channel stream on a stereo device), the audio codec must be transcoded. This is not nearly as intensive as video coding.
|MP2 (layer 2)||AAC (Part 7)|
|MP3 (layer 3)|
1MP3 Mono is incorrectly reported as unsupported and will transcode to AAC.
2Only EAC3 2.0 has been tested.
3OGG containers are not supported and will cause VORBIS to convert.
4Only DTS Mono has been tested.
ATSC Standard for AC-3 and EAC-3.
Subtiles can be a subtle issue for transcoding. Containers have a limited number of subtitles that are supported. If subtitles need to be transcoded, it will happen one of two ways. They can be converted into another supported format (text-based subtitles) or burned into the video (image/lossless based and ASS based) due to the subtitles transcoding not being supported. This is the most intenstive method of transcoding due to two transcodings happening at once; applying the subtitle layer on top of the video layer. Here is a breakdown of common subtitle formats.
|SubRip Text (SRT)||Text||❌||🔶||✅||🔶|
1MP4 containers can only support one embedded subtitle stream. This does not affect external subtitles.
2VTT are supported in an HLS Stream.
3ASS Subtitles are only supported by MKV files. MKV files aren't supported by Firefox. They will always inherently be burned into the video.
4DVB-SUB (SUB + IDX) is another name for VobSub files.
"Forced subtitles are common on movies and only provide subtitles when the characters speak a foreign or alien language, or a sign, flag, or other text in a scene is not translated in the localization and dubbing process. In some cases, foreign dialogue may be left untranslated if the movie is meant to be seen from the point of view of a particular character who does not speak the language in question." - Wikipedia
If the container is unsupported, this will result in remuxing. The video and audio codec will remain intact, but wrapped in a supported container. This is the least intensive process. Most video containers will be remuxed to use the HLS streaming protocol and TS containers. Remuxing shouldn't be a concern even for an RPi3.
1MP4 containers are one of the few containers that will not remux.
2MKV containers can hold nearly any codec, but are not compatible with streaming in Firefox and will remux.
3MKV containers are improperly labeled as WebM in Firefox during playback.
4TS is one of the primary containers for streaming for Jellyfin.