MP4 or MKV with multiple video streams

Hi,

I continue to explore some methods for using GoPro Max .360 directly in Resolve 17.1.1 .
Because a decoder plugin does not exist (yet). I can develop an OpenFX filter for doing that but before I need to add my medias into Resolve.
Resolve does not want .360 but a simple rename or even a symbolic link (under MacOS or Linux) is sufficient since .360 is in fact a standard MP4 file

My problem is how to handle the 2 different video streams (obviously without split them in 2 files)?

I read someone explaining that is possible. But I cannot succeed. Here (better source) Peter Chamberlain never answer .

ffprobe reports the streams like this:

Code: Select all

ffprobe version N-102454-gdeb35a4de8 Copyright (c) 2007-2021 the FFmpeg developers
  built with Apple clang version 12.0.5 (clang-1205.0.22.9)
  configuration: --enable-opencl --enable-videotoolbox --enable-nonfree --enable-libx264 --enable-libx265 --enable-gpl
  libavutil      57.  0.100 / 57.  0.100
  libavcodec     59.  0.100 / 59.  0.100
  libavformat    59.  0.101 / 59.  0.101
  libavdevice    59.  0.100 / 59.  0.100
  libavfilter     8.  0.101 /  8.  0.101
  libswscale      6.  0.100 /  6.  0.100
  libswresample   4.  0.100 /  4.  0.100
  libpostproc    56.  0.100 / 56.  0.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/Users/rlemeill/Desktop/GS010019.360':
  Metadata:
    major_brand     : mp41
    minor_version   : 538120216
    compatible_brands: mp41
    creation_time   : 2021-04-18T10:03:57.000000Z
    firmware        : H19.03.02.00.71
  Duration: 00:08:02.56, start: 0.000000, bitrate: 66425 kb/s
  Stream #0:0(eng): Video: hevc (Main) (hvc1 / 0x31637668), yuvj420p(pc, bt709), 4096x1344 [SAR 1:1 DAR 64:21], 29988 kb/s, 25 fps, 25 tbr, 90k tbn (default)
    Metadata:
      creation_time   : 2021-04-18T10:03:57.000000Z
      handler_name    : GoPro H.265
      vendor_id       : [0][0][0][0]
      encoder         : GoPro H.265 encoder
      timecode        : 10:03:57:08
  Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 189 kb/s (default)
    Metadata:
      creation_time   : 2021-04-18T10:03:57.000000Z
      handler_name    : GoPro AAC 
      vendor_id       : [0][0][0][0]
      timecode        : 10:03:57:08
  Stream #0:2(eng): Data: none (tmcd / 0x64636D74) (default)
    Metadata:
      creation_time   : 2021-04-18T10:03:57.000000Z
      handler_name    : GoPro TCD 
      timecode        : 10:03:57:08
  Stream #0:3(eng): Data: bin_data (gpmd / 0x646D7067), 85 kb/s (default)
    Metadata:
      creation_time   : 2021-04-18T10:03:57.000000Z
      handler_name    : GoPro MET 
  Stream #0:4(eng): Video: hevc (Main) (hvc1 / 0x31637668), yuvj420p(pc, bt709), 4096x1344 [SAR 1:1 DAR 64:21], 29991 kb/s, 25 fps, 25 tbr, 90k tbn (default)
    Metadata:
      creation_time   : 2021-04-18T10:03:57.000000Z
      handler_name    : GoPro H.265
      vendor_id       : [0][0][0][0]
      encoder         : GoPro H.265 encoder
      timecode        : 10:03:57:08
    Side data:
      displaymatrix: rotation of nan degrees
  Stream #0:5(eng): Audio: pcm_s32le (in32 / 0x32336E69), 48000 Hz, 4 channels, s32, 6144 kb/s (default)
    Metadata:
      creation_time   : 2021-04-18T10:03:57.000000Z
      handler_name    : GoPro AMB 
      vendor_id       : [0][0][0][0]


Streams 0:0 and 0:4 are standard hevc main streams but Resolve sees only one video stream but handle correctly the two audio streams and reports it as 5 tracks (1 stereo for stream 0:1 and 4 mono for stream 0:5) (perfect)

With the new (thanks for the startup speedup ) 17.2,
I saw the new MKV supports so I convert the precedent MP4 to MKV with:

Code: Select all

ffmpeg -i in.MP4 -map 0:0 -map 0:1 -map 0:4 -map 0:5  -c copy out.MKV

But no luck Davinci Resolve does not like multiple video streams.

With the new (thanks for the startup speedup ) 17.2,
I saw the new MKV supports so I convert the precedent MP4 to MKV with:

Code: Select all

ffmpeg -i in.MP4 -map 0:0 -map 0:1 -map 0:4 -map 0:5  -c copy out.MKV

But no luck Davinci Resolve does not like multiple video streams.

Today I used a workaround
First I create two files containing only one stream
Front stream:

Code: Select all

FILE=~/Desktop/temp360/in.360
./ffmpeg -y -i $FILE \
    -copy_unknown -map_metadata 0 \
    -map 0:0 \
    -map 0:1 \
    -map 0:2 -tag:d:0 'tmcd' \
    -map 0:3 -tag:d:1 'gpmd' \
    -map 0:5 \
    -metadata:s:0 handler='GoPro H.265' \
    -metadata:s:1 handler='GoPro AAC' \
    -metadata:s:d:0 handler='GoPro TCD' \
    -metadata:s:d:1 handler='GoPro MET' \
    -metadata:s:4 handler='GoPro AMB' \
    -c copy ~/Desktop/temp360/out-p1.mov

Rear stream

Code: Select all

FILE=~/Desktop/temp360/in.360
./ffmpeg -y -i $FILE \
    -copy_unknown -map_metadata 0 \
    -map 0:4 \
    -map 0:1 \
    -map 0:2 -tag:d:0 'tmcd' \
    -map 0:3 -tag:d:1 'gpmd' \
    -map 0:5 \
    -metadata:s:0 handler='GoPro H.265' \
    -metadata:s:1 handler='GoPro AAC' \
    -metadata:s:d:0 handler='GoPro TCD' \
    -metadata:s:d:1 handler='GoPro MET' \
    -metadata:s:4 handler='GoPro AMB' \
    -c copy ~/Desktop/temp360/out-p2.mov

Next I create an equiangular cubemap clip (GoPro's style)
* import the out-p1.mov and out-p2.mov in Resolve
* Create a timeline with dimension 4096x2688 without scaling (GoPro Max streams are 4096x1344) but with output scaled to UHD
* Insert p1 video and audio in timeline
* Insert p2 video only in timeline (audio is same)

*Now it is time to reconstruct the pseudo equiangular cubemap (with the 2x64 bits overlapped areas)
Set transform y=+672 on p1
Set transform y=-672 on p2

Finally I create a compound clip
Select all in the timeline and make a compound clip

And play with it
* I take my compound clip like if it was my imported media
* Apply my new OpenFX GoPro Max to equirectangular
* Apply Reframe360XL

But it can be much more easier without generating the two files if we can select which stream to place in the timeline.