Problem with DJI files, 50fps and tc issues.

[Conniver]

I have a generic proxy workflow, that for some reason wont work with DJI 50fps footage, in this case Ronin 4D.
The timecode is slightly off, enough for it not to work as a proxy (as Davinci Resolve won't recognize it as a proxy)

So the problem is in the DJI RONIN 4D footage, but I need some assistance in how to figure out a workaround.

This is some of the data from media info.
I believe the problem is that in the DJI meta data there is two sets of FPS, once in "Video", and the other in the "Other" tab

ORGINAL:
Video:
Frame rate : 50.000 FPS

Other #1
ID : 6
Type : Time code
Format : QuickTime TC
Duration : 8 s 880 ms
Frame rate : 25.000 FPS
Time code of first frame : 23:55:12:21
Time code of last frame : 23:55:21:17
Time code, stripped : Yes
Language : English
Default : No
Alternate group : 2
Encoded date : UTC 2023-02-06 10:54:48
Tagged date : UTC 2023-02-06 10:54:48

FFASTRANS generated PROXY:
Video tab:
Frame rate : 50.000 FPS

Other
ID : 3
Type : Time code
Format : QuickTime TC
Duration : 8 s 880 ms
Frame rate : 50.000 FPS
Time code of first frame : 23:55:12:21
Time code of last frame : 23:55:21:14
Time code, stripped : Yes
Language : English
Default : No
Encoded date : UTC 2023-08-23 12:36:58
Tagged date : UTC 2023-08-23 12:36:58


Here is the h.264 node generating the proxy.


My workflow has two nodes for h264 encoding, in case one fails it sends it to the built in h.264 encoder.
(Only the DJI ones fails and is sendt to the built in encoder)

The first node is a custom ffmpeg, and looks like this.

Code: Select all

"%s_ffmpeg%" -i "%s_source%" -movflags faststart -map 0:v -map_metadata 0 -sn -s %s_c_res% -g 12 -b:v 6500k -map 0:a:? -b:a 320k -preset fast -c:v h264_nvenc -pix_fmt yuv420p -preset fast -level:v 5.2 -profile high -timecode "%s_start_tc%" "%s_original_path%\Proxy\%s_original_name%.mp4" -y

DJI fails with this error on this node:

Code: Select all

Process exited with error code: 1 (ffmpeg version N-104973-g26dcb05b97 Copyright (c) 2000-2021 the FFmpeg developers
  built with gcc 11.2.0 (Rev5, Built by MSYS2 project)
  configuration:  --disable-static --enable-shared --pkg-config=pkgconf --cc='ccache gcc' --cxx='ccache g++' --ld='ccache g++' --disable-autodetect --enable-amf --enable-bzlib --enable-cuda --enable-cuvid --enable-d3d11va --enable-dxva2 --enable-iconv --enable-lzma --enable-nvenc --enable-zlib --enable-sdl2 --enable-ffnvcodec --enable-nvdec --enable-cuda-llvm --enable-libmp3lame --enable-libopus --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-libdav1d --enable-libaom --disable-debug --enable-fontconfig --enable-libass --enable-libbluray --enable-libfreetype --enable-libmfx --enable-libmysofa --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-amrwbenc --enable-libwebp --enable-libxml2 --enable-libzimg --enable-libshine --enable-gpl --enable-avisynth --enable-libxvid --enable-libopenmpt --enable-version3 --enable-librav1e --enable-libsrt --enable-libgsm --enable-libvmaf --enable-librubberband --enable-mbedtls --extra-cflags=-DLIBTWOLAME_STATIC --extra-libs=-lstdc++ --extra-cflags=-DLIBXML_STATIC --extra-libs=-liconv --disable-w32threads --shlibdir=/local32/bin-video
  libavutil      57. 12.100 / 57. 12.100
  libavcodec     59. 14.100 / 59. 14.100
  libavformat    59. 10.100 / 59. 10.100
  libavdevice    59.  0.101 / 59.  0.101
  libavfilter     8. 20.100 /  8. 20.100
  libswscale      6.  1.101 /  6.  1.101
  libswresample   4.  0.100 /  4.  0.100
  libpostproc    56.  0.100 / 56.  0.100
[mov,mp4,m4a,3gp,3g2,mj2 @ 0136c000] stream 0, timescale not set
[mov,mp4,m4a,3gp,3g2,mj2 @ 0136c000] Stream #5: not enough frames to estimate rate; consider increasing probesize
Guessed Channel Layout for Input Stream #0.1 : stereo
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'D:\FFastrans_TEST\01 - Media\DJI\RONIN_6008x2512_50p.MOV':
  Metadata:
    major_brand     : qt  
    minor_version   : 512
    compatible_brands: qt  
    creation_time   : 2023-02-06T10:54:48.000000Z
    encoder         : DJI RONIN 4D
    com.apple.proapps.manufacturer: DJI
    com.apple.proapps.modelname: RONIN 4D
    org.smpte.rdd18.camera.neutraldensityfilterwheelsetting: 1
    org.smpte.rdd18.camera.shutterspeed_angle: 180.00deg
    org.smpte.rdd18.camera.isosensitivity: 800
    org.smpte.rdd18.camera.whitebalance: 3200K
    org.smpte.rdd18.lens.irisfnumber: F2.8
    org.smpte.rdd18.lens.lenszoom35mmstillcameraequivalent: 24.00mm
    com.apple.proapps.exif.{Exif}.FNumber: 2.800000
    com.apple.proapps.exif.{Exif}.ExposureIndex: 800.000000
    timecode        : 23:55:12:21
  Duration: 00:00:08.92, start: 0.000000, bitrate: 2769910 kb/s
  Stream #0:0[0x1](eng): Video: prores (HQ) (apch / 0x68637061), yuv422p10le(tv, bt709, progressive), 6008x2512, 2776389 kb/s, SAR 1:1 DAR 751:314, 50 fps, 50 tbr, 5k tbn (default)
    Metadata:
      creation_time   : 2023-02-06T10:54:48.000000Z
      handler_name    : VideoHandler
      vendor_id       : appl
      encoder         : Apple ProRes 422 HQ
      timecode        : 23:55:12:21
  Stream #0:1[0x2](eng): Audio: pcm_s24le (in24 / 0x34326E69), 48000 Hz, stereo, s32 (24 bit), 2304 kb/s (default)
    Metadata:
      creation_time   : 2023-02-06T10:54:48.000000Z
      handler_name    : SoundHandler
      vendor_id       : [0][0][0][0]
  Stream #0:2[0x3](eng): Data: none (djmd / 0x646D6A64), 154 kb/s
    Metadata:
      creation_time   : 2023-02-06T10:54:48.000000Z
      handler_name    : DJI meta
  Stream #0:3[0x4](eng): Data: none (dbgi / 0x69676264), 1904 kb/s
    Metadata:
      creation_time   : 2023-02-06T10:54:48.000000Z
      handler_name    : DJI dbgi
  Stream #0:4[0x6](eng): Data: none (tmcd / 0x64636D74), 0 kb/s
    Metadata:
      creation_time   : 2023-02-06T10:54:48.000000Z
      handler_name    : TimeCodeHandler
      timecode        : 23:55:12:21
  Stream #0:5[0x0]: Video: mjpeg (Baseline), yuvj420p(pc, bt470bg/unknown/unknown), 448x186 [SAR 1:1 DAR 224:93], 90k tbr, 90k tbn (attached pic)
Please use -profile:a or -profile:v, -profile is ambiguous
Stream mapping:
  Stream #0:0 -> #0:0 (prores (native) -> h264 (h264_nvenc))
  Stream #0:5 -> #0:1 (mjpeg (native) -> h264 (h264_nvenc))
  Stream #0:1 -> #0:2 (pcm_s24le (native) -> aac (native))
Press [q] to stop, [?] for help
[swscaler @ 05f94240] [swscaler @ 05f9e500] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 05f94240] [swscaler @ 05ff1000] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 05f94240] [swscaler @ 06015900] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 05f94240] [swscaler @ 060a6640] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 05f94240] [swscaler @ 060cf8c0] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 05f94240] [swscaler @ 060b5500] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 05f94240] [swscaler @ 0aa05040] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 05f94240] [swscaler @ 0aa14a80] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 05f94240] [swscaler @ 0aa6eb40] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 05f94240] [swscaler @ 0aad5c00] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 05f94240] [swscaler @ 0ab2b040] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 05f94240] [swscaler @ 0ab76040] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 05f94240] [swscaler @ 0aba8cc0] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 05f94240] [swscaler @ 0ab8d7c0] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 05f94240] [swscaler @ 0ac3f180] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 05f94240] [swscaler @ 0aca4c00] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 05f94240] [swscaler @ 0acf5040] deprecated pixel format used, make sure you did set range correctly
[mp4 @ 03397000] Frame rate very high for a muxer not efficiently supporting it.
Please consider specifying a lower framerate, a different muxer or -vsync 2
[mp4 @ 03397000] Using non-standard frame rate 90000/1
[mp4 @ 03397000] Could not find tag for codec h264 in stream #1, codec not currently supported in container
Could not write header for output file #0 (incorrect codec parameters ?): Invalid argument
Error initializing output stream 0:2 -- 
[aac @ 033c9100] Qavg: nan
Conversion failed!
)

I'm guessing the problem is the two different FPS in the original metadata, 50fps in video, and 25fps in Other.
But I have no clue how to solve this. Or why the DJI footage fails the the first custom ffmpeg node.

Anyone have any clue to what is the problem, and how to fix it.

Thank you.

[emcodem]

Hey Conniver,

Your error message says:

Code: Select all

Stream mapping:
  Stream #0:0 -> #0:0 (prores (native) -> h264 (h264_nvenc))
  Stream #0:5 -> #0:1 (mjpeg (native) -> h264 (h264_nvenc))
  Stream #0:1 -> #0:2 (pcm_s24le (native) -> aac (native))

That means your OUTPUT file gets 2 video tracks and 1 audio track, probably not what you want.

Code: Select all

90k tbr, 90k tbn (attached pic)
...
Using non-standard frame rate 90000/1

Thats kind of high value for framerate, also we see "(attached pic)" which i see first time.
Looking at ffmpeg source code we find this description for "(attached pic)":

Code: Select all

  Add AVStream.attached_pic and AV_DISPOSITION_ATTACHED_PIC,
  used for dealing with attached pictures/cover art.

So it indicates that ffmpeg found some cover image in your source video stream (like there is in some mp3 files).
This cover image "stream" is even reported in the ffmpeg output you wrote:

Code: Select all

 Stream #0:5[0x0]: Video: mjpeg (Baseline), yuvj420p(pc, bt470bg/unknown/unknown), 448x186 [SAR 1:1 DAR 224:93], 90k tbr, 90k tbn (attached pic)

Searching around how to disable this "ATTACHED PIC" stuff in ffmpeg, i found this:
https://superuser.com/questions/1714097 ... ttatchment

Basically saying

Code: Select all

Use -map 0:V in place of -map 0:v to not map cover streams.

Does that work for you?
Did i understand correctly that the inbuilt encoder node works but only the commandline encode fails?

[Conniver]

So at this point the only problem I have is that the timecodes does not match, I get a h264 file from both the "custom" and internal h264 encoder.

Custom:

Code: Select all

"%s_ffmpeg%" -i "%s_source%" -movflags use_metadata_tags -movflags +faststart -map 0:V -map_metadata 0 -sn -s %s_c_res% -g 12 -b:v 6500k -map 0:a:? -b:a 320k -preset fast -c:v h264_nvenc -pix_fmt yuv420p -preset fast -level:v 5.2 -profile high -timecode "%s_start_tc%" "%s_original_path%\Proxy\%s_original_name%.mp4" -y

Time code is the problem now, and only on DJIs 50fps files. They have some strange metadata.

From Mediainfo
Original:

Frame rate : 25.000 FPS
Time code of first frame : 00:38:50:09
Time code of last frame : 00:39:00:02

The framerate in the Video section is 50fps, but 25 in the "other" section of mediainfo.

Proxy:

Frame rate : 50.000 FPS
Time code of first frame : 00:38:50:09
Time code of last frame : 00:38:59:45

The end frame has a slightly different tc in the proxy, making it not usable as a proxy file for editing in Davinci Resolve.

Here is an example of DJI 50fps if anyone is curious.
https://we.tl/t-xqfjov25q9

I realize at this point this is probably an ffmpeg / DJI issue, and not a FFASTRANS problem, but any insight is helpful.

[emcodem]

Hm honestly i have multiple problems with that.
Up in front you have to know that i basically never work with mov directly, the only thing i do with it is to convert it to house format on the ingest side and maybe after all the processing was done, in rare cases, we deliver mov to some customer if they insist.

I don't even understand why the "proxy" actually has a timecode track as we did not tell ffmpeg to generate such a track.
From my perspective a normal way to deal with such stuff is that you first convert whatever you get into your houseformat, e.g. for this stuff you might do XAVC Class 300 for the highres. Then, you can make a proxy from the hires... this way, the proxy generation will work guaranteed because the input format is always the same.

Of course i can check for you who does wrong here, mediainfo or ffmpeg but honestly the workflow as you currently drive it appears very experimental to me, e.g. you can always expect new oddities in the input files which your ingest workflow is not prepared to work with.
Don't get me wrong here, experimental stuff is not bad but i would like to check back with you if you are aware about how ingest workflows usually work and that what you are trying to do takes some special attention?

[admin]

Hi Conniver,

It seems the problem lies in the fact that the timecode track is 25 fps while the video is 50 fps. I have not tried it with DaVinci but in Premiere (using the clip you shared) the frames are doubled from 09 to 18, thus instead of 0:38:50:09 it reads 00:38:50:18. And if I manually use that time code instead in the proxy file I get corresponding timecodes in Premiere. But again, it might be that DaVinci will see it differently. Anyway, applying the doubled frames count in the start TC is simple enough. Take look at this screenshot:

tcfix.png (14.11 KiB) Viewed 1871 times

Try this and see if it helps the situation.

-steinar