Youtube decoder twitter issue

[crispyjones]

I have a simple youtube decoder four step workflow url watchfolder>youtube processor>xdcam-hd processor>delivery. Recently ran a across some twitter video it couldn't deal with here https://twitter.com/SouthMetroPIO/statu ... 6031522816 FFASTrans fails with "job aborted abnormally." When I attempt to complete this same task using the youtube-dl.exe in the FFASTrans processor folder using this command

Code: Select all

youtube-dl.exe  --ffmpeg-location "C:\Users\b----oooo\Desktop\FFAStrans\Processors\ffmpeg\x64" https://twitter.com/SouthMetroPIO/status/948678296031522816

I get the following

Code: Select all

[twitter] 948678296031522816: Downloading webpage
[twitter:card] 948678296031522816: Downloading webpage
[twitter:card] 948678296031522816: Downloading m3u8 information
[twitter:card] 948678296031522816: Downloading mobile webpage
[twitter:card] 948678296031522816: Downloading main script
[twitter:card] 948678296031522816: Downloading mobile API data
[download] Destination: SouthMetroFireRescue - In support of our strategic objectives, South Metro Fire Rescue placed a new engine in service today at Station 39 (Happy Canyon). We constantly strive to provide the best an...-948678296031522816.mp4
ffmpeg version 3.3.3 Copyright (c) 2000-2017 the FFmpeg developers
  built with gcc 7.1.0 (GCC)
  configuration: --disable-static --enable-shared --enable-gpl --enable-version3 --enable-cuda --enable-cuvid --enable-d3d11va --enable-dxva2 --enable-libmfx --enable-nvenc --enable-avisynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libfreetype --enable-libgme --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpe
 --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-libzimg --enable-lzma --enable-zlib
  libavutil      55. 58.100 / 55. 58.100
  libavcodec     57. 89.100 / 57. 89.100
  libavformat    57. 71.100 / 57. 71.100
  libavdevice    57.  6.100 / 57.  6.100
  libavfilter     6. 82.100 /  6. 82.100
  libswscale      4.  6.100 /  4.  6.100
  libswresample   2.  7.100 /  2.  7.100
  libpostproc    54.  5.100 / 54.  5.100
[hls,applehttp @ 00000000004b86a0] Opening 'https://video.twimg.com/ext_tw_video/948678110597165057/pu/vid/0/3000/1280x720/XDHvQsnPNGtNHUvw.ts' for reading
Input #0, hls,applehttp, from 'https://video.twimg.com/ext_tw_video/948678110597165057/pu/pl/1280x720/H-zTPAvOLIP6kMq0.m3u8':
  Duration: 00:00:40.50, start: 0.000000, bitrate: 0 kb/s
  Program 0
    Metadata:
      variant_bitrate : 0
    Stream #0:0: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p, 1280x720, 30 fps, 30 tbr, 90k tbn, 60 tbc
    Metadata:
      variant_bitrate : 0
    Stream #0:1: Audio: aac (LC) ([15][0][0][0] / 0x000F), 44100 Hz, stereo, fltp
    Metadata:
      variant_bitrate : 0
file:SouthMetroFireRescue - In support of our strategic objectives, South Metro Fire Rescue placed a new engine in service today at Station 39 (Happy Canyon). We constantly strive to provide the best an...-948678296031522816.mp4.part: No such file or directory


ERROR: ffmpeg exited with code 1

If you scroll through the ffmpeg call youtube-dl actually found the correct video source here https://video.twimg.com/ext_tw_video/94 ... GtNHUvw.ts. Not sure if this is a FFASTrans problem or youtube-dl.

Again, thanks for this great tool, works on nearly all the video URLs I try. Way better than using sketchy downloader extensions or websites.

[admin]

Hi crispyjones, I just came back from vacation and have not been able to take a look at this. The "job aborted abnormally." is related to the processor-node(s) exiting before it's finished. So THAT is a bug/weakness/safety-mechanism in FFAStrans but youtube-dl might have initiated it. Anyway, I will take a look at the issue. Thanks for reporting!

-steinar

[crispyjones]

I made some progress on this issue. It seems to be an issue with the way twitter URL/shortcuts are created and the Windows 255 character filename limit. If you try and drag the above referenced tweet from the Chrome address bar directly to your desktop, nothing happens. This is because chrome is trying to create a filename greater than 255 characters because the name includes the twitter username+"on twitter"+all the text/emojis in the tweet. If you cheat and simply manually create a windows shortcut the references the short URL text, youtube-dl still kicks out the full length name. This same 255 character limit causes problems (i think) when the file-name variable is handed off to downstream FFASTrans components. I believe I've fixed this on the 9.04 by adding the following youtube-dl switches to the a .conf file referenced by the youtube decoder

--id
--restrict-filenames

This did not work on my 8.2.2 install as adding *any* .conf file to the youtube decoder caused FFASTrans to crash. I am still struggling a bit as I attempted to "upgrade" my online install of 8.2.2 to 9.0.4 and there are some hidden 8.2.2 items that seem to remain and cause problems. Works fine with a "fresh" install.

[admin]

Hi, the problem you reported was fixed in the main 0.9.0 release. How did you upgrade to 0.9.0.4? You need to apply both the main upgrade (0.9.0) and the 0.9.0.4 patch. You can't go directly from 8.2.2 to the 0.9.0.4 patch.

-steinar

[crispyjones]

I see what you mean, looking at the version I believed to be 9.0.4 it really is just 9.0. I thought clicking "latest download" on the front page would be the patched version, I see how it works now. So my attempted upgrade from 8.2.2 to 9.0 failed. What I found was as soon as I would run the 9.0 version, a directory that I had deleted containing the previous version "FFASTrans8.2.2" would be recreated. I didn't want to have the system down for very long so I just went back to 8.2.2 without much investigating.

[crispyjones]

I have the latest 9.1 up and running and I can confirm that long twitter links are now handled correctly *without* the need for a .conf file. Thanks Steinar!

[admin]

Great!

-steinar