Mon Jan 27 11:04:38 2020 UTC ()
Pullup ticket #6124 - requested by leot
net/youtube-dl: update

Revisions pulled up:
- net/youtube-dl/Makefile                                       1.198-1.199
- net/youtube-dl/PLIST                                          1.98
- net/youtube-dl/distinfo                                       1.180-1.181
- net/youtube-dl/patches/patch-youtube__dl_extractor_la7.py     1.1

---
   Module Name:    pkgsrc
   Committed By:   leot
   Date:           Tue Jan 14 20:42:39 UTC 2020

   Modified Files:
           pkgsrc/net/youtube-dl: Makefile PLIST distinfo

   Log Message:
   youtube-dl: Update to 20200115

   pkgsrc changes:
    - Apply a pkgsrc patch to fix la7 extractor (shared upstream)

   Changes:
   20200115
   --------
   Extractors
   * [yourporn] Fix extraction (#21645, #22255, #23459)
   + [canvas] Add support for new API endpoint (#17680, #18629)
   * [ndr:base:embed] Improve thumbnails extraction (#23731)
   + [vodplatform] Add support for embed.kwikmotion.com domain
   + [twitter] Add support for promo_video_website cards (#23711)
   * [orf:radio] Clean description and improve extraction
   * [orf:fm4] Fix extraction (#23599)
   * [safari] Fix kaltura session extraction (#23679, #23670)
   * [lego] Fix extraction and extract subtitle (#23687)
   * [cloudflarestream] Improve extraction
       + Add support for bytehighway.net domain
       + Add support for signed URLs
       + Extract thumbnail
   * [naver] Improve extraction
       * Improve geo-restriction handling
       + Extract automatic captions
       + Extract uploader metadata
       + Extract VLive HLS formats
       * Improve metadata extraction
   - [pandatv] Remove extractor (#23630)
   * [dctp] Fix format extraction (#23656)
   + [scrippsnetworks] Add support for www.discovery.com videos
   * [discovery] Fix anonymous token extraction (#23650)
   * [nrktv:seriebase] Fix extraction (#23625, #23537)
   * [wistia] Improve format extraction and extract subtitles (#22590)
   * [vice] Improve extraction (#23631)
   * [redtube] Detect private videos (#23518)

---
   Module Name:    pkgsrc
   Committed By:   leot
   Date:           Wed Jan 15 08:22:03 UTC 2020

   Added Files:
           pkgsrc/net/youtube-dl/patches: patch-youtube__dl_extractor_la7.py

   Log Message:
   youtube-dl: Add missing patch (should be part of previous update)

   Thanks <wiz>!

---
   Module Name:    pkgsrc
   Committed By:   adam
   Date:           Fri Jan 24 08:08:24 UTC 2020

   Modified Files:
           pkgsrc/net/youtube-dl: Makefile distinfo

   Log Message:
   youtube-dl: updated to 20200124

   version 2020.01.24

   Extractors
   * [youtube] Fix sigfunc name extraction
   * [stretchinternet] Fix extraction
   * [voicerepublic] Fix extraction
   * [azmedien] Fix extraction
   * [businessinsider] Fix jwplatform id extraction
   + [24video] Add support for 24video.vip
   * [ivi:compilation] Fix entries extraction
   * [ard] Improve extraction
       * Simplify extraction
       + Extract age limit and series
       * Bypass geo-restriction
   + [nbc] Add support for nbc multi network URLs
   * [americastestkitchen] Fix extraction
   * [zype] Improve extraction
       + Extract subtitles
       + Support URLs with alternative keys/tokens
       + Extract more metadata
   * [orf:tvthek] Improve geo restricted videos detection
   * [soundcloud] Restore previews extraction


(bsiegert)
diff -r1.196 -r1.196.4.1 pkgsrc/net/youtube-dl/Makefile
diff -r1.97 -r1.97.4.1 pkgsrc/net/youtube-dl/PLIST
diff -r1.178 -r1.178.4.1 pkgsrc/net/youtube-dl/distinfo
diff -r0 -r1.1.2.2 pkgsrc/net/youtube-dl/patches/patch-youtube__dl_extractor_la7.py

cvs diff -r1.196 -r1.196.4.1 pkgsrc/net/youtube-dl/Makefile (expand / switch to unified diff)

--- pkgsrc/net/youtube-dl/Makefile 2019/12/26 22:18:18 1.196
+++ pkgsrc/net/youtube-dl/Makefile 2020/01/27 11:04:38 1.196.4.1
@@ -1,18 +1,18 @@ @@ -1,18 +1,18 @@
1# $NetBSD: Makefile,v 1.196 2019/12/26 22:18:18 leot Exp $ 1# $NetBSD: Makefile,v 1.196.4.1 2020/01/27 11:04:38 bsiegert Exp $
2 2
3# XXX: VERSION_DATE can contains also an optional part that indicates 3# XXX: VERSION_DATE can contains also an optional part that indicates
4# XXX: possible same day revisions. PKGNAME preserves that dotted part as is. 4# XXX: possible same day revisions. PKGNAME preserves that dotted part as is.
5VERSION_DATE= 2019.12.25 5VERSION_DATE= 2020.01.24
6DISTNAME= youtube-dl-${VERSION_DATE} 6DISTNAME= youtube-dl-${VERSION_DATE}
7PKGNAME= ${DISTNAME:S/.//:S/.//} 7PKGNAME= ${DISTNAME:S/.//:S/.//}
8CATEGORIES= net 8CATEGORIES= net
9MASTER_SITES= https://youtube-dl.org/downloads/${VERSION_DATE}/ 9MASTER_SITES= https://youtube-dl.org/downloads/${VERSION_DATE}/
10 10
11MAINTAINER= leot@NetBSD.org 11MAINTAINER= leot@NetBSD.org
12HOMEPAGE= https://ytdl-org.github.io/youtube-dl/ 12HOMEPAGE= https://ytdl-org.github.io/youtube-dl/
13COMMENT= Download videos from youtube.com 13COMMENT= Download videos from youtube.com
14LICENSE= public-domain 14LICENSE= public-domain
15 15
16USE_LANGUAGES= # none 16USE_LANGUAGES= # none
17EGG_NAME= youtube_dl-${VERSION_DATE:S/.0/./g} 17EGG_NAME= youtube_dl-${VERSION_DATE:S/.0/./g}
18WRKSRC= ${WRKDIR}/youtube-dl 18WRKSRC= ${WRKDIR}/youtube-dl

cvs diff -r1.97 -r1.97.4.1 pkgsrc/net/youtube-dl/PLIST (expand / switch to unified diff)

--- pkgsrc/net/youtube-dl/PLIST 2019/12/26 22:12:27 1.97
+++ pkgsrc/net/youtube-dl/PLIST 2020/01/27 11:04:38 1.97.4.1
@@ -1,14 +1,14 @@ @@ -1,14 +1,14 @@
1@comment $NetBSD: PLIST,v 1.97 2019/12/26 22:12:27 leot Exp $ 1@comment $NetBSD: PLIST,v 1.97.4.1 2020/01/27 11:04:38 bsiegert Exp $
2bin/youtube-dl 2bin/youtube-dl
3${PYSITELIB}/${EGG_INFODIR}/PKG-INFO 3${PYSITELIB}/${EGG_INFODIR}/PKG-INFO
4${PYSITELIB}/${EGG_INFODIR}/SOURCES.txt 4${PYSITELIB}/${EGG_INFODIR}/SOURCES.txt
5${PYSITELIB}/${EGG_INFODIR}/dependency_links.txt 5${PYSITELIB}/${EGG_INFODIR}/dependency_links.txt
6${PYSITELIB}/${EGG_INFODIR}/entry_points.txt 6${PYSITELIB}/${EGG_INFODIR}/entry_points.txt
7${PYSITELIB}/${EGG_INFODIR}/top_level.txt 7${PYSITELIB}/${EGG_INFODIR}/top_level.txt
8${PYSITELIB}/youtube_dl/YoutubeDL.py 8${PYSITELIB}/youtube_dl/YoutubeDL.py
9${PYSITELIB}/youtube_dl/YoutubeDL.pyc 9${PYSITELIB}/youtube_dl/YoutubeDL.pyc
10${PYSITELIB}/youtube_dl/YoutubeDL.pyo 10${PYSITELIB}/youtube_dl/YoutubeDL.pyo
11${PYSITELIB}/youtube_dl/__init__.py 11${PYSITELIB}/youtube_dl/__init__.py
12${PYSITELIB}/youtube_dl/__init__.pyc 12${PYSITELIB}/youtube_dl/__init__.pyc
13${PYSITELIB}/youtube_dl/__init__.pyo 13${PYSITELIB}/youtube_dl/__init__.pyo
14${PYSITELIB}/youtube_dl/__main__.py 14${PYSITELIB}/youtube_dl/__main__.py
@@ -1348,29 +1348,26 @@ ${PYSITELIB}/youtube_dl/extractor/openlo @@ -1348,29 +1348,26 @@ ${PYSITELIB}/youtube_dl/extractor/openlo
1348${PYSITELIB}/youtube_dl/extractor/openload.pyo 1348${PYSITELIB}/youtube_dl/extractor/openload.pyo
1349${PYSITELIB}/youtube_dl/extractor/ora.py 1349${PYSITELIB}/youtube_dl/extractor/ora.py
1350${PYSITELIB}/youtube_dl/extractor/ora.pyc 1350${PYSITELIB}/youtube_dl/extractor/ora.pyc
1351${PYSITELIB}/youtube_dl/extractor/ora.pyo 1351${PYSITELIB}/youtube_dl/extractor/ora.pyo
1352${PYSITELIB}/youtube_dl/extractor/orf.py 1352${PYSITELIB}/youtube_dl/extractor/orf.py
1353${PYSITELIB}/youtube_dl/extractor/orf.pyc 1353${PYSITELIB}/youtube_dl/extractor/orf.pyc
1354${PYSITELIB}/youtube_dl/extractor/orf.pyo 1354${PYSITELIB}/youtube_dl/extractor/orf.pyo
1355${PYSITELIB}/youtube_dl/extractor/outsidetv.py 1355${PYSITELIB}/youtube_dl/extractor/outsidetv.py
1356${PYSITELIB}/youtube_dl/extractor/outsidetv.pyc 1356${PYSITELIB}/youtube_dl/extractor/outsidetv.pyc
1357${PYSITELIB}/youtube_dl/extractor/outsidetv.pyo 1357${PYSITELIB}/youtube_dl/extractor/outsidetv.pyo
1358${PYSITELIB}/youtube_dl/extractor/packtpub.py 1358${PYSITELIB}/youtube_dl/extractor/packtpub.py
1359${PYSITELIB}/youtube_dl/extractor/packtpub.pyc 1359${PYSITELIB}/youtube_dl/extractor/packtpub.pyc
1360${PYSITELIB}/youtube_dl/extractor/packtpub.pyo 1360${PYSITELIB}/youtube_dl/extractor/packtpub.pyo
1361${PYSITELIB}/youtube_dl/extractor/pandatv.py 
1362${PYSITELIB}/youtube_dl/extractor/pandatv.pyc 
1363${PYSITELIB}/youtube_dl/extractor/pandatv.pyo 
1364${PYSITELIB}/youtube_dl/extractor/pandoratv.py 1361${PYSITELIB}/youtube_dl/extractor/pandoratv.py
1365${PYSITELIB}/youtube_dl/extractor/pandoratv.pyc 1362${PYSITELIB}/youtube_dl/extractor/pandoratv.pyc
1366${PYSITELIB}/youtube_dl/extractor/pandoratv.pyo 1363${PYSITELIB}/youtube_dl/extractor/pandoratv.pyo
1367${PYSITELIB}/youtube_dl/extractor/parliamentliveuk.py 1364${PYSITELIB}/youtube_dl/extractor/parliamentliveuk.py
1368${PYSITELIB}/youtube_dl/extractor/parliamentliveuk.pyc 1365${PYSITELIB}/youtube_dl/extractor/parliamentliveuk.pyc
1369${PYSITELIB}/youtube_dl/extractor/parliamentliveuk.pyo 1366${PYSITELIB}/youtube_dl/extractor/parliamentliveuk.pyo
1370${PYSITELIB}/youtube_dl/extractor/patreon.py 1367${PYSITELIB}/youtube_dl/extractor/patreon.py
1371${PYSITELIB}/youtube_dl/extractor/patreon.pyc 1368${PYSITELIB}/youtube_dl/extractor/patreon.pyc
1372${PYSITELIB}/youtube_dl/extractor/patreon.pyo 1369${PYSITELIB}/youtube_dl/extractor/patreon.pyo
1373${PYSITELIB}/youtube_dl/extractor/pbs.py 1370${PYSITELIB}/youtube_dl/extractor/pbs.py
1374${PYSITELIB}/youtube_dl/extractor/pbs.pyc 1371${PYSITELIB}/youtube_dl/extractor/pbs.pyc
1375${PYSITELIB}/youtube_dl/extractor/pbs.pyo 1372${PYSITELIB}/youtube_dl/extractor/pbs.pyo
1376${PYSITELIB}/youtube_dl/extractor/pearvideo.py 1373${PYSITELIB}/youtube_dl/extractor/pearvideo.py

cvs diff -r1.178 -r1.178.4.1 pkgsrc/net/youtube-dl/distinfo (expand / switch to unified diff)

--- pkgsrc/net/youtube-dl/distinfo 2019/12/26 22:12:27 1.178
+++ pkgsrc/net/youtube-dl/distinfo 2020/01/27 11:04:38 1.178.4.1
@@ -1,9 +1,10 @@ @@ -1,9 +1,10 @@
1$NetBSD: distinfo,v 1.178 2019/12/26 22:12:27 leot Exp $ 1$NetBSD: distinfo,v 1.178.4.1 2020/01/27 11:04:38 bsiegert Exp $
2 2
3SHA1 (youtube-dl-2019.12.25.tar.gz) = 1658783c89b9e1a941e6fa187f9640c8eab94704 3SHA1 (youtube-dl-2020.01.24.tar.gz) = 29d0dd957ceacb10e773ac8f865c75b9d2ed5bc3
4RMD160 (youtube-dl-2019.12.25.tar.gz) = 288bf6a6dc7c31172babb1923167c900b3e642b9 4RMD160 (youtube-dl-2020.01.24.tar.gz) = 64b072449da28ab53728a35a46454486cd58c3ed
5SHA512 (youtube-dl-2019.12.25.tar.gz) = 951219648597d5e19fd3bcb3c64d8641bffd63cf3c6b4b30332d9148ad8b12c1be252e4cacbcd8169ce502b0fe20b2ff7163620382769b5e83e82d155e4f38c8 5SHA512 (youtube-dl-2020.01.24.tar.gz) = 8f85a218395d2932b26c7b76d5b03cbdf9a7c541415c8d1b805239b6ab1ca74b94fc497f32052a1d93fe20ac760994975a13551dd8c5319f0549af20b70a1a78
6Size (youtube-dl-2019.12.25.tar.gz) = 3153090 bytes 6Size (youtube-dl-2020.01.24.tar.gz) = 3159926 bytes
7SHA1 (patch-setup.py) = a67074ae7cfe5e77847c2f610337ea553eddb69b 7SHA1 (patch-setup.py) = a67074ae7cfe5e77847c2f610337ea553eddb69b
 8SHA1 (patch-youtube__dl_extractor_la7.py) = d104c32abb21f9dfe982c0f42e049aeff0f50d67
8SHA1 (patch-youtube__dl_extractor_rai.py) = ae67a6fb599c90491fd68b72bf71821659a2eca4 9SHA1 (patch-youtube__dl_extractor_rai.py) = ae67a6fb599c90491fd68b72bf71821659a2eca4
9SHA1 (patch-youtube__dl_postprocessor_ffmpeg.py) = f96676170a448d9205d542a7def4beca615a1490 10SHA1 (patch-youtube__dl_postprocessor_ffmpeg.py) = f96676170a448d9205d542a7def4beca615a1490

File Added: pkgsrc/net/youtube-dl/patches/patch-youtube__dl_extractor_la7.py
$NetBSD: patch-youtube__dl_extractor_la7.py,v 1.1.2.2 2020/01/27 11:04:38 bsiegert Exp $

[la7] Fix extraction

Parsing `videoParams' or `videoLa7' JavaScript snippets as JSON is getting too
for js_to_json.  Just extract the `vid' from there and use _search_og_* for all
other data.

Update 2nd test with an existent video.

Closes #23323.

Shared upstream via:

 <https://github.com/ytdl-org/youtube-dl/pull/23641>

--- youtube_dl/extractor/la7.py.orig
+++ youtube_dl/extractor/la7.py
@@ -3,7 +3,6 @@ from __future__ import unicode_literals
 
 from .common import InfoExtractor
 from ..utils import (
-    js_to_json,
     smuggle_url,
 )
 
@@ -31,12 +30,16 @@ class LA7IE(InfoExtractor):
         },
     }, {
         # 'src' is a dictionary
-        'url': 'http://tg.la7.it/repliche-tgla7?id=189080',
-        'md5': '6b0d8888d286e39870208dfeceaf456b',
+        'url': 'http://tg.la7.it/repliche-tgla7?id=300465',
+        'md5': '9440057f4531005c426421487770b756',
         'info_dict': {
-            'id': '189080',
+            'id': '0_i2fn53i6',
             'ext': 'mp4',
-            'title': 'TG LA7',
+            'title': 'Repliche Tgla7',
+            'description': 'Repliche Tgla7',
+            'uploader_id': 'kdla7pillole@iltrovatore.it',
+            'timestamp': 1578231030,
+            'upload_date': '20200105',
         },
     }, {
         'url': 'http://www.la7.it/omnibus/rivedila7/omnibus-news-02-07-2016-189077',
@@ -48,20 +51,19 @@ class LA7IE(InfoExtractor):
 
         webpage = self._download_webpage(url, video_id)
 
-        player_data = self._parse_json(
-            self._search_regex(
-                [r'(?s)videoParams\s*=\s*({.+?});', r'videoLa7\(({[^;]+})\);'],
-                webpage, 'player data'),
-            video_id, transform_source=js_to_json)
+        player_data = self._search_regex(
+            [r'(?s)videoParams\s*=\s*({.+?});', r'videoLa7\(({[^;]+})\);'],
+            webpage, 'player data')
+        vid = self._search_regex(r'vid\s*:\s*"(.+?)",', player_data, 'vid')
 
         return {
             '_type': 'url_transparent',
-            'url': smuggle_url('kaltura:103:%s' % player_data['vid'], {
+            'url': smuggle_url('kaltura:103:%s' % vid, {
                 'service_url': 'http://nkdam.iltrovatore.it',
             }),
             'id': video_id,
-            'title': player_data['title'],
+            'title': self._og_search_title(webpage, default=None),
             'description': self._og_search_description(webpage, default=None),
-            'thumbnail': player_data.get('poster'),
+            'thumbnail': self._og_search_thumbnail(webpage, default=None),
             'ie_key': 'Kaltura',
         }