Merge remote-tracking branch 'upstream/master' into upstream

2025-02-10 20:22:27 +03:00 · 2013-03-03 20:00:55 +08:00 · 2013-03-03 20:00:55 +08:00 · 0963922004
commit 0963922004
parent 20ef501ec9 9479ee5217
15 changed files with 256 additions and 48 deletions
--- a/CHANGELOG.txt
+++ b/CHANGELOG.txt
@ -1,6 +1,27 @@
 Changelog
 =========

+0.3.3
+-----
+
+*Date: 2013-03-01*
+
+* Add support for:
+    - Douban
+    - MioMio
+* Fix issue for:
+    - Tudou
+    - Vimeo
+
+0.3.2
+-----
+
+*Date: 2013-02-22*
+
+* Add support for:
+    - JPopsuki
+* Fix issue for Xiami.
+
 0.3.1
 -----

--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@ -0,0 +1,25 @@
+## How to Contribute
+
+### Report an issue
+
+In case of any encountered problem, always check your network status first. That is, please ensure the video you want to download can be streamed properly in your web browser.
+
+* Keep in mind that some videos on some hosting sites may have a region restriction, e.g., Youku is blocking access to some videos from IP addresses outside mainland China, and YouTube is also blocking some videos in Germany.
+
+Please include:
+
+* Your exact command line, like `you-get -i "www.youtube.com/watch?v=sGwy8DsUJ4M"`. A common mistake is not to escape the `&`. Putting URLs in quotes should solve this problem.
+
+* Your full console output.
+
+* If you executed the command and got no response, please re-run the command with `--debug`, kill the process with keyboard shortcut `Ctrl-C` and include the full console output.
+
+* The output of `you-get --version`, or `git rev-parse HEAD` -- if you are using a Git version (but always remember to keep up-to-date!)
+
+* The output of `python --version`.
+
+* If possible, you may include your IP address and proxy setting information as well.
+
+### Send me a pull request
+
+My time for maintaining this stuff is very limited. If you really want to have support for some site that has not yet been implemented, the best way is to fix it yourself and send me a pull request.
--- a/README.md
+++ b/README.md
@ -1,4 +1,4 @@
-# You-Get
+# You-Get [![Build Status](https://api.travis-ci.org/soimort/you-get.png)](https://travis-ci.org/soimort/you-get)

 [You-Get](https://github.com/soimort/you-get) is a video downloader runs on Python 3. It aims at easing the download of videos on [YouTube](http://www.youtube.com), [Youku](http://www.youku.com)/[Tudou](http://www.tudou.com) (biggest online video providers in China), [ Niconico](http://www.nicovideo.jp), etc., in one script.

@ -6,8 +6,6 @@ See the project homepage <http://www.soimort.org/you-get> for further documentat

 Fork me on GitHub: <https://github.com/soimort/you-get>

-[![Build Status](https://api.travis-ci.org/soimort/you-get.png)](https://travis-ci.org/soimort/you-get)
-
 ## Features

 ### Supported Sites (As of Now)
@ -20,6 +18,7 @@ Fork me on GitHub: <https://github.com/soimort/you-get>
 * Tumblr <http://www.tumblr.com>
 * SoundCloud <http://soundcloud.com>
 * Mixcloud <http://www.mixcloud.com>
+* JPopsuki <http://jpopsuki.tv>
 * Niconico (ニコニコ動画) <http://www.nicovideo.jp>
 * Youku (优酷) <http://www.youku.com>
 * Tudou (土豆) <http://www.tudou.com>
@ -27,10 +26,12 @@ Fork me on GitHub: <https://github.com/soimort/you-get>
 * AcFun <http://www.acfun.tv>
 * bilibili <http://www.bilibili.tv>
 * CNTV (中国网络电视台) <http://www.cntv.cn>
+* Douban (豆瓣) <http://douban.com>
 * ifeng (凤凰视频) <http://v.ifeng.com>
 * iQIYI (爱奇艺) <http://www.iqiyi.com>
 * Joy.cn (激动网) <http://www.joy.cn>
 * Ku6 (酷6网) <http://www.ku6.com>
+* MioMio <http://www.miomio.tv>
 * NetEase (网易视频) <http://v.163.com>
 * PPTV <http://www.pptv.com>
 * QQ (腾讯视频) <http://v.qq.com>
@ -76,10 +77,6 @@ Fork me on GitHub: <https://github.com/soimort/you-get>
    
    $ make install
    
-   Or (on Windows):
-    
-    > setup.py install
-    
   Check if the installation was successful:
    
    $ you-get -V
@ -98,10 +95,6 @@ Fork me on GitHub: <https://github.com/soimort/you-get>
    
    $ make install
    
-   Or (on Windows):
-    
-    > setup.py install
-    
   Check if the installation was successful:
    
    $ you-get -V
@ -110,6 +103,36 @@ Fork me on GitHub: <https://github.com/soimort/you-get>

   Click [here](https://aur.archlinux.org/packages.php\?ID=62576).

+### FAQ (For Windows Users)
+
+* Q: I don't know how to install it on Windows.
+
+* A: Then don't do it. Just put your `you-get` folder into system `%PATH%`.
+
+* Q: I got something like `UnicodeDecodeError: 'gbk' codec can't decode byte 0xb0 in position 1012: illegal multibyte sequence`.
+
+* A: Run `set PYTHONIOENCODING=utf-8`.
+
+## Upgrading
+
+Using Pip:
+
+    $ pip install --upgrade you-get
+
+### Error When Upgrading from Pip
+
+If you see this error:
+
+```
+  File "/usr/lib/python3.3/site-packages/pip-1.2.1-py3.3.egg/pip/backwardcompat.py", line 44, in u
+    return s.decode('utf-8')
+UnicodeDecodeError: 'utf-8' codec can't decode byte 0xae in position 68: invalid start byte
+```
+
+This is an existing bug in Pip 1.2.1. However, this does not affect your upgrading.
+
+In Pip 1.3+, this should be already fixed.
+
 ## Examples (For End-Users)

 Display the information of the video without downloading:
@ -177,6 +200,10 @@ See source code.

 You-Get is licensed under the [MIT license](https://raw.github.com/soimort/you-get/master/LICENSE.txt).

+## Contributing
+
+Please see [CONTRIBUTING.md](https://github.com/soimort/you-get/blob/master/CONTRIBUTING.md).
+


 ***
@ -203,7 +230,7 @@ You-Get基于优酷下载脚本[iambus/youku-lixian](https://github.com/iambus/y

 ### 支持的站点（截至目前）

-已实现对以下视频站点的支持，以后会陆续增加（・∀・）
+已实现对以下站点的支持，以后会陆续增加（・∀・）

 * YouTube <http://www.youtube.com>
 * Vimeo <http://vimeo.com>
@ -213,6 +240,7 @@ You-Get基于优酷下载脚本[iambus/youku-lixian](https://github.com/iambus/y
 * Tumblr <http://www.tumblr.com>
 * SoundCloud <http://soundcloud.com>
 * Mixcloud <http://www.mixcloud.com>
+* JPopsuki <http://jpopsuki.tv>
 * NICONICO动画 <http://www.nicovideo.jp>
 * 优酷 <http://www.youku.com>
 * 土豆 <http://www.tudou.com>
@ -220,10 +248,12 @@ You-Get基于优酷下载脚本[iambus/youku-lixian](https://github.com/iambus/y
 * AcFun <http://www.acfun.tv>
 * bilibili <http://www.bilibili.tv>
 * CNTV <http://www.cntv.cn>
+* 豆瓣 <http://douban.com>
 * 凤凰视频 <http://v.ifeng.com>
 * 爱奇艺 <http://www.iqiyi.com>
 * 激动网 <http://www.joy.cn>
 * 酷6网 <http://www.ku6.com>
+* MioMio <http://www.miomio.tv>
 * 网易视频 <http://v.163.com>
 * PPTV <http://www.pptv.com>
 * 腾讯视频 <http://v.qq.com>
@ -271,10 +301,6 @@ You-Get基于优酷下载脚本[iambus/youku-lixian](https://github.com/iambus/y
    
    $ make install
    
-   或：（适用于Windows）
-    
-    > setup.py install
-    
   检查安装是否成功：
    
    $ you-get -V
@ -293,10 +319,6 @@ You-Get基于优酷下载脚本[iambus/youku-lixian](https://github.com/iambus/y
    
    $ make install
    
-   或：（适用于Windows）
-    
-    > setup.py install
-    
   检查安装是否成功：
    
    $ you-get -V
@ -305,6 +327,36 @@ You-Get基于优酷下载脚本[iambus/youku-lixian](https://github.com/iambus/y

   点击[这里](https://aur.archlinux.org/packages.php\?ID=62576)。

+### FAQ（针对Windows用户)
+
+* Q：我不知道该如何在Windows下安装。
+
+* A：不需要安装。直接把`you-get`目录放到系统`%PATH%`中。
+
+* Q：出现错误提示`UnicodeDecodeError: 'gbk' codec can't decode byte 0xb0 in position 1012: illegal multibyte sequence`。
+
+* A：执行`set PYTHONIOENCODING=utf-8`。
+
+## 升级
+
+使用Pip：
+
+    $ pip install --upgrade you-get
+
+### 从Pip升级时可能的错误
+
+若出现以下错误提示：
+
+```
+  File "/usr/lib/python3.3/site-packages/pip-1.2.1-py3.3.egg/pip/backwardcompat.py", line 44, in u
+    return s.decode('utf-8')
+UnicodeDecodeError: 'utf-8' codec can't decode byte 0xae in position 68: invalid start byte
+```
+
+这被证实是Pip 1.2.1的一个bug。不过，它并不影响到正常的升级。
+
+这在Pip 1.3+中应当已经被修复。
+
 ## 使用方法示例

 ### 如何下载视频
@ -371,3 +423,7 @@ YouTube等国外视频网站的下载，请移步：[rg3/youtube-dl](https://git
 ## 许可证

 You-Get在[MIT License](https://raw.github.com/soimort/you-get/master/LICENSE.txt)下发布。
+
+## 如何参与贡献 / 报告issue
+
+请阅读 [CONTRIBUTING.md](https://github.com/soimort/you-get/blob/master/CONTRIBUTING.md)。
--- a/README.txt
+++ b/README.txt
@ -21,6 +21,7 @@ Supported Sites (As of Now)
 * Tumblr http://www.tumblr.com
 * SoundCloud http://soundcloud.com
 * Mixcloud http://www.mixcloud.com
+* JPopsuki http://jpopsuki.tv
 * Niconico (ニコニコ動画) http://www.nicovideo.jp
 * Youku (优酷) http://www.youku.com
 * Tudou (土豆) http://www.tudou.com
@ -28,10 +29,12 @@ Supported Sites (As of Now)
 * AcFun http://www.acfun.tv
 * bilibili http://www.bilibili.tv
 * CNTV (中国网络电视台) http://www.cntv.cn
+* Douban (豆瓣) http://douban.com
 * ifeng (凤凰视频) http://v.ifeng.com
 * iQIYI (爱奇艺) http://www.iqiyi.com
 * Joy.cn (激动网) http://www.joy.cn
 * Ku6 (酷6网) http://www.ku6.com
+* MioMio http://www.miomio.tv
 * NetEase (网易视频) http://v.163.com
 * PPTV http://www.pptv.com
 * QQ (腾讯视频) http://v.qq.com
@ -79,10 +82,6 @@ Installation
    
    $ make install
    
-   Or (on Windows)::
-    
-    > setup.py install
-    
   Check if the installation was successful::
    
    $ you-get -V
@ -101,10 +100,6 @@ Installation
    
    $ make install
    
-   Or (on Windows)::
-    
-    > setup.py install
-    
   Check if the installation was successful::
    
    $ you-get -V
@ -184,3 +179,8 @@ License
 -------

 You-Get is licensed under the `MIT license <https://raw.github.com/soimort/you-get/master/LICENSE.txt>`_.
+
+Contributing
+------------
+
+Please see `CONTRIBUTING.md <https://github.com/soimort/you-get/blob/master/CONTRIBUTING.md>`_.
--- a/src/you_get/main.py
+++ b/src/you_get/main.py
@ -24,14 +24,17 @@ def url_to_module(url):
        'bilibili': bilibili,
        'cntv': cntv,
        'dailymotion': dailymotion,
+        'douban': douban,
        'facebook': facebook,
        'google': googleplus,
        'iask': sina,
        'ifeng': ifeng,
        'iqiyi': iqiyi,
        'joy': joy,
+        'jpopsuki': jpopsuki,
        'kankanews': bilibili,
        'ku6': ku6,
+        'miomio': miomio,
        'mixcloud': mixcloud,
        'nicovideo': nicovideo,
        'pptv': pptv,
--- a/src/you_get/downloader/init.py
+++ b/src/you_get/downloader/init.py
@ -4,12 +4,15 @@ from .acfun import *
 from .bilibili import *
 from .cntv import *
 from .dailymotion import *
+from .douban import *
 from .facebook import *
 from .googleplus import *
 from .ifeng import *
 from .iqiyi import *
 from .joy import *
+from .jpopsuki import *
 from .ku6 import *
+from .miomio import *
 from .mixcloud import *
 from .netease import *
 from .nicovideo import *
--- a/src/you_get/downloader/douban.py
+++ b/src/you_get/downloader/douban.py
@ -0,0 +1,25 @@
+#!/usr/bin/env python
+
+__all__ = ['douban_download']
+
+from ..common import *
+
+def douban_download(url, output_dir = '.', merge = True, info_only = False):
+    html = get_html(url)
+    
+    titles = re.findall(r'"name":"([^"]*)"', html)
+    real_urls = [re.sub('\\\\/', '/', i) for i in re.findall(r'"rawUrl":"([^"]*)"', html)]
+    
+    for i in range(len(titles)):
+        title = titles[i]
+        real_url = real_urls[i]
+        
+        type, ext, size = url_info(real_url)
+        
+        print_info(site_info, title, type, size)
+        if not info_only:
+            download_urls([real_url], title, ext, size, output_dir, merge = merge)
+
+site_info = "Douban.com"
+download = douban_download
+download_playlist = playlist_not_supported('douban')
--- a/src/you_get/downloader/jpopsuki.py
+++ b/src/you_get/downloader/jpopsuki.py
@ -0,0 +1,20 @@
+#!/usr/bin/env python
+
+__all__ = ['jpopsuki_download']
+
+from ..common import *
+
+def jpopsuki_download(url, output_dir = '.', merge = True, info_only = False):
+    html = get_html(url)
+    
+    title = r1(r'<meta name="title" content="([^"]*)"', html)[:-14]
+    url = "http://jpopsuki.tv%s" % r1(r'<source src="([^"]*)"', html)
+    type, ext, size = url_info(url)
+    
+    print_info(site_info, title, type, size)
+    if not info_only:
+        download_urls([url], title, ext, size, output_dir, merge = merge)
+
+site_info = "JPopsuki.tv"
+download = jpopsuki_download
+download_playlist = playlist_not_supported('jpopsuki')
--- a/src/you_get/downloader/miomio.py
+++ b/src/you_get/downloader/miomio.py
@ -0,0 +1,27 @@
+#!/usr/bin/env python
+
+__all__ = ['miomio_download']
+
+from ..common import *
+
+from .tudou import tudou_download_by_id
+from .youku import youku_download_by_id
+
+def miomio_download(url, output_dir = '.', merge = True, info_only = False):
+    html = get_html(url)
+    
+    title = r1(r'<meta name="description" content="([^"]*)"', html)
+    flashvars = r1(r'flashvars="(type=[^"]*)"', html)
+    
+    t = r1(r'type=(\w+)', flashvars)
+    id = r1(r'vid=([^"]+)', flashvars)
+    if t == 'youku':
+        youku_download_by_id(id, title, output_dir = output_dir, merge = merge, info_only = info_only)
+    elif t == 'tudou':
+        tudou_download_by_id(id, title, output_dir = output_dir, merge = merge, info_only = info_only)
+    else:
+        raise NotImplementedError(flashvars)
+
+site_info = "MioMio.tv"
+download = miomio_download
+download_playlist = playlist_not_supported('miomio')
--- a/src/you_get/downloader/tudou.py
+++ b/src/you_get/downloader/tudou.py
@ -30,15 +30,19 @@ def tudou_download_by_id(id, title, output_dir = '.', merge = True):
 def tudou_download(url, output_dir = '.', merge = True, info_only = False):
    html = get_decoded_html(url)
    
-    iid = r1(r'iid\s*[:=]\s*(\d+)', html)
-    if not iid:
-        tudou_download_playlist(url, output_dir, merge, info_only)
-        return
-    
    title = r1(r'kw\s*[:=]\s*[\'\"]([^\']+?)[\'\"]', html)
    assert title
    title = unescape_html(title)
    
+    vcode = r1(r'vcode\s*[:=]\s*\'([^\']+)\'', html)
+    if vcode:
+        from .youku import youku_download_by_id
+        return youku_download_by_id(vcode, title, output_dir = output_dir, merge = merge, info_only = info_only)
+    
+    iid = r1(r'iid\s*[:=]\s*(\d+)', html)
+    if not iid:
+        return tudou_download_playlist(url, output_dir, merge, info_only)
+    
    tudou_download_by_iid(iid, title, output_dir = output_dir, merge = merge, info_only = info_only)

 def parse_playlist(url):
--- a/src/you_get/downloader/vimeo.py
+++ b/src/you_get/downloader/vimeo.py
@ -24,7 +24,7 @@ def vimeo_download_by_id(id, title = None, output_dir = '.', merge = True, info_
        download_urls([url], title, ext, size, output_dir, merge = merge, faker = True)

 def vimeo_download(url, output_dir = '.', merge = True, info_only = False):
-    id = r1(r'http://\w*vimeo.com[/\w]*/(\d+)$', url)
+    id = r1(r'http://[\w.]*vimeo.com[/\w]*/(\d+)$', url)
    assert id
    
    vimeo_download_by_id(id, None, output_dir = output_dir, merge = merge, info_only = info_only)
--- a/src/you_get/downloader/xiami.py
+++ b/src/you_get/downloader/xiami.py
@ -28,7 +28,7 @@ def location_dec(str):
    return parse.unquote(out).replace("^", "0")

 def xiami_download_song(sid, output_dir = '.', merge = True, info_only = False):
-    xml = get_html('http://www.xiami.com/song/playlist/id/%s/object_name/default/object_id/0' % sid)
+    xml = get_html('http://www.xiami.com/song/playlist/id/%s/object_name/default/object_id/0' % sid, faker = True)
    doc = parseString(xml)
    i = doc.getElementsByTagName("track")[0]
    artist = i.getElementsByTagName("artist")[0].firstChild.nodeValue
--- a/src/you_get/version.py
+++ b/src/you_get/version.py
@ -2,5 +2,5 @@

 __all__ = ['__version__', '__date__']

-__version__ = '0.3.1'
-__date__ = '2013-02-15'
+__version__ = '0.3.3'
+__date__ = '2013-03-01'
--- a/tests/test.py
+++ b/tests/test.py
@ -6,27 +6,51 @@ import unittest
 from you_get import *
 from you_get.__main__ import url_to_module

+def test_urls(urls):
+    for url in urls:
+        url_to_module(url).download(url, info_only = True)
+
 class YouGetTests(unittest.TestCase):
    
+    def test_douban(self):
+        test_urls([
+            "http://site.douban.com/caofang/",
+        ])
+        
    def test_googleplus(self):
-        for url in [ 
-            "http://plus.google.com/111438309227794971277/posts/So6bW37WWtp",
-            "http://plus.google.com/114038303885145553998/posts/7Jkwa35HZu8",
-            "http://plus.google.com/109544372058574620997/posts/Hn9P3Mbuyud",
-            "http://plus.google.com/photos/109544372058574620997/albums/5835145047890484737/5835145057636064194",
+        test_urls([
            "http://plus.google.com/102663035987142737445/posts/jJRu43KQFT5",
            "http://plus.google.com/+%E5%B9%B3%E7%94%B0%E6%A2%A8%E5%A5%88/posts/jJRu43KQFT5",
            "http://plus.google.com/+平田梨奈/posts/jJRu43KQFT5",
            "http://plus.google.com/photos/102663035987142737445/albums/5844078581209509505/5844078587839097874",
            "http://plus.google.com/photos/+%E5%B9%B3%E7%94%B0%E6%A2%A8%E5%A5%88/albums/5844078581209509505/5844078587839097874",
            "http://plus.google.com/photos/+平田梨奈/albums/5844078581209509505/5844078587839097874",
-        ]:
-            url_to_module(url).download(url, info_only = True)
+        ])
+        
+    def test_jpopsuki(self):
+        test_urls([
+            "http://jpopsuki.tv/video/Dragon-Ash---Run-to-the-Sun/8ad7aec604badd0b0798cd999b63ae17",
+        ])
        
    def test_mixcloud(self):
-        for url in [ 
+        test_urls([
            "http://www.mixcloud.com/beatbopz/beat-bopz-disco-mix/",
            "http://www.mixcloud.com/beatbopz/tokyo-taste-vol4/",
            "http://www.mixcloud.com/DJVadim/north-america-are-you-ready/",
-        ]:
-            url_to_module(url).download(url, info_only = True)
+        ])
+        
+    def test_vimeo(self):
+        test_urls([
+            "http://vimeo.com/56810854",
+        ])
+        
+    def test_xiami(self):
+        test_urls([
+            "http://www.xiami.com/song/1769835121",
+        ])
+        
+    def test_youtube(self):
+        test_urls([
+            "http://www.youtube.com/watch?v=pzKerr0JIPA",
+            "http://youtu.be/pzKerr0JIPA",
+        ])
--- a/2
+++ b/2
@ -1,7 +1,7 @@
 #!/usr/bin/env python3

 import os, sys
-sys.path.insert(0, os.path.join((os.path.dirname(os.path.abspath(__file__))), "src"))
+sys.path.insert(0, os.path.join((os.path.dirname(os.path.realpath(__file__))), "src"))

 from you_get import *