mirror of
https://github.com/w-okada/voice-changer.git
synced 2025-01-23 21:45:00 +03:00
490 lines
18 KiB
Plaintext
490 lines
18 KiB
Plaintext
{
|
||
"nbformat": 4,
|
||
"nbformat_minor": 0,
|
||
"metadata": {
|
||
"colab": {
|
||
"name": "VoiceChangerDemo",
|
||
"provenance": [],
|
||
"collapsed_sections": [],
|
||
"authorship_tag": "ABX9TyPgTZqyNueJKS2mFsp/FPoz",
|
||
"include_colab_link": true
|
||
},
|
||
"kernelspec": {
|
||
"name": "python3",
|
||
"display_name": "Python 3"
|
||
},
|
||
"language_info": {
|
||
"name": "python"
|
||
},
|
||
"accelerator": "GPU",
|
||
"gpuClass": "standard"
|
||
},
|
||
"cells": [
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"id": "view-in-github",
|
||
"colab_type": "text"
|
||
},
|
||
"source": [
|
||
"<a href=\"https://colab.research.google.com/github/w-okada/voice-changer/blob/dev/VoiceChangerDemo.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"source": [
|
||
"Voice Changer (デモ版)\n",
|
||
"---\n",
|
||
"\n",
|
||
"このノートはVoice ChangerをColab上で動かすデモ版です。\n",
|
||
"\n",
|
||
"正式版はローカルPCのDocker上で動かすアプリケーションです。\n",
|
||
"\n",
|
||
"正式版は、多くの場合より少ないタイムラグで滑らかに音声を変換できます。\n",
|
||
"\n",
|
||
"詳細な使用方法はこちらの[リポジトリ](https://github.com/w-okada/voice-changer)からご確認ください。\n"
|
||
],
|
||
"metadata": {
|
||
"id": "Lbbmx_Vjl0zo"
|
||
}
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"source": [
|
||
"# GPUを確認\n",
|
||
"GPUを用いたほうが高速に処理が行えます。\n",
|
||
"\n",
|
||
"下記のコマンドでGPUが確認できない場合は、上のメニューから\n",
|
||
"\n",
|
||
"「ランタイム」→「ランタイムの変更」→「ハードウェア アクセラレータ」\n",
|
||
"\n",
|
||
"でGPUを選択してください。"
|
||
],
|
||
"metadata": {
|
||
"id": "oUKi1NYMmXrr"
|
||
}
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"source": [
|
||
"# (1) GPUの確認\n",
|
||
"!nvidia-smi"
|
||
],
|
||
"metadata": {
|
||
"colab": {
|
||
"base_uri": "https://localhost:8080/"
|
||
},
|
||
"id": "vV1t7PBRm-o6",
|
||
"outputId": "465f78ad-bff5-434d-899e-9c426fe025a1"
|
||
},
|
||
"execution_count": 1,
|
||
"outputs": [
|
||
{
|
||
"output_type": "stream",
|
||
"name": "stdout",
|
||
"text": [
|
||
"Thu Nov 3 22:31:14 2022 \n",
|
||
"+-----------------------------------------------------------------------------+\n",
|
||
"| NVIDIA-SMI 460.32.03 Driver Version: 460.32.03 CUDA Version: 11.2 |\n",
|
||
"|-------------------------------+----------------------+----------------------+\n",
|
||
"| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |\n",
|
||
"| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |\n",
|
||
"| | | MIG M. |\n",
|
||
"|===============================+======================+======================|\n",
|
||
"| 0 Tesla T4 Off | 00000000:00:04.0 Off | 0 |\n",
|
||
"| N/A 56C P0 26W / 70W | 0MiB / 15109MiB | 0% Default |\n",
|
||
"| | | N/A |\n",
|
||
"+-------------------------------+----------------------+----------------------+\n",
|
||
" \n",
|
||
"+-----------------------------------------------------------------------------+\n",
|
||
"| Processes: |\n",
|
||
"| GPU GI CI PID Type Process name GPU Memory |\n",
|
||
"| ID ID Usage |\n",
|
||
"|=============================================================================|\n",
|
||
"| No running processes found |\n",
|
||
"+-----------------------------------------------------------------------------+\n"
|
||
]
|
||
}
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"source": [
|
||
"# 使用するモデルとコンフィグファイルの指定\n",
|
||
"\n",
|
||
"使用するトレーニング済みのモデルと、トレーニングで使用したコンフィグファイルのパスを指定してください。\n",
|
||
"\n",
|
||
"多くの場合はGoogle Driveに格納されているファイルを使用すると思います。その場合は、下の(2-2)のセルを実行してドライブをマウントしてください"
|
||
],
|
||
"metadata": {
|
||
"id": "mHvGrgaWnIPA"
|
||
}
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"source": [
|
||
"# (2-1) 使用するモデルとコンフィグファイルの指定\n",
|
||
"CONFIG=\"/content/drive/MyDrive/VoiceChanger/config.json\"\n",
|
||
"MODEL=\"/content/drive/MyDrive/VoiceChanger/G_326000.pth\""
|
||
],
|
||
"metadata": {
|
||
"id": "nSXATMWYb4Ik"
|
||
},
|
||
"execution_count": 2,
|
||
"outputs": []
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 3,
|
||
"metadata": {
|
||
"colab": {
|
||
"base_uri": "https://localhost:8080/"
|
||
},
|
||
"id": "2wxD-gRSMU5R",
|
||
"outputId": "ad46952e-7a9e-4366-b819-f04e3ed3e3d6"
|
||
},
|
||
"outputs": [
|
||
{
|
||
"output_type": "stream",
|
||
"name": "stdout",
|
||
"text": [
|
||
"Mounted at /content/drive\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"# (2-2) Google Driveのマウント\n",
|
||
"from google.colab import drive\n",
|
||
"drive.mount('/content/drive')"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"source": [
|
||
"# リポジトリのクローン\n",
|
||
"リポジトリをクローンします"
|
||
],
|
||
"metadata": {
|
||
"id": "sLBfykjBnjWc"
|
||
}
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"source": [
|
||
"# (3) リポジトリのクローン\n",
|
||
"!git clone --depth 1 https://github.com/isletennos/MMVC_Trainer.git -b v1.3.1.3 /MMVC_Trainer\n",
|
||
"!git clone --depth 1 https://github.com/w-okada/voice-changer.git\n",
|
||
"%cd voice-changer/demo/\n"
|
||
],
|
||
"metadata": {
|
||
"colab": {
|
||
"base_uri": "https://localhost:8080/"
|
||
},
|
||
"id": "86wTFmqsNMnD",
|
||
"outputId": "d654db02-02e7-4305-d538-5b3d977b3cbd"
|
||
},
|
||
"execution_count": 4,
|
||
"outputs": [
|
||
{
|
||
"output_type": "stream",
|
||
"name": "stdout",
|
||
"text": [
|
||
"Cloning into '/MMVC_Trainer'...\n",
|
||
"remote: Enumerating objects: 917, done.\u001b[K\n",
|
||
"remote: Counting objects: 100% (917/917), done.\u001b[K\n",
|
||
"remote: Compressing objects: 100% (828/828), done.\u001b[K\n",
|
||
"remote: Total 917 (delta 3), reused 888 (delta 0), pack-reused 0\u001b[K\n",
|
||
"Receiving objects: 100% (917/917), 53.02 MiB | 18.46 MiB/s, done.\n",
|
||
"Resolving deltas: 100% (3/3), done.\n",
|
||
"Note: checking out '39723b3243e0265b8802e020e4d4d90a4b8b65ee'.\n",
|
||
"\n",
|
||
"You are in 'detached HEAD' state. You can look around, make experimental\n",
|
||
"changes and commit them, and you can discard any commits you make in this\n",
|
||
"state without impacting any branches by performing another checkout.\n",
|
||
"\n",
|
||
"If you want to create a new branch to retain commits you create, you may\n",
|
||
"do so (now or later) by using -b with the checkout command again. Example:\n",
|
||
"\n",
|
||
" git checkout -b <new-branch-name>\n",
|
||
"\n",
|
||
"Cloning into 'voice-changer'...\n",
|
||
"remote: Enumerating objects: 91, done.\u001b[K\n",
|
||
"remote: Counting objects: 100% (91/91), done.\u001b[K\n",
|
||
"remote: Compressing objects: 100% (80/80), done.\u001b[K\n",
|
||
"remote: Total 91 (delta 12), reused 53 (delta 3), pack-reused 0\u001b[K\n",
|
||
"Unpacking objects: 100% (91/91), done.\n",
|
||
"/content/voice-changer/demo\n"
|
||
]
|
||
}
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"source": [
|
||
"# ファイルの配置\n",
|
||
"アプリケーションの挙動を記した設定ファイルをコピーします(4-1)。(4-2)はコピーした設定ファイルを表示しています。もしかしたらうまく動かないときに役立つかもしれません。"
|
||
],
|
||
"metadata": {
|
||
"id": "jmDY8W_fnuSi"
|
||
}
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"source": [
|
||
"# (4-1) 設定ファイルの配置\n",
|
||
"!cp ../template/setting_mmvc_colab.json ../frontend/dist/assets/setting.json\n"
|
||
],
|
||
"metadata": {
|
||
"id": "Bn4kV8TgXp8i"
|
||
},
|
||
"execution_count": 5,
|
||
"outputs": []
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"source": [
|
||
"# (4-2) 設定ファイルの確認\n",
|
||
"!cat ../frontend/dist/assets/setting.json\n"
|
||
],
|
||
"metadata": {
|
||
"colab": {
|
||
"base_uri": "https://localhost:8080/"
|
||
},
|
||
"id": "pjxPsOOaXXTj",
|
||
"outputId": "dd627fc2-35ee-4c45-ac1a-4fd1095f9a12"
|
||
},
|
||
"execution_count": 6,
|
||
"outputs": [
|
||
{
|
||
"output_type": "stream",
|
||
"name": "stdout",
|
||
"text": [
|
||
"{\n",
|
||
" \"app_title\": \"voice-changer\",\n",
|
||
" \"majar_mode\": \"colab\",\n",
|
||
" \"voice_changer_server_url\": \"/test\",\n",
|
||
" \"sample_rate\": 48000,\n",
|
||
" \"buffer_size\": 1024,\n",
|
||
" \"prefix_chunk_size\": 48,\n",
|
||
" \"chunk_size\": 48,\n",
|
||
" \"speakers\": [\n",
|
||
" {\n",
|
||
" \"id\": 100,\n",
|
||
" \"name\": \"ずんだもん\"\n",
|
||
" },\n",
|
||
" {\n",
|
||
" \"id\": 107,\n",
|
||
" \"name\": \"user\"\n",
|
||
" },\n",
|
||
" {\n",
|
||
" \"id\": 101,\n",
|
||
" \"name\": \"そら\"\n",
|
||
" },\n",
|
||
" {\n",
|
||
" \"id\": 102,\n",
|
||
" \"name\": \"めたん\"\n",
|
||
" },\n",
|
||
" {\n",
|
||
" \"id\": 103,\n",
|
||
" \"name\": \"つむぎ\"\n",
|
||
" }\n",
|
||
" ],\n",
|
||
" \"src_id\": 107,\n",
|
||
" \"dst_id\": 100,\n",
|
||
" \"vf_enable\": true,\n",
|
||
" \"voice_changer_mode\": \"realtime\",\n",
|
||
" \"gpu\": 0,\n",
|
||
" \"available_gpus\": [-1, 0, 1, 2, 3, 4],\n",
|
||
" \"screen\": {\n",
|
||
" \"enable_screen\": true,\n",
|
||
" \"backgournd_image_url\": \"./assets/images/bg_natural_sougen.jpg\"\n",
|
||
" },\n",
|
||
" \"avatar\": {\n",
|
||
" \"enable_avatar\": false,\n",
|
||
" \"motion_capture_face\": false,\n",
|
||
" \"motion_capture_upperbody\": false,\n",
|
||
" \"lip_overwrite_with_voice\": false,\n",
|
||
" \"avatar_url\": \"./assets/vrm/zundamon/zundamon.vrm\",\n",
|
||
" \"background_color\": \"#0000dd\",\n",
|
||
" \"chroma_key\": \"#0000dd\",\n",
|
||
" \"avatar_canvas_size\": [1280, 720],\n",
|
||
" \"screen_canvas_size\": [1280, 720]\n",
|
||
" },\n",
|
||
" \"advance\": {\n",
|
||
" \"avatar_draw_skip_rate\": 3,\n",
|
||
" \"screen_draw_skip_rate\": 3,\n",
|
||
" \"visualizer_draw_skip_rate\": 3,\n",
|
||
" \"cross_fade_lower_value\": 0.1,\n",
|
||
" \"cross_fade_offset_rate\": 0.3,\n",
|
||
" \"cross_fade_end_rate\": 0.6,\n",
|
||
" \"cross_fade_type\": 2\n",
|
||
" },\n",
|
||
" \"transcribe\": {\n",
|
||
" \"lang\": \"日本語(ja-JP)\",\n",
|
||
" \"expire_time\": 5\n",
|
||
" }\n",
|
||
"}\n"
|
||
]
|
||
}
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"source": [
|
||
"# モジュールのインストール\n",
|
||
"\n",
|
||
"必要なモジュールをインストールします。"
|
||
],
|
||
"metadata": {
|
||
"id": "8Na2PbLZSWgZ"
|
||
}
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"source": [
|
||
"# (5) 設定ファイルの確認\n",
|
||
"!apt-get install -y espeak libsndfile1-dev &> /dev/null\n",
|
||
"!pip install unidecode &> /dev/null\n",
|
||
"!pip install phonemizer &> /dev/null\n",
|
||
"!pip install retry &> /dev/null\n",
|
||
"!pip install python-socketio &> /dev/null\n",
|
||
"!pip install fastapi &> /dev/null\n",
|
||
"!pip install python-multipart &> /dev/null\n",
|
||
"!pip install uvicorn &> /dev/null\n",
|
||
"!pip install websockets &> /dev/null\n",
|
||
"!pip install pyOpenSSL &> /dev/null"
|
||
],
|
||
"metadata": {
|
||
"id": "LwZAAuqxX7yY"
|
||
},
|
||
"execution_count": 7,
|
||
"outputs": []
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"source": [
|
||
"# サーバの起動\n",
|
||
"\n",
|
||
"サーバを起動します。(6-1)\n",
|
||
"\n",
|
||
"サーバの起動状況を確認します。(6-2) \n",
|
||
"\n",
|
||
"このセルは繰り返し実行することになるのでCtrl+Retでセルを実行してください。\n",
|
||
"\n",
|
||
"アクセスできるようになるまで、１～２分かかるようです。コーヒーでも飲みに行きましょう。\n",
|
||
"\n",
|
||
"下記のようなテキストが表示されたら起動完了です。\n",
|
||
"\n",
|
||
"**`INFO:root:Loaded checkpoint ...`**\n",
|
||
"\n",
|
||
"```\n",
|
||
"INFO:root:Loaded checkpoint '/content/drive/MyDrive/VoiceChanger/G_326000.pth' (iteration 1136)\n",
|
||
"VoiceChanger Initialized (GPU_NUM:1)\n",
|
||
" PHASE1:__main__\n",
|
||
"Start MMVC SocketIO Server\n",
|
||
" CONFIG:/content/drive/MyDrive/VoiceChanger/config.json, MODEL:/content/drive/MyDrive/VoiceChanger/G_326000.pth\n",
|
||
"DEBUG:asyncio:Using selector: EpollSelector\n",
|
||
" Phase name:MMVCServerSIO\n",
|
||
" PHASE3:MMVCServerSIO\n",
|
||
"INFO:root:Loaded checkpoint '/content/drive/MyDrive/VoiceChanger/G_326000.pth' (iteration 1136)\n",
|
||
"```\n",
|
||
"\n"
|
||
],
|
||
"metadata": {
|
||
"id": "-_2OcN9Borke"
|
||
}
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"source": [
|
||
"# (6-1) サーバの起動\n",
|
||
"import random\n",
|
||
"PORT = 10000 + random.randint(1, 9999)\n",
|
||
"LOG_FILE = f\"LOG_FILE_{PORT}\"\n",
|
||
"\n",
|
||
"get_ipython().system_raw(f'python3 MMVCServerSIO.py -p {PORT} -c {CONFIG} -m {MODEL} --colab True >{LOG_FILE} 2>&1 &')\n",
|
||
"#print(f\"PORT:{PORT}, LOG_FILE:{LOG_FILE}\")"
|
||
],
|
||
"metadata": {
|
||
"id": "iNOAB7zISI6J"
|
||
},
|
||
"execution_count": 12,
|
||
"outputs": []
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"source": [
|
||
"# (6-2) サーバの起動確認 (Ctrl+Retで実行)\n",
|
||
"!tail -20 {LOG_FILE}"
|
||
],
|
||
"metadata": {
|
||
"colab": {
|
||
"base_uri": "https://localhost:8080/"
|
||
},
|
||
"id": "chu06KpAjEK6",
|
||
"outputId": "b5047905-b210-45d2-8fef-3dabebc3e7bb"
|
||
},
|
||
"execution_count": 21,
|
||
"outputs": [
|
||
{
|
||
"output_type": "stream",
|
||
"name": "stdout",
|
||
"text": [
|
||
"\u001b[32m Phase name:__main__\u001b[0m\n",
|
||
"\u001b[32m PHASE3:__main__\u001b[0m\n",
|
||
"INFO:root:Loaded checkpoint '/content/drive/MyDrive/VoiceChanger/G_326000.pth' (iteration 1136)\n",
|
||
"VoiceChanger Initialized (GPU_NUM:1)\n",
|
||
"\u001b[32m PHASE1:__main__\u001b[0m\n",
|
||
"\u001b[17mStart MMVC SocketIO Server\u001b[0m\n",
|
||
"\u001b[34m CONFIG:/content/drive/MyDrive/VoiceChanger/config.json, MODEL:/content/drive/MyDrive/VoiceChanger/G_326000.pth\u001b[0m\n",
|
||
"DEBUG:asyncio:Using selector: EpollSelector\n",
|
||
"\u001b[32m Phase name:MMVCServerSIO\u001b[0m\n",
|
||
"\u001b[32m PHASE3:MMVCServerSIO\u001b[0m\n",
|
||
"INFO:root:Loaded checkpoint '/content/drive/MyDrive/VoiceChanger/G_326000.pth' (iteration 1136)\n"
|
||
]
|
||
}
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"source": [
|
||
"# プロキシを起動\n",
|
||
"ウェブサーバへのアクセスをするためのプロキシを起動します。\n",
|
||
"\n",
|
||
"表示されたURLをクリックして開くと別タブでアプリが開きます。\n",
|
||
"\n",
|
||
"Colabなので、ロードにある程度時間がかかります(30秒くらい)。"
|
||
],
|
||
"metadata": {
|
||
"id": "WhxcFLQEpctq"
|
||
}
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"source": [
|
||
"# (7) プロキシを起動\n",
|
||
"from google.colab.output import eval_js\n",
|
||
"proxy = eval_js( \"google.colab.kernel.proxyPort(\" + str(PORT) + \")\" )\n",
|
||
"print(f\"{proxy}front/\")"
|
||
],
|
||
"metadata": {
|
||
"id": "nkRjZm95l87C",
|
||
"colab": {
|
||
"base_uri": "https://localhost:8080/",
|
||
"height": 34
|
||
},
|
||
"outputId": "b86962da-b605-4a4c-cdc4-22337b6a589a"
|
||
},
|
||
"execution_count": 22,
|
||
"outputs": [
|
||
{
|
||
"output_type": "stream",
|
||
"name": "stdout",
|
||
"text": [
|
||
"https://sf6v0yop3nm-496ff2e9c6d22116-13532-colab.googleusercontent.com/front/\n"
|
||
]
|
||
}
|
||
]
|
||
}
|
||
]
|
||
} |