voice-changer/VoiceChangerDemo.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "VoiceChangerDemo",
      "provenance": [],
      "authorship_tag": "ABX9TyNpx+wMQQKflNKso1Aructr",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    },
    "accelerator": "GPU",
    "gpuClass": "standard"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/w-okada/voice-changer/blob/dev/VoiceChangerDemo.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "MMVCプレイヤー（普通版）\n",
        "---\n",
        "\n",
        "このノートはColab上でMMVCのボイチェンを行うノートです。\n",
        "\n",
        "正式版はローカルPCのDocker上で動かすアプリケーションです。\n",
        "\n",
        "正式版は、多くの場合より少ないタイムラグで滑らかに音声を変換できます。\n",
        "\n",
        "詳細な使用方法はこちらの[リポジトリ](https://github.com/w-okada/voice-changer)からご確認ください。\n"
      ],
      "metadata": {
        "id": "Lbbmx_Vjl0zo"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# GPUを確認\n",
        "GPUを用いたほうが高速に処理が行えます。\n",
        "\n",
        "下記のコマンドでGPUが確認できない場合は、上のメニューから\n",
        "\n",
        "「ランタイム」→「ランタイムの変更」→「ハードウェア アクセラレータ」\n",
        "\n",
        "でGPUを選択してください。"
      ],
      "metadata": {
        "id": "oUKi1NYMmXrr"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# (1) GPUの確認\n",
        "!nvidia-smi"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "vV1t7PBRm-o6",
        "outputId": "76db7bf4-f28a-4049-833c-a9f17c1b5472"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Sat Dec  3 10:47:24 2022       \n",
            "+-----------------------------------------------------------------------------+\n",
            "| NVIDIA-SMI 460.32.03    Driver Version: 460.32.03    CUDA Version: 11.2     |\n",
            "|-------------------------------+----------------------+----------------------+\n",
            "| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |\n",
            "| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |\n",
            "|                               |                      |               MIG M. |\n",
            "|===============================+======================+======================|\n",
            "|   0  Tesla T4            Off  | 00000000:00:04.0 Off |                    0 |\n",
            "| N/A   51C    P0    25W /  70W |      0MiB / 15109MiB |      0%      Default |\n",
            "|                               |                      |                  N/A |\n",
            "+-------------------------------+----------------------+----------------------+\n",
            "                                                                               \n",
            "+-----------------------------------------------------------------------------+\n",
            "| Processes:                                                                  |\n",
            "|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |\n",
            "|        ID   ID                                                   Usage      |\n",
            "|=============================================================================|\n",
            "|  No running processes found                                                 |\n",
            "+-----------------------------------------------------------------------------+\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# 使用するモデルとコンフィグファイルの指定\n",
        "\n",
        "使用するトレーニング済みのモデルと、トレーニングで使用したコンフィグファイルのパスを指定してください。\n",
        "\n",
        "多くの場合はGoogle Driveに格納されているファイルを使用すると思います。その場合は、下の(2-2)のセルを実行してドライブをマウントしてください"
      ],
      "metadata": {
        "id": "mHvGrgaWnIPA"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# (2-1) 使用するモデルとコンフィグファイルの指定\n",
        "CONFIG=\"/content/drive/MyDrive/VoiceChanger/config.json\"\n",
        "MODEL=\"/content/drive/MyDrive/VoiceChanger/G_326000.pth\""
      ],
      "metadata": {
        "id": "nSXATMWYb4Ik"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "2wxD-gRSMU5R",
        "outputId": "ff4560ce-c04a-4fe8-bec3-3b93dfdd77ee"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Mounted at /content/drive\n"
          ]
        }
      ],
      "source": [
        "# (2-2) Google Driveのマウント\n",
        "from google.colab import drive\n",
        "drive.mount('/content/drive')"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# リポジトリのクローン\n",
        "リポジトリをクローンします"
      ],
      "metadata": {
        "id": "sLBfykjBnjWc"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# (3) リポジトリのクローン\n",
        "!git clone --depth 1 https://github.com/isletennos/MMVC_Trainer.git -b v1.3.1.5 /MMVC_Trainer\n",
        "!git clone --depth 1 https://github.com/w-okada/voice-changer.git\n",
        "%cd /MMVC_Trainer/monotonic_align \n",
        "!cythonize -3 -i core.pyx  &> /dev/null\n",
        "!mv core.cpython-*.so  monotonic_align &> /dev/null\n",
        "%cd /content/voice-changer/demo/\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "86wTFmqsNMnD",
        "outputId": "35bad20a-b0a8-44c9-a072-10e11a199c50"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Cloning into '/MMVC_Trainer'...\n",
            "remote: Enumerating objects: 917, done.\u001b[K\n",
            "remote: Counting objects: 100% (917/917), done.\u001b[K\n",
            "remote: Compressing objects: 100% (828/828), done.\u001b[K\n",
            "remote: Total 917 (delta 3), reused 889 (delta 0), pack-reused 0\u001b[K\n",
            "Receiving objects: 100% (917/917), 53.02 MiB | 20.41 MiB/s, done.\n",
            "Resolving deltas: 100% (3/3), done.\n",
            "Cloning into 'voice-changer'...\n",
            "remote: Enumerating objects: 108, done.\u001b[K\n",
            "remote: Counting objects: 100% (108/108), done.\u001b[K\n",
            "remote: Compressing objects: 100% (94/94), done.\u001b[K\n",
            "remote: Total 108 (delta 15), reused 75 (delta 3), pack-reused 0\u001b[K\n",
            "Receiving objects: 100% (108/108), 18.79 MiB | 15.05 MiB/s, done.\n",
            "Resolving deltas: 100% (15/15), done.\n",
            "/MMVC_Trainer/monotonic_align\n",
            "/content/voice-changer/demo\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# ファイルの配置\n",
        "アプリケーションの挙動を記した設定ファイルをコピーします(4-1)。(4-2)はコピーした設定ファイルを表示しています。もしかしたらうまく動かないときに役立つかもしれません。"
      ],
      "metadata": {
        "id": "jmDY8W_fnuSi"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# (4-1) 設定ファイルの配置\n",
        "!cp ../template/setting_mmvc_colab.json ../frontend/dist/assets/setting.json\n"
      ],
      "metadata": {
        "id": "Bn4kV8TgXp8i"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# (4-2) 設定ファイルの確認\n",
        "!cat ../frontend/dist/assets/setting.json\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "pjxPsOOaXXTj",
        "outputId": "0aaa53de-2a45-45c1-b0e6-cc6de98873fe"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "{\n",
            "    \"app_title\": \"voice-changer\",\n",
            "    \"majar_mode\": \"colab\",\n",
            "    \"voice_changer_server_url\": \"/test\",\n",
            "    \"sample_rate\": 48000,\n",
            "    \"buffer_size\": 1024,\n",
            "    \"prefix_chunk_size\": 48,\n",
            "    \"chunk_size\": 48,\n",
            "    \"speakers\": [\n",
            "        {\n",
            "            \"id\": 100,\n",
            "            \"name\": \"ずんだもん\"\n",
            "        },\n",
            "        {\n",
            "            \"id\": 107,\n",
            "            \"name\": \"user\"\n",
            "        },\n",
            "        {\n",
            "            \"id\": 101,\n",
            "            \"name\": \"そら\"\n",
            "        },\n",
            "        {\n",
            "            \"id\": 102,\n",
            "            \"name\": \"めたん\"\n",
            "        },\n",
            "        {\n",
            "            \"id\": 103,\n",
            "            \"name\": \"つむぎ\"\n",
            "        }\n",
            "    ],\n",
            "    \"src_id\": 107,\n",
            "    \"dst_id\": 100,\n",
            "    \"vf_enable\": true,\n",
            "    \"voice_changer_mode\": \"realtime\",\n",
            "    \"gpu\": 0,\n",
            "    \"available_gpus\": [-1, 0, 1, 2, 3, 4],\n",
            "    \"screen\": {\n",
            "        \"enable_screen\": true,\n",
            "        \"backgournd_image_url\": \"./assets/images/bg_natural_sougen.jpg\"\n",
            "    },\n",
            "    \"avatar\": {\n",
            "        \"enable_avatar\": false,\n",
            "        \"motion_capture_face\": false,\n",
            "        \"motion_capture_upperbody\": false,\n",
            "        \"lip_overwrite_with_voice\": false,\n",
            "        \"avatar_url\": \"./assets/vrm/zundamon/zundamon.vrm\",\n",
            "        \"background_color\": \"#0000dd\",\n",
            "        \"chroma_key\": \"#0000dd\",\n",
            "        \"avatar_canvas_size\": [1280, 720],\n",
            "        \"screen_canvas_size\": [1280, 720]\n",
            "    },\n",
            "    \"advance\": {\n",
            "        \"avatar_draw_skip_rate\": 3,\n",
            "        \"screen_draw_skip_rate\": 3,\n",
            "        \"visualizer_draw_skip_rate\": 3,\n",
            "        \"cross_fade_lower_value\": 0.1,\n",
            "        \"cross_fade_offset_rate\": 0.3,\n",
            "        \"cross_fade_end_rate\": 0.6,\n",
            "        \"cross_fade_type\": 2\n",
            "    },\n",
            "    \"transcribe\": {\n",
            "        \"lang\": \"日本語(ja-JP)\",\n",
            "        \"expire_time\": 5\n",
            "    }\n",
            "}\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# モジュールのインストール\n",
        "\n",
        "必要なモジュールをインストールします。"
      ],
      "metadata": {
        "id": "8Na2PbLZSWgZ"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# (5) 設定ファイルの確認\n",
        "!apt-get install -y espeak libsndfile1-dev &> /dev/null\n",
        "!pip install unidecode &> /dev/null\n",
        "!pip install phonemizer &> /dev/null\n",
        "!pip install retry &> /dev/null\n",
        "!pip install python-socketio &> /dev/null\n",
        "!pip install fastapi &> /dev/null\n",
        "!pip install python-multipart &> /dev/null\n",
        "!pip install uvicorn &> /dev/null\n",
        "!pip install websockets &> /dev/null\n",
        "!pip install pyOpenSSL &> /dev/null"
      ],
      "metadata": {
        "id": "LwZAAuqxX7yY"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "# サーバの起動\n",
        "\n",
        "サーバを起動します。(6-1)\n",
        "\n",
        "サーバの起動状況を確認します。(6-2) \n",
        "\n",
        "このセルは繰り返し実行することになるのでCtrl+Retでセルを実行してください。\n",
        "\n",
        "アクセスできるようになるまで、１～２分かかるようです。コーヒーでも飲みに行きましょう。\n",
        "\n",
        "下記のようなテキストが表示されたら起動完了です。\n",
        "\n",
        "**`INFO:root:Loaded checkpoint ...`**\n",
        "\n",
        "```\n",
        "INFO:root:Loaded checkpoint '/content/drive/MyDrive/VoiceChanger/G_326000.pth' (iteration 1136)\n",
        "VoiceChanger Initialized (GPU_NUM:1)\n",
        "    PHASE1:__main__\n",
        "Start MMVC SocketIO Server\n",
        "    CONFIG:/content/drive/MyDrive/VoiceChanger/config.json, MODEL:/content/drive/MyDrive/VoiceChanger/G_326000.pth\n",
        "DEBUG:asyncio:Using selector: EpollSelector\n",
        "    Phase name:MMVCServerSIO\n",
        "    PHASE3:MMVCServerSIO\n",
        "INFO:root:Loaded checkpoint '/content/drive/MyDrive/VoiceChanger/G_326000.pth' (iteration 1136)\n",
        "```\n",
        "\n"
      ],
      "metadata": {
        "id": "-_2OcN9Borke"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# (6-1) サーバの起動\n",
        "import random\n",
        "PORT = 10000 + random.randint(1, 9999)\n",
        "LOG_FILE = f\"LOG_FILE_{PORT}\"\n",
        "\n",
        "get_ipython().system_raw(f'python3 MMVCServerSIO.py -t MMVC -p {PORT} -c {CONFIG} -m {MODEL} --colab True >{LOG_FILE} 2>&1 &')\n",
        "#print(f\"PORT:{PORT}, LOG_FILE:{LOG_FILE}\")"
      ],
      "metadata": {
        "id": "iNOAB7zISI6J"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# (6-2) サーバの起動確認 (Ctrl+Retで実行)\n",
        "!tail -20 {LOG_FILE}"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "chu06KpAjEK6",
        "outputId": "c1f8ca20-23e3-42c0-9143-b44498e84bd2"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "\u001b[32m    Phase name:__main__\u001b[0m\n",
            "\u001b[32m    PHASE3:__main__\u001b[0m\n",
            "INFO:root:Loaded checkpoint '/content/drive/MyDrive/VoiceChanger/G_326000.pth' (iteration 1136)\n",
            "VoiceChanger Initialized (GPU_NUM:1)\n",
            "\u001b[32m    PHASE1:__main__\u001b[0m\n",
            "\u001b[17mStart MMVC SocketIO Server\u001b[0m\n",
            "\u001b[34m    CONFIG:/content/drive/MyDrive/VoiceChanger/config.json, MODEL:/content/drive/MyDrive/VoiceChanger/G_326000.pth\u001b[0m\n",
            "DEBUG:asyncio:Using selector: EpollSelector\n",
            "\u001b[32m    Phase name:MMVCServerSIO\u001b[0m\n",
            "\u001b[32m    PHASE3:MMVCServerSIO\u001b[0m\n",
            "INFO:root:Loaded checkpoint '/content/drive/MyDrive/VoiceChanger/G_326000.pth' (iteration 1136)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# プロキシを起動\n",
        "ウェブサーバへのアクセスをするためのプロキシを起動します。\n",
        "\n",
        "表示されたURLをクリックして開くと別タブでアプリが開きます。\n",
        "\n",
        "Colabなので、ロードにある程度時間がかかります(30秒くらい)。"
      ],
      "metadata": {
        "id": "WhxcFLQEpctq"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# (7) プロキシを起動\n",
        "from google.colab.output import eval_js\n",
        "proxy = eval_js( \"google.colab.kernel.proxyPort(\" + str(PORT) + \")\" )\n",
        "print(f\"{proxy}front/\")"
      ],
      "metadata": {
        "id": "nkRjZm95l87C",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "96886f3d-3f88-4392-df41-90514fe4b0bc"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "https://3lddbr773py-496ff2e9c6d22116-11741-colab.googleusercontent.com/front/\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "Jos5WZHGmz4s"
      },
      "execution_count": null,
      "outputs": []
    }
  ]
}