MSPRIOR_Training_Template--Kaggle.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# MSPrior"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**MSPrior** is a multi(scale/stream) prior model for realtime temporal learning using RAVE models created by Antoine Caillon. \n",
    "\\\n",
    "[Source code on Github](https://github.com/caillonantoine/msprior)\n",
    "\\\n",
    "\\\n",
    "**RAVE** is a variational autoencoder for fast and high-quality neural audio synthesis by Antoine Caillon and Philippe Esling.\n",
    "\\\n",
    "[Article on arxiv](https://arxiv.org/abs/2111.05011) & [Source code on Github](https://github.com/acids-ircam/RAVE)\n",
    "\n",
    "----\n",
    "<small>*Note that this notebook has been tested with RAVE models until version 2.2.2 - using models created with version >= 2.3. might not lead to a successful training.*\n",
    "\\\n",
    "\\\n",
    "Last updated: 17.03.2024 \\\n",
    "Author: Martin Heinze"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Install Conda, MSPrior, dependencies\n",
    "\n",
    "This installs all neccessary components and libraries to run this notebook on Kaggle. \\\n",
    "\\\n",
    "*Note: the current version (24.10.2023) of MSPrior training stops at epoch 1000. In order to run the training beyond 1000 epochs, the setup script below downloads a different train.py from devstermarts' fork of MSPrior where the training length can be adjusted to a max amount of epochs. The training cell after the preprocessing step contains command lines for both, the default training as well as the alternative version.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Install Conda\n",
    "!mkdir /kaggle/temp\n",
    "%cd /kaggle/temp\n",
    "!curl -L https://repo.anaconda.com/miniconda/Miniconda3-py39_4.12.0-Linux-x86_64.sh -o miniconda.sh\n",
    "!chmod +x miniconda.sh\n",
    "!sh miniconda.sh -b -p /kaggle/temp/miniconda\n",
    "\n",
    "#Upgrade ipython ipykernel, install ffmpeg\n",
    "!/kaggle/temp/miniconda/bin/pip install --upgrade ipython ipykernel\n",
    "!/kaggle/temp/miniconda/bin/conda install ffmpeg --yes\n",
    "\n",
    "#Install MSPRIOR \n",
    "#Pin to specific release tag if necessary using 'acids-msprior==tag.version‘\n",
    "!/kaggle/temp/miniconda/bin/pip install acids-msprior==1.1.2\n",
    "\n",
    "#Force Torch Reinstall with current Kaggle Kernel compatible CUDA driver version\n",
    "#!/kaggle/temp/miniconda/bin/pip install torch==2.1.0 torchvision==0.16.0 torchaudio==2.1.0 --index-url https://download.pytorch.org/whl/cu118\n",
    "!/kaggle/temp/miniconda/bin/pip install torch torchvision torchaudio --force-reinstall --index-url https://download.pytorch.org/whl/cu118\n",
    "\n",
    "#Download devstermarts train.py (to train > 1000 epochs)\n",
    "!mkdir /kaggle/temp/msprior/\n",
    "!mkdir /kaggle/temp/msprior/msprior_scripts/\n",
    "!curl -L https://raw.githubusercontent.com/devstermarts/msprior/master/msprior_scripts/train.py -o /kaggle/temp/msprior/msprior_scripts/train.py"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Use the filepath to your audio dataset for the --audio flag. This is usually /kaggle/input/your_audio_dataset when you include a dataset uploaded to Kaggle. \\\n",
    "Also add the path of your previously trained RAVE model after the --rave flag. Note that the RAVE model needs to be one exported without the --streaming flag. \\\n",
    "On completion of preprocessing, you will find files lock.mdb, data.mdb and the model in the /kaggle/working/processed folder. \\\n",
    "For all possible flags and parameters of preprocessing, check preprocess.py in https://github.com/caillonantoine/msprior/tree/master/msprior/msprior_scripts \\\n",
    "\\\n",
    "*Note: Preprocessing is neccessary only once per training. When resuming training, make sure to disable this section of the notebook.*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Preprocess the dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!mkdir /kaggle/working/processed\n",
    "\n",
    "%cd /kaggle/working/\n",
    "!/kaggle/temp/miniconda/bin/msprior preprocess \\\n",
    "--audio /kaggle/input/your_audio_dataset \\\n",
    "--out_path /kaggle/working/processed \\\n",
    "--rave /kaggle/input/your_rave_model_without_streaming.ts"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Start training\n",
    "\n",
    "Options for --config are: decoder_only / recurrent / encoder_decoder / encoder_decoder_continuous \\\n",
    "*Note: the last two are dependent on rave2vec, which has not yet been open sourced (24.10.2023)* \\\n",
    "Training progress will be stored in /kaggle/working/runs/name_of_your_training \\\n",
    "For all possible parameters, check train.py in https://github.com/caillonantoine/msprior/tree/master/msprior_scripts and the .gin file to your chosen configuration in https://github.com/caillonantoine/msprior/tree/master/msprior/config \\\n",
    "\\\n",
    "*Note: When resuming training, make sure to disable this section of the notebook.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%cd /kaggle/working/\n",
    "\n",
    "##Use this for original train.py script. Default training length is set to 1000 epochs.\n",
    "#!/kaggle/temp/miniconda/bin/msprior train \\\n",
    "#!/kaggle/temp/miniconda/bin/python /kaggle/temp/msprior/msprior_scripts/train.py \\\n",
    "#--db_path /kaggle/working/processed/ \\\n",
    "#--name name_of_your_training \\\n",
    "#--config recurrent\n",
    "\n",
    "#Default: use devstemarts' train.py. Set flag 'epochs' to any integer, use '-1' for continuous training.\n",
    "!/kaggle/temp/miniconda/bin/python /kaggle/temp/msprior/msprior_scripts/train.py \\\n",
    "--db_path /kaggle/working/processed/ \\\n",
    "--name name_of_your_training \\\n",
    "--config recurrent \\\n",
    "--epochs 2000"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Resume training\n",
    "\n",
    "In order to conveniently resume training RAVE on Kaggle, you can transform the output data of the earlier run into a new dataset from the output tab of your notebook. You then add the new dataset to the notebook when editing (or update dataset, if it has been integrated into the notebook already), disable the above initial, **Preprocess the dataset** and **Start training** sections of the notebook and adjust the below section according to your earlier training details. \\\n",
    "\\\n",
    "*Note: The following section is disabled by default, enable when resuming.*  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "##Copy contents of earlier training to /kaggle/working folder.\n",
    "#!cp -r /kaggle/input/your_earlier_training_run/* /kaggle/working\n",
    "\n",
    "#%cd /kaggle/working/\n",
    "\n",
    "###Use this for original train.py script. Default training length is set to 1000 epochs.\n",
    "##!/kaggle/temp/miniconda/bin/msprior train \\\n",
    "##--db_path /kaggle/working/processed/ \\\n",
    "##--name name_of_your_training \\\n",
    "##--config recurrent \\\n",
    "##--ckpt /kaggle/working/runs/name-of-your-training/version_X/checkpoints/checkpoint-file.ckpt\n",
    "\n",
    "##Default: use devstemarts' train.py. Set flag 'epochs' to any integer, use '-1' for continuous training.\n",
    "#!/kaggle/temp/miniconda/bin/python /kaggle/temp/msprior/msprior_scripts/train.py \\\n",
    "#--db_path /kaggle/working/processed/ \\\n",
    "#--name name_of_your_training \\\n",
    "#--config recurrent \\\n",
    "#--epochs 2000 \\\n",
    "##--ckpt /kaggle/working/runs/name-of-your-training/version_X/checkpoints/checkpoint-file.ckpt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Export model\n",
    "\n",
    "After your training is finished, you can export a model (.ts Torchscript) file. \\\n",
    "Use the --continuous flag if you trained MSPrior with any but a discrete RAVE model. \\\n",
    "The --batch_size=64 flag fixes a current export issue with models trained with recurrent configuration, which has not yet been adressed (24.10.2023). \\\n",
    "Make sure to change the folder on --run flag according to the 'name_of_your_training' from above. \\\n",
    "The exported model will be created in /kaggle/working/runs/name_of_your_training/ \\\n",
    "For all possible parameters, check export.py in https://github.com/caillonantoine/msprior/tree/master/msprior/msprior_scripts"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%cd /kaggle/working/\n",
    "!/kaggle/temp/miniconda/bin/msprior export \\\n",
    "--run /kaggle/working/runs/name_of_your_training \\\n",
    "--continuous \\\n",
    "--batch_size=64"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}