THoR_Finetuning_SemEval2023_t3_1_public.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/nicolay-r/THOR-ECAC/blob/master/THoR_Finetuning_SemEval2023_t3_1_public.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "c4gQyMiAqz9-"
      },
      "source": [
        "# Switch to Python 3.8"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Vkm2dinYp424"
      },
      "outputs": [],
      "source": [
        "!sudo apt-get install python3.8 python3.8-distutils\n",
        "!sudo update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.8 1\n",
        "!sudo update-alternatives --config python3\n",
        "!python3 --version\n",
        "!sudo apt install python3-pip"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "z7ycnMiJm1-g"
      },
      "outputs": [],
      "source": [
        "%cd /content/\n",
        "!rm -rf THOR-ECAC\n",
        "!git clone --branch master https://github.com/nicolay-r/THOR-ECAC.git"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "We install the necessary project requirements"
      ],
      "metadata": {
        "id": "Jo7jDmKiwZt_"
      }
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "9RdnSENQnwZZ"
      },
      "outputs": [],
      "source": [
        "%cd THOR-ECAC\n",
        "!pip install -r requirements.txt"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Download necessary datasets"
      ],
      "metadata": {
        "id": "k3fHWALWtW02"
      }
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "I1HDGM_SsClO"
      },
      "outputs": [],
      "source": [
        "%cd THOR-ECAC\n",
        "!python download_data.py \\\n",
        "  --cause-test \"https://www.dropbox.com/scl/fi/4b2ouqdhgifqy3pmopq08/cause-mult-test.csv?rlkey=tkw0p1e01vezrjbou6v7qh36a&dl=1\" \\\n",
        "  --cause-train \"https://www.dropbox.com/scl/fi/0tlkwbe5awcss2qmihglf/cause-mult-train.csv?rlkey=x9on1ogzn5kigx7c32waudi21&dl=1\" \\\n",
        "  --cause-valid \"https://www.dropbox.com/scl/fi/8zjng2uyghbkpbfcogj6o/cause-mult-valid.csv?rlkey=91dgg4ly7p23e3id2230lqsoi&dl=1\" \\\n",
        "  --state-train \"https://www.dropbox.com/scl/fi/0lokgaeo973wo82ig01hy/state-mult-train.csv?rlkey=tkt1oyo8kwgqs6gp79jn5vbh8&dl=1\" \\\n",
        "  --state-valid \"https://www.dropbox.com/scl/fi/eu4yuk8n61izygnfncnbo/state-mult-valid.csv?rlkey=tlg8rac4ofkbl9o4ipq6dtyos&dl=1\""
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jcsynI374WGl"
      },
      "source": [
        "# Prompt-Tuning Experiments\n"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Launching Stage 1\n",
        "\n",
        "This is the pre-training stage for the LLM, aimed on understanding character state.\n",
        "\n",
        "> **NOTE:** We use `thor_state` engine. Other engines are still available for experiments."
      ],
      "metadata": {
        "id": "4nneW2IAt-i9"
      }
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "y6rb2rt36r3X"
      },
      "outputs": [],
      "source": [
        "!python main.py -r thor_state -es 4 -d state_se24 -bs 4"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Launching Stage 2\n",
        "\n",
        "This is the second stage, aimed on LLM fine-tuning on emotion-cause pairs.\n",
        "\n",
        "Let's consider that we fine-tune the following state, obtained from the stage 1:\n",
        "*  /content/THOR-ECAC/data/save/base_state_se24_1.pth.tar\n",
        "\n",
        "Using three different engines: `prompt_cause`, `thor_cause`, and `thor_cause_rr`."
      ],
      "metadata": {
        "id": "5JqoywEYuAoc"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!python main.py -r prompt_cause -es 4 -d cause_se24 -bs 32 -lp \"/content/THOR-ECAC/data/save/base_state_se24_1.pth.tar\""
      ],
      "metadata": {
        "id": "BvTHKP4WuGmT"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ZwqLcr-S7x-O"
      },
      "outputs": [],
      "source": [
        "!python main.py -r thor_cause -es 4 -d cause_se24 -bs 32 -lp \"/content/THOR-ECAC/data/save/base_state_se24_1.pth.tar\""
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "CMh9k2wM2p7U"
      },
      "outputs": [],
      "source": [
        "!python main.py -r thor_cause_rr -es 4 -d cause_se24 -bs 32 -lp \"/content/THOR-ECAC/data/save/base_state_se24_1.pth.tar\""
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Zz_Xo2emIOXO"
      },
      "source": [
        "# Infer results on the `test` set\n",
        "\n",
        "In this section we infer the results on `test` set in order to later form the Codalab submission.\n",
        "\n",
        "To infer the results on `test` set, we utilize `-i` parameter.\n",
        "\n",
        "> **NOTE:** To load the fine-tuned version of the model for inferring, we can utilize the paramter `-li` instead, which is load by specific epoch index (which is `2` in our case). You can still use `-lp`, but `-li` is shorter."
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "!python main.py -r prompt_cause -d cause_se24 -bs 64 -li 2 -i"
      ],
      "metadata": {
        "id": "uu8jxiTnvG57"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "i0P2RTsOJLmj"
      },
      "outputs": [],
      "source": [
        "!python main.py -r thor_cause -d cause_se24 -bs 64 -li 2 -i"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "MbguuSXGxPLi"
      },
      "outputs": [],
      "source": [
        "!python main.py -r thor_cause_rr -d cause_se24 -bs 64 -li 2 -i"
      ]
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "provenance": [],
      "mount_file_id": "1oeUzE0uGJbb0z9PWdj5i8F2FnfP0G2zC",
      "authorship_tag": "ABX9TyPoJcMA+gI+5Ul6J/71+IXd",
      "include_colab_link": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}