lectures/cap15_2_vector_visualization.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "authorship_tag": "ABX9TyMyyUZXoFEk84H6fbEeYzSI",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/viniciusrpb/cic0269_natural_language_processing/blob/main/lectures/cap15_2_vector_visualization.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "#Capítulo 15 - Arquiteturas Encoder-Decoder\n",
        "\n",
        "## Visualização de Vetores"
      ],
      "metadata": {
        "id": "2y03bxpbrykw"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Objetivo da aula 06/12/2022: gerar uma representação gráfica 2D dos context vectors de um Encoder. Para isso, utilize a técnica [t-Stochastic Distributed Neighbor Embedding (t-SNE)](https://lvdmaaten.github.io/tsne/). O objetivo dessa técnica consiste em projetar um objeto $m$-dimensional como um ponto em um espaço 2D.\n",
        "\n",
        "Observe que no caso dos context vectors do Encoder, deve-se reduzir suas dimensionalidades para 2D utilizando a t-SNE. \n",
        "\n",
        "Recomenda-se utilizar a biblioteca [TSNE do sklearn](http://scikit-learn.org/stable/modules/generated/sklearn.manifold.TSNE.html).\n"
      ],
      "metadata": {
        "id": "G7kcBkVwr1sI"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!pip install tensorflow-datasets\n",
        "!pip install plotly==4.14.3\n",
        "!pip install orca"
      ],
      "metadata": {
        "id": "L3GftSsEJtvn"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "import tensorflow_datasets as tfds\n",
        "import tensorflow as tf\n",
        "import pandas as pd\n",
        "import numpy as np\n",
        "from sklearn.manifold import TSNE\n",
        "from tensorflow.keras.preprocessing.text import Tokenizer\n",
        "from tensorflow.keras.preprocessing.sequence import pad_sequences\n",
        "from tensorflow.keras.optimizers import SGD,Adam\n",
        "from keras.utils.np_utils import to_categorical\n",
        "from keras.models import Sequential,Model\n",
        "from keras.layers import Dense,Embedding,Bidirectional,LSTM,Activation,Input,Dropout,BatchNormalization\n",
        "import plotly.express as px\n",
        "import plotly.graph_objects as go"
      ],
      "metadata": {
        "id": "-9-TJXHBK8nD"
      },
      "execution_count": 39,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "ds_train = tfds.load('ted_hrlr_translate/pt_to_en', split='train',\n",
        "                     shuffle_files=True,\n",
        "                     with_info=True,\n",
        "                     as_supervised=True)\n",
        "\n",
        "ds_valid = tfds.load('ted_hrlr_translate/pt_to_en', split='validation',\n",
        "                     shuffle_files=False,\n",
        "                     with_info=True,\n",
        "                     as_supervised=True)\n",
        "\n",
        "ds_test = tfds.load('ted_hrlr_translate/pt_to_en', split='test',\n",
        "                    shuffle_files=False,\n",
        "                    with_info=True,\n",
        "                    as_supervised=True)"
      ],
      "metadata": {
        "id": "vYUZJSNPJ52-"
      },
      "execution_count": 40,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "def ds2DataFrame(ds):\n",
        "    \n",
        "    dic = {}\n",
        "    dic['pt'] = []\n",
        "    dic['en'] = []\n",
        "    for elem in ds[0]:\n",
        "        port = elem[0].numpy().decode('utf-8')\n",
        "        eng = elem[1].numpy().decode('utf-8')\n",
        "        dic['pt'].append(port)\n",
        "        dic['en'].append(eng)\n",
        "\n",
        "    return pd.DataFrame.from_dict(dic)"
      ],
      "metadata": {
        "id": "XXYCIjKbjoMf"
      },
      "execution_count": 41,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "df_train = ds2DataFrame(ds_train)\n",
        "df_valid = ds2DataFrame(ds_valid)\n",
        "df_test = ds2DataFrame(ds_test)"
      ],
      "metadata": {
        "id": "47EFzFoeKO6G"
      },
      "execution_count": 42,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "df_train"
      ],
      "metadata": {
        "id": "4NlErqdoLOu6",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 424
        },
        "outputId": "f5404917-9b5f-4504-d856-4c15693f4b2b"
      },
      "execution_count": 43,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "                                                      pt  \\\n",
              "0      e quando melhoramos a procura , tiramos a únic...   \n",
              "1                 mas e se estes fatores fossem ativos ?   \n",
              "2       mas eles não tinham a curiosidade de me testar .   \n",
              "3      e esta rebeldia consciente é a razão pela qual...   \n",
              "4      `` `` '' podem usar tudo sobre a mesa no meu c...   \n",
              "...                                                  ...   \n",
              "51780  que a ideia louca é mesmo isso , é louca , e c...   \n",
              "51781  agora , em ambos os casos , não mandei nenhum ...   \n",
              "51782  na linha pontilhada a vermelho , mostramos qua...   \n",
              "51783                     agora , a lagarta não morreu .   \n",
              "51784  os meus alunos têm problemas , problemas socia...   \n",
              "\n",
              "                                                      en  \n",
              "0      and when you improve searchability , you actua...  \n",
              "1                           but what if it were active ?  \n",
              "2                  but they did n't test for curiosity .  \n",
              "3      and this conscious defiance is why i , as an a...  \n",
              "4            you can use everything on the table on me .  \n",
              "...                                                  ...  \n",
              "51780  that the crazy idea is just that , it is crazy...  \n",
              "51781  now in both cases , i did n't send them home a...  \n",
              "51782  in the dotted red line , we show what the adop...  \n",
              "51783                now , the caterpillar did n't die .  \n",
              "51784  my students have problems : social , emotional...  \n",
              "\n",
              "[51785 rows x 2 columns]"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-97f57f98-ec68-4c24-87ed-3dc7b04c13a6\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>pt</th>\n",
              "      <th>en</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>e quando melhoramos a procura , tiramos a únic...</td>\n",
              "      <td>and when you improve searchability , you actua...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>mas e se estes fatores fossem ativos ?</td>\n",
              "      <td>but what if it were active ?</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>mas eles não tinham a curiosidade de me testar .</td>\n",
              "      <td>but they did n't test for curiosity .</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>e esta rebeldia consciente é a razão pela qual...</td>\n",
              "      <td>and this conscious defiance is why i , as an a...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>`` `` '' podem usar tudo sobre a mesa no meu c...</td>\n",
              "      <td>you can use everything on the table on me .</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>51780</th>\n",
              "      <td>que a ideia louca é mesmo isso , é louca , e c...</td>\n",
              "      <td>that the crazy idea is just that , it is crazy...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>51781</th>\n",
              "      <td>agora , em ambos os casos , não mandei nenhum ...</td>\n",
              "      <td>now in both cases , i did n't send them home a...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>51782</th>\n",
              "      <td>na linha pontilhada a vermelho , mostramos qua...</td>\n",
              "      <td>in the dotted red line , we show what the adop...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>51783</th>\n",
              "      <td>agora , a lagarta não morreu .</td>\n",
              "      <td>now , the caterpillar did n't die .</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>51784</th>\n",
              "      <td>os meus alunos têm problemas , problemas socia...</td>\n",
              "      <td>my students have problems : social , emotional...</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>51785 rows × 2 columns</p>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-97f57f98-ec68-4c24-87ed-3dc7b04c13a6')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-97f57f98-ec68-4c24-87ed-3dc7b04c13a6 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-97f57f98-ec68-4c24-87ed-3dc7b04c13a6');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 43
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "num_tokens_encoder = 3\n",
        "num_tokens_decoder = 3\n",
        "num_neurons = 128\n",
        "max_encoder_seq_length = 32\n",
        "max_decoder_seq_length = 32"
      ],
      "metadata": {
        "id": "mQqpbVARMz9Z"
      },
      "execution_count": 44,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Criar o dicionário para as palavras em português"
      ],
      "metadata": {
        "id": "9YlIK_K-DZD-"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "def create_dictionary(df):\n",
        "\n",
        "    word2index = {}\n",
        "    word2index['<OOV>'] = 0\n",
        "    word2index['<BEG>'] = 1\n",
        "    word2index['<END>'] = 2\n",
        "\n",
        "    ind = 3\n",
        "    for text in df:\n",
        "        for word in text.split():\n",
        "            if word not in word2index:\n",
        "                word2index[word] = ind\n",
        "                ind+=1\n",
        "\n",
        "    index2word = {}\n",
        "    for key in word2index:\n",
        "        value = word2index[key]\n",
        "        index2word[value] = key\n",
        "\n",
        "    return word2index,index2word"
      ],
      "metadata": {
        "id": "5qGfsE7WDYjW"
      },
      "execution_count": 45,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [],
      "metadata": {
        "id": "skuNoiTTEGmU"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "vocab_pt,index_pt = create_dictionary(df_train['pt'])\n",
        "vocab_en,index_en = create_dictionary(df_train['en'])\n",
        "\n",
        "vocab_pt_size = len(vocab_pt)\n",
        "vocab_en_size = len(vocab_en)\n",
        "\n",
        "tokenizer_pt = Tokenizer(num_words = vocab_pt_size)\n",
        "tokenizer_pt.fit_on_texts(df_train['pt'])\n",
        "\n",
        "tokenizer_en = Tokenizer(num_words = vocab_en_size)\n",
        "tokenizer_en.fit_on_texts(df_train['en'])"
      ],
      "metadata": {
        "id": "H8FfC5YrJH46"
      },
      "execution_count": 46,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "def prepare_data(df):\n",
        "\n",
        "    X = []\n",
        "    y = []\n",
        "\n",
        "    for sentence in df:\n",
        "        tokens = sentence.split()\n",
        "        lista_x = ['<BEG>']\n",
        "        lista_y = []\n",
        "        for i in range(0,len(tokens)):\n",
        "\n",
        "            lista_y.append(tokens[i])\n",
        "            lista_x.append(tokens[i])\n",
        "            #y.append(tokens[i])\n",
        "            #X.append(tokens[i])\n",
        "\n",
        "        lista_y.append('<END>')\n",
        "\n",
        "        X.append(lista_x)\n",
        "        y.append(lista_y)\n",
        "    \n",
        "    return X,y"
      ],
      "metadata": {
        "id": "c9SC-RUpLbmE"
      },
      "execution_count": 47,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "X_train_pt,y_train_pt = prepare_data(df_train['pt'])\n",
        "X_valid_pt,y_valid_pt = prepare_data(df_train['pt'])\n",
        "\n",
        "X_train_en,y_train_en = prepare_data(df_train['en'])\n",
        "X_valid_en,y_valid_en = prepare_data(df_train['en'])"
      ],
      "metadata": {
        "id": "6h72-qkqPwHw"
      },
      "execution_count": 48,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "y_train_pt[0]"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "tL8qE832P2av",
        "outputId": "6eb2141b-7000-45c4-ed3e-f644e0835521"
      },
      "execution_count": 49,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "['e',\n",
              " 'quando',\n",
              " 'melhoramos',\n",
              " 'a',\n",
              " 'procura',\n",
              " ',',\n",
              " 'tiramos',\n",
              " 'a',\n",
              " 'única',\n",
              " 'vantagem',\n",
              " 'da',\n",
              " 'impressão',\n",
              " ',',\n",
              " 'que',\n",
              " 'é',\n",
              " 'a',\n",
              " 'serendipidade',\n",
              " '.',\n",
              " '<END>']"
            ]
          },
          "metadata": {},
          "execution_count": 49
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "def text2sequences(data,vocab):\n",
        "    corpus_int = []\n",
        "    for instance in data:\n",
        "        instance_int = []\n",
        "        for term in instance:\n",
        "            if term in vocab:\n",
        "                instance_int.append(vocab[term])\n",
        "            else:\n",
        "                instance_int.append(vocab['<OOV>'])\n",
        "        corpus_int.append(instance_int)\n",
        "    \n",
        "    return corpus_int"
      ],
      "metadata": {
        "id": "K5ExQkcjkErG"
      },
      "execution_count": 50,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "train_pt_sequences = text2sequences(X_train_pt,vocab_pt)\n",
        "valid_pt_sequences = text2sequences(X_valid_pt,vocab_pt)\n",
        "#test_pt_sequences = text2sequences(X_test_pt,vocab_pt)\n",
        "\n",
        "train_en_sequences = text2sequences(X_train_en,vocab_en)\n",
        "valid_en_sequences = text2sequences(X_valid_en,vocab_en)"
      ],
      "metadata": {
        "id": "vxX93eWbG7Yg"
      },
      "execution_count": 51,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "trunc_type = 'post'\n",
        "padding_type = 'post'\n",
        "max_length = 64\n",
        "\n",
        "train_pt_padded = pad_sequences(train_pt_sequences, maxlen=max_length, padding=padding_type, truncating=trunc_type)\n",
        "valid_pt_padded = pad_sequences(valid_pt_sequences, maxlen=max_length, padding=padding_type, truncating=trunc_type)\n",
        "\n",
        "train_en_padded = pad_sequences(train_en_sequences, maxlen=max_length, padding=padding_type, truncating=trunc_type)\n",
        "valid_en_padded = pad_sequences(valid_en_sequences, maxlen=max_length, padding=padding_type, truncating=trunc_type)\n",
        "#test_padded = pad_sequences(test_sequences, maxlen=max_length, padding=padding_type, truncating=trunc_type)"
      ],
      "metadata": {
        "id": "YIBQ7daflyDg"
      },
      "execution_count": 52,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "#y_train = text2sequences(y_train_pt,vocab_pt)\n",
        "#y_train = pad_sequences(y_train, maxlen=max_length, padding=padding_type, truncating=trunc_type)\n",
        "#y_train_int = to_categorical(y_train)"
      ],
      "metadata": {
        "id": "LhvA6z5vuQ82"
      },
      "execution_count": 53,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "encoder_inputs = Input(shape=(num_tokens_encoder))\n",
        "encoder_embedding = Embedding(input_dim = vocab_pt_size+3,output_dim = 32)(encoder_inputs)\n",
        "encoder = LSTM(num_neurons, return_state=True)\n",
        "encoder_outputs, state_h, state_c = encoder(encoder_embedding)\n",
        "encoder_states = [state_h, state_c]\n",
        "model = Model(inputs=encoder_inputs,outputs=encoder_outputs)"
      ],
      "metadata": {
        "id": "qaocG-c_kzT8"
      },
      "execution_count": 54,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "#adam = Adam(learning_rate=0.01)\n",
        "#model.compile(loss=\"categorical_crossentropy\",optimizer=adam,metrics=['accuracy'])\n",
        "#history = model.fit(train_pt_padded,y_train,validation_data=(valid_padded,y_valid),batch_size=32,epochs=10)"
      ],
      "metadata": {
        "id": "JL7iPjD-tqHW"
      },
      "execution_count": 55,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "tsne = TSNE(n_components=2,perplexity=5,learning_rate=100.0,metric='euclidean', init='random')\n",
        "X_tsne = tsne.fit_transform(word_vectors)\n",
        "\n",
        "fig = px.scatter(X_tsne, x=0, y=1)#, color=labels.astype(object))\n",
        "fig.show()"
      ],
      "metadata": {
        "id": "mYXVp9r-t4Ct",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 235
        },
        "outputId": "748fae20-d351-4d2d-d945-f21bd223ad47"
      },
      "execution_count": 57,
      "outputs": [
        {
          "output_type": "error",
          "ename": "NameError",
          "evalue": "ignored",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mNameError\u001b[0m                                 Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-57-f1daeb095c47>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0mtsne\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mTSNE\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mn_components\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m2\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0mperplexity\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m5\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0mlearning_rate\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m100.0\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0mmetric\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'euclidean'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0minit\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'random'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 2\u001b[0;31m \u001b[0mX_tsne\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtsne\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mfit_transform\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mword_vectors\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      3\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      4\u001b[0m \u001b[0mfig\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mpx\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mscatter\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mX_tsne\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mx\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0my\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;31m#, color=labels.astype(object))\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      5\u001b[0m \u001b[0mfig\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mshow\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;31mNameError\u001b[0m: name 'word_vectors' is not defined"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Vamos utilizar a classe [Model](https://keras.io/api/models/model/) para construir o modelo completo"
      ],
      "metadata": {
        "id": "q8IcsIjzt9VD"
      }
    }
  ]
}