00_tensorflow_fundamentals.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "00_tensorflow_fundamentals.ipynb",
      "provenance": [],
      "authorship_tag": "ABX9TyO6FNJgCaR2VcNU9mYRmWuS",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/shauryanegi/Neural-Networks/blob/master/00_tensorflow_fundamentals.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "EIqAVW2A5Oo8"
      },
      "source": [
        "##In this notebook we are going to cover some of the most fundamental concepts of tensors using tensorflow.\n",
        "\n",
        "More specefically, we are going to cover:\n",
        "\n",
        "*   Introduction to Tensors\n",
        "*   Getting information from tensors\n",
        "*   Manipulating Tensors\n",
        "*   Tensors and NumPy\n",
        "*   Using @tf.function(a way to speed up your       regular Python functions)\n",
        "*   Using GPU's with Tensorflow (or TPU's)\n",
        "*   Exercises to try yourself!\n",
        "*   List item\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "19umgdvU5ja5"
      },
      "source": [
        "## Introduction to Tensors"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "lhAz-nEz6-vm",
        "outputId": "36a40e88-5d19-403b-b2b5-702c87015354"
      },
      "source": [
        "# Import TensorFlow\n",
        "import tensorflow as tf\n",
        "print(tf.__version__)"
      ],
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "2.5.0\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "g21_D4ZC7Gc_",
        "outputId": "47469e4b-1482-490b-900b-fc90f56bdaeb"
      },
      "source": [
        "#Create tensors with tf.constant()\n",
        "scalar = tf.constant(7)\n",
        "scalar"
      ],
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=int32, numpy=7>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 2
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "bxGxy8dS7gHO",
        "outputId": "2f4ddaf0-0e32-46a4-f77f-34a0374655a1"
      },
      "source": [
        "# Check the number of dimensions in a tensor. (ndim = number of dimensions)\n",
        "scalar.ndim"
      ],
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 3
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "VTytd_lv7_OL",
        "outputId": "45c70b96-3876-4693-fbbf-5cefeefc838b"
      },
      "source": [
        "# Create a vector\n",
        "vector = tf.constant([10,10])\n",
        "vector"
      ],
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2,), dtype=int32, numpy=array([10, 10], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 4
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "8lPqxg1q8Fs9",
        "outputId": "b211e6ed-7d86-4d1a-ee74-ac370446c281"
      },
      "source": [
        "#Check the dimension of our vector\n",
        "vector.ndim"
      ],
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "1"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 5
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "hkxjhQDe8G_W",
        "outputId": "43eb9c69-4152-4e4e-d7df-1ed827e38345"
      },
      "source": [
        "# Create a matrix (has more than one dimension)\n",
        "matrix = tf.constant(([10,7],[10,7]))\n",
        "matrix"
      ],
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2), dtype=int32, numpy=\n",
              "array([[10,  7],\n",
              "       [10,  7]], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 6
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "NwcesfF98fen",
        "outputId": "b1a22c1a-abab-4617-ee17-f25d078462a4"
      },
      "source": [
        "#Check the dimensions of the matrix\n",
        "matrix.ndim"
      ],
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "2"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 7
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "riEDEM798jdu",
        "outputId": "b367f9a3-e8f4-4c1f-dfad-7feda8fb8a11"
      },
      "source": [
        "# Create another matrix\n",
        "\n",
        "another_matrix = tf.constant([[10., 7.], [3., 2.], [8., 9.]],\n",
        "                             dtype=tf.float16)\n",
        "another_matrix\n"
      ],
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 2), dtype=float16, numpy=\n",
              "array([[10.,  7.],\n",
              "       [ 3.,  2.],\n",
              "       [ 8.,  9.]], dtype=float16)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 8
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Sdc3_YH184sj",
        "outputId": "c999faf1-8d9b-4a6a-96df-a184b5bc6e9e"
      },
      "source": [
        "# What's the number of dimensions of another_matrix?\n",
        "another_matrix.ndim"
      ],
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "2"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 9
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "b62ZvoAz9hu_",
        "outputId": "eb0fdc41-0cde-49fa-bd7b-e322ec7de312"
      },
      "source": [
        "# Let us create a tensor\n",
        "\n",
        "tensor = tf.constant([[[1, 2, 3], [2, 3, 4]], [[5, 6, 7], [10, 22,\n",
        "                     12]], [[19, 14, 54], [43, 54, 53]]])\n",
        "tensor"
      ],
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 2, 3), dtype=int32, numpy=\n",
              "array([[[ 1,  2,  3],\n",
              "        [ 2,  3,  4]],\n",
              "\n",
              "       [[ 5,  6,  7],\n",
              "        [10, 22, 12]],\n",
              "\n",
              "       [[19, 14, 54],\n",
              "        [43, 54, 53]]], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 10
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "2U-2q1Nx-G65",
        "outputId": "60371d51-5bf9-46f4-93ae-ab74e06743a5"
      },
      "source": [
        "tensor.ndim"
      ],
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "3"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 11
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fjoqAfrC-2no"
      },
      "source": [
        "What we have learned so far:\n",
        "\n",
        "* Scalar: a single number\n",
        "* Vector: a number with direction\n",
        "* Matrix: a 2-dimensional array of numbers\n",
        "* Tensor: a n-dimensional array (when n can be any number, a 0- dimensional tensor is a scalar, a 1-dimensional tensor is a vector)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FJt_W5hH_f6Z"
      },
      "source": [
        "## Creating Tensors with tf.Variable"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "BfNNuDKM_xUo",
        "outputId": "2d0b25ef-5348-441d-8edd-b8ec50ae929f"
      },
      "source": [
        "#Create the same tensor with tf.Variable\n",
        "changeable_tensor = tf.Variable([10,7])\n",
        "unchangeable_tensor = tf.constant([10,7])\n",
        "\n",
        "print(changeable_tensor)\n",
        "print(unchangeable_tensor)"
      ],
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "<tf.Variable 'Variable:0' shape=(2,) dtype=int32, numpy=array([10,  7], dtype=int32)>\n",
            "tf.Tensor([10  7], shape=(2,), dtype=int32)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 205
        },
        "id": "QV0MOwXaAGju",
        "outputId": "18f5ec0b-5968-4b92-87de-fbb54189f291"
      },
      "source": [
        "# Let's try change one of the elements in changeable_tensor\n",
        "changeable_tensor[0] = 7\n",
        "changeable_tensor"
      ],
      "execution_count": 13,
      "outputs": [
        {
          "output_type": "error",
          "ename": "TypeError",
          "evalue": "ignored",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mTypeError\u001b[0m                                 Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-13-448b1a23d380>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0;31m# Let's try change one of the elements in changeable_tensor\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 2\u001b[0;31m \u001b[0mchangeable_tensor\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;36m7\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      3\u001b[0m \u001b[0mchangeable_tensor\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;31mTypeError\u001b[0m: 'ResourceVariable' object does not support item assignment"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "73KHw-MlAnBC"
      },
      "source": [
        "#How about we try .assign()\n",
        "changeable_tensor[0].assign(7)\n",
        "changeable_tensor"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "S4HGQNUeAyYB"
      },
      "source": [
        "# Let's try change our unchangeable tensor\n",
        "unchangeable_tensor[0].assign(7)\n",
        "\n",
        "#tf.constant does not allow to change our values."
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "on0yVJkIA7pA"
      },
      "source": [
        "### Creating Random Tensors\n",
        "\n",
        "Random tensors are tensors of some arbitary size which contain random numbers."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "JRNEIhxVB6hK",
        "outputId": "d16a2a30-f34f-4375-b010-87414f7cae27"
      },
      "source": [
        "# Create two random (but the same) tensors\n",
        "#Random initilization is important to avoid exploding/vanishing gradients and to make sure that the neural network starts well.\n",
        "\n",
        "\n",
        "random_1 = tf.random.Generator.from_seed(42)  # Set seed for reproducibility\n",
        "random_1 = random_1.normal(shape=(3, 2))  # Uniform distribution\n",
        "random_1\n",
        "\n",
        "random_2 = tf.random.Generator.from_seed(42)\n",
        "random_2 = random_2.normal(shape=(3, 2))\n",
        "\n",
        "# Are they equal?\n",
        "\n",
        "(random_1, random_2, random_1 == random_2)"
      ],
      "execution_count": 14,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<tf.Tensor: shape=(3, 2), dtype=float32, numpy=\n",
              " array([[-0.7565803 , -0.06854702],\n",
              "        [ 0.07595026, -1.2573844 ],\n",
              "        [-0.23193763, -1.8107855 ]], dtype=float32)>,\n",
              " <tf.Tensor: shape=(3, 2), dtype=float32, numpy=\n",
              " array([[-0.7565803 , -0.06854702],\n",
              "        [ 0.07595026, -1.2573844 ],\n",
              "        [-0.23193763, -1.8107855 ]], dtype=float32)>,\n",
              " <tf.Tensor: shape=(3, 2), dtype=bool, numpy=\n",
              " array([[ True,  True],\n",
              "        [ True,  True],\n",
              "        [ True,  True]])>)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 14
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YTO1Nl0xMFB6"
      },
      "source": [
        "#Shuffle the order of elements in a tensor"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "hQZy_qDEMTov",
        "outputId": "a9390772-7dc3-4b9b-adf6-3667334608e3"
      },
      "source": [
        "# Shuffle a tensor (valuable for when you want to shuffle your data so inherent order does not affect learning)\n",
        "\n",
        "not_shuffled = tf.constant([[10, 7], [3, 5], [8, 9]])\n",
        "not_shuffled.ndim\n",
        "\n",
        "# Shuffle our non-shuffled tensor\n",
        "\n",
        "not_shuffled"
      ],
      "execution_count": 15,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 2), dtype=int32, numpy=\n",
              "array([[10,  7],\n",
              "       [ 3,  5],\n",
              "       [ 8,  9]], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 15
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "b3zaqJ_NeWMi",
        "outputId": "fcd6ecf2-edc1-4569-993f-c2c03f94271c"
      },
      "source": [
        "# Shuffle our non-shuffled tensor\n",
        "tf.random.set_seed(42) #global level random seed\n",
        "tf.random.shuffle(not_shuffled) #operation level random seed"
      ],
      "execution_count": 16,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 2), dtype=int32, numpy=\n",
              "array([[ 3,  5],\n",
              "       [ 8,  9],\n",
              "       [10,  7]], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 16
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "PbdzKa1be6OH",
        "outputId": "70207f3f-6671-43cd-aa5e-409e563bf8e8"
      },
      "source": [
        "tf.random.shuffle(not_shuffled)"
      ],
      "execution_count": 17,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 2), dtype=int32, numpy=\n",
              "array([[ 8,  9],\n",
              "       [10,  7],\n",
              "       [ 3,  5]], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 17
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kpiFnjMhgYMo"
      },
      "source": [
        "Other ways to make tensors"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "IWhhniz7iwHg",
        "outputId": "97487aad-df0c-4631-e3fd-7eb485e96b32"
      },
      "source": [
        "#Create a tensor of all ones\n",
        "tf.ones([10,7])"
      ],
      "execution_count": 18,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(10, 7), dtype=float32, numpy=\n",
              "array([[1., 1., 1., 1., 1., 1., 1.],\n",
              "       [1., 1., 1., 1., 1., 1., 1.],\n",
              "       [1., 1., 1., 1., 1., 1., 1.],\n",
              "       [1., 1., 1., 1., 1., 1., 1.],\n",
              "       [1., 1., 1., 1., 1., 1., 1.],\n",
              "       [1., 1., 1., 1., 1., 1., 1.],\n",
              "       [1., 1., 1., 1., 1., 1., 1.],\n",
              "       [1., 1., 1., 1., 1., 1., 1.],\n",
              "       [1., 1., 1., 1., 1., 1., 1.],\n",
              "       [1., 1., 1., 1., 1., 1., 1.]], dtype=float32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 18
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "iphyRMPgi6or",
        "outputId": "8a9e269b-9574-485c-b592-103113a38bbf"
      },
      "source": [
        "#Create a tensor of all zeroes\n",
        "tf.zeros(shape = (3,4))"
      ],
      "execution_count": 19,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 4), dtype=float32, numpy=\n",
              "array([[0., 0., 0., 0.],\n",
              "       [0., 0., 0., 0.],\n",
              "       [0., 0., 0., 0.]], dtype=float32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 19
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mFQNoetdjQW5"
      },
      "source": [
        "### Turn NumPy arrays into tensors\n",
        "\n",
        "The main difference between NumPy arrays and tensors is that tensors can be run on a GPU (much faster for numerical computation)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "kLd-6taSjB5w",
        "outputId": "a5fc0c74-a409-4986-e998-03d2919b43e3"
      },
      "source": [
        "# You can also convert NumPy array into tensors\n",
        "\n",
        "import numpy as np\n",
        "numpy_A = np.arange(1, 25, dtype=np.int32)  # NumPy array between 1 and 25\n",
        "numpy_A\n",
        "\n",
        "# X = tf.constant(some_matrix) #capital for matrix or vector\n",
        "# y = tf.constant(vector) # non capital for vector"
      ],
      "execution_count": 20,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([ 1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15, 16, 17,\n",
              "       18, 19, 20, 21, 22, 23, 24], dtype=int32)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 20
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "e-kHg8dPkX2s",
        "outputId": "1aa0ca04-8033-4315-ac8d-af71b8944a9d"
      },
      "source": [
        "A = tf.constant(numpy_A, shape = (2,3,4)) #shape elements should be equal\n",
        "B = tf.constant(numpy_A)\n",
        "A , B, A.ndim, B.ndim"
      ],
      "execution_count": 21,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<tf.Tensor: shape=(2, 3, 4), dtype=int32, numpy=\n",
              " array([[[ 1,  2,  3,  4],\n",
              "         [ 5,  6,  7,  8],\n",
              "         [ 9, 10, 11, 12]],\n",
              " \n",
              "        [[13, 14, 15, 16],\n",
              "         [17, 18, 19, 20],\n",
              "         [21, 22, 23, 24]]], dtype=int32)>,\n",
              " <tf.Tensor: shape=(24,), dtype=int32, numpy=\n",
              " array([ 1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15, 16, 17,\n",
              "        18, 19, 20, 21, 22, 23, 24], dtype=int32)>,\n",
              " 3,\n",
              " 1)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 21
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vxNneQcYkfQt"
      },
      "source": [
        "## Getting information from tensors\n",
        "\n",
        "When dealing with tensors you probably want to be aware of the following attributes:\n",
        "\n",
        "* Shape\n",
        "* Rank\n",
        "* Axis or Dimension\n",
        "* Size"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "v8ekouHfnR0i",
        "outputId": "08c7bb7a-19de-41b0-d996-a55f8c2fc3d1"
      },
      "source": [
        "#Create a rank 4 tensor (4 dimensions)\n",
        "rank_4_tensor = tf.zeros(shape = [2,3,4,5])\n",
        "rank_4_tensor"
      ],
      "execution_count": 22,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 3, 4, 5), dtype=float32, numpy=\n",
              "array([[[[0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.]],\n",
              "\n",
              "        [[0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.]],\n",
              "\n",
              "        [[0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.]]],\n",
              "\n",
              "\n",
              "       [[[0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.]],\n",
              "\n",
              "        [[0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.]],\n",
              "\n",
              "        [[0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.]]]], dtype=float32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 22
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "gvTLWItUoyL0",
        "outputId": "8e52df28-3fd4-4db2-8c9c-f83f48878a64"
      },
      "source": [
        "rank_4_tensor[0]"
      ],
      "execution_count": 23,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 4, 5), dtype=float32, numpy=\n",
              "array([[[0., 0., 0., 0., 0.],\n",
              "        [0., 0., 0., 0., 0.],\n",
              "        [0., 0., 0., 0., 0.],\n",
              "        [0., 0., 0., 0., 0.]],\n",
              "\n",
              "       [[0., 0., 0., 0., 0.],\n",
              "        [0., 0., 0., 0., 0.],\n",
              "        [0., 0., 0., 0., 0.],\n",
              "        [0., 0., 0., 0., 0.]],\n",
              "\n",
              "       [[0., 0., 0., 0., 0.],\n",
              "        [0., 0., 0., 0., 0.],\n",
              "        [0., 0., 0., 0., 0.],\n",
              "        [0., 0., 0., 0., 0.]]], dtype=float32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 23
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 134
        },
        "id": "cf8JFJANpBi6",
        "outputId": "4ac45c1e-d2c4-412d-891a-902023fbdc40"
      },
      "source": [
        "rank_4_tensor.shape, rank_4_tensor.#ndim,  tf.size(rank_4_tensor)"
      ],
      "execution_count": 24,
      "outputs": [
        {
          "output_type": "error",
          "ename": "SyntaxError",
          "evalue": "ignored",
          "traceback": [
            "\u001b[0;36m  File \u001b[0;32m\"<ipython-input-24-358640c4bf08>\"\u001b[0;36m, line \u001b[0;32m1\u001b[0m\n\u001b[0;31m    rank_4_tensor.shape, rank_4_tensor.#ndim,  tf.size(rank_4_tensor)\u001b[0m\n\u001b[0m                                                                     ^\u001b[0m\n\u001b[0;31mSyntaxError\u001b[0m\u001b[0;31m:\u001b[0m invalid syntax\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "z-0r_bwWpQtT",
        "outputId": "a4f61066-da66-4ed0-9067-dd7fe796d116"
      },
      "source": [
        "# Get various attibutes of our tensor\n",
        "\n",
        "print ('Datatype of every element:', rank_4_tensor.dtype)\n",
        "print ('Number of dimensions: ', rank_4_tensor.ndim)\n",
        "print ('Shape of tensor :', rank_4_tensor.shape)\n",
        "print ('Elements along the 0 axis:', rank_4_tensor.shape[0])\n",
        "print ('Elements along the last axis', rank_4_tensor.shape[-1])\n",
        "print ('Total number of elements in tensors:', tf.size(rank_4_tensor))"
      ],
      "execution_count": 25,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Datatype of every element: <dtype: 'float32'>\n",
            "Number of dimensions:  4\n",
            "Shape of tensor : (2, 3, 4, 5)\n",
            "Elements along the 0 axis: 2\n",
            "Elements along the last axis 5\n",
            "Total number of elements in tensors: tf.Tensor(120, shape=(), dtype=int32)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aqIM9xnzqC7t"
      },
      "source": [
        "Indexing Tensors\n",
        "\n",
        "Tensors can be indexed just like Python lists."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "XDuE4qYLrsN-",
        "outputId": "9a76bc65-ce65-4d55-db2d-2a059a3669ea"
      },
      "source": [
        "#Get the first two elements of each dimension\n",
        "rank_4_tensor[:2, :2, :2, :2]"
      ],
      "execution_count": 26,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2, 2, 2), dtype=float32, numpy=\n",
              "array([[[[0., 0.],\n",
              "         [0., 0.]],\n",
              "\n",
              "        [[0., 0.],\n",
              "         [0., 0.]]],\n",
              "\n",
              "\n",
              "       [[[0., 0.],\n",
              "         [0., 0.]],\n",
              "\n",
              "        [[0., 0.],\n",
              "         [0., 0.]]]], dtype=float32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 26
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "DpHQT_QlsEk9",
        "outputId": "a73afb81-36fe-49ce-861f-610a54670b78"
      },
      "source": [
        "# Get the first element from each dimension from each index except the final one\n",
        "rank_4_tensor[:1,:1,:1,:]"
      ],
      "execution_count": 27,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(1, 1, 1, 5), dtype=float32, numpy=array([[[[0., 0., 0., 0., 0.]]]], dtype=float32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 27
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "_6L2v1vOsjab",
        "outputId": "e7e00995-9ff9-469f-b5f0-f3b6eba37b18"
      },
      "source": [
        "#Create a rank 2 tensor (2 dimensions)\n",
        "rank_2_tensor = tf.constant([[10,7],\n",
        "                            [3,4]])\n",
        "rank_2_tensor.shape, rank_2_tensor.ndim"
      ],
      "execution_count": 28,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(TensorShape([2, 2]), 2)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 28
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "f9Lvm-pVtHsq",
        "outputId": "92e694fe-a735-4ce9-aa04-a7cdd533314f"
      },
      "source": [
        "# Get the last item each of our rank 2 tensor\n",
        "rank_2_tensor, rank_2_tensor[:,-1]"
      ],
      "execution_count": 29,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<tf.Tensor: shape=(2, 2), dtype=int32, numpy=\n",
              " array([[10,  7],\n",
              "        [ 3,  4]], dtype=int32)>,\n",
              " <tf.Tensor: shape=(2,), dtype=int32, numpy=array([7, 4], dtype=int32)>)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 29
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "8Kr_bU77tVZE",
        "outputId": "1a44a239-ed54-4251-ac79-b8accbd0b20b"
      },
      "source": [
        "# Add in extra dimension to our rank 2 tensor\n",
        "rank_3_tensor = rank_2_tensor[:,:, tf.newaxis]\n",
        "rank_3_tensor"
      ],
      "execution_count": 30,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2, 1), dtype=int32, numpy=\n",
              "array([[[10],\n",
              "        [ 7]],\n",
              "\n",
              "       [[ 3],\n",
              "        [ 4]]], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 30
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "pVxrX_fUuD_0",
        "outputId": "e1ffe784-1709-4861-b9f7-5a04c2ccd1a4"
      },
      "source": [
        "# Alternative to tf.newaxis\n",
        "tf.expand_dims(rank_2_tensor , axis = -1) #'-1' means last axis"
      ],
      "execution_count": 31,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2, 1), dtype=int32, numpy=\n",
              "array([[[10],\n",
              "        [ 7]],\n",
              "\n",
              "       [[ 3],\n",
              "        [ 4]]], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 31
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Z9DdcmTptthr",
        "outputId": "add610ee-98f6-40fb-c830-b3bed6ed1319"
      },
      "source": [
        "tf.expand_dims(rank_2_tensor, axis = 0) #Expand the zero axis"
      ],
      "execution_count": 32,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(1, 2, 2), dtype=int32, numpy=\n",
              "array([[[10,  7],\n",
              "        [ 3,  4]]], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 32
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "13dvE_ZduZD5",
        "outputId": "599bf06e-35cc-47b5-b971-93d4a231e475"
      },
      "source": [
        "tf.expand_dims(rank_2_tensor, axis = 1) #Expand the first axis"
      ],
      "execution_count": 33,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 1, 2), dtype=int32, numpy=\n",
              "array([[[10,  7]],\n",
              "\n",
              "       [[ 3,  4]]], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 33
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DpmO67vHuouF"
      },
      "source": [
        "Manipulating Tensor Operations\n",
        "\n",
        "* Basic Operations"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "YRp1rdbavf-G",
        "outputId": "4fffbffc-0708-444f-f858-6c7f5722bc14"
      },
      "source": [
        "# You can add values to a tensor using the addition operator\n",
        "tensor = tf.constant([[10,7],[3,4]])\n",
        "tensor + 10, tf.math.add(tensor, 10)"
      ],
      "execution_count": 34,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<tf.Tensor: shape=(2, 2), dtype=int32, numpy=\n",
              " array([[20, 17],\n",
              "        [13, 14]], dtype=int32)>, <tf.Tensor: shape=(2, 2), dtype=int32, numpy=\n",
              " array([[20, 17],\n",
              "        [13, 14]], dtype=int32)>)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 34
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "seoi8fSuvx1k",
        "outputId": "8bc67996-555e-4147-975f-e8fdddb5d77a"
      },
      "source": [
        "# Other Operations\n",
        "print(tensor * 10 , tf.multiply(tensor, 10))\n",
        "\n",
        "print(tensor / 10, tf.math.divide(tensor,10))\n",
        "\n",
        "print(tensor - 10, tf.math.subtract(tensor, 10))"
      ],
      "execution_count": 35,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "tf.Tensor(\n",
            "[[100  70]\n",
            " [ 30  40]], shape=(2, 2), dtype=int32) tf.Tensor(\n",
            "[[100  70]\n",
            " [ 30  40]], shape=(2, 2), dtype=int32)\n",
            "tf.Tensor(\n",
            "[[1.  0.7]\n",
            " [0.3 0.4]], shape=(2, 2), dtype=float64) tf.Tensor(\n",
            "[[1.  0.7]\n",
            " [0.3 0.4]], shape=(2, 2), dtype=float64)\n",
            "tf.Tensor(\n",
            "[[ 0 -3]\n",
            " [-7 -6]], shape=(2, 2), dtype=int32) tf.Tensor(\n",
            "[[ 0 -3]\n",
            " [-7 -6]], shape=(2, 2), dtype=int32)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qnrOezIUwE6u"
      },
      "source": [
        "# Matrix Multiplication\n",
        "\n",
        "In Machine Learning, matrix multiplication are the most important tensor calculations.\n",
        "\n",
        "There are two rules our tensors (or matrices) need to fulfill if we need to do matrix multiplication.\n",
        "\n",
        "1. Inner dimensions should match\n",
        "2. The resulting matrix has the shape of the outer dimension."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "utqG9jOVCXcW",
        "outputId": "e042de09-53e6-4ff2-c28d-fadace77b816"
      },
      "source": [
        "#Matrix multiplication\n",
        "print(tf.matmul(tensor,tensor)), print(np.dot(tensor,tensor))"
      ],
      "execution_count": 36,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "tf.Tensor(\n",
            "[[121  98]\n",
            " [ 42  37]], shape=(2, 2), dtype=int32)\n",
            "[[121  98]\n",
            " [ 42  37]]\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(None, None)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 36
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "KgJRllVDDBP2",
        "outputId": "71db7c8d-ce59-45ed-94ae-7780ad5d6667"
      },
      "source": [
        "# Matrix multiplication with Python operator \"@\"\n",
        "# @ is used for matrix multiplication\n",
        "tensor @ tensor"
      ],
      "execution_count": 37,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2), dtype=int32, numpy=\n",
              "array([[121,  98],\n",
              "       [ 42,  37]], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 37
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "j4XGkZTaDSgV",
        "outputId": "30643f2d-5d46-4a6a-9135-f468fb2d01fb"
      },
      "source": [
        "tensor.shape"
      ],
      "execution_count": 38,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "TensorShape([2, 2])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 38
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "RRqJArroDfg2",
        "outputId": "2a1eac51-e040-4c80-81a2-5676c37a6746"
      },
      "source": [
        "# Create a tensor of (3,2) \n",
        "X = tf.constant([[1,2,],[3,4],[5,6]])\n",
        "\n",
        "# Create another tensor of (3,2)\n",
        "Y = tf.constant([[7,8],[9,10],[11,12]])\n",
        "\n",
        "X , Y"
      ],
      "execution_count": 39,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<tf.Tensor: shape=(3, 2), dtype=int32, numpy=\n",
              " array([[1, 2],\n",
              "        [3, 4],\n",
              "        [5, 6]], dtype=int32)>, <tf.Tensor: shape=(3, 2), dtype=int32, numpy=\n",
              " array([[ 7,  8],\n",
              "        [ 9, 10],\n",
              "        [11, 12]], dtype=int32)>)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 39
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 246
        },
        "id": "-8eC7tMiDxkK",
        "outputId": "201b5542-2ab7-4ec7-8526-cb510deeee1c"
      },
      "source": [
        "# Try to Matrix multiply tensors of same shape\n",
        "X @ Y"
      ],
      "execution_count": 40,
      "outputs": [
        {
          "output_type": "error",
          "ename": "InvalidArgumentError",
          "evalue": "ignored",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mInvalidArgumentError\u001b[0m                      Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-40-e9e497d9a515>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0;31m# Try to Matrix multiply tensors of same shape\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 2\u001b[0;31m \u001b[0mX\u001b[0m \u001b[0;34m@\u001b[0m \u001b[0mY\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
            "\u001b[0;32m/usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/math_ops.py\u001b[0m in \u001b[0;36mbinary_op_wrapper\u001b[0;34m(x, y)\u001b[0m\n\u001b[1;32m   1232\u001b[0m         \u001b[0;31m#   r_binary_op_wrapper use different force_same_dtype values.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1233\u001b[0m         \u001b[0mx\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0my\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mmaybe_promote_tensors\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mx\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0my\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mforce_same_dtype\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mFalse\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1234\u001b[0;31m         \u001b[0;32mreturn\u001b[0m \u001b[0mfunc\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mx\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0my\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mname\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mname\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   1235\u001b[0m       \u001b[0;32mexcept\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0mTypeError\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mValueError\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0me\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1236\u001b[0m         \u001b[0;31m# Even if dispatching the op failed, the RHS may be a tensor aware\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/math_ops.py\u001b[0m in \u001b[0;36mmatmul_wrapper\u001b[0;34m(a, b, name)\u001b[0m\n\u001b[1;32m   3596\u001b[0m   \u001b[0;32mif\u001b[0m \u001b[0mops\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_numpy_style_type_promotion\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   3597\u001b[0m     \u001b[0;32mreturn\u001b[0m \u001b[0ma\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_matmul\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mb\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 3598\u001b[0;31m   \u001b[0;32mreturn\u001b[0m \u001b[0mmatmul\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0ma\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mb\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mname\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mname\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   3599\u001b[0m \u001b[0mmatmul_wrapper\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__doc__\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mmatmul\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__doc__\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   3600\u001b[0m \u001b[0m_OverrideBinaryOperatorHelper\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmatmul_wrapper\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m\"matmul\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.7/dist-packages/tensorflow/python/util/dispatch.py\u001b[0m in \u001b[0;36mwrapper\u001b[0;34m(*args, **kwargs)\u001b[0m\n\u001b[1;32m    204\u001b[0m     \u001b[0;34m\"\"\"Call target, and fall back on dispatchers if there is a TypeError.\"\"\"\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    205\u001b[0m     \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 206\u001b[0;31m       \u001b[0;32mreturn\u001b[0m \u001b[0mtarget\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    207\u001b[0m     \u001b[0;32mexcept\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0mTypeError\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mValueError\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    208\u001b[0m       \u001b[0;31m# Note: convert_to_eager_tensor currently raises a ValueError, not a\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/math_ops.py\u001b[0m in \u001b[0;36mmatmul\u001b[0;34m(a, b, transpose_a, transpose_b, adjoint_a, adjoint_b, a_is_sparse, b_is_sparse, name)\u001b[0m\n\u001b[1;32m   3488\u001b[0m     \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   3489\u001b[0m       return gen_math_ops.mat_mul(\n\u001b[0;32m-> 3490\u001b[0;31m           a, b, transpose_a=transpose_a, transpose_b=transpose_b, name=name)\n\u001b[0m\u001b[1;32m   3491\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   3492\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/gen_math_ops.py\u001b[0m in \u001b[0;36mmat_mul\u001b[0;34m(a, b, transpose_a, transpose_b, name)\u001b[0m\n\u001b[1;32m   5698\u001b[0m       \u001b[0;32mreturn\u001b[0m \u001b[0m_result\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   5699\u001b[0m     \u001b[0;32mexcept\u001b[0m \u001b[0m_core\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_NotOkStatusException\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0me\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 5700\u001b[0;31m       \u001b[0m_ops\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mraise_from_not_ok_status\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0me\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mname\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   5701\u001b[0m     \u001b[0;32mexcept\u001b[0m \u001b[0m_core\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_FallbackException\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   5702\u001b[0m       \u001b[0;32mpass\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py\u001b[0m in \u001b[0;36mraise_from_not_ok_status\u001b[0;34m(e, name)\u001b[0m\n\u001b[1;32m   6895\u001b[0m   \u001b[0mmessage\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0me\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmessage\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0;34m\" name: \"\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0mname\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mname\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32mNone\u001b[0m \u001b[0;32melse\u001b[0m \u001b[0;34m\"\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   6896\u001b[0m   \u001b[0;31m# pylint: disable=protected-access\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 6897\u001b[0;31m   \u001b[0msix\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mraise_from\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcore\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_status_to_exception\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0me\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcode\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmessage\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   6898\u001b[0m   \u001b[0;31m# pylint: enable=protected-access\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   6899\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.7/dist-packages/six.py\u001b[0m in \u001b[0;36mraise_from\u001b[0;34m(value, from_value)\u001b[0m\n",
            "\u001b[0;31mInvalidArgumentError\u001b[0m: In[0] mismatch In[1] shape: 2 vs. 3: [3,2] [3,2] 0 0 [Op:MatMul]"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 229
        },
        "id": "D3mhspMBEBHn",
        "outputId": "268087ea-7859-4444-9da8-cdb9e4927ea9"
      },
      "source": [
        "#Try to multiply tensors of same shape\n",
        "tf.matmul(X,Y)"
      ],
      "execution_count": 41,
      "outputs": [
        {
          "output_type": "error",
          "ename": "InvalidArgumentError",
          "evalue": "ignored",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mInvalidArgumentError\u001b[0m                      Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-41-cc348e6d8216>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mtf\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmatmul\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mX\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0mY\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
            "\u001b[0;32m/usr/local/lib/python3.7/dist-packages/tensorflow/python/util/dispatch.py\u001b[0m in \u001b[0;36mwrapper\u001b[0;34m(*args, **kwargs)\u001b[0m\n\u001b[1;32m    204\u001b[0m     \u001b[0;34m\"\"\"Call target, and fall back on dispatchers if there is a TypeError.\"\"\"\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    205\u001b[0m     \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 206\u001b[0;31m       \u001b[0;32mreturn\u001b[0m \u001b[0mtarget\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    207\u001b[0m     \u001b[0;32mexcept\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0mTypeError\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mValueError\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    208\u001b[0m       \u001b[0;31m# Note: convert_to_eager_tensor currently raises a ValueError, not a\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/math_ops.py\u001b[0m in \u001b[0;36mmatmul\u001b[0;34m(a, b, transpose_a, transpose_b, adjoint_a, adjoint_b, a_is_sparse, b_is_sparse, name)\u001b[0m\n\u001b[1;32m   3488\u001b[0m     \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   3489\u001b[0m       return gen_math_ops.mat_mul(\n\u001b[0;32m-> 3490\u001b[0;31m           a, b, transpose_a=transpose_a, transpose_b=transpose_b, name=name)\n\u001b[0m\u001b[1;32m   3491\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   3492\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/gen_math_ops.py\u001b[0m in \u001b[0;36mmat_mul\u001b[0;34m(a, b, transpose_a, transpose_b, name)\u001b[0m\n\u001b[1;32m   5698\u001b[0m       \u001b[0;32mreturn\u001b[0m \u001b[0m_result\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   5699\u001b[0m     \u001b[0;32mexcept\u001b[0m \u001b[0m_core\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_NotOkStatusException\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0me\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 5700\u001b[0;31m       \u001b[0m_ops\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mraise_from_not_ok_status\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0me\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mname\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   5701\u001b[0m     \u001b[0;32mexcept\u001b[0m \u001b[0m_core\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_FallbackException\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   5702\u001b[0m       \u001b[0;32mpass\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py\u001b[0m in \u001b[0;36mraise_from_not_ok_status\u001b[0;34m(e, name)\u001b[0m\n\u001b[1;32m   6895\u001b[0m   \u001b[0mmessage\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0me\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmessage\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0;34m\" name: \"\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0mname\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mname\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32mNone\u001b[0m \u001b[0;32melse\u001b[0m \u001b[0;34m\"\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   6896\u001b[0m   \u001b[0;31m# pylint: disable=protected-access\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 6897\u001b[0;31m   \u001b[0msix\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mraise_from\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcore\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_status_to_exception\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0me\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcode\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmessage\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   6898\u001b[0m   \u001b[0;31m# pylint: enable=protected-access\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   6899\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.7/dist-packages/six.py\u001b[0m in \u001b[0;36mraise_from\u001b[0;34m(value, from_value)\u001b[0m\n",
            "\u001b[0;31mInvalidArgumentError\u001b[0m: In[0] mismatch In[1] shape: 2 vs. 3: [3,2] [3,2] 0 0 [Op:MatMul]"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "RCd0Dhn_EDct",
        "outputId": "69e03630-ca93-4cbd-fa3d-8c1133798c73"
      },
      "source": [
        "# Let us change the shape of y\n",
        "tf.reshape(Y, shape = (2,3))"
      ],
      "execution_count": 43,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 3), dtype=int32, numpy=\n",
              "array([[ 7,  8,  9],\n",
              "       [10, 11, 12]], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 43
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "m_dONhuKk0Tr",
        "outputId": "308c5bd4-8f43-4225-8bf0-2526134d08e6"
      },
      "source": [
        "# Try to multiply X by reshaped Y\n",
        "X @tf.reshape(Y, shape = (2,3))"
      ],
      "execution_count": 44,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 3), dtype=int32, numpy=\n",
              "array([[ 27,  30,  33],\n",
              "       [ 61,  68,  75],\n",
              "       [ 95, 106, 117]], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 44
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "K4iSnXdulcHi",
        "outputId": "0a640cc2-9ad0-46ff-fa8c-843e859368af"
      },
      "source": [
        "X.shape , tf.reshape(Y,shape = (2,3)).shape"
      ],
      "execution_count": 45,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(TensorShape([3, 2]), TensorShape([2, 3]))"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 45
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "dG7x9KkJlmKp",
        "outputId": "7e9b8a8c-ae6f-4093-ac3b-f0481304ddcf"
      },
      "source": [
        "tf.matmul(X, tf.reshape(Y, shape = (2,3)))"
      ],
      "execution_count": 46,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 3), dtype=int32, numpy=\n",
              "array([[ 27,  30,  33],\n",
              "       [ 61,  68,  75],\n",
              "       [ 95, 106, 117]], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 46
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "wP4rpejkl8sT",
        "outputId": "23b5e26e-a135-4951-cf34-5f849b649262"
      },
      "source": [
        "tf.reshape(X, shape = (2,3))"
      ],
      "execution_count": 49,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 3), dtype=int32, numpy=\n",
              "array([[1, 2, 3],\n",
              "       [4, 5, 6]], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 49
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "31keWFmElsK1",
        "outputId": "19e69be6-fd82-49a7-d8b0-e088ce639718"
      },
      "source": [
        "# Let us change the shape of X\n",
        "tf.matmul(Y, tf.reshape(X, shape = (2,3)))"
      ],
      "execution_count": 48,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 3), dtype=int32, numpy=\n",
              "array([[ 39,  54,  69],\n",
              "       [ 49,  68,  87],\n",
              "       [ 59,  82, 105]], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 48
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "nL9YR2h9mIqM",
        "outputId": "6a4c4d4a-37d7-41b4-dd0e-646397b73c83"
      },
      "source": [
        "# Can do the same with transpose\n",
        "X , tf.transpose(X), tf.reshape(X, shape = (2,3))"
      ],
      "execution_count": 51,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<tf.Tensor: shape=(3, 2), dtype=int32, numpy=\n",
              " array([[1, 2],\n",
              "        [3, 4],\n",
              "        [5, 6]], dtype=int32)>, <tf.Tensor: shape=(2, 3), dtype=int32, numpy=\n",
              " array([[1, 3, 5],\n",
              "        [2, 4, 6]], dtype=int32)>, <tf.Tensor: shape=(2, 3), dtype=int32, numpy=\n",
              " array([[1, 2, 3],\n",
              "        [4, 5, 6]], dtype=int32)>)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 51
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "wZApAOtYmgR4",
        "outputId": "41f3e48c-b459-4d8f-9625-bd73a8e4ddfc"
      },
      "source": [
        "# Try matrix multiplication with transpose rather than reshape\n",
        "tf.matmul(tf.transpose (X), Y)"
      ],
      "execution_count": 53,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2), dtype=int32, numpy=\n",
              "array([[ 89,  98],\n",
              "       [116, 128]], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 53
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7US_jE2zm_QO"
      },
      "source": [
        "##The dot product\n",
        "\n",
        "Matrix multiplication is also referred as the dot product.\n",
        "\n",
        "You can perform matrix multiplication using:\n",
        "  tf.matmul()\n",
        "  tf.tensordot()"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Idj6n1fwp05c",
        "outputId": "541c90ca-1a98-4e86-efc5-d627b9746761"
      },
      "source": [
        "#Perform the dot product on X and Y (requires X or Y to be transposed)\n",
        "\n",
        "tf.tensordot(tf.transpose(X), Y, axes = 1)"
      ],
      "execution_count": 55,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2), dtype=int32, numpy=\n",
              "array([[ 89,  98],\n",
              "       [116, 128]], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 55
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ckjMRBraqVkY",
        "outputId": "9c092474-194c-426f-91fb-8d2f92488308"
      },
      "source": [
        "# Perform matrix multiplication between X and y (transposed)\n",
        "tf.matmul(X, tf.transpose(Y))"
      ],
      "execution_count": 56,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 3), dtype=int32, numpy=\n",
              "array([[ 23,  29,  35],\n",
              "       [ 53,  67,  81],\n",
              "       [ 83, 105, 127]], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 56
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "S1aaZjl4quTd",
        "outputId": "84ddbe4c-06a0-467c-86e7-7abd76bb8f70"
      },
      "source": [
        "# Perform matrix multiplication between X and Y (reshaped)\n",
        "tf.matmul(X, tf.reshape(Y, shape = (2,3)))"
      ],
      "execution_count": 59,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 3), dtype=int32, numpy=\n",
              "array([[ 27,  30,  33],\n",
              "       [ 61,  68,  75],\n",
              "       [ 95, 106, 117]], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 59
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "beLQJr1Yq4u1",
        "outputId": "814a44e3-b183-4982-cf81-4c381ac340c4"
      },
      "source": [
        "# Check the values of Y, reshape Y and transposed Y\n",
        "\n",
        "print(\"Normal Y:\")\n",
        "print(Y, \"\\n\") # \"\\n is for newline\"\n",
        "\n",
        "print(\"Reshaped Y:\")\n",
        "print(tf.reshape(Y, (2,3)), \"\\n\")\n",
        "\n",
        "print(\"Transposed Y:\")\n",
        "print(tf.transpose(Y), \"\\n\")"
      ],
      "execution_count": 65,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Normal Y:\n",
            "tf.Tensor(\n",
            "[[ 7  8]\n",
            " [ 9 10]\n",
            " [11 12]], shape=(3, 2), dtype=int32) \n",
            "\n",
            "Reshaped Y:\n",
            "tf.Tensor(\n",
            "[[ 7  8  9]\n",
            " [10 11 12]], shape=(2, 3), dtype=int32) \n",
            "\n",
            "Transposed Y:\n",
            "tf.Tensor(\n",
            "[[ 7  9 11]\n",
            " [ 8 10 12]], shape=(2, 3), dtype=int32) \n",
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xB2fjMXhrSxt"
      },
      "source": [
        "### Generally, when perfoming matrix multiplication on two different tensors of different shapes, we would transpose one or the other tensor rather than reshaping to satisfy matrix multiplication rules."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PWwAZ8-YsNxm"
      },
      "source": [
        "## Change the datatype of a tensor"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "jf5DfrhVs0QQ",
        "outputId": "7a1e57d9-f629-4d59-ec61-a779286777e1"
      },
      "source": [
        "# Create a new tensor with default datatype (float 32)\n",
        "B = tf.constant([1.7,7.4])\n",
        "B , B.dtype"
      ],
      "execution_count": 71,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<tf.Tensor: shape=(2,), dtype=float32, numpy=array([1.7, 7.4], dtype=float32)>,\n",
              " tf.float32)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 71
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "4qHnoh_ZtL8n",
        "outputId": "694a12db-d1d4-46ff-feca-b3dc6d69238a"
      },
      "source": [
        "C = tf.constant([7,10])\n",
        "C , C.dtype"
      ],
      "execution_count": 76,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<tf.Tensor: shape=(2,), dtype=int32, numpy=array([ 7, 10], dtype=int32)>,\n",
              " tf.int32)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 76
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "dT6oJVnZtQae",
        "outputId": "29bdefcd-cece-4934-fb2c-a28f9f3f4fc8"
      },
      "source": [
        "# Change from float 32 to float 16 (reduced precision)\n",
        "D = tf.cast(B, dtype = tf.float16)\n",
        "D, D.dtype"
      ],
      "execution_count": 73,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<tf.Tensor: shape=(2,), dtype=float16, numpy=array([1.7, 7.4], dtype=float16)>,\n",
              " tf.float16)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 73
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "TQv9V6KeuCTI",
        "outputId": "7c02f07a-5487-4637-f301-cbba482de19b"
      },
      "source": [
        "# Change from int32 to float32\n",
        "E = tf.cast(C, dtype = tf.float32)\n",
        "E, E.dtype"
      ],
      "execution_count": 75,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<tf.Tensor: shape=(2,), dtype=float32, numpy=array([ 7., 10.], dtype=float32)>,\n",
              " tf.float32)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 75
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "R2Mu_i5-uf7a",
        "outputId": "495bc299-e1cb-462d-ed50-9264f820894a"
      },
      "source": [
        "E_float = tf.cast(E, dtype = tf.float16)\n",
        "E_float"
      ],
      "execution_count": 78,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2,), dtype=float16, numpy=array([ 7., 10.], dtype=float16)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 78
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "iF06iKgnuprA"
      },
      "source": [
        "## Aggregating tensors\n",
        "\n",
        "Aggregating tensors = condensing them from multiple values down to smaller amount of values."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "eUGmb0J2uwgG",
        "outputId": "4c75f22b-01b8-4abd-96bb-04bef8178a82"
      },
      "source": [
        "# Get the absolute values\n",
        "D = tf.constant([-7,-10])\n",
        "D"
      ],
      "execution_count": 79,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2,), dtype=int32, numpy=array([ -7, -10], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 79
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "YmVfIBI9xkIA",
        "outputId": "e98fe911-fe1c-404b-eb30-0f6d963f6638"
      },
      "source": [
        "# Get the absolute values\n",
        "tf.abs(D)"
      ],
      "execution_count": 80,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2,), dtype=int32, numpy=array([ 7, 10], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 80
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2_brKX19xocH"
      },
      "source": [
        "## Let us go through the following forms of aggregation \n",
        "\n",
        "* Minimum\n",
        "* Maximum\n",
        "* Mean\n",
        "* Sum"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "6op0AXnzx6E8",
        "outputId": "52387cf5-1b41-4a23-fc68-20299c2953f9"
      },
      "source": [
        "# Creating a random tensor with values between 0 and 100 of size 50\n",
        "E = tf.constant(np.random.randint(0,100, size = 50))\n",
        "E"
      ],
      "execution_count": 83,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(50,), dtype=int64, numpy=\n",
              "array([50, 86, 13, 32, 47, 87, 86, 12, 62, 37,  1, 19, 78, 36, 98, 31, 58,\n",
              "       80, 62, 89, 30,  4, 71, 97, 51, 16, 98, 94, 86, 24, 56, 24, 42, 17,\n",
              "       79, 66, 36, 36, 34, 28,  1,  1, 23, 26, 48, 60, 61, 10, 40, 56])>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 83
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "vcdQ1Kn3x_F2",
        "outputId": "90bdb838-f0e8-49f6-a2dd-b20e658a5fa6"
      },
      "source": [
        "tf.size(E), E.shape, E.ndim"
      ],
      "execution_count": 84,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<tf.Tensor: shape=(), dtype=int32, numpy=50>, TensorShape([50]), 1)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 84
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "z51a-t0lyQ7O",
        "outputId": "419bc1e5-a773-43f8-def8-5e209c4fcf5b"
      },
      "source": [
        "#Find the minimum\n",
        "tf.reduce_min(E)"
      ],
      "execution_count": 85,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=int64, numpy=1>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 85
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "E-XVtYQVyWXo",
        "outputId": "89d92007-edcb-4676-a477-308b5d559c6d"
      },
      "source": [
        "#Find the maximum\n",
        "tf.reduce_max(E)"
      ],
      "execution_count": 86,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=int64, numpy=98>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 86
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "stZ38GOayflV",
        "outputId": "c8f088db-b31f-4e84-8b58-74f8c918d871"
      },
      "source": [
        "#Find the mean\n",
        "tf.reduce_mean(E)"
      ],
      "execution_count": 87,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=int64, numpy=47>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 87
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "hWVlYfaMyoN8",
        "outputId": "b0b6a9cd-b9fc-4ccf-a47f-aea28cce84fb"
      },
      "source": [
        "#Find the sum\n",
        "tf.reduce_sum(E)"
      ],
      "execution_count": 88,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=int64, numpy=2379>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 88
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "GbLzajs6yqyb",
        "outputId": "e3cafbd8-c960-46d8-aaf4-022cd3fbe599"
      },
      "source": [
        "# Find the Standard Deviation\n",
        "# The input tensor must be in real or complex type. So, we will just convert our data to float\n",
        "\n",
        "E = tf.cast(E, dtype=tf.float16)\n",
        "print(\"The Standard Deviation of E is :\", tf.math.reduce_std(E))  # Standard Deviation\n",
        "\n",
        "# To convert Standard deviation into variance, we will have to square the value.\n",
        "print(\"The Variance of E is :\", tf.math.square(tf.math.reduce_std(E)))  # Variance\n",
        "\n",
        "print(\"The Variance of E is :\", tf.math.reduce_variance(E)) # Variance, the standard way!\n",
        "\n",
        "# Importing Tensorflow probability module\n",
        "import tensorflow_probability as tfp\n",
        "\n",
        "print(\n",
        "    \"The Variance of E using Tensorflow probability is :\", tfp.stats.variance(E)\n",
        ")  # Variance\n"
      ],
      "execution_count": 109,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "The Standard Deviation of E is : tf.Tensor(28.8, shape=(), dtype=float16)\n",
            "The Variance of E is : tf.Tensor(829.5, shape=(), dtype=float16)\n",
            "The Variance of E is : tf.Tensor(829.5, shape=(), dtype=float16)\n",
            "The Variance of E using Tensorflow probability is : tf.Tensor(829.5, shape=(), dtype=float16)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cIs3MDKvzArE"
      },
      "source": [
        "## Find the positional minimum and maximum"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "HtnTBl7w-sVk",
        "outputId": "8475009f-fea3-4cea-a51c-a56fd42637c3"
      },
      "source": [
        "# Create a new tensor for finding positional minimum and maximum.\n",
        "tf.random.set_seed(42)\n",
        "F = tf.random.uniform(shape = [50])\n",
        "F"
      ],
      "execution_count": 111,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(50,), dtype=float32, numpy=\n",
              "array([0.6645621 , 0.44100678, 0.3528825 , 0.46448255, 0.03366041,\n",
              "       0.68467236, 0.74011743, 0.8724445 , 0.22632635, 0.22319686,\n",
              "       0.3103881 , 0.7223358 , 0.13318717, 0.5480639 , 0.5746088 ,\n",
              "       0.8996835 , 0.00946367, 0.5212307 , 0.6345445 , 0.1993283 ,\n",
              "       0.72942245, 0.54583454, 0.10756552, 0.6767061 , 0.6602763 ,\n",
              "       0.33695042, 0.60141766, 0.21062577, 0.8527372 , 0.44062173,\n",
              "       0.9485276 , 0.23752594, 0.81179297, 0.5263394 , 0.494308  ,\n",
              "       0.21612847, 0.8457197 , 0.8718841 , 0.3083862 , 0.6868038 ,\n",
              "       0.23764038, 0.7817228 , 0.9671384 , 0.06870162, 0.79873943,\n",
              "       0.66028714, 0.5871513 , 0.16461694, 0.7381023 , 0.32054043],\n",
              "      dtype=float32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 111
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "wwML_MH_BDCs",
        "outputId": "aaef08a7-3caa-4c14-a5f4-808bb7e55160"
      },
      "source": [
        "#Find the positional maximum\n",
        "tf.argmax(F)"
      ],
      "execution_count": 116,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=int64, numpy=42>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 116
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "fWkZXVyLBIay",
        "outputId": "46cb4a53-6d41-4a0d-f244-60f7d190ae54"
      },
      "source": [
        "#Index on our largest value position\n",
        "F[tf.argmax(F)]"
      ],
      "execution_count": 117,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=float32, numpy=0.9671384>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 117
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "TIFZVwddBbLr",
        "outputId": "dcfb11cd-fe78-45bd-f765-742e93b9512b"
      },
      "source": [
        "#Find the max value of F\n",
        "tf.reduce_max(F)"
      ],
      "execution_count": 118,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=float32, numpy=0.9671384>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 118
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "cPR9_9OUBfl4",
        "outputId": "9b107d03-5b4c-48d5-a664-cec167588d0d"
      },
      "source": [
        "#Check for equality\n",
        "F[tf.argmax(F)] == tf.reduce_max(F)"
      ],
      "execution_count": 121,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=bool, numpy=True>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 121
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Sp07EjTTB5ZS",
        "outputId": "ae6c0e1c-3817-475d-ba9d-a4d6c73e2cfa"
      },
      "source": [
        "# Find the min value of F\n",
        "tf.reduce_min(F)"
      ],
      "execution_count": 123,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=float32, numpy=0.009463668>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 123
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "SIiRWNZaBnaY",
        "outputId": "422487ab-b426-46e4-ddc1-3e3da8ed79dd"
      },
      "source": [
        "#Index on our least value position\n",
        "F[tf.argmin(F)] == tf.reduce_min(F)"
      ],
      "execution_count": 122,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=bool, numpy=True>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 122
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dN5o2iikB3fs"
      },
      "source": [
        "## Squeezing a tensor (REMOVING ALL SINGLE DIMENSIONS)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "X8fjcIKiDRy1",
        "outputId": "89f624e4-fe8f-4597-9c42-cbff623e0d83"
      },
      "source": [
        "# Create a tensor to get started\n",
        "tf.random.set_seed(42)\n",
        "G = tf.constant(tf.random.uniform(shape = [50]), shape = (1,1,1,1,50))\n",
        "G"
      ],
      "execution_count": 127,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(1, 1, 1, 1, 50), dtype=float32, numpy=\n",
              "array([[[[[0.6645621 , 0.44100678, 0.3528825 , 0.46448255, 0.03366041,\n",
              "           0.68467236, 0.74011743, 0.8724445 , 0.22632635, 0.22319686,\n",
              "           0.3103881 , 0.7223358 , 0.13318717, 0.5480639 , 0.5746088 ,\n",
              "           0.8996835 , 0.00946367, 0.5212307 , 0.6345445 , 0.1993283 ,\n",
              "           0.72942245, 0.54583454, 0.10756552, 0.6767061 , 0.6602763 ,\n",
              "           0.33695042, 0.60141766, 0.21062577, 0.8527372 , 0.44062173,\n",
              "           0.9485276 , 0.23752594, 0.81179297, 0.5263394 , 0.494308  ,\n",
              "           0.21612847, 0.8457197 , 0.8718841 , 0.3083862 , 0.6868038 ,\n",
              "           0.23764038, 0.7817228 , 0.9671384 , 0.06870162, 0.79873943,\n",
              "           0.66028714, 0.5871513 , 0.16461694, 0.7381023 , 0.32054043]]]]],\n",
              "      dtype=float32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 127
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "v8RR25iKDl8b",
        "outputId": "c13a77a1-de29-4dfe-bd67-4e6874369de6"
      },
      "source": [
        "G.shape"
      ],
      "execution_count": 128,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "TensorShape([1, 1, 1, 1, 50])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 128
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "JGdYkAYTDyoB",
        "outputId": "69c97154-63d2-441a-cbf2-684575e1dcc0"
      },
      "source": [
        "#Removes dimensions of size 1 from the shape of a tensor\n",
        "G_squeezed = tf.squeeze(G)\n",
        "G_squeezed, G_squeezed.shape"
      ],
      "execution_count": 129,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<tf.Tensor: shape=(50,), dtype=float32, numpy=\n",
              " array([0.6645621 , 0.44100678, 0.3528825 , 0.46448255, 0.03366041,\n",
              "        0.68467236, 0.74011743, 0.8724445 , 0.22632635, 0.22319686,\n",
              "        0.3103881 , 0.7223358 , 0.13318717, 0.5480639 , 0.5746088 ,\n",
              "        0.8996835 , 0.00946367, 0.5212307 , 0.6345445 , 0.1993283 ,\n",
              "        0.72942245, 0.54583454, 0.10756552, 0.6767061 , 0.6602763 ,\n",
              "        0.33695042, 0.60141766, 0.21062577, 0.8527372 , 0.44062173,\n",
              "        0.9485276 , 0.23752594, 0.81179297, 0.5263394 , 0.494308  ,\n",
              "        0.21612847, 0.8457197 , 0.8718841 , 0.3083862 , 0.6868038 ,\n",
              "        0.23764038, 0.7817228 , 0.9671384 , 0.06870162, 0.79873943,\n",
              "        0.66028714, 0.5871513 , 0.16461694, 0.7381023 , 0.32054043],\n",
              "       dtype=float32)>, TensorShape([50]))"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 129
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "J55nUBj-D400"
      },
      "source": [
        "## One-hot encoding tensors"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "K8gtuSfDEJFk",
        "outputId": "d59f5ad7-3164-4acc-9a0f-8bd03a782e0a"
      },
      "source": [
        "# Create a list of indices\n",
        "some_list = [0,1,2,3]  #could be red,green,blue,purple\n",
        "\n",
        "#One hot encode our list of indices\n",
        "tf.one_hot(some_list, depth = 4)"
      ],
      "execution_count": 130,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(4, 4), dtype=float32, numpy=\n",
              "array([[1., 0., 0., 0.],\n",
              "       [0., 1., 0., 0.],\n",
              "       [0., 0., 1., 0.],\n",
              "       [0., 0., 0., 1.]], dtype=float32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 130
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "37mGPaaFGdxK",
        "outputId": "954a1e82-fe23-4740-cbb7-49e4e77411f0"
      },
      "source": [
        "# Specify custom values for one hot encoding\n",
        "tf.one_hot(some_list, depth = 4, on_value = \"I love Deep learning\", off_value =\"I love Tensorflow\")"
      ],
      "execution_count": 132,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(4, 4), dtype=string, numpy=\n",
              "array([[b'I love Deep learning', b'I love Tensorflow',\n",
              "        b'I love Tensorflow', b'I love Tensorflow'],\n",
              "       [b'I love Tensorflow', b'I love Deep learning',\n",
              "        b'I love Tensorflow', b'I love Tensorflow'],\n",
              "       [b'I love Tensorflow', b'I love Tensorflow',\n",
              "        b'I love Deep learning', b'I love Tensorflow'],\n",
              "       [b'I love Tensorflow', b'I love Tensorflow', b'I love Tensorflow',\n",
              "        b'I love Deep learning']], dtype=object)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 132
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yiIQhF1OG4hW"
      },
      "source": [
        "## More Math Functions "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "zUq00DOQHxzm"
      },
      "source": [
        "## Squaring, log, square root"
      ],
      "execution_count": 133,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "LxMVsSSBH1Wi",
        "outputId": "16656b09-06c8-47b0-9a59-b9985d3fed7e"
      },
      "source": [
        "# Create a new tensor\n",
        "H = tf.range(1,10)\n",
        "H"
      ],
      "execution_count": 134,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(9,), dtype=int32, numpy=array([1, 2, 3, 4, 5, 6, 7, 8, 9], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 134
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "5X83cSwSH5x-",
        "outputId": "1901d8fb-7d59-4e58-a862-7c4dba603d24"
      },
      "source": [
        "#Square it\n",
        "tf.square(H)"
      ],
      "execution_count": 135,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(9,), dtype=int32, numpy=array([ 1,  4,  9, 16, 25, 36, 49, 64, 81], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 135
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "6-Z-LyPWH9Lc",
        "outputId": "37f659b9-9c0f-47ba-d08c-95b14f198c8a"
      },
      "source": [
        "# Square root\n",
        "tf.sqrt(tf.cast(H, dtype = tf.float32))"
      ],
      "execution_count": 143,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(9,), dtype=float32, numpy=\n",
              "array([0.99999994, 1.4142134 , 1.7320508 , 1.9999999 , 2.236068  ,\n",
              "       2.4494896 , 2.6457512 , 2.8284268 , 3.        ], dtype=float32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 143
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "LkNdM6FCH-Mk",
        "outputId": "c13885ea-418f-4c43-9761-2b78cb77e9db"
      },
      "source": [
        "tf.math.log(tf.cast(H, dtype = tf.float32))"
      ],
      "execution_count": 149,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(9,), dtype=float32, numpy=\n",
              "array([0.       , 0.6931472, 1.0986123, 1.3862944, 1.609438 , 1.7917595,\n",
              "       1.9459102, 2.0794415, 2.1972246], dtype=float32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 149
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WkCcKZNEIokT"
      },
      "source": [
        "### Tensors and NumPy\n",
        "\n",
        "TensorFlow interacts beautifully with NumPy arrays."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "MziBAlzVSllQ",
        "outputId": "8c1228e9-850c-4c4b-b2f3-41c3545eefba"
      },
      "source": [
        "#Create a tensor directly from a NumPy array\n",
        "J = tf.constant(np.array([3.,7.,10.]))\n",
        "J"
      ],
      "execution_count": 151,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3,), dtype=float64, numpy=array([ 3.,  7., 10.])>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 151
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "DMPEmm1uSuxj",
        "outputId": "87e28d53-b16c-40c4-8789-1b64ba71e78a"
      },
      "source": [
        "#Convert our tensor back to a NumPy array\n",
        "np.array(J), type(np.array(J))"
      ],
      "execution_count": 152,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(array([ 3.,  7., 10.]), numpy.ndarray)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 152
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "J1v8pPEMS2ti",
        "outputId": "788f4d38-1370-46d2-dc12-af68e9960a26"
      },
      "source": [
        "J = tf.constant([3.])\n",
        "J.numpy(), type(J.numpy())"
      ],
      "execution_count": 153,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(array([3.], dtype=float32), numpy.ndarray)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 153
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "HZmo2UOlTFCS",
        "outputId": "e0236e4f-7d9a-4906-ec0b-d187c902c24e"
      },
      "source": [
        "# The default types of each are slighly different\n",
        "numpy_J = tf.constant(np.array([[10.0, 11], [12, 13]]))\n",
        "tensor_J = tf.constant([[10.0, 11], [12, 13]])\n",
        "\n",
        "numpy_J.dtype, tensor_J.dtype"
      ],
      "execution_count": 161,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(tf.float64, tf.float32)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 161
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ZyfqdCJcTlPa",
        "outputId": "db15d4d9-06f8-4510-cc90-ef7df4177368"
      },
      "source": [
        "#Physical devices available\n",
        "tf.config.list_physical_devices() #We are running on a CPU right now."
      ],
      "execution_count": 165,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[PhysicalDevice(name='/physical_device:CPU:0', device_type='CPU')]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 165
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "cPFnxPXEtZxW",
        "outputId": "df7cdd4b-a82a-491b-f177-91a00d474159"
      },
      "source": [
        "#Checking access to GPU\n",
        "import tensorflow as tf\n",
        "tf.config.list_physical_devices()"
      ],
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[PhysicalDevice(name='/physical_device:CPU:0', device_type='CPU'),\n",
              " PhysicalDevice(name='/physical_device:GPU:0', device_type='GPU')]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 1
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "M4rvb5zyuKHG",
        "outputId": "0b3305cf-1d6d-41be-b5ec-ab6f3dfd6677"
      },
      "source": [
        "!nvidia-smi"
      ],
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Sun Jul 11 17:58:53 2021       \n",
            "+-----------------------------------------------------------------------------+\n",
            "| NVIDIA-SMI 470.42.01    Driver Version: 460.32.03    CUDA Version: 11.2     |\n",
            "|-------------------------------+----------------------+----------------------+\n",
            "| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |\n",
            "| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |\n",
            "|                               |                      |               MIG M. |\n",
            "|===============================+======================+======================|\n",
            "|   0  Tesla T4            Off  | 00000000:00:04.0 Off |                    0 |\n",
            "| N/A   35C    P8     9W /  70W |      3MiB / 15109MiB |      0%      Default |\n",
            "|                               |                      |                  N/A |\n",
            "+-------------------------------+----------------------+----------------------+\n",
            "                                                                               \n",
            "+-----------------------------------------------------------------------------+\n",
            "| Processes:                                                                  |\n",
            "|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |\n",
            "|        ID   ID                                                   Usage      |\n",
            "|=============================================================================|\n",
            "|  No running processes found                                                 |\n",
            "+-----------------------------------------------------------------------------+\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "j8obX96CuNEf"
      },
      "source": [
        "### Note : If you have access to a CUDA enabled GPU, TensorFlow would automatically use it whenever possible."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "DQd7utY9urPU"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    }
  ]
}