linear_regression_low_level.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This notebook demonstrates linear regression in TensorFlow, and visualizes the graph and summary data for the variables in TensorBoard. Uncomment code where instructed to learn more as you go along."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# The next three imports help with compatability between\n",
    "# Python 2 and 3\n",
    "from __future__ import absolute_import\n",
    "from __future__ import division\n",
    "from __future__ import print_function\n",
    "\n",
    "import os\n",
    "\n",
    "import numpy as np\n",
    "import pylab\n",
    "import tensorflow as tf\n",
    "\n",
    "# A special command for IPython Notebooks that\n",
    "# intructs Matplotlib to display plots in the notebook\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# This is a directory we'll use to store information\n",
    "# about the graph to later visualize in TensorBoard.\n",
    "# By default, it will be created in the same directory\n",
    "# as this notebook. \n",
    "\n",
    "# Be sure to delete the contents of this directory before\n",
    "# running the script.\n",
    "LOGDIR = './graphs'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "tf.reset_default_graph()\n",
    "sess = tf.Session()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This function will create a noisy dataset that's roughly linear, according to the equation *y = mx + b + noise*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def make_noisy_data(m=0.1, b=0.3, n=100):\n",
    "    x = np.random.rand(n).astype(np.float32)\n",
    "    noise = np.random.normal(scale=0.01, size=len(x))\n",
    "    y = m * x + b + noise\n",
    "    return x, y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Step 1) Create training and testing data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "x_train, y_train = make_noisy_data()\n",
    "x_test, y_test = make_noisy_data()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Plot our training and testing data "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Uncomment the following lines to plot the data.\n",
    "# pylab.plot(x_train, y_train, 'b.')\n",
    "# pylab.plot(x_test, y_test, 'g.')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Step 2) Define placeholders for data we'll later input to the graph."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# tf.name_scope is used to make a graph legible in the TensorBoard graph explorer\n",
    "# shape=[None] means x_placeholder is a one dimensional array of any length. \n",
    "# name='x' gives TensorBoard a display name for this node.\n",
    "with tf.name_scope('input'):\n",
    "    x_placeholder = tf.placeholder(shape=[None], dtype=tf.float32, name='x-input')\n",
    "    y_placeholder = tf.placeholder(shape=[None], dtype=tf.float32, name='y-input')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Uncomment the following lines to see what x_placeholder and y_placeholder are.\n",
    "# print(x_placeholder)\n",
    "# print(y_placeholder)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Step 3) Define our model.\n",
    "\n",
    "Here, we'll use a linear model: *y = mx + b*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "with tf.name_scope('model'):\n",
    "    m = tf.Variable(tf.random_normal([1]), name='m')\n",
    "    b = tf.Variable(tf.random_normal([1]), name='b')\n",
    "    # This is the same as y = tf.add(tf.mul(m, x_placeholder), b), but looks nicer\n",
    "    y = m * x_placeholder + b"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Uncomment the following lines to see what m, b, and y are\n",
    "# print(m)\n",
    "# print(b)\n",
    "# print(y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Step 4) Define a loss function (here, squared error) and an optimizer (here, gradient descent)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "LEARNING_RATE = 0.5\n",
    "\n",
    "with tf.name_scope('training'):\n",
    "    with tf.name_scope('loss'):\n",
    "        loss = tf.reduce_mean(tf.square(y - y_placeholder))\n",
    "    with tf.name_scope('optimizer'):\n",
    "        optimizer = tf.train.GradientDescentOptimizer(LEARNING_RATE)\n",
    "        train = optimizer.minimize(loss) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Uncomment the following 3 lines to see what 'loss', 'optimizer' and 'train' are.\n",
    "# print(\"loss:\", loss)\n",
    "# print(\"optimizer:\", optimizer)\n",
    "# print(\"train_step:\", train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Step 5) Set up TensorBoard"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Write the graph\n",
    "writer = tf.summary.FileWriter(LOGDIR)\n",
    "writer.add_graph(sess.graph)\n",
    "\n",
    "# Attach summaries to Tensors (for TensorBoard visualization)\n",
    "tf.summary.histogram('m', m)\n",
    "tf.summary.histogram('b', b)\n",
    "tf.summary.scalar('loss', loss)\n",
    "\n",
    "# This op will calculate our summary data when run\n",
    "summary_op = tf.summary.merge_all()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Step 6) Initialize variables\n",
    "\n",
    "At this point, our graph is complete. We're now ready to initialize variables, then begin training."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "sess.run(tf.global_variables_initializer())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Step 7) Training"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "TRAIN_STEPS = 201\n",
    "\n",
    "for step in range(TRAIN_STEPS):\n",
    "    \n",
    "    # Session will run two ops:\n",
    "    # - summary_op prepares summary data we'll write to disk in a moment\n",
    "    # - train will use the optimizer to adjust our variables to reduce loss\n",
    "    summary_result, _ = sess.run([summary_op, train], \n",
    "                                  feed_dict={x_placeholder: x_train, \n",
    "                                             y_placeholder: y_train})\n",
    "    # write the summary data to disk\n",
    "    writer.add_summary(summary_result, step)\n",
    "    \n",
    "    # Uncomment the following two lines to watch training happen real time.\n",
    "    if step % 20 == 0:\n",
    "        print(step, sess.run([m, b]))\n",
    "    \n",
    "# close the writer when we're finished using it\n",
    "writer.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Step 8) See the trained values for m and b"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print (\"m: %f, b: %f\" % (sess.run(m), sess.run(b)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Step 9) Use the trained model to make a prediction"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Use the trained model to make a prediction!\n",
    "# Remember that x_placeholder must be a vector, hence [2] not just 2.\n",
    "# We expect the result to be (about): 2 * 0.1 + 0.3 + noise ~= 0.5\n",
    "sess.run(y, feed_dict={x_placeholder: [2]})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Step 10) Start TensorBoard"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# To start TensorBoard, run this command from a terminal\n",
    "# Note: you should run this from the same directory that contains this notebook\n",
    "# or, provide absolute path to the 'graphs' directory\n",
    "# If you are runnining in a virtualenv, be sure to start it\n",
    "# if you open a new terminal window.\n",
    "\n",
    "# $ tensorboard --logdir=graphs\n",
    "\n",
    "# To open TensorBoard, point your browser to http://localhost:6006\n",
    "# Then, click on the tabs for 'scalars', 'distributions', 'histograms', and 'graphs'\n",
    "# to learn more.\n",
    "\n",
    "# If you run into trouble, delete LOGDIR (to clear information from previous runs),\n",
    "# then re-run this script, and restart TensorBoard."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}