1.Basics_NLP.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction - File Opening"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Opening a file"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>To open a file for writing use the built-i open() function. open() returns a file object, and is most commonly used with two arguments.<br/>\n",
    "\n",
    "The syntax is:<br/>\n",
    "file_object = open(filename, mode)<br/>\n",
    "where file_object is the variable to put the file object. The second argument describes the way in which the file will be used.\n",
    "</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "file = open('Data/Basics_NLP/gaur.txt', 'r')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Note-:** The command \"open('newfile.txt', 'r')\" doesn't return the contents of the file. It actually makes something called a \"file object.\" You can think of a file like an old tape drive that you saw on mainframe computers in the 1950s, or even like a DVD player from today. You can move around inside them, and then \"read\" them, but the DVD player is not the DVD the same way the file object is not the file's contents."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Reading a file\n",
    "\n",
    "If you want to return a string containing all characters in the file, you can\n",
    "use file.read()."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Love is everlasting Forgiveness.\n",
      "Having free time is not an opulence, it's a danger.\n",
      "Krishna is the Supreme Personality of Godhead. \n"
     ]
    }
   ],
   "source": [
    "file = open('Data/Basics_NLP/gaur.txt', 'r')\n",
    "\n",
    "print file.read()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also specify how many characters the string should return, by using\n",
    "file.read(n), where \"n\" determines number of characters.<br/>\n",
    "\n",
    "This reads the first 5 characters of data and returns it as a string."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Love is\n"
     ]
    }
   ],
   "source": [
    "file = open('Data/Basics_NLP/gaur.txt', 'r')\n",
    "\n",
    "print file.read(7)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Reading multiple files from a folder\n",
    "\n",
    "The method listdir() returns a list containing the names of the entries in the directory given by path. The list is in arbitrary order. It does not include the special entries '.' and '..' even if they are present in the directory."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "test2.txt\n",
      "test1.txt\n"
     ]
    }
   ],
   "source": [
    "import os\n",
    "\n",
    "for fileName in os.listdir(\"Data/Basics_NLP/dirForBasics_NLP\"):\n",
    "    if fileName.endswith(\".txt\"):\n",
    "        print(fileName)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Reading a file with size greater than RAM\n",
    "\n",
    "If the size of the file is very large then it can't be opened directly and the system may hang in an attempt to do so. So we use lazy loading of the file that reads the data in chunks"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def read_in_chunks(file_object, chunk_size=1024):\n",
    "    \"\"\"Lazy function (generator) to read a file piece by piece.\n",
    "    Default chunk size: 1k.\"\"\"\n",
    "    while True:\n",
    "        data = file_object.read(chunk_size)\n",
    "        if not data:\n",
    "            break\n",
    "        yield data\n",
    "\n",
    "\n",
    "f = open('Data/Basics_NLP/nitai.pdf')\n",
    "for piece in read_in_chunks(f):\n",
    "    read_in_chunks(piece)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Lazy loading of a gzip file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "h1, h2 {margin: 10px 25px 5px;}h2 {font-size: 1.1em;}.filename {font-style: italic;}.exceptionMessage {margin: 10px;border: 1px solid #000;padding: 5px;background-color: #E9E9E9;}.stack,.snippet {margin: 0 25px 10px;}.stack,.snippet {border: 1px solid #ccc;-mox-box-shadow: 0 0 2px rgba(0,0,0,0.2);-webkit-box-shadow: 0 0 2px rgba(0,0,0,0.2);box-shadow: 0 0 2px rgba(0,0,0,0.2);}.error-details {border-top: 1px solid #FFAAAA;-mox-box-shadow: 0 0 2px rgba(0,0,0,0.2);-webkit-box-shadow: 0 0 2px rgba(0,0,0,0.2);box-shadow: 0 0 2px rgba(0,0,0,0.2);border-bottom: 1px solid #FFAAAA;-mox-box-shadow: 0 0 2px rgba(0,0,0,0.2);-webkit-box-shadow: 0 0 2px rgba(0,0,0,0.2);box-shadow: 0 0 2px rgba(0,0,0,0.2);background-color:#FFF3F3;line-height: 1.5;overflow: hidden;padding: 5px;padding-left:25px;}.error-details dt {clear: left;float: left;font-weight: bold;margin-right: 5px;}.error-details dt:after {content: \":\";}.error-details dd {display: block;}.stack {padding: 5px;overflow: auto;height: 150px;}.snippet {background-color: #fff;font-family: monospace;}.snippet .line {display: block;}.snippet .lineNumber {background-color: #ddd;color: #999;display: inline-block;margin-right: 5px;padding: 0 3px;text-align: right;width: 3em;}.snippet .error {background-color: #fff3f3;font-weight: bold;}.snippet .error .lineNumber {background-color: #faa;color: #333;font-weight: bold;}.snippet .line:first-child .lineNumber {padding-top: 5px;}.snippet .line:last-child .lineNumber {padding-bottom: 5px;}\n"
     ]
    }
   ],
   "source": [
    "import gzip\n",
    "f=gzip.open('Data/Basics_NLP/errors.css.gz','rb')\n",
    "file_content=f.read()\n",
    "print file_content"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Rerading the Jupyter Way"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "void show_\n",
      "void datat\n"
     ]
    }
   ],
   "source": [
    "a = !ls Data/Basics_NLP/dirForBasics_NLP\n",
    "    \n",
    "for files in a:\n",
    "    f = open('Data/Basics_NLP/dirForBasics_NLP/'+files).read()\n",
    "    print f[:10]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Practice\n",
    "\n",
    " 1. How do you read a file to the memory and split it line by line\n",
    " 2. How do you read line by line to the memory?\n",
    " \n",
    " Hint : `splitlines() or split()`, `with .. as ..`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "Developer - Pranav Shukla, email - pranavdynamic@gmail.com"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}