forked from mas-dse-ryconrad/DSE200-notebooks
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Tushar
authored and
Tushar
committed
Oct 10, 2017
1 parent
7e88267
commit 8488f9c
Showing
26 changed files
with
21,838 additions
and
0 deletions.
There are no files selected for viewing
Binary file not shown.
Large diffs are not rendered by default.
Oops, something went wrong.
Binary file added
BIN
+679 KB
day_2_github_io_numpy_pandas/025_Memory_hierarchy.slides/MemoryHeirarchy.pdf
Binary file not shown.
1,741 changes: 1,741 additions & 0 deletions
1,741
day_2_github_io_numpy_pandas/02_measuring_performance_of_memory_hierarchy.ipynb
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,201 @@ | ||
| { | ||
| "cells": [ | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "# Comparing the speed of python and numpy\n", | ||
| "\n", | ||
| "first, we define two random vectors" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": { | ||
| "collapsed": false | ||
| }, | ||
| "outputs": [], | ||
| "source": [ | ||
| "import numpy as np\n", | ||
| "from numpy import shape\n", | ||
| "\n", | ||
| "%time\n", | ||
| "L = 1000000\n", | ||
| "a=np.random.uniform(low=-1,high=1,size=L)\n", | ||
| "b=np.random.uniform(low=-1,high=1,size=L)" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": { | ||
| "collapsed": false | ||
| }, | ||
| "outputs": [], | ||
| "source": [ | ||
| "shape(a),shape(b)" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": { | ||
| "collapsed": false | ||
| }, | ||
| "outputs": [], | ||
| "source": [ | ||
| "a[:5],b[:5]" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "Check time for performing the dot product between lists rather than ndarrays." | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "## Computing a dot product using pure python" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": { | ||
| "collapsed": false | ||
| }, | ||
| "outputs": [], | ||
| "source": [ | ||
| "def python_dot(a,b):\n", | ||
| " sum=0\n", | ||
| " for i in xrange(len(a)):\n", | ||
| " sum+=a[i]*b[i]\n", | ||
| " return sum\n", | ||
| " " | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": { | ||
| "collapsed": false | ||
| }, | ||
| "outputs": [], | ||
| "source": [ | ||
| "%time python_dot(a,b)" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "## Computing the dot product using the dot method in numpy" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": { | ||
| "collapsed": false | ||
| }, | ||
| "outputs": [], | ||
| "source": [ | ||
| "%time np.dot(a,b)" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "## Computing the dot product using cython" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": { | ||
| "collapsed": false | ||
| }, | ||
| "outputs": [], | ||
| "source": [ | ||
| "%load_ext Cython" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": { | ||
| "collapsed": false | ||
| }, | ||
| "outputs": [], | ||
| "source": [ | ||
| "%%cython\n", | ||
| "cimport numpy as np # makes numpy available to cython\n", | ||
| "# The following line defines a and b as numpy arrays, cython knows how to deal with those.\n", | ||
| "def cython_dot(np.ndarray[np.float64_t, ndim=1] a,\n", | ||
| " np.ndarray[np.float64_t, ndim=1] b):\n", | ||
| " cdef double sum\n", | ||
| " cdef long i\n", | ||
| " sum=0\n", | ||
| " for i in xrange(a.shape[0]):\n", | ||
| " sum=sum+a[i]*b[i]\n", | ||
| " return sum\n" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": { | ||
| "collapsed": false | ||
| }, | ||
| "outputs": [], | ||
| "source": [ | ||
| "%%time\n", | ||
| "cython_dot(a,b)" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": { | ||
| "collapsed": false | ||
| }, | ||
| "outputs": [], | ||
| "source": [] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": { | ||
| "collapsed": false | ||
| }, | ||
| "outputs": [], | ||
| "source": [] | ||
| } | ||
| ], | ||
| "metadata": { | ||
| "kernelspec": { | ||
| "display_name": "Python 2", | ||
| "language": "python", | ||
| "name": "python2" | ||
| }, | ||
| "language_info": { | ||
| "codemirror_mode": { | ||
| "name": "ipython", | ||
| "version": 2 | ||
| }, | ||
| "file_extension": ".py", | ||
| "mimetype": "text/x-python", | ||
| "name": "python", | ||
| "nbconvert_exporter": "python", | ||
| "pygments_lexer": "ipython2", | ||
| "version": "2.7.12" | ||
| } | ||
| }, | ||
| "nbformat": 4, | ||
| "nbformat_minor": 0 | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,50 @@ | ||
| Place for everything Pandas. | ||
|
|
||
| How to install Pandas? | ||
| ------- | ||
|
|
||
| * [Windows](http://www.youtube.com/watch?v=g4v9_K3Rq3Y) | ||
| * [Linux](http://hdrojas.pythonanywhere.com/static/data/Data%20Analysis%20Kick%20Start.html) | ||
|
|
||
| Lessons | ||
| ------- | ||
|
|
||
| * [01 - Lesson](http://nbviewer.ipython.org/urls/bitbucket.org/hrojas/learn-pandas/raw/master/lessons/01%20-%20Lesson.ipynb) | ||
| * [02 - Lesson](http://nbviewer.ipython.org/urls/bitbucket.org/hrojas/learn-pandas/raw/master/lessons/02%20-%20Lesson.ipynb) | ||
| * [03 - Lesson](http://nbviewer.ipython.org/urls/bitbucket.org/hrojas/learn-pandas/raw/master/lessons/03%20-%20Lesson.ipynb) | ||
| * [04 - Lesson](http://nbviewer.ipython.org/urls/bitbucket.org/hrojas/learn-pandas/raw/master/lessons/04%20-%20Lesson.ipynb) | ||
| * [05 - Lesson](http://nbviewer.ipython.org/urls/bitbucket.org/hrojas/learn-pandas/raw/master/lessons/05%20-%20Lesson.ipynb) | ||
| * [06 - Lesson](http://nbviewer.ipython.org/urls/bitbucket.org/hrojas/learn-pandas/raw/master/lessons/06%20-%20Lesson.ipynb) | ||
| * [07 - Lesson](http://nbviewer.ipython.org/urls/bitbucket.org/hrojas/learn-pandas/raw/master/lessons/07%20-%20Lesson.ipynb) | ||
| * [08 - Lesson](http://nbviewer.ipython.org/urls/bitbucket.org/hrojas/learn-pandas/raw/master/lessons/08%20-%20Lesson.ipynb) | ||
| * [09 - Lesson](http://nbviewer.ipython.org/urls/bitbucket.org/hrojas/learn-pandas/raw/master/lessons/09%20-%20Lesson.ipynb) | ||
| * [10 - Lesson](http://nbviewer.ipython.org/urls/bitbucket.org/hrojas/learn-pandas/raw/master/lessons/10%20-%20Lesson.ipynb) | ||
| * [11 - Lesson](http://nbviewer.ipython.org/urls/bitbucket.org/hrojas/learn-pandas/raw/master/lessons/11%20-%20Lesson.ipynb) | ||
|
|
||
|
|
||
| Cookbooks | ||
| --------- | ||
|
|
||
| * [Compute](http://nbviewer.ipython.org/urls/bitbucket.org/hrojas/learn-pandas/raw/master/lessons/Cookbook%20-%20Compute.ipynb) | ||
| * [Merge](http://nbviewer.ipython.org/urls/bitbucket.org/hrojas/learn-pandas/raw/master/lessons/Cookbook%20-%20Merge.ipynb) | ||
| * [Select](http://nbviewer.ipython.org/urls/bitbucket.org/hrojas/learn-pandas/raw/master/lessons/Cookbook%20-%20Select.ipynb) | ||
| * [Sort](http://nbviewer.ipython.org/urls/bitbucket.org/hrojas/learn-pandas/raw/master/lessons/Cookbook%20-%20Sort.ipynb) | ||
| * [Python 101](http://nbviewer.ipython.org/urls/bitbucket.org/hrojas/learn-pandas/raw/master/lessons/Python_101.ipynb) | ||
|
|
||
| Tutorials | ||
| --------- | ||
|
|
||
| * [Managing Python Packages](https://squareup.com/market/david-rojas-llc/managing-python-packages) | ||
| * [Pandas for Excel Developers](https://squareup.com/market/david-rojas-llc/data-analysis-python-for-excel-developers) | ||
| * [Pandas for SQL Developers](https://squareup.com/market/david-rojas-llc/pandas-for-sql-developers) | ||
| * [How to Deal with Dates in Python](https://squareup.com/market/david-rojas-llc/data-analysis-dates) | ||
| * [Plotting in Pandas](https://squareup.com/market/david-rojas-llc/data-analysis-plotting-in-pandas) | ||
| * [Group By Operations in Pandas](https://squareup.com/market/david-rojas-llc/data-analysis-group-by) | ||
| * [Lambda functions and masks in Pandas](https://squareup.com/market/david-rojas-llc/data-analysis-lambda-and-masks) | ||
| * [more tutorials...](https://squareup.com/market/david-rojas-llc) | ||
|
|
||
| Free Tools | ||
| --------- | ||
|
|
||
| * [pyrs - Convert your Ipython notebooks](https://pyrs.herokuapp.com/) | ||
| * [Pandas Bootcamp (beta) - Practice your data wrangling skills](https://pandasbootcamp.herokuapp.com/) |
Oops, something went wrong.