Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ENH: add google bigquery interface #24

Open
linwoodc3 opened this issue May 30, 2017 · 0 comments
Open

ENH: add google bigquery interface #24

linwoodc3 opened this issue May 30, 2017 · 0 comments

Comments

@linwoodc3
Copy link
Owner

linwoodc3 commented May 30, 2017

Use [pandas.io.gbq](http://pandas.pydata.org/pandas-docs/stable/generated/pandas.read_gbq.html#pandas.read_gbq)

First you will need to pip install: pip install --upgrade google-api-python-client
Here is a working query:

# load keys;  requires you to be registered
keys = json.load(open('/Users/linwood/Desktop/keysforapps/apikeys.txt'))

# setup google creds; not sure if this is required yet; but you need to do it once to authorize the api from your python ecosystem
from apiclient.discovery import build
service = build('bigquery', 'v2', developerKey=keys['google']['apikey']+"2")

# load query in proper SQL syntax as string
from pandas.io import gbq
q="""
SELECT MonthYear,count(*)c,count(IF(Actor1Code LIKE 'MUS',1,null)) c_up
FROM [gdelt-bq.full.events] WHERE EventRootCode = '19'
GROUP BY MonthYear ORDER BY MonthYear;"""


# run the query
df = gbq.read_gbq(q, project_id=<projectid>)
:[out]
Requesting query... ok.
Query running...
Query done.
Cache hit.

Retrieving results...
Got 461 rows.

Total time taken 0.75 s.
Finished at 2017-05-30 10:26:21
@linwoodc3 linwoodc3 changed the title add google bigquery interface ENH: add google bigquery interface Jul 18, 2017
@linwoodc3 linwoodc3 modified the milestone: 0.1.11 Jul 23, 2017
@linwoodc3 linwoodc3 modified the milestones: 0.1.11, Phase 1 Aug 1, 2017
@linwoodc3 linwoodc3 added this to Create repeatable, reliable, stable access to GDELT data in Complete Phase 1 Aug 1, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Complete Phase 1
Create repeatable, reliable, stable a...
Development

No branches or pull requests

1 participant