Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Slip Calculator Jobs for Assignment and Course #1148

Open
wants to merge 11 commits into
base: master
Choose a base branch
from
Open
Prev Previous commit
Next Next commit
save csf file - complete assign route
  • Loading branch information
epai committed May 4, 2017
commit a10376ae52075a3ddb88b6147bb0c73beebe84f8
2 changes: 2 additions & 0 deletions server/controllers/admin.py
Original file line number Diff line number Diff line change
Expand Up @@ -996,6 +996,8 @@ def calculate_slips(cid, aid):
user_id=current_user.id,
assign_id=assign.id,
timescale=timescale,
show_results=form.show_results.data,
result_kind='link',
)
return redirect(url_for('.course_job', cid=cid, job_id=job.id))

Expand Down
1 change: 1 addition & 0 deletions server/forms.py
Original file line number Diff line number Diff line change
Expand Up @@ -564,6 +564,7 @@ class SlipCalculatorForm(BaseForm):
choices=[(c, c.title()) for c in TIMESCALES],
validators=[validators.required()],
description="Time scale for slip calculation.")
show_results = BooleanField('Show Results', default=False)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if this is useful as an option. We should either always show the results, or don't


##########
# Canvas #
Expand Down
40 changes: 24 additions & 16 deletions server/jobs/slips.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,8 @@
import csv

from server import jobs
from server.models import Assignment
from server.models import Assignment, ExternalFile
from server.utils import encode_id, local_time, output_csv_iterable
from server.constants import TIMESCALES

timescales = {'days':86400, 'hours':3600, 'minutes':60}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should go in constants as TIMESCALES, so there's not both TIMESCALES and timescales that have the same keys

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did this originally, but realized if I would then have to use the dictionary keys to fill in the form options, which wouldn't have a guaranteed ordering.

Now that I've thought about it, I think I'll just use an OrderedDict to preserve the ordering of the keys for display on the form.

Expand All @@ -13,21 +14,15 @@ def timediff(created, deadline, timescale):
secs_over = (created - deadline).total_seconds()
return math.ceil(secs_over / timescales[timescale.lower()])

def csv_data(header, rows):
output = io.StringIO()
writer = csv.writer(output, quoting=csv.QUOTE_NONNUMERIC)
writer.writerow(header)
[writer.writerow(row) for row in rows]
return output.getvalue()

@jobs.background_job
def calculate_slips(assign_id, timescale):
def calculate_slips(assign_id, timescale, show_results):
logger = jobs.get_job_logger()
job = jobs.get_current_job()

logger.info('Calculating Slip {}...'.format(timescale.title()))

assignment = Assignment.query.get(assign_id)
course = assignment.course
subms = assignment.course_submissions(include_empty=False)
deadline = assignment.due_date

Expand All @@ -39,12 +34,25 @@ def get_row(subm):

header = ('User Email', 'Slip {} Used'.format(timescale.title()))
rows = (get_row(subm) for subm in subms)
data = csv_data(header, rows)

logger.info(repr(data))

# upload = ExternalFile.upload(csv_data, user_id=1, course_id=1,
# name='temp.okfile', prefix='jobs/example/')

logger.info('Outputting csv...\n')
csv_iterable = output_csv_iterable(header, rows)

logger.info('Uploading...')
created_time = local_time(dt.now(), course, fmt='%m-%d-%I-%M-%p')
csv_name = '{}_{}.csv'.format(assignment.name.replace('/', '-'), created_time)
upload = ExternalFile.upload(csv_iterable,
user_id=job.user.id, course_id=course.id, name=csv_name,
prefix='jobs/slips/{}'.format(course.offering))
logger.info('Saved as: {}'.format(upload.object_name))

download_link = "/files/{}".format(encode_id(upload.id))
logger.info('Download link: {} (see "result" above)\n'.format(download_link))

if show_results:
logger.info('Results:\n')
csv_data = ''.join([row.decode('utf-8') for row in csv_iterable])
logger.info(csv_data)

return download_link


2 changes: 2 additions & 0 deletions server/templates/staff/jobs/calculate_slips.html
Original file line number Diff line number Diff line change
Expand Up @@ -31,8 +31,10 @@ <h1>
<div class="col-md-12">
<div class="box">
<div class="box-body">
<p> Calculate slips and save as a .csv file. </p>
{% call forms.render_form(form, action_text='Calculate Slips') %}
{{ forms.render_field(form.timescale) }}
{{ forms.render_checkbox_field(form.show_results) }}
{% endcall %}
</div>
</div>
Expand Down
10 changes: 10 additions & 0 deletions server/utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -236,6 +236,16 @@ def chunks(l, n):
prev_index = index


def output_csv_iterable(header, rows):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is too similar to generate_csv below to be another function. It has other problems: it uses Windows line endings, and will buffer the whole thing in memory (which isn't really an issue for these jobs, but it defeats the while purpose of using iterables).

Instead, try to use generate_csv. It would looks something like

    csv = generate_csv(subms, header, get_row)

where subms is the thing you're iterating over, and get_row is a function that returns a dict of CSV values.

""" Generate csv string for given header list and list of rows (lists). """
output = StringIO()
writer = csv.writer(output, quoting=csv.QUOTE_NONNUMERIC)
writer.writerow(header)
[writer.writerow(row) for row in rows]
rows = output.getvalue().split('\r\n')
return [bytes(row + '\r\n', 'utf-8') for row in rows]


def generate_csv(query, items, selector_fn):
""" Generate csv export of scores for assignment.
selector_fn: 1 arg function that returns a list of dictionaries
Expand Down