Skip to content

Commit

Permalink
Document the code execution semantics much more carefully.
Browse files Browse the repository at this point in the history 
Also renamed user var/expression functions to match the fields in the
execute request.
  • Loading branch information
@fperez
fperez committed Sep 18, 2010
1 parent 4c7fc69 commit 878faee
Show file tree
Hide file tree
Showing 3 changed files with 136 additions and 46 deletions.
33 changes: 25 additions & 8 deletions IPython/core/interactiveshell.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -1964,10 +1964,17 @@ def _simple_error(self):
etype, value = sys.exc_info()[:2]
return u'[ERROR] {e.__name__}: {v}'.format(e=etype, v=value)

def get_user_variables(self, names):
def user_variables(self, names):
"""Get a list of variable names from the user's namespace.
The return value is a dict with the repr() of each value.
Parameters
----------
names : list of strings
A list of names of variables to be read from the user namespace.
Returns
-------
A dict, keyed by the input names and with the repr() of each value.
"""
out = {}
user_ns = self.user_ns
Expand All @@ -1979,10 +1986,20 @@ def get_user_variables(self, names):
out[varname] = value
return out

def eval_expressions(self, expressions):
def user_expressions(self, expressions):
"""Evaluate a dict of expressions in the user's namespace.
The return value is a dict with the repr() of each value.
Parameters
----------
expressions : dict
A dict with string keys and string values. The expression values
should be valid Python expressions, each of which will be evaluated
in the user namespace.
Returns
-------
A dict, keyed like the input expressions dict, with the repr() of each
value.
"""
out = {}
user_ns = self.user_ns
Expand Down Expand Up @@ -2121,10 +2138,10 @@ def run_cell(self, cell):
If there's more than one block, it depends:
- if the last one is a single line long, run all but the last in
'exec' mode and the very last one in 'single' mode. This makes it
easy to type simple expressions at the end to see computed values.
- otherwise (last one is also multiline), run all in 'exec' mode
- if the last one is no more than two lines long, run all but the last
in 'exec' mode and the very last one in 'single' mode. This makes it
easy to type simple expressions at the end to see computed values. -
otherwise (last one is also multiline), run all in 'exec' mode
When code is executed in 'single' mode, :func:`sys.displayhook` fires,
results are displayed and output prompts are computed. In 'exec' mode,
Expand Down
4 changes: 2 additions & 2 deletions IPython/zmq/ipkernel.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -250,9 +250,9 @@ def execute_request(self, ident, parent):
# or not. If it did, we proceed to evaluate user_variables/expressions
if reply_content['status'] == 'ok':
reply_content[u'user_variables'] = \
shell.get_user_variables(content[u'user_variables'])
shell.user_variables(content[u'user_variables'])
reply_content[u'user_expressions'] = \
shell.eval_expressions(content[u'user_expressions'])
shell.user_expressions(content[u'user_expressions'])
else:
# If there was an error, don't even try to compute variables or
# expressions
Expand Down
145 changes: 109 additions & 36 deletions docs/source/development/messaging.txt
View file
Original file line number Diff line number Diff line change
Expand Up @@ -160,30 +160,23 @@ Message type: ``execute_request``::
'user_expressions' : dict,
}

The ``code`` field contains a single string, but this may be a multiline
string. The kernel is responsible for splitting this into possibly more than
one block and deciding whether to compile these in 'single' or 'exec' mode.
We're still sorting out this policy. The current inputsplitter is capable of
splitting the input for blocks that can all be run as 'single', but in the long
run it may prove cleaner to only use 'single' mode for truly single-line
inputs, and run all multiline input in 'exec' mode. This would preserve the
natural behavior of single-line inputs while allowing long cells to behave more
likea a script. This design will be refined as we complete the implementation.
The ``code`` field contains a single string (possibly multiline). The kernel
is responsible for splitting this into one or more independent execution blocks
and deciding whether to compile these in 'single' or 'exec' mode (see below for
detailed execution semantics).

The ``user_`` fields deserve a detailed explanation. In the past, IPython had
the notion of a prompt string that allowed arbitrary code to be evaluated, and
this was put to good use by many in creating prompts that displayed system
status, path information, and even more esoteric uses like remote instrument
status aqcuired over the network. But now that IPython has a clean separation
between the kernel and the clients, the notion of embedding 'prompt'
maninpulations into the kernel itself feels awkward. Prompts should be a
frontend-side feature, and it should be even possible for different frontends
to display different prompts while interacting with the same kernel.
between the kernel and the clients, the kernel has no prompt knowledge; prompts
are a frontend-side feature, and it should be even possible for different
frontends to display different prompts while interacting with the same kernel.

We have therefore abandoned the idea of a 'prompt string' to be evaluated by
the kernel, and instead provide the ability to retrieve from the user's
namespace information after the execution of the main ``code``, with two fields
of the execution request:
The kernel now provides the ability to retrieve data from the user's namespace
after the execution of the main ``code``, thanks to two fields in the
``execute_request`` message:

- ``user_variables``: If only variables from the user's namespace are needed, a
list of variable names can be passed and a dict with these names as keys and
Expand All @@ -205,32 +198,106 @@ terminal, etc).
empty code string and ``silent=True``.

Execution semantics
Upon completion of the execution request, the kernel *always* sends a
reply, with a status code indicating what happened and additional data
depending on the outcome.
~~~~~~~~~~~~~~~~~~~

The ``code`` field is executed first, and then the ``user_variables`` and
``user_expressions`` are computed. This ensures that any error in the
latter don't harm the main code execution.
When the silent flag is false, the execution of use code consists of the
following phases (in silent mode, only the ``code`` field is executed):

Any error in retrieving the ``user_variables`` or evaluating the
``user_expressions`` will result in a simple error message in the return
fields of the form::
1. Run the ``pre_runcode_hook``.

[ERROR] ExceptionType: Exception message
2. Execute the ``code`` field, see below for details.

3. If #2 succeeds, compute ``user_variables`` and ``user_expressions`` are
computed. This ensures that any error in the latter don't harm the main
code execution.

4. Call any method registered with :meth:`register_post_execute`.

.. warning::

The API for running code before/after the main code block is likely to
change soon. Both the ``pre_runcode_hook`` and the
:meth:`register_post_execute` are susceptible to modification, as we find a
consistent model for both.

To understand how the ``code`` field is executed, one must know that Python
code can be compiled in one of three modes (controlled by the ``mode`` argument
to the :func:`compile` builtin):

*single*
Valid for a single interactive statement (though the source can contain
multiple lines, such as a for loop). When compiled in this mode, the
generated bytecode contains special instructions that trigger the calling of
:func:`sys.displayhook` for any expression in the block that returns a value.
This means that a single statement can actually produce multiple calls to
:func:`sys.displayhook`, if for example it contains a loop where each
iteration computes an unassigned expression would generate 10 calls::

for i in range(10):
i**2

*exec*
An arbitrary amount of source code, this is how modules are compiled.
:func:`sys.displayhook` is *never* implicitly called.

*eval*
A single expression that returns a value. :func:`sys.displayhook` is *never*
implicitly called.


The ``code`` field is split into individual blocks each of which is valid for
execution in 'single' mode, and then:

- If there is only a single block: it is executed in 'single' mode.

- If there is more than one block:

* if the last one is a single line long, run all but the last in 'exec' mode
and the very last one in 'single' mode. This makes it easy to type simple
expressions at the end to see computed values.

* if the last one is no more than two lines long, run all but the last in
'exec' mode and the very last one in 'single' mode. This makes it easy to
type simple expressions at the end to see computed values. - otherwise
(last one is also multiline), run all in 'exec' mode

* otherwise (last one is also multiline), run all in 'exec' mode as a single
unit.

Any error in retrieving the ``user_variables`` or evaluating the
``user_expressions`` will result in a simple error message in the return fields
of the form::

[ERROR] ExceptionType: Exception message

The user can simply send the same variable name or expression for evaluation to
see a regular traceback.

Errors in any registered post_execute functions are also reported similarly,
and the failing function is removed from the post_execution set so that it does
not continue triggering failures.

Upon completion of the execution request, the kernel *always* sends a reply,
with a status code indicating what happened and additional data depending on
the outcome. See :ref:`below <execution_results>` for the possible return
codes and associated data.

The user can simply send the same variable name or expression for
evaluation to see a regular traceback.

Execution counter (old prompt number)
The kernel has a single, monotonically increasing counter of all execution
requests that are made with ``silent=False``. This counter is used to
populate the ``In[n]``, ``Out[n]`` and ``_n`` variables, so clients will
likely want to display it in some form to the user, which will typically
(but not necessarily) be done in the prompts. The value of this counter
will be returned as the ``execution_count`` field of all ``execute_reply```
messages.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The kernel has a single, monotonically increasing counter of all execution
requests that are made with ``silent=False``. This counter is used to populate
the ``In[n]``, ``Out[n]`` and ``_n`` variables, so clients will likely want to
display it in some form to the user, which will typically (but not necessarily)
be done in the prompts. The value of this counter will be returned as the
``execution_count`` field of all ``execute_reply`` messages.

.. _execution_results:

Execution results
~~~~~~~~~~~~~~~~~

Message type: ``execute_reply``::

content = {
Expand Down Expand Up @@ -303,6 +370,11 @@ happens when the kernel was interrupted by a signal.
Kernel attribute access
-----------------------

.. warning::

This part of the messaging spec is not actually implemented in the kernel
yet.

While this protocol does not specify full RPC access to arbitrary methods of
the kernel object, the kernel does allow read (and in some cases write) access
to certain attributes.
Expand Down Expand Up @@ -361,6 +433,7 @@ Message type: ``setattr_reply``::
}



Object information
------------------

Expand Down

0 comments on commit 878faee

Please sign in to comment.