Skip to content

Commit

Permalink
[move-prover] multiple fixes to the memory model (move-language#312)
Browse files Browse the repository at this point in the history
* [move-prover] fix is_parent tests for multiple write-back chains

PR move-language#302 has an error in terms of placing `IsParent` checks for different
borrow chains.

Consider the following case:

```
                        --borrows_from--> C
A --borrows_from--> B --|
                        --borrows_from--> D
```

When references A, B, C, D go out of scope, this will yield two chain of
references:

A --> B --> C and
A --> B --> D

This is correctly captured in move-language#302.

The bug is how to generate the `IsParent` test:

The correct behavior is the following:

```
if IsParent[C](B)
    WriteBack[B](A)
    WriteBack[C](B)

if IsParent[D](B)
    WriteBack[B](A)
    WriteBack[D](B)
```

i.e., branching at node B.

But in move-language#302, I mistakenly used the root of the chain as the branching
point and hence, both the tests become `IsParent[B](A)`, which is not
correct.

This commit fixes this issue by deriving the deterministic factor for
the write-back chain.

Consider the case with the borrowing graph:

```
                  ---> E
          ---> C -|
A ---> B -|       ---> F
          ---> D
```

This will yield three chains:
```
if IsParent[C](B) && IsParent[E](C)
    WriteBack[B](A)
    WriteBack[C](B)
    WriteBack[E](C)

if IsParent[C](B) && IsParent[F](C)
    WriteBack[B](A)
    WriteBack[C](B)
    WriteBack[F](C)

if IsParent[D](B)
    WriteBack[B](A)
    WriteBack[D](B)
```

* [move-prover] improve the peephole optimization for write-back chains

* [move-prover] do not peephole optimize the initial local and global borrows

Live variable analysis allows the following peephole optimization:

```
  $t := call(...)
  x := $t
  <$t is dead>

  ~~~

  x := call(...)
```

This optimization makes perfect sense but unfortunately conflicts with
one policy later: IsParent test can only be applied on two references:

Taking the following code as an example:
```
let v1 = 0;
let v2 = 0;

let r = if (cond) {
  let r1 = &mut v1;
  r1
} else {
  let r2 = &mut v2;
  r2
};
```

References `r1` and `r2` will be optimized away after the live var
analysis. This leaves the reference `r` pointing to two local locations.

And later when translating `IsParent` test to Boogie, we will run into
cases like:

```
IsParent[LocalRoot(v1)@](r)
IsParent[LocalRoot(v2)@](r)
```

And this violates our policy that IsParent can only be applied among
two references.

As a result, we disable this optimization for the initial local/global
borrowing, which leaves the IsParent tests to be
```
IsParent[Reference(r1)@](r)
IsParent[Reference(r2]@](r)
```

An alternative solution to this problem is to implement IsParent tests
between a reference to a `LocalRoot` and a `GloablRoot`. But that is a
more significant change and does not buy much benefits performance-wise.

* [move-prover] use a special location to mark uninitialized mut refs

Mutable references cannot be arbitrarily initialized, they must be
initialized into a state where the IsParent test have to fail if this
reference is not assigned otherwise.

To be specific, consider the folowing example:

```
let v1 = 0;
let v2 = 0;

let r = if (cond) {
    let r1 = &mut v1;
    r1
} else {
    let r2 = &mut v2;
    r2
};
```

This will be translated to:

```
r1 := $uninitialized();
r2 := $uninitialized();

... <omitted> ...

if (cond) {
    r1 := borrow_local(v1);
    r := r1
} else {
    r2 := borrow_local(v2);
    r := r2
}

... <omitted> ...

if (IsParent[r1](r)) {
    WriteBack[r1](r);
    WriteBack[v1](r1);
}
if (IsParent[r2](r)) {
    WriteBack[r2](r);
    WriteBack[v2](r2);
}
```

It is very important to mark `r1` and `r2` and `$uninitialized()`
to ensure that there will be one and only one `IsParent` test activated
when the write-back happens.

The previous approach is to `assume IsEmptyVec(p#$Mutation($t_i));`
for all `t_i` that is a mutable reference used in function locals.

However, setting the `path` vector in the `Mutation` struct to zero
does not guarantee that this mutation, when put under `IsParent` test,
is not going to return false when that mutation is not initialized.

As can be seen in the example above, even after proper assignment, `r1`
and `r2` both have a `path` equal to the length of zero... (because `r`
is borrowing from `r1` and `r2` `as direct borrow, not field or element
borrow).

This commit fixes this issue with an explicit `$Uninitialize()`
constructor for the `$Location` part of a `$Mutation`.

* [move-prover] use an `Uninit` operation to mark uninitialized mut refs

Following from the above commit, instead of marking all mutable
references in the function locals (except the function arguments) as
uninitialized, this commit further restrict the number of mutable
references that can be marked as `Uninit`, hence, reducing the chances
of introducing inconsistent assumptions in the verification.

The basic idea is to leverage the livevar analysis results, and mark
whatever mutable reference that is
1) alive even before the first instruction of the function and
2) is not a parameter,
as uninitialized.

This commit also includes a fix to the concrete spec executor such that
we do not run into errors when a mutable reference is mysterously
destroyed even before it is initialized.

* [move-prover] do not let bind a mutable reference for `EmitEvent`

```
public fun emit_event<T>(handle_ref: &mut EventHandle<T>, msg: T) { ... }
```

Events are verified in Move Prover in a special manner. One example is
that the `EmitEvent` bytecode is especially designed to simulate the
`emit_event` function.

During the invocation of `EmitEvent`, we might be let-binding a mutable
reference which we really do not want as introcuing a mutable reference
via `Assume(Identical)` complicates the live var and borrow analysis
(for example, we will treat the newly bounded reference as uninitialized).

While essentially we do not need a reference to the handle and we can
just let-bound the `$Dereference ( <exp> )` value directly, which is
what is essentially done in this commit.

* [move-prover] update exp files for existing tests

* [move-prover] new tests to summarize the changes

* [move-prover][interpreter] update the concrete semantics with direct ref borrow

This re-enables the failing `if_else.move` test which conditionally (and
directly) assign references to another reference.
  • Loading branch information
meng-xu-cs authored Jul 30, 2022
1 parent 0fda31b commit e19e49a
Show file tree
Hide file tree
Showing 26 changed files with 888 additions and 476 deletions.
1 change: 1 addition & 0 deletions language/evm/move-to-yul/src/functions.rs
Original file line number Diff line number Diff line change
Expand Up @@ -562,6 +562,7 @@ impl<'a> FunctionGenerator<'a> {
| EventStoreDiverge
| OpaqueCallBegin(_, _, _)
| OpaqueCallEnd(_, _, _)
| Uninit
| Havoc(_)
| Stop
| TraceGlobalMem(_) => {}
Expand Down
34 changes: 17 additions & 17 deletions language/move-prover/boogie-backend/src/bytecode_translator.rs
Original file line number Diff line number Diff line change
Expand Up @@ -623,13 +623,6 @@ impl<'env> FunctionTranslator<'env> {
emitln!(writer, "$t{} := _$t{};", idx, idx);
}

// Initialize mutations to have empty paths so $IsParentMutation works correctly.
for i in num_args..fun_target.get_local_count() {
if self.get_local_type(i).is_mutable_reference() {
emitln!(writer, "assume IsEmptyVec(p#$Mutation($t{}));", i);
}
}

// Initial assumptions
if variant.is_verified() {
self.translate_verify_entry_assumptions(fun_target);
Expand Down Expand Up @@ -892,7 +885,15 @@ impl<'env> FunctionTranslator<'env> {
_ => unreachable!(),
})
.collect_vec();
if edge_pattern.len() == 1 {
if edge_pattern.is_empty() {
emitln!(
writer,
"{} := $IsSameMutation({}, {});",
str_local(dests[0]),
str_local(*parent),
src_str
);
} else if edge_pattern.len() == 1 {
emitln!(
writer,
"{} := $IsParentMutation({}, {}, {});",
Expand Down Expand Up @@ -1444,6 +1445,13 @@ impl<'env> FunctionTranslator<'env> {
bytecode
);
}
Uninit => {
emitln!(
writer,
"assume l#$Mutation($t{}) == $Uninitialized();",
srcs[0]
);
}
Destroy => {}
TraceLocal(idx) => {
self.track_local(*idx, srcs[0]);
Expand All @@ -1456,22 +1464,14 @@ impl<'env> FunctionTranslator<'env> {
EmitEvent => {
let msg = srcs[0];
let handle = srcs[1];
let translate_local = |idx: usize| {
let ty = &self.get_local_type(idx);
if ty.is_mutable_reference() {
format!("$Dereference({})", str_local(idx))
} else {
str_local(idx)
}
};
let suffix = boogie_type_suffix(env, &self.get_local_type(msg));
emit!(
writer,
"$es := ${}ExtendEventStore'{}'($es, ",
if srcs.len() > 2 { "Cond" } else { "" },
suffix
);
emit!(writer, "{}, {}", translate_local(handle), str_local(msg));
emit!(writer, "{}, {}", str_local(handle), str_local(msg));
if srcs.len() > 2 {
emit!(writer, ", {}", str_local(srcs[2]));
}
Expand Down
12 changes: 10 additions & 2 deletions language/move-prover/boogie-backend/src/prelude/prelude.bpl
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,9 @@ function {:constructor} $Local(i: int): $Location;
// when mutation ends.
function {:constructor} $Param(i: int): $Location;

// The location of an uninitialized mutation. Using this to make sure that the location
// will not be equal to any valid mutation locations, i.e., $Local, $Global, or $Param.
function {:constructor} $Uninitialized(): $Location;

// A mutable reference which also carries its current value. Since mutable references
// are single threaded in Move, we can keep them together and treat them as a value
Expand Down Expand Up @@ -144,7 +147,12 @@ function {:inline} $ChildMutation<T1, T2>(m: $Mutation T1, offset: int, v: T2):
$Mutation(l#$Mutation(m), ExtendVec(p#$Mutation(m), offset), v)
}

// Return true of the mutation is a parent of a child which was derived with the given edge offset. This
// Return true if two mutations share the location and path
function {:inline} $IsSameMutation<T1, T2>(parent: $Mutation T1, child: $Mutation T2 ): bool {
l#$Mutation(parent) == l#$Mutation(child) && p#$Mutation(parent) == p#$Mutation(child)
}

// Return true if the mutation is a parent of a child which was derived with the given edge offset. This
// is used to implement write-back choices.
function {:inline} $IsParentMutation<T1, T2>(parent: $Mutation T1, edge: int, child: $Mutation T2 ): bool {
l#$Mutation(parent) == l#$Mutation(child) &&
Expand All @@ -158,7 +166,7 @@ function {:inline} $IsParentMutation<T1, T2>(parent: $Mutation T1, edge: int, ch
))))
}

// Return true of the mutation is a parent of a child, for hyper edge.
// Return true if the mutation is a parent of a child, for hyper edge.
function {:inline} $IsParentMutationHyper<T1, T2>(parent: $Mutation T1, hyper_edge: Vec int, child: $Mutation T2 ): bool {
l#$Mutation(parent) == l#$Mutation(child) &&
(var pp := p#$Mutation(parent);
Expand Down
11 changes: 6 additions & 5 deletions language/move-prover/bytecode/src/borrow_analysis.rs
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ use move_model::{
};
use std::{borrow::BorrowMut, collections::BTreeMap, fmt};

#[derive(Debug, Clone, PartialEq, PartialOrd, Eq, Default)]
#[derive(Debug, Clone, Eq, Ord, PartialEq, PartialOrd, Default)]
pub struct BorrowInfo {
/// Contains the nodes which are alive. This excludes nodes which are alive because
/// other nodes which are alive borrow from them.
Expand All @@ -41,9 +41,10 @@ pub struct BorrowInfo {
}

/// Represents a write-back from a source node to a destination node with the associated edge
#[derive(Debug, Clone)]
#[derive(Debug, Clone, Eq, Ord, PartialEq, PartialOrd)]
pub struct WriteBackAction {
pub src: BorrowNode,
/// the `src` of a write-back action must be a reference
pub src: TempIndex,
pub dst: BorrowNode,
pub edge: BorrowEdge,
}
Expand Down Expand Up @@ -116,7 +117,7 @@ impl BorrowInfo {
BorrowNode::LocalRoot(..) | BorrowNode::GlobalRoot(..) => {
trees.push(order);
}
BorrowNode::Reference(..) => {
BorrowNode::Reference(index) => {
if next.is_in_use(node) {
// stop at a live reference
trees.push(order);
Expand All @@ -132,7 +133,7 @@ impl BorrowInfo {
for (parent, edge) in incoming {
let mut appended = order.clone();
appended.push(WriteBackAction {
src: node.clone(),
src: *index,
dst: parent.clone(),
edge: edge.clone(),
});
Expand Down
76 changes: 56 additions & 20 deletions language/move-prover/bytecode/src/clean_and_optimize.rs
Original file line number Diff line number Diff line change
Expand Up @@ -149,7 +149,8 @@ impl<'a> Optimizer<'a> {

// Transform code.
let mut new_instrs = vec![];
for (code_offset, instr) in instrs.into_iter().enumerate() {
let mut should_skip = BTreeSet::new();
for (code_offset, instr) in instrs.iter().enumerate() {
use BorrowNode::*;
use Bytecode::*;
use Operation::*;
Expand All @@ -162,38 +163,73 @@ impl<'a> Optimizer<'a> {
true
}
};
if !new_instrs.is_empty() {
// Perform peephole optimization
match (&new_instrs[new_instrs.len() - 1], &instr) {
(Call(_, _, UnpackRef, srcs1, _), Call(_, _, PackRef, srcs2, _))
if srcs1[0] == srcs2[0] =>
{
// skip this redundant unpack/pack pair.
new_instrs.pop();
continue;

// Perform peephole optimization
match (new_instrs.last(), instr) {
(None, _) => {}
(Some(Call(_, _, UnpackRef, srcs1, _)), Call(_, _, PackRef, srcs2, _))
if srcs1[0] == srcs2[0] =>
{
// skip this redundant unpack/pack pair.
new_instrs.pop();
continue;
}
(Some(Call(_, dests, IsParent(..), srcs, _)), Branch(_, _, _, tmp))
if dests[0] == *tmp
&& !is_unwritten(code_offset as CodeOffset, &Reference(srcs[0])) =>
{
assert!(matches!(instrs[code_offset + 1], Label(..)));
// skip this obsolete IsParent check when all WriteBacks in this block are redundant
let mut block_cursor = code_offset + 2;
let mut skip_branch = true;
loop {
match &instrs[block_cursor] {
Call(_, _, WriteBack(_, _), srcs, _) => {
if is_unwritten(block_cursor as CodeOffset, &Reference(srcs[0])) {
skip_branch = false;
break;
}
// skip redundant write-backs
should_skip.insert(block_cursor);
}
Call(_, _, TraceLocal(_), _, _) => {
// since the previous write-back is skipped, this trace local is redundant as well
should_skip.insert(block_cursor);
}
_ => {
break;
}
}
block_cursor += 1;
}
(Call(_, dests, IsParent(..), srcs, _), Branch(_, _, _, tmp))
if dests[0] == *tmp
&& !is_unwritten(code_offset as CodeOffset, &Reference(srcs[0])) =>
{
// skip this obsolete IsParent check
if skip_branch {
// get rid of the label as well
should_skip.insert(code_offset + 1);
new_instrs.pop();
continue;
}
_ => {}
}
(Some(_), _) => {}
}
// Remove unnecessary WriteBack
match &instr {

// Do not include this instruction if it is marked as skipped
if should_skip.contains(&code_offset) {
continue;
}

// Other cases for skipping the instruction
match instr {
// Remove unnecessary WriteBack
Call(_, _, WriteBack(..), srcs, _)
if !is_unwritten(code_offset as CodeOffset, &Reference(srcs[0])) =>
{
// skip this obsolete WriteBack
continue;
}
_ => {}
}
new_instrs.push(instr);

// This instruction should be included
new_instrs.push(instr.clone());
}
new_instrs
}
Expand Down
3 changes: 3 additions & 0 deletions language/move-prover/bytecode/src/escape_analysis.rs
Original file line number Diff line number Diff line change
Expand Up @@ -295,6 +295,9 @@ impl<'a> TransferFunctions for EscapeAnalysis<'a> {
WriteRef | MoveTo(..) => {
// these operations do not assign any locals
}
Uninit => {
// this operation is just a marker and does not assign any locals
}
Destroy => {
state.remove(&args[0]);
}
Expand Down
15 changes: 15 additions & 0 deletions language/move-prover/bytecode/src/function_data_builder.rs
Original file line number Diff line number Diff line change
Expand Up @@ -210,6 +210,21 @@ impl<'env> FunctionDataBuilder<'env> {
(temp, temp_exp)
}

/// Similar to `emit_let`, but with the temporary created as identical to the dereference of
/// the mutation (if the `def` argument is a mutable reference).
pub fn emit_let_skip_reference(&mut self, def: Exp) -> (TempIndex, Exp) {
let ty = self
.global_env()
.get_node_type(def.node_id())
.skip_reference()
.clone();
let temp = self.new_temp(ty);
let temp_exp = self.mk_temporary(temp);
let definition = self.mk_identical(temp_exp.clone(), def);
self.emit_with(|id| Bytecode::Prop(id, PropKind::Assume, definition));
(temp, temp_exp)
}

/// Emits a new temporary with a havoced value of given type.
pub fn emit_let_havoc(&mut self, ty: Type) -> (TempIndex, Exp) {
let havoc_kind = if ty.is_mutable_reference() {
Expand Down
33 changes: 32 additions & 1 deletion language/move-prover/bytecode/src/livevar_analysis.rs
Original file line number Diff line number Diff line change
Expand Up @@ -216,6 +216,30 @@ impl<'a> LiveVarAnalysis<'a> {
let label_to_code_offset = Bytecode::label_offsets(&code);
let mut transformed_code = vec![];
let mut new_bytecodes = vec![];

// insert marks for uninitialized mutable references
let num_args = self.func_target.get_parameter_count();
if let Some(info) = annotations.get(&0) {
for index in &info.before {
// only mark the mutable references conditionally defined in function body as uninit
if *index >= num_args
&& self
.func_target
.get_local_type(*index)
.is_mutable_reference()
{
transformed_code.push(Bytecode::Call(
self.new_attr_id(),
vec![],
Operation::Uninit,
vec![*index],
None,
));
}
}
}

// optimize the function body
let mut skip_next = false;
for code_offset in 0..code.len() {
if skip_next {
Expand Down Expand Up @@ -252,7 +276,9 @@ impl<'a> LiveVarAnalysis<'a> {
// Drop this load/assign as it is not used.
}
Bytecode::Call(attr_id, dests, oper, srcs, aa)
if code_offset + 1 < code.len() && dests.len() == 1 =>
if code_offset + 1 < code.len()
&& dests.len() == 1
&& !matches!(oper, Operation::BorrowLoc | Operation::BorrowGlobal(..)) =>
{
// Catch the common case where we have:
//
Expand All @@ -263,6 +289,11 @@ impl<'a> LiveVarAnalysis<'a> {
// This is an artifact from transformation from stack to stackless bytecode.
// Copy propagation cannot catch this case because it does not have the
// livevar information about $t.
//
// With one exception: if the called operation is a BorrowLocal or BorrowGlobal (i.e., an operation
// that creates a root mutable reference), do not optimize it away as we need this local/global root
// reference for our IsParent test. An alternative (i.e., one way to get rid of this exception) is
// to support IsParent test against local and global directly, but that is more complicated.
let next_code_offset = code_offset + 1;
if let Bytecode::Assign(_, dest, src, _) = &code[next_code_offset] {
let annotation_at = &annotations[&(next_code_offset as CodeOffset)];
Expand Down
Loading

0 comments on commit e19e49a

Please sign in to comment.