Experimental control-flow operator (#368)

* AST and syntax for delimited control

* stashing for later

* a little more progress

* progress on delimited continuations

* delimit, suspend, and resume implemented (draft)

* example that generates the natural numbers

* fixes

* tinkering

* changed demo to experimental

* comments and name changes

* describe delimited continuations in the README

* renamed Snapshot to Continuation, edits to comments

* Update executable_semantics/ast/statement.h

improve comment for MakeDelimitStmt

Co-authored-by: Dave Abrahams <dabrahams@google.com>

* Update executable_semantics/interpreter/interpreter.cpp

remove snake_case

Co-authored-by: Dave Abrahams <dabrahams@google.com>

* edits to comments, change name of variable

* updates to handle review edits

* trailing whitespace

* fixes to delimited continuations, added more tests, also fixed assignment to do a copy

* improvements from Geoffrey

* new test from Geoffrey, fix for empty blocks

* more suggestions from Geoffrey

* more tests for delimited continuations, renaming some of them

* renamed test files

* improve a comment

* sketch of creating continuation

* initial implementation of shift/reset style continuations

* more documentation

* fix some camel case

* implemented deep copy of continuations, added a test case for it

* fixed a bug and got the recursive test case working

* removed __delimit, polished up __continuation

* back to shallow copy for continuations

* suggestions from Geoffrey

* removed structured binding (for now)

* Update executable_semantics/ast/expression.cpp

Co-authored-by: Geoff Romer <gromer@google.com>

* responses to Geoffrey

Co-authored-by: Dave Abrahams <dabrahams@google.com>
Co-authored-by: Geoff Romer <gromer@google.com>

executable_semantics/BUILD

@@ -25,7 +25,10 @@ cc_library(
 )
 
 EXAMPLES = [
+    "assignment_copy1",
+    "assignment_copy2",
     "block1",
+    "block2",
     "break1",
     "choice1",
     "continue1",
@@ -64,6 +67,14 @@ EXAMPLES = [
     "tuple2",
     "while1",
     "zero",
+    "experimental_continuation1",
+    "experimental_continuation2",
+    "experimental_continuation3",
+    "experimental_continuation4",
+    "experimental_continuation5",
+    "experimental_continuation6",
+    "experimental_continuation7",
+    "experimental_continuation8",
 ]
 
 [genrule(

executable_semantics/README.md

@@ -57,6 +57,8 @@ interpreter.
 [InterpProgram()](interpreter/interpreter.h) runs an abstract machine using the
 [interpreter](interpreter/), as described below.
 
+## Abstract Machine
+
 The abstract machine implements a state-transition system. The state is defined
 by the `State` structure, which includes three components: the procedure call
 stack, the heap, and the function definitions. The `Step` function updates the
@@ -114,20 +116,98 @@ Let `n3` be the sum of `n1` and `n2`.
 
     n3 :: ...
 
-The heap is an array of values. It is used not only for `malloc` but also to
-store anything that is mutable, including function parameters and local
-variables. A pointer is simply an index into the array. The `malloc` expression
-causes the heap to grow (at the end) and returns the index of the last slot. The
-dereference expression returns the nth value of the heap, as specified by the
-dereferenced pointer. The assignment operation stores the value of the
-right-hand side into the heap at the index specified by the left-hand side
+The heap is an array of values. It is used to store anything that is mutable,
+including function parameters and local variables. An address is simply an index
+into the array. The assignment operation stores the value of the right-hand side
+into the heap at the index specified by the address of the left-hand side
 lvalue.
 
-As you might expect, function calls push a new frame on the stack and the
-`return` statement pops a frame off the stack. The parameter passing semantics
-is call-by-value, so the machine applies `CopyVal` to the incoming arguments and
-the outgoing return value. Also, the machine is careful to kill the parameters
-and local variables when the function call is complete.
+Function calls push a new frame on the stack and the `return` statement pops a
+frame off the stack. The parameter passing semantics is call-by-value, so the
+machine applies `CopyVal` to the incoming arguments and the outgoing return
+value. Also, the machine kills the values stored in the parameters and local
+variables when the function call is complete.
+
+## Experimental: Delimited Continuations
+
+Delimited continuations provide a kind of resumable exception with first-class
+continuations. The point of experimenting with this feature is not to say that
+we want delimited continuations in Carbon, but this represents a place-holder
+for other powerful control-flow features that might eventually be in Carbon,
+such as coroutines, threads, exceptions, etc. As we refactor the executable
+semantics, having this feature in place will keep us honest and prevent us from
+accidentally simplifying the interpreter to the point where it can't handle
+features like this one.
+
+Instead of delimited continuations, we could have instead done regular
+continuations with callcc. However, there seems to be a consensus amongst the
+experts that delimited continuations are better than regular ones.
+
+So what are delimited continuations? Recall that a continuation is a
+representation of what happens next in a computation. In the abstract machine,
+the procedure call stack represents the current continuation. A delimited
+continuation is also about what happens next, but it doesn't go all the way to
+the end of the execution. Instead it represents what happens up until control
+reaches the nearest enclosing `__continuation` statement.
+
+The statement
+
+    __continuation <identifier> <statement>
+
+creates a continuation object from the given statement and binds the
+continuation object to the given identifier. The given statement is not yet
+executed.
+
+The statement
+
+    __run <expression>;
+
+starts or resumes execution of the continuation object that results from the
+given expression.
+
+The statement
+
+    __await;
+
+pauses the current continuation, saving the control state in the continuation
+object. Control is then returned to the statement after the `__run` that
+initiated the current continuation.
+
+These three language features are demonstrated in the following example, where
+we create a continuation and bind it to `k`. We then run the continuation twice.
+The first time increments `x` to `1` and the second time increments `x` to `2`,
+so the expected result of this program is `2`.
+
+```carbon
+fn main() -> Int {
+  var Int: x = 0;
+  __continuation k {
+    x = x + 1;
+    __await;
+    x = x + 1;
+  }
+  __run k;
+  __run k;
+  return x;
+}
+```
+
+Note that the control state of the continuation object bound to `k` mutates as
+the program executes. Upon creation, the control state is at the beginning of
+the continuation. After the first `__run`, the control state is just after the
+`__await`. After the second `__run`, the control state is at the end of the
+continuation.
+
+The delimited continuation feature described here is based on the
+`shift`/`reset` style of delimited continuations created by Danvy and Filinsky
+(Abstracting control, ACM Conference on Lisp and Functional Programming, 1990).
+We adapted the feature to operate in a more imperative manner. The
+`__continuation` feature is equivalent to a `reset` followed immediately by a
+`shift` to pause and capture the continuation object. The `__run` feature is
+equivalent to calling the continuation. The `__await` feature is equivalent to a
+`shift` except that it updates the continuation in place.
+
+## Example Programs (Regression Tests)
 
 The [`testdata/`](testdata/) subdirectory includes some example programs with
 golden output.

executable_semantics/ast/expression.cpp

@@ -36,6 +36,14 @@ auto MakeAutoType(int line_num) -> Expression* {
   return t;
 }
 
+// Returns a Continuation type AST node at the given source location.
+auto MakeContinuationType(int line_num) -> Expression* {
+  auto* type = new Expression();
+  type->tag = ExpressionKind::ContinuationT;
+  type->line_num = line_num;
+  return type;
+}
+
 auto MakeFunType(int line_num, Expression* param, Expression* ret)
     -> Expression* {
   auto* t = new Expression();
@@ -150,6 +158,18 @@ auto MakeTuple(int line_num,
   return e;
 }
 
+// Create an AST node for an empty tuple.
+// TODO(geoffromer): remove this and rewrite its callers to use
+// `MakeTuple(line_num, {})`, once that works.
+auto MakeUnit(int line_num) -> Expression* {
+  auto* unit = new Expression();
+  unit->line_num = line_num;
+  unit->tag = ExpressionKind::Tuple;
+  auto* args = new std::vector<std::pair<std::string, Expression*>>();
+  unit->u.tuple.fields = args;
+  return unit;
+}
+
 auto MakeIndex(int line_num, Expression* exp, Expression* i) -> Expression* {
   auto* e = new Expression();
   e->line_num = line_num;
@@ -171,13 +191,13 @@ static void PrintOp(Operator op) {
       std::cout << "-";
       break;
     case Operator::Not:
-      std::cout << "!";
+      std::cout << "not";
       break;
     case Operator::And:
-      std::cout << "&&";
+      std::cout << "and";
       break;
     case Operator::Or:
-      std::cout << "||";
+      std::cout << "or";
       break;
     case Operator::Eq:
       std::cout << "==";
@@ -272,6 +292,9 @@ void PrintExp(const Expression* e) {
     case ExpressionKind::AutoT:
       std::cout << "auto";
       break;
+    case ExpressionKind::ContinuationT:
+      std::cout << "Continuation";
+      break;
     case ExpressionKind::FunctionT:
       std::cout << "fn ";
       PrintExp(e->u.function_type.parameter);

executable_semantics/ast/expression.h

@@ -19,6 +19,7 @@ enum class ExpressionKind {
   GetField,
   Index,
   IntT,
+  ContinuationT,  // The type of a continuation value.
   Integer,
   PatternVariable,
   PrimitiveOp,
@@ -100,6 +101,8 @@ auto MakeGetField(int line_num, Expression* exp, std::string field)
 auto MakeTuple(int line_num,
                std::vector<std::pair<std::string, Expression*>>* args)
     -> Expression*;
+// Create an AST node for an empty tuple.
+auto MakeUnit(int line_num) -> Expression*;
 auto MakeIndex(int line_num, Expression* exp, Expression* i) -> Expression*;
 
 auto MakeTypeType(int line_num) -> Expression*;
@@ -108,6 +111,9 @@ auto MakeBoolType(int line_num) -> Expression*;
 auto MakeFunType(int line_num, Expression* param, Expression* ret)
     -> Expression*;
 auto MakeAutoType(int line_num) -> Expression*;
+// Returns a Continuation type AST node at the given source location,
+// which is the type of a continuation value.
+auto MakeContinuationType(int line_num) -> Expression*;
 
 void PrintExp(const Expression* exp);
 

executable_semantics/ast/statement.cpp

@@ -104,6 +104,36 @@ auto MakeMatch(int line_num, Expression* exp,
   return s;
 }
 
+// Returns an AST node for a continuation statement give its line number and
+// parts.
+auto MakeContinuationStatement(int line_num, std::string continuation_variable,
+                               Statement* body) -> Statement* {
+  auto* continuation = new Statement();
+  continuation->line_num = line_num;
+  continuation->tag = StatementKind::Continuation;
+  continuation->u.continuation.continuation_variable =
+      new std::string(continuation_variable);
+  continuation->u.continuation.body = body;
+  return continuation;
+}
+
+// Returns an AST node for a run statement give its line number and argument.
+auto MakeRun(int line_num, Expression* argument) -> Statement* {
+  auto* run = new Statement();
+  run->line_num = line_num;
+  run->tag = StatementKind::Run;
+  run->u.run.argument = argument;
+  return run;
+}
+
+// Returns an AST node for an await statement give its line number.
+auto MakeAwait(int line_num) -> Statement* {
+  auto* await = new Statement();
+  await->line_num = line_num;
+  await->tag = StatementKind::Await;
+  return await;
+}
+
 void PrintStatement(Statement* s, int depth) {
   if (!s) {
     return;
@@ -177,13 +207,44 @@ void PrintStatement(Statement* s, int depth) {
       PrintStatement(s->u.sequence.stmt, depth);
       if (depth < 0 || depth > 1) {
         std::cout << std::endl;
+      } else {
+        std::cout << " ";
       }
       PrintStatement(s->u.sequence.next, depth - 1);
       break;
     case StatementKind::Block:
-      std::cout << "{" << std::endl;
-      PrintStatement(s->u.block.stmt, depth - 1);
-      std::cout << std::endl << "}" << std::endl;
+      std::cout << "{";
+      if (depth < 0 || depth > 1) {
+        std::cout << std::endl;
+      }
+      PrintStatement(s->u.block.stmt, depth);
+      if (depth < 0 || depth > 1) {
+        std::cout << std::endl;
+      }
+      std::cout << "}";
+      if (depth < 0 || depth > 1) {
+        std::cout << std::endl;
+      }
+      break;
+    case StatementKind::Continuation:
+      std::cout << "continuation " << *s->u.continuation.continuation_variable
+                << " ";
+      if (depth < 0 || depth > 1) {
+        std::cout << std::endl;
+      }
+      PrintStatement(s->u.continuation.body, depth - 1);
+      if (depth < 0 || depth > 1) {
+        std::cout << std::endl;
+      }
+      break;
+    case StatementKind::Run:
+      std::cout << "run ";
+      PrintExp(s->u.run.argument);
+      std::cout << ";";
+      break;
+    case StatementKind::Await:
+      std::cout << "await;";
+      break;
   }
 }
 }  // namespace Carbon

executable_semantics/ast/statement.h

@@ -22,7 +22,10 @@ enum class StatementKind {
   While,
   Break,
   Continue,
-  Match
+  Match,
+  Continuation,  // Create a first-class continuation.
+  Run,           // Run a continuation to the next await or until it finishes..
+  Await,         // Pause execution of the continuation.
 };
 
 struct Statement {
@@ -69,6 +72,15 @@ struct Statement {
       std::list<std::pair<Expression*, Statement*>>* clauses;
     } match_stmt;
 
+    struct {
+      std::string* continuation_variable;
+      Statement* body;
+    } continuation;
+
+    struct {
+      Expression* argument;
+    } run;
+
   } u;
 };
 
@@ -86,6 +98,22 @@ auto MakeContinue(int line_num) -> Statement*;
 auto MakeMatch(int line_num, Expression* exp,
                std::list<std::pair<Expression*, Statement*>>* clauses)
     -> Statement*;
+// Returns an AST node for a continuation statement give its line number and
+// contituent parts.
+//
+//     __continuation <continuation_variable> {
+//       <body>
+//     }
+auto MakeContinuationStatement(int line_num, std::string continuation_variable,
+                               Statement* body) -> Statement*;
+// Returns an AST node for a run statement give its line number and argument.
+//
+//     __run <argument>;
+auto MakeRun(int line_num, Expression* argument) -> Statement*;
+// Returns an AST node for an await statement give its line number.
+//
+//     __await;
+auto MakeAwait(int line_num) -> Statement*;
 
 void PrintStatement(Statement*, int);
 

executable_semantics/interpreter/interpreter.cpp

@@ -7,6 +7,7 @@
 #include <cassert>
 #include <iostream>
 #include <iterator>
+#include <list>
 #include <map>
 #include <optional>
 #include <utility>
@@ -83,6 +84,9 @@ auto CopyVal(const Value* val, int line_num) -> const Value* {
       return MakeFunVal(*val->u.fun.name, val->u.fun.param, val->u.fun.body);
     case ValKind::PtrV:
       return MakePtrVal(val->u.ptr);
+    case ValKind::ContinuationV:
+      // Copying a continuation is "shallow".
+      return val;
     case ValKind::FunctionTV:
       return MakeFunTypeVal(CopyVal(val->u.fun_type.param, line_num),
                             CopyVal(val->u.fun_type.ret, line_num));
@@ -99,6 +103,8 @@ auto CopyVal(const Value* val, int line_num) -> const Value* {
       return MakeVarTypeVal(*val->u.var_type);
     case ValKind::AutoTV:
       return MakeAutoTypeVal();
+    case ValKind::ContinuationTV:
+      return MakeContinuationTypeVal();
     case ValKind::TupleTV: {
       auto new_fields = new VarValues();
       for (auto& field : *val->u.tuple_type.fields) {
@@ -199,8 +205,10 @@ void PrintState(std::ostream& out) {
   PrintStack(state->stack, out);
   out << std::endl << "heap: ";
   PrintHeap(state->heap, out);
-  out << std::endl << "env: ";
-  PrintEnv(CurrentEnv(state), out);
+  if (!state->stack.IsEmpty() && !state->stack.Top()->scopes.IsEmpty()) {
+    out << std::endl << "env: ";
+    PrintEnv(CurrentEnv(state), out);
+  }
   out << std::endl << "}" << std::endl;
 }
 
@@ -242,6 +250,20 @@ auto ValToPtr(const Value* v, int line_num) -> Address {
   }
 }
 
+// Returns *continuation represented as a list of frames.
+//
+// - Precondition: continuation->tag == ValKind::ContinuationV.
+auto ContinuationToVector(const Value* continuation, int sourceLocation)
+    -> std::vector<Frame*> {
+  if (continuation->tag == ValKind::ContinuationV) {
+    return *continuation->u.continuation.stack;
+  } else {
+    std::cerr << sourceLocation << ": runtime error: expected an integer"
+              << std::endl;
+    exit(-1);
+  }
+}
+
 auto EvalPrim(Operator op, const std::vector<const Value*>& args, int line_num)
     -> const Value* {
   switch (op) {
@@ -278,7 +300,7 @@ void InitGlobals(std::list<Declaration>* fs) {
 auto ChoiceDeclaration::InitGlobals(Env& globals) const -> void {
   auto alts = new VarValues();
   for (auto kv : alternatives) {
-    auto t = ToType(line_num, InterpExp(Env(), kv.second));
+    auto t = ToType(this->line_num, InterpExp(Env(), kv.second));
     alts->push_back(make_pair(kv.first, t));
   }
   auto ct = MakeChoiceTypeVal(name, alts);
@@ -496,7 +518,7 @@ auto PatternMatch(const Value* p, const Value* v, Env env,
 void PatternAssignment(const Value* pat, const Value* val, int line_num) {
   switch (pat->tag) {
     case ValKind::PtrV:
-      state->heap[ValToPtr(pat, line_num)] = val;
+      state->heap[ValToPtr(pat, line_num)] = CopyVal(val, line_num);
       break;
     case ValKind::TupleV: {
       switch (val->tag) {
@@ -618,6 +640,7 @@ void StepLvalue() {
     case ExpressionKind::TypeT:
     case ExpressionKind::FunctionT:
     case ExpressionKind::AutoT:
+    case ExpressionKind::ContinuationT:
     case ExpressionKind::PatternVariable: {
       frame->todo.Pop();
       frame->todo.Push(MakeExpToLvalAct());
@@ -743,6 +766,12 @@ void StepExp() {
       act->pos++;
       break;
     }
+    case ExpressionKind::ContinuationT: {
+      const Value* v = MakeContinuationTypeVal();
+      frame->todo.Pop(1);
+      frame->todo.Push(MakeValAct(v));
+      break;
+    }
   }  // switch (exp->tag)
 }
 
@@ -826,10 +855,14 @@ void StepStmt() {
       break;
     case StatementKind::Block: {
       if (act->pos == -1) {
-        auto* scope = new Scope(CurrentEnv(state), std::list<std::string>());
-        frame->scopes.Push(scope);
-        frame->todo.Push(MakeStmtAct(stmt->u.block.stmt));
-        act->pos++;
+        if (stmt->u.block.stmt) {
+          auto* scope = new Scope(CurrentEnv(state), {});
+          frame->scopes.Push(scope);
+          frame->todo.Push(MakeStmtAct(stmt->u.block.stmt));
+          act->pos++;
+        } else {
+          frame->todo.Pop();
+        }
       } else {
         Scope* scope = frame->scopes.Top();
         KillScope(stmt->line_num, scope);
@@ -876,6 +909,43 @@ void StepStmt() {
       }
       frame->todo.Push(MakeStmtAct(stmt->u.sequence.stmt));
       break;
+    case StatementKind::Continuation: {
+      // Create a continuation object by creating a frame similar the
+      // way one is created in a function call.
+      Scope* scope = new Scope(CurrentEnv(state), std::list<std::string>());
+      Stack<Scope*> scopes;
+      scopes.Push(scope);
+      Stack<Action*> todo;
+      todo.Push(
+          MakeStmtAct(MakeReturn(stmt->line_num, MakeUnit(stmt->line_num))));
+      todo.Push(MakeStmtAct(stmt->u.continuation.body));
+      Frame* continuation_frame = new Frame("__continuation", scopes, todo);
+      Address continuation_address =
+          AllocateValue(MakeContinuation({continuation_frame}));
+      // Store the continuation's address in the frame.
+      continuation_frame->continuation = continuation_address;
+      // Bind the continuation object to the continuation variable
+      frame->scopes.Top()->env.Set(*stmt->u.continuation.continuation_variable,
+                                   continuation_address);
+      // Pop the continuation statement.
+      frame->todo.Pop();
+      break;
+    }
+    case StatementKind::Run:
+      // Evaluate the argument of the run statement.
+      frame->todo.Push(MakeExpAct(stmt->u.run.argument));
+      act->pos++;
+      break;
+    case StatementKind::Await:
+      // Pause the current continuation
+      frame->todo.Pop();
+      std::vector<Frame*> paused;
+      do {
+        paused.push_back(state->stack.Pop());
+      } while (!paused.back()->IsContinuation());
+      // Update the continuation with the paused stack.
+      state->heap[paused.back()->continuation] = MakeContinuation(paused);
+      break;
   }
 }
 
@@ -1156,6 +1226,7 @@ void HandleValue() {
         case ExpressionKind::BoolT:
         case ExpressionKind::TypeT:
         case ExpressionKind::AutoT:
+        case ExpressionKind::ContinuationT:
           std::cerr << "internal error, bad expression context in handle_value"
                     << std::endl;
           exit(-1);
@@ -1311,6 +1382,25 @@ void HandleValue() {
           frame->todo.Push(MakeValAct(ret_val));
           break;
         }
+        case StatementKind::Run: {
+          frame->todo.Pop(2);
+          // Push an expression statement action to ignore the result
+          // value from the continuation.
+          Action* ignore_result = MakeStmtAct(
+              MakeExpStmt(stmt->line_num, MakeUnit(stmt->line_num)));
+          ignore_result->pos = 0;
+          frame->todo.Push(ignore_result);
+          // Push the continuation onto the current stack.
+          std::vector<Frame*> continuation_vector =
+              ContinuationToVector(val_act->u.val, stmt->line_num);
+          for (auto frame_iter = continuation_vector.rbegin();
+               frame_iter != continuation_vector.rend(); ++frame_iter) {
+            state->stack.Push(*frame_iter);
+          }
+          break;
+        }
+        case StatementKind::Continuation:
+        case StatementKind::Await:
         case StatementKind::Block:
         case StatementKind::Sequence:
         case StatementKind::Break:

executable_semantics/interpreter/interpreter.h

@@ -29,13 +29,33 @@ struct Scope {
 
 /***** Frames and State *****/
 
+// A frame represents either a function call or a delimited continuation.
 struct Frame {
+  // The name of the function.
   std::string name;
+  // If the frame represents a function call, the bottom scope
+  // contains the parameter-argument bindings for this function
+  // call. The rest of the scopes contain local variables defined by
+  // blocks within the function. The scope at the top of the stack is
+  // the current scope and its environment is the one used for looking
+  // up the value associated with a variable.
   Stack<Scope*> scopes;
+  // The actions that need to be executed in the future of the
+  // current function call. The top of the stack is the action
+  // that is executed first.
   Stack<Action*> todo;
+  // If this frame is the bottom frame of a continuation, then it stores
+  // the address of the continuation.
+  // Otherwise the `continuation` field is the sentinel UINT_MAX.
+  Address continuation;
+  // Returns whether this frame is the bottom frame of a continuation.
+  auto IsContinuation() -> bool { return continuation != UINT_MAX; }
 
   Frame(std::string n, Stack<Scope*> s, Stack<Action*> c)
-      : name(std::move(std::move(n))), scopes(s), todo(c) {}
+      : name(std::move(std::move(n))),
+        scopes(s),
+        todo(c),
+        continuation(UINT_MAX) {}
 };
 
 struct State {
@@ -46,6 +66,8 @@ struct State {
 
 extern State* state;
 
+auto PrintFrame(Frame* frame, std::ostream& out) -> void;
+void PrintStack(Stack<Frame*> ls, std::ostream& out);
 void PrintEnv(Env env);
 auto AllocateValue(const Value* v) -> Address;
 auto CopyVal(const Value* val, int line_num) -> const Value*;

executable_semantics/interpreter/typecheck.cpp

@@ -76,6 +76,7 @@ auto ToType(int line_num, const Value* val) -> const Value* {
     case ValKind::BoolTV:
     case ValKind::IntTV:
     case ValKind::AutoTV:
+    case ValKind::ContinuationTV:
       return val;
     default:
       std::cerr << line_num << ": in ToType, expected a type, not ";
@@ -96,6 +97,8 @@ auto ReifyType(const Value* t, int line_num) -> Expression* {
       return MakeBoolType(0);
     case ValKind::TypeTV:
       return MakeTypeType(0);
+    case ValKind::ContinuationTV:
+      return MakeContinuationType(0);
     case ValKind::FunctionTV:
       return MakeFunType(0, ReifyType(t->u.fun_type.param, line_num),
                          ReifyType(t->u.fun_type.ret, line_num));
@@ -382,10 +385,15 @@ auto TypeCheckExp(Expression* e, TypeEnv env, Env ct_env, const Value* expected,
       }
     }
     case ExpressionKind::IntT:
+      return TCResult(e, MakeIntTypeVal(), env);
     case ExpressionKind::BoolT:
+      return TCResult(e, MakeBoolTypeVal(), env);
     case ExpressionKind::TypeT:
-    case ExpressionKind::AutoT:
       return TCResult(e, MakeTypeTypeVal(), env);
+    case ExpressionKind::AutoT:
+      return TCResult(e, MakeAutoTypeVal(), env);
+    case ExpressionKind::ContinuationT:
+      return TCResult(e, MakeContinuationTypeVal(), env);
   }
 }
 
@@ -500,7 +508,29 @@ auto TypeCheckStmt(Statement* s, TypeEnv env, Env ct_env,
       }
       return TCStatement(MakeReturn(s->line_num, res.exp), env);
     }
-  }
+    case StatementKind::Continuation: {
+      TCStatement body_result =
+          TypeCheckStmt(s->u.continuation.body, env, ct_env, ret_type);
+      Statement* new_continuation = MakeContinuationStatement(
+          s->line_num, *s->u.continuation.continuation_variable,
+          body_result.stmt);
+      env.Set(*s->u.continuation.continuation_variable,
+              MakeContinuationTypeVal());
+      return TCStatement(new_continuation, env);
+    }
+    case StatementKind::Run: {
+      TCResult argument_result = TypeCheckExp(s->u.run.argument, env, ct_env,
+                                              nullptr, TCContext::ValueContext);
+      ExpectType(s->line_num, "argument of `run`", MakeContinuationTypeVal(),
+                 argument_result.type);
+      Statement* new_run = MakeRun(s->line_num, argument_result.exp);
+      return TCStatement(new_run, env);
+    }
+    case StatementKind::Await: {
+      // nothing to do here
+      return TCStatement(s, env);
+    }
+  }  // switch
 }
 
 auto CheckOrEnsureReturn(Statement* stmt, bool void_return, int line_num)
@@ -547,6 +577,10 @@ auto CheckOrEnsureReturn(Statement* stmt, bool void_return, int line_num)
         return CheckOrEnsureReturn(stmt->u.sequence.stmt, void_return,
                                    stmt->line_num);
       }
+    case StatementKind::Continuation:
+    case StatementKind::Run:
+    case StatementKind::Await:
+      return stmt;
     case StatementKind::Assign:
     case StatementKind::ExpressionStatement:
     case StatementKind::While:

executable_semantics/interpreter/value.cpp

@@ -103,6 +103,15 @@ auto MakeAltCons(std::string alt_name, std::string choice_name)
   return v;
 }
 
+// Return a first-class continuation represented a fragment
+// of the stack.
+auto MakeContinuation(std::vector<Frame*> stack) -> Value* {
+  auto* v = new Value();
+  v->tag = ValKind::ContinuationV;
+  v->u.continuation.stack = new std::vector<Frame*>(stack);
+  return v;
+}
+
 auto MakeVarPatVal(std::string name, const Value* type) -> const Value* {
   auto* v = new Value();
   v->tag = ValKind::VarPatV;
@@ -136,6 +145,13 @@ auto MakeTypeTypeVal() -> const Value* {
   return v;
 }
 
+// Return a Continuation type.
+auto MakeContinuationTypeVal() -> const Value* {
+  auto* v = new Value();
+  v->tag = ValKind::ContinuationTV;
+  return v;
+}
+
 auto MakeAutoTypeVal() -> const Value* {
   auto* v = new Value();
   v->tag = ValKind::AutoTV;
@@ -255,6 +271,9 @@ void PrintValue(const Value* val, std::ostream& out) {
     case ValKind::AutoTV:
       out << "auto";
       break;
+    case ValKind::ContinuationTV:
+      out << "Continuation";
+      break;
     case ValKind::PointerTV:
       out << "Ptr(";
       PrintValue(val->u.ptr_type.type, out);
@@ -291,6 +310,14 @@ void PrintValue(const Value* val, std::ostream& out) {
     case ValKind::ChoiceTV:
       out << "choice " << *val->u.choice_type.name;
       break;
+    case ValKind::ContinuationV:
+      out << "continuation[[";
+      for (Frame* frame : *val->u.continuation.stack) {
+        PrintFrame(frame, out);
+        out << " :: ";
+      }
+      out << "]]";
+      break;
   }
 }
 
@@ -314,6 +341,7 @@ auto TypeEqual(const Value* t1, const Value* t2) -> bool {
       return FieldsEqual(t1->u.tuple_type.fields, t2->u.tuple_type.fields);
     case ValKind::IntTV:
     case ValKind::BoolTV:
+    case ValKind::ContinuationTV:
       return true;
     default:
       return false;

executable_semantics/interpreter/value.h

@@ -9,6 +9,7 @@
 #include <vector>
 
 #include "executable_semantics/ast/statement.h"
+#include "executable_semantics/interpreter/stack.h"
 
 namespace Carbon {
 
@@ -38,69 +39,94 @@ enum class ValKind {
   TupleTV,
   StructTV,
   ChoiceTV,
+  ContinuationTV,  // The type of a continuation.
   VarPatV,
-  AltConsV
+  AltConsV,
+  ContinuationV  // A first-class continuation value.
 };
 
+struct Frame;  // used by continuation
+
 struct Value {
   ValKind tag;
   union {
     int integer;
     bool boolean;
+
     struct {
       std::string* name;
       const Value* param;
       Statement* body;
     } fun;
+
     struct {
       const Value* type;
       const Value* inits;
     } struct_val;
+
     struct {
       std::string* alt_name;
       std::string* choice_name;
     } alt_cons;
+
     struct {
       std::string* alt_name;
       std::string* choice_name;
       Address argument;
     } alt;
+
     struct {
       std::vector<std::pair<std::string, Address>>* elts;
     } tuple;
+
     Address ptr;
     std::string* var_type;
+
     struct {
       std::string* name;
       const Value* type;
     } var_pat;
+
     struct {
       const Value* param;
       const Value* ret;
     } fun_type;
+
     struct {
       const Value* type;
     } ptr_type;
+
     struct {
       std::string* name;
       VarValues* fields;
       VarValues* methods;
     } struct_type;
+
     struct {
       std::string* name;
       VarValues* fields;
     } tuple_type;
+
     struct {
       std::string* name;
       VarValues* alternatives;
     } choice_type;
+
     struct {
       std::list<std::string*>* params;
       const Value* type;
     } implicit;
+
+    struct {
+      std::vector<Frame*>* stack;
+    } continuation;
+
   } u;
 };
 
+// Return a first-class continuation represented by the
+// given stack, down to the nearest enclosing `__continuation`.
+auto MakeContinuation(std::vector<Frame*> stack) -> Value*;
 auto MakeIntVal(int i) -> const Value*;
 auto MakeBoolVal(bool b) -> const Value*;
 auto MakeFunVal(std::string name, const Value* param, Statement* body)
@@ -117,6 +143,7 @@ auto MakeVarPatVal(std::string name, const Value* type) -> const Value*;
 
 auto MakeVarTypeVal(std::string name) -> const Value*;
 auto MakeIntTypeVal() -> const Value*;
+auto MakeContinuationTypeVal() -> const Value*;
 auto MakeAutoTypeVal() -> const Value*;
 auto MakeBoolTypeVal() -> const Value*;
 auto MakeTypeTypeVal() -> const Value*;

executable_semantics/syntax/lexer.lpp

@@ -45,6 +45,10 @@ TRUE              "true"
 TYPE              "Type"
 VAR               "var"
 WHILE             "while"
+CONTINUATION_TYPE "__Continuation"
+CONTINUATION      "__continuation"
+RUN               "__run"
+AWAIT             "__await"
 
 identifier    [A-Za-z_][A-Za-z0-9_]*
 integer_literal   [0-9]+
@@ -93,6 +97,10 @@ horizontal_whitespace [ \t\r]
 {TYPE}     { return yy::parser::make_TYPE(context.current_token_position); }
 {VAR}      { return yy::parser::make_VAR(context.current_token_position); }
 {WHILE}    { return yy::parser::make_WHILE(context.current_token_position); }
+{CONTINUATION_TYPE} { return yy::parser::make_CONTINUATION_TYPE(context.current_token_position); }
+{CONTINUATION}    { return yy::parser::make_CONTINUATION(context.current_token_position); }
+{RUN}      { return yy::parser::make_RUN(context.current_token_position); }
+{AWAIT}    { return yy::parser::make_AWAIT(context.current_token_position); }
 
 "=" return yy::parser::make_EQUAL(context.current_token_position);
 "-" return yy::parser::make_MINUS(context.current_token_position);

executable_semantics/syntax/parser.ypp

@@ -110,6 +110,10 @@ void yy::parser::error(
 %token IF
 %token ELSE
 %token WHILE
+%token CONTINUATION_TYPE
+%token CONTINUATION
+%token RUN
+%token AWAIT
 %token BREAK
 %token CONTINUE
 %token RETURN
@@ -180,6 +184,8 @@ expression:
     { $$ = Carbon::MakeTypeType(yylineno); }
 | AUTO
     { $$ = Carbon::MakeAutoType(yylineno); }
+| CONTINUATION_TYPE
+    { $$ = Carbon::MakeContinuationType(yylineno); }
 | paren_expression { $$ = $1; }
 | expression EQUAL_EQUAL expression
     { $$ = Carbon::MakeBinOp(yylineno, Carbon::Operator::Eq, $1, $3); }
@@ -287,6 +293,12 @@ statement:
     { $$ = Carbon::MakeBlock(yylineno, $2); }
 | MATCH "(" expression ")" "{" clause_list "}"
     { $$ = Carbon::MakeMatch(yylineno, $3, $6); }
+| CONTINUATION identifier statement
+    { $$ = Carbon::MakeContinuationStatement(yylineno, $2, $3); }
+| RUN expression ";"
+    { $$ = Carbon::MakeRun(yylineno, $2); }
+| AWAIT ";"
+    { $$ = Carbon::MakeAwait(yylineno); }
 ;
 optional_else:
   // Empty

executable_semantics/testdata/assignment_copy1.6c

@@ -0,0 +1,15 @@
+// Part of the Carbon Language project, under the Apache License v2.0 with LLVM
+// Exceptions. See /LICENSE for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+
+// Test that assignment performs a copy and does not create an alias.
+
+fn main() -> Int {
+  var Int: x = -1;
+  {
+     var Int: y = 0;
+     x = y;
+     // y dies here
+  }
+  return x;
+}

executable_semantics/testdata/assignment_copy1.golden

@@ -0,0 +1 @@
+result: 0

executable_semantics/testdata/assignment_copy2.6c

@@ -0,0 +1,12 @@
+// Part of the Carbon Language project, under the Apache License v2.0 with LLVM
+// Exceptions. See /LICENSE for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+
+// Test that assignment performs a copy and does not create an alias.
+
+fn main() -> Int {
+  var Int: x = 0;
+  var Int: y = x;
+  x = 1;
+  return y;
+}

executable_semantics/testdata/assignment_copy2.golden

@@ -0,0 +1 @@
+result: 0

executable_semantics/testdata/block2.6c

@@ -0,0 +1,11 @@
+// Part of the Carbon Language project, under the Apache License v2.0 with LLVM
+// Exceptions. See /LICENSE for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+
+fn main() -> Int {
+  var Int: x = 0;
+  {
+    // empty block
+  }
+  return x;
+}

executable_semantics/testdata/block2.golden

@@ -0,0 +1 @@
+result: 0

executable_semantics/testdata/experimental_continuation1.6c

@@ -0,0 +1,13 @@
+// Part of the Carbon Language project, under the Apache License v2.0 with LLVM
+// Exceptions. See /LICENSE for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+
+// Test that creating a continuation doesn't do anything.
+
+fn main() -> Int {
+  var Int: x = 0;
+  __continuation k {
+    x = x + 1;
+  }
+  return x;
+}

executable_semantics/testdata/experimental_continuation1.golden

@@ -0,0 +1 @@
+result: 0

executable_semantics/testdata/experimental_continuation2.6c

@@ -0,0 +1,14 @@
+// Part of the Carbon Language project, under the Apache License v2.0 with LLVM
+// Exceptions. See /LICENSE for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+
+// Test creating and running a continuation.
+
+fn main() -> Int {
+  var Int: x = 0;
+  __continuation k {
+    x = x + 1;
+  }
+  __run k;
+  return x;
+}

executable_semantics/testdata/experimental_continuation2.golden

@@ -0,0 +1 @@
+result: 1

executable_semantics/testdata/experimental_continuation3.6c

@@ -0,0 +1,18 @@
+// Part of the Carbon Language project, under the Apache License v2.0 with LLVM
+// Exceptions. See /LICENSE for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+
+// Test pausing a continuation with `__await` and restarting it with
+// `__run`.
+
+fn main() -> Int {
+  var Int: x = 0;
+  __continuation k {
+    x = x + 1;
+    __await;
+    x = x + 2;
+  }
+  __run k;
+  __run k;
+  return x;
+}

executable_semantics/testdata/experimental_continuation3.golden

@@ -0,0 +1 @@
+result: 3

executable_semantics/testdata/experimental_continuation4.6c

@@ -0,0 +1,19 @@
+// Part of the Carbon Language project, under the Apache License v2.0 with LLVM
+// Exceptions. See /LICENSE for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+
+// Assignment for continuations is shallow, so `k2` refers to the same
+// continuation as `k1`.
+
+fn main() -> Int {
+  var Int: x = 0;
+  __continuation k1 {
+    x = x + 1;
+    __await;
+    x = x + 2;
+  }
+  var __Continuation: k2 = k1;
+  __run k1;
+  __run k2;
+  return x;
+}

executable_semantics/testdata/experimental_continuation4.golden

@@ -0,0 +1 @@
+result: 3

executable_semantics/testdata/experimental_continuation5.6c

@@ -0,0 +1,19 @@
+// Part of the Carbon Language project, under the Apache License v2.0 with LLVM
+// Exceptions. See /LICENSE for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+
+// Test access to block-scoped variables upon resuming a continuation.
+
+fn main() -> Int {
+  var Int: y = 0;
+  __continuation k {
+    var Int: x = 0;
+    x = x + 1;
+    __await;
+    x = x + 2;
+    y = x;
+  }
+  __run k;
+  __run k;
+  return y;
+}

executable_semantics/testdata/experimental_continuation5.golden

@@ -0,0 +1 @@
+result: 3

executable_semantics/testdata/experimental_continuation6.6c

@@ -0,0 +1,33 @@
+// Part of the Carbon Language project, under the Apache License v2.0 with LLVM
+// Exceptions. See /LICENSE for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+
+// Test recursive functions inside continuations.
+
+var Int: current = 0;
+
+fn CountUpTo(Int: x) -> Int {
+  if (x == 0) {
+    current = 0;
+    __await;
+    return 0;
+  } else {
+    current = 1 + CountUpTo(x - 1);
+    __await;
+    return current;
+  }
+}
+
+fn main() -> Int {
+  __continuation k {
+    CountUpTo(5);
+  }
+  var Int: sum = 0;
+  var Int: count = 5;
+  while (not (count == 0)) {
+    __run k;
+    sum = sum + current;
+    count = count - 1;
+  }
+  return sum;
+}

executable_semantics/testdata/experimental_continuation6.golden

@@ -0,0 +1 @@
+result: 10

executable_semantics/testdata/experimental_continuation7.6c

@@ -0,0 +1,23 @@
+// Part of the Carbon Language project, under the Apache License v2.0 with LLVM
+// Exceptions. See /LICENSE for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+
+// Test the way in which copying of continuations interacts with data
+// on the stack such as the variable `x`. In this example the copy
+// happens after the variable `x` is created.
+
+var Int: y = 0;
+
+fn main() -> Int {
+  __continuation k1 {
+    var Int: x = 0;
+    x = x + 1;
+    __await;
+    x = x + 2;
+    y = x;
+  }
+  __run k1;
+  var __Continuation: k2 = k1;
+  __run k2;
+  return y;
+}

executable_semantics/testdata/experimental_continuation7.golden

@@ -0,0 +1 @@
+result: 3

executable_semantics/testdata/experimental_continuation8.6c

@@ -0,0 +1,25 @@
+// Part of the Carbon Language project, under the Apache License v2.0 with LLVM
+// Exceptions. See /LICENSE for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+
+// Test the way in which copying of continuations interacts with data
+// on the stack such as the variable `x`. In this example the copy
+// happens before the variable `x` is created, so each continuation
+// creates a different `x`.
+
+var Int: y = 0;
+
+fn main() -> Int {
+  __continuation k1 {
+    var Int: x = 0;
+    x = x + 1;
+    __await;
+    y = x;
+  }
+  var __Continuation: k2 = k1;
+  __run k1;
+  __run k2;
+  __run k1;
+  __run k2;
+  return y;
+}

executable_semantics/testdata/experimental_continuation8.golden

@@ -0,0 +1 @@
+result: 1