Roadmap for 2024 and a retrospective for 2023 (#3564)

We propose a roadmap for 2024 focused on a working Carbon toolchain that
supports Carbon ↔ C++ interop.

---------

Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
Co-authored-by: Richard Smith <richard@metafoo.co.uk>

README.md

@@ -149,18 +149,16 @@ and the language:
 -   A prototype interpreter demo that can both run isolated examples and gives a
     detailed analysis of the specific semantic model and abstract machine of
     Carbon. We call this the [Carbon Explorer](/explorer/).
--   Early components of a full [compiler and toolchain](/toolchain/) that can
-    compile Carbon (and eventually C++ code as well) into standard executable
-    code. This is where most of our current implementation efforts are directed.
-
-If you're interested in contributing, we would love help
-[completing the 0.1 language designs](/docs/project/roadmap.md#complete-design-coverage-of-the-01-languages-necessary-features),
-and
-[completing the Carbon Toolchain implementation of this design](/docs/project/roadmap.md#a-toolchain-that-can-build-a-minimal-mixed-c-and-carbon-program).
-We are also currently working to get more broad feedback and participation from
-the C++ community. Beyond that, we plan to prioritize C++ interoperability and a
-realistic toolchain that implements the 0.1 language and can be used to evaluate
-Carbon in more detail.
+-   An under-development [compiler and toolchain](/toolchain/) that will compile
+    Carbon (and eventually C++ code as well) into standard executable code. This
+    is where most of our current implementation efforts are directed.
+
+If you're interested in contributing, we're currently focused on
+[developing the Carbon toolchain until it can support Carbon ↔ C++ interop](/docs/project/roadmap.md#objective-for-2024-a-working-toolchain-that-supports-c-interop).
+Beyond that, we plan to continue developing the design and toolchain until we
+can ship the
+[0.1 language](/docs/project/milestones.md#milestone-01-a-minimum-viable-product-mvp-for-evaluation)
+and support evaluating Carbon in more detail.
 
 You can see our [full roadmap](/docs/project/roadmap.md) for more details.
 

docs/project/roadmap.md

@@ -10,120 +10,133 @@ SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 
 ## Table of contents
 
--   [Objective for 2023: get ready to evaluate the Carbon Language.](#objective-for-2023-get-ready-to-evaluate-the-carbon-language)
--   [Key results in 2023](#key-results-in-2023)
-    -   [A concrete definition of our Minimum Viable Product for evaluation, the 0.1 language](#a-concrete-definition-of-our-minimum-viable-product-for-evaluation-the-01-language)
-    -   [Complete design coverage of the 0.1 language's necessary features](#complete-design-coverage-of-the-01-languages-necessary-features)
-    -   [Complete 0.1 language implementation coverage in the Carbon Explorer](#complete-01-language-implementation-coverage-in-the-carbon-explorer)
-    -   [A toolchain that can build a minimal mixed C++ and Carbon program](#a-toolchain-that-can-build-a-minimal-mixed-c-and-carbon-program)
+-   [Objective for 2024: a working toolchain that supports C++ interop](#objective-for-2024-a-working-toolchain-that-supports-c-interop)
+-   [Key results in 2024](#key-results-in-2024)
+    -   [Carbon's toolchain implements enough of the language to build realistic code](#carbons-toolchain-implements-enough-of-the-language-to-build-realistic-code)
+    -   [Carbon's toolchain can build C++ code](#carbons-toolchain-can-build-c-code)
+    -   [Carbon's toolchain works with existing, simple C++ build systems](#carbons-toolchain-works-with-existing-simple-c-build-systems)
+    -   [Carbon has a design and toolchain implementation of basic C++ interop](#carbon-has-a-design-and-toolchain-implementation-of-basic-c-interop)
     -   [Give talks at 2-3 conferences covering 3-4 different Carbon topics](#give-talks-at-2-3-conferences-covering-3-4-different-carbon-topics)
--   [Beyond 2023](#beyond-2023)
-    -   [Potential 2024 goals: ship a working 0.1 language for evaluation](#potential-2024-goals-ship-a-working-01-language-for-evaluation)
-    -   [Potential 2025-2026 goals: finish 0.2 language, stop experimenting](#potential-2025-2026-goals-finish-02-language-stop-experimenting)
-    -   [Potential goals _beyond_ 2026: ship 1.0 language & organization](#potential-goals-beyond-2026-ship-10-language--organization)
+    -   [Start building our initial tutorial and introductory material](#start-building-our-initial-tutorial-and-introductory-material)
+-   [Beyond 2024](#beyond-2024)
+    -   [Potential 2025 goals: ship a working 0.1 language for evaluation](#potential-2025-goals-ship-a-working-01-language-for-evaluation)
+    -   [Potential 2026-2027 goals: finish 0.2 language, stop experimenting](#potential-2026-2027-goals-finish-02-language-stop-experimenting)
+    -   [Potential goals _beyond_ 2027: ship 1.0 language & organization](#potential-goals-beyond-2027-ship-10-language--organization)
 
 <!-- tocstop -->
 
-## Objective for 2023: get ready to evaluate the Carbon Language.
+## Objective for 2024: a working toolchain that supports C++ interop
 
-Our focus throughout 2023 will be to get the Carbon Language and project ready
-for serious evaluation of the experiment. There are two aspects to this
-evaluation:
+Our focus for 2024 will be to get the Carbon toolchain working, including C++
+interop. We see three key criteria:
 
-1. The language and tools need to be complete enough to evaluate.
-2. The users and communities we are targeting need both context and awareness of
-   the technical ideas and design principles on which Carbon is built.
+-   Building realistic Carbon code for interesting interop with C++.
+-   Building realistic C++ code for interesting interop with Carbon.
+-   The interop itself to allow a single program mixing the two languages.
 
-## Key results in 2023
+This will both allow folks to explore Carbon using a more traditional and
+realistic compiler model, and allow that exploratory Carbon to lean on C++ for
+libraries and other functionality that doesn't exist in Carbon yet. It will also
+demonstrate how the interop will work in practice.
 
-### A concrete definition of our Minimum Viable Product for evaluation, the 0.1 language
+This objective and focus are oriented around the toolchain and implementation of
+Carbon. We still expect some work on language design, but for its priority to be
+driven largely as a function of being in the critical path of some aspect of our
+implementation work.
 
-While we have talked about our 0.1 language, or our Minimum Viable Product (MVP)
-for evaluation purposes, we need to pin down exactly what this includes. We need
-concrete milestones that need to be reached for us to have confidence in
-potential users and communities being able to evaluate Carbon as a successor to
-C++.
+## Key results in 2024
 
-We expect this to include a reasonably precise set of requirements across:
+### Carbon's toolchain implements enough of the language to build realistic code
 
--   Necessary language features
--   Nice-to-have features
--   Features we can omit and reasonably expect to not obstruct credible
-    evaluation
--   Implementation coverage of those features in the Carbon Explorer to validate
-    the design
--   Implementation coverage of those features in the toolchain
--   Quality of implementation in the toolchain, both in general and for specific
-    features
--   Necessary documentation, strategy, or other supporting material
+This goal is not necessarily about complete support for the entire language
+design, but rather enough of it to support building the realistic and
+interesting Carbon code that interoperates with C++.
 
-We expect this to also include at least enough of Carbon's features and C++
-interop to support [our basic example](/docs/images/snippets.md#mixed).
+Some example language features that we think are key to success here but far
+from an exhaustive list:
 
-Note that we don't expect to finish implementing the 0.1 design in 2023. Our
-goal is to make sufficient progress that we can complete the implementation in
-2024, but there are still many things that can go wrong and cause significant
-delays.
+-   Imports and a working [prelude] (the earliest stages of a standard library)
+-   Operator overloading and dispatch for expressions
+-   Generic types and functions
+-   Templates (likely only partial support and focused on interop use cases)
 
-### Complete design coverage of the 0.1 language's necessary features
+[prelude]: /docs/design#name-lookup-for-common-types
 
-This year we plan to finish the design of the necessary feature-set that we
-define above for the 0.1 language.
+### Carbon's toolchain can build C++ code
 
-### Complete 0.1 language implementation coverage in the Carbon Explorer
+We need the toolchain to be able to build C++ code as if it were Clang in order
+to build the C++ code that Carbon is interoperating with. This isn't about
+building anything new or novel, but about packaging and exposing Clang for this
+purpose.
 
-We expect to complete the level of Carbon Explorer validation of the design
-needed for 0.1 in 2023 so that we have high confidence in the design's cohesion
-and behavior.
+### Carbon's toolchain works with existing, simple C++ build systems
 
-### A toolchain that can build a minimal mixed C++ and Carbon program
+We should be able to drop Carbon's toolchain into at least simple `Makefile` or
+CMake build systems as a replacement for the C++ toolchain and provide a Carbon
+toolchain. This doesn't include supporting everything or even moderately complex
+builds; only the simplest of builds using these build systems need to work at
+first.
+
+### Carbon has a design and toolchain implementation of basic C++ interop
 
 Our end goal is to compile a minimal but non-trivial example of bi-directionally
-mixing C++ and Carbon code such as our main
-[example](https://github.com/carbon-language/carbon-lang/blob/trunk/docs/images/snippets.md#mixed)
-and run it successfully. However, completing everything involved in this example
-isn't expected to be realistic by the end of the year. We expect to work towards
-this example and in rough priority order across the following interop features
-and all the Carbon features they depend on:
+mixing C++ and Carbon code such as our main example and run it successfully.
+However, completing everything involved in this example isn't expected to be
+realistic by the end of the year. We expect to work towards this example and in
+rough priority order across the following interop features and all the Carbon
+features they depend on:
 
 -   Calling C++ functions from Carbon.
 -   Importing concrete C++ types as Carbon types.
--   Using Carbon generics with a C++ type in Carbon.
+-   (stretch) Using Carbon generics with a C++ type in Carbon.
 -   (stretch) Calling Carbon functions from C++.
 -   (stretch) Importing concrete Carbon types into C++.
 
 ### Give talks at 2-3 conferences covering 3-4 different Carbon topics
 
-We want to engage both more broadly and in more depth with the C++ community as
-the Carbon 0.1 language becomes increasingly concrete. This should help set the
-stage for evaluations of Carbon when 0.1 is finished and available. To broaden
-our engagement, we want to give talks at 2-3 conferences spanning 2-3 geographic
-regions.
+We want to continue to engage with the external C++ community as the Carbon
+toolchain becomes a more real and complete toolchain. We specifically want to
+share when interop becomes something people can experiment with and explore.
+
+### Start building our initial tutorial and introductory material
+
+Because of the nature of Carbon's experiment, the tutorial and introductory
+material won't be focused on typical teaching of the language to general
+developers. Instead, it will be focused on enabling C++ developers to start
+evaluating specific aspects of Carbon for interoperating with existing C++
+codebases.
 
-We also want these talks to provide the C++ community a deeper understanding of
-the Carbon language and project, spanning 3-4 different topics. For example, we
-might share the details of the language design, governance, and implementation.
+We only expect to _start_ building this material in 2024. We want to learn what
+any critical gaps are for folks to start evaluating C++ interop and how best to
+close them going into 2025.
 
-## Beyond 2023
+## Beyond 2024
 
 Longer term goals are hard to pin down and always subject to change, but we want
 to give an idea of what kinds of things are expected at a high level further out
-in order to illustrate how the goals and priorities we have in 2023 feed into
+in order to illustrate how the goals and priorities we have in 2024 feed into
 subsequent years.
 
-### Potential 2024 goals: ship a working 0.1 language for evaluation
+### Potential 2025 goals: ship a working [0.1 language] for evaluation
+
+[0.1 language]:
+    /docs/project/milestones.md#milestone-01-a-minimum-viable-product-mvp-for-evaluation
 
 As we adjust our schedule and roadmap to reflect the realistic rate of progress,
 the _earliest_ it seems feasible to have everything we need to evaluate the 0.1
-language is 2024. However, this is just a lower bound. We may discover as we
-progress things that further push out the schedule here. That is the nature of
-an experimental project like Carbon.
+language is 2025. We're starting to be optimistic in 2024 that we'll be able to
+hit this in 2025, but ultimately this remains a lower bound. We may discover as
+we progress things that further push out the schedule here. That is the nature
+of an experimental project like Carbon.
 
 We expect that once we reach this milestone the community will be able to start
 realistically evaluating Carbon as a C++ successor language. Of course, this
 evaluation will take some time.
 
-### Potential 2025-2026 goals: finish 0.2 language, stop experimenting
+### Potential 2026-2027 goals: finish [0.2 language], stop experimenting
+
+[0.2 language]:
+    /docs/project/milestones.md#milestone-02-feature-complete-product-for-evaluation
 
 Once Carbon is moving quickly and getting public feedback, we should be able to
 conclude the experiment. We should know if this is the right direction for
@@ -146,7 +159,10 @@ Some concrete goals that might show up in this time frame:
 -   Create a foundation or similar organization to manage the Carbon project,
     separate from any corporate entities that fund work on Carbon.
 
-### Potential goals _beyond_ 2026: ship 1.0 language & organization
+### Potential goals _beyond_ 2027: ship [1.0 language] & organization
+
+[1.0 language]:
+    /docs/project/milestones.md#milestone-10-no-longer-an-experiment-usable-in-production
 
 A major milestone will be the first version of a production language. We also
 plan to finish transferring all governance of Carbon to an independent open

proposals/p3564.md

@@ -0,0 +1,106 @@
+# Roadmap for 2024 and a retrospective for 2023
+
+<!--
+Part of the Carbon Language project, under the Apache License v2.0 with LLVM
+Exceptions. See /LICENSE for license information.
+SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+-->
+
+[Pull request](https://github.com/carbon-language/carbon-lang/pull/3564)
+
+<!-- toc -->
+
+## Table of contents
+
+-   [Abstract](#abstract)
+-   [Proposal](#proposal)
+-   [Retrospective on 2023](#retrospective-on-2023)
+
+<!-- tocstop -->
+
+## Abstract
+
+We propose a roadmap for 2024 focused on a working Carbon toolchain that
+supports Carbon ↔ C++ interop.
+
+## Proposal
+
+We propose the following goals for 2024:
+
+-   Implementing enough in the toolchain to build meaningful code on the Carbon
+    side of interop.
+-   Teaching the toolchain to compile enough C++ code (by way of Clang) to build
+    meaningful code on the C++ side of interop.
+-   Implementing the interop itself between Carbon and C++ code.
+-   Sharing various aspects of this with the broader Carbon, C++, and open
+    source communities.
+
+This focuses significantly more on implementation than we have done in previous
+years, which is a response to feedback that folks looking at Carbon and trying
+to evaluate or contribute are increasingly blocked on the lack of implementation
+support -- either of language features or compiler functionality. We also think
+the design is far enough along to support this level of implementation, making
+it an effective time to adopt this more focused roadmap.
+
+The full details are in the updated [roadmap] document.
+
+[roadmap]: /docs/project/roadmap.md
+
+## Retrospective on 2023
+
+Our [roadmap for 2023] focused on getting both the language and tools ready for
+evaluation, and building of context in the C++ community to support that
+evaluation. We executed on this really well, but with mixed results.
+
+[Roadmap for 2023]:
+    https://github.com/carbon-language/carbon-lang/blob/840cb1bed7cf9bd57e000cb4a61e986c383d3038/docs/project/roadmap.md
+
+On getting ready for evaluation, we made fantastic progress on getting the
+language (design) ready. We have [milestone definitions], and closed the most
+critical gaps in the design from the start of the year. The remaining gaps are
+either lower risk, almost finished, or really need interop to effectively
+explore.
+
+[milestone definitions]: /docs/project/milestones.md
+
+The tooling side also made fantastic progress, but has a lot further to go and
+forms the focus for 2024.
+
+We also engaged more deeply with the C++ community on the technical
+underpinnings, but the response highlighted that until we have (much) more
+complete tooling, we're going to struggle to deepen our engagement with the
+external community, and especially struggle to broaden that engagement to larger
+communities.
+
+Both the amount remaining on the tooling side of things, and the fact that it
+has proved a limiting factor in external engagement, are driving our priorities
+for this year.
+
+Breaking down the specific key results we aimed at for 2023:
+
+-   A concrete definition of our MVP for evaluation, the 0.1 language
+    -   [Done][mvp-milestone]!
+-   Complete design coverage of the 0.1 language's necessary features
+    -   High risk, non-interop features are mostly done! The rest is taking a
+        back seat to getting the toolchain & interop ready.
+-   Complete 0.1 language implementation in the Carbon Explorer
+    -   There was nuance in this result: we only needed to complete as much
+        design validation as needed to have confidence in the design's cohesion
+        and behavior. We think the explorer got there in 2023 and have
+        [pivoted][toolchain-pivot] fully to the toolchain, so this is somewhat
+        by construction done.
+-   A toolchain that can build a minimal mixed C++ and Carbon program
+    -   As called out in the 2023 roadmap, we didn't expect to be able to
+        complete this by the end of 2023, and indeed, this has turned into our
+        focus for 2024.
+-   Give talks at 2-3 conferences covering 3-4 different Carbon topics
+    -   We only got to one conference, but we did cover 3+ topics this year.
+        We're hoping as we have a more complete toolchain we'll have more
+        relevant topics for more conferences.
+
+[mvp-milestone]:
+    /docs/project/milestones.md#milestone-01-a-minimum-viable-product-mvp-for-evaluation
+[toolchain-pivot]: /proposals/p3532.md
+
+Overall, 2023 was pretty amazing, and we hit some of the most important
+milestones in the project over the past year.