Switch to a brew version of python3.9 (#778)

- brew instead of pyenv to centralize package management
  - can't recall why we used pyenv before, may not have been the best choice.
  - pyenv has been a burden in updating versions, a single `brew upgrade` works better
- python3.9 because it's the latest and greatest, feels weird going to old versions if we're recommending installs.
- bazel 5.0.0 pre-release due to https://github.com/bazelbuild/bazel/commit/b9fc66d327debcfbdb2964afdba35a1cc8919b81
  - earlier versions want `python` to be on the path, which pyenv did but brew does not

For most people this will mean: `rm -rf ~/.pyenv && brew uninstall pyenv && brew install python@3.9 && pip3 install -U pip && pip3 install pre-commit`, plus removing any `pyenv` invocations from the shell `rc` file.

Co-authored-by: Chandler Carruth <chandlerc@gmail.com>

.bazeliskrc

@@ -4,4 +4,6 @@
 
 # Keep pinned to a recent release, listed at
 # https://github.com/bazelbuild/bazel.
-USE_BAZEL_VERSION=4.1.0
+# TODO: Temporarily using pre-release versions due to a python3 fix. This can
+# switch to stable versions post-release.
+USE_BAZEL_VERSION=5.0.0-pre.20210810.4

.github/workflows/tests.yaml

@@ -51,7 +51,7 @@ jobs:
       - uses: actions/setup-python@v2
         with:
           # Match the min version listed in docs/project/contribution_tools.md
-          python-version: '3.6'
+          python-version: '3.9'
 
       # On macOS we need Go and to use it to install Bazelisk.
       - uses: actions/setup-go@v2

docs/project/contribution_tools.md

@@ -17,7 +17,7 @@ contributions.
 -   [Package managers](#package-managers)
     -   [Linux and MacOS](#linux-and-macos)
         -   [Homebrew](#homebrew)
-        -   [Python using `pyenv`](#python-using-pyenv)
+        -   [`python3` and `pip3`](#python3-and-pip3)
     -   [Linux only](#linux-only)
         -   [`go get`](#go-get)
         -   [Cargo (optional)](#cargo-optional)
@@ -89,37 +89,29 @@ Our recommended way of installing is to run
 To get the latest version of `brew` packages, it will be necessary to
 periodically run `brew upgrade`.
 
-#### Python using `pyenv`
+#### `python3` and `pip3`
 
-Carbon requires Python 3.6 or newer. Everything below assumes that `python` or
-`pip` reach the Python 3 tools, not legacy installations of Python 2.
+Carbon requires Python 3.9 or newer. The included `pip3` should typically be
+used for Python package installation rather than other package managers.
 
-We strongly recommend using [pyenv](https://github.com/pyenv/pyenv) to manage
-[Python](python.org) and Python's `pip` package manager. `pip` should typically
-be used for Python package installation rather than other package managers.
+**NOTE**: Carbon will focus support on Homebrew installs of Python 3.9, but it
+may not be necessary if you have Python 3.9 installed another way. If you're
+trying to use a non-Homebrew Python but have issues involving Carbon and Python,
+please try Homebrew's Python.
 
 Our recommended way of installing is:
 
 ```bash
-brew install pyenv
-pyenv install 3.8.5
-pyenv global 3.8.5
+brew install python@3.9
+pip3 install -U pip
 ```
 
-You will also need to update your rc file to add pyenv to your `PATH`; this
-should look like:
+**NOTE**: `pip3` runs may print deprecation warnings referencing
+https://github.com/Homebrew/homebrew-core/issues/76621. These will need to be
+addressed in the future, but as of August 2021 can be ignored.
 
-```bash
-export PYENV_ROOT="$HOME/.pyenv"
-export PATH="$PYENV_ROOT/bin:$PATH"
-eval "$(pyenv init --path)"
-```
-
-Restart the shell (for example, `exec $SHELL`) to get `PATH` updates. If there
-are still issues, check instructions using `pyenv init`.
-
-To get the latest version of `pip` packages, it will be necessary to
-periodically run `pip list --outdated`, then `pip install -U <package>` to
+To get the latest version of `pip3` packages, it will be necessary to
+periodically run `pip3 list --outdated`, then `pip3 install -U <package>` to
 upgrade desired packages. Keep in mind when upgrading that version dependencies
 may mean packages _should_ be outdated, and not be upgraded.
 
@@ -251,7 +243,7 @@ important checks, including formatting.
 Our recommended way of installing is:
 
 ```bash
-pip install pre-commit
+pip3 install pre-commit
 
 # From within each carbon-language git repository:
 pre-commit install

executable_semantics/tests.py

@@ -132,20 +132,6 @@ def _update_golden(test):
 
 def _update_goldens():
     """Runs bazel to update golden files."""
-    # This should typically be called through pyenv due to the shebang. However,
-    # pyenv then modifies the PATH, which affects build caching. In order to
-    # mimic the calling environment for bazel, this strips out PATH entries
-    # which pyenv likely added.
-    # TODO: remove this when/if we're able to add
-    # `--incompatible_strict_action_env=true` to the project .bazelrc, because
-    # that will cause Bazel to ignore PATH.
-    env = os.environ.copy()
-    stripped_path = []
-    for x in env["PATH"].split(":"):
-        if not ("/Cellar/pyenv/" in x or "/.pyenv/versions/" in x):
-            stripped_path.append(x)
-    env["PATH"] = ":".join(stripped_path)
-
     # Load tests from the bzl file. This isn't done through os.listdir because
     # building new tests requires --update_list.
     bzl_content = open(_TEST_LIST_BZL).read()
@@ -159,7 +145,6 @@ def _update_goldens():
             "build",
             "//executable_semantics:golden_tests",
         ],
-        env=env,
     )
 
     print("Updating %d goldens..." % len(tests))

scripts/create_compdb.py

@@ -46,20 +46,6 @@ if not bazel:
         if not shutil.which(bazel):
             sys.exit("Unable to run Bazel")
 
-# This should typically be called through pyenv due to the shebang. However,
-# pyenv then modifies the PATH, which affects build caching. In order to
-# mimic the calling environment for bazel, this strips out PATH entries
-# which pyenv likely added.
-# TODO: remove this when/if we're able to add
-# `--incompatible_strict_action_env=true` to the project .bazelrc, because
-# that will cause Bazel to ignore PATH.
-env = os.environ.copy()
-stripped_path = []
-for x in env["PATH"].split(":"):
-    if not ("/Cellar/pyenv/" in x or "/.pyenv/versions/" in x):
-        stripped_path.append(x)
-env["PATH"] = ":".join(stripped_path)
-
 # Load compiler flags. We do this first in order to fail fast if not run from
 # the workspace root.
 print("Reading the arguments to use...")
@@ -93,7 +79,6 @@ source_files_query = subprocess.run(
     stdout=subprocess.PIPE,
     stderr=subprocess.DEVNULL,
     universal_newlines=True,
-    env=env,
 ).stdout
 source_files = [
     Path(line.split(":")[0]) for line in source_files_query.splitlines()
@@ -135,7 +120,6 @@ generated_file_labels = subprocess.run(
     stdout=subprocess.PIPE,
     stderr=subprocess.DEVNULL,
     universal_newlines=True,
-    env=env,
 ).stdout.splitlines()
 print("Found %d generated files..." % (len(generated_file_labels),))
 
@@ -143,9 +127,7 @@ print("Found %d generated files..." % (len(generated_file_labels),))
 # fail in case there are build errors in the client, and just warn the user
 # that they may be missing generated files.
 print("Building the generated files so that tools can find them...")
-subprocess.run(
-    [bazel, "build", "--keep_going"] + generated_file_labels, env=env
-)
+subprocess.run([bazel, "build", "--keep_going"] + generated_file_labels)
 
 
 # Manually translate the label to a user friendly path into the Bazel output