Просмотр исходного кода

explorer/README.md: update test documentation (#2872)

Update explorer test documentation to match the new tools used.
Adrien Leravat 2 лет назад
Родитель
Сommit
b843096bcd
3 измененных файлов с 51 добавлено и 24 удалено
  1. 48 22
      explorer/README.md
  2. 1 1
      testing/file_test/file_test_base.h
  3. 2 1
      testing/scripts/autoupdate_testdata_base.py

+ 48 - 22
explorer/README.md

@@ -88,10 +88,8 @@ recreate the original value.
 The [`testdata/`](testdata/) subdirectory includes some example programs with
 expected output.
 
-These tests make use of LLVM's
-[lit](https://llvm.org/docs/CommandGuide/lit.html) and
-[FileCheck](https://llvm.org/docs/CommandGuide/FileCheck.html). Tests have
-boilerplate at the top:
+These tests make use of [GoogleTest](https://github.com/google/googletest) with
+Bazel's `cc_test` rules. Tests have boilerplate at the top:
 
 ```carbon
 // Part of the Carbon Language project, under the Apache License v2.0 with LLVM
@@ -99,37 +97,62 @@ boilerplate at the top:
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 // AUTOUPDATE
-// RUN: %{explorer-run}
-// RUN: %{explorer-run-trace}
-// CHECK:result: 0
+// CHECK:STDOUT: result: 7
 
 package ExplorerTest api;
+
+fn Main() -> i32 {
+  return (1 + 2) + 4;
+}
 ```
 
 To explain this boilerplate:
 
 -   The standard copyright is expected.
--   The `AUTOUPDATE` line indicates that `RUN` and `CHECK` lines will be
-    automatically inserted immediately below by the `./lit_autoupdate.py`
-    script.
--   The `RUN` lines indicate two commands for `lit` to execute using the file:
-    one without trace and debug output, one with.
-    -   `RUN:` will be followed by the `not` command when failure is expected.
-        In particular, `RUN: not explorer ...`.
-    -   The full command is in `lit.cfg.py`; it will run explorer and pass
-        results to
-        [`FileCheck`](https://llvm.org/docs/CommandGuide/FileCheck.html).
--   The `CHECK` lines indicate expected output, verified by `FileCheck`.
+-   The `AUTOUPDATE` line indicates that `CHECK` lines matching the output will
+    be automatically inserted immediately below by the
+    `./autoupdate_testdata.py` script.
+-   The `CHECK` lines indicate expected output.
     -   Where a `CHECK` line contains text like `{{.*}}`, the double curly
         braces indicate a contained regular expression.
 -   The `package` is required in all test files, per normal Carbon syntax rules.
 
+### lit tests
+
+The [`lit_testdata/`](lit_testdata/) subdirectory includes other example
+programs.
+
+These tests make use of LLVM's
+[lit](https://llvm.org/docs/CommandGuide/lit.html) and
+[FileCheck](https://llvm.org/docs/CommandGuide/FileCheck.html).
+
+They share most of their header with those in `testdata`, with an additional
+`RUN` rule:
+
+```
+// RUN: %{explorer-run}
+// RUN: %{explorer-run-trace}
+```
+
+The `RUN` lines indicate two commands for `lit` to execute using the file: one
+without trace and debug output, one with.
+
+-   `RUN:` will be followed by the `not` command when failure is expected. In
+    particular, `RUN: not %{explorer-run}`.
+-   The full command is in `lit.cfg.py`; it will run explorer and pass results
+    to [`FileCheck`](https://llvm.org/docs/CommandGuide/FileCheck.html).
+
 ### Useful commands
 
--   `./lit_autodupate.py` -- Updates expected output.
+-   `./autoupdate_testdata.py` -- Updates expected output.
     -   This can be combined with `git diff` to see changes in output.
+-   `autoupdate_lit_testdata.py` -- Updates lit tests expected output.
 -   `bazel test ... --test_output=errors` -- Runs tests and prints any errors.
+-   `bazel test //explorer:file_test.subset --test_arg=explorer/testdata/DIR/FILE.carbon`
+    -- Runs a specific test.
 -   `bazel run testdata/DIR/FILE.carbon.run` -- Runs explorer on the file.
+-   `bazel run testdata/DIR/FILE.carbon.verbose` -- Runs explorer on the file
+    with tracing enabled.
 
 ### Updating fuzzer logic after making AST changes
 
@@ -138,9 +161,12 @@ Please refer to
 
 ## Trace Program Execution
 
-When tracing is turned on (using the `--trace_file=...` option), `explorer`
-prints the state of the program and each step that is performed during
-execution.
+When tracing is turned on (using the `--trace_file=...` option or `.verbose`
+target), `explorer` prints the state of the program and each step that is
+performed during execution.
+
+Printing directly to the standard output using the `--trace_file` option is
+supported by passing `-` in place of a filepath (`--trace_file=-`).
 
 ### State of the Program
 

+ 1 - 1
testing/file_test/file_test_base.h

@@ -31,7 +31,7 @@ namespace Carbon::Testing {
 // ...` and `// CHECK:STDERR: ...` respectively. `[[@LINE+offset]` and
 // `{{regex}}` syntaxes should also work.
 //
-// `lit_autoupdate.py` automatically constructs compatible lines.
+// `autoupdate_testdata.py` automatically constructs compatible lines.
 class FileTestBase : public testing::Test {
  public:
   explicit FileTestBase(const std::filesystem::path& path);

+ 2 - 1
testing/scripts/autoupdate_testdata_base.py

@@ -500,7 +500,8 @@ def main() -> None:
         tests = {test.relative_to(root) for test in parsed_args.tests}
     else:
         print(
-            "HINT: run `lit_autoupdate.py f1 f2 ...` to update specific tests"
+            "HINT: run `autoupdate_testdata.py f1 f2 ...` "
+            "to update specific tests"
         )
         tests = get_tests(parsed_args.testdata)