Branche: trunk

Jon Ross-Perkins 8b907b5a60 Support octal literals (#6910)		il y a 1 mois
..
README.md	a1655b6858 Tuples and tuple indexing (#3646)	il y a 2 ans
comments.md	33f26bd37b Added design of comments in design docs (#2641)	il y a 3 ans
numeric_literals.md	8b907b5a60 Support octal literals (#6910)	il y a 1 mois
string_literals.md	c974a77314 Added newline example and clarifications for string_literals.md (#2966)	il y a 2 ans
symbolic_tokens.md	58c106010c Updates to design docs to reflect accepted proposals (#3254)	il y a 2 ans
whitespace.md	b7df523dc8 Fix cross-file links in non-proposal files (#1010)	il y a 4 ans
words.md	bd4d5805dd Replace `addr` with `ref` in design docs (#6141)	il y a 6 mois

Lexical conventions

Lexical elements

Lexical elements

The first stage of processing a source file is the division of the source file into lexical elements.

A lexical element is one of the following:

a maximal sequence of whitespace characters
a word
a literal:
- a numeric literal
- a string literal
a comment
a symbolic token

The sequence of lexical elements is formed by repeatedly removing the longest initial sequence of characters that forms a valid lexical element, with the following exception:

When a numeric literal immediately follows a . or -> token, with no intervening whitespace, a real literal is never formed. Instead, the token will end no later than the next . character. For example, tuple.1.2 is five tokens, tuple . 1 . 2, not three tokens, tuple . 1.2. However, tuple . 1.2 is lexed as three tokens.

README.md

Lexical conventions

Table of contents

Lexical elements