29 PM_LEX_STATE_BIT_ENDARG,
30 PM_LEX_STATE_BIT_ENDFN,
32 PM_LEX_STATE_BIT_CMDARG,
34 PM_LEX_STATE_BIT_FNAME,
36 PM_LEX_STATE_BIT_CLASS,
37 PM_LEX_STATE_BIT_LABEL,
38 PM_LEX_STATE_BIT_LABELED,
39 PM_LEX_STATE_BIT_FITEM
47 PM_LEX_STATE_NONE = 0,
48 PM_LEX_STATE_BEG = (1 << PM_LEX_STATE_BIT_BEG),
49 PM_LEX_STATE_END = (1 << PM_LEX_STATE_BIT_END),
50 PM_LEX_STATE_ENDARG = (1 << PM_LEX_STATE_BIT_ENDARG),
51 PM_LEX_STATE_ENDFN = (1 << PM_LEX_STATE_BIT_ENDFN),
52 PM_LEX_STATE_ARG = (1 << PM_LEX_STATE_BIT_ARG),
53 PM_LEX_STATE_CMDARG = (1 << PM_LEX_STATE_BIT_CMDARG),
54 PM_LEX_STATE_MID = (1 << PM_LEX_STATE_BIT_MID),
55 PM_LEX_STATE_FNAME = (1 << PM_LEX_STATE_BIT_FNAME),
56 PM_LEX_STATE_DOT = (1 << PM_LEX_STATE_BIT_DOT),
57 PM_LEX_STATE_CLASS = (1 << PM_LEX_STATE_BIT_CLASS),
58 PM_LEX_STATE_LABEL = (1 << PM_LEX_STATE_BIT_LABEL),
59 PM_LEX_STATE_LABELED = (1 << PM_LEX_STATE_BIT_LABELED),
60 PM_LEX_STATE_FITEM = (1 << PM_LEX_STATE_BIT_FITEM),
61 PM_LEX_STATE_BEG_ANY = PM_LEX_STATE_BEG | PM_LEX_STATE_MID | PM_LEX_STATE_CLASS,
62 PM_LEX_STATE_ARG_ANY = PM_LEX_STATE_ARG | PM_LEX_STATE_CMDARG,
63 PM_LEX_STATE_END_ANY = PM_LEX_STATE_END | PM_LEX_STATE_ENDARG | PM_LEX_STATE_ENDFN
70 PM_HEREDOC_QUOTE_NONE,
71 PM_HEREDOC_QUOTE_SINGLE =
'\'',
72 PM_HEREDOC_QUOTE_DOUBLE =
'"',
73 PM_HEREDOC_QUOTE_BACKTICK =
'`',
80 PM_HEREDOC_INDENT_NONE,
81 PM_HEREDOC_INDENT_DASH,
82 PM_HEREDOC_INDENT_TILDE,
262 #define PM_LEX_STACK_SIZE 4
The abstract syntax tree.
@ PM_SHAREABLE_CONSTANT_NODE_FLAGS_EXPERIMENTAL_EVERYTHING
constant writes that should be modified with shareable constant value experimental everything
@ PM_SHAREABLE_CONSTANT_NODE_FLAGS_LITERAL
constant writes that should be modified with shareable constant value literal
@ PM_SHAREABLE_CONSTANT_NODE_FLAGS_EXPERIMENTAL_COPY
constant writes that should be modified with shareable constant value experimental copy
uint16_t pm_node_flags_t
These are the flags embedded in the node struct.
The options that can be passed to parsing.
pm_options_version_t
The version of Ruby syntax that we should be parsing with.
struct pm_locals pm_locals_t
This is a set of local variables in a certain lexical context (method, class, module,...
pm_heredoc_indent_t
The type of indentation that a heredoc uses.
struct pm_context_node pm_context_node_t
This is a node in a linked list of contexts.
uint32_t pm_state_stack_t
A struct that represents a stack of boolean values.
#define PM_LEX_STACK_SIZE
We pre-allocate a certain number of lex states in order to avoid having to call malloc too many times...
struct pm_lex_mode pm_lex_mode_t
When lexing Ruby source, the lexer has a small amount of state to tell which kind of token it is curr...
struct pm_comment pm_comment_t
This is a node in the linked list of comments that we've found while parsing.
pm_lex_state_bit_t
This enum provides various bits that represent different kinds of states that the lexer can track.
pm_lex_state_t
This enum combines the various bits from the above enum into individual values that represent the var...
struct pm_scope pm_scope_t
This struct represents a node in a linked list of scopes.
pm_heredoc_quote_t
The type of quote that a heredoc uses.
void(* pm_encoding_changed_callback_t)(pm_parser_t *parser)
When the encoding that is being used to parse the source is changed by prism, we provide the ability ...
pm_context_t
While parsing, we keep track of a stack of contexts.
@ PM_CONTEXT_CLASS_RESCUE
a rescue statement within a class statement
@ PM_CONTEXT_ELSIF
an elsif clause
@ PM_CONTEXT_DEF_RESCUE
a rescue statement within a method definition
@ PM_CONTEXT_ELSE
an else clause
@ PM_CONTEXT_FOR_INDEX
a for loop's index
@ PM_CONTEXT_CASE_WHEN
a case when statements
@ PM_CONTEXT_BLOCK_RESCUE
a rescue statement within a do..end block
@ PM_CONTEXT_MODULE
a module declaration
@ PM_CONTEXT_DEF_PARAMS
a method definition's parameters
@ PM_CONTEXT_CASE_IN
a case in statements
@ PM_CONTEXT_BLOCK_ELSE
a rescue else statement within a do..end block
@ PM_CONTEXT_LOOP_PREDICATE
the predicate clause of a loop statement
@ PM_CONTEXT_SCLASS
a singleton class definition
@ PM_CONTEXT_UNLESS
an unless statement
@ PM_CONTEXT_POSTEXE
an END block
@ PM_CONTEXT_IF
an if statement
@ PM_CONTEXT_MULTI_TARGET
a multiple target expression
@ PM_CONTEXT_LAMBDA_RESCUE
a rescue statement within a lambda expression
@ PM_CONTEXT_BEGIN_ELSE
a rescue else statement with an explicit begin
@ PM_CONTEXT_NONE
a null context, used for returning a value from a function
@ PM_CONTEXT_CLASS_ELSE
a rescue else statement within a class statement
@ PM_CONTEXT_LAMBDA_ENSURE
an ensure statement within a lambda expression
@ PM_CONTEXT_BLOCK_ENSURE
an ensure statement within a do..end block
@ PM_CONTEXT_CLASS_ENSURE
an ensure statement within a class statement
@ PM_CONTEXT_LAMBDA_BRACES
a lambda expression with braces
@ PM_CONTEXT_MODULE_ELSE
a rescue else statement within a module statement
@ PM_CONTEXT_PARENS
a parenthesized expression
@ PM_CONTEXT_BLOCK_BRACES
expressions in block arguments using braces
@ PM_CONTEXT_DEF_ENSURE
an ensure statement within a method definition
@ PM_CONTEXT_SCLASS_RESCUE
a rescue statement with a singleton class
@ PM_CONTEXT_PREEXE
a BEGIN block
@ PM_CONTEXT_DEFINED
a defined? expression
@ PM_CONTEXT_MODULE_ENSURE
an ensure statement within a module statement
@ PM_CONTEXT_BEGIN_RESCUE
a rescue statement with an explicit begin
@ PM_CONTEXT_UNTIL
an until statement
@ PM_CONTEXT_DEF_ELSE
a rescue else statement within a method definition
@ PM_CONTEXT_FOR
a for loop
@ PM_CONTEXT_PREDICATE
a predicate inside an if/elsif/unless statement
@ PM_CONTEXT_BEGIN_ENSURE
an ensure statement with an explicit begin
@ PM_CONTEXT_SCLASS_ENSURE
an ensure statement with a singleton class
@ PM_CONTEXT_DEFAULT_PARAMS
a method definition's default parameter
@ PM_CONTEXT_LAMBDA_ELSE
a rescue else statement within a lambda expression
@ PM_CONTEXT_CLASS
a class declaration
@ PM_CONTEXT_MAIN
the top level context
@ PM_CONTEXT_LAMBDA_DO_END
a lambda expression with do..end
@ PM_CONTEXT_BEGIN
a begin statement
@ PM_CONTEXT_RESCUE_MODIFIER
a modifier rescue clause
@ PM_CONTEXT_EMBEXPR
an interpolated expression
@ PM_CONTEXT_TERNARY
a ternary expression
@ PM_CONTEXT_DEF
a method definition
@ PM_CONTEXT_SCLASS_ELSE
a rescue else statement with a singleton class
@ PM_CONTEXT_MODULE_RESCUE
a rescue statement within a module statement
@ PM_CONTEXT_BLOCK_KEYWORDS
expressions in block arguments using do..end
@ PM_CONTEXT_WHILE
a while statement
uint8_t pm_scope_parameters_t
The flags about scope parameters that can be set.
uint8_t pm_shareable_constant_value_t
The type of shareable constant value that can be set.
pm_comment_type_t
This is the type of a comment that we've found while parsing.
A data structure that stores a set of strings.
uint32_t pm_constant_id_t
A constant id is a unique identifier for a constant in the constant pool.
A list of byte offsets of newlines in a string.
A generic string type that can have various ownership semantics.
Macro definitions used throughout the prism library.
The encoding interface and implementations used by the parser.
A set of static literal nodes that can be checked for duplicates.
The overall constant pool, which stores constants found while parsing.
This is a node in a linked list of contexts.
pm_context_t context
The context that this node represents.
struct pm_context_node * prev
A pointer to the previous context in the linked list.
This struct defines the functions necessary to implement the encoding interface so we can determine h...
All of the information necessary to store to lexing a heredoc.
size_t ident_length
The length of the heredoc identifier.
pm_heredoc_quote_t quote
The type of quote that the heredoc uses.
pm_heredoc_indent_t indent
The type of indentation that the heredoc uses.
const uint8_t * ident_start
A pointer to the start of the heredoc identifier.
When you are lexing through a file, the lexer needs all of the information that the parser additional...
void * data
This opaque pointer is used to provide whatever information the user deemed necessary to the callback...
When lexing Ruby source, the lexer has a small amount of state to tell which kind of token it is curr...
union pm_lex_mode::@88 as
The data associated with this type of lex mode.
uint8_t terminator
This is the terminator of the list literal.
size_t nesting
This keeps track of the nesting level of the list.
bool interpolation
Whether or not interpolation is allowed in this list.
uint8_t incrementor
When lexing a list, it takes into account balancing the terminator if the terminator is one of (),...
@ PM_LEX_LIST
This state is used when we are lexing a list of tokens, as in a w word list literal or a i symbol lis...
@ PM_LEX_EMBVAR
This state is used when we're lexing a variable that is embedded directly inside of a string with the...
@ PM_LEX_REGEXP
This state is used when a regular expression has been begun and we are looking for the terminator.
@ PM_LEX_DEFAULT
This state is used when any given token is being lexed.
@ PM_LEX_HEREDOC
This state is used when you are inside the content of a heredoc.
@ PM_LEX_EMBEXPR
This state is used when we're lexing as normal but inside an embedded expression of a string.
@ PM_LEX_STRING
This state is used when we are lexing a string or a string-like token, as in string content with eith...
uint8_t breakpoints[11]
This is the character set that should be used to delimit the tokens within the list.
pm_heredoc_lex_mode_t base
All of the data necessary to lex a heredoc.
bool line_continuation
True if the previous token ended with a line continuation.
struct pm_lex_mode * prev
The previous lex state so that it knows how to pop.
bool label_allowed
Whether or not at the end of the string we should allow a :, which would indicate this was a dynamic ...
const uint8_t * next_start
This is the pointer to the character where lexing should resume once the heredoc has been completely ...
enum pm_lex_mode::@87 mode
The type of this lex mode.
size_t * common_whitespace
This is used to track the amount of common whitespace on each line so that we know how much to dedent...
This struct represents an abstract linked list that provides common functionality.
This represents the overall linked list.
This tracks an individual local variable in a certain lexical context, as well as the number of times...
pm_constant_id_t name
The name of the local variable.
pm_location_t location
The location of the local variable in the source.
uint32_t hash
The hash of the local variable.
uint32_t index
The index of the local variable in the local table.
uint32_t reads
The number of times the local variable is read.
This is a set of local variables in a certain lexical context (method, class, module,...
pm_local_t * locals
The nullable allocated memory for the local variables in the set.
uint32_t capacity
The capacity of the local variables set.
uint32_t size
The number of local variables in the set.
This represents a range of bytes in the source string to which a node or token corresponds.
A list of offsets of newlines in a string.
A list of nodes in the source, most often used for lists of children.
This struct represents the overall parser.
const pm_encoding_t * explicit_encoding
When a string-like expression is being lexed, any byte or escape sequence that resolves to a value wh...
pm_lex_state_t lex_state
The current state of the lexer.
uint8_t command_line
The command line flags given from the options.
const pm_encoding_t * encoding
The encoding functions for the current file is attached to the parser as it's parsing so that it can ...
bool partial_script
Whether or not we are parsing a "partial" script, which is a script that will be evaluated in the con...
bool pattern_matching_newlines
This flag indicates that we are currently parsing a pattern matching expression and impacts that calc...
const uint8_t * end
The pointer to the end of the source.
bool recovering
Whether or not we're currently recovering from a syntax error.
pm_node_flags_t integer_base
We want to add a flag to integer nodes that indicates their base.
bool warn_mismatched_indentation
By default, Ruby always warns about mismatched indentation.
pm_constant_pool_t constant_pool
This constant pool keeps all of the constants defined throughout the file so that we can reference th...
bool in_keyword_arg
This flag indicates that we are currently parsing a keyword argument.
pm_state_stack_t do_loop_stack
The stack used to determine if a do keyword belongs to the predicate of a while, until,...
const uint8_t * next_start
This is a special field set on the parser when we need the parser to jump to a specific location when...
pm_static_literals_t * current_hash_keys
The hash keys for the hash that is currently being parsed.
pm_list_t magic_comment_list
The list of magic comments that have been found while parsing.
int lambda_enclosure_nesting
Used to temporarily track the nesting of enclosures to determine if a { is the beginning of a lambda ...
pm_lex_callback_t * lex_callback
This is an optional callback that can be attached to the parser that will be called whenever a new to...
pm_options_version_t version
The version of prism that we should use to parse.
pm_token_t previous
The previous token we were considering.
pm_string_t current_string
This string is used to pass information from the lexer to the parser.
bool parsing_eval
Whether or not we are parsing an eval string.
bool current_regular_expression_ascii_only
True if the current regular expression being lexed contains only ASCII characters.
bool encoding_changed
Whether or not the encoding has been changed by a magic comment.
pm_location_t data_loc
An optional location that represents the location of the END marker and the rest of the content of th...
pm_context_node_t * current_context
The current parsing context.
struct pm_parser::@93 lex_modes
A stack of lex modes.
const uint8_t * start
The pointer to the start of the source.
pm_state_stack_t accepts_block_stack
The stack used to determine if a do keyword belongs to the beginning of a block.
int enclosure_nesting
Tracks the current nesting of (), [], and {}.
pm_list_t error_list
The list of errors that have been found while parsing.
int8_t frozen_string_literal
Whether or not we have found a frozen_string_literal magic comment with a true or false value.
pm_node_list_t * current_block_exits
When parsing block exits (e.g., break, next, redo), we need to validate that they are in correct cont...
const uint8_t * encoding_comment_start
This pointer indicates where a comment must start if it is to be considered an encoding comment.
pm_token_t current
The current token we're considering.
pm_lex_mode_t stack[PM_LEX_STACK_SIZE]
The stack of lexer modes.
pm_list_t warning_list
The list of warnings that have been found while parsing.
const uint8_t * heredoc_end
This field indicates the end of a heredoc whose identifier was found on the current line.
int brace_nesting
Used to track the nesting of braces to ensure we get the correct value when we are interpolating bloc...
pm_encoding_changed_callback_t encoding_changed_callback
When the encoding that is being used to parse the source is changed by prism, we provide the ability ...
int32_t start_line
The line number at the start of the parse.
bool encoding_locked
This is very specialized behavior for when you want to parse in a context that does not respect encod...
pm_lex_mode_t * current
The current mode of the lexer.
pm_list_t comment_list
The list of comments that have been found while parsing.
size_t index
The current index into the lexer mode stack.
pm_string_t filepath
This is the path of the file being parsed.
pm_scope_t * current_scope
The current local scope.
bool command_start
Whether or not we're at the beginning of a command.
pm_newline_list_t newline_list
This is the list of newline offsets in the source file.
bool semantic_token_seen
Whether or not the parser has seen a token that has semantic meaning (i.e., a token that is not a com...
uint32_t node_id
The next node identifier that will be assigned.
This struct represents a node in a linked list of scopes.
struct pm_scope * previous
A pointer to the previous scope in the linked list.
pm_node_list_t implicit_parameters
This is a list of the implicit parameters contained within the block.
pm_shareable_constant_value_t shareable_constant
The current state of constant shareability for this scope.
pm_locals_t locals
The IDs of the locals in the given scope.
pm_scope_parameters_t parameters
This is a bitfield that indicates the parameters that are being used in this scope.
bool closed
A boolean indicating whether or not this scope can see into its parent.
Certain sets of nodes (hash keys and when clauses) check for duplicate nodes to alert the user of pot...
A generic string type that can have various ownership semantics.
This struct represents a token in the Ruby source.