class Prism::Translation::Parser

This class is the entry-point for converting a prism syntax tree into the whitequark/parser gem’s syntax tree. It inherits from the base parser for the parser gem, and overrides the parse* methods to parse with prism and then translate.

Constants

VERSION_3_3

For Ruby 3.3

VERSION_3_4

For Ruby 3.4

Public Instance Methods

default_encoding() click to toggle source

The default encoding for Ruby files is UTF-8.

# File lib/prism/translation/parser.rb, line 33
def default_encoding
  Encoding::UTF_8
end
parse(source_buffer) click to toggle source

Parses a source buffer and returns the AST.

# File lib/prism/translation/parser.rb, line 41
def parse(source_buffer)
  @source_buffer = source_buffer
  source = source_buffer.source

  offset_cache = build_offset_cache(source)
  result = unwrap(Prism.parse(source, filepath: source_buffer.name, version: convert_for_prism(version)), offset_cache)

  build_ast(result.value, offset_cache)
ensure
  @source_buffer = nil
end
parse_with_comments(source_buffer) click to toggle source

Parses a source buffer and returns the AST and the source code comments.

# File lib/prism/translation/parser.rb, line 54
def parse_with_comments(source_buffer)
  @source_buffer = source_buffer
  source = source_buffer.source

  offset_cache = build_offset_cache(source)
  result = unwrap(Prism.parse(source, filepath: source_buffer.name, version: convert_for_prism(version)), offset_cache)

  [
    build_ast(result.value, offset_cache),
    build_comments(result.comments, offset_cache)
  ]
ensure
  @source_buffer = nil
end
tokenize(source_buffer, recover = false) click to toggle source

Parses a source buffer and returns the AST, the source code comments, and the tokens emitted by the lexer.

# File lib/prism/translation/parser.rb, line 71
def tokenize(source_buffer, recover = false)
  @source_buffer = source_buffer
  source = source_buffer.source

  offset_cache = build_offset_cache(source)
  result =
    begin
      unwrap(Prism.parse_lex(source, filepath: source_buffer.name, version: convert_for_prism(version)), offset_cache)
    rescue ::Parser::SyntaxError
      raise if !recover
    end

  program, tokens = result.value
  ast = build_ast(program, offset_cache) if result.success?

  [
    ast,
    build_comments(result.comments, offset_cache),
    build_tokens(tokens, offset_cache)
  ]
ensure
  @source_buffer = nil
end
try_declare_numparam(node) click to toggle source

Since prism resolves num params for us, we don’t need to support this kind of logic here.

# File lib/prism/translation/parser.rb, line 97
def try_declare_numparam(node)
  node.children[0].match?(/\A_[1-9]\z/)
end

Private Instance Methods

build_ast(program, offset_cache) click to toggle source

Build the parser gem AST from the prism AST.

# File lib/prism/translation/parser.rb, line 158
def build_ast(program, offset_cache)
  program.accept(Compiler.new(self, offset_cache))
end
build_comments(comments, offset_cache) click to toggle source

Build the parser gem comments from the prism comments.

# File lib/prism/translation/parser.rb, line 163
def build_comments(comments, offset_cache)
  comments.map do |comment|
    ::Parser::Source::Comment.new(build_range(comment.location, offset_cache))
  end
end
build_offset_cache(source) click to toggle source

Prism deals with offsets in bytes, while the parser gem deals with offsets in characters. We need to handle this conversion in order to build the parser gem AST.

If the bytesize of the source is the same as the length, then we can just use the offset directly. Otherwise, we build an array where the index is the byte offset and the value is the character offset.

# File lib/prism/translation/parser.rb, line 141
def build_offset_cache(source)
  if source.bytesize == source.length
    -> (offset) { offset }
  else
    offset_cache = []
    offset = 0

    source.each_char do |char|
      char.bytesize.times { offset_cache << offset }
      offset += 1
    end

    offset_cache << offset
  end
end
build_range(location, offset_cache) click to toggle source

Build a range from a prism location.

# File lib/prism/translation/parser.rb, line 175
def build_range(location, offset_cache)
  ::Parser::Source::Range.new(
    source_buffer,
    offset_cache[location.start_offset],
    offset_cache[location.end_offset]
  )
end
build_tokens(tokens, offset_cache) click to toggle source

Build the parser gem tokens from the prism tokens.

# File lib/prism/translation/parser.rb, line 170
def build_tokens(tokens, offset_cache)
  Lexer.new(source_buffer, tokens.map(&:first), offset_cache).to_a
end
convert_for_prism(version) click to toggle source

Converts the version format handled by Parser to the format handled by Prism.

# File lib/prism/translation/parser.rb, line 184
def convert_for_prism(version)
  case version
  when 33
    "3.3.0"
  when 34
    "3.4.0"
  else
    "latest"
  end
end
unwrap(result, offset_cache) click to toggle source

If there was a error generated during the parse, then raise an appropriate syntax error. Otherwise return the result.

# File lib/prism/translation/parser.rb, line 117
def unwrap(result, offset_cache)
  result.errors.each do |error|
    next unless valid_error?(error)

    location = build_range(error.location, offset_cache)
    diagnostics.process(Diagnostic.new(error.message, :error, :prism_error, location))
  end
  result.warnings.each do |warning|
    next unless valid_warning?(warning)

    location = build_range(warning.location, offset_cache)
    diagnostics.process(Diagnostic.new(warning.message, :warning, :prism_warning, location))
  end

  result
end
valid_error?(error) click to toggle source

This is a hook to allow consumers to disable some errors if they don’t want them to block creating the syntax tree.

# File lib/prism/translation/parser.rb, line 105
def valid_error?(error)
  true
end
valid_warning?(warning) click to toggle source

This is a hook to allow consumers to disable some warnings if they don’t want them to block creating the syntax tree.

# File lib/prism/translation/parser.rb, line 111
def valid_warning?(warning)
  true
end