ArkScript
A small, fast, functional and scripting language for video games
Ark::internal::Lexer Class Reference

The lexer, in charge of creating a list of tokens. More...

#include <Lexer.hpp>

+ Collaboration diagram for Ark::internal::Lexer:

Public Member Functions

 Lexer (unsigned debug) noexcept
 Construct a new Lexer object. More...
 
void feed (const std::string &code)
 Give code to tokenize and create the list of tokens. More...
 
std::vector< Token > & tokens () noexcept
 Return the list of tokens. More...
 

Private Member Functions

constexpr bool isHexChar (char chr)
 
TokenType guessType (const std::string &value) noexcept
 Helper function to determine the type of a token. More...
 
bool isKeyword (const std::string &value) noexcept
 Check if the value is a keyword in ArkScript. More...
 
bool isIdentifier (const std::string &value) noexcept
 Check if the value can be an identifier in ArkScript. More...
 
bool isOperator (const std::string &value) noexcept
 Check if the value is an operator in ArkScript. More...
 
bool endOfControlChar (const std::string &sequence, char next) noexcept
 Check if a control character / sequence is complete or not. More...
 
void throwTokenizingError (const std::string &message, const std::string &match, std::size_t line, std::size_t col, const std::string &context)
 To throw nice lexer errors. More...
 

Private Attributes

unsigned m_debug
 
std::vector< Tokenm_tokens
 

Detailed Description

The lexer, in charge of creating a list of tokens.

Definition at line 26 of file Lexer.hpp.

Constructor & Destructor Documentation

◆ Lexer()

Ark::internal::Lexer::Lexer ( unsigned  debug)
explicitnoexcept

Construct a new Lexer object.

Parameters
debugthe debug level

Definition at line 15 of file Lexer.cpp.

Member Function Documentation

◆ endOfControlChar()

bool Ark::internal::Lexer::endOfControlChar ( const std::string &  sequence,
char  next 
)
privatenoexcept

Check if a control character / sequence is complete or not.

Parameters
sequencethe sequence without the leading \
nextthe next character to come, maybe, in the sequence
Returns
true
false

Definition at line 301 of file Lexer.cpp.

Referenced by feed().

◆ feed()

void Ark::internal::Lexer::feed ( const std::string &  code)

◆ guessType()

TokenType Ark::internal::Lexer::guessType ( const std::string &  value)
privatenoexcept

Helper function to determine the type of a token.

Parameters
value
Returns
TokenType

Definition at line 262 of file Lexer.cpp.

References Ark::internal::Capture, Ark::internal::GetField, Ark::internal::Identifier, Ark::Utils::isDouble(), Ark::internal::Keyword, Ark::internal::Mismatch, Ark::internal::Number, Ark::internal::Operator, and Ark::internal::Spread.

Referenced by feed().

◆ isHexChar()

constexpr bool Ark::internal::Lexer::isHexChar ( char  chr)
inlineconstexprprivate

Definition at line 54 of file Lexer.hpp.

References Ark::internal::Builtins::String::chr().

◆ isIdentifier()

bool Ark::internal::Lexer::isIdentifier ( const std::string &  value)
privatenoexcept

Check if the value can be an identifier in ArkScript.

Parameters
value
Returns
true
false

Definition at line 291 of file Lexer.cpp.

◆ isKeyword()

bool Ark::internal::Lexer::isKeyword ( const std::string &  value)
privatenoexcept

Check if the value is a keyword in ArkScript.

Parameters
value
Returns
true
false

Definition at line 286 of file Lexer.cpp.

References Ark::internal::keywords.

◆ isOperator()

bool Ark::internal::Lexer::isOperator ( const std::string &  value)
privatenoexcept

Check if the value is an operator in ArkScript.

Parameters
value
Returns
true
false

Definition at line 296 of file Lexer.cpp.

References Ark::internal::operators.

◆ throwTokenizingError()

void Ark::internal::Lexer::throwTokenizingError ( const std::string &  message,
const std::string &  match,
std::size_t  line,
std::size_t  col,
const std::string &  context 
)
private

To throw nice lexer errors.

Parameters
message
match
line
col
context

Definition at line 330 of file Lexer.cpp.

References Ark::internal::makeTokenBasedErrorCtx(), and Ark::Utils::splitString().

Referenced by feed().

◆ tokens()

std::vector< Token > & Ark::internal::Lexer::tokens ( )
noexcept

Return the list of tokens.

Returns
std::vector<Token>&

Definition at line 257 of file Lexer.cpp.

References m_tokens.

Referenced by Ark::internal::Parser::feed().

Member Data Documentation

◆ m_debug

unsigned Ark::internal::Lexer::m_debug
private

Definition at line 51 of file Lexer.hpp.

Referenced by feed().

◆ m_tokens

std::vector<Token> Ark::internal::Lexer::m_tokens
private

Definition at line 52 of file Lexer.hpp.

Referenced by feed(), and tokens().


The documentation for this class was generated from the following files: