Tokenizing Source Code
Lexical analysis (scanning) is the first phase — it converts a stream of characters into a stream of tokens. Each token represents a meaningful unit of the language.
Tokens
Regular Expressions
Finite Automata
Lexical Analysis
Breaking source code into tokens
▶ Continue where you left off
🏆 Track Completed!
🧪 Quick Quiz
What is a regular expression used for in compilers?