Lexemes, Tokens,Trees, and Grammar

• The smallest element/unit/word recognized in the compiled or interpreted language is called a lexeme

  • e.g. getc } { “abc” 1.2 * ++ ,

• Each lexeme is recognized by the Scanner to be in some category; that category is returned as the token associated with the lexeme

  • e.g. ( IDENTIFIER, “getc” )

• The Parser recognizes/builds a hierarchical parse tree of tokens; the parse tree expresses the grammar of the language