Roles of the Lexical analyzer

Lexical analyzer performs below given tasks:

• Helps to identify token into the symbol table
• Removes white spaces and comments from the source program
• Correlates error messages with the source program
• Helps you to expands the macros if it is found in the source program
• Read input characters from the source program

Example of Lexical Analysis, Tokens, Non-Tokens

Consider the following code that is fed to Lexical Analyzer

#include <stdio.h>

   int maximum(int x, int y) {

       // This will compare 2 numbers

       if (x > y)

           return x;

       else {

           return y;

       }

   }

Examples of Tokens created

Examples of Nontokens