I am making a lexer, don't tell me to not do because I already did most of it.
Currently it makes an array of tokens and that's it.

I would like to know, what functions the lexer needs to provide and a brief explanation of what each function needs to do.

I'll accept the most complete list.

An example function would be:

next: Consume the current token and return it