Recognizing terminals in a CFG production previously not defined as tokens.

Posted by kmels on Stack Overflow See other posts from Stack Overflow or by kmels
Published on 2010-05-30T08:39:01Z Indexed on 2010/05/30 8:42 UTC
Read the original article Hit count: 307

Filed under:

compiler

|

code-generation

|

computer-science

|

context-free-grammar

|

parser-generator

I'm making a generator of LL(1) parsers, my input is a CoCo/R language specification. I've already got a Scanner generator for that input. Suppose I've got the following specification:

COMPILER 1.

CHARACTERS

digit="0123456789".

TOKENS
number = digit{digit}. 
decnumber = digit{digit}"."digit{digit}.

PRODUCTIONS

Expression = Term{"+"Term|"-"Term}.      
Term = Factor{"*"Factor|"/"Factor}.       
Factor = ["-"](Number|"("Expression")").
Number = (number|decnumber).

END 1.

So, if the parser generated by this grammar receives a word "1+1", it'd be accepted i.e. a parse tree would be found.

My question is, the character "+" was never defined in a token, but it appears in the non-terminal "Expression". How should my generated Scanner recognize it? It would not recognize it as a token.

Is this a valid input then? Should I add this terminal in TOKENS and then consider an error routine for a Scanner for it to skip it?

How does usual language specifications handle this?

© Stack Overflow or respective owner

Related posts about compiler

IDEA modular problem (jsp)

as seen on Stack Overflow - Search for 'Stack Overflow'
I have project in with 2 separate modules(frontend and backend, first depends on second). When I'm trying to access backend code from frontend code, things going fine. Things turn for the worse when I do the same from jsp. This is stacktrase for simple accessign bean <jsp:useBean id="mybean"… >>> More
Help with a compiler warning: Initialization from distinct Objective-C type when types match

as seen on Stack Overflow - Search for 'Stack Overflow'
Here is the function where I get the compiler warning, I can't seem to figure out what is causing it. Any help is appreciated. -(void)displaySelector{ //warning on the following line: InstanceSelectorViewController *controller = [[InstanceSelectorViewController alloc] initWithCreator:self]; … >>> More
Sometimes Java web app hang

as seen on Stack Overflow - Search for 'Stack Overflow'
The web app occasionally get hang for about 30 seconds when user request a web page, the server's CPU and memory usage are ok, and the jstack shows: "http-9999-3" daemon prio=6 tid=0x552f3400 nid=0xf40 runnable [0x578fc000] java.lang.Thread.State: RUNNABLE at org.eclipse.jdt.internal.compiler… >>> More
Need help regarding one LALR(1) parsing.

as seen on Stack Overflow - Search for 'Stack Overflow'
I am trying to parse a context-free language, called Context Free Art. I have created its parser in Javascript using a YACC-like JS LALR(1) parser generator JSCC. Take the example of following CFA (Context Free Art) code. This code is a valid CFA. startshape A rule A { CIRCLE { s 1} } Notice the… >>> More
Port Compiler Options (8 replies)

as seen on .NET 247 - Search for '.NET 247'
I currently own a license for Crossworks for ARM and would like to compile the port using it. With the code for the CLR now available, is it possible to compile the port with any ARM compiler or are we still restricted to the Keil ARM gcc compilers? >>> More

Related posts about code-generation

AndroMDA maven code generation and JPA Annotations

as seen on Stack Overflow - Search for 'Stack Overflow'
I am using the AndroMDA plugin for maven to generate code from an uml diagram made in MagicDraw. When the code is generated, AndroMDA desings the JPA annotation for the persitence layer. I think that at the compilation process AndroMDA uses Naming Strategies to determine the Table and Column names… >>> More
Creating a user-defined code generation system in PHP

as seen on Stack Overflow - Search for 'Stack Overflow'
My newest project that I'm looking to build with CodeIgniter would require some sort of system that would allow a user to drag and drop pre-defined functions/methods into mini-classes/objects and then run/test them in the browser. So it'd something similar to Scratch. I've designed a relational database… >>> More
Generating new sources via Maven plugin after compile phase

as seen on Stack Overflow - Search for 'Stack Overflow'
I have a Maven project within which I need execute two code generation steps. One generates some Java types, then the second depends on those Java types to generate some more code. Is there a way to have both of these steps happening during my build? At the moment my steps are: execute first code… >>> More
VB6 ImageList Frm Code Generation

as seen on Stack Overflow - Search for 'Stack Overflow'
Note: This is probably a shot in the dark, and its purely out of curiosity that I'm asking. When using the ImageList control from the Microsoft Common Control lib (mscomctl.ocx) I have found that VB6 generates FRM code that doesn't resolve to real property/method names and I am curious as to how… >>> More
Code generation in Eclipse plug-in

as seen on Stack Overflow - Search for 'Stack Overflow'
Hi, I'm developing an Eclipse plug-in and I'd like to insert code automatically... sort of create a sample application automatically when the user presses a button. How can I go about it please? Thanks and regards, Krt_Malta >>> More