Thread: Lexical Analyzer

  1. #1
    Registered User
    Join Date
    Aug 2009
    Posts
    20

    Lexical Analyzer

    Hi everyone !!
    I'm trying to code a Lexical Analyzer in C++, so can anyone tell me where to start ?

  2. #2
    Registered User
    Join Date
    Aug 2009
    Posts
    20
    hellooo????

  3. #3
    Registered User
    Join Date
    Oct 2006
    Posts
    3,445
    start by searching google. after reading the forum rules

  4. #4
    Registered User
    Join Date
    Aug 2009
    Posts
    20
    thanks....

  5. #5
    Registered User
    Join Date
    Aug 2009
    Posts
    20
    well now i have some idea .... like where to start..... thanks for the advice

  6. #6
    C++ Witch laserlight's Avatar
    Join Date
    Oct 2003
    Location
    Singapore
    Posts
    28,413
    If you want to try something interesting then you could try Spirit.
    Quote Originally Posted by Bjarne Stroustrup (2000-10-14)
    I get maybe two dozen requests for help with some sort of programming or design problem every day. Most have more sense than to send me hundreds of lines of code. If they do, I ask them to find the smallest example that exhibits the problem and send me that. Mostly, they then find the error themselves. "Finding the smallest program that demonstrates the error" is a powerful debugging tool.
    Look up a C++ Reference and learn How To Ask Questions The Smart Way

  7. #7
    Guest Sebastiani's Avatar
    Join Date
    Aug 2001
    Location
    Waterloo, Texas
    Posts
    5,708
    Quote Originally Posted by laserlight View Post
    If you want to try something interesting then you could try Spirit.
    A word of caution about using Spirit, though - for complex grammers, long compilation times and slow parsing performance often result. The former is mostly due to the deeply nested template instantiations, and the latter because Spirit was designed foremost to be a feature-rich, flexible library - efficiency was not the primary focus. That isn't to say that it's terribly bloated, either - with something as complicated as C++, it's probably unfair to expect anything other than a hand-coded implementation to run blazingly fast.

    I have read through a good portion of the Spirit source, though, and can honestly vouch that it's an extremely well-designed piece of software - a true masterpiece, actually. Considering all that it has to offer, it's hard to imagine that it could have been implemented with any much less of a footprint, really. So, the bottom line is that, yes, it's certainly well-equipped enough to handle C++ parsing, but probably not sufficient to be used in something like a production-level compiler, obviously.

  8. #8
    Officially An Architect brewbuck's Avatar
    Join Date
    Mar 2007
    Location
    Portland, OR
    Posts
    7,396
    Spirit is a full-blown parser language. Lexical analysis, while it can be viewed as a form of parsing (it is), is really a lot simpler than that. Applying Spirit at the character level seems like overkill, and it's always bothered me that it's not very straightforward to hook Spirit up to a more traditional lexer. You can do it, but you need to be a Spirit expert, and who wants to be one of those?
    Code:
    //try
    //{
    	if (a) do { f( b); } while(1);
    	else   do { f(!b); } while(1);
    //}

  9. #9
    Guest Sebastiani's Avatar
    Join Date
    Aug 2001
    Location
    Waterloo, Texas
    Posts
    5,708
    That's sort of the point of using something like Spirit, though - to do away with the old-style lexer, altogether. Rather than mess with clunky (and error prone) finite state mechanics and specialized subroutines, you simply express the grammer pretty much in it's most direct form, using chains of reusable (and often passive) token-recognizers coupled with the semantic actions that do the actual work of organizing the information as you see fit. This could even entail generating a meta-grammer that is further parsed at a higher level, for instance. So the traditional lexer really isn't at all necessary (or rather, use one or the other, but not both).

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. C name space and lexical scope
    By password636 in forum C Programming
    Replies: 1
    Last Post: 09-22-2009, 09:50 AM
  2. simple lexical analyzer
    By ^Son_Gokou08 in forum C Programming
    Replies: 6
    Last Post: 08-25-2009, 07:55 AM
  3. Scanner? Lexical analyzer? Tokenizer?
    By audinue in forum A Brief History of Cprogramming.com
    Replies: 8
    Last Post: 12-23-2008, 11:32 PM
  4. Lexical analyzer for C
    By nishkarsh in forum C Programming
    Replies: 4
    Last Post: 08-26-2008, 08:05 AM
  5. OpenScript2.0 Spectrum Analyzer
    By jverkoey in forum A Brief History of Cprogramming.com
    Replies: 0
    Last Post: 03-23-2004, 10:01 PM