Best method for counting tokens

This is a discussion on Best method for counting tokens within the C Programming forums, part of the General Programming Boards category; I'm writing a program that counts all of the tokens in a given file ("search"). I can count the number ...

  1. #1
    Registered User
    Join Date
    Jan 2010
    Posts
    12

    Best method for counting tokens

    I'm writing a program that counts all of the tokens in a given file ("search"). I can count the number of "#" signs in the file, but I need to name a number of other tokens... {}()/... and output how many where found in the file.

    Can anyone offer suggestions as to the best method to accomplish this? Right now I am thinking of having each value set up as it's own variable - such as delim1 = #, delim2 = {, and then creating a function from my current code to call each time. Is there a more efficent/faster method? Here is my current code:

    Code:
    #include <stdio.h>
    #include <string.h>
    #define MAX_LINE 80
    
    int main()
    {
    	char line[MAX_LINE];   /*  Max length of line is 80 */
    	char delim[] = "#";       /* Testing just # delim */
        
    	char *words, 
                    *ptr;
    	FILE *fptr;
    	int count = 0;
    	int ch;
    	int linect;
    
    	fptr = fopen ("search", "r");    /* Test file "search" */
    	
    	/* File does not exist or there are not proper permissions set */
    	if (fptr==NULL)
    		printf ("\nCannot open the file\n\n");
    	else	
    	{
    	/* While the file is not at EOF, End of File, loop */
    	while ((ch=fgetc(fptr)) !=EOF)
    	{	
    		/* Reads a line from the file and sets the ptr variable to the line */
    		fgets(line,MAX_LINE,fptr);
    		ptr=line;
    		linect = linect++;
    		 printf("Line reads: %s\n", ptr);
    		/* Loop while looking for delims, when no delims are found it is NULL */
    		while ((words = strtok(ptr, delim)) != NULL)   
    	      {
    			printf("Strtok result: %s\n", words);
    			count++;
    			ptr = NULL;  
    	      }
    	 }
    	printf("\nNumber of delims in the file: %d \n\n",(count-linect)); 
    	fclose(fptr);
    	}   
    	return 0;
    }
    This is working with just one delim on the two lines included in the "search" file below:

    This # is a # test # file.
    End#

  2. #2
    Registered User jeffcobb's Avatar
    Join Date
    Dec 2009
    Location
    Henderson, NV
    Posts
    875
    Well I don't know if this is faster but from the shorter-code-is-simpler dept. something that pulls a character at a time from the file and then hits a switch statement like:
    Code:
    switch(charFromFile)
    {
        case '#':
        {
             hashCounter++;
             break;
        }
        case '{':
        {
             openBraceCounter++;
             break;
        }
    // etc
    }
    
    // then read next character..
    Should be pretty simple and brain-dead to code up...
    C/C++ Environment: GNU CC/Emacs
    Make system: CMake
    Debuggers: Valgrind/GDB

  3. #3
    Registered User
    Join Date
    Jan 2010
    Posts
    12
    Thanks - that makes more sense that what I was thinking... where I'd start with an array filled with different delims and run through counting code EACH time.

  4. #4
    Registered User jeffcobb's Avatar
    Join Date
    Dec 2009
    Location
    Henderson, NV
    Posts
    875
    Quote Originally Posted by Sn0wcra5h View Post
    Thanks - that makes more sense that what I was thinking... where I'd start with an array filled with different delims and run through counting code EACH time.
    Well simple and brain-dead are my forte these days
    C/C++ Environment: GNU CC/Emacs
    Make system: CMake
    Debuggers: Valgrind/GDB

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. on method pointers and inheritance
    By BrownB in forum C++ Programming
    Replies: 2
    Last Post: 03-02-2009, 07:50 PM
  2. Best communication method to thousand childs?
    By Ironic in forum C Programming
    Replies: 8
    Last Post: 11-08-2008, 12:30 AM
  3. C# method
    By siten0308 in forum C# Programming
    Replies: 6
    Last Post: 07-15-2008, 08:01 AM
  4. Overriding a method in C
    By DavidDobson in forum C Programming
    Replies: 1
    Last Post: 07-05-2008, 08:51 AM
  5. Replies: 2
    Last Post: 01-22-2008, 04:22 PM

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21