Thread: how to split a sentence to words

  1. #1
    Registered User
    Join Date
    Sep 2009
    Posts
    33

    how to split a sentence to words

    hi guys..

    i have few problems in splitting a line to words..

    for example, say we enter an address to the system

    say

    56/90, claus street, birmingham

    it should return

    unit number : 54
    street : claus street
    town : birmingham

    i'm new to c++ programming and i have no idea about strtok() function. after reading some books and internet searches, i tried it, with little success.

    here's my effort (albeit a beginner one)

    Code:
    #include <iostream>
    #include <stdio.h>
    #include <string.h>
    
    using namespace std;
    
    int main ()
    {
    
      char str[200];
      char * add;
      cout<<"Enter a String"<<flush<<endl;
      cin>>add;
      while (add != NULL)
      {
        cout<<"string is:"<<add<<endl;
        add = strtok (NULL, " ,.-");
      }
      system ("pause");
      return 0;
    }
    i'm just not sure about the delimiters.. how can i separate the sentence using the delimiters such as tab, space, or '/' key??

    pls help

  2. #2
    Registered User
    Join Date
    Sep 2009
    Posts
    33
    Code:
    #include <stdio.h>
    #include <string.h>
    #include <iostream>
    
    using namespace std;
    
    int main ()
    {
      char str;
      cout<<"Enter a string:"<<flush;
      cin >> str;
      char * pch;
      cout<<"You're splitting"<<str<<endl;
      pch = strtok (str,",.-");
      while (pch != NULL)
      {
        cout<<"%s\n"<<pch<<endl;
        pch = strtok (NULL, " ,.-");
      }
      system ("pause");
      return 0;
    }
    ok here's the latest i've tried.. but still not getting the right result..

    any help?? pls?

  3. #3
    The larch
    Join Date
    May 2006
    Posts
    3,573
    1) You are inputting a single character (previous code attempted to input to an uninitialized pointer).

    2) cin >> only reads input up to the first white-space character (use getline)

    3) you are trying to split the word also at spaces (but street names and such can contain spaces). If you don't somehow escape characters that are also delimiters (if there is no clear format to the input), it might be impossible to determine which part of the string belongs where.
    I might be wrong.

    Thank you, anon. You sure know how to recognize different types of trees from quite a long way away.
    Quoted more than 1000 times (I hope).

  4. #4
    Registered User
    Join Date
    Sep 2009
    Posts
    33
    thanks for that anon..

    still confused with this though..

  5. #5
    Cat without Hat CornedBee's Avatar
    Join Date
    Apr 2003
    Posts
    8,895
    In C++, you should use a istringstream and use >> into a std::string to read the line word by word.
    All the buzzt!
    CornedBee

    "There is not now, nor has there ever been, nor will there ever be, any programming language in which it is the least bit difficult to write bad code."
    - Flon's Law

  6. #6
    Registered User
    Join Date
    Sep 2009
    Posts
    33
    thanks..

    i did this minutes ago.. it compiles ok but i can't see the result as the windows error message pops out..

    Code:
    #include <stdio.h>
    #include <string.h>
    #include <iostream>
    
    using namespace std;
    
    int main ()
    {
      char * add[200];
      cout<<"Enter a string:"<<flush;
      cin.getline(add[200], sizeof(add[200]));
      char *ptr = strtok(add[200], ","); // split the strig at the comma
      cout << "unit num = " << ptr << "\n";
      ptr = strtok( NULL, ",");
      cout << "street address = " << ptr << "\n";
     
      system ("pause");
      return 0;
    }

  7. #7
    Cat without Hat CornedBee's Avatar
    Join Date
    Apr 2003
    Posts
    8,895
    You're placing 200 character pointers on the stack, none of which is initialized. Then you try to read data into the 201st character pointer, meaning that you first access beyond the limits of your pointer array, and then at random memory.

    You just put down code and tweaked until you got no more compile errors, did you?

    Get away from C-style strings. Until you've learned about memory management, you're not ready for them. In other words, forget about strtok. Use std::istringstream and std::string like I said.
    All the buzzt!
    CornedBee

    "There is not now, nor has there ever been, nor will there ever be, any programming language in which it is the least bit difficult to write bad code."
    - Flon's Law

  8. #8
    Registered User
    Join Date
    Sep 2009
    Posts
    33
    hmm.. ok thanks

  9. #9
    Registered User
    Join Date
    Sep 2009
    Posts
    33
    like this??

    Code:
    #include <iostream>
    #include <sstream>
    #include <string>
    using namespace std;
    
    int main()
    {
        char add[200];
        cout<<"Enter an address:"<<flush;
        cin>>add;
        istringstream iss(add);
    
        do
        {
            string sub;
            iss >> sub;
            cout << "Substring: " << sub << endl;
        } while (iss);
    
        system ("pause");
        return 0;
    }
    but it stops after first line..

    like if i enter 89, need street, greenwich.. it only returns
    substring:89
    substring:

    press anything to continue.

  10. #10
    Registered User
    Join Date
    Mar 2009
    Posts
    399
    Code:
    cin >> add
    There's your problem. The >> operator uses whitespace as a delimiter. Use getline instead if you want an entire line.
    Last edited by Memloop; 09-27-2009 at 10:45 AM.

  11. #11
    Cat without Hat CornedBee's Avatar
    Join Date
    Apr 2003
    Posts
    8,895
    And get rid of that char array already!

    Code:
    #include <iostream>
    #include <sstream>
    #include <string>
    using namespace std;
    
    int main()
    {
        string add;
        cout << "Enter an address:" << flush;
        getline(cin, add);
        istringstream iss(add);
    
        string sub;
        while(iss >> sub)
        {
            cout << "Substring: " << sub << endl;
        }
    
        return 0;
    }
    Also, system("pause")? Get a better IDE, like Code::Blocks or Visual Studio.
    All the buzzt!
    CornedBee

    "There is not now, nor has there ever been, nor will there ever be, any programming language in which it is the least bit difficult to write bad code."
    - Flon's Law

  12. #12
    Registered User
    Join Date
    Sep 2009
    Posts
    33
    Code:
    #include <iostream>
    #include <sstream>
    #include <string>
    using namespace std;
    
    int main()
    {
        char add[200];
        cout<<"Enter an address:"<<flush;
        cin.getline(add, 199);
        istringstream iss(add);
    
        do
        {
            string sub;
            iss >> sub;
            cout << "Substring: " << sub << endl;
        } while (iss);
    
        system ("pause");
        return 0;
    }
    yes.. it works now.. thanks guys

    // i have another small question.. how can i sort the address according to their components??

    like 24, michigan street, michigan becomes

    number = 24
    street = michigan street
    town = michigan
    (instead of substring)

    i think this is sufficient already, but i want to make the work look perfect.

  13. #13
    Cat without Hat CornedBee's Avatar
    Join Date
    Apr 2003
    Posts
    8,895
    how can i sort the address according to their components??
    Don't split the string in a loop; instead, split it explicitly into variables, the print the variables however you want.

    But now you've changed the problem, because now you want to split on commas, not spaces.
    All the buzzt!
    CornedBee

    "There is not now, nor has there ever been, nor will there ever be, any programming language in which it is the least bit difficult to write bad code."
    - Flon's Law

  14. #14
    Registered User
    Join Date
    Sep 2009
    Posts
    33
    split explicityly..

    hmm.. can you explain it a bit..

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Begginer Problem: Extracting words from sentence
    By barlas in forum C++ Programming
    Replies: 5
    Last Post: 05-04-2006, 03:17 PM
  2. Beginners Contest #2 For those who wanted more!!
    By ILoveVectors in forum Contests Board
    Replies: 16
    Last Post: 08-12-2005, 12:03 AM
  3. New Theme
    By XSquared in forum A Brief History of Cprogramming.com
    Replies: 160
    Last Post: 04-01-2004, 08:00 PM
  4. Searching for words within a sentence
    By drdroid in forum C++ Programming
    Replies: 4
    Last Post: 02-27-2003, 12:09 AM