Tokenizing a string in C++
Tokenizing a string denotes splitting a string with respect to a delimiter. There are many ways we can tokenize a string. Let’s see each of them:
stringstream
A stringstream associates a string object with a stream allowing you to read from the string as if it were a stream.
Below is the C++ implementation :
// Tokenizing a string using stringstream #include <bits/stdc++.h> using namespace std; int main() { string line = "GeeksForGeeks is a must try"; // Vector of string to save tokens vector <string> tokens; // stringstream class check1 stringstream check1(line); string intermediate; // Tokenizing w.r.t. space ' ' while(getline(check1, intermediate, ' ')) { tokens.push_back(intermediate); } // Printing the token vector for(int i = 0; i < tokens.size(); i++) cout << tokens[i] << '\n'; } |
Output:
GeeksForGeeks is a must try
strtok()
// Splits str[] according to given delimiters. // and returns next token. It needs to be called // in a loop to get all tokens. It returns NULL // when there are no more tokens. char * strtok(char str[], const char *delims);
Below is the C++ implementation :
// C/C++ program for splitting a string // using strtok() #include <stdio.h> #include <string.h> int main() { char str[] = "Geeks-for-Geeks"; // Returns first token char *token = strtok(str, "-"); // Keep printing tokens while one of the // delimiters present in str[]. while (token != NULL) { printf("%s\n", token); token = strtok(NULL, "-"); } return 0; } |
Output:
Geeks for Geeks
Another Example of strtok() :
// C code to demonstrate working of // strtok #include <string.h> #include <stdio.h> // Driver function int main() { // Declaration of string char gfg[100] = " Geeks - for - geeks - Contribute"; // Declaration of delimiter const char s[4] = "-"; char* tok; // Use of strtok // get first token tok = strtok(gfg, s); // Checks for delimeter while (tok != 0) { printf(" %s\n", tok); // Use of strtok // go through other tokens tok = strtok(0, s); } return (0); } |
Output:
Geeks for geeks Contribute
strtok_r()
Just like strtok() function in C, strtok_r() does the same task of parsing a string into a sequence of tokens. strtok_r() is a reentrant version of strtok().
There are two ways we can call strtok_r()
// The third argument saveptr is a pointer to a char * // variable that is used internally by strtok_r() in // order to maintain context between successive calls // that parse the same string. char *strtok_r(char *str, const char *delim, char **saveptr);
Below is a simple C++ program to show the use of strtok_r() :
// C/C++ program to demonstrate working of strtok_r() // by splitting string based on space character. #include<stdio.h> #include<string.h> int main() { char str[] = "Geeks for Geeks"; char *token; char *rest = str; while ((token = strtok_r(rest, " ", &rest;))) printf("%s\n", token); return(0); } |
Output:
Geeks for Geeks
Recommended Posts:
- String slicing in Python to check if a string can become empty by recursive deletion
- Lexicographically smallest string formed by appending a character from first K characters of a string | Set 2
- String Range Queries to find the number of subsets equal to a given String
- Minimal moves to form a string by adding characters or appending string itself
- Create a new string by alternately combining the characters of two halves of the string in reverse
- Find the longest sub-string which is prefix, suffix and also present inside the string | Set 2
- Lexicographically smallest string formed by appending a character from the first K characters of a given string
- Find the longest sub-string which is prefix, suffix and also present inside the string
- Find the character in first string that is present at minimum index in second string
- Check if a string can be converted to another string by replacing vowels and consonants
- Given a string and an integer k, find the kth sub-string when all the sub-strings are sorted according to the given condition
- Minimum deletions from string to reduce it to string with at most 2 unique characters
- Find the count of palindromic sub-string of a string in its sorted form
- Minimum number of given operations required to convert a string to another string
- Find length of longest subsequence of one string which is substring of another string
If you like GeeksforGeeks and would like to contribute, you can also write an article using contribute.geeksforgeeks.org or mail your article to contribute@geeksforgeeks.org. See your article appearing on the GeeksforGeeks main page and help other Geeks.
Please Improve this article if you find anything incorrect by clicking on the "Improve Article" button below.


