strspn problems with '#' character.

I have noticed a problem that I can't explain.

When using `strspn`, it won't recognize the pound symbol unless it's the first character in the string.

Create 2 files:

Code:

---

touch "#here" "there#"

---

Compile this code:

Code:

---

#include <stdio.h>
#include <string.h>

int main (int argc, char *argv[])
{
    printf("Location: %lu\n", (argv[1], "#"));
    return(0);
}

---

When you run `program \#here` it lists the location as 1, which is correct. When you run `program there#` it says the position is 0, which is wrong.

I'm using `strspn` to search for a bunch of special characters, and every other character seems to work with it, except the pound symbol. I understand I won't be able to change this behaviour, but why is it happening?

You left out strspn in your example code, right?

However, what shell do you use to call your program?

> printf("Location: %lu\n", (argv[1], "#"));
What!?

Code:

---

$ gcc -Wall bar.c
bar.c: In function ‘main’:
bar.c:6:39: warning: left-hand operand of comma expression has no effect [-Wunused-value]
    6 |    printf("Location: %lu\n", (argv[1], "#"));
      |                                      ^
bar.c:6:25: warning: format ‘%lu’ expects argument of type ‘long unsigned int’, but argument 2 has type ‘char *’ [-Wformat=]
    6 |    printf("Location: %lu\n", (argv[1], "#"));
      |                      ~~^    ~~~~~~~~~~~~~~
      |                        |            |
      |                        |            char *
      |                        long unsigned int
      |                      %s

---

> When you run `program \#here` it lists the location as 1, which is correct. When you run `program there#` it says the position is 0, which is wrong.
Maybe read the manual page again.

Quote:

---

The strspn() function calculates the length (in bytes) of the initial segment of s which consists entirely of bytes in accept.

---

As written, all you're doing is counting how many leading # there are in the string.
The number and placement of # anywhere else in the string is irrelevant.

Oh, right. However, # is also a character with special meaning in shell languages. It introduces comments for example in bash or PowerShell.

Quote:

---

Originally Posted by aGerman

You left out strspn in your example code, right?

However, what shell do you use to call your program?

---

Sorry, must have cut the function name instead of copying it.

Code:

---

#include <stdio.h>
#include <string.h>
 
int main (int argc, char *argv[])
{
    printf("Location: %lu\n", strspn(argv[1], "#"));

    return(0);

}

---

I use Bash shell, and I am starting to think this may be the problem. I just don't know why it's happening with filenames.

strcspn does what you want. The 'c' stands for compliment, and it counts how many leading characters of your string are not any of the given characters. It returns the length of the string if none of the given characters are in the string.

Code:

---

    size_t n = strcspn(argv[1], "#");
    if (n < strlen(argv[1]))
        printf("First # is at position %zu\n", n);  // %zu is correct for size_t
    else
        printf("# is not in the string\n");

---

You could also use strchr and subtract the original string start from the result (if not NULL).

Ok, this must be related to `strspn`:

Code:

---

#include <stdio.h>
#include <string.h>

int main (int argc, char *argv[])
{
    printf("Location: %lu\n", strspn(argv[1], "#"));

    for(size_t x = 0; x < strlen(argv[1]); x++)
        putchar(argv[1][x]);

    return(0);
}

---

The loop shows that the pound symbol is still a pound symbol inside the program. I guess I won't use `strspn`, I'll have to step through each individual character.

Quote:

---

Originally Posted by john.c

strcspn does what you want. The 'c' stands for compliment, and it counts how many leading characters of your string are not any of the given characters. It returns the length of the string if none of the given characters are in the string.

Code:

---

    size_t n = strcspn(argv[1], "#");
    if (n < strlen(argv[1]))
        printf("First # is at position %zu\n", n);  // %zu is correct for size_t
    else
        printf("# is not in the string\n");

---

You could also use strchr and subtract the original string start from the result (if not NULL).

---

Ty, I will look into these.

Ok, `strcspn` doesn't work. `strcspn` does the exact opposite of my first problem. Yes, it solves the problem of the location with `there#`, but not `#here`. And `strchr` won't work without extra effort. Here is the full scope of what I am trying to do.

I want to remove special characters from filenames. If it is marked for tv, the special character is encoded, to keep the filename safe for processing. It's nice to have an '!' displayed in a filename, without the filename having an '!'. And if it isn't for tv, the character is removed by replacing it with a '.'.

Code:

---

#define SPECIAL_CHARS      "':,\"?&!()$/#;_<>*%|^@"

void fix_file (char *file, int tv_flag)
{
// Declare variables.
    char new_filename[FILENAME_SIZE] = {0};

// Loop through all characters.
    for(size_t x = 0; x < strlen(file); x++)
// If it is a special character.
        if(strspn(&file[x], SPECIAL_CHARS) == 1)
// If it needs to be encoded.
            if(tv_flag == FLAG_YES)
                add_character(file[x], new_filename);
            else
            {
// Remove character.
                new_filename[strlen(new_filename) + 1] = '\0';
                new_filename[strlen(new_filename)] = '.';
            }
        else
        {
// Keep the character.
            new_filename[strlen(new_filename) + 1] = '\0';
            new_filename[strlen(new_filename)] = file[x];
        }
}

---

If I go the `strchr` route, which would work, I would need a bunch of tests for all the special characters I'm searching for. I was hoping to use this function, because I liked the ability of skipping directory entires '.' and '..'

Code:

---

// Skip '.' and '..' entries.
        if(dir->d_name[strspn(dir->d_name, ".")] == '\0')
            continue;

---

I guess I could make one test for '#' with `strchr` outside of the other search.

Thanks for your input guys.

What do you mean by

Quote:

---

with `there#`, but not `#here`

---

with `#here` strcspn says the position is 0, so it works.

EDIT:
If file[x] is a single character (I don't know what the & is for), then you could use strchr backwards, strchr(SPECIAL_CHARS, file[x])

If it returns NULL then file[x] is not a special char.

You could also maybe look at strpbrk

strpbrk - cppreference.com

Quote:

---

Originally Posted by john.c

What do you mean by with `#here` strcspn says the position is 0, so it works.

---

Your right. My local copy of `man strcspn` does not list what errors, or what happens if it is not found. I read somewhere that `strspn` returns 0 when not found. But, I wrongly assumed that 0 in `strcspn` had the same meaning. Ty for clearing this up, this will work.

Quote:

---

Originally Posted by john.c

If file[x] is a single character (I don't know what the & is for)

---

The & is because both functions require a string to search. This is why I was testing for 1, it means the first character is special. If the character is not special, I have to copy the character as is.

Quote:

---

Originally Posted by john.c

then you could use strchr backwards, strchr(SPECIAL_CHARS, file[x])

---

That fixes that problem. Thanks.

Quote:

---

Originally Posted by Yonut

The & is because both functions require a string to search. This is why I was testing for 1, it means the first character is special. If the character is not special, I have to copy the character as is.

---

But strspn returns 1 only when the first character is special and the next character is not special. So strspn("##foo", SPECIAL_CHARS) will return 2.

strspn and strcspn are the wrong tools for this job, as you've found.

strspn() and strcspn() counts how many chars of a set are or aren't in the original string since it's beginning:

Code:

---

  size_t sz = strspn( "#!#abc", "#!" ); // will return 3 because '#' and '!' are the 3 first chars.

---

strcspn() is complementary to this span count. It will count how many chars from the set aren't in the original string, since its beginning:

Code:

---

  size_t sz = strcspn( "abc#!#", "#!" ); // will return 3 because 'a', 'b' and 'c' aren't '#' or '!'

---

That's it. If you use:

Code:

---

  size_t sz = strspn ( "abc#", "#!" ); // will return 0, because 'a' isn't '#' or '!'.

---

Quote:

---

Originally Posted by christop

But strspn returns 1 only when the first character is special and the next character is not special. So strspn("##foo", SPECIAL_CHARS) will return 2.

strspn and strcspn are the wrong tools for this job, as you've found.

---

Actually, following the course of the conversation, you would notice that I changed my use of `strspn` to `strcsnp`, which does the following:

1) If there are no special characters, it returns the length of the original string.
2) If it does start with a special character, it will return 0.

That seems to be what I need. Can you expand on your comment:

Quote:

---

Originally Posted by christop

strspn and strcspn are the wrong tools for this job

---

This is what I test to see if I need to change the filename:

Code:

---

if(strlen(filename) != strcspn(filename, SPECIAL_CHARS))

---

And this is what I use while changing the filename.

Code:

---

for(size_t x = 0; x < strlen(filename); x++)
    if(strchr(SPECIAL_CHARS, filename[x]) != NULL)

---

I'm going character by character. If the next 2 characters are special, that's great, but I'm still only dealing with the first character. The second one can wait till next round.

Consider this:

1. Create a const, static table of character mappings. It would map the entire 8-bit character range to either itself or '.'

2. On entry, loop over each character in the input string. Pass the input chars through the table.

3. If the table value is '.', then

3a. If TV, then call the add_character function as now.

3b. If no TV, store the dot.

4. If the table value is not '.', just store the result.


This brings two questions to mind. First, what is your policy for "out of range" characters, like all the ASCII values below space? Is it okay to have a filename with newlines or tabs in it? Second, what is the status of dot in filenames? If you have a filename like foo.txt, is the dot going to be added to the SPECIAL characters list? Or is dot already handled before this function is called?

Quote:

---

Originally Posted by aghast

This brings two questions to mind. First, what is your policy for "out of range" characters, like all the ASCII values below space? Is it okay to have a filename with newlines or tabs in it? Second, what is the status of dot in filenames? If you have a filename like foo.txt, is the dot going to be added to the SPECIAL characters list? Or is dot already handled before this function is called?

---

If the questions are from you wanting me to 'consider this', then maybe you should answer them.

As for non-displayed ASCII characters, how many files have you downloaded or created that have non-displayed ASCII characters in them? And why are you trying to create them that way?!?

And the period, or 'dot' as you put it, in the filenames are interpreted as spaces. And a period can be encoded for things like `Mr.`, `vs.` or even '...'. Extensions are ignored.

Quote:

---

Originally Posted by aghast

is the dot going to be added to the SPECIAL characters list

---

It's not in the list, and no it won't be added.

Quote:

---

Originally Posted by Yonut

If the questions are from you wanting me to 'consider this', then maybe you should answer them.

As for non-displayed ASCII characters, how many files have you downloaded or created that have non-displayed ASCII characters in them? And why are you trying to create them that way?!?

And the period, or 'dot' as you put it, in the filenames are interpreted as spaces. And a period can be encoded for things like `Mr.`, `vs.` or even '...'. Extensions are ignored.

It's not in the list, and no it won't be added.

---

I cannot answer the questions because they are policy questions, not technical ones. Nothing I have shown you will fail to work depending on how you decide - the questions are just things that invite a "management" decision rather than a "programming" decision.

Non-displayed ASCII characters are such a problem that the GNU project has modified the CLI of many standard utilities to support accepting filenames delimited with ASCII NUL bytes, in an attempt to handle every possible flavor of filename. (Except, obviously, those with NULs in. ;-) Look for utilities that take a -0 (dash zero) flag.

Handling the periods as spaces is fine, that would fit into the lookup table without any extra effort.

Something like this:

Code:

---

#define DOT(ch)  [ch] = '.'
#define SELF(ch) [ch] = (ch)
#define NOP(ch) [ch] = 0

static char Fix_chars[256] = {
// Non-graphic ASCII chars
NOP(0), NOP(1), NOP(2), /* lots of these */
SELF(' '), ... SELF('A'), ... SELF('Z'), ... SELF('a'), ... SELF('z'),
// Specials
DOT('\''), DOT(','), DOT(';'), DOT('"'), ..., DOT('@'),
// High-bit chars? I have no idea.
};


int
fix_filename(
    const char *filename,
    int for_tv)
{
    char * new_name = sm_malloc(PATH_MAX);

    for (char *in = filename, *out=new_name; *in != `\0'; ++in) {   
        int fix = Fix_chars[*in];
        switch (fix) {
        case 0: // skip the character entirely
            break;
        case '.': // special processing
            *out = '\0'; // end string here so add_character can find it
            add_character(*in, new_name);
            // Move forward to end-of-string, which might be *out already,
            // if add_character() adds nothing.
            while (*out != '\0')
                ++out;
            break;
        default:
            *out++ = fix;
            break;
        }
    }
    *out = '\0';
    // optionally shrink allocated buffer
    char *n2 = realloc(new_name, strlen(new_name) + 1);
    return n2 ? n2 : new_name;
}

---

Quote:

---

Originally Posted by aghast

Code:

---

#define DOT(ch)  [ch] = '.'
...

---

---

Don't remember asking people to do something for me. In fact, the OP was about how `strspn` has trouble handling the pound symbol `#`.

And for the 'management' portion of your code, what happens when the file starts with a special character, and it's changed to a period, which makes the file hidden? Also each folder has at least one hidden file which lists the last played file in a given directory. So I don't display hidden files, which mean I would eventually lose storage space to files that just disappear.

It's nice you want to help me by giving me a fish. That's not my style. There are two types of people on the internet offering help... thinkers and regurgitators. Thinkers will help you understand how to fix the problem, regurgitators will just give you the answer. Because you don't have a proper understand of the full scope of my problem, you can't possibly offer me a completed solution.

But thanks?

Quote:

---

Originally Posted by Yonut

what happens when the file starts with a special character, and it's changed to a period, which makes the file hidden?

---

Are you renaming the files themselves, or just displaying their names in some sort of user interface? If the former, I'd be careful not to overwrite existing files. For example, if you have two files "foo@bar" and "foo.bar", if you rename the first file it'll clobber the second one. If the latter, then it's up to your application whether it hides a file or not. If the actual filename doesn't start with a period, then don't hide it. Period.

Quote:

---

Originally Posted by christop

Are you renaming the files themselves, or just displaying their names in some sort of user interface? If the former, I'd be careful not to overwrite existing files. For example, if you have two files "foo@bar" and "foo.bar", if you rename the first file it'll clobber the second one. If the latter, then it's up to your application whether it hides a file or not. If the actual filename doesn't start with a period, then don't hide it. Period.

---

Why does it have to be the same process? Can't one fix the names and another display them.

And I'm using `stat` before `rename` with the new filename, to make sure it isn't replacing an existing file. Why would I have included it in my short example that displays my original problem?

I don't know about you, but sometimes I want to make a filename safe without obscuring what the original filename was. Getting rid of leading periods would hide the original filename. Instead, the smart thing would be, because this can be an automated program, would be to change the files with a leading period, but warn the user (or in a log file) about that instance. Then the file is safe, the file can be reverted back to the original filename, and it wouldn't be lost because there'd be a record of it's existence and location.

For the record, the original problem has been solved, and I just seem to be explaining things that aren't important. I may not return to this post. But thanks to those that helped.

Quote:

---

Originally Posted by Yonut

I don't know about you, but sometimes I want to make a filename safe without obscuring what the original filename was.

---

OK, but I still don't understand what you mean by "safe". All characters that you might find in a file name are "safe" (otherwise the OS wouldn't have let you use them in a file name in the first place).

I just assumed you wanted to escape certain characters for HTML or some other user interface that treats those characters as special. That makes more sense to me than renaming a file from one safe name to another safe name. So if you're renaming files to make them safe (for some definition of "safe") then you're probably solving the problem in the wrong place, and the real problem is how the file names are handled in the shell (or a shell script). At least in Bash (and other Bourn shell derivatives) It's easy enough to handle any file name properly:

Code:

---

ls -l "$filename"

---

That will work for any value of $filename, whether it has a pound sign or an asterisk or a linefeed or what have you.

Aren't you forgetting about charsets? On SysV systems is common the use of UTF-8, so a filename like ":D.txt" is a valid name. It works, including, on Windows (where a modified version of UTF-16 is used)...

The only thing "safe" about filenames is that, in C/C++, they end with '\0'.

Quote:

---

Originally Posted by christop

Quote:

---

Originally Posted by Yonut

I don't know about you, but sometimes I want to make a filename safe without obscuring what the original filename was.

---

OK, but I still don't understand what you mean by "safe". All characters that you might find in a file name are "safe" (otherwise the OS wouldn't have let you use them in a file name in the first place).

---

Really?!? So an exclamation mark (!) in a filename is safe to you? Or it's safe to you because you either quoted it or escaped the offending character. A safe filename does not need to be quoted, or escaped and can be directly sent to a windows machine (if needed) without any problems with special characters on different OSs. When something is safe, you don't have to do anything to it or worry about it, ever.

Quote:

---

Originally Posted by christop

That makes more sense to me than renaming a file from one safe name to another safe name.

---

Why would I rename a safe filename to another safe filename?!? Am I being trolled here or is this a legitimate question to you?!?

Quote:

---

Originally Posted by christop

So if you're renaming files to make them safe (for some definition of "safe") then you're probably solving the problem in the wrong place, and the real problem is how the file names are handled in the shell (or a shell script). At least in Bash (and other Bourn shell derivatives) It's easy enough to handle any file name properly

---

Ahh you mean by quoting it. Yes, taking that extra step would make it an unsafe filename 'safe'. And that would work for Bash, but I'm pretty sure I didn't come to a c programming forum because I'm writing this in Bash.

Quote:

---

Originally Posted by christop

That will work for any value of $filename, whether it has a pound sign or an asterisk or a linefeed or what have you.

---

You sure like to put ridiculous characters in your filenames. Glad I have a way to guard against it.

Just out of curiosity, are you a Millennial or younger?

Spaces in a file name need to be escaped or quoted when used in a shell. Do you consider spaces to be "unsafe"? Are spaces "special"?

Quote:

---

Originally Posted by Yonut

Just out of curiosity, are you a Millennial or younger?

---

Why would my age matter? I've been putting spaces in file names since I've started using *nix in the '90s, and I've never thought of them as "unsafe". I also used spaces in file names back when I still used Windows (which has supported spaces and long file names since '95). I never thought of them as unsafe in Windows either. Otherwise, Windows folders like "Documents and Settings" and "Program Files" would be considered "dangerous".

My Unix systems professor in college threw a fit when I wrote defensive shell scripts with quotes around file name variables. My rationale is that any user might use my script, even someone migrating to/from Windows, where a file name might look like "/mnt/to/windows/Documents and Settings/Joe Bob/Cat Video!.wmv". Better to support "unsafe" characters than to force a user to rename their files only because your script can't handle them.