[Herra Tohtori]

So, as I came to notice this fine afternoon, the behaviour of FS2_Open regarding filenames longer than 32 characters has changed.

Previously, it apparently just failed to read them with no warnings or other adverse effects as far as I could tell, but recent builds (don't know what revision the change was implemented in) this changed; debug builds now show an Assertion Failed! window and then crash with a Windows error message, while standard builds just crash with Windows error message.

This is mainly an issue with people who mod stuff since it's fairly easy to surpass 32 character limit with files in WIP directories, since I at least tend to name them in somewhat descriptive manner which easily pushes the name over the limit. For example if I had a texture called "jupiterboxcombined_realistic.tga" in my Jupiter skybox WIP dir and that caused the following assertion failure:

Code: [Select]

Assert: 0
File: safe_strings.cpp
Line: 70
ERANGE: String error @ d:\scp testing\code\cfile\cfilesystem.cpp (570). Please Report
Trying to put into 32 byte buffer:
jupiterboxcombined_realistic.tga

<no module>! KiFastSystemCallRet
<no module>! WaitForSingleObject + 18 bytes
<no module>! <no symbol>
<no module>! <no symbol>
<no module>! <no symbol>
<no module>! <no symbol>
<no module>! <no symbol>
<no module>! <no symbol>
<no module>! <no symbol>
<no module>! <no symbol>
<no module>! <no symbol>
<no module>! <no symbol>
<no module>! <no symbol>
<no module>! <no symbol>
<no module>! <no symbol>
<no module>! RegisterWaitForInputIdle + 73 bytes

At least debug builds give a warning; however I was trying to get SSE2 builds working and the builds just kept crashing, and I was quite convinced it was an issue related to SSE2 specifically, and since the SSE2 build didn't come with corresponding debug build I just kept using older build that didn't register long file names visibly.

It wasn't until I tried a regular recent build and it refused to run too that I realized the issue wasn't the SSE2 but rather something changed in the recent builds, and running debug build clarified the issue. Moving long file names out of the directories that FS2_Open reads on startup solved the issue, but now I need to go through all my WIP dirs and check if any of the file names there exceed the limit.

Anyway, just figured to give a heads-up on the subject. Yeah, I know, running debug would have shown what the problem was immediately. What can I say, I was fixated on the SSE2 support being the most probable cause of problem since as far as I could tell it was the main change between the builds.

[Fury]

And I suggested someone could write a small neat app to recursively scan a directory for files longer than 32 characters. Short googling didn't find any.

[castor]

Here's the linux version

Code: [Select]

find . | awk -F / '{if (length($(NF)) > 32) print}'


[karajorma]

When you get a crash you're meant to report it. It even says so at the top of the error message.

[Herra Tohtori]

When you get a crash you're meant to report it. It even says so at the top of the error message.

Yes, and I reported it as soon as I saw the error message. Regular builds don't show the error, they just crash with Windows error message. Like I said, the reason why I didn't immediately try the debug build was my false assumption that the problem was related to SSE2 instructions, and I simply didn't try the normal versions of the build until this evening...

[Rodo]

It could be related to this patch.

not quite sure though...

[Echelon9]

As portej05 hasn't had the chance to step in here:

The 'safe_strings' related informative errors are the result of work Portej05 has done to ensure the FS2_Open engine doesn't silently do a class of memory corruption operations. The error occurs because it's warning you that something occurred to breach the size of a buffer; which previously wouldn't have been reported and lead to a number of hard to find bugs.

The 32 character limit is one of the most readily accessible sections of code that triggers safe_strings at the moment, AFAIK there isn't any easy way to simply bump that limit up without breaking something else.

[Iss Mneur]

And I suggested someone could write a small neat app to recursively scan a directory for files longer than 32 characters. Short googling didn't find any.

Here is a precompiled win32 version with code.

Drop the .exe into the root directory, and run it.  It then generates a file 'names{unixtime}.txt' with the path and the filename of any file that is 32 characters or longer including the extension.

The .exe was compiled with VC++ 2008 Express on Windows Vista x64.  It should work on all versions of windows including x86.

Code: ("file_name_length_checker.cpp") [Select]

#include <time.h>
#include <stdio.h>
#include <stdlib.h>
#include <sys/types.h>
#include <io.h>

#if _DEBUG
#define DEBUG 1
#endif

#define OUTFILENAMESIZE 50
#define MAXLENGTHOFNAME 32


int name_to_long(char * name) {
int counter = 0;
while (name[counter] != NULL) counter++;
if (counter >= MAXLENGTHOFNAME) {
return 1;
} else {
return 0;
}
}

void filesearch(FILE *outfile, char* dir) {
_finddata_t* data = (_finddata_t*)malloc(sizeof(_finddata_t));
char * searchpattern;
if (NULL == dir) {
searchpattern = "*";
} else {
searchpattern = (char*)malloc(sizeof(char)*_MAX_PATH);
// \\ is specific to windows but then so is _findfirst and next
sprintf_s(searchpattern, sizeof(char)*_MAX_PATH, "%s\\*", dir);
}

printf("Searching: %s\n", searchpattern);

intptr_t files = _findfirst(searchpattern, data);
if (NULL == files) {
printf("Error searching files.");
return;
}
do {
// don't try . and ..
if ( data->name[0] == '.' && data->name[1] == NULL
|| data->name[0] == '.' && data->name[1] == '.' && data->name[2] == NULL) {
// do nothing
} else {
#if DEBUG
printf("Found: %s of type %x\n", data->name, data->attrib);
#endif
if ( data->attrib & _A_SUBDIR ) {
#if DEBUG
printf("Found directory. Recursing into.\n");
#endif
char* newdir = (char*)malloc(sizeof(char)*_MAX_PATH);
if ( NULL == dir ) {
sprintf_s(newdir,sizeof(char)*_MAX_PATH, "%s", data->name);
} else {
sprintf_s(newdir,sizeof(char)*_MAX_PATH, "%s\\%s", dir, data->name);
}
filesearch(outfile, newdir);
free(newdir);
} else {
if (name_to_long(data->name) == 1) {
if (NULL == dir) {
fprintf(outfile, "%s\n\r", data->name);
} else {
fprintf(outfile, "%s\\%s\n\r", dir, data->name);
}
}
}
}
} while(0 == _findnext(files, data));
_findclose(files);
if (NULL != dir) free(searchpattern);
}

int main(char ** argv, int argc) {
time_t t = time(NULL);

FILE *outfile;
char* outfilename = (char *)malloc(sizeof(char)*OUTFILENAMESIZE);
sprintf_s(outfilename, sizeof(char)*OUTFILENAMESIZE, "names%d.txt", t);
printf("Storing file names found into: %s\n", outfilename);
outfile = fopen(outfilename, "w");
if (NULL == outfile) {
fprintf(stderr, "Unable to open file to store filenames.\n");
scanf_s("%s");
return 1;
}

filesearch(outfile, NULL);

fclose(outfile);
// scanf_s("%s");
return 0;

}

Edit: Fixed app description to reflect how the program actually works.

[attachment deleted by Tolwyn]

[Fury]

Nice first post. Thanks, seems to work.

[portej05]

The next builds should support safe_strings errors in both release and debug modes (rather than just crashing with a 'divide by zero' error in release mode).
I've made the changes and confirmed they work under VS2008 - platform maintainers please check that this still compiles and works right.

The CRT functions will NOT be used even if they are present.

[taylor]

Be aware that filenames need to be less than 32 characters.  The max length is actually 31 characters, with 32 being the nul terminator.

Both of the tests for filename length posted earlier still allow for filenames that are too long by one character.  Just FYI.

[portej05]

Both of the tests for filename length posted earlier still allow for filenames that are too long by one character.  Just FYI.

The engine will now pick them up and die with an error.

[Herra Tohtori]

The file name checker posted earlier seems to work correctly then:

Code: [Select]

#define MAXLENGTHOFNAME 32

int name_to_long(char * name) {
int counter = 0;
while (name[counter] != NULL) counter++;
if (counter >= MAXLENGTHOFNAME) {
return 1;
} else {
return 0;
}
}
If filename is 32 characters long or longer, it returns 1. I checked, and it did list the exactly 32 character long filenimes.

As a side note: Good thing FS2_Open doesn't read the .svn directories on mediaVP svn dir...

[Iss Mneur]

If filename is 32 characters long or longer, it returns 1. I checked, and it did list the exactly 32 character long filenimes.

Ya, it seems I wrote the description incorrectly.  I have fixed my original post to reflect what the code actually does which according to Taylor is the actually the correct solution.

Stupid off by one errors....

[Herra Tohtori]

Yeah, well. As they say, there are 10 kinds of people:

1. those who start listing from "1", and...

1. those who start listing from "0".

[Aardwolf]

Yeah, well. As they say, there are 10 kinds of people:

1. those who start listing from "1", and...

1. those who start listing from "0".

Hahaha, I get it!