[portej05]

This is a public post as we'd like to receive the maximum feedback possible on this code.
The tests included in the code are minimal, however we'd like people to have a play and see how they go with it.

ANY AND ALL FEEDBACK IS WELCOME, WANTED AND HIGHLY APPRECIATED (if you wish to keep questions/comments private, we can be found at #scp on EsperNet or you can PM portej05)

The following files should be compiled and linked and the tests run.
There should be no compile errors or warnings (sorry non-VC guys, you'll need to do a search/replace __debugbreak with whatever your platform uses).
VC6 users should receive some sort of platform warning about double freeing a pointer.
All other users should hit a breakpoint at line 200 of scpmemorytrack.h

As always, ENJOY!

Edit: Updated to fix a bunch of problems we've seen in VC6 builds
Edit2: Thanks for taking part!
Edit3: Thanks to some incredible detective work by IssMneur, this is open once again (description in new post)

[attachment deleted by admin]

[Aardwolf]

I was a bit confused by the instructions, but I've got it sorted out now.

I had assumed that this had something to do with the existing FSO code, when it is in fact its own project... a "unit test" of a feature that has not yet been put into a test build.

[Bobboau]

what is this?

[Aardwolf]

This is a unit test of something they want to implement for most of the codebase, I think. They want to test on all sorts of systems before they do it, of course, because otherwise they're going to have a heckuva lot of bug complaints to deal with.

They're planning on overloading the new operator because it's bad somehow. I didn't even know that was possible until I heard about it here

[Bobboau]

so is this for tracking memory leaks?

[chief1983]

The current memory tracking apparently has issues (apparently it hasn't ever been completely accurate), and they're going to try to sort them out with this new allocator.

[portej05]

There's a number of reasons for developing these:
1) We've (currently) got no way of tracking new/delete operations
2) We're using more and more C++ constructs in FSO, and we've noticed that occasionally someone has used malloc followed by memset(0) on an object containing a vector or other construct which initialises to a non-zero state
3) There is now the possibility of implementing and creating different allocator styles. We're experimenting with an O(1) allocator, pool allocators and other types of allocators for regularly used objects
4) We'll have a complete call stack for allocated objects (not currently implemented - that's tracking, not allocation)

We're aiming for performance improvements, code quality improvements and debugging improvements.

Aardwolf: It's not that it's bad, it's that it isn't trackable in FSO at present
Also, it's weakly linked, so the way we do it is possible (you can overload it in some quite interesting ways)

Also, this is not a unit test, this is a compile test. We're hoping people will play with it a little, and hopefully trip it up in ways that we haven't been able to imagine.

[Echelon9]

We're hoping people will play with it a little, and hopefully trip it up in ways that we haven't been able to imagine.

Let me at it (along with the peculiar trio GCC 4.2, LLVM and Clang)! hehehehe

[Echelon9]

Does not cleanly compile with G++ 4.2.1 on OS X for me. The attached patch was applied to address the __debugbreak() issue and the location of malloc.h on OS X.

Code: [Select]

$ g++ --version
i686-apple-darwin10-g++-4.2.1 (GCC) 4.2.1 (Apple Inc. build 5646) (dot 1)
Copyright (C) 2007 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

$ g++ trackertest.cpp -o trackertest
In file included from scpmemory.h:91,
                 from trackertest.cpp:4:
scppatterns.h: In member function ‘T* Singleton<T, SingletonAllocator_T>::operator->()’:
scppatterns.h:56: error: expected primary-expression before ‘>’ token
scppatterns.h:56: error: expected primary-expression before ‘)’ token
scppatterns.h: In destructor ‘Singleton<T, SingletonAllocator_T>::~Singleton()’:
scppatterns.h:64: error: expected primary-expression before ‘>’ token
In file included from scpmemory.h:93,
                 from trackertest.cpp:4:
scpmemorytrack.h: In member function ‘T* memory::SCP_untracked_allocator<T>::allocate(size_t) const’:
scpmemorytrack.h:77: error: ‘length_error’ is not a member of ‘std’
In file included from trackertest.cpp:4:
scpmemory.h: In member function ‘Object_T* memory::StaticAllocator<StaticAllocator_T>::Allocate()’:
scpmemory.h:167: error: expected primary-expression before ‘>’ token
scpmemory.h:167: error: expected primary-expression before ‘)’ token
scpmemory.h: In member function ‘Object_T* memory::StaticAllocator<StaticAllocator_T>::Copy(const Object_T&)’:
scpmemory.h:174: error: expected primary-expression before ‘>’ token
scpmemory.h: In member function ‘void memory::StaticAllocator<StaticAllocator_T>::Deallocate(Object_T*)’:
scpmemory.h:181: error: expected primary-expression before ‘>’ token
scpmemory.h: In member function ‘Object_T* memory::StaticAllocator<StaticAllocator_T>::AllocateArray(size_t)’:
scpmemory.h:188: error: expected primary-expression before ‘>’ token
scpmemory.h: In member function ‘void memory::StaticAllocator<StaticAllocator_T>::DeallocateArray(Object_T*)’:
scpmemory.h:195: error: expected primary-expression before ‘>’ token
[attachment deleted by admin]

[portej05]

Due to VC6 issues, the current allocator design is being withdrawn.
Thanks for taking part!

[Goober5000]

Hopefully not as such.  There's still a fair bit of discussion going on in the SCP IRC channel.  Stay tuned.

[portej05]

IssMneur managed to chase down that VC6 doesn't like templated functions which don't use template arguments in their call parameters.
Allocators have been modified to take this into account.

Please test to destruction (note, tracking component is NOT implemented in this module presently).

[KeldorKatarn]

If I were you guys, I'd seriously consider dropping VC6 support if it is corrupting development like this. I know it was one of the most loved C/C++ IDEs around, but it was also the least standard conform C/C++ IDE around, and with free versions of Visual C++ express around and other great free IDEs around I think you should consider this.

This is just meant as an advice. I don't care if you keep supporting VC6, but in the long run I think it will cause you more troubles than it solved. Especially since it most likely will keep you from ever using the new upcoming C++ standard features and more modern libraries.

A lot of development studios didn't like to do that also, but did in the end. I know the express versions of VC++ aren't idea for development, but they are good enough for people to build the engine, and for development there are other great free IDEs like Eclipse or Code::Blocks and from what I know you already support the latter one.

[Goober5000]

We have been considering it.  So far we've managed to work through the problems, and we're continuing to do so.

The latest trunk code still compiles and runs fine on VC6 and Win98.  I'm in the process of testing the allocators.

[Goober5000]

Apparently it's necessary to #include "scpcompiler.h" in each file, or the symbols won't be #define'd correctly.  This is the case even if scpcompiler.h is #included in one of the files you include.

Anyway, with that done, I get the following errors in VC6...

Code: [Select]

--------------------Configuration: allocators_test - Win32 Debug--------------------
Compiling...
scpmemory.cpp
scpnewdelete.cpp
trackertest.cpp
c:\my documents\visual studio projects\visual c++\allocators_test\trackertest.cpp(50) : error C2275: 'Object' : illegal use of this type as an expression
        c:\my documents\visual studio projects\visual c++\allocators_test\trackertest.cpp(26) : see declaration of 'Object'
c:\my documents\visual studio projects\visual c++\allocators_test\trackertest.cpp(50) : error C2059: syntax error : ')'
c:\my documents\visual studio projects\visual c++\allocators_test\trackertest.cpp(53) : error C2275: 'Object' : illegal use of this type as an expression
        c:\my documents\visual studio projects\visual c++\allocators_test\trackertest.cpp(26) : see declaration of 'Object'
c:\my documents\visual studio projects\visual c++\allocators_test\trackertest.cpp(62) : error C2275: 'Object' : illegal use of this type as an expression
        c:\my documents\visual studio projects\visual c++\allocators_test\trackertest.cpp(26) : see declaration of 'Object'
Error executing cl.exe.

allocators_test.exe - 4 error(s), 0 warning(s)

[chief1983]

Yeah that's about where we were a week ago when I was testing it.

[Iss Mneur]

For the record, I can get the above code (posted 2010-01-05, 11:17:59) that portej05 posted to compile and run (as far is it is supposed to according to portej05) with only the below modifications.

Code: (extract of trackertest.cpp) [Select]

int main( )
{
using namespace memory;
Object* ptr = Heap->Allocate( (Object*)NULL );
ptr->d.DoSomething( );

Heap->Deallocate( ptr );

std::vector< Object > vector_test;
for ( size_t i = 0; i < 100; i++ )
{
Object temp;
vector_test.push_back( temp );
}

Heap->Deallocate( ptr );

return 0;
}


[chief1983]

Still had warnings though didn't it?

[Iss Mneur]

Still had warnings though didn't it?

Yes, warnings about the "identifier was truncated to '255' characters in the debug information".  I don't think there is much we can do about these.

Code: [Select]

Compiling...
scpmemory.cpp
scpnewdelete.cpp
trackertest.cpp
c:\program files (x86)\microsoft visual studio\vc98\include\vector(39) : warning C4786: 'std::reverse_iterator<std::basic_string<char,std::char_traits<char>,std::allocator<char> > const *,std::basic_string<char,std::char_traits<char>,std::allocator<
char> >,std::basic_string<char,std::char_traits<char>,std::allocator<char> > const &,std::basic_string<char,std::char_traits<char>,std::allocator<char> > const *,int>' : identifier was truncated to '255' characters in the debug information
        c:\program files (x86)\microsoft visual studio\vc98\include\vector(39) : while compiling class-template member function '__thiscall std::vector<std::basic_string<char,std::char_traits<char>,std::allocator<char> >,std::allocator<std::basic_st
ring<char,std::char_traits<char>,std::allocator<char> > > >::std::vector<std::basic_string<char,std::char_traits<char>,std::allocator<char> >,std::allocator<std::basic_string<char,std::char_traits<char>,std::allocator<char> > > >(const std::allocato
r<std::basic_string<char,std::char_traits<char>,std::allocator<char> > > &)'
c:\program files (x86)\microsoft visual studio\vc98\include\vector(39) : warning C4786: 'std::reverse_iterator<std::basic_string<char,std::char_traits<char>,std::allocator<char> > *,std::basic_string<char,std::char_traits<char>,std::allocator<char>
>,std::basic_string<char,std::char_traits<char>,std::allocator<char> > &,std::basic_string<char,std::char_traits<char>,std::allocator<char> > *,int>' : identifier was truncated to '255' characters in the debug information
        c:\program files (x86)\microsoft visual studio\vc98\include\vector(39) : while compiling class-template member function '__thiscall std::vector<std::basic_string<char,std::char_traits<char>,std::allocator<char> >,std::allocator<std::basic_st
ring<char,std::char_traits<char>,std::allocator<char> > > >::std::vector<std::basic_string<char,std::char_traits<char>,std::allocator<char> >,std::allocator<std::basic_string<char,std::char_traits<char>,std::allocator<char> > > >(const std::allocato
r<std::basic_string<char,std::char_traits<char>,std::allocator<char> > > &)'
c:\program files (x86)\microsoft visual studio\vc98\include\vector(39) : warning C4786: 'std::vector<std::basic_string<char,std::char_traits<char>,std::allocator<char> >,std::allocator<std::basic_string<char,std::char_traits<char>,std::allocator<cha
r> > > >::vector<std::basic_string<char,std::char_traits<char>,std::allocator<char> >,std::allocator<std::basic_string<char,std::char_traits<char>,std::allocator<char> > > >' : identifier was truncated to '255' characters in the debug information
c:\program files (x86)\microsoft visual studio\vc98\include\vector(60) : warning C4786: 'std::vector<std::basic_string<char,std::char_traits<char>,std::allocator<char> >,std::allocator<std::basic_string<char,std::char_traits<char>,std::allocator<cha
r> > > >::~vector<std::basic_string<char,std::char_traits<char>,std::allocator<char> >,std::allocator<std::basic_string<char,std::char_traits<char>,std::allocator<char> > > >' : identifier was truncated to '255' characters in the debug information
Linking...

allocatortest.exe - 0 error(s), 4 warning(s)


[Goober5000]

Anyway, with that done, I get the following errors in VC6...

Okay, ignore what I said.  It looks like all I need to do to make it work (aside from #include'ing scpcompiler.h) is to remove the (Object_T*) casts of NULL.

Once that's done, I get the debug warnings about identifiers being truncated.  But that's been present in trunk ever since Kazan added std::vector many moons ago, so it's not really a problem.