I don't want to go through the additional trouble of setting up a testing tracker when it's not needed.
So here's my idea: Commission a fairly informal group of bug testers with two main objectives:
Make detailed reportsThis is pretty obvious, and pretty simple to do actually; make a page of your usual computer set up and stick it on a page somewhere. Get used to taking screenshots and an image editting program, and figure out some way to upload images quickly and easily. If you can, FRED a mission to examine the whatever-it-is in more detail.
The point of the detailed results is to make the problem reproducable for anyone using it.
Break stuffThis is why QA gets paid. When a coder posts a new build, it's usually because it's got a new feature, or feature(s).
These need to be broken. How? Purposely do weird stuff. Stress-test the feature. If it lets you put in a numerical value that should be positive, try negative numbers. Try the feature every which way you can.
But all in all, what really needs to be done is for the feature to be used and tested, right then, before it's a couple months later and everyone's moved on.
This is also a great way to learn how to mod through experimentation, because whatever you do wrong is only helping; if it's an understandable mistake, it should be documented and fixed.
It's not necessary for every individual to bugtest every build; if there were only three people bugtesting, one doing the first of every three builds, the next doing the second of every three builds, the third doing the third, but each tested every feature thoroughly, the quality of fs2_open would leap dramatically.
Another thing that could be done is, while some testers were doing the rotating thing, other testers tried to break values in various tables and detailed what worked and what didn't. (This would be things like trying a ship name that's too long, an afterburner image that doesn't exist, ridiculously high values, etc). Invalid table values that crash FS2 are usually notoriously easy to fix, but make things much more moddable because you can quickly throw in a warning telling the modder what exactly went wrong, rather than insulting their mother and popping up an opton to tell Microsoft about it (Or whatever their particular OS does to tell them it crashed)
So, thoughts? Anyone still interested in volunteering?

Inqui, is there any chance of getting a tester-only QA forum set up on indiegames?