Video game development, as an industry, is incredibly immature. At a recent Boston IGDA meeting, the IGDA spokesperson, that Della Roca guy, was advocating people start doing rapid and iterative development.
A software development methodology that is at least 15 years old in the form he was advocating. We play catch up in development, we play catch up in marketing and public relations.
Focus groups and representative slices ARE the way to get useful feedback. Seeking people outside the typical eager beta user is critical for a titles success. You can't build any product that only satisfies a small percentage of your target market and expect it to succeed. So you can't let those people dictate development. You can't always be reacting to those criticisms. Drop everything and add an economy model to FS2 is not the way to get anything substative done in a project like the SCP, and the SCP itself is entirely focused on satisfying "hardcore" fans. If it can't work there (consistently), it can't work anywhere, especially a commercial venture.
We had a bull**** filter when we did Lore. We had community relations people, and then we had meetings where we actually sifted thru the drek that was most of the comments. We were definitely using the community for feedback and ideas, but you have to have that bull**** filter if you are going to succeed (not that I would call Lore a financial success by any stretch). Even with our filter, we ended up with some bad design choices because we paid TOO much attention to those hardcore players. Stuff that didn't become evident until we had large scale battles, or non hardcore people finally playing. Balance issues, level design, you name it. We should have stuck more to our guns than we did, and it ended up costing us.
Far too many developers build their designs around reacting to perceptions of a very, very small slice of folks.
Build a game, make it playable, then find
of DIFFERENT people to play it and use THAT feedback.
Sadly, that will take just as long to sink in for this industry as "rapid development" is taking
re-reading what I posted and it sounds like I agree with the article 100%, I don't, there IS a place for hardcore feedback, its just got to be weighted equally with other feedback. I'm like 95% with him.