Hard Light Productions Forums

Community Projects => The FreeSpace Wiki Project => Topic started by: General Battuta on November 07, 2010, 09:06:41 am

Title: Should campaigns be given numerical rankings on the Wiki?
Post by: General Battuta on November 07, 2010, 09:06:41 am
I want to get a general sense of feeling on the topic.

EDIT: be aware that a component score is not possible at all right now since the script is apparently not open to customization.
Title: Re: Should campaigns be given numerical rankings on the Wiki?
Post by: TopAce on November 07, 2010, 09:10:32 am
We already had this discussion several times before.

And I said in the thread at the FSWiki board, rating not only is inaccurate, but in the numbers they exist in this community, the outcome is not representative. Let's deal with it: This is not the Internet Movie Database where even the least popular products are voted on 100-150 times.
Title: Re: Should campaigns be given numerical rankings on the Wiki?
Post by: General Battuta on November 07, 2010, 09:11:35 am
We already had this discussion several times before.

And I said in the thread at the FSWiki board, rating not only is inaccurate, but in the numbers they exist in this community, it is not representative. Let's deal with it: This is not the Internet Movie Database.

This debate is happening again right now in the 'best mod' thread, which is why I want to have this poll.

I am not advocating any position (well, okay, I'm advocating 'no').
Title: Re: Should campaigns be given numerical rankings on the Wiki?
Post by: TopAce on November 07, 2010, 09:12:37 am
Yes, I'm following that thread. It's too bad that it got derailed into this subject.
Title: Re: Should campaigns be given numerical rankings on the Wiki?
Post by: Mobius on November 07, 2010, 09:13:25 am
I still think a component score would work, as I expect the values given by voters to be more representative that way. I may be wrong, though.

EDIT: I request this poll be linked by admins to news (right above "home", "help", "search", etc. etc.) to get as many votes as possible.
Title: Re: Should campaigns be given numerical rankings on the Wiki?
Post by: General Battuta on November 07, 2010, 09:22:28 am
Component score is not possible.
Title: Re: Should campaigns be given numerical rankings on the Wiki?
Post by: Mobius on November 07, 2010, 09:25:07 am
Not with the Wiki's rating bar. It's still possible to create polls anywhere else (I suggest using Game Warden's poll system), work a bit on statistics, and post the results on each campaign's article on the Wiki as normal text. It's more complex and requires a lot of reliability (unless we decide to use Game Warden).
Title: Re: Should campaigns be given numerical rankings on the Wiki?
Post by: General Battuta on November 07, 2010, 09:26:58 am
Unless the polls came with a sampling frame analysis I would edit them out of every article I saw them in.
Title: Re: Should campaigns be given numerical rankings on the Wiki?
Post by: headdie on November 07, 2010, 09:28:42 am
I went component score because at the end of the day not everybody look for the same things. 

I still stand by my position that an anonymous rating system would be beneficial for new members, those of us who are regulars to the board have a reasonable idea what's good or not even if we haven't played it because of the feedback we have read.  But people who are new to the HLP or dont browse it often don't have this advantage so will find it harder to find mods they would like.

EDIT: I request this poll be linked by admins to news (right above "home", "help", "search", etc. etc.) to get as many votes as possible.

I support this to if possible to implement
Title: Re: Should campaigns be given numerical rankings on the Wiki?
Post by: Mobius on November 07, 2010, 09:31:25 am
Unless the polls came with a sampling frame analysis I would edit them out of every article I saw them in.

The text will link to Game Warden's polls so that people will be always updated on how the votes change.
Title: Re: Should campaigns be given numerical rankings on the Wiki?
Post by: General Battuta on November 07, 2010, 09:34:24 am
Unless the polls came with a sampling frame analysis I would edit them out of every article I saw them in.

The text will link to Game Warden's polls so that people will be always updated on how the votes change.

That's not a sampling frame analysis.
Title: Re: Should campaigns be given numerical rankings on the Wiki?
Post by: Mobius on November 07, 2010, 09:38:47 am
Why not?
Title: Re: Should campaigns be given numerical rankings on the Wiki?
Post by: The E on November 07, 2010, 09:52:21 am
Game Warden? No disrespect, but that place is dead. (And I would argue that it doesn't need a resurrection).

And it would make the problem even worse, as separating the voting site from the subject being voted on is not a good idea, I think.

Also, if I understand the term correctly, "sampling frame analysis" means metadata about the voters. Gathering that is a necessary first step to ensure statistical validity.
Title: Re: Should campaigns be given numerical rankings on the Wiki?
Post by: General Battuta on November 07, 2010, 09:57:34 am
Why not?

Do you know any sampling statistics? The_E has it right.
Title: Re: Should campaigns be given numerical rankings on the Wiki?
Post by: Qent on November 07, 2010, 10:28:13 am
So... we're voting on whether or not to vote? :wtf: (I know this is just for opinions, but still.)

I think being able to vote from 1-10 with the option of adding a review would be good, but that it has no place on the wiki. Now I think I said this before, but isn't all this already on FSMods?
Title: Re: Should campaigns be given numerical rankings on the Wiki?
Post by: The E on November 07, 2010, 10:33:12 am
Yes, it is. And it's hardly ever used for campaigns that are not available for download there.
Title: Re: Should campaigns be given numerical rankings on the Wiki?
Post by: Droid803 on November 07, 2010, 12:02:30 pm
Yes, it is. And it's hardly ever used for campaigns that are not available for download there.

It's hardly used period.
Title: Re: Should campaigns be given numerical rankings on the Wiki?
Post by: Thaeris on November 07, 2010, 12:27:14 pm
Let's look at it this way...

When I want to find a campaign to play, I load up the campaign list on the Wiki. I read the description, the requirements, and any veteran comments. I get far more out of the veteran comments than any numeric rating. Furthermore, if something is rated poorly, I want to understand why its rated poorly.

Thus, this is my conclusion - keep the campaign listing the way it is. It works just fine, and if you are wanting to play something, you are more than capable of using your own judgement in determining if you'll like it.

Otherwise, if you must include a numeric rating with a campaign, any voter must include a comment with the said vote. This might be comparable to ratings you find on iTunes or other such download sites. In that manner, you'll help prevent any empty imput for or against a campaign; empty imput does no one much good, after all.
Title: Re: Should campaigns be given numerical rankings on the Wiki?
Post by: Droid803 on November 07, 2010, 12:45:37 pm
Agreed.
It's a lot more helpful to know why someone didn't like something, as I might actually like it in contrast.
Title: Re: Should campaigns be given numerical rankings on the Wiki?
Post by: General Battuta on November 07, 2010, 12:48:12 pm
+1 to thaeris
Title: Re: Should campaigns be given numerical rankings on the Wiki?
Post by: Snail on November 07, 2010, 01:05:53 pm
Also, an unweighted scale would probably cause people to vote either 1 or 10 (out of 10) in order to get a greater impact on the results.
Title: Re: Should campaigns be given numerical rankings on the Wiki?
Post by: Qent on November 07, 2010, 01:18:12 pm
Also, an unweighted scale would probably cause people to vote either 1 or 10 (out of 10) in order to get a greater impact on the results.
I disagree. Unless Battuta says people do that. :P I know I don't.
Title: Re: Should campaigns be given numerical rankings on the Wiki?
Post by: TopAce on November 07, 2010, 01:38:08 pm
Also, an unweighted scale would probably cause people to vote either 1 or 10 (out of 10) in order to get a greater impact on the results.

There'll always be haters and unconditional admirers. If you belong to either group, it's a strong incentive to vote. If you simply played the campaign and found it adequately fun with some glitches, you'll just forget about it and move along.
Title: Re: Should campaigns be given numerical rankings on the Wiki?
Post by: General Battuta on November 07, 2010, 01:42:53 pm
Yeah, I am with TopAce.
Title: Re: Should campaigns be given numerical rankings on the Wiki?
Post by: Lucika on November 07, 2010, 03:07:16 pm
[...] if you say that - for instance - a 1 to 10 system wouldn't be accurate enough because people would mostly vote 1s and 10s, how can you argue for a tiered system which is basically just a rebranded 1 to 3 system?

Don't worry, no one will fire up the FS Wiki and go for a campaign that he (or she :D) dislikes just to vote it down. Personally I have no problems with the 1-100 scale idea, but if I'd be the one who decides, I'd make more (and longer) "text reviews" - basically expanded player comments on the Wiki page.

My vote went for flat score though.
Title: Re: Should campaigns be given numerical rankings on the Wiki?
Post by: Snail on November 07, 2010, 03:48:39 pm
My assumptions were based on pie graphs of user ratings on gaming sites.
Title: Re: Should campaigns be given numerical rankings on the Wiki?
Post by: Black Wolf on November 08, 2010, 03:26:49 am
OK, well if these poll figures are accurate, and I suppose they probably are, there seems to be much stronger support against doing this for campaigns, or at least as a standard addition. And while I disagree with the result, I do respect majority opinion... more or less. :p
So here's how I think I'm going to handle it. I still want to implement the script for mods (i.e. new ships), and this seems to have been a lot less controversial. So, once I get back to my main comp, and once Goober puts the script into the wiki PHP, I'll figure out the best way to apply it there (I'm hoping I can use a template of some kind – that'll make it a hell of a lot easier), and then I'm going to integrate the functionality into the modding portal to try to make that a bit more interesting/useful.

However, the functionality, once it's integrated into the wiki, can't and wont be limited just to mods. Thus, any campaign that chooses to put a rating bar on their page will be able to do so. Admittedly, without full coverage over every campaign it's far less useful, but it can still be used on a per-campaign basis in addition to some kind of text review section.

If, however, enough campaigns do end up opting in, we may want to re-open this discussion at some future point.
Title: Re: Should campaigns be given numerical rankings on the Wiki?
Post by: Dilmah G on November 08, 2010, 04:00:40 am
I voted yes, but anyway. I think that course of action sounds alright.

I think we could achieve perhaps what the numerical rankings aimed to achieve with campaigns by encouraging people to write reviews for the campaigns; I recall when I first got here and started to find my way around this confusing list of campaigns on the wiki (wasn't that confusing in the end though, after I worked out someone had taken the time out to categorize them), I based a lot of my decision on whether to play a campaign based on the comments on it. Full blown reviews, even if they're only a few paragraphs would probably give people a good idea about the campaign.

You could even front page the better reviews, but that's another can of worms entirely.
Title: Re: Should campaigns be given numerical rankings on the Wiki?
Post by: headdie on November 08, 2010, 04:11:32 am
I agree that given public consensus an opt in system is best for the community for the time being
Title: Re: Should campaigns be given numerical rankings on the Wiki?
Post by: TrashMan on November 08, 2010, 06:21:36 am
I'd say Tier rankings + required comments.
Title: Re: Should campaigns be given numerical rankings on the Wiki?
Post by: General Battuta on November 08, 2010, 08:30:33 am
I would not support an opt-in system. If these ratings are considered diagnostic than the absence of information on a campaign will be taken as a disincentive to play it. Having no rating would be as bad as having a low one.
Title: Re: Should campaigns be given numerical rankings on the Wiki?
Post by: Black Wolf on November 08, 2010, 08:55:12 am
Well, you're kind of out of luck. If the functionality exists on the wiki then it can be applied on any basis, or on any page that  someone wants to use it. Just like the random flag allows not only the random screenies on the front page but also the application of, for example, a random mods section on the modding portal, or a random campaign-specific screenshot on the page of a given campaign, a rating bar applied to mods can't be limited by any system I know of to just mods.
If someone chooses to put one on their campaign page there's nothing anyone can do about it, and there's nothing anyone should want to do about it. After all, while the wiki as a whole is collaborative, it's really up to the individual campaign authors how they choose to display their campaigns.

What's not going to be happening, given expressed community opinion, will be the applying of the rating bar to all campaign pages. Realistically, the likelihood is that only a relatively small number of people will choose to use it - it'll only become a problem for those not utilizing this feature if a significant majority of campaign pages chucked one up, which, if this poll is accurate, is highly unlikely.
Title: Re: Should campaigns be given numerical rankings on the Wiki?
Post by: General Battuta on November 08, 2010, 09:00:12 am
I don't believe the individual campaign authors have particular control over their pages. As a wiki they're open to community editing.

The fact that this system apparently cannot be modified or controlled by the user wanting to implement it is a big no-no for me.

Quote
If someone chooses to put one on their campaign page there's nothing anyone can do about it

except delete it

Quote
and there's nothing anyone should want to do about it

except that the Wiki has community enforced standards, for example the veteran comments policy.
Title: Re: Should campaigns be given numerical rankings on the Wiki?
Post by: Rodo on November 08, 2010, 09:01:53 am
I say :yes: , lots of people come round here asking for advise... which one to play and which one not to, this can only mean one thing: the actual system is not working properly or it's not as helpfull as it should be to newbies.
A simple score will make things easier for them, also they can always go back and play any campaigns left behind because of a poor review ranking.
Title: Re: Should campaigns be given numerical rankings on the Wiki?
Post by: General Battuta on November 08, 2010, 09:03:07 am
This is not Metacritic. We do not have sufficient N to get powerful, convergent samples.

If the bar could remain hidden until 20 or 50 votes were cast, maybe, but we can't do that.
Title: Re: Should campaigns be given numerical rankings on the Wiki?
Post by: Mobius on November 08, 2010, 11:47:46 am
I'd say to wait until we get approximately 50 votes.
Title: Re: Should campaigns be given numerical rankings on the Wiki?
Post by: NGTM-1R on November 08, 2010, 03:34:19 pm
Well, you're kind of out of luck.

The poll has a categoric no winning over a qualified no. You're kind of out of luck.
Title: Re: Should campaigns be given numerical rankings on the Wiki?
Post by: General Battuta on November 08, 2010, 03:37:34 pm
Agreed. There's inadequate support for any kind of change from status quo right now.
Title: Re: Should campaigns be given numerical rankings on the Wiki?
Post by: Lucika on November 08, 2010, 03:50:28 pm
Sorry, but the same people who say that we don't have enough votes and voters to "properly" use this feature are the same people who say '23 out of 37 people say no, so the idea is definitely worthless'?

Hypocrisy, good to see you again!
Title: Re: Should campaigns be given numerical rankings on the Wiki?
Post by: NGTM-1R on November 08, 2010, 03:55:10 pm
Sorry, but the same people who say that we don't have enough votes and voters to "properly" use this feature are the same people who say '23 out of 37 people say no, so the idea is definitely worthless'?

Hypocrisy, good to see you again!

See, we actually recognize the possibility that people will check here. Certainly more than actually check the wiki.
Title: Re: Should campaigns be given numerical rankings on the Wiki?
Post by: General Battuta on November 08, 2010, 04:07:39 pm
Sorry, but the same people who say that we don't have enough votes and voters to "properly" use this feature are the same people who say '23 out of 37 people say no, so the idea is definitely worthless'?

Hypocrisy, good to see you again!

You ****ed up your statistics. I will give you a lesson on sampling science now.

Anyone who gives enough of a **** to vote on campaign quality on the wiki is also likely to vote here, which makes this a good metric not only of how large the voter pool is but what they think of it.

That gives this poll a decent **** at being a representative sample of possible wiki voters. It says nothing about whether possible wiki voters is a representative sample of campaign players, which is required for the polls to be diagnostic.
Title: Re: Should campaigns be given numerical rankings on the Wiki?
Post by: Black Wolf on November 08, 2010, 04:29:54 pm
I don't believe the individual campaign authors have particular control over their pages. As a wiki they're open to community editing.

If I went and deleted the reviews section from the WiH page on the basis that they might make people want to play it more than, say, ASW (http://www.hard-light.net/wiki/index.php/Ancient-Shivan_War_%28campaign%29), which doesn't have a review section, you'd (quite rightly), call it vandalism. And so, this:


Quote
If someone chooses to put one on their campaign page there's nothing anyone can do about it

except delete it

Would also be vandalism, if the rating bar was added by the creator of the campaign as a legitimate attempt to generate feedback.

Well, you're kind of out of luck.

The poll has a categoric no winning over a qualified no. You're kind of out of luck.

The poll refects community views, which is fine for old campaigns and such, the 'ownership' of which (or at least responsibility for) has essentially been ceded to the community because the original creators left. So a consensus view is appropriate for dealing with them. It's not appropriate to be applied when the individuals or teams who created the campaigns are still around, and can decide for themselves on things like this.
Title: Re: Should campaigns be given numerical rankings on the Wiki?
Post by: General Battuta on November 08, 2010, 04:42:13 pm
Quote
If I went and deleted the reviews section from the WiH page on the basis that they might make people want to play it more than, say, ASW, which doesn't have a review section, you'd (quite rightly), call it vandalism.

The addition of a reviews section was requested by the community after discussion. I'm hardly the creator on the WiH page anyway. Adding a reviews page to ASW would absolutely be a good use of time; removing them would contradict what the community seems to have agreed on. If you read the changelog on the WiH page you'll note that I actually went to pains to skew the reviews negative.

Quote
Would also be vandalism, if the rating bar was added by the creator of the campaign as a legitimate attempt to generate feedback.

It would no more be vandalism than removing a lengthy blog-style veteran comment by the campaign creator. The community has agreed that blog-style veteran comments are a bad idea. Campaign creators do not, I believe, have any special privilege over their own pages; the wiki is a community effort.

Quote
The poll refects community views, which is fine for old campaigns and such, the 'ownership' of which (or at least responsibility for) has essentially been ceded to the community because the original creators left. So a consensus view is appropriate for dealing with them. It's not appropriate to be applied when the individuals or teams who created the campaigns are still around, and can decide for themselves on things like this.

This is an entirely new assertion that I do not believe has precedent. The wiki is, as a wiki, open to editing by all. Will it be split into subdomains?

I don't at the moment recognize any special privilege that campaign creators have over their own wiki pages. This notion feels very much out of left field to me, and more appropriate for a campaign website than a wiki environment. Maybe the ratings bar could go there instead?

Additionally, you've now abandoned the original purpose of the scale, which was to provide diagnostic data to newcomers so they could decide what campaigns to play.

If only a portion of campaigns receive the diagnostic data, the following problem arises:

1. The data is not presented, because it is not available for all campaigns, and is therefore no longer diagnostic of quality.

2. The data is presented, and is taken as diagnostic of quality, in which case campaigns that do not have a ranking are either ignored or recommended as 'unranked' which is as much a slight as 'screening out of competition' or 'racing for completion, no time.'

Just seems like it's not really doing much any more.
Title: Re: Should campaigns be given numerical rankings on the Wiki?
Post by: Lucika on November 08, 2010, 04:59:18 pm
Sorry, but the same people who say that we don't have enough votes and voters to "properly" use this feature are the same people who say '23 out of 37 people say no, so the idea is definitely worthless'?

Hypocrisy, good to see you again!

You ****ed up your statistics. I will give you a lesson on sampling science now.

Anyone who gives enough of a **** to vote on campaign quality on the wiki is also likely to vote here, which makes this a good metric not only of how large the voter pool is but what they think of it.

That gives this poll a decent **** at being a representative sample of possible wiki voters. It says nothing about whether possible wiki voters is a representative sample of campaign players, which is required for the polls to be diagnostic.

While I have my reasons to disagree with your statement above, it is currently totally irrelevant to the fact that you dismiss the idea based on a vote count that you yourself wouldn't consider adequate for rating a campaign.

As for your statement that I have quoted, please take a look at the registrated members count. ~10000. Noting the fact that we've had over one and a half million page downloads just last month, I think saying that 10% of the members are active would be an understatement. For the purposes of illustration, I'll stick with that.

The problem is that you've failed to notice the fact that those who've voted here are those who seriously oppose or support the idea. This is fair to say based on the relatively low amount of votes and the fact that the poll is not that old. This is essentially the same as getting, a few WiH-lovers and WiH-haters together to vote. No one from the middle ground, just them. Obviously those who might not consider the issue we're discussing here important enough to vote in this topic are not necessarily ignoring the chance to fill the opinion on a given campaign.

50 votes - which you would consider acceptable - are ~0.5% of the total member list of HLP, and taking the relatively pessimistic calculation of mine, still only ~5%.

Or you seriously believe that only 129 people wanted to see the Hatsepshut upgraded in almost two years? That would be a laughable claim, and essentially similar to your statement above. Voters about the voting system on the forum are not the same as people who want to vote. Totally different groups.

By the way, unless you are as dirt-conservative and slow as FIFA, you might consider a test run for this. Implement on some of the major campaigns and see the results a month later.

Of course, we can keep on arguing on semantics and statistical details all day until someone closes this topic, but I don't think that this will get us any closer to the goal of solving this issue.
Title: Re: Should campaigns be given numerical rankings on the Wiki?
Post by: General Battuta on November 08, 2010, 05:03:32 pm
While I have my reasons to disagree with your statement above, it is currently totally irrelevant to the fact that you dismiss the idea based on a vote count that you yourself wouldn't consider adequate for rating a campaign.

Again, you've ****ed up your statistics. The sampling population for a campaign is the players of the campaign. The sampling population for a decision about how to vote on the wiki is people who visit the wiki and are likely to vote. That second number is much smaller, and this vote count is probably moving towards significance on it.

The rest of your comments are built on a bad assumption, namely wild guesswork about population sizes, and therefore discardable. Anyone who cares enough to vote on the wiki should, I think, care enough to vote here, and I've done everything possible to promote the poll short of highlighting it. Your suggestions of sampling error are of course relevant; this is not a scientific poll, but it is the best we have, and we cannot reject the null (that it is representative) without tendentious data (which your error hypothesis is not.)

However:

Quote
50 votes - which you would consider acceptable - are ~0.5% of the total member list of HLP, and taking the relatively pessimistic calculation of mine, still only ~5%.

Actually, in many populations 5% is vastly more than is required to get a representative sample. Just a cool little statistics tip. However, this does not scale linearly with the population size so it probably doesn't apply here.

The issue is, I think, pretty close to resolved.
Title: Re: Should campaigns be given numerical rankings on the Wiki?
Post by: Black Wolf on November 08, 2010, 05:06:11 pm
Quote
If I went and deleted the reviews section from the WiH page on the basis that they might make people want to play it more than, say, ASW, which doesn't have a review section, you'd (quite rightly), call it vandalism.

The addition of a reviews section was requested by the community after discussion. I'm hardly the creator on the WiH page anyway. Adding a reviews page to ASW would absolutely be a good use of time; removing them would contradict what the community seems to have agreed on. If you read the changelog on the WiH page you'll note that I actually went to pains to skew the reviews negative.

I've got no issues with what you (or anyone) did to the WiH page. I like it. It's a good wiki page.

It would no more be vandalism than removing a lengthy blog-style veteran comment by the campaign creator. The community has agreed that blog-style veteran comments are a bad idea. Campaign creators do not, I believe, have any special privilege over their own pages; the wiki is a community effort.

Well, I do believe that creators have special rights over how they put their campaigns up on the wiki - see my Frontlines page for an example. FSWiki isn't wikipaedia, the rules aren't as rigid, and that, IMO, is a good thing.

Additionally, you've now abandoned the original purpose of the scale, which was to provide diagnostic data to newcomers so they could decide what campaigns to play.

If only a portion of campaigns receive the diagnostic data, the following problem arises:

1. The data is not presented, because it is not available for all campaigns, and is therefore no longer diagnostic of quality.

2. The data is presented, and is taken as diagnostic of quality, in which case campaigns that do not have a ranking are either ignored or recommended as 'unranked' which is as much a slight as 'screening out of competition' or 'racing for completion, no time.'

Just seems like it's not really doing much any more.

Yes, the original plan has been abandoned, because the poll seemed to support not utilizing the rating bar in that way. And since not all campaigns will be represented, obviously any kind of ranking would be inappropriate. But for indivdual campaigns who want to put a bar on their page and get a rating for their own campagn, independent of all others, there shouldn't be any problem IMO.
Title: Re: Should campaigns be given numerical rankings on the Wiki?
Post by: General Battuta on November 08, 2010, 05:09:06 pm
I do really like what you've done with Frontlines, it's an excellent piece of work.

And I understand the argument you're making for individual rating bars, but...since in the end it relies on us presenting the newcomer with data and qualifiers anyway, I'm not sure it's enough of a meaningful departure from the status quo to be worth it. And I think short review blurbs a la the WiH page are more useful than numerical data.

You're making headway on me but I'm not sold.

(And if it becomes an issue, I absolutely support edit neutrality. I do not think campaign creators should get special privilege over their pages. But I understand that's not a major point of contention right here.)

Oh and my point above re: the reviews section was to distinguish between a practice the community has endorsed (reviews section) and one they've decided to do away with (long, silly veteran comments).
Title: Re: Should campaigns be given numerical rankings on the Wiki?
Post by: Black Wolf on November 09, 2010, 02:55:04 am
Well, in shopping hte idea of rating bars around, there's been general consensus that rating bars for individual mods would be good. I also think that - it's what I wanted it for in the first place. My point htough is that once the script is in the wiki, it wont be possible to keep it off campaign pages without a rule that says "You absolutely must not put one of these bars on your campaigns wiki page." And that's not a rule I think we should have on our wiki. We should be encouraging people to make their pages unique and interesting as possible (See Shade's opening post on this thread (http://www.hard-light.net/forums/index.php?topic=70979.0)), and that's made harder with every prohibition and limitation that we put on pages.
Title: Re: Should campaigns be given numerical rankings on the Wiki?
Post by: Mobius on November 09, 2010, 11:29:18 am
Quote
50 votes - which you would consider acceptable - are ~0.5% of the total member list of HLP, and taking the relatively pessimistic calculation of mine, still only ~5%.

Actually, in many populations 5% is vastly more than is required to get a representative sample. Just a cool little statistics tip. However, this does not scale linearly with the population size so it probably doesn't apply here.

The issue is, I think, pretty close to resolved.

Except we're more or less busy working on several projects ---> not an accurate sample.
Title: Re: Should campaigns be given numerical rankings on the Wiki?
Post by: General Battuta on November 09, 2010, 11:36:03 am
Quote
50 votes - which you would consider acceptable - are ~0.5% of the total member list of HLP, and taking the relatively pessimistic calculation of mine, still only ~5%.

Actually, in many populations 5% is vastly more than is required to get a representative sample. Just a cool little statistics tip. However, this does not scale linearly with the population size so it probably doesn't apply here.

The issue is, I think, pretty close to resolved.

Except we're more or less busy working on several projects ---> not an accurate sample.

Is there a language barrier here? What point are you trying to make and how does it relate to what I'm saying? Right in the text you quoted, you can see me saying 'so it probably doesn't apply here'. Was that somehow unclear?
Title: Re: Should campaigns be given numerical rankings on the Wiki?
Post by: Mobius on November 09, 2010, 11:48:22 am
We need more votes. As I suggested earlier, we need to pimp up this poll to focus everyone's attention on it. I want to know what the players/lurkers (not the developers!) think about the matter because the points of view of those categories may differ.

I hope it's more understandable now. ;)
Title: Re: Should campaigns be given numerical rankings on the Wiki?
Post by: TopAce on November 09, 2010, 11:59:41 am
Lurkers and casual players will find a simple 1-10 voting range very convenient. Developers would prefer opinions put in words, not numbers. That's the big difference.
Title: Re: Should campaigns be given numerical rankings on the Wiki?
Post by: Mobius on November 09, 2010, 12:05:02 pm
That's why I think a simple voting system should be fine. I strongly believe we must care about those lurkers' opinions, not our own.
Title: Re: Should campaigns be given numerical rankings on the Wiki?
Post by: General Battuta on November 09, 2010, 12:11:58 pm

Votes ranging from 0 to 10 are also a bit inaccurate IMHO, as people tend to give 10s and 0s out the n00bish way without caring too much about the consequences. Any alternatives?

That's why I think a simple voting system should be fine. I strongly believe we must care about those lurkers' opinions, not our own.

You're contradicting yourself.
Title: Re: Should campaigns be given numerical rankings on the Wiki?
Post by: Mobius on November 09, 2010, 12:15:02 pm
I'm not. Isn't 0-100, or Very bad-Excellent equally simple? Isn't that practical enough for lurkers?

Heh, I'd like a 0-15 or 0-30. 0-10 is too common it may end up being misused, while 0-15 at least leads voters to think about what they're really doing.
Title: Re: Should campaigns be given numerical rankings on the Wiki?
Post by: General Battuta on November 09, 2010, 12:16:01 pm
BlackWolf cannot modify the script, so it's out of the question.
Title: Re: Should campaigns be given numerical rankings on the Wiki?
Post by: Mobius on November 09, 2010, 12:17:36 pm
But we can still look for alternatives, right?
Title: Re: Should campaigns be given numerical rankings on the Wiki?
Post by: The E on November 09, 2010, 01:04:05 pm
Even as a simple consumer of games, I find that numeric ratings are misleading and worthless. When I consider buying a game, I make my decisions based on written reviews, not the numeric score that gets pinned on them at the end. Same goes for campaigns.
Title: Re: Should campaigns be given numerical rankings on the Wiki?
Post by: Mobius on November 09, 2010, 02:17:04 pm
That's what you do. It doesn't mean the gaming community as a whole does the same - let me explain why.

As staff member of a site devoted to videogaming, I have access to stats regarding site visits and preferences. I can ensure you that reviews are severely underestimated and underused: people hardly read them and prefer numerical rankings more.
Title: Re: Should campaigns be given numerical rankings on the Wiki?
Post by: TopAce on November 09, 2010, 02:19:59 pm
Quote
people hardly read them [reviews] and prefer numerical rankings more.

How do you know?
Title: Re: Should campaigns be given numerical rankings on the Wiki?
Post by: General Battuta on November 09, 2010, 02:21:36 pm
You all have my feelings on the matter. I don't think further discussion is going to be very productive, especially not with TopAce (who handles so much wiki stuff) against it.
Title: Re: Should campaigns be given numerical rankings on the Wiki?
Post by: TopAce on November 09, 2010, 02:25:30 pm
I have as much impact on the matter as any regular member here. I don't have any privileges.
Title: Re: Should campaigns be given numerical rankings on the Wiki?
Post by: Mobius on November 09, 2010, 02:32:00 pm
Quote
people hardly read them [reviews] and prefer numerical rankings more.

How do you know?

As I said, I have access to that data. People tend to read FAQs more and buy games after hearsay: those who use reviews constitute a minority (15.9% is the value I can see right now).

That's why I stopped reviewing space and flights sims - it wasn't worth the effort.
Title: Re: Should campaigns be given numerical rankings on the Wiki?
Post by: Droid803 on November 09, 2010, 04:56:24 pm
I'm of the persuasion that a numerical score will not help less-played campaigns more noticed. It will not help popular campaigns that are widely recommended be played more. It will not stop people from asking for recommendations.

Ergo, it is useless, worthless, and a waste of everyone's time and effort.

In fact, I can imagine it having the opposite effect. Less-played campaigns will be even more forgotten as noone will bother to vote for it or whatever.

I am of the firm belief someone's work is worth more than some stupid number.

If a flat numerical score like game reviews ever comes into effect, rest assured that not only will I not participate, but I may as well quit FS modding. I do not want my work judged based off of an arbitrary number. Thank you.
Title: Re: Should campaigns be given numerical rankings on the Wiki?
Post by: General Battuta on November 09, 2010, 05:00:52 pm
I am pretty much in that camp.
Title: Re: Should campaigns be given numerical rankings on the Wiki?
Post by: Black Wolf on November 09, 2010, 05:51:50 pm
If a flat numerical score like game reviews ever comes into effect, rest assured that not only will I not participate, but I may as well quit FS modding. I do not want my work judged based off of an arbitrary number. Thank you.

If you don't want to use it, you wont be forced to. That's what this thread was about.

And threatening to take you bat and ball and go home is just childish. :doubt:
Title: Re: Should campaigns be given numerical rankings on the Wiki?
Post by: Narwhal on November 09, 2010, 07:05:54 pm
I agree threats like "If this passes, I leave" are not very mature.

I used to be in favor of such a rating - even started a thread asking your opinion on it - but after reading all the opinions I was convinced by the "other side" and I am now  moderately against.

If it was to be done, it would need in my opinion a very strict implementation :

- Like / Dislike only answers, no numerical rating
- Number of votes
- Only completed campaign, i.e. someone (me?) would have to make a separate page with completed campaigns only. Actually - that was one of my projects, but even though I have some time I always find better occupations. I don't want people to vote on unfinished / demo / partial release, and then skewing the result when the campaign is finished.
- No way to vote when you have selected a "best campaigns" filter. This skews results an awful lot "Oh, yes ! This campaign was good ! I will vote for it too."

[I am still in favor of a short-list of outstanding campaigns, but for other reasons and that's another issue]
Title: Re: Should campaigns be given numerical rankings on the Wiki?
Post by: NGTM-1R on November 09, 2010, 07:29:58 pm
If you don't want to use it, you wont be forced to. That's what this thread was about.

That's like saying if you suppress one person's free speech, it doesn't effect anyone else. It doesn't work for the same reason. The legal term is "chilling effect" and Droid quite ably explained the problem in his post. Read the following again.

In fact, I can imagine it having the opposite effect. Less-played campaigns will be even more forgotten as noone will bother to vote for it or whatever.

Now refute that and we can say you actually have an argument.
Title: Re: Should campaigns be given numerical rankings on the Wiki?
Post by: Black Wolf on November 10, 2010, 04:55:50 am
If you don't want to use it, you wont be forced to. That's what this thread was about.

That's like saying if you suppress one person's free speech, it doesn't effect anyone else. It doesn't work for the same reason. The legal term is "chilling effect" and Droid quite ably explained the problem in his post. Read the following again.

In fact, I can imagine it having the opposite effect. Less-played campaigns will be even more forgotten as noone will bother to vote for it or whatever.

Now refute that and we can say you actually have an argument.

Prove that it will happen and you might have an argument. You're making assumptions that are yet to be tested. Hell, you're the one advocating "repression", to use your own words. Consider:

I'm yet to find anyone who has an objection to a rating bar for individual ship mods. That's what I wanted it for in the first place, and that's a project I'm going to take on personally.
Once the script is implemented, the functionality will be employable by anyone, for any purpose they wish, just like any other tools on the wiki.
What you're proposing is to forbid people from using this functionality, if they so choose, to generate feedback about their particular campaign. There are currently no rules forbidding the use of functionality on the wiki, and there flat out shouldn't be any rules like that.

Let me also remind people of the current implementation plan to address some of Narwhal's concerns:

  • This will not be being applied wholesale to all campaigns, so there's no chance of incomplete ones getting it. Well, I suppose there is, if someone put one on on a campaign page which was as yet unreleased, however I plan to word the template to make it very obvious that the rating is meant to be about the experience of playing the campaign, so it wont apply to unreleased campaigns.
  • It will be entirely at the discretion of the creator of any campaign if he wants to implement one of the bars.
  • No top ten lists or anything like it will be compiled due to concerns raised about these. The only place the scores will be visible will be on the pages of the campaigns which choose to use the bar. This is a direct result of it not being applied wholesale, since it would obviously not be representative.
  • The script does show how many votes have contributed to the aggregate score.
The other factor I want to address is the idea that seems to be developing that there is some kind of zero sum game between a numeric rating bar and written reviews. There isn't. A lot of people who've played a campaign will likely not want to sit down and write a review, but they might be willing to click a few buttons and rate it. This gives the authors more feedback, admittedly of a limited type, but of a type that they have chosen to receive. Moreover, the template can be written like this:

"If you have played |CAMPAIGN NAME| please take the time to ratet he campaign, and possibly write a short review it in the space below:"

This could very well increase the number of written reviews as people will feel the need to justfy their score. And yes, I realize that that's an equally untested assertion to the one I was annoyed about at the beginning of the post, but it's not the basis for my entire argument, so I'm happy enough with it.
Title: Re: Should campaigns be given numerical rankings on the Wiki?
Post by: General Battuta on November 10, 2010, 07:45:27 am
I'm still thinking I will remove them where I see them. The community has indicated it does not want them. You may say that a campaign creator should have the right to deploy anything he wants, but Wiki standards hold: the campaign creator does not have the right to post blog-style veteran comments per se. If the community doesn't want them I think they're a no-go.
Title: Re: Should campaigns be given numerical rankings on the Wiki?
Post by: Black Wolf on November 10, 2010, 07:59:59 am
Well then you're going to put me in an awkward position if you do. But I guess we'll cross that bridge when we come to it.
Title: Re: Should campaigns be given numerical rankings on the Wiki?
Post by: General Battuta on November 10, 2010, 08:05:58 am
I don't know. I do see the point you're making.
Title: Re: Should campaigns be given numerical rankings on the Wiki?
Post by: Angelus on November 15, 2010, 01:53:11 pm
I voted no.
Title: Re: Should campaigns be given numerical rankings on the Wiki?
Post by: Vidmaster on November 30, 2010, 11:34:36 am
I WANT MORE PEOPLE TO WRITE REVIEWS. I'd "campaign reviews" by people, linked to the campaign's main article. No scores permitted, just a description aka review of the personal experience with the thing.
Title: Re: Should campaigns be given numerical rankings on the Wiki?
Post by: Nohiki on November 30, 2010, 11:39:38 am
Definitely. Even the verbal ratings are too individual, and from them you can at least understand what that specific user didn't like.
Title: Re: Should campaigns be given numerical rankings on the Wiki?
Post by: TopAce on November 30, 2010, 12:45:36 pm
I WANT MORE PEOPLE TO WRITE REVIEWS. I'd "campaign reviews" by people, linked to the campaign's main article. No scores permitted, just a description aka review of the personal experience with the thing.

I'd post a new thread about it in the Missions and Campaigns forum. It's a better place for this kind of thing. Those who visit the Wiki board are already aware of this possibility.
Title: Re: Should campaigns be given numerical rankings on the Wiki?
Post by: Mobius on December 26, 2010, 02:14:22 pm
I haven't followed the debate for quite some time, but I have something to add:

No top ten lists or anything like it will be compiled due to concerns raised about these. The only place the scores will be visible will be on the pages of the campaigns which choose to use the bar. This is a direct result of it not being applied wholesale, since it would obviously not be representative.
The script does show how many votes have contributed to the aggregate score.

The other factor I want to address is the idea that seems to be developing that there is some kind of zero sum game between a numeric rating bar and written reviews. There isn't. A lot of people who've played a campaign will likely not want to sit down and write a review, but they might be willing to click a few buttons and rate it. This gives the authors more feedback, admittedly of a limited type, but of a type that they have chosen to receive. Moreover, the template can be written like this:

In my opinion, if these can be done, it'll be very, very nice.