Scoring System
I knew that as soon as I create this Blog, that the issue of scoring on the site will be brought up into the discussion of every article that I ever post.
So, I thought I would address the issue with my first actual article.
I hope, whether you're an author, or a reader, you'll read this article with an open mind and a bit of understanding. I've dealt with the subject for the longest time, since the scoring system started on the site in 1999. And rest assured, ever since the start, the subject is brought to my attention by an author or a reader a minimum of once a month, if not more. Needless to say, I've put quite a bit of thought into it over the years, and in this article, I'll try to share the results of these thoughts.
The concept of scoring an artistic work like stories by people is a difficult one to tackle.
Ideally, scores on the site would enable any reader to take a story's score as a measure of what to expect of the story in any facet. The score should tell how well the story is written, how well the story's story is told, how extensive the plot is, etc... So theoretically, the score should help you pick out the best stories for reading and theoretically, it should be infallible.
From a reader's point of view: the scores should tell him/her how much they're GOING to enjoy the story if they decide to read it.
From the author's point of view: the scoring mechanism should give him/her 10s and everybody else a 6 or lower. Of course, unless the author is trying to find a story to read, then the score should reflect this need, temporarily at least.
But the reality is, there could never be a way for a single score to reflect every facet of the story. There could never be a single way to account for different people's tastes and education levels, let alone account for their personal fetishes. For example, some people like cheating wives stories and score them higher than anything else, others think those stories are an abomination and the issue should not be discussed without mentioning the stoning of the wife at the hand of the raging masses; so they always score any story that fails to do that with a 5 or less vote. Does that mean the story is not good? Does it means it's not written well? Does the score take into account your personal stance on the genre?
I wanted to build that ideal system mentioned above. So the first iteration of the scoring mechanism, who's existence is probably not remembered by anybody, was a reflection of the 'Celestial Reviews' scoring method. The reader gets to select three separate scores, one for each of the criteria: 'Technical quality', 'Plot & Character' and 'Appeal to Reviewer'; the points range was from 1 to 10.
This mechanism stayed on the site for a whole week and, at the time, the site had about a thousand stories. During its existence for that whole week, the system gathered exactly 5 votes. Yes, you've read correctly, that's 'FIVE' votes in total.
I can't really tell why it was such a failure. Was it because it required too much work from the reader? Was it because most readers didn't think they were qualified enough to make judgment on subjects like 'Plot and Character' and 'Technical Quality'?
I don't know, but I know it was a miserable failure, and it needed rethinking on my part to make it work; it required me to lower my expectations from the readers.
The result of the rework is the current system. As soon as it was online, scores came pouring in. So the participation problem was resolved.
Soon after, like within a week, I came to the realization that there is something extremely wrong with the scoring mechanism. Scores were fluctuating wildly, and it was evident that what I built at the time was a weak system, and it was being abused. At the time, the system that I used to run the site was very easy to use and implement, but very limited in what it allowed me to do. The only thing that I could do at the time is to track the last IP address that cast a vote for the story and stop them from voting again, but since it tracked only the last vote, when somebody else voted, the first voter could vote again. Not perfect, but better than the initial system. That system stayed online from the fall of 1999 to June of 2001. At the end of that period only 4 stories had a score higher than 9.
It took me that long to bring up my skill level to use a more powerful system. In June of 2001 I finished the re-implementation of the site using PHP and MySQL and I instituted user registration. Now, the site could tell who's voting, and could keep track of who voted for what story and stopped them from voting again. So while most people think that I use the site to harvest email addresses to sell to spammers (which I don't), I had to do it so that people couldn't abuse the voting system easily; it's not perfect, but it works well enough to make it a viable system.
The most recent change since then was about a month ago, when I changed the location of the voting mechanism to above the 'To be Continued...' or 'The End' line and changed the wording of the form itself. This recent change had a tremendous effect on the rate of participation. The rate of participation went up to almost double the rate of participation pre-change.
Inflation of scores
Many people are pissed about the seemingly artificial high scores, and have commented on how unrealistic they think they are.
Unfortunately, there is no easy way around this one. It has to do with human psychology. People tend to be nice and illogical simultaneously.
First, they don't like hurting authors on purpose, so if they like the story, they give it a high score, and if they don't like the story, many people would abstain from voting. I've been told by many people that they won't vote at all if they don't think that the story is good and deserves a 9 or 10. So the tendency for votes to be on the higher side of things can't be avoided for this very peculiar phenomenon.
Second, they tend to be affected by existing scores for the story in question and the scores for other stories. So for example, if John Doe is reading the story 'Mary Had a Little Caboose' and the story has an existing score of 9.2. And John Doe had already read the story 'House in the Mountains', and it has a score of 9.5. John Doe liked 'Mary Had a Little Caboose' more than 'House in the Mountains', then his vote will be a 10 most likely, regardless of what he really thinks of the story 'Mary Had a Little Caboose' on its own; maybe a 9. He's now looking at it in context of the other story's score.
That effect causes an ever rising top score on the top list and cannot be broken unless new readers are introduced into the site, and they bring with them a fresh perspective.
Score Manipulation
I've often received messages from authors that somebody must be tampering with scores, or somebody has a grudge against them, and casting multiple low votes to lower their stories' scores. While this is possible, it's not possible at the scale that some people think.
The site is equipped with vote manipulation busting system. While anybody can create multiple accounts to try to influence scores, the system renders any such endeavor almost moot. The voting system on the site keeps track of the time that the story has been accessed last and the time of the vote. If the interval is too short (that is measured depending on the story's length), then the vote is nullified. So for anybody to cast multiple votes for multiple stories using multiple accounts, that person has to spend a lot of time to do it; and I do mean a LOT of time.
When story scores change, they change because of valid votes 99% of the time. So if you're an author and your story's score drops, take it like a man and accept the fact that somebody doesn't like your story. No story can please every person on every level.
Over the years, I've received many suggestions to what I can do to make the scoring system 'better'.
For now, the scoring mechanism is staying the same way it is. There will be no change, unless someone can come up with a scheme that somehow can read the minds of all those who've read the story reflects their thoughts and opinion of the story in perspective of the reader who's checking the score and trying to decide whether to read or not, in one, easy-to-understand, decimal number.
Although, it doesn't simply end here.
I need to change the reviewing system on the site to enable any reader to submit an extensive review of any story. My thoughts so far have explored re-implementing the reviewing system in parallel with a second score for stories that is derived from those reviews. So each story will have two scores: the existing score, let's call it the popular score and a new score and let's call it a reviewers' score.
So far this idea has shown merit, and it beats all other ideas that's I've had myself, or were suggested to me.
However, such a system presents its own challenges and limitations, so for now it's in the analysis phase. It may or may not proceed into the planning and then implementation phases.
Update: 2004-11-01 : 11:25 am
The future reviewing system will be accessible to anybody (whenever it's done). At the end of a story, the reader will have the current voting form and a link to submit a complete review.
Reviews will be moderated, as in somebody have to approve the review before it shows up on the site and affect a story's score; we don't need reviews like 'complete crap 1, 1, 1' or 'I liked it 10, 10, 10', there is no point in those. The only way to really avoid silly or stupid reviews is to moderate them. Reviews should contain valid constructive criticism or a developed opinion.
No anonymous reviews, the reviewer's user ID will show up on top of each review.
While the idea of restricting reviews to premier members does have merits, it would be too restrictive, and many of the qualified and willing to post people would be shutout. So the feature will be practically crippled; so no restriction on who can and cannot review.
So, I thought I would address the issue with my first actual article.
I hope, whether you're an author, or a reader, you'll read this article with an open mind and a bit of understanding. I've dealt with the subject for the longest time, since the scoring system started on the site in 1999. And rest assured, ever since the start, the subject is brought to my attention by an author or a reader a minimum of once a month, if not more. Needless to say, I've put quite a bit of thought into it over the years, and in this article, I'll try to share the results of these thoughts.
The concept of scoring an artistic work like stories by people is a difficult one to tackle.
Ideally, scores on the site would enable any reader to take a story's score as a measure of what to expect of the story in any facet. The score should tell how well the story is written, how well the story's story is told, how extensive the plot is, etc... So theoretically, the score should help you pick out the best stories for reading and theoretically, it should be infallible.
From a reader's point of view: the scores should tell him/her how much they're GOING to enjoy the story if they decide to read it.
From the author's point of view: the scoring mechanism should give him/her 10s and everybody else a 6 or lower. Of course, unless the author is trying to find a story to read, then the score should reflect this need, temporarily at least.
But the reality is, there could never be a way for a single score to reflect every facet of the story. There could never be a single way to account for different people's tastes and education levels, let alone account for their personal fetishes. For example, some people like cheating wives stories and score them higher than anything else, others think those stories are an abomination and the issue should not be discussed without mentioning the stoning of the wife at the hand of the raging masses; so they always score any story that fails to do that with a 5 or less vote. Does that mean the story is not good? Does it means it's not written well? Does the score take into account your personal stance on the genre?
History
I wanted to build that ideal system mentioned above. So the first iteration of the scoring mechanism, who's existence is probably not remembered by anybody, was a reflection of the 'Celestial Reviews' scoring method. The reader gets to select three separate scores, one for each of the criteria: 'Technical quality', 'Plot & Character' and 'Appeal to Reviewer'; the points range was from 1 to 10.
This mechanism stayed on the site for a whole week and, at the time, the site had about a thousand stories. During its existence for that whole week, the system gathered exactly 5 votes. Yes, you've read correctly, that's 'FIVE' votes in total.
I can't really tell why it was such a failure. Was it because it required too much work from the reader? Was it because most readers didn't think they were qualified enough to make judgment on subjects like 'Plot and Character' and 'Technical Quality'?
I don't know, but I know it was a miserable failure, and it needed rethinking on my part to make it work; it required me to lower my expectations from the readers.
The result of the rework is the current system. As soon as it was online, scores came pouring in. So the participation problem was resolved.
Soon after, like within a week, I came to the realization that there is something extremely wrong with the scoring mechanism. Scores were fluctuating wildly, and it was evident that what I built at the time was a weak system, and it was being abused. At the time, the system that I used to run the site was very easy to use and implement, but very limited in what it allowed me to do. The only thing that I could do at the time is to track the last IP address that cast a vote for the story and stop them from voting again, but since it tracked only the last vote, when somebody else voted, the first voter could vote again. Not perfect, but better than the initial system. That system stayed online from the fall of 1999 to June of 2001. At the end of that period only 4 stories had a score higher than 9.
It took me that long to bring up my skill level to use a more powerful system. In June of 2001 I finished the re-implementation of the site using PHP and MySQL and I instituted user registration. Now, the site could tell who's voting, and could keep track of who voted for what story and stopped them from voting again. So while most people think that I use the site to harvest email addresses to sell to spammers (which I don't), I had to do it so that people couldn't abuse the voting system easily; it's not perfect, but it works well enough to make it a viable system.
The most recent change since then was about a month ago, when I changed the location of the voting mechanism to above the 'To be Continued...' or 'The End' line and changed the wording of the form itself. This recent change had a tremendous effect on the rate of participation. The rate of participation went up to almost double the rate of participation pre-change.
Remarks about the system
Inflation of scores
Many people are pissed about the seemingly artificial high scores, and have commented on how unrealistic they think they are.
Unfortunately, there is no easy way around this one. It has to do with human psychology. People tend to be nice and illogical simultaneously.
First, they don't like hurting authors on purpose, so if they like the story, they give it a high score, and if they don't like the story, many people would abstain from voting. I've been told by many people that they won't vote at all if they don't think that the story is good and deserves a 9 or 10. So the tendency for votes to be on the higher side of things can't be avoided for this very peculiar phenomenon.
Second, they tend to be affected by existing scores for the story in question and the scores for other stories. So for example, if John Doe is reading the story 'Mary Had a Little Caboose' and the story has an existing score of 9.2. And John Doe had already read the story 'House in the Mountains', and it has a score of 9.5. John Doe liked 'Mary Had a Little Caboose' more than 'House in the Mountains', then his vote will be a 10 most likely, regardless of what he really thinks of the story 'Mary Had a Little Caboose' on its own; maybe a 9. He's now looking at it in context of the other story's score.
That effect causes an ever rising top score on the top list and cannot be broken unless new readers are introduced into the site, and they bring with them a fresh perspective.
Score Manipulation
I've often received messages from authors that somebody must be tampering with scores, or somebody has a grudge against them, and casting multiple low votes to lower their stories' scores. While this is possible, it's not possible at the scale that some people think.
The site is equipped with vote manipulation busting system. While anybody can create multiple accounts to try to influence scores, the system renders any such endeavor almost moot. The voting system on the site keeps track of the time that the story has been accessed last and the time of the vote. If the interval is too short (that is measured depending on the story's length), then the vote is nullified. So for anybody to cast multiple votes for multiple stories using multiple accounts, that person has to spend a lot of time to do it; and I do mean a LOT of time.
When story scores change, they change because of valid votes 99% of the time. So if you're an author and your story's score drops, take it like a man and accept the fact that somebody doesn't like your story. No story can please every person on every level.
Author and Reader Suggestions for Scoring Enhancements
Over the years, I've received many suggestions to what I can do to make the scoring system 'better'.
- Forcing readers to vote: Does not work and would make the site less appealing. Nobody likes to be forced to do anything.
- Formulas that involve download counts, size of the story, number of chapters of the story, consistency of number of downloads from one chapter to another; They all don't work. Most authors post their stories on other sites, so a reader can read one chapter here and one chapter on ASSTR and one chapter on EWP and another on ASSM. So the counts would be skewed and unreliable at best. Number of chapters? Well, how does that affect a long and repetitive story in comparison with a shorter story that is built tighter? How does the chapter length get counted? Can't force a format on the author, so that is out of the window. Everybody knows that length and chapter count don't correspond in any way, shape or form to the quality of the story. Every formula has an optimum variable combination, which would boost a story's score to the max and that will have no real reflection on how good the story is. So we're back to square one.
Anything that somehow messes with the reader's evaluation of the story is not acceptable, after all, the score should reflect the readers' opinions and not a morphed version of it. - Allowing people to change their scores. Well, it's feasible, but very resource intensive and messes with the site's ability to thwart manipulation. However, I implemented the system to allow a reader to vote again after 125 days, so you could cast a different vote if the serial degenerates over time or improves over time.
Conclusion
For now, the scoring mechanism is staying the same way it is. There will be no change, unless someone can come up with a scheme that somehow can read the minds of all those who've read the story reflects their thoughts and opinion of the story in perspective of the reader who's checking the score and trying to decide whether to read or not, in one, easy-to-understand, decimal number.
Although, it doesn't simply end here.
I need to change the reviewing system on the site to enable any reader to submit an extensive review of any story. My thoughts so far have explored re-implementing the reviewing system in parallel with a second score for stories that is derived from those reviews. So each story will have two scores: the existing score, let's call it the popular score and a new score and let's call it a reviewers' score.
So far this idea has shown merit, and it beats all other ideas that's I've had myself, or were suggested to me.
However, such a system presents its own challenges and limitations, so for now it's in the analysis phase. It may or may not proceed into the planning and then implementation phases.
Update: 2004-11-01 : 11:25 am
The future reviewing system will be accessible to anybody (whenever it's done). At the end of a story, the reader will have the current voting form and a link to submit a complete review.
Reviews will be moderated, as in somebody have to approve the review before it shows up on the site and affect a story's score; we don't need reviews like 'complete crap 1, 1, 1' or 'I liked it 10, 10, 10', there is no point in those. The only way to really avoid silly or stupid reviews is to moderate them. Reviews should contain valid constructive criticism or a developed opinion.
No anonymous reviews, the reviewer's user ID will show up on top of each review.
While the idea of restricting reviews to premier members does have merits, it would be too restrictive, and many of the qualified and willing to post people would be shutout. So the feature will be practically crippled; so no restriction on who can and cannot review.
Comments
My scoring formula was a joke... I just wanted to make that clear :)
I think writers and readers go through a phase of 'This score isn't high enough compared to this story.'
Like Lazeez says, a lot of this is human nature.
People are looking for different things, and even score complainers are looking for a system that makes 'THEIR' favorite stories highest.
No score will ever compare to having a reader write me that they 'cried' because of a story or that a musician picked up his guitar again because of my Stringbreaker series.
The scores are like the cherry on ice cream... it's nice but I hope you weren't there for the cherry.
ElSol
The only revision to the system that I'd suggest is to offer two scores for long-running serial novels: one for "incomplete" stories; and one for a "final" score -- only counting scores from after the work is complete. I personally hesitate to score serials until the things is finished -- what if the ending stinks, despite a brilliant opening? What if the author takes a disconnected mish-mash of unrelated threads and ties them together in the last three chapters in a brilliant ending which changes the complexion of the entire story? Yet participation scores for serials are probably the most important (who wants to start up with a 1000K+ serial novel consistently scoring 6-8?).
Minor Bitch and Moan:
Some of us (how many others, I don't know) pull stories down from the site, then read them off-line, using a PDA or eBook reader. (it's _much_ easier on the eyes, and portable. Beats the hell out of actually listening to the asshole next to you in a management meeting drone on and on about the same issue for the nth time, and you can look like you're taking notes on your PDA :-). )
Forex, I wrote a little perl script to turn a selection of stories into a single big file suitable for my REB-1100 reader or conversion by MobiPocket Publisher, organized by author (Lazeez, if you want a copy, let me know, but it's a bit of a kludge, and has evolved with the site as you've changed things). I create a new group of "current" stories about once a week or so.
But the scoring system limits my scoring participation. If I sit down a couple of times per week to score the stories I've read off-line, the votes come too quickly (I would score 10+ stories in a couple of minutes), and aren't counted -- even though I've read the stories thoroughly and completely, and would be giving honest scores.
Despite its limitations, however, the scoring _is_ somewhat helpful to me as a reader. I would never have read stories about "Bowling" or "Rings" if they hadn't come to my attention as scoring well over a long period of time, because the names and descriptions didn't appeal to me -- and now those two ongoing serials are among my favorites on the site.
-Ghiti
Keeping in mind, I'm a new author, and have lurked in various groups long enough to have learned to take the scoring systems of ANY site with a grain of salt:
I think one of the things that needs to be kept in mind by authors is that scores aren't necessarily reflective of what we'd LIKE them to be reflective of. Partly because of human nature and psychology, as Lazeez said, and partly because of the nature of the work, many times the scores will be reflective of a given authors popularity, rather than the quality of a specific story.
Obviously, there's some exceptions to that -- without mentioning names, there's some well-known and respected authors out there who generally score fairly high, but start writing a story which may not appeal to many readers and therefore takes a hit in the scoring. That said though, I would be surprised if some of the authors that almost always get 9's and 10's on their stories started posting and getting constant 5's and 6's, regadless of how good or bad the story is. I think a lot of readers will judge a story FIRST on its individual quality, but then on its author's popularity among his/her other works, and they'll take BOTH of those factors into account when picking a score.
I'll even fess up to doing this once or twice. There's stories I've read by some that I didn't like all that much, but I still gave them a higher rating than I would have it they were by someone I've never heard of, on the basis of who wrote them. Is it good or right? Probably not. Have I done it before? Absolutely.
Welcome to human nature.
My point being: I think it would help if the authors among us would keep in mind that the scoring system isn't perfect, and it's limited as much by the nature of the readers as it is by the technical basis of the system. Focus on the feedback and comments you get from your readers, and pay more attention to what THEY say about your work than what the score says, and everyone will be happier.
I'd love it if Elizabeth suddenly hit the 'Top Scores' list. I don't expect it to happen, but I'd certainly be overjoyed -- but I'd be happier because it would give me greater exposure and presumably introduce more readers to my story, rather than specifically because of the score that it would take to get there.
That's my view, at least. I'm sure many of you will disagree, and that's fine, but that's how I look at it.
Lazeez, all the discussion about scoring aside, thank you for the years of dedication and hard work on the site -- I've appreciated it for years, both as a reader and now as an author. Bravo.
I like Lazeez's thought about 'popular' scores, *and* 'reviewer' scores. I was wondering if both scores could be posted. That would give a clearer picture, regardless of actual numbers.
The only problem I see, is that there would need to be a great many more 'reveiwers' that are now extant.
How do we get more reviewers, and more reviews? How many reviews per reviewer (certainly not more than one, per story) should be allowed, if there is to be a limit at all?
Also, should there be 'credentials' required of reviewers?
I believe there should be at least minimal credentials. What they should be, I leave to you all. Of course, there won't be a concensus, but in this matter the majority could rule.
The end result should be a 'fair' evaluation.
A) Popular response to a story should be encouraged.
B) Knowledgable reviews should be available to as many stories as possible.
I hope many more of us will write, espousing opinions. Perhaps a compromise can be reached that we all can accept.
TeNderLoin (JIM7)
:)))
I like the idea of two scores, one for the general readers and one for reviewers. I like the latter because it wouldn't be anonymous. As a reader, I could skim the reviews and know which ones to discount and which to really pay attention to. The hard part is that there seem to be very few reviews as it is.
As for the current scoring system, my problem with it is that it doesn't capture the distribution. If one logs on and checks scores often enough, one can mathematically back out individual votes. I did so for a while on one story and found that the distribution was about thirty 9's and 10's and two 4's. This implies that the story is quite good overall, but two people didn't like it (or were voting simply to lower the average).
Those two outliers can pull the average down signficantly. For example, thirty votes of 10 and two votes of 4 give the same average (9.65) as twenty votes of 10 and twelve votes of 9. The median average doesn't tell me if the reader's consensus is lower (the latter) or if there are one or two strong dislikes balancing out other strong likes (the former). Maybe having the mode average or some other limited distribution would help, but more reviewers would solve the problem as well.
When I read reviews I see scores on the 1-to-10 scale for several categories. I would like to see a similar system for just maybe just three categories such as Originality, Style, AND Grammer. Each would have a pull down menu as the current one but with a diffenert scale such as (5 - Try Again), (6 - Needs Work), (7 - Average), (8 - Above Average), (9 - Excellent), (10 - Why Isn't This Published?) I know eliminating 0-4 seems to be a bit too much in the Kumbyia Camp (Crap) but it eliminated the haters purposely giving too low a score just to see a very low number.
Also I would like to change a few of my scores as I voted too low or high previously considering the whole of the works in the categories of my choosing.
Finally, I would like to suggest that unfinished works should not be able to be scored until finished. I am guilty of rating a story much higher in the beggining and then discovering it went downhill. I also have rated something low and have been tremendoulsy impressed when the author pulled himself out of it.
Sincerly,
Nine Mason Jars
I agree that the scroing system isnt quite as good as we would all like, maybe reworking the scores to 5 levels, Hate/Dislike/Neutral/Like/Love. would fix it up a little, then work a numeric value off the levels obtained. if you reworked all the scores that have currently been provided by readers, like mentioned above, most people put a 5 as the bottom end. if someone really hated the story enough to put a 2 or 3, it just adds one point to the dislike level, no great fluctuation on the overal scheme of things.
Also if someone is willing to be a reviewer, open up the scoring for them, allow for them to score on all the story's structure (plot, character development etc etc), since they are willing to review a story and not just read it, they *might* also be more willing to score on a range of things.
/MrZvon out.
For what it's worth, I think your scoring system is impartial and works just fine. Some of my stories get high schores and others got shredded. The scores tell me what works and what doesn't, and that makes me a better writer.
You mentioned reviewers. I think it would be a novel idea if every person could be considered a "reviewer." By doing so, you create a "reviewer" page associated with each member that lists every vote that they gave to every story that they voted on. Then I, as a reader, could choose a few favorite "reviewers" (who may not even realize they are being regarded as such) who have similar tastes as me and read the stories that they have given high scores to...
Hungry Guy
The ability to vote again, on LONG stories is great, but I think 125 days is too long. More logical might be 90 days, considering how prolific and freqent many long story writers are.
Thanks for the chance to comment. CEH
On serials instead of having a fixed number of days before a chance of a second vote might it be fairer if it were after a set number of chapters (or words). Some authors are very prolific, and I suspect disciplined, and add one or more chapters a week while others a chapter every few months.
Whatever you decide you have my thanks for providing a great site.
SOL editor
I would seriously consider reviewing stories I am reading or have read. I would explain what my criteria for a rating is. Or I would use the guidelines that Lazeez set up. I understand the need for an average reviewers score. I think the individual reviews need to be available to the readers. I have a technical background and readers need to know that a review by St John will be very critical of poor science. And poor grammar. With the help of volunteer editors, there is no excuse for poor grammar. Not all editors are technically competent, so some poor science may end up in final draft.
You should see the E-mails between my author and myself. I have to keep reminding myself it is his story and he has final say.
P.S. Edited "A Better World" by Robert Wilson. Am editing "Found in Space" by same author. Am not on new editors list because am busy enough with "Found in Space".
St John
Lazeez said:
I need to change the reviewing system on the site to enable any reader to submit an extensive review of any story. My thoughts so far have explored re-implementing the reviewing system in parallel with a second score for stories that is derived from those reviews. So each story will have two scores: the existing score, let's call it the popular score and a new score and let's call it a reviewers' score.
Got one simple idea to throw at you here about the rating system. Rather than rework the ratings SYSTEM to accept emotive votes (hate/neutral/love), just post a blurb about the voting box:
Your opinion is IMPORTANT!
Please tell us how you feel about this story:
(1=Evil! 3=No thanks 7=Pretty Good 10=Holy Grail!)
I mean, lot of what you all have said is that the problems with the scoring system is it's meanings of the numbers are ambiguous. So, slap meanings on them!
(Though, really should either slap a 0 on low end or remove 10 from the high for balance, but what the hey!)
Now, ideally, there would be an adjustable slider at the end of each story that keeps track of a person's vote (per person per story), so if the story starts failing, the person can adjust his vote for that story. But, that takes a lot of 'puter to implement that for everyone, so since I have no better ideas I'll just leave THAT topic at that.
As for the reviewer thing ... was it El SOl that commented after Hungry Dog? Apologies, whomever I cite. You mentioned that HD's idea was okay, but that some might wish to remain anonymous reviewers. Well, make it not so much a requirement but an enticement: give reviewers site bonuses -- authors get more downloads and stuff if they contribute, if I recall correctly, so why not for reviewers, too? Though, not sure if that should just be open to all, or only to those who fill basic qualifications; question of risking flamers versus fact that those WANTING more downloads more likely to be "better read" (halfways intelligent and/or educated).
By the way, do registered editors get any download bonuses? I can get a couple authors to vouch for me that I'm active, if so. ;)
khyranleander (KB)
I've given the scoring system some thought myself...
I don't believe the problem is the scoring system, but that the scoring system is the only thing.
What SOL DESPERATELY needs is a set of selfless non-writers but good readers that consitently do 1 or 2 reviews a week... and even 'update' reviews for serials.
We need the SOL Celeste to step forward... and in truth we need about 4 or 5 of them.
People with varying tastes, peeves, 'writing preferences', genre preferences... and yes, we even need that arrogant-prig of a reviewer that every author hates until they review your story nicely.
Anyway... that's my two cents.
ElSol
ps. If you ask why I don't do the reviews... 'cause I give at the office. :)
Perhaps a two vote system would be good, but if you're having trouble getting people to vote as it is, that might not work.
For myself, I'd love it if folks would occasionally review one of my stories -- as long as the review could be responded to.
Laz, you're doing an amazing job. Thanks for all you're time.
~Pen~
If I don't finish it becuase its too bad to read easily then I vote it low.
What vote I give a story is usally 10 if I don't think the author could have improved the story [I let a few minor spellings/typos go, but not ifrequently I mention them in the comment.]
I vote 9 if I really enjoyed it, but there were small bits that could have been done better.
I vote 8 if I enjoyed the story, but there were some areas that could be improved [several typos/spellings will earn a story that would otherwise have got a 10 about an 8]
7 is, for me, a high average story. Good, but not exceptonal.
6 is about average, nothing special but nothing particularly wrong with it.
5 is low average
4 is really very poor. If the story is difficult to read because of poor structure, spelling, etc. I will not mark it higher than 4
3 is bad. I read to the end, but really didn't enjoy it, or I couldn't read to the end.
2 is abysmal
1 is if it is virtually impossible to read.
In practice, I vote most of the stories I read between 7 and 10, with some 6s. I don't recall ever voting a 2 or a 1.
Chris 'Awkward' McKenna
If we're going to do mini-reviews; I would prefer they be done by Authors or Premier members only.
This makes it a peer review situation, or one where the reader cares enough to fork over the significantly cheap price.
Premier and Authors are more honest in their ratings; I've discovered this since I began to post stories to Premier side of the site first. This works both ways; some times the scores go up and sometimes they go down.
----
As to scores... I think readers also take sub-genres into account.
I can write a 'less than perfect' incest story and get most readers to hit the ten; case in point The Wolf Summers, if I compare it to 'straight incest stroke' stories it's probably in the right scoring range even I think that technically the writing is atrocious.
ElSol
For example, a 1 and a 10, to me, are the two extremes. My life will be forever changed by having read this work. Okay, that's a little overboard, but most people will get what I mean by that. A 5 should reflect an average score. Unfortunatly, it appears that on this site, anything below an 8 is trash, 8 to 9 is average, and above 9 is readable. That's not a good representation
A 10 is perfection, beyond any possible improvement. The highest I ever give a story is 9. A truly abysmal story might get a 2 or a 3. A 1 should be as rare as a 10.
I'd also like to see revoting. As others have mentioned. In long stories, If I rank something a 8 or 9, and the quality suffers in later chapters, I want to be able to adequately reflect that in my vote.
my 2¢
-Harris
Now the clincher:
Chapter 11
I did poopy in my pants!
[the end]
I'd like to see the eventual score for THAT story!
(I doubt any author has done this; the worst I can see is an abandoned story.)
What would the eventual score reflect? The quality of those first ten chapters, or the readers' inflamed reaction to the last chapter. Does one chapter make an entire story a 1?
Just a thought...
There are serials - one is running right now - where the first couple of installments were pretty good, and since then (another dozen or so) the writing has become stilted and the plot completely predictable. It is possible that the 9.74 (now 9.73) rating is what would be given if people who voted early could vote again having read the whole mess, but I doubt it
Second: there are no benchmarks for scoring. "I like it" seems to equal 9 or 10, "I don't like it" is almost anything
The idea of parallel voting lines is most attractive: if you want to write a review (following whatever guidelines for reviews might be promulgated) your vote is recorded in one tally, and if you just want to read it then your vote goes in another tally.
In the past couple of weeks well written, funny stories written by authors whose work I remember appeared on SOL and were trashed in the voting. Had I not recognized the authors' names I wouldn't have read them - but having read them, I can only wonder what in the hell people are looking for - anything more than preternaturally sexy 14 year olds and stroke stroke stroke seems to be actively punished. I doubt seriously that anyone who had to think about such stories and the quality of writing and storytelling that went into them would have been giving low scores - and if such scores were given,justifying them
"I like it" seems to equal 9 or 10, "I don't like it" is almost anything'
This seems to echo what somebody said earlier. Just say you like something or don't like it.
Additionally, maybe the code can figure out an author's score by the percentage of people that approve of the majority of his stories. I'm not good at math, but there must be some formula that could combine a set of scores and come up with a total score" for that author.
I think voting a story low because it has, say m/m sex in it, even though the story codes warned the voter about the content is patently unfair. I think "don't like" should refer to the quality of the story, not your personal opinion of of what the characters do.
Lastly, can an author vote for his own stories? Is that even fair? Can any authors answer this, and tell me if you voted on your own stories? (Oh, no! I never did that, but I know somebody that did! Cmon.. fess up.)
cPuppy
Some comments:
a) Author's can see their average score... Lazeez has a formula that I believe works something...
Sum of all (Votes for a Story * Score) divided by total number of votes
Hence I have a general score 9.63 (15 stories that have a total of 4574 votes)
I remember there being a way that I SAW other writer's average score.
I've always thought this was a good statistic to have availabe for the general readership for author's with more than 5 stories.
b) Yes... I vote for myself and yes, I give every story a Ten (well, except for one which I REALLY didn't think deserved it).
If you have a question as to why I can talk about fairness when I don't treat my own stories that way...
A six-eight page AMR chapter generally takes;
2 hours for a first draft
1 1/2 hours for a second draft
1 hour for a third draft
1/2 for a fourth draft
2 days of biting my nails while my proofreaders/editors
get back to me.
1-3 hours for the final draft
This is not to say ANYTHING about the mental outlining.
Personally, I think I can give myself a ten.
An AVERAGE short story of 12-24 pages
6-12 hours first draft
4 hours second draft
2 hours third draft
2 hours fourth draft
1 hour final presentation draft for FRC
2 days of kicking myself for letting ANYONE touch my story even just to proofread it.
1 day of thinking about these people that don't actually understand how perfect the story is and realizing that MAYBE they have a point and I should change something
2 hours Posting draft
---
This is the part where someone says I don't have a right to click on that ten, right?
If someone's going to put in the work... then they have the right to think whatever they want, even if in all reality their work is a piece of shite!
---
Finally, Lazeez, if you're reading this... you're not going to be doing all the moderating will you?
That's a lot of work to take on.
ElSol
ZAGAT surveys of restaurants has three basic categories in which a patron is asked tro provide a rating. Now, if there's anything as complicated as sex and bhow its portrayed, it has to be food and how & where its served.
If the Zagats can do it with 3 criteria RATED 1-2-3, why cant you?
So here are my 3 cents:
1.Authors should be prohibited from voting.
2.Premium subscribers should be able to vote in three categories:
* CREATIVITY AND ORIGINALITY OF PLOT.STORYLINE.
* QUALITY OF THE WRITING.
* QUALITY OF THE SEXUAL DEPICTIONS.
3. All readers should be able to see the three ratings, in order to judge their own views against a standard, arbitrary though it be.
thanks for the chance to participate.
Whata bout setting up a different Zagat scale for Premium subscribers
----
JimC --
At first, I thought that the hate-mail was bad; although actually I don't get any was bad.
The WORST is the passive-aggressive crap.
'Now, this chapter was significantly better than your last one...'
and my favorite;
"The story has stopped going in the direction you originally intended... I've stopped reading."
------
As a writer, I feel incomplete in many ways unless someone gets offended at some point by my writing.
Wait until my 'Christian Religious Fantasy' story; I'm getting my 'I'm Catholic... you're going to hell' rebuttal templates.
----
Plus have you ever seen anyone hate something truly worthless?
ElSol
> 1.Authors should be prohibited from
> voting.
Would you tell a mother that she can't say her baby is beautiful? That only OTHER people are allowed to make that judgement.
It's 1 vote over 125 days after the 1st time most of us don't remember to do it again... you want to know why?
Because a GOOD writer can't read their own shite after awhile... they've grown and every little mistake is exploded ten-fold.
> 2.Premium subscribers should be able to
> vote in three categories:
> * CREATIVITY AND ORIGINALITY OF
> PLOT.STORYLINE.
> * QUALITY OF THE WRITING.
> * QUALITY OF THE SEXUAL DEPICTIONS.
The problem is you need a system that gets used... notice what Lazeez says -- it will not be used.
People won't go vote for the future of their country, do you think they'll go through three clicks to get this done?
I can't get through a 'shitty' story long enough for my vote of 'Oh My God! Learn to write in English!' to count, which is basically a public service.
-----
Lazeez seems to have made up his mind how to move forward... I'm cool with except for the fact that moderating that might be a lot of work for him.
ElSol
When the reviewing system gets implemented, god knows how long that will be, I will be asking for volunteer moderators. I can't moderate all that stuff. My plate is more than full enough already, I can't pile up more stuff.
There will be a need for a minimum of two moderators, each review must be approved by two moderators. I may pick up the slack, but I can't fill the position of a full time review moderator.
Although, from past experience, I don't expect to see enough reviews to keep me completely busy if I decide to become a moderator, but then again, the site is way bigger than what it was in 1999 and there are way more users.
Glad to see this discussion, it's giving me some basis for some things that can be done in the future in order to make things more meaningful.
> that truly manages a viewer's "average" and take it
> into account. That is, if a reader votes 50 times and > consistently votes 50 out of 50 10's then that vote
> would count as a 5 as "average".
This will never work.
Reason being, the way some people read.
I'm a writer, 20 hours of my 'free' time are taking up in keeping up with my own writing.
A story has to be truly impressive for me to read it; case in point, the Top 20 serials... I can't get through the first chapter.
Therefore if a writer captures me enough to make me take time out of my writing to read... then he/she fucking damn well deserves a TEN+.
You're penalizing my scoring because I'm much faster to recognize shit, shit that smells like roses, plastic shit, stuff that looks and smells great but tastes like shit... and finally stuff that other people might think is creme brulee but I still think is shit.
ElSol
I think part of the problem with the scoring system is that a 1-10 rating has a different meaning to different people. For instance, to make a 10 on my scale, i'd have to want to read a story about four times (more if its very short); but someone else may rate a story on how quickly it makes them...ahem...satisfied. The multi-vote reviews help there, but i still find too many 10/10/10 postings for my liking.
OK, so here's my wild idea to add to the bunch. If you want to place a vote, you have to wait at least an hour (or two, or six, or whatever) after you read the story, but before it disappears from counting toward your limit of 16. This would discourage the knee-jerk "I finished this story, that must mean i really like it," and it would also force people to really consider how much they enjoyed the story versus all the others they had read.---And this might not work at all, because it would definitely reduce the number of votes (after all, who wants to remember to come back in an hour?) but those who did come back would probably only have the highest praise or the lowest condemnation.
Moliere's comment is intriguing.
Perhaps there could be a "Volunteer Reviewer's" section. Each portion of the section could be dedicated to a particular genre, accessed by links.
This would especially be helpful with 'Short-Shorts', that never make it to the 'Update' page, because they are just one chapter.
If the "Volunteer Reviewer's" page had a link on the 'Home Page', it would be as easy to access as the 'Update' page is, now.
Are there any further comments as to Moliere's idea?
TeNderLoin
:)))
The one problem with this, as already stated, is that I (an dI am sure many others do the same) don't (or simply can't) read stories on a monitor. For many reasons - ability to read and concentrate on screens or as simply as paying for online time.
Particularly if we have found one that already has many chapters - if you have the constraint of 'pay by the hour' internet conneciton, you can't always afford to simply leave it connected - so, what do you do?
You save it. You find a 10, 20 , 30 or 60 chapter story - you are going to capture each chapter, save it and then either read it or print it (I tend to do the latter).
As for - when do you score a story?
Recently there have been a number of very good general stories which are still being added to.
Do I rate them on what I have read so far (v.good in almost all cases) or, as was suggested, wait until Chapter 11 gets published to feel that big 'let down'? (And this happened with one sotry recently - have been waiting for weeks for the next chapter only to have the Final Chapter posted and - hey, what's this? It's finished all of a sudden? What a let down... In this case I would have rated the story, prior to the last chapter a 9 or 10 (because it WAS a very good story) but (and there is always a but) - the last chapter was so disappointing that if I had voted on it (I haven't btw) I would now only give it a 2 or 3.
Where do you draw the line?
Personally - reviews yes - good idea - but they should be open to ANYONE to post a review - maybe even, as already suggested - rewards for providing reviews - the more you do, the bigger the rewards???
And in the long tem - multi-part stories should have Chapter Scores - like my 'let down' above - given the last chapter, would I recommend it to anyone? Probably not - last week yes, but now - no - it left too many things hanging (excuse the pun...)
StoryO'Mine
Then if you scored it as such, and were able to score it as such - "4 - not worth reading" - then your voting habits would still be counted accordingly.
>Therefore if a writer captures me enough to make me >take time out of my writing to read... then he/she >fucking damn well deserves a TEN+.
That doesn't make any sense. That suggests that every single thing you read is absolutely perfect on a comparative scale, when you're admitting that you're not reading enough to make a real comparison based on what else is on the site. Plus the idea of a score going "TEN+" adds merit to the notion that even within your reading there are levels of comparison, except your scale (based upon your own choices of reading) begins much higher. Your average begins higher, that doesn't mean it isn't your average and that it shouldn't be scored as such. Note: A person who consistently scores things as 4's across the board will be curving their own average up "TEN+" when they actually like something, so there will be some compression at either extreme end. The idea would be to move more scores to the middle, because while that would be painful to authors it would make the scores off the middle more meaningful.
>You're penalizing my scoring because I'm much faster >to recognize shit, shit that smells like roses, >plastic shit, stuff that looks and smells great but >tastes like shit... and finally stuff that other >people might think is creme brulee but I still think >is shit.
No, I'm penalizing your scoring because you're not using your fast shit smelling senses to score something as such when you see it. If the only things you scored on SoL were gems and turds, your scores would move toward the middle for each still just on the basis of your voting habits. If SoL suddenly started hosting only turds or only gems then your voting habits would assumedly reflect that, making your higher votes more meaningful or your low ones more meaningful down the line respectively.
I think the key issue would be this timer Lazeez is talking about. I think it would pretty much either have to go or be a variable affecting your vote in my system - because as you say - sometimes you can just look at the first paragraph and know something is a bunch of jumbled words right from the get go.
Removing it entirely would open the system up for abuse again, but simply making the timer a "short list" for nuancing the scoring might work. If someone spends more than X amount of time reading then their score would count as a 100% vote, but if they spend less than X time then their vote would count for 50% of a vote, or some such. If someone were running around being an idiot and seeing how fast they could score everything as a 1 in my system, they'd still be registering as an average vote after a while a half a vote at a time.
RE: Scoring
My preference (for serials) with a single vote is to give it a 10 to encourage the authors to continue. Having 125 days makes this type of activity more viable since you can adjust your vote later.
You've certainly put a lot of time and effort into this.
After reading your explanation of the scoring phenomenon, an idea immediately came to mind.
1.Continue the single vote "popular" voting system as it stands.
RE: Scoring
My preference (for serials) with a single vote is to give it a 10 to encourage the authors to continue. Having 125 days makes this type of activity more viable since you can adjust your vote later.
You've certainly put a lot of time and effort into this.
After reading your explanation of the scoring phenomenon, an idea immediately came to mind.
1.Continue the single vote "popular" voting system as it stands.
RE Scoring 2.
ooops, I tried to use the tab key and it sent the comment before I was done......
As I was saying....an idea came to mind of combining to of your ideas.
A. Continue the "popular" single vote system.
B. Institute the multilevel 'Celestial Reviews' methoed as you had before as part and parcel of your reviewing system. If anybody has the motivation to write a real review, they should be perfectly capable a selecting some score at the end of that process. This would also serve to keep "dablers", "diletantes" and the like from misusing the review system.
The result would be a popular score and a reviewers score.....
...for what its worth.
> After reading a few more of these replies, plus Lazeez's update, I realized
> that I was looking at this the wrong way. My suggestion about an "approval" > percentage was more about ME (as an author) finding out how many people > that saw my story description, checked the codes, and decided to read the
> story liked what they read. This is a statistic that would be very meaningful to > ME (as an author).
I have an opinion on this... SHOCKER THAT!... it follows my opinion on 'constructive criticism'.
In many ways, an SOL score is meaningless to you unless you agree with it.
So if you think 'Lucky Tickets' is better than 'Lucky Stiff', their relative scores are going to seem off. This travels outward obviously... if your scores don't sit well (whether up or down) you're going to have a problem.
And it goes even farther out, my scores will never sit well while their higher than Steiner's Aftermath until I think a story is as good, better, or different enough to make the relative scores understandable.
> I don't look for reading material in Recent High Scores or Top Downloads.
> Why? I'm always seeing the same twenty stories on those lists. Those
> interminable 100+ chapter stories that are daunting just in their size.
It's gotten a little better since the new influx of readers reduced it to the point where you no longer need a 9.7 to get on the 'board'.
I wonder if its too intensive to reduce the mark to 15 days since last update... that might let a few others get on it and unlock the top of it a little when writers take a break.
> If the interpretation is shown, and the actual score is never shown, would
> anybody be hurt (author, reader)?
Yeah, then we couldn't keep score, dude :)
Oh... come on... you know you'd do the "I'm Number 1... I'm Number 1" dance if you hit the top of the Top 20 Serial board.
That's almost as delicious as the emails that you're like "I didn't need to know that shit about your life! Oh... hey, attached picture! Uh... tell me more, sweetie."
ElSol
Terry's suggestion to use statistical modeling holds the most water. I would approach it in a slightly different way though, based on how psychological tests are scored and normed.
First off, move away from 1-10 as a scale, it's too confusing. Switch to a new scale of 1-5, but don't CALL it '1' and '5' on the form. Just have 5 radio buttons sitting in a horizontal line with 'Hated it' on the left and 'Loved it' on the right. As a bonus, you can easily move previous scores by doing a simple conversion
"1 or 2" becomes a "1"
"3 or 4" becomes a "2"
"5 or 6" becomes a "3" etc
This is called a "Likert scale" and pops up in a LOT of psychological tests.
Next, change how the scores are weighted based on the voter's previous record. Someone who always votes 'loved it' and never votes anything else; those scores don't really give you any information. Voters who vary their scores whether they 'tend' toward negative reviews or 'tend' toward positive reviews will give you valuable information by using Z-scores.
Last, the results shown for a test score become an average of the Z-scores. In the end, you have a statistically and psychologically valid measure of what people think about stories with enough differentiation for the scores to be meaningful. I'd even code it for you, just to see if the formula works for the data you have!
(I don't have a blogger account. It's Switch Blayde.)
1. Authors should not be able to score their own story. Why would they want to? I assume authors want to know how their story rated. If they thought their own story was less than a 10 they'd change it until they thought it was "perfect." So an author will always rate their story a 10. My vote... author cannot rate their own story.
2. I think a multi-part story should have a rating for each part and one for the entire story. I don't rate parts because I don't know if the whole story would get an equal rating to the part I just read. It would also be good for an author to see how one part compared to another (a good learning experience).
3. Ignoring a person's rating if they haven't been in the story for a period of time has faults. There are many times I don't finish a story in one sitting. Sometimes I go back and finish it... and then cast my vote. Now I know that vote may be ignored.
4. My biggest problem with the scoring is its credibility. I read a story the other day that was so poorly written it was almost impossible to follow. But it had a rating over 9. I think people must rate the genre (subject matter) more than anything else. That's not right and doesn't help the author (or other readers) much. Because of it, I'm not sure what value the rating system has. I'd rather have fewer people rate stories in exchange for meaningful ratings.
5. As an author, I'd like to see more than an average score. The average should be posted like it is, but when I look at my stats I'd like to see, by story, the number of 1's, 2's, 3's, etc. Two 5's give the same average number as a 9 and a 1, but they have significantly different meanings to me. Two 5's mean the story was mediocre. A 9 and a 1 means someone loved it and someone hated it.
Switch Blayde
Yes they do tell you something (and this is a problem when 'scoring' is voluntary.
It can mean that the person taking the trouble to score DOES like that type of story - it can also mean that they are very selective in what they read and, just possibly, they only read stories that they do actually like.
OTOH you will get the opposite effect - people who read crap stories (sorry - only way to say it) and therefore think they are so bad or bad enough that they can't be bothered voting for them - they see the voting as a waste of time. This last attitude needs to be worked on rather than the scoring system itself.
And yes - agree - authors should NOT be allowed to vote on their own stories under any circumstances.
Authors should, however, be able to get a breakdown of voting for their stories - not necessarily who voted, but what value each user voted with - how many 10's, 9's, 8's etc etc.
> And yes - agree - authors should NOT be allowed to
> vote on their own stories under any circumstances.
Let me be perfectly clear on this:
I WILL resent this and while other writers might not want to voice it, I know I'm not the only one that would.
This is going to buy negative feelings because other people... a majority of which are non-writers so don't understand... are allowed to judge what a writer has spent hours of THEIR free time putting together and gives away.
When somebody hits that vote that drops their score... a writer cannot help but feel that THEY are being judged.
It's 1 vote that makes strict SOL writers (like me) feel like the the process is now complete and from that point on the story belongs to the reader too.
I'd rather see the scoring system removed than have someone tell me I can't cast a vote for my own story.
ElSol
Would it help to have a list somewhere of specific scoring criteria? A reader would finish the story, come to the score form, and see a little link Scoring Criteria Readme. Clicking the link would bring him or her to a short page outlining what the scoring system is supposed to represent - an overview, and expansion on the Excellent, good, fair, average, etc. list.
This may not prevent bias and abuse, but it will at least assist readers who are not really sure what they're voting on. This might clarify things for the "10 if I liked it, 5 if it was okay, and 0 if I didn't like it" scoring crowd.
Here's an example of what this criteria might look like:
10 - vote a ten if the story was consistently perfect in story-telling, stroke value, spelling, grammar, and mechanics.
9 - vote a nine if this story was almost perfect, but had a few minor flaws here and there.
8 - vote an eight if you thought this story was fairly close to perfect but could a few tweaks here and there.
7 - vote a seven if you thought this story was above average, with forgivable flaws, but requiring some enhancements to make it perfect
...
0 - vote a zero if you thought this story was the worst thing you've ever read in your life and you wish the author would be electrocuted by his own keyboard - if this story is completely devoid of any recognizable coherent thought, and has the spelling and grammar of a four-year-old.
and so on.
Further, if Lazeez decides taht an 8 means "you thought this story was fairly close to perfect but needs few tweaks here and there," and people follow it, would that be fair to people that already scored the story before this guide was created and had a different idea what an 8 means? I think that would be rather rude! Oh, no! All those 8s should have been 7s!
I'm sure that the score is supposed to rate a story's quality but I have no idea how anybody comes up with a score. Can you point to a single story on storiesonline that deserves a 2, and another one that deserves a 1? For that matter, one that deserves a 3, 4, or 5? What IS the difference there? How bad must a story be to get a 4, but not a 3?
I have a feeling that the only scores that are cast are 0 and 6-10. I'm pretty sure a user's score for a story isn't recorded, but added to the aggregate for a story (which is why the site doesn't allow a person to change his score). However, the raw vote information would be useful, just to know the standard deviation! Hey.. let's go all out an have ALL scores stored, and if you click on a story's score, you get to a page that shows who voted and what each person scored for this story! After all, if you are posting your feelings about a story by voting in the first place, then you should be willing to stand behind what you've voted.
And how about a "people that liked this story, also liked..." link?
> and mechanics.
Two problems...
a) The simple one; the last time I checked storiesonline.net was not strictly a sex-only story so insisting that a story have a particular 'stroke' value would deny many great stories a ten.
I didn't read Aftermath for it's stroke value; nor did stroke have anything to do with my rating.
b) Perfection is impossible... no story should be rated a 10 by any reader by your measure.
> After all, if you are posting your feelings about a
> story by voting in the first place, then you should
> be willing to stand behind what you've voted.
Can you say will kill most if not all voting on SOL... there's a hard enough problem getting people to vote without adding a false sense of accountability.
----
Now on to individual chapters having scores.
From a writing perspective, this is a shitty idea.
Case in point from my own experience: I wrote AriaX with the endpoint and how the ENTIRE story worked together.
The posting of the first chapter got: 9.4 rating
The posting of the second chapter dropped it to: 9.34
Prior to posting the last chapter there were 86 votes
The posting of the last chapter raised it to a: 9.62 at 192 votes.
There were two factors involved in the jump at the end, it was released into general consumption {not premier} and a reader was able to get the ENTIRE package.
What's the true score... the 'individual' chapters or the 'score' achieved after the readers could see everything or the fair combination of both as shown by the SOL score.
I have my opinion, other people would have others.
I don't want new writers and 'old' ones to be put in an expectation loop because individual chapters have a score... just because something looks bad to the reader doesn't mean it doesn't fit perfectly into the overall story a writer like me has in mind.
If I had my druthers, I would have the option on SOL for specific stories not to be rated until the last chapter.
For long serials, I might not choose the option but for 1-5 chapter mini-serials it would be useful to see what the effect of getting the 'whole' story is.
By the way, I choose to post the indy chapters for short serials as a way of supporting SOL's premier site... otherwise I would probably opt to not post until I could pass everything over.
ElSol
(snip)
> Now on to individual chapters having scores.
> From a writing perspective, this is a shitty idea.
> Case in point from my own experience: I wrote AriaX
> with the endpoint and how the ENTIRE story worked
> together.
My point exactly. I think it does an injustice to rate an incomplete story. What I said was that it was not a good idea to rate a work in progress, but if SOL wanted to do so, there should be a rating for each chapter AND a final rating. Right now there's a rating for whatever chapter the story is up to and that's the only rating the story is given, even after more chapters are written and it completes. At least I think that's how it works.
Switch
Now to the ratings. Someone posted a blog believing no one ever gives a rating on the low end, like a 1 or 2. I recently gave one (I think it was a 2). Why? If a story is that bad, I don't bother voting. I guess I don't like to give that low a rating and feel better simply not giving a vote. But this story was almost unreadable. For example, the use of pronouns was such that I often had to read a sentence several times just to figure out who was who. I never even got to the end of the story. I tried because the subject was one I enjoy, but the story was written so poorly it wasn't worth the bother. So based on what I previously said, why did I rate it? Because it had a very high rating! I still can't fathom how it got that rating. I lost any faith I had with SOL's rating system when I saw that story's rating.
Finally, about providing the rater's id. Absolutely not! There is no reason for it and it will surely reduce the number of ratings (and probably the ratings as well).
> rate an incomplete story. What I said was that it was > not a good idea to rate a work in progress, but if
> SOL wanted to do so, there should be a rating for
> each chapter AND a final rating. Right now there's a > rating for whatever chapter the story is up to and
> that's the only rating the story is given, even after > more chapters are written and it completes. At least > I think that's how it works.
It might be an injustice to rate an incomplete story... but it is a disservice to writers especially new ones who are just learning certain things to rate each individual chapter.
Certain things make a chapter better than others... action, sex, intrigue...yadda, yadda.
Those scores can be seen as 'approval' ratings by authors (new and old) and rating chapters individually may push writers into a less 'wholly' developed story for better scoring chapters.
Like some artists think that services like Mac's music site pushes them towards 'singles' rather than albums.
In my opinion rather than score chapters individually, it's better to allow the writer to choose that an individual story cannot be rated until it's over.
Then the writer has to choose between a score that might convince readers to give a serial a chance, or dealing with less readers until his story is in the form he believes it's meant to have.
The present system of rating incomplete stories is a fair comprimise between rating individual chapters and just rating the whole story if the writer.
But having individual chapters EACH have a score is bad idea in my opinion.
ElSol
I do think if a story or a chapter is reposted the votes should reset. I think some authors repost just to raise the number of reads and votes.
Best Wishes all,
Iolanthe Dance
The Return - posted between June 19 and July 20th - 12 chapters which, in themselve, were very good - each cchapter was basically self-contained and teh ongoing story/plot was reasonably well thought out. My score would have been an 8 or 9 (a 10, to me at least, would be absolute perfection and would have to be an exceptional story regardles of what it is all about).
But then Chapter 13 - didn;t come along until Oct 13 - after 7 to 14 day cycles for posting, nearly 3 months elapsed. This also turned out to be the final chapter.
And the outcome - a shocker (IMHO) - a shorter chapter than most and it was an obvious attempt to wind the story up as quickly as possible - to the extent that if I rated the entire story, this one chapter alone would have made me give it nothing more than a 5 - BUT, the preceding chapters WERE worth reading.
So what should I do - rate it based on how let down I believe readers are from because of this final chapter? Or rate it per chapter? Or both?
StoryO'Mine
Best Wishes,
Iolanthe Dance
If readers cannot change their vote, then there is a disincentive to rate a story early on... it will take 125 days before a high score (for a story that started out nice) can be "corrected" by a new score.
Short stories (or long stories that are posted in a single block) are the exceptions to this rule.
Maybe votes should reset after a story is completed? (I canot see how this would work), but it would reflect that a score that rates an entire story should be more indicative of the story as a whole.
Somebody mentioned that the difference between a score of 2 and 3 is meaningless, and later on, somebody said that he scored a story a "2" because it was unreadable. If that was the case, why wasn't the unreadable story rated a 0 or 1?
Until this whole thread came up, the lowest I scored anybody was 5. From what some people said, I could be guilty of "grading off the curve." (I cannot see somebody taking the time to actually write a story and sticking his neck out and posting it for the world to see, and having everybody send him zeros. That sounds needlessly cruel.) Could the scores be limited to 5-10, with another selection "Didn't like it" (which would become a zero). Average scores less than five would not be shown, but described as "below par."
On the web, Amazon has a similar service to SOL (although it is a commercial site). When you get to the story you are interested in, you can see a random mini-review of it from a reader (these are moderated), with the ability to see ALL the reviews, sorted by date. The site also says that people that purchased this book, also purchased some other books, and you can see things that you may not have realized were out there. I remember reading somebody suggesting this earlier.
As for review moderators, let people volunteer to be a moderator and/or reviewer in their personal profile. Every week, the site randomly picks out so many people, and gives them a chance to look at the mini-reviews that were posted, and either approve or reject them. It takes more than one moderator to approve a review (let's say 2). In addition, if 4 moderators disapprove of a review, it gets tossed. Switch the moderators every week, and make rules like "moderators cannot post reviews when they are moderating", and "moderators cannot review their own stories" and you should have something that works for the most part, and should be self-correcting.
All these techniques have been used on other sites with varying success.
How I chose a score:
If I really like the story and it is grammatically correct as well it gets a 10.
If I really like the story, but it contains some spelling and grammer errors it gets a 9.
If I really like the story, but it has lots of errors it gets an 8.
If I don't like the story or it contains so many errors that it is cumbersome or distracting to read I don't finish reading it and it gets no score.
I don't read stories that don't appeal to me by reason of the story codes. It wouldn't make sense to read a story that I didn't think I would enjoy. I barely have enough time to read stories that I think might interest me as it is.
As far as the scoring scheme itself goes, I'd like to see 2 catagories, one for the storyline, plot, characters, etc. and one for "technical" merit, spelling, grammer, etc.
> it was unreadable. If that was the case, why wasn't
> the unreadable story rated a 0 or 1?
I thought the story line (plot) was okay. If there were different categories to rate, I would have rated the story line a 6 or 7 and the quality of writing a 1 (I don't think there's a 0).
Since there is only one rating, I came up with a 2 (I gave the author credit for the story line). I was going to give it a 3, but the current score was so unbeliebably high (in my opinion much too high) that I dropped it to a 2. This brings up another problem with the rating system. I shouldn't have been biased by the other voting, but I'm human after all.
> chapter? Or rate it per chapter? Or both?
I seem not to be getting across the negative effective that rating each chapter could have... so let's try a different angle.
What benefit is gained by rating each individidual chapter?
Let's take your example... 8 chapters that were absolutely the best, each chapter a 10+ then as a someone so eloquently put...
Poopy in the pants!
Let's crunch the numbers --
8 chapters at a 10 = 80 points
At the end you state you believe the story ended up being a 5 because of the 'poopy pants' incident...
80+ X/ 9 = 5
80 + X = 45
X = 45 -80
X = -35
So in your mind the last chapter rated a -35 (or thereabouts since probably not EACH chapter would have gotten an 10)...
What's the point of reading 8 REALLY Good chapter in a Really Good story who's ending rates a -35?
The chapter is just a building block, I don't walk up to a house and tell the owner:
"Hey really nice bricks... but your house is the fucking ugliest thing I've ever seen! Who designed this piece of shit Godzilla with his tail?"
Oh... yeah the 'really nice bricks' does a lot!
Rating the individual serves no purpose, it only hurts authors and misleads reader.
----
Here's how I think of 'pre-completion' scoring?
It is how a reader rewards an author in the hope that they will keep writing, and finish something they like without any 'poopy incidents... click on the ten when you get the chance as a way of saying 'Thanks' for a good time so far.
While email is more effective... this is at least something.
ElSol
> going to give it a 3, but the current score was so
> unbeliebably high (in my opinion much too high) that > I dropped it to a 2. This brings up another problem
> with the rating system. I shouldn't have been biased > by the other voting, but I'm human after all.
It's not just you... I won't read a 9.7+ story, not even my own.
My expectations are too high.
ElSol
I am not a fan of individual chapters getting a vote, for the same reasons that El Sol commented on. In a long serial, some chapters are necessary as building blocks.
By the same token, I don't want to see a story that I thought was off to a very good start and voted a 9 on, to turn into total crap, and the author gets to keep the 9. Maybe I now think this story is a 6, because it is average. To get that score, I need to vote a 3, to balance out the 9 I gave it before. I'd like to be able to change my vote, not vote on a chapter.
If a writer badly miscodes a story (Such as labelling it a MF Rom, and it is really a MF Violent Rape), then they deserve a harsh spanking. I don't care how good the story is, if you give me a totally unexpected squick, you earned a massive downgrading in score.
Some readers seemed to give their favorite authors a TON of leeway when it comes to a story. These fans don't care if the story is nearly identical to a previous one, except for location changes and name changes. They don't care about plot problems, spelling errors, grammatical errors, etc. Their objectivity has gone out the window. And you can't correct this by changing the voting system.
Garylian
Looking at the scores for stories published, most seem to be in the eights and above. So by definition, does that make eight the new five?
I mean, if the average story vote is, say 8.25 ( for discussion purposes), then that would have to be the new average and anything above that was good and any thing below was poor.
Surely if you want to run a scale out of ten, you have to limit the use of extremes?
By this I mean: You can vote as many 4, 5’s and 6’s as you want over a week, but you can only use 7 and 3 eight times a week, 8 and 2 four times a week, 9 and 1 twice a week and 10 an 0 once week.
If there was a limit on the amount of times you could vote ‘10’ then inevitably you are going to save it for a story that rightly deserves it.
Let’s face it, its stupid having an average on a 0 to10 scale higher than 5.
You might argue that the stories deserve their scores, fine, but these stories must be taken in context with other stories posted on the site. If all the stories are of the same high calibre, then by default they become ‘Average’.
A limit MUST be placed on the amount of high and low scores that can be given out to stop them being devalued. If everyone that was involved in war was given a Victoria Cross or whatever the American (Or any other foreign nation) equivalent is, then what would you give to people of outstanding bravery?
If you had an infinite amount of money to spend, nothing would have any value as you could go out and buy another (Art work and ‘one offs’ aside). If you work hard for something you appreciate it all the more dont you. It’s why people rock climb when they could take a cable car or other easier means to the summit. Without a sense of achievement there is no value.
That’s why a strict limit should be placed on eights and above and twos and below (0 being the equivalent of a cat running over the keyboard and the resultant garbage being posted)
As many of you have already noticed, there are a lot of stories being scored eight and above which should never have scored that high.
So you may now argue what’s to stop you only voting once a week and that vote being a 10? (Or 0 if you hate someone).
Well the simple answer is that you have to earn it. Earn it?
Here is a suggestion:
Vote two times a 4, 5 or 6 and you earn a chance to vote a 7 or 3
Vote two times a 7 or 3 and get a chance to vote an 8 or a 2
Vote two times an 8 or 2 and get a chance to vote with a 9 or a 1
Vote two times a 9 or 1 and get a chance to vote with a 10 or 0
So in order to vote with a ten or zero you must have voted at least 30 times.
Now if you had to vote at least thirty times to earn the chance to vote with a 10, you would make sure it was a damn good story.
Most people have an ‘account’ to use the site, so apart from the programming involved there is nothing to stop peoples accounts from also keeping tally of how many ‘points’ they have earned in the same way it keeps a tally of how many stories you access.
It’s just an idea, but really, you should not be able to give 10s (and 0s) out so easily.
Pixy
Then you are just wasting everyones time. If a story deserves a lower score then a lower score should be given.
I think scoring can be looked at philosophically. It depends on what you, as an author, are trying to do.
Some authors are trying to write the "Great American Novel" (you can substitute whatever country you're from), or the "Great American Erotic Novel", as it were.
Others, like myself, are niche writers, who know we are only going to appeal to a restricted clientele.
Based on the feedback I get after I post each story, there are indications I have a core group of fans who like the KIND of stories I write, and vote accordingly. My scores almost always peak early and then modereate as people who aren't necessarily in my niche give a story a try.
But I have gotten a lot of feedback that says things like "I'm not into [this or that], but I liked your characters, and the way the story developed." I have written the same kinds of feedback myself. That tells you people like the STYLE of your writing.
Of course there are some people who can't get past the spelling errors, or the punctuation (I love to abuse an elipse, as anyone who'se read one of my stories can attest) or whatever technical aspect of writing is their thing, and they're never happy. I can understand that, having been turned off by too many errors, and having closed many a window without finishing or voting on a story.
But that's one way of voting, in a round about sort of way. When I see that maybe 10% of the people who've read a story actually took the time, or had enough interest to vote on it, that tells me a little bit about the size of my niche.
The rest of them said "No thanks, I'm moving on". But they didn't tank my story by giving it a low score just because they think incest (or whatever) is sick. They just moved on, trying to find better writing, or a different kink, or whatever it is they're looking for.
Some people might say that's the same thing as running a popularity contest.
But I know why I write, and I know why I post. I post stories just FOR that core group of people who have the same kink I have. And my goal is for their hands to be too busy in their laps to close that window before they get to the voting box.
The scores my stories get give me some indication as to how successful that goal is. The number of people who don't vote at all, or 'taste, but don't swallow' gives me another indication. The number of people who've made me a favorite author gives me still another indication.
That's why I like SOL and post here. Scoring will always have a subjective element, and at SOL you have more than one way to get an idea of how you're doing.
All this is to say that the idea of having a 'regular' group of reviewers try to review all the different sub groups within the genre, won't work well. There are too many niches, and too many reasons why people write, and too many types of readers for any select group of reviewers to serve everybody well.
I like the idea of having reviews, but if we're going to have them, let the average reader do them, moderated for meaningful content. Then we'll have yet another indicator of how we're doing, how we can do it better, and what we might want to read when we're not writing.
I belong to many other site's and keep SOL as my prime site. there are a veried amout of stories , yet some are not finished, witch I do wish they would get with the program. As a member I would and will follw the writing's of many author's that their stories interest me. they hook you in and seem to never finish a story. Now some years later I for one am still waiting. Whats with you guy's Lazee is doing his job, why can't you.
The scoring is just that it's to let us know where we list, to help us improve our writing's. Really is it nasasiry to be so slow that you can keep up with us.
I answer that for you writers.
YES, you can't or won't keep up . because you are afaid that you will be blinded with writers block.
Or even not have a thought in your little mind, We feed you with prise in hopes that you will be posting more chapters. but no you try something else. Some are good, put you forget the ones that are great. Every writer has a point where it's hard to come up with something new. won't treat us reads like fool's we know there are some who just don't care. GET WITH THE PROGRAM, FOOL. write and we will READ.
Thanks once again to LAZEE, for his affords in managing this site. keep the faith, I do.
if you out there don't like what I said Email me ,I'll tell you where too go.
thanks again...
the one and only
fintinman