Scoring System Changes
Since implementing the scoring system on the site in 1999, it has been the only controversial part of the whole site. Many users found it useful, some didn't trust it and some ignored it.
To authors it's even more troublesome. Many authors expect the score to tell them how they did in their writing, they wanted it to reflect the effort that they put into the story regardless of the story's content and its subject's appeal to the readers.
Those concerns and expectations are not something that I can really address. Authors need to simply realize that the score simply reflects how much a reader liked the story and whether they recommend that others read it too. It's like a thumbs up signal.
However, there is a problem with the scoring system that I can address: Score compression.
Score compression is when votes, like they are now on the site, tend to be mostly on one end of the scale. Last check revealed that the median for all scores on the site is 8.62!
A median of 8.62 means that half the stories on the site have a score of 8.62 and more. That means about 8000 stories have about 1.2 points spread. That means anything below 9 didn't get a good score. 8.62 is so close to the top, it's making scores meaningless.
The reasons for this compression are multiple.
So, I'm introducing two changes to the system to be rolled out gradually.
The first change is the wording accompanying the number scores in the vote form and I'm removing the numbers. I'm proposing the following as the new list:
Amazing; Impossible to Improve
Excellent Story
Great Story
Good Story
Not Bad
Some Good, Some Bad
Not Good
Pretty Bad
Hated it
You Call this a Story!?
This way it's not mixed signals. The old list was a bit misleading to authors as it implied that the score may represent the readers' judgement on how well the story is written. Words like 'Needs Work', imply that the reader noticed the errors in the story and commenting on them.
This list is not final. I'm open to suggestions of a better wording that improves the distinction in your minds about the meaning of the score you're casting.
The second change is the more drastic one. I'm replacing the current scores with weighed scores.
The new scores shown on the site will reflect the story's score relative to the median of all scores on the site. This will have the effect of lowering all scores. I've implemented the formulas that calculate the weighed scores and here is a sample of scores and their new values:
Old Score -> New Score
(average) -> (weighed)
10 -> 10
9.85 -> 9.56
9.5 -> 8.55
9 -> 7.10
8.62 -> 6
7 -> 4.93
6 -> 4.28
5 -> 3.62
4 -> 2.96
3 -> 2.31
1 -> 1
One thing to remember, the weighed score is relative to the current median. So a story's score may change even if it received no new votes. If the median changes, then the story's score will change.
Hopefully, the wording change will make the votes that readers cast more reasonable, so that automatic 10s change to something more meaningful.
One problem I don't know how to address is the fact that the more recent the story is on the site, the higher the average score is; this is related to the psychological effect of higher and higher scores. So if you have a reasonable solution, I'm all ears.
I know that scoring is a controversial subject but let's all try to be as objective and reasonable as possible. I'm trying to make the system work for everybody the best possible way. I appreciate everybody's contributions.
And before you fire off your reply, one thing I will not do, I won't ever scrap the voting system. So don't even suggest it.
To authors it's even more troublesome. Many authors expect the score to tell them how they did in their writing, they wanted it to reflect the effort that they put into the story regardless of the story's content and its subject's appeal to the readers.
Those concerns and expectations are not something that I can really address. Authors need to simply realize that the score simply reflects how much a reader liked the story and whether they recommend that others read it too. It's like a thumbs up signal.
However, there is a problem with the scoring system that I can address: Score compression.
Score compression is when votes, like they are now on the site, tend to be mostly on one end of the scale. Last check revealed that the median for all scores on the site is 8.62!
A median of 8.62 means that half the stories on the site have a score of 8.62 and more. That means about 8000 stories have about 1.2 points spread. That means anything below 9 didn't get a good score. 8.62 is so close to the top, it's making scores meaningless.
The reasons for this compression are multiple.
- Some readers never vote anything but 10; they're nice people, they don't want to hurt the author's feelings.
- Some readers vote only for stories they like. For stories they don't like, they abstain from voting.
- The psychological effect of high scores. The higher the scores the higher the readers will tend to vote.
So, I'm introducing two changes to the system to be rolled out gradually.
The first change is the wording accompanying the number scores in the vote form and I'm removing the numbers. I'm proposing the following as the new list:
Amazing; Impossible to Improve
Excellent Story
Great Story
Good Story
Not Bad
Some Good, Some Bad
Not Good
Pretty Bad
Hated it
You Call this a Story!?
This way it's not mixed signals. The old list was a bit misleading to authors as it implied that the score may represent the readers' judgement on how well the story is written. Words like 'Needs Work', imply that the reader noticed the errors in the story and commenting on them.
This list is not final. I'm open to suggestions of a better wording that improves the distinction in your minds about the meaning of the score you're casting.
The second change is the more drastic one. I'm replacing the current scores with weighed scores.
The new scores shown on the site will reflect the story's score relative to the median of all scores on the site. This will have the effect of lowering all scores. I've implemented the formulas that calculate the weighed scores and here is a sample of scores and their new values:
Old Score -> New Score
(average) -> (weighed)
10 -> 10
9.85 -> 9.56
9.5 -> 8.55
9 -> 7.10
8.62 -> 6
7 -> 4.93
6 -> 4.28
5 -> 3.62
4 -> 2.96
3 -> 2.31
1 -> 1
One thing to remember, the weighed score is relative to the current median. So a story's score may change even if it received no new votes. If the median changes, then the story's score will change.
Hopefully, the wording change will make the votes that readers cast more reasonable, so that automatic 10s change to something more meaningful.
One problem I don't know how to address is the fact that the more recent the story is on the site, the higher the average score is; this is related to the psychological effect of higher and higher scores. So if you have a reasonable solution, I'm all ears.
I know that scoring is a controversial subject but let's all try to be as objective and reasonable as possible. I'm trying to make the system work for everybody the best possible way. I appreciate everybody's contributions.
And before you fire off your reply, one thing I will not do, I won't ever scrap the voting system. So don't even suggest it.
Comments
As a reader, I will glance at about 50% of the posts of new stories, that's usually because the story codes turn me off. If they do, I skip the tale. Other than that, you're almost guaranteed that I'll read at least a few paragraphs and make up my mind if I'll read any further. Votes and scores mean nothing to me because I've often read a few paragraphs and moved on before any votes appear.
Oh yeah, for your first place, if you plan on using some term instead of a number, why not use a term like "Deserves Publication"?
I'm not sure that this is really a problem though. Because I don't have the time to read every story on the site, I really only consider reading stories with a 9.0 rating or higher (unless something in the description really appeals to me).
Changing the system as you describe would just mean that my "threshold" becomes lower. It's not going to really change anything about how I read (as long as older stories get re-ranked also).
As far as voting goes, I find the 10 point scale to be a bit of an overkill. I think that I'd prefer more of a 1 to 5 stars sort of approach. Since it seems that the real problem is that so few readers actually enter a score, (most stories only get about a 4% or 5% voting response) anything you can do to encourage people to vote would be a good thing.
Furthermore, I don't know if this is possible, but what about taking into account the person doing the scoring? For example, if a story gets a 9 from someone who usually gives out 10s, that should be worth less than a 9 from someone who usually gives out 7s. This is especially true for those few trolls that like to give out 1s to everybody.
unfair for the very reason you stated in your article. DROP SCORING and let the readers decide if they want to read a story by its description. I have not voted on a number of stories not because they were bad, but because they failed to capture me. All scoring is subjective
I do have one recommendation, that each reader automatically registers as a five, unless and until they vote differently. I think that would encourage people to vote, but it could be a mite of a database challenge.
I also like the idea of saying that the top score possible read "Deserves Publication"
That sounds 'cool' to me. ;-)
The simplest answer is to just disallow voting until the story is completed, but I doubt you'd like that and honestly I don't either. An alternative is a system that adjusts total score downward depending on how many votes the story has accrued. Alternately, if a single membership logs multiple votes on the same story, perhaps some or all of THOSE could be adjusted downwards. This is more fair but would probably be much harder to program (I failed my C++ class, so all I know is, this shit's hard).
The median score on a slow, usually longer story is 9.17. And that includes stories before inflated scores.
As a multiple chapter story goes on, later votes are suspect. Why did someone vote after chapter eleven of a continuing story.
I think scores are often more of an encouragement to a writer to continue, rather than an honest evaluation. Low scores don't encourage a writer to continue, since it's about the only reinforcement and feedback they get.
A writer of a 9.5 story still might get only two or three comments on a new chapter of a continuing story. And readers that voted prior, have burned their vote to encourage.
I understand the balance between information and encouragement for the author makes having a scoring difficult.
And I almost never read a story that I don't download the chapter. Downloading doesn't encourage voting.
With one vote per story, scores are difficult to understand as multipart stories go on.
Nothing is perfect.
rougher63
But, everyone is in the same boat so, 'whatever'. :-)
I like the new wording. That aside, personally I don't think the scoring system is 'broke', so I can't see the point in 'fixing it'. The only thing I compare scores with are my own stories. And they 'reflect the scores people gave them'. The only thing that peeves me is the 'bomber' who scores '1' and distorts the score. I think scores outside a standard deviation (or two) from the mean 'for that story' should be auto-rejected, but what do I know.
Good luck!
SN
One thing I'd like to see included in some way is the technical side of writing - - things like spelling, wrong words, and basic good grammar. I have completely given up on some stories that may have had a good story line but were so full of the errors I mentioned that they were simply too difficult to read. In some of these cases I've sent messages to the author suggesting a fresh edit or other corrections. Very hesitantly, as I know it would complicate matters tremendously, could I suggest a two track scoring system? One for theme, characters, etc. and one for grammar, spelling, and other technical issues. This could solve, in part, the situation mentioned by another poster about not scoring stories that don't "grab" you.
I also agree that stories with many chapters shouldn't be scored before at least some reasonable percentage of the story has been published. It has been my practice not to score a story until I've finished it. That may be a disservice to several authors who produce extremely long stories but I see no other way to be fair in scoring. Some stories that started very slowly and barely had enough "hook" to keep me around turned out to be excellent. Had I scored them based on the first chapter or so they would have received a score that didn't reflect the quality of the story.
Seems like I'm echoing some others here. :)
In the end, you learn to know what the median is, and know that something above it is likely worth reading. This will just change that number from 8.5+ to something lower. It will eventually level out, but it's going to knock down the downloads for a while until people get used to the changes.
-Emjay
Dwight
I have read many stories where the writing is an abomination but the plot and the characters are interestiong. Of course, there are also many stories in which the writing flows as easily as water making its way downhill but the story content is so warped that the reader is in a quandry as to how to vote. Dual scoring would solve that problem for me.
A 1..10 (or 1..5) system seems like it should be a good guide, but in this case it's hard to do because people don't vote objectively. For the various reasons you described, the numbers are currently... unreliable.
Also, 'quality' is difficult to measure here. Consider the various tastes of readers, and it becomes very difficult to have a single number indicate how good -- i.e. 'how much I will enjoy' -- a story is. It can be the best-written snuff story in the world, and it'll creep me the hell out . Yes, story codes help guide here ('hmm, includes snuff. I don't think I'll read this after all'), but if I can readily ignore people who vote high for snuff stories I'll get 'better results' from the voting system.
What I'd like to see is an affinity system. Show me what people who like the same stories I do like. This can be done several ways, but may require that votes be logged by userid (something that should probably be done anyway, to prevent fraudulent voting).
1. Amazon style. "people who liked this story also liked ...". Well-solved problem.
2. Profile comparison. "People who scored stories similarly to you also scored these ones in this way". Not as well-solved, but doable.
3. Explicit recommendation, strictly opt-in. "The following readers liked...", then let the reader see how these readers rated other stories. Invasive compared to other solutions; even strictly opt-in it feels nasty.
You could even expand things a little, though this may scare the tinfoil hat crowd. Many people only read stories that look like they will interesting. Log (user, story, completed). Then you can show "people who read this story also read..." and rate them accordingly. 'completed', above, is a flag indicating that the reader read to the end of the story. If someone reads all 118 chapters of _Jack and Jill_ by Old Fart, or all 167 chapters of _Dance of a Lifetime_ by Frank Downey, it indicates they actually found the stories interesting. Even if they don't log a rating, this can still be taken as a 'vote' of sorts. Reading four chapters of either of these, then quitting, is another type of vote.
Anyway, it all depends on the data available. If voting is currently just a big bucket of 'number of votes' and 'cumulative total' the above suggestions won't be terribly workable. If data gets logged in sufficient detail, they're doable.
(To further complicate things, if we allow users to weight various story codes and authors then it can produce a weighted score. This could be something of an explicit method of doing the above.)
CWatson: fwiw, I *taught* my C++ course :) Yep, programming geek.
billyi47
Furthermore, BOTH ideas allow "score bombing", when a troll and/or malicious fellow author gets tons of people to open the story and then close it again, thus sendings its ratings plummeting.
It's a mess.
The suggestion that all unvoting readers register as a 5 would have a rather interesting impact on stories. Combined with weighted rankings, it would tend bring the story's score further away from 5 (i.e. either up or down, depending on their average score) if they had more votes. This has the interesting impact of causing the "nice" people who only vote on stories they like to give implicit input to stories they don't. It also causes people like me who occasionally spend 30 seconds reading a story before deciding not to read it to give unfair votes. I think this is a bad idea.
The suggestion of allowing explicit recommendation could be done without the nastiness of opt-in/opt-out by basing it purely on the favorites lists and reviews. This would tend to miss a lot of stuff, though.
I think having a dual-scoring system is a bad idea. People are too lazy to vote already. Multi-scoring is fine for reviewers. Keep in mind that voting requires a mental effort by the voter to make a decision on exactly how much they liked the story (which is easier if the story is really great or really bad, hence the "nice" people who only give 10s).
On a related subject, I seems to me that long stories have a higher median grade than shorter stories. Any comment on that observation?
Lazeez, thank you for your efforts.
1 - allow voting for each chapter and provide the results with the statics you are already providing. This way if a story is improving or headed down hill, you can have a better idea of what is happening. There might also be a way to show a summary to the reader so he will know if a story is improving or declining. (Rising like a rocket or falling like a stone).
2 - Require anyone who casts a vote more than 2 points off the current score for the story to provide a reason. It should be anonymous, but if no reason is given, or if the reason does not seem reasonable to Lazeez (or his designated judge) the vote does not count. (PS-lack of sex should not be a valid reason)
3 - On multi-part stories, require at least 3 chapters be posted before voting is allowed (unless the story has fewer chapters).
4 - Set up separate scores for Content, Character Development and Grammar.
5 - Definitely set up a code or category for 'needs editing'.
Now, in the middle of this, I have to say that I'm one of those who probably helps skew the curve. I seldom vote unless I can give the story a 10. My reason for doing this is to not allow myself to be tempted to bring down another's score for personal or selfish reasons. When I do vote, I will tell the author that I did and why I thought enough of his/her story to deserve a 10.
For those that deserve less, I usually just tell the author how the story could be improved, but I don't vote. Only once did I think a story was so bad that I gave it an honest vote of a 2.
billyis6yhb
As far as telling the author one`s thoughts, any response that is not fanatically ranting about the world shaking brilliance of the endeavor, is often met with insult and derision. I rarely comment now as even the the most benign observation is not worth the result. Why not state whether the author actually wants a response that is the least bit critical? I have no talent for writing, however, I am capable of determining if a piece is readable.
If I think a story is badly written I will vote low.
To me, this type of "scoring" is much fairer than any type of numerical system. However, I do agree with other comments that the "mechanics" of the writing (grammer, proper use of apostrophes, commas, word choice, etc.) should have its own separate "scoring" area. While I am learning to (mostly) ignore many of the more common errors, they still get in the way of my enjoyment of the story.
Having said that, I wonder how valid the scores would be if a certain of top scores and bottom scores are thrown out as being prejudiced. I guess it works in athletic competitions.
Another useful stat for determining quality is the number of downloads in relation to the number of chapters. Lots of downloads and few chapters means a lot of people read it. The other way around of course means the opposite.
A really nifty feature I'd like would be to automate that calculation. A simple near-static solution that'd be really cool would be to calculate an average number of downloads for previous chapters when a new one is posted, and list the change from the previous chapter's total. That'd give, at a glance, an indication of whether a story is gaining readers, or losing them. (This of course presupposes that the download total always counts individual chapters, rather than counting each "session" once regardless of how many chapters were read...)
At any rate, fixing the scores isn't a matter of just weighting the existing ones, or even putting permanent weighting on them in place. To really fix the scores, you'd need to weight the scores of each individual user - and alert them to the fact that unless they spread out their scores, all those tens they're giving out count as fives since they're the median score awarded...
2. I would suggest a system by which an author can be protected from a few low votes that pull a good score down. It might be from readers who resent the subject or theme of the work, promoting their favorite authors by hurting others with good scores, or other reasons.
Perhaps a practice of dropping the lowest votes before calculating the average, or eliminating votes that below a stastical parameter, say 3 std. dev. below the mean.
3. I would also like to see, in Author Stats, the distribution of votes by story. That is, how many 10's, 9's etc.
Thanks.
It seems to me that most of the problems with scoring are the trolls that love to tear any story down and our friendly folks that give a 10 to everything.
Why not just discard a percentage of both those that vote high and those that vote low. Then you have a more meaningful value to work with. This I think is a viable solution that would be easy to implement.
Secondly, I'd like to be able to change my vote at a later date...as many times as I deem it necessary. I did a very popular author a disservice by ranking a story low after the first chapter..mainly because I didn't think he was going about the tail in a good manner...boy was I wrong. I learned two things. Give the author a chance (I don't rate serials until at least the third chapter is out..sometimes more) and you can't change your score.
If I feel like that, I won't vote as it is not fair. I am not being objective enough.
I do use both content identifiers and scores to determine whether a story is worth reading and find it frustrating when a story that is barely literate still gets a high score, particularly when you have such excellent writers who may get similar scores. The system needs revision, but regrettably I have no realistic suggestions.
Jon
I like the idea of waiting till the story is completed until voting, but that doesn't seem reasonable. How about not allowing voting for multi-chapter stories until either 50 KB are posted, or the story is concluded (if shorter)?
I understand how readers give high scores to encourage writers, don't know what can be done about that. I also, tend to not vote for stories I find unacceptable, and probably also give higher scores than merited to encourage writers.
More use should be made of emailing authors to make suggestions, etc.
Another possibility that you did not mention is to have more than one score like in the reviews. I would find that very helpfull
That said, why not set anyone who reads a story a default score of 5? If 10% of the readers vote, scores would oscillate between 4 and 6, which wouldn't matter because the score is weighted anyways...
I am not a programmer, so I don't know if what I am going to suggest is possible.
The voting as it stands is an overall score. What if you added the Plot, Stroke and Technical Quality (like in the reviews) scoring options as a seperate box so those so inclined could use it. If the point of scoring is to help the Author then this would be a great help. If it is only to help the potential reader then it is useful but not as important.
I have read many of the Blogs and forums connected to many of the popular (and not so popular) authors and a common theme is the universal desire for feedback. So if a reader is predisposed to send a comment via email they will probably spend the extra time to vote on a few extra categories. Some who won't take the time to write might still take the time to click a few times for the extra categories. Even if they don't it is really nothing lost anyway.
I also think that despite the unfairness of short stories recieving less votes than a multi-chapter story, that you should be able to vote on each chapter. This would help the author understand the trends for the story chapter by chapter.
Since there is no way to create a perfect system, I believe that helping the author recieve more feedback is the priority. If there are clever mathematical ways to wheight or otherwise help adjust scores between one chapter and multiple chapters, I don't know, but I think not allowing a vote until later in the story or casting only one vote for multiple chapters is not really helpful to the author.
Since there is no way (I can see) of taking away the "human element" of bias then maybe giving more options for voting will help offset undesireable voting practices.
Good luck and thank you for your time.
I really enjoy the way that you handle your business and the care that you take to provide a great experience for both the readers and the authors. Your quest to improve the voting is but one example of this care.
Keith had some great comments about the issue of voting that I find very compelling and I agree with many of his comments. (dot)B also raised some good points.
Personally, I think that the old system of numbers and their description was better than the current plan to just use comments, especially the comments that are proposed. I don’t think that descriptors like “Good Story”, “Not Bad”, “Some Good, Some Bad”, “Not Good”, and “Pretty Bad” tell either the author or other readers anything of substance. I don’t remember the comments with the old voting system, so I may have just shot myself in the foot. As one other person commented, what does “Amazing; Impossible to Improve” really mean? Yes, the numbers are inflated, but at least they are an indicator to fellow readers, just as the number of downloads is an indicator (does that really mean downloads as in xxx number of people pulled down the zip file or does it count the number of people who have/are reading the story?)
I think that the best voting system would be one that is simple and tells people how much one would like the author to keep on developing the characters and plot. For example,
1) I will not read more;
2) I will continue to read;
3) Story captured my imagination.
I've never liked scoring systems on any site, because you simply can't make them fair. You can't KNOW who honestly didn't like the story and who is just being a jerk.
Scores don't really tell you anything, either. I've had someone e-mail me recently and complain that they didn't like my story but they kept reading it because of the high scoring. Like it's MY fault they let that influence them. Readers should judge on summary and actual content, not numbers. I certainly can't help the scores someone gives me.
And NO WAY should "timeliness of updates" be a factor that can be voted on. Penalizing someone for writers' block is rude and just plain wrong. The people who write here do it for free, and they have lives. They shouldn't be punished if that gets in the way of updating in what someone else thinks is a timely manner.
The same goes for 0, 1, and 2.
If a story is that bad it has no place on this site.
b.
Can you trap the "trolls that like to give out 1s to everybody". I'd just toss those votes, myself.
I admit to giving very high votes on average. But usually because if finished the story and really liked it. So your proposal dilutes those votes.
One problem not addressed by either the numbers or equivalent phrases is the lack of content. I have read in several places authors asking for more explicit feedback - What did you like/dislike. More work, but a system that lets a reader flag attributes they like and those they don't? I very often write a note to the author about what I like most. Clicking a few radio buttons in a form could really provide more information.
Hal
1) Each user can only cast one vote per story, and if the user wants to change their vote later on, they can.
2) A user must provide comments when casting a vote. This splits the divide between usefulness to authors and usefulness to other readers.
3) Comments and votes are made public. This makes voters accountable, gives authors feedback in a way pure numbers cannot, and also, for readers, allows us to get a sense of what the votes of others mean.
As a reader, I honestly have no problems with the current scoring system. Yes, votes are "inflated," but there are many good reasons why this is the case -- why the distribution of votes would not be normal -- and all it means is that I get to decide for myself what the threshold number is for a story to capture my interest.
I do like the suggested descriptions for the votes, with two caveats. Anything can be improved, so for 10 I would suggest something like "Amazing, Absolute Best of Its Category"; similarly 5 is a little vague, and I would suggest "Some Good, but Equal Amount Bad."
One thing I would change is "Not Bad" to something like "Needs improvement/Editor" and I like .b's idea of automatic 5's until a different score is posted, but keeping track of several million readers votes could be rather cumbersome.
Another idea would be for a user to be forced to vote to read more than ... say ... 50%, or 5 chapters, of a multi chapter story. (Or an automatic vote of 7 ... after all, if they DIDN'T like it they wouldn't continue reading.)
If you DO look at automatic votes, how about rewarding authors who put inordinate amounts of time in their stories, like cmsix, Morgan, or John Smith (whose story 'Robin" is nearly 7MB and has been an ongoing serial for 2 1/2 years) by assigning an automatic 9 or 10 to a reader who reads more than 20 chapters.
Another option is Asa Strong's idea to DISCARD a percentage (10% sounds good) of the highest and lowest scores before averaging the rest. This would severely discourage the trolls who only give 1's or 10's and force people to be fairer in their voting. As an addition to this, if you discarded 25% of the highest and 10% of the lowest scores for all stories then change all remaining scores to the new average, you wouldn't need a NEW scoring system.
I do like the idea of splitting scores, a la reviews, for plot, stroke, technical, and overall.
I suspect that people who don't like a long story don't bother skipping to the end of it, just to vote. So the only votes are those readers who enjoyed it enough to read to the end.
I'm not overly thrilled by these changes. It's not that hard to tell which stories are likely to be good.
Thanks,
Ersatz
If I get bored with a story, I quit reading it. Usually, I allow myself to be bored for a couple of chapters on a long one before dropping it in case it gets better.
I tend to write multi-chapter stories. I also tend to start out slowly and build once I get into it. Quite frankly, it bothers me when a bunch of people vote on the first chapter of a story of mine. They are offering an opinion about something they know almost nothing about. (Insert political comment here.)
Would a story of mine survive if nobody voted for it for the first 20 chapters? I doubt it.
I know that the fact that this blog entry has been posted means that a change is going to be made. I'll go through another learning curve and figure out that anything above 7.3 or 8.7 is a potentially readable story and continue bookmarking. I might as well bite the bullet and see if scoring can be improved. With this in mind, I have a couple of suggestions for long stories. If it's a one chapter deal, people should be able to vote once and only once.
First, set up your description of each voting level however you wish. Most awesome, bestest ever, I don't care. It should reflect an accurate description of what you're attempting to rate.
How about a dual score? Allow everyone who reads a chapter to be allowed to vote on that chapter. There will be an overall score which reflects all the votes. There will also be a current score that reflects only votes made on that latest chapter. The writeup in the scoring section should inform the voter that even though he's voting after reading that particular chapter that the vote should be for the whole story so far. An overall 8.3 and a current 9.4 would tell me it started out slow (or unpopular) and got better. Of course, only people who had read that far would vote for the current chapter. I never said it was perfect.
Another possibility is to use cookies to allow a reader to update his score. I don't know if this is possible – I just know what I want the mighty computer to do. Reader A reads the first chapter, gives it an 8. A few weeks later, he reads chapter 2, thinks the story's 9 material, he votes 9, the 8 he voted earlier is upgraded in the score. Each person only gets one vote, he's just able to modify it as he reads more of the story.
A word about every author's favorite – the troll. We all get them, we all complain about them. All I can offer is a comparison. I'm working on my fourth page of listings as Old Fart, have 4 stories under another name and co-author another 4. That's 40 stories, most of them completed. I get emails every day telling me what a good writer I am. When I look at a person whose only creativity consists of voting a 1 and dropping somebody's score, I feel sorry for the guy. I wonder how many emails he gets praising him for his contributions to the site.
The OLD FART
The new system just replaces one simplistic point system with a different one. A rating based on words rather than numbers is better, but only goes part way.
I see two things that effect scores.
1. As stated earlier new stories enjoy a honeymoon period with high initial scores.
2. The other is that many stories with high pruriant interest average higher scores. This is often without regard to quality or quantity.
I have seen several very mean emails from anonymous readers about stories that had no sex in them, even when the stories stated there was no sex in them
Quality should not be about the stroke value. It should be about imaginative writing, the proper use of English, the editing and yes about how long a story is. Writing and editing is hard work. Ratings should reward good work.
How can you compare a full feature Cmsix or Al Steiner novel to a 5k story of any ilk? I don't think you can.
I rarely read any stories of less than 50K in size. I can't stand to just get into a story and then have it end.
A decent rating system needs to take into account many factors.
Quality of Writing
Imagination
Size of Story
Character Development
Editing
This would better reflect the authors efforts.
Score compression and scores are one in the same problem from my POV.
Very seldom does a story rate a 10. But I often give 10's because that's the only way I can really send a signal that the story was good.
Yet every so often I rate a story a 1 because so many people have rated it high - a 9.5 or something and it's really (IMO) an 8 and the only way I have to reduce the score is to be one of the first couple dozen to rate a story and rate it down.
Often, stories that are otherwise pretty good have some grammatical oopses - you know the kind - where you're reading along and instead of using the contraction for you are the author uses your instead. Coarse for course, and a half dozen more of the same sorts of oopses.
I have nothing to give but a 10 or a nine. So if the story is well constructed it gets a 10 - if the grammar oopses and story construction (logical progression) isn't the best it gets a 9.
But maybe it really deserves a 9.4. Pretty good story, but not outstanding.
So the suggestion is to institute 10ths into the scoring system and see if the range changes.
Another writer suggested a dual scoring system. That might work if one score was for the overall story likability - plot, characters and setting and one was for construction that includes grammar, spelling and logic.
I'd still suggest scoring based on 10ths though.
Finally, if the story deserves a 10 it should stay a 10. If the story is 'only' a 9 it should not be compressed to a 7.4 (or whatever)
Bottom line? I use scores to see what I want to read. As another poster said - I seldom - and I mean really seldom - even glance at a story that is rated less 9. If the description sounds good I might give an 8.8 or above a glance. But if the author has spelling or grammar issues - I'm outa-there like a shot and I may give the abomination a 1 to boot.
To authors - want a good score (regardless of the system used), 1) run the damn spell checker with the grammar checker turned on. 2) Ask for help from an editor - not a proof reader alone - to help with logic continuity, words that are spelled correctly but misused and homonyms that don't belong.
Finally, another poster suggested that raters would have to post a comment. Nah. I won't even rate a story unless it's outstanding. Not worth my time to compose something for a story that has problems.
Any "discarding" of votes will result in me not voting.
A story with 175 votes at 9.6 does not equal a story with 175000 votes at 9.6.
About weighted scores; My thought is that I have learned through experience what the votes represent under the current system. All the changes will do, in my mind, is make me learn all over again what the numbers really represent. It is all relative.
So will the rating system changes end up being beneficial? Only time will tell. Call it the great experiment.
As to the Descriptions; If you are just asking people to vote on wether they like the "story" as in plot then this is an improvement over the old way.
I like changing the descriptions for 1 thru 10. I agree with those disliking "Amazing; Impossible to Improve" for a 10. I propose dropping "impossible to improve" and just use "Amazing" or some other superlative. Maybe: "One of the best I've ever read"
Good luck!
One of the problems that I have with the current system, is that it is easy for people to tank a story, by giving it a 1, just to lower the score. I have seen this happen time and time again, where a perfectly good story gets a bad score, because a reader or group of readers give it a low score just to sway the score.
Another problem is that people may score a story low based on the first chapter, and to an epic novel, that's not really fair. There is also no way to change your score if your opinion of the story changes. One way to fix this problem, is to only let a story get voted on if it's completed.
Personally, I never vote on a story unless it's completed. I have read a lot of stories that start OK, but for some reason, go in the toilet long before the end.
Another problem with scoring, is the second way stories are rated, and that's by the number of "downloads" it gets. The thing that I have noticed, is that the more chapters a story has, the more downloads it can get if the story is uploaded or read 1 chapter at a time. Each chapter's downloads get added to the total. However, if a story gets read all in one sitting, or is uploaded complete, it only gets 1 download for the whole thing.
Regarding the "timeliness of updates" thing that was mentioned, that's a bad idea for many reasons. That said, however, I won't ever post another story until it's done, and at least in the editing stage, if not ready to post as complete.
Anyway, that's my 2 pennies worth.
I like the idea of being able to give and get votes on a story I've written or read.
I agree with others here who say that voting on a multi-chapter story should be done ONLY when the story is complete and I think only those who read the entire story should vote on it. I write this as someone who has written both long and short stories.
I also agree that a Two-part scoring system sounds attractive to me. One score for the story (plot, character, etc.) and one score for the technicals (Typos, Spelling, punctuation, etc.) My longest story is my best rated story. Sadly, it also is my worst edited story. For this I apologize but it's true.
One other thing I agree with that others have mentioned. Is it possible to track who is voting for what? Maybe have the vote of someone registered on the site count more or differently than someone who is logging a vote anonymously. Combined with allowing only a single vote per story, no matter how long it is and forcing readers to vote at the END of the story, this could stop people voting mulitple times and would stop the 'drag' caused by those who always give stories a ten and those who always give a story a one. Just an idea.
I also like the idea of the dual (or more) voting system. Voting on the technical merits (or lack of) some stories has long been my dream, and I've gotten lazy over the years and usually don't find the effort worth it to write a comment to an author to tell them that a spellchecker isn't an optional piece of software for them. Similarly, once divorced of excruciatingly bad plots or plot implementations some people probably deserve the credit of being able to just sit down and knock out an effortless looking piece that doesn't do anything for you.
Someone mentioned the Amazon ranking/recommendation system. This would be really exciting, but I'm unclear on how much trouble it would be or how much value it might actually have. The trouble, as I see it, is that the story codes probably already fill that role to some extent already. If you're an avid reader of monkey scat porn then you're probably already filtering out everything but your hobby based on the codes, and/or you're going to merely end up making recommendations along the lines of the story codes + scoring. That might add something, but I don't know it would add enough to make it worthwhile for Lazeez to implement it.
Finally, I think the people who suggest alternatives to "impossible to improve" are spot on. I comfort myself by knowing I'm a critical sort of person, and I simply don't know if I've ever read anything that I couldn't imagine something to be improved upon somewhere. The single vote per story per account (changeable)issue also sounds like a good idea.
As an author I’ve never cared or bothered with reviews, critics, reader scores or reader comments. They are not why I write.
My suggestion would be to dump the whole voting system.
But this is your site and you and others seem to want it, so have at it. For me, I’ll go on using the story description written in the author’s own words to determine if I’m interested in reading a story or not.
has its problems and so "Outstanding" maybe better...
A problem I find is where the story is released as a work in progress the author asks for comments on the work as is...
these work in progress scores ...
should be tracked separately to the Final/finished work..
maybe you should consider as some others have suggested taking the
individual's scoring into account...
possibly including not actually
using a score until the individual has registered vote for 5/10 stories...
to address the area of Story vs Style/technique/proof reading/spelling etc
why not have a similar set of scale boxes for these... they should just be reported to the author on a vote basis.. and not
averaged..
excellent site
thank you
That would be me, I think.
This would be really exciting, but I'm unclear on how much trouble it would be or how much value it might actually have.
"How much trouble" depends on what data's currently tracked. If scoring is "total accumulated" and "total votes" (I suspect this is how it's currently done) then the extant scoring information wouldn't be useful for this. However, a new scoring mechanism that allows Amazon-like behavior (and incidentally changing votes over time) is pretty simple. Space-intensive, but straightforward.
The trouble, as I see it, is that the story codes probably already fill that role to some extent already. If you're an avid reader of monkey scat porn then you're probably already filtering out everything but your hobby based on the codes, and/or you're going to merely end up making recommendations along the lines of the story codes + scoring.
Anyone interested in monkey scat porn is has a fairly narrow set of codes to look for. Other codes (rom, slow, MF) are almost useless for searching. Heh, I found a site that had stories marked 'piv' and it confused the hell out of me, until I discovered it meant 'penis in vagina'. You'd think that would be a meaningless code.
*ahem*
Anyway, codes can be useful for coarse filtering, but it doesn't handle subjective evaluation. Most codes are binary and give you an idea of content, but not *quality*. The current numeric system doesn't really do a good job of indicating quality because of value inflation and compression.
That might add something, but I don't know it would add enough to make it worthwhile for Lazeez to implement it.
I think it would add quite a bit. As I said, the current numeric system isn't a lot of use, I think. A affinity-based system could be quite valuable.
I don't much *care* what Mr. Monkey Scat Pornlover likes. If I can find out what people who like a story by Nick Scipio, Frank Downey, or Al Steiner like, well *that* is something that will lead me to other stories I can expect to enjoy.
1. Create a voting system like SoL’s email responses. Once a week the member when he/she signs on will get a page that lists ALL non-voted stories read or downloaded by that member in the past week or since last sign on which ever is longer. On this ‘sign on’ page show story name, description/codes and dropdown score bar. If the person signing on has elected to use the category voting system, display the categories instead of a single score option. Have a vote button at the bottom of the page that inputs the score(s) and sends the member to the page originally requested/bookmarked.
2. The problem with automatic scoring of 5 is, we only get ONE vote per story. I have read some stories that were wonderful, but the ending absolutely sucked. Three Pieces of Silver is one example. Give us one during the story and one to replace that score after it is (final). That would encourage me to vote on a story in progress knowing that my vote would not be wasted on a story with a lousy ending. OR, allow each member to vote as each chapter comes out, and a final score to replace the chapter scores.
3. Create a weighted system that scores grammar/spelling, story plot, stroke and how well you like the story (“Its 3am and I cant put this story down!”). Allow the votes collected in this system to count as two or three votes in the overall score.
4. OR, keep the simplified voting and create # 2. only allowing authors, authors/editors, authors/editors/select-members(i.e. ME! lol) to vote in the categorized.
Now as for the rescoring…
If a story stayed on the All Time Long/Short Classics lists for more than x weeks then give it a higher weighted score to start off. Depending on the number of weeks it stayed and where it stayed in the list.
Display each stories current score until it has at least x number of votes under the new system. Then show an asterisk or remove the asterisk next to the score to indicate it is a new score. After a few months, just show the new score.
JR|away (Fawks)
1. It's simple enough to avoid scaring off potential voters.
2. It nullifies the "vote nicely" tendency since you don't have a single figure to compare to.
3. You could have a seperate catergory ("did not vote") to capture the lazy ones as well.
4. If you've got a record of previous votes, old stories can be easily converted.
Personally, I don't think weighting the votes will work. People will continue to vote biased, but they'll take their mark off the new, weighted score.
I think the actual algorithm used was a Bayesian rating scheme where the rating was = ( (avg_num_votes * avg_rating) + (this_num_votes * this_rating) ) / (avg_num_votes + this_num_votes).
This means that only once you have a typical number of votes can you see that the stories score will be halfway between the voters score and the average score. This also helps to not have the inflation you see.
How about letting people score like they do now, but let the author decide when they can score (they can always vote, if the story is marked finished).
But instead of the raw score, show the percentile. So instead of showing 9.35 you show, that this story is better than or equal to 75% of all other stories.
I also like the idea of multiple scoring parameters: idea, execution, technical style.
Also, as an author and statistics nut, I would really like to see the flow of votes, so a time (or voting-number) depending graph would be cool. The reader could also use that for gauging if the story has lost its focus (a lot of the really long serials could be a lot shorter imho, as they are just recycling the idea over and over again).
The other thing that might be nice, and one person wrote something close to this, is to be able to vote just on a chapter or section and a vote for the story over all.
That or being able to change your vote as one chapter can really effect an entire story and if you've already voted you can't change it.
I think that would make things a little more acurate.
I read a lot of stories which have a well developed plot and a great story but drive me to distraction with spelling, grammar, and other "proofreading" errors. I hate to downgrade the story score because of the mechanics but they would really be much more enjoyable if someone would proofread them.
Your new descriptive ratings are an improvement. However, no story is impossible to improve upon. I'll never give a story that rating.
Maybe there could be another list with multiple boxes where the voter could check "other" items such as poor grammar, poor spelling, excessive time between chapters, etc. etc., allow multiple items to be checked, and just report the number of voters who checked each item.
It would also be nice to close the hole which lets a person vote multiple times for/against the same story.
I've seen a couple of good idea's mentioned here. But unfortunately I haven't got the time available to read them all.
It would probably be too difficult to put into operation, but I think the idea of two separate votes on every story is a good one. One vote for the stories technical interpretation (that would keep the grammar boys happy and might encourage some more writers to use editors) and one vote on the stories actual content itself. I'm not sure how you would then rate the stories, maybe an average of both.
There are authors on this site who are extremely good technical writers, but the content of some of their stories isn't to everyone's liking. There are authors whose technical grasp of English is wanting (and I don't exclude myself from this group). But who's stories to some readers are compelling to read.
If every story that is opened, also had an automatic middle of the road vote on them. It might encourage people to go down and lower the vote, if the story didn't hold their interest or the presentation was so bad that they felt they could not continue reading. If they did get to the end of the tale, raise or lower the votes accordingly. It's just my idea to encourage those who don’t bother to vote to do so.
Also unusually high or low votes that are entered by fans or trolls should be either automatically filtered and brought into line with the general voting on the story or should be discarded completely. I would suggest that the current voting position is displayed at the end of the story as well. It might encourage readers who disagree with the current score to vote either way.
The Wanderer
I will freely admit to not voting for stories that are either poorly written or just don't hold my interest. So, just too make things a little bit more difficult, weighting the voting might even the system.
I choose stories on some basic criteria. i) stories codes ii) decription/blurb iii) length (the longer the better) iv) age of story (earlier stories with a rating of 7 plus are usually worth 10 or 11 compared to some of the latest postings. The greatest problem with any rating system is that people will use it for personal attacks on authours. Anyone brave enough to post their work should be encouraged not flamed because of it. However anyone that can come up with a fair and balanced system should patent it, as it will be a first.
I agree with most of the posts that I don't normally read anything less than an 9. Until I run out, then I troll for good descriptions or previously read authors. Yet, even then I normally won't read something that has a 7.5 with a good number of voters.
I would like to make a suggestion.
I think voting should be tied into commenting. You should not be able to vote, unless you have made and sent a comment to the author. This would be a pain for the reader, but it would certainly be a deterrent to the "easy clickers". It would be like saying, "To mark a vote, you need to care enough to send a comment to the author."
;-) just my 3.4cents (inflation ya know)
~Jay
I looked over the various descriptions and voted accordingly. I think the votes were mostly 6's and 7's. These are on stories I enjoyed and read all the way through. The screen would come up telling that the score was now [9.21 – 9.51] on the ones I did.
With the old wording, I would do exactly the same thing. I would read the descriptions and vote accordingly. I recall a few 8's in the past, but most of my voting has been 9 or 10.
If I spend 30 hours a week writing “Jack and Jill – The Second Book” and end up with a score of 7 or 8 simply because we're making it “fair” (see graduated income tax, minimum wage, big brother knows better than you how to run your life), I doubt I'll continue with it.
There's just something about seeing a 9.27, inflated though it may be, that makes me want to continue to write. I call it appreciation.
The OLD FART
I have been keeping track of the scores recieved by my latest story and approximately 10% of the readers put it in their libraries.
Only about 4% have voted so far about what I would expect since its a serial.
Knight Ranger
Even the number of downloads isn't really as indicative of the popularity of a story as authors think. I go into a story and read the first few paragraphs then decide I don't like it. I quit reading and back out before getting to the bottom of the page where the voting form is found. I counted as a download even though I didn't read enough of it to matter. A good teaser generates a lot of that sort of download, even if the story or writing is junk.
I judge the stories I pick to read by the scores given, on other readers recommendations, and on the story codes included with the teaser. I seldom vote on stories. Ones I do vote on get high scores. That's happened maybe 10 times in the approximately 2 years I've been reading on SOL but has not happened in probably the last year. I never vote low scores because I never make it to the form. I tend to prefer longer stories, and do not see how anyone can vote after just one or two chapters. You can judge the writing, but not the story in that case. Instead of voting, I send authors constructive criticism and or point out typos. If I took the time to do that, I like what I read, even if I thought there was room for improvement (there is always room for that).
Canyondick
How about adding something on the user preferences page for us readers that don't care to participate in voting so we can choose not to see scores and download numbers in the story list. If I really like a story I send a comment to the author in the box at the bottom of the page.
I've seen lots of ideas toward the changes. Many of them are good. An equal number of them are impractical. I don't envy you as you try to distill these ideas down.
I wonder if it would be appropriate for a very small comment box to be included with the voting. I don't want to discourage people from actually writing the authors and giving them long and glowing regards, or lengthy feedback. For example, when a person leaves feedback on eBay there is a small comment box. Sure, there will be a lot of "WOW what a great story". But there may be some useful comments like "Spelling mistakes distracting - liked story otherwise."
I also wonder if it would be practicable to allow longer stories to have two voting tiers. One set for voting along the way as the chapters are posted, and one set for those who get to the end of the story. I see a few benefits to this. Some people are just not going to like the story subject, and will want to vote before the story end. Sometimes a story starts out great, and I'll give it a decent score - but then the author loses inspiration and the story tanks in the end. I'd like to see both scores presented.
I'm sure whatever scoring system we end up with will have a lot of thought put into it.
Thank you for your hard work.
Emily
Require a vote before a reader is allowed to view the next story. If a reader abandons a story and logs off, require a vote at the next logon. Multiple choice. Codes, stinks, not worth my time etc.
There are some stories where I will not vote after reading the first chapter posted. These stories usually involve unfamiliar authors.
How about NOT making it retroactive.
and after a story makes some many votes, or the score stabilizes, freeze it, then age won't wither...
Have you considered allowing voting without commenting? There have been times when I've considered voting but really didn't have much to say about the story. Let's face it, "Great story," or whatever really doesn't tell the writer much. I appreciate both positive and critical comments on my stories, but when you have a couple of thousand people reading and only four or five voting, generally only those who really enjoyed or hated it are going to comment.
I'm not sure it's possible to make the voting 'fair' (non-inflated), unless you have a voting population who is motivated to vote in the first place without being pushed to do so, and then have a system where the votes were rationed into whatever histogram it's desired for the votes to be. For example, they're permitted to only give 5% of their votes as a 10, 10% as 9, 15% as 8, etc. But, you don't want a system where someone has an incentive to find another story to lowball just so they can give an extra 10 to a new story.
Perhaps add an 11 category for "Really Really liked it", and limit that choice to X times per week or Y times per month. That kind of feedback might be informative to readers and authors, since it provides an idea of how many readers spent a limited vote choice on a particular story.
Another kind of vote could be to allow readers to rate stories within each author - either to pick which of that author's stories they would recommend to other readers, or rate their top 3 in order. This vote could be changed at any time to replace a previous vote. It might be necessary to limit this vote to stories that have been finished for a set period of time, to avoid inflated votes given to whatever is the author's current story.
This feedback would be helpful to readers when looking at a new author's stories, as a recommendation of which story to start first.
It also might be feedback to authors to know which of their stories are liked the most. I suspect that quite a lot of authors would find that their highest scored story would not be the one that gets the most "i like this one the best" votes.
I'm not sure it's a good idea to statistically downgrade votes or change their weight, as that can discourage votes if they think their votes don't count equally with everyone else's. Or, it could cause people to inflate their vote intentionally so that it downgrade to what they feel it should be.
Everyone is already familiar with the current voting system, so it would take a while before readers would be comfortable that they know what a 7 or 8 or 9 means under the new system. It seems to me that readers evalute current voting more like a relative preference than a story being "excellent" or "very good", so perhaps keep the existing vote system, but also have an option to tell the reader that a story was in the 90th percentile of scores, top 5%, etc.
maroon
If you really want to change the scoring system and make it more meaningful for both the readers and the authors try this. For stories with more than twenty votes provide a distribution of the scores as a percentage of total votes. This works best if you reduce the scoring range from 1 though 10 to 1 through 5. Using this system the following is a sample score.
Average Score 4.25; 5’s 55 percent; 4’s 30 percent; 3’s 7 percent, 2’s 1 percent, and 1’s 7 percent.
This way the reader and the author can separate out the troll factor. The same system works with 1 though 10 but it would require a larger display space to show all categories.
An easy way to do public reviews might help the authors get more feedback also. Or maybe a bbs where authors can get dialog on their stories and chapters.
I agree that as it stands now, it's very hard to tell the difference between a 9.2 and a 9.7; but that's okay too because peoples' basis for their votes can be wildly different. Someone can love Avatar for the relationships and tolerate the BDSM. Others might love it for the BDSM and tolerate the religious aspects. The only way to help that is to be able to leave a comment or review with your score.
More of a problem than inflated scores is radical scores that change a rating. How about throwing out all scores that are more than 2 sd's above the average for that story after so many votes are cast. If a story is running at 7.0 after 20 votes, and somone casts a 5.0, throw it out. Obviously a disgruntled reader. Conversly, in the same example, someone casts a 10.0, throw it out. Obviously someone just inflating scores.
How about also allowing votes after each chapter, with the above proviso? Sometimes a story gets better or worse, but you have alreadt cast your vote.
I think if you reduce your scores, you are going to lose authors. Many of us are not going to put all the tiome and effort into this for a 7.0, that would have been a 9.0 before. What difference does it make, anyway, if the scroes are inflated in your opinion? People are not going to continue reading a bad story anyway. I have seen many more examples of a 7.0 that is a good story in my opinion, than a 9.5 that stunk.
Or, if you want to make voting meaningful, how about: to vote, you have to write a comment. The cokmments are more useful to us than the votes anyway.
Just some ideas and thoughts.
unknown1000u2
I do think that the scores are over inflated, and that's a problem. When I read a story, if I don't like the story, I don't always finish it. Sometimes I'll still vote on it though, often with a low score.
I give high scores to stories I like, but only if they don't have a bunch of grammar and spelling problems.
Personally, I'll read something even if the score is low, if I like the content.
Anyway, I think this move will cost a lot of the authors a lot of new readers, but, like I said, it's your site.
I agree with your post, go for it. One thing I wouldn't mind is a way to leave a short comment on stories similar to Amazon's reader's reviews. I know you have the review pages already, so, maybe a link from the story itself to the corresponding Reviews page(s)?
Regarding the comment on older stories getting shorted because of the upwards drift over time in median scoring, could this be taken into account so that a slightly revised formula be applied based on that year's median?
And maybe I'm wrong in this ... but I see my 9s eventually becoming 8s and my 8s becoming 7s and my 7s becoming 6s just to make/keep things "fair" ... Somehow that's not inspiring at all as a writer.
As for voting ...
Under the old system I rated stories 9s or 10s ... and sometimes 8s. With the wording of the new system, the stories I read will be getting 9s and 8s and 7s. With the way 10 reads now, I can't see myself giving it anymore ... unless there's something out there that really knocks me out of my chair.
No matter what you do, I still think you're going to have:
a) readers who will give 10s and 1s for whatever reasons
b) readers who give scores that are low, but won't write and tell the author why
c) readers who won't vote
The same old thing, but with different numbers that will simply be lower. And how inspiring are these lower numbers going to be to new authors? Not to mention us older authors who enjoy seeing a 9 even if it is inflated now and again.
I don't mind an 8 or 7 under the old system ... I knew what to expect when I posted the stories ... but under the new one an 8 somehow doesn't look as nice.
Seems to me you'd be just as well off to drop the 10 and don't let any votes go lower than 5.
Stormy Weather
I would like to make a suggestion.
I think voting should be tied into commenting. You should not be able to vote, unless you have made and sent a comment to the author. This would ... certainly be a deterrent to the "easy clickers". It would be like saying, "To mark a vote, you need to care enough to send a comment to the author."
I commend Lazeez for his constant attempts to make StoriesOnline better. And he is right, the current voter's seem to opt for a 10 if they like a story, and either a 9 or an 8 if they don't. Of course, there are a few faddists who muck things up with deliberately low scores for perfectly good stories too.
By now, I realize that my own writing has leveled off, and while one story may be better than the next, they're not that far apart. Therefore, I'd much prefer hearing from the reader.
We've come to expect fairness in everything we do. I'm sorry folks but in MHO fairness only works in the stories we write. Everything else in our life is in our own abilities. (sorry for the rant-it's been a bad week.)
Personally, I look to the number of downloads for my stories not the score. Even if the reader only looks at a paragraph or two that means something to me.
For multi-chapter stories I look for a increase in downloads with every chapter posted. To me that means I've held their attention one more time. That means more to me than the than any score.
The other coment is that I like the improved comment section. Good, bad or indifferent, it's all good to me.
So I guess I'd vote for, leave the scoring system as is or get rid of it all together. Someone will always be unhappy, no matter what you do.
Allen Wilson
Lets say I download a story to day,
next time I log on and select a story to download, - I'm reminded that I owe a vote, - and the vote-page for the story pops up - in a way so that I can't download without voting, - something like clicking on a link on the crack- and porno-sites...?
Personly I use the scores in selecting stories to read, as I seldom go for a story below 9, if I do,it has to have the quite right description.
I'm OK with the new system. My only comment would be to delete the last three options. Quite frankly, I don't need the comments of anyone to the effect that the story is "Pretty bad", or "You call this a story?" Those people I want to tell "you got what you fucking paid me for -- which was nothing."
So, Go with your word descriptions but I'd recommend deleting those truly negative options.
Tom Land
One mark for artisic merit: plot development, character develoment, originality of story etc.
Another mark for technical merit: grammar, punctuation, spelling, sentence structure etc.
The two marks can be given equal or different weightings to arrive at a composite score.
gbin269
The total of the score, whether a 9 or a 2, doesn't matter much to me. There are thousands of readers out there who read and don't vote or comment; I get about one note per 500 downloads.
The current system is okay as far as numbers; the intended improvement will help weed out some of the trolls. I'd like to know if I can improve my stories, but I'm dependent on feedback for that to happen. If a reader downloads it and doesn't vote or comment, should his/her opinion count?
I like the new wording, except for the top and bottom. I don't have any suggestions for replacements, sorry :)
Janna Leonard
The only "problem" I have ever had with voting is there are a couple of voters that seem to dislike me personally and always vote a 1. I have never read a story that is bad enough to warrant a 1 so I take that personally. I know that there isn't anything you can do about people like that so I just accept it a spart of being an author.
Please keep up the good work and I for one appreciate your site.
One problem with your suggested approach is that this alienates the voters opinion of the process in the first place. I truly think that if you subtract points from what votes already voted, you will alienate more voters from the process. And If I am understanding correctly all future votes will only be counted at the lower values. I tell you what. If I am the first person to vote on a story and I gave the story an 8. Then the results come back, that I am the first vote and the overall rating for the story is now a 7. Well I would go ballistic to find my vote didn't count for what I voted. To overcame this, I would either vote all 10s or not at all. And could easily see others voting 1's or not at all in repose to the change.
The most objectionable part however, is that a someone else story doing poorly is going to make my story look like it is doing poorly.
So maybe I am thinking more about the unintended consequences of the change. In either case, the only real choice I can see is to restart the voting process from scratch under the new value system. That and forget the weighted scouring.
I can't tell you for sure what my reaction will be if my story's drop one or two points in value overnight. The only thing I can say for sure is that I will not be a happy with it.
Timm
Have you thought about decile scoring? By that I mean, regardless of the numerical score (or word score) after a period of time -- 1 month? 3 months? -- the stories would be rescored as 10s if they were in the top 10%, 9s if they were in the second 10%, and so forth.
I don't know if you save scores for any period of time, but if you did for a long enough period the computer could convert each score to its relevant decile location and then the total scores for a story could be recast.
It would eliminate the vote compression that's certainly a problem now.
Other thoughts for the descriptions:- Highly recommended,
will read again, enjoyable,worth a read,Killed time, have seen better, not worth reading , not worth writing, Ugggh!
All subjective but they pick up on the emotions and feelings generated by the works
New descriptions as just applied are better but note that the follow-on member bookmark system is still using the original numeric score. Also, lists such as the Rising Stars report still rely on numeric score and may become broken as the scores are redefined.
Finally, I would suggest moving away from representing score numerically in order to break away from the current system and prevent adverse comparisons when the weighted scoring drops the new scoring levels; see comment above about rising stars etc.
Whatever the outcomes, thanks for what you do for us - it is appreciated.
- S
That said I have to say that to me a "great" story is surely better than an "excellent" story.
I think your first sugestion is a good idea. I do not completely understand your second suggestion. So I cannot comment upon that.
That said, I usually do not read stories with themes that I do not like. So naturally I end up reading stories which I will like and so the stories end up with a rating of 8-10. That is probably the reason for high scores.
I wish you could come up with a system to allow scores on (1) the value of the storyline, (2) the entertainment value of the story as written, and (3) the distraction of poor grammar, with the combined (or averaged) score being the actual score for the story from that reader. I suppose this is too complex to be implemented. Thus said, your suggestions are an improvement. If I can't have it my way, I'd rather have it as you suggest!
Second, you may want to consider automatic ratings based on reader behavior. For example, when Al Steiner posts, I would bet that 90% of readers read his story within a few hours. They tend to know approximately when he will post the next chapter, and look for it every day as the time approaches. You could come up with something that would rate a story based on how many people continue to read a story as new chapters are posted. Poorly written stories tend to be forgotten, and readers tend to ignore them when they see them post an update.
In the above case the story would perhaps end up with a very low rating and yet be very appealing to the few.
One way around this may be to give the number of votes for excellent, very good, etc, right down to the poor, horrible, useless end of the voting range. This way people can at least see that the story appealed greatly to some readers.
so any score will always be selective and baised.
i think that a voting like with the reviews is better!
just my 2p, cheers
ham
I come to the site to read the stories. I pay the fees for access. I do not want to be required to vote. I do not want to be required to jump through hoops just to read. Does this make me a troll, or as one of the authors here indicates of some readers, an asshole?
Just what is a troll, that entity that so many of the authors are condemning? Is that anyone who does not hold their writing in the same high esteem as they do?
Should there not be a category about stories that are too awful to finish reading?
There are many good stories here, but there are many more that are just a mess. I do not come here expecting to find marvels of great literature. That is what I find in libraries, at least to some extent. I come here for a particular type of literature normally frowned upon by polite society.
Anything that makes coming here less of a pleasure and more of a chore, will not be good for the site.
Requiring comments to the author can be a waste of time. Good praise is always welcomed. Any criticism is just the troll at work, to be ignored.
Ignore many of the votes, high low or whatever? Why bother to vote at all if that is to be?
There are some proposed complicated and amazing vote evaluation, weighting, and super voodoo accounting ideas popping up. Just how complicated should a porn site be?
Hide the votes until the reader has read the story and then voted? Just what would the purpose of the vote anyway? I thought the idea is to serve as a general guide to help people find the stories that they want to read. How could you reconcile yourself if you find out after reading it and see the other votes, that you enjoyed a bad story?
I find the votes of others interesting but usually of little value. When I see a story that is one massive paragraph with score of 8 or higher, I begin to wonder if I am the only one who noticed.
The authors say that they want comments. I am quick to thank a writer for sharing their efforts. I appreciate the effort to create and present their work, especially when there is no return.
However, if there is nothing praiseworthy there, does it serve any purpose to comment? I have been cursed by several writers who would not know the difference between a comma and an amoeba.
Allow people who care to vote on story and technical ability. These are just guideline opinions anyway.
I would very much like the ability to revise a vote on a multi-chapter story, or else at least vote seperatel on each chapter. Someone mentioned the possibility of a voting form (or better, perhaps just a link to one) on every page, for when one drops -- for whatever reason -- the particular story one has been reading.
Multiple catagories for evaluation (eg. the story itself, the quality of the writing, plot and character, overall satisfaction, etc) might be a good *optional* addition to a general all-purpose vote -- but I wonder how many catagories would be sufficient, or excessive.
Is a perpetually recalculating curve really better than an occasion adjustment? I don't know, myself. It does sound like a perpetually self-adjusting bell-curve mechanism might bother some readers/writers, who may feel it denigrates the readers' input.
The descriptions themselves are pretty good -- but I read the first one as "Impossible to Remove", which while awkward, makes more sence to me than "Impossible to Improve". I thought the proposal of "Future Classic" was going in the right direction. One quibble: I would consider "Great" to be higher praise than "Excellent"; but maybe that's just me.
I have posted stories that have scored highly until one person decided to put in a very low score. The average then dropped like a stone.
Averaging out does not in point of fact, give a particularly accurate representation of whether most of the voters liked or disliked the stories they have voted for, it merely states the median.
It may well be too much information for the site to hold on to, but if you're going to provide a service, it would be more helpful for us authors to know how many voters said 'great' and how many said 'rubbish!' as well as those in between.
At least then, we would know the majority view, or be able to see the scoring trends.
I also like the idea of scoring serials (those published in chapters or bits at a time) on a chapter by chapter basis. As someone said to me, many serials have a tendancy to be waffle and never reach a conclusion.
In addition, it may be as well to have a two tier scoring system whereby an author is marked on the spelling, punctuation and presentation (as this is one thing that will make a story easy or difficult to read) and another for the story itself.
I know it's difficult to rate stories at times as their appeal is subjective, however, it's nice to see that it has been realised that the current method is somewhat flawed.
That being said, why all the fuss and bother. As the wise muse said, 'If it ain't broke, why try to fix it?'
That effectively washes out all the "always scores 10" voters, while still allowing extraordinary stories to be recognized.
I want to read the story off-line, and then come back to score it. Currently, I cannot do this without increasing my downloads for a story I have already read. This makes me not want to score the story. Maybe you can add a score button on a story after I have read it, and don't subtract from my download totals. Or, you might want to allow scoring for each chapter. And then average the total for each chapter at the end. But that would make more voting, and I don't see the average person doing that.
You might want to have a simple and complex score. That is, simple based on what you have listed above. Complex, as another score, that would allow details to be asked (i.e. story plot, uniqueness, grammar, etc.). I would envision the complex score to be something that would help the author (and editors) and that it could be weighted and combined with the simple score, to give a better picture of the story type. I have scored wonderful stories a little lower, because of bad or no editing. While this is indicative of the entire story, the author does not see my reasoning, and is not as pleased, even though it might not be all his fault (editors see things the author cannot see, that's why they are there). Think of the complex score as what your English professor would give, and the simple score as what most individuals think of the story. Both have merit, but neither gives a complete picture.
I do not score any story based on anything other than the story. Although I have never scored anything lower than a 5 (If it is that bad, I drop out of the story long before I get a chance to score it. But I know that others do score for multiple reasons. That being said, you might want to be very specific in what you ask. Some will still lie, but with more details, you should be able to get a better picture of the entire story.
1. Perhaps the weighting could be done against the average score of a publication year to work against the upward drift?
2. It would be helpful to post not the mean score, but the median. That would eliminate the troll votes, and it is a statistically more reliable value.
3. In multi-chapter stories, the scores accumulated could be deleted when the story is finished. Everyone who reads the final chapter can vote for the overall score.
4. To weigh in the number of votes or the number of downloads, as suggested here, would skew the scores in favour of established authors with a big following and against niché stories.
5. Authors could chose to publish their story as niché story (all the non-straight sex stories which seem to draw the ire of of straight laced readers), and their scores could be weighed against the average in that group. A lot of well-written stories get lower scores because they do not match the popular tastes.
Just my 2ç.
Argon
Is it possible that we just have a great quantity of really good authors at SOL? If you rated the singing of everyone in the Mormon Tabernacle Choir, would you be surprised if they all scored above nine?
If the purpose for scores is "Bragging rights", then yes, change the system. If the purpose for scores is to indicate an interesting and well written story, I say it's just fine for that already.
I also second the desire to set the bar for a "10" very high, so that this score doesn't get abused. Personally, I think something like "Deserves Print Publication" might work. Ranking a story's content and writing separately sounds interesting, but in practice I suspect it'll only further cloud over issues, since it'll involve even further participation from readers.
Lastly, a plea should be made to all readers to not only score the stories they like, but also score the ones that let them down, even if they haven't finished the story. In this way, they are strengthening the viability of the scoring system and potentially saving other readers' time.
All of this is very subjective. Stories strike an emotional response with readers, and our life experiences, and where we are at in our lives, will affect those responses.
Having said all this, I like what the_Scot suggested, to separate out the content, the style, etc. There are stories that can strike a chord, yet which suffer from grammar and incomplete and disjointed sentences. I might really like the story, but will not enjoy the telling of it.
But the site is still the best around, and I truly appreciate the effor5ts of Lazeez and the authors!
Sorry for all the trouble.
Would these authors object to the added voting catagories on top of the usual overall score? Adding a separate box under the usual voting box that had plot, technical quality, and perhaps stroke?
Also, were they objecting to voting on every chapter in a serial that is for there use only, again seperate from the overall score?
The main thing is these additions would not affect current or past scoring that is shown to the readers.
I think these are good ideas, if they are not too much work for you personally.
Thanks for trying to improve your site.
I am writing one of those multi chapter stories. So if I was writing for the numbers, here’s how I would do it. Write small chapters and post them two times a week. Readers love chapters coming out fast. Small chapters mean more of them, so more chances to vote. The day you post is also important to votes. Readers pick up stories on certain days, so if you post then, you will get higher scores. If you want to play that game, you can win. That is, if that is what you are writing for. Anyone who knows my story, knows that is not what I write for.
I wish the voting could show different things. Have talked about it, but the consciences was, it is hard enough to get the reader to vote in the first place. If they had to vote with several categories, it might cut the number down further.
I feel the idea of how things are being changed is very good. My only problem is that the new stories have been getting better scores. I don’t think they are better than stores that were posted several years ago; it is just a change in how the reader is voting. Older stories could suffer with a lack of readers looking at them, if their scoring dropped with the averaging system talked about. (I am thinking of some of Al Steiner’s wonderful stories. They were the ones that got me started writing.) Maybe if the average for each year of a stories post was the benchmark and then the adjustment made from that. Without knowing how hard different types of information is obtainable, it is hard to give suggestions about how to make that work effectively.
I don’t think that there is going to be a system that works for everyone. I am pleased to know that this isn’t the first time the scoring system has been revised. It probably won’t be the last. For me, personally, it shows that there is a care for the overall website. Lazeez’s commitment of time and energy is comendable.
This decision is MUCH harder in the gap between 9 and 10 and when the score is lower than 4. Just my way of thinking. Scoring from 1 to 100 would (yes I am deliberately leaving out a 0 score) would make a lot more sence, and the math would not have to change all that much.
Weighted scores are not a good answer, especially as you point out that a story's score can change even if it has not received any new votes. Confusion reighns supreme!
A suggestion for multi-part stories, one that I am sure would result in a bit more work for Lazeez, but would make a lot more sense to my mind anyways:
+----------------+-----------------+-------------+---------+---------+
| Chapter # | Date Posted | Downloads | Votes | Score |
+----------------+-----------------+-------------+---------+---------+
This would be visible only if you open the story, the Updates page would show the over all score of the story (which could just be the average of the chapter scores). This would give the author and the readers an idea of how the story is progressing.
The Carnie