Derailed
Well, it has been an interesting experience to say the least.
If you haven't read my two most recent entries about changes to the scoring system on the site, you should read them.
The gist of this blog entry is to tell you that ALL changes to the scoring system have been canceled and reverted.
A recap of what happened.
For the last couple of years, story scores on the site have been edging up higher and higher due to various psychological reasons. The median for all scores of last year is about 8.87. Which means that half of the stories are scoring more than 8.87 and the other half below 8.87. The results of this median is that story scores are in general, extremely high.
Over the last year, I kept hearing from many authors how ridiculous the scores are becoming, and from readers how they can't trust the system to guide them to the better stories. The phrase 'I don't read anything that is not scoring over 9.25' came up very frequently.
So, because of these things, I set out to correct what's causing these issues. I planned to do two things:
The first is to change the wording of the voting form to be more consistent and somewhat realistic of what those votes reflect, which is the reader's feeling about the stories they read. The current wording is somewhat confusing, some grades are related to feelings, some others are related to the quality of the work. I seem to have made the grave mistake of wording the top score (10) entry in a way that few readers really could give a 10 but to the really best of stories (which logically is the way it should be).
The second was to create a different score evaluation system that took into consideration what date the story was posted and the average of all the stories posted in the same period of time. The new system would have taken care of the over-time creep up of scores and made the playing field between old and new, more level. It would have also leveled the playing field between older stories that were scored using the old wording and the newer stories scored using the new wording. There wouldn't have been a large discrepancy between the meaning of older stories and newer ones.
This new valuation system would have shifted the median of scores down to a more logical median. My goal was to have a median of 6.
I guess I aimed too far down.
On December 1st, 2006, I announced the upcoming changes, and implemented the vote form wording change.
The wording change had its intended effect and stories posted after the change had a more moderate, more reasonable score average. Many seemed pleased with the change, although, I only received two approvals, one from an author and the other from a reader.
However, the opposition was far more vocal and came from authors. I received many worried notes and nearly ten authors asked for their stories to be removed if the score weighing were to take place.
Since I'm not in the business of pissing everybody off - and to me, ten vocal opposing people represent a 100 opposing silent ones - I canceled the score weighing system implementation.
I left the new wording in the forms and the newer stories continued to be scored lower on average. I received few notes from authors angry about unfairness of the lower scores and one demanded that I remove his stories from the site.
I agree that having lower average scores is not fair, considering that readers won't really know or care why the older stories have higher score average than the newer ones. They'll understand the drop as a drop in story quality. I even received one note from a reader asking why all the newer stories were crap (based on scores).
So, last night, I reversed the change in the voting form, and in an ironic twist, had to use the same code that I created to calculate a weighed scoring system to raise all the score of the stories posted after the wording change to a level more in line with the scores of stories posted before the change.
And now a week after the start of the changes, we're back to the same old system that many mocked and distrusted and complained about. We're back to a kindergarten-like system where everybody is a winner and everybody gets an A for their effort.
I don't know what I'm going to do next, but I'll think of something to keep the scores under control. I really don't want to see the scores on the site compress any further and lowest scoring stories reach 9+ and high scoring stories reach 9.9+.
If anybody has any suggestions to make the system work better, without confusing the hell out of authors, I'm all ears. And please, no suggestions of a complicated score weighing system. I had the perfect one and it was rejected. If there were supporters for the new system that I planned and announced, they mostly kept it to themselves.
Update: Please see my following entry about the subject.
If you haven't read my two most recent entries about changes to the scoring system on the site, you should read them.
The gist of this blog entry is to tell you that ALL changes to the scoring system have been canceled and reverted.
A recap of what happened.
For the last couple of years, story scores on the site have been edging up higher and higher due to various psychological reasons. The median for all scores of last year is about 8.87. Which means that half of the stories are scoring more than 8.87 and the other half below 8.87. The results of this median is that story scores are in general, extremely high.
Over the last year, I kept hearing from many authors how ridiculous the scores are becoming, and from readers how they can't trust the system to guide them to the better stories. The phrase 'I don't read anything that is not scoring over 9.25' came up very frequently.
So, because of these things, I set out to correct what's causing these issues. I planned to do two things:
The first is to change the wording of the voting form to be more consistent and somewhat realistic of what those votes reflect, which is the reader's feeling about the stories they read. The current wording is somewhat confusing, some grades are related to feelings, some others are related to the quality of the work. I seem to have made the grave mistake of wording the top score (10) entry in a way that few readers really could give a 10 but to the really best of stories (which logically is the way it should be).
The second was to create a different score evaluation system that took into consideration what date the story was posted and the average of all the stories posted in the same period of time. The new system would have taken care of the over-time creep up of scores and made the playing field between old and new, more level. It would have also leveled the playing field between older stories that were scored using the old wording and the newer stories scored using the new wording. There wouldn't have been a large discrepancy between the meaning of older stories and newer ones.
This new valuation system would have shifted the median of scores down to a more logical median. My goal was to have a median of 6.
I guess I aimed too far down.
On December 1st, 2006, I announced the upcoming changes, and implemented the vote form wording change.
The wording change had its intended effect and stories posted after the change had a more moderate, more reasonable score average. Many seemed pleased with the change, although, I only received two approvals, one from an author and the other from a reader.
However, the opposition was far more vocal and came from authors. I received many worried notes and nearly ten authors asked for their stories to be removed if the score weighing were to take place.
Since I'm not in the business of pissing everybody off - and to me, ten vocal opposing people represent a 100 opposing silent ones - I canceled the score weighing system implementation.
I left the new wording in the forms and the newer stories continued to be scored lower on average. I received few notes from authors angry about unfairness of the lower scores and one demanded that I remove his stories from the site.
I agree that having lower average scores is not fair, considering that readers won't really know or care why the older stories have higher score average than the newer ones. They'll understand the drop as a drop in story quality. I even received one note from a reader asking why all the newer stories were crap (based on scores).
So, last night, I reversed the change in the voting form, and in an ironic twist, had to use the same code that I created to calculate a weighed scoring system to raise all the score of the stories posted after the wording change to a level more in line with the scores of stories posted before the change.
And now a week after the start of the changes, we're back to the same old system that many mocked and distrusted and complained about. We're back to a kindergarten-like system where everybody is a winner and everybody gets an A for their effort.
I don't know what I'm going to do next, but I'll think of something to keep the scores under control. I really don't want to see the scores on the site compress any further and lowest scoring stories reach 9+ and high scoring stories reach 9.9+.
If anybody has any suggestions to make the system work better, without confusing the hell out of authors, I'm all ears. And please, no suggestions of a complicated score weighing system. I had the perfect one and it was rejected. If there were supporters for the new system that I planned and announced, they mostly kept it to themselves.
Update: Please see my following entry about the subject.
Comments
I was not in favor of the scoring changes when they were announced, but I decided to work within them. I'm a lot happier now.
Speaking for myself, we authors are more liberal in acknowledging a need for a thick skin than actually having one. Our writing reflects hours of hard work and exposing our private thoughts and feelings. It isn't easy to get beaten up and a system that promotes that outcome is tough to swallow.
If scoring reforms are to be implemented I hope that they reward author creativity and writing quality. I would hope that authors could be encouraged to go beyond formula plots and worn out language, as well as to edit carefully.
A suggested reform would be to place a "Voting Guide" somewhere on the site that suggests to readers some factors that they might consider when casting a vote. Some of the points above could be applicable.
The system certainly lends itself to high scores. A partial explanation is non-voting by readers with negative opinions of the story who stop reading rather than continue to the end. The strength of the system is simplicity and consistency.
Thanks for doing a fine job managing this site. It is undoubtedly sometimes frustrating and thankless.
Autumn Writer
Personally, I've stopped even looking at scores, while I used to do it, the amount of 9+ grades makes anything worthless. It resembles the idiot olympics more then anything else.
I'm not sure whats a better system, with so many authors resisting the changes.
I'm tempted to say damn those authors, full steam ahead, but without knowing who is involved, I can't say much.
by authors.
Suggestion: if you're an author or reviewer, then you can be presumed to be more literate than a non-author. Your vote should count as much as, say, five non-author votes.
The basic problem seems to be that many of the so-called readers are really, well, borderline illiterate. They rate stories based solely on the subject matter, rather than on the author's skill.
There should be some way of rating
readers. Perhaps some sort of elementary literacy test. If they fail, they're not allowed to vote.
Liked new wording and hate seeing your work being unappreciated, but I can see why there would be such a opposition from authors. I what could be worth trying is to have 2 scores one that all users vote on and other where only votes of reviewers, authors and maybe premium users are taken into account that would hopefully give somewhat more meaningful score. You could run some queries and let us know what the results are.
All the best
Daniel
I'm sorry that there was an outpouring in opposition to the changes. Changes in general are difficult, authors tend to be pretty protective of their stories, and I think there may have been more than a little ego involved in protecting their scores.
In general, I don't pay a lot of attention to the stories I read- actually the story codes and and the little blurb (along with the author) are more likely to attract me.
Anyhow- thanks, Lazeez for all the hard work you've put into this site. I doubt if anyone really knows just how much goes on behind the scenes to make this a premier site for posting stories.
Merry Christmas/ Happy Holidays/ Happy Hannukah, etc. to all readers and authors.
John Darkscar
Because authors have an expectation of a certain result that they have been recieving for years, they resist the change you are trying to make. I've read cmsix's blog and Oldfart's forum as well and I can understand and sympathize with how they and the other authors feel. I can also see where the voting has reached a point of absurdity.
I guess it will all come down to what is more important in the long run: to have a scoring system that hides outstanding achievment as well as mediocrity or to leave things the way they are and make the current group of disgrunteled authors happy.
I don't want it thought that I am saying that cmsix's or Oldfart's work is mediocre. I have been a long time reader of there works and have emailed them to say so. It was wonderful to see more from each author.
But, if one is looking at this as objectively as possible, I would think one must admit that that current voting system is seriously flawed. If there should be change, which is always painful, then only Lazee can decide what, when and how that change should be implimented. I would ask those resistant to 'any' change try to open up your ming and continue the dialogue to find a comprimsie. There needs to be a starting point or the voting system will go from the currently absurd to the absolutely useless.
I feel that the real dilemma is finding a way to lower the expectations of the current authors while trying to make this a better system for all future users.
I don't know if there is a way to do this fairly to all but I do know that no change is unfair to all, even those who wish the voting would remain as it is. In the long run the authors who resist this change the most will be just as affected; as the future readers of this site will no longer have a way to find the outstanding works of the cmsix's and Oldfarts of this site.
I am trying to promote thoughtful dialouge with no acrimoy. I used cmsix and oldfart as examples because of there public views in there blogs and forum. I mean no slight to them personnaly.
thank you.
I've been thinking about this problem for a long time. If I had the time and resources, I have some ideas that I think might lead to a reasonably fair system.
In any case, I applaud you for at least trying.
Ace
I am not an author, but I believe that the authors would benefit from having the separate scores. As a reader, I would also benefit because it would give me some insight as to why the overall score is what it is.
Nothing can be done about them, though - although perhaps randomizing the order of the scores on the voting form would take care of some of them, as was sort of suggested... So, looking at the other problem - that of those stories which would recieve a low vote, if readers didn't quit before getting down to the voting form. That has a fairly simple solution, I believe: set up the story display page to always have voting available on the bottom line of the screen. Thus, when someone wants to give up in disgust on a story, they can vote without having to scroll to the end of the story first. That might help in lowering average scores in a way that - hopefully - less authors object to.
My suggestion is that maybe you could do both? Continue the old, reinstituted scoring system, and then beside it either add both the new rating system & the new, honesty-weighed scoring sytem, OR, just add the new mean-averaged scoring system. Or, maybe a link to 'show this writer/story with mean-averaged scoring'.
I NEVER use the current score/rating system (either to judge what to read, or to 'comment') because it's patently ridiculous. Theodore Sturgeon's infamous-but-oh-so-true dictum that '90% of ANYTHING is crap' applies even more-so to attempted artistic endeavors, & every single one I noted was 9+ before stopping paying any attention to scores entirely. Some might say, 'treat 9.5, or even 9.7, as a 5'; but the judgment of scorers who rate EVERYTHING as 9+ isn't to be trusted. Of the 1000s I've read, ever, only a dozen or maybe a baker's-dozen at most, would be 9+ out of 10--only 20% or so would even qualify as 'author' vs 'writer'. Don't get me wrong, many of my all-time favorite stories/writers wouldn't be, ALLTOGETHER, scored above a 6 or 7! though they're still quite engaging, entertaining, &/or intriguing. This is NOT being 'harsh', or 'unsupportive'!, rather the opposite!--EVERYONE, readers & writers both, benefit from truthful & honest review.
Then I read of Lazeez' superb plan to correct scoring to a mean!, INGENIOUS! Not only that, but he plans to combine this far more true-scoring with a practical, pragmatic rating system! WITH THE 2 COMBINED, at last, something usable, sensible & shrewd. Also, as-is is good, 'kiss' principle definitely applies--the proposed scoring & rating system & that's it. [A truly honest, real-world, authentic mean would be 4-, though realistically the most I could hope for would be a 5 mean, I could live with--& more importantly have some RESPEST for--the proposed mean of 6.]
Even though SOL supposedly exists primarily for the sake of READERS, not writers; I understand that you must cater to those submitting material, so I don't blame you for the cancellation--but to those who threatened to withdraw, I call you: "COWARD"!; obviously not really wanting to improve & rather coast along in a cushy world-view of your own ego's making. Those threatening to leave because of honest scoring & rating, along with those who whine & wheedle about their ratings on the current asinine scoring system, aren't worth reading anyway--& if you could get by without 'em I'd say 'good riddance'.
Lazeez, I salute you, & thank you, for the attempt. I have a great deal of respect for you because of it.
w_newd
nonof_urbiz@sbcglobal.net
As for the wording change, I didn't think that would be a big deal, but to see my current story plummet from 9.75 to 9.71 in four days shows that a lot of people were exercising more judgment in their scores. The problem with that, though, are all those longer running serials with thousands of votes. Like I said, it's an imperfect world.
Gina Marie
Basically, I look at the description and codes and see if the story appeals to me. If so, I look at the score. If the score is above about 8.8 then I will give it a chance.
Note that I don't really care what the score is, just that it hasn't been considered "junk" by everyone.
In that spirit, perhaps (in addition to the numeric score) you could also color code a story based on whether its score is above, at, or below the current average of scores being given this year (or month or whatever).
A "better than most" story could be shown in green, an "about average" story shown in yellow, and a "not so good" story in red.
Under this scheme, readers would be drawn to the best stories and authors would have a sort of simple "thumbs-up/thumbs-down" feedback (which would be independant of any score inflation).
While I am mostly a reader, I do have one story posted, which received a 9.22 (which is either "great" or "barely readable" depending on how people vote here). :)
LordGarth
Well as part of the "silent majority" I feel particularly embarrassed and guilty about not commending your earlier efforts. Though I wrote a comment about how I thought renormalizing the scores might not work, I found your new voting categories, which had explicit descriptions for story quality, very useful. In particular, it made it explicit the difference between a "Good" (8) and "Amazing" (10). I'm very sorry that you were, effectively, blackmailed into changing your site back to the former system because it might lower scores in the future.
Alas, I think you're trying to create a rational voting system to compensate for highly irrational voters. I found reading the comments to your earlier blogs to be highly illuminating -- there was a large number of posts by people who proudly boasted that they indeed voted nothing but 10's. Though it is obvious, I'm not sure that all the complaining authors realize that, for a story to have a score of 9.5, at a minimum 50% of the voters had to have voted 10's. And this is for a score that, by consensus, was the threshold for "very good" stories. Even with the normalization scheme you proposed, one can't rescale a 10 to a lower score and pretend it to be fair.
The only way to create reasonable scoring system is to start with reasonable voters. I'm afraid you're out of luck on that one...
As always, your attempts to improve your excellent site are appreciated.
Within a week, NanoVirus was #1.
Rah-Rah, cmsix.
Sure, authors aren't getting paid. But a garbage 10 vote isn't payment, either. A well written comment of "thank you very much for writing such a good story" is worth 100 10 votes. But the ego of some authors to HAVE to be #1, is insane.
Unfortunately, I think Gina Marie hit the nail on the head. No matter what you do, people are going to be unhappy. Especially authors that really put a lot of weight on the scores their stories receive. (Which I can understand.)
I liked the way the new scoring was turning out, but it sure wasn't going to address the older stories on the site. And I wasn't crazy with the weighting old scores, because voting patterns work in whacky ways. Different stories score differently, depending on subject matter and the rapidity of posting. Longer serials would be affected, and stories currently in progress would really show strange results.
If it would be possible to have both scores posted, that would work for me. I'd like to see both, because I'd trust the newer scores to give me a more true indication of what readers thought. I'm tired of the "TenFest" we've been under for the past few years.
Also, I don't think premium members and reviewers should have their scores carry more weight. Reviewers (I am one of them... the tough one!) have a way of scoring, and yet most of them only score 9's or 10's. So, you won't see anything different from them. And, scoring isn't more accurate because a premium account holder voted. It just means they had more disposable income they could send to Lazeez for his efforts. Neither set of people probably show any real difference in their voting from the statistics that Lazeez showed.
It matters not at all whether English is a first language. If a tale is told with mangled language, it is not going to be read - at least not by many.
The other option is to use the 3 part system used by reviewers. I think that may well cover all the bases - and let potential readers have more info.
DirtyH
I need to be educated by the writers - what constitutes a satisfactory score. Is it a particular number? Is it a number that exceeds his/her last offering? Is it a number that exceeds that of another author? Please help me understand what you want from a numeric scoring system that will make you happy enough to continue writing in a quality and timely manner. Thanks
My best to all
justaguy!
What is needed is the ability to vote once by a reader for any story but also allow them to change their voting for the same story.
More important, however, is to find a way to disallow or discourage trolls from voting.
Perhaps, making it compulsory for any voter to type in a short message why they vote to far below (or above) the average a story has achieved is one way. If some troll is determined to mess around, let him waste his time. And to make sure he's not just doing a copy and paste job from a blanket note, force him to send the message to the author so an author could at least bring the abuse to attention.
I've heard so many times Authors complaining about no feedback. Well, tie in the voting process to feedback system. If a reader is inclined to vote, why shouldn't he be forced to send feedback.
Another, suggested that the voters should have proficiency, and I suspect it reflect a similar sentiment about the absurdity of the voting that has been going on, and will continue. I'd like to see something done along those lines. Maybe allow readers who wants to vote to take a simple grammar or other test, and based on their scores let them have a special account that allows them to vote. I know this sounds elitist, but hey... it gets really tiring to see trolls and others messing up with the system and turning it into something useless to both readers and authors. Readers suffer and complain they can't find good stories. Authors suffer because their work isn't appreciated or evaluated on a better and a level playing field.
As it is, most scores are not worth any consideration. I've been using download numbers, names of authors recommended, and similar information rather than the scores for a long time!
Maybe Lazeez should consider the alternative scoring system and enforce both the old and the new when someone is voting, and let the readers see the differences in both scores. And if he's going to start using both systems, I would prefer that the new system to have some checks and balances, such as enforcing voters to send feedback, or allowing them to vote only if they have gone through some kind of proficiency. That kind of precautionary methods might just show to everybody what system is really worth attention!
anon
One of the biggest problems is that only about 5% of readers actually vote. Making the voting more complicated will mean less people vote, causing the "trolls" and "cheerleaders" to affect the results even more than they do now.
If a change is to be made, I would think it would need to be something that will encourage more people to vote. Perhaps a simpler sort of Thumbs-Up or Thumbs-Down vote would be better.
Though I think the best "change" would be to add a "color code" as I suggested above. This would not require changing anybody's scores. It would just be an additional way of showing what stories are getting better scores than others.
LordGarth
Wah!
I want to take my ball and go home.
Wah!
If a story has a good summary and subject codes, it will do much more to encourage me to read it than the score. But I will at least consider reading anything over a 9.0.
Your continuing efforts in providing one of the very best story sites on the web is greatly appreciated. Maybe sometimes you try too hard for perfection.
I am supprised that some of the authors would like to pull their work because of the changes you proposed but that would have been their choices. I would think that the email "feedback" would be more important than the scores but since I am not an author I don't truly understand.
One of the best bosses I ever had told me that the best question you can have is often the one you "don't" ask. Using that philosophy, maybe what you "don't tell us" may be the best thing you ever say!!!!!! This is your site so do what you think is best for the good of your site.
Thanks again for all your extraordinary efforts.
RickM
May I make a slightly different point about the voting system and the unrealistically high votes given. We are only allowed to vote once on a story. This seems fair except that some longer running stories improve or deteriorate over a period of time and we cannot adjust our vote. One or two authors in particular seem to aim for high download figures by padding out a story and writing shorter and shorter chapters. I would like to be able to give a lower mark if a story deteriorates and (sometimes) a higher mark if a story improves. Is it possible for the system to permit further voting after, say, ten chapters?
Otherwise the solution for the reader would be to wait until a story has finished (and some take years) before voting. This would not be satisfactory for authors who would receive no feedback via voting whilst they are writing.
Am I alone in thinking that repeat voting could be beneficial?
Frank
However, the other story site that I occasionally visit has no voting mechanism, and it can be quite frustrating. I often end up wasting quite a lot of time "sifting" through poorly written junk in search of something decent. Your voting system, while obviously not perfect, greatly reduces this problem.
In short, despite all the complaints, your site is by far the best one around.
Thanks for all your hard work and dedication.
LordGarth
As an author here at SOL, I'd like to comment on the scoring system snafu.
At first I will admit to some worry about my scores...then I thought about it some more and realized that I could not or rather, should not worry about 'scores'
I don't write for 'scores'. Scores do not pay me, they do not establish who or what I am as a writer, and they don't really have a bearing on my ability to write.
Scores usually just show that a story is either:
A, Loved
B. Hated
or
C. Indifferent
Yes everyone wants a top scoring story online. Yes, every writer wants to be the 'best' at something.
Me, I decided that scores don't really matter that much. The truth is in the writing of the author, and a good author will write, generally, a good story.
The readers of the stories know this. The readers can either get into a certain story or not.
That a few authors shouted to take it back or they'd go play elsewhere are rather childish reactions.
If they have to worry over a score that is 'real world' or a score that is lop sided in their favor, then, and I say this in as even a way as I can...they may want to concentrate more on their writing and less on the scoring.
I, as a writer, have had outstanding scores and some real stinkers for scores.
I realized after the dust settled that yes...the stinkers did stink. I admit that.
I am not a professional writer, and I do not make a living writing. If I did perhaps 'scores' would matter...in some way...but as the type of writer I am though...
I do believe in a 'fair' treatment of scoring...the real world dictates that not everyone 'wins' Even though it seems that more and more youth sports are pushing in that direction.
Real world is...do a good job you 'MAY' win. Do a lousy job...you will lose. SO WHAT? That's LIFE.
I'm sorry that not everyone who submits stories to Storiesonline isn't an Ernest Hemingway or a Curt Vonegut. It is real life though...there are winners and there are losers.
That doesn't mean that the winners will always win and the losers always lose however, and I'd encourage the authors who had issues with the scoring system to reflect on that.
Work at improving yourselves and your crafting of those stories, taking the lumps as they come.
Really.
An aspect of the scoring that hasn't been mentioned is that the results determine which stories appear in the top lists. This has a big impact on readership, and ultimately results in higher scores because of the stories' association with the list.
Having made a few brief appearances on a few of "the lists" I can attest to the increase in numbers that goes along with it. I also believe that many readers vote for their favorite authors' stories to help push them onto "the lists".
Mine usually fall a little short of "the lists" so logically, I should have favored any revision that would have shaken things up. I couldn't do that, however, because it is easier to work from a known position.
There are a lot of aspects, and it is just academic now. It might help people understand authors' points of view more clearly.
AW
Let's think about this for a second. When you go into the bookstore to buy a book, you don't have scores to go by. You pick up the book, you read what it's about, and if it sounds interesting to you, you buy it and read it. Scores don't tell you what the story is ABOUT. So what you get is the assumption that a high score means good and a low score means bad, which can cause people to skip over something they might actually like.
And we spend plenty of time writing it, you can spend a little time wading through it. Again, when you go to the bookstore, you have to look through the shelves.
I think the attitude towards writers who wanted their stories removed is silly. It's our work, we spend hours on it, and if we decide for whatever reason that we no longer want it on this site, that's our choice. You can't have a story site without stories, after all. Some of the comments I've seen people make have been just as immature and childish as they're accusing authors of being.
Either get some empathy or go write for hours yourself. Then see how much fun it is to have your work and yourself trashed.
There's an old saying about letting the lunatics run the asylum. Bad idea. Personally, I'd much prefer a more honest voting system than one that skews all scores into the 80+ percent. So you get a lower score. Big deal. As Lazeez said, just adjust your mental scale. If you're getting 7s and 8s when you used to get 9.5 or more, suddenly an 8 is an incredible score! Wow! I'm getting 8s! Awesome. Then, when you get a 9 or more it's fantastic!
If someone wants to take their ball and go home, let them. Where else are they gonna go? This is the story site on the net as far as I'm concerned. If they want to participate, they have to play by the established rules. Welcome to the real world, right?
Some suggestions though. Have you considered letting readers vote using a slider instead of a drop list. Put zero on the far left and ten on the far right and let the slider change colour from black to red as the score gets "hotter". It would make the logic more fuzzy, sliding the thing from the centre up or down outside of the graduated degrees of one to ten. It wouldn't solve the problem of cheerleaders and trolls, but for people who wanna vote by "feel" instead of by number, it's a great idea.
And what about setting the scores to a default for every download, a default of say 6, the desired medium? Let readers vote with their download, and then adjust it up or down once they've read it? It would encourage more people to vote if they really didn't want to leave the default score at 6. "Uhg! Terrible! Where's the voting form!?"
You might also set up "bonus tags" at the end of a story. Besides giving a numeric score, set a check box for "excellent technical quality" and "suberb story telling" and "fantastic stroke factor", etc. and then list the number of checks recieved along with the score. This would give browsers additional reference for their downloading choices: "Among those who voted, 65% tagged this as a superbly told story." etc.
If system resources allow, offline readers could be shown a page of stories that they've recently downloaded, with drop lists, and give them the option to quickly score them if they choose, on their way to downloading a new batch.
Finally, if system resources allow, it would be helpful for browsers to be shown a general score next to an author's name for his overall body of work -- his or her overall weighted score, and if the checkbox thing is implimented, pop in little icons for the different author's strengths: an abc icon for technical quality, a book icon for story quality, and a thumbs up icon for stroke factor. That way you can see at-a-glance that one particular person is known for his story telling, or his technical quality, or his stroke factor.
Anyway, just a few thoughts. Hope it helps. Don't give up on the new system. Long term, it'll be more rewarding to get an 8 or a 9 if it's more rare. Let the whiners whine and the complainers complain, and then let them get used to the new system and adjust their mental scales. *It would help if you implemented the changes along with some new bells and whistles like "Author icons". If they need the ego strokes so bad, give 'em those. ;>
-PB1
Can we all just tone it down a little?
These are free stories on a mostly free site and everyone is acting like this is about a life and death struggle! Come on.
There are more important things in the world to worry about. It's time for a reality check. I'm betting that regardless of what Lazeez decides to do, a year from now 99.99% of the vistors to this site won't remember or care what is decided about this whole issue.
sheesh.
My fisrt story I found was cmsix " John and Argent". I liked it and from there I took the time to read the rest of what he had.
What got me out to other stories is the Favorites listed by other Authors. If I liked the story I had just read, I would see what the Author like from other writers. Thats how I found out about Al Steiner's stories. They are a must read. Maybe it would be nice to see other Authors posting they favorite writers from SOL.
As for the scoring, I do look at it, but like I said before, I perfer the longer stories. So size does matter.
As for the downloads, I dont look at those at all. If I read a story, Click, download goes up one, and I read just one line or paragraph, and leave it at that.
And I am one who votes, I give out 8 and 9 to most I read, and 1 and 2 to those I could not finish.
In closing, I pray that you all can keep the stories coming out. I will wait, that all I ever do is wait. Hurry up and wait ( sorry, inside joke for those who know)
Jim Fields
Also, instead of reporting the average score, have you considered reporting the median instead of the mean? That might help with some of the grade creep with high scores having a disproportinate effect.
I like the idea of showing both the "old-style" and "adjusted" scores together. Perhaps after a year of seeing both scores, people would not only have gradually become acustomed to the new system, but also have seen for themselves that the new system is more informative and useful than the old one.
Is this a practible solution? I certainly would find the change an improvement, and those who are addicted to the present system's inflated ratings would, with such an adjustment period built in, be much less likely to feel blind-sided or hurt by new, lower, but more meaninful numbers.
However, I wasn't one of those who asked to have his stories pulled.
I don't know about other writers, but I write because I enjoy writing. I don't write to please others; I write to please ME! I don't care much about the score - I do care about downloads, but even having 1000 people read what I wrote is better than nothing.
Yet like other writers I've had days when I got hit by trolls and bitched - hey, some days I've seen a score of 9.8 drop to a 9.5 in an hour and I've bitched. Then there are days when I got over 100 anonymous emails in just hours, almost all of them vulgar and all of them super critical and again I bitched. Then I got smart, cut off the e-mail feed from SOL and redirected people to my yahoo site, if they wanted to e-mail me, they can do it from there.
But guess what? Even when I was annoyed and complaining, I still posted another chapter when it was done.
So I'll carry on writing and perhaps bitching. However, I don't have a clue how to repair the scores. In fact, I don't care that much because I've decided the scores are for readers, not writers.
Whatever you do, Best of luck Lazeez.
.B
The ones that threaten to leave.. yeah they get lots of downloads... but one of them (im not naming names).. I don't read his stories because Lots of ppl like his stories that I don't.. he usually gets a high 9 when I think he deserves a 3 or 4.. and I vote after I read the first chapter to see if he's improved or not ... side tracked.
Nothing you do is going to make everyone happy. What WILL happen over time though -- if another website opens up.. and gets authors to cross-post over there, and their voting system works.. people will leave... gradually over time......
well anyways. my 2 cents... wish you kept the system.
I like many stories, but for wildly different reasons - some because they happen to hit one of my pet fetishes, and even if the story is basically crap, I still enjoy it. Others that don't get me off at all are simply so well written as to be unforgettable, and in some the plot or characters are brilliantly done. Stories I don't like can be because of the subject (the story codes help me avoid most of those), because they're written in the 2nd person (I know the "you" is not me, so I can never really enjoy one of those, no matter what it is about or how well written it is), others are simply so badly written that I cannot make my way past the first page or two. Others are just too short (the size indication helps me avoid most of those - I like something that will keep my mind occupied for more than 5 minutes...)
How can I possibly express any of that with a simple 1..10 (whether expressed
as a number or a menu of words, which I
know is going to be translated into a 1..10 anyway). I cannot. So I will not bother to try. Or not until I have
the ability to give more meaningful information (to other readers).
I agree with the comments that part of the restistance to the change was just because it was a change, and any change is bad (a pretty silly philosophy, since everything is always changing, but one deeply ingrained in many people.) That said, I also did not really believe that the proposed change would help anything much, it was too small a change. If you're going to suffer the pains (and complaints) from attempting any change, make it all worthwhile - go for something wildly radical, instead of just a minor enhancement, which was really that was proposed.
While I am here, probably for the only time, I would like to complement Laseez on the site - it is definitely the best on the net, and you don't need just my subjective opinion for that, just take a look on asstr at the Golden Clit nominations and see what site most of the stories nominated come from. It has been this way for years, well done (to both Laseez and the authors who recognise that this is the best place to be).
One last comment - It keeps being repeated that the authors here are only paid for their stories by the comments
(adoration really I guess) that they receive. Is that really true? Don't
authors (or at least the frequent authors) get given Premier Membership for free? Non-authors pay $N a month for the same benefits. That's $N a month those authors are being paid for their stories, it might not be much, and it might not be disposable income, but it isn't nothing.
BAD GUD
[ ] [ ] Grimer And Speling
[ ] [ ] Plot
[ ] [ ] Plausibility
[ ] [ ] Strokability
Come to think of it I'm sort of describing the review system. Crap. Well, then, a mini review system with the 3 to 6 most important indicators. The results could be shown with the story to the readers, or it could be private to the author as structured feedback.
- Ritalin Underdose
Good job, Lazeez. Keep it up.
Just My two cents.
I really appreciate all you do here Lazeez, It's a great site and my favorite.
With the useless and meaningless scores we have now, a little bit of rational thinking clarifies who stands where and for what reason. More than that when some of them start singing the tune “I’ll take my ball and leave” then you know what the problem is. What really came as a personal surprise to me is that some of them are writers whose works I have read and enjoyed and with their really high scores and high downloads, I’m not sure I could reconcile their attitudes unless it’s an ego thing.
I’m also surprised that Lazeez caved into such demands. If someone wants to leave let them leave. I ask you, how many good story sites that are FREE that are around? Let me hear a count of sites please! Less than the number of fingers on one hand, right? In fact much less than that.
Take the biggest one of them, ASSTR. Badly organized. No scoring. No download statistics of any kind. You want to find a story, you need to use a search system that is slow, cumbersome, and quite many times, it’s broken or taken offline! Yet, it does work. And it has its own abuse problems. Some authors attempt frequent uploads of their work, just to keep current on the “Recent updates” list so that they can gain visibility. Some of their efforts are very blatant, like uploading a list of what’s changed, or a graphic, or... well, you’ve got the idea!
And when someone comes up and says “I’m gonna take my ball,” I don’t see the reason not to let them go. Consider their reasons: my scores will go down and I don’t want it! When those scores have lost any meaning, and when there’s a better substitute to alleviate the situation, it’s obvious what is the unspoken reason, and combine that with the threat... well, I think it is time to see how serious they are, especially when we all know that there aren’t really viable alternatives for good story sites. Hell, all publishing houses have a monopoly, and they exercise their power. Just the stated reason and concerns over lower scores tells us how much ego stroke is involved, so that makes me wonder if the same egos will be able to find an alternative place to stroke their egos.
Let’s really think about that for a second, shall we? What are the viable options for some of the authors who don’t want to play? Huh?
I think that Lazeez can do a service to his site, majority of the authors and the readers. He can even do that without upsetting at least some of the dissenters, byt keeping the old scoring system and ammending it with the new, just to make his point --not that he needs to, since we all know the “score” there :-) The numbers speak for themselves!
And here’s another thing to chew on. This site has been good for everybody! All the readers and authors. The authors have a place to get visibility for their works and readers get a chance to read them. If some long time writers complain about change and go to such great lengths to prevent it, when we know the said change would alleviate a well known problem to some degree, then it does make you wonder the loyalty besides all the other factors that come into play or their stated reasons. Is this site for the authors and readers, or is it the place for a handful tooting their own horn looking for ego satisfaction?
No. I’m not very happy with the old scoring system. No, I don’t like the idea of pulling the old scores by means of an artificial manner. No, I am not satisfied with the proposed new system, but I can see it will help a bit, especially side by side with the old scores!
But what will help is having some system where votes are controlled for abuse, and perhaps weighted in by other factors. Give the voters the chance to change their votes or revoke a vote for a story. Make sure that if the reader doesn’t finish reading the story, the vote is cancelled. I’m surprised to see that writers are complaining about too few voters and want to have a system where more people vote, and then they state about cfheerleaders or trolls voting too freely high scores or voting too low with malicious intent. Uhh, what exactly do you want? Unless cheerleaders and trols are prevented, I think it will be beneficial to have less number of voters who may vote reasonably. In fact, make the scoring system such that a reader has to vote on several categories (technique, story idea, plot, storke content, etc). Make sure that they’re accountable for their votes (like compulsory feedback).
Alternatively, if you want more voters, then force people to vote when they access a story, more importantly, make it compulsory for them to vote when they access the last chapter of a story! I mean, how difficult is it really to display only half of the last chapter before enforcing the reader to vote before he can continue? It may not guarantee some one has read all the chapters, but it’s better than votes that are given during the progress of the story. Perhaps weigh in the votes cast after the last chapter! That should really count for something!
And while a story is in progress, only list the download numbers, the number of voters, but don’t give out the score, except an indication (a color code of below average, average, above average or something similar), but no thumbs up or down to indicate if the scores are going up or down whcih might predispose the voters towards a particular direction. And when the story is complete, compile the votes. It could be a combination of votes given or changed when the story was in progress + the final vote by a reader at the fnal chapter (or a weighted final chapter vote).
And for lists, etc... you can always set up several other lists such as completed stories and stories in progress. Further divide the stories by length as short (less than 100 kb), medium (100kb - 500 kb), long (longer than 500 KB), which should help separate them from each other and also deal with hidden scores, but allowing for a general indication of how they are doing (below avrg, avrg, above avrg), with the relevant other statistics (downloads, no. of votes, etc.)
This might prevent comparison between shoter and much longer stories where a comaprison of scores is not going to be meaningfully competitive!
I also find the weekly lists a bit restrictive! It’s useful, but if someone posts a chapter middle of the week, he’s already at a disadvantage to make the downloads lists. Yeah, I’m ignoring the list by score, because the scores are meaningless, which leaves me only the download stats to look at, and for most authors, it’s the list that they will try to make it into!
Why not have another list, a monthly or something, which weighs in the downloads, and also weight that with number of votes and voters for a length of time (a month is prety reasonable, and it should keep a total of downloads, votes and scores for a month). So it will take into account only the votes (and the consequent average score) and downloads in that period of time, instead of a running total of votes and scores and such!
Anon
I say, let them take their ball and go home. Most of them will be really surprised when they aren't missed that much. Out of sight becomes out of mind.
Not that she threatened this, but an example is Gina Marie. I personally think she is an incredibly gifted writer of erotica. But, she got involved in her own website for authors, and also working on non-erotica, which took up a huge chunk of her time. I have a login to her website, but I almost never go. I've been interested in Tangent, but I haven't started to read it, and won't until it is fully published here on SOL. It's gotten to the point that I don't think about her when I think of my favorite authors, because she had become "out of sight, out of mind". I still think she is one of the best, but when I come to SOL daily, I don't look for her. And yet I desperately want to see how S&M and LAH end.
The same thing happened to NetWolf when he pulled one of his "I'm mad so I am going away" events. He wasn't missed as much as he probably thought he would be, even on his own website.
So, if the authors threaten to leave, let them. As one other poster put it, the innates are running the aslyum.
When an author is holding their prick in one hand and scrolling through their scores/downloads with the other, you can't reason with them. And you shouldn't bother trying. Just give them a handi-wipe for their monitor, and let them go jerk off on their own. They will soon get bored and come back, wanting the adulation they will only get here.
They have no real comprehension of how powerful SOL is. If they go, most won't follow them. Their stories are just a drop in the bucket, and nobody has more readers than SOL. Going to some site where only 500 people a day log in won't stroke their ego enough.
Call their bluff, Lazeez.
Just record who votes what for each story and allow the reader to see that list. Then all artificial inflation/deflation is visible and can be used to judge.
Then if you want to make it really usefull to the reader, allow them to filter specific user IDs out of the voting averages being displayed. Then the beholder gets a custom tuned scoring that will over time come to present him stories that are likely to be enjoyed by him.
As for objective meaning of votes (contests)... it is a myth that voting has any legitimacy at all (in any context).
As for the voting: I don't give a crap as a reader or writer. I commend you for not censoring unlike other erotic story sites on the internet.
I would absolutely LOVE to have the system changed so that readers can revoke whatever vote they have previously given a story, and vote again if they decide that there is a reason to. Something like that of [anidb.info] just for a quick example, though you might also implement a limit to how often you can revoke and re-vote.
Of course, this could potentially leave an opening for some vote manipulation, not to mention that I am completely ignoring any thoughts about the setup and resources required to keep track of it all. So I guess it's really nothing more than a pipe dream.
But quite frankly, very little annoys me more than giving a favourable recommendation and quick feedback (in the form of voting) to one or more ongoing series, only to find out what had seemed to be a well thought out and executed story... Later turns into ridiculously drawn out and boring pointless fluff bloated self-adulation fest where you can't even remember the names of anyone in the harem of the Mary Sue self-insert main character anymore.
Quite honestly, if the authors who threatened to pull their stories against anything other than the current 10 fest ALSO happen to be the authors of the same sort of tripe as I described above (which is quite likely, with the requisite kindergarden "I won't play if I won't win" mentality), then in my eyes it's truly not much of a loss at all.
It'd be just like the Darwin Awards, authors improving the story pool at SOL by removing themselves from it. LOL!
That said, I agree with PleasureBoy1 that it might make sense to talk out the ideas more, to make changes that decrease people's perception of the raw number of the score, and to throw authors a bone or two -- maybe have bronze/silver/gold stars for authors who have a certain number of stories or KB written above an 8 (using the proposed scores) or something like that.
I've heard the expression:
"Let no good deed go unpunished!"
Now one can see how it might apply...
Thank you for your efforts on our behalf.
LaMonte
If the new system is (was) harsher, then I say it should go ahead. If authors threaten to leave, let them. They will probably come back anyway once they have cooled off. Storiesonline is still the best place on the web that I have found for stories. It is also very user friendly.
Lets face it, we all know the type. Everyone has someone at work who whinges and moans all the time, repeatedly saying how they are going to leave and get another job. Yet these individuals never do. In fact they are normally the last to leave a company (Death, retirement or sacking is normally the only way you get rid of them.)
As such we should treat them the same way we treat them at work. By ignoring them and fervently hoping that they do in fact leave.
My stance may seem harsh, but as a writer on storesonline (19 stories and counting) I don't want to be molly-codled. I welcome an honest and fair scoring system. I see it as an aid to improvement, not fuel for a ridiculous ego.
And before anyone nitpicks at my story count, I wrote five stories under a different name. Because, perversely, I did not trust my story scores (they were too bloody high!) and I wanted to start afresh.
In fact, to be honest, If the situation canot be remedied, I suggest the scoring system be removed completely. Lets be honest here, It's pretty much useless as it stands at the moment.
My rant over.
Pixy
While I can appreciate the feelings of some authors that they would dislike seeing lower scores than they are used to receiving on their older works, they should consider that receiving more balanced scores, would, with intelligent feedback, both point them in direction of better writing and more of a target to shoot for in improving the scores they receive after implementation of the new system.
All that aside, let me thank you for the care and energy you put into maintaining the best story site I've found online.
Like at least one author has commented here, the only way you are going to get a new system to stick is to draw a line in the sand (say 01 January 2007) and say that all stories with at least one chapter posted before that date use the current (old) system (even for voters who read it later than this). All stories posted after this date use a new system.
Make the new system noticably different to the current one so that it is obvious that the score is not just a read across. The easiest way that I can think to do this is to make the new system score out of 20 rather than 10, this also allows more range of expression. My off-the-top-of-my-head suggested scores would be:
20 - Perfect, impossible to improve
19 - Almost perfect
18 - Outsanding
17 - Excellent
16 - Extremely good
15 - Very, very good
14 - Very good
13 - Good
12 - Alright
11 - More good than bad
10 - Neither good nor bad
9 - More bad than good
8 - Poor
7 - Very poor
6 - Bad
5 - Very Bad
4 - Very, Very Bad
3 - Extremely bad
2 - Atrocious
1 - Nothing good about it at all
This should please authors with egos as they'll be getting bigger numbers. You'll need to display the scores as what they are out of, so people don't think a old-style scored story that got 9 is poor. 9/10 and 9/20 are very different (perhaps also colourcode the scores - /10 in green /20 in blue - for even more differential). An author should not have any complaints about their story score changing from "9.71" to "9.71/10".
Another suggestion is to categorise all stories into one of two types "stoke" and "plot". A stroke story that gets a 9/10 gets it for a very different reason to a plot story that gets a 9/10, and its a bit like comparing chalk and cheese.
A third suggestion is to allow more people to give stories review-type scores. Don't give it to everyone, but allow people to request the ability. Restrict it to people who have voted on at least 20 stories since they registered. If you need the money then maybe restrict it to preimier members (it might get more people to sign up). These people will also vote in the normal way, but will provide more detailed feedback to authors (allow an author to opt-out of receiving this type of feedback if they don't want it perhaps).
2- Any method of adjusting scores on a curve or against an average ASSUMES that half of the stories are below average quality. A word type score (vs just a number)bettershows the overall quality of stories on SOL.
Story scores have become pretty much useless, except as ego-inflation devices.
cmsix, what will you do when the AVERAGE story score is 9.98? Will a 9.99 satisfy your craving?
Perhaps a new scale to reflect the top-end compression would be appropriate:
10.00 -- cmsix material
09.95 -- superb, cannot be improved
09.90 -- astonishingly superior
09.85 -- superior
09.80 -- merely superior
09.75 -- very excellent
09.70 -- merely excellent
09.65 -- almost excellent
09.60 -- very good
09.55 -- good
09.50 -- don't bother
But then again, why bother at all?
cmsix, whose works I have enjoyed for the time I've been a member here, and the many other authors who I envy and enjoy (to the point where their work convinced me that a premier membership was essential to support this outstanding site!) has persuaded me with his outbursts to change my scoring approach.
Anything below an 8 is meaningless. A 9 is simply average, and a 10 has lost its meaning through inflation.
There is now little point in voting. So I will no longer vote.
Instead, I'll "vote" with my comment to the author. I have been doing this frequently, and will now do so more frequently.
And for those authors who feel that spelling and punctuation and proofreading and such are not so necessary ... I'll send along a comment about that, as well.
Gray
I can tell you cmsix and Asa Strong won't be be getting feedback from me, either!
Lazeez, thank you again for putting up with some of the kindergardeners you have for authors. I think your site is the best of its kind on the internet.
kandlbecker
I always thought it was a guide for the potential reader, not as a reward for the author.
As a reader, I just want to be able to see if others are liking a story or not (so that I can decide whether this will be one of the 2 or 3 stories I get to read today).
I think the 10 point scale is too detailed (an 8 out of 10 does not mean the same thing to a lot of people).
I would prefer to see a simpler voting scheme, sort of...
Do you recommend this story ?
-Yes, read it.
-Maybe, if the codes appeal to you.
-No, skip it.
Even if the score inflation could be avoided completely, a ten point scale leaves way too much up to interpretation.
Let the reviewers have the 10 point scale, where they can give detailed info as to why a story is an 8.
2.} I admit that I rarely vote any thing less then an 8 because if I have been willing to follow a story through 5 or even 100 chapters and have kept reading it has been that good a story. if it isn't that good I will have quit reading before its complete. I guess it shows I'm more interested in the multi-chapter stories then the shorter stories in most cases
If there was a way to display the drop off of readership from the first few chapters to last few, this would be of more assistance to me in deciding to read something then the present system. The only other way is if the voting could be revised to reflect a vote per chapter and also a composite score for the whole story. then I could vote as a story is posted knowing that if the author starts posting three line chapters or making some strange illogical twist, i have recourse to reflect my lack of satisfaction.
While i should probably score a poor score to a story i am abandoning I don't feel it fair to score an incomplete story even if it has lost me.
Now if the scoring is primarily for the authors satisfaction then leave the system alone and let the authors be happy with their high scores many readers may not trust.
In any case irregardless of the scoring issue it is an excellent site
I am a premium member and think it is worth every penny to support a site that has given me so much pleasure over the years.
gmax
Here's an idea, though: for stories with multiple chapters, tie the scores to the number of chapters read by each person. It stands to reason that we will read more of stories we like.
And screw the annoyed authors. I like complex, regressive, cool weighting systems, and I LOVED the new wording on the voting forms.
Just my 2c.
--CircumSpect
I liked the new scoring system. The median of the scores needs to be lowered. Screw the people that don't want changes done.
I commend your efforts to bring sanity to a scoring system. I have stories on this site, and I don't give a good damn about numbers. A note from a reader is worth a hundred votes anytime, and perhaps if the reader didn't like the content he/she gave it a low score. I would urge those people to read the description and the codes.
What I would prefer is a way of separating content from execution from sensuality. Maybe "Did you like the story", "Was it well-written, grammatically correct and/or readable", and lastly "Did it make you horny?"
I don't pay any attention to scores when choosing a story to read. I read the blurb and the codes, and if they pique my interest I'll try a few chapters.
I like well-developed characters who are real and human, with real feelings. A nice plot, not too thick, not too thin - kind of like Grandma's vegetable soup. And realistic sex - no twelve-inch dicks and gaping vaginas on 9 year olds. In short, something that could have happened in real life and a good author has let me share those moments.
I realize not everyone likes the same thing, and that's why different authors write different stories, but their common ground is content, execution and sexual excitement. Perhaps give each sector a maximum of 3.5 points, then add the total?
I don't know how to score effectively. If I really like the story, I shoot an e-mail to the author and give it a ten. If I don't like it, I don't bother reading it, unless it's truly horribly constructed. Then I will e-mail the author and encourage them to enlist the aid of an editor. The story is there, the presentation is bad.
Rest assured no matter what you do, Lazeez, someone won't like it, but I appreciate having a forum to post my stories, little-read as they are. I will continue to read and post and add my voice whenever possible.
Thanks for a great site and all the work you've put into it. I do hope you think of a way for common-sense voting to occur.
Sincerely,
Janna Leonard
As has been said in many forums, high scores on SOL seem not to represent good stories.
Often badly written, often with poor grammer or spelling, characterless or stroke stories score highly while those which are better developed and well edited score less well.
The comment by someone of having an authors/reviewers only scoring system, perhaps alongside the present system, would be a better guide for those that want a good literary experience and not just something to use to keep their right hand busy.
If you do this, I hope it will include all authors, not just those who've written reams hosted on your site.
As an ego-stroking device for authors, the voting system isn't broken. (Except for those authors who are bright enough to figure out that outscoring the average story on the site by .07 isn't very impressive.)
But for readers, votes/scores are meaningless. I've personally encountered too much crap to believe that the 'average' story on the site is near-perfect. The problem is, to be fair, all scores would have to be wiped out, and start fresh with a new system. Preferably one that limits the number of 10s that any one voter can issue. Or one that just lets everyone on the site rank their favorite 100 stories, and does some BCS-style manipulation of the data to generate scores for stories.
There were some good and kind authors who told me to get better, as my story was worth writing. It is for these people that I am still writing today.
What has this got to do with scoring and the huge debate over it?
1. If you don’t like changes to a site where you can post your story for free and get all of the benefits you receive from SOL, then you probably should leave. No one should be that big headed. If you are that fucking good, try and get yourself published in print.
2. There is a problem with scoring and those that oppose changes to that are the ones that are making the problem. You heard me right, YOU are making a problem. It does not help the readers to see a raft of stories that are all above 9.7. What happens when everyone just starts voting 10’s for everything, because, isn’t that what the authors want anyway? If every story was scored a 10 then why even have the scoring? Maybe we should have it hidden from view; so that the authors only see it and they can feel good about their score, without realizing that everyone else is getting the same thing.
3. So why not start with every story being a 10 and then as the votes come in, it will lower it? No, I’m not being serious, but it does make you wonder where the scoring is going, doesn’t it? Here’s an idea, just like the supermarkets, we could pay for placement of our stories. We could buy a higher score, buy it’s placement on the top 20 list. Extra income for Lazeez and the author who needs that recognition can have it. Ok, that sounds a bit to much like the beauty pageant scandals, but it is an amusing thought.
4. I think Lazeez took the message wrong. Most of us that didn’t complain or comment about the changes, didn’t do so for a reason. There is a lot of hard work being done on his part and we appreciate his work, and the site. We didn’t know that, after the said changes were going to be made, that we wouldn’t need to encourage what we thought was going to happen. Those that didn’t want it, thought, were indeed vocal about it. That was predictable. I was planning to roll with whatever happened with the scoring changes. It was being done in the interest of the reader, and not the author. Authors should take note of that, because if this site only makes the author feel good and the readers don’t trust it, they will go away. What good will it be if we post, but no one is there to read? The authors can go to another site and then Lazeez will have an empty site that he has worked hard at. This happens in business and I can see the same sort of thing happening here. If you, as an author, are too pig headed that you would rather have it your way, no matter what, then how is someone supposed to respect you? I know people like that and I don’t care what they say. Their opinion is not counted, in my book, as the only person they are concerned with is themselves. (See point #1 for my view on where you should go)
5. After all is said and done, one person said it well. Everyone is taking this too seriously and tempers are getting out of hand. My view is my view. I’m expressing it, because others are and I felt that I had something to say, that was a bit different. I am not out to piss off other authors, although I may well have done so. If so, they need a better sense of humor. After all, if they can dish it out, they’d better be ready to take it. My biggest concern, and I hope I have stated it well enough above but it bears repeating, is this: the health and well being of this site and the ease and enjoyment that readers who come here, is and should be Lazeez’s paramount priority. Without the readers, where would we all be? We know the answer to that question. We’d be on another site complaining about other issues!
It's probably a little late to chime in on this (in fact I had not been checking SOL for two weeks because of RL commitments and missed the whole thing), but a thought did occur to me. No matter what scoring system is put into place, it will always be skewed towards higher scores. The simple reason is this: People who don't like a story are not going to finish it. They're going to stop in the middle and move on to something else and thus never score it. This leaves you with the people that actually like the story and thus are more likely to finish it to the end and leave a higher score.
So, really, I'm at the point now where I don't care what scoring system is in place anymore, because it gives, at best, only rough estimate of what people think about it. The emailed comments tell you a hell of a lot more than the score does.
-- AStrangeGeek
I've also been copyediting/proofreading for dstar for almost a year now, so it's not as if I'm unaware of authors' concerns, though I don't write much myself. (I need to change that…)
I must say that the current scores are, as has been said by others, essentially useless outside of a very narrow range. The proposed scoring system really felt like a temporary fix, not a proper solution, though the only real way to find out would be to run with it for a while. Having said that, here's my suggestion (which is mostly an amalgamation of many other people's suggestions):
1) Use a system with fewer ratings for 'casual' readers, i.e. "Wholly Reccommended", "Good if you like this sort of thing", and "Bad" ratings. The wording of these could, of course, be changed, but I think that any more than five ratings will result in score compression anyway. I don't think I've actually voted lower than an eight yet, though that's mostly because anything lower than what I'd give a seven I wouldn't get through—I like long stories, and I can read about ten chapters before I'll give up. All of these would fall into one of the top two categories, so I'd argue that a five-point scale where one is 'bad' and five is 'wholly reccommended' (and the average is 'good if you like this sort of thing') would be more effective. This still doesn't help the troll/cheerleader effect that much, but it means that people can really only say 'good/average/bad' which seems to be the current practice anyway.
2) Have a more elaborate system that people can opt to use which takes more into account than an essentially ineffective single-axis rating system; fiction has at least half a dozen aspects that contribute to quality, so some sort of weighted system that evaluates those would help. As has been suggested, some sort of 'literacy test' would make scores from a system like this more reliable; pgdp.net (which does distributed proofreading for Project Gutenberg) has some similar requirements for getting into the later stages of the process. A reasonable threshold number of votes and a 5-15 minute test should work, though having someone essentially spam votes across a number of stories to get to the test could be a problem. This would also give authors better feedback on what people think needs work, without requiring a written comment; I know I don't really want to write someone a comment saying "Go get an editor, you need one!", but scoring their story highly in the less-technical aspects and low in the others is an indicator of that sort of thing. Sending a comment that says the same is potentially more work, but if I'm interested in looking for more editing work, I could look for stories that have that scoring profile, check them out, and send the authors of the interesting ones an email—this isn't at all possible with the current system. The downside is that getting an editor and solving those problems would make the score a poor reflection of the current quality of the story, but if it's a serial, some adaptation of (5) would have to be applied; a re-post of a shorter story might be set up to lock the current score and allow new votes, or something similar. Also, those of us who are picky about grammar and usage would probably appreciate such a score; those who aren't could ignore it, of course.
3 & 4) PleasureBoy1's batch-download voting suggestion, and the bonus tags. The exact mechanics of the bonus tags might be a bit difficult; displaying them for anyone who got one checked would be essentially useless, but a useful threshold might be difficult to determine.
5) Some way to make voting fairer for stories that aren't posted in toto. There are really two solutions possible: revoting, and a one-vote-per-chapter type of solution, which would require some form of weighting scheme to produce a final composite score. Some way to show scoring trends for long stories that are in progress would be very useful here, as well. For the revoting solution, I'd suggest showing the score that the reader previously gave, and allowing them to change it each time a new chapter was posted. Revoting would require more resources, and might not even be useful if you switch to a trinary voting system from the current decimal system, as suggested in (1).
6) Some form of aggregate score per-author (Another PleasureBoy1 suggestion); I favor weighting by length, maybe with some sort of down-weighting for stories in progress; ignoring inactive stories is probably a good idea here. Having the 'bonus tags' displayed per-author would also be nice; some reasonable percentage of an author's stories would have to have earned a tag for it to show up, almost certainly somewhere in the 25-75% range.
Please note that I haven't gone through and done any sort of analysis of how what I'm proposing could be abused, aside from a few simple observations.
One further suggestion that I've noticed people making would be to keep the current scores displayed for a while, so people can see the difference that the new rating system makes. The new rating system, if all of these suggestions were implemented, would involve a display of a 'reviewer' score, the general readership score, and a few icons; it shouldn't be hard to fit this into the current display scheme. Color-coding by score-relative-to-median is an interesting concept as well, but that could be added later if it is found that the median is appreciably higher than desired.
My only other observation is that many people are saying that they don't vote if they give up on the story; some way to allow people to vote on the stories that they've read but not voted on yet would be nice…maybe only tracked for the last two days, or something. (One day is probably not fair; I tend to be reading around when the download counter resets, just because I'm up around that time.)
Ok, that's a bit over a thousand words...I think I need to let someone else talk now. ;-)
Quickly looking at the scores, 8.4% of stories are scored 7 or below, 17.4% between 7 and 8, 44% between 8 and 9, and 31% above 9. This basically says to me that, if we assume 25% bad and 25% good, we've got a three-vote system going that's reasonably effective. Assuming reliable scores, that essentially means that anything above a 9 is pretty good, and anything below an 8 isn't worth reading. I don't see any real way to guarantee reliable scores, but having ten steps in the voting scale doesn't seem to be helping any. Allowing more precise vote placement in the spectrum will only make the problem worse, not better. Less is likely to help, but, as always, there are no guarantees.
Having a multi-dimensional rating scheme (as suggested in my (2)) is really not something that would be useful unless lots of people were using it, and I have no idea how many people actually would...at a guess, nowhere near enough to make it worth implementing, though I do enjoy building systems like that. The balancing is the real problem.
Here's a variation on old and new scoring systems.
1. freeze all old scores and display them.
2. adjust old scores (by the method or similar Lazeez proposed) and display them
3. use the new score system and display them
4. Make sure (3) is set up of different categories that measure technical aspects (grammar, typos, etc) as well as content(plot, story idea, story development), and categorical and an aggregate results are displayed.
5. devise a system for bonus points or recommendation system (by other authors, reviewers or some people selected on the basis of valid criteria such as a voluntary proficiency test), which may reflect more balanced presentation of relative merits of stories, and perhaps reduce voting abuse (by trolls + cheerleaders).
6. AFAIK, My Library, favorite stories, favorite authors functions are only available to premium members. Maybe a new list similar to those should be created if it doesn't exist, or a few of the existing lists should be made available to all users (something like favorite author, favorite story, recommended stories). These should be used as a basis to make up "Top XXX" list and/or for bonus points and such.
First I think Lazeez should set up a poll among authors and readers (separately), whether old system is of any use or not. This should put to rest the issue of how much of value or use the old scoring system. Then follow it by polls on proposed scoring system changes, and whether to keep old and new system running together for a while.
Earlier on in one of my messages I proposed that the voting should be tied to feedback or something... and after thinking it over, I wanted to ratify it a bit.
If someone is going to vote, then they should be forced to write a short note that is explanatory relating why they voted the way they did. The message should be something factual such as "Too many typos... I stopped counting after the 7th typo. Awkward sentence structure... Tenses are mixed up and difficult to follow," and similar sort. These could be easily verified by a quick look. I mean, an author knows his work, and there're also authors and editors, reviewers and such, who may be called upon to referee, but I doubt it would come to it. If, for example, you get a low vote because the voter claims you have 4 or 5 typos in the last chapter voted which consist of more than 15,000 words, that's an unreasonable low vote. OTOH, same number of typos in 2000 word chapter tells there's some problem. And if the voter has made a misleading or wrongful statement with respect to what is supposed to be factual, then the vote should be discarded.
The message could also be somewhat subjective, such as "I didn't like xxx because xxx." This one is the more difficult item, because it's easy to cheat and put in some double-talk or generalities, instead of something more substantial, something that one could point at in the story and say, "yeah, ok! it's a valid point!" or conversely, "There's no such thing in the text, so your allusion is invalid!" Perhaps, a good way to deal with this last very subjective type of messages would be to allow the author reject or accept the vote (I will come to that particular mechanism shortly!)
OK, let's talk about vote discarding!
I think it should be under the control of the author, since he knows what he put in the story, and is in a position to raise objections to a particular vote, especially for the factual type of voting reasons! Also some system of verification could be set up involving other authors, editors, reviewers, or even readers who has read the chapter. Besides that an internal check mechanism could be algorythmically established very easily for votes by comparing them to the avreage score of the story and average score of ALL stories and determinign the deviation of the vote from those values. Perhaps this might even be used to flag the particular votes, before the author gets involved in a process of objecting rejecting the vote!
Furthermore, I think with compulsory commenting for votes being cast, the list of comments and votes should be made available publicly. And here's another variation and possible solution to the problem of objections /rejections from an author. In the list where the votes and the accompanying comments are being displayed, the author can accept or reject manually certain votes and let an adjusted average to be displayed. Since msot such objections would be related to low votes, I would expect the list to be sorted by lowest to highest vote (with their accompanying comments from the voter). A reader can then look at the votes rejected and browse through the related comments, and might get a good feel for whether the author is being selective with his rejections or very subjective (oofhand refusal) or somewhere between the two. And I think this might help force authors to be more objective and honest (as wella s forcing the voters to be more careful), while presenting a "best of both world reasonable" average score as an alternative to the scores we're seeing now.
In fact, if such a system is adopted, and both types of scores (all votes accounted for vs. adjusted score where some votes are being discarded by an author), I bet you the readers will be mighty curious why there is such a discrepancy between all votes and author approved votes. A quick look at a table of votes and comments and the status flag (indicating author's rejection) should be very telling!
Personally, I would love to see a 10 vote being accompanied by a "thought provoking and very deep" comment for a stupid stroke story which has nothing going for it. Should be good for a quick laugh if nothing else! :-)
To be able to implement such a system where individual votes and their comments are tracked, the voter should be identified. Perhaps by issuing IDs by SOL to readers like certificate (at least to protect privacy). This would also help identify abusers; those that have majority of their votes being rejected by most of the authors for whom they vote! A resulting action might be either an automatic process that immediately puts all votes for all their stories into a a rejected status (perhaps such an indicator could be made next to the voter to alert both the readers looking at comment + voter + vote list of a story; which would also help the author to see that the particular voter is identified by the system as a possible abuser!) Is it really difficult to add a flag next to a user to specify their voter status such as "can vote", "all votes are flagged", "possible problems" or some similar manner, basd on rejections vs number of votes cast?
I think such a system would be very effective in culling out the trolls and bring back some kind of order to the current situation, and it has at least a sort of relative checks and balances, heavily factored in the author's favor (but for the discriminating reader, they wll get an idea about the author in how reejcts votes and such and make their own determination in regards to how an author's preferred score is formed and how reliable it is). Best of all, the system forces the voter and author to apply some self-restraint, self censure!
Another aspect of a list of votes + comments is that the comments could act as a sort of recommendation, if the author is given the option to flag some of the comments bhecause he thinks that those comments (with theri associated votes) reflect some aspect of his story the best, or more applicable as a valid comment about his story. Such flagged comments might make it into a mini-recommendation list that a reader can peruse. The author could be limited to choose only 10 or 20 such comments for the mini-recommendations list! Of course, the vote + comment list should be sortable to alolow the author to make his selections so that mini recommendation list could be generated.
Links should be made available for vote + comment list, mini recommendation list, and each story should have its scores and author-adjusted scores in various lists!
My suggestion-- apology if someone has said it before-- would be to put both scores up, the absolute score and the weighted score. I would find the latter more interesting (although I pretty much estimate it anyway when I see the absolute score). But in the end, any broadly-based voting system is going to reflect the broadest taste, the most basic Penthouse-style tale of lucky seduction out of the blue, set in a modern realistic setting; those of us who like to explore other things will simply have to be content with lower ratings, fewer scores, and the occasional piece of fan mail telling us that our story really meant something to somebody (and made them horny).
In any case, thanks again Lazeez for everything, and shame on folks who turned petulant and threatened to withdraw their content if the scoring system changed. I'd like to know how many of them sent you a note of thanks when the decision was reversed. I bet I can guess.
Would it be possible for us to have the original score, and the new weighted score, each in its own column? And use the new voting system so that the votes show up accordingly?
Then, after an appropriate amount of time, we can see what kind of results we get are.
Please, be serious. If all scores reached a 9.9 plateau, he wouldn't be happy unless he had a 10.3!
Well, I never thought I'd write this, but after observing all the commotion since the advent of “Derailed”, here I am.
I've been involved with SOL for just under three years. First as a reader, then author, then (briefly) a reviewer. I don't think you can find anyone who's had anything to do with the site in any capacity who thinks Lazeez is bad webmaster. Even those of us who have been complaining that he's been out to get us with the scoring change. But he is a webmaster. He's not an author or a reader when he's being a webmaster. His job is to create a successful site.
As an author, I take pride in my work. It may be a work of love, but it is work. If you don't think so, sit down at your keyboard and write something worth reading. That said, I feel everything I write deserves to be rated somewhere between Very Good and Excellent. 9 and 10.
As a reader, I'm primarily interested in stories that are Very Good and Excellent. The first thing I look at is the description, then the score, then the codes. If the score is between 8.5 and 9, I take a closer look at the description and a real close look at the codes. Below 8.5 and it will usually get tossed.
Bottom line – the current scoring works for me, both as a reader and an author. And I've pretty much boiled them down to
WORTH MY TIME
MAY BE WORTH MY TIME
NOT WORTH MY TIME
I don't use means, medians or anything else. I've learned to work with what's there. And I may have made some subconscious adjustments along the way as the scores crept up.
Next point – Lazeez has made many changes to the way things were over the past few years. Some I didn't think were needed or flat out disagreed with at the time. I can't think of a single one of them right now. He puts a lot of thought into what he changes, there's always a valid reason for it and it always seems to work. And if there's a bug or two, it gets handled.
Next – You force people to do something in order to vote, they won't vote. Giving IQ or voter proficiency tests, forcing a voter to justify his vote, even having multiple categories where he has to rate stroke, content, plot, grammar (ar, not er); any of these improvements will improve the votes out of existence. And I can't recall any author requesting fewer votes in his blog.
Reiterate – It's HIS SITE. It's not a popularity contest. He saw what he thought was a valid problem with HIS SITE and told us what was happening and why. And then he opened it up to discussion. That was his first mistake. The second was trying to convince people of the rightness of doing it. The third was bowing down to it. It's a fucking erotic story site, not the Clinton White House. The valid measure is how many hits, members, etc. it gets, not what the vocal minority with some vested interest says. An author has a right to go elsewhere if he so chooses. Personally, I don't think any 1 or 10 authors is powerful enough to really harm the site. When's the last time Frank Downey published an update? Has the number of hits dropped as a result?
Personally, if it's going to be changed, and it looks like it needs to be, I like the Yes / Maybe / No rating system I mentioned earlier. I'd be more likely to click a “Don't Read” tab of something I was quitting in the fourth paragraph than going down to the end and hitting a 1 or a 2. The same with an “I don't like it but you pervs might” tab (that was sarcasm, you perv). Maybe institute a Comments option, similar to the Reviews option. People write short comments of why they liked or didn't like it and the potential reader can click on the tab just like he does with a review tab now. Of course, this has the potential of degrading into a reader love fest, as it has with Lit.
I said that I thought readers should be able to change their votes on multi-chapter stories in a previous comment. I hate to see the votes pile up on the first chapter of one of my stories because the voters have no idea what they're voting on. But I don't want to have no votes for the weeks or months it takes to complete the story, either. If I've already voted on a story, the voting option doesn't come up. There should be a way to have it modify my vote as I get further into the story. If not, there should be a way to list all votes versus votes made after the latest chapter. Hell, I'd like to be able to revise my vote on a single chapter story I read 6 months ago.
I don't think revising the current system is going to do something. What we had was the government saying “Your money buys too much. We're going to raise the prices of everything by 30% so it will be more consistent with statistical data.” People rebelled. If we just converted to using sand as a medium of exchange, there might be some complaining to start with but people would start digging soon enough.
The OLD FART