Scoring Changes Follow Up

I canceled the planned changes. Too many authors asked for the removal of their stories from the site. Sorry for all the trouble.


This is a follow up to my previous blog entry about the changes to the scoring system. If you haven't read that one, please check it to see what this whole thing is about.

This follow up is to address as many of the comments that have been received so far as possible.

A simple clarification: the voting system itself is not really changing. It still works the same way. It's the results representation that's changing to allow for a clearer distinction between tiny variations in the scores. I'm just shifting the median for all scores from whatever it is now, to an artificial one of 6. For example, the current top scoring serials on the site contains eight stories with a score of 9.77. The new representation would simply magnify smaller variations within the .77 bit.

As for suggestions offered, there were plenty, and that's good.

Few things to clarify with regards to the nature of the site, to shed light on why some things are the way they are and why I can't/won't change some things related.

The site gets accessed from all over the world. In most places, internet access is not unlimited. Many, many readers pay for every minute that they spend on the site. So a large chunk of the story accesses are for quick downloads to read offline. Can't force those readers to vote. Voting works only when reading a story while on line.

Many of those world wide readers don't have English as their first language, hell, I don't have English as a first language. More than half of the readers don't feel and definitely aren't qualified to judge the grammar and sentence composition of the text their reading. Can't force them to cast a grammar vote. However, they can tell whether they like a story or not.

Things not doable:

* Forcing readers to vote: Not good.
Readers should never feel that they must vote. This action would cause a lot of junk voting. It would be worse than not voting.

* Forcing readers to comment: Not good.
5% of all readers vote and less than 1% actually comment; trying to force those numbers up will drive people away from the site. Not good.

* Dropping a certain percentage of votes from the top end and the bottom end: That would require keeping individual votes indefinitely. Not doable; requires too much resources.
Unless everybody is willing to chip in for a larger disk array for the site ($15,000 +) and for the cost of hosting it ($600 per month), then it is not possible to keep votes indefinitely.

* Allowing readers to change their vote later and allowing voting for stories previously downloaded: Not doable.
The site has a system in place for blocking score manipulation. Those changes would break it completely and make scores open to easy manipulation. That's a bad thing.

* Changing voting method for an additional criteria like grammar: Not exactly fair.
Older stories that had their votes cast already would be at a severe disadvantage. Plus it would require readers to vote for multiple criteria.

* Disallowing votes for serials until their completed: Not fair.
Many authors rely on votes to give them motivation to write. No votes means way less feedback.
Plus, doing that would create even more bias towards serials. If scoring is only allowed after a story is completed, the only those who stuck with the story till then end would vote, which by default means they liked it and their votes would automatically be very high.

* Automatic vote casting for non voters: Not Good.
Since there is such a large difference between the number of downloads and the number of votes, casting a 6 for each non-voter means that the scores will never go above 6.5 or below 5.5, that's even worse than it is now.

Things Doable:

Adding an additional voting panel for individual chapters. The results of this panel would be simply sent to the author, but not displayed on the site.

Adding an additional voting panel for grammar and stuff. The results of this panel would be simply sent to the author, but not displayed on the site.

----

Thanks to Aleph Null's suggestion. He provided the solution that I needed for the new system to be more fair for older stories. It's so simple, I can't believe that I didn't think of it first. The new system will calculate the median for each year and then calculate each story's weighed score depending on when it was posted or last updated.

----

Everybody seems to think that I'm doing this as a spur of the moment thing. I'm not. I created the initial code more than a year ago. I knew it would piss a lot of authors off. After all, having your scores go down from 9+ to 7+ is a bummer.

I've been thinking about the issue and monitoring the median for the last year and a half. And now, the median has reached a ridiculous level. The effect of the extremely high average of the scores was evident in the comments posted. Many said that they don't read anything that has a score below 9. Why is that?

From the authors comments, it was clear again that the authors' expectations of the system are misplaced. Every author wants the system to be the equivalent of the film critics. Unfortunately, it's not and it can never be. It's like a poll at the door exit from the theatre.

Just look at the reviews. There are 30 people on the site able to post a review for a story. I would invite everybody to count how many reviews are submitted per month.

I tried the multi-criteria voting system in 1999 where it asked readers to rate three things: story line, quality and appeal. In its first week 8 votes were cast. Just Eight in a whole WEEK!

It was an abject failure.

People don't want to think about it. It has to be a single easy choice. Anything other than that would be used by a slim minority of those already voting.

The new display method would be closer to showing what people are thinking instead of showing what they're doing.

As for the 'Impossible to improve' option. I know it sounds ridiculous, but it's on purpose.

The reason for it is best illustrated in Stormy Weather's response:

Under the old system I rated stories 9s or 10s ... and sometimes 8s. With the wording of the new system, the stories I read will be getting 9s and 8s and 7s. With the way 10 reads now, I can't see myself giving it anymore... unless there's something out there that really knocks me out of my chair.

The new wording is meant to keep the 10 for those who knock your socks off with their work. How do you really reward those authors that put so much work into their stories and have a great creativity that results in truly great story? Do you give them the same as you're giving everybody else?

Is a 9.5 really meaningful when almost everybody is getting over 9.2?

I want people to stop and think for a bit before casting that vote.

And it seems that the new wording is being fairly effective. From a sampling of the most recently posted stories, the scores seem to be a bit more realistic.

The first two weeks after the new score display is implemented will be very rough, especially on me. I know I will hear about some extreme displeasure with what's happening, and I'm definitely NOT looking forward to it.

We'll all just need to get used to the new numbers. Lower our mental line in the sand for the new scores from 9 to 7 and everything will be fine soon.



Update:

I've been refining the system before full deployment and got some interesting numbers.

I've defined a set periods that the system will use to define which median value to apply to a story and got the following:

+------------+------------+--------+
| From Date | To Date | Median |
+------------+------------+--------+
| 1998-01-01 | 2001-07-01 | 8.25 |
| 2001-06-30 | 2003-01-01 | 8.44 |
| 2002-12-31 | 2004-01-01 | 8.60 |
| 2003-12-31 | 2005-01-01 | 8.65 |
| 2004-12-31 | 2006-01-01 | 8.93 |
| 2005-12-31 | 2006-12-01 | 8.87 |
| 2006-12-01 | 2008-01-01 | 8.33 |
+------------+------------+--------+

The 2001-07 date corresponds with when the system went from no-login anybody can vote as many times as they wished for any story, to the log in system where nobody could normally vote more than once per story.

The 2006-12-01 date corresponds with the wording change in the forms.

As you can see, there is a definite rise in the general voting over the years. the only anomaly is the difference between 2005 and 2006, the median dropped from 8.93 to 8.87. The explanation may not be obvious, but the drop corresponds with moving the form from below 'the end'/to be continued' line to above it and hiding the score of the story in the form so readers couldn't readily compare the story's existing score and didn't have as much of an incentive to go higher.

The change is needed to future proof scores. Say an author posts a story this year that scores 9.7 which is pretty good now. If nothing is done, and the scores keep creeping up, this same story score, which now is pretty good, in two years would look lame compared to stories posted in two years.

At the current rate of score creep up, in a couple of years, stories that score below 9.25 would be crap and the top end would reach 9.9+.

If going unchecked, the scores will reach a level that everybody would be forced to vote a 10 for anything otherwise it would be below everything else at the time.

And for those worrying about the old scores, not to worry, internally, the system will still work as it is now. The same scores are kept internally, but they'll be displayed in an interpretted way. I'm not sure yet, but I may show both scores in the authors stats pages along with the median that each story is compared against.

Comments

Anonymous said…
LOL- I'm sure you will indeed get some truly vitriolic comments.

For my part, I'm a bit sad that my scores will drop, but can console myself with the thought that the relative improvement across the 3 stories I have posted will remain.

I understand what you're doing and why, and I truly have no complaint. It's your site, I'm allowed to use it for free to both read and post my scribbling and I'm grateful for that. Thanks for all the effort you put into maintaining things and improving them.

Best wishes for a happy Holiday (or Christmas, is thats your religious persuasion)Season to you, Lazeez, and to all the authors who have given me the pleasure of reading their works, and to the readers for their attentions and kind comments.
Anonymous said…
I'm writing this comment as a non-writer but frequent reader and voter.

First off, I vote on almost every story I read (and I read a lot). I generally read stories scoring > 9.0 (and whose subject matter interests me), and consider such stories as "OK"; stories scoring > 9.5 I consider as "good." This metric seems well in-step with others who have already commented (on the earlier blog).

When I score stories, I rate "OK" stories as an 8. Stories I consider to be "good" (5-10%) I score as a 9, and the few (~1%) of stories I consider to be "excellent" I rate as a 10. I rarely score stories below a 7, but this is primarily because I rarely read stories that would generate scores below a 7. So, by means of the stories I select to read, I end up scoring stories significantly skewed to the higher values, in spite of the fact that I tend to vote "meaner" than the average reader.

As long as people continue to give 10's at the drop of a hat, the skewed scores will continue to occur. I'm afraid I don't think that calling a 10 an "Amazing" is not going to change how people vote.

Additionally, I'm afraid that "renormalizing" the scores will cause a discrepancy between the scores of "pre-normalization" and "post-normalization" stories, because of the reasons I just stated. If this is true, then the stories scored with the new descriptions will end up with the same scores they had under the old one, and thus "pre-normalization" stories of comparable quality will be substantially lower in score than those scored under the new system.

Furthermore, to be able to compare "pre-" and "post-" normalization scores will work only if people vote in the distribution anticipated by the renormaliztion (and which I fear will not be the case).

Therefore, I think that the old scores with the (ridiculously) compressed scoring is probably the best that can be done, because it works and it is self-consistent. I know that a story with a score of 9.5 will be, on average, "better" than a story with a score of 9.0. And furthermore, with experience I have a decent understanding of how "good" a story with a score of 9.3 will be (again, on average).

My suggestion for changes to the scoring system involves multiple-chapter stories (a concept that has been mentioned by several posters already). I would like to be able to vote twice for such stories, once as chapters are being submitted, and a second time at the conclusion of the story (ideally with the second vote superseding the first one). This is due to the fact that my scoring of a story might change significantly during the course of a story (e.g. if the conclusion to a story is particularly good or particularly bad). I find that I generally won't vote until the end of such stories (so that I can cast my "final" vote), but also rely on "early voters" to determine whether or not I should read such a story in the first place! This makes me quite the hypocrite!

The easiest way to implement this (from Lazeez's POV) might be to have two seperate scores for a multiple-chapter story -- one "in progress" score, and one "final" score. I hope this isn't in the "impossible" category.

Finally, I've read many comments by authors who indicate that they use the score as a metric to determine how the readers feel about their stories. While this is obviously very important, it is also important to remember that is the primary system used to pick out interesting stories to read. This is particularly important for new visitors to the site, who will not know which authors they are likely to find most appealing to them, and as such will use the score as the only metric by which to screen out likely stories.

I'd like to thank Lazeez for his constant efforts to improve his already wonderful site!
Anonymous said…
Well, now I understand much better what you're up against. I just thank you for all you have done. When I can afford it I will be getting a premiere membership. It's the least I can do to support your fantastic site.

I hope you can survive the coming storm as you try to implement your ideas. I know I will just have to adjust my scale of how I view scores.

You have given me a lot of food for thought. My voting will become more carefully considered.

Good luck Lazeez.
Anonymous said…
As a relatively new writer, I need scores to keep me going. If there is a sudden slash of all the scores of my stories, I'm not certain what I'll do, but I do know Ive been thinking seriously about moving to a site where feedback is common and open. There are no votes at the other site, but there don't seem to be as many thugs either. I think the owner has found a way to keep them away somehow. There is a new one that does that and I've been reading it with the idea of posting there. Perhaps when I get more confident, I'll come back, but for now I won't be posting any new stories here. Be seeing you and best of luck, it was a great site for me to get started on. Thanks.
Anonymous said…
I'm sure you're going to deal with a lot of crap from people Laz. I'm sorry for that. While I'm not happy that my scores will be dropping, I love this site, and will absolutely continue to post here. Your work is greatly appreciated by a lot of people, authors and writers alike.

I don't write just for scores. What they tell me is if people like what I'm writing, but I've found that I write what I write, and if people don't like it well... that's that. I will go(and in some respects already have gone) commercial some day. I count this site as a major help to me in the regard, since I can practice here, and play with concepts that amuse me while (hopefully) improving my writing.

It's a tough change, but it's your site, and I thank you for the chance to post here.

Dark Pen
Anonymous said…
Hmmm. You said you'd be using median by year, but this seems unfair. The median this month will be lower than the median during the rest of this year due to the changes.
Anonymous said…
Speaking for myself, I seldom vote for stories unless I really like them and I never pay any attention to the scores. If I really like a story, I may comment on content and grammar and express my appreciation for a job well done. I seldom give negative comments on a story unless grammar/ spelling is so bad that I can't continue to read it (which unfortunately is all too frequent an occurrence).
The current system has seemed adequate to me, but your proposed changes will work for me just as well...
Anonymous said…
Whatever happens PLEASE don't loose the old scores. If worse comes to worse I would like us to have the option of returning to a system we have come to accept and understand, faults and all. Better the Devil we know... .

Because I fear and hate change I have not commented on this issue until now, and this is only to request that we be able to return to the 50's where and when all was good and Made In America..huh?

Anyway Lazeez, only history will tell if you were a true seer or a fool who shot himself in the foot.

With admiration for the job you have done these many years, Merry Christmas.
Anonymous said…
Shakes Peer2B here:

Like most people, I'm uncomfortable with change. Also, like most people, I can adapt.

A scoring system that actually creates a point spread would, on an intellectual level, be nice. Seeing my scores go down would take some getting used to - BUT it occurs to me that having a more equitable distribution of scores might have a beneficial effect for authors.

Most of us have experienced having someone who really hates our stories going through and hitting us with very low scores that bring the overall scores way down. With a more even distribution of scores, it seems to me that this would be much harder to accomplish...

Just a thought...
Anonymous said…
In all honesty, what you plan do to the scores is simply manipulation. As things sit sit now, the scores mean little to me unless I am the recipient of a troll invasion where several people vote low out of spite. In other words, I'm used to seeing the scores on my stories bounce like a yoyo.
I honestly doubt if anything you do will really have any long lasting effect. I expect that inside of a week, the trolls and slash artists will have conjured up a new way to 'beat the system' and in a month, the 'nicer than nice' guys will have found themselves some reason to flood some writers scores with 10s.
Being somewhat pessimistic, I'm afraid that 'grading to a curve' may well have the same effect it has had in the school system. There, all it really did was increase the scores of the mediocre students and encourage them to remain below par.
I'm sorry, but having taken the time to think about this, I'd suggest leaving the scoring system alone. However, this is your site and we readers and writers are your guests, so do what you wish. I doubt if it will hurt anyone in any major way, but don't be surprised if inside of a year, human nature hasn't defeated your changes.
Anonymous said…
As a very long time reader and fledgling author, I understand there are inequities in the current system of scoring. I too, tend to give 10's to my favorites but, more importantly, I give feedback to the authors.

As an author, I look for the feedback. If someone cares enough to actually contact me about my story, I listen and, if possible, respond.

Sure, I like to see a high score for my story but if I had my druthers, I would hope for feedback.

We will get along with whatever system is in place and adapt. Just keep reading!

Steve Rust
Anonymous said…
Progress is good. .

Because I have given up on all but long stories I have not had to vote recently as I do not vote until the end. A chapter by chapter vote is good but how would it score before the end? As I have said before could there be a voting form on say chapter 2 which gives a work in progress vote to encourage (or otherwise)the authors then nothing until after the last chapter when a story's score goes back to zero until votes are again cast?

BTW how about a donation page - I don't feel the need for Premier membership but would not mind sending you the odd piece of gold, silver or copper from time to time?

I have only given the top score once as things have to be subjective as in
Highly recommended
will read again
enjoyable
worth a read
nothing special
Killed time
have seen better
not worth reading
not worth writing
Ugggh!
Anonymous said…
I've read both entries about the new scoring system, and I'm not really sure how it's going to affect things, especially in terms of recomputing a story's rating based on a composite median.

I'd strongly suggest that, for a interim period at least, you display a story's unweighted score (i.e., the one it used to have, or would have had, under the old system) along with the newly adjusted one, so that the unwary reader can have some time to adjust to what the new score really means, in terms of story quality.
Anonymous said…
I didn't mean to imply that I gave 10s as a regular thing under the old system. I only gave the 10s to the stories that knocked me off my seat and that's what I'll continue to do; with the same rare frequency as I've always done. And will end up giving fewer 9s and more 8s and 7s because of how things are worded, which seems to be the goal of the system.

Stormy Weather
Anonymous said…
What bothers me about this whole scoring hoopla is the idea that we voters are idiots, and so some mathamatical system needs to in place to convert our scores into what they would be if we weren't so dumb.

If I vote a story a score that's the score I want posted, not some statistically engineered, bell curved median.

What the hell, if my vote is going to be manipulated anyway, why vote? I think the Commies invented this form of voting or worse, the French.

Joe J.
Anonymous said…
Will a troll vote of '1' have a larger or smaller effect under the new system?

SN
Anonymous said…
Thanks Lazeez,

I have been reading SOL since 1999 and always used the full range of votes, but often felt out-of-the norm for giving a moderately good story with problems a 6. Generally this kind of story would average the high 8's. So I am in complete agreement with your approach. Thank you for your efforts to make this a more relevant system.

As for English not being your first Language - geez, could a fooled me. Considering your obvious webmaster skills, I am thus forced to conclude you are a bot--after all the only language you appear to master better than English is HTML! (Just kidding :>)

We all stand in awe of your achievements. Thank you,
kandlbecker
Newsman said…
I would recommend that you do something to prevent multiple votes from the same IP address under multiple acocunts within say 1 hour. That would help prevent frudulent votes. Sure it would cause problems for people who share an internet connection with their spouse/SO, however, I don't see that as a huge problem.
What I *do* see as a problem is when people set out to manipulate the scores and create multiple accounts with multiple email addresses. Considering how easy it is to get multiple accounts from Yahoo, Hotmail, Google, etc. it would be child's play to create multiple free accounts, and thereby allow multiple fraudulent votes.
Preventing multiple votes from the same IP within 30 minutes to an hour would allow people who share an internet connection to vote, but reduce the fraud.
Anonymous said…
I guess my complaints are the old tired ones that cause me to cringe and want to jump out of a story when I see someone use "their" for "they're" (check your intro Laz) or "loose" for "lose" (this one seems to be getting more common). These are the types of things that really detract from stories in my mind. Is it because people don't like copy editors or that there aren't enough editors? I know there are some ESL issues, but when I see word transpositions, I want to vote a story to the basement.

I agree with most of the comments in that I try not to read the garbage on the site and tend not to vote on stories where I can't stomach the excessive typos.

I also feel frustrated as when providing comments because less than half ever get a response. I realize authors are here to write, but if you don't acknowledge comments, you discourage people from making comments. I don't necessarily want a long personal email back, but a thanks for your comment message would tell me that you at least got my message and bothered to read it. Knowing that, I'm a lot more likely to send comments in the future.

I like the voting system, as flawed as it may be. I spend most of my Internet time here and appreciate all of Lazeez's efforts in maintaining a responsive community.

To all the authors, I thank you for your efforts and hope you will continue to write, post and respond to reader comments.
Anonymous said…
In answer to Shakes Peer2B and his comment about low scores counting for less, that idea is dead wrong. Effectively Lazeez is placing a negative bias on ALL scores, so under the new system a score of 1 has the effect of being a vote of approximately .76 or even less. What that effectively means is that the slime who vote a 1 to knock an author down will have a heyday.
Anonymous said…
I hope you don't get flamed. It is your site and I have used it for many years. Whatever you decide will be fine with me.

Tom
Anonymous said…
I have seen the score of my story, A Different Kind of Lifestyle, drop significantly over the past twenty four hours. One might think that I would be upset, but I'm not. I wrote to Lazeez about a year ago concerned about score inflation. I'd say that he found a solution. Others may not be so understanding, but in the long term it is a good thing. All I have to say is -- Good work Lazeez.
Anonymous said…
I try and take the time to both vote and write a short note to the author about their story. I have tried to send a note to almost all of the long stories that I have read.

I also tend to vote higher than that I probably should but the reason for this is I am not paying for the stories and the author is not being paid to write the story. I feel that I can give the author a bit of slack in their story for all their hard work and effort.

I also tend to not vote for poorly written stories since when I start to read a story if it does not catch my interest I do not continue or vote on the story. I also do not vote on a story where I just do not like the subject manor.

Thanks to all the authors for all your hard work and effort in writing your stories and taking the time.
Anonymous said…
Sometimes I read a story that in my opinion doesn't merit a nine, for example, but is better than an eight. Is it possible to have an 8.5? Just a thought confuse the issue.

Lazeez, perhaps hard feeling will be soothed if you double what you're paying everyone now ;-)
Anonymous said…
By the way, the first story I ever posted on SOL received a rotten score. Tom Land wrote me and suggested a rewrite and ways to improve the story. I followed his advice and the score improved immensely. I've never forgotten his kindness to a new writer.
Anonymous said…
Throughout the discussion on this topic, many times, scores of 1 and "trolls" have been mentioned.

Honestly, I haven't often given a story a score that low - I think maybe twice - but I have voted less than 5 a fair number of times. There were varying reasons. Sometimes, the inflated scores were just so inappropriate for the story, I felt I had to do my part to bring the score into a more reasonable range. (Not that it works unless there weren't many votes.) Other times, it was because - due to a complete lack of editting - the story truly approached being "a jumbled mess of words" or however the 1 was phrased.

Rarely, I admit, the low score was 'punishment' for the author being dishonest in the story description/codes. If anyone feels that was out of line, all I can say is that if they'd been honest, maybe they wouldn't have gotten someone who wasn't likely to enjoy their stories - or likely to vote poorly on their stories - to read them. If I were to write a snuff story, and not warn those people who dislike such that that's what my story was about, I deserve all of them to vote low, IMAO. But that's another rant.

Seriously though, sometimes, low scores are meritted. Not everyone can be Frank Downey, or Al Steiner, or cmsix, or [insert favorite author here].

Now honestly, I don't really care about scores. I've mostly ignored them for some time now. If a story is long, completed, and has an appropriate set of codes for my tastes, I'll read it. Or at least attempt to, in the cases of the horrendously written ones. And when a story moves me to vote, I vote. Which means that the average stories I usually skip voting, and my scores tend to not have a lot in the 6-8 range, although when a story is bad enough for me to quit reading, I'll skip to the end and vote low. But I don't choose stories on the basis of scores. I think anyone who does is making a mistake, but it's theirs to make. Maybe, under this new system, that'll change. I liked the idea of just showing relative rankings, since I suspect that's how many vote... but it doesn't matter that much to me. And as has been mentioned, people will figure out how to skew this system before christmas anyway.
Anonymous said…
When I win the lottery, or Paris Hilton forces me to marry her so that we can take care of my love child I assure you that I'll see about getting you that 15 grand and sustainment funds, Lazeez.

Don't get down on yourself because of the people who don't understand the complexity and worthiness of what you're doing. Some people don't seem to respect what a valuable, inexpensive, and massive resource you've got here, easily presented and comparing more than favorably with almost each and every other erotica website on the internet. You're doing each of us a valuable and appreciated service, juggling the normal administrative duties of the site with egos and sensibilities of the authors and readers? You deserve a medal.
Anonymous said…
OK then. That's pretty clear. I guess everybodys' "perfect solutions" are like every other plan to fix the world. ;)

Don't get too wound up about it. In the end, almost everybody will forget what the fuss was all about, and why it seemed so %&^$#! important.

PS: and Thanks! for doing all this.
Anonymous said…
I'm still thinking that the voting scores are much less important to me than the number of votes (i.e. people who've voted). And while I'm at it, I might as well plug my favorite solution again:

For each story, on posting of a new chapter, take the current "downloaded" score and divide by the previous number of existing chapters. The result will be a static score of how many people downloaded and read the story so far. Store the result in the database table, and every time it is computed, store the difference as well.

Example: Story A has 60 chapters, and 6600 downloads when the author posts a new chapter. Dividing one by the other in the right order gives us a score of 110 readers. Since the system is new, the old score is zero, and so the listing until the next chapter will say 110 (+110). This is static - the calculation is done once every upload, not on page access.

The next time a chapter is uploaded, there's 6771 downloads and 61 existing chapters, for a score of 111. The new listing will be 111 (+1) - the story has gained one reader.
Anonymous said…
As a reader on SOL, I have learned what my personal cutoff point is for stories that I won't bother to read. If the score is too low, that usually means that its a mess grammatically, or it jumps around too much, without any transitions, or in some other way is so painful to look at that it is rendered unreadable.
As a writer on SOL, the scores are a handy way to set my own personal benchmark, an incentive to code the story honestly and to keep the action hot.
As a reader, I don't really care what you do to the scoring system, a really badly written story is still not going to get read by me. It's score is likely to be inflated by those who don't mind the poor craftmanship but do like the storyline.
As a writer, I love the idea of separate voting panels for chapters. It sounds like a great way for an author to see if he / she is maintaining or improving the quality of the story as they move forward with it.
But what I really like, both as a reader and a writer, is the idea for a voting panel on the technical quality of the story. They do it that way for Figure Skating, and I think it would work quite nicely for writing as well.
I don't know how many times I've written to an author and urged them to use an editor because I loved a plot line but couldn't stay with the story because the grammar and puncuation were so terrible.
A private technical score would be a great addition to the site for a writer, I think.
And lastly, on a personal note, I am sooo glad that I don't have your job. The kind of passion and dedication that you bring to running this site is a rare and wonderful thing in this day and age. Keep up the great work!
Unknown said…
Lazeez,

Thank you so much for the thought and work that you have put into making SOL as good as it is. I have a healthy respect for people who are able to get things done.

I understand what you are going after with the scoring changes and I appreciate your willingness to put yourself out on the line to make things work for more people.

I agree with your idea of a seperate instant send "Quick Comment" to an author (one that does not reflect on the site). As an author, instant reaction is great and I guess I would format it with just some radio button selection (or drop down menu). Those items for the quick comments would be (not in any particular order): please get an editor, like the concept/learn grammar & spelling, like the concept...etc.

I think that quick comments such as these could be added to and improved, but they would cover the majority of complaint as to technical and could give and author the feedback. Just have it show up on the statistic's page in the authors/editors section.

Warmest regards, Large_Pianist
Vinnie Tesla said…
I'm pretty psyched about the move, but since over half my stories have below 8 already, I'm not really an author with a lot to lose.

I do find that I agree with the commenter on your previous post who suggested that a smaller overall range might be desirable. I can't really defend it logically, but my intuition is that readers would be readier to vote 2/5 than they currently are 4/10. The latter leaves too much room for hair-splitting.
Anonymous said…
Well as a "reader-only" member of the site, I have to say Thank You for all your work Lazeez. What you are attempting is not easy and often unappreciated. It is good to see from most of the comments that people here are willing to give the new "voting wording" a shot.

From the way I read things, I think it is important for people to remember that it is "voting wording" and not a change in the voting system. A 1 is still a 1 and a 7 is still a 7 and a 10 is still a 10. Scores below 6 are not going to suddenly cause more damage to a vote than they did; they will be averaged in just like they were before. It seems to me that the new system is an attempt to help people be a bit more constructive (and dare I say, honest) in their criticism, which I think is a good thing.

Again, I thank you Lazeez for all your work on this site.
Anonymous said…
Andrew Johns here.

I didn't get a chance to post on the original, and figured I might as well pop in here now, instead of posting at the end of 131 comments!

The old scoring system had problems, we all know. And leaving it the way it is will just perpetuate the score creep.

It seems a ton of authors are worried about troll attacks. I discussed this with Lazeez years ago, and as far as I know, he still has a mechanism in place to monitor it.

If a score is more than 4 points below the current average (after a certain number of votes have been tallied), it becomes a 6. So that troll vote of a 1 becomes a 6, softening the intended troll attack.

I am a huge fan of the propossed changes. I hate that scoring has become such a joke over the years. When "Aftermath" was #1 while it was being published, it carried a score of 9.21 at the end of its writing. (Lazeez keeps great stats!) This was back when people could vote multiple times, and troll votes of 1 counted as 1's. Still, 9.21 for the best story (at the time by voters) made sense. There really was a sense of "wow, this is a great story".

Now, people vote for authors. Scoring has become a pep rally for authors.

I wish I could suggest that "scores <5 or >8 requier a comment", but Lazeez has already pointed out correctly that it would simple cause problems for those with metered time access.

I'm the toughest egg to crack as a reviewer. People that bother to read reviews know that if I give a story a 10, I must have *really* liked it. If I gave it a 9 or 8, it was pretty good. If I gave it a 6, it was purely average. And if I went below 6... Well, let's just say that if you printed it out, you have emergency toilet paper.

Trying to be fair with a review takes a whole heck of a lot more energy and effort than a simple score. Which is why when several authors say they blow off reviews, it makes you wonder why you bother. Many authors DO respond to a review, though, and I let them pick my brain at whatever detail level they want. Those authors get real feedback.

A score isn't feedback, no matter how you slice it. It's become a meaningless number. If you are writing and worried about the score, or wondering how to make it creep up another .05, you are just killing yourself for something that has no relative merit.

I still wish premium members could have a Top 20 list. Stories on that list are given a weighted score, and THAT score is available to other premium members. A vote of 10 means nothing these days. Having to only be able to keep 20 stories on a list... Getting a story on there is an accomplishment!
Anonymous said…
I like the idea as a reader of allowing premier members to have a top 10 or top 20 list with the number of times a story is in that list only available to authors and other premier members.

As to the changes in scoring, I agree that, scores have started getting compacted at the top. Unlike some who consider others incompetent to give a story its proper vote and thus give a story a 1, I don't pay attention the votes a story has when I am actually giving my vote.

Knight Ranger
Anonymous said…
I'll repeat my suggestion from the previous post: normalize the scores against the user issueing the scores.

If a user consistently gives a score of 9, then normalize against the 9. If the user's average score is a 6, use that.

(And please, spare us the "enormous disk requirements" thing. If you don't want to do something, say so. Disk is cheap.)
Anonymous said…
SOrry to be a wet blanket, but I read a lot and never vote. I also do not look at the votes my stories get. Elsewhere I do not allow voting if I have that option.

This is because a simple 1-10 scale means absolutely nothing to me.

I have been for some years involved in a voting system to grade performances in a sport. We calibrate the judges and adjust the ir submited scoresheets accordingly.

I fully understand that a proper muti-facet voting system is utterly impractical, and I do not have any way of calibrating the voters (and nor do you) to know that A's 5 is B's 9. That is why I ignore the current system.
WILLOBIE said…
Grade inflation is everywhere. I like the idea of separating story quality and grammar/diction, but, as a reader, I want to see both sets of ratings. A strong story can be ruined for me by inept editing.
Lazeez said…
Scoring system changes have been canceled. Too many authors asked for the removal of their stories from the site if the change were to take place.

Sorry for all the trouble.
Anonymous said…
(I posted this on the original "scoring changes" post as well...)

Oh well...

Would these authors object to the added voting catagories on top of the usual overall score? Adding a separate box under the usual voting box that had plot, technical quality, and perhaps stroke?

Also, were they objecting to voting on every chapter in a serial that is for there use only, again seperate from the overall score?

The main thing is these additions would not affect current or past scoring that is shown to the readers.

I think these are good ideas, if they are not too much work for you personally.

Thanks for trying to improve your site.
Anonymous said…
Personally I'd just like to give a great big FUCK YOU to the whiny dick authors that wanted to remove their stories. Up yours guys, if I had a list of your names I'd make sure that your pathetic grandstanding really paid off with the shitty scores you've proved that you richly deserved.

Again, fuck you all you cocksucking bastards.
Anonymous said…
Lazee.

I want to thank you for all your efforts to make this a site I don't mind paying to use. There are lots out there but few that take the time to always make it better for the reader as well as the author.

I am not an author. I do visit SOL every day and currently have over 60 ongoing stories bookmaked that I follow. I don't read too many short stories unfortunately and do not vote until the story is finished. What I always try to do is let the author know what I think of their work. It could be as simple as "I really enjoyed that one" or letting them know how it affected me personally. I also don't always comment after each chapter....maybe I should...don't know?

The scoring system is not perfect but unless you developed a 5 page questionairre and covered all aspects of the emotional and intellectual gambet, it never will be. The problems are not really with the system but the readers understanding of how it was intended to be used. I have read many comments where a "good" story gets a 9.. come on that's a little much. A good story for me might get a 8 but, you have to leave room for the really good and great stories that left you emotionally drained and wanting more (there are a number of these quality ones on this site).

Anyway got to stop my rant and get back to what I enjoy most and that's reading....

This is the best story site on the WWW so lets appreciate all the efforts by Lazee and not get bent out of shape over something as petty as scoring....hey why not take it out all together..... bet that would really piss of the crybaby authors.

RickM
Anonymous said…
Perhaps a change that would be more acceptable would be to draw a line under the current system on a given date (01/01/2007 for the sake of argument), and all stories posted from that date onwards are scorred on the new system - perhaps giving marks out of 20 to allow a greater range of expression. All stories where the first chapter was posted on or before 31/12/2006 would use the current scoring system with no changes. Obviously you would have to mark which system a score relates to, so instead of a story being shown to have a sore of "9.77" it would be shown as having a score of "9.77/10" or "9.77/20" (obviously the former would be a much better story).
Anonymous said…
Well it's pretty easy to see why "voter turnout" is often low, and why the scores that do get through are on the high end.

Most readers are guys reading with one hand, probably with a child, significant other, or parental figure asleep in the next room. When the story "wows" them so to speak, it's all over. They close it, dispose of the resultant mess, and sneak off to bed (back to work) or whatever. In the case of the truly compelling stories, where it's not just about the stroke factor, they bookmark it, come back to it again, and again, and when they finally get to the end it's a ten, because they stuck with it so long, as opposed to the 20-30 other stories they just glanced at and closed because it didn't do anything for them.

Hence "voter turnout" is low, and the scores are high. The only real remedy for this this is to have the scoring window at the bottom of every chapter, or even in a floating sidebar. I've skipped over voting a story really low many times simply because I didn't feel like navigating to the last page to cast my vote. (I've also skipped voting for a few because they were so good I had a mess to clean up and had to close the browser while I was afk.) A score card at the side or end of every chapter page would solve this right quick, though it may be interpretted by some as nagging the reader. I'd just think of it as convenient. Overall votes per download would be much higher, and overall scores for the stories would be much lower, more realistic. And authors would get tons more honest (including negative) feedback, if that's your goal.

This is the downside to an improved voting system. Getting much more negative feedback would discourage some authors. Not everybody wants to hear what the 95% of people who didn't vote really think of their work. And some authors don't particularly care what a reader thinks who couldn't even be bothered to read the whole thing. So it's a toss-up I guess.

Personally, when I'm browsing for something to read, I use the story codes first, sorted by score descending, and then I look for a catchy synopsis. Inside my mind, I see the entire spectrum of scores from 8 to 10. Anything below 8 is a zero. Around 9 is a 5/10 for me, and 9.5 is a 7.5/10. I've just gotten used to the automatic scaling my brain does. ((Score - 8) / 2) * 10. -shrug- Works for me. I'm sure anyone who's been around long enough has a similar mental filter. There's just not enough time in a week for stories less than a 9.3 or so, though they may be good stories. (A few of mine are far below that level.)

As for my own scores? I tend to look more at downloads per chapter, as opposed to straight score. I've always been curious about the demographics of my readers, but that would be kind of intrusive. Age? Gender?

Anyway, just my opinion. Lazeez's got a great site going here, and there's very few improvements that could be made.

-pleasure boy 1
Anonymous said…
Lazeez,
Thanks for considering my suggestion. Does this mean my story gets a 0.1 boost before the adjustment that won't take place? (just kidding)

I think you are on the right track. I don't understand why authors would withdraw stories but I guess we are a temperamental lot ;-) ((says the author with one, count 'em one, story posted to date, but I've got another one in the pipeline!))

Finally, I liked pooperman's sugestion about voting twice for longer stories. I suggest a variation where I could change my vote later as chapters are posted. I don't know how the data is kept, so this may require a great deal of work, but seems like it would allow early voting on a new serial with allowance to adjust later if it improves or declines.

Finally, I hope the new score descriptions will stay. Thanks, Lazeez for running a great site and for all the time and effort you put in. Keep it up.
Anonymous said…
I think you have seen a problem which does truly exist, you have
analysed it well, and you have come up with a series of actions that should solve the problem over time. Thank you for the excellent site, and for all the excellent work you do here. This is a site which is truly in the best traditions of the original internet.
Anonymous said…
I had been scoring nearly every story I read on SOL to which I could give a 10 or, but only if it increased the posted score, a 9. I do not recall ever finishing a story with a score below 8 and usually I would ignore stories with scores below 8.5. This system worked for me.

Now I am totally confused. There appears to be a new scoring system, but the old scores seem to still be present on prior stories. When I see a score now, I do not know what it means -- is it an old score or a new score.

As virtually no story is impossible to improve, the top score seems unachievable. I scored two stories under the new system, but then realized that the old scores still seem to be in place on preexisting stories. Thus, I feel I did a disservice to the authors.

I have decided to stop scoring stories until I can comprehend what is happening with scores. I cannot use the new scoring system when inconsistent scores still are posted on older stories as that is a disservice to authors. Now, I do not know what a score means when I look at it -- is it an old score or a new score or a score where readers have ignored the new vebiage?

Just one reader's reactions.
Wino

Popular posts from this blog

Reader Feedback 101

Final Decisions

Scoring Changes Implementation Follow Up