Scoring Changes Implementation Follow Up
It's been ten days since the implementation of the new voting system, so I thought I would keep everybody up-to-date with how things are shaping up.
First thing to report is that voting is up. The number of basic votes cast per day added to the number of TPA votes cast per day is about 30% higher than the previous week's number of basic votes cast. So it seems that many people were not using the old system and were encouraged by the new one. So for those who were afraid that the new voting system will lead to people moving away from voting, rest assured, it didn't happen.
Second, the voting form wording change is affecting scores mildly. The median of votes cast after the change is lower. It currently stands at 8.00 compared to 8.82 prior to the change. Of course, the Q Score is compensating for this difference and the Q Scores are more consistent.
The number of TPA votes is about 30% of the basic votes number. So, it's quite high. I didn't expect it to be this high. So, I'm not going to keep individual TPA votes indefinitely. I can't. I haven't decided how long to keep them, I'll wait for future development to guide this decision.
Implementation changes:
The Appeal component of the TPA is now a part of the Q Score. It's processed as a basic score.
During the implementation, I changed the minimum number of votes needed to show the score on the site from 10 to 15.
The expected developments are that some authors and few readers are still trying to find a concrete relationship between the Q Score and the regular score and since they don't have any hard numbers, many are just ending up being confused and already declaring the new system to be a failure.
My advice to those who are trying to correlate the scores is: Don't. You don't have the numbers and if I gave you the exact formula, you will still be missing the data used in the formula to calculate things.
Yes, relative positions of stories may change between the old score and the Q Score and that's because different posting and updating dates of stories affect the Q Score a lot. So unless you can dig the data from the database, the exact formula won't be much help.
The unexpected development is that so far, I'm getting a lot more support for the Q Score from readers and few authors. I've received many requests to remove the old scores completely and just leave the Q Score and the TPA (many of those requests came from premier members). Reasons given in the requests are that the score next to the Q Score are confusing things and that with all the data presented in the tables, there is too much info to scan through making the site harder to use.
I'm starting to lean in that direction already. If I receive more requests of the same, I will start the process of removing the old average scores and just leave the Q Scores. When/If I start that change I will make it so that the TPA score is also hidden if the number of votes is below 15.
First thing to report is that voting is up. The number of basic votes cast per day added to the number of TPA votes cast per day is about 30% higher than the previous week's number of basic votes cast. So it seems that many people were not using the old system and were encouraged by the new one. So for those who were afraid that the new voting system will lead to people moving away from voting, rest assured, it didn't happen.
Second, the voting form wording change is affecting scores mildly. The median of votes cast after the change is lower. It currently stands at 8.00 compared to 8.82 prior to the change. Of course, the Q Score is compensating for this difference and the Q Scores are more consistent.
The number of TPA votes is about 30% of the basic votes number. So, it's quite high. I didn't expect it to be this high. So, I'm not going to keep individual TPA votes indefinitely. I can't. I haven't decided how long to keep them, I'll wait for future development to guide this decision.
Implementation changes:
The Appeal component of the TPA is now a part of the Q Score. It's processed as a basic score.
During the implementation, I changed the minimum number of votes needed to show the score on the site from 10 to 15.
The expected developments are that some authors and few readers are still trying to find a concrete relationship between the Q Score and the regular score and since they don't have any hard numbers, many are just ending up being confused and already declaring the new system to be a failure.
My advice to those who are trying to correlate the scores is: Don't. You don't have the numbers and if I gave you the exact formula, you will still be missing the data used in the formula to calculate things.
Yes, relative positions of stories may change between the old score and the Q Score and that's because different posting and updating dates of stories affect the Q Score a lot. So unless you can dig the data from the database, the exact formula won't be much help.
The unexpected development is that so far, I'm getting a lot more support for the Q Score from readers and few authors. I've received many requests to remove the old scores completely and just leave the Q Score and the TPA (many of those requests came from premier members). Reasons given in the requests are that the score next to the Q Score are confusing things and that with all the data presented in the tables, there is too much info to scan through making the site harder to use.
I'm starting to lean in that direction already. If I receive more requests of the same, I will start the process of removing the old average scores and just leave the Q Scores. When/If I start that change I will make it so that the TPA score is also hidden if the number of votes is below 15.
Comments
Long Stories - If I vote early using the new system (to give the author some feedback) can I change my vote as things progress?
Or should I wait until the end (as now) unless, in my opinion, the story is a total waste of space and I am not going to get to the end.
My proposal is that after a certain number of chapters are published, the story could be voted again by anyone that has already voted. I do not mean to disregard the old vote, but to allow a new vote. Some people may say that is is not fair to allow more than vote for a story, but my opinion is that very long stories deserve multiple votes to encourage early voting.
I am very reluctant to vote on a story until it is finished, because I do not have a crystal ball to determine if the rest of the story will rate as what I have read so far.
I have been using the fact that I can change the TPA score as an opportunity to be able to vote early and still change my vote at a later time.
If voting more that once for a story (as I proposed above) is a problem, another possible solution is to keep the score for unfinished stories so that it can be changed, and only lock it in place after the story is finished. This would probably be better than the earlier proposal, but I do not know if it is technically feasible to keep all the scores for a long period of time (some stories, like Cammie Sue book 3, have been ongoing and active for over a year).
I'd like to see an FAQ section regarding this new TPA scoring on the help page. It's so new and I'm sure you're getting al lot of the same questions. It might be helpful to both authors and readers.
sourdough
They have also changed my voting behaviour. Under the old scoring method, I would always wait until the story was completed before voting. Now I vote early and adjust as the story progresses. So far I've only made 1 significant change, but there's always those situations where a story starts off great and then plummets quickly (in either style or substance), or where a plot twist changes a mundane read into a nail biter.
There's my $0.02.
If a reader does not like a story, would he continue reading to the end just vote on it?
Very unlikely.
If the reader does not like a story, would he only read part of the story and then vote without finishing, maybe missing out on redeeming factors?
Most unfair!
Will readers finish a story that they enjoy and vote it low?
Again very unlikely.
Possible assumption: The only stories most likely to be voted on will be the ones that readers like, read to the end, and feel compelled to recommend by voting. So where is the surprise that scores tend to be high?
Jon
For new stories I think it is probably the most valuable of all the scores out there. My belief is if a reader spends the time to vote on three sections, they are more likely think about their votes.
I look at the TPA scoring as the best indicator for me to initially judge a story.
John Smith
My question is, what about the writers? Are story submissions up? I seriously doubt it. I wouldn't be surprised if they are down a fair bit actually.
My personal holy grail was the perfect story, a 10. That was never feasible but I could aim for it.
But now, the best I could hope for is a 9. And it might take me 20 years to write it.
SN
e.g. my first story had an Av score of 9.6 (from the smallest voting base) and received a Q score of 8.65
My second story - Av score 9.65 (from a larger, double, voting base), scored a Q score of 8.6? (And it has been improved constantly since first publication --- ahh, love that word!!)
Of the three stories I have so far submitted, Story two is by far the best, but with the Q score only showing it will appear to be the worst and may be bypassed by discerning readers. The sighting of the Av score alongside modifies that assumption somewhat.
Hold the line for a while please mate.
A second point - if stories already have a swag of votes in the Av/Q scores will they still be required to wait for 15 votes before the TPA scores are shown? It is hard to get readers of older completed stories to vote. (I've been watching the d/l, and the number of votes cast compared to d/l is sparse after the story is completed.)
The older stories have already earned their way in number of votes cast, additional votes in TPA format should be revealed as soon as they are received.
Doug
One other thing I would Question: Why is the A of the TPA used as the score that gets counted for the other scoring? There have been a couple of stories that I have given a technical score of 6 or 7 but then given a 9 or even a 10 for appeal. Even though I really like a story I would prefer that the TPA was AVERAGED to get the result used for the other scores. I like the TPA scoring a lot, especially because I can adjust my score later, but I think some weight at least should be given to the T and P part of the equation. Otherwise the score may get artificially inflated (or deflated in some cases where I just don't like a story that is excellent technically). How would that be any different from the system that the Q score and TPA are designed to improve on?
One other thing I would Question: Why is the A of the TPA used as the score that gets counted for the other scoring? There have been a couple of stories that I have given a technical score of 6 or 7 but then given a 9 or even a 10 for appeal. Even though I really like a story I would prefer that the TPA was AVERAGED to get the result used for the other scores. I like the TPA scoring a lot, especially because I can adjust my score later, but I think some weight at least should be given to the T and P part of the equation. Otherwise the score may get artificially inflated (or deflated in some cases where I just don't like a story that is excellent technically). How would that be any different from the system that the Q score and TPA are designed to improve on?
For me as a reader the T is the most important. I won't read a story that is not well done.
Second is the plot and characterization. Same as above - if it's not written well I won't even finish reading it.
The A part is more or less or no value to me. I frequently read stories outside of my main interest - that's how one learns and grows. If you felt you had to average them I would go 30% 50% and 20% for TPA.
I fell the Q scores are less than useless. A score should be a constant, not something that is all over the map. A story written this year and the same story written two years ago has the same inherent perceived quality.
Rather than pick a point for each year as the base why not just use the overall total history of the site?
One thing I don't understand about leveling by year: is it the year a story is written or the year each single vote is cast. If I wrote a story three years ago and it got 100 votes that year, and 50 each subsequent year, how do you calculate the results?
I decide to read a story primarily by author. I do look at the score but never at the TPA or the Q score.
I don't think there is any argument that can be made for giving a well written and plotted story a six. That's like getting a "D" in a Creative Writing class for a story that rates a B+ or A-!
Several people in this blog entry have commented that they particularly like this feature, as they do not want to read stories that are "not well done" (by which I assume they mean that either poor grammar, spelling or formatting). This is quite reasonable. But just what denotes "well done," then?
According to the voting form, the "T" should be judged based on a stories' "Spelling, grammar and overall readability." I find this a very difficult metric to judge, except in a binary fashion. For me, there are stories that are painful for me to read (be it because of spelling, too-short paragraphs, poor grammar, etc.), and stories that are "OK" -- that is, not painful to read. OK, perhaps I like certain stories (or, more likely, some authors) more because I like their particular style -- does this characterization belong in this score? To me, this is more in the "Appeal."
Borrowing from Stephen King, imagine someone created a story consisting of nothing but sentences of "All work and no play makes Jack a dull boy" (go read or watch "The Shining" if you're missing this reference). Let's assume this sentence is bundled into technically correct paragraphs, with variable number of sentences in each paragraph.
Is this hypothetical story worthy of a "10" in the "T" category? Why not -- it's spelled correctly, each sentence is grammatically correct, and it's readable (if boring). It obeys all the Elements of Style.
In my voting with the new categories, I find that I give most stories that I consider to be "OK" in the technical merit category an "8" because I don't know what else to vote. Mentally, I know what a "10" should be in the "P=Plot" category, and very much what a "10" should be in the "A=Appeal" category. But what is the gold standard in the "Technical Merit" category?
As others have indicated, I think Lazeez needs to create a FAQ on exactly how these categories should be separated and graded, and (with apologies to Lazeez) upload this FAQ ASAP if he's considering abandoning the old scoring system in the near future.
I don't believe that authors' interests have been considered. It is legitimate for authors to have concerns.
Some call us "whiny" or say "good riddance" to us. I don't think that this vocal minority has experienced slaving over a work for hours to make it the best it can be. They should try it.
After one contentious story, the negative scores and the intentional flak from some of the readers were a pain in the butt. However, I put up with those, but had to make it almost impossible for the slaggers to send me vicious e-mails by insisting that they join my Yahoo group if they wanted to e-mail me. Unfortunately that cut out a lot of my best feedback, so I had refer to downloads and the somewhat less than ideal scoring system.
Now anyone who is honest in their voting policy is penalized and their vote is downgraded to some none-existent, theoretical position that is supposed to give a result that is somehow more fair. My question is, "MORE FAIR TO WHOM!"
It sure as the dickens doesn't look fair to me and I've been lucky not to be kicked too hard by it. Even so, I've certainly seen the effect.
So, I've come to a decision.
When all the series that I am now writing on SoL are complete, that's IT!
NO MORE.
.B
The scoring system now in place is an exercise that promotes mediocrity. There is no real standard applied no matter which score you choose to use.
The TPA score makes the assumption that any reader is qualified to assess a story. Unfortunately while many may have the background to do so, many others wouldn't know a semicolon from a baseball bat.
The Q score is so arbitrary as to be meaningless. It is so dependent upon factors outside the authors ability to change as to be of no use to an author.
The old 1-10 score had its problems too. There is no doubt it is a popularity vote. It is also a way for certain readers to cause any number of problems. But at least it was consistent and could be somewhat evaluated by an author.
I honestly feel that this quest to satisfy some perceived need for readers to have more input will have but one outcome. Those authors who take their work seriously and work hard producing what the readers consume will drift to sites that are more in tune with their needs.
In the long run, the readers of SOL are the ones that will suffer. There is no shortage of sites that accept submissions from armature authors. They will simply move to one of them.
In my opinion, SOL has never been a very good site for authors. The posting schedule is hit and miss. Sometimes, corrected chapters take a long time to be posted. The instance of anonymous users by Lazeez ensures that there will be abuse, and the list goes on.
I have to agree with .B that there is no incentive for me, as an author to continue to post on SOL. It just isn't worth the effort.
Asa Strong
Reversing the scoring drop-down was overkill in my opinion, but that will balance in time, making that minor gripe irrelevant.
Regardless, I still find SOL a fine environment to post my work on, with far more (quality & quantity ) insightful feedback than I usually get elsewhere. The posting time is lightning quick for me, compared to what I'm used to elsewhere as well. Being able to fix errors and see them posted within a few hours is wonderful to me, after having to see a story drop in score for days before that critical "why didn't I read it through one more time" error correction gets posted.
Question: Is the Qscore what the toplists for score is based upon now? You mention that the A part of the TPA is added to the Qscore, but if the toplist is based upon the average score, then you're losing out on all the TPA scores with respects to the toplists, and there are a [i]lot[/i] of TPA scores now that it has been introduced.
Naturally, the TPA is of great value to an author as feedback in and of itself, but let's face it, toplists bring readers in, and we all want that too. If the TPA isn't being applied toward our position in the toplist, we're getting bitten in the butt a little there.
I tend to agree that it should be the average of the TPA score, and not simply the A part which should be added to the Qscore.
I post on this site and Literotica, as well as a 'for pay' site. With respect to Literotica and SOL the differences are these:
Literotica brings more readers, by far to the stories. SOL is better at helping readers find authors and at giving older stories some life.
SOL has far shorter queue times (2 hrs. vs 8 days.) Literotica has a simple 1-5 scoring system. There is also a means of public and private comments. authors can turn off voting and/or public comments on a story by story basis.
SOL's presentation of stories such as coding, teasers and listing is far better.
Literotica can get a little goofy with some of their commercial products. SOL has none of that.
There are many crossover authors and readers. If you are unaware of it, check it out at Literotica.com. You don't need a membership to peruse the site.
To the Purvv, writers make mistakes, and new writers try to do better and improve their stories as they learn the craft of storytelling. I send in more reposts than most and Lazeez has NEVER hinted annoyance or baulked at posting them as quickly as my new posts.
I still stand by the points I made in my previous post.
Doug
The TPA scores I like.
The Q scores? Any system that manipulates a vote by some mysterious formula to adjust it and alter it away from the voters intent is, I think, suspect.
Doug
I've read stories from many sites... and even though a free account on SOL doesn't have the usefulness it once did, it still beats any other site.
On what other site is searching by codes as useful? On what other site does there exist so much information designed to (hopefully, or in theory at least) allow readers to choose stories they are likely to enjoy?
And personally, the most important... what other site has such a readable format for stories? Every other site I've seen imposes fonts, font sizes, color schemes, text widths and such that result in a far less enjoyable reading experience.
So, I suppose that some authors will leave. And, in the cases of those with loyal fan bases, their readers will follow them to whatever other site they post to. However, I doubt that any other site will expose them to as many readers as SOL does, and I doubt that every reader from SOL will be willing to put up with the formatting of the other sites. So, if these authors are willing to lose readers, just because something meant to be a tool for readers - the scores - aren't as useful to their egos... well, that's their call. Some, I'll still read... but the others... well, their choices have consequences.
I do consider the changing of everyone's votes to be insulting and I think it simply encourages further abuse of the system. However, I also think that it is a way to increase the number of 'stroke' stories on the site. Since I find I don't write 'stroke' stories well, I feel uncomfortable with the result and now feel that my stories no longer fit.
On top of that, at the end of November, my weekly downloads of the present story I'm working on was well over 3000. This week it was 1700+. During the same time for a story of similar length and subject, but on another site, my weekly downloads have doubled to a total download of almost 20,000 per week. By the way, that site is NOT an erotic site and I do NOT use the same 'nom de plume' there.
I am an old man and although I'm retired, I have a limited time to write each week. So, if you were me, which site would you support - the one where your downloads were falling and you felt uncomfortable, or the one where your downloads were skyrocketing and you got decent feedback and honest criticism, not vulgar insults?
.B
Now, as to the points he raised: I actually agree with some of them. I mean, I know that my votes mean less, under the new system... but then, they meant next to nothing under the old system, too. Because people voting wouldn't bother to think about any actually criteria, and hand out 10s to everything, even piles of unmitigated crap. Hopefully under the new system, that won't happen as much... but we shall see. It's disturbing that I've seen so many 10s in the new scores, honestly. The QScores are doing, however, what they were intended to do: ensure that only rare stories get 9+ scores. Which is as it should be.
On the TPA score - you only post an integer value, that, I would guess, represents the mean of the scores in each category, no?
Do you round up or truncate the integer?
After a story gets a single vote in a category less than a 10, will it be a 9 or less forever?
Is the 10 a rara avis, that can only be a 10 until the first '9' (or lesser) vote?
bc
All authors, thank you very much for sharing your labors. I for one appreciate the opportunity to read what you are willing to share. All scoring is purely subjective and should be relegated to that consideration. If you enjoyed writing a story, then why should you worry too much if everyone does not find it to be their particular interest? I may be very wrong, but if an author does not enjoy writing, why write? If you are writing to please a particular audience, why bother if you do not find pleasure in so doing?
Please do not let the numbers put out by 5% of the readers be what decides whether you stay around or not. Most of us probably do not know how to grade a story other than whether or not we liked it. Most stories have something about them that catches someone`s interest, but you would be a miracle worker to be able to please everyone with every story.
It's killing downloads and votes for large chunks of - mostly new - authors, and will only grow worse once the TPA is changed to the 15 display range as well.
Dave
A story can be very high in grammar (T), but be an uninspiring and dull plot(P) and have little personal appeal (A). A story like that may have a vote like 9,5,5.
I think many voters are giving the same score for each category regardless of what each category means, so they vote 5,5,5 instead.
There is nothing one can do about it except hope more voters think before clicking submit.
I really like that I can update scores as I read subsequent chapters. I sometimes misunderstand where an author is going with a story and am overenthusiastic initially. Thus, I am more likely to uppdate the score I give to a story to lower scores than higher. On the other hand, I am much more willing to score a story with only a few chapters posted.
I personally preferred the simpler voting system.
I genuinely like both sites for different reasons. As a newbie, I just wanted to point out my experience with the new system.
There are stories that are getting 8,9,9 or 9,9,9 when they are technically utter garbage and the plot is a huge yawner from start to finish with nothing innovative in it at all.
The average story on SoL should be getting 5,5,5. When you'd see something get 9,9,9 you should darn near wet yourself in your eagerness to read it.
Unfortunately, these days, anything that is mostly legible is scoring in the 8's or 9's at least, making the score only useful for weeding out the worst of the worst which scores less than that.
Overinflated scoring is bad for everyone. The writer doesn't get the proper signals sent to him (ie, "practice more, you write like a drugged chimp") and goes around thinking he's Shakespeare reincarnated, and the readers have to wade hip-deep through dreck to find the few nuggets of gold buried in the pile of high scorers.
Not sure if there is any good solution for that, but it's still a pain.
/Croft