1. Here you will find official announcements, updates and Developer blogs. These announcement are also linked in the Official SotA Discord server.
    We encourage comments from the community! To keep the announcements official, we ask that comment threads be created in the General forums for player input.

                                                 Thanks!

Testing Incentives Program Suspension

Discussion in 'Announcements' started by DarkStarr, Aug 21, 2017.

Thread Status:
Not open for further replies.
  1. DarkStarr

    DarkStarr Executive Producer Moderator SOTA Developer

    Messages:
    6,697
    Likes Received:
    45,640
    Trophy Points:
    202
    Gender:
    Male
    Greetings Avatars,

    About a year ago we started the Testing Incentives Program in an effort to reward “a solid record of reproducible bugs with detailed information (ex. coordinates and screenshots) as a minimum. In addition to total number of bugs, player rank is also determined through other items like working on the Bug Brigade, providing editing feedback on release documents, etc.”

    I want to thank everyone who participated in the program. Your efforts are truly appreciated and we could not have gotten where we are today without your help.

    This program was at many times successful and lots of rewards were handed out. However, there was a high volume of minor, duplicate, non-reproducible, or non-detailed bugs created. Reviewing that high volume of bugs to find the “reproducible bugs with detailed information” cost us a lot of internal QA staff time in the process. As an example, just this past Friday our QA team spent the entire day reviewing the bug forums and were only able to find 7 total previously unreported bugs, of which only 3 were even high priority, none were critical. After careful consideration, we do not believe this is an efficient use of your backer funds, and because of that, we have suspended the Testing Incentives Program until further notice. The rewards given out last release were the final round.

    Once again I want to extend a huge amount of gratitude to everyone who has ever written up a bug about the game and I hope that everyone continues to do so to help make the game a better experience.
     
  2. LoneStranger

    LoneStranger Avatar

    Messages:
    3,047
    Likes Received:
    4,776
    Trophy Points:
    165
    Gender:
    Male
    Location:
    Petaluma, CA
    Well, I have to say that this is disappointing.
     
    Bom, zyxe, Leostemplar and 12 others like this.
  3. GrayFog

    GrayFog Avatar

    Messages:
    674
    Likes Received:
    1,400
    Trophy Points:
    93
    Yup, good move imo.

    I know i wouldn't want to dig through all the bug reports for hours/days only to figure out a handful are actually new :D
    A waste of dev time.
     
  4. Daxxe Diggler

    Daxxe Diggler Avatar

    Messages:
    2,704
    Likes Received:
    5,736
    Trophy Points:
    153
    Gender:
    Male
    Location:
    Virtue Oasis - Hidden Vale
    But I had rewards I was still hoping to get. :(

    I understand the reasons though and the duplicates and previously posted bugs were getting out of hand.
     
    Net, FrostII and amaasing like this.
  5. Fister Magee

    Fister Magee Avatar

    Messages:
    2,577
    Likes Received:
    3,833
    Trophy Points:
    165
    Gender:
    Male
    Location:
    Spite
    It's a shame there were so many duplicates being submitted... why was that, because old bugs should have been fixed long ago??
     
    Bom, Ice Queen, Leostemplar and 12 others like this.
  6. Dinsoo

    Dinsoo Avatar

    Messages:
    796
    Likes Received:
    2,634
    Trophy Points:
    105
    Gender:
    Male
    Location:
    Minneapolis
    It says something that so many duplicates were submitted. Why don't you fix them? hehe
     
    Bom, Baratan and Moiseyev Trueden like this.
  7. Attenwood

    Attenwood Portalarian Emeritus

    Messages:
    6,911
    Likes Received:
    5,866
    Trophy Points:
    153
    More like...
    Player 1 finds and submits a bug for X issue in a Release.
    Players 2-9999 submit duplicate reports for the same issue discovered and reported by Player 1.
     
    Chatele, Hornpipe, Sorthious and 6 others like this.
  8. Tiina Onir

    Tiina Onir Avatar

    Messages:
    1,052
    Likes Received:
    1,811
    Trophy Points:
    125
    Location:
    Bramble, South Paladis
    So it sounds like searching and merging needs improved (and maybe getting the community to do the merging). Unfortunately neither of these are things ANY forum software is good at.
     
  9. Daxxe Diggler

    Daxxe Diggler Avatar

    Messages:
    2,704
    Likes Received:
    5,736
    Trophy Points:
    153
    Gender:
    Male
    Location:
    Virtue Oasis - Hidden Vale
    Yes, the "duplicates" problem was the same new bug being reported over and over by many people in the same release.

    I guess people just don't like to search through the already reported bugs before posting the same one again... or they just hoped they would get credit for the "Program".
     
  10. Sheamus McGuinness

    Sheamus McGuinness Avatar

    Messages:
    600
    Likes Received:
    1,669
    Trophy Points:
    93
    I will always love you scorpion chair.... RIP
     
  11. amaasing

    amaasing Avatar

    Messages:
    479
    Likes Received:
    1,154
    Trophy Points:
    43
    Gender:
    Female
    Location:
    Colorado
    Have mixed feelings on this. :) I was really wanting the Ant Farm but by the time I loaded the Test Release each month and started playing there were a couple of players who had already submitted about 50 bugs each so lots of my 'testing' was reading through those to determine if mine was a duplicate. The down side is that those were the players who got all the incentives and the rest of us who worked just as hard to Test and locate bugs got 5 COTOs for testing.

    This is NOT a complaint. I have enjoyed and appreciate the COTOs for sure. I just never felt that the system was quite working to benefit all who tested.

    Maybe another method of rewarding testers can be determined. Or not. I remember the days when we tested just to make the game better. :) (But I really did want that Ant Farm.......) :)
     
    Bom, Mishikal, Katrina Bekers and 5 others like this.
  12. Poor game design

    Poor game design Avatar

    Messages:
    18,208
    Likes Received:
    35,487
    Trophy Points:
    153
    Gender:
    Male
    Will anything change as far as being able to report bugs normally? (i.e. using /bug to create a bug report with the expectation that it will get fixed)

    Also, I'm not upset about the suspension of this program. But I know I've submitted many bugs that have been said to have either been by design or have never been fixed (for whatever reason). From my perspective, the ongoing communication of what the design is, needs to improve if you want to get better bug reporting from the community.

    Here, I'll give you a perfect example. I have reported multiple times that staves do not lose durability when you cast spells, they only lose durability when you use them to physically attack something. Because this is a known bug and I was told it would be fixed many releases ago, I stopped reporting it many releases ago. Yet it's still not fixed.

    The point I'm trying to make is that I can give many examples like this where people report things that are not fixed or the development team appears unresponsive to the problem (sometimes saying clear bugs are "by design" without real communication as to what the design is or that they actually understand the problem). What I'm trying to say is that overall the act of reporting bugs is not always efficiently managed by Portalarium and the result is often (imo) wasted time and money not fixing things that should've been fixed years ago while hiding behind "it's by design" or not fully understanding the problem.

    Please take this under consideration as you look to improve the way QA and DEV handle bugs in the future.
     
    Bom, Preachyr, MrBlight and 4 others like this.
  13. 2112Starman

    2112Starman Avatar

    Messages:
    3,620
    Likes Received:
    8,009
    Trophy Points:
    165
    And of course my complaint that I never received my rewards after 1 full year of hard testing probably didn't help with the demise of this. They did just end up giving me the tier 1 which I appreciated (this release)... but felt I had done enough for higher tiers. Hell, I created 30+ *VIDEOS* alone to show them the bugs. Lots of my time.
     
    Bom likes this.
  14. Spoon

    Spoon Avatar

    Messages:
    8,439
    Likes Received:
    23,674
    Trophy Points:
    165
    Gender:
    Male
    Location:
    Sweden
    Fully understandable.


    *nods and moves on*
     
    Kaisa, Sean Silverfoot, Net and 2 others like this.
  15. Attenwood

    Attenwood Portalarian Emeritus

    Messages:
    6,911
    Likes Received:
    5,866
    Trophy Points:
    153
    No. In-game systems to direct players to the forums to submit bugs by using /bug for the normal Live forums and /devbug for the appropriate QA forums will continue to work as they have.

    Ideally as less time is spent by our internal team sifting through mounds and mounds of unreproable or duplicated reports, we will have more time to be able to give better responses to issues that are at first reported bugged then deemed "by design". I won't weigh down this thread going into a discussion about the durability loss of Staves, but feel free to reach out directly and I'll be happy to give you an answer. :)
     
    Frederick Glasgow, Net and amaasing like this.
  16. Daxxe Diggler

    Daxxe Diggler Avatar

    Messages:
    2,704
    Likes Received:
    5,736
    Trophy Points:
    153
    Gender:
    Male
    Location:
    Virtue Oasis - Hidden Vale
    Sheamus, you can always get one in Offline mode if you really really want one. I know it's not the same thing, but let me know if you are interested.
     
    Bom and Sheamus McGuinness like this.
  17. Dinsoo

    Dinsoo Avatar

    Messages:
    796
    Likes Received:
    2,634
    Trophy Points:
    105
    Gender:
    Male
    Location:
    Minneapolis
    I feel like this is an attempt to say, "Hey, we already know about that. Stop reporting it!"
     
    Katrina Bekers, Net and 3devious like this.
  18. Jens_T

    Jens_T Avatar

    Messages:
    643
    Likes Received:
    1,487
    Trophy Points:
    93
    Location:
    Zurich
    I understand the issue with duplicate bug reportings, I just had wished that this would have been addressed early on by providing a better tool for current release bugs (rather than a forum) and a bug tracker for old issues that are known and worked on.
    Still, can't help that it feels bad.
     
    Bom, Net, LoneStranger and 2 others like this.
  19. Tiina Onir

    Tiina Onir Avatar

    Messages:
    1,052
    Likes Received:
    1,811
    Trophy Points:
    125
    Location:
    Bramble, South Paladis
    We're still submitting bug reports (rewards be damned), so I'd STILL like to see it addressed in this way!
     
    Bom and Net like this.
  20. Lexie

    Lexie Bear Queen Emeritus

    Messages:
    839
    Likes Received:
    2,901
    Trophy Points:
    93
    There is a certain degree of duplicate reports that we anticipated and expected. Not everyone is going to submit things under the same wording, and some thing can be described several ways. We deal with that internally, as well. The concern was with the severity of it. One example (of many) would be the pink plant texture that was showing in some PoTs by the docks. The 'pink plant' reports flooded the forums so much it delayed our reaction time on getting to all the other bug reports. We were still going through all those reports by the time it was fixed. Yet one could tell it was a known issue by looking at the post right below it.
     
Thread Status:
Not open for further replies.