Testing Incentives Program Reinstated (as a Lottery)

Discussion in 'Announcements' started by DarkStarr, Sep 20, 2017.

  1. DarkStarr

    DarkStarr Executive Producer SOTA Developer

    Messages:
    6,254
    Likes Received:
    42,137
    Trophy Points:
    202
    Gender:
    Male
    Greetings Avatars,

    As we announced recently, we had to temporarily suspend the Testing Incentives Program due to the amount of internal bandwidth it was consuming. This was due to the high volume of bugs the program generated as players were incentivized to create quantity over quality to gain rewards.

    We do want to reward efforts to test on the QA server because we do understand that time spent there is time lost on advancing your character on live. At the same time we do not want to repeat the mistakes of the past, so instead of rewarding quantity we are going to create a lottery. Each release we will randomly reward 4 of each of the following rewards (one prize per player):
    • 5 Gold Crowns
    • Ant Farm
    • Antennae
    • Bug Cloak
    • Set of Bug Paintings
    In order to qualify you must meet the following requirements:
    • Install the QA version of the game
    • Login and play on QA for at least 4 hours per release
    • Report at least 1 bug in the QA section of the forums
      • Note that players will only be submitted for the lottery once per QA cycle. Continued bug reporting will not count toward additional submissions into the lottery.
    Thanks Avatars!
     
  2. liz_the_wiz

    liz_the_wiz Bug Brigade - Bug Hunter

    Messages:
    945
    Likes Received:
    1,669
    Trophy Points:
    93
    Gender:
    Female
    I find myself disappointed that so many of the rewards people worked so hard for are now just a lottery.
     
  3. Weins201

    Weins201 Avatar

    Messages:
    6,362
    Likes Received:
    10,110
    Trophy Points:
    153
    so every player who reports a bug and hs 4 hours get one of the awards 4 of them ???
    or only 4 players who enter have their names submitted get a reward?
     
    Moiseyev Trueden likes this.
  4. liz_the_wiz

    liz_the_wiz Bug Brigade - Bug Hunter

    Messages:
    945
    Likes Received:
    1,669
    Trophy Points:
    93
    Gender:
    Female
    @Weins201 I believe "4 of each of the following rewards" so thats 20 people will get prizes
     
  5. Rufus D`Asperdi

    Rufus D`Asperdi Avatar

    Messages:
    5,890
    Likes Received:
    14,488
    Trophy Points:
    165
    Gender:
    Male
    Location:
    Austin, TX
    Is it allowable to post bug reports to the QA Section of the Forums with Alternate Accounts? This practice being generally prohibited in the Forums.
     
  6. Dinsoo

    Dinsoo Avatar

    Messages:
    614
    Likes Received:
    2,083
    Trophy Points:
    105
    Gender:
    Male
    Location:
    Minneapolis
    Cool! Hopefully there won't be too many complaints and a lot of bugs reported.
     
    Fionwyn Wyldemane and amaasing like this.
  7. Barugon

    Barugon Avatar

    Messages:
    8,685
    Likes Received:
    14,686
    Trophy Points:
    153
    Gender:
    Male
    Location:
    Mystic Isle
    Ok....
     
    Dirk Hammerstrike likes this.
  8. Koldar

    Koldar Avatar

    Messages:
    1,956
    Likes Received:
    4,894
    Trophy Points:
    125
    Location:
    Novia
    This kind of swings the pendulum too far in the other direction. At least before I knew I could count on 5 cotos for 4 hours of time. Now, I have a chance at receiving 5 cotos after I spend 4 hours on the server and report at least one bug. Not a large incentive for that level of time committed. This also encourages people to stop reporting after they report 1 bug. If you don't want people to report all of the bugs they find, this seems like the way to do it. I know if I found 5 or 6 bugs, I'd save them for the next 5 or 6 months.

    I guess I fail to see the problems with the previous program. I understand you didn't have the bandwidth to keep up with bug reports....but that problem itself should guide you to higher level business decisions that need to be made.
     
    Last edited: Sep 20, 2017
  9. Waxillium

    Waxillium Avatar

    Messages:
    3,342
    Likes Received:
    9,160
    Trophy Points:
    165
    Gender:
    Male
    Location:
    Rift's End
    Too bad this can't be made into an in game daily quest reward. Have a bug tracker NPC at the entrance of every scene. Use key words to identify the type of bug. Enter the details. Finish quest. Data sorted on the back end. Avatar awarded bug hunter points. Turn points in for items.
     
  10. Daxxe Diggler

    Daxxe Diggler Bug Brigade - Bug Hunter

    Messages:
    2,274
    Likes Received:
    4,961
    Trophy Points:
    153
    Gender:
    Male
    Location:
    Virtue Oasis - Hidden Vale
    @DarkStarr - Thank you for reinstating the program. I think this will encourage people playing on QA to at least find one bug to report.

    I do have a question though... What if we happen to win the lottery for an item we already have achieved? Will we be able to choose an alternate prize instead? Or, will duplicates just become something to put on our vendors?
     
  11. Sir Leonard

    Sir Leonard Localization Team

    Messages:
    341
    Likes Received:
    772
    Trophy Points:
    43
    Gender:
    Male
    Location:
    King's Landing
    Feeling the same here... also i don't think this will help on testing QUALITY, people will just log in in multiple accounts and report 1 bug at each, spliting the issue the maximum they can, so to be able to game the system.
    Too bad i don't use multiple accounts.
     
  12. LoneStranger

    LoneStranger Avatar

    Messages:
    3,047
    Likes Received:
    4,772
    Trophy Points:
    165
    Gender:
    Male
    Location:
    Petaluma, CA
    This seems to be a solution not for massive amounts of bug reports, but rather the time it takes to give the awards each month. I wish some time could have been spent automating the awards process instead.
     
  13. amaasing

    amaasing Avatar

    Messages:
    468
    Likes Received:
    1,128
    Trophy Points:
    43
    Gender:
    Female
    Location:
    Colorado
    Well, from reading the comments I suppose it proves you can satisfy some of the people some of the time but never all of the people all of the time.

    Personally, before this announcement we were not getting anything, so not sure why the 'grumpy butts'. I was okay with the original incentives and was okay with them being taken away. And I am certainly okay with whatever you guys come up with for incentives (or not) for testing. I think some have forgotten that testing is one of the things we 'signed on for' during the early releases. Good incentives, Bad incentives, No incentives - we should all be testing and not complaining about the incentives.
     
  14. redfish

    redfish Avatar

    Messages:
    10,590
    Likes Received:
    25,820
    Trophy Points:
    165
    @amaasing

    I'm not personally grumpy about it... but I would have to be honest and say I'm not sure this would incentivize me to participate... because I'm still giving up 4 hours I could use to play the live server. I'll try to make time if I can, but I can't guarantee it.
     
    Ajivani, By Tor and Moiseyev Trueden like this.
  15. LoneStranger

    LoneStranger Avatar

    Messages:
    3,047
    Likes Received:
    4,772
    Trophy Points:
    165
    Gender:
    Male
    Location:
    Petaluma, CA
    If I worked four hours of overtime, I'd be rolling in COTOs.
     
  16. redfish

    redfish Avatar

    Messages:
    10,590
    Likes Received:
    25,820
    Trophy Points:
    165
    Re: the availability of the rewards. The only thing that bothered me a bit is the program was ended abruptly without advance notice, so I didn't have the same chance to do the work to get the rewards that I expected.

    I don't really have a problem with them coming back in the form of the lottery. People should remember those with multiple accounts will have to put four hours on each account without botting, which isn't fun. I guess it depends on how worth that effort ends up being.

    Its just not necessarily going to be incentive enough for everyone else, either, if it isn't worth the effort to use alts to get multiple lottery entries.
     
  17. Bushmaster

    Bushmaster Avatar

    Messages:
    515
    Likes Received:
    900
    Trophy Points:
    63
    Location:
    Colorado
    I dub thee "The Incentives For Some But Not All Program!"

    Knowing how I am always on the wrong side of the RNG, I will not partake.
     
  18. Spoon

    Spoon Bug Brigade - Bug Hunter

    Messages:
    8,127
    Likes Received:
    22,775
    Trophy Points:
    165
    Gender:
    Male
    Location:
    Sweden
    @DarkStarr
    @Attenwood
    @Serafina
    @Lexie


    Let me quickly add some proposed changes before this gets rolling which could make this a little more fun and get closer to the quality goal.




    My first proposal is that since the rewards from the old system was in tiers you should have less of tier2 and more of tier1 where also the QA was much harder to get.
    This would preserve the perceived effort<>value that those who participated in the last program worked for. So would give them a tip of the hat.
    Having such tiers is also something which gets people to continue getting lottery tickets since even if you had a smaller lottery win you'd still want a bigger one.
    It would cost you nothing but be worth a lot.

    Proposal:
    10 * pack of 5 crowns
    8 * Antenna (was tier 1 on live)
    5 * Ant farm (was tier 1 on QA)
    2 * Bug Cloak (was tier 2 on live)
    1 * Bug paintings (was tier 2 on QA)



    My second proposal would be to have minimum requirements for the QA Bug reports to qualify. This since the stated goal is quality but your proposed system doesn't reinforce it.
    A) No duplicates.
    This was the issue with the last system, then you should build the new to prevent that.
    B) Use the [/devbug] command, paste that into the report and fill in ALL the fields.
    QA will thank you.
    C) No feedback. (Feedback is encouraged but doesn't qualify for the lottery).
    So that you don't get "bug" reports that the mines don't have enough nodes etc.
    D) QA have the mandate to disqualify a bug report.
    So that you eliminate lawyering that they qualified through ABC but it still was a troll report like the outhouses don't stink or something. It would be bad for the system if any bug report regardless of content would qualify.

    My third proposal is for QA to have the mandate to add a second lottery ticket to extra high quality bug reports or reporters that go above and beyond. If that was rare (1-3 per release) and since it is just a ticket that shouldn't be too much.
    My thought here is that if these quality reports would get tagged as such, [QA+] maybe, then those could be used as examples on how to do it for others and thus raise the bar. It would also be that extra pat on the shoulder that gets some to give it that little bit of extra which is worth so much for you. If they did that for at least one per release then @Berek could write something short in the update about "Quality Bug Report of the Release" with a thank you or something.
    Such immaterial rewards are usually worth a whole lot more to the type of people you want to encourage. Plus it gives the spotlight where you want it.

    Happy hunting

    Spoon
     
    Last edited: Sep 21, 2017
  19. Black Knight

    Black Knight Avatar

    Messages:
    74
    Likes Received:
    149
    Trophy Points:
    20
    Gender:
    Male
    Location:
    Denmark
    As soon as you create a system
    People start gaming it :)

    Aside from that, great to hear that you do appreciate the testers, and I commend you for trying to provide some incentives.
     
  20. Alley Oop

    Alley Oop Bug Brigade – Bug Hunter

    Messages:
    6,317
    Likes Received:
    10,737
    Trophy Points:
    153
    i'm not too worried. they'd have to hit the four-hour mark with each character.