1. Here you will find official announcements and updates. These announcements are also linked in the Official SotA Discord server.
    We encourage comments from the community! To keep the announcements official, we ask that comment threads be created in the General forums for player input.

                                                 Thanks!

Testing Incentives Program Suspension

Discussion in 'Announcements' started by DarkStarr, Aug 21, 2017.

Thread Status:
Not open for further replies.
  1. Lexie

    Lexie Bear Queen Emeritus

    Messages:
    842
    Likes Received:
    2,902
    Trophy Points:
    93
    @redfish
    I think today's changes alone will help things a lot. I fully stand behind our company's decision on this. We could have had the best reporting system and tools in the world, but it wouldn't have necessarily stopped some of the repeated behavior we were seeing. I try to get to every post, but as you can imagine, it is a lot of work and time spent.

    As corny as this is, Sota would not be the game it is today if it wasn't for players submitting information on the problems they experienced. I don't want anyone who reported bugs to ever think their input was a waste of time or that their work is not appreciated. We will still be collecting bug information, of course. Thank you, thank you, thank you, redfish - and others - for what you do.
     
  2. Barugon

    Barugon Avatar

    Messages:
    15,678
    Likes Received:
    24,293
    Trophy Points:
    153
    Gender:
    Male
    Well, I'm not going to stop reporting bugs. I might make extra effort to ensure that I'm not posting something that has already been reported though.
     
    Net and Jens_T like this.
  3. Sir Leonard

    Sir Leonard Localization Team

    Messages:
    344
    Likes Received:
    779
    Trophy Points:
    43
    Gender:
    Male
    Location:
    King's Landing
    Fully understand this, but i am also one of the guys that never understood why stick with forums and rely on forum tools to bug reporting.

    There are several tools more specialized on bug triage and if we'd just have access to it (maybe make it Dev+ related), would be easier to not submit duplicate bugs, cause they would all be visible in the same place along with status.
    The forum system just made it dreadful to search for previous reports, too much partitions and old releases to go through, you got lost on subjects and titles.

    A way to sumbit standarized versions more reliable is required. I know bug brigade was doing kind of a job like this, but still don't know about how efficient is to dig into forum threads anyway.
     
  4. null2

    null2 Avatar

    Messages:
    373
    Likes Received:
    1,152
    Trophy Points:
    55
    We definitely appreciate the hard work that you all go through to work with players to reproduce issues. I will miss the incentive program, but will continue to test and submit bugs.
     
    Rinaldi, Daxxe Diggler, Net and 2 others like this.
  5. Lexie

    Lexie Bear Queen Emeritus

    Messages:
    842
    Likes Received:
    2,902
    Trophy Points:
    93
    YAY!

    I love how you tend to submit a lot of combat related bugs. Those are usually the hardest bugs to report, IMHO. You have a good eye for noticing patterns and abnormal behavior in very fast paced situations. And how others report deco bugs, or item bugs. Some are more visual bug detail oriented. It's definitely a team effort.
     
  6. Fionwyn Wyldemane

    Fionwyn Wyldemane Avatar

    Messages:
    1,841
    Likes Received:
    6,079
    Trophy Points:
    125
    Gender:
    Female
    Location:
    Chicago, IL
    Like many other backers, I was submitting bug reports before the program and will likely continue submitting reports.

    Here's my question about all this: is suspending the program actually going to resolve the underlying problem of too many duplicates being reported? I don't think so. Maybe you're hoping less people will report bugs if there is no incentive to do so?

    My humble suggestion is NOT to suspend the program entirely, but keep it as a program for Dev+ ONLY.
     
  7. Tiina Onir

    Tiina Onir Avatar

    Messages:
    1,103
    Likes Received:
    1,900
    Trophy Points:
    125
    Location:
    Bramble, South Paladis
    To be clear, I'm not trying to belittle the work you (and company) are doing, I just think the community might be able to help more if we had the right tools to do so. If we could take some of that load off your team, they could focus on the things we CAN'T help with.

    I'll admit I don't do bug reports on dev, but even on Live I think that having proper bug-reporting system would be useful.
     
    Sentinel2 and Net like this.
  8. Net

    Net Avatar

    Messages:
    3,727
    Likes Received:
    11,178
    Trophy Points:
    153
    Nooooo, I was working hoping to get that bug spray emote...

    I really hope the program will be back with some modifications:
    - there should be transparent system to see how many "bug points" we got
    - it should be easy to locate previously submitted bugs and see their status (low priority, estimated fix in release 47...)
    - people should get bug points for improving on previously posted reports, adding additional steps to reproduce them, noticing the issue elsewhere

    The forums are mess and looking for duplicates especially in older forums is something dedicate bug brigade member can do, but not something for average person. Several of my friends just do not want to report any bugs because it is a lot of time writing proper bug report and it is usually not quick fix. Also it is hard not to post duplicates between releases. Some things bother me each release and at least in the past my experience sadly was that devs were not even aware of my previous bug reports. That seemed to change, but even though now devs are aware of bug reports, some old bugs have not been fixed yet.

    And another thing are known issue, another document that is good for bug hunters to filter out bug reports but not something normal person to spend their time looking for if their issue is known.

    I really hope that Portalarium will work on making bug submitting easier and more effective. I hope they will reward those, who produce original and detailed bug reports. I think that small reward for the rtrouble for everyone would be nice and worth the extra duplicate reports if they were merged before they reach devs. Reproducing some rare bugs and helping narrow down the settings and hardwware that is causing (or not causing) them is something that requires a lot of time and effort but is not currently rewarded.

    Anyway the best reward for submitting bug report is seeing the bug fixed. Which is not always the case (I understand that some bugs are low priority, but it still sucks if you deal with the same issue for three releases and nothing changes despite continuous reporting).

    PS: Getting lower number of bug reports does not mean there is less bugs in the game.


    PPS: Of course I will continue post bugs, I will just be super annoyed when I run out of bugs and the reward program is restarted:D
     
    Last edited: Aug 21, 2017
  9. Tahru

    Tahru Avatar

    Messages:
    4,800
    Likes Received:
    12,170
    Trophy Points:
    165
    Gender:
    Male
    Location:
    Spite
    Have you guys considered using a bug tracking system that is friendly to the Internet? I know you use JIRA and it is probably butt slow and there is a fear of it getting polluted with junk. So, using another bug tracker could work well as an intermediary, something with features to help prevent duplicates.
     
    Sentinel2 and Net like this.
  10. Net

    Net Avatar

    Messages:
    3,727
    Likes Received:
    11,178
    Trophy Points:
    153
    Now that QA is open to everyone I doubt they wil lack testers for at least several releases. So there is no need for COTOs for playing QA though they were nice reward and helped me pay my taxes.
     
    Last edited: Aug 21, 2017
    Sean Silverfoot and Sentinel2 like this.
  11. Sentinel2

    Sentinel2 Avatar

    Messages:
    1,109
    Likes Received:
    1,811
    Trophy Points:
    113
    That's been one of my big issues with bug reporting. I've searched the forums and apparently my search terms didn't quite match how others reported the bug I submitted. I've asked Attenwood a couple years ago how we should search for bugs to avoid dupes. He said just to search the forums. :)

    There are tools out there that have already solved this problem. We could be better utilized IMO to take off some of the pressure from Portalarium devs. We have a lot of talented people who are willing to help!
     
    GreyMouser Skye, Tiina Onir and Net like this.
  12. Sentinel2

    Sentinel2 Avatar

    Messages:
    1,109
    Likes Received:
    1,811
    Trophy Points:
    113
    If we're not getting Jira access, maybe setup a separate instance of Jira for public bug reporting? Or setup bugzilla (or whatever secondary product).

    Anything other than searching the forums. That's obviously not the solution ;-)
     
    Vladamir Begemot and Net like this.
  13. Net

    Net Avatar

    Messages:
    3,727
    Likes Received:
    11,178
    Trophy Points:
    153
    There are more issues with duplicates. Even if I find bug that was reported severla releases ago, I still need to know:
    a) does the team know about the bug?
    b) was it low priority and is yet to receive fix?
    c) was it fix, but broke in subsequent release
    d) was it never properly fixed

    It would be great to see status of bugs based on their number, to check if those "fixed" bugs are really fixed. And to know if devs know about the bug but due to small team and more serious bugs did not have time to work on it. forum tags and bug ids do help a bit, but forum is not good place for bug searching and reporting.
     
    discipleofweb, Sentinel2 and FrostII like this.
  14. Tiina Onir

    Tiina Onir Avatar

    Messages:
    1,103
    Likes Received:
    1,900
    Trophy Points:
    125
    Location:
    Bramble, South Paladis
    In fairness, Portalarium's complaint seems to be (from what I can tell) duplicates in the same release.

    That said, I totally agree that bug reporting works best when it's a two-way street. I report the bug, but from that point I don't know what's happening with it (particularly between releases). What kind of priority is it given? Is there a target fix release? When release day comes, has it been fixed? This type of information helps bug-reporting immensely. Without it, devs can think something's fixed, while we think it's just not been gotten to yet.
     
  15. Berek

    Berek Portalarian Emeritus Dev Emeritus

    Messages:
    3,957
    Likes Received:
    12,761
    Trophy Points:
    165
    Gender:
    Male
    Location:
    Austin, TX
    Thank you Barugon!

    We would be happy to consider other tools. I agree that the forums are not ideal here by any stretch of the imagination. It's all we got atm though.
     
  16. Alleine Dragonfyre

    Alleine Dragonfyre Avatar

    Messages:
    1,750
    Likes Received:
    4,695
    Trophy Points:
    125
    Gender:
    Female
    Location:
    Immortal City
    This is what I found discouraging about reporting bugs. I know it's hard to sift through all this information for devs and so because I couldn't really find duplicates to what I found, I figured I'd not submit to reduce the noise. I'm new and chances are what I was seeing, others already have anyways.
     
    Duke Gréagóir likes this.
  17. Alleine Dragonfyre

    Alleine Dragonfyre Avatar

    Messages:
    1,750
    Likes Received:
    4,695
    Trophy Points:
    125
    Gender:
    Female
    Location:
    Immortal City
    I don't understand why you feel less eyes on the game is a good thing.
     
  18. Lexie

    Lexie Bear Queen Emeritus

    Messages:
    842
    Likes Received:
    2,902
    Trophy Points:
    93
    No problem, as no belittling was interpreted by me :)

    The best thing our bug reporting community can do right now is this:

    1) Read the following link and spread the awareness:
    https://docs.google.com/document/d/1IgbhfeXt8mTLZaY7cXGf3m-Pf-3BodoRK1-sVuHlHr8/pub

    2) Encourage other bug reporters to use the QA tools that we DO have now, and that is by using /bug on live or /devbug on QA. I've been pushing for more to use it, and some are. Although not as many are receptive to using it as I would like, despite the fact it really does cut down on the amount of typing they need to do ;) Some took to using it, although others continued to submit bug reports without it, giving me tad bit of pushback - from time to time.

    For those who are not aware, when people type /bug at the location of the problem it copies the bug report template AND auto-fills out some of the information. It also opens a forum browser link to the QA bug forums.

    At this point I just want people who do report to use the QA tools we have now :) MANY are EXCELLENT with their reports and following the rules, and having everyone follow the same standard of reporting is my next goal.
     
    Last edited: Aug 21, 2017
  19. Vladamir Begemot

    Vladamir Begemot Avatar

    Messages:
    6,194
    Likes Received:
    12,076
    Trophy Points:
    153
    Gender:
    Male
    @Lexie does /debug on QA launch the QA forums? I've always just used it to see FPS. If that's the case, oh man the headaches I would have been saved by not typing /bug, then browsing over the the QA forums every time!

    And yes, everyone should use /bug, it saves a boatload of time!
     
    Net likes this.
  20. Womby

    Womby Avatar

    Messages:
    3,299
    Likes Received:
    12,165
    Trophy Points:
    153
    Location:
    South Australia
    So it seems like there were two main problems:
    1. reporting of the same bug by multiple testers
    2. reporting of bugs that are virtually impossible to reproduce

    I know that I was unable to reproduce some of the bugs that I have reported - for example, the client hanging and incorrect reporting of container weight in the user interface.
    Should we not report bugs that we know happen rarely, and are virtually impossible to reproduce? It seems like it might be more important to concentrate on bugs that players are more likely to come across.
    On the other hand, it seems like their existence should be made known, even if no action is taken in the short-term.
    I guess in future I could mark these "Information only - couldn't reproduce.", and anyone else who sees it can add to the thread. That could be a way to determine how much of a problem it is, without wasting dev time on a bug that can't reliably be reproduced. Maybe over time, a pattern might emerge.
     
    Net likes this.
Thread Status:
Not open for further replies.