Suggestion: enhanced QA incentives.

I was listening to chat and it cropped up that the QA server population can be quite sparse. It's clear there are some dedicated testers, but as Vlad says elsewhere, maybe with such small numbers testing, certain things that are likely to fail with intense use in groups or in busy zones don't get a hammering?

Confession time: thus far I have not tested. Its a bit of a drag setting up the QA branch in Steam, then reverting to live, but mainly I don't test as when I want to play I want to play on live and make progress. To a lesser extent I want to see the finished features on release for the first time as polished new content to get the full experience.

However it seems we are all needed to help test. Experiencing new polished content isn't going to happen if it isn't tested thoroughly by significant numbers of folk in all different ways first. Port are tiny and don't have paid, professional Testers so rely on us.

What would get me onto the QA server?

- firstly I think I ought to anyway. We're a small population of players at the moment and I feel I need to dive in and help.

- Maybe I don't have the right to whine when stuff is broken or not functioning optimally on live if I haven't gotten my hands dirty on QA?

- monthly release cycles are so onerous on content quality at release and given the majority are in favour of this release cycle, then it has to be supported on QA with bodies.

But despite this and more most don't help out and test, I don't (or didn't)

What I suggest to the Devs is that they incentivise QA like crazy -

- pass over your total accumulated (including used) exp points on QA to live for you to use

- award 1 COTO per QA session over an hour or something similar

- Any artefact or rare drops from Boss kills applied on live as well

- Registering and playing on QA grants titles and rewards in live

- Managed group content led by GMs and Devs to test Group play and new scenes and content

- fishing expeditions and dance parties to test load ballance and performance
- more...

Thoughts?

 

Excellent ideas. Perhaps they could package some or all of your gains (loot, gold, XP, etc) in a Bundle that you can claim on Live.

Oh and bring back /zone while they're at it, as your friends teleport list will be much smaller.

 

Maybe when they rework the quest system they can make it so that on QA there are test script quests which when completed yield rewards on prod.

 

Well, FWIW I won an Ant Farm this release. It's pretty cool, it's got little animated ants.

The QA rewards are some of the rarest in game I believe, due to them not being able to be bought. If I were into selling things for cash it would be pretty valuable.

It's not a guarantee, but for anyone into collecting or having sellable assets, it's a good thing to spend some hours on already.

I just wish the Corpion chair was still being awarded, even occasionally! I can't believe I didn't get it in time.

 

I have an ant farm. I don't know why I got it anymore but it's my favorite thing. that said raffle tickets for rewards are kinda BS. Hours logged-> XP earned would be nice tho.

 

Maybe they could keep track of the experience you earn on QA and give that to your avatar on live.

[edit] The idea here is to remove a barrier to people playing on QA, which is having to put aside any progress in order to spend time on QA. They could even incentivize it by having double experience on QA.

 

I have a Royal Pest Controller title. That's my favorite thing from testing.

 

I'm hopeful it morphs into more than a lottery.

I've submitted bugs. And I'm glad to do it. I have a vested interest in making this the best game possible. Especially for fellow Linux users

Still, it would be nice if the lottery were done away with. I can't see this process being any better than it's predecessor.

 

They were doing some of the things you suggest, but as usual, the rotten apples spoiled the bunch.

As a quick for instance, they used to give cotos away for time spent QA'ing, but then people just logged in and collected cotos on live without really doing anything.

The lottery system came about because they couldn't come up with a non-exploitable way to give rewards for true participation.

 

I am (still) testing - like others - every month on QA - well, over the last years.
[You can read about in the bug sections - most time I am hunting quests ]
I assume I got almost all the special stuff which can be won while testing. But - after 4 years not all ...
The special created bug trophies are really nice (well, until now no "Crown of the Bughunter" or the "Bughunter Spear - with special Poison bonus" can be earned, but who knows ... someday?)

I am not a fan of raffles, because it's always a kind of coincidence.

Just an example:
if you found 10 bugs on QA in the 4 hours which was said we have to test at least - and another one found only 1 bug, may be he was the winner of this QA this month.

My suggestions for a fair play:
A calculable rule would be fine, like:

1 hour of testing grants 1 COTO.
2 hours grants 3 COTOs.
...
maximum X hours / player

So every player could decide if he wants to spend time, knowing about a granted reward.

This only honors the time someone stays in game, not if he really searches bugs...

So let's do something more interesting for a long term success - make bug hunting an event

Count the bugs! And add them to the rewards list.

After someone found an amount of bugs he could redeem depending on the amount ... (just examples ...)
10 bugs: Antenna
20 bugs: Cloak
30 bugs: ...

100 bugs: Ant Farm

500 bugs: Giant Ant Pet

750 bugs: special Ant Armor ...

1000 bugs: Special designed Ant Basement / Dungeon / House ...

This would be a fair and amazing bug hunting system to encourage players to invest their time

And I think, it would be -again- a feature which can't be found in another game ...

 

Bug hunts can be a bonus but shouldn't be the only thing to reward testing time on QA.

Verifying that something was fixed or confirming other things aren't broken is also valid QA testing. Merely installing the QA client and running it is a test in itself, if only to check for 98% load bug or CTDs on startup or exit. Those activities don't generate bugs but is also time spent by a player on QA.

Even something as simple as exiting and re-entering instances, without doing anything else, would be checking on performance degradation. Or running around Central Brittany to see if FPS dropped. Those activities are harder to track and correlate but again is time being spent testing. But they trivially calculate your AFK time as idle time. And they can change it so AFK happens much, much quicker on QA.

Any non-AFK time spent on QA can also be rewarded in some way and it can be capped in some way if needed.
Any gains in QA can be bundled and claimed on Live and this too can be capped in some way if needed.
Any bugs found can be rewarded using whatever gamification scheme they come up with.
They can do a lot more as they track a ton of stuff, but it's likely overkill.

 

I use to do a lot of bug reporting but then they made the QA server and stopped rewards on the live version. I'm on a data cap so I dont have the luxury of playing on QA very often. I still report some bugs I find but would love to see those that catch them on live rewarded as well.

 

I dont think this dev team has time to put into QA, even back when it was huge. Back when we had rewards, I personally submitted dozens of actual videos showing bugs as well as many many more reports. All I ever got was the basic reward. I miss attenwood, he was a go getter on this stuff.

 

Before the change to the "lottery" system, it was a lot of Dev work to get bug reports filtered and prizes awarded. I do wish they had systems in place to make QA better for testing, but I don't think this is a short term goal.

What I'd suggest is:

• Add a "premium currency" that you can collect during QA. This will be a quest item, so can't be sold or traded.
• Add this "premium currency" to normal loot packages on the QA client, so players can just play and still get rewarded. Make them rare enough that they are fun to discover, but not CotO-rare. Like 1-2 per hour, on average.
• Add additional "premium currency" to items that you want heavy focus on during QA. For example, if you want us to test dungeon building, make them randomly spawn inside of player-made dungeons in design mode.
• After the server is closed, the amount of this premium currency is totaled and goes into the pool for rewards. Each is basically a ticket for the random rewards. I'd also say to give players a small xp bonus (500xp per currency?) and/or CotOs (1 CotO per currency?) as an additional motivation.

This can all run fairly automatedly, so little need for Devs to monitor (other than abuse prevention). It also feels rewarding when you can do something you already like to do, uncover bugs, and always get something for your time.

 

I have been doing QA for about 6 months or so. Mainly i just deal in quests/recipes and new zones. I dont dwell in the combat stuff. I think there are more out there better suited for that aspect. I enjoy going in and seeing the new content first hand and not for a spoiler aspect. I just like doing it. Its fun to find bugs...but not fun when they are found and not fixed by the time the game goes live. Some of my bug reports might be trivial to some....but i feel any bug reported how high end or low end should be checked and fixed. That is after all what we are there for. This last release...R60 i found quite a few really important bugs. Missing items from scenes and such. So i felt good about my hard work on checking things out. Still a few things that need to be fixed but not game stopping. Now if they would just fix the issue at hand I will be happy

 

I was under the impression they monitored our activity on the QA server. So you couldn't just login and sit there.

At least that was my understanding.

 

and what would stop you from logging in, just futzing around for a couple of hours, not really testing anything, and logging off?

Note that they also said they have things in place to detect macroers... and yet, I have watched people macro away at gathering water...

 

Sometimes futzing is needed, it catches bugs that wouldn't be seen if you weren't futzing. On the months I'm really on the ball I try to test a bunch of things that are my normal activities but not necessarily part of the release, as well as things I can do that relate to the release. That's how I find lots of UI issues.

And the rules are that you have to submit at least one good bug, so if you're just there without doing any actual submissions, because futzing didn't find anything this month, then you aren't in the drawing.

 

I'm not talking about the drawing... I'm talking about the situation BEFORE the drawing system existed, and some of the reasons they shifted to it. Another, the workload from sorting through the myriad of (and often, duplicate) bug reports that were being submitted from people who WERE actually testing, has been mentioned previously.

 

I found plenty of bugs "futzing" around as you mentioned it. Regressions happen

And I just reported a suspected macro player the other day. It looked rather odd. Even brought in a buddy to look in case I was off.