My opinion of the number 1 thing to fix

Let me start by saying I enjoy this game, i'm a big fan of sandboxes. I also don't think i should enjoy everything in a sandbox, if I enjoy EVERYTHING in the sandbox the sandbox it is probably not broad enough. To me sandboxes are a place for players to create their own fun, as long as a section of community is enjoying a part of the game then i'm happy that part of the game exists, even if i personally don't enjoy it.

The biggest issue i have with this game is the level QA that is done.
This is an over arching problem to me that lets down lots of things, some of which I personally don't care about, but as said above in a sandbox that is irrelevant. I understand that lots of testing resource is perhaps not within reach of Portalium. But there should be a minimum quality of each "feature set" that as a customer i can expect.

Do I expect the game to be bug free? NO.
The only way software is bug free is if it is so simple that you can easy test all situations, a classic hello world program is probably the only bug free software i have ever written.

So in the interest of constructive feedback, I would make the following suggestion. Feel free to comment on this (and hopefully) develop it, or suggest a better alternative.

Suggestion : Restructure the way you do and the incentive for QA testing by players.

This is not a "quick" solution, but i hope it is a solution that can start with baby steps and grow, with a huge advantage that at least initially should not need developer resource. So can be spread around the team for any member with some work bandwidth or even a community member to do.

Step 1
Write Smoke tests for feature sets of the game. In the company i work for a smoke test is a test of minimum quality. If a build does not pass the smoke then it does not get released until it at least passes that test. For example if i was writing a smoke test for something like the amazon website, my smoke test would be i can see the homepage, search for product, get a result, click buy, and go through the payment system and that get put into the back end system for distribution. If a user can not find what they want and pay for it then I would lose a lot of money so that is a the minimum quality i would be prepared to live with.

Write equivalent "scripted" (i mean in text not code) for different areas of the game.

So for the magic mover for example you could have a smoke test of (though i hope written with more care and attention than i gave this example)
1. Ensure you have a house claimed with deco, including at least one non empty containers.
2. Put the house in the magic mover.
3. Confirm it shows up the propert manager.
4. Use the property manager and put the house back on the lot.
5. Does everything appear to be present, did things move?
6. Check that a container still has things in it.
7. Put it back in the magic mover.
8. Move it to another town, did it cost the correct number of COTOs?
9. Did it move correctly to the new town.
10. Pull up the lot and ensure all items returned to the local bank.

Then just ask for any comments as a catch all.

Other smoke tests would be crafting using batches and singles, refining using batches and singles (as uses more skills), going to scene and killing a creature with a given set of skills (checking each can damage it). With the quest reset functionality (which you should always have available on QA, even if done before) you can do the same with quests.

Step 2
Give each smoke test a point value based on how long it takes to complete the smoke test. This is because your customers are gamers, therefore you should expect everything you create to be "gamed". So if one smoke test takes ages and another a short time, (most) people will just do the short one if it is for the same reward.

Step 3
Publish the smoke tests and when people complete the smoke tests in QA they "earn" the smoke test point value.

Step 4
Run the points like the raffle, so those that do the most/longest tests get a better chance of winning. The player can do their own testing too for balance and stuff that smoke testing is not very good at and give points for this, but only if they do a minimum amount of points in smoke tests first.

These smoke tests then become an ever growing and more refined minimum standard of testing for that feature set. As time goes on the smoke tests can be reviewed and updated, as bugs are found add a test to the smoke test if they are considered critical. With this approach quality should always go up over time.

It is also a great resource for the specifications and success criteria for any automated testing added further down the line.


Regards
Spung

 

Very well thought out. I agree that something of this nature needs to happen.

 

They did have a different reward system. Players gamed it. It was not productive. I did not reach the highest-level rewards because I wasn't over-doing it, but I still keep the title up because it's cool.

I dunno, perhaps if they had caps on the number of "points" you could get every month, that would have kept some people from basically hogging all the bugs as quickly as possible, and posting bugs without even checking for duplicates.

Also, players are not on the clock like regular game testers would be. I go in and try stuff out when I can. I certainly can't be there for a daily post-release patch cycle.

 

The methodology that Spungwa outlined seems to have a maximum amount of smoke points someone could receive built into it.

If each thing that they want tested is worth a certain number of points. They can easily control how many points are given for each task. Especially if each task can only be done once to receive points. This way players would go through each task kind of like quests building up as many points as they can to try to earn rewards.

Yes the people that do more tasks would have a better chance of winning rewards but even a person that only does one or two has still got a chance to win.
And as far as people gaming the system this is one instance I would love to see it. If you're only able to get points once per task they would have to do multiple different tasks set forth by the developers to earn the points. Completing the goal of testing whatever is needed.

In my mind this would require the developers to have some tools to automatically record players doing these actions. And not solely relying on them sending bug reports.