Public Stats API

Any word on this stat being public? @Undone Did you fix those problems @Umuri mentioned? I think this is one of the most interesting stats that should be public!

 

@devilcult

Code:

const elasticsearch = require('elasticsearch')
const urlencode = require('urlencode')

var client = new elasticsearch.Client({
  host: 'www.shroudoftheavatar.com:9200/'
})

const request = {
  "size": 2,
  "sort": [
    {
      "@timestamp": {
        "order": "desc",
        "unmapped_type": "boolean"
      }
    }
  ],
  "query": {
    "bool": {
      "must": [
        {
          "query_string": {
            "query": "PlayerName: 'devilcut'",
            "analyze_wildcard": "true"
          }
        },
        {
          "range": {
            "@timestamp": {
              "gte": "1494501864000",
              "format": "epoch_millis"
            }
          }
        }
      ],
      "must_not": []
    }
  },
}

client.search({ method: 'GET', source: JSON.stringify(request) })
  .then(body => { console.log(body.hits.hits) })
  .catch(error => console.log(error))

 

Don't worry, it will be coming back, in some form or another.
However the problems were not "breaks" or "bugs", they were intentional changes that Undone did, to help secure data and protect players privacy, which I wholeheartedly encouraged and agree with. People don't realize how much their data reveals sometime, when properly correlated with other data. It's why sharing data on the internet is so powerful to marketers nowadays.

I will continue to work with Undone to ensure that any data interpretations I find fit with Portalarium's desired level of sharing, and that we properly protect player data from unintentional reveals.

Short version: It will be back. Players shouldn't have all the data, pvpers will abuse it, so will griefers, so we gotta make it safe.

 

When this was first released, it was very easy to keep the result set below 10,000 using the time window. I'm now very concerned about doing that going forward.

Should we expect the correct timestamps to return for the non-PVP related data? If not, how should we control the size of the result set? i.e. if I want to retrieve all the crafting data for an hour if there are 12,000 "ItemGained_Crafting" events in that hour?

Thank you

 

Thanks @Umuri I understand privacy concerns and exploits must be prevented first.
However I was just asking for the number of players online, just a number with no extra data

 

Please consider randomizing the timestamps with in the 15-30m window. That would at least distribute it more and while hopefully provide enough protection.

Alternatively, you could consider pushing the json to S3 or other cloud store that would allow us to pull all the data without adding load to your systems. That would mean we have to process the data ourselves instead of using your systems. But, that has many benefits.

Thank you for getting this going and dealing with all of us demanding lot. . I am very grateful that the your team is working with us.

 

@Undone Given that 'price' is different depending on whether one is buying or selling to NPCs, could we also have the 'value' as it would be reported in a users inventory?

 

@Undone: just saw this yesterday and played with it a bit last night. Totally awesome, I'll certainly be using it. I also agree that it would be greatly helpful if a short description could be added for each of the LocationEvent types. Particularly confusing are:

• ItemGained_ExplodeItem_Merchant
• ItemGained_Merchant
• ItemGained_Merchant (why is this on the list twice? typo?)
• ItemDestroyed_Merchant

I'd also love to see this data (possibly anonymised) exposed for player-vendors.

 

@Undone What is the threat model for the exploits you are trying to prevent with the timestamp rounding? I haven't seen any explanation, other than Umuri's vague explanations about possible PVP focused players having an advantage of some kind.

For example, if your worried about PVP focused players using in-game location data to hunt other vulnerable players. I have already found a work around that allows me to track player locations without accurate time stamps using a machine learning technique called classifiers. So, if that is the threat your mitigating with obscuring the timestamps, it doesn't work.

Time is an important component of log data, obfuscating times shouldn't be done lightly. Especially if it doesn't address the threat your trying to protect against.

I realize that I'm making an assumption about the treat model.

 

Opting in also shows everyone where you like to play, or are playing.



Perhaps opting in to Categories Such as:
-PvP Kill stats
-PvP Death stats
-Suicides

etc.


Maybe I'd want to participate in some activities, but not have everyone know how many COTOs I purchased?

 

@Undone If I make a map that accurately shows player locations in realtime without using timestamps, will you turn them back on and admit that your security controls don't actually work or make sense? For example, the correct compensating control to loot table discovery is to make your loot generating routine use an random number generator or other source of entropy, not to hide the values for loot generation.

 

QFT... and thank you...

 

@Undone "If you wish to build a real time map of player locations then you're likely going to get me to just drop PositionUpdates from the list or just disable the API all together seeing as you'll be querying the stack far too frequently. These sorts of things are not what this API was meant for."

Haha, so because you don't understand security controls your going to punish users of the API. Awesome attitude.

 

I think the potential for economic abuse of this data needs to be considered.

For example, I can pull every artifact dropped in the last two weeks and see which components of the warlock chain have dropped the least and know that they will be worth more as they are harder to find.