1. This forum section contains Blog style threads from the Developers, and/or threads with details about what they have been working on, previews of upcoming content, and sometimes backstory associated with the content. Players are permitted to reply to these threads if the developer has indicated it is permissible by leaving the thread unlocked. Please note that TOS applies, and any abuse or trolling will be moderated.
    Dismiss Notice

Public Stats API

Discussion in 'Developer Work and Blog posts' started by Undone, May 9, 2017.

Thread Status:
Not open for further replies.
  1. Ben_Hroth

    Ben_Hroth Avatar

    Messages:
    261
    Likes Received:
    739
    Trophy Points:
    28
    I must be a dummy... I'm not seeing artifacts, and I was curious about them (not for abuse, but just trying to find more artifacts!) Can you give me a hint on a specific Archetype that matches to an artifact?
     
  2. Elnoth

    Elnoth Avatar

    Messages:
    724
    Likes Received:
    1,557
    Trophy Points:
    93
    I just used -st Artifacts

    It lets me see the new artifacts that have dropped too:

    [​IMG]
     
    hroth and Alley Oop like this.
  3. DavidDC

    DavidDC Programmer Moderator SOTA Developer

    Messages:
    1,532
    Likes Received:
    3,236
    Trophy Points:
    113
    Gender:
    Male
    It let you see artifacts in the db but those havent dropped yet i think..
     
    hroth likes this.
  4. Elnoth

    Elnoth Avatar

    Messages:
    724
    Likes Received:
    1,557
    Trophy Points:
    93
    they dropped - thats a sample of the 1000+ lines I pulled of every artifact that dropped in the last 2 weeks. For example - the boots artifact listed in that pic has this info:

    • [122].ypos">"ypos": "160.573",
    • [122].zpos">"zpos": "15.395",
    • [122].xpos">"xpos": "233.12",
    • [122].Archetype">"Archetype": "Equipable_Archetypes/Artifacts/Release41/ArchetypeArmor_Boots_Of_Skulking",
    • [122].Quantity">"Quantity": "1",
    • [122].EconomyGoldDelta">"EconomyGoldDelta": "8",
    • [122].ItemId">"ItemId": "591275e9abb70d05c0fb4375",
    • "@timestamp": "2017-05-10T02:07:37.000Z",
    • [122].SceneName">"SceneName": "The Fall",
    • [122].LocationEvent">"LocationEvent": "ItemGained_Loot",
    • [122].timestamp">"timestamp": "May 9 21:07:37"
     
    hroth likes this.
  5. DavidDC

    DavidDC Programmer Moderator SOTA Developer

    Messages:
    1,532
    Likes Received:
    3,236
    Trophy Points:
    113
    Gender:
    Male
    yeah saw it, tried with archetype but i t didnt work i was doing it wrong... oh well i was hoping loot was part of the mysteries
     
  6. Umuri

    Umuri Avatar

    Messages:
    527
    Likes Received:
    1,828
    Trophy Points:
    63
    More pages added to Umuri's SotaSTATS.
    Economy based ones:
    Raw materials (1 hr, 24 hr, ALL)
    http://umuri.com/SOTA/Stats/Top25/ALL-RawMaterialsGained.php
    Gold by Scene (1 hr, 24 hr, ALL)
    http://umuri.com/SOTA/Stats/Top25/ALL-SceneGoldGained.php
    Oracle Tests Passed (1 hr, 24hr, ALL)
    http://umuri.com/SOTA/Stats/Top25/ALL-OracleTestPassed.php
    Interesting enough that means we can assume (20% error rate) that 600 people passed their oracle test in the last 24 hours! Nifty!

    Re: the discussion on the rest.
    @tuerdkughter - @Undone is quite aware of how it can be reconstructed, however rate limiting the API as discussed will limit that particular information leak. Althouhg it will still allow you to altert when certain scenes are active (such a shardfalls). Which just means an easy solution is to not report PlayerMove events in PVP Zones. I've been letting him know of any exploitable datasets/pairings that are available and what possible ways to fix it there would be while preserving our access, since my goal is as always putting everyone on equal footing. I would much rather we lose access to data than to have certain people have unfair advantages.

    IMO I will disagree with Undone that loot tables are worth worrying about as the important ones have already been reconstructed through manual testing/records. PVP information (Logon/logoff times, hunting zones, which zones have people hunting right now) is way more important to make sure we properly santize.

    Also, remember, this API will probably be locked down once he's done setting it up/testing it, and odds are it will be put behing auth tokens to track who uses what, and ensure proper usage. So applications that query every 15 seconds will probably be scrutinized and revoked before they get too far. :)

    I'm personally still pushing for hoping he gets a time delay feature working, so that we are 24 hours or so behind on logs. Thats still plenty for analytics while removing most PVP advantage.

    Also, in the interest of information sharing, since none of the others have bothered mentioning it yet: The R41 artifacts have only dropped so far in "The Fall". For those who don't get to dig in the data. I omitted artifacts from my scene list pending chris saying it was ok to list em but since everyone else is yapping about em, lets make it fair.
     
    Last edited: May 12, 2017
    smack, Spoon and Tiina Onir like this.
  7. DavidDC

    DavidDC Programmer Moderator SOTA Developer

    Messages:
    1,532
    Likes Received:
    3,236
    Trophy Points:
    113
    Gender:
    Male
    if you see an item lootgenerated in a shardfall last hour you still know there might be someone all depending the amount of lootgenerated there was...
     
    Elnoth likes this.
  8. Tiina Onir

    Tiina Onir Avatar

    Messages:
    1,103
    Likes Received:
    1,900
    Trophy Points:
    125
    Location:
    Bramble, South Paladis
    *snipped to what I was responding to*

    I also agree, the loot tables are always being worked out via manual testing, whereas PVP info will more affect individual players.

    I definitely agree with you, wholeheartedly, on a time delay. For the purposes of the game stats I'm interested in, running a day late is just fine.

    I can't imagine listing where artifacts are dropping as being verboten, as we can kind of see the pattern (at least somewhat) just from talking amongst out selves in any of the larger guilds. Besides, it seems they are using them to encourage people to go to certain areas.
     
  9. DavidDC

    DavidDC Programmer Moderator SOTA Developer

    Messages:
    1,532
    Likes Received:
    3,236
    Trophy Points:
    113
    Gender:
    Male
    Some SceneName like Novia_R4_Desert01_01 arent searchable for some reason when used with archetype... AND SceneName.. But we can still see them if we go global without specifiyng a scene name or just by specifying a scenaname alone, any reason why?

    Edit: Trying to delete my post but cant... i was using a = instead of : all day thats why it didnt work .. ;p
     
    Last edited: May 13, 2017
  10. Scoffer

    Scoffer Avatar

    Messages:
    905
    Likes Received:
    2,651
    Trophy Points:
    93
    Just to clarify as I'm a little confused.

    Some of this data should be hidden because those people who are PvP flagged would be able to find other likeminded pvp folk and go fight?
    Isn't that the point of PvP?
    Hasn't there been numerous posts on the forum already about no one being available to pvp?
     
  11. Tiina Onir

    Tiina Onir Avatar

    Messages:
    1,103
    Likes Received:
    1,900
    Trophy Points:
    125
    Location:
    Bramble, South Paladis
    Just because you're open to PvP, doesn't mean you want people to be able to use out-of-game-information to track you down. It's the one thing I've seen, so far, as being an unfair advantage, and it's worth putting a delay on the feed to keep it from being exploited.
     
  12. ikickbutt

    ikickbutt Avatar

    Messages:
    45
    Likes Received:
    100
    Trophy Points:
    8
    Ummm - couldn't this get exploited by pvp guilds? ie 'theres someone in a pvp zone lets gank em' ?
     
    Tiina Onir likes this.
  13. that_shawn_guy

    that_shawn_guy Bug Hunter

    Messages:
    1,415
    Likes Received:
    3,753
    Trophy Points:
    125
    Location:
    earth... mostly
    If someone ask opt-in to being anonymous, is there a good reason why their location data should not also be removed? If that data would be used to find busy zones, could that not be handled by always showing their location at 0,0,0?

    From what I can gather it is the location with timestamp data that is the majority concern. I'm trying to find a way to solve that issue without reducing the value of all of the other data.
     
  14. Tiina Onir

    Tiina Onir Avatar

    Messages:
    1,103
    Likes Received:
    1,900
    Trophy Points:
    125
    Location:
    Bramble, South Paladis
    Yep, that's why some kind of delay to make the info less useful for that is proposed.

    Removing the location data prevents generating accurate area heat-maps (which is exactly the kind of statistical information we want to facilitate, I think). Removing the timestamps doesn't do that, nor does putting a delay on the feed.
     
    Wintermute of CoF likes this.
  15. that_shawn_guy

    that_shawn_guy Bug Hunter

    Messages:
    1,415
    Likes Received:
    3,753
    Trophy Points:
    125
    Location:
    earth... mostly
    If this were purely PvP location related data, I would agree. However, removing timestamps or severely delaying crafting and economic data considerably reduces it's value.
     
  16. Phil Scott

    Phil Scott Avatar

    Messages:
    74
    Likes Received:
    41
    Trophy Points:
    20
    Gender:
    Male
    Given that this area is going to be .... detailed I think that it would be a good idea to create a new forum area. This would allow topics to be split up.
     
    that_shawn_guy likes this.
  17. Phil Scott

    Phil Scott Avatar

    Messages:
    74
    Likes Received:
    41
    Trophy Points:
    20
    Gender:
    Male
    I agree, delay any data that has player location (but do keep the exact time) but for crafting and economic entries we do need live data.
     
    that_shawn_guy likes this.
  18. Phil Scott

    Phil Scott Avatar

    Messages:
    74
    Likes Received:
    41
    Trophy Points:
    20
    Gender:
    Male
    Do Archetypes have a 1 to 1 relationship to the Item Name, if so could we get a table at some point to show people the names they see in game. If not could we get an attribute/key for the public name.
     
    that_shawn_guy likes this.
  19. Jaanelle DeJure

    Jaanelle DeJure Avatar

    Messages:
    1,845
    Likes Received:
    4,276
    Trophy Points:
    113
    Does the API tell us anything that would help quantify the "Gap" between new players and vets?

    It would be nice to have some hard data on this. That way we don't need to endlessly speculate about it in the forums.
     
  20. Leinad Laurelwreath

    Leinad Laurelwreath Avatar

    Messages:
    253
    Likes Received:
    575
    Trophy Points:
    28
    Gender:
    Male
    Location:
    Jaaanaford
    This data, as well as the game online population, is one of the questions that will get you *crickets* as an answer. I'd even prefer a "No, because X reason" but you'll get crickets. Only @Chris knows this.

    (someone prove me wrong, please!)
     
Thread Status:
Not open for further replies.