1. This forum section contains Blog style threads from the Developers, and/or threads with details about what they have been working on, previews of upcoming content, and sometimes backstory associated with the content. Players are permitted to reply to these threads if the developer has indicated it is permissible by leaving the thread unlocked. Please note that TOS applies, and any abuse or trolling will be moderated.
    Dismiss Notice

Public Stats API

Discussion in 'Developer Work and Blog posts' started by Undone, May 9, 2017.

Thread Status:
Not open for further replies.
  1. Tiina Onir

    Tiina Onir Avatar

    Messages:
    1,103
    Likes Received:
    1,900
    Trophy Points:
    125
    Location:
    Bramble, South Paladis
    This is definitely the right place.

    @Rowell could tell you more, but I think sotatraders depends on the users to updated their vendors, because player vendor information is NOT accessible via API. There are a few of us that have wanted anonymized historical data for a while, but there is concern about it not being perfectly anonymized, and being able to use it to gain too much of an advantage.
     
    Sonja likes this.
  2. Rowell

    Rowell Avatar

    Messages:
    612
    Likes Received:
    1,281
    Trophy Points:
    105
    Gender:
    Female
    Location:
    Brittany Fields
    I'm mainly pulling data about items (created items, destroyed items, items sold to vendors, items bought from vendors). I map (as best I can) the archaic item names the API uses to items in my database, and use the vendor prices gleaned from the SOTA API to populate my base (vendor) prices. I simply make a call to the API with one of the following queries:

    http://shroudoftheavatar.com:9200/_all/_search?q=@timestamp:>now-1h AND LocationEvent:ItemGained_Merchant&_source=&size=500&pretty
    http://shroudoftheavatar.com:9200/_all/_search?q=@timestamp:>now-1h AND LocationEvent:ItemDestroyed_Merchant&_source=&size=500&pretty
    http://shroudoftheavatar.com:9200/_all/_search?q=@timestamp:>now-1h AND LocationEvent:ItemGained_Crafting&_source=&size=500&pretty
    http://shroudoftheavatar.com:9200/_all/_search?q=@timestamp:>now-1h AND LocationEvent:ItemDestroyed_Crafting&_source=&size=500&pretty

    These get different aspects of item prices of the last 500 items over the last hour. The returned data will have to be read in and unpacked (it's JSON).
    One thing that does not exist (at this point) is that player vendor information (current or historical) does not exist. A few telethons ago, I asked if the SOTA API would ever include that data, and the response was basically "Yes, eventually"... but the devs have a lot on their plate, so I'm not going to hold my breath to get it any time soon. Otherwise, my site would have a better snap shot of the pulse of the economy.

    Earlier in this thread, there are several links which describe what all the parameters are acceptable to the SOTA API. Take a peek at those, and see if there's anything that you could find useful.
     
    Sonja likes this.
  3. Warrior B'Patrick

    Warrior B'Patrick Avatar

    Messages:
    1,222
    Likes Received:
    2,409
    Trophy Points:
    113
    Gender:
    Male
    Location:
    Athens, Georgia, USA
    Greetings all. Thank you @Tiina Onir and @Rowell this was very helpful. I understand how the data could be used in a non-beneficial ways and fully agree, it is not the data that is bad it is the way it is manipulated. Since what I am trying to do is city and player vendor specific I think I will just have to spend the time running around the town manually gathering data. I do think it is funny though that they talk about a 25 hr delay on showing data but I can run around the town and gather it manually much quicker than that. Just need a way to parse the vendor list to a text file and I would not need the API.

    Thanks again and I will look thru the thread again for those links. I do want to thank both of you for your sites. I have visited both of them and found then useful.
     
    Sonja likes this.
  4. Tiina Onir

    Tiina Onir Avatar

    Messages:
    1,103
    Likes Received:
    1,900
    Trophy Points:
    125
    Location:
    Bramble, South Paladis
    Having been a part of a full vendor survey last quarter, I can tell you covering all of novia certainly takes more then a few hours man-hours. Because of that, there will be much dancing from me when/if we ever get the ability to get statistical data about player-vendor sales.

    And thanks for the kind words, it's always heartening to know it's being found helpful.
     
    Last edited: Jun 6, 2018
  5. Rowell

    Rowell Avatar

    Messages:
    612
    Likes Received:
    1,281
    Trophy Points:
    105
    Gender:
    Female
    Location:
    Brittany Fields
    @Undone Any word on the possibility of having Player Vendor Sales (time delayed) as a data set tracked and available in the SOTA API ??
     
    Aeryk, Tiina Onir and that_shawn_guy like this.
  6. Undone

    Undone Trap Master Moderator SOTA Developer

    Messages:
    346
    Likes Received:
    848
    Trophy Points:
    43
    No word as of yet sorry. They've got me working on other top secret important stuff =O
     
    Jaesun, Rentier and Lained like this.
  7. Rowell

    Rowell Avatar

    Messages:
    612
    Likes Received:
    1,281
    Trophy Points:
    105
    Gender:
    Female
    Location:
    Brittany Fields
    Hopefully it's cool top secret important stuff. :)
     
  8. Undone

    Undone Trap Master Moderator SOTA Developer

    Messages:
    346
    Likes Received:
    848
    Trophy Points:
    43
    That all depends on your definition of cool =)
     
    Jaesun, Tiina Onir and Rentier like this.
  9. xadoor

    xadoor Avatar

    Messages:
    261
    Likes Received:
    337
    Trophy Points:
    28
    Do we really want to be able to query for all the vender data? isn't part of the game supposed to be the time spent wandering around looking what prices are? I think that is 100% THE GAME for some folks as they try to buy/sell for profit.
     
    Lord Trady of Blix and Lained like this.
  10. Rowell

    Rowell Avatar

    Messages:
    612
    Likes Received:
    1,281
    Trophy Points:
    105
    Gender:
    Female
    Location:
    Brittany Fields

    The reason for the request isn't to post what's on vendors in real time (or even close to real time). The reason for the request is to aggregate prices so that they can be shown when costing recipe costing. For example, if I wanted to know how much it would cost to make a Study Chair, I could aggregate all the prices from all across the world, then show the lowest price, the highest price, and the median average price for the components...and also show the lowest price, highest price and the median average price that a Sturdy Chair sells for....without ever showing who the vendors are with the lowest/highest prices for those items.

    It's mainly used as an aid for crafters.
     
  11. DavidDC

    DavidDC Programmer Moderator SOTA Developer

    Messages:
    1,532
    Likes Received:
    3,236
    Trophy Points:
    113
    Gender:
    Male
    You say in your own post its to aggregate price and know which is lowest or highest. Thats not crafting, its to be more competitive with others in pricing. Those that doesnt have access to the api will be left behind. Those that have access to that api will go in a pricing war and everything will turn bad or will sky rocket down to the bottom to a point that crafter wont make any penny out of their crafting and will just stop doing so.
     
    Veronique of Lowlands likes this.
  12. that_shawn_guy

    that_shawn_guy Bug Hunter

    Messages:
    1,415
    Likes Received:
    3,753
    Trophy Points:
    125
    Location:
    earth... mostly
    There are some of here trying to use the API to build tool precisely to make sure everyone has access to the same data.

    I think it's a bit of stretch to say knowledge of buy/sell prices will cause a long term crash in market value. In an open market, prices rise an fall with perceived value. But, in general, markets tend to be much more stable with full knowledge. Every stock market has detailed prices for current/historic buy/sell prices. That open knowledge does not seem to hinder them in anyway.
     
  13. DavidDC

    DavidDC Programmer Moderator SOTA Developer

    Messages:
    1,532
    Likes Received:
    3,236
    Trophy Points:
    113
    Gender:
    Male
    Cant compare real stock value with game value, totally not the same. In real stock if you want a barrel of petrol u gonna have to go dig it yourself to match the price of the current market, which is impossible. In the game, if you want to craft a chair, getting the wood is doable by everyone in a few click. Thats why real stock dont care to give out their numbers, the system behind to get those ressource is so complex and out of reach for most people that they arent scared of competition. But if you do enter in competition with them, they will do everything possible to screw you. Also the real world is way bigger than the game world. The population is also not the same. A gold mine in alberta and a gold mine in china can clearly have succesful business. The transport of the gold from a country to another is not worth it so they can make their own business in their local area. In game, everything is accessible by everyone and the population is a million time lower.

    I made an access to the api available for everyone and people are mainly using it after every release to know where new recipes are and what monster drop what item. Where is the fun when there is no secret? The filter right now is that people have to spend in game time to walk around and see price, they also have to use their head to remember those price and can always be surprised by what they find in a town or another. With a program, you get all those filters out, people will stick in their town and just get all he want to know in a click without even moving. And it will turn in a "who got the best api tool win the game" market. Even with delay in result, if you get the last month price and its not efficient then why even implement it? Im 15000% sure its a bad idea. The api right now is subtily killing the game. Instead of having people walking around in never visited area in hope to find something good, they just check the api if something special drop there, no, then move to the next part. Having more addition to it wont give better result. Id turn it off and forget it if i was Undone. Too bad i am not ;)
     
    LiquidBlaze likes this.
  14. Rowell

    Rowell Avatar

    Messages:
    612
    Likes Received:
    1,281
    Trophy Points:
    105
    Gender:
    Female
    Location:
    Brittany Fields
    What are you talking about, have's and have not's? A web site posting all this information for ALL to see isn't creating a segmented class of people that have an unfair advantage. And seeing the lowest/highest/median prices on items 1) lets crafters know the general price range for raw materials and components, and 2) lets crafters know the price range to price items when selling on their vendor. Nothing wrong with that.

    What you shouldn't be able to see is "Vendor Bob sells Item-X for 25g, and that's the lowest price. Here's his location." That's crossing the line, IMHO. People will still explore different areas, because they don't know the location of the vendors.
     
  15. that_shawn_guy

    that_shawn_guy Bug Hunter

    Messages:
    1,415
    Likes Received:
    3,753
    Trophy Points:
    125
    Location:
    earth... mostly
    I don't see any issue seeing what Bob has listed on his vendor. Just because Bob has the lowest price and everyone could rush over and buy something cheap doesn't mean Bob can keep it in stock for very long. There are plenty of great deals out there where people just want to get rid of something well below "market value" and happen to be in a part of town that gets little traffic. What is wrong with giving their deal more visibility?

    Obviously, the "don't include my name" flag should still apply to this data. But, why not give the option to the player instead of hiding this data?

    What wrong with know that some vendors have higher buy price on their vendor in Novia Market and a different vendor in Ardoris has a lower sell price? Isn't this how economies work?
     
    Archer and Feeyo like this.
  16. Feeyo

    Feeyo Avatar

    Messages:
    1,716
    Likes Received:
    2,520
    Trophy Points:
    113
    Location:
    Aelasar’s Forest
    As how I understood the request is to publish vendor sold data, not the actual items still for sale with prices.

    Although having the API expose all vendor data to the public would be awesome!
    If we then could use the API to also sent in buy and sell orders, we could create an out of SotA exchange system. I would so love to do this :D
     
    Archer and that_shawn_guy like this.
  17. that_shawn_guy

    that_shawn_guy Bug Hunter

    Messages:
    1,415
    Likes Received:
    3,753
    Trophy Points:
    125
    Location:
    earth... mostly
    This is across the line for me. Transaction should be in game only.
    Of course, this ignores the fact that the "API" we are talking about here is currently just a neutered ElasticSearch with event data in it and is therefor one-way... :)
     
    Feeyo likes this.
  18. Feeyo

    Feeyo Avatar

    Messages:
    1,716
    Likes Received:
    2,520
    Trophy Points:
    113
    Location:
    Aelasar’s Forest
    hehe - yeah I know, I was just getting too excited again.
     
    that_shawn_guy likes this.
  19. xadoor

    xadoor Avatar

    Messages:
    261
    Likes Received:
    337
    Trophy Points:
    28
    the unintended consequences of making vender prices/sales avail thru the api would be huge I bet.
    1) I could list items really cheap/expensive in towns nobody goes into to drive prices
    2) I could purchase my own things with alts
    3) who knows what else
     
  20. that_shawn_guy

    that_shawn_guy Bug Hunter

    Messages:
    1,415
    Likes Received:
    3,753
    Trophy Points:
    125
    Location:
    earth... mostly
    If the information was easily accessible you would probably sell out pretty quick.

    Sure. But, how long could you keep that up?

    You are right. But, how does less information not have the same risk.
     
Thread Status:
Not open for further replies.