Public Stats API

Discussion in 'Developer Depot' started by Undone, May 9, 2017.

  1. Kardall

    Kardall Avatar

    Messages:
    22
    Likes Received:
    54
    Trophy Points:
    3
    I do not use NEST no. I use url queries in my C# Application.

    Nothing I tried to get to work with C# worked with NEST, even using a GET and setting the Body to a normal query would error out so... newp.

    Only way I thought about it, is installing the C# Python interpreter, and capturing results from the public api python scripts (or writing your own)
     
    Roycestein Kaelstrom likes this.
  2. Tiina Onir

    Tiina Onir Avatar

    Messages:
    1,094
    Likes Received:
    1,881
    Trophy Points:
    125
    Location:
    Bramble, South Paladis
    Sorry, my experiments have all be in Python, as that's what my website written in.
     
    Roycestein Kaelstrom likes this.
  3. Black Tortoise

    Black Tortoise Avatar

    Messages:
    1,964
    Likes Received:
    3,661
    Trophy Points:
    125
    Gender:
    Male
    Location:
    Storm's Reach
    Its also pretty trivial content to parse with any JSON library, an Elastic Search wrapper isnt really needed. I just make raw HTTP requests to it, and turn the body into JSON, then turn it into whatever other datatype suits my needs.
     
    Roycestein Kaelstrom likes this.
  4. Roycestein Kaelstrom

    Roycestein Kaelstrom Avatar

    Messages:
    4,631
    Likes Received:
    10,256
    Trophy Points:
    153
    Yeah, that's what I ended up doing. @Lained hook me up with the querystring syntax for the HTTP request. Not ideal, but oh well...
     
  5. Kardall

    Kardall Avatar

    Messages:
    22
    Likes Received:
    54
    Trophy Points:
    3
    This does work, however, some queries are easier to do with the ES library.

    Unless JSON was being created while you were learning how it worked, you probably use SOME kind of Library. However, the fact we can't use the POST and include a body of json using NEST is kind of frustrating.

    Right now since you cannot hammer the server or it blocks you, then you really can't do 'real time graphs' and such anyway. When they open it up, yes... yes yes yes, you can bet there will be some wicked things coming out... I promise.
     
    Roycestein Kaelstrom likes this.
  6. Rowell

    Rowell Avatar

    Messages:
    611
    Likes Received:
    1,280
    Trophy Points:
    105
    Gender:
    Female
    Location:
    Brittany Fields
    In the archetypes for items in the SOTA API, there are many items that are hard to pin down as to their actual item type. They end with an ambiguous typing, which makes it hard to discren what they are. Items like "Ingredients/Components_Wood/Handle/ItemComponent_Handle_MapleB". What makes that different from "Ingredients/Components_Wood/Handle/ItemComponent_Handle_MapleA"?
    Same with the following archetype endings:

    - MapleA
    - MapleB
    - PineA
    - PineB
    - CopperA
    - CopperB
    - IronA
    - IronB
    - CottonA
    - CottonB

    So far, I've been able to map about 40% of the archetypes that I've captured from the SOTA API. But there are still a lot that I have no idea how to map, due to the ambiguous nature of the archetype naming convention. Any help that you could give would be greatly appreciated. Thanks!
     
    Roycestein Kaelstrom likes this.
  7. brogan

    brogan Avatar

    Messages:
    71
    Likes Received:
    63
    Trophy Points:
    8
    @Rowell

    MapleA = Hard Maple
    MapleB = Rock Maple
    PineA = Grey Pine
    PineB = Crimson Pine
    CopperA = Bronze
    CopperB = Constantan
    IronA = Meteoric Iron
    IronB = White Iron
    CottonA = Carapacian
    CottonB = Fustian
    LeatherA = Hardened Leather
    LeatherB = Supple Leather
     
  8. that_shawn_guy

    that_shawn_guy Bug Hunter

    Messages:
    1,263
    Likes Received:
    3,368
    Trophy Points:
    125
    Location:
    earth... mostly
    Any interest, as a community, to start assembling this type of data in https://github.com/SotACommunity/Database or something else under https://github.com/SotACommunity?

    I feel like many of us are duplicating work on our own that could, if we shared it, make life easier for everyone try to build something.

    I had intended to flesh out what is there a lot more. But, IRL won again...
     
  9. Rowell

    Rowell Avatar

    Messages:
    611
    Likes Received:
    1,280
    Trophy Points:
    105
    Gender:
    Female
    Location:
    Brittany Fields
    To Date, www.sotatraders.com has 1474 items in its database, and 1128 recipes. About 40% of the items (587) have been mapped to the SOTA API Archetypes. The site allows users to add and update items and recipes, so there's the little need (potentially) for administrators to manually hand-jam updates all the time (just verify them and click a button to update the site). It's all stored in a MS-SQL database.
     
  10. Umuri

    Umuri Avatar

    Messages:
    528
    Likes Received:
    1,830
    Trophy Points:
    63
    Sorry for the delay on this one, but we cannot track this number because the api does not expose experience gained events with any actually useful data such as amount. Due to the data blanking, the API only exposes that experience was gained as an event, and the location/timestamp, no amount data.
    It's only useful as a second ordinal timing seperator.
     
  11. xadoor

    xadoor Avatar

    Messages:
    261
    Likes Received:
    337
    Trophy Points:
    28
    Roycestein Kaelstrom likes this.
  12. Rowell

    Rowell Avatar

    Messages:
    611
    Likes Received:
    1,280
    Trophy Points:
    105
    Gender:
    Female
    Location:
    Brittany Fields
    Belated thanks, @brogan ... I completely missed your post from a few weeks ago! This is extremely helpful, and should allow me to connect at least another hundred items from my DB to the API.
     
    Roycestein Kaelstrom likes this.
  13. xadoor

    xadoor Avatar

    Messages:
    261
    Likes Received:
    337
    Trophy Points:
    28
    I purchased 2 tickets and then verified that the stats api still had only single ticket purchases so unless someone has an explanation I declare it broken.
     
    Roycestein Kaelstrom likes this.
  14. Rowell

    Rowell Avatar

    Messages:
    611
    Likes Received:
    1,280
    Trophy Points:
    105
    Gender:
    Female
    Location:
    Brittany Fields
    SotaTraders uses about 14 tables for items and recipes. That's everything from the items and recipes themselves, to the modifying information (like town and NPC's they're found in/on, and the ingredients used in each recipe, etc). I don't know an easy way to post all of this data in a generic way that anyone joining the "consortium" could use it. They would have to use the same general relationships between each table. The tables could easily be exported to a spreadsheet. Then others can just copy/paste that data into the database of their choice (or use an import script to do so).
     
  15. that_shawn_guy

    that_shawn_guy Bug Hunter

    Messages:
    1,263
    Likes Received:
    3,368
    Trophy Points:
    125
    Location:
    earth... mostly
    That is reason I started putting data in as JSON files in directory hierarchy. It's a common format and should provide an easy way for developers either use directly or build an import utility from it for their own apps.
     
  16. Tiina Onir

    Tiina Onir Avatar

    Messages:
    1,094
    Likes Received:
    1,881
    Trophy Points:
    125
    Location:
    Bramble, South Paladis
    It should be scriptable. I already use plpgsql functions to address the multitude of tables on sotadb.info as JSON. As long as the format is well defined and we guarantee uniqueness on certain fields, it should be okay. That said, this line of thought is probably outside the scope of this thread, and deserves it's own thread.
     
    Roycestein Kaelstrom likes this.
  17. Nellie

    Nellie Avatar

    Messages:
    5
    Likes Received:
    6
    Trophy Points:
    3
    Location:
    Hong Kong
    Hi,

    I managed to fire up ElasticHQ and run some queries against the Elasticsearch. It's pretty neat.

    e.g. Ubuntu 16.04

    # docker pull elastichq/elasticsearch-hq
    # docker run --rm --name=sota --network=host elastichq/elasticsearch-hq

    Connect via browser to: http://<Your Linux IP>:5000
    Enter 'http://www.shroudoftheavatar.com:9200' as address to connect to.
    Example: https://imgur.com/a/p2R1bRb

    # docker stop sota


    -Nellie-
     
  18. Roycestein Kaelstrom

    Roycestein Kaelstrom Avatar

    Messages:
    4,631
    Likes Received:
    10,256
    Trophy Points:
    153
    This is a useful tool as an alternative from having to build up a web app to render the data from elastic. Thanks for sharing.
     
  19. Undone

    Undone Trap Master Moderator SOTA Developer

    Messages:
    346
    Likes Received:
    848
    Trophy Points:
    43
    This is fantastic thanks for sharing!
     
  20. Warrior B'Patrick

    Warrior B'Patrick Avatar

    Messages:
    1,050
    Likes Received:
    2,084
    Trophy Points:
    113
    Gender:
    Male
    Location:
    Athens, Georgia, USA
    Greetings all, I am a asking for some beginning help to start using the API. I have just started to learn elasticsearch and would like to know if there is a better way then going thru the training that "Learning Elasticsearch 5.0" and "Elasticsearch 5 cookbook" provide? Also the queries are easy enough but how do you get a listing of all the datapoints? For instance... I have a new vendor in a POT and would like to run a query for player vendors items and pricing. Time frame does not matter it is only for historical reference. Is there a datapoint for each POT city? I know there is one for player vendors because of SOTAtraders site.

    If this is not the right place to post this please let me know and thank you all for any help you can offer.
     
    Tiina Onir and that_shawn_guy like this.