Public Stats API

Discussion in 'Developer Depot' started by Undone, May 9, 2017.

  1. that_shawn_guy

    that_shawn_guy Avatar

    Messages:
    973
    Likes Received:
    2,566
    Trophy Points:
    105
    Location:
    earth... mostly
    Code:
    python download_scroll.py -o playerkilled.json -tf 30 -st "PlayerKilledByPlayer OR PlayerKilledBySelf"
    Traceback (most recent call last):
      File "download_scroll.py", line 39, in <module>
        data_dump()
      File "download_scroll.py", line 29, in data_dump
        for result in scan_result:
      File "/usr/local/lib/python2.7/site-packages/elasticsearch/helpers/__init__.py", line 324, in scan
        client.clear_scroll(body={'scroll_id': [scroll_id]}, ignore=(404, ))
      File "/usr/local/lib/python2.7/site-packages/elasticsearch/client/utils.py", line 73, in _wrapped
        return func(*args, params=params, **kwargs)
      File "/usr/local/lib/python2.7/site-packages/elasticsearch/client/__init__.py", line 1047, in clear_scroll
        params=params, body=body)
      File "/usr/local/lib/python2.7/site-packages/elasticsearch/transport.py", line 318, in perform_request
        status, headers, data = connection.perform_request(method, url, params, body, ignore=ignore, timeout=timeout)
      File "/usr/local/lib/python2.7/site-packages/elasticsearch/connection/http_urllib3.py", line 128, in perform_request
        self._raise_error(response.status, raw_data)
      File "/usr/local/lib/python2.7/site-packages/elasticsearch/connection/base.py", line 124, in _raise_error
        raise HTTP_EXCEPTIONS.get(status_code, TransportError)(status_code, error_message, additional_info)
    elasticsearch.exceptions.AuthorizationException: TransportError(403, u'<html><body><h1>403 Forbidden</h1>\nRequest forbidden by administrative rules.\n</body></html>\n')
     
  2. Balec Fares deCani

    Balec Fares deCani Bug Brigade - Bug Hunter

    Messages:
    1,352
    Likes Received:
    4,040
    Trophy Points:
    125
    Gender:
    Male
    Hello World! (got a response using curl)
     
    Black Tortoise likes this.
  3. Undone

    Undone Trap Master Moderator SOTA Developer

    Messages:
    345
    Likes Received:
    841
    Trophy Points:
    43
    Ah yes it looks like scroll is currently disabled. I would use download_quick.py for now; I'll look into this tomorrow.

    I've disabled the functionality in the script for now (simple print and exit messages) in the repo.
     
    Black Tortoise likes this.
  4. Andel Rho

    Andel Rho Avatar

    Messages:
    17
    Likes Received:
    11
    Trophy Points:
    3
    When do we get more attributes?
     
  5. Balec Fares deCani

    Balec Fares deCani Bug Brigade - Bug Hunter

    Messages:
    1,352
    Likes Received:
    4,040
    Trophy Points:
    125
    Gender:
    Male
    Quick and dirty tutorial:

    It's pretty easy to get started using this.
    download and install python: https://www.python.org/downloads/
    open cmd prompt, cd to scripts directory in your new python location
    run: pip install elasticsearch
    download and install SotA sample scripts from OP: https://github.com/rthompsonj/SotAPublicStatsQuery
    cd to where you put the SotA sample scripts
    run the sample script using one of the provided example commands: https://github.com/rthompsonj/SotAPublicStatsQuery/blob/master/README.md
    Look in the folder where you put the SotA scripts for the newly created output file.

    some possible alterations to the above:
    remove the first word "python" when running the sample script in last step
    change the long datatype to int in the SotA file: query_builder.py
     
    Last edited: May 9, 2017
    Vladamir Begemot, Bom, smack and 2 others like this.
  6. Odin BloodBayne

    Odin BloodBayne Avatar

    Messages:
    114
    Likes Received:
    369
    Trophy Points:
    30
    Gender:
    Male
    Location:
    Isle Of Valhalla
  7. Earl Atogrim von Draken

    Earl Atogrim von Draken Avatar

    Messages:
    6,364
    Likes Received:
    12,164
    Trophy Points:
    165
    Gender:
    Male
    Can we get an API key so not everybody and his (insert random animal) can read out what we did as soon as we flag for the API?
     
  8. oplek

    oplek Avatar

    Messages:
    1,009
    Likes Received:
    2,546
    Trophy Points:
    113
    Gender:
    Male
    The endpoint (both http and https) are timing out for me. Is it just me?
     
    devilcult likes this.
  9. devilcult

    devilcult Avatar

    Messages:
    1,076
    Likes Received:
    2,263
    Trophy Points:
    113
    Did it crash or i just cant access it..


    Code:
    Elasticsearch TRACE: 2017-05-10T12:30:58Z
      -> HEAD http://shroudoftheavatar.com:9200/
    
      <- 403
     
  10. Odin BloodBayne

    Odin BloodBayne Avatar

    Messages:
    114
    Likes Received:
    369
    Trophy Points:
    30
    Gender:
    Male
    Location:
    Isle Of Valhalla
    Can merchant name be added to the "ItemGained_Merchant" response?

    Code:
    {"ItemId": "59129d6dabb70d05c00088d9", "SceneName": "Novia_R6_City_StormsReach", "xpos": "80.02", "zpos": "148.812", "timestamp": "May 9 23:56:12", "PricePerUnit": "6", "ypos": "38.188", "Price": "1200", "EconomyGoldDelta": "-600", "LocationEvent": "ItemGained_Merchant", "Archetype": "Ingredients/Reagents/ItemReagent_SulfurousAsh", "@timestamp": "2017-05-10T04:56:12.000Z", "Quantity": "200"}
     
    Browncoat Jayson and iansmellios like this.
  11. Elnoth

    Elnoth Avatar

    Messages:
    703
    Likes Received:
    1,519
    Trophy Points:
    93
    Interesting - I get 23 hits on Elnoth, even when my name is set to Anonymous. All of these were when my elemental pet got a kill and recorded an entry in "LocationEvent": "MonsterKilledByMonster"

    [​IMG]
     
    Last edited: May 10, 2017
  12. HogwinHD

    HogwinHD Avatar

    Messages:
    587
    Likes Received:
    1,721
    Trophy Points:
    75
    Gender:
    Male
    Location:
    Somewhere.....
    Can someone please make a video tutorial, step by step for those of us who dont speak Code etc? would be a great help :)
     
    Knightwolf and Caree Death like this.
  13. Frederick Glasgow

    Frederick Glasgow Avatar

    Messages:
    735
    Likes Received:
    2,067
    Trophy Points:
    93
    I am not going to opt in to having my name seen,what goes on in my keep basement secret lab is between me and the occasional spider.;)o_O:eek:
     
  14. oplek

    oplek Avatar

    Messages:
    1,009
    Likes Received:
    2,546
    Trophy Points:
    113
    Gender:
    Male
    To answer my own question...

    Looks like my web host is blocking the port. When I run locally, I can connect. It's experimental, but hopefully the plan is to run it on a standard port at some time in the future.
     
  15. null2

    null2 Avatar

    Messages:
    374
    Likes Received:
    1,153
    Trophy Points:
    55
    iansmellios and Alley Oop like this.
  16. pinzasso

    pinzasso Avatar

    Messages:
    97
    Likes Received:
    126
    Trophy Points:
    8
    Last edited: May 10, 2017
    Aeryk, Bom, Black Tortoise and 3 others like this.
  17. oplek

    oplek Avatar

    Messages:
    1,009
    Likes Received:
    2,546
    Trophy Points:
    113
    Gender:
    Male
  18. pinzasso

    pinzasso Avatar

    Messages:
    97
    Likes Received:
    126
    Trophy Points:
    8
    That is the default ElasticSearch port (9200).
     
  19. oplek

    oplek Avatar

    Messages:
    1,009
    Likes Received:
    2,546
    Trophy Points:
    113
    Gender:
    Male
    Yes, I (now) know. That doesn't make it a standard port.... I'm not sure where the disconnect is here. It doesn't become a standard port because one company decided to use it. For HTTP requests, 80 and 443 are the standard ports... maybe arguably 8080 for a distant third for non-public/production servers.

    So, if I want to create a website that ties into this, I have a number of choices. I can either,
    1. Take the time/money to move my websites to another host, who hopefully also isn't blocking the non-standard 9200 too
    2. Not have a website
    3. Run the website off a local machine, which includes a bunch of problems to solve.
    Their use of a non-standard port is a limiting factor for me. Maybe I'm the only one, and not worth the effort to change, but at least I can put in the request to normalize the API's configuration a bit more.
     
  20. oplek

    oplek Avatar

    Messages:
    1,009
    Likes Received:
    2,546
    Trophy Points:
    113
    Gender:
    Male
    But I think that's the last I'll say about that. I do look forward to seeing what I can do with it.