1. This forum section contains Blog style threads from the Developers, and/or threads with details about what they have been working on, previews of upcoming content, and sometimes backstory associated with the content. Players are permitted to reply to these threads if the developer has indicated it is permissible by leaving the thread unlocked. Please note that TOS applies, and any abuse or trolling will be moderated.
    Dismiss Notice

Public Stats API

Discussion in 'Developer Work and Blog posts' started by Undone, May 9, 2017.

Thread Status:
Not open for further replies.
  1. that_shawn_guy

    that_shawn_guy Bug Hunter

    Messages:
    1,409
    Likes Received:
    3,738
    Trophy Points:
    125
    Location:
    earth... mostly
    Code:
    python download_scroll.py -o playerkilled.json -tf 30 -st "PlayerKilledByPlayer OR PlayerKilledBySelf"
    Traceback (most recent call last):
      File "download_scroll.py", line 39, in <module>
        data_dump()
      File "download_scroll.py", line 29, in data_dump
        for result in scan_result:
      File "/usr/local/lib/python2.7/site-packages/elasticsearch/helpers/__init__.py", line 324, in scan
        client.clear_scroll(body={'scroll_id': [scroll_id]}, ignore=(404, ))
      File "/usr/local/lib/python2.7/site-packages/elasticsearch/client/utils.py", line 73, in _wrapped
        return func(*args, params=params, **kwargs)
      File "/usr/local/lib/python2.7/site-packages/elasticsearch/client/__init__.py", line 1047, in clear_scroll
        params=params, body=body)
      File "/usr/local/lib/python2.7/site-packages/elasticsearch/transport.py", line 318, in perform_request
        status, headers, data = connection.perform_request(method, url, params, body, ignore=ignore, timeout=timeout)
      File "/usr/local/lib/python2.7/site-packages/elasticsearch/connection/http_urllib3.py", line 128, in perform_request
        self._raise_error(response.status, raw_data)
      File "/usr/local/lib/python2.7/site-packages/elasticsearch/connection/base.py", line 124, in _raise_error
        raise HTTP_EXCEPTIONS.get(status_code, TransportError)(status_code, error_message, additional_info)
    elasticsearch.exceptions.AuthorizationException: TransportError(403, u'<html><body><h1>403 Forbidden</h1>\nRequest forbidden by administrative rules.\n</body></html>\n')
     
  2. Balec Fares deCani

    Balec Fares deCani Avatar

    Messages:
    1,343
    Likes Received:
    4,015
    Trophy Points:
    125
    Gender:
    Male
    Hello World! (got a response using curl)
     
    Black Tortoise likes this.
  3. Undone

    Undone Trap Master Moderator SOTA Developer

    Messages:
    346
    Likes Received:
    848
    Trophy Points:
    43
    Ah yes it looks like scroll is currently disabled. I would use download_quick.py for now; I'll look into this tomorrow.

    I've disabled the functionality in the script for now (simple print and exit messages) in the repo.
     
    Black Tortoise likes this.
  4. Andel Rho

    Andel Rho Avatar

    Messages:
    17
    Likes Received:
    11
    Trophy Points:
    3
    When do we get more attributes?
     
  5. Balec Fares deCani

    Balec Fares deCani Avatar

    Messages:
    1,343
    Likes Received:
    4,015
    Trophy Points:
    125
    Gender:
    Male
    Quick and dirty tutorial:

    It's pretty easy to get started using this.
    download and install python: https://www.python.org/downloads/
    open cmd prompt, cd to scripts directory in your new python location
    run: pip install elasticsearch
    download and install SotA sample scripts from OP: https://github.com/rthompsonj/SotAPublicStatsQuery
    cd to where you put the SotA sample scripts
    run the sample script using one of the provided example commands: https://github.com/rthompsonj/SotAPublicStatsQuery/blob/master/README.md
    Look in the folder where you put the SotA scripts for the newly created output file.

    some possible alterations to the above:
    remove the first word "python" when running the sample script in last step
    change the long datatype to int in the SotA file: query_builder.py
     
    Last edited: May 9, 2017
    Vladamir Begemot, Bom, smack and 2 others like this.
  6. Odin BloodBayne

    Odin BloodBayne Avatar

    Messages:
    114
    Likes Received:
    370
    Trophy Points:
    30
    Gender:
    Male
    Location:
    Isle Of Valhalla
  7. Earl Atogrim von Draken

    Earl Atogrim von Draken Avatar

    Messages:
    6,331
    Likes Received:
    12,109
    Trophy Points:
    165
    Gender:
    Male
    Can we get an API key so not everybody and his (insert random animal) can read out what we did as soon as we flag for the API?
     
  8. oplek

    oplek Avatar

    Messages:
    1,238
    Likes Received:
    3,017
    Trophy Points:
    113
    Gender:
    Male
    The endpoint (both http and https) are timing out for me. Is it just me?
     
    devilcult likes this.
  9. DavidDC

    DavidDC Programmer Moderator SOTA Developer

    Messages:
    1,532
    Likes Received:
    3,236
    Trophy Points:
    113
    Gender:
    Male
    Did it crash or i just cant access it..


    Code:
    Elasticsearch TRACE: 2017-05-10T12:30:58Z
      -> HEAD http://shroudoftheavatar.com:9200/
    
      <- 403
     
  10. Odin BloodBayne

    Odin BloodBayne Avatar

    Messages:
    114
    Likes Received:
    370
    Trophy Points:
    30
    Gender:
    Male
    Location:
    Isle Of Valhalla
    Can merchant name be added to the "ItemGained_Merchant" response?

    Code:
    {"ItemId": "59129d6dabb70d05c00088d9", "SceneName": "Novia_R6_City_StormsReach", "xpos": "80.02", "zpos": "148.812", "timestamp": "May 9 23:56:12", "PricePerUnit": "6", "ypos": "38.188", "Price": "1200", "EconomyGoldDelta": "-600", "LocationEvent": "ItemGained_Merchant", "Archetype": "Ingredients/Reagents/ItemReagent_SulfurousAsh", "@timestamp": "2017-05-10T04:56:12.000Z", "Quantity": "200"}
     
    Browncoat Jayson and iansmellios like this.
  11. Elnoth

    Elnoth Avatar

    Messages:
    724
    Likes Received:
    1,557
    Trophy Points:
    93
    Interesting - I get 23 hits on Elnoth, even when my name is set to Anonymous. All of these were when my elemental pet got a kill and recorded an entry in "LocationEvent": "MonsterKilledByMonster"

    [​IMG]
     
    Last edited: May 10, 2017
  12. HogwinHD

    HogwinHD Avatar

    Messages:
    610
    Likes Received:
    1,730
    Trophy Points:
    105
    Gender:
    Male
    Location:
    Somewhere.....
    Can someone please make a video tutorial, step by step for those of us who dont speak Code etc? would be a great help :)
     
    Knightwolf and Caree Death like this.
  13. Frederick Glasgow

    Frederick Glasgow Avatar

    Messages:
    729
    Likes Received:
    2,052
    Trophy Points:
    93
    I am not going to opt in to having my name seen,what goes on in my keep basement secret lab is between me and the occasional spider.;)o_O:eek:
     
  14. oplek

    oplek Avatar

    Messages:
    1,238
    Likes Received:
    3,017
    Trophy Points:
    113
    Gender:
    Male
    To answer my own question...

    Looks like my web host is blocking the port. When I run locally, I can connect. It's experimental, but hopefully the plan is to run it on a standard port at some time in the future.
     
  15. null2

    null2 Avatar

    Messages:
    373
    Likes Received:
    1,152
    Trophy Points:
    55
  16. pinzasso

    pinzasso Avatar

    Messages:
    97
    Likes Received:
    126
    Trophy Points:
    8
    Last edited: May 10, 2017
    Aeryk, Bom, Black Tortoise and 3 others like this.
  17. oplek

    oplek Avatar

    Messages:
    1,238
    Likes Received:
    3,017
    Trophy Points:
    113
    Gender:
    Male
  18. pinzasso

    pinzasso Avatar

    Messages:
    97
    Likes Received:
    126
    Trophy Points:
    8
    That is the default ElasticSearch port (9200).
     
  19. oplek

    oplek Avatar

    Messages:
    1,238
    Likes Received:
    3,017
    Trophy Points:
    113
    Gender:
    Male
    Yes, I (now) know. That doesn't make it a standard port.... I'm not sure where the disconnect is here. It doesn't become a standard port because one company decided to use it. For HTTP requests, 80 and 443 are the standard ports... maybe arguably 8080 for a distant third for non-public/production servers.

    So, if I want to create a website that ties into this, I have a number of choices. I can either,
    1. Take the time/money to move my websites to another host, who hopefully also isn't blocking the non-standard 9200 too
    2. Not have a website
    3. Run the website off a local machine, which includes a bunch of problems to solve.
    Their use of a non-standard port is a limiting factor for me. Maybe I'm the only one, and not worth the effort to change, but at least I can put in the request to normalize the API's configuration a bit more.
     
  20. oplek

    oplek Avatar

    Messages:
    1,238
    Likes Received:
    3,017
    Trophy Points:
    113
    Gender:
    Male
    But I think that's the last I'll say about that. I do look forward to seeing what I can do with it.
     
Thread Status:
Not open for further replies.