Public Stats API

Discussion in 'Developer Depot' started by Undone, May 9, 2017.

  1. Lord Andernut

    Lord Andernut Bug Brigade - Bug Hunter

    Messages:
    3,333
    Likes Received:
    10,074
    Trophy Points:
    165
    Gender:
    Male
    Location:
    New Britannian Market
    So I ran the pvp_stats.pv which resulted in this:

    C:\Python\Python36-32\Scripts\SotAPublicStatsQuery-master\examples>pvp_stats.py
    Traceback (most recent call last):
    File "C:\Python\Python36-32\Scripts\SotAPublicStatsQuery-master\examples\pvp_stats.py", line 3, in <module>
    import pandas as pd
    ModuleNotFoundError: No module named 'pandas'


    I want to sort of dig around just to see something like how many people are going to my POT (New Britannian Market) and how many things are being sold/bought etc. Just seems interesting to me :)
     
    Lained likes this.
  2. Odin BloodBayne

    Odin BloodBayne Avatar

    Messages:
    114
    Likes Received:
    369
    Trophy Points:
    30
    Gender:
    Male
    Location:
    Isle Of Valhalla
    Sounds like you need pandas installed and probably bokeh

    pip install pandas bokeh
     
  3. Balec Fares deCani

    Balec Fares deCani Bug Brigade - Bug Hunter

    Messages:
    1,352
    Likes Received:
    4,040
    Trophy Points:
    125
    Gender:
    Male
    Those require a couple additional modules (Note that this script requires the pandas and bokeh python packages.). Maybe start with commands under the Examples heading.
     
  4. Undone

    Undone Trap Master Moderator SOTA Developer

    Messages:
    345
    Likes Received:
    841
    Trophy Points:
    43
    Things are currently limited to location-based events, we are looking into expanding this.

    I hope examples like this are helpful!

    This has been taken care of. I wasn't aware that pet kills were flagged as MonsterKilledByMonster events; they have been removed and should now be filtered out.
     
    Black Tortoise likes this.
  5. Undone

    Undone Trap Master Moderator SOTA Developer

    Messages:
    345
    Likes Received:
    841
    Trophy Points:
    43
    I mention this in the README posted on the repository that you need pandas and bokeh to produce that visualization. It's just a simple example of how one might pull down the data and produce some interactive web visualizations.
     
    Lord Andernut likes this.
  6. Lord Andernut

    Lord Andernut Bug Brigade - Bug Hunter

    Messages:
    3,333
    Likes Received:
    10,074
    Trophy Points:
    165
    Gender:
    Male
    Location:
    New Britannian Market
    Tried Example script:
    C:\Python\Python36-32\Scripts\SotAPublicStatsQuery-master>download_quick.py -o riseLoot.json -tf 0.5 -st "SceneName:The Rise"
    Traceback (most recent call last):
    File "C:\Python\Python36-32\Scripts\SotAPublicStatsQuery-master\download_quick.py", line 31, in <module>
    data_dump()
    File "C:\Python\Python36-32\Scripts\SotAPublicStatsQuery-master\download_quick.py", line 16, in data_dump
    request = construct_request(es, search_term, time_frame)
    File "C:\Python\Python36-32\Scripts\SotAPublicStatsQuery-master\utils\query_builder.py", line 30, in construct_request
    "gte": long(gte_time),
    NameError: name 'long' is not defined

    C:\Python\Python36-32\Scripts\SotAPublicStatsQuery-master>

    I assume this is where this quote comes in.... I'm not really sure how to change the long datatype to int. I tried editing query_build.pv and I didn't see a "long datatype" - I'm just using a default editor to look (IDLE)
     
  7. Undone

    Undone Trap Master Moderator SOTA Developer

    Messages:
    345
    Likes Received:
    841
    Trophy Points:
    43
    Are you using a 32bit version of python? I'm curious as to why long() is not defined....hrm

    To change it to int you simply change from long(<stuff>) to int(<stuff>).
     
  8. Lord Andernut

    Lord Andernut Bug Brigade - Bug Hunter

    Messages:
    3,333
    Likes Received:
    10,074
    Trophy Points:
    165
    Gender:
    Male
    Location:
    New Britannian Market
    I downloaded version 3.6.1 - I don't recall an option to use 32 or 64 bit when installing.

    Edit - The directory is this:

    Python36-32

    So that may be version 3.6 and 32-bit
     
  9. Lord Andernut

    Lord Andernut Bug Brigade - Bug Hunter

    Messages:
    3,333
    Likes Received:
    10,074
    Trophy Points:
    165
    Gender:
    Male
    Location:
    New Britannian Market
    Am I editing download_quick? I don't see "long" anywhere in the file when I edit it.

    This is what I see:

    [​IMG]

    I'm script illiterate. I knew dos in and out and like to think it will help, but I'm illiterate with this stuff and just bumbling through :p
     
  10. Undone

    Undone Trap Master Moderator SOTA Developer

    Messages:
    345
    Likes Received:
    841
    Trophy Points:
    43
    Yea that's a 32 bit version of python; are you running a 64bit version of windows? Personally I recommend people use the anaconda python distribution as it comes packaged with most everything one could ever need in one neat package (although it likely comes with a bunch of stuff you may not need but it wont hurt).
     
  11. Undone

    Undone Trap Master Moderator SOTA Developer

    Messages:
    345
    Likes Received:
    841
    Trophy Points:
    43
    The long you are looking for in the utils/query_builder.py file.
     
  12. Undone

    Undone Trap Master Moderator SOTA Developer

    Messages:
    345
    Likes Received:
    841
    Trophy Points:
    43
    I just pushed an update to do a try/except so hopefully it will do the int() cast automatically for you. Pull the latest and let me know if that helps!
     
  13. Lord Andernut

    Lord Andernut Bug Brigade - Bug Hunter

    Messages:
    3,333
    Likes Received:
    10,074
    Trophy Points:
    165
    Gender:
    Male
    Location:
    New Britannian Market
    64-bit windows 10 pro. I'll go with your suggested install later at home, thanks (using work computer atm to peek at this stuff).

    Thanks, this generated a json file with no errors when I change long to int.

    I'll take a look again in 3-4 hours and see if I can figure out how to grab statistics on my POT :)

    Thanks, will try later - have a performance in a little bit to prepare for :)
     
  14. Poor game design

    Poor game design Avatar

    Messages:
    18,208
    Likes Received:
    35,487
    Trophy Points:
    153
    Gender:
    Male
    I'd like anything that occures in my POT to be excluded from this data or to be anonymous. How do I do that?

    @Undone
     
    Black Tortoise likes this.
  15. Poor game design

    Poor game design Avatar

    Messages:
    18,208
    Likes Received:
    35,487
    Trophy Points:
    153
    Gender:
    Male
    I'd like anything that occures in my POT to be excluded from this data or to be anonymous. How do I do that?

    @Undone
     
  16. Ben_Hroth

    Ben_Hroth Avatar

    Messages:
    261
    Likes Received:
    739
    Trophy Points:
    28
    @Undone - Just wanted to confirm that "price" and any commerce is only the NPC Vendor stuff right now... I have been experimenting and cannot see purchases made from PC Vendors or trades.
     
  17. Ristra

    Ristra Avatar

    Messages:
    3,951
    Likes Received:
    5,458
    Trophy Points:
    153
    Location:
    Athens
    Would making your POT anonymous do the trick?

    Instead of zone info: anonymous POT could be toggled
     
  18. Umuri

    Umuri Avatar

    Messages:
    529
    Likes Received:
    1,832
    Trophy Points:
    63
    The disconnect is that what you are making is -NOT- an HTTP request. It is a RESTful API request that spouts out JSON.
    So yes, it would actually be breaking standard to use port 80 for such a use, because it is specifically not a website. That you can access it via such syntax is just a happenstance of similarity. 8080 is one of the most common ports for http-like interfaces or secondary access over the http protocol.

    To put another way, you do not want people's browsers accessing this as a matter of practice, they won't make sense of it and it is neither user friendly nor user intuitive outside of those who can code.

    If your host has that blocked, please contact your host, as odds are there's settings to be enabled on your side that allows such tunnels to be put out, and they just must be configured. I know one of the hosts i've used in the past merely requires a whitelist for the connections you wish to make outbound while the rest have all allowed it or had a quick control panel option that was disabled by default in instances where i didn't have complete server access.
     
  19. Ben_Hroth

    Ben_Hroth Avatar

    Messages:
    261
    Likes Received:
    739
    Trophy Points:
    28
    Without a nefarious PvP-like mind, Baron, it doesn't seem like the information being produced is very intrusive as it stands.

    It shows position info of anonymous avatars (unless they have opted-in), any NPC vendor transactions in anonymous fashion (even if they've opted in), and anonymous kill/loot statistics (even if they've opted in).
     
  20. OxNull

    OxNull Avatar

    Messages:
    118
    Likes Received:
    291
    Trophy Points:
    18
    Location:
    Brookside
    is it possible to get a "KilledByAbility" attribute added?

    Thanks for your work towards this @Undone
     
    Titania Xylia and that_shawn_guy like this.