Umuri's SotaSTATS - Making the API Accessible to all!

Alright, after way too many delays and me dropping off the map for over a month past when it should have deployed, i'd like to announce the 1.0 version of SotaSTATS is released.

SotaSTATS

All the main features I wanted for 1.0 work, to a decent degree of speed.
Things you will find here:
Raw API Data -> Find under codex. This is the slowest feature of the sight, but can be used to pull any raw data the API has provided.
Overlayed API data on maps -> Find under codex once you've specified a date range (max 48 hours) a location, and at least an event type. There will be a link on the left saying Overlay Codex data. Example
Statistics -> This will give you more refined statistics based on 4 main categories:
Monsters - Data on monsters, what they drop, where they spawn, as people populate them hp/etc.
Items - Where you can get them, what they are, destroy/creation totals over the past few days
Scenes - What appears there (monsters/items)
Misc - Lottery Stats, Coto usage stats

And more to be found as you explore around.
If you find something you want to share, just post the link. SotaSTATS tries to ensure the links contain all needed information, to show others what you see.

In addition, I would like to announce that SotaSTATS uses a demo of a new login tech i've been working on.
SotaSTATS requires NO SIGN UP. But if you want to login, you can login IN-GAME, without ever revealing anything other than your character name. This new login method will allow us to make guild websites/auction houses/events/etc and have a provable method of limiting it to 1 per in-game account. As well as other nifty features. You don't have to trust me with passwords, usernames, etc. It's all handled in-game on portalarium's servers!

Now, once you login, you'll get access to more features such as more codex entries returned on searches, access to the list of what you've killed over the past 30 days.
More importantly, once you login you can start submitting updates to the SotaSTATS internal library.
Right now this is only monsters. To do this, go to the stats tab, pull up the monster, then go to the specific scene. You'll see its stats on the left (hp/focus/exp/etc). You can submit edits to that if you are logged in, that help update data for others.

I will be expanding editing abilities to include names (so we can use better names other than the rediculous internal names the game uses).

Oh yeah and i tried to get it to not look entirely bad which is my trademark, and somewhat play nice on mobile, so if you have mobile complaints, let me hear it!

ONE MORE THING: The loot tables are estimates and guesses, and should not be taken as word of god. They have lots of false positives. I just thought throwing something together would be better than nothing, as Atos has gone out of his way (rightly so) to make matching up loot to kills difficult. They'll get better as I iterately progress the algorithm that matches up.

So yeah, have fun, more updates to come.
I WILL be fixing speed over the next few days and watching for any big performance hangups through real usage.

 

Nice work

 

Great job!

I notice that the translated names are being returned as individual objects/mobs. I can only imagine that would affect overall stats by splitting things up by translated name.

 

It does, however I already have a system in-place to merge them and soft-link different names as translations of a main base object.
However, this requires people flagging what is translations of what. An example of this is the Ferocious Red Spider
Example

I have not made an interface for this, as it's a one-off easy to do job, that I may hire-out to players for in-game gold.

 

I wonder if there is a set of internal resource files that could be parsed into a database to get it all in one-fell-swoop.

 

Monday - 7:31 pm - Well you all hit it enough that the host decided it was "too busy" and killed it without contacting me.
I am very pissed at them and am working to resolve it, hopefully it will be back up within 48 hours.

Update:
Wednesday 11:36 am - I have it tenatively back up, having built an entire new server for the dba, migrated over, then doing lots of upgrades to deal with the new architecture since the web and db are now far apart. It'll be probably not until tonight that it's fully caught up to current data, so expect things to be wonky.

 

Hi, it feels like the Codex search should be defaulting to the most recent searchable time period instead of the earliest one.

And is there a list of search terms somewhere? Like all available EventTypes?

Thanks,

And this is very exciting, still.

 

@Umuri in case you’re still developing sotastats would it be possible to add a report on logged in players?
There’s a lot of discussion on player count and @Chris pointed to the API for this kind of information - but writing the scripts is beyond me.

 

So not just me then? People asking how many people play, people get mad and link to here, yet I can't find current or historical people playing.

 

Can you provide me a quote/link to where he says this? As far as i know, there is nothing in the API that would easily provide this information without making a few inferences/assumptions.

I had such a thing that made those when it first come out, but i will say unless somethings changed since i last looked there is no easy/foolproof way to do this to give really solid numbers
I used the # of people who complete their oracle quest as a "rough" estimate.
Keep in mind this is not 1000%, I would assign a rough error rate of at most 5% to this number, as some of the things this picks up are quest turn ins that aren't the oracle quest. But the running average can be taken to reduce it to what is "most likely" just the oracle quests.

This does not tell you how many people play, but merely how many accounts get their oracle quest each day for 500 gold.
You can make your own arguments on how many accounts per person, how many are bots, how many people play without collecting the oracle each day, etc etc.

However, below is the requested data, query time 2808 seconds.
You can interpret the trends yourselves, however the TLDR is that we went from ~800 a day to ~1000, so we could estimate assuming all else constant, we've gained 20% since may 2017.


2017-04-28 766
2017-04-29 840
2017-04-30 806
2017-05-01 769
2017-05-02 801
2017-05-03 723
2017-05-04 652
2017-05-05 756
2017-05-06 796
2017-05-07 772
2017-05-08 719
2017-05-09 725
2017-05-10 681
2017-05-11 706
2017-05-12 747
2017-05-13 728
2017-05-14 750
2017-05-15 713
2017-05-16 702
2017-05-17 726
2017-05-18 733
2017-05-19 705
2017-05-20 753
2017-05-21 737
2017-05-22 739
2017-05-23 678
2017-05-24 703
2017-05-25 735
2017-05-26 740
2017-05-27 744
2017-05-28 753
2017-05-29 743
2017-05-30 696
2017-05-31 748
2017-06-01 790
2017-06-02 770
2017-06-03 793
2017-06-04 821
2017-06-05 726
2017-06-06 729
2017-06-07 706
2017-06-08 731
2017-06-09 727
2017-06-10 781
2017-06-11 814
2017-06-12 732
2017-06-13 731
2017-06-14 739
2017-06-15 727
2017-06-16 733
2017-06-17 728
2017-06-18 756
2017-06-19 698
2017-06-20 705
2017-06-21 709
2017-06-22 686
2017-06-23 708
2017-06-24 695
2017-06-25 696
2017-06-26 681
2017-06-27 676
2017-06-28 698
2017-06-29 785
2017-06-30 871
2017-07-01 862
2017-07-02 808
2017-07-03 796
2017-07-04 839
2017-07-05 773
2017-07-06 817
2017-07-07 828
2017-07-08 880
2017-07-09 861
2017-07-10 812
2017-07-11 833
2017-07-12 885
2017-07-13 863
2017-07-14 833
2017-07-15 900
2017-07-16 896
2017-07-17 826
2017-07-18 845
2017-07-19 862
2017-07-20 831
2017-07-21 772
2017-07-22 843
2017-07-23 884
2017-07-24 805
2017-07-25 793
2017-07-26 799
2017-07-27 817
2017-07-28 850
2017-07-29 902
2017-07-30 903
2017-07-31 804
2017-08-01 835
2017-08-02 839
2017-08-03 839
2017-08-04 822
2017-08-05 831
2017-08-06 883
2017-08-07 858
2017-08-08 836
2017-08-09 851
2017-08-10 846
2017-08-11 850
2017-08-12 891
2017-08-13 900
2017-08-14 843
2017-08-15 842
2017-08-16 901
2017-08-17 860
2017-08-18 877
2017-08-19 883
2017-08-20 901
2017-08-21 830
2017-08-22 858
2017-08-23 842
2017-08-24 841
2017-08-25 840
2017-08-26 892
2017-08-27 958
2017-08-28 803
2017-08-29 484
2017-09-01 92
2017-09-02 952
2017-09-03 941
2017-09-04 965
2017-09-05 855
2017-09-06 884
2017-09-07 864
2017-09-08 897
2017-09-09 896
2017-09-10 896
2017-09-11 865
2017-09-12 872
2017-09-13 880
2017-09-14 911
2017-09-15 889
2017-09-16 923
2017-09-17 926
2017-09-18 859
2017-09-19 893
2017-09-20 861
2017-09-21 843
2017-09-22 853
2017-09-23 914
2017-09-24 897
2017-09-25 879
2017-09-26 848
2017-09-27 865
2017-09-28 878
2017-09-29 866
2017-09-30 881
2017-10-01 881
2017-10-02 861
2017-10-03 855
2017-10-04 859
2017-10-05 872
2017-10-06 865
2017-10-07 899
2017-10-08 920
2017-10-09 888
2017-10-10 844
2017-10-11 894
2017-10-12 889
2017-10-13 881
2017-10-14 887
2017-10-15 915
2017-10-16 899
2017-10-17 875
2017-10-18 902
2017-10-19 866
2017-10-20 865
2017-10-21 884
2017-10-22 893
2017-10-23 860
2017-10-24 861
2017-10-25 822
2017-10-26 852
2017-10-27 863
2017-10-28 892
2017-10-29 880
2017-10-30 868
2017-10-31 881
2017-11-01 878
2017-11-02 900
2017-11-03 898
2017-11-04 941
2017-11-05 955
2017-11-06 860
2017-11-07 900
2017-11-08 875
2017-11-09 902
2017-11-10 901
2017-11-11 943
2017-11-12 947
2017-11-13 907
2017-11-14 895
2017-11-15 910
2017-11-16 914
2017-11-17 839
2017-11-18 930
2017-11-19 943
2017-11-20 859
2017-11-21 942
2017-11-22 913
2017-11-23 898
2017-11-24 891
2017-11-25 912
2017-11-26 906
2017-11-27 871
2017-11-28 922
2017-11-29 924
2017-11-30 938
2017-12-01 929
2017-12-02 996
2017-12-03 976
2017-12-04 903
2017-12-05 879
2017-12-06 922
2017-12-07 903
2017-12-08 925
2017-12-09 926
2017-12-10 904
2017-12-11 855
2017-12-12 906
2017-12-13 882
2017-12-14 879
2017-12-15 850
2017-12-16 876
2017-12-17 904
2017-12-18 877
2017-12-19 902
2017-12-20 880
2017-12-21 818
2017-12-22 855
2017-12-23 806
2017-12-24 843
2017-12-25 790
2017-12-26 840
2017-12-27 855
2017-12-28 929
2017-12-29 901
2017-12-30 874
2017-12-31 890
2018-01-01 879
2018-01-02 885
2018-01-03 890
2018-01-04 873
2018-01-05 877
2018-01-06 880
2018-01-07 909
2018-01-08 881
2018-01-09 899
2018-01-10 932
2018-01-11 954
2018-01-12 886
2018-01-13 923
2018-01-14 915
2018-01-15 944
2018-01-16 945
2018-01-17 938
2018-01-18 912
2018-01-19 901
2018-01-20 904
2018-01-21 940
2018-01-22 854
2018-01-23 886
2018-01-24 934
2018-01-25 909
2018-01-26 916
2018-01-27 886
2018-01-28 975
2018-01-29 907
2018-01-30 919
2018-01-31 931
2018-02-01 959
2018-02-02 981
2018-02-03 984
2018-02-04 1012
2018-02-05 966
2018-02-06 1029
2018-02-07 987
2018-02-08 802
2018-02-09 149
2018-02-10 967
2018-02-11 1020
2018-02-12 942
2018-02-13 978
2018-02-14 943
2018-02-15 973
2018-02-16 960
2018-02-17 1032
2018-02-18 981
2018-02-19 959
2018-02-20 960
2018-02-21 1016
2018-02-22 1043
2018-02-23 1052
2018-02-24 1041
2018-02-25 975
2018-02-26 929
2018-02-27 986
2018-02-28 1031
2018-03-01 1036
2018-03-02 1093
2018-03-03 1120
2018-03-04 1095
2018-03-05 1083
2018-03-06 1063
2018-03-07 1081
2018-03-08 1020
2018-03-09 1033

 

@Umuri thank you for the reply. @Chris pointed to the API as part of a discussion following this MMO article: http://massivelyop.com/2018/03/06/s...laims-higher-concurrency-heading-into-launch/

 

Not to mention on other sites devs have simply said "Go check out the Public API!" whenever someone questions numbers. But, seems like the API can't do that, huh?

 

Perhaps this is a new functionality, @Chris @Undone can you help us here? It would be great if we had hard facts instead of speculation

 

Okay..
So whats the give or take retention rate of new players past 2 months between persistance and now?
What are the plans to increase this %? How can the community help?
And would you give actual #s? And %s? Or any actual information?

 

I don't do the daily oracle quest, and many don't. So it works as an increased activity tracker but not of course for absolutes.

 

I am online every day but often forget my oracle quest too..and my blessings....and to repair.....I'd leave town without my head if it were not attached.
I wish there was an easier way to get this actual number of players...I don't understand why its something that seems to be intentionally 'hidden' ...kinda feel the same about 'xp to next level' though, and virtue values ....when stuff gets hidden from me it just makes me suspicious.

 

Generic housecleaning in progress, speeding up some things that have hit performance issues when we had to move to new harddrives due to how big it was getting.
But a new page courtesy of @Vladamir Begemot , the economy quick-page:
Economy QuickPage
Quickly and easily see quantity data across various time aggregates for various main resources.

 

Thanks @Umuri, it's looking great and getting greater!

 

Due to the last few releases rendering me unable to play SOTA even in offline mode, I am declaring EOL for sotaSTATS, and making parts of the code available on github.
https://github.com/umuri/SotaSTATS
The poller/database populator at least is complete, so you can use that to start pulling your own copy of the data to play with.

If individuals want the historical dump from SotaSTATS I will deal with those on a case by case basis as its hundreds of gb in size for the dataset to be transferred.
Hope you all got some use out of it, and as always the code sucks.