Public Stats API

Ultimately, I think you'll have to have an enforced limitation.

I don't know if it's possible with Elasticsearch, but it might be nice if we could limit our request to just the fields we want. It's a shame to trafnsfer 20 lines of stats, only to discard 16.

 

I don't. My point was what I may be interested in may be different from what @Umari is interested in.

Each of them is probably useful to someone, but (at least when I played around with it) I never used the entire output. If we could request just the fields we were interested in, I know I would (maybe it's already possible, and I just don't know elasticsearch well enough? this is my first time really working with it, and I've only gotten the time to do light dabbling so far). A similar example, when I query my local DB, I never do a "SELECT *" I always list the fields I'm interested in.

 

Part of me thinks you should not allow the script kiddies access to this kind of data and vet people with legitimate coding skills to be used on legitimate sites for data use (say kinda like SOTAMAP).

 

I realize you're saying only part of you, but how do you vet them? Ultimately, any vetting process amplifies @Umuri's complaint that this is a resource that only a select few (limited by programming background) can use, and gain an advantage with. Then it would be, not just limited by our experience, but also the vetting process.

(complaint may be a strong word... maybe "observation")

 

To limit what fields you want, include the parameter _source= in your query.
Example query below that only returns the archetype, for example:
http://shroudoftheavatar.com:9200/_all/_search?q=LocationEvent:ItemGained_Loot&_source=Archetype

Ugh definitely not. those are some of the more useful fields.
I know i've been slow getting the fruits out to everyone, i'm out of town 3+ days a week right now until mid june so my time is a little limited, but some really cool stuff is coming next week (hopefully tuesday or wednesday at latest) when i can push the entirety of SotaSTATS live. The top25s (currently defunct due to switchover) were just a small sample of some of the nifty stats that this system allows us.


Re: Bandwidth hogs, i know i was one of the people, sheerly because of the breadth of all the different data i'm pulling, although I did just rework the entire system to be much better about how it pulls (it was getting tons of duplicates when time started getting fudged). But I will definitely kill functionality rather than have them kill the API, been keeping in touch with undone to see how its been going. I know the heatmaps alone are the bulk of the data requests and i'll definitely kill those as features if it means keeping the rest of the api accessible.

 

Nope. I definitely do need ItemId.

 

I think a lot of people would find ItemId useful. It helps distinguish between zero value items and items that have value (ie. stolen items from scenery vs. looted items that both have the same name).

I haven't messed around with this API stuff yet to know what can be done with it... but I do know that distinguishing between two same-named items is important to be able to do for any crafting site or wiki site, etc. that may have intentions of displaying some info based off of this data.

By the same token, the zpos, xpos, and ypos can all show location data that these websites can use. I would think those would be very important to keep.

 

Will itemid will become (more) important when things like item affinity come online? It could possibly be the link between a specific item type and what additional abilities it has. Does it have any additional properties for crafted items?

 

I think the key is that they are having to do this because they dont have the manpower to keep a dev busy creating a new web front end for people to be able to see this data in a more organized way that everyone can see. This just fired up and as undone said, 30 gigs a day are uploading with only the few people doing it now, what happens if 50000 people are playing this game in a year and they have 100 times more people doing this. This just seems like it should be limited to community orientated people who provide a community service like Sotamap and Umuri (who I bet integrates some into his widely used HUD). With the big $$$ behind this game, I think this system is going to be used by a lot of individuals for their own use only to gain financial advantage in the game for themselves, Im not sure they had this kind of thing planned. However, if their servers and bandwidth can handle it without effecting players in the game... go for it

I'm considering doing so for myself. I still have a Dell R720 with 128 gigs of ram in my home lab with esxi installed, I can throw up a SQL server VM again on server 2012 R2 (or several) and start pounding that front end to mine date. I have a 250 meg connection so I can probably hit it pretty hard. Im not sure if they want me doing that though, even though I have a degree in CS, I have been mostly in high end infrastructure over the past 25 years. I'm apt to make a lot of mistakes... aka... I am the script kiddy.

Data is power

 

People WILL use it to their advantage, that's why it has to be open to everyone. What's to stop someone from starting a open service, as a front to collect other data? If I thought of that, I guarantee others will. On the flip side, how do you start a service to the community, without access to the information you need to run it?

The best counter to people getting an unfair advantage is democratising access. As more people have access, it's more likely that someone like @Umuri will make it available. "Two people can keep a secret... if one of them is dead"

The bandwidth/load consideration is a technical matter, rate and request-size limits should solve that. @Umuri's comment about how to select individual fields is great. @Undone: It might be helpful to have a best-practices guide to give people an idea of the rates and so how to limit fields doesn't get lost in this thread.

 

nice! you are awesome! That looks awesome, you should continue expanding the site, you are already in a good position with that site to have a great community site.

I love the pwned by Avara choice lol.. as you and I were talking about his the other night, me running my arss away from him across the entire zone as it was dropping 50 hp dots on me somehow (I LIVED!!!)

 

@devilcult I have few creative ideas of how to take advantage of ItemId in my own app(s).