Community-wide XML online data source for Items, Crafting, etc.?

There are a number of data bases with items and crafting recipes. Is all their item data coming from a single source, or are they collecting it all on their own?

Is there any value in coming up with a single community-wide XML data source for all information for the items, recipes, NPCs, etc.? Is this already being done? Has this been discussed before?

Can the various online tools people have and are creating ingest XML data sources if made available online?

https://www.w3schools.com/xml/default.asp

I see several efforts / sites making online tools to work with this type of information. The creativity in these sites is fantastic! Except... they all pretty much need the same source data and appear to me to independently gather and verify that information.

Maybe I am wrong, and they all get their information from SotAWiki ? Or, is there an API data connection to the SotA server for us to query/receive this information? I'm not tech smart on that.

If not, the source information is in the game and anyone walking around (even non-programmers) can help gather and verify information (items, recipes, NPC locations, even NPC prices). Perhaps we just need a simple place to "put" the data and then it become available as a data feed (XML formatted I imagine) to all the tool sites and developers out there.

Certainly to do this, superior beings (beyond basic fishermen like me) with detailed item knowledge would need to come up with the XML structure to support the types of data being collected and used.

For example from https://www.sitepoint.com/really-good-introduction-xml/





Thoughts?

 

It would probably be a boon to lots of people. Crafted items would need their components. They could have recipe also, but those change and are a spoiler, so maybe not.

For it to be useful on 3rd party sites they would need permission, and it would have to be fast or they would have to download it after every release.

 

i started a json version of the same basic idea. https://github.com/shawnbutts/sotadata. i got started with recipes and crafting stuff. haven't had much time to work on it though.

 

This is really good. May I suggest putting tools and fuel into their own category within each recipe?

The reason is tools only get used at 1/100th per use, and fuels get used even on failure. So it would be much easier for complex apps to know that instead of having to categorize at runtime.

 

i think collecting the data is a bigger challenge than anything on the technical side.

 

I agree with this too. We would need a group of people, working together, able to sense what information needs collecting, has probably changed, how often to verify, collect and verify that information, etc. We'd need a user-based login system to add information to the central data base source.

A process might be needed ...

1. Many people SUBMIT data base updates - info is staged to show what has "changed", by whom, and when
2. Other people VERIFY the data - info is examined for accuracy and accepted into the master data base
3. Some people can eventually be Submitters and Verifiers if they have a track record of accuracy and diligence

So yes, Gathering Data is probably the hardest part and requires many people. Yet even new people just joining the game who find errors in the various online tools using the source data can help us maintain data.

Thus the data source can really become "community-based" and anyone can help out.

Perhaps we'd end up with a good cadre of people willing to spend time on data gathering, motivate others to help, etc. We kind of did this with Hospitallers (many people helping people in game), ... maybe people could become an "Accumulator" (a person who accumulates: to gather or collect) -- and people can turn on a SotA title "Accumulator" when they are out wandering around in the game checking on information. -- Maybe the title could use the Latin word "Legulus" which means gatherer. - A collector, gatherer, picker. ref: https://en.wiktionary.org/wiki/legulus

Lots of ideas with the end goal to gather and store accurate information for everyone to use.

 

github does all of those. this was part of the reason why i started this on github. since it would mostly be developers consuming this data, github is likely something they are already using.

 

I will happily contribute. Json is a better choice than xml, glad you went with that. What about the changes I suggested?

And I won't be able to start until late January. Haven't even been able to play in 2 months.

 

Anything worth doing, is worth doing right. I've created https://github.com/SotACommunity with an empty "Database" repo.

 

https://www.w3schools.com/js/js_json_xml.asp

I'm learning as we go. I've worked with JSON with JavaScript client programming. Looks like it is a good way to go according to W3Schools.

I am not a developer, so I have not used github, but I know basically what it does. https://github.com/SotACommunity is a space to do what? Put the actual data online in a JSON format? Or is it where developers build the downloadable code to help us host the JSON data somewhere else on the Internet which can be Linked (like an HTML .js linked source) and ingested into tool databases and such? I guess I'm asking sort of "what are the steps" to achieve this perceived goal to have an online data source, from a program management level?

 

We can start assembling the raw data in https://github.com/SotACommunity/Database. From there we can create other projects under https://github.com/SotACommunity/ to work with the collected data or pretty much whatever we want.

Github has a lot of features that make it easy to see who added what, manage feature requests, and generally make running a coding project easier. For example, i started an discussion in https://github.com/SotACommunity/Database/issues/1 so we can gather input for how we want to layout the data. This same system can be used to track bugs, feature requests, etc.

Once we have useful data, anyone can pull it into their own projects.

 

http://www.sotatraders.com uses a SQL database with 3 main tables (Items, Recipes and Recipe Ingredients), along with a few helper tables to track everything from vendor location, to region, to what properties an item has.
The site also has a section for submitting new and updating incorrect items/recipes.
With that SQL data, it can be output into just about any format, or queried on in all manner of ways.