Jump to content
Sign in to follow this  
NastyGamer

API Improvements

Recommended Posts

Some changes I think the API might benefit from:

  1. Upper cap on API usage should be replaced with Rate Limits. Assign a higher rate rimit for resource-intensive API endpoints... so that people don't spam that endpoint. The current system is actually as bad as having no limits. The quotas are already high enough after buying a cheap VIP in-game. What if I concurrently try to access nations API from 1000 different threads/processes. It's much easier to do that than it looks like especially since there is a free tier in AWS and Google Cloud >_> This should actually result in much needed reduced API usage for the server imo.
  2. After the extended deletion rule changes Alex made, the API has started to bloat with unnecessary objects in Nations, All-Cities, and Nation-Military API.
    Some stats to prove that:
    Total nations in the API: 13813
    Total nations active in last 45 days: 6732
    Total nations that should have been deleted previously (above 45 days inactivity): 7081
    Total nations inactive above 90 days (in automated VM): 4668

    The extra 7081 nations have slowed down Nations API endpoint by almost half. It used to take me almost 25s to fetch the Nations API earlier this year... it's now around 50s. (can be different according to internet speeds... point to be noted is that it has slowed down). At least those 4668 nation objects do not need to be present in the API. Same applies for the All-Cities and Nation-Military API.
    Either change how the game handles automated VMs (preferred) or just exclude them from the API.

  3. Fetching the status of Powered key of a nation city atm means, getting all the city ids, either through All-Cities or Nation API and then parsing through each city separately. Now, this looks ok for smaller sheets and such, but when the sheet/bot/code is dealing with 100+ nations, it means on an average calling the API 8 times for each of those 100+ nations. (if 8 is the average cities). The number shoots into thousands easily. Checking City power is a very important factor for categorizing "profitable" and active nations imo. It makes sense to add a powered key either into All-Cities API (preferred) or the Nation/Nations API (like the number of powered cities I guess). This will again reduce load since I'll be not calling your City endpoint thousands of times.

  4. War-Attacks API is buggy (numbers are not accurate) and has pretty much fricked up design. I cannot comment on it cause I never used it because of its weird design. I ask more experienced developers to please tell Alex on how to improve this one. It's not useless atm, but it's buggy and more poorly designed relative to Alex's other API endpoints.

  5. @Alexplease read this: http://php.net/manual/en/language.types.intro.php. I am amazed at your API's data types.

  6. Idk how your backend behaves but I think large API results should be cached by your server (if it's not being done already). I am quite ready to lose a few minutes of updated data in return for a faster API response time. I don't need to know if a trade was done 1 minute ago as long as I am getting the data within 10-15 minutes.
    Reason for this change: "Real Time" large API responses are usually more structured (paginated) than your API. I don't expect you to do that so instead lose the real time for a few minutes delay. Fetch data in the backend every 15min or so and just dump that JSON whenever someone accesses that endpoint (this is specifically for Nations, All-Cities, Nation-Military, Trade-History.... the larger ones). This is more of personal opinion without any numbers to back whether it will be faster or not and I am sure people will disagree with it and I'll be glad to read why not.

I'll keep editing, adding suggestions here. Meanwhile feel free to add more suggestions here or correct the ones suggested.

  • Like 2

Share this post


Link to post
Share on other sites

My three:
 

  1. Add a "30davgprice" value or "7davgprice" value to the tradeprice API. It would just be the average price of that resource over the last 7 and 30 days.
  2. Double commas in All-Cities API! Currently I can see 15 of these... We shouldn't have to fix your invalid JSON in our code.
  3. Instead of a hard-coded 5s or so delay in the tradeprice API, have it return data from 5s ago instead. Or even better, remove the delay altogether, it's honestly more annoying than useful for preventing against scripted trades (P&W market doesn't change quickly enough for 5s delay to be effective IMO)

...and please fix the data types, you shouldn't be storing numbers as strings in your DB anyway.

Edited by JDTech
  • Like 1

Share this post


Link to post
Share on other sites

There is only 1 proper response to this.

"F"

But really, tbh I haven't been touching the nations api that much, I just assumed I was experiencing this delay since I slapped a synchronized onto it to threadsafe it until I felt like touching it again

Share this post


Link to post
Share on other sites

if we could get the trade prices api condensed into one instead of requiring a hit for each resource, that'd be pretty alright too 👌

Share this post


Link to post
Share on other sites
19 hours ago, NastyGamer said:

Some changes I think the API might benefit from:

  1. Upper cap on API usage should be replaced with Rate Limits. Assign a higher rate rimit for resource-intensive API endpoints... so that people don't spam that endpoint. The current system is actually as bad as having no limits. The quotas are already high enough after buying a cheap VIP in-game. What if I concurrently try to access nations API from 1000 different threads/processes. It's much easier to do that than it looks like especially since there is a free tier in AWS and Google Cloud >_> This should actually result in much needed reduced API usage for the server imo.
  2. After the extended deletion rule changes Alex made, the API has started to bloat with unnecessary objects in Nations, All-Cities, and Nation-Military API.
    Some stats to prove that:
    Total nations in the API: 13813
    Total nations active in last 45 days: 6732
    Total nations that should have been deleted previously (above 45 days inactivity): 7081
    Total nations inactive above 90 days (in automated VM): 4668

    The extra 7081 nations have slowed down Nations API endpoint by almost half. It used to take me almost 25s to fetch the Nations API earlier this year... it's now around 50s. (can be different according to internet speeds... point to be noted is that it has slowed down). At least those 4668 nation objects do not need to be present in the API. Same applies for the All-Cities and Nation-Military API.
    Either change how the game handles automated VMs (preferred) or just exclude them from the API.

  3. Fetching the status of Powered key of a nation city atm means, getting all the city ids, either through All-Cities or Nation API and then parsing through each city separately. Now, this looks ok for smaller sheets and such, but when the sheet/bot/code is dealing with 100+ nations, it means on an average calling the API 8 times for each of those 100+ nations. (if 8 is the average cities). The number shoots into thousands easily. Checking City power is a very important factor for categorizing "profitable" and active nations imo. It makes sense to add a powered key either into All-Cities API (preferred) or the Nation/Nations API (like the number of powered cities I guess). This will again reduce load since I'll be not calling your City endpoint thousands of times.

  4. War-Attacks API is buggy (numbers are not accurate) and has pretty much fricked up design. I cannot comment on it cause I never used it because of its weird design. I ask more experienced developers to please tell Alex on how to improve this one. It's not useless atm, but it's buggy and more poorly designed relative to Alex's other API endpoints.

  5. @Alexplease read this: http://php.net/manual/en/language.types.intro.php. I am amazed at your API's data types.

  6. Idk how your backend behaves but I think large API results should be cached by your server (if it's not being done already). I am quite ready to lose a few minutes of updated data in return for a faster API response time. I don't need to know if a trade was done 1 minute ago as long as I am getting the data within 10-15 minutes.
    Reason for this change: "Real Time" large API responses are usually more structured (paginated) than your API. I don't expect you to do that so instead lose the real time for a few minutes delay. Fetch data in the backend every 15min or so and just dump that JSON whenever someone accesses that endpoint (this is specifically for Nations, All-Cities, Nation-Military, Trade-History.... the larger ones). This is more of personal opinion without any numbers to back whether it will be faster or not and I am sure people will disagree with it and I'll be glad to read why not.

I'll keep editing, adding suggestions here. Meanwhile feel free to add more suggestions here or correct the ones suggested.

I am not sure about #6, caching wont really help out if its not properly implemented but I think, that caching would go against our need for a real time API. He should add pagination to it or more better filtering. But, filtering would require it to adopt ODATA standards of even graphql support.

On 2: just no. We need the ability to pull in "automated vms" for tracking purposes.

I will reply more to this thread after work.

Share this post


Link to post
Share on other sites
5 hours ago, ThommyZero said:

I honestly am not even sure what the API does.

 

Can someone help me please?

it's like a record/website, but on the internet and for computers, it returns information about the game in a readable format (humans/computers) based on the url & parameters you use.

http://politicsandwar.wikia.com/wiki/Category:API

Share this post


Link to post
Share on other sites
20 hours ago, Jroc said:

I am not sure about #6, caching wont really help out if its not properly implemented but I think, that caching would go against our need for a real time API. He should add pagination to it or more better filtering. But, filtering would require it to adopt ODATA standards of even graphql support.

On 2: just no. We need the ability to pull in "automated vms" for tracking purposes.

I will reply more to this thread after work.

Well, I do agree that some people need the ability to pull in automated VMs, but it has indeed slowed down the API with a lot of useless nations. Maybe he could add a URL GET parameter to filter out only active nations (not in automated VMs).

Also with #6, I was trying to propose to let go of the "real time" path and take up the cache path. Pagination is highly unlikely with Alex's current API development history. And it might result in much faster response times (again I am just speculating. maybe I am overestimating the benefit cause SQL queries are already pretty damn fast. Someone experienced with working with APIs or Alex could comment on the response times). But the fact remains, that the large APIs have considerably slowed down (for some it has gone from 10s to 20s. For me 25s to 50s cause prob server location and latency and internet speed... idk). I'd really like Alex to fix it.

Also, I've heard Alex say somewhere that API already uses a lot of his server resources. Instead of blaming us to constantly hitting his API for changes, maybe he could do some work to force reduce resource usage of the API (webhooks, pagination, filters, cache, CDN, rate-limits etc.)

Share this post


Link to post
Share on other sites
On 1/11/2019 at 11:07 PM, Dynamic said:

If major changes are to be made, I would like to see proper versioning in the url structure. It would be super helpful for those using an old API and smooth a transition over.

I really like this article: https://www.baeldung.com/rest-versioning

Since now we're making so high demands from Alex, I'd say include Pagination for those big APIs too lol. 
.

.

Anyways, @Dynamic don't have such high hopes. lol. I'd be happier if Alex just logs every change he makes to the API (which he doesn't atm. he leaves it to the community to discover the changes).

Share this post


Link to post
Share on other sites
On 1/13/2019 at 6:32 AM, NastyGamer said:

Since now we're making so high demands from Alex, I'd say include Pagination for those big APIs too lol. 
.

.

Anyways, @Dynamic don't have such high hopes. lol. I'd be happier if Alex just logs every change he makes to the API (which he doesn't atm. he leaves it to the community to discover the changes).

@NastyGamer you wouldn't think separating the PHP API files into two folders would be that difficult...

  • Upvote 1

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

  • Recently Browsing   0 members

    No registered users viewing this page.

×

Important Information

By using this site, you agree to our Terms of Use and the Guidelines of the game and community.