General DiscussionHTTP Get request 429

Why do you even need a database of your stats. Just come here to look at them. They store them just fine.

Sorry for the late reply. I am only just chasing this issue up now, because it has happened to another site I want to use now.
so it looks like its something wrong with my hosting provider.

essentially, I want to get a copy of my dotabuff, and store the details I want on a MYSQL server.
So when people load up my personal webpage, they can see my total number of games is 240 (just an example)
it needs to be stored and updated in my database so that my webpage load times to not suffer.
If I am pulling data from DotaBUFF, CSGOStats, Fitbit website etc. it would take 20 seconds + before my page even loads.
Storing them in my database makes the request pretty much instantanious.

Anyway, I am looking into this issue with my web hostings support forum

You can extract the contents to a .csv using http get which can then be imported to a SQL table.

429 means you were rate limited. Keep the number of requests per hour reasonable and it won't happen.

@Havoc Badger
I know how to use these functions and I was using it successfully before I had to raise this ticket
I left the project for 2 weeks, and when I came back I was getting errors, I thought that it could have been Google scraper bots that caused me to request the pages too many times, however this is not the case anymore, because I implemented a specific URL to request the api (have to include "?ddos=1" in the URL) which has been included for the last 16 days, and my API still gets no results.

@Jason
Thanks for the reply, this is the sort of response I was initially looking for
however I have made a total of 3 requests in the last 16 days via my API and I am no longer getting 429 requests, I just get nothing at all.
Finally, as testing, I saved my whole dotabuff page, and stored it in my dropbox as a website.
Whenever my API fails the dotabuff API, it goes and pulls my dropbox stored profile perfectly
which means it is one of 2 issues

- DotaBUFF has put my ISP on a blacklist for sending get_file_contents() requests
- My webhosting has blacklisted external calls to Dotabuff.com

I have a ticket opened with my webhost as well.
I will hopefully update this issue when I find out more

PSA, I plan to be making maximum 1 request every 6 hours if this issue gets resolved.

For anyone curious, my API is here on pastebin to expire in 1 month
http://pastebin.com/qvk3tu38

And here is an example of it being uploaded (the actual site)
http://www.p337.x10host.com/api/dotabuff.php?ddos=1&dotabuffid=36369500

Your script is requesting http://www.dotabuff.com/players/76561197996635228 - note the community id instead of player id (which would be 36369500). If your client isn't setup to follow HTTP 302 redirects it won't give you the HTML you're looking for.

Try pointing it at http://www.dotabuff.com/players/36369500

Thanks, but the code isnt the issue, both my Steam community ID and the dotabuff ID were working before this change happened.

the link to my dotabuff API (http://www.p337.x10host.com/api/dotabuff.php?ddos=1&dotabuffid=36369500) has a debug line currently included in it to echo the URL used in get_file_contents() before it is auctioned.

Again, I really raised this ticket becuase I wanted to know if there were any overzealous rules set up by you guys, but it sounds like you dont.

my hosting provider has some new anti-ddos rules that are quite strict, so I think it might be those guys.

EDIT: I think my best option is to create a new free webhosting solution and see if my code works there or not.
I will get back with the results of that shortly

bump because i want to make a data crawler too.

or you guys could just make it so you can set time limits for your stats/trends. (instead of Last 1 Month it can be Since Jan to Feb)