{"id":1106,"date":"2013-04-27T11:55:02","date_gmt":"2013-04-27T11:55:02","guid":{"rendered":"http:\/\/www.nargalzius.com\/blog\/?p=1106"},"modified":"2017-02-10T01:51:31","modified_gmt":"2017-02-10T01:51:31","slug":"universal-proxy","status":"publish","type":"post","link":"http:\/\/nargalzius.com\/blog\/archives\/2013\/04\/27\/universal-proxy","title":{"rendered":"Universal Proxy"},"content":{"rendered":"

If you would notice this site’s front page<\/a>, I’ve now replaced a bunch of parts of the side-bar (right side of the homepage) with some social feeds of networks I’m active on. This was an unintended effect of creating a caching proxy<\/em> for work.<\/p>\n

A proxy is a script that pulls data from another domain and outputs the data as is. It may seem silly; to require something to simply mirror the data of another site, when you can simply pull the data from the source itself. Logically, this is true, but in the intertubes,<\/em> nothing is really simple. <\/p>\n

You have to wrestle with domain implementation policies, wherein certain stuff simply doesn’t work because the application, browser, server won’t allow it as a matter of security.<\/p>\n

Flash as an example, can be annoyingly strict about pulling data from a different domain. If you build and test your app locally there’s no problem, but once you upload that and run it, the same exact algorithm may or may not work depending on the policies of both the plugin and sometimes even the server itself.<\/p>\n

A proxy placed in your localhost solves this by ensuring that the data your “application” is accessing comes from the same domain.<\/p>\n

Now a caching<\/em> proxy is something more; just like with the concept of caching in general<\/a> it allows a feed to be pulled from the source via the proxy, but then it’s written to a file locally, and have an expiration on it. As long as it’s not “expired”, then the proxy will simply re-use the file it had generated instead of pulling the feed from the source for a set period of time.<\/p>\n

This is extremely useful (and in my case, the reason why I made it) when dealing with rate-limited stuff. All social networks have their own rate limitations; if a user\/application is polling their APIs too often, they will block access temporarily to prevent their servers from being overloaded with requests (similar to, if not tantamount to a DoS attack<\/a>)<\/p>\n

Practical Example<\/h1>\n

So as an example, let’s say this site is a high traffic site, where hundreds of visitors visit it per minute.<\/strong> Let’s also assume I have the proper infrastructure to handle the traffic.<\/p>\n

Now my social feed on the side polls the various social networks, and I’ve implemented it via JQuery – so it’s kinda realtime when you load it. Meaning whenever a person visits the site, it will load the page, then run the JQuery script that then pulls the required feeds from the different social networks. As soon as that’s complete, it then loads them into their respective slots.<\/p>\n

Now imagine that happening hundreds of times per minute.<\/strong> Twitter as an example is one of the stricter services to work with; and has a rate limit<\/a> of about 15 API calls every 15 mintues 1 <\/sup><\/a>Or roughly one call per minute. Call more often than that, it will<\/em> block you<\/strong> [temporarily].<\/span><\/span>