What is a Proxy Server?
A proxy server is a computer system or program that operates as a go between for a user, application, or computer and another user, application, or computer. So it acts on behalf of another. This can be very helpful when running a web spider.
What are the uses of a proxy such as when running a web spider?
- The most likely reason for using a proxy on the Internet is to conceal the identity of the requesting user, computer or program. It the early days of Internet computing this was primarily for security purposes. However, as ROBOT or BOT programs as well as Black Hat SEO techniques have become more prevalent it has become more for the reason of concealing identity. The concealing of identity becomes required when a source requesting user, computer or program has been identified as not wanted and banded from the website or server. The caller then wants to conceal the identity so that it can gain information like other users do.
- Another use of proxy computers is to speed up access to information. For example, if you are on an aircraft it is likely that the proxy server on-board has already “cached” a copy of the front page of MSN, Yahoo and Google. This way each requester does not need to access the primary source which is located on the ground. Vast amounts of information can be stored on a proxy or caching server that is close to the requester.
- In addition, proxy servers can be used to audit the use of outbound or inbound resources. So in other words, if you are web surfing from the office, the IT Department’s proxy server may be recording your every move. You might just get a visit from someone in personnel about your time management and social habits. Likewise, a hotel can monitor use of its Internet Resources and apply a charge to those who want to use them or a least make them agree to legal terms and conditions before using them.
- A proxy can also be used to access resources for content that governments or businesses are blocking their citizens or employees from seeing.