Tipp of the day: Download trouble

Sometimes you try to download a file with CURL or a HTTPSocket and you receive an error 403 (forbidden). Often this happens because the webserver does a few checks to make sure you are a real visitor and not a mass download tool.
Typically a webserver will check the referrer in the http request. Often they check for referrers and redirect people without referrer to a start page to avoid deep linking. Second webservers like to check for the user agent and if your user agent shows things like CURL, they may block you. Some servers set cookies to verify you visited a page with ads before you download something. So you may need to carry a cookie for the downloads.
So best you capture the http request from the browser so you see values they pass for referrer, user agent and cookie string. Than you can use those values and build your download like it.

Update: Another thing to check is the accept header in the http request. The biggest plugin in space...