The response stream has other stuff that you don't necessarily want. You really just want the raw data of the imgage file, so you can use:
new MemoryStream(new WebClient().DownloadData("http://address/file.ico"));
Probably the website is protected against hot linking. Since images are large in size in compare to html pages and websites pays for their used bandwidth, this is a mechanism for protecting websites from bsndwidth theft, by other websites.
The idea is using some kind of authentication and make sure a real user is asking for the image. This is usually done by cookies on the page which contains the image. You need to get those cookies and send them with your request. You need to add a
CookieContainer to your request, then request the page the image resides and at the end retrieve the image (use the same request for all steps, cookies would be added and used automatically).