Geeks With Blogs

News Awarded Microsoft MVP C#.NET - 2007, 2008 and 2009

I am born in Bangladesh and currently live in Melbourne, Australia. I am a Microsoft Certified Application Developer MCAD Chartered Member (C# .Net)and born in Bangladesh.
I am founder and Chief Executive Officer of
Simplexhub, a highly experienced software development company based in Melbourne Australia and Dhaka, Bangladesh. Co-founder and core developer of Pageflakes
Simplexhub, is on its mission to build a smart virtual community in Bangladesh and recently launched beta an ASP.NET MVC application written in C#.NET.

Some of My Articles
Flexible and Plugin based .Net Application..
Mass Emailing Functionality with C#, .NET 2.0, and Microsoft® SQL Server 2005 Service Broker'
Write your own Code Generator or Template Engine in .NET
Shahed Khan blog

WebRequest is the abstract base class for the .NET Framework's request/response model for accessing data from the Internet.

To get content of a website, in .NET 1.0. we used to use WebRequest, which is good and also works asynchronously.

public static string GetContent(string url)
System.Net.WebRequest request = System.Net.WebRequest.Create(url);
using (System.Net.WebResponse response = request.GetResponse())
  using (System.IO.StreamReader reader =new System.IO.StreamReader(response.GetResponseStream()))
   return reader.ReadToEnd();

But in .NET 2.0, we can also use the WebClient class. It can also work asynchronous and works the same as the other one.

public static string GetContent(string url)
using (System.Net.WebClient client =new System.Net.WebClient())
  return client.DownloadString(url);

We can use any of the above method for web scrapeing in .NET. But the second approach is probably more cleaner.

Posted on Monday, June 9, 2008 3:28 PM | Back to top

Comments on this post: System.Net.WebClient().DownloadString(url) for Web Scrapeing

# re: System.Net.WebClient().DownloadString(url) for Web Scrapeing
Requesting Gravatar...
you could even reduce that to:
new System.Net.WebClient().DownloadString(url);

WebClient does not implement iDisposable so it is not necessary to wrap in a using statement. One line is even "more cleaner" than 7, no?

Finally, I believe scraping has no e. (not scrapeing).

thanks for the post!
Left by Rusty on Jul 28, 2008 4:04 AM

Your comment:
 (will show your gravatar)

Copyright © Shahed Khan | Powered by: