Geeks With Blogs

News Awarded Microsoft MVP C#.NET - 2007, 2008 and 2009


I am born in Bangladesh and currently live in Melbourne, Australia. I am a Microsoft Certified Application Developer MCAD Chartered Member (C# .Net)and born in Bangladesh.
I am founder and Chief Executive Officer of
Simplexhub, a highly experienced software development company based in Melbourne Australia and Dhaka, Bangladesh. Co-founder and core developer of Pageflakes www.pageflakes.com.
Simplexhub, is on its mission to build a smart virtual community in Bangladesh and recently launched beta realestatebazaar.com.bd an ASP.NET MVC application written in C#.NET.


Some of My Articles
Flexible and Plugin based .Net Application..
Mass Emailing Functionality with C#, .NET 2.0, and Microsoft® SQL Server 2005 Service Broker'
Write your own Code Generator or Template Engine in .NET
Shahed Khan blog

WebRequest is the abstract base class for the .NET Framework's request/response model for accessing data from the Internet.

To get content of a website, in .NET 1.0. we used to use WebRequest, which is good and also works asynchronously.

public static string GetContent(string url)
{
System.Net.WebRequest request = System.Net.WebRequest.Create(url);
using (System.Net.WebResponse response = request.GetResponse())
{
  using (System.IO.StreamReader reader =new System.IO.StreamReader(response.GetResponseStream()))
  {
   return reader.ReadToEnd();
  }
}
}

But in .NET 2.0, we can also use the WebClient class. It can also work asynchronous and works the same as the other one.

public static string GetContent(string url)
{
using (System.Net.WebClient client =new System.Net.WebClient())
{
  return client.DownloadString(url);
}
}

We can use any of the above method for web scrapeing in .NET. But the second approach is probably more cleaner.

Posted on Monday, June 9, 2008 3:28 PM | Back to top


Comments on this post: System.Net.WebClient().DownloadString(url) for Web Scrapeing

# re: System.Net.WebClient().DownloadString(url) for Web Scrapeing
Requesting Gravatar...
you could even reduce that to:
new System.Net.WebClient().DownloadString(url);

WebClient does not implement iDisposable so it is not necessary to wrap in a using statement. One line is even "more cleaner" than 7, no?

Finally, I believe scraping has no e. (not scrapeing).

thanks for the post!
Left by Rusty on Jul 28, 2008 4:04 AM

Your comment:
 (will show your gravatar)


Copyright © Shahed Khan | Powered by: GeeksWithBlogs.net