problem with crawl many url in .net: Server IP not ping. maybe bandwidth or http connection limit ex

Posted by Hamid on Stack Overflow See other posts from Stack Overflow or by Hamid
Published on 2010-05-13T08:50:36Z Indexed on 2010/05/13 8:54 UTC
Read the original article Hit count: 319

Filed under:

c#

|

bandwidth

|

bandwidth-throttling

|

httpwebrequest

|

windows-services

Hi to all

I develop web crawling service (windows service / multi-thread) . its work fine, but sometimes my server network not response. and i can't ping server IP (from internet), but can ping by other network card (local ip) that not access to internet.

after i open server with remote desktop and stop crawling service. i could ping.

What's my problem? Bandwidth limit or max connection limit exceed or ???
how to prevent this issue?

Note: when this problem occur, i open browser for browse web site, but can't open any website!!!

Could you please help me.

Thanks in advanced

© Stack Overflow or respective owner

Related posts about c#

.NET WebRequest.PreAuthenticate not quite what it sounds like

as seen on West-Wind - Search for 'West-Wind'
I’ve run into the problem a few times now: How to pre-authenticate .NET WebRequest calls doing an HTTP call to the server – essentially send authentication credentials on the very first request instead of waiting for a server challenge first? At first glance this sound like it should be easy:… >>> More
HttpWebRequest and Ignoring SSL Certificate Errors

as seen on West-Wind - Search for 'West-Wind'
Man I can't believe this. I'm still mucking around with OFX servers and it drives me absolutely crazy how some these servers are just so unbelievably misconfigured. I've recently hit three different 3 major brokerages which fail HTTP validation with bad or corrupt certificates at least according to… >>> More
The dynamic Type in C# Simplifies COM Member Access from Visual FoxPro

as seen on West-Wind - Search for 'West-Wind'
I’ve written quite a bit about Visual FoxPro interoperating with .NET in the past both for ASP.NET interacting with Visual FoxPro COM objects as well as Visual FoxPro calling into .NET code via COM Interop. COM Interop with Visual FoxPro has a number of problems but one of them at least got a lot… >>> More
Dynamic Type to do away with Reflection

as seen on West-Wind - Search for 'West-Wind'
The dynamic type in C# 4.0 is a welcome addition to the language. One thing I’ve been doing a lot with it is to remove explicit Reflection code that’s often necessary when you ‘dynamically’ need to walk and object hierarchy. In the past I’ve had a number of ReflectionUtils that used string based expressions… >>> More
Finding a Relative Path in .NET

as seen on West-Wind - Search for 'West-Wind'
Here’s a nice and simple path utility that I’ve needed in a number of applications: I need to find a relative path based on a base path. So if I’m working in a folder called c:\temp\templates\ and I want to find a relative path for c:\temp\templates\subdir\test.txt I want to receive back subdir\test… >>> More

Related posts about bandwidth

Implement QoS/Bandwidth Management or Upgrade Bandwidth?

as seen on Server Fault - Search for 'Server Fault'
A question that I'm faced with currently. Here's my setup: Cisco ASA 5510 15Mbps Internet Connection @ $1350/month The bandwidth was originally meant for 35-45 people but we've grown quite quickly to roughly 60-65 people. Needless to say, when I check bandwidth logs it's almost always spiked at… >>> More
Wireless AAA for a small, bandwidth-limited hotel.

as seen on Server Fault - Search for 'Server Fault'
We (the tech I work with and myself) live in a remote northern town where Internet access is somewhat of a luxury, and bandwidth is quite limited. Here, overage charges ranging from few hundreds, to few thousands of dollars a month, is not uncommon. I myself incur regular monthly charges just through… >>> More
How to scale bandwidth of a startup website?

as seen on Server Fault - Search for 'Server Fault'
I would like to host a website using my home internet connection, with multiple computers acting as nodes of web server, db, apache cassandra clusters, and memcached clusters. When this website gets to the point where I outgrow my slow home internet connection, what is the easiest/best way to scale… >>> More
Bandwidth.com Launches Free Hosted PBX for SMBs

as seen on Internet.com - Search for 'Internet.com'
Phonebooth Free provides customers with 200 inbound calling minutes a month at no charge. >>> More
Ask How-To Geek: How Can I Monitor My Bandwidth Usage?

as seen on How to geek - Search for 'How to geek'
If you’re lucky you enjoy wide open internet access with out restriction (or restrictions so high you would have to work all month to meet them). If you’re not so lucky, you’ve got an ISP with heavy caps. Today we help out a reader working under such a cap. Latest Features How-To… >>> More