Search Results

Search found 59112 results on 2365 pages for 'web internet'.

Page 34/2365 | < Previous Page | 30 31 32 33 34 35 36 37 38 39 40 41  | Next Page >

  • Detecting 'stealth' web-crawlers

    - by Jacco
    What options are there to detect web-crawlers that do not want to be detected? (I know that listing detection techniques will allow the smart stealth-crawler programmer to make a better spider, but I do not think that we will ever be able to block smart stealth-crawlers anyway, only the ones that make mistakes.) I'm not talking about the nice crawlers such as googlebot and Yahoo! Slurp. I consider a bot nice if it: identifies itself as a bot in the user agent string reads robots.txt (and obeys it) I'm talking about the bad crawlers, hiding behind common user agents, using my bandwidth and never giving me anything in return. There are some trapdoors that can be constructed updated list (thanks Chris, gs): Adding a directory only listed (marked as disallow) in the robots.txt, Adding invisible links (possibly marked as rel="nofollow"?), style="display: none;" on link or parent container placed underneath another element with higher z-index detect who doesn't understand CaPiTaLiSaTioN, detect who tries to post replies but always fail the Captcha. detect GET requests to POST-only resources detect interval between requests detect order of pages requested detect who (consistently) requests https resources over http detect who does not request image file (this in combination with a list of user-agents of known image capable browsers works surprisingly nice) Some traps would be triggered by both 'good' and 'bad' bots. you could combine those with a whitelist: It trigger a trap It request robots.txt? It doest not trigger another trap because it obeyed robots.txt One other important thing here is: Please consider blind people using a screen readers: give people a way to contact you, or solve a (non-image) Captcha to continue browsing. What methods are there to automatically detect the web crawlers trying to mask themselves as normal human visitors. Update The question is not: How do I catch every crawler. The question is: How can I maximize the chance of detecting a crawler. Some spiders are really good, and actually parse and understand html, xhtml, css javascript, VB script etc... I have no illusions: I won't be able to beat them. You would however be surprised how stupid some crawlers are. With the best example of stupidity (in my opinion) being: cast all URLs to lower case before requesting them. And then there is a whole bunch of crawlers that are just 'not good enough' to avoid the various trapdoors.

    Read the article

  • Testing an iphone web app on windows

    - by JoseMarmolejos
    When developing web apps for the iphone on a mac you can test your app in either Iphoney or the apple supplied simulator; bot of them are excellent for the task but are only available for macs. So I have to ask, are windows alternative for these iphone simulators? So far I could only find this one.

    Read the article

  • Best practices for sending automated daily emails from web service

    - by Tauren
    I am running a web service that currently sends confirmation emails out to new users via the gmail smtp servers. As I'm only getting a few new users each day, this hasn't been a problem. I've recently added new features to the webapp that will require a customized message to be sent out to each user every day. Think of this as similar to the regular messages LinkedIn sends out that give you a status report on the activity in your network. Every user's message will be different. With thousands of users, this means thousands of unique messages will be sent each day. Edit: I've since found that these types of email are called "transactional or relationship messages". Spamtacular has a good article on differentiating between marketing and transactional email. I don't think using gmail's smtp servers will cut it anymore, but I don't know that for sure. I don't know what gmail's maximum outgoing messages per account is (it might be 100/day), but they limit outgoing mail to 500 recipients per message. I'm not sending a single message to 500 recipients, but I'm going to be sending 1000's of customized messages with each recipient getting one per day. I'm interested to learn any best practices for doing this (especially for Java-based webapps). Here are some of my thoughts and concerns on it: Should I set up my own outgoing mail server? If I do this, it seems like I'll have all sorts of other issues to worry about, such as preventing mail server abuse, monitoring bounces, allowing ways to opt-out of emails, etc. Are there any tools or services to help with this? Maybe something like OpenEMM or a services like MailChimp? But those seem focused more toward email marketing campaigns. I don't think I should have the webapp itself handle sending emails as it currently is for new user signups. I'm thinking I should setup a separate messaging server that can access the same backend/datastore as the webapp. Thoughts on this? Should I consider setting up some sort of message queueing service to help with this, such as JMS, RabbitMQ, ActiveMQ, etc.? Do I need to provide users a way to opt-out? Do I need to flag these as bulk messages? I don't really consider these email marketing messages, but I'm unsure what is considered appropriate or proper netiquette. Any advice is appreciated. I'm also very interested in open source tools or web services that simplify things and could help me to ramp up as quickly as possible. Thanks!

    Read the article

  • How to create Ror style Restful routing in Asp.net MVC Web Api

    - by Jas
    How to configure routing in asp.net web api, to that I can code for the following actions in my ApiController inherited class? |======================================================================================| |Http Verb| Path | Action | Used for | |======================================================================================| | GET | /photos | index | display a list of all photos | | GET | /photos/new | new | return an HTML form for creating a new photo | | POST | /photos/ | create | create a new photo | | GET | /photos/:id | show | display a specific photo | | GET | /photos/:id/edit | edit | return an HTML form for editing a photo | | PUT | /photos/:id | update | update a specific photo | | DELETE | /photos/:id | destroy | delete a specific photo |

    Read the article

  • How to create a setup for a web application with Web Platform Installer

    - by Nasser Hajloo
    I have a large Web Application ( ErPwith 11 subsystem) and I want tocreate a setup for itwith Microsoft WebPI. Currently We send our application for customers once a week (for weekly updates). We usefollowing tools in this application, So How to create a setup for out project toconfigure it in client IIS automatically List item .netFramework 3.5 SQL server 2008 Asp.net C# NHibernate Log4net castleProxy SQL Server Reporting Services (RDL) Visual Studio Client Reports (RDLC) Javascript JQuery

    Read the article

  • Problem when creating Web Service

    - by Polaris
    I am using Visual Studio 2008 SP1 and framework 3.5 sp1 on Windows XP sp3. I have Java service which I consume in my .NET application. When attemp to add web service I get next error: The operation could not be complited.An attempt was made to load a program with an incorrect format But on another machine everything works fine. There is not any error in Java service. I think error in VS. Is anybody have seen this error before?

    Read the article

  • technologies beside scaling web applications in a distributed nature

    - by wik
    Hello, I am interested in theory to scale web applications in a distributed nature, i.e. when there is some platform/stack can be extended by others applications running on different servers, etc. I am researching this field and feels the lack of the right keywords :) Interesting concepts found so far: opensocial through API, like shopify does (shopify it's a hosted ecommerce solution) semantic web not quite sure about this one Am I on the right way or am I lost anything? :) Thanks.

    Read the article

  • Making a Web Gui to design a Garden Layout

    - by paddydub
    I would like to design a web page Gui where users can design a simple interactive garden. The user would pick a template design and receive price estimates based on the design template and the dimensions entered. I'd like the user to be able to move items such as plants, stones and be able to adjust the dimensions of the grass, paving. I'm thinking i could make it using flash but I would like to know there are any other ways I could use to implement this?

    Read the article

  • Stateful EJBs in web application?

    - by Sebastien Lorber
    Hello I never used stateful EJBs. I understand that a stateful EJB can be useful with a java client. But i wonder: in which case to use them on a web application? And how? Should we put these stateful beans in Session (because of stateless http)? Is it a good practice? (without debating too much about stateful vs stateless)

    Read the article

  • Web based client for Amazon S3

    - by Dick Lebavo
    We are looking for a secure online solution to access our files stored on Amazon S3. We have about 3K files, mostly media and documents, that we need to make available to our employees on the move. We don't want to develop anything in-house if there is an existing solution. Please note that our employees are not technologically minded , so a simple web based upload/download GUI would work the best.

    Read the article

  • How to decide on what hardware to deploy web application

    - by Yuval A
    Suppose you have a web application, no specific stack (Java/.NET/LAMP/Django/Rails, all good). How would you decide on which hardware to deploy it? What rules of thumb exist when determining how many machines you need? How would you formulate parameters such as concurrent users, simultaneous connections and DB read/write ratio to a decision on how much, and which, hardware you need? Any resources on this issue would be very helpful...

    Read the article

  • Back out plan for a Web App

    - by nobody
    We need a back out plan for a web app whose first maintenance release is going to production soon. The issue we are facing is even if we back out new EAR and deploy old one , the data which was keyed in using new release would not support old business rules(current), since there is enormous changes in business rules. Can you suggest how do we tackle this issue?

    Read the article

  • File structure for PHP-based website.

    - by John Berryman
    I'm building a PHP-based web app for the first time and I haven't found anything to pattern it after. At this point I'm mostly curious about how the files should be arranged into directories so that development of the website can be manageable. This includes javascript scripts, images, stylesheets, cgi scripts, html files, pure php files that define common functions, etc. Question: Can someone point me to an explanation about how such a website is typically organized on the server?

    Read the article

  • Internet Explorer-like clicking sound without using Explorer

    - by Tom
    When I switch on my Windows 7 laptop a few minutes after turning it on I hear a single click sound like the one Internet Explorer gives during browsing. I don't use Explorer at all. I use Firefox. What can it be then? Is it coming from Explorer running in the background? Or is it not related to Explorer at all? Has anyone else experienced something like this?

    Read the article

  • Calculating and billing IP multicast usage on the Internet

    - by obvio171
    I've been searching for the reasons why IP multicast isn't widely supported on the Public Internet, and a commonly-cited reason is the difficulty ISPs have in tracking Multicast usage for later billing. Given this difficulty, since ISPs control the routers and they're not forced to support Multicast (as per IPv4), they just disable it. I couldn't find what this difficulty was though. Since an ISP has full control of any inbound and outbound traffic, be it Unicast or Multicast, what's the difficulty in tracking and billing the latter that does not exist in the former?

    Read the article

  • Internet Sharing via router

    - by mak4pi
    Question: How to setup Internet Sharing among two computers (one Mac 10.5.6, one Ubuntu 9.10) via a router? Situation: The Mac is using AirPort (wireless interface) to connect to router as 192.168.1.101. The Ubuntu is using an ethernet cable to connect to router as 192.168.1.104. Visual: Mac )))))) Router ======= Ubuntu Any ideas are greatly appreciated. Thanks.

    Read the article

  • Intermittent loss of internet connectivity with good wireless signal

    - by rsheart
    I recently bought a HP touch smart running windows 7. At random times, internet connectivity is lost while the wireless signal remains strong. The only way to restore connectivity is to restart the PC. Nothing else works. Have many other PC's on the same router also running windows 7 and no problems there. Have changed routers and problem remains. I have reinstalled the network drivers with no help. Any thoughts?

    Read the article

  • Internet Connection losing

    - by Mehdi Golchin
    When my internet connection is idle for a while about 5 mins, the connection will be lost. It merely happens when the connection is idle, not during download. I'm using an ADSL service by a HUAWEI smartAX MT882a ADSL modem. Any help will be appreciated

    Read the article

  • Troubleshooting an unstable internet connection

    - by Konrad Rudolph
    My MacBook Pro running OS X (10.9, but I had the same problem before) is connected to a Belkin router via WiFi and, using Virgin Media as the ISP, to the internet. The connection is extremely unstable – on some days, I get a ping timeout every few seconds. In addition, some domains seem to suffer general connectivity issues. For instance, I often find that while the youtube.com website loads, none of the videos (which are hosted on a separate domain) do. At other times, videos load but always fail to buffer, even though the actual connection speed is ok, even though I’ve disabled dash playback. Since I’m living in a rented room and the ISP contract isn’t actually mine I’ve got only limited possibilities of addressing the problem. In particular, I have no access to the router configuration and my non tech savvy landlady, while sympathetic, is not in a great hurry to hand the problem over to the ISP’s customer support. What’s more, I seem to be the only person in the house experiencing these problems – but I can imagine that this is simply because I’m the only one who’s using the internet continuously. I’m searching for specific tests that might be able to pinpoint – and ideally solve – the problem. So far all I’ve managed to do is establish that Virgin is routing my traffic in mysterious ways. Here’s an excerpt from traceroute google.co.uk. It’s worth mentioning that the host name doesn’t seem to matter a lot, the trace route is always the same. traceroute: Warning: google.co.uk has multiple addresses; using 62.254.36.148 traceroute to google.co.uk (62.254.36.148), 64 hops max, 52 byte packets 1 (192.168.2.1) 1.112 ms 1.300 ms 2.359 ms 2 10.100.32.1 (10.100.32.1) 11.926 ms 10.217 ms 24.987 ms 3 cmbg-core-1a-ae3-610.network.virginmedia.net (80.1.202.93) 28.809 ms * 66.653 ms 4 popl-bb-1b-ae16-0.network.virginmedia.net (212.43.163.141) 13.759 ms 126.504 ms 20.472 ms 5 nrth-bb-1b-et-010-0.network.virginmedia.net (62.253.175.57) 28.357 ms 16.398 ms 42.387 ms 6 nrth-bb-1c-ae1-0.network.virginmedia.net (62.253.174.110) 27.441 ms 15.622 ms 12.044 ms 7 lutn-icdn-1-ae0-0.network.virginmedia.net (62.253.175.82) 16.678 ms 28.463 ms 28.253 ms 8 * * * 9 * * * 10 * * * ^C If I let it, this goes on until the end of time. It never seems to reach a destination. Is this normal? A friend living in the same town who is also with Virgin Media has a more conventional traceroute output: 7 hops to google.co.uk, all of which send the ICMP TIME_EXCEEDED response. The obvious fix – rebooting the router – doesn’t seem to help. As far as I can tell, the WiFi connection is stable (I can always ping the router) so the problem is further downstream. I’ve tried using an alternative DNS before (OpenDNS) but if anything, this made things worse. In fact, it made all Google services nigh unreachable.

    Read the article

< Previous Page | 30 31 32 33 34 35 36 37 38 39 40 41  | Next Page >