Search Results

Search found 325 results on 13 pages for 'tasty spider men'.

Page 2/13 | < Previous Page | 1 2 3 4 5 6 7 8 9 10 11 12  | Next Page >

  • building Mozilla Spider Monkey on Ubuntu

    - by Hussain
    Hi. I'm trying to build spider monkey on ubuntu 10.04 (lucid). However, when I run autoconf2.13 on the js/src directory, it tells me there is no configure.in file. I can't just do the usual ./configure make sudo make install , either. What's up with it?

    Read the article

  • SphinxSearch or a spider - which one to choose?

    - by r2b2
    Hello, here is my problem: We own SiteA and SiteB and they share the same server and database where we have full control. SiteC , siteD and siteE are some of the sites we own as well but reside on a different web hosts. The goal is to create a unified search functionality for all of the sites mentioned above. That is if somebody search for a term in SiteA, the search result will automatically come up with results from SiteB,SiteC,SiteD and Site E too. The search results should be shown under the website they were found in. All these websites content are stored in their own databases. If I use SphinxSearch to index the above sites,I would then require those sites that we dont have complete control with to setup a web service where i can download a database dump or csv file for indexing. Im not quite sure about how a sphider will come into play here so need your opinion. Sphinx or a spider? THanks!

    Read the article

  • Is there any reason to allow Yahoo! Slurp to crawl my site?

    - by James Skemp
    I thought a year or more ago Yahoo! would be using another search engine for results, and no longer using their own Slurp bot. However, a couple of the sites I manage Yahoo! Slurp continues to crawl pages, and seems to ignore the Gone status code when returned (as it keeps coming back). Is there any reason why I wouldn't want to block Yahoo! Slurp via robots.txt or by IP (since it tends to ignore robots.txt in some cases anyways)? I've confirmed that when the bot does hit it is from Yahoo! IPs, so I believe this is a legit instance of the bot. Is Yahoo Search the same as Bing Search now? is a related question, but I don't think it completely answers whether one should add a new block of the bot.

    Read the article

  • Which specific programming activities do women, on average, perform better than men? [closed]

    - by blueberryfields
    Following a recent discussion with female associates in hiring positions for software development/engineering positions, I found out that this kind of information would be incredibly useful to helping make sure that the workforce shows a gender balance. So I went looking. I've found various literature speaking about risk-taking behaviour and patterns, and other statistical differences between men and women when it comes to work performance. See for example this article related to hedge fund management. I have yet to see any such comparison in the computing field. To restate the question: Which specific programming activities do women, on average, perform better than men? Please back up your answers with specific details, preferably by linking to relevant research or, failing that, explaining what you're basing the information on.

    Read the article

  • Storing URLs while Spidering

    - by itemio
    I created a little web spider in python which I'm using to collect URLs. I'm not interested in the content. Right now I'm keeping all the visited URLs in a set in memory, because I don't want my spider to visit URLs twice. Of course that's a very limited way of accomplishing this. So what's the best way to keep track of my visited URLs? Should I use a database? * which one? MySQL, sqlite, postgre? * how should I save the URLs? As a primary key trying to insert every URL before visiting it? Or should I write them to a file? * one file? * multiple files? how should I design the file-structure? I'm sure there are books and a lot of papers on this or similar topics. Can you give me some advice what I should read?

    Read the article

  • E: Sub-process /usr/bin/dpkg returned an error code (1)

    - by Joel
    I cant install or uppdate anything on my system 12.04 I get the error... installArchives() failed: dpkg: error processing libqt4-xmlpatterns (--configure): libqt4-xmlpatterns:amd64 4:4.8.1-0ubuntu4.2 cannot be configured because libqt4-xmlpatterns:i386 is in a different version (4:4.8.1-0ubuntu4.3) dpkg: error processing libqt4-xmlpatterns:i386 (--configure): libqt4-xmlpatterns:i386 4:4.8.1-0ubuntu4.3 cannot be configured because libqt4-xmlpatterns:amd64 is in a different version (4:4.8.1-0ubuntu4.2) dpkg: dependency problems prevent configuration of libqt4-declarative:i386: libqt4-declarative:i386 depends on libqt4-xmlpatterns (= 4:4.8.1-0ubuntu4.3); however: Package libqt4-xmlpatterns:i386 is not configured yet. dpkg: error processing libqt4-declarative:i386 (--configure): dependency problems - leaving unconfigured dpkg: dependency problems prevent configuration of libqt4-declarative: libqt4-declarative depends on libqt4-xmlpatterns (= 4:4.8.1-0ubuntu4.3); however: Version of libqt4-xmlpatterns on system is 4:4.8.1-0ubuntu4.2. dpkg: error processing libqt4-declarative (--configure): dependency problems - leaving unconfigured dpkg: dependency problems prevent configuration of libqtgui4:i386: libqtgui4:i386 depends on libqt4-declarative (= 4:4.8.1-0ubuntu4.3); however: Package libqt4-declarative:i386 is not configured yet. dpkg: error processing libqtgui4:i386 (--configure): dependency problems - leaving unconfigured dpkg: dependency problems prevent configuration of libqtgui4: No apport report written because the error message indicates its a followup error from a previous failure. No apport report written because the error message indicates its a followup error from a previous failure. No apport report written because MaxReports is reached already No apport report written because MaxReports is reached already No apport report written because MaxReports is reached already No apport report written because MaxReports is reached already libqtgui4 depends on libqt4-declarative (= 4:4.8.1-0ubuntu4.3); however: Package libqt4-declarative is not configured yet. dpkg: error processing libqtgui4 (--configure): dependency problems - leaving unconfigured dpkg: dependency problems prevent configuration of libqt4-designer: libqt4-designer depends on libqtgui4 (= 4:4.8.1-0ubuntu4.3); however: Package libqtgui4 is not configured yet. dpkg: error processing libqt4-designer (--configure): dependency problems - leaving unconfigured dpkg: dependency problems prevent configuration of libqt4-designer:i386: libqt4-designer:i386 depends on libqtgui4 (= 4:4.8.1-0ubuntu4.3); however: Package libqtgui4:i386 is not configured yet. dpkg: error processing libqt4-designer:i386 (--configure): dependency problems - leaving unconfigured dpkg: dependency problems prevent configuration of libqt4-opengl: libqt4-opengl depends on libqtgui4 (= 4:4.8.1-0ubuntu4.3); however: Package libqtgui4 is not configured yet. dpkg: error processing libqt4-opengl (--configure): dependency problems - leaving unconfigured dpkg: dependency problems prevent configuration of libqt4-opengl:i386: libqt4-opengl:i386 depends on libqtgui4 (= 4:4.8.1-0ubuntu4.3); however: Package libqtgui4:i386 is not configured yet. dpkg: error processing libqt4-opengl:i386 (--configure): dependency problems - leaving unconfigured dpkg: dependency problems prevent configuration of libqt4-qt3support: libqt4-qt3support depends on libqt4-designer (= 4:4.8.1-0ubuntu4.3); however: Package libqt4-designer is not configured yet. libqt4-qt3support depends on libqtgui4 (= 4:4.8.1-0ubuntu4.3); however: Package libqtgui4 is not configured yet. dpkg: error processing libqt4-qt3support (--configure): dependency problems - leaving unconfigured dpkg: dependency problems prevent configuration of libqt4-qt3support:i386: libqt4-qt3support:i386 depends on libqt4-designer (= 4:4.8.1-0ubuntu4.3); however: Package libqt4-designer:i386 is not configured yet. libqt4-qt3support:i386 depends on libqtgui4 (= 4:4.8.1-0ubuntu4.3); however: Package libqtgui4:i386 is not configured yet. dpkg: error processing libqt4-qt3support:i386 (--configure): dependency problems - leaving unconfigured dpkg: dependency problems prevent configuration of libqt4-scripttools:i386: libqt4-scripttools:i386 depends on libqtgui4 (= 4:4.8.1-0ubuntu4.3); however: Package libqtgui4:i386 is not configured yet. dpkg: error processing libqt4-scripttools:i386 (--configure): dependency problems - leaving unconfigured dpkg: dependency problems prevent configuration of libqt4-svg: libqt4-svg depends on libqtgui4 (= 4:4.8.1-0ubuntu4.3); however: Package libqtgui4 is not configured yet. dpkg: error processing libqt4-svg (--configure): dependency problems - leaving unconfigured dpkg: dependency problems prevent configuration of libqt4-svg:i386: libqt4-svg:i386 depends on libqtgui4 (= 4:4.8.1-0ubuntu4.3); however: Package libqtgui4:i386 is not configured yet. dpkg: error processing libqt4-svg:i386 (--configure): dependency problems - leaving unconfigured Errors were encountered while processing: libqt4-xmlpatterns libqt4-xmlpatterns:i386 libqt4-declarative:i386 libqt4-declarative libqtgui4:i386 libqtgui4 libqt4-designer libqt4-designer:i386 libqt4-opengl libqt4-opengl:i386 libqt4-qt3support libqt4-qt3support:i386 libqt4-scripttools:i386 libqt4-svg libqt4-svg:i386 Error in function: dpkg: dependency problems prevent configuration of libqt4-declarative: libqt4-declarative depends on libqt4-xmlpatterns (= 4:4.8.1-0ubuntu4.3); however: Version of libqt4-xmlpatterns on system is 4:4.8.1-0ubuntu4.2. dpkg: error processing libqt4-declarative (--configure): dependency problems - leaving unconfigured dpkg: error processing libqt4-xmlpatterns (--configure): libqt4-xmlpatterns:amd64 4:4.8.1-0ubuntu4.2 cannot be configured because libqt4-xmlpatterns:i386 is in a different version (4:4.8.1-0ubuntu4.3) dpkg: error processing libqt4-xmlpatterns:i386 (--configure): libqt4-xmlpatterns:i386 4:4.8.1-0ubuntu4.3 cannot be configured because libqt4-xmlpatterns:amd64 is in a different version (4:4.8.1-0ubuntu4.2) dpkg: dependency problems prevent configuration of libqtgui4: libqtgui4 depends on libqt4-declarative (= 4:4.8.1-0ubuntu4.3); however: Package libqt4-declarative is not configured yet. dpkg: error processing libqtgui4 (--configure): dependency problems - leaving unconfigured dpkg: dependency problems prevent configuration of libqt4-declarative:i386: libqt4-declarative:i386 depends on libqt4-xmlpatterns (= 4:4.8.1-0ubuntu4.3); however: Package libqt4-xmlpatterns:i386 is not configured yet. dpkg: error processing libqt4-declarative:i386 (--configure): dependency problems - leaving unconfigured dpkg: dependency problems prevent configuration of libqt4-svg: libqt4-svg depends on libqtgui4 (= 4:4.8.1-0ubuntu4.3); however: Package libqtgui4 is not configured yet. dpkg: error processing libqt4-svg (--configure): dependency problems - leaving unconfigured dpkg: dependency problems prevent configuration of libqt4-opengl: libqt4-opengl depends on libqtgui4 (= 4:4.8.1-0ubuntu4.3); however: Package libqtgui4 is not configured yet. dpkg: error processing libqt4-opengl (--configure): dependency problems - leaving unconfigured dpkg: dependency problems prevent configuration of libqt4-designer: libqt4-designer depends on libqtgui4 (= 4:4.8.1-0ubuntu4.3); however: Package libqtgui4 is not configured yet. dpkg: error processing libqt4-designer (--configure): dependency problems - leaving unconfigured dpkg: dependency problems prevent configuration of libqt4-qt3support: libqt4-qt3support depends on libqt4-designer (= 4:4.8.1-0ubuntu4.3); however: Package libqt4-designer is not configured yet. libqt4-qt3support depends on libqtgui4 (= 4:4.8.1-0ubuntu4.3); however: Package libqtgui4 is not configured yet. dpkg: error processing libqt4-qt3support (--configure): dependency problems - leaving unconfigured dpkg: dependency problems prevent configuration of libqtgui4:i386: libqtgui4:i386 depends on libqt4-declarative (= 4:4.8.1-0ubuntu4.3); however: Package libqt4-declarative:i386 is not configured yet. dpkg: error processing libqtgui4:i386 (--configure): dependency problems - leaving unconfigured dpkg: dependency problems prevent configuration of libqt4-svg:i386: libqt4-svg:i386 depends on libqtgui4 (= 4:4.8.1-0ubuntu4.3); however: Package libqtgui4:i386 is not configured yet. dpkg: error processing libqt4-svg:i386 (--configure): dependency problems - leaving unconfigured dpkg: dependency problems prevent configuration of libqt4-opengl:i386: libqt4-opengl:i386 depends on libqtgui4 (= 4:4.8.1-0ubuntu4.3); however: Package libqtgui4:i386 is not configured yet. dpkg: error processing libqt4-opengl:i386 (--configure): dependency problems - leaving unconfigured dpkg: dependency problems prevent configuration of libqt4-designer:i386: joel@Joel-PC:~$ sudo apt-get install -f [sudo] password for joel: Läser paketlistor... Färdig Bygger beroendeträd Läser tillståndsinformation... Färdig Korrigerar beroenden.... Färdig Följande paket har installerats automatiskt och är inte längre nödvändiga: kde-l10n-sv language-pack-kde-sv-base language-pack-kde-zh-hans-base calligra-l10n-engb calligra-l10n-sv calligra-l10n-zhcn language-pack-kde-en kde-l10n-engb language-pack-kde-sv language-pack-zh-hans-base kde-l10n-zhcn language-pack-zh-hans language-pack-kde-zh-hans language-pack-kde-en-base Använd "apt-get autoremove" för att ta bort dem. Följande ytterligare paket kommer att installeras: libqt4-xmlpatterns Följande paket kommer att uppgraderas: libqt4-xmlpatterns 1 att uppgradera, 0 att nyinstallera, 0 att ta bort och 22 att inte uppgradera. 15 är inte helt installerade eller borttagna. Behöver hämta 0 B/1 033 kB arkiv. Efter denna åtgärd kommer ytterligare 0 B utrymme användas på disken. Vill du fortsätta [J/n]? J dpkg: fel vid hantering av libqt4-xmlpatterns (--configure): libqt4-xmlpatterns:amd64 4:4.8.1-0ubuntu4.2 cannot be configured because libqt4-xmlpatterns:i386 is in a different version (4:4.8.1-0ubuntu4.3) dpkg: fel vid hantering av libqt4-xmlpatterns:i386 (--configure): libqt4-xmlpatterns:i386 4:4.8.1-0ubuntu4.3 cannot be configured because libqt4-xmlpatterns:amd64 is in a different version (4:4.8.1-0ubuntu4.2) dpkg: beroendeproblem förhindrar konfigurering av libqt4-declarative:i386: libqt4-declarative:i386 är beroende av libqt4-xmlpatterns (= 4:4.8.1-0ubuntu4.3), men: Paketet libqt4-xmlpatterns:i386 har inte konfigurerats ännu. dpkg: fel vid hantering av libqt4-declarative:i386 (--configure): beroendeproblem - lämnar okonfigurerad dpkg: beroendeproblem förhindrar konfigurering av libqt4-declarative: libqt4-declarative är beroende av libqt4-xmlpatterns (= 4:4.8.1-0ubuntu4.3), men: Versionen av libqt4-xmlpatterns på systemet är 4:4.8.1-0ubuntu4.2. dpkg: fel vid hantering av libqt4-declarative (--configure): beroendeproblem - lämnar okonfigurerad dpkg: beroendeproblem förhindrar konfigurering av libqtgui4:i386: libqtgui4:i386 är beroende av libqt4-declarative (= 4:4.8.1-0ubuntu4.3), men: Paketet libqt4-declarative:i386 har inte konfigurerats ännu. dpkg: fel vid hantering av libqtgui4:i386 (--configure): beroendeproblem - lämnar okonfigurerad dpkg: beroendeproblem förhindrar koIngen apport-rapport skrevs därför att felmeddelandet indikerar att det är ett efterföljande fel från ett tidigare problem. Ingen apport-rapport skrevs därför att felmeddelandet indikerar att det är ett efterföljande fel från ett tidigare problem. Ingen apport-rapport skrevs därför att felmeddelandet indikerar att det är ett efterföljande fel från ett tidigare problem. Ingen apport-rapport skrevs därför att felmeddelandet indikerar att det är ett efterföljande fel från ett tidigare problem. Ingen apport-rapport skrevs därför att felmeddelandet indikerar att det är ett efterföljande fel från ett tidigare problem. Ingen apport-rapport skrevs därför att felmeddelandet indikerar att det är ett efterföljande fel från ett tidigare problem. nfigurering av libqtgui4: libqtgui4 är beroende av libqt4-declarative (= 4:4.8.1-0ubuntu4.3), men: Paketet libqt4-declarative har inte konfigurerats ännu. dpkg: fel vid hantering av libqtgui4 (--configure): beroendeproblem - lämnar okonfigurerad dpkg: beroendeproblem förhindrar konfigurering av libqt4-designer: libqt4-designer är beroende av libqtgui4 (= 4:4.8.1-0ubuntu4.3), men: Paketet libqtgui4 har inte konfigurerats ännu. dpkg: fel vid hantering av libqt4-designer (--configure): beroendeproblem - lämnar okonfigurerad dpkg: beroendeproblem förhindrar konfigurering av libqt4-designer:i386: libqt4-designer:i386 är beroende av libqtgui4 (= 4:4.8.1-0ubuntu4.3), men: Paketet libqtgui4:i386 har inte konfigurerats ännu. dpkg: fel vid hantering av libqt4-designer:i386 (--configure): beroendeproblem - lämnar okonfigurerad dpkg: beroendeproblem förhindrar konfigurering av libqt4-opengl: libqt4-opengl är beroende av libqtgui4 (= 4:4.8.1-0ubuntu4.3), men: Paketet libqtgui4 har inte konfigurerats ännu. dpkg: fel vid hantering av libqt4-opengl (--configure): beroendeproblem - lämnar okonfigurerad dpkg: beroendeproblem förhindrar konfigurering av libqt4-opengl:i386: libqt4-opengl:i386 är beroende av libqtgui4 (= 4:4.8.1-0ubuntu4.3), men: Paketet libqtgui4:i386 har inte konfigurerats ännu. dpkg: fel vid hantering av libqt4-opengl:i386 (--configure): beroendeproblem - lämnar okonfigurerad dpkg: beroendeproblem förhindrar konfigurering av libqt4-qt3support: libqt4-qt3support är beroende av libqt4-designer (= 4:4.8.1-0ubuntu4.3), men: Paketet libqt4-designer har inte konfigurerats ännu. libqt4-qt3support är beroende av libqtgui4 (= 4:4.8.1-0ubuntu4.3), men: Paketet libqtgui4 har inte konfigurerats ännu. dpkg: fel vid hantering av libqt4-qt3support (--configure): beroendeproblem - lämnar okonfigurerad dpkg: beroendeproblem förhindrar konfigurering av libqt4-qt3support:i386: libqt4-qt3support:i386 är beroende av libqt4-designer (= 4:4.8.1-0ubuntu4.3), men: Paketet libqt4-designer:i386 har inte konfigurerats ännu. libqt4-qt3support:i386 är beroende av libqtgui4 (= 4:4.8.1-0ubuntu4.3), men: Paketet libqtgui4:i386 har inte konfigurerats ännu. dpkg: fel vid hantering av libqt4-qt3support:i386 (--configure): beroendeproblem - lämnar okonfigurerad dpkg: beroendeproblem förhindrar konfigurering av libqt4-scripttools:i386: libqt4-scripttools:i386 är beroende av libqtgui4 (= 4:4.8.1-0ubuntu4.3), men: Paketet libqtgui4:i386 har inte konfigurerats ännu. dpkg: fel vid hantering av libqt4-scripttools:i386 (--configure): beroendeproblem - lämnar okonfigurerad dpkg: beroendeproblem förhindrar konfigurering av libqt4-svg: libqt4-svg är beroende av libqtgui4 (= 4:4.8.1-0ubuntu4.3), men: Paketet libqtgui4 har inte konfigurerats ännu. dpkg: fel vid hantering av libqt4-svg (--configure): beroendeproblem - lämnar okonfigurerad dpkg: beroendeproblem förhindrar konfigurering av libqt4-svg:i386: libqt4-svg:i386 är beroende av libqtgui4 (= 4:4.8.1-0ubuntu4.3), men: Paketet libqtgui4:i386 har inte konfigurerats ännu. dpkg: fel vid hantering av libqt4-svg:i386 (--configure): beroendeproblem - lämnar okonfigurerad Fel uppstod vid hantering: libqt4-xmlpatterns libqt4-xmlpatterns:i386 libqt4-declarative:i386 libqt4-declarative libqtgui4:i386 libqtgui4 libqt4-designer libqt4-designer:i386 libqt4-opengl libqt4-opengl:i386 libqt4-qt3support libqt4-qt3support:i386 libqt4-scripttools:i386 libqt4-svg libqt4-svg:i386 E: Sub-process /usr/bin/dpkg returned an error code (1)

    Read the article

  • wget: Turn Off Forced .html Retreival

    - by Mike B
    When performing a recursive download, I specify a pattern via the -R parameter for wget to reject, but if this file is a HTML file, wget downloads the file regardless of whether or not it matches the pattern. e.g. wget -r -R "*dynamicfile*" example.com still retrieves files such as example.com/dynamicfile1.html Is there a way to prevent this?

    Read the article

  • How do I rate limit google's crawl of my class C IP block?

    - by Zak
    I have several sites in a class C network that all get crawled by google on a pretty regular basis. Normally this is fine. However, when google starts crawling all the sites at the same time, the small set of servers that back this IP block can take a pretty big hit on load. With google webmaster tools, you can rate limit the googlebot on a given domain, but I haven't found a way to limit the bot across an IP network yet. Anyone have experience with this? How did you fix it?

    Read the article

  • Is it true that the Google Spider gives the most relevance of a search result to the first 68 characters of the <title>?

    - by leeand00
    I am reading documentation about my CMS and it states that an HTML page <title> tag is really important in SEO. It states that the Google Spider gives the most relevance to the first 68 characters of a site title. (68 characters being the number of characters that Google will display in it's search engine result pages,) Can anyone verify this is still true? I read in The Information Diet that content farms were getting too good at gaming Google's algorithm for collecting and posting SERPs and so google had to change the search algorithm.

    Read the article

  • What does the arxiv.org anti-bot "search and destroy" actually do?

    - by Brian Campbell
    The lanl.arxiv.org math and scientific preprint service (formerly known as xxx.lanl.gov) has a strict policy against bots that ignore its robots.txt, Robots Beware. On that page, the have a link labelled with "Click here to initiate automated 'seek-and-destroy' against your site", which is forbidden by their robots.txt but presumably badly behaved robots will follow it, and reap the consequences. The question, what are the actual consequences? I have never had the guts to actually click on that link to see what it does. What can they be doing that is both effective and legal?

    Read the article

  • Social Engagement: One Size Doesn't Fit Anyone

    - by Mike Stiles
    The key to achieving meaningful social engagement is to know who you’re talking to, know what they like, and consistently deliver that kind of material to them. Every magazine for women knows this. When you read the article titles promoted on their covers, there’s no mistaking for whom that magazine is intended. And yet, confusion still reigns at many brands as to exactly whom they want to talk to, what those people want to hear, and what kind of content they should be creating for them. In most instances, the root problem is brands want to be all things to all people. Their target audience…the world! Good luck with that. It’s 2012, the age of aggregation and custom content delivery. To cope with the modern day barrage of information, people have constructed technological filters so that content they regard as being “for them” is mostly what gets through. Even if your brand is for men and women, young and old, you may want to consider social properties that divide men from women, and young from old. Yes, a man might find something in a women’s magazine that interests him. But that doesn’t mean he’s going to subscribe to it, or buy even one issue. In fact he’ll probably never see the article he’d otherwise be interested in, because in his mind, “This isn’t for me.” It wasn’t packaged for him. News Flash: men and women are different. So it’s a tall order to craft your Facebook Page or Twitter handle to simultaneously exude the motivators for both. The Harris Interactive study “2012 Connecting and Communicating Online: State of Social Media” sheds light on the differing social behaviors and drivers. -65% of women (vs. 59% of men) stay glued to social because they don’t want to miss anything. -25% of women check social when they wake up, before they check email. Only 18% of men check social before e-mail. -95% of women surveyed belong to Facebook vs. 86% of men. -67% of women log in to Facebook once a day or more vs. 54% of men. -Conventional wisdom is Pinterest is mostly a woman-thing, right? That may be true for viewing, but not true for sharing. Men are actually more likely to share on Pinterest than women, 23% to 10%. -The sharing divide extends to YouTube. 68% of women use it mainly for consumption, as opposed to 52% of men. -Women are as likely to have a Twitter account as men, but they’re much less likely to check it often. 54% of women check it once a week compared to 2/3 of men. Obviously, there are some takeaways from this depending on your target. Women don’t want to miss out on anything, so serialized content might be a good idea, right? Promotional posts that lead to a big payoff could keep them hooked. Posts for women might be better served first thing in the morning. If sharing is your goal, maybe male-targeted content is more likely to get those desired shares. And maybe Twitter is a better place to aim your male-targeted content than Facebook. Some grocery stores started experimenting with male-only aisles. The results have been impressive. Why? Because while it’s true men were finding those same items in the store just fine before, now something has been created just for them. They have a place in the store where they belong. Each brand’s strategy and targets are going to differ. The point is…know who you’re talking to, know how they behave, know what they like, and deliver content using any number of social relationship management targeting tools that meets their expectations. If, however, you’re committed to a one-size-fits-all, “our content is for everybody” strategy (or even worse, a “this is what we want to put out and we expect everybody to love it” strategy), your content will miss the mark for more often than it hits. @mikestilesPhoto via stock.schng

    Read the article

  • web spidering/crawling, can i do it or just search engines?

    - by bboyreason
    i already had a question answered about web-scraping with wget. but as i read a little more, i realize i may be looking for a web-crawling program. particularly the part about web-crawlers being able to get specific data like links or, in my case, products. all of the products on my site have the following naming convention, website.com/uniqueAlphaNumericID.html as far as i know, no dynamic content generation is being used and only one page per one item in the above format. should i just be thinking about: wget website.com | grep *.html or should i be looking into spiders/crawlers?

    Read the article

  • Response code for Chinese spiders? [closed]

    - by pt2ph8
    My server is being "attacked" by Chinese spiders that don't respect the rules in my robots.txt. They are being very aggressive and using a lot of resources, so I'm going to set up some rules in nginx to block them by user agent. Question: which response code should I return, 403, 444 (empty response in nginx) or something else? I'm wondering how the spiders will react to different status codes. What's the best practice?

    Read the article

  • Legality, terms of service for performing a web crawl

    - by Berlin Brown
    I was going to crawl a site for some research I was collecting. But, apparently the terms of service is quite clear on the topic. Is it illegal to now "follow" the terms of service. And what can the site normally do? Here is an example clause in the TOS. Also, what about sites that don't provide this particular clause. Restrictions: "use any robot, spider, site search application, or other automated device, process or means to access, retrieve, scrape, or index the site" It is just research? Edit: "OK, from the standpoint of designing an efficient crawler. Should I provide some form of natural language engine to read terms of service and then abide by them."

    Read the article

  • Worried about spiders repeatedly hitting high-demand page

    - by Matt Thrower
    Due to some rather bizarre architectural considerations I've had to set up something that really ought to run as a console application as a web page. It does the job of writing a large variety of text files and xml feeds from our site data for various other services to pick up so obviously it takes a little while to run and is pretty processor intensive. However, before I deploy it I'm rather worried that it might get hit repeatedly by spiders and the like. It's fine for the data to be re-written but continual hits on this page are going to trigger performance issues for obvious reasons. Is this something I ought to worry about? Or in reality is spider traffic unlikely to be intensive enough to cause problems?

    Read the article

  • Maximum page fetch with maximum bandwith

    - by Ehsan
    Hi I want to create an application like a spider I've implement fetching page as the following code in multi-thread application but there is two problem 1) I want to use my maximum bandwidth to send/receive request, how should I config my request to do so (Like Download Accelerator application and the like) cause I heard the normal application will use 66% of the available bandwidth. 2) I don't know what exactly HttpWebRequest.KeepAlive do, but as its name implies I think i can create a connection to a website and without closing the connection sending another request to that web site using existing connection. does it boost performance or Im wrong ?? public PageFetchResult Fetch() { PageFetchResult fetchResult = new PageFetchResult(); try { HttpWebRequest req = (HttpWebRequest)HttpWebRequest.Create(URLAddress); HttpWebResponse resp = (HttpWebResponse)req.GetResponse(); Uri requestedURI = new Uri(URLAddress); Uri responseURI = resp.ResponseUri; string resultHTML = ""; byte[] reqHTML = ResponseAsBytes(resp); if (!string.IsNullOrEmpty(FetchingEncoding)) resultHTML = Encoding.GetEncoding(FetchingEncoding).GetString(reqHTML); else if (!string.IsNullOrEmpty(resp.CharacterSet)) resultHTML = Encoding.GetEncoding(resp.CharacterSet).GetString(reqHTML); req.Abort(); resp.Close(); fetchResult.IsOK = true; fetchResult.ResultHTML = resultHTML; } catch (Exception ex) { fetchResult.IsOK = false; fetchResult.ErrorMessage = ex.Message; } return fetchResult; }

    Read the article

  • Mamimum page fetch with maximum bandwith

    - by Ehsan
    Hi I want to create an application like a spider I've implement fetching page as the following code in multi-thread application but there is two problem 1) I want to use my maximum bandwidth to send/receive request, how should I config my request to do so (Like Download Accelerator application and the like) cause I heard the normal application will use 66% of the available bandwidth. 2) I don't know what exactly HttpWebRequest.KeepAlive do, but as its name implies I think i can create a connection to a website and without closing the connection sending another request to that web site using existing connection. does it boost performance or Im wrong public PageFetchResult Fetch() { PageFetchResult fetchResult = new PageFetchResult(); try { HttpWebRequest req = (HttpWebRequest)HttpWebRequest.Create(URLAddress); HttpWebResponse resp = (HttpWebResponse)req.GetResponse(); Uri requestedURI = new Uri(URLAddress); Uri responseURI = resp.ResponseUri; if (Uri.Equals(requestedURI, responseURI)) { string resultHTML = ""; byte[] reqHTML = ResponseAsBytes(resp); if (!string.IsNullOrEmpty(FetchingEncoding)) resultHTML = Encoding.GetEncoding(FetchingEncoding).GetString(reqHTML); else if (!string.IsNullOrEmpty(resp.CharacterSet)) resultHTML = Encoding.GetEncoding(resp.CharacterSet).GetString(reqHTML); req.Abort(); resp.Close(); fetchResult.IsOK = true; fetchResult.ResultHTML = resultHTML; } else { URLAddress = responseURI.AbsoluteUri; relayPageCount++; if (relayPageCount > 5) { fetchResult.IsOK = false; fetchResult.ErrorMessage = "Maximum page redirection occured."; relayPageCount = 0; return fetchResult; } req.Abort(); resp.Close(); return Fetch(); } } catch (Exception ex) { fetchResult.IsOK = false; fetchResult.ErrorMessage = ex.Message; } return fetchResult; }

    Read the article

  • Detecting 'stealth' web-crawlers

    - by Jacco
    What options are there to detect web-crawlers that do not want to be detected? (I know that listing detection techniques will allow the smart stealth-crawler programmer to make a better spider, but I do not think that we will ever be able to block smart stealth-crawlers anyway, only the ones that make mistakes.) I'm not talking about the nice crawlers such as googlebot and Yahoo! Slurp. I consider a bot nice if it: identifies itself as a bot in the user agent string reads robots.txt (and obeys it) I'm talking about the bad crawlers, hiding behind common user agents, using my bandwidth and never giving me anything in return. There are some trapdoors that can be constructed updated list (thanks Chris, gs): Adding a directory only listed (marked as disallow) in the robots.txt, Adding invisible links (possibly marked as rel="nofollow"?), style="display: none;" on link or parent container placed underneath another element with higher z-index detect who doesn't understand CaPiTaLiSaTioN, detect who tries to post replies but always fail the Captcha. detect GET requests to POST-only resources detect interval between requests detect order of pages requested detect who (consistently) requests https resources over http detect who does not request image file (this in combination with a list of user-agents of known image capable browsers works surprisingly nice) Some traps would be triggered by both 'good' and 'bad' bots. you could combine those with a whitelist: It trigger a trap It request robots.txt? It doest not trigger another trap because it obeyed robots.txt One other important thing here is: Please consider blind people using a screen readers: give people a way to contact you, or solve a (non-image) Captcha to continue browsing. What methods are there to automatically detect the web crawlers trying to mask themselves as normal human visitors. Update The question is not: How do I catch every crawler. The question is: How can I maximize the chance of detecting a crawler. Some spiders are really good, and actually parse and understand html, xhtml, css javascript, VB script etc... I have no illusions: I won't be able to beat them. You would however be surprised how stupid some crawlers are. With the best example of stupidity (in my opinion) being: cast all URLs to lower case before requesting them. And then there is a whole bunch of crawlers that are just 'not good enough' to avoid the various trapdoors.

    Read the article

  • Slide Men&uacute; con Jquery &amp; Asp.net

    - by Jason Ulloa
    En este post, trabajaremos una parte que en ocasiones se nos hace un “mundo”, la creación de menús en nuestras aplicaciones web. Nuestro objetivo será evitar la utilización de elementos que puedan ocasionar que la página se vuelva un poco lenta, para ello utilizaremos jquery que viene siendo una herramienta muy semejante a ajax para crear nuestro menú. Para crear nuestro menús de ejemplo necesitaremos de tres elementos: 1. CSS, para aplicar los estilos. 2. Jquery para realizar las animaciones. 3. Imágenes para armar los menús. Nuestro primer Paso: Será agregar la referencias a nuestra página, para incluir los CSS y los Scripts. 1: <link rel="stylesheet" type="text/css" href="Styles/jquery.hrzAccordion.defaults.css" /> 2: <link rel="stylesheet" type="text/css" href="Styles/jquery.hrzAccordion.examples.css" /> 3: <script type="text/javascript" src="JS/jquery-1.3.2.js"></script> 1:  2: <script type="text/javascript" src="JS/jquery.easing.1.3.js"> 1: </script> 2: <script type="text/javascript" src="JS/jquery.hrzAccordion.js"> 1: </script> 2: <script type="text/javascript" src="JS/jquery.hrzAccordion.examples.js"> </script> Nuestro segundo paso: Será la definición del html que contendrá los elementos de tipo imagen y el texto. 1: <li> 2: <div class="handle"> 3: <img src="images/title1.png" /></div> 4: <img src="images/image_test.gif" align="left" /> 5: <h3> 6: Contenido 1</h3> 7: <p> 8: Contenido de Ejemplo 1.<br> 9: <br> 10: Agregue todo el contenido aquí</p> 11: </li> En el código anterior, hemos definido un elemento que contendrá una imagen que se mostrará dentro del menú una vez desplegado. Una etiqueta H3 de html que tendrá el Título y un elemento <p> para definir el parrado de texto. Como vemos es algo realmente sencillo. Si queremos agregar mas elementos, será nada mas copiar el div anterior y agregar nuevo contenido. Al final, nuestro menú debe lucir algo así: Por último, les dejo el ejemplo para descargar

    Read the article

  • Turning on collision crashes game

    - by MomentumGaming
    I am getting a null pointer excecption to both my sprite and level. I am working on my mob class, and when I try to move him and the move function is called, the game crashes after checking collision with a null pointer excecption. Taking out the one line that actually checks if the tile located in front of it fixes the problem. Also, if i keep collision ON but don't move the position of the mob (the spider) the game works fine. I will have collision, and the spider appears on the screen, only problem is, getting it to move causes this nasty error that i just can't fix. true Exception in thread "Display" java.lang.NullPointerException at com.apcompsci.game.entity.mob.Mob.collision(Mob.java:67) at com.apcompsci.game.entity.mob.Mob.move(Mob.java:38) at com.apcompsci.game.entity.mob.spider.update(spider.java:58) at com.apcompsci.game.level.Level.update(Level.java:55) at com.apcompsci.game.Game.update(Game.java:128) at com.apcompsci.game.Game.run(Game.java:106) at java.lang.Thread.run(Unknown Source) Here is my renderMob mehtod: public void renderMob(int xp,int yp,Sprite sprite,int flip) { xp -= xOffset; yp-=yOffset; for(int y = 0; y<32; y++) { int ya = y + yp; int ys = y; if(flip == 2||flip == 3)ys = 31-y; for(int x = 0; x<32; x++) { int xa = x + xp; int xs = x; if(flip == 1||flip == 3)xs = 31-x; if(xa < -32 || xa >=width || ya<0||ya>=height) break; if(xa<0) xa =0; int col = sprite.pixels[xs+ys*32]; if(col!= 0x000000) pixels[xa+ya*width] = col; } } } My spider class which determines the sprite and where I control movement, also rendering the spider onto the screen, when I increment ya to move the sprite, I get the crash, but without ya++, it runs flawlessly with a spider sprite on screen: package com.apcompsci.game.entity.mob; import com.apcompsci.game.entity.mob.Mob.Direction; import com.apcompsci.game.graphics.Screen; import com.apcompsci.game.graphics.Sprite; import com.apcompsci.game.level.Level; public class spider extends Mob{ Direction dir; private Sprite sprite; private boolean walking; public spider(int x, int y) { this.x = x <<4; this.y = y <<4; sprite = sprite.spider_forward; } public void update() { int xa = 0, ya = 0; ya++; if(ya<0) { sprite = sprite.spider_forward; dir = Direction.UP; } if(ya>0) { sprite = sprite.spider_back; dir = Direction.DOWN; } if(xa<0) { sprite = sprite.spider_side; dir = Direction.LEFT; } if(xa>0) { sprite = sprite.spider_side; dir = Direction.LEFT; } if(xa!= 0 || ya!= 0) { System.out.println("true"); move(xa,ya); walking = true; } else{ walking = false; } } public void render(Screen screen) { screen.renderMob(x, y, sprite, 0); } } This is th mob class that contains the move() method that is called in the spider class above. This move method calls the collision method. tile and sprite comes up null in the debugger: package com.apcompsci.game.entity.mob; import java.util.ArrayList; import java.util.List; import com.apcompsci.game.entity.Entity; import com.apcompsci.game.entity.projectile.DemiGodProjectile; import com.apcompsci.game.entity.projectile.Projectile; import com.apcompsci.game.graphics.Sprite; public class Mob extends Entity{ protected Sprite sprite; protected boolean moving = false; protected enum Direction { UP,DOWN,LEFT,RIGHT } protected Direction dir; public void move(int xa,int ya) { if(xa != 0 && ya != 0) { move(xa,0); move(0,ya); return; } if(xa>0) dir = Direction.RIGHT; if(xa<0) dir = Direction.LEFT; if(ya>0)dir = Direction.DOWN; if(ya<0)dir = Direction.UP; if(!collision(xa,ya)){ x+= xa; y+=ya; } } public void update() { } public void shoot(int x, int y, double dir) { //dir = Math.toDegrees(dir); Projectile p = new DemiGodProjectile(x, y,dir); level.addProjectile(p); } public boolean collision(int xa,int ya) { boolean solid = false; for(int c = 0; c<4; c++) { int xt = ((x+xa) + c % 2 * 14 - 8 )/16; int yt = ((y+ya) + c / 2 * 12 +3 )/16; if(level.getTile(xt, yt).solid()) solid = true; } return solid; } public void render() { } } Finally, here is the method in which i call the add() method for the spider to add it to the level: protected void loadLevel(String path) { try{ BufferedImage image = ImageIO.read(SpawnLevel.class.getResource(path)); int w = width =image.getWidth(); int h = height = image.getHeight(); tiles = new int[w*h]; image.getRGB(0, 0, w,h, tiles,0, w); } catch(IOException e){ e.printStackTrace(); System.out.println("Exception! Could not load level file!"); } add(new spider(20,45)); } I don't think i need to include the level class but just in case, I have provided a gistHub link for better context. It contains all of the full classes listed above , plus my entity class and maybe another. Thanks for the help if you decide to do so, much appreciated! Also, please tell me if i'm in the wrong section of stackeoverflow, i figured that since this is the gamign section that it belonged but debugging code normally goes into the general section.

    Read the article

  • How to track humans, but not spiders?

    - by johnnietheblack
    I am writing a nice little PHP/MySQL/JS ad system for my site - charging by impression. Therefore, my clients would like to be sure that they aren't paying for impressions caused by robots / spiders / etc. Is there a decently effective way to decipher between human and robot, without doing something obstructive like a captcha, or requesting "no index" or something? Basically, is there some sort of "name tag" that legit spiders have on so that my site knows their presence, therefore letting me react accordingly? And, if there isn't...how do ad companies like Google defend against this?

    Read the article

  • JDOM 1.1: hyphen is not a valid comment character

    - by Stefan Kendall
    I'm using tagsoup to clean some HTML I'm scraping from the internet, and I'm getting the following error when parsing through pages with comments: The data "- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - " is not legal for a JDOM comment: Comment data cannot start with a hyphen. I'm using JDOM 1.1, and here's the code that does the actual cleaning: SAXBuilder builder = new org.jdom.input.SAXBuilder("org.ccil.cowan.tagsoup.Parser"); // build // Don't check the doctype! At our usage rate, we'll get 503 responses // from the w3. builder.setEntityResolver(dummyEntityResolver); Reader in = new StringReader(str); org.jdom.Document doc = builder.build(in); String cleanXmlDoc = new org.jdom.output.XMLOutputter().outputString(doc); Any idea what's going wrong, or how to fix this? I need to be able to parse pages with long comment strings of <!--------- data ------------>

    Read the article

< Previous Page | 1 2 3 4 5 6 7 8 9 10 11 12  | Next Page >