Search Results

Search found 42919 results on 1717 pages for 'web crawler'.

Page 125/1717 | < Previous Page | 121 122 123 124 125 126 127 128 129 130 131 132  | Next Page >

• How to scrape the first paragraph from a wikipedia page?

  - by David

  Let's say I want to grab the first paragraph in this wikipedia page. How do I get the principal text between the title and contents box using XPath or DOM & PHP or something similar? Is there any php library for that? I don't want to use the api because it's a bit complex. Note: i just need that to add a widget under my pages that displays related info from Wikipedia.

  Read the article

• How to mock test a web service in PHPUnit across multiple tests?

  - by scraton

  I am attempting to test a web service interface class using PHPUnit. Basically, this class makes calls to a SoapClient object. I am attempting to test this class in PHPUnit using the "getMockFromWsdl" method described here: http://www.phpunit.de/manual/current/en/test-doubles.html#test-doubles.stubbing-and-mocking-web-services However, since I want to test multiple methods from this same class, every time I setup the object, I also have to setup the mock WSDL SoapClient object. This is causing a fatal error to be thrown: Fatal error: Cannot redeclare class xxxx in C:\web\php5\PEAR\PHPUnit\Framework\TestCase.php(1227) : eval()'d code on line 15 How can I use the same mock object across multiple tests without having to regenerate it off the WSDL each time? That seems to be the problem.

  Read the article

• What should I consider when deploying a new web farm?

  - by tsilb

  My web app has been chugging along just great in Production for years with one App server and one Web server. Now we're moving to a multi-server environment with 2 App and 3 Web servers. I have enough time to make changes before the go-live. As a Developer, what considerations should I take into account from coding, deployment, and architectural/ecosystem management perspectives? Already on my list: Remove tight-coupling between servers Applicable files (i.e. downloadables) stored in IMAGE fields in SQL instead of files on app server Deployment: Take out node out of the farm at a time

  Read the article

• The type or namespace name 'Oledb' does not exist in the namespace 'System.Data' error on Web Servic

  - by Pankaj Kumar

  Hi everyone... i have a webservice that i want to test by typing the url in the address bar in the web browser localhost:1981/myProject/admin/autocomplete.asmx and when i do this it gives this compilation error CS0234: The type or namespace name 'Oledb' does not exist in the namespace 'System.Data' (are you missing an assembly reference?) i know this is because we added this in our web.config <add namespace="System.Data.Oledb"/> <add namespace ="System.Data"/> in the namespaces section..... when i call this web service through ajax it works but if i try to test it it gives this error. Is there any way to prevent this?

  Read the article

• Scrapy - Follow RSS links

  - by Tupak Goliam

  Hello, I was wondering if anyone ever tried to extract/follow RSS links using SgmlLinkExtractor/CrawlSpider. I can't get it to work... I am using the following rule: rules = ( Rule(SgmlLinkExtractor(tags=('link',), attrs=False), follow=True, callback='parse_article'), ) (having in mind that rss links are located in the link tag). I am not sure how to tell SgmlLinkExtractor to extract the text() of the link and not to search the attributes ... Any help is welcome, Thanks in advance

  Read the article

• Is there a sample set of web log data available for testing analysis against?

  - by Peter

  Sorry if this isn't strictly speaking a programming question, but I figure my best chance of success would be to ask here. I'm developing some web log file analysis algorithms, but to date I only have access to a fairly small amount of web log data to process. One algorithm I want to use makes some assumptions about 'the shape' of typical web log data, and so I'd like to test it against a larger 'exemplar' - perhaps the logs of a busy site with a good distribution of traffic from different sources etc. Is there a set of such data available somewhere? Thanks for any help.

  Read the article

• Is there a way to crawl all facebook fan pages?

  - by user220755

  Is there a way to crawl all facebook fan pages and collect some information? like for example crawling facebook fan pages and save their names, or how many fans, etc? Or at least, do you have a hint of how this could be possibly done?

  Read the article

• How can I isolate the form controls in a ASP Web User Control from the rest of the page's form contr

  - by Justin808

  I have a Web User Control I created for authentication. The web user control is inside the box below. Clicking any button (1 or 2) below works correct as it goes to the correct c# button click event in the code behind file. If I press enter on fields a or b it goes to the correct callback (button1's) if I press enter on field c it still goes to button1's callback, not button2's How can I give my web user control a nice self contained for and view state etc, so it wont mess with the remainder of the page's form? +--------------+ | User: __a___ | | Pass: __b___ | | [button1]| +--------------+ Prompt:______c______ [button2]

  Read the article

• Java website on Tomcat PHP website on Apache - how to get PHP web pages into Java web pages?

  - by Venkat

  We have a Java web application deployed on Tomcat. We also setup Apache and mod_proxy_ajp to route web requests (port 80/443) to Tomcat. We would like to deploy a PHP application on the same Apache server - probably under a subdirectory (/var/www/ourapp). Now we would like to access & display web pages from PHP application within web pages generated by Java application. Planning to implement Single Sign-on as well. Example: Web page from java has (JQuery Tabs) and we like to display the PHP web page within a tab while all other HTML comes from java application. Can you please give a overall picture of how to proceed about this? Mainly 1. how we should install/setup our PHP application on same Apache server which is used to route web requests to Tomcat? i.e. either setup sub domain or install in sub directory 2. How to bring PHP pages into present web pages (generated by java). Can we use AJAX requests or should go for Java PHP Bridge/ Querces such applications? Thank you for your time in advance. Regards.

  Read the article

• Problems and solution for Developing a connected web and desktop application?

  - by Taz

  hi, I am trying to develop a web application(Using ASP.NET and c#) that uses a specific database hosted on web server. I will have another desktop application that will use a local database. Both databases have same structure and data at start up. Then databases will change when users add data to web application and an employee adds data to the desktop application. After a while I have to sync both databases. What will be best way to do this? Is there any opensource example/ starter kit to start with? Thanks.

  Read the article

• What ASP.NET Web Config entries could limit certain file access by date and time?

  - by Dr. Zim

  What entries in a web.config could allow certain files to become publicly accessible after a certain date and time? Specifically, we have these files starting with AB_.jpg where the _ could be anything. We put them in a folder on April 27th for example, but they shouldn't be accessible until April 30th at 11:59:59 PM. I think the web.config in part works like Unix's FTP .htaccess file to define file security. For example, this web.config entry allows directory browsing: <?xml version="1.0" encoding="UTF-8"?> <configuration> <system.webServer> <directoryBrowse enabled="true" /> </system.webServer> </configuration>

  Read the article

• Webservice and ORM Framework?

  - by Sebastian

  Does anybody know a good web framework that includes an ORM mapper and allows straight forward implementation of web services? I'm looking for a framework written in PHP or C++. I'm looking for the following features (not all of them required, some will do nicely) data definition in one place used by database and web service WSDL generation XML output/JSON output boilerplate code generation So what I would like is a framework that let's me specify the objects, the web service functions on those objects and then generate everything that is required leaving me to fill the business logic (connecting the database to the web service). Anything like that out there? Background information for why I need this: I'm looking into creating a web project: the client is a rich web application that fetches all its data using AJAX. It will be completely custom made using only a low level javascript library. The server back end is supposed to serve static content and javascript (basically the rich web application) and to provide a RESTful web service API (which I would like to implement using aforementioned framework).

  Read the article

• how to prevent all crawlers except good ones (google, bing, yahoo) access website content?

  - by tranhuyhung

  I just want to let Google, Bing, Yahoo crawl my website to build indexes. But I do not want my opposite website use crawling service to steal my website content. What should I do?

  Read the article

• is it possible to extract all PDFs from a site

  - by deming

  given a URL like www.mysampleurl.com is it possible to crawl through the site and extract links for all PDFs that might exist? I've gotten the impression that Python is good for this kind of stuff. but is this feasible to do? how would one go about implementing something like this? also, assume that the site does not let you visit something like www.mysampleurl.com/files/

  Read the article

• how to scrawl file hosting website with scrapy in python?

  - by Veryel Hua

  Can anyone help me to figure out how to scrawl file hosting website like filefactory.com? I don't want to download all the file hosted but just to index all available files with scrapy. I have read the tutorial and docs with respect to spider class for scrapy. If I only give the website main page as the begining url I wouldn't not scrawl the whole site, because the scrawling depends on links but the begining page seems not point to any file pages. That's the problem I am thinking and any help would be appreciated!

  Read the article

• Best practice when working with web services that return objects?

  - by Raj

  Hello, I'm currently working with web services that return objects such as a list of files e.g. File array. I wanted to know whether its best practice to bind this type of object directly to my front end code for example a repeater/listview or whether to first parse it into my own list of "file class" e.g. customFiles[] If the web service changes then it will break my front end code, however if I create my own CustomFile class, then i would only need to change my code in one place to fix the issue, but it just seems like a lot of extra work to create the same classes from a web service, i wanted to know what is the best practice for this type of work. thanks Raj.

  Read the article

• I can't change a value, for a web service using an axiom client!

  - by adrian7

  Hi.I just started learning about web serices. I succesfuly installed Tomcat, and deployed axis2 on it. I followed the tutorial from here http://ws.apache.org/axis2/1_1_1/quickstartguide.html . I made the StockQuoteService web service, moved the .aar file in the WEB-INF/services folder and tested it - it worked all fine. Then I made a client for the servce. I took the source code of the file AXIOMClient.java from the dir described here http://ws.apache.org/axis2/1_1_1/quickstartguide.html#clientaxiom , imported the classes in Eclipse Java IDE and runned it. I tested and the update method works from the browser when calling http://localhost:8080/axis2/services/StockQuoteService/update?symbol=WSO&price=144 But from my java client update method is not working at all, when calling getPrice I always receive 42.00. What am I doing wrong? or, how can I trace what is happening?

  Read the article

• Is it possible to run C++ binded with SDL+OpenGL code on a web browser?

  - by unknownthreat

  My client wants her website to have an application that renders 3D (light 3D stuff, we are drawing only flat squares in 3D world) but web programming is not my thing. So I am looking for something that can run a C++ program from a web browser. But I think, if this is the case, then the client side must download the program first, and that's not what I want. The client should only be able to use this application only on the website. I came across Google Native Client, which claims that it can run x86 native code in web applications. I haven't decide whether it is worth it or not and I don't know whether this is what I want or not, so I decided to ask experienced people about this. If I want to have something like this, is what I said above possible? Or I completely need other languages like Flex because it does not worth the trouble? Or is Google Native Client suitable for doing something like this?

  Read the article

• Model View Presenter plus ASP.NET Web Service; where does the asmx live?

  - by Dirk

  I've been slowly transitioning from a traditional web forms architecture to the MVP pattern (Passive View). So far, it's been fairly easy to implement b/c the views I've dealt with have all employed a classic PostBack model. However, I've come across my first view that will refresh portions of itself via web services. I can't grok where the web service should live (Presenter I think) or how to expose that asmx end point to my View while still maintaining the clean separation of concerns/testability that MVP affords me. I've searched far and wide for some examples on how this might be implemented and have come up with nothing. Please help!

  Read the article

• Most efficient way for testing links

  - by Burnzy

  I'm currently developping an app that is going through all the files on a server and checking every single hrefs to check wether they are valid or not. Using a WebClient or a HttpWebRequest/HttpWebResponse is kinda overkilling the process because it downloads the whole page each time, which is useless, I only need to check if the link do not return 404. What would be the most efficient way? Socket seems to be a good way of doing it, however I'm not quite sure how this works. Thanks for sharing your expertise!

  Read the article

• Robots.txt with one site but two domains

  - by Dofs

  I have a website which has two domains added. Both domains point to the root of the website. Is it possible to alter the robots.txt so that one of the domains doesn't get crawled, while the other still does?

  Read the article

• How do i get a directory listing for a folder on the web?

  - by JimDel

  How do I get a directory listing for a folder on the web? I'm looking to download a group of small files from a folder on the web. I can do it easily with a a single file but I'm not sure how to do it for multiple files. If there was something similar to the code below but for a folder on the web I think I can do it. private void button1_Click(object sender, EventArgs e) { DirectoryInfo di = new DirectoryInfo("c:/myFolder"); FileInfo[] rgFiles = di.GetFiles("*.*"); foreach (FileInfo fi in rgFiles) { //Do Something with each of them } } Thanks

  Read the article

• IIS Web Farm Framework servers are automatically set to "unavailable" even when they are healthy... And they never return to the available state!

  - by JohannesH

  I have 2 web farm configurations, one with 2 member servers and one with 3 member servers. I have health monitoring set up on both farms and the monitoring tool reports all servers as being healthy. However after a while all the servers are marked as being "Unavailable" and "Healthy" in the "Monitoring and Management" screen (in the "Servers" screen they are all listed with "Yes" in the "Ready for Load Balancing" column). Viewing the event log on both the web farm controller or any of farm servers doesn't reveal anything interesting. there are no warnings or errors in the period where the servers became unavailable. There are a couple of informational events about the worker process getting shut down due to inactivity but I don't hope this is the cause since that would mean that the farms will die during the night when the load is low. Am I missing something? EDIT: Btw, I think its very odd that the application pool shuts down on the servers since the health monitoring system is polling an aspx page on each server. Shouldn't that keep them going? EDIT2: Now I've also experienced this problem with the RTW version of Web Farm Framework 2.

  Read the article

• I need to create a web service using .NET-- where should I start?

  - by j-t-s

  Hi All I am trying to implement a feature. But I've never had anything to do with "Web Services" before, other than using them. I have a desktop application, and I want that application to be able to sort of "post" some information (i.e. email address, username, user-selected options (just plain text) etc) to an application or "web service" on my ASP.NET web server. Can somebody please guide me in the right direction? How would I accomplish this? Thank you :) Jason

  Read the article

• Current state of Erlang web development? Frameworks, Template languages

  - by pommonico

  Django is to Python as Rails is to Ruby. What is the current front-runner for Erlang? Recent Erlang web activity has resulted in many new frameworks but they are all still moving targets. ErlyWeb is very far along but now seems to be dead. If you were to invest significant capital into a new web product (i.e. not a hobby, you use your own money and you use other's investment in you with expectation of payback), would you choose Erlang as a web platform? If so, what framework and technologies would you choose?

  Read the article

< Previous Page | 121 122 123 124 125 126 127 128 129 130 131 132  | Next Page >