Search Results

Search found 16086 results on 644 pages for 'screen scraping'.

Page 69/644 | < Previous Page | 65 66 67 68 69 70 71 72 73 74 75 76  | Next Page >

  • CasperJS Load next page in loop

    - by SquiresSquire
    I've been working on a script which collates the scores for a list of user from a website. One problem is though, I'm trying to load the next page in the while loop, but the function is not being loaded... this.thenOpen("http://www.url.com?ul=" + currentName + "&sortdir=desc&sort=lastfound", function (id) { return function () { this.capture("Screenshots/" + json.username[id] + ".png"); if (!casper.exists(x("//*[contains(text(), 'That username does not exist in the system')]"))) { if (casper.exists(x('//*[@id="ctl00_ContentBody_ResultsPanel"]/table[2]'))){ this.thenEvaluate(tgsagc.tagNextLink); tgsagc.cacheCount = 0; tgsagc.continue = true; this.echo("------------ " + json.username[id] + " ------------"); while (tgsagc.continue) { this.then(function(){ this.evaluate(tgsagc.tagNextLink); var findDates, pageNumber; pageNumber = this.evaluate(tgsagc.pageNumber); findDates = this.evaluate(tgsagc.getFindDates); this.echo("Found " + findDates.length + " on page " + pageNumber); tgsagc.checkFinds(findDates); this.echo(tgsagc.cacheCount + " Caches for " + json.username[id]); this.echo("Continue? " + tgsagc["continue"]); return this.click("#tgsagc-link-next"); }); } leaderboard[json.username[id]] = tgsagc.cacheCount; console.log("Final Count: " + leaderboard[json.username[id]]); console.log(JSON.stringify(leaderboard)); } else { this.echo("------------ " + json.username[id] + " ------------"); this.echo("0 Caches Found"); leaderboard[json.username[id]] = 0; console.log(JSON.stringify(leaderboard)); } } else { this.echo("------------ " + json.username[id] + " ------------"); this.echo("No User found with that Username"); leaderboard[json.username[id]] = null; console.log(JSON.stringify(leaderboard)); }

    Read the article

  • C# WebClient - View source question

    - by Jim
    I'm using a C# WebClient to post login details to a page and read the all the results. The page I am trying to load includes flash (which, in the browser, translates into HTML). I'm guessing it's flash to avoid being picked up by search engines??? The flash I am interested in is just text (not an image/video) etc and when I "View Selection Source" in firefox I do actually see the text, within HTML, that I want to see. (Interestingly when I view the source for the whole page I do not see the text, within HTML, that I want to see. Could this be related?) Currently after I have posted my login details, and loaded the HTML back, I see the page which does NOT show the flash HTML (as if I had viewed source for the whole page). Thanks in advance, Jim PS: I should point out that the POST is actually working, my log in is successful.

    Read the article

  • Add a script tag to HTML with JSoup preserve path tags

    - by Bhuvnesh Pratap
    I am adding a new script tag to my DOM with JSoup like this : rootElement.after("<script src='<?=base_url()?>js/read.js'></script>") but what I end up getting is this : <script src="&lt;?=base_url()?&gt;js/read.js"></script> I intend to preserve this "<?=base_url()?>" and achieve something like: <script src="<?=base_url()>js/read.js"></script> string here , How could I do this ?

    Read the article

  • Programmatically login to a website and redirect the user to the logged in page?

    - by Santhosh
    Hi, Right now, I have all the employees of my company login to an external website using the company id, username and a password. We are trying to integrate it into an intranet portal which should provide seamless access to this website without requiring the user to enter these credentials. Is there any way of doing this programmatically (.NET C#)? Very similar to screenscraping, Can I simulate the appropriate POST action and then redirect the user to the logged in page? Any help is appreciated. Thanks.

    Read the article

  • Extract anything that looks like links from large amount of data in python

    - by Riz
    Hi, I have around 5 GB of html data which I want to process to find links to a set of websites and perform some additional filtering. Right now I use simple regexp for each site and iterate over them, searching for matches. In my case links can be outside of "a" tags and be not well formed in many ways(like "\n" in the middle of link) so I try to grab as much "links" as I can and check them later in other scripts(so no BeatifulSoup\lxml\etc). The problem is that my script is pretty slow, so I am thinking about any ways to speed it up. I am writing a set of test to check different approaches, but hope to get some advices :) Right now I am thinking about getting all links without filtering first(maybe using C module or standalone app, which doesn't use regexp but simple search to get start and end of every link) and then using regexp to match ones I need.

    Read the article

  • How to grab data on website?

    - by Doug
    So, often, I check my accounts for different numbers. For example, my affiliate accounts- i check for cash increase. I want to program a script where it can login to all these websiets and then grab the money value for me and display it on one page. How can I program this?

    Read the article

  • convert a textview, including those contents off the screen, to bitmap

    - by user623318
    Hi, I want to save(export) contents of MyView, which extends TextView, into a bitmap. I followed the code: [this][1]. It works fine when the size of the text is small. But when there are lots of texts, and some of the content is out of the screen, what I got is only what showed in the screen. Then I add a "layout" in my code: private class MyView extends TextView{ public MyView(Context context) { super(context); // TODO Auto-generated constructor stub } public Bitmap export(){ Layout l = getLayout(); int width = l.getWidth() + getPaddingLeft() + getPaddingRight(); int height = l.getHeight() + getPaddingTop() + getPaddingBottom(); Bitmap viewBitmap = Bitmap.createBitmap(width, height, Bitmap.Config.ARGB_8888); Canvas canvas = new Canvas(viewBitmap); setCursorVisible(false); layout(0, 0, width, height); draw(canvas); setCursorVisible(true); return viewBitmap; } } Now the strange thing happened: The first time I invoke "export"(I use an option key to do that), I got contents only on the screen. When I invoke "export" again, I got complete contents, including those out of the screen. Why? How to "export" a view, including contents cannot be showed on the screen? Thank you! [1]: http://www.techjini.com/blog/2010/02/10/quicktip-how-to-convert-a-view-to-an-image-android/ this

    Read the article

  • What must I learn to parse dynamic HTML sites with PHP?

    - by butteff
    What I must to learn to write php web-site grabber (parser)? It must collect information from other websites, such as as weather forecast, wiki "on this day", some news and other useful and interesting "every day" information! what i must to read for writing m3u player on php? sorry for my bad english

    Read the article

  • How do I open a hardware accelarated DirectX window on a secondary screen

    - by user567021
    I'm looking to create a hardware accelarated DirectX (9 at the moment) window on a secondary screen. This screen is connected to the same graphics display as the primary screen (at least at the moment). Currently, when I try to open the window on the secondary screen based on window position or by dragging it there, CPU usage jumps by about 10%, which seems to indicate that windows is switching to a software fallback rather than the hardware accelaration. Machine is windows XP running a NVIDIA graphics card (varying cards as this runs on several machines), with the latest driver. It's also running CUDA at the same time to produce the images if that matters. Programming language is c++, manual window and message queue creation, no tookbox used at the moment to manage the GUI Thanks

    Read the article

  • Distinguish between screen timeout and power-button-press?

    - by Nick
    Hi, my app does something when the screen goes black, but I want it to only carry out that task if the screen was turned off "by itself", through a screen timeout - NOT when the user presses the power-button. Is there any way to distinguish between those two events? ACTION_SCREEN_OFF obviously fires in both cases, and I haven't found any other intents that might match what I'm looking for. Thanks for your help, Nick

    Read the article

  • Prevent UIWebView sliding off the screen (iPhone)

    - by DisposableHeart
    I have a full screen UIWebView component on the screen with fixed size content, so no scrolling required. When I nudge the screen the content in moving up or down. The same effect can one experience all over the iPhone when there is a scroll-enabled component. I would like to prevent this happening in my application. Please shed some light...

    Read the article

  • Screen capture code produces black bitmap

    - by wadetandy
    I need to add the ability to take a screenshot of the entire screen, not just the current window. The following code produces a bmp file with the correct dimensions, but the image is completely black. What am I doing wrong? void CaptureScreen(LPCTSTR lpszFilePathName) { BITMAPFILEHEADER bmfHeader; BITMAPINFO *pbminfo; HBITMAP hBmp; FILE *oFile; HDC screen; HDC memDC; int sHeight; int sWidth; LPBYTE pBuff; BITMAP bmp; WORD cClrBits; RECT rcClient; screen = GetDC(0); memDC = CreateCompatibleDC(screen); sHeight = GetDeviceCaps(screen, VERTRES); sWidth = GetDeviceCaps(screen, HORZRES); //GetObject(screen, sizeof(BITMAP), &bmp); hBmp = CreateCompatibleBitmap ( screen, sWidth, sHeight ); // Retrieve the bitmap color format, width, and height. GetObject(hBmp, sizeof(BITMAP), (LPSTR)&bmp) ; // Convert the color format to a count of bits. cClrBits = (WORD)(bmp.bmPlanes * bmp.bmBitsPixel); if (cClrBits == 1) cClrBits = 1; else if (cClrBits bmiHeader.biSize = sizeof(BITMAPINFOHEADER); pbminfo-bmiHeader.biWidth = bmp.bmWidth; pbminfo-bmiHeader.biHeight = bmp.bmHeight; pbminfo-bmiHeader.biPlanes = bmp.bmPlanes; pbminfo-bmiHeader.biBitCount = bmp.bmBitsPixel; if (cClrBits bmiHeader.biClrUsed = (1bmiHeader.biCompression = BI_RGB; // Compute the number of bytes in the array of color // indices and store the result in biSizeImage. // The width must be DWORD aligned unless the bitmap is RLE // compressed. pbminfo-bmiHeader.biSizeImage = ((pbminfo-bmiHeader.biWidth * cClrBits +31) & ~31) /8 * pbminfo-bmiHeader.biHeight; // Set biClrImportant to 0, indicating that all of the // device colors are important. pbminfo-bmiHeader.biClrImportant = 0; CreateBMPFile(lpszFilePathName, pbminfo, hBmp, memDC); } void CreateBMPFile(LPTSTR pszFile, PBITMAPINFO pbi, HBITMAP hBMP, HDC hDC) { HANDLE hf; // file handle BITMAPFILEHEADER hdr; // bitmap file-header PBITMAPINFOHEADER pbih; // bitmap info-header LPBYTE lpBits; // memory pointer DWORD dwTotal; // total count of bytes DWORD cb; // incremental count of bytes BYTE *hp; // byte pointer DWORD dwTmp; int lines; pbih = (PBITMAPINFOHEADER) pbi; lpBits = (LPBYTE) GlobalAlloc(GMEM_FIXED, pbih-biSizeImage); // Retrieve the color table (RGBQUAD array) and the bits // (array of palette indices) from the DIB. lines = GetDIBits(hDC, hBMP, 0, (WORD) pbih-biHeight, lpBits, pbi, DIB_RGB_COLORS); // Create the .BMP file. hf = CreateFile(pszFile, GENERIC_READ | GENERIC_WRITE, (DWORD) 0, NULL, CREATE_ALWAYS, FILE_ATTRIBUTE_NORMAL, (HANDLE) NULL); hdr.bfType = 0x4d42; // 0x42 = "B" 0x4d = "M" // Compute the size of the entire file. hdr.bfSize = (DWORD) (sizeof(BITMAPFILEHEADER) + pbih-biSize + pbih-biClrUsed * sizeof(RGBQUAD) + pbih-biSizeImage); hdr.bfReserved1 = 0; hdr.bfReserved2 = 0; // Compute the offset to the array of color indices. hdr.bfOffBits = (DWORD) sizeof(BITMAPFILEHEADER) + pbih-biSize + pbih-biClrUsed * sizeof (RGBQUAD); // Copy the BITMAPFILEHEADER into the .BMP file. WriteFile(hf, (LPVOID) &hdr, sizeof(BITMAPFILEHEADER), (LPDWORD) &dwTmp, NULL); // Copy the BITMAPINFOHEADER and RGBQUAD array into the file. WriteFile(hf, (LPVOID) pbih, sizeof(BITMAPINFOHEADER) + pbih-biClrUsed * sizeof (RGBQUAD), (LPDWORD) &dwTmp, ( NULL)); // Copy the array of color indices into the .BMP file. dwTotal = cb = pbih-biSizeImage; hp = lpBits; WriteFile(hf, (LPSTR) hp, (int) cb, (LPDWORD) &dwTmp,NULL); // Close the .BMP file. CloseHandle(hf); // Free memory. GlobalFree((HGLOBAL)lpBits); }

    Read the article

  • How to extract the data from a website using java?

    - by giri
    Hi I am familier with java programming language I like to extract the data from a website and store it to my database running on my machine.Is that possible in java.If so which API I should use. For example the are number of schools listed on a website How can I extract that data and store it to my database using java.

    Read the article

  • getting text that will be displayed to user from html

    - by gordatron
    Bit of a random one, i am wanting to have a play with some NLP stuff and I would like to: Get all the text that will be displayed to the user in a browser from HTML. My ideal output would not have any tags in it and would only have fullstops (and any other punctuation used) and new line characters, though i can tolerate a fairly reasonable amount of failure in this (random other stuff ending up in output). If there was a way of inserting a newline or full stop in situations where the content was likely not to continue on then that would be considered an added bonus. e.g: items in an ul or option tag could be separated by full stops (or to be honest just ignored). I am working Java, but would be interested in seeing any code that does this. I can (and will if required) come up with something to do this, just wondered if there was anything out there like this already, as it would probably be better than what I come up with in an afternoon ;-). An example of the code I might write if I do end up doing this would be to use a SAX parser to find content in p tags, strip it of any span or strong etc tags, and add a full stop if I hit a div or another p without having had a fullstop. Any pointers or suggestions very welcome.

    Read the article

  • utf-8 convertion doesn't work always

    - by Marco Piccinni
    I searched into other stack before to type here and I didn't find anythong similar. I have to scrape different utf-8 webpages which contain text like "Oggi è una bellissima giornata" the problem is on the characther "è" I extract this text with jtidy and xpath query expression and I convert it with byte[] content = filteredEncodedString.getBytes("utf-8"); String result = new String(content,"utf-8"); where filteredEncodedString contains the text "Oggi è una bellissima giornata". This procedures works on the most webpages analyzed so far but in some case it doesn't extract a utf-8 string. Page encoding is always the same as the text is similar. Any ideas about the problem? thanks Marco

    Read the article

  • Split screen in iPad

    - by AAT
    I working on a iPAD only app which requires me to split the screen such a way that: (1) there are 2 parts on the screen divided vertically (2) On the left side, user can communicate using a chat (3) On the right side, user can see continuous streaming data I am not sure (A) how can I do two tasks simultaneously (B)how to split the screen (is Split view the way to achieve both of these?) Thank you.

    Read the article

  • is it possible to use a python scrapper in a website?

    - by Tom
    I want to scrap a website and use that content in a website of my own. I am just wondering if that can be done with python 2.7, and if so how? If not, do I have to use JavaScript to scrap it? And do you have a good place to learn how to do that or good libraries for it. For those of you wondering, the website I am scrapping is legal, and they allow for this to be done. I have searched all over but apparently nobody tries to implement these scrappers that they write. I can write a web scrapper in python just fine. Say my scrapper scraps a name from a wikipedia page (John Doe for example), how can I use that name that I get in my website? Another update, I have found pjsrape and PhantomJS. I have only found one stack overflow post and the github examples with aren't very intuitive. If anybody has any experience or better ways to do it I would very much appreciate it

    Read the article

< Previous Page | 65 66 67 68 69 70 71 72 73 74 75 76  | Next Page >