Search Results

Search found 45804 results on 1833 pages for 'large files'.

Page 101/1833 | < Previous Page | 97 98 99 100 101 102 103 104 105 106 107 108  | Next Page >

  • Using a large list of terms, search through page text and replace words with links

    - by dunc
    A while ago I posted this question asking if it's possible to convert text to HTML links if they match a list of terms from my database. I have a fairly huge list of terms - around 6000. The accepted answer on that question was superb, but having never used XPath, I was at a loss when problems started occurring. At one point, after fiddling with code, I somehow managed to add over 40,000 random characters to our database - the majority of which required manual removal. Since then I've lost faith in that idea and the more simple PHP solutions simply weren't efficient enough to deal with the amount of data and the quantity of terms. My next attempt at a solution is to write a JS script which, once the page has loaded, retrieves the terms and matches them against the text on a page. This answer has an idea which I'd like to attempt. I would use AJAX to retrieve the terms from the database, to build an object such as this: var words = [ { word: 'Something', link: 'http://www.something.com' }, { word: 'Something Else', link: 'http://www.something.com/else' } ]; When the object has been built, I'd use this kind of code: //for each array element $.each(words, function() { //store it ("this" is gonna become the dom element in the next function) var search = this; $('.message').each( function() { //if it's exactly the same if ($(this).text() === search.word) { //do your magic tricks $(this).html('<a href="' + search.link + '">' + search.link + '</a>'); } } ); } ); Now, at first sight, there is a major issue here: with 6,000 terms, will this code be in any way efficient enough to do what I'm trying to do?. One option would possibly be to perform some of the overhead within the PHP script that the AJAX communicates with. For instance, I could send the ID of the post and then the PHP script could use SQL statements to retrieve all of the information from the post and match it against all 6,000 terms.. then the return call to the JavaScript could simply be the matching terms, which would significantly reduce the number of matches the above jQuery would make (around 50 at most). I have no problem with the script taking a few seconds to "load" on the user's browser, as long as it isn't impacting their CPU usage or anything like that. So, two questions in one: Can I make this work? What steps can I take to make it as efficient as possible? Thanks in advance,

    Read the article

  • Looking for nice Javascript/jQuery code for displaying large tables

    - by misha-moroshko
    I have an HTML table which may contain thousands of rows (number of columns is not a problem here). I would like to be able to browse this table easily and be able to do the following: Decide how many rows will be presented Jump to the next/previous X number of rows Scroll the table using the scroll bars to any desired line Be able to customize/extend easily this Javascript/jQuery code Has anyone seen something similar ? Thank you very much !

    Read the article

  • how to handle large dataset like sproutcore

    - by Nik
    Hello all, I really don't have any substantial code to show here, actually, that's kinda why I am writing: I looked at the SproutCore demo, especially the Collection demo, on http://demo.sproutcore.com/sample_controls/, and am amazed by its loading 200,000 records to the page so easily. I tried using Rails to provide 200,000 records and in a completely blank HTML page with <% @projects.each do |p| % <%= p.title % <% end % that freezes the browser for seconds on my m1530 laptop with 4gb ram and t7700 256gb ssd. Yet the sproutcore demo does not freeze and takes less than 3 seconds to load. What do you think the one technique they are using to enable this is? Thanks!

    Read the article

  • Locking DB w/ Large Reads (Ruby-on-Rails/Heroku)

    - by Splashlin
    Currently I have a Web API running on Heroku that is constantly writing information we're collecting from other data sources (currently theres about half a GB of data and it's growing very quickly). We're looking to add a reporting system on top of the current database that we can use to extract useful information out of the DB. The problem is that when we're running reports we're locking the DB and any other sites communicating with the DB are timing out. Does anyone have any solutions on how to solve this type of issue? Amazon RDS seems to have some interesting stuff with database replication but I don't know if that will solve my problems. Any advice would be greatly appreciated. Thanks

    Read the article

  • Need advice on cron job'ing a very large process

    - by Arms
    I have a PHP script that grabs data from an external service and saves data to my database. I need this script to run once every minute for every user in the system (of which I expect to be thousands). My question is, what's the most efficient way to run this per user, per minute? At first I thought I would have a function that grabs all the user Ids from my database, iterate over the ids and perform the task for each one, but I think that as the number of users grow, this will take longer, and no longer fall within 1 minute intervals. Perhaps I should queue the user Ids, and perform the task individually for each one? In which case, I'm actually unsure of how to proceed. Thanks in advance for any advice.

    Read the article

  • Reducing lag when downloading large amount of data from webpage

    - by Mahir
    I am getting data via RSS feeds and displaying each article in a table view cell. Each cell has an image view, set to a default image. If the page has an image, the image is to be replaced with the image from the article. As of now, each cell downloads the source code from the web page, causing the app to lag when I push the view controller and when I try scrolling. Here is what I have in the cellForRowAtIndexPath: method. NSString * storyLink = [[stories objectAtIndex: storyIndex] objectForKey: @"link"]; storyLink = [storyLink stringByTrimmingCharactersInSet:[NSCharacterSet whitespaceAndNewlineCharacterSet]]; NSString *sourceCode = [NSString stringWithContentsOfURL:[NSURL URLWithString:storyLink] encoding:NSUTF8StringEncoding error:&error]; NSString *startPt = @"instant-gallery"; NSString *startPt2 = @"<img src=\""; if ([sourceCode rangeOfString:startPt].length != 0) { //webpage has images // find the first "<img src=...>" tag starting from "instant-gallery" NSString *trimmedSource = [sourceCode substringFromIndex:NSMaxRange([sourceCode rangeOfString:startPt])]; trimmedSource = [trimmedSource substringFromIndex:NSMaxRange([trimmedSource rangeOfString:startPt2])]; trimmedSource = [trimmedSource substringToIndex:[trimmedSource rangeOfString:@"\""].location]; NSURL *url = [NSURL URLWithString:trimmedSource]; NSData *data = [NSData dataWithContentsOfURL:url]; UIImage *image = [UIImage imageWithData:data]; cell.picture.image = image; Someone suggested using NSOperationQueue. Would this way be a good solution? EDIT: - (UITableViewCell *)tableView:(UITableView *)tableView cellForRowAtIndexPath:(NSIndexPath *)indexPath { static NSString *MyIdentifier = @"FeedCell"; LMU_LAL_FeedCell *cell = [tableView dequeueReusableCellWithIdentifier:MyIdentifier]; if (cell == nil) { NSArray *nib = [[NSBundle mainBundle] loadNibNamed:@"LMU_LAL_FeedCell" owner:self options:nil]; cell = (LMU_LAL_FeedCell*) [nib objectAtIndex:0]; } int storyIndex = [indexPath indexAtPosition: [indexPath length] - 1]; NSString *untrimmedTitle = [[stories objectAtIndex: storyIndex] objectForKey: @"title"]; cell.title.text = [untrimmedTitle stringByTrimmingCharactersInSet:[NSCharacterSet whitespaceCharacterSet]]; CGSize maximumLabelSize = CGSizeMake(205,9999); CGSize expectedLabelSize = [cell.title.text sizeWithFont:cell.title.font constrainedToSize:maximumLabelSize]; //adjust the label to the the new height. CGRect newFrame = cell.title.frame; newFrame.size.height = expectedLabelSize.height; cell.title.frame = newFrame; //position frame of date label CGRect dateNewFrame = cell.date.frame; dateNewFrame.origin.y = cell.title.frame.origin.y + cell.title.frame.size.height + 1; cell.date.frame = dateNewFrame; cell.date.text = [self formatDateAtIndex:storyIndex]; dispatch_queue_t someQueue = dispatch_queue_create("cell background queue", NULL); dispatch_async(someQueue, ^(void){ NSError *error = nil; NSString * storyLink = [[stories objectAtIndex: storyIndex] objectForKey: @"link"]; storyLink = [storyLink stringByTrimmingCharactersInSet:[NSCharacterSet whitespaceAndNewlineCharacterSet]]; NSString *sourceCode = [NSString stringWithContentsOfURL:[NSURL URLWithString:storyLink] encoding:NSUTF8StringEncoding error:&error]; NSString *startPt = @"instant-gallery"; NSString *startPt2 = @"<img src=\""; if ([sourceCode rangeOfString:startPt].length != 0) { //webpage has images // find the first "<img src=...>" tag starting from "instant-gallery" NSString *trimmedSource = [sourceCode substringFromIndex:NSMaxRange([sourceCode rangeOfString:startPt])]; trimmedSource = [trimmedSource substringFromIndex:NSMaxRange([trimmedSource rangeOfString:startPt2])]; trimmedSource = [trimmedSource substringToIndex:[trimmedSource rangeOfString:@"\""].location]; NSURL *url = [NSURL URLWithString:trimmedSource]; NSData *data = [NSData dataWithContentsOfURL:url]; UIImage *image = [UIImage imageWithData:data]; dispatch_async(dispatch_get_main_queue(), ^(void){ cell.picture.image = image; }); }) //error: expected expression } return cell; //error: expected identifier } //error extraneous closing brace

    Read the article

  • How to determine on which file system a file was created in Java

    - by rafrafUk
    Hi Everyone! I get files in different formats coming from different systems that I need to import into our database. Part of the import process it to check the line length to make sure the format is correct. We seem to be having issues with files coming from UNIX systems where one character is added. I suspect this is due to the return carriage being encoded differently on UNIX and windows platform. Is there a way to detect on which file system a file was created, other than checking the last character on the line? Or maybe a way of reading the files as text and not binary which I suspect is the issue? Thanks Guys !

    Read the article

  • Java combine parents of two large inheritance chains

    - by Soylent Green
    I have two parent classes in a huge project, let's say ClassA and ClassB. Each class has many subclasses, which in turn have many subclasses, which in turn have many subclasses, etc. My task is to "marry" these two "families" so that both inherit from a SINGLE parent. I need to essentially make ClassA and ClassB one class (parent) to both of their combined subclasses (children). ClassA and ClassB both currently implement Serializable. I am currently trying to make both inheritance chains inherit from ClassA, and then copy all functions and data members from ClassB into ClassA. This is tedious, and I think a terrible solution. What would be the CORRECT way to solve this problem?

    Read the article

  • Ideal way/architecture to deliver large data over Web Services

    - by zengr
    We are trying to design 6 web services, which will serve another client component. The client component requires data from the web service we are implementing. Now, the problem is, there is not 1 WS we are implementing, there is one WS which the client component hits, this initiates a series (5 more) of WSs which gather data from their respective data stores and finally provide the data back to the original WS, which then delivers the data back to the client component. So, if the requested data becomes huge, then, this will be a serious problem for our internal communication channel. So, what do you guys suggest? What can be done to avoid overloading of the communication channel between the internal WS and at the same time, also delivering the data to the client component.

    Read the article

  • Read large amount of data from file in Java

    - by Crozin
    Hello I've got text file that contains 1 000 002 numbers in following formation: 123 456 1 2 3 4 5 6 .... 999999 100000 Now I need to read that data and allocate it to int variables (the very first two numbers) and all the rest (1 000 000 numbers) to an array int[]. It's not a hard task, but - it's horrible slow. My first attempt was java.util.Scanner: Scanner stdin = new Scanner(new File("./path")); int n = stdin.nextInt(); int t = stdin.nextInt(); int array[] = new array[n]; for (int i = 0; i < n; i++) { array[i] = stdin.nextInt(); } It works as excepted but it takes about 7500 ms to execute. I need to fetch that data in up to several hundred of milliseconds. Then I tried java.io.BufferedReader: Using BufferedReader.readLine() and String.split() I got the same results in about 1700 ms, but it's still too many. How can I read that amount of data in less that 1 second? The final result should be equal to: int n = 123; int t = 456; int array[] = { 1, 2, 3, 4, ..., 999999, 100000 };

    Read the article

  • Large amount of constants in Java

    - by Lars D
    I need to include about 1 MByte of data in a Java application, for very fast and easy access in the rest of the source code. My main background is not Java, so my initial idea was to convert the data directly to Java source code, defining 1MByte of constant arrays, classes (instead of C++ struct) etc., something like this: public final/immutable/const MyClass MyList[] = { { 23012, 22, "Hamburger"} , { 28375, 123, "Kieler"} }; However, it seems that Java does not support such constructs. Is this correct? If yes, what is the best solution to this problem?

    Read the article

  • Getting the Item Count of a large sharepoint list in fastest way

    - by sooraj
    I am trying to get the count of the items in a sharepoint document library programatically. The scale I am working with is 30-70000 items. We have usercontrol in a smartpart to display the count . Ours is a TEAM site. This is the code to get the total count SPList VoulnterrList = web.Lists[ListTitle]; SPQuery query = new SPQuery(); query.ViewAttributes = "Scope=\"Recursive\""; string queries = "<Where><Eq><FieldRef Name='ApprovalStatus' /><Value Type='Choice'>Pending</Value></Eq></Where>"; query.Query = queries; SPListItemCollection lstitemcollAssoID = VoulnterrList.GetItems(query); lblCount.Text = "Total Proofs: " + VoulnterrList.Items.Count.ToString() + " Pending Proofs: " + lstitemcollAssoID.Count.ToString(); The problem is this has serious performance issue it takes 75 to 80 sec to load the page. if we comment this page load will decrees to 4 sec. Any better approch for this problem Ours is sharepoint 2007

    Read the article

  • Selecting from a Large Table SQL 2005

    - by Eugene
    I have a SQL table it has more than 1000000 rows, and I need to select with the query as you can see below: SELECT DISTINCT TOP (200) COUNT(1) AS COUNT, KEYWORD FROM QUERIES WITH(NOLOCK) WHERE KEYWORD LIKE '%Something%' GROUP BY KEYWORD ORDER BY 'COUNT' DESC Could you please tell me how can I optimize it to speed up the execution process? Thank you for useful answers.

    Read the article

  • Make backup of large site with 100,000+ files/images

    - by niggles
    I tried backing up our site today using the Unix 'cp' command and ended up getting our office blocked out by PLESK - it added my ip to /etc/hosts.deny as it thought I was flooding the server. After Tech support fixed the issue, they suggested I go folder by folder to back it up, but there's about 10,000 folders on the site totaling 1/2 a terabyte, each with multiple sub-folders, so this isn't viable. Basically I want to be able to mirror the domain on another domain we've got set up on the same dedicated server so I can test with live images (the bulk of our content). Any suggestions e.g adding some rules to open_base_dir and getting PHP to recursively copy the folders to the other domain (remember it's on the same dedicated box so it just needs to traverse the directory, not FTP things).

    Read the article

  • Linking to a Large address aware DLL.

    - by Canopus
    Suppose I have a DLL which is built with LARGEADDRESSAWARE linker flag set. Now I have an application dynamically linking to this DLL. Does this make my application LARGEADDRESSAWARE? If not then, does it make sense to have this flag set for any DLL?

    Read the article

  • Breaking up large DataGridView for printing

    - by Hal
    Hey, I've got a single-row, 40 column-long DataGridView that i need to print. Since i can neither print it directly (because A4 sheets won't cut it ;)) nor adjust its width to the width of the page itself (because the headers look terrible), i wanted to break the DataGridView to 4 separate pieces and display 10 columns per row (imagine: column 1 to 10 in the first line, column 11 to 21 four or five lines below, etc...). Is there an easy way to do this? I was leaning towards a more manual approach (using fors), but i'd love to know if there's a more elegant way. Cheers

    Read the article

  • Downloading Large JSON File to local file using Java

    - by user1279675
    I'm attempting to download a JSON from the following URL - http://api.crunchbase.com/v/1/companies.js - to a local file. I'm using Java 1.7 and the following JSON Libraries - http://www.json.org/java/ - to attempt to make it work. Here's my code: public static void download(String address, String localFileName) { OutputStream out = null; URLConnection conn = null; InputStream in = null; try { URL url = new URL(address); out = new BufferedOutputStream( new FileOutputStream(localFileName)); conn = url.openConnection(); in = conn.getInputStream(); byte[] buffer = new byte[1024]; int numRead; long numWritten = 0; while ((numRead = in.read(buffer)) != -1) { out.write(buffer, 0, numRead); numWritten += numRead; System.out.println(buffer.length); System.out.println(" " + buffer.hashCode()); } System.out.println(localFileName + "\t" + numWritten); } catch (Exception exception) { exception.printStackTrace(); } finally { try { if (in != null) { in.close(); } if (out != null) { out.close(); } } catch (IOException ioe) { } } } When I run the code everything seems to work until midway through the loop the program seems to stop and not continue reading the JSON Object. Does anyone know why this would stop reading? How could I fix the issue?

    Read the article

  • Calculating average (AVG) and grouping by week on large data set takes too long

    - by caioiglesias
    I'm getting average prices by week on 7 million rows, it's taking around 30 seconds to get the job done. This is the simple query: SELECT AVG(price) as price, yearWEEK(FROM_UNIXTIME(timelog)) as week from pricehistory where timelog > $range and product_id = $id GROUP BY week The only week that actually gets data changed and is worth averaging every time is always the last one, so this calculation for the whole period is a waste of resources. I just wanted to know if mysql has a tool to help out on this.

    Read the article

  • Declaring two large 2d arrays gives segmentation fault.

    - by pfdevil
    Hello, i'm trying to declare and allocate memory for two 2d-arrays. However when trying to assign values to itemFeatureQ[39][16816] I get a segmentation vault. I can't understand it since I have 2GB of RAM and only using 19MB on the heap. Here is the code; double** reserveMemory(int rows, int columns) { double **array; int i; array = (double**) malloc(rows * sizeof(double *)); if(array == NULL) { fprintf(stderr, "out of memory\n"); return NULL; } for(i = 0; i < rows; i++) { array[i] = (double*) malloc(columns * sizeof(double *)); if(array == NULL) { fprintf(stderr, "out of memory\n"); return NULL; } } return array; } void populateUserFeatureP(double **userFeatureP) { int x,y; for(x = 0; x < CUSTOMERS; x++) { for(y = 0; y < FEATURES; y++) { userFeatureP[x][y] = 0; } } } void populateItemFeatureQ(double **itemFeatureQ) { int x,y; for(x = 0; x < FEATURES; x++) { for(y = 0; y < MOVIES; y++) { printf("(%d,%d)\n", x, y); itemFeatureQ[x][y] = 0; } } } int main(int argc, char *argv[]){ double **userFeatureP = reserveMemory(480189, 40); double **itemFeatureQ = reserveMemory(40, 17770); populateItemFeatureQ(itemFeatureQ); populateUserFeatureP(userFeatureP); return 0; }

    Read the article

  • Prevent RegEx Hang on Large Matches...

    - by developerjay
    This is a great regular expression for dates... However it hangs indefinitely on this one page I tried... I wanted to try this page ( http://pleac.sourceforge.net/pleac%5Fpython/datesandtimes.html ) for the fact that it does have lots of dates on it and I want to grab all of them. I don't understand why it is hanging when it doesn't on other pages... Why is my regexp hanging and/or how could I clean it up to make it better/efficient ? Python Code: monthnames = "(?:Jan\w*|Feb\w*|Mar\w*|Apr\w*|May|Jun\w?|Jul\w?|Aug\w*|Sep\w*|Oct\w*|Nov(?:ember)?|Dec\w*)" pattern1 = re.compile(r"(\d{1,4}[\/\\\-]+\d{1,2}[\/\\\-]+\d{2,4})") pattern4 = re.compile(r"(?:[\d]*[\,\.\ \-]+)*%s(?:[\,\.\ \-]+[\d]+[stndrh]*)+[:\d]*[\ ]?(PM)?(AM)?([\ \-\+\d]{4,7}|[UTCESTGMT\ ]{2,4})*"%monthnames, re.I) patterns = [pattern4, pattern1] for pattern in patterns: print re.findall(pattern, s) btw... when i say im trying it against this site.. I'm trying it against the webpage source.

    Read the article

  • hadoop - large database query

    - by Mastergeek
    Situation: I have a Postgres DB that contains a table with several million rows and I'm trying to query all of those rows for a MapReduce job. From the research I've done on DBInputFormat, Hadoop might try and use the same query again for a new mapper and since these queries take a considerable amount of time I'd like to prevent this in one of two ways that I've thought up: 1) Limit the job to only run 1 mapper that queries the whole table and call it good. or 2) Somehow incorporate an offset in the query so that if Hadoop does try to use a new mapper it won't grab the same stuff. I feel like option (1) seems more promising, but I don't know if such a configuration is possible. Option(2) sounds nice in theory but I have no idea how I would keep track of the mappers being made and if it is at all possible to detect that and reconfigure. Help is appreciated and I'm namely looking for a way to pull all of the DB table data and not have several of the same query running because that would be a waste of time.

    Read the article

  • Intersection() and Except() is too slow with large collections of custom objects

    - by Theo
    I am importing data from another database. My process is importing data from a remote DB into a List<DataModel> named remoteData and also importing data from the local DB into a List<DataModel> named localData. I am then using LINQ to create a list of records that are different so that I can update the local DB to match the data pulled from remote DB. Like this: var outdatedData = this.localData.Intersect(this.remoteData, new OutdatedDataComparer()).ToList(); I am then using LINQ to create a list of records that no longer exist in remoteData, but do exist in localData, so that I delete them from local database. Like this: var oldData = this.localData.Except(this.remoteData, new MatchingDataComparer()).ToList(); I am then using LINQ to do the opposite of the above to add the new data to the local database. Like this: var newData = this.remoteData.Except(this.localData, new MatchingDataComparer()).ToList(); Each collection imports about 70k records, and each of the 3 LINQ operation take between 5 - 10 minutes to complete. How can I make this faster? Here is the object the collections are using: internal class DataModel { public string Key1{ get; set; } public string Key2{ get; set; } public string Value1{ get; set; } public string Value2{ get; set; } public byte? Value3{ get; set; } } The comparer used to check for outdated records: class OutdatedDataComparer : IEqualityComparer<DataModel> { public bool Equals(DataModel x, DataModel y) { var e = string.Equals(x.Key1, y.Key1) && string.Equals(x.Key2, y.Key2) && ( !string.Equals(x.Value1, y.Value1) || !string.Equals(x.Value2, y.Value2) || x.Value3 != y.Value3 ); return e; } public int GetHashCode(DataModel obj) { return 0; } } The comparer used to find old and new records: internal class MatchingDataComparer : IEqualityComparer<DataModel> { public bool Equals(DataModel x, DataModel y) { return string.Equals(x.Key1, y.Key1) && string.Equals(x.Key2, y.Key2); } public int GetHashCode(DataModel obj) { return 0; } }

    Read the article

  • Permissions for Large Variables to Be Sent Via Stored Procedures (SQL Server)

    - by Joe Majewski
    I can't figure out a way to allow more than 4000 bytes to be received at once via a call to a stored procedure. I am storing images in the table that are around 15 - 20 kilobytes each, but upon getting them and displaying them to the page, they are always exactly 3.91 KB in size (or 4000 bytes). Do stored procedures have a limit on how much data can be sent at once? I double-checked my data, and I am indeed only receiving the first 4000 characters from the varbinary(MAX) field. Is there a permission setting to allow more than 4k bytes at once?

    Read the article

< Previous Page | 97 98 99 100 101 102 103 104 105 106 107 108  | Next Page >