Search Results

Search found 5638 results on 226 pages for 'scheduling algorithm'.

Page 89/226 | < Previous Page | 85 86 87 88 89 90 91 92 93 94 95 96  | Next Page >

• algorithm q: Fuzzy matching of structured data

  - by user86432

  I have a fairly small corpus of structured records sitting in a database. Given a tiny fraction of the information contained in a single record, submitted via a web form (so structured in the same way as the table schema), (let us call it the test record) I need to quickly draw up a list of the records that are the most likely matches for the test record, as well as provide a confidence estimate of how closely the search terms match a record. The primary purpose of this search is to discover whether someone is attempting to input a record that is duplicate to one in the corpus. There is a reasonable chance that the test record will be a dupe, and a reasonable chance the test record will not be a dupe. The records are about 12000 bytes wide and the total count of records is about 150,000. There are 110 columns in the table schema and 95% of searches will be on the top 5% most commonly searched columns. The data is stuff like names, addresses, telephone numbers, and other industry specific numbers. In both the corpus and the test record it is entered by hand and is semistructured within an individual field. You might at first blush say "weight the columns by hand and match word tokens within them", but it's not so easy. I thought so too: if I get a telephone number I thought that would indicate a perfect match. The problem is that there isn't a single field in the form whose token frequency does not vary by orders of magnitude. A telephone number might appear 100 times in the corpus or 1 time in the corpus. The same goes for any other field. This makes weighting at the field level impractical. I need a more fine-grained approach to get decent matching. My initial plan was to create a hash of hashes, top level being the fieldname. Then I would select all of the information from the corpus for a given field, attempt to clean up the data contained in it, and tokenize the sanitized data, hashing the tokens at the second level, with the tokens as keys and frequency as value. I would use the frequency count as a weight: the higher the frequency of a token in the reference corpus, the less weight I attach to that token if it is found in the test record. My first question is for the statisticians in the room: how would I use the frequency as a weight? Is there a precise mathematical relationship between n, the number of records, f(t), the frequency with which a token t appeared in the corpus, the probability o that a record is an original and not a duplicate, and the probability p that the test record is really a record x given the test and x contain the same t in the same field? How about the relationship for multiple token matches across multiple fields? Since I sincerely doubt that there is, is there anything that gets me close but is better than a completely arbitrary hack full of magic factors? Barring that, has anyone got a way to do this? I'm especially keen on other suggestions that do not involve maintaining another table in the database, such as a token frequency lookup table :). This is my first post on StackOverflow, thanks in advance for any replies you may see fit to give.

  Read the article

• Is there a faster way to parse through a large file with regex quickly?

  - by Ray Eatmon

  Problem: Very very, large file I need to parse line by line to get 3 values from each line. Everything works but it takes a long time to parse through the whole file. Is it possible to do this within seconds? Typical time its taking is between 1 minute and 2 minutes. Example file size is 148,208KB I am using regex to parse through every line: Here is my c# code: private static void ReadTheLines(int max, Responder rp, string inputFile) { List<int> rate = new List<int>(); double counter = 1; try { using (var sr = new StreamReader(inputFile, Encoding.UTF8, true, 1024)) { string line; Console.WriteLine("Reading...."); while ((line = sr.ReadLine()) != null) { if (counter <= max) { counter++; rate = rp.GetRateLine(line); } else if(max == 0) { counter++; rate = rp.GetRateLine(line); } } rp.GetRate(rate); Console.ReadLine(); } } catch (Exception e) { Console.WriteLine("The file could not be read:"); Console.WriteLine(e.Message); } } Here is my regex: public List<int> GetRateLine(string justALine) { const string reg = @"^\d{1,}.+\[(.*)\s[\-]\d{1,}].+GET.*HTTP.*\d{3}[\s](\d{1,})[\s](\d{1,})$"; Match match = Regex.Match(justALine, reg, RegexOptions.IgnoreCase); // Here we check the Match instance. if (match.Success) { // Finally, we get the Group value and display it. string theRate = match.Groups[3].Value; Ratestorage.Add(Convert.ToInt32(theRate)); } else { Ratestorage.Add(0); } return Ratestorage; } Here is an example line to parse, usually around 200,000 lines: 10.10.10.10 - - [27/Nov/2002:16:46:20 -0500] "GET /solr/ HTTP/1.1" 200 4926 789

  Read the article

• R: Forecast package: Automatic algorithm for composite model involving ETS and AR

  - by phanikishan

  Hey, I would like to write a code involving automatic selection of a best composite model using ETS as well as autoregressive models. What is the criteria I should base my selection on? Also if I'm using the auto.arima function for deducing number of AR terms and corresponding coefficients from the forecast package in R, does my input series necessarily have to be stationary? or the value for d would be automatically selected thus returning a non-stationary model? Thanks, Phani

  Read the article

• What is it about Fibonacci numbers?

  - by Ian Bishop

  Fibonacci numbers have become a popular introduction to recursion for Computer Science students and there's a strong argument that they persist within nature. For these reasons, many of us are familiar with them. They also exist within Computer Science elsewhere too; in surprisingly efficient data structures and algorithms based upon the sequence. There are two main examples that come to mind: Fibonacci heaps which have better amortized running time than binomial heaps. Fibonacci search which shares O(log N) running time with binary search on an ordered array. Is there some special property of these numbers that gives them an advantage over other numerical sequences? Is it a density quality? What other possible applications could they have? It seems strange to me as there are many natural number sequences that occur in other recursive problems, but I've never seen a Catalan heap.

  Read the article

• What's the "Hello World!" of genetic algorithms good for?

  - by JohnIdol

  I found this very cool C++ sample , literally the "Hello World!" of genetic algorithms. I so decided to re-code the whole thing in C# and this is the result. Now I am asking myself: is there any practical application along the lines of generating a target string starting from a population of random strings? EDIT: my buddy on twitter just tweeted that "is useful for transcription type things such as translation. Does not have to be Monkey's". I wish I had a clue.

  Read the article

• How does Amazon's Statistically Improbable Phrases work?

  - by ??iu

  How does something like Statistically Improbable Phrases work? According to amazon: Amazon.com's Statistically Improbable Phrases, or "SIPs", are the most distinctive phrases in the text of books in the Search Inside!™ program. To identify SIPs, our computers scan the text of all books in the Search Inside! program. If they find a phrase that occurs a large number of times in a particular book relative to all Search Inside! books, that phrase is a SIP in that book. SIPs are not necessarily improbable within a particular book, but they are improbable relative to all books in Search Inside!. For example, most SIPs for a book on taxes are tax related. But because we display SIPs in order of their improbability score, the first SIPs will be on tax topics that this book mentions more often than other tax books. For works of fiction, SIPs tend to be distinctive word combinations that often hint at important plot elements. For instance, for Joel's first book, the SIPs are: leaky abstractions, antialiased text, own dog food, bug count, daily builds, bug database, software schedules One interesting complication is that these are phrases of either 2 or 3 words. This makes things a little more interesting because these phrases can overlap with or contain each other.

  Read the article

• Fast way to manually mod a number

  - by Nikolai Mushegian

  I need to be able to calculate (a^b) % c for very large values of a and b (which individually are pushing limit and which cause overflow errors when you try to calculate a^b). For small enough numbers, using the identity (a^b)%c = (a%c)^b%c works, but if c is too large this doesn't really help. I wrote a loop to do the mod operation manually, one a at a time: private static long no_Overflow_Mod(ulong num_base, ulong num_exponent, ulong mod) { long answer = 1; for (int x = 0; x < num_exponent; x++) { answer = (answer * num_base) % mod; } return answer; } but this takes a very long time. Is there any simple and fast way to do this operation without actually having to take a to the power of b AND without using time-consuming loops? If all else fails, I can make a bool array to represent a huge data type and figure out how to do this with bitwise operators, but there has to be a better way.

  Read the article

• Is there a name for the technique of using base-2 numbers to encode a list of unique options?

  - by Lunatik

  Apologies for the rather vague nature of this question, I've never been taught programming and Google is rather useless to a self-help guy like me in this case as the key words are pretty ambiguous. I am writing a couple of functions that encode and decode a list of options into a Long so they can easily be passed around the application, you know this kind of thing: 1 - Apple 2 - Orange 4 - Banana 8 - Plum etc. In this case the number 11 would represent Apple, Orange & Plum. I've got it working but I see this used all the time so assume there is a common name for the technique, and no doubt all sorts of best practice and clever algorithms that are at the moment just out of my reach.

  Read the article

• Constructing colours for maximum contrast

  - by Martin

  I want to draw some items on screen, each item is in one of N sets. The number of sets changes all the time, so I need to calculate N different colours which are as different as possible (to make it easy to identify what is in which set). So, for example with N = 2 my results would be black and white. With three I guess I would get all red, all green, all blue. For all four, it's less obvious what the correct answer is, and this is where I'm having trouble.

  Read the article

• Pointer-based binary heap implementation

  - by Derek Chiang

  Is it even possible to implement a binary heap using pointers rather than an array? I have searched around the internet (including SO) and no answer can be found. The main problem here is that, how do you keep track of the last pointer? When you insert X into the heap, you place X at the last pointer and then bubble it up. Now, where does the last pointer point to? And also, what happens when you want to remove the root? You exchange the root with the last element, and then bubble the new root down. Now, how do you know what's the new "last element" that you need when you remove root again?

  Read the article

• google maps : given a point, how to find all points at a given road distance

  - by amarsh-anand

  In my app, the GPS picks the location of the vehicle. It is then supposed to put markers at all points where the vehicle could be if it drives for 1 KM in any direction (note that the roads may fork many times within his 1KM reach). Can someone suggest me how to do this. Thanks in advance.

  Read the article

• Revisions: algorithm and data structure

  - by SODA

  Hi, I need ideas for structuring and processing data with revisions. For example, I have a database of objects (e.g. cars). Each object has a number of properties, which can be arbitrary, so there's no a set schema to describe these objects. These objects are probably saved as key-value pairs. Now I need to change property of an object. I don't want to completely rewrite it - I want to be able to go back and see history of changes to these properties, that's why I want to add new property and keep the old one (so I guess a timestamp would do the job of telling which property is the latest). At the same time I want to be able to get info about any object in a snap, with only latest versions of each of the properties. Any ideas what would be the best approach? At least please point me in the right direction. Thanks!

  Read the article

• Does anyone know how to do an "inverse" trilinear interpolation?

  - by Kaladze

  Trilinear interpolation approximates the value of a point (x, y, z) inside a cube using the values at the cube vertices. I´m trying to do an "inverse" trilinear interpolation. Knowing the values at the cube vertices and the value attached to a point how can I find (x, y, z)? Any help would be highly appreciated. Thank you!

  Read the article

• How to find largest common sub-tree in the given two binary search trees?

  - by Bhushan

  Two BSTs (Binary Search Trees) are given. How to find largest common sub-tree in the given two binary trees? EDIT 1: Here is what I have thought: Let, r1 = current node of 1st tree r2 = current node of 2nd tree There are some of the cases I think we need to consider: Case 1 : r1.data < r2.data 2 subproblems to solve: first, check r1 and r2.left second, check r1.right and r2 Case 2 : r1.data > r2.data 2 subproblems to solve: - first, check r1.left and r2 - second, check r1 and r2.right Case 3 : r1.data == r2.data Again, 2 cases to consider here: (a) current node is part of largest common BST compute common subtree size rooted at r1 and r2 (b)current node is NOT part of largest common BST 2 subproblems to solve: first, solve r1.left and r2.left second, solve r1.right and r2.right I can think of the cases we need to check, but I am not able to code it, as of now. And it is NOT a homework problem. Does it look like?

  Read the article

• Using Ruby, what is the most efficient way to check if any key in a hash matches any values within a

  - by Jason

  I want to compare the keys in a hash of parameters against an array of elements for a match. For example: params = {"key1", "key2", "key3"} params_to_match = ["key2","key3"] I could do this, but I'm sure there is a much more elegant way to acheive the same result params.each_key{|key| if params_to_match.include?(key.to_s) return end }

  Read the article

• How to sort a list so that managers are always ahead of their subordinates (How do I do a topologica

  - by James Black

  I am working on a project using Groovy, and I would like to take an array of employees, so that no manager follows their subordinates in the array. The reason being that I need to add people to a database and I would prefer not to do it in two passes. So, I basically have: <employees> <employee> <employeeid>12</employeeid> <manager>3</manager> </employee> <employee> <employeeid>1</employeeid> <manager></manager> </employee> <employee> <employeeid>3</employeeid> <manager>1</manager> </employee> </employees> So, it should be sorted as such: employeeid = 1 employeeid = 3 employeeid = 12 The first person should have a null for managers. I am thinking about a binary tree representation, but I expect it will be very unbalanced, and I am not certain the best way to do this using Groovy properly. Is there a way to do this that isn't going to involve using nested loops?

  Read the article

• Programming Contest Question: Counting Polyominos

  - by Martijn Courteaux

  Hi, An example question for a programming contest was to write a program that finds out how much polyominos are possible with a given number of stones. So for two stones (n = 2) there is only one polyominos: XX You might think this is a second solution: X X But it isn't. The polyominos are not unique if you can rotate them. So, for 4 stones (n = 4), there are 7 solutions: X X XX X X X X X X XX X XX XX XX X X X XX X X XX The application has to be able to find the solution for 1 <= n <=10 PS: Using the list of polyominos on Wikipedia isn't allowed ;) EDIT: Of course the question is: How to do this in Java, C/C++, C#

  Read the article

• Is there a good radixsort-implementation for floats in C#

  - by CommuSoft

  I have a datastructure with a field of the float-type. A collection of these structures needs to be sorted by the value of the float. Is there a radix-sort implementation for this. If there isn't, is there a fast way to access the exponent, the sign and the mantissa. Because if you sort the floats first on mantissa, exponent, and on exponent the last time. You sort floats in O(n).

  Read the article

• Algorithm for sentence analysis and tokenization

  - by Andrea Nagar

  I need to analyze a document and compile statistics as to how many times each a sequence of words is used (so the analysis is not on single words but of batch of recurring words). I read that compression algorithms do something similar to what I want - creating dictionaries of blocks of text with a piece of information reporting its frequency. It should be something similar to http://www.codeproject.com/KB/recipes/Patterns.aspx Do you have anything written in C#?

  Read the article

• Latex: vertical line in lstlistings

  - by Helltone

  I want to have a vertical line for indentation in the lstlisting environment, similar to what one can get in algorithm2e. I tried doing something like the code below, but the the |'s are not contiguous and the result is ugly. \lstset{ ... showtabs=true, tabsize=3, tab=\hfill$|$\hfill, ... }

  Read the article

• Aligning music notes using String matching algorithms or Dynamic Programming

  - by Dolphin

  Hi I need to compare 2 sets of musical pieces (i.e. a playing-taken in MIDI format-note details extracted and saved in a database table, against sheet music-taken into XML format). When evaluating playing against sheet music (i.e.note details-pitch, duration, rhythm), note alignment needs to be done - to identify missed/extra/incorrect/swapped notes that from the reference (sheet music) notes. I have like 1800-2500 notes in one piece approx (can even be more-with polyphonic, right now I'm doing for monophonic). So will I have to have all these into an array? Will it be memory overloading or stack overflow? There are string matching algorithms like KMP, Boyce-Moore. But note alignment can also be done through Dynamic Programming. How can I use Dynamic Programming to approach this? What are the available algorithms? Is it about approximate string matching? Which approach is much productive? String matching algos like Boyce-Moore, or dynamic programming? How can I assess which is more effective? Greatly appreciate any insight or suggestions Thanks in advance

  Read the article

• Count subset of binary pattern ..

  - by mr.bio

  Hi there . I have a A=set of strings and a B=seperate string. I want to count the number of occurences in from B in A. Example : A: 10001 10011 11000 10010 10101 B: 10001 result would be 3.(10001 is a subset of 10001,10011,10101) So i need a function that takes a set and string and returns an int. int myfunc(set<string> , string){ int result; // My Brain is melting return result ; }

  Read the article

• Need help understanding this MATLAB function implementation

  - by Runner

  http://www.cc.gatech.edu/classes/AY2000/cs7495_fall/participants/sashag/ps3/getchaincode.m offset = N/2; %offset between ajacent pixels I don't understand what the above mean?

  Read the article

• How can I generate an "unlimited" world?

  - by snowlord

  I would like to create a game with an endless (in reality an extremely large) world in which the player can move about. Whether or not I will ever get around to implement the game is one matter, but I find the idea interesting and would like some input on how to do it. The point is to have a world where all data is generated randomly on-demand, but in a deterministic way. Currently I focus on a large 2D map from which it should be possible to display any part without knowledge about the surrounding parts. I have implemented a prototype by writing a function that gives a random-looking, but deterministic, integer given the x and y of a pixel on the map (see my recent question about this function). Using this function I populate the map with "random" values, and then I smooth the map using a simple filter based on the surrounding pixels. This makes the map dependent on a few pixels outside its edge, but that's not a big problem. The final result is something that at least looks like a map (especially with a good altitude color map). Given this, one could maybe first generate a coarser map which is used to generate bigger differences in altitude to create mountain ranges and seas. Anyway, that was my idea, but I am sure that there exist ways to do this already and I also believe that given the specification, many of you can come up with better ideas. EDIT: Forgot the link to my question.

  Read the article

• Load balancing and shedulling algorithms .NET

  - by Lukas Šalkauskas

  Hello there, so here is my problem: I have several different configuarion servers. I have different calculations (jobs), I can predict how long approx. each job will take to be caclulated. Also I have priorities. My question is how to keep all machines loaded 99-100% and shedule the jobs in the best way. Each machine can do several calculations at the time. Jobs are pushed to the machine. Central machine knows current load of each machine. Also I would like to to assign some king of machine learning here, because I will know statistics of each job (started, finished, cpu load etc.). How to distribute jobs(calculations) in the best possible way, also keep in mind priority. Any suggestions ? Ideas ? Algorithms ?

  Read the article

< Previous Page | 85 86 87 88 89 90 91 92 93 94 95 96  | Next Page >