Search Results

Search found 3828 results on 154 pages for 'mathematical optimization'.

Page 142/154 | < Previous Page | 138 139 140 141 142 143 144 145 146 147 148 149  | Next Page >

• algorithm q: Fuzzy matching of structured data

  - by user86432

  I have a fairly small corpus of structured records sitting in a database. Given a tiny fraction of the information contained in a single record, submitted via a web form (so structured in the same way as the table schema), (let us call it the test record) I need to quickly draw up a list of the records that are the most likely matches for the test record, as well as provide a confidence estimate of how closely the search terms match a record. The primary purpose of this search is to discover whether someone is attempting to input a record that is duplicate to one in the corpus. There is a reasonable chance that the test record will be a dupe, and a reasonable chance the test record will not be a dupe. The records are about 12000 bytes wide and the total count of records is about 150,000. There are 110 columns in the table schema and 95% of searches will be on the top 5% most commonly searched columns. The data is stuff like names, addresses, telephone numbers, and other industry specific numbers. In both the corpus and the test record it is entered by hand and is semistructured within an individual field. You might at first blush say "weight the columns by hand and match word tokens within them", but it's not so easy. I thought so too: if I get a telephone number I thought that would indicate a perfect match. The problem is that there isn't a single field in the form whose token frequency does not vary by orders of magnitude. A telephone number might appear 100 times in the corpus or 1 time in the corpus. The same goes for any other field. This makes weighting at the field level impractical. I need a more fine-grained approach to get decent matching. My initial plan was to create a hash of hashes, top level being the fieldname. Then I would select all of the information from the corpus for a given field, attempt to clean up the data contained in it, and tokenize the sanitized data, hashing the tokens at the second level, with the tokens as keys and frequency as value. I would use the frequency count as a weight: the higher the frequency of a token in the reference corpus, the less weight I attach to that token if it is found in the test record. My first question is for the statisticians in the room: how would I use the frequency as a weight? Is there a precise mathematical relationship between n, the number of records, f(t), the frequency with which a token t appeared in the corpus, the probability o that a record is an original and not a duplicate, and the probability p that the test record is really a record x given the test and x contain the same t in the same field? How about the relationship for multiple token matches across multiple fields? Since I sincerely doubt that there is, is there anything that gets me close but is better than a completely arbitrary hack full of magic factors? Barring that, has anyone got a way to do this? I'm especially keen on other suggestions that do not involve maintaining another table in the database, such as a token frequency lookup table :). This is my first post on StackOverflow, thanks in advance for any replies you may see fit to give.

  Read the article

• Is there a way to increase the efficiency of shared_ptr by storing the reference count inside the co

  - by BillyONeal

  Hello everyone :) This is becoming a common pattern in my code, for when I need to manage an object that needs to be noncopyable because either A. it is "heavy" or B. it is an operating system resource, such as a critical section: class Resource; class Implementation : public boost::noncopyable { friend class Resource; HANDLE someData; Implementation(HANDLE input) : someData(input) {}; void SomeMethodThatActsOnHandle() { //Do stuff }; public: ~Implementation() { FreeHandle(someData) }; }; class Resource { boost::shared_ptr<Implementation> impl; public: Resource(int argA) explicit { HANDLE handle = SomeLegacyCApiThatMakesSomething(argA); if (handle == INVALID_HANDLE_VALUE) throw SomeTypeOfException(); impl.reset(new Implementation(handle)); }; void SomeMethodThatActsOnTheResource() { impl->SomeMethodThatActsOnTheHandle(); }; }; This way, shared_ptr takes care of the reference counting headaches, allowing Resource to be copyable, even though the underlying handle should only be closed once all references to it are destroyed. However, it seems like we could save the overhead of allocating shared_ptr's reference counts and such separately if we could move that data inside Implementation somehow, like boost's intrusive containers do. If this is making the premature optimization hackles nag some people, I actually agree that I don't need this for my current project. But I'm curious if it is possible.

  Read the article

• Unpredictable CCK field name in returned View data

  - by AK

  I'm using views_get_view_result to directly access the data in a view. I've stumbled upon this odd behavior where cck fields are prefixed with the first field name as a query optimization. Explained here. What's bizarre though is that fields are named differently depending on whether I retrieve that data as Anonymous or as Admin. I'm pretty sure all my permissions are set up, and the view itself has no restrictions. What is going on here? This is a big problem since I can't know how to retrieve a field. Here's a dump of the two view results. Notice that node_data_field_game_date_field_game_home_score_value != node_data_field_game_official_field_game_home_score_value. // View as Admin stdClass Object ( [nid] => 3191 [node_data_field_game_date_field_game_date_value] => 2010-03-27T00:00:00 [node_type] => game [node_vid] => 5039 [node_data_field_game_date_field_game_official_value] => 0 [node_node_data_field_game_home_team_title] => TeamA [node_node_data_field_game_home_team_nid] => 3396 [node_data_field_game_date_field_game_home_score_value] => 68 [node_node_data_field_game_visitor_team_title] => TeamB [node_node_data_field_game_visitor_team_nid] => 3442 [node_data_field_game_date_field_game_visitor_score_value] => 118 ) // View as Anonymous stdClass Object ( [nid] => 3191 [node_data_field_game_date_field_game_date_value] => 2010-03-27T00:00:00 [node_type] => game [node_vid] => 5039 [node_data_field_game_official_field_game_official_value] => 0 [node_node_data_field_game_home_team_title] => TeamA [node_node_data_field_game_home_team_nid] => 3396 [node_data_field_game_official_field_game_home_score_value] => 68 [node_node_data_field_game_visitor_team_title] => TeamB [node_node_data_field_game_visitor_team_nid] => 3442 [node_data_field_game_official_field_game_visitor_score_value] => 118 )

  Read the article

• Odd C++ template behaviour with static member vars

  - by jon hanson

  This piece of code is supposed to calculate an approximation to e (i.e. the mathematical constant ~ 2.71828183) at compile-time, using the following approach; e1 = 2 / 1 e2 = (2 * 2 + 1) / (2 * 1) = 5 / 2 = 2.5 e3 = (3 * 5 + 1) / (3 * 2) = 16 / 6 ~ 2.67 e4 = (4 * 16 + 1) / (4 * 6) = 65 / 24 ~ 2.708 ... e(i) = (e(i-1).numer * i + 1) / (e(i-1).denom * i) The computation is returned via the result static member however, after 2 iterations it yields zero instead of the expected value. I've added a static member function f() to compute the same value and that doesn't exhibit the same problem. #include <iostream> #include <iomanip> // Recursive case. template<int ITERS, int NUMERATOR = 2, int DENOMINATOR = 1, int I = 2> struct CalcE { static const double result; static double f () {return CalcE<ITERS, NUMERATOR * I + 1, DENOMINATOR * I, I + 1>::f ();} }; template<int ITERS, int NUMERATOR, int DENOMINATOR, int I> const double CalcE<ITERS, NUMERATOR, DENOMINATOR, I>::result = CalcE<ITERS, NUMERATOR * I + 1, DENOMINATOR * I, I + 1>::result; // Base case. template<int ITERS, int NUMERATOR, int DENOMINATOR> struct CalcE<ITERS, NUMERATOR, DENOMINATOR, ITERS> { static const double result; static double f () {return result;} }; template<int ITERS, int NUMERATOR, int DENOMINATOR> const double CalcE<ITERS, NUMERATOR, DENOMINATOR, ITERS>::result = static_cast<double>(NUMERATOR) / DENOMINATOR; // Test it. int main (int argc, char* argv[]) { std::cout << std::setprecision (8); std::cout << "e2 ~ " << CalcE<2>::result << std::endl; std::cout << "e3 ~ " << CalcE<3>::result << std::endl; std::cout << "e4 ~ " << CalcE<4>::result << std::endl; std::cout << "e5 ~ " << CalcE<5>::result << std::endl; std::cout << std::endl; std::cout << "e2 ~ " << CalcE<2>::f () << std::endl; std::cout << "e3 ~ " << CalcE<3>::f () << std::endl; std::cout << "e4 ~ " << CalcE<4>::f () << std::endl; std::cout << "e5 ~ " << CalcE<5>::f () << std::endl; return 0; } I've tested this with VS 2008 and VS 2010, and get the same results in each case: e2 ~ 2 e3 ~ 2.5 e4 ~ 0 e5 ~ 0 e2 ~ 2 e3 ~ 2.5 e4 ~ 2.6666667 e5 ~ 2.7083333 Why does result not yield the expected values whereas f() does? According to Rotsor's comment below, this does work with GCC, so I guess the question is, am i relying on some type of undefined behaviour with regards to static initialisation order, or is this a bug with Visual Studio?

  Read the article

• Initial capacity of collection types, i.e. Dictionary, List

  - by Neil N

  Certain collection types in .Net have an optional "Initial Capacity" constructor param. i.e. Dictionary<string, string> something = new Dictionary<string,string>(20); List<string> anything = new List<string>(50); I can't seem to find what the default initial capacity is for these objects on MSDN. If I know I will only be storing 12 or so items in a dictionary, doesn't it make sense to set the initial capacity to something like 20? My reasoning is, assuming the capacity grows like it does for a StringBuiler, which doubles each time the capacity is hit, and each re-allocation is costly, why not pre-set the size to something you know will hold your data, with some extra room just in case? If the initial capacity is 100, and I know I will only need a dozen or so, it seems as though the rest of that allocated RAM is allocated for nothing. Please spare me the "premature optimization" speil for the O(n^n)th time. I know it won't make my apps any faster or save any meaningful amount of memory, this is mostly out of curiosity.

  Read the article

• Fastest way to become a MySQL expert?

  - by Kerry

  I have been using MySQL for years, mainly on smaller projects until the last year or so. I'm not sure if it's the nature of the language or my lack of real tutorials that gives me the feeling of being unsure if what I'm writing is the proper way for optimization purposes and scaling purposes. While self-taught in PHP I'm very sure of myself and the code I write, easily can compare it to others and so on. With MySQL, I'm not sure whether (and in what cases) an INNER JOIN or LEFT JOIN should be used, nor am I aware of the large amount of functionality that it has. While I've written code for databases that handled tens of millions of records, I don't know if it's optimum. I often find that a small tweak will make a query take less than 1/10 of the original time... but how do I know that my current query isn't also slow? I would like to become completely confident in this field in the ability to optimize databases and be scalable. Use is not a problem -- I use it on a daily basis in a number of different ways. So, the question is, what's the path? Reading a book? Website/tutorials? Recommendations?

  Read the article

• gcc -finline-functions behaviour?

  - by user176168

  I'm using gcc with the -finline-functions optimization for release builds. In order to combat code bloat because I work on an embedded system I want to say don't inline particular functions. The obvious way to do this would be through function attributes ie attribute(noinline). The problem is this doesn't seem to work when I switch on the global -finline-functions optimisation which is part of the -O3 switch. It also has something to do with it being templated as a non templated version of the same function doesn't get inlined which is as expected. Has anybody any idea of how to control inlining when this global switch is on? Here's the code: #include <cstdlib> #include <iostream> using namespace std; class Base { public: template<typename _Type_> static _Type_ fooT( _Type_ x, _Type_ y ) __attribute__ (( noinline )); }; template<typename _Type_> _Type_ Base::fooT( _Type_ x, _Type_ y ) { asm(""); return x + y; } int main(int argc, char *argv[]) { int test = Base::fooT( 1, 2 ); printf( "test = %d\n", test ); system("PAUSE"); return EXIT_SUCCESS; }

  Read the article

• How can I modify my Shunting-Yard Algorithm so it accepts unary operators?

  - by KingNestor

  I've been working on implementing the Shunting-Yard Algorithm in JavaScript for class. Here is my work so far: var userInput = prompt("Enter in a mathematical expression:"); var postFix = InfixToPostfix(userInput); var result = EvaluateExpression(postFix); document.write("Infix: " + userInput + "<br/>"); document.write("Postfix (RPN): " + postFix + "<br/>"); document.write("Result: " + result + "<br/>"); function EvaluateExpression(expression) { var tokens = expression.split(/([0-9]+|[*+-\/()])/); var evalStack = []; while (tokens.length != 0) { var currentToken = tokens.shift(); if (isNumber(currentToken)) { evalStack.push(currentToken); } else if (isOperator(currentToken)) { var operand1 = evalStack.pop(); var operand2 = evalStack.pop(); var result = PerformOperation(parseInt(operand1), parseInt(operand2), currentToken); evalStack.push(result); } } return evalStack.pop(); } function PerformOperation(operand1, operand2, operator) { switch(operator) { case '+': return operand1 + operand2; case '-': return operand1 - operand2; case '*': return operand1 * operand2; case '/': return operand1 / operand2; default: return; } } function InfixToPostfix(expression) { var tokens = expression.split(/([0-9]+|[*+-\/()])/); var outputQueue = []; var operatorStack = []; while (tokens.length != 0) { var currentToken = tokens.shift(); if (isNumber(currentToken)) { outputQueue.push(currentToken); } else if (isOperator(currentToken)) { while ((getAssociativity(currentToken) == 'left' && getPrecedence(currentToken) <= getPrecedence(operatorStack[operatorStack.length-1])) || (getAssociativity(currentToken) == 'right' && getPrecedence(currentToken) < getPrecedence(operatorStack[operatorStack.length-1]))) { outputQueue.push(operatorStack.pop()) } operatorStack.push(currentToken); } else if (currentToken == '(') { operatorStack.push(currentToken); } else if (currentToken == ')') { while (operatorStack[operatorStack.length-1] != '(') { if (operatorStack.length == 0) throw("Parenthesis balancing error! Shame on you!"); outputQueue.push(operatorStack.pop()); } operatorStack.pop(); } } while (operatorStack.length != 0) { if (!operatorStack[operatorStack.length-1].match(/([()])/)) outputQueue.push(operatorStack.pop()); else throw("Parenthesis balancing error! Shame on you!"); } return outputQueue.join(" "); } function isOperator(token) { if (!token.match(/([*+-\/])/)) return false; else return true; } function isNumber(token) { if (!token.match(/([0-9]+)/)) return false; else return true; } function getPrecedence(token) { switch (token) { case '^': return 9; case '*': case '/': case '%': return 8; case '+': case '-': return 6; default: return -1; } } function getAssociativity(token) { switch(token) { case '+': case '-': case '*': case '/': return 'left'; case '^': return 'right'; } } It works fine so far. If I give it: ((5+3) * 8) It will output: Infix: ((5+3) * 8) Postfix (RPN): 5 3 + 8 * Result: 64 However, I'm struggling with implementing the unary operators so I could do something like: ((-5+3) * 8) What would be the best way to implement unary operators (negation, etc)? Also, does anyone have any suggestions for handling floating point numbers as well? One last thing, if anyone sees me doing anything weird in JavaScript let me know. This is my first JavaScript program and I'm not used to it yet.

  Read the article

• I'm having an issue to use GLshort for representing Vertex, and Normal.

  - by Xylopia

  As my project gets close to optimization stage, I notice that reducing Vertex Metadata could vastly improve the performance of 3D rendering. Eventually, I've dearly searched around and have found following advices from stackoverflow. Using GL_SHORT instead of GL_FLOAT in an OpenGL ES vertex array How do you represent a normal or texture coordinate using GLshorts? Advice on speeding up OpenGL ES 1.1 on the iPhone Simple experiments show that switching from "FLOAT" to "SHORT" for vertex and normal isn't tough, but what troubles me is when you're to scale back verticies to their original size (with glScalef), normals are multiplied by the reciprocal of the scale. Natural remedy for this is to multiply the normals w/ scale before you submit to GPU. Then, my short normals almost become 0, because the scale factor is usually smaller than 0. Duh! How do you use "short" for both vertex and normal at the same time? I've been trying this and that for about a full day, but I could only go for "float vertex w/ byte normal" or "short vertex w/ float normal" so far. Your help would be truly appreciated.

  Read the article

• Reduce durability in MySQL for performance

  - by Paul Prescod

  My site occasionally has fairly predictable bursts of traffic that increase the throughput by 100 times more than normal. For example, we are going to be featured on a television show, and I expect in the hour after the show, I'll get more than 100 times more traffic than normal. My understanding is that MySQL (InnoDB) generally keeps my data in a bunch of different places: RAM Buffers commitlog binary log actual tables All of the above places on my DB slave This is too much "durability" given that I'm on an EC2 node and most of the stuff goes across the same network pipe (file systems are network attached). Plus the drives are just slow. The data is not high value and I'd rather take a small chance of a few minutes of data loss rather than have a high probability of an outage when the crowd arrives. During these traffic bursts I would like to do all of that I/O only if I can afford it. I'd like to just keep as much in RAM as possible (I have a fair chunk of RAM compared to the data size that would be touched over an hour). If buffers get scarce, or the I/O channel is not too overloaded, then sure, I'd like things to go to the commitlog or binary log to be sent to the slave. If, and only if, the I/O channel is not overloaded, I'd like to write back to the actual tables. In other words, I'd like MySQL/InnoDB to use a "write back" cache algorithm rather than a "write through" cache algorithm. Can I convince it to do that? If this is not possible, I am interested in general MySQL write-performance optimization tips. Most of the docs are about optimizing read performance, but when I get a crowd of users, I am creating accounts for all of them, so that's a write-heavy workload.

  Read the article

• SQL Structure of DB table with different types of columns

  - by Dmitry Dvornikov

  I have a problem with the optimization of the structure of the database. I'll try to explain it exactly. I create a project, where we can add different values??, but this values must have different types of the columns in the database (eg, int, double , varchar). What is the best way to store the different types of values ??in the database. In the project I'm using Propel 1.6. The point is availability to add value with 'int', 'varchar' and other columns types, to search the table was efficient. In total, I have two ideas. The first is to create a table of "value", which will have columns: "id ", "value_int", "value_double", "value_varchar", etc - with the corresponding column types. Depending on the type of values??, records will be saved with the value in the appropriate column (the rest will be NULL). The second solution is to create separate tables such as "value_int", "value_varchar" etc. There would be columns: "id", "value", which correspond to the relevant types of "value" (ie, such as int, varchar, etc). I must admit that I do not believe any of the above solutions, originally I was thinking about one table "value", where the column would be a "text" type - but this solution would probably be even worse. I would like to know your opinion on this topic, maybe something else would be better. Thanks in advance. EDIT: For example : We have three tables: USER: [table of users] * id * name FIELD: [table of profile fields - where the column 'type' is the type of field, eg int or varchar) * id * type * name VALUE : * id * User_id - ( FK user.id ) * Field_id - ( FK field.id ) * value So we have in each row an user in USER table, and the profile is stored in the VALUE table. Bit each profile field may have a different type (column 'type' in the FIELD table), and based on that I would want this value to add to the appropriate column of the appropriate type.

  Read the article

• RegQueryValueEx not working with a Release version but working fine with Debug

  - by Nux

  Hi. I'm trying to read some ODBC details form a registry and for that I use RegQueryValueEx. The problem is when I compile the release version it simply cannot read any registry values. The code is: CString odbcFuns::getOpenedKeyRegValue(HKEY hKey, CString valName) { CString retStr; char *strTmp = (char*)malloc(MAX_DSN_STR_LENGTH * sizeof(char)); memset(strTmp, 0, MAX_DSN_STR_LENGTH); DWORD cbData; long rret = RegQueryValueEx(hKey, valName, NULL, NULL, (LPBYTE)strTmp, &cbData); if (rret != ERROR_SUCCESS) { free(strTmp); return CString("?"); } strTmp[cbData] = '\0'; retStr.Format(_T("%s"), strTmp); free(strTmp); return retStr; } I've found a workaround for this - I disabled Optimization (/Od), but it seems strange that I needed to do that. Is there some other way? I use Visual Studio 2005. Maybe it's a bug in VS? Almost forgot - the error code is 2 (as the key wouldn't be found).

  Read the article

• Strict pointer aliasing: is access through a 'volatile' pointer/reference a solution?

  - by doublep

  On the heels of a specific problem, a self-answer and comments to it, I'd like to understand if it is a proper solution, workaround/hack or just plain wrong. Specifically, I rewrote code: T x = ...; if (*reinterpret_cast <int*> (&x) == 0) ... As: T x = ...; if (*reinterpret_cast <volatile int*> (&x) == 0) ... with a volatile qualifier to the pointer. Let's just assume that treating T as int in my situation makes sense. Does this accessing through a volatile reference solve pointer aliasing problem? For a reference, from specification: [ Note: volatile is a hint to the implementation to avoid aggressive optimization involving the object because the value of the object might be changed by means undetectable by an implementation. See 1.9 for detailed semantics. In general, the semantics of volatile are intended to be the same in C++ as they are in C. — end note ] EDIT: The above code did solve my problem at least on GCC 4.5.

  Read the article

• What runs faster? Wordpress or Drupal 6.x?

  - by electblake

  So... I run a pretty large Wordpress blog. Currently it gets around 20k+ pageviews a day, and its always a struggle to keep the bad boy running quickly - I currently run a vps.net with CentOS 5.3 I am also Drupal developer by trade so I love the CMS Framework for its versatility and the portability (I can take work from one site and implement on another with great ease) MY QUESTION IS: What is faster then? Wordpress 3.x & Drupal 6.x I'd love to migrate my site to Drupal to be able to roll out new features etc (which I find awkward to do in Wordpress) but I am scared that Drupal may not be able to handle the traffic. Any opinions? I know that some major players use Drupal - as Dries documents well on his blog but I'm not under any illusions that Drupal can be a real hog. Thanks for any/all help! Please try to avoid server optimization talk unless it pertains to Wordpress or Drupal 6.x specifically, I love to learn more about optimizations but I do want to sort out which platform is quicker :) p.s - I realize the fastest option is to use a lower-level framework (with less overhead) like CakePHP etc but assume that isn't an option ;)

  Read the article

• Multithreaded linked list traversal

  - by Rob Bryce

  Given a (doubly) linked list of objects (C++), I have an operation that I would like multithread, to perform on each object. The cost of the operation is not uniform for each object. The linked list is the preferred storage for this set of objects for a variety of reasons. The 1st element in each object is the pointer to the next object; the 2nd element is the previous object in the list. I have solved the problem by building an array of nodes, and applying OpenMP. This gave decent performance. I then switched to my own threading routines (based off Windows primitives) and by using InterlockedIncrement() (acting on the index into the array), I can achieve higher overall CPU utilization and faster through-put. Essentially, the threads work by "leap-frog'ing" along the elements. My next approach to optimization is to try to eliminate creating/reusing the array of elements in my linked list. However, I'd like to continue with this "leap-frog" approach and somehow use some nonexistent routine that could be called "InterlockedCompareDereference" - to atomically compare against NULL (end of list) and conditionally dereference & store, returning the dereferenced value. I don't think InterlockedCompareExchangePointer() will work since I cannot atomically dereference the pointer and call this Interlocked() method. I've done some reading and others are suggesting critical sections or spin-locks. Critical sections seem heavy-weight here. I'm tempted to try spin-locks but I thought I'd first pose the question here and ask what other people are doing. I'm not convinced that the InterlockedCompareExchangePointer() method itself could be used like a spin-lock. Then one also has to consider acquire/release/fence semantics... Ideas? Thanks!

  Read the article

• Threads are blocked in malloc and free, virtual size

  - by Albert Wang

  Hi, I'm running a 64-bit multi-threaded program on the windows server 2003 server (X64), It run into a case that some of the threads seem to be blocked in the malloc or free function forever. The stack trace is like follows: ntdll.dll!NtWaitForSingleObject() + 0xa bytes ntdll.dll!RtlpWaitOnCriticalSection() - 0x1aa bytes ntdll.dll!RtlEnterCriticalSection() + 0xb040 bytes ntdll.dll!RtlpDebugPageHeapAllocate() + 0x2f6 bytes ntdll.dll!RtlDebugAllocateHeap() + 0x40 bytes ntdll.dll!RtlAllocateHeapSlowly() + 0x5e898 bytes ntdll.dll!RtlAllocateHeap() - 0x1711a bytes MyProg.exe!malloc(unsigned __int64 size=0) Line 168 C MyProg.exe!operator new(unsigned __int64 size=1) Line 59 + 0x5 bytes C++ ntdll.dll!NtWaitForSingleObject() ntdll.dll!RtlpWaitOnCriticalSection() ntdll.dll!RtlEnterCriticalSection() ntdll.dll!RtlpDebugPageHeapFree() ntdll.dll!RtlDebugFreeHeap() ntdll.dll!RtlFreeHeapSlowly() ntdll.dll!RtlFreeHeap() MyProg.exe!free(void * pBlock=0x000000007e8e4fe0) C BTW, the param values passed to the new operator is not correct here maybe due to optimization. Also, at the same time, I found in the process Explorer, the virtual size of this program is 10GB, but the private bytes and working set is very small (<2GB). We did have some threads using virtualalloc but in a way that commit the memory in the call, and these threads are not blocked. m_pBuf = VirtualAlloc(NULL, m_size, MEM_COMMIT, PAGE_READWRITE); ...... VirtualFree(m_pBuf, 0, MEM_RELEASE); This looks strange to me, seems a lot of virtual space is reserved but not committed, and malloc/free is blocked by lock. I'm guessing if there's any corruptions in the memory/object, so plan to turn on gflag with pageheap to troubleshoot this. Does anyone has similar experience on this before? Could you share with me so I may get more hints? Thanks a lot!

  Read the article

• What is the optimum way to select the most dissimilar individuals from a population?

  - by Aaron D

  I have tried to use k-means clustering to select the most diverse markers in my population, for example, if we want to select 100 lines I cluster the whole population to 100 clusters then select the closest marker to the centroid from each cluster. The problem with my solution is it takes too much time (probably my function needs optimization), especially when the number of markers exceeds 100000. So, I will appreciate it so much if anyone can show me a new way to select markers that maximize diversity in my population and/or help me optimize my function to make it work faster. Thank you # example: library(BLR) data(wheat) dim(X) mdf<-mostdiff(t(X), 100,1,nstart=1000) Here is the mostdiff function that i used: mostdiff <- function(markers, nClust, nMrkPerClust, nstart=1000) { transposedMarkers <- as.array(markers) mrkClust <- kmeans(transposedMarkers, nClust, nstart=nstart) save(mrkClust, file="markerCluster.Rdata") # within clusters, pick the markers that are closest to the cluster centroid # turn the vector of which markers belong to which clusters into a list nClust long # each element of the list is a vector of the markers in that cluster clustersToList <- function(nClust, clusters) { vecOfCluster <- function(whichClust, clusters) { return(which(whichClust == clusters)) } return(apply(as.array(1:nClust), 1, vecOfCluster, clusters)) } pickCloseToCenter <- function(vecOfCluster, whichClust, transposedMarkers, centers, pickHowMany) { clustSize <- length(vecOfCluster) # if there are fewer than three markers, the center is equally distant from all so don't bother if (clustSize < 3) return(vecOfCluster[1:min(pickHowMany, clustSize)]) # figure out the distance (squared) between each marker in the cluster and the cluster center distToCenter <- function(marker, center){ diff <- center - marker return(sum(diff*diff)) } dists <- apply(transposedMarkers[vecOfCluster,], 1, distToCenter, center=centers[whichClust,]) return(vecOfCluster[order(dists)[1:min(pickHowMany, clustSize)]]) } }

  Read the article

• Javascript, IE, Strings, and Performance problems

  - by Infinity

  Hey guys, So we have this product, and it's really slow in IE. We've already applied a lot of the practices advised by the IE guys themselves (like this, and this), and try to sacrifice clean code for performance in the critical parts like DOM manipulation. However, as you can see in this IE profiler screenshot.. Just "String" is the biggest offender. Almost 750ms of exclusive time. Does this mean IE is spending 750ms just instantiating Strings? I also read this stuff on the Opera dev blog: A build script can remove whitespace, comments, replace strings with Array lookups (to avoid MSIE creating a string object for every single instance of a string — even in conditions) But no more info regarding this. Anyone can clarify? It seems like IE has to create a full String instance every time you have " " in your code, which could explain this, but I don't know what the array lookup optimization would look like. BTW- we don't really do much of string concatenation anywhere in the code. The library we use is MooTools 1.2.4 Any suggestions will be appreciated! Thx

  Read the article

• Can someone recommend a good tutorial on MySQL indexes, specifically when used in an order by clause

  - by Philip Brocoum

  I could try to post and explain the exact query I'm trying to run, but I'm going by the old adage of, "give a man a fish and he'll eat for a day, teach a man to fish and he'll eat for the rest of his life." SQL optimization seems to be very query-specific, and even if you could solve this one particular query for me, I'm going to have to write many more queries in the future, and I'd like to be educated on how indexes work in general. Still, here's a quick description of my current problem. I have a query that joins three tables and runs in 0.2 seconds flat. Awesome. I add an "order by" clause and it runs in 4 minutes and 30 seconds. Sucky. I denormalize one table so there is one fewer join, add indexes everywhere, and now the query runs in... 20 minutes. What the hell? Finally, I don't use a join at all, but rather a subquery with "where id in (...) order by" and now it runs in 1.5 seconds. Pretty decent. What in God's name is going on? I feel like if I actually understood what indexes were doing I could write some really good SQL. Anybody know some good tutorials? Thanks!

  Read the article

• Can you dynamically combine multiple conditional functions into one in Python?

  - by erich

  I'm curious if it's possible to take several conditional functions and create one function that checks them all (e.g. the way a generator takes a procedure for iterating through a series and creates an iterator). The basic usage case would be when you have a large number of conditional parameters (e.g. "max_a", "min_a", "max_b", "min_b", etc.), many of which could be blank. They would all be passed to this "function creating" function, which would then return one function that checked them all. Below is an example of a naive way of doing what I'm asking: def combining_function(max_a, min_a, max_b, min_b, ...): f_array = [] if max_a is not None: f_array.append( lambda x: x.a < max_a ) if min_a is not None: f_array.append( lambda x: x.a > min_a ) ... return lambda x: all( [ f(x) for f in f_array ] ) What I'm wondering is what is the most efficient to achieve what's being done above? It seems like executing a function call for every function in f_array would create a decent amount of overhead, but perhaps I'm engaging in premature/unnecessary optimization. Regardless, I'd be interested to see if anyone else has come across usage cases like this and how they proceeded. Also, if this isn't possible in Python, is it possible in other (perhaps more functional) languages?

  Read the article

• Many users, many cpus, no delays. Good for cloud?

  - by Eric

  I wish to set up a CPU-intensive time-important query service for users on the internet. A usage scenario is described below. Is cloud computing the right way to go for such an implementation? If so, what cloud vendor(s) cater to this type of application? I ask specifically, in terms of: 1) pricing 2) latency resulting from: - slow CPUs, instance creations, JIT compiles, etc.. - internal management and communication of processes inside the cloud (e.g. a queuing process and a calculation process) - communication between cloud and end user 3) ease of deployment A usage scenario I am expecting is: - A typical user sends a query (XML of size around 1K) once every 30 seconds on average. - Each query requires a numerical computation of average time 0.2 sec and max time 1 sec on a 1 GHz Pentium. The computation requires no data other than the query itself and is performed by the same piece of code each time. - The delay a user experiences between sending a query and receiving a response should be on average no more than 2 seconds and in general no more than 5 seconds. - A background save to a DB of the response should occur (not time critical) - There can be up to 30000 simultaneous users - i.e., on average 1000 queries a second, each requiring an average 0.2 sec calculation, so that would necessitate around 200 CPUs. Currently I'm look at GAE Java (for quicker deployment and less IT hassle) and EC2 (Speed and price optimization) as options. Where can I learn more about the right way to set ups such a system? past threads, different blogs, books, etc.. BTW, if my terminology is wrong or confusing, please let me know. I'd greatly appreciate any help.

  Read the article

• Strange results about C++11 memory model (Relaxed ordering)

  - by Dancing_bunny

  I was testing the example in the memory model of the Anthony Williams's book "C++ Concurrency" #include<atomic> #include<thread> #include<cassert> std::atomic_bool x,y; std::atomic_int z; void write_x_then_y() { x.store(true, std::memory_order_relaxed); y.store(true, std::memory_order_relaxed); } void read_y_then_x() { while(!y.load(std::memory_order_relaxed)); if(x.load(std::memory_order_relaxed)) { ++z; } } int main() { x = false; y = false; z = 0; std::thread a(write_x_then_y); std::thread b(read_y_then_x); a.join(); b.join(); assert(z.load()!=0); } According to the explanation, relaxed operations on difference variables (here x and y) can be freely reordered. However, I repeated running the problem for more than several days. I never hit the situation that the assertion (assert(z.load()!=0);) fires. I just use the default optimization and compile the code using g++ -std=c++11 -lpthread dataRaceAtomic.cpp Does anyone actually try it and hit the assertion? Could anyone give me an explanation about my test results? BTW, I also tried the version without using the atomic type, I got the same result. Currently, both programs are running healthily. Thanks.

  Read the article

• Reordering Variadic Parameters

  - by void-pointer

  I have come across the need to reorder a variadic list of parameters that is supplied to the constructor of a struct. After being reordered based on their types, the parameters will be stored as a tuple. My question is how this can be done so that a modern C++ compiler (e.g. g++-4.7) will not generate unnecessary load or store instructions. That is, when the constructor is invoked with a list of parameters of variable size, it efficiently pushes each parameter into place based on an ordering over the parameters' types. Here is a concrete example. Assume that the base type of every parameter (without references, rvalue references, pointers, or qualifiers) is either char, int, or float. How can I make it so that all the parameters of base type char appear first, followed by all of those of base type int (which leaves the parameters of base type float last). The relative order in which the parameters were given should not be violated within sublists of homogeneous base type. Example: foo::foo() is called with arguments float a, char&& b, const float& c, int&& d, char e. The tuple tupe is std::tuple<char, char, int, float, float>, and it is constructed like so: tuple_type{std::move(b), e, std::move(d), a, c}. Consider the struct defined below, and assume that the metafunction deduce_reordered_tuple_type is already implemented. How would you write the constructor so that it works as intended? If you think that the code for deduce_reodered_tuple_type, would be useful to you, I can provide it; it's a little long. template <class... Args> struct foo { // Assume that the metafunction deduce_reordered_tuple_type is defined. typedef typename deduce_reordered_tuple_type<Args...>::type tuple_type; tuple_type t_; foo(Args&&... args) : t_{reorder_and_forward_parameters<Args>(args)...} {} }; Edit 1 The technique I describe above does have applications in mathematical frameworks that make heavy use of expression templates, variadic templates, and metaprogramming in order to perform aggressive inlining. Suppose that you wish to define an operator that takes the product of several expressions, each of which may be passed by reference, reference to const, or rvalue reference. (In my case, the expressions are conditional probability tables and the operation is the factor product, but something like matrix multiplication works suitably as well.) You need access to the data provided by each expression in order to evaluate the product. Consequently, you must move the expressions passed as rvalue references, copy the expressions passed by reference to const, and take the addresses of expressions passed by reference. Using the technique I describe above now poses several benefits. Other expressions can use uniform syntax to access data elements from this expression, since all of the heavy-lifting metaprogramming work is done beforehand, within the class. We can save stack space by grouping the pointers together and storing the larger expressions towards the end of the tuple. Implementing certain types of queries becomes much easier (e.g. check whether any of the pointers stored in the tuple aliases a given pointer). Thank you very much for your help!

  Read the article

• Can g++ fill uninitialized POD variables with known values?

  - by Bob Lied

  I know that Visual Studio under debugging options will fill memory with a known value. Does g++ (any version, but gcc 4.1.2 is most interesting) have any options that would fill an uninitialized local POD structure with recognizable values? struct something{ int a; int b; }; void foo() { something uninitialized; bar(uninitialized.b); } I expect uninitialized.b to be unpredictable randomness; clearly a bug and easily found if optimization and warnings are turned on. But compiled with -g only, no warning. A colleague had a case where code similar to this worked because it coincidentally had a valid value; when the compiler upgraded, it started failing. He thought it was because the new compiler was inserting known values into the structure (much the way that VS fills 0xCC). In my own experience, it was just different random values that didn't happen to be valid. But now I'm curious -- is there any setting of g++ that would make it fill memory that the standard would otherwise say should be uninitialized?

  Read the article

• Calculate the number of ways to roll a certain number

  - by helloworld

  I'm a high school Computer Science student, and today I was given a problem to: Program Description: There is a belief among dice players that in throwing three dice a ten is easier to get than a nine. Can you write a program that proves or disproves this belief? Have the computer compute all the possible ways three dice can be thrown: 1 + 1 + 1, 1 + 1 + 2, 1 + 1 + 3, etc. Add up each of these possibilities and see how many give nine as the result and how many give ten. If more give ten, then the belief is proven. I quickly worked out a brute force solution, as such int sum,tens,nines; tens=nines=0; for(int i=1;i<=6;i++){ for(int j=1;j<=6;j++){ for(int k=1;k<=6;k++){ sum=i+j+k; //Ternary operators are fun! tens+=((sum==10)?1:0); nines+=((sum==9)?1:0); } } } System.out.println("There are "+tens+" ways to roll a 10"); System.out.println("There are "+nines+" ways to roll a 9"); Which works just fine, and a brute force solution is what the teacher wanted us to do. However, it doesn't scale, and I am trying to find a way to make an algorithm that can calculate the number of ways to roll n dice to get a specific number. Therefore, I started generating the number of ways to get each sum with n dice. With 1 die, there is obviously 1 solution for each. I then calculated, through brute force, the combinations with 2 and 3 dice. These are for two: There are 1 ways to roll a 2 There are 2 ways to roll a 3 There are 3 ways to roll a 4 There are 4 ways to roll a 5 There are 5 ways to roll a 6 There are 6 ways to roll a 7 There are 5 ways to roll a 8 There are 4 ways to roll a 9 There are 3 ways to roll a 10 There are 2 ways to roll a 11 There are 1 ways to roll a 12 Which looks straightforward enough; it can be calculated with a simple linear absolute value function. But then things start getting trickier. With 3: There are 1 ways to roll a 3 There are 3 ways to roll a 4 There are 6 ways to roll a 5 There are 10 ways to roll a 6 There are 15 ways to roll a 7 There are 21 ways to roll a 8 There are 25 ways to roll a 9 There are 27 ways to roll a 10 There are 27 ways to roll a 11 There are 25 ways to roll a 12 There are 21 ways to roll a 13 There are 15 ways to roll a 14 There are 10 ways to roll a 15 There are 6 ways to roll a 16 There are 3 ways to roll a 17 There are 1 ways to roll a 18 So I look at that, and I think: Cool, Triangular numbers! However, then I notice those pesky 25s and 27s. So it's obviously not triangular numbers, but still some polynomial expansion, since it's symmetric. So I take to Google, and I come across this page that goes into some detail about how to do this with math. It is fairly easy(albeit long) to find this using repeated derivatives or expansion, but it would be much harder to program that for me. I didn't quite understand the second and third answers, since I have never encountered that notation or those concepts in my math studies before. Could someone please explain how I could write a program to do this, or explain the solutions given on that page, for my own understanding of combinatorics? EDIT: I'm looking for a mathematical way to solve this, that gives an exact theoretical number, not by simulating dice

  Read the article

< Previous Page | 138 139 140 141 142 143 144 145 146 147 148 149  | Next Page >