Search Results

Search found 20224 results on 809 pages for 'query optimization'.

Page 178/809 | < Previous Page | 174 175 176 177 178 179 180 181 182 183 184 185  | Next Page >

  • Optimizing BeautifulSoup (Python) code

    - by user283405
    I have code that uses the BeautifulSoup library for parsing, but it is very slow. The code is written in such a way that threads cannot be used. Can anyone help me with this? I am using BeautifulSoup for parsing and than save into a DB. If I comment out the save statement, it still takes a long time, so there is no problem with the database. def parse(self,text): soup = BeautifulSoup(text) arr = soup.findAll('tbody') for i in range(0,len(arr)-1): data=Data() soup2 = BeautifulSoup(str(arr[i])) arr2 = soup2.findAll('td') c=0 for j in arr2: if str(j).find("<a href=") > 0: data.sourceURL = self.getAttributeValue(str(j),'<a href="') else: if c == 2: data.Hits=j.renderContents() #and few others... c = c+1 data.save() Any suggestions? Note: I already ask this question here but that was closed due to incomplete information.

    Read the article

  • How to insert zeros between bits in a bitmap?

    - by anatolyg
    I have some performance-heavy code that performs bit manipulations. It can be reduced to the following well-defined problem: Given a 13-bit bitmap, construct a 26-bit bitmap that contains the original bits spaced at even positions. To illustrate: 0000000000000000000abcdefghijklm (input, 32 bits) 0000000a0b0c0d0e0f0g0h0i0j0k0l0m (output, 32 bits) I currently have it implemented in the following way in C: if (input & (1 << 12)) output |= 1 << 24; if (input & (1 << 11)) output |= 1 << 22; if (input & (1 << 10)) output |= 1 << 20; ... My compiler (MS Visual Studio) turned this into the following: test eax,1000h jne 0064F5EC or edx,1000000h ... (repeated 13 times with minor differences in constants) I wonder whether i can make it any faster. I would like to have my code written in C, but switching to assembly language is possible. Can i use some MMX/SSE instructions to process all bits at once? Maybe i can use multiplication? (multiply by 0x11111111 or some other magical constant) Would it be better to use condition-set instruction (SETcc) instead of conditional-jump instruction? If yes, how can i make the compiler produce such code for me? Any other idea how to make it faster? Any idea how to do the inverse bitmap transformation (i have to implement it too, bit it's less critical)?

    Read the article

  • changing the serialization procedure for a graph of objects (.net framework)

    - by pierusch
    Hello I'm developing a scientific application using .net framework. The application depends heavily upon a large data structure (a tree like structure) that has been serialized using a standard binaryformatter object. The graph structure looks like this: <serializable()>Public class BigObjet inherits list(of smallObject) end class <serializable()>public class smallObject inherits list(of otherSmallerObjects) end class ... The binaryFormatter object does a nice job but it's not optimized at all and the entire data structure reaches around 100Mb on my filesystem. Deserialization works too but it's pretty slow (around 30seconds on my quad core). I've found a nice .dll on codeproject (see "optimizing serialization...") so I wrote a modified version of the classes above overriding the default serialization/deserialization procedure reaching very good results. The problem is this: I can't lose the data previosly serialized with the old version and I'd like to be able to use the new serialization/deserialization method. I have some ideas but I'm pretty sure someone will be able to give me a proper and better advice ! use an "helper" graph of objects who takes care of the entire serialization/deserialization procedure reading data from the old format and converting them into the classes I nedd. This could work but the binaryformatter "needs" to know the types being serialized so........ :( modify the "old" graph to include a modified version of serialization procedure...so I'll be able to deserialize old file and save them with the new format......this doesn't sound too good imho. well any help will be higly highly appreciated :)

    Read the article

  • How to simplify my code... 2D NSArray in Objective C...?

    - by Tattat
    self.myArray = [NSArray arrayWithObjects: [NSArray arrayWithObjects: [self d], [self generateMySecretObject],nil], [NSArray arrayWithObjects: [self generateMySecretObject], [self generateMySecretObject],nil],nil]; for (int k=0; k<[self.myArray count]; k++) { for(int s = 0; s<[[self.myArray objectAtIndex:k] count]; s++){ [[[self.myArray objectAtIndex:k] objectAtIndex:s] setAttribute:[self generateSecertAttribute]]; } } As you can see this is a simple 2*2 array, but it takes me lots of code to assign the NSArray in very first place, because I found that the NSArray can't assign the size at very beginning. Also, I want to set attribute one by one. I can't think of if my array change to 10*10. How long it could be. So, I hope you guys can give me some suggestions on shorten the code, and more readable. thz

    Read the article

  • Fast read of certain bytes of multiple files in C/C++

    - by Alejandro Cámara
    I've been searching in the web about this question and although there are many similar questions about read/write in C/C++, I haven't found about this specific task. I want to be able to read from multiple files (256x256 files) only sizeof(double) bytes located in a certain position of each file. Right now my solution is, for each file: Open the file (read, binary mode): fstream fTest("current_file", ios_base::out | ios_base::binary); Seek the position I want to read: fTest.seekg(position*sizeof(test_value), ios_base::beg); Read the bytes: fTest.read((char *) &(output[i][j]), sizeof(test_value)); And close the file: fTest.close(); This takes about 350 ms to run inside a for{ for {} } structure with 256x256 iterations (one for each file). Q: Do you think there is a better way to implement this operation? How would you do it?

    Read the article

  • Python Profiling In Windows, How do you ignore Builtin Functions

    - by Tim McJilton
    I have not been capable of finding this anywhere online. I was looking to find out using a profiler how to better optimize my code, and when sorting by which functions use up the most time cumulatively, things like str(), print, and other similar widely used functions eat up much of the profile. What is the best way to profile a python program to get the user-defined functions only to see what areas of their code they can optimize? I hope that makes sense, any light you can shed on this subject would be very appreciated.

    Read the article

  • dataset using where condition from another query

    - by refer
    here's my code. I have a dataset that has its values and then i run an independent query which uses the same tables as dataset. now when i run the dataset, i want it to get a where condition on the query result. here's the code - sql = "SELECT ID, name FROM books WITH(NOLOCK) WHERE id =" & Session("ID") ds = FillDataset(sql) Sql1 = "SELECT n.id as id,a.name as title FROM books n WITH(NOLOCK)" & _ "LEFT JOIN new_books a WITH(NOLOCK) ON a.id = n.book_id " rd = ExecuteReader(SqlCnn, SqlStr) Now after this i make an htmltable and all the cells are loaded with the data from datareader(rd). but 1 dropdown is loaded from the dataset. i want that dropdown to have the selectedvalue which is the value from the datareader(rd). how can i do that?

    Read the article

  • building objects from xml file at runtime and intializing, in one pass?

    - by KaluSingh Gabbar
    I have to parse the XML file and build objects representation based on that, now once I get all these data I create entries in various database for these data objects. I have to do second pass over that for value as in the first pass all I could do is build the assets in various databases. and in second pass I get the values for all the data and put it in the database. I have a feeling that this can be done in a single pass but I just want to see what are your opinions. As I am just a student who started with professional work, experienced ppl please help. Can someone who have ideas or done similar work, please provide some light on the topic so that I can think over the possibility of the work and get the prototype going based on your suggestion. Thanks a lot for your precious time, I honestly appreciate it.

    Read the article

  • Slow query with unexpected scan

    - by zerkms
    Hello I have this query: SELECT * FROM SAMPLE SAMPLE INNER JOIN TEST TEST ON SAMPLE.SAMPLE_NUMBER = TEST.SAMPLE_NUMBER INNER JOIN RESULT RESULT ON TEST.TEST_NUMBER = RESULT . TEST_NUMBER WHERE SAMPLED_DATE BETWEEN '2010-03-17 09:00' AND '2010-03-17 12:00' the biggest table here is RESULT, contains 11.1M records. The left 2 tables about 1M. this query works slowly (more than 10 minutes) and returns about 800 records. executing plan shows clustered index scan over all 11M records. RESULT.TEST_NUMBER is a clustered primary key. if I change 2010-03-17 09:00 to 2010-03-17 10:00 - i get about 40 records. it executes for 300ms. and plan shows clustered index seek if i replace * in SELECT clause to RESULT.TEST_NUMBER (covered with index) - then all become fast in first case too. this points to hdd io issues, but doesn't clarifies changing plan. so, any ideas?

    Read the article

  • Ruby Challenge - efficiently change the last character of every word in a sentence to a capital

    - by emson
    Hi All I recently was challenged to write some Ruby code to change the last character of every word in a sentence into a capital. Such that the string: "script to convert the last letter of every word to a capital" becomes "scripT tO converT thE lasT letteR oF everY worD tO A capitaL" This was my optimal solution however I'm sure you wizards have much better solutions and I would be really interested to hear them. "script to convert the last letter of every word to a capital".split.map{|w|w<<w.slice!(-1).chr.upcase}.join' ' For those interested as to what is going on here is an explanation. split will split the sentence up into an array, the default delimiter is a space and with Ruby you don't need to use brackets here. map the array from split is passed to map which opens a block and process each word (w) in the array. the block slice!(s) off the last character of the word and converts it to a chr (a character not ASCII code) and then capitalises upcase it. This character is now appended << to the word which is missing the sliced last letter. Finally the array of words is now join together with a ' ' to reform the sentence. Enjoy

    Read the article

  • How do I make a sql query where fields are the result of a different query?

    - by CRP
    I have two tables, the first is like this: f1 | f2 | f3 | f4 ----------------- data.... the second contains info about the fields of the first: field | info ------------ f1 a f2 b f3 a etc. I would like to query the first table selecting the fields with a query on the second. So, for example, I might want to get data for fields where info is equal to "a", thus I would do "select f1, f3 from first_table". How do I do this programmatically? I was thinking about something along the lines of select (select fields from second_table where info='a') from first_table Thanks Chris

    Read the article

  • Biggest performance improvement you've had with the smallest change?

    - by JoelFan
    What's the biggest performance improvement you've had with the smallest change? For example, I once improved the performance of a certain page on a high-profile web app by a factor of 10, just by moving "where customerID = ?" to a different place inside a complicated SQL statement (before my change it had been selecting all customers in a join, then later selecting out the desired customer).

    Read the article

  • Retrieving Spotlight query used to open document in application

    - by vicvicvic
    If you search for something that opens Preview.app (e.g. a PDF file) in Spotlight, the application not only opens the file but actually copies the Spotlight query into the application's search field (for deeper search). iPhoto and Mail.app also do this. This is a really nice feature. Unfortunately, Apple does not document how it's implemented. Opening a document from Spotlight appears to simply send application:openFile to the application delegate. Is it possible to retrieve the user's Spotlight query somehow? Or is this a hardcoded feature of Spotlight?

    Read the article

  • What algorithm can I use to determine points within a semi-circle?

    - by khayman218
    I have a list of two-dimensional points and I want to obtain which of them fall within a semi-circle. Originally, the target shape was a rectangle aligned with the x and y axis. So the current algorithm sorts the pairs by their X coord and binary searches to the first one that could fall within the rectangle. Then it iterates over each point sequentially. It stops when it hits one that is beyond both the X and Y upper-bound of the target rectangle. This does not work for a semi-circle as you cannot determine an effective upper/lower x and y bounds for it. The semi-circle can have any orientation. Worst case, I will find the least value of a dimension (say x) in the semi-circle, binary search to the first point which is beyond it and then sequentially test the points until I get beyond the upper bound of that dimension. Basically testing an entire band's worth of points on the grid. The problem being this will end up checking many points which are not within the bounds.

    Read the article

  • "variable tracking" is eating my compile time!

    - by wowus
    I have an auto-generated file which looks something like this... static void do_SomeFunc1(void* parameter) { // Do stuff. } // Continues on for another 4000 functions... void dispatch(int id, void* parameter) { switch(id) { case ::SomeClass1::id: return do_SomeFunc1(parameter); case ::SomeClass2::id: return do_SomeFunc2(parameter); // This continues for the next 4000 cases... } } When I build it like this, the build time is enormous. If I inline all the functions automagically into their respective cases using my script, the build time is cut in half. GCC 4.5.0 says ~50% of the build time is being taken up by "variable tracking" when I use -ftime-report. What does this mean and how can I speed compilation while still maintaining the superior cache locality of pulling out the functions from the switch? EDIT: Interestingly enough, the build time has exploded only on debug builds, as per the following profiling information of the whole project (which isn't just the file in question, but still a good metric; the file in question takes the most time to build): Debug: 8 minutes 50 seconds Release: 4 minutes, 25 seconds

    Read the article

  • Is code clearness killing application performance?

    - by Jorge Córdoba
    As today's code is getting more complex by the minute, code needs to be designed to be maintainable - meaning easy to read, and easy to understand. That being said, I can't help but remember the programs that ran a couple of years ago such as Winamp or some games in which you needed a high performance program because your 486 100 Mhz wouldn't play mp3s with that beautiful mp3 player which consumed all of your CPU cycles. Now I run Media Player (or whatever), start playing an mp3 and it eats up a 25-30% of one of my four cores. Come on!! If a 486 can do it, how can the playback take up so much processor to do the same? I'm a developer myself, and I always used to advise: keep your code simple, don't prematurely optimize for performance. It seems that we've gone from "trying to get it to use the least amount of CPU as possible" to "if it doesn't take too much CPU is all right". So, do you think we are killing performance by ignoring optimizations?

    Read the article

  • How to query data from a password protected https website

    - by Addie
    I'd like my application to query a csv file from a secure website. I have no experience with web programming so I'd appreciate detailed instructions. Currently I have the user login to the site, manually query the csv, and have my application load the file locally. I'd like to automate this by having the user enter his login information, authenticating him on the website, and querying the data. The application is written in C# .NET. The url of the site is: https://www2.emidas.com/default.asp. I've tested the following code already and am able to access the file once the user has already authenticated himself and created a manual query. System.Net.WebClient Client = new WebClient(); Stream strm = Client.OpenRead("https://www3.emidas.com/users/<username>/file.csv"); Here is the request sent to the site for authentication. I've angle bracketed the real userid and password. POST /pwdVal.asp HTTP/1.1 Accept: image/jpeg, application/x-ms-application, image/gif, application/xaml+xml, image/pjpeg, application/x-ms-xbap, application/vnd.ms-excel, application/vnd.ms-powerpoint, application/msword, application/x-shockwave-flash, */* User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; InfoPath.2; Tablet PC 2.0; OfficeLiveConnector.1.4; OfficeLivePatch.1.3; .NET4.0C; .NET4.0E) Content-Type: application/x-www-form-urlencoded Accept-Encoding: gzip, deflate Cookie: ASPSESSIONID<unsure if this data contained password info so removed>; ClientId=<username> Host: www3.emidas.com Content-Length: 36 Connection: Keep-Alive Cache-Control: no-cache Accept-Language: en-US client_id=<username>&password=<password>

    Read the article

  • What to do with a big image that's slowing website loading down significantly

    - by Dave
    Hi I'm working on a website that's already been designed by someone else. The designer has used a big image (900x700 100KB) which contains a big logo right across the top, then the background for two columns. This image loads every time a page is loaded as it forms the basis for the website. What should I do with it to improve loading time? I'm considering splitting it up into two or more images, especially the logo on the top. Does splitting up images like that decrease loading time in any significant way? Thanks -edit: Also, all the images are .jpg, would changing this to .gif or .png help anything?

    Read the article

  • Why does php show error for my SQL query

    - by ZincX
    UPDATE: My mistake - I made a typo. Nevermind this question. I'm using php to update a mysql database. The resultant query I'm using when i print it out on my webpage before executing is as follows: INSERT INTO perch2_content_items (itemOrder, regionID, pageID, itemRev, itemID, itemJSON, itemSearch ) SELECT MAX(itemOrder)+1, 105, 81, 11, 118, 'json', 'search' FROM perch2_content_items WHERE regionID=105 When I copy and paste this query directly into the phpmyadmin SQL interface, it works fine. The table gets updated. However, when I try to execute it using my php code as follows, it throws an error. $insertToPerch = "INSERT INTO perch2_content_items (itemOrder, regionID, pageID, itemRev, itemID, itemJSON, itemSearch ) SELECT MAX(itemOrder)+1, $regionID, $pageID, $regionRev, $newItemID, 'json', 'search' FROM perch2_content_items WHERE regionID=$regionID"; mysql_query(insertToPerch) or die(mysql_error()); The error I'm getting is: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'insertToPerch' at line 1 Can anybody help me figure out why it is failing.

    Read the article

  • why is there extra using where in execution plan of query

    - by user366534
    I see plan of query: EXPLAIN SELECT * FROM `subscribers` WHERE state =4 AND date_added < '2010-12-23 11:47:45' It shows: id select_type table type possible_keys key key_len ref rows Extra 1 SIMPLE subscribers range state_date_added state_date_added 9 NULL 8 Using where Here is indexes of table: Table Non_unique Key_name Seq_in_index Column_name Collation Cardinality Sub_part Packed Null Index_type Comment subscribers 0 PRIMARY 1 subscriber_id A 382039 NULL NULL BTREE subscribers 0 email_list_id 1 email_address A 191019 NULL NULL BTREE subscribers 0 email_list_id 2 list_id A 382039 NULL NULL BTREE subscribers 1 FK_list_id 1 list_id A 10 NULL NULL BTREE subscribers 1 state_date_added 1 state A 12 NULL NULL BTREE subscribers 1 state_date_added 2 date_added A 8128 NULL NULL BTREE The last two lines describes index what is supposed for the query. Why is there in extra column using where? Even If I fetch only state and date_added column, it has in extra column: Using where; Using index. I understand why it has using index, but I don't understand Using where here.

    Read the article

  • Performing complex query with Dynamics CRM 4.0

    - by dub
    Hi, I have two custom entites, Product and ProductType, linked together in many-to-one relationship. Product has a lookup field to ProductType. I'm trying to write a query to fetch Type1 products with a price over 100, and Type2 products with a price lower than 100. Here's how I would do it in SQL : select * from Product P inner join ProductType T on T.Id = P.TypeId where (T.Code = 'Type1' and P.Price >= 100) or (T.Code = 'Type2' and P.Price < 100) I can't figure out a way to build a QueryExpression to do exactly that. I know I could do it with two queries, but I'd like to minimize roundtrips to the server. Is there a way to perform that query in only one operation ? Thanks!

    Read the article

  • Whats wrong with my SQL query?

    - by William
    I'm trying to set up a query that shows the first post of each thread and is ordered by the date of the last post in each thread. I got the first part down with this query: SELECT * FROM ( SELECT Min( ID ) AS MinID FROM test_posts GROUP BY Thread )tmin JOIN test_posts ON test_posts.ID = tmin.MinID Now I need to figure out how to call the last post of each thread into a table, than use that table to order the first tables results. So far I got this, but it doesn't work. SELECT * FROM ( SELECT Min( ID ) AS MinID FROM test_posts GROUP BY Thread )tmin JOIN test_posts ON test_posts.ID = tmin.MinID ORDER BY (SELECT MAX( ID ) AS MaxID, Thread, MAX( Date ) FROM test_posts GROUP BY Thread )tmax tmax.Date

    Read the article

< Previous Page | 174 175 176 177 178 179 180 181 182 183 184 185  | Next Page >