Search Results

Search found 11166 results on 447 pages for 'justin standard'.

Page 155/447 | < Previous Page | 151 152 153 154 155 156 157 158 159 160 161 162  | Next Page >

  • How to optimize my PageRank calculation?

    - by asmaier
    In the book Programming Collective Intelligence I found the following function to compute the PageRank: def calculatepagerank(self,iterations=20): # clear out the current PageRank tables self.con.execute("drop table if exists pagerank") self.con.execute("create table pagerank(urlid primary key,score)") self.con.execute("create index prankidx on pagerank(urlid)") # initialize every url with a PageRank of 1.0 self.con.execute("insert into pagerank select rowid,1.0 from urllist") self.dbcommit() for i in range(iterations): print "Iteration %d" % i for (urlid,) in self.con.execute("select rowid from urllist"): pr=0.15 # Loop through all the pages that link to this one for (linker,) in self.con.execute("select distinct fromid from link where toid=%d" % urlid): # Get the PageRank of the linker linkingpr=self.con.execute("select score from pagerank where urlid=%d" % linker).fetchone()[0] # Get the total number of links from the linker linkingcount=self.con.execute("select count(*) from link where fromid=%d" % linker).fetchone()[0] pr+=0.85*(linkingpr/linkingcount) self.con.execute("update pagerank set score=%f where urlid=%d" % (pr,urlid)) self.dbcommit() However, this function is very slow, because of all the SQL queries in every iteration >>> import cProfile >>> cProfile.run("crawler.calculatepagerank()") 2262510 function calls in 136.006 CPU seconds Ordered by: standard name ncalls tottime percall cumtime percall filename:lineno(function) 1 0.000 0.000 136.006 136.006 <string>:1(<module>) 1 20.826 20.826 136.006 136.006 searchengine.py:179(calculatepagerank) 21 0.000 0.000 0.528 0.025 searchengine.py:27(dbcommit) 21 0.528 0.025 0.528 0.025 {method 'commit' of 'sqlite3.Connecti 1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler 1339864 112.602 0.000 112.602 0.000 {method 'execute' of 'sqlite3.Connec 922600 2.050 0.000 2.050 0.000 {method 'fetchone' of 'sqlite3.Cursor' 1 0.000 0.000 0.000 0.000 {range} So I optimized the function and came up with this: def calculatepagerank2(self,iterations=20): # clear out the current PageRank tables self.con.execute("drop table if exists pagerank") self.con.execute("create table pagerank(urlid primary key,score)") self.con.execute("create index prankidx on pagerank(urlid)") # initialize every url with a PageRank of 1.0 self.con.execute("insert into pagerank select rowid,1.0 from urllist") self.dbcommit() inlinks={} numoutlinks={} pagerank={} for (urlid,) in self.con.execute("select rowid from urllist"): inlinks[urlid]=[] numoutlinks[urlid]=0 # Initialize pagerank vector with 1.0 pagerank[urlid]=1.0 # Loop through all the pages that link to this one for (inlink,) in self.con.execute("select distinct fromid from link where toid=%d" % urlid): inlinks[urlid].append(inlink) # get number of outgoing links from a page numoutlinks[urlid]=self.con.execute("select count(*) from link where fromid=%d" % urlid).fetchone()[0] for i in range(iterations): print "Iteration %d" % i for urlid in pagerank: pr=0.15 for link in inlinks[urlid]: linkpr=pagerank[link] linkcount=numoutlinks[link] pr+=0.85*(linkpr/linkcount) pagerank[urlid]=pr for urlid in pagerank: self.con.execute("update pagerank set score=%f where urlid=%d" % (pagerank[urlid],urlid)) self.dbcommit() This function is 20 times faster (but uses a lot more memory for all the temporary dictionaries) because it avoids the unnecessary SQL queries in every iteration: >>> cProfile.run("crawler.calculatepagerank2()") 64802 function calls in 6.950 CPU seconds Ordered by: standard name ncalls tottime percall cumtime percall filename:lineno(function) 1 0.004 0.004 6.950 6.950 <string>:1(<module>) 1 1.004 1.004 6.946 6.946 searchengine.py:207(calculatepagerank2 2 0.000 0.000 0.104 0.052 searchengine.py:27(dbcommit) 23065 0.012 0.000 0.012 0.000 {meth 'append' of 'list' objects} 2 0.104 0.052 0.104 0.052 {meth 'commit' of 'sqlite3.Connection 1 0.000 0.000 0.000 0.000 {meth 'disable' of '_lsprof.Profiler' 31298 5.809 0.000 5.809 0.000 {meth 'execute' of 'sqlite3.Connectio 10431 0.018 0.000 0.018 0.000 {method 'fetchone' of 'sqlite3.Cursor' 1 0.000 0.000 0.000 0.000 {range} But is it possible to further reduce the number of SQL queries to speed up the function even more?

    Read the article

  • Software tools to automatically decrypt a file, whose encryption algorithm (and/or encryption keys)

    - by Andrew
    I have an idea for encryption that I could program fairly easily to encrypt some local text file. Given that my approach is novel, and does not use any of the industry standard encryption techniques, would I be able to test the strength of my encryption using 'cracker' apps or suchlike? Or do all those tools rely on advanced knowledge of the encryption process (or intercepted 'keys'), meaning I'd have to build my own cracker for testing?

    Read the article

  • How to increase the startup speed of the delphi app?

    - by Olaf
    What do you do to increase startup speed (or to decrease startup time) of your Delphi app? Other than application specific, is there a standard trick that always works? Note: I'm not talking about fast algorithms or the likes. Only the performance increase at startup, in terms of speed.

    Read the article

  • Parse vBulletin's BB Code in PHP

    - by Josh K
    I would like a function that parses BB Code from vBulletin into a standard HTML markup. Without using the PEAR library or the PECL extension. Specifically the trouble I'm having is matching [quote=My Name]. The name 'My Name' isn't enclosed by anything and can contain spaces.

    Read the article

  • Node.js appears to be missing the multipart module

    - by Brenton Alker
    I am trying to parse form data, including upload files with a node.js http server. All of the tutorial type articles I have found use a require("multipart"); to include the multipart module, but when I try the same I get: Error: Cannot find module 'multipart' I also can't find it in the current api docs (though it is in the google cached version). So, has this module been removed from the standard installation or is there something else that does the job?

    Read the article

  • Regex | validation error

    - by MMRUser
    I'm trying to validate a USA mobile number, since I'm using pre-built javascript validation library I just replaced this regex validation with the previous one which comes with the validation library. previous validation regex: "telephone":{ "regex":"/^[0-9\-\(\)\ ]{10,10}$/", "alertText":"* Invalid phone number"}, This works like 2126661234 but not in USA standard. After I changed: "telephone":{ "regex":"/^[2-9]\d{2}-\d{3}-\d{4}$/", "alertText":"* Invalid phone number"}, Now every entry I get an error even if I enter 212-666-1234 I really don't know what is the wrong, so I'm expecting some help.

    Read the article

  • Using the web browser control

    - by Khou
    Using the standard .NET web browser control how do you... 1)Auto POST (ie submit) a form 2) Detect form has been posted and the browser has been landed on the specified page (this is to check that the user has logged in) 3) detect form input value on a change event

    Read the article

  • How to count differences between two files on linux?

    - by Zsolt Botykai
    Hi all, I need to work with large files and must find differences between two. And I don't need the different bits, but the number of differences. For the differ rows I come up with diff --suppress-common-lines --speed-large-files -y File1 File2 | wc -l And it works, but is there a better way to do it? And how to count the exact number of differences (with standard tools like bash, diff, awk, sed some old version of perl)? Thanks in advance

    Read the article

  • java IO to copy one File to another

    - by Aly
    I have two Java.io.File objects file1 and file2. I want to copy the contents from file1 to file2. Is there an standard way to do this without me having to create a method that reads file1 and write to file2

    Read the article

  • Non-modal WinForms FontDialog?

    - by McKenzieG1
    Is it possible to display the WinForms "font picker" dialog non-modally? Or is there another font picker other than the standard one that can be used non-modally? Our application has many windows, and users who frequently need to interrupt what they are doing and switch to another window to look at something. This tends to lead to "buried dialog" scenarios with modal dialogs, where the UI is unresponsive, but it isn't immediately apparent why, because the modal dialog that has captured the focus is behind another window.

    Read the article

  • Difference between Java SE/EE/ME?

    - by Richard Knop
    Which one should I install when I want to start learning Java? I going to start with some basics so I will create simple programs that create files, directories, edit XML files and so on, nothing too complex for now. I guess Java SE (Standard Edition) is the one I should install on my Windows 7 desktop? I already have Komodo IDE which I will use to write Java code.

    Read the article

  • Connection Pooling in J2SE?

    - by Zombies
    It is hard to find a resource on this without finding J2EE, but here is what I am looking for: All I have is a standard J2SE app, it makes a lot of DB connections. I simply need to make use of a connection pool. Any suggestions?

    Read the article

  • iPhone UIViewController goes under status bar

    - by Mantas
    I have UIView and UIController view. My is standard 320x460 view. In applicationDidFinishLaunching I do [window addSubview:[controller view]]; What is weird, UIView goes under the status bar (like there's missing outlet). However, if I rotate iPhone to the side and then back, it shows up ok. Is this an expected behavior (I bet I can ix it by setting offset) or am I doing smth wrong?

    Read the article

  • How do you adjust the frame or vertical alignment of a UIBarButtonItem contained by a UIToolbar inst

    - by akaii
    Horizontal positioning of UIBarButtonItems is no problem, I can simply pad the space with fixed/flexible space items. However, I can't seem to adjust the toolbar item vertically. UIToolbar has no alignment property, and UIBarButtonItem has no way of setting its frame. I need to do this because we're using a mix of custom icons created using initWithImage, and standard icons created with initWithBarButtonSystemItem. The custom icons aren't being centered properly (they're offset upwards, relative to the system icons, which are centered properly), so the toolbar looks awkward.

    Read the article

  • Square Brackets in XSL-FO

    - by Igman
    I am attempting to create a list in XSL-FO using a square bracket. I have been able to get it working using the standard unicode bullet character (&#8226;) but I just can't seem to get it working for square brackets. I have tried using &#9632;, but that does not seem to work. It is important that i can get the square bullets working because I am matching an existing file format.Any help in getting this working would be greatly appreciated.

    Read the article

  • ASP.NET MVC Render View to String

    - by Ben
    I'm currently using NVelocity to render some email templates. I'm curious how I can do this with the standard ASP.NET MVC view engine i.e. render a view as a string. Basically I want to pass get the string result of a model binded view and pass this to iTextSharp to generate a PDF. Thanks. Ben

    Read the article

  • Creating a second form page for a has_many relationship

    - by Victor Martins
    I have an Organization model that has_many users through affiliations. And, in the form of the organization ( the standard edit ) I use semanting_form_for and semantic_fields_for to display the organization fields and affiliations fields. But I wish to create a separete form just to handle the affiliations of a specific organization. I was trying to go to the Organization controller and create a an edit_team and update_team methods then on the routes create those pages, but it's getting a mess and not working. am I on the right track?

    Read the article

  • Sphinx, reStructuredText show\hide code snippets

    - by Adam Matan
    Hi, I've been documenting a software package using Sphinx and reStructuredText. Within my documents, there are some long code snippets. I want to be able to have them hidden as default, with a little "Show\Hide" button that would expand them (Example). Is there a standard way to do that? If not, I think I will suggest this feature to the developers. Thanks, Adam

    Read the article

  • Showing windows (file/directory) context menu in c# winforms application

    - by Axarydax
    Hi, I'm trying to display standard file/directory context menu in WinForms application - I'd like to enable the user to do things with files in my application as he could in Explorer. There should be two options popup the context menu with some system call read everything from the system context menu and insert it into my context menu Could you please point me in the right direction? Thanks

    Read the article

< Previous Page | 151 152 153 154 155 156 157 158 159 160 161 162  | Next Page >