Search Results

Search found 29949 results on 1198 pages for 'large scale website'.

Page 165/1198 | < Previous Page | 161 162 163 164 165 166 167 168 169 170 171 172  | Next Page >

  • Simulating user activities on a GMail page

    - by Vitaliy
    I create a program that simulates me browsing to gmail, entering the user name and password and clicking the submit button. All this with C#. I would appreciate two kinds of answers: One that tells how to do this programaticaly. Since I may be interested in automating more sophisticated user activities. On that tells me about a program that already does that. Thanks!!

    Read the article

  • Python: How to read huge text file into memory

    - by asmaier
    I'm using Python 2.6 on a Mac Mini with 1GB RAM. I want to read in a huge text file $ ls -l links.csv; file links.csv; tail links.csv -rw-r--r-- 1 user user 469904280 30 Nov 22:42 links.csv links.csv: ASCII text, with CRLF line terminators 4757187,59883 4757187,99822 4757187,66546 4757187,638452 4757187,4627959 4757187,312826 4757187,6143 4757187,6141 4757187,3081726 4757187,58197 So each line in the file consists of a tuple of two comma separated integer values. I want to read in the whole file and sort it according to the second column. I know, that I could do the sorting without reading the whole file into memory. But I thought for a file of 500MB I should still be able to do it in memory since I have 1GB available. However when I try to read in the file, Python seems to allocate a lot more memory than is needed by the file on disk. So even with 1GB of RAM I'm not able to read in the 500MB file into memory. My Python code for reading the file and printing some information about the memory consumption is: #!/usr/bin/python # -*- coding: utf-8 -*- import sys infile=open("links.csv", "r") edges=[] count=0 #count the total number of lines in the file for line in infile: count=count+1 total=count print "Total number of lines: ",total infile.seek(0) count=0 for line in infile: edge=tuple(map(int,line.strip().split(","))) edges.append(edge) count=count+1 # for every million lines print memory consumption if count%1000000==0: print "Position: ", edge print "Read ",float(count)/float(total)*100,"%." mem=sys.getsizeof(edges) for edge in edges: mem=mem+sys.getsizeof(edge) for node in edge: mem=mem+sys.getsizeof(node) print "Memory (Bytes): ", mem The output I got was: Total number of lines: 30609720 Position: (9745, 2994) Read 3.26693612356 %. Memory (Bytes): 64348736 Position: (38857, 103574) Read 6.53387224712 %. Memory (Bytes): 128816320 Position: (83609, 63498) Read 9.80080837067 %. Memory (Bytes): 192553000 Position: (139692, 1078610) Read 13.0677444942 %. Memory (Bytes): 257873392 Position: (205067, 153705) Read 16.3346806178 %. Memory (Bytes): 320107588 Position: (283371, 253064) Read 19.6016167413 %. Memory (Bytes): 385448716 Position: (354601, 377328) Read 22.8685528649 %. Memory (Bytes): 448629828 Position: (441109, 3024112) Read 26.1354889885 %. Memory (Bytes): 512208580 Already after reading only 25% of the 500MB file, Python consumes 500MB. So it seem that storing the content of the file as a list of tuples of ints is not very memory efficient. Is there a better way to do it, so that I can read in my 500MB file into my 1GB of memory?

    Read the article

  • Using Hibernate's ScrollableResults to slowly read 90 million records

    - by at
    I simply need to read each row in a table in my MySQL database using Hibernate and write a file based on it. But there are 90 million rows and they are pretty big. So it seemed like the following would be appropriate: ScrollableResults results = session.createQuery("SELECT person FROM Person person") .setReadOnly(true).setCacheable(false).scroll(ScrollMode.FORWARD_ONLY); while (results.next()) storeInFile(results.get()[0]); The problem is the above will try and load all 90 million rows into RAM before moving on to the while loop... and that will kill my memory with OutOfMemoryError: Java heap space exceptions :(. So I guess ScrollableResults isn't what I was looking for? What is the proper way to handle this? I don't mind if this while loop takes days (well I'd love it to not). I guess the only other way to handle this is to use setFirstResult and setMaxResults to iterate through the results and just use regular Hibernate results instead of ScrollableResults. That feels like it will be inefficient though and will start taking a ridiculously long time when I'm calling setFirstResult on the 89 millionth row... UPDATE: setFirstResult/setMaxResults doesn't work, it turns out to take an unusably long time to get to the offsets like I feared. There must be a solution here! Isn't this a pretty standard procedure?? I'm willing to forgo Hibernate and use JDBC or whatever it takes. UPDATE 2: the solution I've come up with which works ok, not great, is basically of the form: select * from person where id > <offset> and <other_conditions> limit 1 Since I have other conditions, even all in an index, it's still not as fast as I'd like it to be... so still open for other suggestions..

    Read the article

  • Auto load a specific link at various time intervals

    - by user228837
    Here's what I need to do. I'm using Google Chrome. I have page that auto-reloads every 5 seconds using a script: javascript: timeout=prompt("Set timeout [s]"); current=location.href; if(timeout>0) setTimeout('reload()',1000*timeout); else location.replace(current); function reload() { setTimeout('reload()',1000*timeout); fr4me='<frameset cols=\'*\'>\n<frame src=\''+current+'\'/>'; fr4me+='</frameset>'; with(document){write(fr4me);void(close())}; } I found that script by Googling. The reason why the page auto-reloads every 5 seconds is I'm waiting for a specific link or url to appear in the page. It appears at random times. Once I see the link I'm waiting for, I immediately click the link. That's fine. But I want more. What I want is the page will auto-reload and I want it to auto-detect the the link I'm waiting for. Once the script finds the link I'm waiting for, it automatically loads that link on a new tab or page. For example, I'm auto-reloading www.example.com. I'm waiting for a specific url "BUY NOW". When the page auto-reloads, it checks if there's a url "BUY NOW". If it sees one, it should automatically open that link. Thanks.

    Read the article

  • How to update multiple elements with one MooTools Request.HTML call

    - by Mario
    Does anyone know if, using one Request.HTML call from MooTools, it is possible to somehow update more than one element in a webpage? The current call I have is: var req = new Request.HTML({update: $('content')}).get('../latest_events'); This updates the content div in my page with the "../latest_events" page. Is there a way to update other divs with the "../latest_events" page using this same call, or do I have to just use separate calls?

    Read the article

  • Does whitespace in the title tag affect SEO?

    - by amelvin
    The site I'm working on uses Umbraco and has xslt macros to generate dynamic page title tags - but the title tags generated contain lots of whitespace and linefeeds. Now these macros can be changed so I'm sure that the contents of the title tag can be condensed, but at this stage of development we'd rather not do any work that is not essential. I've checked W3 and Google but I'm struggling to find something conclusive on whitespace. So I'd like to ask is a title tag formatted like this: <title> Sitename - The official blah blah blah - Section - Section Search Results </title> any worse for SEO than: <title>Sitename - The official blah blah blah - Section - Section Search Results</title> ... and are there any other implications to leaving the title tag with whitespace in it?

    Read the article

  • How to upload a huge(GBs) file to a webserver

    - by Aman Jain
    Hi I want to create a web interface that will allow users to upload huge files. These files are actually vmdk files(Virtual Machine Disks), and they can be multi GB in size. I think this makes the situation different from normal file uploads(10-20MB file uploads). So any advise on how to achieve the same in an efficient and fault tolerant way, is appreciated. Thanks Aman Jain

    Read the article

  • Redirects or Host Headers for domain names & WWW

    - by Rob
    If you web site address is: http://www.example.org should you set up an IIS web site for http://example.org with a permanent redirect to the www address or simply add the second Url as a host header? If you where to add the host header would that affect SEO?

    Read the article

  • Geopraphic Charting controls for Websites

    - by Ian
    Hi All, I need to create dash boards showing geographic regions and show sales, hot spots etc on a map. What have you tried and what do you recommend? I like the look of both Fusion Charts and Dundas I will be using asp.net for the site but any control's or library's including flash or javascript are good options. Most important is the look and feel followed by functionality in South Africa. After my last post looking for commercial mapping solutions, it looks like they are very expensive and now I am investigating alternatives to full mapping solutions. thanks

    Read the article

  • webpage accessibility

    - by amit upadhyay
    hi, i have a strange requirement. I want to access a page on my site through a link say www.abc.com/downloads/file.txt but i also want that if anybody enters only www.abc.com/downloads it should not be accessible or it should display access denied. How can i do that???????

    Read the article

  • Storing varchar(max) & varbinary(max) together - Problem?

    - by Tony Basallo
    I have an app that will have entries of both varchar(max) and varbinary(max) data types. I was considering putting these both in a separate table, together, even if only one of the two will be used at any given time. The question is whether storing them together has any impact on performance. Considering that they are stored in the heap, I'm thinking that having them together will not be a problem. However, the varchar(max) column will be probably have the text in row table option set. I couldn't find any performance testing or profiling while "googling bing," probably too specific a question? The SQL Server 2008 table looks like this: Id ParentId Version VersionDate StringContent - varchar(max) BinaryContent - varbinary(max) The app will decide which of the two columns to select for when the data is queried. The string column will much used much more frequently than the binary column - will this have any impact on performance?

    Read the article

  • ASP.NET 3.5 Web Site stopped importing System namespace by default

    - by Alexis
    I have a VB Web Site project that has recently (and mysteriously) stopped importing the "System" namespace by default. I'm having to either place a "Imports System" line at the top of each code behind, or preface everything with "System", which is fairly annoying, not to mention redundant. I can't for the life of me figure out how to get the System namespace back to being imported by default. I've already checked to see that WINDOWS\Microsoft.NET\Framework\v2.0.50727\CONFIG\web.config contains the <add namespace="System"/> line--it does. That was my best lead. I'm tearing my hair out. Does anyone have any suggestions?

    Read the article

  • Tomcat Exception-Type Ignoring Specific Exception for More General

    - by David Marks
    For one type of exception, IOException, I want to display one page. For all other exceptions I have a default error page. In my web.xml I have things setup like this: java.io.IOException /queryException.jsp java.lang.Exception /error.jsp The problem is the error.jsp is the only page that ever shows, even if an IOException is thrown. The order the tags appear in doesn't matter; if I remove the java.lang.Exception tag though, I can get queryException to show for IOExceptions. What is the solution here? How can I keep a general error page for all exceptions EXCEPT for those with specific pages?

    Read the article

  • Importing wikipedia database dumb - kills navicat - anyone got any ideas?

    - by Ali
    Ok guys I've downloaded the wikipedia xml dump and its a whopping 12 GB of data :\ for one table and I wanted to import it into mysql databse on my localhost - however its a humongous file 12GB and obviously navicats taking its sweet time in importing it or its more likely its hanged :(. Is there a way to include this dump or atleast partially at most you know bit by bit. Let me correct that its 21 GB of data - not that it helps :\ - does any one have any idea of importing humongous files like this into MySQL database.

    Read the article

  • How to protect/monitor your site from crawling by malicious user

    - by deathy
    Situation: Site with content protected by username/password (not all controlled since they can be trial/test users) a normal search engine can't get at it because of username/password restrictions a malicious user can still login and pass the session cookie to a "wget -r" or something else. The question would be what is the best solution to monitor such activity and respond to it (considering the site policy is no-crawling/scraping allowed) I can think of some options: Set up some traffic monitoring solution to limit the number of requests for a given user/IP. Related to the first point: Automatically block some user-agents (Evil :)) Set up a hidden link that when accessed logs out the user and disables his account. (Presumably this would not be accessed by a normal user since he wouldn't see it to click it, but a bot will crawl all links.) For point 1. do you know of a good already-implemented solution? Any experiences with it? One problem would be that some false positives might show up for very active but human users. For point 3: do you think this is really evil? Or do you see any possible problems with it? Also accepting other suggestions.

    Read the article

  • Point driven forum

    - by Dofs
    Hi, A while a go I saw a good post on SO about a user who wanted to create a site using a point system similar to SO, but I havn't been able to find it since. Can anyone direct me to a good article or post about creating point driven sites/forums?

    Read the article

  • How would you handle making an array or list that would have more entries than the standard implemen

    - by faceless1_14
    I am trying to create an array or list that could handle in theory, given adequate hardware and such, as many as 100^100 BigInteger entries. The problem with using an array or standard list is that they can only hold Integer.MAX_VALUE number of entries. How would you work around this limitations? A whole new class/interface? A wrapper for list? another data type entirely?

    Read the article

  • AnkhSVN: No Versioned Parent Directories when adding a Web Site

    - by CJM
    I've recently been trying out Ankhsvn and I quite like it, and more importantly, it generally likes me. But I've hit a problem adding existing VS projects to subversion using AnkhSVN. Most Windows and Web projects are added fine, but I have a problem when I try to a Websites (as opposed to Web Applications), and in these cases I get the following error: No versioned parent directories After some googling, I've come across other people with the same problems and there have been one or two suggests for solving the problem, but non so far have worked. AnkhSVN is popular enough, and although web applications are often preferred to websites these days, I'm hoping that someone here has a ready solution available.

    Read the article

  • registration 0.8 alpha activation problem

    - by craphunter
    Got the following error: Exception Type: TypeError at /accounts/account/activate/success/ Exception Value: activate() takes at least 2 non-keyword arguments (1 given) My view: def activate(request, backend, template_name='registration/activation_complete.html', success_url=None, extra_context=None, **kwargs): backend = get_backend(backend) account = backend.activate(request, **kwargs) if account: if success_url is None: to, args, kwargs = backend.post_activation_redirect(request, account) return redirect(to, *args, **kwargs) else: return redirect(success_url) if extra_context is None: extra_context = {} context = RequestContext(request) for key, value in extra_context.items(): context[key] = callable(value) and value() or value return render_to_response(template_name, kwargs, context_instance=context) My url: urlpatterns = patterns('', url(r'^activate/complete/$', direct_to_template, { 'template': 'registration/activation_complete.html' }, name='registration_activation_complete'), # Activation keys get matched by \w+ instead of the more specific # [a-fA-F0-9]{40} because a bad activation key should still get to the view; # that way it can return a sensible "invalid key" message instead of a # confusing 404. url(r'^activate/(?P<activation_key>\w+)/$', activate, { 'backend': 'registration.backends.default.DefaultBackend' }, name='registration_activate'), url(r'^register/$', register, { 'backend': 'registration.backends.default.DefaultBackend' }, name='registration_register'), url(r'^register/complete/$', direct_to_template, { 'template': 'registration/registration_complete.html' }, name='registration_complete'), url(r'^register/closed/$', direct_to_template, { 'template': 'registration/registration_closed.html' }, name='registration_disallowed'), (r'', include('registration.auth_urls')), url(r'^account/activate/(?P<activation_key>\w+)/$', 'registration.views.activate', {'success_url': 'account/activate/success/'}, name='registration_activate2'), url(r'^account/activate/success/$', direct_to_template, {'template': 'registration/activation_complete.html'}, name='registration_activation_complete'), ) What do I do wrong? Thanks!

    Read the article

  • CSS Vertical Spacing Issue (Margin?) - Vertical Space Between Two Divs

    - by Sootah
    I've got the layout of my site mostly done. On the domain I'm testing this layout on - http://networkgenius.org/ - there is a vertical space separating the content-wrapper div and the menu div. I have the margins set to 0px for the vertical spacing, and this is the only area that is separated like it is. I'd like the white content area to be pressed up right against the bottom of the menu, so that none of the body's background shows between them. What am I doing wrong?

    Read the article

< Previous Page | 161 162 163 164 165 166 167 168 169 170 171 172  | Next Page >