Search Results

Search found 9017 results on 361 pages for 'efficient storage'.

Page 259/361 | < Previous Page | 255 256 257 258 259 260 261 262 263 264 265 266  | Next Page >

• What's the fastest way to scrape a lot of pages in php?

  - by Yegor

  I have a data aggregator that relies on scraping several sites, and indexing their information in a way that is searchable to the user. I need to be able to scrape a vast number of pages, daily, and I have ran into problems using simple curl requests, that are fairly slow when executed in rapid sequence for a long time (the scraper runs 24/7 basically). Running a multi curl request in a simple while loop is fairly slow. I speeded it up by doing individual curl requests in a background process, which works faster, but sooner or later the slower requests start piling up, which ends up crashing the server. Are there more efficient ways of scraping data? perhaps command line curl?

  Read the article

• C++ Thread Safe Integer

  - by Paul Ridgway

  Hello everyone, I have currently created a C++ class for a thread safe integer which simply stores an integer privately and has public get a set functions which use a boost::mutex to ensure that only one change at a time can be applied to the integer. Is this the most efficient way to do it, I have been informed that mutexes are quite resource intensive? The class is used a lot, very rapidly so it could well be a bottleneck... Googleing C++ Thread Safe Integer returns unclear views and oppinions on the thread safety of integer operations on different architectures. Some say that a 32bit int on a 32bit arch is safe, but 64 on 32 isn't due to 'alignment' Others say it is compiler/OS specific (which I don't doubt). I am using Ubuntu 9.10 on 32 bit machines, some have dual cores and so threads may be executed simultaneously on different cores in some cases and I am using GCC 4.4's g++ compiler. Thanks in advance...

  Read the article

• Get value from ajax-loaded form before submitting

  - by ldvldv

  Hi, I have an app that loads an ajax form when a button is clicked. I then want to check a value in that ajax loaded form called "comment_content" so that I don't send an ajax request if the comment_content is empty. I can capture the form submission using the live method, however, I can't capture the form element because it needs an element. I was looking at livequery but was wondering if there is a more efficient way by using live or if that I shouldn't even bother? Your help is very much appreciated. $("#comment_submit").live('click', function(){ var comment = $("#comment_content").html(); if(comment == ""){ $("#notice").html("").stop(true, true); $("#notice").append("<p>Are you going to write something in that comment?</p>"); $('#notice').slideDown().delay(2000).slideUp(); return false; } return true; });

  Read the article

• Collision detection by sliding against a plane in XNA

  - by Bevin

  Hello, I am attempting to engineer a collision detection algorithm for a custom Minecraft client I'm making. Basically, the entire world is made up of cubes, and the player (or camera) needs to be able to stand on and move against these cubes. The result I want is illustrated in this image: The green line is the player's movement vector. When the player is brushing up against a plane of one of the cubes, I want the vector to change to one that is perpendicular with the plane. The vector should, however, keep all of it's velocity in the plane's direction, yet lose all velocity towards the plane. I hope I've made my question clear. What is the best and most efficient way to implement a collision detection system like this? Also, will a system like this allow for a simple gravity component?

  Read the article

• Time complexity O() of isPalindrome()

  - by Aran

  I have this method, isPalindrome(), and I am trying to find the time complexity of it, and also rewrite the code more efficiently. boolean isPalindrome(String s) { boolean bP = true; for(int i=0; i<s.length(); i++) { if(s.charAt(i) != s.charAt(s.length()-i-1)) { bP = false; } } return bP; } Now I know this code checks the string's characters to see whether it is the same as the one before it and if it is then it doesn't change bP. And I think I know that the operations are s.length(), s.charAt(i) and s.charAt(s.length()-i-!)). Making the time-complexity O(N + 3), I think? This correct, if not what is it and how is that figured out. Also to make this more efficient, would it be good to store the character in temporary strings?

  Read the article

• Determining whether a file is a duplicate

  - by Todd R

  Is there a reliable way to determine whether or not two files are the same? For example, two files with the same size and type may or may not be the same binarilly (yeah, I know it's not really a word). I assume that comparing one or two checksums of the files will help, but I wonder: How reliable are checksums at determining whether two files are different; what are the chances of two different files having the same checksum? Would reliability increase by applying additional checksum comparisons? Which checksum algorithm(s) would be the most efficient and/or reliable? Any ideas, suggestions or thoughts are appreciated! P.S. The code for this is being written in Java running on a nix system, but generic or platform agnostic input is most helpful.

  Read the article

• Creating a "mountable" File System, where to start?

  - by Mike Curry

  A friend and I are thinking about creating a simple file system for learning purposes. We're going to write it in C/C++, and try to get it to a mountable state from within linux. We've both been coding or over 16 years (32 combined), so I suppose its just a matter of finding some documentation, and a ton of learning. My question is, where could I find out more information? (Documentation for creating a file system, requirements of mounting a file system in linux, etc) Where do we start? Edit: I should also mention, this would not be a boot-able file system, just a file system used for storage, though I am not too sure if that matters or not.

  Read the article

• Store data in file system rather than SQL or Oracle database.

  - by nunu

  Hi All, As I am working on Employee Management system, I have two table (for example) in database as given below. EmployeeMaster (DB table structure) EmployeeID (PK) | EmployeeName | City MonthMaster (DB table structure) Month | Year | EmployeeID (FK) | PrenentDays | BasicSalary Now my question is, I want to store data in file system rather than storing data in SQL or ORACLE. I want my data in file system storage for Insert, Edit and Delete opration with keeping relation with objects too. I am a C# developer, Could anybody have thoughts or idea on it. (To store data in file system with keeping relations between them) Thanks in advance. Any ideas on it?

  Read the article

• With the attachment_fu rails plugin, is there any way to delete files uploaded to Amazon S3?

  - by Eric Nguyen

  Let's say I'm using attachment_fu to attach profile pics to user profiles in a system, with Amazon S3 used as the actual file storage. When users upload new profile pics, I'd like to replace the attached file with the new one. I can do this within my database (i.e. the file metadata) easily, but attachment_fu doesn't seem to provide methods for deleting the files from S3. Am I missing something, or am I approaching this the wrong way? Many thanks!

  Read the article

• Is there an optimal way to render images in cocoa? Im using setNeedsDisplay

  - by Edward An

  Currently, any time I manually move a UIImage (via handling the touchesMoved event) the last thing I call in that event is [self setNeedsDisplay], which effectively redraws the entire view. My images are also being animated, so every time a frame of animation changes, i have to call setNeedsDisplay. I find this to be horrific since I don't expect iphone/cocoa to be able to perform such frequent screen redraws very quickly. Is there an optimal, more efficient way that I could be doing this? Perhaps somehow telling cocoa to update only a particular region of the screen (the rect region of the image)?

  Read the article

• smallest perimiter rectangle with given integer area and integer sides

  - by remuladgryta

  Given an integer area A, how can one find integer sides w and h of a rectangle such that w*h = A and w+h is as small as possible? I'd rather the algorithm be simple than efficient (although within reasonable efficiency). What would be the best way to accomplish this? Finding out the prime factors of A, then combining them in some way that tries to balance w and h? Finding the two squares with integer sides with areas closest to A and then somehow interpolating between them? Any other method i'm not thinking of?

  Read the article

• PHP/MySQL: Storing and retrieving UUIDS

  - by Greg

  I'm trying to add UUIDs to a couple of tables, but I'm not sure what the best way to store/retrieve these would be. I understand it's far more efficient to use BINARY(16) instead of VARCHAR(36). After doing a bit of research, I also found that you can convert a UUID string to binary with: UNHEX(REPLACE(UUID(),'-','')) Pardon my ignorance, but is there an easy way to this with PHP and then turn it back to a string, when needed, for readability? Also, would it make much difference if I used this as a primary key instead of auto_increment? EDIT: Found part of the answer: $bin = pack("h*", str_replace('-', '', $guid)); How would you unpack it?

  Read the article

• Rails diff model config in dev or prod environment

  - by Denis

  Hi, I've a model which use paperclip, in dev env I want to store files on the file system. In production I want to store them on my s3 account. How to configure my model to reflet this difference? Here is my model class Photo < ActiveRecord::Base has_attached_file :photo, :styles => { :medium => "200x200>", :thumb => "100x100>" }, :storage => :s3, :s3_credentials => "#{Rails.root}/config/s3.yml", :path => "/:style/:filename" end

  Read the article

• Write data into .txt file created by CFileDialog, in C++

  - by younevertell

  I wanna Write data into .txt file created by CFileDialog, in C++. The problem I am facing is that below codes doesn't work, although there is no build error. The .txt file created by CFileDialog can not be found for some reason. What's wrong the code? what's the efficient way to Write data into .txt file created by CFileDialog, in C++? Thanks CFileDialog dlg(FALSE, NULL, NULL, OFN_OVERWRITEPROMPT, _T("My Data File (*.txt)|*.txt||")); if(dlg.DoModal() != IDOK) return; CString filename = dlg.GetPathName(); ofstream outfile (filename); int mydata = 10; outfile << "my data:" << mydata << endl; outfile.close();

  Read the article

• Exploring search options for PHP

  - by Joshua

  I have innoDB table using numerous foreign keys, but we just want to look up some basic info out of it. I've done some research but still lost. 1) How can I tell if my host has Sphinx installed already? I don't see it as an option for table storage method (i.e. innodb, myisam). 2) Zend_Search_Lucene, responsive enough for AJAX functionality of millions of records? 3) Mirror my innoDB with a myisam? Make every innodb transaction end with a write to the myisam, then use 1:1 lookups? How would I do this automagically? This should make MyISAM ACID-compliant and free(er) from corruption no? 4) PostgreSQL fulltext queries don't even look like SQL to me wtf, I don't have time to learn a new SQL syntax I need noob options 5) ???????????????????? This is high volume site on a decently-equipped VPS Thanks very much for any ideas.

  Read the article

• Python: Huge file reading by using linecache Vs normal file access open()

  - by user335223

  Hi, I am in a situation where multiple threads reading the same huge file with mutliple file pointers to same file. The file will have atleast 1 million lines. Eachline's length varies from 500 characters to 1500 characters. There won't "write" operations on the file. Each thread will start reading the same file from different lines. Which is the efficient way..? Using the Python's linecache or normal readline() or is there anyother effient way?

  Read the article

• Is it possible to transfer data between html pages driven by spring web flow?

  - by Easwaramoorthy Kanagaraj

  I am aware of passing data between jsp in spring web flow. Is it possible to transfer data between html pages driven by spring web flow. I don't want to use the HTML5 local storage capabilities. Example: Page 1: Search box for an employee id. Page 2: Search result for the employee details. Two ways that I could think: Get the employee details in page 1 by ajax and pass the result to the page two. Pass the employee id to page 2 and get the result by ajax in onload. In both case I need to pass any variable/data. I am confused in doing this. Is there anything in the Spring webflow using which I could do this? Thanks in advance, Easwar

  Read the article

• Servlet 3 spec and ThreadLocal

  - by mindas

  As far as I know, Servlet 3 spec introduces asynchronous processing feature. Among other things, this will mean that the same thread can and will be reused for processing another, concurrent, HTTP request(s). This isn't revolutionary, at least for people who worked with NIO before. Anyway, this leads to another important thing: no ThreadLocal variables as a temporary storage for the request data. Because if the same thread suddenly becomes the carrier thread to a different HTTP request, request-local data will be exposed to another request. All of that is my pure speculation based on reading articles, I haven't got time to play with any Servlet 3 implementations (Tomcat 7, GlassFish 3.0.X, etc.). So, the questions: Am I correct to assume that ThreadLocal will cease to be a convenient hack to keep the request data? Has anybody played with any of Servlet 3 implementations and tried using ThreadLocals to prove the above? Apart from storing data inside HTTP Session, are there any other similar easy-to-reach hacks you could possibly advise?

  Read the article

• C# - Inserting multiple rows using a stored procedure

  - by user177883

  I have a list of objects, this list contains about 4 million objects. there is a stored proc that takes objects attributes as params , make some lookups and insert them into tables. what s the most efficient way to insert this 4 million objects to db? How i do : -- connect to sql - SQLConnection ... foreach(var item in listofobjects) { SQLCommand sc = ... // assign params sc.ExecuteQuery(); } THis has been really slow. is there a better way to do this?

  Read the article

• Any strategies for assessing the trade-off between CPU loss and memory gain from compression of data

  - by indiehacker

  Are very large TextProperties a burden? Should they be compressed? Say I have a information stored in 2 attributes of type TextProperty in my datastore entities. The strings are always the same length of 65,000 characters and have lots of repeating integers, a sample appearing as follows: entity.pixel_idx = 0,0,0,0,0,0,0,0,0,1,1,1,1,1,1,1,1,1,1,1,5,5,5,5,5,5,5,5,5,5,5,5....etc. entity.pixel_color = 2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,1,1,1,1,1,1,1,1,1,1,1,1,...etc. So these above could also be represented using much less storage memory by compressing say using only each integer and the length of its series ( '0,8' for '0,0,0,0,0,0,0,0') but then its takes time and CPU to compress and decompress? Any general ideas? Are there some tricks for testing different attempts to the problem?

  Read the article

• How to initialise a STL vector/list with a class without invoking the copy constructor

  - by Warpspace

  I have a C++ program that uses a std::list containing instances of a class. If I call e.g. myList.push_back(MyClass(variable)); it goes through the process of creating a temporary variable, and then immediately copies it to the vector, and afterwards deletes the temporary variable. This is not nearly as efficient as I want, and sucks when you need a deep copy. I would love to have the constructor of my class new something and not have to implement a copy constructor just to allocate my memory for the second time and waste runtime. I'd also rather not have to immediately find the class instance from the vector/list and then manually allocate the memory (or do something horrible like allocate the memory in the copy constructor itself). Is there any way around this (I'm not using Visual Studio BTW)?

  Read the article

• Theory of Game Interface Design

  - by anon

  Anyone know of a good book on Game Interface Design (not game play mechanics; the actual UI). I'm particular interested in theories of cognition, and how game interfaces are designed to allow the enduser efficient communication with the game (whether it in FPS, RTS, or so on). In a modern game, the amount of information conveyed to the user, the amount of choices the user can make; and the support for the user to make said decisions is simply astounding (think UIs for Starcraft II / WoW). Any insights into this would be greatly appreciated.

  Read the article

• Treating a fat webservice in .net 3.5 c#

  - by Chris M

  I'm dealing with an obese 3rd party webservice that returns about 3mb of data for a simple search results, about 50% of the data in that response is junk. Would it make sense then to remap this data to my own result object and ditch the response so I'm storing 1-2 mb in memory for filtering and sorting rather than using the web-responses own object and using 2-4 or am I missing a point? So far I've been accessing the webservice from a separate project and using a new class to provide the interaction and to handle the persistence so my project looks like this |- Web (mvc2 proj) |- DAL (database/storage fluent-nhibernate) |- SVCGateway (interaction layer + webservice related models) |- Services -------------- |- Tests |- Specs I'm trying to make the application behave fast and I also need to store the result set temporarily in case a customer goes to view the product and wants to go back to the results. (Service returns only 500 of possible 14K results). So basically I'm looking for confirmation that I'm doing the right thing in pushing the results into my own objects or if I'm breaking some rule or even if there's a better way of handling it. Thanks

  Read the article

• Prims vs Polys: what are the pros and cons of each?

  - by Richard Inglis

  I've noticed that most 3d gaming/rendering environments represent solids as a mesh of (usually triangular) 3d polygons. However some examples, such as Second Life, or PovRay use solids built from a set of 3d primitives (cube, sphere, cone, torus etc) on which various operations can be performed to create more complex shapes. So my question is: why choose one method over the other for representing 3d data? I can see there might be benefits for complex ray-tracing operations to be able to describe a surface as a single mathematical function (like PovRay does), but SL surely isn't attempting anything so ambitious with their rendering engine. Equally, I can imagine it might be more bandwidth-efficient to serve descriptions of generalised solids instead of arbitrary meshes, but is it really worth the downside that SL suffers from (ie modelling stuff is really hard, and usually the results are ugly) - was this just a bad decision made early in SL's development that they're now stuck with? Or is it an artefact of what's easiest to implement in OpenGL?

  Read the article

• Facebook user_id as MongoDB BSON ObjectId?

  - by MattDiPasquale

  I'm rebuilding Lovers on Facebook with Sinatra & Redis. I like Redis because it doesn't have the long (12-byte) BSON ObjectIds and I am storing sets of Facebook user_ids for each user. The sets are requests_sent, requests_received, & relationships, and they all contain Facebook user ids. I'm thinking of switching to MongoDB because I want to use it's geospatial indexing. If I do, I'd want to use the FB user ids as the _id field because I want the sets to be small and I want the JSON responses to be small. But, is the BSON ObjectId better (more efficient for MongoDB) to use than just an integer (fb user_id)?

  Read the article

< Previous Page | 255 256 257 258 259 260 261 262 263 264 265 266  | Next Page >