Search Results

Search found 65101 results on 2605 pages for 'big data'.

Page 39/2605 | < Previous Page | 35 36 37 38 39 40 41 42 43 44 45 46  | Next Page >

  • How can I store all my level data in a single file instead of spread out over many files?

    - by Jon
    I am currently generating my level data, and saving to disk to ensure that any modifications done to the level are saved. I am storing "chunks" of 2048x2048 pixels into a file. Whenever the player moves over a section that doesn't have a file associated with the position, a new file is created. This works great, and is very fast. My issue, is that as you are playing the file count gets larger and larger. I'm wondering what are techniques that can be used to alleviate the file count, without taking a performance hit. I am interested in how you would store/seek/update this data in a single file instead of multiple files efficiently.

    Read the article

  • What is the most complicated data structure you have used in a practical situation?

    - by Fanatic23
    The germ for this question came up from a discussion I was having with couple of fellow developers from the industry. Turns out that in a lot of places project managers are wary about complex data structures, and generally insist on whatever exists out-of-the-box from standard library/packages. The general idea seems to be like use a combination of whats already available unless performance is seriously impeded. This helps keeping the code base simple, which to the non-diplomatic would mean "we have high attrition, and newer ones we hire may not be that good". So no bloom filter or skip-lists or splay trees for you CS junkies. So here's the question (again): Whats the most complicated data structure you did or used in office? Helps get a sense of how good/sophisticated real world software are.

    Read the article

  • How can I protect my save data from casual hacking?

    - by Danran
    What options are there for saving game data in a secure manner? I'm interested in solutions specifically tailored for C++. I'm looking for something that is fast and easy to use. I'm only concerned about storing simple information such as Which levels are and are not unlocked The user's score for each level I'm curious again to know what's out there to use, any good libraries to use that give me nice, secure game data files that the average player can't mess with. I just found this here which looks very nice, but it would be great to get some opinions on potential other libraries/options out there.

    Read the article

  • What data-structure/algorithm will allow me to send a list of key/value dictionaries using the least amount of bits?

    - by user12365
    I have server objects that have corresponding client objects. The data to be kept in sync is inside the server object's key/value dictionary. To keep the client objects in sync with the sever objects, I want the server to send the key/value dictionary every frame for each object. What data-structure/algorithm will allow me to send a list of key/value dictionaries using the least amount of bits? Bonus constraint 1: For each type of object, the values of some keys change more often than others. Bonus constraint 2: Memory usage on the server side is relatively expensive.

    Read the article

  • Data structures for storing finger/stylus movements in drawing application?

    - by mattja?øb
    I have a general question about creating a drawing application, the language could be C++ or ObjectiveC with OpenGL. I would like to hear what are the best methods and practices for storing strokes data. Think of the many iPad apps that allow you to draw with your finger (or a stylus) or any other similar function on a desktop app. To summarize, the data structure must: be highly responsive to the movement store precise values (close in space / time) usable for rendering the strokes with complex textures (textures based on the dynamic of the stroke etc) exportable to a text file for saving/loading

    Read the article

  • How does Google store search trends in backend?

    - by Achshar
    Google trends shows what query has been searched how many times and some other properties of the said query. But how is this data stored in a database? Storing a new row for every search does not seem right. They also tell the query on a time graph, so they must have some way to look for individual searches made by users, but the number of queries they get every day, it does not feel right that they would store every search in a database row along with a time-stamp. This does not apply to just Google trends or Google in general but any other big site that gets awful number of queries and then has tools to see them in depth. I am not an expert on this but I am interested to know some high level structure of how things work behind the scenes.

    Read the article

  • Android: Having trouble creating a subclass of application to share data with multiple Activities

    - by Mike
    Hello, I just finished a couple of activities in my game and now I was going to start to wire them both up to use real game data, instead of the test data I was using just to make sure each piece worked. Since multiple Activities will need access to this game data, I started researching the best way to pass this data to my Activities. I know about using putExtra with intents, but my GameData class has quite a bit of data and not just simple key value pairs. Besides quite a few basic data types, it also has large arrays. I didn't really want to try and pass all that, unless I can pass the entire object, instead of just key/data pairs. I read the following post and thought it would be the way to go, but so far, I haven't got it to work. Android: How to declare global variables? I created a simple test app to try this method out, but it keeps crashing and my code seems to look the same as in the post above - except I changed the names. Here is the error I am getting. Can someone help me understand what I am doing wrong? 12-23 00:50:49.762: ERROR/AndroidRuntime(608): Caused by: java.lang.ClassCastException: android.app.Application It crashes on the following statement: GameData newGameData = ((GameData)getApplicationContext()); Here is my code: package mrk.examples.StaticGameData; import android.app.Application; public class GameData extends Application { private int intTest; GameData () { intTest = 0; } public int getIntTest(){ return intTest; } public void setIntTest(int value){ intTest = value; } } // My main activity package mrk.examples.StaticGameData; import android.app.Activity; import android.content.Intent; import android.os.Bundle; import android.util.Log; public class StaticGameData extends Activity { int intStaticTest; /** Called when the activity is first created. */ @Override public void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); setContentView(R.layout.main); GameData newGameData = ((GameData)getApplicationContext()); newGameData.setIntTest(0); intStaticTest = newGameData.getIntTest(); Log.d("StaticGameData", "Well: IntStaticTest = " + intStaticTest); newGameData.setIntTest(1); Log.d("StaticGameData", "Well: IntStaticTest = " + intStaticTest + " newGameData: " + newGameData.getIntTest()); Intent intentNew = new Intent(this, PassData2Activity.class); startActivity (intentNew); } } // My test Activity to see if it can access the data and its previous state from the last activity package mrk.examples.StaticGameData; import android.app.Activity; import android.os.Bundle; import android.util.Log; public class PassData2Activity extends Activity { @Override public void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); setContentView(R.layout.main); GameData gamedataPass = ((GameData)getApplicationContext()); Log.d("PassData2Activity", "IntTest = " + gamedataPass.getIntTest()); } } Below is the relevant portion of my manifest: <application android:icon="@drawable/icon" android:label="@string/app_name"> <activity android:name=".StaticGameData" android:label="@string/app_name"> <intent-filter> <action android:name="android.intent.action.MAIN" /> <category android:name="android.intent.category.LAUNCHER" /> </intent-filter> </activity> <activity android:name=".PassData2Activity"></activity> </application> <application android:name=".GameData" android:icon="@drawable/icon" android:label="@string/app_name"> </application> Thanks in advance for helping me understand why this code is crashing. Also, if you think this is just the wrong approach to let multiple activities have access to the same data, please give your suggestion. Please keep in mind that I am talking about quite a few variables and some large arrays.

    Read the article

  • Best practice? iphone: sync data

    - by Andy Jacobs
    So i'm working on a project where there is data visualization. My ultimate goal is that i have a set of data shipped with the download of the iphone app. But i want it connected to a backend, that if the iphone has a connection with the internet. it can sync the changes from the backend. The syncing is no problem or the connection between the backend & the iphone. But what should i use as data storage on my iphone? what is the best way. my data is purely text and doesn't have to be secure. But it's main feature should be updating certain parts of data ( adding and deleting are not so important ) so what is the easiest (read: least time consuming development ) or the best way? sqlite? plist? ..?

    Read the article

  • Explaining persistent data structures in simple terms

    - by Jason Baker
    I'm working on a library for Python that implements some persistent data structures (mainly as a learning exercise). However, I'm beginning to learn that explaining persistent data structures to people unfamiliar with them can be difficult. Can someone help me think of an easy (or at least the least complicated) way to describe persistent data structures to them? I've had a couple of people tell me that the documentation that I have is somewhat confusing. (And before anyone asks, no I don't mean persistent data structures as in persisted to the file system. Google persistent data structures if you're unclear on this.)

    Read the article

  • Aging Data Structure in C#

    - by thelsdj
    I want a data structure that will allow querying how many items in last X minutes. An item may just be a simple identifier or a more complex data structure, preferably the timestamp of the item will be in the item, rather than stored outside (as a hash or similar, wouldn't want to have problems with multiple items having same timestamp). So far it seems that with LINQ I could easily filter items with timestamp greater than a given time and aggregate a count. Though I'm hesitant to try to work .NET 3.5 specific stuff into my production environment yet. Are there any other suggestions for a similar data structure? The other part that I'm interested in is aging old data out, If I'm only going to be asking for counts of items less than 6 hours ago I would like anything older than that to be removed from my data structure because this may be a long-running program.

    Read the article

  • Data mining logs to locate a bug

    - by gooli
    I'm working on a data distribution application which receives data from a source and distributes that data to multiple target application. After successfully distributing several messages each second for 8 days, it missed a single message and did not deliver it properly to the clients. As I was looking at the logs I tried to find something there that was special for the time the miss happend - either in the data, its rate or some other condition but couldn't find anything. Is there any data mining technique I can use to identify how that specific event differs from other events?

    Read the article

  • How to build a data flow?

    - by salvationishere
    I am running Visual Studio 2008, the SSIS Tutorial described on: http://msdn.microsoft.com/en-us/library/ms167106.aspx I finished all of the tasks but am getting following errors: Error 1 Validation error. Extract Sample Currency Data: Extract Sample Currency Data: input column "CurrencyAlternateKey" (123) has lineage ID 55 that was not previously used in the Data Flow task. Lesson 1.dtsx 0 0 Error 2 Validation error. Extract Sample Currency Data SSIS.Pipeline: input column "CurrencyAlternateKey" (123) has lineage ID 55 that was not previously used in the Data Flow task. Lesson 1.dtsx 0 0 Can you tell what I need to do to make this build without errors?

    Read the article

  • Parsing content-disposion header's filename in multipart/from-data

    - by Artyom
    Hello According to RFC, in multipart/form-data content-disposition header filename field receives as parameter HTTP quoted string - string between quites where character '\' can escape any other ascii character. Problem web browsers don't do it. IE6 sends: Content-Disposition: form-data; name="file"; filename="z:\tmp\test.txt" Instead of expected Content-Disposition: form-data; name="file"; filename="z:\\tmp\\test.txt" Which should be parsed as z:tmptest.txt according to rules instead of z:\tmp\test.txt. Firefox, Konqueror and Chrome don't escape " characters for example: Content-Disposition: form-data; name="file"; filename=""test".txt" Instead of expected Content-Disposition: form-data; name="file"; filename="\"test\".txt" So... how would you suggest to deal with this issue?

    Read the article

  • Last Observation Carried Forward In a data frame?

    - by Tal Galili
    Hi all, I wish to implement a "Last Observation Carried Forward" for a data set I am working on which has missing values at the end of it. Here is a simple code to do it (question after it): LOCF <- function(x) { # Last Observation Carried Forward (for a left to right series) LOCF <- max(which(!is.na(x))) # the location of the Last Observation to Carry Forward x[LOCF:length(x)] <- x[LOCF] return(x) } # example: LOCF(c(1,2,3,4,NA,NA)) LOCF(c(1,NA,3,4,NA,NA)) Now this works great for simple vectors. But if I where to try and use it on a data frame: a <- data.frame(rep("a",4), 1:4,1:4, c(1,NA,NA,NA)) a t(apply(a, 1, LOCF)) # will make a mess It will turn my data frame into a character matrix. Can you think of a way to do LOCF on a data.frame, without turning it into a matrix? (I could use loops and such to correct the mess, but would love for a more elegant solution) Cheers, Tal

    Read the article

  • How to preserve data integrity while minimizing the transmission size

    - by user1500578
    we have sensors in the wild that send their data to a server every day via TCP/IP, either through 3G or through satellite for the physical layer. The sensors can automatically switch from one to the other depending on their location and the quality of the signal with the local 3G operator. Given that the 3G and satellite communications are very expensive, we want to minimize the amount of data to send. But also, we want to protect ourselves from lost data. What would be the best strategy to ensure with reasonable certainty that the integrity of our data is preserved, while minimizing the amount of redundancy, i.e the amount of data transmitted ? I've read about the zfec codec, but I'm not sure if we need to transmit all the chunks, or if we need to send a hash code along each chunk.

    Read the article

  • Merging MySQL Structure and Data

    - by Shahid
    I have a MySQL database running on a deployment machine which also contains data. Then I have another MySQL database which has evolved in terms of STRUCTURE + DATA for some time. I need a way to merge the changes (ONLY) for both structure and data to the DB in deployment machine without disturbing the existing data. Does anyone know of a tool available which can do this safely. I have had a look at a few comparison tools but I need a tool which can automate the merge operation. Note also that most of the data in the tables is in BINARY so I can't use many file comparison tools. Does any one know of a solution to this? thanks

    Read the article

  • JavaScript data grid for millions of rows

    - by Rudiger
    I need to present millions of rows of data to users in a grid using JavaScript. I don't want the user thinking about viewing finite pieces of data; it should appear that all of the data is available. Rather than downloading all the data at once, small chunks are downloaded as the user comes to them (by scrolling, keying in row numbers, searching). The rows will not be edited through this front end, so read-only answers are acceptable. What JavaScript data grids are best for this kind of seamless paging?

    Read the article

  • Letting Wcf data service implement another contract

    - by Wasim
    Hi all , I have a wcf data service with the standart configuration . I want to add another functionality to it , so I built a contract interface and let my wcf data service implements it . Now I see in the service the InitializeService method , and the contract interface methods . When I come to connect the service , I get an error , that there is no end point declared to the contract I added . How can do that ? examples ? links ? I choosed to add the contract interface to the wcf data service and not adding another service , because the client application uses the wcf data service generated objects , and I want to use the same obkject to make operations not related to data , for more coplex processing . If I do the methods in another service , then I have types incompatibility . Thanks in advance ...

    Read the article

  • Advice on simple efficient way to store web form data when no db/auth required

    - by ted776
    Hi, I have a situation where I need to provide an efficient way to process and store comments submitted via a web form. I would normally use PHP and either MySQL or XML to store the data, but this is slightly different in that this web form will only be temporarily available in a closed LAN environment, and all i need to do is process the form data and store it a format which can be accessed by another application on the LAN (Adobe Director). Each request made by the Director app should pop the stack of data. I'm wondering how best to store the data for this type of situation as it's not something I would normally do. I'm thinking possibly storing the data in an XML file, but any advice would be great!

    Read the article

  • JavaScript - question regarding data structure

    - by orokusaki
    I'm trying to calculate somebody's bowel health based on a points system. I can edit the data structure, or logic in any way. I'm just trying to write a function and data structure to handle this ability. Pseudo calculator function: // Bowel health calculator var points = 0; If age is from 30 and 34: points += 1 If age is from 35 and 40: points += 2 If daily BMs is from 1 and 3: points -= 1 If daily BMs is from 4 and 6: points -= 2 return points; Pseudo data structure: var points_map = { age: { '30-34': 1, '35-40': 2 }, dbm: { '1-3': -1, '4-6': -2 } }; I have a full spreadsheet of data like this, and I am trying to write a DRY version of code and a DRY version of this data (ie, probably not a string for the '30-34', etc) in order to handle this sort of thing without a humongous number of switch statements.

    Read the article

  • Cloud HUGE data storage options?

    - by ToughPal
    Hi, Does anyone have a good suggestion on how to do video recording? We have a camera that can record and then stream live video to a server. So this means we can have 1000's of cameras sending data 24X7 for recording. We will store data for over 7 / 14 / 30 days depending on the package. Per day if a camera is sending data to the server then it will store 1.5GB. So that means there is a traffic of 1.5GB / day / camera Total monthly 45GB / month / camera (Data + bandwidth for one camera) Please let me know the most cost effective way to get this data stored? Thanks!

    Read the article

  • How to Embed/Link binary data into a C++ DLL

    - by CrimsonX
    So I have a Visual Studio 2008 project which has a large amount of binary data that it is currently referencing. I would like to package the binary data much like you can do with C# by adding it as a "resource" and compiling it as a DLL. Lets say all my data has an extension of ".data" and is currently being read from the visual studio project. Is there a way that you can compile or link the data into the .dll which it is calling? I've looked at some of the google link for this and so far I haven't come up with anything - the only possible solution I've come up with is to use something like ResGen to create a .resources file and then link it using AssemblyLinker with /Embed or /Link flags. I dont think it'd work properly though because I dont have text files to create the .resources files, but rather binary files themselves. Any advice?

    Read the article

  • Managing test data for Junit tests.

    - by nobody
    Hi, We are facing one problem in managing test data(xmls which is used to create mock objects). The data which we have currently has been evolved over a long period of time. Each time we add a new functionality or test case we add new data to test that functionality. Now, the problem is when the business requirement changes the format( like length or format of a variable) or any change which the test data doesn't support , we need to change the entire test data which is 100s of MBs in size. Could anyone suggest a better method or process to overcome this problem? Any suggestion would be appreciated.

    Read the article

  • How to extract data from Google Analytics and build a data warehouse (webhouse) from it?

    - by nkaur301
    I have click stream data such as referring URL, top landing pages, top exit pages and metrics such as page views, number of visits, bounces all in Google Analytics. I am required to build a data warehouse from scratch(which I believe is known as web-house) from this data. My questions are:- 1)Is it possible? Every day data increases (some in terms of metrics or measures such as visits and some in terms of new referring sites), how would the process of loading the warehouse go about? 2)What ETL tool would help me to achieve this? Pentaho I believe has a way to pull out data from Google Analytics, has anyone used it? How does that process go? Any references, links would be appreciated besides answers.

    Read the article

  • SQL SERVER – 5 Tips for Improving Your Data with expressor Studio

    - by pinaldave
    It’s no secret that bad data leads to bad decisions and poor results.  However, how do you prevent dirty data from taking up residency in your data store?  Some might argue that it’s the responsibility of the person sending you the data.  While that may be true, in practice that will rarely hold up.  It doesn’t matter how many times you ask, you will get the data however they decide to provide it. So now you have bad data.  What constitutes bad data?  There are quite a few valid answers, for example: Invalid date values Inappropriate characters Wrong data Values that exceed a pre-set threshold While it is certainly possible to write your own scripts and custom SQL to identify and deal with these data anomalies, that effort often takes too long and becomes difficult to maintain.  Instead, leveraging an ETL tool like expressor Studio makes the data cleansing process much easier and faster.  Below are some tips for leveraging expressor to get your data into tip-top shape. Tip 1:     Build reusable data objects with embedded cleansing rules One of the new features in expressor Studio 3.2 is the ability to define constraints at the metadata level.  Using expressor’s concept of Semantic Types, you can define reusable data objects that have embedded logic such as constraints for dealing with dirty data.  Once defined, they can be saved as a shared atomic type and then re-applied to other data attributes in other schemas. As you can see in the figure above, I’ve defined a constraint on zip code.  I can then save the constraint rules I defined for zip code as a shared atomic type called zip_type for example.   The next time I get a different data source with a schema that also contains a zip code field, I can simply apply the shared atomic type (shown below) and the previously defined constraints will be automatically applied. Tip 2:     Unlock the power of regular expressions in Semantic Types Another powerful feature introduced in expressor Studio 3.2 is the option to use regular expressions as a constraint.   A regular expression is used to identify patterns within data.   The patterns could be something as simple as a date format or something much more complex such as a street address.  For example, I could define that a valid IP address should be made up of 4 numbers, each 0 to 255, and separated by a period.  So 192.168.23.123 might be a valid IP address whereas 888.777.0.123 would not be.   How can I account for this using regular expressions? A very simple regular expression that would look for any 4 sets of 3 digits separated by a period would be:  ^[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}$ Alternatively, the following would be the exact check for truly valid IP addresses as we had defined above:  ^(25[0-5]|2[0-4][0-9]|1[0-9]{2}|[1-9]?[0-9])\.(25[0-5]|2[0-4][0-9]|1[0-9]{2}|[1-9]?[0-9])\.(25[0-5]|2[0-4][0-9]|1[0-9]{2}|[1-9]?[0-9])\.(25[0-5]|2[0-4][0-9]|1[0-9]{2}|[1-9]?[0-9])$ .  In expressor, we would enter this regular expression as a constraint like this: Here we select the corrective action to be ‘Escalate’, meaning that the expressor Dataflow operator will decide what to do.  Some of the options include rejecting the offending record, skipping it, or aborting the dataflow. Tip 3:     Email pattern expressions that might come in handy In the example schema that I am using, there’s a field for email.  Email addresses are often entered incorrectly because people are trying to avoid spam.  While there are a lot of different ways to define what constitutes a valid email address, a quick search online yields a couple of really useful regular expressions for validating email addresses: This one is short and sweet:  \b[A-Z0-9._%+-]+@[A-Z0-9.-]+\.[A-Z]{2,4}\b (Source: http://www.regular-expressions.info/) This one is more specific about which characters are allowed:  ^([a-zA-Z0-9_\-\.]+)@((\[[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.)|(([a-zA-Z0-9\-]+\.)+))([a-zA-Z]{2,4}|[0-9]{1,3})(\]?)$ (Source: http://regexlib.com/REDetails.aspx?regexp_id=26 ) Tip 4:     Reject “dirty data” for analysis or further processing Yet another feature introduced in expressor Studio 3.2 is the ability to reject records based on constraint violations.  To capture reject records on input, simply specify Reject Record in the Error Handling setting for the Read File operator.  Then attach a Write File operator to the reject port of the Read File operator as such: Next, in the Write File operator, you can configure the expressor operator in a similar way to the Read File.  The key difference would be that the schema needs to be derived from the upstream operator as shown below: Once configured, expressor will output rejected records to the file you specified.  In addition to the rejected records, expressor also captures some diagnostic information that will be helpful towards identifying why the record was rejected.  This makes diagnosing errors much easier! Tip 5:    Use a Filter or Transform after the initial cleansing to finish the job Sometimes you may want to predicate the data cleansing on a more complex set of conditions.  For example, I may only be interested in processing data containing males over the age of 25 in certain zip codes.  Using an expressor Filter operator, you can define the conditional logic which isolates the records of importance away from the others. Alternatively, the expressor Transform operator can be used to alter the input value via a user defined algorithm or transformation.  It also supports the use of conditional logic and data can be rejected based on constraint violations. However, the best tip I can leave you with is to not constrain your solution design approach – expressor operators can be combined in many different ways to achieve the desired results.  For example, in the expressor Dataflow below, I can post-process the reject data from the Filter which did not meet my pre-defined criteria and, if successful, Funnel it back into the flow so that it gets written to the target table. I continue to be impressed that expressor offers all this functionality as part of their FREE expressor Studio desktop ETL tool, which you can download from here.  Their Studio ETL tool is absolutely free and they are very open about saying that if you want to deploy their software on a dedicated Windows Server, you need to purchase their server software, whose pricing is posted on their website. Reference: Pinal Dave (http://blog.SQLAuthority.com) Filed under: Pinal Dave, PostADay, SQL, SQL Authority, SQL Query, SQL Scripts, SQL Server, SQL Tips and Tricks, T SQL, Technology

    Read the article

< Previous Page | 35 36 37 38 39 40 41 42 43 44 45 46  | Next Page >