Search Results

Search found 63736 results on 2550 pages for 'sample data'.

Page 111/2550 | < Previous Page | 107 108 109 110 111 112 113 114 115 116 117 118  | Next Page >

  • large amount of data in many text files - how to process?

    - by Stephen
    Hi, I have large amounts of data (a few terabytes) and accumulating... They are contained in many tab-delimited flat text files (each about 30MB). Most of the task involves reading the data and aggregating (summing/averaging + additional transformations) over observations/rows based on a series of predicate statements, and then saving the output as text, HDF5, or SQLite files, etc. I normally use R for such tasks but I fear this may be a bit large. Some candidate solutions are to 1) write the whole thing in C (or Fortran) 2) import the files (tables) into a relational database directly and then pull off chunks in R or Python (some of the transformations are not amenable for pure SQL solutions) 3) write the whole thing in Python Would (3) be a bad idea? I know you can wrap C routines in Python but in this case since there isn't anything computationally prohibitive (e.g., optimization routines that require many iterative calculations), I think I/O may be as much of a bottleneck as the computation itself. Do you have any recommendations on further considerations or suggestions? Thanks

    Read the article

  • iPhones SDK: Setting a relationship property object using core data?

    - by Harkonian
    I'm using core data in my app. I have two entities that are related: EntityA and EntityB. EntityA has a property of type "relationship" with EntityB. In addition, both of these entities are defined classes (not the default NSManagedObject). I'm inserting a new object into my data like this: EntityA *newEntityA = [NSEntityDescription insertNewObjectForEntityForName:@"EntityA" inManagedObjectContext:self.managedObjectContext]; newEntityA.name = @"some name"; newEntityA.entityB.name = @"some other name"; The problem is entityB.name is null. Even if I add an NSLog() statement right after assigning the value, it is null. What is the proper way of setting my "name" property of EntityB when EntityB is a property of EntityA?

    Read the article

  • Is there an existing solution to the multithreaded data structure problem?

    - by thr
    I've had the need for a multi-threaded data structure that supports these claims: Allows multiple concurrent readers and writers Is sorted Is easy to reason about Fulfilling multiple readers and one writer is a lot easier, but I really would wan't to allow multiple writers. I've been doing research into this area, and I'm aware of ConcurrentSkipList (by Lea based on work by Fraser and Harris) as it's implemented in Java SE 6. I've also implemented my own version of a concurrent Skip List based on A Provably Correct Scalable Concurrent Skip List by Herlihy, Lev, Luchangco and Shavit. These two implementations are developed by people that are light years smarter then me, but I still (somewhat ashamed, because it is amazing work) have to ask the question if these are the two only viable implementations of a concurrent multi reader/writer data structures available today?

    Read the article

  • How to auto-increment reference number persistently when NSManagedObjects created in core-data.

    - by KayKay
    In my application i am using core-data to store information and saving these data to the server using web-connectivity i have to use MySql. Basically what i want to do is to keep track of number of NSManagedObject already created and Whenever i am adding new NSManagedObject, based on that counting it will assign the class a Int_value which will act as primary_key in MySql. For examaple, there are already 10 NSManagedobjects, and when i will add new one it will assign it "11" as primary_key. these value will have to be increasing because there is no deleting of NSManagedObject. From my approach its about static member in applicationDelegate whose initial value can be any integer but should be incremented by one(like auto-increment) everytime new NSManagedObject is created and also it should be persistent. I am not clear how to do this, please give me suggestions. Thanks in advance.

    Read the article

  • Proper binding data to combobox and handling its events.

    - by Wodzu
    Hi guys. I have a table in SQL Server which looks like this: ID Code Name Surname 1 MS Mike Smith 2 JD John Doe 3 UP Unknown Person and so on... Now I want to bind the data from this table into the ComboBox in a way that in the ComboBox I have displayed value from the Code column. I am doing the binding in this way: SqlDataAdapter sqlAdapter = new SqlDataAdapter("SELECT * FROM dbo.Users ORDER BY Code", MainConnection); sqlAdapter.Fill(dsUsers, "Users"); cbxUsers.DataSource = dsUsers.Tables["Users"]; cmUsers = (CurrencyManager)cbxUsers.BindingContext[dsUsers.Tables["Users"]]; cbxUsers.DisplayMember = "Code"; And this code seems to work. I can scroll through the list of Codes. Also I can start to write code by hand and ComboBox will autocomplete the code for me. However, I wanted to put a label at the top of the combobox to display Name and Surname of the currently selected user code. My line of though was like that: "So, I need to find an event which will fire up after the change of code in combobox and in that event I will get the current DataRow..." I was browsing through the events of combobox, tried many of them but without a success. For example: private void cbxUsers_SelectionChangeCommitted(object sender, EventArgs e) { if (cmUsers != null) { DataRowView drvCurrentRowView = (DataRowView)cmUsers.Current; DataRow drCurrentRow = drvCurrentRowView.Row; lblNameSurname.Text = Convert.ToString(drCurrentRow["Name"]) + " " + Convert.ToString(drCurrentRow["Surname"]); } } This give me a strange results. Firstly when I scroll via mouse scroll it doesn't return me the row wich I am expecting to obtain. For example on JD it shows me "Mike Smith", on MS it shows me "John Doe" and on UP it shows me "Mike Smith" again! The other problem is that when I start to type in ComboBox and press enter it doesn't trigger the event. However, everything works as expected when I bind data to lblNameSurname.Text in this way: lblNameSurname.DataBindings.Add("Text", dsusers.Tables["Users"], "Name"); The problem here is that I can bind only one column and I want to have two. I don't want to use two labels for it (one to display name and other to display surname). So, what is the solution to my problem? Also, I have one question related to the data selection in ComboBox. Now, when I type something in the combobox it allows me to type letters that are not existing in the list. For example, I start to type "J" and instead of finishing with "D" so I would have "JD", I type "Jsomerandomtexthere". Combobox will allow that but such item does not exists on the list. In other words, I want combobox to prevent user from typing code which is not on the list of codes. Thanks in advance for your time.

    Read the article

  • How to manipulate data in View using Asp.Net Mvc RC 2?

    - by Picflight
    I have a table [Users] with the following columns: INT SmallDateTime Bit Bit [UserId], [BirthDate], [Gender], [Active] Gender and Active are Bit that hold either 0 or 1. I am displaying this data in a table on my View. For the Gender I want to display 'Male' or 'Female', how and where do I manipulate the 1's and 0's? Is it done in the repository where I fetch the data or in the View? For the Active column I want to show a checkBox that will AutoPostBack on selection change and update the Active filed in the Database. How is this done without Ajax or jQuery?

    Read the article

  • Where would you document standardized complex data that is passed between many objects and methods?

    - by Eli
    Hi All, I often find myself with fairly complex data that represents something that my objects will be working on. For example, in a task-list app, several objects might work with an array of tasks, each of which has attributes, temporal expressions, sub tasks and sub sub tasks, etc. One object will collect data from web forms, standardize it into a format consumable by the class that will save them to the database, another object will pull them from the database, put them in the standard format and pass them to the display object, or the update object, etc. The data itself can become a fairly complex series of arrays and sub arrays, representing a 'task' or list of tasks. For example, the below might be one entry in a task list, in the format that is consumable by the various objects that will work on it. Normally, I just document this in a file somewhere with an example. However, I am thinking about the best way to add it to something like PHPDoc, or another standard doc system. Where would you document your consumable data formats that are for many or all of the objects / methods in your app? Array ( [Meta] => Array ( //etc. ) [Sched] => Array ( [SchedID] => 32 [OwnerID] => 2 [StatusID] => 1 [DateFirstTask] => 2011-02-28 [DateLastTask] => [MarginMonths] => 3 ) [TemporalExpressions] => Array ( [0] => Array ( [type] => dw [TemporalExpID] => 3 [ord] => 2 [day] => 6 [month] => 4 ) [1] => Array ( [type] => dm [TemporalExpID] => 32 [day] => 28 [month] => 2 ) ) [Task] => Array ( [SchedTaskID] => 32 [SchedID] => 32 [OwnerID] => 2 [UserID] => 5 [ClientID] => 9 [Title] => Close Prior Year [Body] => [DueTime] => ) [SubTasks] => Array ( [101] => Array ( [SchedSubTaskID] => 101 [ParentST] => [RootT] => 32 [UserID] => 2 [Title] => Review Profit and Loss by Class [Body] => [DueDiff] => 0 ) [102] => Array ( [SchedSubTaskID] => 102 [ParentST] => [RootT] => 32 [UserID] => 2 [Title] => Review Balance Sheet [Body] => [DueDiff] => 0 ) [103] => Array ( [SchedSubTaskID] => 103 [ParentST] => [RootT] => 32 [UserID] => 2 [Title] => Review Current Year for Prior Year Expenses to Accrue [Body] => Look at Journal Entries that are templates as well. [DueDiff] => 0 ) [104] => Array ( [SchedSubTaskID] => 104 [ParentST] => [RootT] => 32 [UserID] => 2 [Title] => Review Prior Year Membership from 11/1 - 12/31 to Accrue to Current Year [Body] => [DueDiff] => 0 ) [105] => Array ( [SchedSubTaskID] => 105 [ParentST] => [RootT] => 32 [UserID] => 2 [Title] => Enter Vacation Accrual [Body] => [DueDiff] => 0 ) [106] => Array ( [SchedSubTaskID] => 106 [ParentST] => 105 [RootT] => 32 [UserID] => 2 [Title] => Email Peter requesting Vacation Status of Employees at Year End [Body] => We need Employee Name, Rate and Days of Vacation left to use. We also need to know if the employee used any of the prior year's vacation. [DueDiff] => 43 ) [107] => Array ( [SchedSubTaskID] => 107 [ParentST] => [RootT] => 32 [UserID] => 2 [Title] => Grants Receivable at Year End [Body] => [DueDiff] => 0 ) [108] => Array ( [SchedSubTaskID] => 108 [ParentST] => 107 [RootT] => 32 [UserID] => 2 [Title] => Email Peter Requesting if there were and Grants Receivable at year end [Body] => [DueDiff] => 43 ) ) )

    Read the article

  • How to keep track of NSManagedObjects created in core-data persistently.

    - by KayKay
    In my application i am using core-data to store information and saving these data to the server using web-connectivity i have to use MySql. Basically what i want to do is to keep track of number of NSManagedObject already created and Whenever i am adding new NSManagedObject, based on that counting it will assign the class a Int_value which will act as primary_key in MySql. For examaple, there are already 10 NSManagedobjects, and when i will add new one it will assign it "11" as primary_key. these value will have to be increasing because there is no deleting of NSManagedObject. From my approach its about static member in applicationDelegate whose initial value can be any integer but should be incremented by one everytime new NSManagedObject is created and also it should be persistent. I am not clear how to do this, please give me suggestions. Thanks in advance.

    Read the article

  • How is 'processing credit card data' defined (PCI)?

    - by Chris
    If i have a web application and i receive credit card data transmitted via a POST request by a web browser over HTTPS and instantly open a socket (SSL) to a remote PCI compilant card processor to forward the data and wait for a response, am i allowed to do that? or is this receiving the data with my application and forwarding it already subject of "processing credit card data"? if i create an iframe that is displayed in a client browser to enter cc data and this iframe posts the data via HTTPS to remote card processor (directly!) is this already a case of processing credit card data? even if my application code 'doesnt touch' the entered data with any event handlers? i'm interested in the definition "credit card data processing". when does it start to be a cc data processing application? can somebody maybe point me to that section in PCI-DSS standard that clearly defines when you start to 'be a processing application'? Thanks,

    Read the article

  • R mlbench exapmle

    - by Johan B.
    I heard R is the "de facto" language amongst statistical software developers, and I'm giving it a try. I already know the basics, but it still looks "weird" to me (a C developer). I think it would be very useful to see a working example to see how a real R program is built. I thought that an R solution for any of the mlbench problems would be optimal, because I'm already familiar with it and it would allow me to compare it to other languages, but any other "toy problem" example is welcome. Thank you.

    Read the article

  • Can someone please debug this Windows Azure Application?

    - by Vimvq1987
    Here's the myTODO project from codeplex: myTODO project I added any necessary libraries, added storage, changed obsolete types/methods, but everything went wrong when I debug it. An exception was thrown here (in TableStorage.cs): public IEnumerable<TElement> ExecuteWithRetries(RetryPolicy retry) { IEnumerable<TElement> ret = null; if (retry == null) { throw new ArgumentNullException("retry"); } retry(() => { try { ret = _query.Execute(); } catch (InvalidOperationException e) { if (TableStorageHelpers.CanBeRetried(e)) { throw new TableRetryWrapperException(e); } throw; } }); return ret; } I'm using Visual Studio 2008, SQL server 2008, Windows Azure SDK v1.1. Can anyone please debug this project for me, or suggest me someway to get it working. This request is urgent. Any helps are much appreciated. PS: If you can't download these file, please let me know, I'll upload to another hosts.

    Read the article

  • transforming binary data using ssis and sql server 2008

    - by Rick
    Hello All - I have a task to import/transform and extract zipped binary files that contain both text data as well as embeded binary data. Within the data is data that is relational in nature and needs to be processed into a defined database structure. Currently I have a C# single threaded app that essentially grabs all the files from the directory (currently there is 13K files of varying sizes) and extracts the data on a single thread line by line inserts to the database. As you could imagine this is a very slow process and unacceptable. There are several different parsing routines used depending on the header record in the file. There are potentially upto a million rows per file when all the data is extracted to the row level of detail. Follow on task is to parse those rows into their appropriate tables based on is content. i.e. the textual content has to be parsed further into "buckets" of like data in the database. That about sums up the big picture. Now for the problem task list. How do i iterate through a packet of data using SSIS? In the app the file is decompressed and then is parsed using streams data type and byte arrays and is routed to the required parsing routine based on the header data of each packet. There is bit swapping involved as well. Should i wrap up the app code into a script task(s) and let it do the custom processing? The data is seperated by year and the sql server tables is partitioned by year as well. I need to be able to "catch" bad file data as well and process by hand most likely. Should i simply load the zipped file to sql as a blob and parse the file with T-SQL? Would that be multi threaded if done that way? Not sure how to do the parsing in tsql that is involved here. Which do you think would be faster? Potentially the data that is currently processed via files could come to us via a socket. Can SSIS collect that data in real time? How would i go about setting that up? Processing these new files from the directorys will become a daily task. I can manage the data once i get it to sql server. Getting it there in a timely fashion seems to be the long pole in the tent for me. I would appreciate any comments or suggestions from the group. Rick

    Read the article

  • remove duplicates from object array data java

    - by zahir hussain
    hi i want to know how to remove duplicates in object. for example cat c[] = new cat[10]; c[1].data = "ji"; c[2].data = "pi"; c[3].data = "ji"; c[4].data = "lp"; c[5].data = "ji"; c[6].data = "pi"; c[7].data = "jis"; c[8].data = "lp"; c[9].data = "js"; c[10].data = "psi"; i would like to remove the duplicates value from object array. thanks and advance

    Read the article

  • Mapping enum with fluent nhibernate

    - by Puneet
    I am following the http://wiki.fluentnhibernate.org/Getting%5Fstarted tutorial to create my first NHibernate project with Fluent NHibernate I have 2 tables 1) Account with fields Id AccountHolderName AccountTypeId 2) AccountType with fields Id AccountTypeName Right now the account types can be Savings or Current So the table AccountTypes stores 2 rows 1 - Savings 2 - Current For AccoutType table I have defined enum public enum AccountType { Savings=1, Current=2 } For Account table I define the entity class public class Account { public virtual int Id {get; private set;} public virtual string AccountHolderName {get; set;} public virtual string AccountType {get; set;} } The fluent nhibernate mappings are: public AgencyMap() { Id(o => o.Id); Map(o => o.AccountHolderName); Map(o => o.AccountType); } When I try to run the solution, it gives an exception - InnerException = {"(XmlDocument)(2,4): XML validation error: The element 'class' in namespace 'urn:nhibernate-mapping-2.2' has incomplete content. List of possible elements expected: 'meta, subselect, cache, synchronize, comment, tuplizer, id, composite-id' in namespace 'ur... I guess that is because I have not speciofied any mapping for AccountType. The questions are: How can I use AccountType enum instead of a AccountType class? Maybe I am going on wrong track. Is there a better way to do this? Thanks!

    Read the article

  • What are the Options for Storing Hierarchical Data in a Relational Database?

    - by orangepips
    Good Overviews One more Nested Intervals vs. Adjacency List comparison: the best comparison of Adjacency List, Materialized Path, Nested Set and Nested Interval I've found. Models for hierarchical data: slides with good explanations of tradeoffs and example usage Representing hierarchies in MySQL: very good overview of Nested Set in particular Hierarchical data in RDBMSs: most comprehensive and well organized set of links I've seen, but not much in the way on explanation Options Ones I am aware of and general features: Adjacency List: Columns: ID, ParentID Easy to implement. Cheap node moves, inserts, and deletes. Expensive to find level (can store as a computed column), ancestry & descendants (Bridge Hierarchy combined with level column can solve), path (Lineage Column can solve). Use Common Table Expressions in those databases that support them to traverse. Nested Set (a.k.a Modified Preorder Tree Traversal) First described by Joe Celko - covered in depth in his book Trees and Hierarchies in SQL for Smarties Columns: Left, Right Cheap level, ancestry, descendants Compared to Adjacency List, moves, inserts, deletes more expensive. Requires a specific sort order (e.g. created). So sorting all descendants in a different order requires additional work. Nested Intervals Combination of Nested Sets and Materialized Path where left/right columns are floating point decimals instead of integers and encode the path information. Bridge Table (a.k.a. Closure Table: some good ideas about how to use triggers for maintaining this approach) Columns: ancestor, descendant Stands apart from table it describes. Can include some nodes in more than one hierarchy. Cheap ancestry and descendants (albeit not in what order) For complete knowledge of a hierarchy needs to be combined with another option. Flat Table A modification of the Adjacency List that adds a Level and Rank (e.g. ordering) column to each record. Expensive move and delete Cheap ancestry and descendants Good Use: threaded discussion - forums / blog comments Lineage Column (a.k.a. Materialized Path, Path Enumeration) Column: lineage (e.g. /parent/child/grandchild/etc...) Limit to how deep the hierarchy can be. Descendants cheap (e.g. LEFT(lineage, #) = '/enumerated/path') Ancestry tricky (database specific queries) Database Specific Notes MySQL Use session variables for Adjacency List Oracle Use CONNECT BY to traverse Adjacency Lists PostgreSQL ltree datatype for Materialized Path SQL Server General summary 2008 offers HierarchyId data type appears to help with Lineage Column approach and expand the depth that can be represented.

    Read the article

  • List of fundamental data structures - what am I missing?

    - by jboxer
    I've been studying my fundamental data structures a bunch recently, trying to make sure I've got them down cold. By "fundamental", I mean the real basic ones. Fancy ones like Red-Black Trees and Bloom Filters are clearly worth knowing, but they're usually either enhancements of fundamental ones (Red-Black Trees are binary search trees with special properties to keep them balanced) or they're only useful in very specific situations (Bloom Filters). So far, I'm "fluent" in the following data structures: Arrays Linked Lists Stacks/Queues Binary Search Trees Heaps/Priority Queues Hash Tables However, I feel like I'm missing something. Are there any fundamental ones that I'm forgetting about? EDIT: Added these after posting the question Strings (suggested by catchmeifyoutry) Sets (suggested by Peter) Graphs (suggested by Nick D and aJ) B-Trees (Suggested by tloach) I'm a little on-the-fence about whether these are too fancy or not, but I think they're different enough from the fundamental structures (and important enough) to be worth studying as fundamental.

    Read the article

  • How to manupilate data in VIew using Asp.Net Mvc RC 2?

    - by Picflight
    I have a table [Users] with the following columns: INT SmallDateTime Bit Bit [UserId], [BirthDate], [Gender], [Active] Gender and Active are Bit that hold either 0 or 1. I am displaying this data in a table on my View. For the Gender I want to display 'Male' or 'Female', how and where do I manipulate the 1's and 0's? Is it done in the repository where I fetch the data or in the View? For the Active column I want to show a checkBox that will AutoPostBack on selection change and update the Active filed in the Database. How is this done without Ajax or jQuery?

    Read the article

  • Subset generation by rules

    - by Sazug
    Let's say that we have a 5000 users in database. User row has sex column, place where he/she was born column and status (married or not married) column. How to generate a random subset (let's say 100 users) that would satisfy these conditions: 40% should be males and 60% - females 50% should be born in USA, 20% born in UK, 20% born in Canada, 10% in Australia 70% should be married and 30% not. These conditions are independent, that is we cannot do like this: (0.4 * 0.5 * 0.7) * 100 = 14 users that are males, born in USA and married (0.4 * 0.5 * 0.3) * 100 = 6 users that are males, born in USA and not married. Is there an algorithm to this generation?

    Read the article

  • If I CFRelease() an image in core data, how do I get it back?

    - by Sam
    My iphone app plays a slide show made up of 5 user images.  These images are stored using core data.  I was noticing that memory was building up every time a different slide show was played and it was not releasing any of the previously played slide shows.   These images are showing up in Object Allocations as CFData. So I tried releasing this data in the dealloc method CFRelease(slideshow.image1); CFRelease(slideshow.image2); CFRelease(slideshow.image3); CFRelease(slideshow.image4); CFRelease(slideshow.image5); This releases the previous slideshow great...BUT when I go back to view that same slideshow again, it crashes.   I am guessing that I need to alloc/init these images again, but I am not sure how?  Or maybe I should be managing this memory in a different way?

    Read the article

  • How do I protect Dynamic data pages using ASP.NET Authentication?

    - by ProfK
    I have a site where most of my pages are arranged in business area folders, e.g. Activations, Outdoors, Branding. Each folder has a small web.config that protects the contents against access by people without a role for that business area. However, basic admin for most business areas is done via Dynamic Data pages. These are only basically protected by not appearing in the menu unless the user has the correct role, but they are still accessible directly via URL, because of the {table}/{Action} routing used by Dynamic Data. What can I do to protect these pages against direct access?

    Read the article

< Previous Page | 107 108 109 110 111 112 113 114 115 116 117 118  | Next Page >