Search Results

Search found 6048 results on 242 pages for 'linq to dataset'.

Page 17/242 | < Previous Page | 13 14 15 16 17 18 19 20 21 22 23 24  | Next Page >

  • Building a custom (dynamic) dataset and grid

    - by marko.ivanovski.nz
    Hi, I'm in the process of building a dynamic table in which you can add/remove rows & columns so it varies in size depending on what the user wants. Its purpose is to store properties for a product, but there can be from 1 to 10 different properties(columns) per product, and multiple instances(rows) of the product as well. Here's a screenshot of what I mean http://i40.tinypic.com/nbqkxc.jpg As you can see I need the structure to be completely up to the client which is where I'm getting stuck. I've started writing a custom DataSet that has "add column" & "add row" buttons, and have built a custom Table with Textboxes in each cell which builds from that dataset. I have no idea how to store the data on submit though, and to make it even more complex I need to store this in the database as a string which I think I can do by converting it to XML. Any help is appreciated, I think I just need a pointer in the right direction and am happy to do research from there. Thanks in advance. Marko

    Read the article

  • DataReader-DataSet Hybrid solution

    - by G33kKahuna
    My solution architects and I have exhausted both pure Dataset and Datareader solutions. Basically we have a Microsoft.NET 2.0 windows service application that pulls data based on a query and processes additional tasks per record; almost a poor mans workflow system. The recordsets are broader (in terms of the columns) and deeper (in terms of number of records). We observed that DataSet performs much better in terms of performance but runs into contraints as # of records increase say 100K+ we start seeing System.OutOfMemoryException on a 4G machine with processModel configured to run at memoryLimit set to 85. Since this is a multi-threaded app, there could be multiple threads processing different queries and building different DataSets, so we run into the exception sooner in that case DataReader on the other hand works but is a lot slower and hits other contraints; if there is some sort of disconnect it has to start over again or leaves open connections on the DB side and worst case takes down the service completely etc. So, we decided the best option would be some sort of hybrid solution. I'm open to guidance and suggestions. Are there any hybrid solutions available? Any other suggestions

    Read the article

  • Convert XML to DataSet with rows

    - by lidermin
    Hi, I have an xml file like this: <result> <customer> <id>1</id> <name>A</name> </customer> <customer> <id>2</id> <name>B</name> </customer> </result> So I need that data filled on a DataSet, here is my code: var reader = new StringReader(xmldoc.InnerXml); dsDatos.ReadXml(reader); The problem is that it's filling the dataset with two Tables each one with a single row. But I need a single Table with the two rows. What am I doing wrong? PD: I'm using C# and I don't want to iterate through the XML file, I want to use the ReadXml method. Thanks for your time.

    Read the article

  • updating dataset using join and bindingsource?

    - by netadictos
    Hi, I have created a dataset and in the designer I have created the relations and foreign keys that exist in the database. Basically, I have a product that has a relationship to a table of prices. The keyfield they share is IdProduct in the Prices table. In the Fill/Get of the product I return the Price field. I also have a DataGrid that uses a BindingSource which uses this table. Everything displays correctly and when I double click on a row within the datagrid I then open up a tabbed form that contains a detailed view of the record selected. The user at this point is able to make changes to the record and they are properly propogated back to the BindingSource. The problem is that the TableAdapter does not contain the appopriate update, therefore I am not able to call the TableAdapter.Update method with the dataset as I would had I created a tableadapter not using a join. How am I best to handle this situation. At the same time I cannot get any modified row: dTiendasDs.ProductosDataTable modified = (dTiendasDs.ProductosDataTable) dTiendasDs.Productos.GetChanges(DataRowState.Modified); modified is always null Thanks,

    Read the article

  • Entity Framework LINQ Query using Custom C# Class Method - Once yes, once no - because executing on the client or in SQL?

    - by BrooklynDev
    I have two Entity Framework 4 Linq queries I wrote that make use of a custom class method, one works and one does not: The custom method is: public static DateTime GetLastReadToDate(string fbaUsername, Discussion discussion) { return (discussion.DiscussionUserReads.Where(dur => dur.User.aspnet_User.UserName == fbaUsername).FirstOrDefault() ?? new DiscussionUserRead { ReadToDate = DateTime.Now.AddYears(-99) }).ReadToDate; } The linq query that works calls a from after a from, the equivalent of SelectMany(): from g in oc.Users.Where(u => u.aspnet_User.UserName == fbaUsername).First().Groups from d in g.Discussions select new { UnReadPostCount = d.Posts.Where(p => p.CreatedDate > DiscussionRepository.GetLastReadToDate(fbaUsername, p.Discussion)).Count() }; The query that does not work is more like a regular select: from d in oc.Discussions where d.Group.Name == "Student" select new { UnReadPostCount = d.Posts.Where(p => p.CreatedDate > DiscussionRepository.GetLastReadToDate(fbaUsername, p.Discussion)).Count(), }; The error I get is: LINQ to Entities does not recognize the method 'System.DateTime GetLastReadToDate(System.String, Discussion)' method, and this method cannot be translated into a store expression. My question is, why am I able to use my custom GetLastReadToDate() method in the first query and not the second? I suppose this has something to do with what gets executed on the db server and what gets executed on the client? These queries seem to use the GetLastReadToDate() method so similarly though, I'm wondering why would work for the first and not the second, and most importantly if there's a way to factor common query syntax like what's in the GetLastReadToDate() method into a separate location to be reused in several different places LINQ queries. Please note all these queries are sharing the same object context.

    Read the article

  • Use LINQ, to Sort and Filter items in a List<ReturnItem> collection, based on the values within a Li

    - by Daniel McPherson
    This is tricky to explain. We have a DataTable that contains a user configurable selection of columns, which are not known at compile time. Every column in the DataTable is of type String. We need to convert this DataTable into a strongly typed Collection of "ReturnItem" objects so that we can then sort and filter using LINQ for use in our application. We have made some progress as follows: We started with the basic DataTable. We then process the DataTable, creating a new "ReturnItem" object for each row This "ReturnItem" object has just two properties: ID ( string ) and Columns( List(object) ). The properties collection contains one entry for each column, representing a single DataRow. Each property is made Strongly Typed (int, string, datetime, etc). For example it would add a new "DateTime" object to the "ReturnItem" Columns List containing the value of the "Created" Datatable Column. The result is a List(ReturnItem) that we would then like to be able to Sort and Filter using LINQ based on the value in one of the properties, for example, sort on "Created" date. We have been using the LINQ Dynamic Query Library, which gets us so far, but it doesn't look like the way forward because we are using it over a List Collection of objects. Basically, my question boils down to: How can I use LINQ, to Sort and Filter items in a List(ReturnItem) collection, based on the values within a List(object) property which is part of the ReturnItem class?

    Read the article

  • subset complete or balance dataset in r

    - by SHRram
    I have a dataset that unequal number of repetition. I want to subset a data by removing those entries that are incomplete (i.e. replication less than maximum). Just small example: set.seed(123) mydt <- data.frame (name= rep ( c("A", "B", "C", "D", "E"), c(1,2,4,4, 3)), var1 = rnorm (14, 3,1), var2 = rnorm (14, 4,1)) mydt name var1 var2 1 A 2.439524 3.444159 2 B 2.769823 5.786913 3 B 4.558708 4.497850 4 C 3.070508 2.033383 5 C 3.129288 4.701356 6 C 4.715065 3.527209 7 C 3.460916 2.932176 8 D 1.734939 3.782025 9 D 2.313147 2.973996 10 D 2.554338 3.271109 11 D 4.224082 3.374961 12 E 3.359814 2.313307 13 E 3.400771 4.837787 14 E 3.110683 4.153373 summary(mydt) name var1 var2 A:1 Min. :1.735 Min. :2.033 B:2 1st Qu.:2.608 1st Qu.:3.048 C:4 Median :3.120 Median :3.486 D:4 Mean :3.203 Mean :3.688 E:3 3rd Qu.:3.446 3rd Qu.:4.412 Max. :4.715 Max. :5.787 I want to get rid of A, B, E from the data as they are incomplete. Thus expected output: name var1 var2 4 C 3.070508 2.033383 5 C 3.129288 4.701356 6 C 4.715065 3.527209 7 C 3.460916 2.932176 8 D 1.734939 3.782025 9 D 2.313147 2.973996 10 D 2.554338 3.271109 11 D 4.224082 3.374961 Please note the dataset is big, the following may not a option: mydt[mydt$name == "C",] mydt[mydt$name == "D", ]

    Read the article

  • LINQ to SQL exception: System.OutOfMemoryException

    - by Adam
    Not sure why I keep getting an OutOfMemory exception. I'm using ASP.NET MVC with LINQ to SQL. Here's some of the stack trace: [OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.] System.Runtime.CompilerServices.RuntimeHelpers._CompileMethod(IntPtr method) +0 System.Reflection.Emit.DynamicMethod.CreateDelegate(Type delegateType) +7652553 System.Data.Linq.SqlClient.ObjectReaderCompiler.Compile(SqlExpression expression, Type elementType) +442 System.Data.Linq.SqlClient.SqlProvider.GetReaderFactory(SqlNode node, Type elemType) +100 System.Data.Linq.SqlClient.SqlProvider.System.Data.Linq.Provider.IProvider.Execute(Expression query) +253 System.Data.Linq.Table1.System.Linq.IQueryProvider.Execute(Expression expression) +49 System.Linq.Queryable.Single(IQueryable1 source, Expression`1 predicate) +301 WorkGrabber.Web.Models.WorkGrabberDataContext.GetJob(Int32 id) +233 WorkGrabber.Web.Controllers.BidsController.New(Int32 jobId) +19

    Read the article

  • What's the next big thing after LINQ?

    - by Leniel Macaferi
    I started using LINQ (Language Integrated Query) when it was still in beta, more specifically Microsoft .NET LINQ Preview (May 2006). Almost 4 years have passed and here we are using LINQ in a lot of projects for the most diverse tasks. I even wrote my final college project based on LINQ. You see how I like it. LINQ and more recently PLINQ (Parallel LINQ) give our jobs a great boost when it comes to more programming power and less lines of code leading us to more expressive and readable code. I keep thinking what could be the next big language improvement for C# after LINQ. I know there are some promissing language features coming as Code Contracts, etc, but nothing having the impact that LINQ had. What do you think could be the next big thing?

    Read the article

  • LINQ – TakeWhile and SkipWhile methods

    - by nmarun
    I happened to read about these methods on Vikram's blog and tried testing it. Somehow when I saw the output, things did not seem to add up right. I’m writing this blog to show the actual workings of these methods. Let’s take the same example as showing in Vikram’s blog and I’ll build around it. 1: int[] numbers = { 5, 4, 1, 3, 9, 8, 6, 7, 2, 0 }; 2:  3: foreach(var number in numbers.TakeWhile(n => n < 7)) 4: { 5: Console.WriteLine(number); 6: } Now, the way I (incorrectly) read the upper bound condition in the foreach loop was: ‘Give me all numbers that pass the condition of n<7’. So I was expecting the answer to be: 5, 4, 1, 3, 2, 0. But when I run the application, I see only: 5, 4, 1,3. Turns out I was wrong (happens at least once a day). The documentation on the method says ‘Returns elements from a sequence as long as a specified condition is true. To show in code, my interpretation was the below code’: 1: foreach (var number in numbers) 2: { 3: if (number < 7) 4: { 5: Console.WriteLine(number); 6: } 7: } But the actual implementation is: 1: foreach(var number in numbers) 2: { 3: if(number < 7) 4: { 5: Console.WriteLine(number); 6: break; 7: } 8: } So there it is, another situation where one simple word makes a difference of a whole world. The SkipWhile method has been implemented in a similar way – ‘Bypasses elements in a sequence as long as a specified condition is true and then returns the remaining elements’ and not ‘Bypasses elements in a sequence where a specified condition is true and then returns the remaining elements’. (Subtle.. very very subtle). It’s feels strange saying this, but hope very few require to read this article to understand these methods.

    Read the article

  • LINQ for SQL Developers and DBA’s

    - by AtulThakor
    Firstly I’d just like to thank the guys who organise the SQL Server User Group (Martin/Tony/Chris) and for giving me the opportunity to speak at the recent event. Sorry about the slides taking so long but here they are along with some extra information. Firstly the demo’s were all done using LINQPad 4.0 which can be downloaded here: http://www.linqpad.net/ There are 2 versions 3.5/4.0 With 3.5 you should be able to replicate the problem I showed where a query using a parameter which is X characters long would create a different execution plan to a query which uses a parameter which is Y characters long, otherwise I would just use 4.0 The sample database used is AdventureWorksLT2008 which can be downloaded from here: http://msftdbprodsamples.codeplex.com/releases/view/37109 The scripts have been named so that you can select the appropriate way to run them i.e.: C# expression / C#statement, each script can be run individually be highlighting the query and clicking the play symbol or hitting F5. Scripts and Slides: http://sqlblogcasts.com/blogs/atulthakor/An%20Introduction%20to%20LINQ.zip Please don't hesitate in sending any questions via email/twitter, I’ll try my best to answer your questions! Thanks, Atul

    Read the article

  • Grouping data in LINQ with the help of group keyword

    - by vik20000in
    While working with any kind of advanced query grouping is a very important factor. Grouping helps in executing special function like sum, max average etc to be performed on certain groups of data inside the date result set. Grouping is done with the help of the Group method. Below is an example of the basic group functionality.     int[] numbers = { 5, 4, 1, 3, 9, 8, 6, 7, 2, 0 };         var numberGroups =         from num in numbers         group num by num % 5 into numGroup         select new { Remainder = numGroup.Key, Numbers = numGroup };  In the above example we have grouped the values based on the reminder left over when divided by 5. First we are grouping the values based on the reminder when divided by 5 into the numgroup variable.  numGroup.Key gives the value of the key on which the grouping has been applied. And the numGroup itself contains all the records that are contained in that group. Below is another example to explain the same. string[] words = { "blueberry", "abacus", "banana", "apple", "cheese" };         var wordGroups =         from num in words         group num by num[0] into grp         select new { FirstLetter = grp.Key, Words = grp }; In the above example we are grouping the value with the first character of the string (num[0]). Just like the order operator the group by clause also allows us to write our own logic for the Equal comparison (That means we can group Item by ignoring case also by writing out own implementation). For this we need to pass an object that implements the IEqualityComparer<string> interface. Below is an example. public class AnagramEqualityComparer : IEqualityComparer<string> {     public bool Equals(string x, string y) {         return getCanonicalString(x) == getCanonicalString(y);     }      public int GetHashCode(string obj) {         return getCanonicalString(obj).GetHashCode();     }         private string getCanonicalString(string word) {         char[] wordChars = word.ToCharArray();         Array.Sort<char>(wordChars);         return new string(wordChars);     } }  string[] anagrams = {"from   ", " salt", " earn", "  last   ", " near "}; var orderGroups = anagrams.GroupBy(w => w.Trim(), new AnagramEqualityComparer()); Vikram  

    Read the article

  • Enhancing performance in Entity Framework applications by precompiling LINQ to Entities queries

    - by nikolaosk
    This is going to be the tenth post of a series of posts regarding ASP.Net and the Entity Framework and how we can use Entity Framework to access our datastore. You can find the first one here , the second one here , the third one here , the fourth one here , the fifth one here ,the sixth one here ,the seventh one here ,the eighth one here and the ninth one here . I have a post regarding ASP.Net and EntityDataSource . You can read it here .I have 3 more posts on Profiling Entity Framework applications...(read more)

    Read the article

  • LINQ: Enhancing Distinct With The SelectorEqualityComparer

    - by Paulo Morgado
    On my last post, I introduced the PredicateEqualityComparer and a Distinct extension method that receives a predicate to internally create a PredicateEqualityComparer to filter elements. Using the predicate, greatly improves readability, conciseness and expressiveness of the queries, but it can be even better. Most of the times, we don’t want to provide a comparison method but just to extract the comaprison key for the elements. So, I developed a SelectorEqualityComparer that takes a method that extracts the key value for each element. Something like this: public class SelectorEqualityComparer<TSource, Tkey> : EqualityComparer<TSource> where Tkey : IEquatable<Tkey> { private Func<TSource, Tkey> selector; public SelectorEqualityComparer(Func<TSource, Tkey> selector) : base() { this.selector = selector; } public override bool Equals(TSource x, TSource y) { Tkey xKey = this.GetKey(x); Tkey yKey = this.GetKey(y); if (xKey != null) { return ((yKey != null) && xKey.Equals(yKey)); } return (yKey == null); } public override int GetHashCode(TSource obj) { Tkey key = this.GetKey(obj); return (key == null) ? 0 : key.GetHashCode(); } public override bool Equals(object obj) { SelectorEqualityComparer<TSource, Tkey> comparer = obj as SelectorEqualityComparer<TSource, Tkey>; return (comparer != null); } public override int GetHashCode() { return base.GetType().Name.GetHashCode(); } private Tkey GetKey(TSource obj) { return (obj == null) ? (Tkey)(object)null : this.selector(obj); } } Now I can write code like this: .Distinct(new SelectorEqualityComparer<Source, Key>(x => x.Field)) And, for improved readability, conciseness and expressiveness and support for anonymous types the corresponding Distinct extension method: public static IEnumerable<TSource> Distinct<TSource, TKey>(this IEnumerable<TSource> source, Func<TSource, TKey> selector) where TKey : IEquatable<TKey> { return source.Distinct(new SelectorEqualityComparer<TSource, TKey>(selector)); } And the query is now written like this: .Distinct(x => x.Field) For most usages, it’s simpler than using a predicate.

    Read the article

  • how to update child records when updating the Master table using Linq [closed]

    - by user20358
    I currently use a general repositry class that can update only a single table like so public abstract class MyRepository<T> : IRepository<T> where T : class { protected IObjectSet<T> _objectSet; protected ObjectContext _context; public MyRepository(ObjectContext Context) { _objectSet = Context.CreateObjectSet<T>(); _context = Context; } public IQueryable<T> GetAll() { return _objectSet.AsQueryable(); } public IQueryable<T> Find(Expression<Func<T, bool>> filter) { return _objectSet.Where(filter); } public void Add(T entity) { _objectSet.AddObject(entity); _context.ObjectStateManager.ChangeObjectState(entity, System.Data.EntityState.Added); _context.SaveChanges(); } public void Update(T entity) { _context.ObjectStateManager.ChangeObjectState(entity, System.Data.EntityState.Modified); _context.SaveChanges(); } public void Delete(T entity) { _objectSet.Attach(entity); _context.ObjectStateManager.ChangeObjectState(entity, System.Data.EntityState.Deleted); _objectSet.DeleteObject(entity); _context.SaveChanges(); } } For every table class generated by my EDMX designer I create another class like this public class CustomerRepo : MyRepository<Customer> { public CustomerRepo (ObjectContext context) : base(context) { } } for any updates that I need to make to a particular table I do this: Customer CustomerObj = new Customer(); CustomerObj.Prop1 = ... CustomerObj.Prop2 = ... CustomerObj.Prop3 = ... CustomerRepo.Update(CustomerObj); This works perfectly well when I am updating just to the specific table called Customer. Now if I need to also update each row of another table which is a child of Customer called Orders what changes do I need to make to the class MyRepository. Orders table will have multiple records for a Customer record and multiple fields too, say for example Field1, Field2, Field3. So my questions are: 1.) If I only need to update Field1 of the Orders table for some rows based on a condition and Field2 for some other rows based on a different condition then what changes I need to do? 2.) If there is no such condition and all child rows need to be updated with the same value for all rows then what changes do I need to do? Thanks for taking the time. Look forward to your inputs...

    Read the article

  • LINQ – Skip() and Take() methods

    - by nmarun
    I had this issue recently where I have an array of integers and I’m doing some Skip(n) and then a Take(m) on the collection. Here’s an abstraction of the code: 1: int[] numbers = { 5, 4, 1, 3, 9, 8, 6, 7, 2, 0 }; 2: var taken = numbers.Skip(3).Take(3); 3: foreach (var i in taken) 4: { 5: Console.WriteLine(i); 6: } The output is as expected: 3, 9, 8 – skip the first three and then take the next three items. But, what happens if I do something like: 1: var taken = numbers.Skip(7).Take(5); In English – skip the first seven and the take the next 5 items from an array that contains only 10 elements. Think it’ll throw the IndexOutOfRangeException exception? Nope. These extension methods are a little smarter than that. Even though the user has requested more elements than what exists in the collection, the Take method only returns the first three thereby making the output of the program as: 7, 2, 0. The scenario is handled similarly when you do: 1: var taken = numbers.Take(5).Skip(7); This one takes the first 5 elements from the numbers array and then skips 7 of them. This is what is looks like in the debug mode: Just wanted to share this behavior.

    Read the article

  • Outer Join is not working in Linq Query: The method 'Join' cannot follow the method 'SelectMany' or is not supported

    - by Scorpion
    I am writing the Linq query as below: But on run its throwing the following error: The method 'Join' cannot follow the method 'SelectMany' or is not supported. Try writing the query in terms of supported methods or call the 'AsEnumerable' or 'ToList' method before calling unsupported methods. LINQ from a in AccountSet join sm in new_schoolMemberSet on a.AccountId equals sm.new_OrganisationId.Id into ps from suboc in ps.DefaultIfEmpty() join sr in new_schoolRoleSet on suboc.new_SchoolRoleId.Id equals sr.new_schoolRoleId where sr.new_name == "Manager" where a.new_OrganisationType.Value == 430870007 select new { a.AccountId, a.new_OrganisationType.Value } I am expecting the result as below: I never used the Outer join in Linq before. So please correct me if I am doing it wrong. Thanks

    Read the article

  • Will this LINQ-TO-SQL query fetch all records from the table ?

    - by Puneet Dudeja
    public long GetNewCRN() { return ((from c in DataContext.GetTable<Cust_Master>() select c.CUSTSERH_CRN).Max() + 1); } Will this Linq to Sql query fetch all records from the table first and then select the maximum of the column ? If yes, then isn't it a bad idea using linq to sql instead of normal SqlCommand ? Or is there any other way of doing it in linq to sql ? When I attach Console.Out, I see nothing(command prompt does not even open). But when I include following:- context.Log = new System.IO.StreamWriter("d:\\abcd.txt"); I get an error, that "The process can not access the file because it is being used by another process" and that process is "w3wp.exe". How can I see the sql commands being executed by DataContext then ?

    Read the article

  • Hype and LINQ

    - by Tony Davis
    "Tired of querying in antiquated SQL?" I blinked in astonishment when I saw this headline on the LinqPad site. Warming to its theme, the site suggests that what we need is to "kiss goodbye to SSMS", and instead use LINQ, a modern query language! Elsewhere, there is an article entitled "Why LINQ beats SQL". The designers of LINQ, along with many DBAs, would, I'm sure, cringe with embarrassment at the suggestion that LINQ and SQL are, in any sense, competitive ways of doing the same thing. In fact what LINQ really is, at last, is an efficient, declarative language for C# and VB programmers to access or manipulate data in objects, local data stores, ORMs, web services, data repositories, and, yes, even relational databases. The fact is that LINQ is essentially declarative programming in a .NET language, and so in many ways encourages developers into a "SQL-like" mindset, even though they are not directly writing SQL. In place of imperative logic and loops, it uses various expressions, operators and declarative logic to build up an "expression tree" describing only what data is required, not the operations to be performed to get it. This expression tree is then parsed by the language compiler, and the result, when used against a relational database, is a SQL string that, while perhaps not always perfect, is often correctly parameterized and certainly no less "optimal" than what is achieved when a developer applies blunt, imperative logic to the SQL language. From a developer standpoint, it is a mistake to consider LINQ simply as a substitute means of querying SQL Server. The strength of LINQ is that that can be used to access any data source, for which a LINQ provider exists. Microsoft supplies built-in providers to access not just SQL Server, but also XML documents, .NET objects, ADO.NET datasets, and Entity Framework elements. LINQ-to-Objects is particularly interesting in that it allows a declarative means to access and manipulate arrays, collections and so on. Furthermore, as Michael Sorens points out in his excellent article on LINQ, there a whole host of third-party LINQ providers, that offers a simple way to get at data in Excel, Google, Flickr and much more, without having to learn a new interface or language. Of course, the need to be generic enough to deal with a range of data sources, from something as mundane as a text file to as esoteric as a relational database, means that LINQ is a compromise and so has inherent limitations. However, it is a powerful and beautifully compact language and one that, at least in its "query syntax" guise, is accessible to developers and DBAs alike. Perhaps there is still hope that LINQ can fulfill Phil Factor's lobster-induced fantasy of a language that will allow us to "treat all data objects, whether Word files, Excel files, XML, relational databases, text files, HTML files, registry files, LDAPs, Outlook and so on, in the same logical way, as linked databases, and extract the metadata, create the entities and relationships in the same way, and use the same SQL syntax to interrogate, create, read, write and update them." Cheers, Tony.

    Read the article

  • Importing a large dataset into a database

    - by peaceful
    I'm a beginning programmer in the relevant areas to this question, so if possible, it'd be helpful to avoid assuming I know a lot already. I'm trying to import the OpenLibrary dataset into a local Postgres database. After it's imported, I plan to use it as a starting seed for a Ruby on Rails application that will include information on books. The OpenLibrary datasets are available here, in a modified JSON format: http://openlibrary.org/dev/docs/jsondump I only need very basic information for my application, much less than what is provided in the dumps. I'm only trying to get out book titles, author names, and relationships between books and authors. Below are two typical entries from their dataset, the first for an author, and the second for a book (they seem to have an entry for each edition of a book). The entries seem to lead off with a primary key, and then with a type, before including the actual JSON database dump. /a/OL2A /type/author {"name": "U. Venkatakrishna Rao", "personal_name": "U. Venkatakrishna Rao", "last_modified": {"type": "/type/datetime", "value": "2008-09-10 08:44:01.978456"}, "key": "/a/OL2A", "birth_date": "1904", "type": {"key": "/type/author"}, "id": 99, "revision": 3} /b/OL345M /type/edition {"publishers": ["Social Science Research Project, Dept. of Geography, University of Dacca"], "pagination": "ii, 54 p.", "title": "Land use in Fayadabad area", "lccn": ["sa 65000491"], "subject_place": ["East Pakistan", "Dacca region."], "number_of_pages": 54, "languages": [{"comment": "initial import", "code": "eng", "name": "English", "key": "/l/eng"}], "lc_classifications": ["S471.P162 E23"], "publish_date": "1963", "publish_country": "pk ", "key": "/b/OL345M", "authors": [{"birth_date": "1911", "name": "Nafis Ahmad", "key": "/a/OL302A", "personal_name": "Nafis Ahmad"}], "publish_places": ["Dacca, East Pakistan"], "by_statement": "[by] Nafis Ahmad and F. Karim Khan.", "oclc_numbers": ["4671066"], "contributions": ["Khan, Fazle Karim, joint author."], "subjects": ["Land use -- East Pakistan -- Dacca region."]} The size of the uncompressed dumps are enormous, about 2GB for the authors list, and 18GB for the book editions list. OpenLibrary does not provide any tools for this themselves, they provide a simple unoptimized Python script for reading in sample data (which unlike the actual dumps comes in pure JSON format), but they estimate if that was modified for use on their actual data it would take 2 months (!) to finish loading the data. How can I read this into the database? I assume I'll need to write a program to do this. What language and any guidance on how I should do it to finish in a reasonable amount of time? The only scripting language I have any experience with is Ruby.

    Read the article

  • Dataset and SQLCe in .net 4

    - by Tyler Edwards
    I'm trying to build a SQLCe table structure dynamically using a dataset as the temporary table structure... Is there a SqlConnection string that will work with SqlCe? There used to be a SqlCeConnection...but I don't find that in the 4.0 framework? Has something replaced this? If the answer is L2S or Entity Framework...can I add columns to SqlCe tables at runtime? If so...how?

    Read the article

  • Transfer DataReader rows to a CrystalReport DataSet

    - by lennie
    Please help me as I am a newbie using VB.NET. I have been asked to transfer rows from a DataReader into a Crystal Report dataset. Here are the coding: Private Sub BtnTransfer() dim strsql as string = "Select OrderID, OrderDate from ORDERS" dim DS as new dataset1 '<---crysal report dataset1.xsd dim DR as SqlDataReader dim RW as DataRow = DS.Tables(0).NewRow sqlconn = new sqlconnection(ConnString) sqlcmd = new sqlCommand(strSql, sqlconn) sqlcmd.Connection.open() DR = sqlcmd.ExecuteReader(CommandBehaviour.CloseConnection) Do while DR.Read RW("OrderID") = DR("OrderID") RW("OrderDate") = DR(OrderDate") DS.Tables(0).Rows.ADD(RW) Loop End sub

    Read the article

  • Dataset to Database

    - by user181421
    Hello, I'm working with framework 2.0. I have a dataset with many rows that were entered by the user and I need to write them to the database. What is the right way to do it? TY

    Read the article

< Previous Page | 13 14 15 16 17 18 19 20 21 22 23 24  | Next Page >