Search Results

Search found 1402 results on 57 pages for 'dataset'.

Page 13/57 | < Previous Page | 9 10 11 12 13 14 15 16 17 18 19 20  | Next Page >

  • How to manage a large dataset using Spring MySQL and RowCallbackHandler

    - by rmarimon
    I'm trying to go over each row of a table in MySQL using Spring and a JdbcTemplate. If I'm not mistaken this should be as simple as: JdbcTemplate template = new JdbcTemplate(datasource); template.setFetchSize(1); // template.setFetchSize(Integer.MIN_VALUE) does not work either template.query("SELECT * FROM cdr", new RowCallbackHandler() { public void processRow(ResultSet rs) throws SQLException { System.out.println(rs.getString("src")); } }); I get an OutOfMemoryError because it is trying to read the whole thing. Any ideas?

    Read the article

  • Appending Dataset in Core Data execution of Update through iTuneStore

    - by Yoon Lee
    So I have completed my code work. This is first time releasing the app through iTuneStore. Current state of reading Core Data (.sqlite) file is already prefetched (already has information like apple's 'Reciepie' program). Assuming I have successfully released through apple store, and decide to update my application to existing users. Say I have sqlite contents but it contains bit more information than previous SQLite file under same structure. Question 1. Every time update held to the existing user, does it removes previous ones and move new updated application? Question 2. if it is not, then HOW can I append the existing sql value?

    Read the article

  • DataSetProvider - DataSet to ClientDataSet

    - by nomad311
    I am trying to take data from a TMSQuery which is connected to my DB and populate a ClientDataSet with some of that data using a DataSetProvider. My problem is that I will need to modify some of this data before it can go into my ClientDataSet. The ClientDataSet has persistent fields that will not match up with the raw DB data. I can't even get a string from the DB into a memo field in ClientDataSet. The ClientDataSet is a part of my data tier so I will need to conform the data from the DB to the ClientDataSet field by field (well most will be able to go right through, but many will require routing and/or conversion). Does anyone have experience with this?

    Read the article

  • How can I visualise a "broken" hierarchical dataset?

    I have a reasonably large datatable structured something like this: StaffNo Grade Direct Boss2 Boss3 Boss4 Boss5 Boss6 ------- ----- ----- ----- ----- ----- ----- ----- 10001 1 10002 10002 10057 10094 10043 10099 10002 2 10057 NULL 10057 10094 10043 10099 10003 1 10004 10004 10057 10094 10043 10099 10004 2 10057 NULL 10057 10094 10043 10099 10057 3 10094 NULL NULL 10094 10043 10099 etc.... i.e. a unique id , their level (grade) in the hierarchy, a record of their bosses ID and the IDs of the supervisors above. (The 2,3,4, etc refers to the boss at that particular grade). The system relies on a strict hierarchy - if you are my boss (/parent) then your boss must be my grandparent. Unfortunately this rule is not enforced within the data model and the data ultimately comes from other systems which don't even know about the rule, let alone observe it. So you and I may share the same boss, but our bosses boss won't be the same. note: I cannot change the data model I cannot fix the data at source. So (for the moment) I have to fix the data once it's in place. Once a fortnight someone will do something which breaks the model and I'll need to modify the procs slightly to resolve. Not ideal, but I'm stuck with this for the next six months. Anyway, specific queries are easy to produce but I find it hard to keep track of the bigger picutre. The application which sits on this runs without complaint regardless but navigating around the system becoming extraordinarily confusing. So my question is: Can anyone recommend a tool (or technique) for generating some kind of "broken tree" diagram in this sort of circumstances? I don't want something that will fix things for me, or attempt statistical analysis but at least something that will give a visual indication of how broken it is at any one time. Note : At the moment this is in a SQL Server database but I'm open to ideas utilising C#, Perl or Python.

    Read the article

  • Generic dataset handling library

    - by Pep.
    Hello, I want to build a generic Perl module for handling and analysing biomedical character separated datasets and which can, most certain, be used on any kind of datasets that contain a mixture of categorical (A,B,C,..) and continuous (1.2,3,881..) and identifier (XXX1,XXX2...). The plan is to have people initialize the module and then use some arguments to point to the data file(s), the place were the analysis reports should be placed and the structure of the data. By structure of data I mean which variable is in which place and its name/type. And this is where I need some enlightenment. I am baffled how to do this in a clean way. Obviously, having people create a simple schema file, be it XML or some other format would be the cleanest but maybe not all people enjoy doing something like this. The solutions I can think of are: Create a configuration file in XML or similar and with a prespecified format. Pass the information during initialization of the module. Use the first row of the data as headers and try to guess types (ouch) Surely there must be a "canonical" way of doing this that is also usable and efficient. Thanks p.

    Read the article

  • Hibernate design to speed up querying of large dataset

    - by paddydub
    I currently have the below tables representing a bus network mapped in hibernate, accessed from a Spring MVC based bus route planner I'm trying to make my route planner application perform faster, I load all the above tables into Lists to perform the route planner logic. I would appreciate if anyone has any ideas of how to speed my performace Or any suggestions of another method to approach this problem of handling a large set of data Coordinate Connections Table (INT,INT,INT, DOUBLE)( Containing 50,000 Coordinate Connections) ID, FROMCOORDID, TOCOORDID, DISTANCE 1 1 2 0.383657 2 1 17 0.173201 3 1 63 0.258781 4 1 64 0.013726 5 1 65 0.459829 6 1 95 0.458769 Coordinate Table (INT,DECIMAL, DECIMAL) (Containing 4700 Coordinates) ID , LAT, LNG 0 59.352669 -7.264341 1 59.352669 -7.264341 2 59.350012 -7.260653 3 59.337585 -7.189798 4 59.339221 -7.193582 5 59.341408 -7.205888 Bus Stop Table (INT, INT, INT)(Containing 15000 Stops) StopID RouteID COORDINATEID 1000100001 100 17 1000100002 100 18 1000100003 100 19 1000100004 100 20 1000100005 100 21 1000100006 100 22 1000100007 100 23 This is how long it takes to load all the data from each table: stop.findAll = 148ms, stops.size: 15670 Hibernate: select coordinate0_.COORDINATEID as COORDINA1_2_, coordinate0_.LAT as LAT2_, coordinate0_.LNG as LNG2_ from COORDINATES coordinate0_ coord.findAll = 51ms , coordinates.size: 4704 Hibernate: select coordconne0_.COORDCONNECTIONID as COORDCON1_3_, coordconne0_.DISTANCE as DISTANCE3_, coordconne0_.FROMCOORDID as FROMCOOR3_3_, coordconne0_.TOCOORDID as TOCOORDID3_ from COORDCONNECTIONS coordconne0_ coordinateConnectionDao.findAll = 238ms ; coordConnectioninates.size:48132 Hibernate Annotations @Entity @Table(name = "STOPS") public class Stop implements Serializable { @Id @GeneratedValue(strategy = GenerationType.AUTO) @Column(name = "STOPID") private int stopID; @Column(name = "ROUTEID", nullable = false) private int routeID; @ManyToOne(fetch = FetchType.LAZY) @JoinColumn(name = "COORDINATEID", nullable = false) private Coordinate coordinate; } @Table(name = "COORDINATES") public class Coordinate { @Id @GeneratedValue @Column(name = "COORDINATEID") private int CoordinateID; @Column(name = "LAT") private double latitude; @Column(name = "LNG") private double longitude; } @Entity @Table(name = "COORDCONNECTIONS") public class CoordConnection { @Id @GeneratedValue @Column(name = "COORDCONNECTIONID") private int CoordinateID; @ManyToOne(fetch = FetchType.LAZY) @JoinColumn(name = "FROMCOORDID", nullable = false) private Coordinate fromCoordID; @ManyToOne(fetch = FetchType.LAZY) @JoinColumn(name = "TOCOORDID", nullable = false) private Coordinate toCoordID; @Column(name = "DISTANCE", nullable = false) private double distance; }

    Read the article

  • SQL query duration is longer for smaller dataset?

    - by entens
    I received reports that a my report generating application was not working. After my initial investigation, I found that the SQL transaction was timing out. I'm mystified as to why the query for a smaller selection of items would take so much longer to return results. Quick query (averages 4 seconds to return): SELECT * FROM Payroll WHERE LINEDATE >= '04-17-2010'AND LINEDATE <= '04-24-2010' ORDER BY 'EMPLYEE_NUM' ASC, 'OP_CODE' ASC, 'LINEDATE' ASC Long query (averages 1 minute 20 seconds to return): SELECT * FROM Payroll WHERE LINEDATE >= '04-18-2010'AND LINEDATE <= '04-24-2010' ORDER BY 'EMPLYEE_NUM' ASC, 'OP_CODE' ASC, 'LINEDATE' ASC I could simply increase the timeout on the SqlCommand, but it doesn't change the fact the query is taking longer than it should. Why would requesting a subset of the items take longer than the query that returns more data? How can I optimize this query?

    Read the article

  • Using data from a dataset in styles with Mapnik

    - by Alex King
    I've setup Mapnik to connect to a PostGIS database, and display geometry. I'd like to have a column in my database called opacity, and use it as the opacity for that row of geometry when Mapnik renders it. So far, I've only found information on how to display text from the database, and how to use filters to display different styles when database values are within parameters. Nothing about how to use the values directly inside of a style or layer though - is this possible?

    Read the article

  • Update thousands of records in a DataSet to SQL Server

    - by MSIL
    I have half a million records in a data set of which 50,000 are updated. Now I need to commit the updated records back to the SQL Server 2005 Database. What is the best and efficient way to do this considering the fact that such updates could be frequent (though concurrency is not an issue but performance is)

    Read the article

  • how to handle large dataset like sproutcore

    - by Nik
    Hello all, I really don't have any substantial code to show here, actually, that's kinda why I am writing: I looked at the SproutCore demo, especially the Collection demo, on http://demo.sproutcore.com/sample_controls/, and am amazed by its loading 200,000 records to the page so easily. I tried using Rails to provide 200,000 records and in a completely blank HTML page with <% @projects.each do |p| % <%= p.title % <% end % that freezes the browser for seconds on my m1530 laptop with 4gb ram and t7700 256gb ssd. Yet the sproutcore demo does not freeze and takes less than 3 seconds to load. What do you think the one technique they are using to enable this is? Thanks!

    Read the article

  • How to use SqlDataSource for filling combobox as well as datatable or Dataset

    - by Shantanu Gupta
    I am trying to fetch a column value from a datasource when some value is selected from a dropdownlist on its change event. <asp:DropDownList ID="ddlCityName" runat="server" DataSourceID="dsCity" DataTextField="CityName" DataValueField="CityID" AutoPostBack="True" OnTextChanged="CityName_OnTextChanged"> </asp:DropDownList> <asp:SqlDataSource ID="dsCity" runat="server" ConnectionString="<%$ ConnectionStrings:GmapConnectionString %>" SelectCommand="SELECT * FROM [vcity]" ></asp:SqlDataSource> Here I want to fetch any other column's value that is not binded to a ddlCityName from sqldatasource. I have four columns in datasource i.e. name, id, address, phno. I want to fetch an address of a person who selects some value from ddl.

    Read the article

  • Transform LINQ Dataset into a Matrix for export

    - by Mad Halfling
    Hi folks, I've got a data table with columns in which include Item, Category and Value (and others, but those are the only relevant ones for this problem) that I access via LINQ in a C# ASP.Net MVC app. I want to transform these into a matrix and output that as a CSV file to pull into Excel as matrix with the items down the side, the categories across the top and the values in the row cells. However, I don't know how many, or what, categories there will be in this table, nor will there always be a record for each item/category combination. I've written this by looping round, getting my "master category" list, then looking again for each item, filling in either blank or Value, depending on whether the item/category record exists, but as there are currently 27000 records in the table, this isn't as fast as I'd like. Is there a slicker and faster way I can do this, maybe via LINQ (firing into a quicker SQL statement so the DB server can do the leg-work), or will any method essentially come back to what I am doing? Thx MH

    Read the article

  • The most efficient way to calculate an integral in a dataset range

    - by Annalisa
    I have an array of 10 rows by 20 columns. Each columns corresponds to a data set that cannot be fitted with any sort of continuous mathematical function (it's a series of numbers derived experimentally). I would like to calculate the integral of each column between row 4 and row 8, then store the obtained result in a new array (20 rows x 1 column). I have tried using different scipy.integrate modules (e.g. quad, trpz,...). The problem is that, from what I understand, scipy.integrate must be applied to functions, and I am not sure how to convert each column of my initial array into a function. As an alternative, I thought of calculating the average of each column between row 4 and row 8, then multiply this number by 4 (i.e. 8-4=4, the x-interval) and then store this into my final 20x1 array. The problem is...ehm...that I don't know how to calculate the average over a given range. The question I am asking are: Which method is more efficient/straightforward? Can integrals be calculated over a data set like the one that I have described? How do I calculate the average over a range of rows?

    Read the article

  • how to visualize (value, count) dataset with thousands data points

    - by user510040
    I have a file with 2 numeric columns: value and count. File may have 5000 rows. I do plot(value, count) to find the shape of distribution. But because there are too many data points the picture is not very clear. Do you know better visualization approach? Probably histograms or barplot with grouping close values on x axis will be the better way to look on data? I cannot figure out the syntax of using histogram or barplot for my case.

    Read the article

  • Is this OleDbDataAdapter bug

    - by ????
    It doesn't look to me OleDbDataAdapter should throw an exception on trying to fill DataSet for a db table column of type decimal(28,3). The message is "The numerical value is too large to fit into a 96 bit decimal". Could you just check this, I have no significant experience with ADO.NET and OLE DB components? The VB.NET code we have in the application is this: Dim dbDataSet As New DataSet Dim dbDataAdapter As OleDbDataAdapter Dim dbCommand As OleDbCommand Dim conn As OleDbConnection Dim connectionString As String 'parts where connectionString is set conn = New OleDbConnection(connectionString) 'part where sqlQuery is set but it ends up being "SELECT Price As 'Price' From PricebookView" - Price is of type decimal(28,3) dbCommand = New OleDbCommand(sqlQuery, conn) dbCommand.CommandTimeout = cmdTimeout dbDataAdapter = New OleDbDataAdapter(dbCommand) dbDataAdapter.Fill(dbDataSet) The last line is where the exception is thrown and the top of the stack trace is: at System.Data.ProviderBase.DbBuffer.ReadNumeric(Int32 offset) at System.Data.OleDb.ColumnBinding.Value_NUMERIC() at System.Data.OleDb.ColumnBinding.Value() at System.Data.OleDb.OleDbDataReader.GetValues(Object[] values) at System.Data.ProviderBase.DataReaderContainer.CommonLanguageSubsetDataReader.GetValues(Object[] values) at System.Data.ProviderBase.SchemaMapping.LoadDataRow() at System.Data.Common.DataAdapter.FillLoadDataRow(SchemaMapping mapping) at System.Data.Common.DataAdapter.FillFromReader(DataSet dataset, DataTable datatable, String srcTable, DataReaderContainer dataReader, Int32 startRecord, Int32 maxRecords, DataColumn parentChapterColumn, Object parentChapterValue) at System.Data.Common.DataAdapter.Fill(DataSet dataSet, String srcTable, IDataReader dataReader, Int32 startRecord, Int32 maxRecords) at System.Data.Common.DbDataAdapter.FillInternal(DataSet dataset, DataTable[] datatables, Int32 startRecord, Int32 maxRecords, String srcTable, IDbCommand command, CommandBehavior behavior) at System.Data.Common.DbDataAdapter.Fill(DataSet dataSet, Int32 startRecord, Int32 maxRecords, String srcTable, IDbCommand command, CommandBehavior behavior) at System.Data.Common.DbDataAdapter.Fill(DataSet dataSet) ... I am not sure why does it try to set the value to Int32. Thank you for the time !

    Read the article

  • Slow Update/insert into SQL Server CE using LinqToDatasets

    - by Vaccano
    I have a mobile app that is using LinqToDatasets to update/insert into a SQL Server CE 3.5 File. My Code looks like this: // All the MyClass Updates MyTableAdapter myTableAdapter = new MyTableAdapter(); foreach (MyClassToInsert myClass in updates.MyClassChanges) { // Update the row if it is already there int result = myTableAdapter.Update(myClass.FirstColumn, myClass.SecondColumn, myClass.FirstColumn); // If the row was not there then insert it. if (result == 0) { myTableAdapter.Insert(myClass.FirstColumn, myClass.SecondColumn); } } This code is used to keep the hand held database in sync with the server database. Problem is if it is a full update (first time for example) there are a lot of updates (about 125). That makes this code (and more loops like it take a very long time (I have three such loops that take over 30 seconds each). Is there a faster or better way to do updates/inserts like this? (I did see this Codeplex Project, but I could not see how to make it work with both updates and inserts.)

    Read the article

  • Why doesn't EnumerableRowCollection<DataRow>.Select() compile like this?

    - by David Fox
    This works: from x in table.AsEnumerable() where x.Field<string>("something") == "value" select x.Field<decimal>("decimalfield"); but, this does not: from x in table.AsEnumerable() .Where(y=>y.Field<string>("something") == "value") .Select(y=>y.Field<decimal>("decimalfield")); I also tried: from x in table.AsEnumerable() .Where(y=>y.Field<string>("something") == "value") .Select(y=>new { name = y.Field<decimal>("decimalfield") }); Looking at the two overloads of the .Select() method, I thought the latter two should both return EnumerableRowCollection, but apparently I am wrong. What am I missing?

    Read the article

  • python union of 2 nested lists with index

    - by sbas
    I want to get the union of 2 nested lists plus an index to the common values. I have two lists like A = [[1,2,3],[4,5,6],[7,8,9]] and B = [[1,2,3,4],[3,3,5,7]] but the length of each list is about 100 000. To A belongs an index vector with len(A): I = [2,3,4] What I want is to find all sublists in B where the first 3 elements are equal to a sublist in A. In this example I want to get B[0] returned ([1,2,3,4]) because its first three elements are equal to A[0]. In addition, I also want the index to A[0] in this example, that is I[0]. I tried different things, but nothing worked so far :( First I tried this: Common = [] for i in range(len(B)): if B[i][:3] in A: id = [I[x] for x,y in enumerate(A) if y == B[i][:3]][0] ctdCommon.append([int(id)] + B[i]) But that takes ages, or never finishes Then I transformed A and B into sets and took the union from both, which was very quick, but then I don't know how to get the corresponding indices Does anyone have an idea?

    Read the article

  • Using LINQ to fetch result from nested SQL queries

    - by Shantanu Gupta
    This is my first question and first day in Linq so bit difficult day for me to understand. I want to fetch some records from database i.e. select * from tblDepartment where department_id in ( select department_id from tblMap where Guest_Id = @GuestId ) I have taken two DataTable. i.e. tblDepartment, tblMap Now I want to fetch this result and want to store it in third DataTable. How can I do this. I have been able to construct this query up till now after googling. var query = from myrow in _dtDepartment.AsEnumerable() where myrow.Field<int>("Department_Id") == _departmentId select myrow; Please provide me some link for learning Linq mainly for DataTables and DataSets. EDIT: I have got a very similar example here but i m still not able to understand how it is working. Please put some torch on it.

    Read the article

  • VBNet2008 Transfer DATAREADER rows to CrystalReport DataSet1.xsd

    - by lennie
    Hi Good Guys, I need your help. Please help me as I am a newbie using VB.NET. I have been asked to transfer DATAREADER rows into Crystal Report DATASET1.XSD Here are the coding Private Sub BtnTransfer() dim strsql as string = "Select OrderID, OrderDate from ORDERS" dim DS as new dataset1 '<---crysal report dataset1.xsd dim DR as SqlDataReader dim RW as DataRow = DS.Tables(0).NewRow sqlconn = new sqlconnection(ConnString) sqlcmd = new sqlCommand(strSql, sqlconn) sqlcmd.Connection.open() DR = sqlcmd.ExecuteReader(CommandBehaviour.CloseConnection) Do while DR.READ RW("OrderID") = DR("OrderID") RW("OrderDate") = DR(OrderDate") DS.Tables(0).Rows.ADD(RW) Loop End sub

    Read the article

< Previous Page | 9 10 11 12 13 14 15 16 17 18 19 20  | Next Page >