Search Results

Search found 14354 results on 575 pages for 'existing records'.

Page 105/575 | < Previous Page | 101 102 103 104 105 106 107 108 109 110 111 112  | Next Page >

  • LINQ to SQL - Tracking New / Dirty Objects

    - by Joseph Sturtevant
    Is there a way to determine if a LINQ object has not yet been inserted in the database (new) or has been changed since the last update (dirty)? I plan on binding my UI to LINQ objects (using WPF) and need it to behave differently depending whether or not the object is already in the database. MyDataContext context = new MyDataContext(); MyObject obj; if (new Random().NextDouble() > .5) obj = new MyObject(); else obj = context.MyObjects.First(); // How can I distinguish these two cases? The only simple solution I can think of is to set the primary key of new records to a negative value (my PKs are an identity field and will therefore be set to a positive integer on INSERT). This will only work for detecting new records. It also requires identity PKs, and requires control of the code creating the new object. Is there a better way to do this? It seems like LINQ must be internally tracking the status of these objects so that it can know what to do on context.SubmitChanges(). Is there some way to access that "object status"? Clarification Apparently my initial question was confusing. I'm not looking for a way to insert or update records. I'm looking for a way, given any LINQ object, to determine if that object has not been inserted (new) or has been changed since its last update (dirty).

    Read the article

  • BackgroundWorker From ASP.Net Application

    - by Kevin
    We have an ASP.Net application that provides administrators to work with and perform operations on large sets of records. For example, we have a "Polish Data" task that an administrator can perform to clean up data for a record (e.g. reformat phone numbers, social security numbers, etc.) When performed on a small number of records, the task completes relatively quickly. However, when a user performs the task on a larger set of records, the task may take several minutes or longer to complete. So, we want to implement these kinds of tasks using some kind of asynchronous pattern. For example, we want to be able to launch the task, and then use AJAX polling to provide a progress bar and status information. I have been looking into using the BackgroundWorker class, but I have read some things online that make me pause. I would love to get some additional advice on this. For example, I understand that the BackgroundWorker will actually use the thread pool from the current application. In my case, the application is an ASP.Net web site. I have read that this can be a problem because when the application recycles, the background workers will be terminated. Some of the jobs I mentioned above may take 3 minutes, but others may take a few hours. Also, we may have several hundred administrators all performing similar operations during the day. Will the ASP.Net application thread pool be able to handle all of these background jobs efficiently while still performing it's normal request processing? So, I am trying to determine if using the BackgroundWorker class and approach is right for our needs. Should I be looking at an alternative approach? Thanks and sorry for such a long post! Kevin

    Read the article

  • php cli script hangs with no messages

    - by julio
    Hi-- I've written a PHP script that runs via SSH and nohup, meant to process records from a database and do stuff with them (eg. process some images, update some rows). It works fine with small loads, up to maybe 10k records. I have some larger datasets that process around 40k records (not a lot, I realize, but it adds up to a lot of work when each record requires the download and processing of up to 50 images). The larger datasets can take days to process. Sometimes I'll see in my debug logs memory errors, which are clear enough-- but sometimes the script just appears to "die" or go zombie on me. My tail of the debug log just stops, with no error messages, the tail of the nohup log ends with no error, and the process is still showing in a ps list, looking like this-- 26075 pts/0 S 745:01 /usr/bin/php ./import.php but no work is getting done. Can anyone give me some ideas on why a process would just quit? The obvious things (like a php script timeout and memory issues) are not a factor, as far as I can tell. Thanks for any tips PS-- this is hosted on a godaddy VDS (not my choice). I am sort of suspecting that godaddy has some kind of limits that might kick in on me despite what overrides I put in the code (such as set_time_limit(0);).

    Read the article

  • Paging with Find using Active Record

    - by Brian Rizzo
    I can't seem to find an answer to this question or a good example of how to accomplish what I am trying to do. I'm sure it's been posted or explained somewhere, but I am having trouble finding the exact solution I need. I am using ActiveRecord in Subsonic 3.0.0.3. When I do something like recordset = VehicleModel.Find(x => x.Model.StartsWith(SearchText)); I get back an IList of VehicleModel objects (or more simply a recordset), this is fine until I return too many records. I also cannot order the returned set of records (my grid will do this fine, but i'm sure it will be too slow if i have too many records). Being that Find is returning an IList there isn't much that I can run directly against this (again I may be overlooking something simple so please don't kill me). My question is can someone explain how to find data like i am above, sort it and get a page of data where a page is of size n? Am I going about this wrong? Am I even close to being on the right track?

    Read the article

  • How can I stop an auto-generated Linq to SQL class from loading ALL data?

    - by Gary McGill
    I have an ASP.NET MVC project, much like the NerdDinner tutorial example. (I'm using MVC 2, but followed the NerdDinner tutorial in order to create it). As per the instructions in part 3 of the tutorial, I've created a Linq-to-SQL model of my database by creating a "Linq to SQL Classes" (.dbml) surface, and dropping my database tables onto it. The designer has automatically added relationships between the generated classes based on my database tables. Let's say that my classes are as per the NerdDinner example, so I have Dinner and RSVP tables, where each Dinner record is associated with many RSVP records - hence in the generated classes, the Dinner object has a RSVPs property which is a list of RSVP objects. My problem is this: it appears (and I'd be gladly proved wrong on this) that as soon as I access a Dinner object, it's loading all of the corresponding RSVP objects, even if I don't use the RSVPs member. First question: is this really the default behavior for the generated classes? In my particular situation, the object graph contains many more tables (which have an order of magnitude more records), and so this is disastrous behaviour - I'd be loading tons of data when all I want to do is show the details of a single parent record. Second question: are there any properties exposed through the designer UI that would let me modify this behavior? (I can't find any). Third question: I've seen a description of how to control the loading of related records in a DataContext by using a DataShape object associated with the DataContext. Is that what I'm meant to do, and if so are there any tutorials like the NerdDinner one that would show not only how to do it, but also suggest a 'pattern' for normal use?

    Read the article

  • How can I stop an auto-generated Linq to SQL class from loading ALL data?

    - by Gary McGill
    DUPLICATE of http://stackoverflow.com/questions/2433422/how-can-i-stop-an-auto-generated-linq-to-sql-class-from-loading-all-data post answers there! I have an ASP.NET MVC project, much like the NerdDinner tutorial example. (I'm using MVC 2, but followed the NerdDinner tutorial in order to create it). As per the instructions in part 3 of the tutorial, I've created a Linq-to-SQL model of my database by creating a "Linq to SQL Classes" (.dbml) surface, and dropping my database tables onto it. The designer has automatically added relationships between the generated classes based on my database tables. Let's say that my classes are as per the NerdDinner example, so I have Dinner and RSVP tables, where each Dinner record is associated with many RSVP records - hence in the generated classes, the Dinner object has a RSVPs property which is a list of RSVP objects. My problem is this: it appears (and I'd be gladly proved wrong on this) that as soon as I access a Dinner object, it's loading all of the corresponding RSVP objects, even if I don't use the RSVPs member. First question: is this really the default behavior for the generated classes? In my particular situation, the object graph contains many more tables (which have an order of magnitude more records), and so this is disastrous behaviour - I'd be loading tons of data when all I want to do is show the details of a single parent record. Second question: are there any properties exposed through the designer UI that would let me modify this behavior? (I can't find any). Third question: I've seen a description of how to control the loading of related records in a DataContext by using a DataShape object associated with the DataContext. Is that what I'm meant to do, and if so are there any tutorials like the NerdDinner one that would show not only how to do it, but also suggest a 'pattern' for normal use?

    Read the article

  • PostgreSQL - Why are some queries on large datasets so incredibly slow

    - by Brad Mathews
    Hello, I have two types of queries I run often on two large datasets. They run much slower than I would expect them to. The first type is a sequential scan updating all records: Update rcra_sites Set street = regexp_replace(street,'/','','i') rcra_sites has 700,000 records. It takes 22 minutes from pgAdmin! I wrote a vb.net function that loops through each record and sends an update query for each record (yes, 700,000 update queries!) and it runs in less than half the time. Hmmm.... The second type is a simple update with a relation and then a sequential scan: Update rcra_sites as sites Set violations='No' From narcra_monitoring as v Where sites.agencyid=v.agencyid and v.found_violation_flag='N' narcra_monitoring has 1,700,000 records. This takes 8 minutes. The query planner refuses to use my indexes. The query runs much faster if I start with a set enable_seqscan = false;. I would prefer if the query planner would do its job. I have appropriate indexes, I have vacuumed and analyzed. I optimized my shared_buffers and effective_cache_size best I know to use more memory since I have 4GB. My hardware is pretty darn good. I am running v8.4 on Windows 7. Is PostgreSQL just this slow? Or am I still missing something? Thanks! Brad

    Read the article

  • Parsing large delimited files with dynamic number of columns

    - by annelie
    Hi, What would be the best approach to parse a delimited file when the columns are unknown before parsing the file? The file format is Rightmove v3 (.blm), the structure looks like this: #HEADER# Version : 3 EOF : '^' EOR : '~' #DEFINITION# AGENT_REF^ADDRESS_1^POSTCODE1^MEDIA_IMAGE_00~ // can be any number of columns #DATA# agent1^the address^the postcode^an image~ agent2^the address^the postcode^^~ // the records have to have the same number of columns as specified in the definition, however they can be empty etc #END# The files can potentially be very large, the example file I have is 40Mb but they could be several hundred megabytes. Below is the code I had started on before I realised the columns were dynamic, I'm opening a filestream as I read that was the best way to handle large files. I'm not sure my idea of putting every record in a list then processing is any good though, don't know if that will work with such large files. List<string> recordList = new List<string>(); try { using (FileStream fs = new FileStream(path, FileMode.Open, FileAccess.Read)) { StreamReader file = new StreamReader(fs); string line; while ((line = file.ReadLine()) != null) { string[] records = line.Split('~'); foreach (string item in records) { if (item != String.Empty) { recordList.Add(item); } } } } } catch (FileNotFoundException ex) { Console.WriteLine(ex.Message); } foreach (string r in recordList) { Property property = new Property(); string[] fields = r.Split('^'); // can't do this as I don't know which field is the post code property.PostCode = fields[2]; // etc propertyList.Add(property); } Any ideas of how to do this better? It's C# 3.0 and .Net 3.5 if that helps. Thanks, Annelie

    Read the article

  • How to read a database record with a DataReader and add it to a DataTable

    - by Olga
    Hello I have some data in a Oracle database table(around 4 million records) which i want to transform and store in a MSSQL database using ADO.NET. So far i used (for much smaller tables) a DataAdapter to read the data out of the Oracle DataBase and add the DataTable to a DataSet for further processing. When i tried this with my huge table, there was a outofmemory exception thrown. ( I assume this is because i cannot load the whole table into my memory) :) Now i am looking for a good way to perform this extract/transfer/load, without storing the whole table in the memory. I would like to use a DataReader and read the single dataRecords in a DataTable. If there are about 100k rows in it, I would like to process them and clear the DataTable afterwards(to have free memory again). Now i would like to know how to add a single datarecord as a row to a dataTable with ado.net and how to completly clear the dataTable out of memory: My code so far: Dim dt As New DataTable Dim count As Int32 count = 0 ' reads data records from oracle database table' While rdr.Read() 'read n records and add them to a dataTable' While count < 10000 dt.Rows.Add(????) count = count + 1 End While 'transform data in the dataTable, and insert it to the destination' ' flush the dataTable after insertion' count = 0 End While Thank you very much for your response!

    Read the article

  • searching XML documents using php

    - by dbomb101
    I am trying to make a search function using the combination of DOM, PHP and XML. I got something up and running but the problem is that my search function will only accept exact terms, on top of this am wondering if the method I picked the most efficient $searchTerm = "Lupe"; $doc = new DOMDocument(); foreach (file('musicInformation.xml')as $node) { $xmlString .= trim($node); } $doc->loadXML($xmlString); $records = $doc->documentElement->childNodes; $records = $doc->getElementsByTagName("musicdetails"); foreach( $records as $record ) { $artistnames = $record->getElementsByTagName("artistname"); $artistname = $artistnames->item(0)->nodeValue; $recordnames = $record->getElementsByTagName("recordname"); $recordname = $recordnames->item(0)->nodeValue; $recordtypes = $record->getElementsByTagName("recrodtype"); $recordtype = $recordtypes->item(0)->nodeValue; $formats = $record->getElementsByTagName("format"); $format = $formats->item(0)->nodeValue; $prices = $record->getElementsByTagName("price"); $price = $prices->item(0)->nodeValue; if($searchTerm == $artistname|| $searchTerm == $recordname || $searchTerm == $recordtype ||$searchTerm == $format || $searchTerm == $price) { echo "$artistname - $recordname - $recordtype - $format -$price\n"; }

    Read the article

  • Self-referencing tables in Linq2Sql

    - by J-Man
    Hi, I've seen a lot of questions on self-referencing tables in Linq2Sql and how to eagerly load all child records for a particular root object. I've implemented a temporary solution by accessing all underlying properties, but you can see that this doesn't do the performance any good. The thing is though, that all records are correlated with each-other using a correlation GUID. Example below: RootElement - Id: 1 - ParentId: null - CorrelationId: 4D68E512-4B55-44f4-BA5A-174B630A03DD ChildElement1 - Id: 2 - ParentId: 1 - CorrelationId: 4D68E512-4B55-44f4-BA5A-174B630A03DD ChildElement2 - Id: 3 - ParentId: 2 - CorrelationId: 4D68E512-4B55-44f4-BA5A-174B630A03DD ChildElement1 - Id: 4 - ParentId: 2 - CorrelationId: 4D68E512-4B55-44f4-BA5A-174B630A03DD In my case, I do have access to the correlationId, so I can retrieve all of my records by performing the following query: from element in db.Elements where element.CorrelationId == '4D68E512-4B55-44f4-BA5A-174B630A03DD' select element; But, of course, I want these elements associated with each other by executing this query: from element in db.Elements where element.CorrelationId == '4D68E512-4B55-44f4-BA5A-174B630A03DD' && element.ParentId == null select element; My question is: is it possible to combine the results the first query as some sort of 'caching mechanism' for the query where I get the root element? Thanks for the input. J.

    Read the article

  • ASP.NET MVC: How to validate an Ajax form with a specified UpdateTargetID?

    - by Bryan Roth
    I'm trying to figure out how to show validation errors after a user submits an Ajax form that has its UpdateTargetID property set. I'm stumped on how to update the Ajax form with the validation errors without returning the Create PartialView into the results div. If the form is valid, then it should return the Records PartialView. Create.ascx <% Using Ajax.BeginForm("Create", "Record", New Record With {.UserID = Model.UserID}, New AjaxOptions With { .UpdateTargetId = "results", .LoadingElementId = "loader" })%> Date Located <%= Html.TextBoxFor(Function(model) model.DateLocated)%> <%= Html.ValidationMessageFor(Function(model) model.DateLocated) %> Description <%= Html.TextBoxFor(Function(model) model.Description)%> <%= Html.ValidationMessageFor(Function(model) model.Description) %> <input id="btnSave" type="submit" value="Create" /> <span id="loader" class="loader">Saving...</span> <%End Using%> Records.ascx <div id="results"> ... </div> RecordController.vb Function Create(ByVal newRecord As Record) As ActionResult ValidateRecord(newRecord) If Not ModelState.IsValid Then Return PartialView("Create", newRecord) End If _repository.Add(newRecord) _repository.Save() Dim user = _repository.GetUser(newRecord.UserID) Return PartialView("Records", user) End Function

    Read the article

  • javascript table sorting/paging (client-side). How big is too big?

    - by Aheho
    I'm using a jQuery plugin called Tablesorter to do client-side sorting of a log table in one of my applications. I am also making use of the tablepager add-in. I really like the responsiveness that client-side sorting and paging brings to the party. I also like how you don't have to hit the web server or database repeatedly. However I can see that, in time, the log I'm displaying could grow quite large. I'm sure there comes a point where client-side paging and sorting is going to be impractical. What point will this technique begin to collapse under it's own weight? 500 records? 2000 records? 10,000 records? EDIT: In nutshell, what criteria would you use to determine if you are going to use client-side sorting/paging as opposed to server-side paging? Does the size of expected result set factor into your decision? Where is the tipping point?

    Read the article

  • Looking for advice on importing large dataset in sqlite and Cocoa/Objective-C

    - by jluckyiv
    I have a fairly large hierarchical dataset I'm importing. The total size of the database after import is about 270MB in sqlite. My current method works, but I know I'm hogging memory as I do it. For instance, if I run with Zombies, my system freezes up (although it will execute just fine if I don't use that Instrument). I was hoping for some algorithm advice. I have three hierarchical tables comprising about 400,000 records. The highest level has about 30 records, the next has about 20,000, the last has the balance. Right now, I'm using nested for loops to import. I know I'm creating an unreasonably large object graph, but I'm also looking to serialize to JSON or XML because I want to break up the records into downloadable chunks for the end user to import a la carte. I have the code written to do the serialization, but I'm wondering if I can serialize the object graph if I only have pieces in memory. Here's pseudocode showing the basic process for sqlite import. I left out the unnecessary detail. [database open]; [database beginTransaction]; NSArray *firstLevels = [[FirstLevel fetchFromURL:url retain]; for (FirstLevel *firstLevel in firstLevels) { [firstLevel save]; int id1 = [firstLevel primaryKey]; NSArray *secondLevels = [[SecondLevel fetchFromURL:url] retain]; for (SecondLevel *secondLevel in secondLevels) { [secondLevel saveWithForeignKey:id1]; int id2 = [secondLevel primaryKey]; NSArray *thirdLevels = [[ThirdLevel fetchFromURL:url] retain]; for (ThirdLevel *thirdLevel in thirdLevels) { [thirdLevel saveWithForeignKey:id2]; } [database commit]; [database beginTransaction]; [thirdLevels release]; } [secondLevels release]; } [database commit]; [database release]; [firstLevels release];

    Read the article

  • SQL Server: Output an XML field as tabular data using a stored procedure

    - by Pawan
    I am using a table with an XML data field to store the audit trails of all other tables in the database. That means the same XML field has various XML information. For example my table has two records with XML data like this: 1st record: <client> <name>xyz</name> <ssn>432-54-4231</ssn> </client> 2nd record: <emp> <name>abc</name> <sal>5000</sal> </emp> These are the two sample formats and just two records. The table actually has many more XML formats in the same field and many records in each format. Now my problem is that upon query I need these XML formats to be converted into tabular result sets. What are the options for me? It would be a regular task to query this table and generate reports from it. I want to create a stored procedure to which I can pass that I need to query "<emp>" or "<client>", then my stored procedure should return tabular data.

    Read the article

  • SqlBulkCopy slow as molasses

    - by Chris
    I'm looking for the fastest way to load bulk data via c#. I have this script that does the job but slow. I read testimonies that SqlBulkCopy is the fastest. 1000 records 2.5 seconds. files contain anywhere near 5000 records to 250k What are some of the things that can slow it down? Table Def: CREATE TABLE [dbo].[tempDispositions]( [QuotaGroup] [varchar](100) NULL, [Country] [varchar](50) NULL, [ServiceGroup] [varchar](50) NULL, [Language] [varchar](50) NULL, [ContactChannel] [varchar](10) NULL, [TrackingID] [varchar](20) NULL, [CaseClosedDate] [varchar](25) NULL, [MSFTRep] [varchar](50) NULL, [CustEmail] [varchar](100) NULL, [CustPhone] [varchar](100) NULL, [CustomerName] [nvarchar](100) NULL, [ProductFamily] [varchar](35) NULL, [ProductSubType] [varchar](255) NULL, [CandidateReceivedDate] [varchar](25) NULL, [SurveyMode] [varchar](1) NULL, [SurveyWaveStartDate] [varchar](25) NULL, [SurveyInvitationDate] [varchar](25) NULL, [SurveyReminderDate] [varchar](25) NULL, [SurveyCompleteDate] [varchar](25) NULL, [OptOutDate] [varchar](25) NULL, [SurveyWaveEndDate] [varchar](25) NULL, [DispositionCode] [varchar](5) NULL, [SurveyName] [varchar](20) NULL, [SurveyVendor] [varchar](20) NULL, [BusinessUnitName] [varchar](25) NULL, [UploadId] [int] NULL, [LineNumber] [int] NULL, [BusinessUnitSubgroup] [varchar](25) NULL, [FileDate] [datetime] NULL ) ON [PRIMARY] and here's the code private void BulkLoadContent(DataTable dt) { OnMessage("Bulk loading records to temp table"); OnSubMessage("Bulk Load Started"); using (SqlBulkCopy bcp = new SqlBulkCopy(conn)) { bcp.DestinationTableName = "dbo.tempDispositions"; bcp.BulkCopyTimeout = 0; foreach (DataColumn dc in dt.Columns) { bcp.ColumnMappings.Add(dc.ColumnName, dc.ColumnName); } bcp.NotifyAfter = 2000; bcp.SqlRowsCopied += new SqlRowsCopiedEventHandler(bcp_SqlRowsCopied); bcp.WriteToServer(dt); bcp.Close(); } }

    Read the article

  • Search implementation dilemma: full text vs. plain SQL

    - by Ethan
    I have a MySQL/Rails app that needs search. Here's some info about the data: Users search within their own data only, so searches are narrowed down by user_id to begin with. Each user will have up to about five thousand records (they accumulate over time). I wrote out a typical user's records to a text file. The file size is 2.9 MB. Search has to cover two columns: title and body. title is a varchar(255) column. body is column type text. This will be lightly used. If I average a few searches per second that would be surprising. It's running an a 500 MB CentOS 5 VPS machine. I don't want relevance ranking or any kind of fuzziness. Searches should be for exact strings and reliably return all records containing the string. Simple date order -- newest to oldest. I'm using the InnoDB table type. I'm looking at plain SQL search (through the searchlogic gem) or full text search using Sphinx and the Thinking Sphinx gem. Sphinx is very fast and Thinking Sphinx is cool, but it adds complexity, a daemon to maintain, cron jobs to maintain the index. Can I get away with plain SQL search for a small scale app?

    Read the article

  • Auditing in Entity Framework.

    - by Gabriel Susai
    After going through Entity Framework I have a couple of questions on implementing auditing in Entity Framework. I want to store each column values that is created or updated to a different audit table. Rightnow I am calling SaveChanges(false) to save the records in the DB(still the changes in context is not reset). Then get the added | modified records and loop through the GetObjectStateEntries. But don't know how to get the values of the columns where their values are filled by stored proc. ie, createdate, modifieddate etc. Below is the sample code I am working on it. //Get the changed entires( ie, records) IEnumerable<ObjectStateEntry> changes = context.ObjectStateManager.GetObjectStateEntries(EntityState.Modified); //Iterate each ObjectStateEntry( for each record in the update/modified collection) foreach (ObjectStateEntry entry in changes) { //Iterate the columns in each record and get thier old and new value respectively foreach (var columnName in entry.GetModifiedProperties()) { string oldValue = entry.OriginalValues[columnName].ToString(); string newValue = entry.CurrentValues[columnName].ToString(); //Do Some Auditing by sending entityname, columnname, oldvalue, newvalue } } changes = context.ObjectStateManager.GetObjectStateEntries(EntityState.Added); foreach (ObjectStateEntry entry in changes) { if (entry.IsRelationship) continue; var columnNames = (from p in entry.EntitySet.ElementType.Members select p.Name).ToList(); foreach (var columnName in columnNames) { string newValue = entry.CurrentValues[columnName].ToString(); //Do Some Auditing by sending entityname, columnname, value } }

    Read the article

  • Update mapping table in Linq

    - by Gary McGill
    I have a table Customers with a CustomerId field, and a table of Publications with a PublicationId field. Finally, I have a mapping table CustomersPublications that records which publications a customer can access - it has two fields: CustomerId field PublicationId. For a given customer, I want to update the CustomersPublications table based on a list of publication ids. I want to remove records in CustomersPublications where the PublicationId is not in the list, and add new records where the PublicationId is in the list but not already in the table. This would be easy in SQL, but I can't figure out how to do it in Linq. For the delete part, I tried: var recordsToDelete = dataContext.CustomersPublications.Where ( cp => (cp.CustomerId == customerId) && ! publicationIds.Contains(cp.PublicationId) ); dataContext.CustomersPublications.DeleteAllOnSubmit(recordsToDelete); ... but that didn't work. I got an error: System.NotSupportedException: Method 'Boolean Contains(Int32)' has no supported translation to SQL So, I tried using Any(), as follows: var recordsToDelete = dataContext.CustomersPublications.Where ( cp => (cp.CustomerId == customerId) && ! publicationIds.Any(p => p == cp.PublicationId) ); ... and this just gives me another error: System.NotSupportedException: Local sequence cannot be used in LINQ to SQL implementation of query operators except the Contains() operator Any pointers? [I have to say, I find Linq baffling (and frustrating) for all but the simplest queries. Better error messages would help!]

    Read the article

  • SQL Query: How to determine "Seen during N hour" if given two DateTime time stamps?

    - by efess
    Hello all. I'm writing a statistics application based off a SQLite database. There is a table which records when users Login and Logout (SessionStart, SessionEnd DateTimes). What i'm looking for is a query that can show what hours user have been logged in, in sort of a line graph way- so between the hours of 12:00 and 1:00AM there were 60 users logged in (at any point), between the hours of 1:00 and 2:00AM there were 54 users logged in, etc... And I want to be able to run a SUM of this, which is why I can't bring the records into .NET and iterate through them that way. I've come up with a rather primative approach, a subquery for each hour of the day, however this approach has proved to be slow and slow. I need to be able to calculate this for a couple hundred thousand records in a split second.. SELECT case when (strftime('%s',datetime(date(sessionstart), '+0 hours')) > strftime('%s',sessionstart) AND strftime('%s',datetime(date(sessionstart), '+0 hours')) < strftime('%s',sessionend)) OR (strftime('%s',datetime(date(sessionstart), '+1 hours')) > strftime('%s',sessionstart) AND strftime('%s',datetime(date(sessionstart), '+1 hours')) < strftime('%s',sessionend)) OR (strftime('%s',datetime(date(sessionstart), '+0 hours')) < strftime('%s',sessionstart) AND strftime('%s',datetime(date(sessionstart), '+1 hours')) > strftime('%s',sessionend)) then 1 else 0 end as hour_zero, ... hour_one, ... hour_two, ........ hour_twentythree FROM UserSession I'm wondering what better way to determine if two DateTimes have been seen durring a particular hour (best case scenario, how many times it has crossed an hour if it was logged in multiple days, but not necessary)? The only other idea I had is have a "hour" table specific to this, and just tally up the hours the user has been seen at runtime, but I feel like this is more of a hack than the previous SQL. Any help would be greatly appreciated!

    Read the article

  • SSIS - Bulk Update at Database Field Level

    - by Adam
    Hello, Here's our mission: Receive files from clients. Each file contains anywhere from 1 to 1,000,000 records. Records are loaded to a staging area and business-rule validation is applied. Valid records are then pumped into an OLTP database in a batch fashion, with the following rules: If record does not exist (we have a key, so this isn't an issue), create it. If record exists, optionally update each database field. The decision is made based on one of 3 factors...I don't believe it's important what those factors are. Our main problem is finding an efficient method of optionally updating the data at a field level. This is applicable across ~12 different database tables, with anywhere from 10 to 150 fields in each table (original DB design leaves much to be desired, but it is what it is). Our first attempt has been to introduce a table that mirrors the staging environment (1 field in staging for each system field) and contains a masking flag. The value of the masking flag represents the 3 factors. We've then put an UPDATE similar to... UPDATE OLTPTable1 SET Field1 = CASE WHEN Mask.Field1 = 0 THEN Staging.Field1 WHEN Mask.Field1 = 1 THEN COALESCE( Staging.Field1 , OLTPTable1.Field1 ) WHEN Mask.Field1 = 2 THEN COALESCE( OLTPTable1.Field1 , Staging.Field1 ) ... As you can imagine, the performance is rather horrendous. Has anyone tackled a similar requirement? We're a MS shop using a Windows Service to launch SSIS packages that handle the data processing. Unfortunately, we're pretty much novices at this stuff.

    Read the article

  • In PHP... best way to turn string representation of a folder structure into nested array

    - by Greg Frommer
    Hi everyone, I looked through the related questions for a similar question but I wasn't seeing quite what I need, pardon if this has already been answered already. In my database I have a list of records that I want represented to the user as files inside of a folder structure. So for each record I have a VARCHAR column called "FolderStructure" that I want to identify that records place in to the folder structure. The series of those flat FolderStructure string columns will create my tree structure with the folders being seperated by backslashes (naturally). I didn't want to add another table just to represent a folder structure... The 'file' name is stored in a separate column so that if the FolderStructure column is empty, the file is assumed to be at the root folder. What is the best way to turn a collection of these records into a series of HTML UL/LI tags... where each LI represents a file and each folder structure being an UL embedded inside it's parent?? So for example: file - folderStructure foo - bar - firstDir blue - firstDir/subdir would produce the following HTML: <ul> <li>foo</li> <ul> <li> bar </li> <ul> <li> blue </li> </ul> </ul> </ul> Thanks

    Read the article

  • jqGrid Sort or Search does not work with columns having json dot notation

    - by rsmoorthy
    I have this jqGrid: $("#report").jqGrid( { url: '/py/db?coll=report', datatype: 'json', height: 250, colNames: ['ACN', 'Status', 'Amount'], colModel: [ {name:'acn', sortable:true}, {name:'meta.status', sortable:true}, {name:amount} ], caption: 'Show Report', rownumbers: true, gridview: true, rowNum: 10, rowList: [10,20,30], pager: '#report_pager', viewrecords: true, sortname: 'acn', sortorder: "desc", altRows: true, loadonce: true, mtype: "GET", rowTotal: 1000, jsonReader: { root: "rows", page: "page", total: "total", records: "records", repeatitems: false, id: "acn" } }); Notice that the column 'meta.status' is in JSON dot notation and accordingly the data sent from the server is like this: {"page": "1", "total": "1", "records": "5", "rows": [ {"acn":1,"meta": {"status":"Confirmed"}, "amount": 50}, {"acn":2,"meta": {"status":"Started"}, "amount": 51}, {"acn":3,"meta": {"status":"Stopped"}, "amount": 52}, {"acn":4,"meta": {"status":"Working"}, "amount": 53}, {"acn":5,"meta": {"status":"Started"}, "amount": 54} ] } The problems are of two fold: Sorting does not work on columns with dot notation, here "meta.status". It does not even show the sortable icons on the column header, and nothing happens even if the header is clicked. Sorting does not work, whether loadonce is true or false. If I try Searching (after setting loadonce to true) for the column meta.status (other columns without dot notation is okay), then it throws up a javascript error like this. Any help? Thanks Moorthy

    Read the article

  • Is there a way to increase performance on my simple textfilter?

    - by djerry
    Hey guys, I'm writing a filter that will pick out items. I have a list of Objects. The objects contain a number, name and some other irrelevant items. At the moment, the list contains 200 items. When typing in a textbox, i'm looking if the string matches a part of the number/name of the objects in the list. If so, add them to the listbox. Here's the code for my textbox textchanged event : private void txtTelnumber_TextChanged(object sender, TextChangedEventArgs e) { lstOverview.Items.Clear(); string data = ""; foreach (ucTelListItem telList in _allUsers) { data = telList.User.H323 + telList.user.E164; if (data.Contains(txtTelnumber.Text)) lstOverview.Items.Add(telList); } } I sometimes see a little delay when entering a character, especially when i go from 4 records to 200 records (so when i had a filter and 4 records matched, and i backspace and the whole list appears again). My list is a list of usercontrols, cause i found it takes less time to load the usercontrols from a list, then to have to initialize a new usercontrol each time. Can i do something about the code, or is it just adding the usercontrol the listbox that causes the small delay (small delay = <1 sec)? Thanks in advance.

    Read the article

  • Searching a column containing CSV data in a MySQL table for existence of input values

    - by Adarsh R
    Hi, I have a table say, ITEM, in MySQL that stores data as follows: ID FEATURES -------------------- 1 AB,CD,EF,XY 2 PQ,AC,A3,B3 3 AB,CDE 4 AB1,BC3 -------------------- As an input, I will get a CSV string, something like "AB,PQ". I want to get the records that contain AB or PQ. I realized that we've to write a MySQL function to achieve this. So, if we have this magical function MATCH_ANY defined in MySQL that does this, I would then simply execute an SQL as follows: select * from ITEM where MATCH_ANY(FEAURES, "AB,PQ") = 0 The above query would return the records 1, 2 and 3. But I'm running into all sorts of problems while implementing this function as I realized that MySQL doesn't support arrays and there's no simple way to split strings based on a delimiter. Remodeling the table is the last option for me as it involves lot of issues. I might also want to execute queries containing multiple MATCH_ANY functions such as: select * from ITEM where MATCH_ANY(FEATURES, "AB,PQ") = 0 and MATCH_ANY(FEATURES, "CDE") In the above case, we would get an intersection of records (1, 2, 3) and (3) which would be just 3. Any help is deeply appreciated. Thanks

    Read the article

< Previous Page | 101 102 103 104 105 106 107 108 109 110 111 112  | Next Page >