Search Results

Search found 7311 results on 293 pages for 'rows'.

Page 63/293 | < Previous Page | 59 60 61 62 63 64 65 66 67 68 69 70  | Next Page >

• Rearrange array

  - by bradenkeith

  Array starts like this: Array ( [SMART Board] => Array ( [0] => sb1 [1] => sb2 [2] => sb3 ) [Projector] => Array ( [0] => pr1 [1] => pr2 [2] => pr3 ) [Speakers] => Array ( [0] => sp1 [1] => sp2 [2] => sp3 ) [Splitter] => Array ( [0] => spl1 [1] => spl2 [2] => spl3 ) [Wireless Slate] => Array ( [0] => ws1 [1] => ws2 [2] => ws3 ) ) The keys are used as my table columns titles. Their individual arrays are to carry the column information. I have split it up even further with array_slice and array_merge_recursive to look pretty as 2 arrays - 1 holds the column names, the other looks like this: Array ( [0] => sb1 [1] => sb2 [2] => sb3 [3] => pr1 [4] => pr2 [5] => pr3 [6] => sp1 [7] => sp2 [8] => sp3 [9] => spl1 [10] => spl2 [11] => spl3 [12] => ws1 [13] => ws2 [14] => ws3 ) However, when trying to write the table I'm getting keys 0, 1, 2 as the column data, then row break then 3, 4, 5 then row break... etc. I need it to be 0, 3, 6, 9, 12 row break 1, 4, 7, 10, 13 row break etc... How can I either rearrange my array to allow for this, or rewrite how the data goes into the table so that the correct information lines up with the appropriate column? foreach(unserialize($dl->data) as $data){ //first 3 are specialinstructions, system, and room $uniques = array_slice($data,0,3); $rows = array_slice($data,3); $rows2 = array_merge_recursive($rows2, $rows); //get the specialinstructions, system, and room foreach($uniques as $unique){ echo $unique."<br/>"; }///foreach uniques echo "<br>"; }///foreach unserialized $numberofrooms = count($rows2[key($rows2)]); $numberofproducts = count($rows2); print_r($rows2); unset($rows); //write the individual rows foreach($rows2 as $header=>$rowset){ $headers[] = $header; foreach($rowset as $row){ $rows[] = $row; }//foreach rowset }//foreach rows2 echo "<p>"; print_r($rows); echo '<table class="data-table"> <caption>DL</caption> <thead><tr>'; foreach($headers as $header){ echo "<th>".$header."</th>"; } echo '</tr></thead>'; echo '<tbody>'; $i = 0; foreach($rows as $row){ if($i == 3 || $i == 0){ echo "<tr>"; $i = 1; } echo '<td>'.$row.'</td>'; if($i == 2){ echo "</tr>"; } $i++; } echo '</tbody></table>';

  Read the article

• Is there a maximum number of input controls that can be used on an HTML form?

  - by Rich

  I have an ambitious requirement for an asp.net 2.0 web page that contains a table (gridview), and each row in the grid contains 6 select (dropdown) controls for data entry. The number of rows that will be displayed is dependent upon the user's search parameters, which are specified in another area of the page. Unfortunately, with the default (and even basic) search parameters specified, the grid could contain several hundred rows. I've noticed that the browser, in this case IE8, starts behaving rather erratically once I reach a large number of rows -- no documented evidence for the number of rows where this begins to be a problem. For example, trying to view the source of the page results in a message from IE stating that there was a problem with the page that forced the browser to reload it, and I never get the source. Obviously the page loads and renders rather slowly also. I know that my solution is probably going to involve paging the gridview such that it only displays 20 or so rows per page, and I'll have to write code to handle the saving of changes in the dropdown values when the user changes pages. I can probably turn off viewstate on the gridview also. However, the question I really want to pose is this -- has anyone seen a documented rule indicating the maximum number of input controls that an HTML browser form is supposed to be able to contain? I could not find anything on the Internet after doing a search, and I suspect the answer may be whatever the browser can handle based on the machine configuration it is running on. Any rules of thumb you use? Thanks for any suggestions. Rich

  Read the article

• How to write this in better way?

  - by dario

  Hi all. Let's look at this code: IList<IHouseAnnouncement> list = new List<IHouseAnnouncement>(); var table = adapter.GetData(); //get data from repository object -> DataTable if (table.Rows.Count >= 1) { for (int i = 0; i < table.Rows.Count-1; i++) { var anno = new HouseAnnouncement(); anno.Area = float.Parse(table.Rows[i][table.powierzchniaColumn].ToString()); anno.City = table.Rows[i][table.miastoColumn].ToString(); list.Add(anno); } } return list; Is it better way to write this in less code and better fashion (must be :-) )? Maybe using labda (but let mi know how)? Thanks in advance!

  Read the article

• Reading in a file - Warning Message in R

  - by Sheila

  I have a file that has 22268 rows BY 2521 columns. When I try to read in the file using this line of code: file <- read.table(textfile, skip=2, header=TRUE, sep="\t", fill=TRUE, blank.lines.skip=FALSE) I get the following error: Warning message: In scan(file, what, nmax, sep, dec, quote, skip, nlines, na.strings, : number of items read is not a multiple of the number of columns I also used this command to see what rows had an incorrect number of columns: x <-count.fields("train.gct", sep="\t", skip=2) which(x != 2521) and got back a list of about 20 rows that were incorrect. Is there a way to fill these rows with NA values? I thought that is what the "fill" parameter does in the read.table function, but it doesn't appear so. Any help would be greatly appreciated. Thank you.

  Read the article

• Customer wants some data to appear after you later delete rows. System giant / not my creation. Fast

  - by John Sullivan

  This is a fairly common problem, it probably has a name, I just don't know what it is. A.) User sees obscure piece of information in Row B of L_OBSCURE_INFO displayed on some screen at a certain point. It is in table L_Obscure_info. B.) Under certain circumstances we want to correctly delete data in L_OBSCURE_INFO. Unfortunately, nobody accounted for the fact that the user might want to backtrack and see some random piece of information that was most recently in L_OBSCURE_INFO. C.) The system is enormous and L_OBSCURE_INFO is used all the time. You have no idea what the ramifications are of implementing some kind of hack and whatever you do, you don't want to introduce more bugs. I think the best approach would be to create an L_OBSCURE_INFO_HISTORY table and record a record in there every time you change data. But god help your ensuring it's accurate in this system where L_OBSCURE_INFO is being touched everywhere and you don't have time to implement L_OBSCURE_INFO_HISTORY. Is there a particularly easy, clever design solution for this kind of problem -- basically an elegant database hack? If not, is this kind of design problem under a particular class of problems or have a name?

  Read the article

• Deleting object in NSMutableSet Core Data

  - by Luke

  Hi I am having trouble deleting an object in an NSMutableSet using core Data... I am trying to delete a "player" object in the second section of my tableview. I am getting the error Invalid update: invalid number of rows in section 1. The number of rows contained in an existing section after the update (6) must be equal to the number of rows contained in that section before the update (6), plus or minus the number of rows inserted or deleted from that section (0 inserted, 1 deleted) and plus or minus the number of rows moved into or out of that section (0 moved in, 0 moved out Take a look at my code. - (void)tableView:(UITableView *)tableView commitEditingStyle:(UITableViewCellEditingStyle)editingStyle forRowAtIndexPath:(NSIndexPath *)indexPath { if (editingStyle == UITableViewCellEditingStyleDelete) { if (indexPath.section==0){ }else{ Player *p = [self.fetchedResultsController.fetchedObjects objectAtIndex: indexPath.section]; [_team removePlayersObject:p]; [_team.players removeObject:p]; AppDelegate *delegate = [[UIApplication sharedApplication] delegate]; _managedObjectContext = delegate.managedObjectContext; [_managedObjectContext deleteObject:p]; [self.tableView deleteRowsAtIndexPaths:[NSArray arrayWithObject:indexPath] withRowAnimation:UITableViewRowAnimationTop]; } } } - (NSInteger)tableView:(UITableView *)tableView numberOfRowsInSection:(NSInteger)section { switch(section){ case 0: return 7; case 1: return [self.fetchedResultsController.fetchedObjects count]; } return 0; }

  Read the article

• datagridviewcomboboxcolumn with datasource issue?

  - by Sarrrva

  i have some propblem in datagridviewcombobocolumn with custom datasource property in vb.net. when i add datasource it does not populate in datagridview combobox column it giving nothing.. any one please help me out from this problem... code comboboxcell: Public Overrides Sub InitializeEditingControl(ByVal rowIndex As Integer, ByVal initialFormattedValue As Object, ByVal dataGridViewCellStyle As DataGridViewCellStyle) ' Set the value of the editing control to the current cell value. MyBase.InitializeEditingControl(rowIndex, initialFormattedValue, dataGridViewCellStyle) Dim ctl As ComboEditingControl = CType(DataGridView.EditingControl, ComboEditingControl) ctl.DropDownStyle = ComboBoxStyle.DropDown ctl.AutoCompleteSource = AutoCompleteSource.ListItems ctl.AutoCompleteMode = System.Windows.Forms.AutoCompleteMode.Suggest If (Me.DataGridView.Rows(rowIndex).Cells(0).Value <> Nothing) Then Dim GetValueFromRowToUseForBuildingCombo As String = Me.DataGridView.Rows(rowIndex).Cells(0).Value.ToString() ctl.Items.Clear() Dim dt As New DataTable() Try dt = TryCast(DirectCast(Me.DataGridView.Columns(ColumnIndex), ComboColumn).DataSource, DataTable) Catch ex As Exception MsgBox("error") End Try If (dt Is Nothing) Then ctl.Items.Add("") Else Dim thing As DataRow For Each thing In dt.Rows ctl.Items.Add(thing(0).ToString) Next End If If Me.Value Is Nothing Then ctl.SelectedIndex = -1 Else ctl.SelectedItem = Me.Value End If ctl.EditingControlDataGridView = Me.DataGridView End If End Sub from code: Dim widgets As New WidgetDataHandler Dim obj = widgets.GetAllWigetTypes() Dim dt As New DataTable Dim ListofmyObjects As New List(Of widget_types)(obj) Dim objList As New cObjectToTable(Of widget_types)(ListofmyObjects) dt = objList.GetTable() Dim obj1 For Each obj1 In obj blPersons.Add(obj1) Next Dim col1 As New DataGridViewTextBoxColumn col1.DisplayIndex = 0 col1.DataPropertyName = "Id" col1.HeaderText = "Id" dgvi00.Columns.Add(col1) Dim col2 As New ComboColumn col2.DisplayIndex = 1 col2.SortMode = DataGridViewColumnSortMode.Automatic col2.HeaderText = "Name" col2.DataPropertyName = "Name" col2.ToolTipText = "Select something from my combo" Dim dst As New DataSet 'Dim dt1 As New DataTable 'dt1.Columns.Add(col2.HeaderText) 'For Each thing In dt.Rows ' MsgBox(thing(1).ToString) ' dt1.Rows.Add(thing(1).ToString) 'Next dst.Tables.Add(dt) col2.DataSource = dst.Tables(0) col2.DisplayMember = "Name" Me.dgvi00.Columns.AddRange(col2) dgvi00.DataSource = blPersons.BindingSource 'setup the bindings for the binding navigator Dim bn As New _365_Media_Library.BindingNavigatorWithFilter bn.Dock = DockStyle.Bottom bn.GripStyle = ToolStripGripStyle.Hidden Me.Controls.Add(bn) bn.BindingSource = blPersons.BindingSource note : its working good in standalone application regards and thanks sarva

  Read the article

• PHP loop position

  - by Jordan Pagaduan

  Can someone help me on this. I'm made an image uploader and i want the image to make another tr if it reach to 5 pics so it will not overflow. Here is my code: Can someone help me on this. I'm made an image uploader and i want the image to make another tr if it reach to 5 pics so it will not overflow. Here is my code: $dbc = mysql_connect("localhost" , "root" , "") or die (mysql_error()); mysql_select_db('blog_data') or die (mysql_error()); $sql = "SELECT * FROM img_uploaded"; $result = mysql_query($sql); while($rows=mysql_fetch_array($result)) { if ($rows) { echo "<tr><td><img src='user_images/".$rows['img_name'] . "' width='100' height='100'></td></tr>"; } else { echo "<td><img src='user_images/".$rows['img_name'] . "' width='100' height='100'></td>"; } }

  Read the article

• Writing to CSV issue in Spyder

  - by 0003

  I am doing the Kaggle Titanic beginner contest. I generally work in Spyder IDE, but I came across a weird issue. The expected output is supposed to be 418 rows. When I run the script from terminal the output I get is 418 rows (as expected). When I run it in Spyder IDE the output is 408 rows not 418. When I re-run it in the current python process, it outputs the expected 418 rows. I posted a redacted portion of the code that has all of the relevant bits. Any ideas? import csv import numpy as np csvFile = open("/train.csv","ra") csvFile = csv.reader(csvFile) header = csvFile.next() testFile = open("/test.csv","ra") testFile = csv.reader(testFile) testHeader = testFile.next() writeFile = open("/gendermodelDebug.csv", "wb") writeFile = csv.writer(writeFile) count = 0 for row in testFile: if row[3] == 'male': do something to row writeFile.writerow(row) count += 1 elif row[3] == 'female': do something to row writeFile.writerow(row) count += 1 else: raise ValueError("Did not find a male or female in %s" % row)

  Read the article

• Which is more efficient in mysql, a big join or multiple queries of single table?

  - by Tom Greenpoint

  I have a mysql database like this Post – 500,000 rows (Postid,Userid) Photo – 200,000 rows (Photoid,Postid) About 50,000 posts have photos, average 4 each, most posts do not have photos. I need to get a feed of all posts with photos for a userid, average 50 posts each. Which approach would be more efficient? 1: Big Join select * from post left join photo on post.postid=photo.postid where post.userid=123 2: Multiple queries select * from post where userid=123 while (loop through rows) { select * from photo where postid=row[postid] }

  Read the article

• VB.NET Abstract Property

  - by ElPresidente

  I have an abstract "GridBase" class with two types of derived classes "DetailGrid" and "HeaderGrid". Respectively, one is comprised of "DetailRow" objects and the other "HeaderRow" objects. Both of those inherit from a "RowBase" abstract class. What I am trying to do is the following: Public MustInherit Class GridBase Private pRows As List(Of RowBase) Public ReadOnly Property Rows As List(Of RowBase) Get Return pRows End Get End Property End Class Public Class DetailGrid Inherits GridBase End Class In this scenario, I want DetailGrid.Rows to return a list of DetailRow. I want HeaderRow.Rows to return a list of HeaderRow. Am I on the right track with this or should the Rows property not be included in the GridBase class?

  Read the article

• Where should I handle the exceptions, in the BLL, DAL or PL ?

  - by Puneet Dudeja

  Which is the best place to handle the exceptions ? BLL, DAL or PL ? Should I allow the methods in the DAL and BLL to throw the exceptions up the chain and let the PL handle them? or should I handle them at the BLL ? e.g If I have a method in my DAL that issues "ExecuteNonQuery" and updates some records, and due to one or more reason, 0 rows are affected. Now, how should I let my PL know that whether an exception happened or there really was no rows matched to the condition. Should I use "try catch" in my PL code and let it know through an exception, or should I handle the exception at DAL and return some special code like (-1) to let the PL differentiate between the (exception) and (no rows matched condition i.e. zero rows affected) ?

  Read the article

• postgres min function performance

  - by wutzebaer

  hi i need the lowest value for runnerId this query: SELECT "runnerId" FROM betlog WHERE "marketId" = '107416794' ; takes 80ms (1968 result rows) this SELECT min("runnerId") FROM betlog WHERE "marketId" = '107416794' ; takes 1600ms is there a faster way to find the minimum, or should i calc the min in my java programm? "Result (cost=100.88..100.89 rows=1 width=0)" " InitPlan 1 (returns $0)" " -> Limit (cost=0.00..100.88 rows=1 width=9)" " -> Index Scan using runneridindex on betlog (cost=0.00..410066.33 rows=4065 width=9)" " Index Cond: ("runnerId" IS NOT NULL)" " Filter: ("marketId" = 107416794::bigint)" CREATE INDEX marketidindex ON betlog USING btree ("marketId" COLLATE pg_catalog."default"); another idea SELECT "runnerId" FROM betlog WHERE "marketId" = '107416794' ORDER BY "runnerId" LIMIT 1 >1600ms SELECT "runnerId" FROM betlog WHERE "marketId" = '107416794' ORDER BY "runnerId" >>100ms how can a limit slow the query down?

  Read the article

• mysql row counter

  - by David19801

  Hello, I have a mysql table. It has auto increment on the id. but I regularly delete rows so the numbers are all over the place. I need to get the last n rows out, but because of deletions, the common way of using the max of the autoincremented id column doesn't work well... 1 - Is their another way to get the bottom 50? 2 - Is their a way to get rows by actual row number? so if I have 4 rows labelled 1,2,3,4 delete row 2 then it will become 1,2,3 rather than 1,3,4?

  Read the article

• Fetching Cassandra row keys

  - by knorv

  Assume a Cassandra datastore with 20 rows, with row keys named "r1" .. "r20". Questions: How do I fetch the row keys of the first ten rows (r1 to r10)? How do I fetch the row keys of the next ten rows (r11 to r20)? I'm looking for the Cassandra analogy to: SELECT row_key FROM table LIMIT 0, 10; SELECT row_key FROM table LIMIT 10, 10;

  Read the article

• Autopruning after a specified amount of row are created?

  - by Rob

  Basic question, sorry. Basically, I have a script that creates a MySQL entry each time someone visits the page. A logging script. However, I want to make it autoprune after, say, 100 visits. For example in pseudo code: if amount of rows > 100 { delete rows 1+ until amount of rows == 100 } So in a nutshell, each time a new row is added after 100, it needs to automatically remove the row with the smallest id (My primary key).

  Read the article

• Update MySQl table onDrop?

  - by dougvt

  Hi all. I am writing a PHP/MySQL application (using CodeIgniter) that uses some jQuery functionality for dragging table rows. I have a table in which the user can drag rows to the desired order (kind of a queue for which I need to preserve the rank of each row). I've been trying to figure out how to (and whether I should) update the database each time the user drops a row, in order to simplify the UI and avoid a "Save" button. I have the jQuery working and can send a serialized list back to the server onDrop, but is it good design practice to run an update query this often? The table will usually have 30-40 rows max, but if the user drags row 1 far down the list, then potentially all the rows would need to be updated to update the rank field. I've been wondering whether to send a giant query to the server, to loop through the rows in PHP and update each row with its own Update query, to send a small serialized list to a stored procedure to let the server do all the work, or perhaps a better method I haven't considered. I've read that stored procedures in MySQL are not very efficient and use a separate process for each call. Any advice as to the right solution here? Thanks very much for your help!

  Read the article

• How to remove row which has one or more empty or null cell ?

  - by Harikrishna

  I have datagridview on my winform. I am displaying records in the datagridview. Now after displaying the records on the datagridview, I want to remove the row from datagridview which has one or more empy cells that is no value in the cell for that row. So for that I am checking each cell for every row if there is any cell empty or null then I remove that rows using RemoveAt() function. My code is : for (int i = 0; i < dataGridView1.Rows.Count - 1; i++) { for (int j = 0; j < dataGridView1.Columns.Count; j++) { if (string.IsNullOrEmpty(dataGridView1.Rows[i].Cells[j].Value.ToString())) { dataGridView1.Rows.RemoveAt(i); break; } } }

  Read the article

• How to predict result set row count?

  - by Saurabh Kumar

  I have an application where I create a big SQL query dynamically for SQL server 2008. This query is based on various search criteria which the user might give such as search by lastname, firstname, ssn etc. The requirement is that if the user gives a condition due to which the formed query might return a lot of rows(configurable for max N rows), then the application must send back a message instead to the user saying that he needs to refine his search query as the existing query will return too many rows. I would not want to bring back say, 5000 rows to the client and then discard that data just to show the user an error. What is an efficient way to tackle this issue?

  Read the article

• For a DataGridView, how do I get the values from each row?

  - by David L

  I am wondering what is the best way to go iterate through all the rows in a datagridview and get the values from the cells. Here is what I am thinking of doing, but I don't really like it because if I rearrange the columns then the code will also have to be changed. for (int i = 0; i < dataGridView.RowCount; i++) { string Value0 = dataGridView1.Rows[i].Cells[0]; string Value1 = dataGridView1.Rows[i].Cells[1]; string Value2 = dataGridView1.Rows[i].Cells[2]; }

  Read the article

• How to pass dynamic id text box value to another page without refreshing with jquery and php

  - by linlin

  $('.btncomment').click(function() { var id = $(this).attr('id'); $.post('SaveTopicInformation.php',{tid:commentform.(topic_+id).value, topicdetail:commentform.(topicdetail_+id).value,userid:commentform.(user_+id).value}); }); $userid=$rows['UserID']; $topicid=$rows['TopicID']; ? " " class="commentAlink"Comment " " value=""/ " value=""/ " cols="50" rows="5" "Cancel " value="Comment" / ?

  Read the article

• SQL SERVER – Guest Posts – Feodor Georgiev – The Context of Our Database Environment – Going Beyond the Internal SQL Server Waits – Wait Type – Day 21 of 28

  - by pinaldave

  This guest post is submitted by Feodor. Feodor Georgiev is a SQL Server database specialist with extensive experience of thinking both within and outside the box. He has wide experience of different systems and solutions in the fields of architecture, scalability, performance, etc. Feodor has experience with SQL Server 2000 and later versions, and is certified in SQL Server 2008. In this article Feodor explains the server-client-server process, and concentrated on the mutual waits between client and SQL Server. This is essential in grasping the concept of waits in a ‘global’ application plan. Recently I was asked to write a blog post about the wait statistics in SQL Server and since I had been thinking about writing it for quite some time now, here it is. It is a wide-spread idea that the wait statistics in SQL Server will tell you everything about your performance. Well, almost. Or should I say – barely. The reason for this is that SQL Server is always a part of a bigger system – there are always other players in the game: whether it is a client application, web service, any other kind of data import/export process and so on. In short, the SQL Server surroundings look like this: This means that SQL Server, aside from its internal waits, also depends on external waits and settings. As we can see in the picture above, SQL Server needs to have an interface in order to communicate with the surrounding clients over the network. For this communication, SQL Server uses protocol interfaces. I will not go into detail about which protocols are best, but you can read this article. Also, review the information about the TDS (Tabular data stream). As we all know, our system is only as fast as its slowest component. This means that when we look at our environment as a whole, the SQL Server might be a victim of external pressure, no matter how well we have tuned our database server performance. Let’s dive into an example: let’s say that we have a web server, hosting a web application which is using data from our SQL Server, hosted on another server. The network card of the web server for some reason is malfunctioning (think of a hardware failure, driver failure, or just improper setup) and does not send/receive data faster than 10Mbs. On the other end, our SQL Server will not be able to send/receive data at a faster rate either. This means that the application users will notify the support team and will say: “My data is coming very slow.” Now, let’s move on to a bit more exciting example: imagine that there is a similar setup as the example above – one web server and one database server, and the application is not using any stored procedure calls, but instead for every user request the application is sending 80kb query over the network to the SQL Server. (I really thought this does not happen in real life until I saw it one day.) So, what happens in this case? To make things worse, let’s say that the 80kb query text is submitted from the application to the SQL Server at least 100 times per minute, and as often as 300 times per minute in peak times. Here is what happens: in order for this query to reach the SQL Server, it will have to be broken into a of number network packets (according to the packet size settings) – and will travel over the network. On the other side, our SQL Server network card will receive the packets, will pass them to our network layer, the packets will get assembled, and eventually SQL Server will start processing the query – parsing, allegorizing, generating the query execution plan and so on. So far, we have already had a serious network overhead by waiting for the packets to reach our Database Engine. There will certainly be some processing overhead – until the database engine deals with the 80kb query and its 20 subqueries. The waits you see in the DMVs are actually collected from the point the query reaches the SQL Server and the packets are assembled. Let’s say that our query is processed and it finally returns 15000 rows. These rows have a certain size as well, depending on the data types returned. This means that the data will have converted to packages (depending on the network size package settings) and will have to reach the application server. There will also be waits, however, this time you will be able to see a wait type in the DMVs called ASYNC_NETWORK_IO. What this wait type indicates is that the client is not consuming the data fast enough and the network buffers are filling up. Recently Pinal Dave posted a blog on Client Statistics. What Client Statistics does is captures the physical flow characteristics of the query between the client(Management Studio, in this case) and the server and back to the client. As you see in the image, there are three categories: Query Profile Statistics, Network Statistics and Time Statistics. Number of server roundtrips–a roundtrip consists of a request sent to the server and a reply from the server to the client. For example, if your query has three select statements, and they are separated by ‘GO’ command, then there will be three different roundtrips. TDS Packets sent from the client – TDS (tabular data stream) is the language which SQL Server speaks, and in order for applications to communicate with SQL Server, they need to pack the requests in TDS packets. TDS Packets sent from the client is the number of packets sent from the client; in case the request is large, then it may need more buffers, and eventually might even need more server roundtrips. TDS packets received from server –is the TDS packets sent by the server to the client during the query execution. Bytes sent from client – is the volume of the data set to our SQL Server, measured in bytes; i.e. how big of a query we have sent to the SQL Server. This is why it is best to use stored procedures, since the reusable code (which already exists as an object in the SQL Server) will only be called as a name of procedure + parameters, and this will minimize the network pressure. Bytes received from server – is the amount of data the SQL Server has sent to the client, measured in bytes. Depending on the number of rows and the datatypes involved, this number will vary. But still, think about the network load when you request data from SQL Server. Client processing time – is the amount of time spent in milliseconds between the first received response packet and the last received response packet by the client. Wait time on server replies – is the time in milliseconds between the last request packet which left the client and the first response packet which came back from the server to the client. Total execution time – is the sum of client processing time and wait time on server replies (the SQL Server internal processing time) Here is an illustration of the Client-server communication model which should help you understand the mutual waits in a client-server environment. Keep in mind that a query with a large ‘wait time on server replies’ means the server took a long time to produce the very first row. This is usual on queries that have operators that need the entire sub-query to evaluate before they proceed (for example, sort and top operators). However, a query with a very short ‘wait time on server replies’ means that the query was able to return the first row fast. However a long ‘client processing time’ does not necessarily imply the client spent a lot of time processing and the server was blocked waiting on the client. It can simply mean that the server continued to return rows from the result and this is how long it took until the very last row was returned. The bottom line is that developers and DBAs should work together and think carefully of the resource utilization in the client-server environment. From experience I can say that so far I have seen only cases when the application developers and the Database developers are on their own and do not ask questions about the other party’s world. I would recommend using the Client Statistics tool during new development to track the performance of the queries, and also to find a synchronous way of utilizing resources between the client – server – client. Here is another example: think about similar setup as above, but add another server to the game. Let’s say that we keep our media on a separate server, and together with the data from our SQL Server we need to display some images on the webpage requested by our user. No matter how simple or complicated the logic to get the images is, if the images are 500kb each our users will get the page slowly and they will still think that there is something wrong with our data. Anyway, I don’t mean to get carried away too far from SQL Server. Instead, what I would like to say is that DBAs should also be aware of ‘the big picture’. I wrote a blog post a while back on this topic, and if you are interested, you can read it here about the big picture. And finally, here are some guidelines for monitoring the network performance and improving it: Run a trace and outline all queries that return more than 1000 rows (in Profiler you can actually filter and sort the captured trace by number of returned rows). This is not a set number; it is more of a guideline. The general thought is that no application user can consume that many rows at once. Ask yourself and your fellow-developers: ‘why?’. Monitor your network counters in Perfmon: Network Interface:Output queue length, Redirector:Network errors/sec, TCPv4: Segments retransmitted/sec and so on. Make sure to establish a good friendship with your network administrator (buy them coffee, for example J ) and get into a conversation about the network settings. Have them explain to you how the network cards are setup – are they standalone, are they ‘teamed’, what are the settings – full duplex and so on. Find some time to read a bit about networking. In this short blog post I hope I have turned your attention to ‘the big picture’ and the fact that there are other factors affecting our SQL Server, aside from its internal workings. As a further reading I would still highly recommend the Wait Stats series on this blog, also I would recommend you have the coffee break conversation with your network admin as soon as possible. This guest post is written by Feodor Georgiev. Read all the post in the Wait Types and Queue series. Reference: Pinal Dave (http://blog.SQLAuthority.com) Filed under: Pinal Dave, PostADay, Readers Contribution, SQL, SQL Authority, SQL Query, SQL Server, SQL Tips and Tricks, SQL Wait Stats, SQL Wait Types, T SQL

  Read the article

• SQL SERVER – Concurrency Basics – Guest Post by Vinod Kumar

  - by pinaldave

  This guest post is by Vinod Kumar. Vinod Kumar has worked with SQL Server extensively since joining the industry over a decade ago. Working on various versions from SQL Server 7.0, Oracle 7.3 and other database technologies – he now works with the Microsoft Technology Center (MTC) as a Technology Architect. Let us read the blog post in Vinod’s own voice. Learning is always fun when it comes to SQL Server and learning the basics again can be more fun. I did write about Transaction Logs and recovery over my blogs and the concept of simplifying the basics is a challenge. In the real world we always see checks and queues for a process – say railway reservation, banks, customer supports etc there is a process of line and queue to facilitate everyone. Shorter the queue higher is the efficiency of system (a.k.a higher is the concurrency). Every database does implement this using checks like locking, blocking mechanisms and they implement the standards in a way to facilitate higher concurrency. In this post, let us talk about the topic of Concurrency and what are the various aspects that one needs to know about concurrency inside SQL Server. Let us learn the concepts as one-liners: Concurrency can be defined as the ability of multiple processes to access or change shared data at the same time. The greater the number of concurrent user processes that can be active without interfering with each other, the greater the concurrency of the database system. Concurrency is reduced when a process that is changing data prevents other processes from reading that data or when a process that is reading data prevents other processes from changing that data. Concurrency is also affected when multiple processes are attempting to change the same data simultaneously. Two approaches to managing concurrent data access: Optimistic Concurrency Model Pessimistic Concurrency Model Concurrency Models Pessimistic Concurrency Default behavior: acquire locks to block access to data that another process is using. Assumes that enough data modification operations are in the system that any given read operation is likely affected by a data modification made by another user (assumes conflicts will occur). Avoids conflicts by acquiring a lock on data being read so no other processes can modify that data. Also acquires locks on data being modified so no other processes can access the data for either reading or modifying. Readers block writer, writers block readers and writers. Optimistic Concurrency Assumes that there are sufficiently few conflicting data modification operations in the system that any single transaction is unlikely to modify data that another transaction is modifying. Default behavior of optimistic concurrency is to use row versioning to allow data readers to see the state of the data before the modification occurs. Older versions of the data are saved so a process reading data can see the data as it was when the process started reading and not affected by any changes being made to that data. Processes modifying the data is unaffected by processes reading the data because the reader is accessing a saved version of the data rows. Readers do not block writers and writers do not block readers, but, writers can and will block writers. Transaction Processing A transaction is the basic unit of work in SQL Server. Transaction consists of SQL commands that read and update the database but the update is not considered final until a COMMIT command is issued (at least for an explicit transaction: marked with a BEGIN TRAN and the end is marked by a COMMIT TRAN or ROLLBACK TRAN). Transactions must exhibit all the ACID properties of a transaction. ACID Properties Transaction processing must guarantee the consistency and recoverability of SQL Server databases. Ensures all transactions are performed as a single unit of work regardless of hardware or system failure. A – Atomicity C – Consistency I – Isolation D- Durability Atomicity: Each transaction is treated as all or nothing – it either commits or aborts. Consistency: ensures that a transaction won’t allow the system to arrive at an incorrect logical state – the data must always be logically correct.  Consistency is honored even in the event of a system failure. Isolation: separates concurrent transactions from the updates of other incomplete transactions. SQL Server accomplishes isolation among transactions by locking data or creating row versions. Durability: After a transaction commits, the durability property ensures that the effects of the transaction persist even if a system failure occurs. If a system failure occurs while a transaction is in progress, the transaction is completely undone, leaving no partial effects on data. Transaction Dependencies In addition to supporting all four ACID properties, a transaction might exhibit few other behaviors (known as dependency problems or consistency problems). Lost Updates: Occur when two processes read the same data and both manipulate the data, changing its value and then both try to update the original data to the new value. The second process might overwrite the first update completely. Dirty Reads: Occurs when a process reads uncommitted data. If one process has changed data but not yet committed the change, another process reading the data will read it in an inconsistent state. Non-repeatable Reads: A read is non-repeatable if a process might get different values when reading the same data in two reads within the same transaction. This can happen when another process changes the data in between the reads that the first process is doing. Phantoms: Occurs when membership in a set changes. It occurs if two SELECT operations using the same predicate in the same transaction return a different number of rows. Isolation Levels SQL Server supports 5 isolation levels that control the behavior of read operations. Read Uncommitted All behaviors except for lost updates are possible. Implemented by allowing the read operations to not take any locks, and because of this, it won’t be blocked by conflicting locks acquired by other processes. The process can read data that another process has modified but not yet committed. When using the read uncommitted isolation level and scanning an entire table, SQL Server can decide to do an allocation order scan (in page-number order) instead of a logical order scan (following page pointers). If another process doing concurrent operations changes data and move rows to a new location in the table, the allocation order scan can end up reading the same row twice. Also can happen if you have read a row before it is updated and then an update moves the row to a higher page number than your scan encounters later. Performing an allocation order scan under Read Uncommitted can cause you to miss a row completely – can happen when a row on a high page number that hasn’t been read yet is updated and moved to a lower page number that has already been read. Read Committed Two varieties of read committed isolation: optimistic and pessimistic (default). Ensures that a read never reads data that another application hasn’t committed. If another transaction is updating data and has exclusive locks on data, your transaction will have to wait for the locks to be released. Your transaction must put share locks on data that are visited, which means that data might be unavailable for others to use. A share lock doesn’t prevent others from reading but prevents them from updating. Read committed (snapshot) ensures that an operation never reads uncommitted data, but not by forcing other processes to wait. SQL Server generates a version of the changed row with its previous committed values. Data being changed is still locked but other processes can see the previous versions of the data as it was before the update operation began. Repeatable Read This is a Pessimistic isolation level. Ensures that if a transaction revisits data or a query is reissued the data doesn’t change. That is, issuing the same query twice within a transaction cannot pickup any changes to data values made by another user’s transaction because no changes can be made by other transactions. However, this does allow phantom rows to appear. Preventing non-repeatable read is a desirable safeguard but cost is that all shared locks in a transaction must be held until the completion of the transaction. Snapshot Snapshot Isolation (SI) is an optimistic isolation level. Allows for processes to read older versions of committed data if the current version is locked. Difference between snapshot and read committed has to do with how old the older versions have to be. It’s possible to have two transactions executing simultaneously that give us a result that is not possible in any serial execution. Serializable This is the strongest of the pessimistic isolation level. Adds to repeatable read isolation level by ensuring that if a query is reissued rows were not added in the interim, i.e, phantoms do not appear. Preventing phantoms is another desirable safeguard, but cost of this extra safeguard is similar to that of repeatable read – all shared locks in a transaction must be held until the transaction completes. In addition serializable isolation level requires that you lock data that has been read but also data that doesn’t exist. Ex: if a SELECT returned no rows, you want it to return no. rows when the query is reissued. This is implemented in SQL Server by a special kind of lock called the key-range lock. Key-range locks require that there be an index on the column that defines the range of values. If there is no index on the column, serializable isolation requires a table lock. Gets its name from the fact that running multiple serializable transactions at the same time is equivalent of running them one at a time. Now that we understand the basics of what concurrency is, the subsequent blog posts will try to bring out the basics around locking, blocking, deadlocks because they are the fundamental blocks that make concurrency possible. Now if you are with me – let us continue learning for SQL Server Locking Basics. Reference: Pinal Dave (http://blog.sqlauthority.com) Filed under: PostADay, SQL, SQL Authority, SQL Performance, SQL Query, SQL Server, SQL Tips and Tricks, T SQL, Technology Tagged: Concurrency

  Read the article

• Denali Paging–Key seek lookups

  - by Dave Ballantyne

  In my previous post “Denali Paging – is it win.win ?” I demonstrated the use of using the Paging functionality within Denali.  On reflection,  I think i may of been a little unfair and should of continued always planned to continue my investigations to the next step. In Pauls article, he uses a combination of ctes to first scan the ordered keys which is then filtered using TOP and rownumber and then uses those keys to seek the data.  So what happens if we replace the scanning portion of the code with the denali paging functionality. Heres the original procedure,  we are going to replace the functionality of the Keys and SelectedKeys ctes : CREATE  PROCEDURE dbo.FetchPageKeySeek         @PageSize   BIGINT,         @PageNumber BIGINT AS BEGIN         -- Key-Seek algorithm         WITH    Keys         AS      (                 -- Step 1 : Number the rows from the non-clustered index                 -- Maximum number of rows = @PageNumber * @PageSize                 SELECT  TOP (@PageNumber * @PageSize)                         rn = ROW_NUMBER() OVER (ORDER BY P1.post_id ASC),                         P1.post_id                 FROM    dbo.Post P1                 ORDER   BY                         P1.post_id ASC                 ),                 SelectedKeys         AS      (                 -- Step 2 : Get the primary keys for the rows on the page we want                 -- Maximum number of rows from this stage = @PageSize                 SELECT  TOP (@PageSize)                         SK.rn,                         SK.post_id                 FROM    Keys SK                 WHERE   SK.rn > ((@PageNumber - 1) * @PageSize)                 ORDER   BY                         SK.post_id ASC                 )         SELECT  -- Step 3 : Retrieve the off-index data                 -- We will only have @PageSize rows by this stage                 SK.rn,                 P2.post_id,                 P2.thread_id,                 P2.member_id,                 P2.create_dt,                 P2.title,                 P2.body         FROM    SelectedKeys SK         JOIN    dbo.Post P2                 ON  P2.post_id = SK.post_id         ORDER   BY                 SK.post_id ASC; END; and here is the replacement procedure using paging: CREATE  PROCEDURE dbo.FetchOffsetPageKeySeek         @PageSize   BIGINT,         @PageNumber BIGINT AS BEGIN         -- Key-Seek algorithm         WITH    SelectedKeys         AS      (                 SELECT  post_id                 FROM    dbo.Post P1                 ORDER   BY post_id ASC                 OFFSET  @PageSize * (@PageNumber-1) ROWS                 FETCH NEXT @PageSize ROWS ONLY                 )         SELECT  P2.post_id,                 P2.thread_id,                 P2.member_id,                 P2.create_dt,                 P2.title,                 P2.body         FROM    SelectedKeys SK         JOIN    dbo.Post P2                 ON  P2.post_id = SK.post_id         ORDER   BY                 SK.post_id ASC; END; Notice how all i have done is replace the functionality with the Keys and SelectedKeys CTEs with the paging functionality. So , what is the comparative performance now ?. Exactly the same amount of IO and memory usage , but its now pretty obvious that in terms of CPU and overall duration we are onto a winner.    

  Read the article

• Solving Big Problems with Oracle R Enterprise, Part II

  - by dbayard

  Part II – Solving Big Problems with Oracle R Enterprise In the first post in this series (see https://blogs.oracle.com/R/entry/solving_big_problems_with_oracle), we showed how you can use R to perform historical rate of return calculations against investment data sourced from a spreadsheet.  We demonstrated the calculations against sample data for a small set of accounts.  While this worked fine, in the real-world the problem is much bigger because the amount of data is much bigger.  So much bigger that our approach in the previous post won’t scale to meet the real-world needs. From our previous post, here are the challenges we need to conquer: The actual data that needs to be used lives in a database, not in a spreadsheet The actual data is much, much bigger- too big to fit into the normal R memory space and too big to want to move across the network The overall process needs to run fast- much faster than a single processor The actual data needs to be kept secured- another reason to not want to move it from the database and across the network And the process of calculating the IRR needs to be integrated together with other database ETL activities, so that IRR’s can be calculated as part of the data warehouse refresh processes In this post, we will show how we moved from sample data environment to working with full-scale data.  This post is based on actual work we did for a financial services customer during a recent proof-of-concept. Getting started with the Database At this point, we have some sample data and our IRR function.  We were at a similar point in our customer proof-of-concept exercise- we had sample data but we did not have the full customer data yet.  So our database was empty.  But, this was easily rectified by leveraging the transparency features of Oracle R Enterprise (see https://blogs.oracle.com/R/entry/analyzing_big_data_using_the).  The following code shows how we took our sample data SimpleMWRRData and easily turned it into a new Oracle database table called IRR_DATA via ore.create().  The code also shows how we can access the database table IRR_DATA as if it was a normal R data.frame named IRR_DATA. If we go to sql*plus, we can also check out our new IRR_DATA table: At this point, we now have our sample data loaded in the database as a normal Oracle table called IRR_DATA.  So, we now proceeded to test our R function working with database data. As our first test, we retrieved the data from a single account from the IRR_DATA table, pull it into local R memory, then call our IRR function.  This worked.  No SQL coding required! Going from Crawling to Walking Now that we have shown using our R code with database-resident data for a single account, we wanted to experiment with doing this for multiple accounts.  In other words, we wanted to implement the split-apply-combine technique we discussed in our first post in this series.  Fortunately, Oracle R Enterprise provides a very scalable way to do this with a function called ore.groupApply().  You can read more about ore.groupApply() here: https://blogs.oracle.com/R/entry/analyzing_big_data_using_the1 Here is an example of how we ask ORE to take our IRR_DATA table in the database, split it by the ACCOUNT column, apply a function that calls our SimpleMWRR() calculation, and then combine the results. (If you are following along at home, be sure to have installed our myIRR package on your database server via  “R CMD INSTALL myIRR”). The interesting thing about ore.groupApply is that the calculation is not actually performed in my desktop R environment from which I am running.  What actually happens is that ore.groupApply uses the Oracle database to perform the work.  And the Oracle database is what actually splits the IRR_DATA table by ACCOUNT.  Then the Oracle database takes the data for each account and sends it to an embedded R engine running on the database server to apply our R function.  Then the Oracle database combines all the individual results from the calls to the R function. This is significant because now the embedded R engine only needs to deal with the data for a single account at a time.  Regardless of whether we have 20 accounts or 1 million accounts or more, the R engine that performs the calculation does not care.  Given that normal R has a finite amount of memory to hold data, the ore.groupApply approach overcomes the R memory scalability problem since we only need to fit the data from a single account in R memory (not all of the data for all of the accounts). Additionally, the IRR_DATA does not need to be sent from the database to my desktop R program.  Even though I am invoking ore.groupApply from my desktop R program, because the actual SimpleMWRR calculation is run by the embedded R engine on the database server, the IRR_DATA does not need to leave the database server- this is both a performance benefit because network transmission of large amounts of data take time and a security benefit because it is harder to protect private data once you start shipping around your intranet. Another benefit, which we will discuss in a few paragraphs, is the ability to leverage Oracle database parallelism to run these calculations for dozens of accounts at once. From Walking to Running ore.groupApply is rather nice, but it still has the drawback that I run this from a desktop R instance.  This is not ideal for integrating into typical operational processes like nightly data warehouse refreshes or monthly statement generation.  But, this is not an issue for ORE.  Oracle R Enterprise lets us run this from the database using regular SQL, which is easily integrated into standard operations.  That is extremely exciting and the way we actually did these calculations in the customer proof. As part of Oracle R Enterprise, it provides a SQL equivalent to ore.groupApply which it refers to as “rqGroupEval”.  To use rqGroupEval via SQL, there is a bit of simple setup needed.  Basically, the Oracle Database needs to know the structure of the input table and the grouping column, which we are able to define using the database’s pipeline table function mechanisms. Here is the setup script: At this point, our initial setup of rqGroupEval is done for the IRR_DATA table.  The next step is to define our R function to the database.  We do that via a call to ORE’s rqScriptCreate. Now we can test it.  The SQL you use to run rqGroupEval uses the Oracle database pipeline table function syntax.  The first argument to irr_dataGroupEval is a cursor defining our input.  You can add additional where clauses and subqueries to this cursor as appropriate.  The second argument is any additional inputs to the R function.  The third argument is the text of a dummy select statement.  The dummy select statement is used by the database to identify the columns and datatypes to expect the R function to return.  The fourth argument is the column of the input table to split/group by.  The final argument is the name of the R function as you defined it when you called rqScriptCreate(). The Real-World Results In our real customer proof-of-concept, we had more sophisticated calculation requirements than shown in this simplified blog example.  For instance, we had to perform the rate of return calculations for 5 separate time periods, so the R code was enhanced to do so.  In addition, some accounts needed a time-weighted rate of return to be calculated, so we extended our approach and added an R function to do that.  And finally, there were also a few more real-world data irregularities that we needed to account for, so we added logic to our R functions to deal with those exceptions.  For the full-scale customer test, we loaded the customer data onto a Half-Rack Exadata X2-2 Database Machine.  As our half-rack had 48 physical cores (and 96 threads if you consider hyperthreading), we wanted to take advantage of that CPU horsepower to speed up our calculations.  To do so with ORE, it is as simple as leveraging the Oracle Database Parallel Query features.  Let’s look at the SQL used in the customer proof: Notice that we use a parallel hint on the cursor that is the input to our rqGroupEval function.  That is all we need to do to enable Oracle to use parallel R engines. Here are a few screenshots of what this SQL looked like in the Real-Time SQL Monitor when we ran this during the proof of concept (hint: you might need to right-click on these images to be able to view the images full-screen to see the entire image): From the above, you can notice a few things (numbers 1 thru 5 below correspond with highlighted numbers on the images above.  You may need to right click on the above images and view the images full-screen to see the entire image): The SQL completed in 110 seconds (1.8minutes) We calculated rate of returns for 5 time periods for each of 911k accounts (the number of actual rows returned by the IRRSTAGEGROUPEVAL operation) We accessed 103m rows of detailed cash flow/market value data (the number of actual rows returned by the IRR_STAGE2 operation) We ran with 72 degrees of parallelism spread across 4 database servers Most of our 110seconds was spent in the “External Procedure call” event On average, we performed 8,200 executions of our R function per second (110s/911k accounts) On average, each execution was passed 110 rows of data (103m detail rows/911k accounts) On average, we did 41,000 single time period rate of return calculations per second (each of the 8,200 executions of our R function did rate of return calculations for 5 time periods) On average, we processed over 900,000 rows of database data in R per second (103m detail rows/110s) R + Oracle R Enterprise: Best of R + Best of Oracle Database This blog post series started by describing a real customer problem: how to perform a lot of calculations on a lot of data in a short period of time.  While standard R proved to be a very good fit for writing the necessary calculations, the challenge of working with a lot of data in a short period of time remained. This blog post series showed how Oracle R Enterprise enables R to be used in conjunction with the Oracle Database to overcome the data volume and performance issues (as well as simplifying the operations and security issues).  It also showed that we could calculate 5 time periods of rate of returns for almost a million individual accounts in less than 2 minutes. In a future post, we will take the same R function and show how Oracle R Connector for Hadoop can be used in the Hadoop world.  In that next post, instead of having our data in an Oracle database, our data will live in Hadoop and we will how to use the Oracle R Connector for Hadoop and other Oracle Big Data Connectors to move data between Hadoop, R, and the Oracle Database easily.

  Read the article

< Previous Page | 59 60 61 62 63 64 65 66 67 68 69 70  | Next Page >