Search Results

Search found 40744 results on 1630 pages for 'sql interview questions a'.

Page 191/1630 | < Previous Page | 187 188 189 190 191 192 193 194 195 196 197 198  | Next Page >

  • Most efficient way to maintain a 'set' in SQL Server?

    - by SEVEN YEAR LIBERAL ARTS DEGREE
    I have ~2 million rows or so of data, each row with an artificial PK, and two Id fields (so: PK, ID1, ID2). I have a unique constraint (and index) on ID1+ID2. I get two sorts of updates, both with a distinct ID1 per update. 100-1000 rows of all-new data (ID1 is new) 100-1000 rows of largely, but not necessarily completely overlapping data (ID1 already exists, maybe new ID1+ID2 pairs) What's the most efficient way to maintain this 'set'? Here are the options as I see them: Delete all the rows with ID1, insert all the new rows (yikes) Query all the existing rows from the set of new data ID1+ID2, only insert the new rows Insert all the new rows, ignore inserts that trigger unique constraint violations Any thoughts?

    Read the article

  • i need to do a view in sql that returns the latest invoice date for each company

    - by dave haughton
    hi, i have a company table that is dbo.companies and has companyId as a column. I also have an invoice table that is dbo.invoices with invoicecompanyId column (which is the same as the companyId on the other table) and it also has a column called invoicedate. What i am mtrying to achieve is a view of each companyid with the corresponding latest invoice date for all the companies i have. i have done the following but i dont know how to filter for the latest invoice, it returns all invoices from all companies and i need latest invoice for all companies SELECT TOP (100) PERCENT 'A' + SUBSTRING('000000', 1, 6 - LEN(CAST(dbo.companies.companyId AS varchar(10)))) + CAST(dbo.companies.companyId AS varchar(10)) AS Client_ID, dbo.invoices.invoiceDate AS S_Inv_Date FROM dbo.invoices INNER JOIN dbo.companies ON dbo.invoices.invoiceCompanyId = dbo.companies.companyId ORDER BY Client_ID can you help please ta

    Read the article

  • How can I use SQL Server's full text search across multiple rows at once?

    - by Morbo
    I'm trying to improve the search functionality on my web forums. I've got a table of posts, and each post has (among other less interesting things): PostID, a unique ID for the individual post. ThreadID, an ID of the thread the post belongs to. There can be any number of posts per thread. Text, because a forum would be really boring without it. I want to write an efficient query that will search the threads in the forum for a series of words, and it should return a hit for any ThreadID for which there are posts that include all of the search words. For example, let's say that thread 9 has post 1001 with the word "cat" in it, and also post 1027 with the word "hat" in it. I want a search for cat hat to return a hit for thread 9. This seems like a straightforward requirement, but I don't know of an efficient way to do it. Using the regular FREETEXT and CONTAINS capabilities for N'cat AND hat' won't return any hits in the above example because the words exist in different posts, even though those posts are in the same thread. (As far as I can tell, when using CREATE FULLTEXT INDEX I have to give it my index on the primary key PostID, and can't tell it to index all posts with the same ThreadID together.) The solution that I currently have in place works, but sucks: maintain a separate table that contains the entire concatenated post text of every thread, and make a full text index on THAT. I'm looking for a solution that doesn't require me to keep a duplicate copy of the entire text of every thread in my forums. Any ideas? Am I missing something obvious?

    Read the article

  • SQLce create table "index field" explanation (SQL Management Studio) ?

    - by bretddog
    Hi, I'm new to databases, and now creating SQLCE database in Management Studio. There is a value in brackets; [UQ_Users_0000000000000028], which seems a bit random to me, so would just like to ask if someone could explain this field? Is it just simply required to be a unique field? Is there any reason why I would want to change it to something else than what SSMS scripts? cheers! CREATE TABLE [Users] ( [UserID] int NOT NULL ..... CREATE UNIQUE INDEX [UQ__Users__0000000000000028] ON [Users] ([UserID] ASC);

    Read the article

  • What is the best way to partition large tables in SQL Server?

    - by RyanFetz
    In a recent project the "lead" developer designed a database schema where "larger" tables would be split across two seperate databases with a view on the main database which unioned the two seperate database-tables together. The main database is what the application was driven off of so these tables looked and felt like ordinary tables (except some quirkly things around updating). This seemed like a HUGE performance problem. We do see problems with performance around these tables but nothing to make him change his mind about his design. Just wondering what is the best way to do this, or if it is even worth doing?

    Read the article

  • How do I do pivoting in this query in SQL?

    - by dewacorp.alliances
    Hi there I have this table like this: Name; Amount1, Amount, Rate1, Rate2 Test; 1000; 2000; 1.0; 2.0 I want to display into: Parameter; Amount1; Rate1; Total 'Parameter 1'; 1000; 1.0; 1000 'Parameter 2'; 2000; 2.0; 4000 BTW ... I am using SQL2K5. All I can think of is CURSOR. Any other solution in elegant way? Thanks

    Read the article

  • Find existence of number in a sorted list in constant time? (Interview question)

    - by Rich
    I'm studying for upcoming interviews and have encountered this question several times (written verbatim) Find or determine non existence of a number in a sorted list of N numbers where the numbers range over M, M N and N large enough to span multiple disks. Algorithm to beat O(log n); bonus points for constant time algorithm. First of all, I'm not sure if this is a question with a real solution. My colleagues and I have mused over this problem for weeks and it seems ill formed (of course, just because we can't think of a solution doesn't mean there isn't one). A few questions I would have asked the interviewer are: Are there repeats in the sorted list? What's the relationship to the number of disks and N? One approach I considered was to binary search the min/max of each disk to determine the disk that should hold that number, if it exists, then binary search on the disk itself. Of course this is only an order of magnitude speedup if the number of disks is large and you also have a sorted list of disks. I think this would yield some sort of O(log log n) time. As for the M N hint, perhaps if you know how many numbers are on a disk and what the range is, you could use the pigeonhole principle to rule out some cases some of the time, but I can't figure out an order of magnitude improvement. Also, "bonus points for constant time algorithm" makes me a bit suspicious. Any thoughts, solutions, or relevant history of this problem?

    Read the article

  • My tricky SQL Update query not working so well.

    - by Spooks
    I am trying to update a table in my database with another row from another table. I have two parameters one being the ID and another being the row number (as you can select which row you want from the GUI) this part of the code works fine, this returns one column of a single row. (select txtPageContent FROM (select *, Row_Number() OVER (ORDER BY ArchiveDate asc) as rowid from ARC_Content Where ContentID = @ContentID) as test Where rowid = @rowID) its just when i try to add the update/set it won't work. I am probably missing something update TBL_Content Set TBL_Content.txtPageContent = (select txtPageContent FROM (select *, Row_Number() OVER (ORDER BY ArchiveDate asc) as rowid from ARC_Content Where ContentID = @ContentID) as test Where rowid = @rowID) Thanks for the help! (i have tried top 1 with no avail)

    Read the article

  • SQL Server FTS: possible to get information how/why rows were matched?

    - by jimmy_keen
    Is it possible to get the information why/how given row returned by FTS query was matched (or which substring caused row to match)? For example, consider simpliest table with id and text columns, with FTS index on the later one. SELECT * FROM Example WHERE CONTAINS(text, 'FORMSOF(INFLECTIONAL, jump)'); This examplary query could return, say row {1, 'Jumping Jack'}. Now, is it possible to somehow get information that this very row was matched because of 'Jumping' word? It doesn't even have to be exact information, more of a which substring caused row to match. Why I'm asking - I got C# app that builds up those queries basing on user input (keywords to search for), and I need the very basic information why/how row was matched back, to use further in C# code. If it's not possible, any alternatives?

    Read the article

  • How can I get reports for last 24h in SQL?

    - by newbie
    I need to get all reports made in last 24h, table has CreatedDate column, so I need to check in database that report was created in last 24h. I know I can use getdate() to get current date, but how can I minus 24h from that attrbiute and then compare that date with CretedDate?

    Read the article

  • Why cant i use the field user in SQL Server 8?

    - by acidzombie24
    Maybe not literally but the query below gets an error near user. If i change it to userZ it works. WHY can i not use that name? Is there a way to specific its a field instead of a keyword? (or whatever it is) create table Post2 ( id INTEGER PRIMARY KEY NOT NULL, title nvarchar(max) NOT NULL, body nvarchar(max) NOT NULL, user integer REFERENCES Post1(id));

    Read the article

  • Why does SQL 2005 SSIS component install fail?

    - by Ducain
    I am trying to install SSIS on our production SQL 2005 SP2 box. Each time I try, the install/setup screen results in failure, starting with the native client, and moving on down. Screen shots below show what I see: Here is the result of clicking on the status link to the right of the native client after the install failed: === Verbose logging started: 3/28/2012 16:38:08 Build type: SHIP UNICODE 3.01.4000.4042 Calling process: C:\Program Files\Microsoft SQL Server\90\Setup Bootstrap\setup.exe === MSI (c) (DC:00) [16:38:08:875]: Resetting cached policy values MSI (c) (DC:00) [16:38:08:875]: Machine policy value 'Debug' is 0 MSI (c) (DC:00) [16:38:08:875]: ******* RunEngine: ******* Product: {F9B3DD02-B0B3-42E9-8650-030DFF0D133D} ******* Action: ******* CommandLine: ********** MSI (c) (DC:00) [16:38:08:875]: Client-side and UI is none or basic: Running entire install on the server. MSI (c) (DC:00) [16:38:08:875]: Grabbed execution mutex. MSI (c) (DC:00) [16:38:08:875]: Cloaking enabled. MSI (c) (DC:00) [16:38:08:875]: Attempting to enable all disabled priveleges before calling Install on Server MSI (c) (DC:00) [16:38:08:875]: Incrementing counter to disable shutdown. Counter after increment: 0 MSI (s) (90:F0) [16:38:08:875]: Grabbed execution mutex. MSI (s) (90:D4) [16:38:08:875]: Resetting cached policy values MSI (s) (90:D4) [16:38:08:875]: Machine policy value 'Debug' is 0 MSI (s) (90:D4) [16:38:08:875]: ******* RunEngine: ******* Product: {F9B3DD02-B0B3-42E9-8650-030DFF0D133D} ******* Action: ******* CommandLine: ********** MSI (s) (90:D4) [16:38:08:875]: Machine policy value 'DisableUserInstalls' is 0 MSI (s) (90:D4) [16:38:08:890]: Warning: Local cached package 'C:\WINDOWS\Installer\65eb99.msi' is missing. MSI (s) (90:D4) [16:38:08:890]: User policy value 'SearchOrder' is 'nmu' MSI (s) (90:D4) [16:38:08:890]: User policy value 'DisableMedia' is 0 MSI (s) (90:D4) [16:38:08:890]: Machine policy value 'AllowLockdownMedia' is 0 MSI (s) (90:D4) [16:38:08:890]: SOURCEMGMT: Media enabled only if package is safe. MSI (s) (90:D4) [16:38:08:890]: SOURCEMGMT: Looking for sourcelist for product {F9B3DD02-B0B3-42E9-8650-030DFF0D133D} MSI (s) (90:D4) [16:38:08:890]: SOURCEMGMT: Adding {F9B3DD02-B0B3-42E9-8650-030DFF0D133D}; to potential sourcelist list (pcode;disk;relpath). MSI (s) (90:D4) [16:38:08:890]: SOURCEMGMT: Now checking product {F9B3DD02-B0B3-42E9-8650-030DFF0D133D} MSI (s) (90:D4) [16:38:08:890]: SOURCEMGMT: Media is enabled for product. MSI (s) (90:D4) [16:38:08:890]: SOURCEMGMT: Attempting to use LastUsedSource from source list. MSI (s) (90:D4) [16:38:08:890]: SOURCEMGMT: Trying source C:\Program Files\Microsoft SQL Server\90\Setup Bootstrap\Cache\. MSI (s) (90:D4) [16:38:08:890]: SOURCEMGMT: Source is invalid due to invalid package code (product code doesn't match). MSI (s) (90:D4) [16:38:08:890]: Note: 1: 1706 2: -2147483646 3: sqlncli.msi MSI (s) (90:D4) [16:38:08:890]: SOURCEMGMT: Processing net source list. MSI (s) (90:D4) [16:38:08:890]: Note: 1: 1706 2: -2147483647 3: sqlncli.msi MSI (s) (90:D4) [16:38:08:890]: SOURCEMGMT: Processing media source list. MSI (s) (90:D4) [16:38:09:921]: SOURCEMGMT: Trying media source F:\. MSI (s) (90:D4) [16:38:09:921]: Note: 1: 2203 2: F:\sqlncli.msi 3: -2147287038 MSI (s) (90:D4) [16:38:09:921]: SOURCEMGMT: Source is invalid due to missing/inaccessible package. MSI (s) (90:D4) [16:38:09:921]: Note: 1: 1706 2: -2147483647 3: sqlncli.msi MSI (s) (90:D4) [16:38:09:921]: SOURCEMGMT: Processing URL source list. MSI (s) (90:D4) [16:38:09:921]: Note: 1: 1402 2: UNKNOWN\URL 3: 2 MSI (s) (90:D4) [16:38:09:921]: Note: 1: 1706 2: -2147483647 3: sqlncli.msi MSI (s) (90:D4) [16:38:09:921]: Note: 1: 1706 2: 3: sqlncli.msi MSI (s) (90:D4) [16:38:09:921]: SOURCEMGMT: Failed to resolve source MSI (s) (90:D4) [16:38:09:921]: MainEngineThread is returning 1612 MSI (c) (DC:00) [16:38:09:921]: Decrementing counter to disable shutdown. If counter >= 0, shutdown will be denied. Counter after decrement: -1 MSI (c) (DC:00) [16:38:09:921]: MainEngineThread is returning 1612 === Verbose logging stopped: 3/28/2012 16:38:09 === Here is the log visible when I click the failed status for MSXML6: === Verbose logging started: 3/28/2012 16:38:12 Build type: SHIP UNICODE 3.01.4000.4042 Calling process: C:\Program Files\Microsoft SQL Server\90\Setup Bootstrap\setup.exe === MSI (c) (DC:58) [16:38:12:250]: Resetting cached policy values MSI (c) (DC:58) [16:38:12:250]: Machine policy value 'Debug' is 0 MSI (c) (DC:58) [16:38:12:250]: ******* RunEngine: ******* Product: {56EA8BC0-3751-4B93-BC9D-6651CC36E5AA} ******* Action: ******* CommandLine: ********** MSI (c) (DC:58) [16:38:12:250]: Client-side and UI is none or basic: Running entire install on the server. MSI (c) (DC:58) [16:38:12:250]: Grabbed execution mutex. MSI (c) (DC:58) [16:38:12:250]: Cloaking enabled. MSI (c) (DC:58) [16:38:12:250]: Attempting to enable all disabled priveleges before calling Install on Server MSI (c) (DC:58) [16:38:12:250]: Incrementing counter to disable shutdown. Counter after increment: 0 MSI (s) (90:58) [16:38:12:265]: Grabbed execution mutex. MSI (s) (90:DC) [16:38:12:265]: Resetting cached policy values MSI (s) (90:DC) [16:38:12:265]: Machine policy value 'Debug' is 0 MSI (s) (90:DC) [16:38:12:265]: ******* RunEngine: ******* Product: {56EA8BC0-3751-4B93-BC9D-6651CC36E5AA} ******* Action: ******* CommandLine: ********** MSI (s) (90:DC) [16:38:12:265]: Machine policy value 'DisableUserInstalls' is 0 MSI (s) (90:DC) [16:38:12:265]: Warning: Local cached package 'C:\WINDOWS\Installer\ce6d56e.msi' is missing. MSI (s) (90:DC) [16:38:12:265]: User policy value 'SearchOrder' is 'nmu' MSI (s) (90:DC) [16:38:12:265]: User policy value 'DisableMedia' is 0 MSI (s) (90:DC) [16:38:12:265]: Machine policy value 'AllowLockdownMedia' is 0 MSI (s) (90:DC) [16:38:12:265]: SOURCEMGMT: Media enabled only if package is safe. MSI (s) (90:DC) [16:38:12:265]: SOURCEMGMT: Looking for sourcelist for product {56EA8BC0-3751-4B93-BC9D-6651CC36E5AA} MSI (s) (90:DC) [16:38:12:265]: SOURCEMGMT: Adding {56EA8BC0-3751-4B93-BC9D-6651CC36E5AA}; to potential sourcelist list (pcode;disk;relpath). MSI (s) (90:DC) [16:38:12:265]: SOURCEMGMT: Now checking product {56EA8BC0-3751-4B93-BC9D-6651CC36E5AA} MSI (s) (90:DC) [16:38:12:265]: SOURCEMGMT: Media is enabled for product. MSI (s) (90:DC) [16:38:12:265]: SOURCEMGMT: Attempting to use LastUsedSource from source list. MSI (s) (90:DC) [16:38:12:265]: SOURCEMGMT: Trying source d:\2a2ac35788eea9066bae01\. MSI (s) (90:DC) [16:38:12:265]: Note: 1: 2203 2: d:\2a2ac35788eea9066bae01\msxml6.msi 3: -2147287037 MSI (s) (90:DC) [16:38:12:265]: SOURCEMGMT: Source is invalid due to missing/inaccessible package. MSI (s) (90:DC) [16:38:12:265]: Note: 1: 1706 2: -2147483647 3: msxml6.msi MSI (s) (90:DC) [16:38:12:265]: SOURCEMGMT: Processing net source list. MSI (s) (90:DC) [16:38:12:265]: Note: 1: 1706 2: -2147483647 3: msxml6.msi MSI (s) (90:DC) [16:38:12:265]: SOURCEMGMT: Processing media source list. MSI (s) (90:DC) [16:38:12:296]: SOURCEMGMT: Trying media source F:\. MSI (s) (90:DC) [16:38:12:296]: Note: 1: 2203 2: F:\msxml6.msi 3: -2147287038 MSI (s) (90:DC) [16:38:12:296]: SOURCEMGMT: Source is invalid due to missing/inaccessible package. MSI (s) (90:DC) [16:38:12:296]: Note: 1: 1706 2: -2147483647 3: msxml6.msi MSI (s) (90:DC) [16:38:12:296]: SOURCEMGMT: Processing URL source list. MSI (s) (90:DC) [16:38:12:296]: Note: 1: 1402 2: UNKNOWN\URL 3: 2 MSI (s) (90:DC) [16:38:12:296]: Note: 1: 1706 2: -2147483647 3: msxml6.msi MSI (s) (90:DC) [16:38:12:296]: Note: 1: 1706 2: 3: msxml6.msi MSI (s) (90:DC) [16:38:12:296]: SOURCEMGMT: Failed to resolve source MSI (s) (90:DC) [16:38:12:296]: MainEngineThread is returning 1612 MSI (c) (DC:58) [16:38:12:296]: Decrementing counter to disable shutdown. If counter >= 0, shutdown will be denied. Counter after decrement: -1 MSI (c) (DC:58) [16:38:12:296]: MainEngineThread is returning 1612 === Verbose logging stopped: 3/28/2012 16:38:12 === When I click on the failed status for SSIS, no log file appears at all. To be honest, I'm not even sure where to start on this one - never guessed it would be so much trouble to add a component right from the disk. Any help or pointers whatsoever would be greatly appreciated. If any more details are needed, please ask - I'd be glad to add them.

    Read the article

  • Bulk inserting best way to about it? + Helping me understand fully what I found so far

    - by chobo2
    Hi So I saw this post here and read it and it seems like bulk copy might be the way to go. http://stackoverflow.com/questions/682015/whats-the-best-way-to-bulk-database-inserts-from-c I still have some questions and want to know how things actually work. So I found 2 tutorials. http://www.codeproject.com/KB/cs/MultipleInsertsIn1dbTrip.aspx#_Toc196622241 http://www.codeproject.com/KB/linq/BulkOperations_LinqToSQL.aspx First way uses 2 ado.net 2.0 features. BulkInsert and BulkCopy. the second one uses linq to sql and OpenXML. This sort of appeals to me as I am using linq to sql already and prefer it over ado.net. However as one person pointed out in the posts what he just going around the issue at the cost of performance( nothing wrong with that in my opinion) First I will talk about the 2 ways in the first tutorial I am using VS2010 Express, .net 4.0, MVC 2.0, SQl Server 2005 Is ado.net 2.0 the most current version? Based on the technology I am using, is there some updates to what I am going to show that would improve it somehow? Is there any thing that these tutorial left out that I should know about? BulkInsert I am using this table for all the examples. CREATE TABLE [dbo].[TBL_TEST_TEST] ( ID INT IDENTITY(1,1) PRIMARY KEY, [NAME] [varchar](50) ) SP Code USE [Test] GO /****** Object: StoredProcedure [dbo].[sp_BatchInsert] Script Date: 05/19/2010 15:12:47 ******/ SET ANSI_NULLS ON GO SET QUOTED_IDENTIFIER ON GO ALTER PROCEDURE [dbo].[sp_BatchInsert] (@Name VARCHAR(50) ) AS BEGIN INSERT INTO TBL_TEST_TEST VALUES (@Name); END C# Code /// <summary> /// Another ado.net 2.0 way that uses a stored procedure to do a bulk insert. /// Seems slower then "BatchBulkCopy" way and it crashes when you try to insert 500,000 records in one go. /// http://www.codeproject.com/KB/cs/MultipleInsertsIn1dbTrip.aspx#_Toc196622241 /// </summary> private static void BatchInsert() { // Get the DataTable with Rows State as RowState.Added DataTable dtInsertRows = GetDataTable(); SqlConnection connection = new SqlConnection(connectionString); SqlCommand command = new SqlCommand("sp_BatchInsert", connection); command.CommandType = CommandType.StoredProcedure; command.UpdatedRowSource = UpdateRowSource.None; // Set the Parameter with appropriate Source Column Name command.Parameters.Add("@Name", SqlDbType.VarChar, 50, dtInsertRows.Columns[0].ColumnName); SqlDataAdapter adpt = new SqlDataAdapter(); adpt.InsertCommand = command; // Specify the number of records to be Inserted/Updated in one go. Default is 1. adpt.UpdateBatchSize = 1000; connection.Open(); int recordsInserted = adpt.Update(dtInsertRows); connection.Close(); } So first thing is the batch size. Why would you set a batch size to anything but the number of records you are sending? Like I am sending 500,000 records so I did a Batch size of 500,000. Next why does it crash when I do this? If I set it to 1000 for batch size it works just fine. System.Data.SqlClient.SqlException was unhandled Message="A transport-level error has occurred when sending the request to the server. (provider: Shared Memory Provider, error: 0 - No process is on the other end of the pipe.)" Source=".Net SqlClient Data Provider" ErrorCode=-2146232060 Class=20 LineNumber=0 Number=233 Server="" State=0 StackTrace: at System.Data.Common.DbDataAdapter.UpdatedRowStatusErrors(RowUpdatedEventArgs rowUpdatedEvent, BatchCommandInfo[] batchCommands, Int32 commandCount) at System.Data.Common.DbDataAdapter.UpdatedRowStatus(RowUpdatedEventArgs rowUpdatedEvent, BatchCommandInfo[] batchCommands, Int32 commandCount) at System.Data.Common.DbDataAdapter.Update(DataRow[] dataRows, DataTableMapping tableMapping) at System.Data.Common.DbDataAdapter.UpdateFromDataTable(DataTable dataTable, DataTableMapping tableMapping) at System.Data.Common.DbDataAdapter.Update(DataTable dataTable) at TestIQueryable.Program.BatchInsert() in C:\Users\a\Downloads\TestIQueryable\TestIQueryable\TestIQueryable\Program.cs:line 124 at TestIQueryable.Program.Main(String[] args) in C:\Users\a\Downloads\TestIQueryable\TestIQueryable\TestIQueryable\Program.cs:line 16 InnerException: Time it took to insert 500,000 records with insert batch size of 1000 took "2 mins and 54 seconds" Of course this is no official time I sat there with a stop watch( I am sure there are better ways but was too lazy to look what they where) So I find that kinda slow compared to all my other ones(expect the linq to sql insert one) and I am not really sure why. Next I looked at bulkcopy /// <summary> /// An ado.net 2.0 way to mass insert records. This seems to be the fastest. /// http://www.codeproject.com/KB/cs/MultipleInsertsIn1dbTrip.aspx#_Toc196622241 /// </summary> private static void BatchBulkCopy() { // Get the DataTable DataTable dtInsertRows = GetDataTable(); using (SqlBulkCopy sbc = new SqlBulkCopy(connectionString, SqlBulkCopyOptions.KeepIdentity)) { sbc.DestinationTableName = "TBL_TEST_TEST"; // Number of records to be processed in one go sbc.BatchSize = 500000; // Map the Source Column from DataTabel to the Destination Columns in SQL Server 2005 Person Table // sbc.ColumnMappings.Add("ID", "ID"); sbc.ColumnMappings.Add("NAME", "NAME"); // Number of records after which client has to be notified about its status sbc.NotifyAfter = dtInsertRows.Rows.Count; // Event that gets fired when NotifyAfter number of records are processed. sbc.SqlRowsCopied += new SqlRowsCopiedEventHandler(sbc_SqlRowsCopied); // Finally write to server sbc.WriteToServer(dtInsertRows); sbc.Close(); } } This one seemed to go really fast and did not even need a SP( can you use SP with bulk copy? If you can would it be better?) BatchCopy had no problem with a 500,000 batch size.So again why make it smaller then the number of records you want to send? I found that with BatchCopy and 500,000 batch size it took only 5 seconds to complete. I then tried with a batch size of 1,000 and it only took 8 seconds. So much faster then the bulkinsert one above. Now I tried the other tutorial. USE [Test] GO /****** Object: StoredProcedure [dbo].[spTEST_InsertXMLTEST_TEST] Script Date: 05/19/2010 15:39:03 ******/ SET ANSI_NULLS ON GO SET QUOTED_IDENTIFIER ON GO ALTER PROCEDURE [dbo].[spTEST_InsertXMLTEST_TEST](@UpdatedProdData nText) AS DECLARE @hDoc int exec sp_xml_preparedocument @hDoc OUTPUT,@UpdatedProdData INSERT INTO TBL_TEST_TEST(NAME) SELECT XMLProdTable.NAME FROM OPENXML(@hDoc, 'ArrayOfTBL_TEST_TEST/TBL_TEST_TEST', 2) WITH ( ID Int, NAME varchar(100) ) XMLProdTable EXEC sp_xml_removedocument @hDoc C# code. /// <summary> /// This is using linq to sql to make the table objects. /// It is then serailzed to to an xml document and sent to a stored proedure /// that then does a bulk insert(I think with OpenXML) /// http://www.codeproject.com/KB/linq/BulkOperations_LinqToSQL.aspx /// </summary> private static void LinqInsertXMLBatch() { using (TestDataContext db = new TestDataContext()) { TBL_TEST_TEST[] testRecords = new TBL_TEST_TEST[500000]; for (int count = 0; count < 500000; count++) { TBL_TEST_TEST testRecord = new TBL_TEST_TEST(); testRecord.NAME = "Name : " + count; testRecords[count] = testRecord; } StringBuilder sBuilder = new StringBuilder(); System.IO.StringWriter sWriter = new System.IO.StringWriter(sBuilder); XmlSerializer serializer = new XmlSerializer(typeof(TBL_TEST_TEST[])); serializer.Serialize(sWriter, testRecords); db.insertTestData(sBuilder.ToString()); } } So I like this because I get to use objects even though it is kinda redundant. I don't get how the SP works. Like I don't get the whole thing. I don't know if OPENXML has some batch insert under the hood but I do not even know how to take this example SP and change it to fit my tables since like I said I don't know what is going on. I also don't know what would happen if the object you have more tables in it. Like say I have a ProductName table what has a relationship to a Product table or something like that. In linq to sql you could get the product name object and make changes to the Product table in that same object. So I am not sure how to take that into account. I am not sure if I would have to do separate inserts or what. The time was pretty good for 500,000 records it took 52 seconds The last way of course was just using linq to do it all and it was pretty bad. /// <summary> /// This is using linq to sql to to insert lots of records. /// This way is slow as it uses no mass insert. /// Only tried to insert 50,000 records as I did not want to sit around till it did 500,000 records. /// http://www.codeproject.com/KB/linq/BulkOperations_LinqToSQL.aspx /// </summary> private static void LinqInsertAll() { using (TestDataContext db = new TestDataContext()) { db.CommandTimeout = 600; for (int count = 0; count < 50000; count++) { TBL_TEST_TEST testRecord = new TBL_TEST_TEST(); testRecord.NAME = "Name : " + count; db.TBL_TEST_TESTs.InsertOnSubmit(testRecord); } db.SubmitChanges(); } } I did only 50,000 records and that took over a minute to do. So I really narrowed it done to the linq to sql bulk insert way or bulk copy. I am just not sure how to do it when you have relationship for either way. I am not sure how they both stand up when doing updates instead of inserts as I have not gotten around to try it yet. I don't think I will ever need to insert/update more than 50,000 records at one type but at the same time I know I will have to do validation on records before inserting so that will slow it down and that sort of makes linq to sql nicer as your got objects especially if your first parsing data from a xml file before you insert into the database. Full C# code using System; using System.Collections.Generic; using System.Linq; using System.Text; using System.Xml.Serialization; using System.Data; using System.Data.SqlClient; namespace TestIQueryable { class Program { private static string connectionString = ""; static void Main(string[] args) { BatchInsert(); Console.WriteLine("done"); } /// <summary> /// This is using linq to sql to to insert lots of records. /// This way is slow as it uses no mass insert. /// Only tried to insert 50,000 records as I did not want to sit around till it did 500,000 records. /// http://www.codeproject.com/KB/linq/BulkOperations_LinqToSQL.aspx /// </summary> private static void LinqInsertAll() { using (TestDataContext db = new TestDataContext()) { db.CommandTimeout = 600; for (int count = 0; count < 50000; count++) { TBL_TEST_TEST testRecord = new TBL_TEST_TEST(); testRecord.NAME = "Name : " + count; db.TBL_TEST_TESTs.InsertOnSubmit(testRecord); } db.SubmitChanges(); } } /// <summary> /// This is using linq to sql to make the table objects. /// It is then serailzed to to an xml document and sent to a stored proedure /// that then does a bulk insert(I think with OpenXML) /// http://www.codeproject.com/KB/linq/BulkOperations_LinqToSQL.aspx /// </summary> private static void LinqInsertXMLBatch() { using (TestDataContext db = new TestDataContext()) { TBL_TEST_TEST[] testRecords = new TBL_TEST_TEST[500000]; for (int count = 0; count < 500000; count++) { TBL_TEST_TEST testRecord = new TBL_TEST_TEST(); testRecord.NAME = "Name : " + count; testRecords[count] = testRecord; } StringBuilder sBuilder = new StringBuilder(); System.IO.StringWriter sWriter = new System.IO.StringWriter(sBuilder); XmlSerializer serializer = new XmlSerializer(typeof(TBL_TEST_TEST[])); serializer.Serialize(sWriter, testRecords); db.insertTestData(sBuilder.ToString()); } } /// <summary> /// An ado.net 2.0 way to mass insert records. This seems to be the fastest. /// http://www.codeproject.com/KB/cs/MultipleInsertsIn1dbTrip.aspx#_Toc196622241 /// </summary> private static void BatchBulkCopy() { // Get the DataTable DataTable dtInsertRows = GetDataTable(); using (SqlBulkCopy sbc = new SqlBulkCopy(connectionString, SqlBulkCopyOptions.KeepIdentity)) { sbc.DestinationTableName = "TBL_TEST_TEST"; // Number of records to be processed in one go sbc.BatchSize = 500000; // Map the Source Column from DataTabel to the Destination Columns in SQL Server 2005 Person Table // sbc.ColumnMappings.Add("ID", "ID"); sbc.ColumnMappings.Add("NAME", "NAME"); // Number of records after which client has to be notified about its status sbc.NotifyAfter = dtInsertRows.Rows.Count; // Event that gets fired when NotifyAfter number of records are processed. sbc.SqlRowsCopied += new SqlRowsCopiedEventHandler(sbc_SqlRowsCopied); // Finally write to server sbc.WriteToServer(dtInsertRows); sbc.Close(); } } /// <summary> /// Another ado.net 2.0 way that uses a stored procedure to do a bulk insert. /// Seems slower then "BatchBulkCopy" way and it crashes when you try to insert 500,000 records in one go. /// http://www.codeproject.com/KB/cs/MultipleInsertsIn1dbTrip.aspx#_Toc196622241 /// </summary> private static void BatchInsert() { // Get the DataTable with Rows State as RowState.Added DataTable dtInsertRows = GetDataTable(); SqlConnection connection = new SqlConnection(connectionString); SqlCommand command = new SqlCommand("sp_BatchInsert", connection); command.CommandType = CommandType.StoredProcedure; command.UpdatedRowSource = UpdateRowSource.None; // Set the Parameter with appropriate Source Column Name command.Parameters.Add("@Name", SqlDbType.VarChar, 50, dtInsertRows.Columns[0].ColumnName); SqlDataAdapter adpt = new SqlDataAdapter(); adpt.InsertCommand = command; // Specify the number of records to be Inserted/Updated in one go. Default is 1. adpt.UpdateBatchSize = 500000; connection.Open(); int recordsInserted = adpt.Update(dtInsertRows); connection.Close(); } private static DataTable GetDataTable() { // You First need a DataTable and have all the insert values in it DataTable dtInsertRows = new DataTable(); dtInsertRows.Columns.Add("NAME"); for (int i = 0; i < 500000; i++) { DataRow drInsertRow = dtInsertRows.NewRow(); string name = "Name : " + i; drInsertRow["NAME"] = name; dtInsertRows.Rows.Add(drInsertRow); } return dtInsertRows; } static void sbc_SqlRowsCopied(object sender, SqlRowsCopiedEventArgs e) { Console.WriteLine("Number of records affected : " + e.RowsCopied.ToString()); } } }

    Read the article

  • Putting indexes in separate filegroup kills our queries

    - by womp
    Can anyone shed some light on this? On our dev boxes, our database resides entirely in the PRIMARY filegroup, and everything works fine. On one of our production servers, recently upgraded from 2005 to 2008, we noticed it was performing slower than it should. On this machine, there are two filegroups - PRIMARY and INDEXES. Both filegroups contain 1 file per logical volume, one logical volume per CPU, (and each logical volume is a RAID 10 of 4 physical disks). We isolated a few queries that were performing fast on the dev boxes and slow (up to 40x slower) on the production machine. Turned out these queries were using the non-clustered indexes that resided in the INDEXES filegroup. Tweaking some of the queries to only use clustered indexes that were in the PRIMARY filegroup dropped their times back to normal. As a final confirmation, we redeployed the same database on the same machine to have everything in PRIMARY, and things went back to normal! Here's the statistics output of one of the queries, run identically on the machine with different filegroup configurations (table names changed to protect the innocent): FAST (everything in PRIMARY filegroup): (3 row(s) affected) Table '0'. Scan count 2, logical reads 14, ... Table '1'. Scan count 0, logical reads 0, ... Table '1'. Scan count 0, logical reads 0, ... Table '2'. Scan count 2, logical reads 7, ... Table '3'. Scan count 2, logical reads 1012, ... Table '4'. Scan count 1, logical reads 3, ... SQL Server Execution Times: CPU time = 437 ms, elapsed time = 445 ms. SLOW (indexes split into their own filegroup): (3 row(s) affected) Table '0'. Scan count 209, logical reads 428, ... Table '1'. Scan count 0, logical reads 0,... Table '2'. Scan count 1021, logical reads 9043,.... Table '3'. Scan count 209, logical reads 105754, .... Table '4'. Scan count 0, logical reads 0, .... Table '5'. Scan count 1, logical reads 695, ... **Table '#46DA8CA9'. Scan count 205, logical reads 205, ...** Table '6'. Scan count 6, logical reads 436, ... Table '7'. Scan count 1, logical reads 12,.... SQL Server Execution Times: CPU time = 17581 ms, elapsed time = 17595 ms. Notice the weird temp table and extra tables involved in the slow query. It seems clear that having a second file group is making SQL Server batty with choosing an execution plan. What the heck is going on?

    Read the article

  • Attempting Unauthorized operation - SQL 2008 R2 install

    - by Fred L
    I've been banging against this for a few days. Keep getting this unauthorized error when trying to install SQL 2008 R2 on a Windows 7 machine. I've changed permissions on the key, does not fix... Created an admin user, gave specific permissions on that key, does not fix... Disabled all firewalls, installed from a local admin, does not fix... I'm out of patience and ideas! :) Help? 2012-07-06 13:09:11 Slp: Sco: Attempting to set value AppName 2012-07-06 13:09:11 Slp: SetValue: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\VSTAHostConfig\SSIS_ScriptComponent\2.0, Name = AppName 2012-07-06 13:09:11 Slp: Sco: Attempting to create base registry key HKEY_LOCAL_MACHINE, machine 2012-07-06 13:09:11 SSIS: Processing Registry ACLs for SID 'S-1-5-21-2383144575-3599344511-819193542-1074' 2012-07-06 13:09:11 Slp: Sco: Attempting to open registry subkey SOFTWARE\Microsoft\Microsoft SQL Server\100 2012-07-06 13:09:11 SSIS: Setting permision on registry key HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft SQL Server\100. 2012-07-06 13:09:11 Slp: Sco: Attempting to replace account with sid in security descriptor D:(A;OICI;KR;;;S-1-5-21-2383144575-3599344511-819193542-1074) 2012-07-06 13:09:11 Slp: ReplaceAccountWithSidInSddl -- SDDL to be processed: D:(A;OICI;KR;;;S-1-5-21-2383144575-3599344511-819193542-1074) 2012-07-06 13:09:11 Slp: ReplaceAccountWithSidInSddl -- SDDL to be returned: D:(A;OICI;KR;;;S-1-5-21-2383144575-3599344511-819193542-1074) 2012-07-06 13:09:11 Slp: Sco: Attempting to set security descriptor D:(A;OICI;KR;;;S-1-5-21-2383144575-3599344511-819193542-1074) 2012-07-06 13:09:11 Slp: Sco: Attempting to normalize security descriptor D:(A;OICI;KR;;;S-1-5-21-2383144575-3599344511-819193542-1074) 2012-07-06 13:09:11 Slp: Sco: Attempting to replace account with sid in security descriptor D:(A;OICI;KR;;;S-1-5-21-2383144575-3599344511-819193542-1074) 2012-07-06 13:09:11 Slp: ReplaceAccountWithSidInSddl -- SDDL to be processed: D:(A;OICI;KR;;;S-1-5-21-2383144575-3599344511-819193542-1074) 2012-07-06 13:09:11 Slp: ReplaceAccountWithSidInSddl -- SDDL to be returned: D:(A;OICI;KR;;;S-1-5-21-2383144575-3599344511-819193542-1074) 2012-07-06 13:09:11 Slp: Sco: Attempting to normalize security descriptor D:(A;OICI;KR;;;S-1-5-21-2383144575-3599344511-819193542-1074) 2012-07-06 13:09:11 Slp: Sco: Attempting to replace account with sid in security descriptor D:(A;OICI;KR;;;S-1-5-21-2383144575-3599344511-819193542-1074) 2012-07-06 13:09:11 Slp: ReplaceAccountWithSidInSddl -- SDDL to be processed: D:(A;OICI;KR;;;S-1-5-21-2383144575-3599344511-819193542-1074) 2012-07-06 13:09:11 Slp: ReplaceAccountWithSidInSddl -- SDDL to be returned: D:(A;OICI;KR;;;S-1-5-21-2383144575-3599344511-819193542-1074) 2012-07-06 13:09:11 Slp: Prompting user if they want to retry this action due to the following failure: 2012-07-06 13:09:11 Slp: ---------------------------------------- 2012-07-06 13:09:11 Slp: The following is an exception stack listing the exceptions in outermost to innermost order 2012-07-06 13:09:11 Slp: Inner exceptions are being indented 2012-07-06 13:09:11 Slp: 2012-07-06 13:09:11 Slp: Exception type: Microsoft.SqlServer.Configuration.Sco.ScoException 2012-07-06 13:09:11 Slp: Message: 2012-07-06 13:09:11 Slp: Attempted to perform an unauthorized operation. 2012-07-06 13:09:11 Slp: Data: 2012-07-06 13:09:11 Slp: WatsonData = 100 2012-07-06 13:09:11 Slp: DisableRetry = true 2012-07-06 13:09:11 Slp: Inner exception type: System.UnauthorizedAccessException 2012-07-06 13:09:11 Slp: Message: 2012-07-06 13:09:11 Slp: Attempted to perform an unauthorized operation. 2012-07-06 13:09:11 Slp: Stack: 2012-07-06 13:09:11 Slp: at System.Security.AccessControl.Win32.GetSecurityInfo(ResourceType resourceType, String name, SafeHandle handle, AccessControlSections accessControlSections, RawSecurityDescriptor& resultSd) 2012-07-06 13:09:11 Slp: at System.Security.AccessControl.NativeObjectSecurity.CreateInternal(ResourceType resourceType, Boolean isContainer, String name, SafeHandle handle, AccessControlSections includeSections, Boolean createByName, ExceptionFromErrorCode exceptionFromErrorCode, Object exceptionContext) 2012-07-06 13:09:11 Slp: at Microsoft.SqlServer.Configuration.Sco.SqlRegistrySecurity..ctor(ResourceType resourceType, SafeRegistryHandle handle, AccessControlSections includeSections) 2012-07-06 13:09:11 Slp: at Microsoft.SqlServer.Configuration.Sco.SqlRegistrySecurity.Create(InternalRegistryKey key) 2012-07-06 13:09:11 Slp: at Microsoft.SqlServer.Configuration.Sco.InternalRegistryKey.GetAccessControl() 2012-07-06 13:09:11 Slp: at Microsoft.SqlServer.Configuration.Sco.InternalRegistryKey.SetSecurityDescriptor(String sddl, Boolean overwrite) 2012-07-06 13:09:11 Slp: ---------------------------------------- 2012-07-06 13:09:24 Slp: User has chosen to retry this action 2012-07-06 13:09:24 Slp: Sco: Attempting to normalize security descriptor D:(A;OICI;KR;;;S-1-5-21-2383144575-3599344511-819193542-1074) 2012-07-06 13:09:24 Slp: Sco: Attempting to replace account with sid in security descriptor D:(A;OICI;KR;;;S-1-5-21-2383144575-3599344511-819193542-1074) 2012-07-06 13:09:24 Slp: ReplaceAccountWithSidInSddl -- SDDL to be processed: D:(A;OICI;KR;;;S-1-5-21-2383144575-3599344511-819193542-1074) 2012-07-06 13:09:24 Slp: ReplaceAccountWithSidInSddl -- SDDL to be returned: D:(A;OICI;KR;;;S-1-5-21-2383144575-3599344511-819193542-1074) 2012-07-06 13:09:24 Slp: Sco: Attempting to normalize security descriptor D:(A;OICI;KR;;;S-1-5-21-2383144575-3599344511-819193542-1074) 2012-07-06 13:09:24 Slp: Sco: Attempting to replace account with sid in security descriptor D:(A;OICI;KR;;;S-1-5-21-2383144575-3599344511-819193542-1074) 2012-07-06 13:09:24 Slp: ReplaceAccountWithSidInSddl -- SDDL to be processed: D:(A;OICI;KR;;;S-1-5-21-2383144575-3599344511-819193542-1074) 2012-07-06 13:09:24 Slp: ReplaceAccountWithSidInSddl -- SDDL to be returned: D:(A;OICI;KR;;;S-1-5-21-2383144575-3599344511-819193542-1074) 2012-07-06 13:09:24 Slp: Prompting user if they want to retry this action due to the following failure: 2012-07-06 13:09:24 Slp: ----------------------------------------

    Read the article

  • Source Control and SQL Development &ndash; Part 3

    - by Ajarn Mark Caldwell
    In parts one and two of this series, I have been specifically focusing on the latest version of SQL Source Control by Red Gate Software.  But I have been doing source-controlled SQL development for years, long before this product was available, and well before Microsoft came out with Database Projects for Visual Studio.  “So, how does that work?” you may wonder.  Well, let me share some of the details of how we do it where I work… The key to this approach is that everything is done via Transact-SQL script files; either natively written T-SQL, or generated.  My preference is to write all my code by hand, which forces you to become better at your SQL syntax.  But if you really prefer to use the Management Studio GUI to make database changes, you can still do that, and then you use the Generate Scripts feature of the GUI to produce T-SQL scripts afterwards, and store those in your source control system.  You can generate scripts for things like stored procedures and views by right-clicking on the database in the Object Explorer, and Choosing Tasks, Generate Scripts (see figure 1 to the left).  You can also do that for the CREATE scripts for tables, but that does not work when you have a table that is already in production, and you need to make just a simple change, such as adding a new column or index.  In this case, you can use the GUI to make the table changes, and then instead of clicking the Save button, click the Generate Change Script button (). Then, once you have saved the change script, go ahead and execute it on your development database to actually make the change.  I believe that it is important to actually execute the script rather than just click the Save button because this is your first test that your change script is working and you didn’t somehow lose a portion of the change. As you can imagine, all this generating of scripts can get tedious and tempting to skip entirely, so again, I would encourage you to just get in the habit of writing your own Transact-SQL code, and then it is just a matter of remembering to save your work, just like you are in the habit of saving changes to a Word or Excel document before you exit the program. So, now that you have all of these script files, what do you do with them?  Well, we organize ours into folders labeled ChangeScripts, Functions, Views, and StoredProcedures, and those folders are loaded into our source control system.  ChangeScripts contains all of the table and index changes, and anything else that is basically a one-time-only execution.  Of course you want to write your scripts with qualifying logic so that if a script were accidentally run more than once in a database, it would not crash nor corrupt anything; but these scripts are really intended to be run only once in a database. Once you have your initial set of scripts loaded into source control, then making changes, such as altering a stored procedure becomes a simple matter of checking out your CREATE PROCEDURE* script, editing it in SSMS, saving the change, executing the script in order to effect the change in your database, and then checking the script back in to source control.  Of course, this is where the lack of integration for source control systems within SSMS becomes an irritation, because this means that in addition to SSMS, I also have my source control client application running to do the check-out and check-in.  And when you have 800+ procedures like we do, that can be quite tedious to locate the procedure I want to change in source control, check it out, then locate the script file in my working folder, open it in SSMS, do the change, save it, and the go back to source control to check in.  Granted, it is not nearly as burdensome as, say, losing your source code and having to rebuild it from memory, or losing the audit trail that good source control systems provide.  It is worth the effort, and this is how I have been doing development for the last several years. Remember that everything that the SQL Server Management Studio does in modifying your database can also be done in plain Transact-SQL code, and this is what you are storing.  And now I have shown you how you can do it all without spending any extra money.  You already have source control, or can get free, open-source source control systems (almost seems like an oxymoron, doesn’t it) and of course Management Studio is free with your SQL Server database engine software. So, whether you spend the money on tools to make it easier, or not, you now have no excuse for not using source control with your SQL development. * In our current model, the scripts for stored procedures and similar database objects are written with an IF EXISTS…DROP… at the top, followed by the CREATE PROCEDURE… section, and that followed by a section that assigns permissions.  This allows me to run the same script regardless of whether the procedure previously existed in the database.  If the script was only an ALTER PROCEDURE, then it would fail the first time that procedure was deployed to a database, unless you wrote other code to stub it if it did not exist.  There are a few different ways you could organize your scripts for deployment, each with its own trade-offs, but I think it is absolutely critical that whichever way you organize things, you ensure that the same script is run throughout the deployment cycle, and do not allow customizations to creep in between TEST and PROD.  If you do, then you have broken the integrity of your deployment process because what you deployed to PROD was not exactly the same as what was tested in TEST, so you effectively have now released untested code into PROD.

    Read the article

< Previous Page | 187 188 189 190 191 192 193 194 195 196 197 198  | Next Page >