Search Results

Search found 130394 results on 5216 pages for 'run a sql script file in c'.

Page 87/5216 | < Previous Page | 83 84 85 86 87 88 89 90 91 92 93 94  | Next Page >

  • Why does Windows share permissions change file permissions?

    - by Andrew Rump
    When you create a (file system) share (on windows 2008R2) with access for specific users does it changes the access rights to the files to match the access rights to the share? We just killed our intranet web site when sharing the INetPub folder (to a few specified users). It removed the file access rights for authenticated users, i.e., the user could not log in using single signon (using IE & AD)! Could someone please tell me why it behaves like this? We now have to reapply the access right every time we change the users in the share killing the site in the process every time!

    Read the article

  • cannot unset env variables from script

    - by w00t
    Hi, I am trying to unset all environment variables from within a script. The script runs fine but if I run env it still shows all the variables set. If I run the command from CLI, it works and the variables are unset. unset `env | awk -F= '/^\w/ {print $1}' | xargs` Have any idea how to run this from a script? Also, have any idea how to source /etc/profile from a script? This doesn't work either. I need to set variables with same names but different paths, depending on the instances my users need.

    Read the article

  • awk + remove all not uniq lines except the first line uniq (FILE NAME)

    - by yael
    hi I have the following file How to remove by sed all FILE NAME lines except the first uniq FILE NAME For example need to remove all FILE NAME lines from the file except the first: FILE NAME: /dir1/dir2/dir3/dir4/dir5/file FILE NAME: /dirA/dirB/dirC/dirD/dirE/file the file: FILE NAME: /dir1/dir2/dir3/dir4/dir5/file PARAMETER NAME: blablabla TARGET FILE: 12 SOURCE FILE: 565 FILE NAME: /dir1/dir2/dir3/dir4/dir5/file PARAMETER NAME: blablabla TARGET FILE: 18 SOURCE FILE: 552 FILE NAME: /dir1/dir2/dir3/dir4/dir5/file PARAMETER NAME: blablabla TARGET FILE: 14 SOURCE FILE: 559 FILE NAME: /dirA/dirB/dirC/dirD/dirE/file PARAMETER NAME: blablabla TARGET FILE: 134 SOURCE FILE: 344 FILE NAME: /dirA/dirB/dirC/dirD/dirE/file PARAMETER NAME: blablabla TARGET FILE: 13 SOURCE FILE: 445 FILE NAME: /dirA/dirB/dirC/dirD/dirE/file PARAMETER NAME: blablabla TARGET FILE: 13 SOURCE FILE: 434

    Read the article

  • Server 2012 file server security - File access through software only?

    - by user1607914
    I have just setup a server 2012 essentials R2 box as a test bed for my small business. Its running Essentials R2 as a VM on a Essentials R2 host. I have set it up to be the DC and DHCP server and joined a client. My next task is to use it as a file server but I am not quite sure how to set it up. All the client interactions with the file server will be through client side software. I therefore do not want any access to the files apart from through the software. I have looked at the ntfs permissions and there is a permission called "List Folder/Read Data" however I am unsure this will do what I want. Is it possible to give access to the files but not allow access through the windows explorer / command line?

    Read the article

  • Server 2008 R2 file access permissions

    - by Napster100
    I'm finding it awkward to sort out permissions for file sharing and access on my LAN. I've created an account on the server node (as a normal user) and shared a drive that has 2 folders at the root, one is for personal file storage and the other shared files, if I connect to the shared area from a workstation running windows 7 and log-in using the account I created on the server, I can look through directories but can't look in some (which I wanted as I changed the permissions for that to happen), but my problem is although the permissions are set for this user account to have full control of the specific folder I can't create a folder in that area or upload files to that folder. Could someone explain why this is? Thanks in advanced

    Read the article

  • Script _name_ for this shop?

    - by Michael
    Those two web shops are using same script I guess. http://www.ipmart.com and http://www.gsm-traders.com anybody knows what kind of script they are using? Or at least recommend some script similar to this?

    Read the article

  • How to Collect Debug Info for Oracle SQL Developer

    - by thatjeffsmith
    In a perfect world, there would be no software bugs. Developers would always test their code. QA would find any scenarios and bugs the developers hadn’t already thought of. Regression tests would be complete and flawless. But alas, we can only afford to pay mere humans here, so we will have bugs from time to time. Or sometimes you are trying to do something the software wasn’t designed for, or perhaps your machine has exhausted it’s resources trying to build the un-buildable. When you run into problems, you will need help. Developers need your help so they can help you. Surprisingly enough, feedback like this isn’t very helpful: Your program isn’t working. How can I make it work? When you are ready to work with us on the SQL Developer OTN forum, you will most likely be asked to run SQL Developer and capture the output from the command console. In case you need help with this, ere’s a step-by-step process you can follow in Windows 7 (should work in XP too.) Open a windows command window Start – Run – CMD Once it’s open, click on the window icon and select ‘Defaults.’ Change the default buffer size to be something bigger, much bigger. Set the CMD window default buffer size HIGHER Note: you only need to do this once. Navigate to your SQL Developer Installation Folder Instead of running the ‘sqldeveloper.exe’ file in the root directory, we are going to go several sub-directories down. Find the ‘bin’ sub-directory and run the ‘sqldeveloper.exe’ there. When you do this, a CMD window will open, and then you’ll see the SQL Developer application load. The SQL Developer bin directory - run the tool from here and get a logging window Use SQL Developer as normal, until it ‘breaks’ or ‘hangs’ Now, you are ready to grab the nitty-gritty information that MIGHT tell the developer what is going wrong or happening in your scenario. Click back into the CMD window Send a Ctrl+Break or a Ctrl+Pause. If you on a newer laptop that doesn’t have this key, be sure to check the ‘Fn’ subset of keys. If you need to map the BREAK or PAUSE buttons, this article might help. You can also try the on-screen keyboard in windows – just type ‘OSK’ in your START – RUN prompt. Copy the logging information from the command window – all of it We need this information, help us get it! Open a case with Oracle Support or Start a Thread on the Forums Or email me. If you’re on my blog reading this, it’s the least I can do to help Now, before you hit ‘Send’ or ‘Post’ or ‘Submit’ – be sure to add a brief description of what you were doing in the application when you ran into the problem. Even if you were doing ‘nothing,’ let us know how many connections you had open, what windows were active, etc. The more you can tell us, the higher your odds go up to getting a quick fix or at least an answer as to what is happening. Also include the following information: The version of SQL Developer you are running The version of the JDK you are using The OS you are using The version of Oracle you are connected to Now, don’t be surprised if you get asked to upgrade to a supported configuration, say ‘version 3.1 and the 1.6 JDK.’ Supporting older versions of software is fun, and while we enjoy a challenge, it may be easier for you to upgrade your way out of the problem at hand.

    Read the article

  • SQL Server source control from Visual Studio

    - by David Atkinson
    Developers have long since had to context switch between two IDEs, Visual Studio for application code development and SQL Server Management Studio for database development. While this is accepted, especially given the richness of the database development feature set in SSMS, loading a separate tool can seem a little overkill. This is where SQL Connect comes in. This is an add-in to Visual Studio that provides a connected development experience for the SQL Server developer. Connected database development involves modifying a development sandbox database, as opposed to offline development, where SQL text files are modified independently of the database. One of the main complaints of Data Dude (VS DBPro) is that it enforces the offline approach. This gripe is what SQL Connect addresses. If you don't already use SQL Source Control, you can get up and running with SQL Connect by adding a new project to your Visual Studio solution as follows: Then choose your existing development database and you're ready to go. If you already use SQL Source Control, you will need to link SQL Connect to your existing database scripts folder repository, so SQL Connect and SQL Source Control can be used collaboratively (note that SQL Source Control v.3.0.9.18 or later is required). Locate the repository (this can be found in the Setup tab in SQL Source Control). .and create a working folder for it (here I'm using TortoiseSVN). Back in Visual Studio, locate the SQL Connect panel (in the View menu if it hasn't auto loaded) and select Import SQL Source Control project Locate your working folder and click Import. This creates a Red Gate database project under your solution: From here you can modify your development database, and manage your changes in source control. To associate your development database with the project, right click on the project node, select Properties, set the database and Save. Now you're ready to make some changes. Locate the object you'd like to modify in the Solution Explorer, and double click it to invoke a query window or table designer. You also have the option to edit the creation SQL directly using Edit SQL File in Project. Keeping the development database and Visual Studio project in sync is as easy as clicking on a button. One you've made your change, you can use whichever mechanism you choose to commit to source control. Here I'm using the free open-source AnkhSVN to integrate Subversion with Visual Studio. Maintaining your database in a Visual Studio solution means that you can commit database changes and application code changes in the same changeset. This is desirable if you have continuous integration set up as you want to ensure that all files related to a change are committed atomically, so you avoid an interim "broken build". More discussion on SQL Connect and its benefits can be found in the following article on Simple Talk: No More Disconnected SQL Development in Visual Studio The SQL Connect project team is currently assessing the backlog for the next development effort, and they'd appreciate your feature suggestions, as well as your votes on their suggestions site: http://redgate.uservoice.com/forums/140800-sql-connect-for-visual-studio- A 28-day free trial of SQL Connect is available from the Red Gate website. Technorati Tags: SQL Server

    Read the article

  • javax script how to call a function in JavaScript from Java

    - by Dan Howard
    Hi All, I'm trying to call a function in JavaScript via Java. This works fine when directly reading a script as a string but I'm using CompiledScripts. When I do this with a compiled script it gives me method not found if I also add bindings. Without bindings it works but of course the function fails because it needs the bindings. Any ideas? CompiledScript script = ... get script.... Bindings bindings = script.getEngine().createBindings(); Logger scriptLogger = LogManager.getLogger("TEST_SCRIPT"); bindings.put("log", scriptLogger); //script.eval(bindings); -- this way fails script.eval(); // -- this way works Invocable invocable = (Invocable) script.getEngine(); invocable.invokeFunction(methodName); TIA

    Read the article

  • close file with fclose() but file still in use

    - by Marco
    Hi all, I've got a problem with deleting/overwriting a file using my program which is also being used(read) by my program. The problem seems to be that because of the fact my program is reading data from the file (output.txt) it puts the file in a 'in use' state which makes it impossible to delete or overwrite the file. I don't understand why the file stays 'in use' because I close the file after use with fclose(); this is my code: bool bBool = true while(bBool){ //Run myprogram.exe tot generate (a new) output.txt //Create file pointer and open file FILE* pInputFile = NULL; pInputFile = fopen("output.txt", "r"); // //then I do some reading using fscanf() // //And when I'm done reading I close the file using fclose() fclose(pInputFile); //The next step is deleting the output.txt if( remove( "output.txt" ) == -1 ){ //ERROR }else{ //Succesfull } } I use fclose() to close the file but the file remains in use by my program until my program is totally shut down. What is the solution to free the file so it can be deleted/overwrited? In reality my code isn't a loop without an end ; ) Thanks in advance! Marco Update Like ask a part of my code which also generates the file 'in use'. This is not a loop and this function is being called from the main(); Here is a piece of code: int iShapeNr = 0; void firstRun() { //Run program that generates output.txt runProgram(); //Open Shape data file FILE* pInputFile = NULL; int iNumber = 0; pInputFile = fopen("output.txt", "r"); //Put all orientations of al detected shapes in an array int iShapeNr = 0; int iRotationBuffer[1024];//1024 is maximum detectable shapes, can be changed in RoboRealm int iXMinBuffer[1024]; int iXMaxBuffer[1024]; int iYMinBuffer[1024]; int iYMaxBuffer[1024]; while(feof(pInputFile) == 0){ for(int i=0;i<9;i++){ fscanf(pInputFile, "%d", &iNumber); fscanf(pInputFile, ","); if(i == 1) { iRotationBuffer[iShapeNr] = iNumber; } if(i == 3){//xmin iXMinBuffer[iShapeNr] = iNumber; } if(i == 4){//xmax iXMaxBuffer[iShapeNr] = iNumber; } if(i == 5){//ymin iYMinBuffer[iShapeNr] = iNumber; } if(i == 6){//ymax iYMaxBuffer[iShapeNr] = iNumber; } } iShapeNr++; } fflush(pInputFile); fclose(pInputFile); } The while loop parses the file. The output.txt contains sets of 9 variables, the number of sets is unknown but always in sets of 9. output.txt could contain for example: 0,1,2,3,4,5,6,7,8,8,7,6,5,4,1,2,3,0

    Read the article

  • script to dynamically fix ophaned users after db restore

    - by JJgates
    After performing a database restore, I want to run a dynamic script to fix ophaned users. My script below loops through all users that are displayed after executing sp_change_users_login 'report' and apply "alter user [username] with login = [username]" to fix SID conflicts verses static go statements. Although, I'm getting an "incorrect syntax error on line 15." can't figure out why... DECLARE @Username varchar(100), @cmd varchar(100) DECLARE userLogin_cursor CURSOR FAST_FORWARD FOR SELECT UserName = name FROM sysusers WHERE issqluser = 1 and (sid IS NOT NULL AND sid <> 0×0) AND suser_sname(sid) IS NULL ORDER BY name FOR READ ONLY OPEN userLogin_cursor FETCH NEXT FROM userLogin_cursor INTO @Username WHILE @@fetch_status = 0 BEGIN SET @cmd = ‘ALTER USER ‘+@username+‘ WITH LOGIN ‘+@username EXECUTE(@cmd) FETCH NEXT FROM userLogin_cursor INTO @Username END CLOSE userLogin_cursor DEALLOCATE userLogin_cursor

    Read the article

  • Auto Generated - CREATE Table Script in SQL 2008 throws error

    - by jack
    I scripted the tables in my dev database using SQL 2008 - generate scripts option (Datbase-right click-Tasks-Generate Scripts) and ran it on the staging database, but the script throws the below error for each table Msg 102, Level 15, State 1, Line 1 Incorrect syntax near '('. Msg 319, Level 15, State 1, Line 15 Incorrect syntax near the keyword 'with'. If this statement is a common table expression, an xmlnamespaces clause or a change tracking context clause, the previous statement must be terminated with a semicolon. Below is the script for one of my table ALTER TABLE [dbo].[Customer]( [CustomerID] [int] IDENTITY(1,1) NOT NULL, [FirstName] [nvarchar](500) NULL, [LastName] [nvarchar](500) NULL, [DateOfBirth] [datetime] NULL, [EmailID] [nvarchar](200) NULL, [ContactForOffers] [bit] NULL, CONSTRAINT [PK_Customer] PRIMARY KEY CLUSTERED ( [CustomerID] ASC )WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY] ) ON [PRIMARY] GO Any help will be much appreciated.

    Read the article

  • exim redirect mail to a script

    - by DevilWAH
    I am looking to set up a mail relay so that any mail sent to for example @example.com gets parsed be a PHP script. I know to do this for indervidual address I could do some thing like (from the following web site http://evolt.org/incoming_mail_and_php) Con figure an Aliases such as script: |/our/script.php then any emails sent to [email protected] will be passed to the script given.. but how can I make it that every email to @example.com is passed to the script? thank you

    Read the article

  • How to find missing alpha values in sets of data within same table in SQL

    - by Jeff
    I have a table of many values where one column has the WO Number, and another column has the Resource ID. I need to be able to find all the WO numbers that do not have a resource value of "RW". Here is an example of the typical information. I need to be able to know that work order 5678 does not have an "RW" Resource ID. WO Number - Resource ID 1234 - IN 1234 - WE 1234 - AS 1234 - RW 5678 - PR 5678 - WE 5678 - IN 5678 - AS

    Read the article

  • Come play in the SQL Server 2008 R2 Hosted Trial virtual lab!

    - by ssqa.net
    In continuation to SQL_Server_2008_R2 release date announcement you can access a complete, integrated Microsoft SQL Server 2008 R2, SharePoint 2010, and Office 2010 environment… right from your desktop. SQL Server 2008 R2 Hosted Trial Our Hosted Trial makes it easy for you to experience new features without any need for configuration or additional work. Register now to try out up to seven labs: SQL Server 2008 R2 – Multi Server Management SQL Server 2008 R2 – PowerPivot SQL Server 2008 R2 – Reporting...(read more)

    Read the article

  • Linq To SQL: Behaviour for table field which is NotNull and having Default value or binding

    - by kaushalparik27
    I found this something interesting while wandering over community which I would like to share. The post is whole about: DBML is not considering the table field's "Default value or Binding" setting which is a NotNull. I mean the field which can not be null but having default value set needs to be set IsDbGenerated = true in DBML file explicitly.Consider this situation: There is a simple tblEmployee table with below structure: The fields are simple. EmployeeID is a Primary Key with Identity Specification = True with Identity Seed = 1 to autogenerate numeric value for this field. EmployeeName and their EmailAddress to store in rest of 2 fields. And the last one is "DateAdded" with DateTime datatype which doesn't allow NULL but having Default Value/Binding with "GetDate()". That means if we don't pass any value to this field then SQL will insert current date in "DateAdded" field.So, I start with a new website, add a DBML file and dropped the said table to generate LINQ To SQL context class. Finally, I write a simple code snippet to insert data into the tblEmployee table; BUT, I am not passing any value to "DateAdded" field. Because I am considering SQL Server's "Default Value or Binding (GetDate())" setting to this field and understand that SQL will insert current date to this field.        using (TestDatabaseDataContext context = new TestDatabaseDataContext())        {            tblEmployee tblEmpObjet = new tblEmployee();            tblEmpObjet.EmployeeName = "KaushaL";            tblEmpObjet.EmployeeEmailAddress = "[email protected]";            context.tblEmployees.InsertOnSubmit(tblEmpObjet);            context.SubmitChanges();        }Here comes the twist when application give me below error:  This is something not expecting! From the error it clearly depicts that LINQ is passing NULL value to "DateAdded" Field while according to my understanding it should respect Sql Server's "Default value or Binding" setting for this field. A bit googling and I found very interesting related to this problem.When we set Primary Key to any field with "Identity Specification" Property set to true; DBML set one important property "IsDbGenerated=true" for this field. BUT, when we set "Default Value or Biding" property for some field; we need to explicitly tell the DBML/LINQ to let it know that this field is having default binding at DB side that needs to be respected if I don't pass any value. So, the solution is: You need to explicitly set "IsDbGenerated=true" for such field to tell the LINQ that the field is having default value or binding at Sql Server side so, please don't worry if i don't pass any value for it.You can select the field and set this property from property window in DBML Designer file or write the property in DBML.Designer.cs file directly. I have attached a working example with required table script with this post here. I hope this would be helpful for someone hunting for the same. Happy Discovery!

    Read the article

  • Keeping an Eye on Your Storage

    - by Fatherjack
    There are plenty of resources that advise you about looking for signs that your storage hardware is having problems. SQL Server Alerts for 823, 824 and 825 are covered here by Paul Randall of SQL Skills: http://www.sqlskills.com/blogs/paul/a-little-known-sign-of-impending-doom-error-825/ and here by me: https://www.simple-talk.com/blogs/2011/06/27/alerts-are-good-arent-they/. Now until very recently I wasn’t aware that there was a different way to track the 823 + 824 errors. It was by complete chance that I happened to be searching about in the msdb database when I found the suspect_pages table. Running a query against it I got zero rows. This, as it turns out is a good thing. Highlighting the table name and pressing F1 got me nowhere – Is it just me or does Books Online fail to load properly for no obvious reason sometimes? So I typed the table name into the search bar and got my local version of http://msdn.microsoft.com/en-us/library/ms174425.aspx. From that we get the following description: Contains one row per page that failed with a minor 823 error or an 824 error. Pages are listed in this table because they are suspected of being bad, but they might actually be fine. When a suspect page is repaired, its status is updated in the event_type column. So, in the table we would, on healthy hardware, expect to see zero rows but on disks that are having problems the event_type column would show us what is going on. Where there are suspect pages on the disk the rows would have an event_type value of 1, 2 or 3, where those suspect pages have been restored, repaired or deallocated by DBCC then the value would be 4, 5 or 7. Having this table means that we can set up SQL Monitor to check the status of our hardware as we can create a custom metric based on the query below: USE [msdb] go SELECT COUNT(*) FROM [dbo].[suspect_pages] AS sp All we need to do is set the metric to collect this value and set an alert to email when the value is not 1 and we are then able to let SQL Monitor take care of our storage. Note that the suspect_pages table does not have any updates concerning Error 825 which the links at the top of the page cover in more detail. I would suggest that you set SQL Monitor to alert on the suspect_pages table in addition to other taking other measures to look after your storage hardware and not have it as your only precaution. Microsoft actually pass ownership and administration of the suspect_pages table over to the database administrator (Manage the suspect_pages Table (SQL Server)) and in a surprising move (to me at least) advise DBAs to actively update and archive data in it. The table will only ever contain a maximum of 1000 rows and once full, new rows will not be added. Keeping an eye on this table is pretty important, although In my opinion, if you get to 1000 rows in this table and are not already waiting for new disks to be added to your server you are doing something wrong but if you have 1000 rows in there then you need to move data out quickly because you may be missing some important events on your server.

    Read the article

  • can't connect to Sql Sever Management Express 2012

    - by Rare-Man
    i installed Sql Sever Management Express 2012 , but when i try to connect in Sql management studio enviroment , i have this error . TITLE: Connect to Server Cannot connect to .. ADDITIONAL INFORMATION: A network-related or instance-specific error occurred while establishing a connection to SQL Server. The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote connections. (provider: Named Pipes Provider, error: 40 - Could not open a connection to SQL Server) (Microsoft SQL Server, Error: 2) For help, click: http://go.microsoft.com/fwlink?ProdName=Microsoft%20SQL%20Server&EvtSrc=MSSQLServer&EvtID=2&LinkId=20476 The system cannot find the file specified BUTTONS: OK ................................................................................... and in during installion i dont have option for select cluster !! this is my SQL Server Configuration Manager , my sql server service is empty ... And when get Remove a Failover Cluster Node , this error happened . http://oi57.tinypic.com/2lrvat.jpg

    Read the article

  • Lync using SQL 2008 R2 SP1 - Publish Topology error

    - by EKS
    Error that shows in the web page opened by the Topology builder: Error: An error occurred: "Microsoft.Rtc.Common.Data.SqlConnectionException" "A network-related or instance-specific error occurred while establishing a connection to SQL Server. The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote connections. (provider: Named Pipes Provider, error: 40 - Could not open a connection to SQL Server)" Looking in the log file: ( I assume this is the acutal error making it STOP) Installed SQL Server 2005 Backward Compatibility version is 8.05.1054 Error: SQL Server 2005 Backward Compatibility is not installed or its version is not high enough. SQL Server 2005 Backward Compatability SP2 or higher must be installed. I have installed SQLServer2005_BC_x64.msi from MS, and cant seem to find this SP2 version. SQL server is a 2008 R2 SP1. I have also tested with a 2008 SP3 SQL server same error. Named pippes output via ( OSQL /L) SQL-2008-1 SQL2

    Read the article

  • SQL connection is too slow

    - by user66905
    We have a business web application in ASP.NET + SQL Server 2008. In the beginning, SQL Server and IIS were on the same machine. Now we bought another machine. Current configuration is IIS machine plus SQL Server machine, and they are connected by a 1gb LAN connection. With this configuration our web application is slower than before. Max bandwidth is 1-2% of network, about 15mbps. When we use another threads to the same SQL Server from the same IIS machine, network use is higher. So this is no problem with SQL Server. Ho we can make higher bandwidth for this SQL connection? Specs: .Net 3.5 SQL Server 2008 Standard file transfer can use 100% of LAN SQL connection by TCP/IP protocol SQL logins Pool tested with enable and

    Read the article

  • The blocking nature of aggregates

    - by Rob Farley
    I wrote a post recently about how query tuning isn’t just about how quickly the query runs – that if you have something (such as SSIS) that is consuming your data (and probably introducing a bottleneck), then it might be more important to have a query which focuses on getting the first bit of data out. You can read that post here.  In particular, we looked at two operators that could be used to ensure that a query returns only Distinct rows. and The Sort operator pulls in all the data, sorts it (discarding duplicates), and then pushes out the remaining rows. The Hash Match operator performs a Hashing function on each row as it comes in, and then looks to see if it’s created a Hash it’s seen before. If not, it pushes the row out. The Sort method is quicker, but has to wait until it’s gathered all the data before it can do the sort, and therefore blocks the data flow. But that was my last post. This one’s a bit different. This post is going to look at how Aggregate functions work, which ties nicely into this month’s T-SQL Tuesday. I’ve frequently explained about the fact that DISTINCT and GROUP BY are essentially the same function, although DISTINCT is the poorer cousin because you have less control over it, and you can’t apply aggregate functions. Just like the operators used for Distinct, there are different flavours of Aggregate operators – coming in blocking and non-blocking varieties. The example I like to use to explain this is a pile of playing cards. If I’m handed a pile of cards and asked to count how many cards there are in each suit, it’s going to help if the cards are already ordered. Suppose I’m playing a game of Bridge, I can easily glance at my hand and count how many there are in each suit, because I keep the pile of cards in order. Moving from left to right, I could tell you I have four Hearts in my hand, even before I’ve got to the end. By telling you that I have four Hearts as soon as I know, I demonstrate the principle of a non-blocking operation. This is known as a Stream Aggregate operation. It requires input which is sorted by whichever columns the grouping is on, and it will release a row as soon as the group changes – when I encounter a Spade, I know I don’t have any more Hearts in my hand. Alternatively, if the pile of cards are not sorted, I won’t know how many Hearts I have until I’ve looked through all the cards. In fact, to count them, I basically need to put them into little piles, and when I’ve finished making all those piles, I can count how many there are in each. Because I don’t know any of the final numbers until I’ve seen all the cards, this is blocking. This performs the aggregate function using a Hash Match. Observant readers will remember this from my Distinct example. You might remember that my earlier Hash Match operation – used for Distinct Flow – wasn’t blocking. But this one is. They’re essentially doing a similar operation, applying a Hash function to some data and seeing if the set of values have been seen before, but before, it needs more information than the mere existence of a new set of values, it needs to consider how many of them there are. A lot is dependent here on whether the data coming out of the source is sorted or not, and this is largely determined by the indexes that are being used. If you look in the Properties of an Index Scan, you’ll be able to see whether the order of the data is required by the plan. A property called Ordered will demonstrate this. In this particular example, the second plan is significantly faster, but is dependent on having ordered data. In fact, if I force a Stream Aggregate on unordered data (which I’m doing by telling it to use a different index), a Sort operation is needed, which makes my plan a lot slower. This is all very straight-forward stuff, and information that most people are fully aware of. I’m sure you’ve all read my good friend Paul White (@sql_kiwi)’s post on how the Query Optimizer chooses which type of aggregate function to apply. But let’s take a look at SQL Server Integration Services. SSIS gives us a Aggregate transformation for use in Data Flow Tasks, but it’s described as Blocking. The definitive article on Performance Tuning SSIS uses Sort and Aggregate as examples of Blocking Transformations. I’ve just shown you that Aggregate operations used by the Query Optimizer are not always blocking, but that the SSIS Aggregate component is an example of a blocking transformation. But is it always the case? After all, there are plenty of SSIS Performance Tuning talks out there that describe the value of sorted data in Data Flow Tasks, describing the IsSorted property that can be set through the Advanced Editor of your Source component. And so I set about testing the Aggregate transformation in SSIS, to prove for sure whether providing Sorted data would let the Aggregate transform behave like a Stream Aggregate. (Of course, I knew the answer already, but it helps to be able to demonstrate these things). A query that will produce a million rows in order was in order. Let me rephrase. I used a query which produced the numbers from 1 to 1000000, in a single field, ordered. The IsSorted flag was set on the source output, with the only column as SortKey 1. Performing an Aggregate function over this (counting the number of rows per distinct number) should produce an additional column with 1 in it. If this were being done in T-SQL, the ordered data would allow a Stream Aggregate to be used. In fact, if the Query Optimizer saw that the field had a Unique Index on it, it would be able to skip the Aggregate function completely, and just insert the value 1. This is a shortcut I wouldn’t be expecting from SSIS, but certainly the Stream behaviour would be nice. Unfortunately, it’s not the case. As you can see from the screenshots above, the data is pouring into the Aggregate function, and not being released until all million rows have been seen. It’s not doing a Stream Aggregate at all. This is expected behaviour. (I put that in bold, because I want you to realise this.) An SSIS transformation is a piece of code that runs. It’s a physical operation. When you write T-SQL and ask for an aggregation to be done, it’s a logical operation. The physical operation is either a Stream Aggregate or a Hash Match. In SSIS, you’re telling the system that you want a generic Aggregation, that will have to work with whatever data is passed in. I’m not saying that it wouldn’t be possible to make a sometimes-blocking aggregation component in SSIS. A Custom Component could be created which could detect whether the SortKeys columns of the input matched the Grouping columns of the Aggregation, and either call the blocking code or the non-blocking code as appropriate. One day I’ll make one of those, and publish it on my blog. I’ve done it before with a Script Component, but as Script components are single-use, I was able to handle the data knowing everything about my data flow already. As per my previous post – there are a lot of aspects in which tuning SSIS and tuning execution plans use similar concepts. In both situations, it really helps to have a feel for what’s going on behind the scenes. Considering whether an operation is blocking or not is extremely relevant to performance, and that it’s not always obvious from the surface. In a future post, I’ll show the impact of blocking v non-blocking and synchronous v asynchronous components in SSIS, using some of LobsterPot’s Script Components and Custom Components as examples. When I get that sorted, I’ll make a Stream Aggregate component available for download.

    Read the article

  • The blocking nature of aggregates

    - by Rob Farley
    I wrote a post recently about how query tuning isn’t just about how quickly the query runs – that if you have something (such as SSIS) that is consuming your data (and probably introducing a bottleneck), then it might be more important to have a query which focuses on getting the first bit of data out. You can read that post here.  In particular, we looked at two operators that could be used to ensure that a query returns only Distinct rows. and The Sort operator pulls in all the data, sorts it (discarding duplicates), and then pushes out the remaining rows. The Hash Match operator performs a Hashing function on each row as it comes in, and then looks to see if it’s created a Hash it’s seen before. If not, it pushes the row out. The Sort method is quicker, but has to wait until it’s gathered all the data before it can do the sort, and therefore blocks the data flow. But that was my last post. This one’s a bit different. This post is going to look at how Aggregate functions work, which ties nicely into this month’s T-SQL Tuesday. I’ve frequently explained about the fact that DISTINCT and GROUP BY are essentially the same function, although DISTINCT is the poorer cousin because you have less control over it, and you can’t apply aggregate functions. Just like the operators used for Distinct, there are different flavours of Aggregate operators – coming in blocking and non-blocking varieties. The example I like to use to explain this is a pile of playing cards. If I’m handed a pile of cards and asked to count how many cards there are in each suit, it’s going to help if the cards are already ordered. Suppose I’m playing a game of Bridge, I can easily glance at my hand and count how many there are in each suit, because I keep the pile of cards in order. Moving from left to right, I could tell you I have four Hearts in my hand, even before I’ve got to the end. By telling you that I have four Hearts as soon as I know, I demonstrate the principle of a non-blocking operation. This is known as a Stream Aggregate operation. It requires input which is sorted by whichever columns the grouping is on, and it will release a row as soon as the group changes – when I encounter a Spade, I know I don’t have any more Hearts in my hand. Alternatively, if the pile of cards are not sorted, I won’t know how many Hearts I have until I’ve looked through all the cards. In fact, to count them, I basically need to put them into little piles, and when I’ve finished making all those piles, I can count how many there are in each. Because I don’t know any of the final numbers until I’ve seen all the cards, this is blocking. This performs the aggregate function using a Hash Match. Observant readers will remember this from my Distinct example. You might remember that my earlier Hash Match operation – used for Distinct Flow – wasn’t blocking. But this one is. They’re essentially doing a similar operation, applying a Hash function to some data and seeing if the set of values have been seen before, but before, it needs more information than the mere existence of a new set of values, it needs to consider how many of them there are. A lot is dependent here on whether the data coming out of the source is sorted or not, and this is largely determined by the indexes that are being used. If you look in the Properties of an Index Scan, you’ll be able to see whether the order of the data is required by the plan. A property called Ordered will demonstrate this. In this particular example, the second plan is significantly faster, but is dependent on having ordered data. In fact, if I force a Stream Aggregate on unordered data (which I’m doing by telling it to use a different index), a Sort operation is needed, which makes my plan a lot slower. This is all very straight-forward stuff, and information that most people are fully aware of. I’m sure you’ve all read my good friend Paul White (@sql_kiwi)’s post on how the Query Optimizer chooses which type of aggregate function to apply. But let’s take a look at SQL Server Integration Services. SSIS gives us a Aggregate transformation for use in Data Flow Tasks, but it’s described as Blocking. The definitive article on Performance Tuning SSIS uses Sort and Aggregate as examples of Blocking Transformations. I’ve just shown you that Aggregate operations used by the Query Optimizer are not always blocking, but that the SSIS Aggregate component is an example of a blocking transformation. But is it always the case? After all, there are plenty of SSIS Performance Tuning talks out there that describe the value of sorted data in Data Flow Tasks, describing the IsSorted property that can be set through the Advanced Editor of your Source component. And so I set about testing the Aggregate transformation in SSIS, to prove for sure whether providing Sorted data would let the Aggregate transform behave like a Stream Aggregate. (Of course, I knew the answer already, but it helps to be able to demonstrate these things). A query that will produce a million rows in order was in order. Let me rephrase. I used a query which produced the numbers from 1 to 1000000, in a single field, ordered. The IsSorted flag was set on the source output, with the only column as SortKey 1. Performing an Aggregate function over this (counting the number of rows per distinct number) should produce an additional column with 1 in it. If this were being done in T-SQL, the ordered data would allow a Stream Aggregate to be used. In fact, if the Query Optimizer saw that the field had a Unique Index on it, it would be able to skip the Aggregate function completely, and just insert the value 1. This is a shortcut I wouldn’t be expecting from SSIS, but certainly the Stream behaviour would be nice. Unfortunately, it’s not the case. As you can see from the screenshots above, the data is pouring into the Aggregate function, and not being released until all million rows have been seen. It’s not doing a Stream Aggregate at all. This is expected behaviour. (I put that in bold, because I want you to realise this.) An SSIS transformation is a piece of code that runs. It’s a physical operation. When you write T-SQL and ask for an aggregation to be done, it’s a logical operation. The physical operation is either a Stream Aggregate or a Hash Match. In SSIS, you’re telling the system that you want a generic Aggregation, that will have to work with whatever data is passed in. I’m not saying that it wouldn’t be possible to make a sometimes-blocking aggregation component in SSIS. A Custom Component could be created which could detect whether the SortKeys columns of the input matched the Grouping columns of the Aggregation, and either call the blocking code or the non-blocking code as appropriate. One day I’ll make one of those, and publish it on my blog. I’ve done it before with a Script Component, but as Script components are single-use, I was able to handle the data knowing everything about my data flow already. As per my previous post – there are a lot of aspects in which tuning SSIS and tuning execution plans use similar concepts. In both situations, it really helps to have a feel for what’s going on behind the scenes. Considering whether an operation is blocking or not is extremely relevant to performance, and that it’s not always obvious from the surface. In a future post, I’ll show the impact of blocking v non-blocking and synchronous v asynchronous components in SSIS, using some of LobsterPot’s Script Components and Custom Components as examples. When I get that sorted, I’ll make a Stream Aggregate component available for download.

    Read the article

  • Be the surgeon

    - by Rob Farley
    It’s a phrase I use often, especially when teaching, and I wish I had realised the concept years earlier. (And of course, fits with this month’s T-SQL Tuesday topic, hosted by Argenis Fernandez) When I’m sick enough to go to the doctor, I see a GP. I used to typically see the same guy, but he’s moved on now. However, when he has been able to roughly identify the area of the problem, I get referred to a specialist, sometimes a surgeon. Being a surgeon requires a refined set of skills. It’s why they often don’t like to be called “Doctor”, and prefer the traditional “Mister” (the history is that the doctor used to make the diagnosis, and then hand the patient over to the person who didn’t have a doctorate, but rather was an expert cutter, typically from a background in butchering). But if you ask the surgeon about the pain you have in your leg sometimes, you’ll get told to ask your GP. It’s not that your surgeon isn’t interested – they just don’t know the answer. IT is the same now. That wasn’t something that I really understood when I got out of university. I knew there was a lot to know about IT – I’d just done an honours degree in it. But I also knew that I’d done well in just about all my subjects, and felt like I had a handle on everything. I got into developing, and still felt that having a good level of understanding about every aspect of IT was a good thing. This got me through for the first six or seven years of my career. But then I started to realise that I couldn’t compete. I’d moved into management, and was spending my days running projects, rather than writing code. The kids were getting older. I’d had a bad back injury (ask anyone with chronic pain how it affects  your ability to concentrate, retain information, etc). But most of all, IT was getting larger. I knew kids without lives who knew more than I did. And I felt like I could easily identify people who were better than me in whatever area I could think of. Except writing queries (this was before I discovered technical communities, and people like Paul White and Dave Ballantyne). And so I figured I’d specialise. I wish I’d done it years earlier. Now, I can tell you plenty of people who are better than me at any area you can pick. But there are also more people who might consider listing me in some of their lists too. If I’d stayed the GP, I’d be stuck in management, and finding that there were better managers than me too. If you’re reading this, SQL could well be your thing. But it might not be either. Your thing might not even be in IT. Find out, and then see if you can be a world-beater at it. But it gets even better, because you can find other people to complement the things that you’re not so good at. My company, LobsterPot Solutions, has six people in it at the moment. I’ve hand-picked those six people, along with the one who quit. The great thing about it is that I’ve been able to pick people who don’t necessarily specialise in the same way as me. I don’t write their T-SQL for them – generally they’re good enough at that themselves. But I’m on-hand if needed. Consider Roger Noble, for example. He’s doing stuff in HTML5 and jQuery that I could never dream of doing to create an amazing HTML5 version of PivotViewer. Or Ashley Sewell, a guy who does project management far better than I do. I could go on. My team is brilliant, and I love them to bits. We’re all surgeons, and when we work together, I like to think we’re pretty good! @rob_farley

    Read the article

  • Be the surgeon

    - by Rob Farley
    It’s a phrase I use often, especially when teaching, and I wish I had realised the concept years earlier. (And of course, fits with this month’s T-SQL Tuesday topic, hosted by Argenis Fernandez) When I’m sick enough to go to the doctor, I see a GP. I used to typically see the same guy, but he’s moved on now. However, when he has been able to roughly identify the area of the problem, I get referred to a specialist, sometimes a surgeon. Being a surgeon requires a refined set of skills. It’s why they often don’t like to be called “Doctor”, and prefer the traditional “Mister” (the history is that the doctor used to make the diagnosis, and then hand the patient over to the person who didn’t have a doctorate, but rather was an expert cutter, typically from a background in butchering). But if you ask the surgeon about the pain you have in your leg sometimes, you’ll get told to ask your GP. It’s not that your surgeon isn’t interested – they just don’t know the answer. IT is the same now. That wasn’t something that I really understood when I got out of university. I knew there was a lot to know about IT – I’d just done an honours degree in it. But I also knew that I’d done well in just about all my subjects, and felt like I had a handle on everything. I got into developing, and still felt that having a good level of understanding about every aspect of IT was a good thing. This got me through for the first six or seven years of my career. But then I started to realise that I couldn’t compete. I’d moved into management, and was spending my days running projects, rather than writing code. The kids were getting older. I’d had a bad back injury (ask anyone with chronic pain how it affects  your ability to concentrate, retain information, etc). But most of all, IT was getting larger. I knew kids without lives who knew more than I did. And I felt like I could easily identify people who were better than me in whatever area I could think of. Except writing queries (this was before I discovered technical communities, and people like Paul White and Dave Ballantyne). And so I figured I’d specialise. I wish I’d done it years earlier. Now, I can tell you plenty of people who are better than me at any area you can pick. But there are also more people who might consider listing me in some of their lists too. If I’d stayed the GP, I’d be stuck in management, and finding that there were better managers than me too. If you’re reading this, SQL could well be your thing. But it might not be either. Your thing might not even be in IT. Find out, and then see if you can be a world-beater at it. But it gets even better, because you can find other people to complement the things that you’re not so good at. My company, LobsterPot Solutions, has six people in it at the moment. I’ve hand-picked those six people, along with the one who quit. The great thing about it is that I’ve been able to pick people who don’t necessarily specialise in the same way as me. I don’t write their T-SQL for them – generally they’re good enough at that themselves. But I’m on-hand if needed. Consider Roger Noble, for example. He’s doing stuff in HTML5 and jQuery that I could never dream of doing to create an amazing HTML5 version of PivotViewer. Or Ashley Sewell, a guy who does project management far better than I do. I could go on. My team is brilliant, and I love them to bits. We’re all surgeons, and when we work together, I like to think we’re pretty good! @rob_farley

    Read the article

< Previous Page | 83 84 85 86 87 88 89 90 91 92 93 94  | Next Page >