Search Results

Search found 2972 results on 119 pages for 'paul jan'.

Page 10/119 | < Previous Page | 6 7 8 9 10 11 12 13 14 15 16 17  | Next Page >

• insert/update/delete with xml in .net 3.5

  - by Radhi

  Hello guys, i have "UserProfile" in my site which we have stored in xml fromat of xml is as below <UserProfile xmlns=""> <BasicInfo> <Title value="Basic Details" /> <Fields> <UserId title="UserId" display="No" right="Public" value="12" /> <EmailAddress title="Email Address" display="Yes" right="Public" value="[email protected]" /> <FirstName title="First Name" display="Yes" right="Public" value="Radhi" /> <LastName title="Last Name" display="Yes" right="Public" value="Patel" /> <DisplayName title="Display Name" display="Yes" right="Public" value="Radhi Patel" /> <RegistrationStatusId title="RegistrationStatusId" display="No" right="Public" value="10" /> <RegistrationStatus title="Registration Status" display="Yes" right="Private" value="Registered" /> <CountryName title="Country" display="Yes" right="Public" value="India" /> <StateName title="State" display="Yes" right="Public" value="Maharashtra" /> <CityName title="City" display="Yes" right="Public" value="Mumbai" /> <Gender title="Gender" display="Yes" right="Public" value="FeMale" /> <CreatedBy title="CreatedBy" display="No" right="Public" value="0" /> <CreatedOn title="CreatedOn" display="No" right="Public" value="Nov 27 2009 3:08PM " /> <ModifiedBy title="ModifiedBy" display="No" right="Public" value="12" /> <ModifiedOn title="ModifiedOn" display="No" right="Public" value="Feb 18 2010 1:43PM " /> <LogInStatusId title="LogInStatusId" display="No" right="Public" value="1" /> <LogInStatus title="LogIn Status" display="Yes" right="Private" value="Free" /> <ProfileImagePath title="Profile Pic" display="No" right="Public" value="~/Images/96.jpg" /> </Fields> </BasicInfo> <PersonalInfo> <Title value="Personal Details" /> <Fields> <Nickname title="Nick Name" display="Yes" right="Public" value="Rahul" /> <NativeLocation title="Native" display="Yes" right="Public" value="Amreli" /> <DateofAnniversary title="Anniversary Dt." display="Yes" right="Public" value="12/29/2008" /> <BloodGroupId title="BloodGroupId" display="No" right="Public" value="25" /> <BloodGroupName title="Blood Group" display="Yes" right="Public" value="25" /> <MaritalStatusId title="MaritalStatusId" display="No" right="Public" value="34" /> <MaritalStatusName title="Marital status" display="Yes" right="Private" value="34" /> <DateofDeath title="Death dt" display="Yes" right="Private" value="" /> <CreatedBy title="CreatedBy" display="No" right="Public" value="12" /> <CreatedOn title="CreatedOn" display="No" right="Public" value="Jan 6 2010 2:59PM " /> <ModifiedBy title="ModifiedBy" display="No" right="Public" value="12" /> <ModifiedOn title="ModifiedOn" display="No" right="Public" value="3/10/2010 5:34:14 PM" /> </Fields> </PersonalInfo> <FamilyInfo> <Title value="Family Details" /> <Fields> <GallantryHistory title="Gallantry History" display="Yes" right="Public" value="Gallantry history rahul" /> <Ethinicity title="Ethinicity" display="Yes" right="Public" value="ethnicity rahul" /> <KulDev title="KulDev" display="Yes" right="Public" value="Krishna" /> <KulDevi title="KulDevi" display="Yes" right="Public" value="Khodiyar" /> <Caste title="Caste" display="Yes" right="Private" value="Brhamin" /> <SunSignId title="SunSignId" display="No" right="Public" value="20" /> <SunSignName title="SunSignName" display="Yes" right="Public" value="Scorpio" /> <CreatedBy title="CreatedBy" display="No" right="Public" value="12" /> <CreatedOn title="CreatedOn" display="No" right="Public" value="Dec 29 2009 4:59PM " /> <ModifiedBy title="ModifiedBy" display="No" right="Public" value="12" /> <ModifiedOn title="ModifiedOn" display="No" right="Public" value="3/10/2010 6:29:56 PM" /> </Fields> </FamilyInfo> <HobbyInfo> <Title value="Hobbies/Interests" /> <Fields> <AbountMe title="Abount Me" display="Yes" right="Public" value="Naughty.... " /> <Hobbies title="Hobbies" display="Yes" right="Public" value="Dance, Music, decoration, Shopping" /> <Food title="Food" display="Yes" right="Public" value="Maxican salsa, Pizza, Khoya kaju " /> <Movies title="Movies" display="Yes" right="Public" value="day after tommorrow. wake up sid. avatar" /> <Music title="Music" display="Yes" right="Public" value="Chu kar mere man ko... wake up sid songs, slow music, apgk songs" /> <TVShows title="TV Shows" display="Yes" right="Public" value="business bazzigar, hanah montana" /> <Books title="Books" display="Yes" right="Public" value="mystry novels" /> <Sports title="Sports" display="Yes" right="Public" value="Badminton" /> <Will title="Will" display="Yes" right="Public" value="do photography, to have my own super home... and i can decorate it like anything..." /> <FavouriteQuotes title="Favourite Quotes" display="Yes" right="Public" value="Live like a king nothing lasts forever. not even your troubles smooth sea do not makes skillfull sailors" /> <CremationPrefernces title="Cremation Prefernces" display="Yes" right="Public" value="" /> <CreatedBy title="CreatedBy" display="No" right="Public" value="12" /> <CreatedOn title="CreatedOn" display="No" right="Public" value="Feb 24 2010 2:13PM " /> <ModifiedBy title="ModifiedBy" display="No" right="Public" value="12" /> <ModifiedOn title="ModifiedOn" display="No" right="Public" value="Mar 2 2010 4:34PM " /> </Fields> </HobbyInfo> <PermenantAddr> <Title value="Permenant Address" /> <Fields> <Address title="Address" display="Yes" right="Public" value="Test Entry" /> <CityId title="CityId" display="No" right="Public" value="93" /> <CityName title="City" display="Yes" right="Public" value="Chennai" /> <StateId title="StateId" display="No" right="Public" value="89" /> <StateName title="State" display="Yes" right="Public" value="Tamil Nadu" /> <CountryId title="CountryId" display="No" right="Public" value="108" /> <CountryName title="Country" display="Yes" right="Public" value="India" /> <ZipCode title="ZipCode" display="Yes" right="Private" value="360019" /> <CreatedBy title="CreatedBy" display="No" right="Public" value="12" /> <CreatedOn title="CreatedOn" display="No" right="Public" value="Jan 6 2010 1:29PM " /> <ModifiedBy title="ModifiedBy" display="No" right="Public" value="0" /> <ModifiedOn title="ModifiedOn" display="No" right="Public" value=" " /> </Fields> </PermenantAddr> <PresentAddr> <Title value="Present Address" /> <Fields> <Address title="Ethinicity" display="Yes" right="Public" value="" /> <CityId title="CityId" display="No" right="Public" value="" /> <CityName title="City" display="Yes" right="Public" value="" /> <StateId title="StateId" display="No" right="Public" value="" /> <StateName title="State" display="Yes" right="Public" value="" /> <CountryId title="CountryId" display="No" right="Public" value="" /> <CountryName title="Country" display="Yes" right="Public" value="" /> <ZipCode title="ZipCode" display="Yes" right="Public" value="" /> <CreatedBy title="CreatedBy" display="No" right="Public" value="" /> <CreatedOn title="CreatedOn" display="No" right="Public" value="" /> <ModifiedBy title="ModifiedBy" display="No" right="Public" value="" /> <ModifiedOn title="ModifiedOn" display="No" right="Public" value="" /> </Fields> </PresentAddr> <ContactInfo> <Title value="Contact Details" /> <Fields> <DayPhoneNo title="Day Phone" display="Yes" right="Public" value="" /> <NightPhoneNo title="Night Phone" display="Yes" right="Public" value="" /> <MobileNo title="Mobile No" display="Yes" right="Private" value="" /> <FaxNo title="Fax No" display="Yes" right="CUG" value="" /> <CreatedBy title="CreatedBy" display="No" right="Public" value="12" /> <CreatedOn title="CreatedOn" display="No" right="Public" value="Jan 5 2010 12:37PM " /> <ModifiedBy title="ModifiedBy" display="No" right="Public" value="12" /> <ModifiedOn title="ModifiedOn" display="No" right="Public" value="Feb 17 2010 1:37PM " /> </Fields> </ContactInfo> <EmailInfo> <Title value="Alternate Email Addresses" /> <Fields /> </EmailInfo> <AcademicInfo> <Title value="Education Details" /> <Fields> <Record right="Public"> <Education title="Education" display="Yes" value="Full Attendance" /> <Institute title="Institute" display="Yes" value="Attendance" /> <PassingYear title="Passing Year" display="Yes" value="2000" /> <IsActive title="IsActive" display="No" value="false" /> <CreatedBy title="CreatedBy" display="No" value="12" /> <CreatedOn title="CreatedOn" display="No" value="Dec 31 2009 12:41PM " /> <ModifiedBy title="ModifiedBy" display="No" value="12" /> <ModifiedOn title="ModifiedOn" display="No" value="Dec 31 2009 12:41PM " /> </Record> <Record right="Public"> <Education title="Education" display="Yes" value="D.C.E." /> <Institute title="Institute" display="Yes" value="G.P.G" /> <PassingYear title="Passing Year" display="Yes" value="2005" /> <IsActive title="IsActive" display="No" value="true" /> <CreatedBy title="CreatedBy" display="No" value="12" /> <CreatedOn title="CreatedOn" display="No" value="Dec 31 2009 12:45PM " /> <ModifiedBy title="ModifiedBy" display="No" value="12" /> <ModifiedOn title="ModifiedOn" display="No" value="Dec 31 2009 12:45PM " /> </Record> <Record right="Public"> <Education title="Education" display="Yes" value="MCSE" /> <Institute title="Institute" display="Yes" value="MCSE" /> <PassingYear title="Passing Year" display="Yes" value="2009" /> <IsActive title="IsActive" display="No" value="true" /> <CreatedBy title="CreatedBy" display="No" value="12" /> <CreatedOn title="CreatedOn" display="No" value="Dec 31 2009 6:12PM " /> <ModifiedBy title="ModifiedBy" display="No" value="12" /> <ModifiedOn title="ModifiedOn" display="No" value="Mar 2 2010 4:33PM " /> </Record> <Record right="Public"> <Education title="Education" display="Yes" value="H.S.C." /> <Institute title="Institute" display="Yes" value="G.H.S.E.B." /> <PassingYear title="Passing Year" display="Yes" value="2002" /> <IsActive title="IsActive" display="No" value="true" /> <CreatedBy title="CreatedBy" display="No" value="12" /> <CreatedOn title="CreatedOn" display="No" value="Dec 31 2009 6:17PM " /> <ModifiedBy title="ModifiedBy" display="No" value="12" /> <ModifiedOn title="ModifiedOn" display="No" value="Dec 31 2009 6:17PM " /> </Record> <Record right="Public"> <Education title="Education" display="Yes" value="S.S.C." /> <Institute title="Institute" display="Yes" value="G.S.E.B." /> <PassingYear title="Passing Year" display="Yes" value="2000" /> <IsActive title="IsActive" display="No" value="true" /> <CreatedBy title="CreatedBy" display="No" value="12" /> <CreatedOn title="CreatedOn" display="No" value="Dec 31 2009 6:17PM " /> <ModifiedBy title="ModifiedBy" display="No" value="12" /> <ModifiedOn title="ModifiedOn" display="No" value="Dec 31 2009 6:17PM " /> </Record> </Fields> </AcademicInfo> <AchievementInfo> <Title value="Achievement Details" /> <Fields> <Record right="Public"> <Awards title="Award" display="Yes" value="Test Entry" /> <FieldOfAward title="Field Of Award" display="Yes" value="Test Entry" /> <Tournament title="Tournament" display="Yes" value="Test Entry" /> <AwardDescription title="Description" display="Yes" value="Test Entry" /> <AwardYear title="Award Year" display="Yes" value="2002" /> <IsActive title="IsActive" display="No" value="true" /> <CreatedBy title="CreatedBy" display="No" value="12" /> <CreatedOn title="CreatedOn" display="No" value="Dec 31 2009 3:51PM " /> <ModifiedBy title="ModifiedBy" display="No" value="12" /> <ModifiedOn title="ModifiedOn" display="No" value="Dec 31 2009 3:51PM " /> </Record> <Record right="Public"> <Awards title="Award" display="Yes" value="Test Entry" /> <FieldOfAward title="Field Of Award" display="Yes" value="Test Entry" /> <Tournament title="Tournament" display="Yes" value="Test Entry" /> <AwardDescription title="Description" display="Yes" value="Test Entry" /> <AwardYear title="Award Year" display="Yes" value="2005" /> <IsActive title="IsActive" display="No" value="true" /> <CreatedBy title="CreatedBy" display="No" value="12" /> <CreatedOn title="CreatedOn" display="No" value="Jan 8 2010 10:19AM " /> <ModifiedBy title="ModifiedBy" display="No" value="12" /> <ModifiedOn title="ModifiedOn" display="No" value="Jan 8 2010 10:19AM " /> </Record> <Record right="Public"> <Awards title="Award" display="Yes" value="Test Entry3" /> <FieldOfAward title="Field Of Award" display="Yes" value="Test Entry3" /> <Tournament title="Tournament" display="Yes" value="Test Entry3" /> <AwardDescription title="Description" display="Yes" value="Test Entry3" /> <AwardYear title="Award Year" display="Yes" value="2007" /> <IsActive title="IsActive" display="No" value="true" /> <CreatedBy title="CreatedBy" display="No" value="12" /> <CreatedOn title="CreatedOn" display="No" value="Dec 31 2009 11:47AM " /> <ModifiedBy title="ModifiedBy" display="No" value="12" /> <ModifiedOn title="ModifiedOn" display="No" value="Dec 31 2009 11:47AM " /> </Record> <Record right="Public"> <Awards title="Award" display="Yes" value="Test Entry4" /> <FieldOfAward title="Field Of Award" display="Yes" value="Test Entry4" /> <Tournament title="Tournament" display="Yes" value="Test Entry4" /> <AwardDescription title="Description" display="Yes" value="Test Entry3" /> <AwardYear title="Award Year" display="Yes" value="2000" /> <IsActive title="IsActive" display="No" value="false" /> <CreatedBy title="CreatedBy" display="No" value="12" /> <CreatedOn title="CreatedOn" display="No" value="Dec 31 2009 11:47AM " /> <ModifiedBy title="ModifiedBy" display="No" value="12" /> <ModifiedOn title="ModifiedOn" display="No" value="Dec 31 2009 11:47AM " /> </Record> </Fields> </AchievementInfo> <ProfessionalInfo> <Title value="Professional Details" /> <Fields> <Record right="Public"> <Occupation title="Occupation" display="Yes" value="Test Entry" /> <Organization title="Organization" display="Yes" value="Test Entry" /> <ProjectsDescription title="Description" display="Yes" value="Test Entry" /> <Duration title="Duration" display="Yes" value="26" /> <IsActive title="IsActive" display="No" value="false" /> <CreatedBy title="CreatedBy" display="No" value="12" /> <CreatedOn title="CreatedOn" display="No" value="Jan 4 2010 3:01PM " /> <ModifiedBy title="ModifiedBy" display="No" value="12" /> <ModifiedOn title="ModifiedOn" display="No" value="Jan 4 2010 3:01PM " /> </Record> <Record right="Public"> <Occupation title="Occupation" display="Yes" value="Test Entry" /> <Organization title="Organization" display="Yes" value="Test Entry" /> <ProjectsDescription title="Description" display="Yes" value="Test Entry" /> <Duration title="Duration" display="Yes" value="10" /> <IsActive title="IsActive" display="No" value="false" /> <CreatedBy title="CreatedBy" display="No" value="12" /> <CreatedOn title="CreatedOn" display="No" value="Jan 4 2010 3:01PM " /> <ModifiedBy title="ModifiedBy" display="No" value="12" /> <ModifiedOn title="ModifiedOn" display="No" value="Jan 4 2010 3:01PM " /> </Record> <Record right="Public"> <Occupation title="Occupation" display="Yes" value="Test Entry" /> <Organization title="Organization" display="Yes" value="Test Entry" /> <ProjectsDescription title="Description" display="Yes" value="Test Entry" /> <Duration title="Duration" display="Yes" value="15" /> <IsActive title="IsActive" display="No" value="false" /> <CreatedBy title="CreatedBy" display="No" value="12" /> <CreatedOn title="CreatedOn" display="No" value="Jan 4 2010 3:01PM " /> <ModifiedBy title="ModifiedBy" display="No" value="12" /> <ModifiedOn title="ModifiedOn" display="No" value="Jan 4 2010 3:01PM " /> </Record> </Fields> </ProfessionalInfo> </UserProfile> now for tags like PersonalInfo,contactInfo,Address there will be only one record, but for tags like "OtherInfo","academicInfo","ProfessionalInfo" there will be multiple records so in xml there are tags for that. now to edit tags having one record only for one user i did coding like: private void FillFamilyInfoControls(XElement rootElement) { ddlSunsign.DataBind(); XElement parentElement; string xPathQuery = "FamilyInfo/Fields"; parentElement = rootElement.XPathSelectElement(xPathQuery); txtGallantryHistory.Text = parentElement.Element("GallantryHistory").Attribute("value").Value; txtEthinicity.Text = parentElement.Element("Ethinicity").Attribute("value").Value; txtKulDev.Text = parentElement.Element("KulDev").Attribute("value").Value; txtKulDevi.Text = parentElement.Element("KulDevi").Attribute("value").Value; txtCaste.Text = parentElement.Element("Caste").Attribute("value").Value; ddlSunsign.SelectedValue = parentElement.Element("SunSignId").Attribute("value").Value; ddlSunsign.SelectedItem.Text = parentElement.Element("SunSignName").Attribute("value").Value; } private XElement UpdateFamilyInfoXML(XElement rootElement) { XElement parentElement; string xPathQuery = "FamilyInfo/Fields"; parentElement = rootElement.XPathSelectElement(xPathQuery); parentElement.Element("GallantryHistory").Attribute("value").Value = txtGallantryHistory.Text; parentElement.Element("Ethinicity").Attribute("value").Value = txtEthinicity.Text; parentElement.Element("KulDev").Attribute("value").Value = txtKulDev.Text; parentElement.Element("KulDevi").Attribute("value").Value = txtKulDevi.Text; parentElement.Element("Caste").Attribute("value").Value = txtCaste.Text; parentElement.Element("SunSignId").Attribute("value").Value = ddlSunsign.SelectedItem.Value; parentElement.Element("SunSignName").Attribute("value").Value = ddlSunsign.SelectedItem.Text; parentElement.Element("ModifiedBy").Attribute("value").Value = UMSession.CurrentLoggedInUser.UserId.ToString(); parentElement.Element("ModifiedOn").Attribute("value").Value = System.DateTime.Now.ToString(); return rootElement; //rootElement.XPathSelectElement(xPathQuery) = parentElement; } these 2 functions i have used to update xml and to get data from xml and fill into controls. but for the tags where there are multiple records... i am not able to find any solution /control using which i an do it easily. coz my field's value is in attribute named "Value" when in the examples i saw its between opening and closing tag.. i have 2 methods to do so. 1 . i make one page to edit a single record and pass id of record to that page on editing. open the page in iframe on same page to edit and in page_load get data from database for id passed and fill it in control. 2 . i store xml on server as physical file and use it in page i opened in iframe to edit the record. so i am confused... can anybody please guide me that what shpould i do to let user provide interface to edit this xml data/user profile ?

  Read the article

• Cardinality Estimation Bug with Lookups in SQL Server 2008 onward

  - by Paul White

  Cost-based optimization stands or falls on the quality of cardinality estimates (expected row counts).  If the optimizer has incorrect information to start with, it is quite unlikely to produce good quality execution plans except by chance.  There are many ways we can provide good starting information to the optimizer, and even more ways for cardinality estimation to go wrong.  Good database people know this, and work hard to write optimizer-friendly queries with a schema and metadata (e.g. statistics) that reduce the chances of poor cardinality estimation producing a sub-optimal plan.  Today, I am going to look at a case where poor cardinality estimation is Microsoft’s fault, and not yours. SQL Server 2005 SELECT th.ProductID, th.TransactionID, th.TransactionDate FROM Production.TransactionHistory AS th WHERE th.ProductID = 1 AND th.TransactionDate BETWEEN '20030901' AND '20031231'; The query plan on SQL Server 2005 is as follows (if you are using a more recent version of AdventureWorks, you will need to change the year on the date range from 2003 to 2007): There is an Index Seek on ProductID = 1, followed by a Key Lookup to find the Transaction Date for each row, and finally a Filter to restrict the results to only those rows where Transaction Date falls in the range specified.  The cardinality estimate of 45 rows at the Index Seek is exactly correct.  The table is not very large, there are up-to-date statistics associated with the index, so this is as expected. The estimate for the Key Lookup is also exactly right.  Each lookup into the Clustered Index to find the Transaction Date is guaranteed to return exactly one row.  The plan shows that the Key Lookup is expected to be executed 45 times.  The estimate for the Inner Join output is also correct – 45 rows from the seek joining to one row each time, gives 45 rows as output. The Filter estimate is also very good: the optimizer estimates 16.9951 rows will match the specified range of transaction dates.  Eleven rows are produced by this query, but that small difference is quite normal and certainly nothing to worry about here.  All good so far. SQL Server 2008 onward The same query executed against an identical copy of AdventureWorks on SQL Server 2008 produces a different execution plan: The optimizer has pushed the Filter conditions seen in the 2005 plan down to the Key Lookup.  This is a good optimization – it makes sense to filter rows out as early as possible.  Unfortunately, it has made a bit of a mess of the cardinality estimates. The post-Filter estimate of 16.9951 rows seen in the 2005 plan has moved with the predicate on Transaction Date.  Instead of estimating one row, the plan now suggests that 16.9951 rows will be produced by each clustered index lookup – clearly not right!  This misinformation also confuses SQL Sentry Plan Explorer: Plan Explorer shows 765 rows expected from the Key Lookup (it multiplies a rounded estimate of 17 rows by 45 expected executions to give 765 rows total). Workarounds One workaround is to provide a covering non-clustered index (avoiding the lookup avoids the problem of course): CREATE INDEX nc1 ON Production.TransactionHistory (ProductID) INCLUDE (TransactionDate); With the Transaction Date filter applied as a residual predicate in the same operator as the seek, the estimate is again as expected: We could also force the use of the ultimate covering index (the clustered one): SELECT th.ProductID, th.TransactionID, th.TransactionDate FROM Production.TransactionHistory AS th WITH (INDEX(1)) WHERE th.ProductID = 1 AND th.TransactionDate BETWEEN '20030901' AND '20031231'; Summary Providing a covering non-clustered index for all possible queries is not always practical, and scanning the clustered index will rarely be optimal.  Nevertheless, these are the best workarounds we have today. In the meantime, watch out for poor cardinality estimates when a predicate is applied as part of a lookup. The worst thing is that the estimate after the lookup join in the 2008+ plans is wrong.  It’s not hopelessly wrong in this particular case (45 versus 16.9951 is not the end of the world) but it easily can be much worse, and there’s not much you can do about it.  Any decisions made by the optimizer after such a lookup could be based on very wrong information – which can only be bad news. If you think this situation should be improved, please vote for this Connect item. © 2012 Paul White – All Rights Reserved twitter: @SQL_Kiwi email: [email protected]

  Read the article

• Can any postfix guru assist me determine how emails are still being sent via my server from unauthorized sources?

  - by Dave

  Hi all, I'm getting a little concerned as I run a small server hosting a number of websites and manage the email for a few dozen people. Just recently though I've had a couple of notifications from spamcop alerting me that spam has been sent from my server, and when I have a look over the logs from time to time I can indeed see that there are many repeated attempts of mail being sent from my server. Most of the time it gets knocked back from the destination servers but sometimes its getting through. Unfortunately I'm not linux or postfix expert, I can get by but had though I had my machine locked down quite securely, I don't allow relaying, when I check the online DNS/MX tools they tend to report my server as being OK so I'm not sure where to take it now and hoping someone might be able to throw me a few pointers. I get lots of entries like this in my MAIL.INFO log Jan 2 08:39:34 Debian-50-lenny-64-LAMP postfix/qmgr[15993]: 66B88257C12F: from=<>, size=3116, nrcpt=1 (queue active) Jan 2 08:39:34 Debian-50-lenny-64-LAMP postfix/qmgr[15993]: 614C2257C1BC: from=<[email protected]>, size=2490, nrcpt=3 (queue active) and Jan 7 16:09:37 Debian-50-lenny-64-LAMP postfix/error[6471]: 0A316257C204: to=<[email protected]>, relay=none, delay=384387, delays=384384/3/0/0.01, dsn=4.0.0, status=deferred (delivery temporarily suspended: host mx.fakemx.net[46.4.35.23] refused to talk to me: 421 mx.fakemx.net Service Unavailable) Jan 7 16:09:37 Debian-50-lenny-64-LAMP postfix/error[6470]: 5848C257C20D: to=<[email protected]>, relay=none, delay=384373, delays=384370/3/0/0.01, dsn=4.0.0, status=deferred (delivery temporarily suspended: host mx.fakemx.net[46.4.35.23] refused to talk to me: 421 mx.fakemx.net Service Unavailable) then there tends to be connection timeouts, so from what I see even though I had relaying disabled.. something is getting by and trying to send.. So if you can help that will be greatly appreciated, and any further logging/config info I can supply. Thanks

  Read the article

• Lost Permission on Files using wrong chmod syntax Centos 5.5

  - by alloutfallout

  Hello, I was trying to remove write permissions on an entire directory, and I used the incorrect command: chmod 644 -r sites/default I meant to type chmod -R 644 sites/default The result was this: chmod: cannot access `644': No such file or directory $ ls -als sites total 24 4 drwxr-xr-x 5 user group 4096 Jan 11 10:54 . 4 drwxrwxr-x 14 user group 4096 Jan 11 10:11 .. 4 drwxr-xr-x 4 user group 4096 Jan 5 01:25 all 4 d-w------- 3 user group 4096 Jan 11 10:43 default 4 -rw-r--r-- 1 user group 1849 Apr 15 2010 example.sites.php I fixed the permissions on the default folder with $ chmod 644 sites/default But, the following ls shows a all the files with red backgrounds and question marks. I can't access any files unless I am root. $ ls -als sites/default total 0 ? ?--------- ? ? ? ? ? . ? ?--------- ? ? ? ? ? .. ? ?--------- ? ? ? ? ? default.settings.php ? ?--------- ? ? ? ? ? files ? ?--------- ? ? ? ? ? settings.php When I log in as root, I can edit all of the files, and their permissions appear correctly. I do not know how to undo the damage caused by using -r with chmod instead of -R. Any Suggestions?

  Read the article

• Apache22 on FreeBSD - Starts, does not respond to requests

  - by NuclearDog

  Hey folks! I'm running Apache 2.2.17 with the peruser MPM on FreeBSD 8.2-RC1 on Amazon's EC2 (so it's XEN). It was installed from ports. My problem is that, although Apache is running, listening for, and accepting connections, it doesn't actually respond to any or show them in the log at all. If I telnet to the port it's listening on and type out an HTTP request: GET / HTTP/1.1 Host: asdfasdf And hit enter a couple of times, it just sits there... Nothing. No response requesting with a browser either. There doesn't appear to be anything helpful in the error log: [Sun Jan 09 16:56:24 2011] [warn] Init: Session Cache is not configured [hint: SSLSessionCache] [Sun Jan 09 16:56:25 2011] [notice] Digest: generating secret for digest authentication ... [Sun Jan 09 16:56:25 2011] [notice] Digest: done [Sun Jan 09 16:56:25 2011] [notice] Apache/2.2.17 (FreeBSD) mod_ssl/2.2.17 The access log stays empty: root:/var/log# wc httpd-access.log 0 0 0 httpd-access.log root:/var/log# I've tried with accf_http and accf_data both enabled and disabled, and with both the stock configuration and my customized config. I also tried uninstalling apache22-peruser-mpm and just installing straight apache22... Still no luck. I tried removing all of the LoadModule lines from httpd.conf and just re-enabled the ones that were necessary to parse the config. Ended up with only the following loaded: root:/usr/local/etc/apache22# /usr/local/sbin/apachectl -M Loaded Modules: core_module (static) mpm_peruser_module (static) http_module (static) so_module (static) authz_host_module (shared) log_config_module (shared) alias_module (shared) Syntax OK root:/usr/local/etc/apache22# Same results. Apache is definitely what's listening on port 80: root:/usr/local/etc/apache22# sockstat -4 | grep httpd root httpd 43789 3 tcp4 6 *:80 *:* root httpd 43789 4 tcp4 *:* *:* root:/usr/local/etc/apache22# And I know it's not a firewall issue as there is nothing running locally, and connecting from the local box to 127.0.0.1:80 results in the same issue. Does anyone have any idea what's going on? Why it would be doing this? I've exhausted all of my debugging expertise. :/ Thanks for any suggestions!

  Read the article

• In 10.10, USB 3.0 PCI Express card recognized by lspci but not lsusb or dmesg. How to fix?

  - by Paul

  Asus N PC, runs 10.10 x86_64 The Asus N comes with 4 usb 2.0 ports, each labelled 2.0 on the case. Attempting to add two usb 3.0 ports to be provided by a generic usb 3.0 pci express card installed in the pci expres slot. The new card says usb 3.0 and has the blue ports. The card is installed into the laptop unpowered, then the laptop is powered on and boots normally. Nothing happens when a USB 3.0 flash drive is inserted into the usb 3.0 port. uname -a Linux drpaulbrewer-N90SV 2.6.35.8 #1 SMP Fri Jan 14 15:54:11 EST 2011 x86_64 GNU/Linux lspci -v 00:00.0 Host bridge: Silicon Integrated Systems [SiS] 671MX Subsystem: ASUSTeK Computer Inc. Device 1b27 Flags: bus master, medium devsel, latency 64 Kernel modules: sis-agp 00:01.0 PCI bridge: Silicon Integrated Systems [SiS] PCI-to-PCI bridge (prog-if 00 [Normal decode]) Flags: bus master, fast devsel, latency 0 Bus: primary=00, secondary=01, subordinate=01, sec-latency=0 I/O behind bridge: 0000d000-0000dfff Memory behind bridge: fa000000-fdefffff Prefetchable memory behind bridge: 00000000d0000000-00000000dfffffff Capabilities: [d0] Express Root Port (Slot+), MSI 00 Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit- Capabilities: [f4] Power Management version 2 Capabilities: [70] Subsystem: Silicon Integrated Systems [SiS] PCI-to-PCI bridge Kernel driver in use: pcieport 00:02.0 ISA bridge: Silicon Integrated Systems [SiS] SiS968 [MuTIOL Media IO] (rev 01) Flags: bus master, medium devsel, latency 0 00:02.5 IDE interface: Silicon Integrated Systems [SiS] 5513 [IDE] (rev 01) (prog-if 80 [Master]) Subsystem: ASUSTeK Computer Inc. Device 1b27 Flags: bus master, medium devsel, latency 128 I/O ports at 01f0 [size=8] I/O ports at 03f4 [size=1] I/O ports at 0170 [size=8] I/O ports at 0374 [size=1] I/O ports at ffe0 [size=16] Capabilities: [58] Power Management version 2 Kernel driver in use: pata_sis 00:03.0 USB Controller: Silicon Integrated Systems [SiS] USB 1.1 Controller (rev 0f) (prog-if 10 [OHCI]) Subsystem: ASUSTeK Computer Inc. Device 1b27 Flags: bus master, medium devsel, latency 64, IRQ 20 Memory at f9fff000 (32-bit, non-prefetchable) [size=4K] Kernel driver in use: ohci_hcd 00:03.1 USB Controller: Silicon Integrated Systems [SiS] USB 1.1 Controller (rev 0f) (prog-if 10 [OHCI]) Subsystem: ASUSTeK Computer Inc. Device 1b27 Flags: bus master, medium devsel, latency 64, IRQ 21 Memory at f9ffe000 (32-bit, non-prefetchable) [size=4K] Kernel driver in use: ohci_hcd 00:03.3 USB Controller: Silicon Integrated Systems [SiS] USB 2.0 Controller (prog-if 20 [EHCI]) Subsystem: ASUSTeK Computer Inc. Device 1b27 Flags: bus master, medium devsel, latency 64, IRQ 22 Memory at f9ffd000 (32-bit, non-prefetchable) [size=4K] Capabilities: [50] Power Management version 2 Kernel driver in use: ehci_hcd 00:04.0 Ethernet controller: Silicon Integrated Systems [SiS] 191 Gigabit Ethernet Adapter (rev 02) Subsystem: ASUSTeK Computer Inc. Device 11f5 Flags: bus master, medium devsel, latency 0, IRQ 19 Memory at f9ffcc00 (32-bit, non-prefetchable) [size=128] I/O ports at cc00 [size=128] Capabilities: [40] Power Management version 2 Kernel driver in use: sis190 Kernel modules: sis190 00:05.0 IDE interface: Silicon Integrated Systems [SiS] SATA Controller / IDE mode (rev 03) (prog-if 8f [Master SecP SecO PriP PriO]) Subsystem: ASUSTeK Computer Inc. Device 1b27 Flags: bus master, medium devsel, latency 64, IRQ 17 I/O ports at c800 [size=8] I/O ports at c400 [size=4] I/O ports at c000 [size=8] I/O ports at bc00 [size=4] I/O ports at b800 [size=16] I/O ports at b400 [size=128] Capabilities: [58] Power Management version 2 Kernel driver in use: sata_sis Kernel modules: sata_sis 00:06.0 PCI bridge: Silicon Integrated Systems [SiS] PCI-to-PCI bridge (prog-if 00 [Normal decode]) Flags: bus master, fast devsel, latency 0 Bus: primary=00, secondary=02, subordinate=02, sec-latency=0 Memory behind bridge: fdf00000-fdffffff Capabilities: [b0] Subsystem: Silicon Integrated Systems [SiS] Device 0004 Capabilities: [c0] MSI: Enable+ Count=1/1 Maskable- 64bit+ Capabilities: [d0] Express Root Port (Slot+), MSI 00 Capabilities: [f4] Power Management version 2 Kernel driver in use: pcieport 00:07.0 PCI bridge: Silicon Integrated Systems [SiS] PCI-to-PCI bridge (prog-if 00 [Normal decode]) Flags: bus master, fast devsel, latency 0 Bus: primary=00, secondary=03, subordinate=06, sec-latency=0 I/O behind bridge: 0000e000-0000efff Memory behind bridge: fe000000-febfffff Prefetchable memory behind bridge: 00000000f6000000-00000000f8ffffff Capabilities: [b0] Subsystem: Silicon Integrated Systems [SiS] Device 0004 Capabilities: [c0] MSI: Enable+ Count=1/1 Maskable- 64bit+ Capabilities: [d0] Express Root Port (Slot+), MSI 00 Capabilities: [f4] Power Management version 2 Kernel driver in use: pcieport 00:0f.0 Audio device: Silicon Integrated Systems [SiS] Azalia Audio Controller Subsystem: ASUSTeK Computer Inc. Device 17b3 Flags: bus master, medium devsel, latency 0, IRQ 18 Memory at f9ff4000 (32-bit, non-prefetchable) [size=16K] Capabilities: [50] Power Management version 2 Kernel driver in use: HDA Intel Kernel modules: snd-hda-intel 01:00.0 VGA compatible controller: nVidia Corporation G96 [GeForce GT 130M] (rev a1) (prog-if 00 [VGA controller]) Subsystem: ASUSTeK Computer Inc. Device 2021 Flags: bus master, fast devsel, latency 0, IRQ 16 Memory at fc000000 (32-bit, non-prefetchable) [size=16M] Memory at d0000000 (64-bit, prefetchable) [size=256M] Memory at fa000000 (64-bit, non-prefetchable) [size=32M] I/O ports at dc00 [size=128] [virtual] Expansion ROM at fde80000 [disabled] [size=512K] Capabilities: [60] Power Management version 3 Capabilities: [68] MSI: Enable- Count=1/1 Maskable- 64bit+ Capabilities: [78] Express Endpoint, MSI 00 Capabilities: [b4] Vendor Specific Information: Len=14 <?> Kernel driver in use: nvidia Kernel modules: nvidia-current, nouveau, nvidiafb 02:00.0 Network controller: Atheros Communications Inc. AR928X Wireless Network Adapter (PCI-Express) (rev 01) Subsystem: Device 1a3b:1067 Flags: bus master, fast devsel, latency 0, IRQ 16 Memory at fdff0000 (64-bit, non-prefetchable) [size=64K] Capabilities: [40] Power Management version 2 Capabilities: [50] MSI: Enable- Count=1/1 Maskable- 64bit- Capabilities: [60] Express Legacy Endpoint, MSI 00 Capabilities: [90] MSI-X: Enable- Count=1 Masked- Kernel driver in use: ath9k Kernel modules: ath9k 03:00.0 USB Controller: NEC Corporation uPD720200 USB 3.0 Host Controller (rev 03) (prog-if 30) Flags: bus master, fast devsel, latency 0, IRQ 10 Memory at febfe000 (64-bit, non-prefetchable) [size=8K] Capabilities: [50] Power Management version 3 Capabilities: [70] MSI: Enable- Count=1/8 Maskable- 64bit+ Capabilities: [90] MSI-X: Enable- Count=8 Masked- Capabilities: [a0] Express Endpoint, MSI 00 lsusb Bus 003 Device 002: ID 0b05:1751 ASUSTek Computer, Inc. BT-253 Bluetooth Adapter Bus 003 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub Bus 002 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub Bus 001 Device 004: ID 0bda:0158 Realtek Semiconductor Corp. USB 2.0 multicard reader Bus 001 Device 002: ID 04f2:b071 Chicony Electronics Co., Ltd 2.0M UVC Webcam / CNF7129 Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub dmesg trying to post dmesg exceeded the stackexchange posting limit of 30K... but nothing there is usb 3.0

  Read the article

• BIOS flash XP, 1 long beep, 2 short beeps, over&over

  - by Paul

  BIOS issue on HP dv9233cl laptop, wiped drive of Vista, loaded XP, not all the drives loaded. Went to the HP website, downloaded all drivers for this laptop. Started loading them. Loaded WIN Flash HP Network System BIOS Window SP42187. After a minute a low resolution screen appeared stating "It is now safe to turn off the computer" I waited a minute and half. Turned it off. Let it set 10 seconds try to start and No screen images at all and a nasty loud long beep 2 short beeps, 2 seconds of silence and it happens over & over again. I have unplugged/removed battery, still same problem, Any sugg.... Thx.. Paul

  Read the article

• LTO 3 tape drive needing repaired

  - by DO it all Paul

  We have an IBM LTO 3 tape drive that needs repaired and with the £400 price tag i'm having to shop around for quotes. My question is has anyone actually repaired one before and how was in done? The first error LED was showing a 6, then i cleared the mangled tape only for it to start flashing alternate 'o' on the 7 segment display, simliar to a half 8, flashing top to bottom and it would just flash away like that coupled with a flashing amber light. I tried a reset holding the eject button for it to show an 'r' the go back to flashing again as before. I checked the IBM solutions for the codes but this flashing isn't documented at all. Would be great if anyone had any experience in this area. Thank you, Paul

  Read the article

• HTTP downloads slow - FTP of same file very fast - Windows 2003

  - by Paul Hinett

  I am having some issues with download speeds on my site via http, i am averaging around 70kbps downloading a file that is around 70mb. But if i connect to my server via FTP and download the same file on the same computer / connection i am averaging about 300+kbps. I know my server has alot of connections at any one time, probably around 400 connections. My server has a 1gbps connection to the internet so there is plenty of bandwidth available, as proven with the FTP. I have no throttling of any kind enabled in IIS. If interested there is a test file here you can download to check the speed: http://filesd.house-mixes.com/test.zip I am based in the UK and the server is in Washington, USA if that makes any difference. Paul

  Read the article

• Setting up routing for MS DirectAccess to a VMWare EsXi Host

  - by Paul D'Ambra

  I'm trying to set up DirectAccess on a virtual machine so I can demonstrate it's value and then if need be add a physical machine to host it. I'm hitting a problem because the Direct Access machine (DA01) needs to have 2 public addresses actually configured on the external adapter but there is a Zyxel Zywall USG300 between the VMware ESXi host and the outside world. I've summarised my setup in this diagram If I ping from the LAN to 212.x.y.89 I get a response but if I ping from the VM I get destination host unreachable. I used "route add 212.x.y.89 192.c.d.1" and get request timed out. At that point I see outbound traffic allowed on the Zyxel firewall but nothing coming back. I'm past my understanding of routing and VMWare so am not sure how to tie down where my problem lies (or even if this setup is possible). So any help massively appreciated. Paul

  Read the article

• KVM Guest not reachable from host

  - by Paul

  Hello, I'm running Ubuntu server 9.10, installed KVM etc. Created the bridge network following instructions on help.ubuntu.com/community/KVM/Networking Created a windows 2008 guest using virt-install command line (using virt-manager GUI from a remote Ubuntu desktop would not let me select the ISO location). I can however use a remote virt-manager to connect to the guest and complete the windows install. Within windows 2008 I changed the IP address but cannot ping from outside world. The bridge network appears fine - I'm not sure what else to look at! Here is the interfaces file: The loopback network interface auto lo iface lo inet loopback The primary network interface auto eth0 iface eth0 inet manual # auto br0 iface br0 inet static address 60.234.64.50 netmask 255.255.255.248 network 60.234.0.0 broadcast 60.234.0.255 gateway 60.234.64.49 bridge_ports eth0 bridge_stp off bridge_fd 0 bridge_maxwait 0 auto eth1 iface eth1 inet static address 192.168.12.2 netmask 255.255.255.0 broadcast 192.168.12.255 The ip of the windows server is 60.234.64.52 What else should I check? Regards Paul.

  Read the article

• Silverlight Version 4 latest build for Win7 64bit and WinXP 32bit

  - by Paul

  I have a requirement where a few people need the latest version of Silverlight 4 installed. I know the latest version is 5.xx... but apparently with some new software we're having installed we have to use version 4 After a bit of googling i can see that the latest version is... Build 4.1.10329.0 Released May 8, 2012 We have a mix of Win7 64-bit machines and WinXP 32-bit machines. Q: Is there a different version for each OS or the same one fits all. (This seems strangely hard to decipher by googling) Q: Does anyone know where i can download the latest version 4? Microsoft do not seem to offer it anymore unless i'm just not finding it. Q: Is there a separate browser version of it or will installing it also handle any browser needs (our new software will be browser based) Any pointers much appreciated. Paul

  Read the article

• Connecting my iPhone to iTunes causes my Acer laptop to crash

  - by Paul Sheldrake

  Hello I have an Acer Travelmate 8200 laptop and whenever I connect my Iphone to it, it crashes with the BSOD(Blue Screen Of Death). I have figured out that if I delete all the pictures in my phone I can get it to connect but that is not a ideal long term solution. I also read that it may be a conflict with the built in web-cam I have but I've upgraded the driver and I still get the crashing problem. Any suggestions would be appreciated! Thanks Paul! edit: Here is the BSOD message I get

  Read the article

• No HDMI audio - Windows 8 - ASUS H81M-PLUS

  - by Paul Wright

  I have an issue with HDMI audio on Windows 8 using an ASUS H81M-PLUS motherboard (without an external GFX card). There are many forum posts advising you to go into playback devices and setting HDMI to be default - I have done this. To eliminate what works and what doesn't work: I have not been able to get sound from my HDTV using HDMI. I have used this HDMI cable with my PS3, so this cable should be fine. I am able to use the HDMI cable in extended mode, so that I have two monitors (including the TV), just no audio. This HDMI cable goes straight from the motherboard to the TV. Below I have included 'Device manager', and 'Playback Devices' (Sound). Device Manager Playback Devices, showing disabled and disconnected devices I am at a loss. I have uninstalled all drivers, and then rebooted and made windows look for the correct ones, made sure the HDMI device was default. Thanks, Paul

  Read the article

• i accidentally deleted the recovery folder on a partition (win vista home)

  - by paul

  i accidentally deleted the recovery folder on the recovery partition (win vista home) i think it was some sort of scheduled maintenance of some program that i did not configure properly? oops... lol i called toshiba and they said i needed to buy a recovery program, which i didnt bother doing. I bought a legal copy of vista and would like to install the correct files and in a way that when my computer starts looking for files it will eventually find them or i can point to the partition. i'm pretty sure it's not a matter of copy and paste (is it?) thanks Paul

  Read the article

• Creating different margins on the first page of a word template

  - by Paul

  I have a letterhead template and I need the first page left margin to be larger than subsequent pages. I've seen the option of placing a text box or image box in the header to push the text but this ends up throwing off the tabs and bullet list indentation markers. I thought of setting up the first page using two columns and pushing the text to start on the second column but I can't seem to find a way to get the text to switch back to 1 column on the second page when it is created from text overflowing. Does anyone know how something like this is possible? Thanks in advance, Paul

  Read the article

• Disable OS X Portable Home Directories for specific hosts for all users, not just individuals?

  - by Paul Nendick

  Would it be possible to block any and all Portable Home Directory services for specific hosts? Something like MCX's "MobileAccountNeverAsk-" but for the whole workstation? We have a network with both portable and stationary machines. I'd like our users to be able to use all machines, going portable on the MacBook but not being bothering with syncing when logged into stationary iMacs or Mac Pros. The Open Directory servers are running Snow Leopard (for now) and all clients are running Lion. Thanks! Paul

  Read the article

• Fun with Aggregates

  - by Paul White

  There are interesting things to be learned from even the simplest queries.  For example, imagine you are given the task of writing a query to list AdventureWorks product names where the product has at least one entry in the transaction history table, but fewer than ten. One possible query to meet that specification is: SELECT p.Name FROM Production.Product AS p JOIN Production.TransactionHistory AS th ON p.ProductID = th.ProductID GROUP BY p.ProductID, p.Name HAVING COUNT_BIG(*) < 10; That query correctly returns 23 rows (execution plan and data sample shown below): The execution plan looks a bit different from the written form of the query: the base tables are accessed in reverse order, and the aggregation is performed before the join.  The general idea is to read all rows from the history table, compute the count of rows grouped by ProductID, merge join the results to the Product table on ProductID, and finally filter to only return rows where the count is less than ten. This ‘fully-optimized’ plan has an estimated cost of around 0.33 units.  The reason for the quote marks there is that this plan is not quite as optimal as it could be – surely it would make sense to push the Filter down past the join too?  To answer that, let’s look at some other ways to formulate this query.  This being SQL, there are any number of ways to write logically-equivalent query specifications, so we’ll just look at a couple of interesting ones.  The first query is an attempt to reverse-engineer T-SQL from the optimized query plan shown above.  It joins the result of pre-aggregating the history table to the Product table before filtering: SELECT p.Name FROM ( SELECT th.ProductID, cnt = COUNT_BIG(*) FROM Production.TransactionHistory AS th GROUP BY th.ProductID ) AS q1 JOIN Production.Product AS p ON p.ProductID = q1.ProductID WHERE q1.cnt < 10; Perhaps a little surprisingly, we get a slightly different execution plan: The results are the same (23 rows) but this time the Filter is pushed below the join!  The optimizer chooses nested loops for the join, because the cardinality estimate for rows passing the Filter is a bit low (estimate 1 versus 23 actual), though you can force a merge join with a hint and the Filter still appears below the join.  In yet another variation, the < 10 predicate can be ‘manually pushed’ by specifying it in a HAVING clause in the “q1” sub-query instead of in the WHERE clause as written above. The reason this predicate can be pushed past the join in this query form, but not in the original formulation is simply an optimizer limitation – it does make efforts (primarily during the simplification phase) to encourage logically-equivalent query specifications to produce the same execution plan, but the implementation is not completely comprehensive. Moving on to a second example, the following query specification results from phrasing the requirement as “list the products where there exists fewer than ten correlated rows in the history table”: SELECT p.Name FROM Production.Product AS p WHERE EXISTS ( SELECT * FROM Production.TransactionHistory AS th WHERE th.ProductID = p.ProductID HAVING COUNT_BIG(*) < 10 ); Unfortunately, this query produces an incorrect result (86 rows): The problem is that it lists products with no history rows, though the reasons are interesting.  The COUNT_BIG(*) in the EXISTS clause is a scalar aggregate (meaning there is no GROUP BY clause) and scalar aggregates always produce a value, even when the input is an empty set.  In the case of the COUNT aggregate, the result of aggregating the empty set is zero (the other standard aggregates produce a NULL).  To make the point really clear, let’s look at product 709, which happens to be one for which no history rows exist: -- Scalar aggregate SELECT COUNT_BIG(*) FROM Production.TransactionHistory AS th WHERE th.ProductID = 709;   -- Vector aggregate SELECT COUNT_BIG(*) FROM Production.TransactionHistory AS th WHERE th.ProductID = 709 GROUP BY th.ProductID; The estimated execution plans for these two statements are almost identical: You might expect the Stream Aggregate to have a Group By for the second statement, but this is not the case.  The query includes an equality comparison to a constant value (709), so all qualified rows are guaranteed to have the same value for ProductID and the Group By is optimized away. In fact there are some minor differences between the two plans (the first is auto-parameterized and qualifies for trivial plan, whereas the second is not auto-parameterized and requires cost-based optimization), but there is nothing to indicate that one is a scalar aggregate and the other is a vector aggregate.  This is something I would like to see exposed in show plan so I suggested it on Connect.  Anyway, the results of running the two queries show the difference at runtime: The scalar aggregate (no GROUP BY) returns a result of zero, whereas the vector aggregate (with a GROUP BY clause) returns nothing at all.  Returning to our EXISTS query, we could ‘fix’ it by changing the HAVING clause to reject rows where the scalar aggregate returns zero: SELECT p.Name FROM Production.Product AS p WHERE EXISTS ( SELECT * FROM Production.TransactionHistory AS th WHERE th.ProductID = p.ProductID HAVING COUNT_BIG(*) BETWEEN 1 AND 9 ); The query now returns the correct 23 rows: Unfortunately, the execution plan is less efficient now – it has an estimated cost of 0.78 compared to 0.33 for the earlier plans.  Let’s try adding a redundant GROUP BY instead of changing the HAVING clause: SELECT p.Name FROM Production.Product AS p WHERE EXISTS ( SELECT * FROM Production.TransactionHistory AS th WHERE th.ProductID = p.ProductID GROUP BY th.ProductID HAVING COUNT_BIG(*) < 10 ); Not only do we now get correct results (23 rows), this is the execution plan: I like to compare that plan to quantum physics: if you don’t find it shocking, you haven’t understood it properly :)  The simple addition of a redundant GROUP BY has resulted in the EXISTS form of the query being transformed into exactly the same optimal plan we found earlier.  What’s more, in SQL Server 2008 and later, we can replace the odd-looking GROUP BY with an explicit GROUP BY on the empty set: SELECT p.Name FROM Production.Product AS p WHERE EXISTS ( SELECT * FROM Production.TransactionHistory AS th WHERE th.ProductID = p.ProductID GROUP BY () HAVING COUNT_BIG(*) < 10 ); I offer that as an alternative because some people find it more intuitive (and it perhaps has more geek value too).  Whichever way you prefer, it’s rather satisfying to note that the result of the sub-query does not exist for a particular correlated value where a vector aggregate is used (the scalar COUNT aggregate always returns a value, even if zero, so it always ‘EXISTS’ regardless which ProductID is logically being evaluated). The following query forms also produce the optimal plan and correct results, so long as a vector aggregate is used (you can probably find more equivalent query forms): WHERE Clause SELECT p.Name FROM Production.Product AS p WHERE ( SELECT COUNT_BIG(*) FROM Production.TransactionHistory AS th WHERE th.ProductID = p.ProductID GROUP BY () ) < 10; APPLY SELECT p.Name FROM Production.Product AS p CROSS APPLY ( SELECT NULL FROM Production.TransactionHistory AS th WHERE th.ProductID = p.ProductID GROUP BY () HAVING COUNT_BIG(*) < 10 ) AS ca (dummy); FROM Clause SELECT q1.Name FROM ( SELECT p.Name, cnt = ( SELECT COUNT_BIG(*) FROM Production.TransactionHistory AS th WHERE th.ProductID = p.ProductID GROUP BY () ) FROM Production.Product AS p ) AS q1 WHERE q1.cnt < 10; This last example uses SUM(1) instead of COUNT and does not require a vector aggregate…you should be able to work out why :) SELECT q.Name FROM ( SELECT p.Name, cnt = ( SELECT SUM(1) FROM Production.TransactionHistory AS th WHERE th.ProductID = p.ProductID ) FROM Production.Product AS p ) AS q WHERE q.cnt < 10; The semantics of SQL aggregates are rather odd in places.  It definitely pays to get to know the rules, and to be careful to check whether your queries are using scalar or vector aggregates.  As we have seen, query plans do not show in which ‘mode’ an aggregate is running and getting it wrong can cause poor performance, wrong results, or both. © 2012 Paul White Twitter: @SQL_Kiwi email: [email protected]

  Read the article

• I see no LOBs!

  - by Paul White

  Is it possible to see LOB (large object) logical reads from STATISTICS IO output on a table with no LOB columns? I was asked this question today by someone who had spent a good fraction of their afternoon trying to work out why this was occurring – even going so far as to re-run DBCC CHECKDB to see if any corruption had taken place.  The table in question wasn’t particularly pretty – it had grown somewhat organically over time, with new columns being added every so often as the need arose.  Nevertheless, it remained a simple structure with no LOB columns – no TEXT or IMAGE, no XML, no MAX types – nothing aside from ordinary INT, MONEY, VARCHAR, and DATETIME types.  To add to the air of mystery, not every query that ran against the table would report LOB logical reads – just sometimes – but when it did, the query often took much longer to execute. Ok, enough of the pre-amble.  I can’t reproduce the exact structure here, but the following script creates a table that will serve to demonstrate the effect: IF OBJECT_ID(N'dbo.Test', N'U') IS NOT NULL DROP TABLE dbo.Test GO CREATE TABLE dbo.Test ( row_id NUMERIC IDENTITY NOT NULL,   col01 NVARCHAR(450) NOT NULL, col02 NVARCHAR(450) NOT NULL, col03 NVARCHAR(450) NOT NULL, col04 NVARCHAR(450) NOT NULL, col05 NVARCHAR(450) NOT NULL, col06 NVARCHAR(450) NOT NULL, col07 NVARCHAR(450) NOT NULL, col08 NVARCHAR(450) NOT NULL, col09 NVARCHAR(450) NOT NULL, col10 NVARCHAR(450) NOT NULL, CONSTRAINT [PK dbo.Test row_id] PRIMARY KEY CLUSTERED (row_id) ) ; The next script loads the ten variable-length character columns with one-character strings in the first row, two-character strings in the second row, and so on down to the 450th row: WITH Numbers AS ( -- Generates numbers 1 - 450 inclusive SELECT TOP (450) n = ROW_NUMBER() OVER (ORDER BY (SELECT 0)) FROM master.sys.columns C1, master.sys.columns C2, master.sys.columns C3 ORDER BY n ASC ) INSERT dbo.Test WITH (TABLOCKX) SELECT REPLICATE(N'A', N.n), REPLICATE(N'B', N.n), REPLICATE(N'C', N.n), REPLICATE(N'D', N.n), REPLICATE(N'E', N.n), REPLICATE(N'F', N.n), REPLICATE(N'G', N.n), REPLICATE(N'H', N.n), REPLICATE(N'I', N.n), REPLICATE(N'J', N.n) FROM Numbers AS N ORDER BY N.n ASC ; Once those two scripts have run, the table contains 450 rows and 10 columns of data like this: Most of the time, when we query data from this table, we don’t see any LOB logical reads, for example: -- Find the maximum length of the data in -- column 5 for a range of rows SELECT result = MAX(DATALENGTH(T.col05)) FROM dbo.Test AS T WHERE row_id BETWEEN 50 AND 100 ; But with a different query… -- Read all the data in column 1 SELECT result = MAX(DATALENGTH(T.col01)) FROM dbo.Test AS T ; …suddenly we have 49 LOB logical reads, as well as the ‘normal’ logical reads we would expect. The Explanation If we had tried to create this table in SQL Server 2000, we would have received a warning message to say that future INSERT or UPDATE operations on the table might fail if the resulting row exceeded the in-row storage limit of 8060 bytes.  If we needed to store more data than would fit in an 8060 byte row (including internal overhead) we had to use a LOB column – TEXT, NTEXT, or IMAGE.  These special data types store the large data values in a separate structure, with just a small pointer left in the original row. Row Overflow SQL Server 2005 introduced a feature called row overflow, which allows one or more variable-length columns in a row to move to off-row storage if the data in a particular row would otherwise exceed 8060 bytes.  You no longer receive a warning when creating (or altering) a table that might need more than 8060 bytes of in-row storage; if SQL Server finds that it can no longer fit a variable-length column in a particular row, it will silently move one or more of these columns off the row into a separate allocation unit. Only variable-length columns can be moved in this way (for example the (N)VARCHAR, VARBINARY, and SQL_VARIANT types).  Fixed-length columns (like INTEGER and DATETIME for example) never move into ‘row overflow’ storage.  The decision to move a column off-row is done on a row-by-row basis – so data in a particular column might be stored in-row for some table records, and off-row for others. In general, if SQL Server finds that it needs to move a column into row-overflow storage, it moves the largest variable-length column record for that row.  Note that in the case of an UPDATE statement that results in the 8060 byte limit being exceeded, it might not be the column that grew that is moved! Sneaky LOBs Anyway, that’s all very interesting but I don’t want to get too carried away with the intricacies of row-overflow storage internals.  The point is that it is now possible to define a table with non-LOB columns that will silently exceed the old row-size limit and result in ordinary variable-length columns being moved to off-row storage.  Adding new columns to a table, expanding an existing column definition, or simply storing more data in a column than you used to – all these things can result in one or more variable-length columns being moved off the row. Note that row-overflow storage is logically quite different from old-style LOB and new-style MAX data type storage – individual variable-length columns are still limited to 8000 bytes each – you can just have more of them now.  Having said that, the physical mechanisms involved are very similar to full LOB storage – a column moved to row-overflow leaves a 24-byte pointer record in the row, and the ‘separate storage’ I have been talking about is structured very similarly to both old-style LOBs and new-style MAX types.  The disadvantages are also the same: when SQL Server needs a row-overflow column value it needs to follow the in-row pointer a navigate another chain of pages, just like retrieving a traditional LOB. And Finally… In the example script presented above, the rows with row_id values from 402 to 450 inclusive all exceed the total in-row storage limit of 8060 bytes.  A SELECT that references a column in one of those rows that has moved to off-row storage will incur one or more lob logical reads as the storage engine locates the data.  The results on your system might vary slightly depending on your settings, of course; but in my tests only column 1 in rows 402-450 moved off-row.  You might like to play around with the script – updating columns, changing data type lengths, and so on – to see the effect on lob logical reads and which columns get moved when.  You might even see row-overflow columns moving back in-row if they are updated to be smaller (hint: reduce the size of a column entry by at least 1000 bytes if you hope to see this). Be aware that SQL Server will not warn you when it moves ‘ordinary’ variable-length columns into overflow storage, and it can have dramatic effects on performance.  It makes more sense than ever to choose column data types sensibly.  If you make every column a VARCHAR(8000) or NVARCHAR(4000), and someone stores data that results in a row needing more than 8060 bytes, SQL Server might turn some of your column data into pseudo-LOBs – all without saying a word. Finally, some people make a distinction between ordinary LOBs (those that can hold up to 2GB of data) and the LOB-like structures created by row-overflow (where columns are still limited to 8000 bytes) by referring to row-overflow LOBs as SLOBs.  I find that quite appealing, but the ‘S’ stands for ‘small’, which makes expanding the whole acronym a little daft-sounding…small large objects anyone? © Paul White 2011 email: [email protected] twitter: @SQL_Kiwi

  Read the article

• When is a Seek not a Seek?

  - by Paul White

  The following script creates a single-column clustered table containing the integers from 1 to 1,000 inclusive. IF OBJECT_ID(N'tempdb..#Test', N'U') IS NOT NULL DROP TABLE #Test ; GO CREATE TABLE #Test ( id INTEGER PRIMARY KEY CLUSTERED ); ; INSERT #Test (id) SELECT V.number FROM master.dbo.spt_values AS V WHERE V.[type] = N'P' AND V.number BETWEEN 1 AND 1000 ; Let’s say we need to find the rows with values from 100 to 170, excluding any values that divide exactly by 10.  One way to write that query would be: SELECT T.id FROM #Test AS T WHERE T.id IN ( 101,102,103,104,105,106,107,108,109, 111,112,113,114,115,116,117,118,119, 121,122,123,124,125,126,127,128,129, 131,132,133,134,135,136,137,138,139, 141,142,143,144,145,146,147,148,149, 151,152,153,154,155,156,157,158,159, 161,162,163,164,165,166,167,168,169 ) ; That query produces a pretty efficient-looking query plan: Knowing that the source column is defined as an INTEGER, we could also express the query this way: SELECT T.id FROM #Test AS T WHERE T.id >= 101 AND T.id <= 169 AND T.id % 10 > 0 ; We get a similar-looking plan: If you look closely, you might notice that the line connecting the two icons is a little thinner than before.  The first query is estimated to produce 61.9167 rows – very close to the 63 rows we know the query will return.  The second query presents a tougher challenge for SQL Server because it doesn’t know how to predict the selectivity of the modulo expression (T.id % 10 > 0).  Without that last line, the second query is estimated to produce 68.1667 rows – a slight overestimate.  Adding the opaque modulo expression results in SQL Server guessing at the selectivity.  As you may know, the selectivity guess for a greater-than operation is 30%, so the final estimate is 30% of 68.1667, which comes to 20.45 rows. The second difference is that the Clustered Index Seek is costed at 99% of the estimated total for the statement.  For some reason, the final SELECT operator is assigned a small cost of 0.0000484 units; I have absolutely no idea why this is so, or what it models.  Nevertheless, we can compare the total cost for both queries: the first one comes in at 0.0033501 units, and the second at 0.0034054.  The important point is that the second query is costed very slightly higher than the first, even though it is expected to produce many fewer rows (20.45 versus 61.9167). If you run the two queries, they produce exactly the same results, and both complete so quickly that it is impossible to measure CPU usage for a single execution.  We can, however, compare the I/O statistics for a single run by running the queries with STATISTICS IO ON: Table '#Test'. Scan count 63, logical reads 126, physical reads 0. Table '#Test'. Scan count 01, logical reads 002, physical reads 0. The query with the IN list uses 126 logical reads (and has a ‘scan count’ of 63), while the second query form completes with just 2 logical reads (and a ‘scan count’ of 1).  It is no coincidence that 126 = 63 * 2, by the way.  It is almost as if the first query is doing 63 seeks, compared to one for the second query. In fact, that is exactly what it is doing.  There is no indication of this in the graphical plan, or the tool-tip that appears when you hover your mouse over the Clustered Index Seek icon.  To see the 63 seek operations, you have click on the Seek icon and look in the Properties window (press F4, or right-click and choose from the menu): The Seek Predicates list shows a total of 63 seek operations – one for each of the values from the IN list contained in the first query.  I have expanded the first seek node to show the details; it is seeking down the clustered index to find the entry with the value 101.  Each of the other 62 nodes expands similarly, and the same information is contained (even more verbosely) in the XML form of the plan. Each of the 63 seek operations starts at the root of the clustered index B-tree and navigates down to the leaf page that contains the sought key value.  Our table is just large enough to need a separate root page, so each seek incurs 2 logical reads (one for the root, and one for the leaf).  We can see the index depth using the INDEXPROPERTY function, or by using the a DMV: SELECT S.index_type_desc, S.index_depth FROM sys.dm_db_index_physical_stats ( DB_ID(N'tempdb'), OBJECT_ID(N'tempdb..#Test', N'U'), 1, 1, DEFAULT ) AS S ; Let’s look now at the Properties window when the Clustered Index Seek from the second query is selected: There is just one seek operation, which starts at the root of the index and navigates the B-tree looking for the first key that matches the Start range condition (id >= 101).  It then continues to read records at the leaf level of the index (following links between leaf-level pages if necessary) until it finds a row that does not meet the End range condition (id <= 169).  Every row that meets the seek range condition is also tested against the Residual Predicate highlighted above (id % 10 > 0), and is only returned if it matches that as well. You will not be surprised that the single seek (with a range scan and residual predicate) is much more efficient than 63 singleton seeks.  It is not 63 times more efficient (as the logical reads comparison would suggest), but it is around three times faster.  Let’s run both query forms 10,000 times and measure the elapsed time: DECLARE @i INTEGER, @n INTEGER = 10000, @s DATETIME = GETDATE() ; SET NOCOUNT ON; SET STATISTICS XML OFF; ; WHILE @n > 0 BEGIN SELECT @i = T.id FROM #Test AS T WHERE T.id IN ( 101,102,103,104,105,106,107,108,109, 111,112,113,114,115,116,117,118,119, 121,122,123,124,125,126,127,128,129, 131,132,133,134,135,136,137,138,139, 141,142,143,144,145,146,147,148,149, 151,152,153,154,155,156,157,158,159, 161,162,163,164,165,166,167,168,169 ) ; SET @n -= 1; END ; PRINT DATEDIFF(MILLISECOND, @s, GETDATE()) ; GO DECLARE @i INTEGER, @n INTEGER = 10000, @s DATETIME = GETDATE() ; SET NOCOUNT ON ; WHILE @n > 0 BEGIN SELECT @i = T.id FROM #Test AS T WHERE T.id >= 101 AND T.id <= 169 AND T.id % 10 > 0 ; SET @n -= 1; END ; PRINT DATEDIFF(MILLISECOND, @s, GETDATE()) ; On my laptop, running SQL Server 2008 build 4272 (SP2 CU2), the IN form of the query takes around 830ms and the range query about 300ms.  The main point of this post is not performance, however – it is meant as an introduction to the next few parts in this mini-series that will continue to explore scans and seeks in detail. When is a seek not a seek?  When it is 63 seeks © Paul White 2011 email: [email protected] twitter: @SQL_kiwi

  Read the article

• Fun with Aggregates

  - by Paul White

  There are interesting things to be learned from even the simplest queries.  For example, imagine you are given the task of writing a query to list AdventureWorks product names where the product has at least one entry in the transaction history table, but fewer than ten. One possible query to meet that specification is: SELECT p.Name FROM Production.Product AS p JOIN Production.TransactionHistory AS th ON p.ProductID = th.ProductID GROUP BY p.ProductID, p.Name HAVING COUNT_BIG(*) < 10; That query correctly returns 23 rows (execution plan and data sample shown below): The execution plan looks a bit different from the written form of the query: the base tables are accessed in reverse order, and the aggregation is performed before the join.  The general idea is to read all rows from the history table, compute the count of rows grouped by ProductID, merge join the results to the Product table on ProductID, and finally filter to only return rows where the count is less than ten. This ‘fully-optimized’ plan has an estimated cost of around 0.33 units.  The reason for the quote marks there is that this plan is not quite as optimal as it could be – surely it would make sense to push the Filter down past the join too?  To answer that, let’s look at some other ways to formulate this query.  This being SQL, there are any number of ways to write logically-equivalent query specifications, so we’ll just look at a couple of interesting ones.  The first query is an attempt to reverse-engineer T-SQL from the optimized query plan shown above.  It joins the result of pre-aggregating the history table to the Product table before filtering: SELECT p.Name FROM ( SELECT th.ProductID, cnt = COUNT_BIG(*) FROM Production.TransactionHistory AS th GROUP BY th.ProductID ) AS q1 JOIN Production.Product AS p ON p.ProductID = q1.ProductID WHERE q1.cnt < 10; Perhaps a little surprisingly, we get a slightly different execution plan: The results are the same (23 rows) but this time the Filter is pushed below the join!  The optimizer chooses nested loops for the join, because the cardinality estimate for rows passing the Filter is a bit low (estimate 1 versus 23 actual), though you can force a merge join with a hint and the Filter still appears below the join.  In yet another variation, the < 10 predicate can be ‘manually pushed’ by specifying it in a HAVING clause in the “q1” sub-query instead of in the WHERE clause as written above. The reason this predicate can be pushed past the join in this query form, but not in the original formulation is simply an optimizer limitation – it does make efforts (primarily during the simplification phase) to encourage logically-equivalent query specifications to produce the same execution plan, but the implementation is not completely comprehensive. Moving on to a second example, the following query specification results from phrasing the requirement as “list the products where there exists fewer than ten correlated rows in the history table”: SELECT p.Name FROM Production.Product AS p WHERE EXISTS ( SELECT * FROM Production.TransactionHistory AS th WHERE th.ProductID = p.ProductID HAVING COUNT_BIG(*) < 10 ); Unfortunately, this query produces an incorrect result (86 rows): The problem is that it lists products with no history rows, though the reasons are interesting.  The COUNT_BIG(*) in the EXISTS clause is a scalar aggregate (meaning there is no GROUP BY clause) and scalar aggregates always produce a value, even when the input is an empty set.  In the case of the COUNT aggregate, the result of aggregating the empty set is zero (the other standard aggregates produce a NULL).  To make the point really clear, let’s look at product 709, which happens to be one for which no history rows exist: -- Scalar aggregate SELECT COUNT_BIG(*) FROM Production.TransactionHistory AS th WHERE th.ProductID = 709;   -- Vector aggregate SELECT COUNT_BIG(*) FROM Production.TransactionHistory AS th WHERE th.ProductID = 709 GROUP BY th.ProductID; The estimated execution plans for these two statements are almost identical: You might expect the Stream Aggregate to have a Group By for the second statement, but this is not the case.  The query includes an equality comparison to a constant value (709), so all qualified rows are guaranteed to have the same value for ProductID and the Group By is optimized away. In fact there are some minor differences between the two plans (the first is auto-parameterized and qualifies for trivial plan, whereas the second is not auto-parameterized and requires cost-based optimization), but there is nothing to indicate that one is a scalar aggregate and the other is a vector aggregate.  This is something I would like to see exposed in show plan so I suggested it on Connect.  Anyway, the results of running the two queries show the difference at runtime: The scalar aggregate (no GROUP BY) returns a result of zero, whereas the vector aggregate (with a GROUP BY clause) returns nothing at all.  Returning to our EXISTS query, we could ‘fix’ it by changing the HAVING clause to reject rows where the scalar aggregate returns zero: SELECT p.Name FROM Production.Product AS p WHERE EXISTS ( SELECT * FROM Production.TransactionHistory AS th WHERE th.ProductID = p.ProductID HAVING COUNT_BIG(*) BETWEEN 1 AND 9 ); The query now returns the correct 23 rows: Unfortunately, the execution plan is less efficient now – it has an estimated cost of 0.78 compared to 0.33 for the earlier plans.  Let’s try adding a redundant GROUP BY instead of changing the HAVING clause: SELECT p.Name FROM Production.Product AS p WHERE EXISTS ( SELECT * FROM Production.TransactionHistory AS th WHERE th.ProductID = p.ProductID GROUP BY th.ProductID HAVING COUNT_BIG(*) < 10 ); Not only do we now get correct results (23 rows), this is the execution plan: I like to compare that plan to quantum physics: if you don’t find it shocking, you haven’t understood it properly :)  The simple addition of a redundant GROUP BY has resulted in the EXISTS form of the query being transformed into exactly the same optimal plan we found earlier.  What’s more, in SQL Server 2008 and later, we can replace the odd-looking GROUP BY with an explicit GROUP BY on the empty set: SELECT p.Name FROM Production.Product AS p WHERE EXISTS ( SELECT * FROM Production.TransactionHistory AS th WHERE th.ProductID = p.ProductID GROUP BY () HAVING COUNT_BIG(*) < 10 ); I offer that as an alternative because some people find it more intuitive (and it perhaps has more geek value too).  Whichever way you prefer, it’s rather satisfying to note that the result of the sub-query does not exist for a particular correlated value where a vector aggregate is used (the scalar COUNT aggregate always returns a value, even if zero, so it always ‘EXISTS’ regardless which ProductID is logically being evaluated). The following query forms also produce the optimal plan and correct results, so long as a vector aggregate is used (you can probably find more equivalent query forms): WHERE Clause SELECT p.Name FROM Production.Product AS p WHERE ( SELECT COUNT_BIG(*) FROM Production.TransactionHistory AS th WHERE th.ProductID = p.ProductID GROUP BY () ) < 10; APPLY SELECT p.Name FROM Production.Product AS p CROSS APPLY ( SELECT NULL FROM Production.TransactionHistory AS th WHERE th.ProductID = p.ProductID GROUP BY () HAVING COUNT_BIG(*) < 10 ) AS ca (dummy); FROM Clause SELECT q1.Name FROM ( SELECT p.Name, cnt = ( SELECT COUNT_BIG(*) FROM Production.TransactionHistory AS th WHERE th.ProductID = p.ProductID GROUP BY () ) FROM Production.Product AS p ) AS q1 WHERE q1.cnt < 10; This last example uses SUM(1) instead of COUNT and does not require a vector aggregate…you should be able to work out why :) SELECT q.Name FROM ( SELECT p.Name, cnt = ( SELECT SUM(1) FROM Production.TransactionHistory AS th WHERE th.ProductID = p.ProductID ) FROM Production.Product AS p ) AS q WHERE q.cnt < 10; The semantics of SQL aggregates are rather odd in places.  It definitely pays to get to know the rules, and to be careful to check whether your queries are using scalar or vector aggregates.  As we have seen, query plans do not show in which ‘mode’ an aggregate is running and getting it wrong can cause poor performance, wrong results, or both. © 2012 Paul White Twitter: @SQL_Kiwi email: [email protected]

  Read the article

• Regular Expressions Cookbook Is in The Money—Win a Copy

  - by Jan Goyvaerts

  Regular Expressions Cookbook has sold enough copies to pay for the advance royalty Steven and I received. To celebrate we're giving away five copies of the book.

  Read the article

• What’s New in The Second Edition of Regular Expressions Cookbook

  - by Jan Goyvaerts

  %COOKBOOKFRAME% The second edition of Regular Expressions Cookbook is a completely revised edition, not just a minor update. All of the content from the first edition has been updated for the latest versions of the regular expression flavors and programming languages we discuss. We’ve corrected all errors that we could find and rewritten many sections that were either unclear or lacking in detail. And lack of detail was not something the first edition was accused of. Expect the second edition to really dot all i’s and cross all t’s. A few sections were removed. In particular, we removed much talk about browser inconsistencies as modern browsers are much more compatible with the official JavaScript standard. There is plenty of new content. The second edition has 101 more pages, bringing the total to 612. It’s almost 20% bigger than the first edition. We’ve added XRegExp as an additional regex flavor to all recipes throughout the book where XRegExp provides a better solution than standard JavaScript. We did keep the standard JavaScript solutions, so you can decide which is better for your needs. The new edition adds 21 recipes, bringing the total to 146. 14 of the new recipes are in the new Source Code and Log Files chapter. These recipes demonstrate techniques that are very useful for manipulating source code in a text editor and for dealing with log files using a grep tool. Chapter 3 which has recipes for programming with regular expressions gets only one new recipe, but it’s a doozy. If anyone has ever flamed you for using a regular expression instead of a parser, you’ll now be able to tell them how you can create your own parser by mixing regular expressions with procedural code. Combined with the recipes from the new Source Code and Log Files chapter, you can create parsers for whatever custom language or file format you like. If you have any interest in regular expressions at all, whether you’re a beginner or already consider yourself an expert, you definitely need a copy of the second edition of Regular Expressions Cookbook if you didn’t already buy the first. If you did buy the first edition, and you often find yourself referring back to it, then the second edition is a very worthwhile upgrade. You can buy the second edition of Regular Expressions Cookbook from Amazon or wherever technical books are sold. Ask for ISBN 1449319432.

  Read the article

• Heaps of Trouble?

  - by Paul White NZ

  If you’re not already a regular reader of Brad Schulz’s blog, you’re missing out on some great material.  In his latest entry, he is tasked with optimizing a query run against tables that have no indexes at all.  The problem is, predictably, that performance is not very good.  The catch is that we are not allowed to create any indexes (or even new statistics) as part of our optimization efforts. In this post, I’m going to look at the problem from a slightly different angle, and present an alternative solution to the one Brad found.  Inevitably, there’s going to be some overlap between our entries, and while you don’t necessarily need to read Brad’s post before this one, I do strongly recommend that you read it at some stage; he covers some important points that I won’t cover again here. The Example We’ll use data from the AdventureWorks database, copied to temporary unindexed tables.  A script to create these structures is shown below: CREATE TABLE #Custs ( CustomerID INTEGER NOT NULL, TerritoryID INTEGER NULL, CustomerType NCHAR(1) COLLATE SQL_Latin1_General_CP1_CI_AI NOT NULL, ); GO CREATE TABLE #Prods ( ProductMainID INTEGER NOT NULL, ProductSubID INTEGER NOT NULL, ProductSubSubID INTEGER NOT NULL, Name NVARCHAR(50) COLLATE SQL_Latin1_General_CP1_CI_AI NOT NULL, ); GO CREATE TABLE #OrdHeader ( SalesOrderID INTEGER NOT NULL, OrderDate DATETIME NOT NULL, SalesOrderNumber NVARCHAR(25) COLLATE SQL_Latin1_General_CP1_CI_AI NOT NULL, CustomerID INTEGER NOT NULL, ); GO CREATE TABLE #OrdDetail ( SalesOrderID INTEGER NOT NULL, OrderQty SMALLINT NOT NULL, LineTotal NUMERIC(38,6) NOT NULL, ProductMainID INTEGER NOT NULL, ProductSubID INTEGER NOT NULL, ProductSubSubID INTEGER NOT NULL, ); GO INSERT #Custs ( CustomerID, TerritoryID, CustomerType ) SELECT C.CustomerID, C.TerritoryID, C.CustomerType FROM AdventureWorks.Sales.Customer C WITH (TABLOCK); GO INSERT #Prods ( ProductMainID, ProductSubID, ProductSubSubID, Name ) SELECT P.ProductID, P.ProductID, P.ProductID, P.Name FROM AdventureWorks.Production.Product P WITH (TABLOCK); GO INSERT #OrdHeader ( SalesOrderID, OrderDate, SalesOrderNumber, CustomerID ) SELECT H.SalesOrderID, H.OrderDate, H.SalesOrderNumber, H.CustomerID FROM AdventureWorks.Sales.SalesOrderHeader H WITH (TABLOCK); GO INSERT #OrdDetail ( SalesOrderID, OrderQty, LineTotal, ProductMainID, ProductSubID, ProductSubSubID ) SELECT D.SalesOrderID, D.OrderQty, D.LineTotal, D.ProductID, D.ProductID, D.ProductID FROM AdventureWorks.Sales.SalesOrderDetail D WITH (TABLOCK); The query itself is a simple join of the four tables: SELECT P.ProductMainID AS PID, P.Name, D.OrderQty, H.SalesOrderNumber, H.OrderDate, C.TerritoryID FROM #Prods P JOIN #OrdDetail D ON P.ProductMainID = D.ProductMainID AND P.ProductSubID = D.ProductSubID AND P.ProductSubSubID = D.ProductSubSubID JOIN #OrdHeader H ON D.SalesOrderID = H.SalesOrderID JOIN #Custs C ON H.CustomerID = C.CustomerID ORDER BY P.ProductMainID ASC OPTION (RECOMPILE, MAXDOP 1); Remember that these tables have no indexes at all, and only the single-column sampled statistics SQL Server automatically creates (assuming default settings).  The estimated query plan produced for the test query looks like this (click to enlarge): The Problem The problem here is one of cardinality estimation – the number of rows SQL Server expects to find at each step of the plan.  The lack of indexes and useful statistical information means that SQL Server does not have the information it needs to make a good estimate.  Every join in the plan shown above estimates that it will produce just a single row as output.  Brad covers the factors that lead to the low estimates in his post. In reality, the join between the #Prods and #OrdDetail tables will produce 121,317 rows.  It should not surprise you that this has rather dire consequences for the remainder of the query plan.  In particular, it makes a nonsense of the optimizer’s decision to use Nested Loops to join to the two remaining tables.  Instead of scanning the #OrdHeader and #Custs tables once (as it expected), it has to perform 121,317 full scans of each.  The query takes somewhere in the region of twenty minutes to run to completion on my development machine. A Solution At this point, you may be thinking the same thing I was: if we really are stuck with no indexes, the best we can do is to use hash joins everywhere. We can force the exclusive use of hash joins in several ways, the two most common being join and query hints.  A join hint means writing the query using the INNER HASH JOIN syntax; using a query hint involves adding OPTION (HASH JOIN) at the bottom of the query.  The difference is that using join hints also forces the order of the join, whereas the query hint gives the optimizer freedom to reorder the joins at its discretion. Adding the OPTION (HASH JOIN) hint results in this estimated plan: That produces the correct output in around seven seconds, which is quite an improvement!  As a purely practical matter, and given the rigid rules of the environment we find ourselves in, we might leave things there.  (We can improve the hashing solution a bit – I’ll come back to that later on). Faster Nested Loops It might surprise you to hear that we can beat the performance of the hash join solution shown above using nested loops joins exclusively, and without breaking the rules we have been set. The key to this part is to realize that a condition like (A = B) can be expressed as (A <= B) AND (A >= B).  Armed with this tremendous new insight, we can rewrite the join predicates like so: SELECT P.ProductMainID AS PID, P.Name, D.OrderQty, H.SalesOrderNumber, H.OrderDate, C.TerritoryID FROM #OrdDetail D JOIN #OrdHeader H ON D.SalesOrderID >= H.SalesOrderID AND D.SalesOrderID <= H.SalesOrderID JOIN #Custs C ON H.CustomerID >= C.CustomerID AND H.CustomerID <= C.CustomerID JOIN #Prods P ON P.ProductMainID >= D.ProductMainID AND P.ProductMainID <= D.ProductMainID AND P.ProductSubID = D.ProductSubID AND P.ProductSubSubID = D.ProductSubSubID ORDER BY D.ProductMainID OPTION (RECOMPILE, LOOP JOIN, MAXDOP 1, FORCE ORDER); I’ve also added LOOP JOIN and FORCE ORDER query hints to ensure that only nested loops joins are used, and that the tables are joined in the order they appear.  The new estimated execution plan is: This new query runs in under 2 seconds. Why Is It Faster? The main reason for the improvement is the appearance of the eager Index Spools, which are also known as index-on-the-fly spools.  If you read my Inside The Optimiser series you might be interested to know that the rule responsible is called JoinToIndexOnTheFly. An eager index spool consumes all rows from the table it sits above, and builds a index suitable for the join to seek on.  Taking the index spool above the #Custs table as an example, it reads all the CustomerID and TerritoryID values with a single scan of the table, and builds an index keyed on CustomerID.  The term ‘eager’ means that the spool consumes all of its input rows when it starts up.  The index is built in a work table in tempdb, has no associated statistics, and only exists until the query finishes executing. The result is that each unindexed table is only scanned once, and just for the columns necessary to build the temporary index.  From that point on, every execution of the inner side of the join is answered by a seek on the temporary index – not the base table. A second optimization is that the sort on ProductMainID (required by the ORDER BY clause) is performed early, on just the rows coming from the #OrdDetail table.  The optimizer has a good estimate for the number of rows it needs to sort at that stage – it is just the cardinality of the table itself.  The accuracy of the estimate there is important because it helps determine the memory grant given to the sort operation.  Nested loops join preserves the order of rows on its outer input, so sorting early is safe.  (Hash joins do not preserve order in this way, of course). The extra lazy spool on the #Prods branch is a further optimization that avoids executing the seek on the temporary index if the value being joined (the ‘outer reference’) hasn’t changed from the last row received on the outer input.  It takes advantage of the fact that rows are still sorted on ProductMainID, so if duplicates exist, they will arrive at the join operator one after the other. The optimizer is quite conservative about introducing index spools into a plan, because creating and dropping a temporary index is a relatively expensive operation.  It’s presence in a plan is often an indication that a useful index is missing. I want to stress that I rewrote the query in this way primarily as an educational exercise – I can’t imagine having to do something so horrible to a production system. Improving the Hash Join I promised I would return to the solution that uses hash joins.  You might be puzzled that SQL Server can create three new indexes (and perform all those nested loops iterations) faster than it can perform three hash joins.  The answer, again, is down to the poor information available to the optimizer.  Let’s look at the hash join plan again: Two of the hash joins have single-row estimates on their build inputs.  SQL Server fixes the amount of memory available for the hash table based on this cardinality estimate, so at run time the hash join very quickly runs out of memory. This results in the join spilling hash buckets to disk, and any rows from the probe input that hash to the spilled buckets also get written to disk.  The join process then continues, and may again run out of memory.  This is a recursive process, which may eventually result in SQL Server resorting to a bailout join algorithm, which is guaranteed to complete eventually, but may be very slow.  The data sizes in the example tables are not large enough to force a hash bailout, but it does result in multiple levels of hash recursion.  You can see this for yourself by tracing the Hash Warning event using the Profiler tool. The final sort in the plan also suffers from a similar problem: it receives very little memory and has to perform multiple sort passes, saving intermediate runs to disk (the Sort Warnings Profiler event can be used to confirm this).  Notice also that because hash joins don’t preserve sort order, the sort cannot be pushed down the plan toward the #OrdDetail table, as in the nested loops plan. Ok, so now we understand the problems, what can we do to fix it?  We can address the hash spilling by forcing a different order for the joins: SELECT P.ProductMainID AS PID, P.Name, D.OrderQty, H.SalesOrderNumber, H.OrderDate, C.TerritoryID FROM #Prods P JOIN #Custs C JOIN #OrdHeader H ON H.CustomerID = C.CustomerID JOIN #OrdDetail D ON D.SalesOrderID = H.SalesOrderID ON P.ProductMainID = D.ProductMainID AND P.ProductSubID = D.ProductSubID AND P.ProductSubSubID = D.ProductSubSubID ORDER BY D.ProductMainID OPTION (MAXDOP 1, HASH JOIN, FORCE ORDER); With this plan, each of the inputs to the hash joins has a good estimate, and no hash recursion occurs.  The final sort still suffers from the one-row estimate problem, and we get a single-pass sort warning as it writes rows to disk.  Even so, the query runs to completion in three or four seconds.  That’s around half the time of the previous hashing solution, but still not as fast as the nested loops trickery. Final Thoughts SQL Server’s optimizer makes cost-based decisions, so it is vital to provide it with accurate information.  We can’t really blame the performance problems highlighted here on anything other than the decision to use completely unindexed tables, and not to allow the creation of additional statistics. I should probably stress that the nested loops solution shown above is not one I would normally contemplate in the real world.  It’s there primarily for its educational and entertainment value.  I might perhaps use it to demonstrate to the sceptical that SQL Server itself is crying out for an index. Be sure to read Brad’s original post for more details.  My grateful thanks to him for granting permission to reuse some of his material. Paul White Email: [email protected] Twitter: @PaulWhiteNZ

  Read the article

• Replacement Text Syntax for JavaScript’s String.replace()

  - by Jan Goyvaerts

  A RegexBuddy user told me that he couldn’t easily find a detailed explanation of the replacement text syntax supported by the String.replace() function in JavaScript. I had to admin that my own web page about JavaScript’s regular expression support was also lacking. I’ve now added a new Replacement Syntax section that has all the details. I’ll summarize it here: $1: Text matched by the first capturing group or the literal text $1 if the regex has no capturing groups. $99: Text matched by the 99th capturing group if the regex has 99 or more groups. Text matched by the 9th capturing group followed by a literal 9 if the regex has 9 or more but less than 99 groups. The literal text $99 if the regex has fewer than 9 groups. $+: Text matched by the highest-numbered capturing group. Replaced with nothing if the highest-numbered group didn’t participate in the match. $&: Text matched by the entire regex. You cannot use $0 for this. $` (backtick): Text to the left of the regex match. $' (single quote): Text to the right of the regex match. $_: The entire subject string.

  Read the article

< Previous Page | 6 7 8 9 10 11 12 13 14 15 16 17  | Next Page >