Search Results

Search found 415 results on 17 pages for 'predicate'.

Page 16/17 | < Previous Page | 12 13 14 15 16 17  | Next Page >

• ROracle support for TimesTen In-Memory Database

  - by Sherry LaMonica

  Today's guest post comes from Jason Feldhaus, a Consulting Member of Technical Staff in the TimesTen Database organization at Oracle.  He shares with us a sample session using ROracle with the TimesTen In-Memory database.  Beginning in version 1.1-4, ROracle includes support for the Oracle Times Ten In-Memory Database, version 11.2.2. TimesTen is a relational database providing very fast and high throughput through its memory-centric architecture.  TimesTen is designed for low latency, high-volume data, and event and transaction management. A TimesTen database resides entirely in memory, so no disk I/O is required for transactions and query operations. TimesTen is used in applications requiring very fast and predictable response time, such as real-time financial services trading applications and large web applications. TimesTen can be used as the database of record or as a relational cache database to Oracle Database. ROracle provides an interface between R and the database, providing the rich functionality of the R statistical programming environment using the SQL query language. ROracle uses the OCI libraries to handle database connections, providing much better performance than standard ODBC.The latest ROracle enhancements include: Support for Oracle TimesTen In-Memory Database Support for Date-Time using R's POSIXct/POSIXlt data types RAW, BLOB and BFILE data type support Option to specify number of rows per fetch operation Option to prefetch LOB data Break support using Ctrl-C Statement caching support Times Ten 11.2.2 contains enhanced support for analytics workloads and complex queries: Analytic functions: AVG, SUM, COUNT, MAX, MIN, DENSE_RANK, RANK, ROW_NUMBER, FIRST_VALUE and LAST_VALUE Analytic clauses: OVER PARTITION BY and OVER ORDER BY Multidimensional grouping operators: Grouping clauses: GROUP BY CUBE, GROUP BY ROLLUP, GROUP BY GROUPING SETS Grouping functions: GROUP, GROUPING_ID, GROUP_ID WITH clause, which allows repeated references to a named subquery block Aggregate expressions over DISTINCT expressions General expressions that return a character string in the source or a pattern within the LIKE predicate Ability to order nulls first or last in a sort result (NULLS FIRST or NULLS LAST in the ORDER BY clause) Note: Some functionality is only available with Oracle Exalytics, refer to the TimesTen product licensing document for details. Connecting to TimesTen is easy with ROracle. Simply install and load the ROracle package and load the driver. > install.packages("ROracle") > library(ROracle) Loading required package: DBI > drv <- dbDriver("Oracle") Once the ROracle package is installed, create a database connection object and connect to a TimesTen direct driver DSN as the OS user. > conn <- dbConnect(drv, username ="", password="", dbname = "localhost/SampleDb_1122:timesten_direct") You have the option to report the server type - Oracle or TimesTen? > print (paste ("Server type =", dbGetInfo (conn)$serverType)) [1] "Server type = TimesTen IMDB" To create tables in the database using R data frame objects, use the function dbWriteTable. In the following example we write the built-in iris data frame to TimesTen. The iris data set is a small example data set containing 150 rows and 5 columns. We include it here not to highlight performance, but so users can easily run this example in their R session. > dbWriteTable (conn, "IRIS", iris, overwrite=TRUE, ora.number=FALSE) [1] TRUE Verify that the newly created IRIS table is available in the database. To list the available tables and table columns in the database, use dbListTables and dbListFields, respectively. > dbListTables (conn) [1] "IRIS" > dbListFields (conn, "IRIS") [1] "SEPAL.LENGTH" "SEPAL.WIDTH" "PETAL.LENGTH" "PETAL.WIDTH" "SPECIES" To retrieve a summary of the data from the database we need to save the results to a local object. The following call saves the results of the query as a local R object, iris.summary. The ROracle function dbGetQuery is used to execute an arbitrary SQL statement against the database. When connected to TimesTen, the SQL statement is processed completely within main memory for the fastest response time. > iris.summary <- dbGetQuery(conn, 'SELECT SPECIES, AVG ("SEPAL.LENGTH") AS AVG_SLENGTH, AVG ("SEPAL.WIDTH") AS AVG_SWIDTH, AVG ("PETAL.LENGTH") AS AVG_PLENGTH, AVG ("PETAL.WIDTH") AS AVG_PWIDTH FROM IRIS GROUP BY ROLLUP (SPECIES)') > iris.summary SPECIES AVG_SLENGTH AVG_SWIDTH AVG_PLENGTH AVG_PWIDTH 1 setosa 5.006000 3.428000 1.462 0.246000 2 versicolor 5.936000 2.770000 4.260 1.326000 3 virginica 6.588000 2.974000 5.552 2.026000 4 <NA> 5.843333 3.057333 3.758 1.199333 Finally, disconnect from the TimesTen Database. > dbCommit (conn) [1] TRUE > dbDisconnect (conn) [1] TRUE We encourage you download Oracle software for evaluation from the Oracle Technology Network. See these links for our software: Times Ten In-Memory Database,  ROracle.  As always, we welcome comments and questions on the TimesTen and  Oracle R technical forums.

  Read the article

• Objective-C Out of scope problem

  - by davbryn

  Hi, I'm having a few problems with some Objective-C and would appreciate some pointers. So I have a class MapFileGroup which has the following simple interface (There are other member variables but they aren't important): @interface MapFileGroup : NSObject { NSMutableArray *mapArray; } @property (nonatomic, retain) NSMutableArray *mapArray; mapArray is @synthesize'd in the .m file. It has an init method: -(MapFileGroup*) init { self = [super init]; if (self) { mapArray = [NSMutableArray arrayWithCapacity: 10]; } return self; } It also has a method for adding a custom object to the array: -(BOOL) addMapFile:(MapFile*) mapfile { if (mapfile == nil) return NO; mapArray addObject:mapfile]; return YES; } The problem I get comes when I want to use this class - obviously due to a misunderstanding of memory management on my part. In my view controller I declare as follows: (in the @interface): MapFileGroup *fullGroupOfMaps; With @property @property (nonatomic, retain) MapFileGroup *fullGroupOfMaps; Then in the .m file I have a function called loadMapData that does the following: MapFileGroup *mapContainer = [[MapFileGroup alloc] init]; // create a predicate that we can use to filter an array // for all strings ending in .png (case insensitive) NSPredicate *caseInsensitivePNGFiles = [NSPredicate predicateWithFormat:@"SELF endswith[c] '.png'"]; mapNames = [unfilteredArray filteredArrayUsingPredicate:caseInsensitivePNGFiles]; [mapNames retain]; NSEnumerator * enumerator = [mapNames objectEnumerator]; NSString * currentFileName; NSString *nameOfMap; MapFile *mapfile; while(currentFileName = [enumerator nextObject]) { nameOfMap = [currentFileName substringToIndex:[currentFileName length]-4]; //strip the extension mapfile = [[MapFile alloc] initWithName:nameOfMap]; [mapfile retain]; // add to array [fullGroupOfMaps addMapFile:mapfile]; } This seems to work ok (Though I can tell I've not got the memory management working properly, I'm still learning Objective-C); however, I have an (IBAction) that interacts with the fullGroupOfMaps later. It calls a method within fullGroupOfMaps, but if I step into the class from that line while debugging, all fullGroupOfMaps's objects are now out of scope and I get a crash. So apologies for the long question and big amount of code, but I guess my main question it: How should I handle a class with an NSMutableArray as an instance variable? What is the proper way of creating objects to be added to the class so that they don't get freed before I'm done with them? Many thanks

  Read the article

• IXmlSerializable Dictionary

  - by Shimmy

  I was trying to create a generic Dictionary that implements IXmlSerializable. Here is my trial: Sub Main() Dim z As New SerializableDictionary(Of String, String) z.Add("asdf", "asd") Console.WriteLine(z.Serialize) End Sub Result: <?xml version="1.0" encoding="utf-16"?><Entry key="asdf" value="asd" /> I placed a breakpoint on top of the WriteXml method and I see that when it stops, the writer contains no data at all, and IMHO it should contain the root element and the xml declaration. <Serializable()> _ Public Class SerializableDictionary(Of TKey, TValue) : Inherits Dictionary(Of TKey, TValue) : Implements IXmlSerializable Private Const EntryString As String = "Entry" Private Const KeyString As String = "key" Private Const ValueString As String = "value" Private Shared ReadOnly AttributableTypes As Type() = New Type() {GetType(Boolean), GetType(Byte), GetType(Char), GetType(DateTime), GetType(Decimal), GetType(Double), GetType([Enum]), GetType(Guid), GetType(Int16), GetType(Int32), GetType(Int64), GetType(SByte), GetType(Single), GetType(String), GetType(TimeSpan), GetType(UInt16), GetType(UInt32), GetType(UInt64)} Private Shared ReadOnly GetIsAttributable As Predicate(Of Type) = Function(t) AttributableTypes.Contains(t) Private Shared ReadOnly IsKeyAttributable As Boolean = GetIsAttributable(GetType(TKey)) Private Shared ReadOnly IsValueAttributable As Boolean = GetIsAttributable(GetType(TValue)) Private Shared ReadOnly GetElementName As Func(Of Boolean, String) = Function(isKey) If(isKey, KeyString, ValueString) Public Function GetSchema() As System.Xml.Schema.XmlSchema Implements System.Xml.Serialization.IXmlSerializable.GetSchema Return Nothing End Function Public Sub WriteXml(ByVal writer As XmlWriter) Implements IXmlSerializable.WriteXml For Each entry In Me writer.WriteStartElement(EntryString) WriteData(IsKeyAttributable, writer, True, entry.Key) WriteData(IsValueAttributable, writer, False, entry.Value) writer.WriteEndElement() Next End Sub Private Sub WriteData(Of T)(ByVal attributable As Boolean, ByVal writer As XmlWriter, ByVal isKey As Boolean, ByVal value As T) Dim name = GetElementName(isKey) If attributable Then writer.WriteAttributeString(name, value.ToString) Else Dim serializer As New XmlSerializer(GetType(T)) writer.WriteStartElement(name) serializer.Serialize(writer, value) writer.WriteEndElement() End If End Sub Public Sub ReadXml(ByVal reader As XmlReader) Implements IXmlSerializable.ReadXml Dim empty = reader.IsEmptyElement reader.Read() If empty Then Exit Sub Clear() While reader.NodeType <> XmlNodeType.EndElement While reader.NodeType = XmlNodeType.Whitespace reader.Read() Dim key = ReadData(Of TKey)(IsKeyAttributable, reader, True) Dim value = ReadData(Of TValue)(IsValueAttributable, reader, False) Add(key, value) If Not IsKeyAttributable AndAlso Not IsValueAttributable Then reader.ReadEndElement() Else reader.Read() While reader.NodeType = XmlNodeType.Whitespace reader.Read() End While End While reader.ReadEndElement() End While End Sub Private Function ReadData(Of T)(ByVal attributable As Boolean, ByVal reader As XmlReader, ByVal isKey As Boolean) As T Dim name = GetElementName(isKey) Dim type = GetType(T) If attributable Then Return Convert.ChangeType(reader.GetAttribute(name), type) Else Dim serializer As New XmlSerializer(type) While reader.Name <> name reader.Read() End While reader.ReadStartElement(name) Dim value = serializer.Deserialize(reader) reader.ReadEndElement() Return value End If End Function Public Shared Function Serialize(ByVal dictionary As SerializableDictionary(Of TKey, TValue)) As String Dim sb As New StringBuilder(1024) Dim sw As New StringWriter(sb) Dim xs As New XmlSerializer(GetType(SerializableDictionary(Of TKey, TValue))) xs.Serialize(sw, dictionary) sw.Dispose() Return sb.ToString End Function Public Shared Function Deserialize(ByVal xml As String) As SerializableDictionary(Of TKey, TValue) Dim xs As New XmlSerializer(GetType(SerializableDictionary(Of TKey, TValue))) Dim xr As New XmlTextReader(xml, XmlNodeType.Document, Nothing) Return xs.Deserialize(xr) xr.Close() End Function Public Function Serialize() As String Dim sb As New StringBuilder Dim xw = XmlWriter.Create(sb) WriteXml(xw) xw.Close() Return sb.ToString End Function Public Sub Parse(ByVal xml As String) Dim xr As New XmlTextReader(xml, XmlNodeType.Document, Nothing) ReadXml(xr) xr.Close() End Sub End Class

  Read the article

• IXmlSerializable Dictionary problem

  - by Shimmy

  I was trying to create a generic Dictionary that implements IXmlSerializable. Here is my trial: Sub Main() Dim z As New SerializableDictionary(Of String, String) z.Add("asdf", "asd") Console.WriteLine(z.Serialize) End Sub Result: <?xml version="1.0" encoding="utf-16"?><Entry key="asdf" value="asd" /> I placed a breakpoint on top of the WriteXml method and I see that when it stops, the writer contains no data at all, and IMHO it should contain the root element and the xml declaration. <Serializable()> _ Public Class SerializableDictionary(Of TKey, TValue) : Inherits Dictionary(Of TKey, TValue) : Implements IXmlSerializable Private Const EntryString As String = "Entry" Private Const KeyString As String = "key" Private Const ValueString As String = "value" Private Shared ReadOnly AttributableTypes As Type() = New Type() {GetType(Boolean), GetType(Byte), GetType(Char), GetType(DateTime), GetType(Decimal), GetType(Double), GetType([Enum]), GetType(Guid), GetType(Int16), GetType(Int32), GetType(Int64), GetType(SByte), GetType(Single), GetType(String), GetType(TimeSpan), GetType(UInt16), GetType(UInt32), GetType(UInt64)} Private Shared ReadOnly GetIsAttributable As Predicate(Of Type) = Function(t) AttributableTypes.Contains(t) Private Shared ReadOnly IsKeyAttributable As Boolean = GetIsAttributable(GetType(TKey)) Private Shared ReadOnly IsValueAttributable As Boolean = GetIsAttributable(GetType(TValue)) Private Shared ReadOnly GetElementName As Func(Of Boolean, String) = Function(isKey) If(isKey, KeyString, ValueString) Public Function GetSchema() As System.Xml.Schema.XmlSchema Implements System.Xml.Serialization.IXmlSerializable.GetSchema Return Nothing End Function Public Sub WriteXml(ByVal writer As XmlWriter) Implements IXmlSerializable.WriteXml For Each entry In Me writer.WriteStartElement(EntryString) WriteData(IsKeyAttributable, writer, True, entry.Key) WriteData(IsValueAttributable, writer, False, entry.Value) writer.WriteEndElement() Next End Sub Private Sub WriteData(Of T)(ByVal attributable As Boolean, ByVal writer As XmlWriter, ByVal isKey As Boolean, ByVal value As T) Dim name = GetElementName(isKey) If attributable Then writer.WriteAttributeString(name, value.ToString) Else Dim serializer As New XmlSerializer(GetType(T)) writer.WriteStartElement(name) serializer.Serialize(writer, value) writer.WriteEndElement() End If End Sub Public Sub ReadXml(ByVal reader As XmlReader) Implements IXmlSerializable.ReadXml Dim empty = reader.IsEmptyElement reader.Read() If empty Then Exit Sub Clear() While reader.NodeType <> XmlNodeType.EndElement While reader.NodeType = XmlNodeType.Whitespace reader.Read() Dim key = ReadData(Of TKey)(IsKeyAttributable, reader, True) Dim value = ReadData(Of TValue)(IsValueAttributable, reader, False) Add(key, value) If Not IsKeyAttributable AndAlso Not IsValueAttributable Then reader.ReadEndElement() Else reader.Read() While reader.NodeType = XmlNodeType.Whitespace reader.Read() End While End While reader.ReadEndElement() End While End Sub Private Function ReadData(Of T)(ByVal attributable As Boolean, ByVal reader As XmlReader, ByVal isKey As Boolean) As T Dim name = GetElementName(isKey) Dim type = GetType(T) If attributable Then Return Convert.ChangeType(reader.GetAttribute(name), type) Else Dim serializer As New XmlSerializer(type) While reader.Name <> name reader.Read() End While reader.ReadStartElement(name) Dim value = serializer.Deserialize(reader) reader.ReadEndElement() Return value End If End Function Public Shared Function Serialize(ByVal dictionary As SerializableDictionary(Of TKey, TValue)) As String Dim sb As New StringBuilder(1024) Dim sw As New StringWriter(sb) Dim xs As New XmlSerializer(GetType(SerializableDictionary(Of TKey, TValue))) xs.Serialize(sw, dictionary) sw.Dispose() Return sb.ToString End Function Public Shared Function Deserialize(ByVal xml As String) As SerializableDictionary(Of TKey, TValue) Dim xs As New XmlSerializer(GetType(SerializableDictionary(Of TKey, TValue))) Dim xr As New XmlTextReader(xml, XmlNodeType.Document, Nothing) Return xs.Deserialize(xr) xr.Close() End Function Public Function Serialize() As String Dim sb As New StringBuilder Dim xw = XmlWriter.Create(sb) WriteXml(xw) xw.Close() Return sb.ToString End Function Public Sub Parse(ByVal xml As String) Dim xr As New XmlTextReader(xml, XmlNodeType.Document, Nothing) ReadXml(xr) xr.Close() End Sub End Class

  Read the article

• How to optimize Core Data query for full text search

  - by dk

  Can I optimize a Core Data query when searching for matching words in a text? (This question also pertains to the wisdom of custom SQL versus Core Data on an iPhone.) I'm working on a new (iPhone) app that is a handheld reference tool for a scientific database. The main interface is a standard searchable table view and I want as-you-type response as the user types new words. Words matches must be prefixes of words in the text. The text is composed of 100,000s of words. In my prototype I coded SQL directly. I created a separate "words" table containing every word in the text fields of the main entity. I indexed words and performed searches along the lines of SELECT id, * FROM textTable JOIN (SELECT DISTINCT textTableId FROM words WHERE word BETWEEN 'foo' AND 'fooz' ) ON id=textTableId LIMIT 50 This runs very fast. Using an IN would probably work just as well, i.e. SELECT * FROM textTable WHERE id IN (SELECT textTableId FROM words WHERE word BETWEEN 'foo' AND 'fooz' ) LIMIT 50 The LIMIT is crucial and allows me to display results quickly. I notify the user that there are too many to display if the limit is reached. This is kludgy. I've spent the last several days pondering the advantages of moving to Core Data, but I worry about the lack of control in the schema, indexing, and querying for an important query. Theoretically an NSPredicate of textField MATCHES '.*\bfoo.*' would just work, but I'm sure it will be slow. This sort of text search seems so common that I wonder what is the usual attack? Would you create a words entity as I did above and use a predicate of "word BEGINSWITH 'foo'"? Will that work as fast as my prototype? Will Core Data automatically create the right indexes? I can't find any explicit means of advising the persistent store about indexes. I see some nice advantages of Core Data in my iPhone app. The faulting and other memory considerations allow for efficient database retrievals for tableview queries without setting arbitrary limits. The object graph management allows me to easily traverse entities without writing lots of SQL. Migration features will be nice in the future. On the other hand, in a limited resource environment (iPhone) I worry that an automatically generated database will be bloated with metadata, unnecessary inverse relationships, inefficient attribute datatypes, etc. Should I dive in or proceed with caution?

  Read the article

• Serializing Class Derived from Generic Collection yet Deserializing the Generic Collection

  - by Stacey

  I have a Repository Class with the following method... public T Single<T>(Predicate<T> expression) { using (var list = (Models.Collectable<T>)System.Xml.Serializer.Deserialize(typeof(Models.Collectable<T>), FileName)) { return list.Find(expression); } } Where Collectable is defined.. [Serializable] public class Collectable<T> : List<T>, IDisposable { public Collectable() { } public void Dispose() { } } And an Item that uses it is defined.. [Serializable] [System.Xml.Serialization.XmlRoot("Titles")] public partial class Titles : Collectable<Title> { } The problem is when I call the method, it expects "Collectable" to be the XmlRoot, but the XmlRoot is "Titles" (all of object Title). I have several classes that are collected in .xml files like this, but it seems pointless to rewrite the basic methods for loading each up when the generic accessors do it - but how can I enforce the proper root name for each file without hard coding methods for each one? The [System.Xml.Serialization.XmlRoot] seems to be ignored. When called like this... var titles = Repository.List<Models.Title>(); I get the exception <Titlesxmlns=''> was not expected. The XML is formatted such as. .. <?xml version="1.0" encoding="utf-16"?> <Titles xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema"> <Title> <Id>442daf7d-193c-4da8-be0b-417cec9dc1c5</Id> </Title> </Titles> Here is the deserialization code. public static T Deserialize<T>(String xmlString) { System.Xml.Serialization.XmlSerializer XmlFormatSerializer = new System.Xml.Serialization.XmlSerializer(typeof(T)); StreamReader XmlStringReader = new StreamReader(xmlString); //XmlTextReader XmlFormatReader = new XmlTextReader(XmlStringReader); try { return (T)XmlFormatSerializer.Deserialize(XmlStringReader); } catch (Exception e) { throw e; } finally { XmlStringReader.Close(); } }

  Read the article

• Dynamic Linq help, different errors depending on object passed as parameter?

  - by sah302

  I have an entityDao that is inherbited by everyone of my objectDaos. I am using Dynamic Linq and trying to get some generic queries to work. I have the following code in my generic method in my EntityDao : public abstract class EntityDao<ImplementationType> where ImplementationType : Entity { public ImplementationType getOneByValueOfProperty(string getProperty, object getValue){ ImplementationType entity = null; if (getProperty != null && getValue != null) { LCFDataContext lcfdatacontext = new LCFDataContext(); //Generic LINQ Query Here entity = lcfdatacontext.GetTable<ImplementationType>().Where(getProperty + " =@0", getValue).FirstOrDefault(); //.Where(getProperty & "==" & CStr(getValue)) } //lcfdatacontext.SubmitChanges() //lcfdatacontext.Dispose() return entity; } }         Then I do the following method call in a unit test (all my objectDaos inherit entityDao): [Test] public void getOneByValueOfProperty() { Accomplishment result = accomplishmentDao.getOneByValueOfProperty("AccomplishmentType.Name", "Publication"); Assert.IsNotNull(result); } The above passes (AccomplishmentType has a relationship to accomplishment) Accomplishment result = accomplishmentDao.getOneByValueOfProperty("Description", "Can you hear me now?"); Accomplishment result = accomplishmentDao.getOneByValueOfProperty("LocalId", 4); Both of the above work Accomplishment result = accomplishmentDao.getOneByValueOfProperty("Id", New Guid("95457751-97d9-44b5-8f80-59fc2d170a4c"))       Does not work and says the following: Operator '=' incompatible with operand types 'Guid' and 'Guid Why is this happening? Guid's can't be compared? I tried == as well but same error. What's even moreso confusing is that every example of Dynamic Linq I have seen simply usings strings whether using the parameterized where predicate or this one I have commented out: //.Where(getProperty & "==" & CStr(getValue)) With or without the Cstr, many datatypes don't work with this format. I tried setting the getValue to a string instead of an object as well, but then I just get different errors (such as a multiword string would stop comparison after the first word). What am I missing to make this work with GUIDs and/or any data type? Ideally I would like to be able to just pass in a string for getValue (as I have seen for every other dynamic LINQ example) instead of the object and have it work regardless of the data Type of the column.

  Read the article

• whats wrong in this LINQ synatx?

  - by Saurabh Kumar

  Hi, I am trying to convert a SQL query to LINQ. Somehow my count(distinct(x)) logic does not seem to be working correctly. The original SQL is quite efficient(or so i think), but the generated SQL is not even returning the correct result. I am trying to fix this LINQ to do what the original SQL is doing, AND in an efficient way as the original query is doing. Help here would be really apreciated as I am stuck here :( SQL which is working and I need to make a comparable LINQ of: SELECT [t1].[PersonID] AS [personid] FROM [dbo].[Code] AS [t0] INNER JOIN [dbo].[phonenumbers] AS [t1] ON [t1].[PhoneCode] = [t0].[Code] INNER JOIN [dbo].[person] ON [t1].[PersonID]= [dbo].[Person].PersonID WHERE ([t0].[codetype] = 'phone') AND ( ([t0].[CodeDescription] = 'Home') AND ([t1].[PhoneNum] = '111') OR ([t0].[CodeDescription] = 'Work') AND ([t1].[PhoneNum] = '222') ) GROUP BY [t1].[PersonID] HAVING COUNT(DISTINCT([t1].[PhoneNum]))=2 The LINQ which I made is approximately as below: var ids = context.Code.Where(predicate); var rs = from r in ids group r by new { r.phonenumbers.person.PersonID} into g let matchcount=g.Select(p => p.phonenumbers.PhoneNum).Distinct().Count() where matchcount ==2 select new { personid = g.Key }; Unfortunately, the above LINQ is NOT generating the correct result, and is actually internally getting generated to the SQL shown below. By the way, this generated query is also reading ALL the rows(about 19592040) around 2 times due to the COUNTS :( Wich is a big performance issue too. Please help/point me to the right direction. Declare @p0 VarChar(10)='phone' Declare @p1 VarChar(10)='Home' Declare @p2 VarChar(10)='111' Declare @p3 VarChar(10)='Work' Declare @p4 VarChar(10)='222' Declare @p5 VarChar(10)='2' SELECT [t9].[PersonID], ( SELECT COUNT(*) FROM ( SELECT DISTINCT [t13].[PhoneNum] FROM [dbo].[Code] AS [t10] INNER JOIN [dbo].[phonenumbers] AS [t11] ON [t11].[PhoneType] = [t10].[Code] INNER JOIN [dbo].[Person] AS [t12] ON [t12].[PersonID] = [t11].[PersonID] INNER JOIN [dbo].[phonenumbers] AS [t13] ON [t13].[PhoneType] = [t10].[Code] WHERE ([t9].[PersonID] = [t12].[PersonID]) AND ([t10].[codetype] = @p0) AND ((([t10].[codetype] = @p1) AND ([t11].[PhoneNum] = @p2)) OR (([t10].[codetype] = @p3) AND ([t11].[PhoneNum] = @p4))) ) AS [t14] ) AS [cnt] FROM ( SELECT [t3].[PersonID], ( SELECT COUNT(*) FROM ( SELECT DISTINCT [t7].[PhoneNum] FROM [dbo].[Code] AS [t4] INNER JOIN [dbo].[phonenumbers] AS [t5] ON [t5].[PhoneType] = [t4].[Code] INNER JOIN [dbo].[Person] AS [t6] ON [t6].[PersonID] = [t5].[PersonID] INNER JOIN [dbo].[phonenumbers] AS [t7] ON [t7].[PhoneType] = [t4].[Code] WHERE ([t3].[PersonID] = [t6].[PersonID]) AND ([t4].[codetype] = @p0) AND ((([t4].[codetype] = @p1) AND ([t5].[PhoneNum] = @p2)) OR (([t4].[codetype] = @p3) AND ([t5].[PhoneNum] = @p4))) ) AS [t8] ) AS [value] FROM ( SELECT [t2].[PersonID] FROM [dbo].[Code] AS [t0] INNER JOIN [dbo].[phonenumbers] AS [t1] ON [t1].[PhoneType] = [t0].[Code] INNER JOIN [dbo].[Person] AS [t2] ON [t2].[PersonID] = [t1].[PersonID] WHERE ([t0].[codetype] = @p0) AND ((([t0].[codetype] = @p1) AND ([t1].[PhoneNum] = @p2)) OR (([t0].[codetype] = @p3) AND ([t1].[PhoneNum] = @p4))) GROUP BY [t2].[PersonID] ) AS [t3] ) AS [t9] WHERE [t9].[value] = @p5 Thanks!

  Read the article

• .NET: Is there a way to finagle a default namespace in an XPath 1.0 query?

  - by Cheeso

  I'm building a tool that performs xpath 1.0 queries on XHTML documents. The requirement to use a namespace prefix in the query is killing me. The query looks like this: html/body/div[@class='contents']/div[@class='body']/ div[@class='pgdbbyauthor']/h2[a[@name][starts-with(.,'Quick')]]/ following-sibling::ul[1]/li/a (all on one line) ...which is bad enough, except because it's xpath 1.0, I need to use an explicit namespace prefix on each QName, so it looks like this: ns1:html/ns1:body/ns1:div[@class='contents']/ns1:div[@class='body']/ ns1:div[@class='pgdbbyauthor']/ns1:h2[ns1:a[@name][starts-with(.,'Quick')]]/ following-sibling::ns1:ul[1]/ns1:li/ns1:a To set up the query, I do something like this: var xpathDoc = new XPathDocument(new StringReader(theText)); var nav = xpathDoc.CreateNavigator(); var xmlns = new XmlNamespaceManager(nav.NameTable); foreach (string prefix in xmlNamespaces.Keys) xmlns.AddNamespace(prefix, xmlNamespaces[prefix]); XPathNodeIterator selection = nav.Select(xpathExpression, xmlns); But what I want is for the xpathExpression to use the implicit default namespace. Is there a way for me to transform the unadorned xpath expression, after it's been written, to inject a namespace prefix for each element name in the query? I'm thinking, anything between two slashes, I could inject a prefix there. Excepting of course axis names like "parent::" and "preceding-sibling::" . And wildcards. That's what I mean by "finagle a default namespace". Is this hack gonna work? Addendum Here's what I mean. suppose I have an xpath expression, and before passing it to nav.Select(), I transform it. Something like this: string FixupWithDefaultNamespace(string expr) { string s = expr; s = Regex.Replace(s, "^(?!::)([^/:]+)(?=/)", "ns1:$1"); // beginning s = Regex.Replace(s, "/([^/:]+)(?=/)", "/ns1:$1"); // stanza s = Regex.Replace(s, "::([A-Za-z][^/:*]*)(?=/)", "::ns1:$1"); // axis specifier s = Regex.Replace(s, "\\[([A-Za-z][^/:*\\(]*)(?=[\\[\\]])", "[ns1:$1"); // predicate s = Regex.Replace(s, "/([A-Za-z][^/:]*)(?!<::)$", "/ns1:$1"); // end s = Regex.Replace(s, "^([A-Za-z][^/:]*)$", "ns1:$1"); // edge case s = Regex.Replace(s, "([-A-Za-z]+)\\(([^/:\\.,\\)]+)(?=[,\\)])", "$1(ns1:$2"); // xpath functions return s; } This actually works for simple cases I tried. To use the example from above - if the input is the first xpath expression, the output I get is the 2nd one, with all the ns1 prefixes. The real question is, is it hopeless to expect this Regex.Replace approach to work, as the xpath expressions get more complicated?

  Read the article

• What to Expect in Rails 4

  - by mikhailov

  Rails 4 is nearly there, we should be ready before it released. Most developers are trying hard to keep their application on the edge. Must see resources: 1) @sikachu talk: What to Expect in Rails 4.0 - YouTube 2) Rails Guides release notes: http://edgeguides.rubyonrails.org/4_0_release_notes.html There is a mix of all major changes down here: ActionMailer changes excerpt: Asynchronously send messages via the Rails Raise an ActionView::MissingTemplate exception when no implicit template could be found ActionPack changes excerpt Added controller-level etag additions that will be part of the action etag computation Add automatic template digests to all CacheHelper#cache calls (originally spiked in the cache_digests plugin) Add Routing Concerns to declare common routes that can be reused inside others resources and routes Added ActionController::Live. Mix it in to your controller and you can stream data to the client live truncate now always returns an escaped HTML-safe string. The option :escape can be used as false to not escape the result Added ActionDispatch::SSL middleware that when included force all the requests to be under HTTPS protocol ActiveModel changes excerpt AM::Validation#validates ability to pass custom exception to :strict option Changed `AM::Serializers::JSON.include_root_in_json' default value to false. Now, AM Serializers and AR objects have the same default behaviour Added ActiveModel::Model, a mixin to make Ruby objects work with AP out of box Trim down Active Model API by removing valid? and errors.full_messages ActiveRecord changes excerpt Use native mysqldump command instead of structure_dump method when dumping the database structure to a sql file. Attribute predicate methods, such as article.title?, will now raise ActiveModel::MissingAttributeError if the attribute being queried for truthiness was not read from the database, instead of just returning false ActiveRecord::SessionStore has been extracted from Active Record as activerecord-session_store gem. Please read the README.md file on the gem for the usage Fix reset_counters when there are multiple belongs_to association with the same foreign key and one of them have a counter cache Raise ArgumentError if list of attributes to change is empty in update_all Add Relation#load. This method explicitly loads the records and then returns self Deprecated most of the 'dynamic finder' methods. All dynamic methods except for find_by_... and find_by_...! are deprecated Added ability to ActiveRecord::Relation#from to accept other ActiveRecord::Relation objects Remove IdentityMap ActiveSupport changes excerpt ERB::Util.html_escape now escapes single quotes ActiveSupport::Callbacks: deprecate monkey patch of object callbacks Replace deprecated memcache-client gem with dalli in ActiveSupport::Cache::MemCacheStore Object#try will now return nil instead of raise a NoMethodError if the receiving object does not implement the method, but you can still get the old behavior by using the new Object#try! Object#try can't call private methods Add ActiveSupport::Deprecations.behavior = :silence to completely ignore Rails runtime deprecations What are the most important changes for you?

  Read the article

• How do I call a function name that is stored in a hash in Perl?

  - by Ether

  I'm sure this is covered in the documentation somewhere but I have been unable to find it... I'm looking for the syntactic sugar that will make it possible to call a method on a class whose name is stored in a hash (as opposed to a simple scalar): use strict; use warnings; package Foo; sub foo { print "in foo()\n" } package main; my %hash = (func => 'foo'); Foo->$hash{func}; If I copy $hash{func} into a scalar variable first, then I can call Foo->$func just fine... but what is missing to enable Foo->$hash{func} to work? (EDIT: I don't mean to do anything special by calling a method on class Foo -- this could just as easily be a blessed object (and in my actual code it is); it was just easier to write up a self-contained example using a class method.) EDIT 2: Just for completeness re the comments below, this is what I'm actually doing (this is in a library of Moose attribute sugar, created with Moose::Exporter): # adds an accessor to a sibling module sub foreignTable { my ($meta, $table, %args) = @_; my $class = 'MyApp::Dir1::Dir2::' . $table; my $dbAccessor = lcfirst $table; eval "require $class" or do { die "Can't load $class: $@" }; $meta->add_attribute( $table, is => 'ro', isa => $class, init_arg => undef, # don't allow in constructor lazy => 1, predicate => 'has_' . $table, default => sub { my $this = shift; $this->debug("in builder for $class"); ### here's the line that uses a hash value as the method name my @args = ($args{primaryKey} => $this->${\$args{primaryKey}}); push @args, ( _dbObject => $this->_dbObject->$dbAccessor ) if $args{fkRelationshipExists}; $this->debug("passing these values to $class -> new: @args"); $class->new(@args); }, ); } I've replaced the marked line above with this: my $pk_accessor = $this->meta->find_attribute_by_name($args{primaryKey})->get_read_method_ref; my @args = ($args{primaryKey} => $this->$pk_accessor); PS. I've just noticed that this same technique (using the Moose meta class to look up the coderef rather than assuming its naming convention) cannot also be used for predicates, as Class::MOP::Attribute does not have a similar get_predicate_method_ref accessor. :(

  Read the article

• SQL SERVER – 5 Tips for Improving Your Data with expressor Studio

  - by pinaldave

  It’s no secret that bad data leads to bad decisions and poor results.  However, how do you prevent dirty data from taking up residency in your data store?  Some might argue that it’s the responsibility of the person sending you the data.  While that may be true, in practice that will rarely hold up.  It doesn’t matter how many times you ask, you will get the data however they decide to provide it. So now you have bad data.  What constitutes bad data?  There are quite a few valid answers, for example: Invalid date values Inappropriate characters Wrong data Values that exceed a pre-set threshold While it is certainly possible to write your own scripts and custom SQL to identify and deal with these data anomalies, that effort often takes too long and becomes difficult to maintain.  Instead, leveraging an ETL tool like expressor Studio makes the data cleansing process much easier and faster.  Below are some tips for leveraging expressor to get your data into tip-top shape. Tip 1:     Build reusable data objects with embedded cleansing rules One of the new features in expressor Studio 3.2 is the ability to define constraints at the metadata level.  Using expressor’s concept of Semantic Types, you can define reusable data objects that have embedded logic such as constraints for dealing with dirty data.  Once defined, they can be saved as a shared atomic type and then re-applied to other data attributes in other schemas. As you can see in the figure above, I’ve defined a constraint on zip code.  I can then save the constraint rules I defined for zip code as a shared atomic type called zip_type for example.   The next time I get a different data source with a schema that also contains a zip code field, I can simply apply the shared atomic type (shown below) and the previously defined constraints will be automatically applied. Tip 2:     Unlock the power of regular expressions in Semantic Types Another powerful feature introduced in expressor Studio 3.2 is the option to use regular expressions as a constraint.   A regular expression is used to identify patterns within data.   The patterns could be something as simple as a date format or something much more complex such as a street address.  For example, I could define that a valid IP address should be made up of 4 numbers, each 0 to 255, and separated by a period.  So 192.168.23.123 might be a valid IP address whereas 888.777.0.123 would not be.   How can I account for this using regular expressions? A very simple regular expression that would look for any 4 sets of 3 digits separated by a period would be:  ^[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}$ Alternatively, the following would be the exact check for truly valid IP addresses as we had defined above:  ^(25[0-5]|2[0-4][0-9]|1[0-9]{2}|[1-9]?[0-9])\.(25[0-5]|2[0-4][0-9]|1[0-9]{2}|[1-9]?[0-9])\.(25[0-5]|2[0-4][0-9]|1[0-9]{2}|[1-9]?[0-9])\.(25[0-5]|2[0-4][0-9]|1[0-9]{2}|[1-9]?[0-9])$ .  In expressor, we would enter this regular expression as a constraint like this: Here we select the corrective action to be ‘Escalate’, meaning that the expressor Dataflow operator will decide what to do.  Some of the options include rejecting the offending record, skipping it, or aborting the dataflow. Tip 3:     Email pattern expressions that might come in handy In the example schema that I am using, there’s a field for email.  Email addresses are often entered incorrectly because people are trying to avoid spam.  While there are a lot of different ways to define what constitutes a valid email address, a quick search online yields a couple of really useful regular expressions for validating email addresses: This one is short and sweet:  \b[A-Z0-9._%+-]+@[A-Z0-9.-]+\.[A-Z]{2,4}\b (Source: http://www.regular-expressions.info/) This one is more specific about which characters are allowed:  ^([a-zA-Z0-9_\-\.]+)@((\[[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.)|(([a-zA-Z0-9\-]+\.)+))([a-zA-Z]{2,4}|[0-9]{1,3})(\]?)$ (Source: http://regexlib.com/REDetails.aspx?regexp_id=26 ) Tip 4:     Reject “dirty data” for analysis or further processing Yet another feature introduced in expressor Studio 3.2 is the ability to reject records based on constraint violations.  To capture reject records on input, simply specify Reject Record in the Error Handling setting for the Read File operator.  Then attach a Write File operator to the reject port of the Read File operator as such: Next, in the Write File operator, you can configure the expressor operator in a similar way to the Read File.  The key difference would be that the schema needs to be derived from the upstream operator as shown below: Once configured, expressor will output rejected records to the file you specified.  In addition to the rejected records, expressor also captures some diagnostic information that will be helpful towards identifying why the record was rejected.  This makes diagnosing errors much easier! Tip 5:    Use a Filter or Transform after the initial cleansing to finish the job Sometimes you may want to predicate the data cleansing on a more complex set of conditions.  For example, I may only be interested in processing data containing males over the age of 25 in certain zip codes.  Using an expressor Filter operator, you can define the conditional logic which isolates the records of importance away from the others. Alternatively, the expressor Transform operator can be used to alter the input value via a user defined algorithm or transformation.  It also supports the use of conditional logic and data can be rejected based on constraint violations. However, the best tip I can leave you with is to not constrain your solution design approach – expressor operators can be combined in many different ways to achieve the desired results.  For example, in the expressor Dataflow below, I can post-process the reject data from the Filter which did not meet my pre-defined criteria and, if successful, Funnel it back into the flow so that it gets written to the target table. I continue to be impressed that expressor offers all this functionality as part of their FREE expressor Studio desktop ETL tool, which you can download from here.  Their Studio ETL tool is absolutely free and they are very open about saying that if you want to deploy their software on a dedicated Windows Server, you need to purchase their server software, whose pricing is posted on their website. Reference: Pinal Dave (http://blog.SQLAuthority.com) Filed under: Pinal Dave, PostADay, SQL, SQL Authority, SQL Query, SQL Scripts, SQL Server, SQL Tips and Tricks, T SQL, Technology

  Read the article

• Rockmelt, the technology adoption model, and Facebook's spare internet

  - by Roger Hart

  Regardless of how good it is, you'd have to have a heart of stone not to make snide remarks about Rockmelt. After all, on the surface it looks a lot like some people spent two years building a browser instead of just bashing out a Chrome extension over a wet weekend. It probably does some more stuff. I don't know for sure because artificial scarcity is cool, apparently, so the "invitation" is still in the post*. I may in fact never know for sure, because I'm not wild about Facebook sign-in as a prerequisite for anything. From the video, and some initial reviews, my early reaction was: I have a browser, I have a Twitter client; what on earth is this for? The answer, of course, is "not me". Rockmelt is, in a way, quite audacious. Oh, sure, on launch day it's Bay Area bar-chat for the kids with no lenses in their retro specs and trousers that give you deep-vein thrombosis, but it's not really about them. Likewise,  Facebook just launched Google Wave, or something. And all the tech snobbery and scorn packed into describing it that way is irrelevant next to what they're doing with their platform. Here's something I drew in MS Paint** because I don't want to get sued: (see: The technology adoption lifecycle) A while ago in the Guardian, John Lanchester dusted off the idiom that "technology is stuff that doesn't work yet". The rest of the article would be quite interesting if it wasn't largely about MySpace, and he's sort of got a point. If you bolt on the sentiment that risk-averse businessmen like things that work, you've got the essence of Crossing the Chasm. Products for the mainstream market don't look much like technology. Think for  a second about early (1980s ish) hi-fi systems, with all the knobs and fiddly bits, their ostentatious technophile aesthetic. Then consider their sleeker and less (or at least less conspicuously) functional successors in the 1990s. The theory goes that innovators and early adopters like technology, it's a hobby in itself. The rest of the humans seem to like magic boxes with very few buttons that make stuff happen and never trouble them about why. Personally, I consider Apple's maddening insistence that iTunes is an acceptable way to move files around to be more or less morally unacceptable. Most people couldn't care less. Hence Rockmelt, and hence Facebook's continued growth. Rockmelt looks pointless to me, because I aggregate my social gubbins with Digsby, or use TweetDeck. But my use case is different and so are my enthusiasms. If I want to share photos, I'll use Flickr - but Facebook has photo sharing. If I want a short broadcast message, I'll use Twitter - Facebook has status updates. If I want to sell something with relatively little hassle, there's eBay - or Facebook marketplace. YouTube - check, FB Video. Email - messaging. Calendaring apps, yeah there are loads, or FB Events. What if I want to host a simple web page? Sure, they've got pages. Also Notes for blogging, and more games than I can count. This stuff is right there, where millions and millions of users are already, and for what they need it just works. It's not about me, because I'm not in the big juicy area under the curve. It's what 1990s portal sites could never have dreamed of achieving. Facebook is AOL on speed, crack, and some designer drugs it had specially imported from the future. It's a n00b-friendly gateway to the internet that just happens to serve up all the things you want to do online, right where you are. Oh, and everybody else is there too. The price of having all this and the social graph too is that you have all of this, and the social graph too. But plenty of folks have more incisive things to say than me about the whole privacy shebang, and it's not really what I'm talking about. Facebook is maintaining a vast, and fairly fully-featured training-wheels internet. And it makes up a large proportion of the online experience for a lot of people***. It's the entire web (2.0?) experience for the early and late majority. And sure, no individual bit of it is quite as slick or as fully-realised as something like Flickr (which wows me a bit every time I use it. Those guys are good at the web), but it doesn't have to be. It has to be unobtrusively good enough for the regular humans. It has to not feel like technology. This is what Rockmelt sort of is. You're online, you want something nebulously social, and you don't want to faff about with, say, Twitter clients. Wow! There it is on a really distracting sidebar, right in your browser. No effort! Yeah - fish nor fowl, much? It might work, I guess. There may be a demographic who want their social web experience more simply than tech tinkering, and who aren't just getting it from Facebook (or, for that matter, mobile devices). But I'd be surprised. Rockmelt feels like an attempt to grab a slice of Facebook-style "Look! It's right here, where you already are!", but it's still asking the mature market to install a new browser. Presumably this is where that Facebook sign-in predicate comes in handy, though it'll take some potent awareness marketing to make it fly. Meanwhile, Facebook quietly has the entire rest of the internet as a product management resource, and can continue to give most of the people most of what they want. Something that has not gone un-noticed in its potential to look a little sinister. But heck, they might even make Google Wave popular.     *This was true last week when I drafted this post. I got an invite subsequently, hence the screenshot. **MS Paint is no fun any more. It's actually good in Windows 7. Farewell ironically-shonky diagrams. *** It's also behind a single sign-in, lending a veneer of confidence, and partially solving the problem of usernames being crummy unique identifiers. I'll be blogging about that at some point.

  Read the article

• Resolving data redundancy up front

  - by okeofs

  Introduction As all of us do when confronted with a problem, the resource of choice is to ‘Google it’. This is where the plot thickens. Recently I was asked to stage data from numerous databases which were to be loaded into a data warehouse. To make a long story short, I was looking for a manner in which to obtain the table names from each database, to ascertain potential overlap.   As the source data comes from a SQL database created from dumps of a third party product,  one could say that there were +/- 95 tables for each database.   Yes I know that first instinct is to use the system stored procedure “exec sp_msforeachdb 'select "?" AS db, * from [?].sys.tables'”. However, if one stops to think about this, it would be nice to have all the results in a temporary or disc based  table; which in itself , implies additional labour. This said,  I decided to ‘re-invent’ the wheel. The full code sample may be found at the bottom of this article.   Define a few temporary tables and variables   declare @SQL varchar(max); declare @databasename varchar(75) /* drop table ##rawdata3 drop table #rawdata1 drop table #rawdata11 */ -- A temp table to hold the names of my databases CREATE TABLE #rawdata1 (    database_name varchar(50) ,    database_size varchar(50),    remarks Varchar(50) )     --A temp table with the same database names as above, HOWEVER using an --Identity number (recNO) as a loop variable. --You will note below that I loop through until I reach 25 (see below) as at --that point the system databases, the reporting server database etc begin. --1- 24 are user databases. These are really what I was looking for. --Whilst NOT the best solution,it works and the code was meant as a quick --and dirty. CREATE TABLE #rawdata11 (    recNo int identity(1,1),    database_name varchar(50) ,    database_size varchar(50),    remarks Varchar(50) )   --My output table showing the database name and table name CREATE TABLE ##rawdata3 (    database_name varchar(75) ,    table_name varchar(75), )   Insert the database names into a temporary table I pull the database names using the system stored procedure sp_databases   INSERT INTO #rawdata1 EXEC sp_databases Go   Insert the results from #rawdata1 into a table containing a record number  #rawdata11 so that I can LOOP through the extract   INSERT into #rawdata11 select * from  #rawdata1   We now declare 3 more variables:  @kounter is used to keep track of our position within the loop. @databasename is used to keep track of the’ current ‘ database name being used in the current pass of the loop;  as inorder to obtain the tables for that database we  need to issue a ‘USE’ statement, an insert command and other related code parts. This is the challenging part. @sql is a varchar(max) variable used to contain the ‘USE’ statement PLUS the’ insert ‘ code statements. We now initalize @kounter to 1 .   declare @kounter int; declare @databasename varchar(75); declare @sql varchar(max); set @kounter = 1   The Loop The astute reader will remember that the temporary table #rawdata11 contains our  database names  and each ‘database row’ has a record number (recNo). I am only interested in record numbers under 25. I now set the value of the temporary variable @DatabaseName (see below) .Note that I used the row number as a part of the predicate. Now, knowing the database name, I can create dynamic T-SQL to be executed using the sp_sqlexec stored procedure (see the code in red below). Finally, after all the tables for that given database have been placed in temporary table ##rawdata3, I increment the counter and continue on. Note that I used a global temporary table to ensure that the result set persists after the termination of the run. At some stage, I plan to redo this part of the code, as global temporary tables are not really an ideal solution.    WHILE (@kounter < 25)  BEGIN  select @DatabaseName = database_name from #rawdata11 where recNo = @kounter  set @SQL = 'Use ' + @DatabaseName + ' Insert into ##rawdata3 ' + + ' SELECT table_catalog,Table_name FROM information_schema.tables' exec sp_sqlexec  @Sql  SET @kounter  = @kounter + 1  END   The full code extract   Here is the full code sample.   declare @SQL varchar(max); declare @databasename varchar(75) /* drop table ##rawdata3 drop table #rawdata1 drop table #rawdata11 */ CREATE TABLE #rawdata1 (    database_name varchar(50) ,    database_size varchar(50),    remarks Varchar(50) ) CREATE TABLE #rawdata11 (    recNo int identity(1,1),    database_name varchar(50) ,    database_size varchar(50),    remarks Varchar(50) ) CREATE TABLE ##rawdata3 (    database_name varchar(75) ,    table_name varchar(75), )   INSERT INTO #rawdata1 EXEC sp_databases go INSERT into #rawdata11 select * from  #rawdata1 declare @kounter int; declare @databasename varchar(75); declare @sql varchar(max); set @kounter = 1 WHILE (@kounter < 25)  BEGIN  select @databasename = database_name from #rawdata11 where recNo = @kounter  set @SQL = 'Use ' + @DatabaseName + ' Insert into ##rawdata3 ' + + ' SELECT table_catalog,Table_name FROM information_schema.tables' exec sp_sqlexec  @Sql  SET @kounter  = @kounter + 1  END    select * from ##rawdata3  where table_name like '%SalesOrderHeader%'

  Read the article

• How to trace a function array argument in DTrace

  - by uejio

  I still use dtrace just about every day in my job and found that I had to print an argument to a function which was an array of strings.  The array was variable length up to about 10 items.  I'm not sure if the is the right way to do it, but it seems to work and is not too painful if the array size is small.Here's an example.  Suppose in your application, you have the following function, where n is number of item in the array s.void arraytest(int n, char **s){    /* Loop thru s[0] to s[n-1] */}How do you use DTrace to print out the values of s[i] or of s[0] to s[n-1]?  DTrace does not have if-then blocks or for loops, so you can't do something like:    for i=0; i<arg0; i++        trace arg1[i]; It turns out that you can use probe ordering as a kind of iterator. Probes with the same name will fire in the order that they appear in the script, so I can save the value of "n" in the first probe and then use it as part of the predicate of the next probe to determine if the other probe should fire or not.  So the first probe for tracing the arraytest function is:pid$target::arraytest:entry{    self->n = arg0;}Then, if I want to print out the first few items of the array, I first check the value of n.  If it's greater than the index that I want to print out, then I can print that index.  For example, if I want to print out the 3rd element of the array, I would do something like:pid$target::arraytest:entry/self->n > 2/{    printf("%s",stringof(arg1 + 2 * sizeof(pointer)));}Actually, that doesn't quite work because arg1 is a pointer to an array of pointers and needs to be copied twice from the user process space to the kernel space (which is where dtrace is). Also, the sizeof(char *) is 8, but for some reason, I have to use 4 which is the sizeof(uint32_t). (I still don't know how that works.)  So, the script that prints the 3rd element of the array should look like:pid$target::arraytest:entry{    /* first, save the size of the array so that we don't get            invalid address errors when indexing arg1+n. */    self->n = arg0;}pid$target::arraytest:entry/self->n > 2/{    /* print the 3rd element (index = 2) of the second arg. */    i = 2;    size = 4;    self->a_t = copyin(arg1+size*i,size);    printf("%s: a[%d]=%s",probefunc,i,copyinstr(*(uint32_t *)self->a_t));}If your array is large, then it's quite painful since you have to write one probe for every array index.  For example, here's the full script for printing the first 5 elements of the array:#!/usr/sbin/dtrace -spid$target::arraytest:entry{        /* first, save the size of the array so that we don't get           invalid address errors when indexing arg1+n. */        self->n = arg0;}pid$target::arraytest:entry/self->n > 0/{        i = 0;        size = sizeof(uint32_t);        self->a_t = copyin(arg1+size*i,size);        printf("%s: a[%d]=%s",probefunc,i,copyinstr(*(uint32_t *)self->a_t));}pid$target::arraytest:entry/self->n > 1/{        i = 1;        size = sizeof(uint32_t);        self->a_t = copyin(arg1+size*i,size);        printf("%s: a[%d]=%s",probefunc,i,copyinstr(*(uint32_t *)self->a_t));}pid$target::arraytest:entry/self->n > 2/{        i = 2;        size = sizeof(uint32_t);        self->a_t = copyin(arg1+size*i,size);        printf("%s: a[%d]=%s",probefunc,i,copyinstr(*(uint32_t *)self->a_t));}pid$target::arraytest:entry/self->n > 3/{        i = 3;        size = sizeof(uint32_t);        self->a_t = copyin(arg1+size*i,size);        printf("%s: a[%d]=%s",probefunc,i,copyinstr(*(uint32_t *)self->a_t));}pid$target::arraytest:entry/self->n > 4/{        i = 4;        size = sizeof(uint32_t);        self->a_t = copyin(arg1+size*i,size);        printf("%s: a[%d]=%s",probefunc,i,copyinstr(*(uint32_t *)self->a_t));} If the array is large, then your script will also have to be very long to print out all values of the array.

  Read the article

• Best practices for using the Entity Framework with WPF DataBinding

  - by Ken Smith

  I'm in the process of building my first real WPF application (i.e., the first intended to be used by someone besides me), and I'm still wrapping my head around the best way to do things in WPF. It's a fairly simple data access application using the still-fairly-new Entity Framework, but I haven't been able to find a lot of guidance online for the best way to use these two technologies (WPF and EF) together. So I thought I'd toss out how I'm approaching it, and see if anyone has any better suggestions. I'm using the Entity Framework with SQL Server 2008. The EF strikes me as both much more complicated than it needs to be, and not yet mature, but Linq-to-SQL is apparently dead, so I might as well use the technology that MS seems to be focusing on. This is a simple application, so I haven't (yet) seen fit to build a separate data layer around it. When I want to get at data, I use fairly simple Linq-to-Entity queries, usually straight from my code-behind, e.g.: var families = from family in entities.Family.Include("Person") orderby family.PrimaryLastName, family.Tag select family; Linq-to-Entity queries return an IOrderedQueryable result, which doesn't automatically reflect changes in the underlying data, e.g., if I add a new record via code to the entity data model, the existence of this new record is not automatically reflected in the various controls referencing the Linq query. Consequently, I'm throwing the results of these queries into an ObservableCollection, to capture underlying data changes: familyOC = new ObservableCollection<Family>(families.ToList()); I then map the ObservableCollection to a CollectionViewSource, so that I can get filtering, sorting, etc., without having to return to the database. familyCVS.Source = familyOC; familyCVS.View.Filter = new Predicate<object>(ApplyFamilyFilter); familyCVS.View.SortDescriptions.Add(new System.ComponentModel.SortDescription("PrimaryLastName", System.ComponentModel.ListSortDirection.Ascending)); familyCVS.View.SortDescriptions.Add(new System.ComponentModel.SortDescription("Tag", System.ComponentModel.ListSortDirection.Ascending)); I then bind the various controls and what-not to that CollectionViewSource: <ListBox DockPanel.Dock="Bottom" Margin="5,5,5,5" Name="familyList" ItemsSource="{Binding Source={StaticResource familyCVS}, Path=., Mode=TwoWay}" IsSynchronizedWithCurrentItem="True" ItemTemplate="{StaticResource familyTemplate}" SelectionChanged="familyList_SelectionChanged" /> When I need to add or delete records/objects, I manually do so from both the entity data model, and the ObservableCollection: private void DeletePerson(Person person) { entities.DeleteObject(person); entities.SaveChanges(); personOC.Remove(person); } I'm generally using StackPanel and DockPanel controls to position elements. Sometimes I'll use a Grid, but it seems hard to maintain: if you want to add a new row to the top of your grid, you have to touch every control directly hosted by the grid to tell it to use a new line. Uggh. (Microsoft has never really seemed to get the DRY concept.) I almost never use the VS WPF designer to add, modify or position controls. The WPF designer that comes with VS is sort of vaguely helpful to see what your form is going to look like, but even then, well, not really, especially if you're using data templates that aren't binding to data that's available at design time. If I need to edit my XAML, I take it like a man and do it manually. Most of my real code is in C# rather than XAML. As I've mentioned elsewhere, entirely aside from the fact that I'm not yet used to "thinking" in it, XAML strikes me as a clunky, ugly language, that also happens to come with poor designer and intellisense support, and that can't be debugged. Uggh. Consequently, whenever I can see clearly how to do something in C# code-behind that I can't easily see how to do in XAML, I do it in C#, with no apologies. There's been plenty written about how it's a good practice to almost never use code-behind in WPF page (say, for event-handling), but so far at least, that makes no sense to me whatsoever. Why should I do something in an ugly, clunky language with god-awful syntax, an astonishingly bad editor, and virtually no type safety, when I can use a nice, clean language like C# that has a world-class editor, near-perfect intellisense, and unparalleled type safety? So that's where I'm at. Any suggestions? Am I missing any big parts of this? Anything that I should really think about doing differently?

  Read the article

• Overflow exception while performing parallel factorization using the .NET Task Parallel Library (TPL

  - by Aviad P.

  Hello, I'm trying to write a not so smart factorization program and trying to do it in parallel using TPL. However, after about 15 minutes of running on a core 2 duo machine, I am getting an aggregate exception with an overflow exception inside it. All the entries in the stack trace are part of the .NET framework, the overflow does not come from my code. Any help would be appreciated in figuring out why this happens. Here's the commented code, hopefully it's simple enough to understand: class Program { static List<Tuple<BigInteger, int>> factors = new List<Tuple<BigInteger, int>>(); static void Main(string[] args) { BigInteger theNumber = BigInteger.Parse( "653872562986528347561038675107510176501827650178351386656875178" + "568165317809518359617865178659815012571026531984659218451608845" + "719856107834513527"); Stopwatch sw = new Stopwatch(); bool isComposite = false; sw.Start(); do { /* Print out the number we are currently working on. */ Console.WriteLine(theNumber); /* Find a factor, stop when at least one is found (using the Any operator). */ isComposite = Range(theNumber) .AsParallel() .Any(x => CheckAndStoreFactor(theNumber, x)); /* Of the factors found, take the one with the lowest base. */ var factor = factors.OrderBy(x => x.Item1).First(); Console.WriteLine(factor); /* Divide the number by the factor. */ theNumber = BigInteger.Divide( theNumber, BigInteger.Pow(factor.Item1, factor.Item2)); /* Clear the discovered factors cache, and keep looking. */ factors.Clear(); } while (isComposite); sw.Stop(); Console.WriteLine(isComposite + " " + sw.Elapsed); } static IEnumerable<BigInteger> Range(BigInteger squareOfTarget) { BigInteger two = BigInteger.Parse("2"); BigInteger element = BigInteger.Parse("3"); while (element * element < squareOfTarget) { yield return element; element = BigInteger.Add(element, two); } } static bool CheckAndStoreFactor(BigInteger candidate, BigInteger factor) { BigInteger remainder, dividend = candidate; int exponent = 0; do { dividend = BigInteger.DivRem(dividend, factor, out remainder); if (remainder.IsZero) { exponent++; } } while (remainder.IsZero); if (exponent > 0) { lock (factors) { factors.Add(Tuple.Create(factor, exponent)); } } return exponent > 0; } } Here's the exception thrown: Unhandled Exception: System.AggregateException: One or more errors occurred. --- > System.OverflowException: Arithmetic operation resulted in an overflow. at System.Linq.Parallel.PartitionedDataSource`1.ContiguousChunkLazyEnumerator.MoveNext(T& currentElement, Int32& currentKey) at System.Linq.Parallel.AnyAllSearchOperator`1.AnyAllSearchOperatorEnumerator`1.MoveNext(Boolean& currentElement, Int32& currentKey) at System.Linq.Parallel.StopAndGoSpoolingTask`2.SpoolingWork() at System.Linq.Parallel.SpoolingTaskBase.Work() at System.Linq.Parallel.QueryTask.BaseWork(Object unused) at System.Linq.Parallel.QueryTask.<.cctor>b__0(Object o) at System.Threading.Tasks.Task.InnerInvoke() at System.Threading.Tasks.Task.Execute() --- End of inner exception stack trace --- at System.Linq.Parallel.QueryTaskGroupState.QueryEnd(Boolean userInitiatedDispose) at System.Linq.Parallel.SpoolingTask.SpoolStopAndGo[TInputOutput,TIgnoreKey](QueryTaskGroupState groupState, PartitionedStream`2 partitions, SynchronousChannel`1[] channels, TaskScheduler taskScheduler) at System.Linq.Parallel.DefaultMergeHelper`2.System.Linq.Parallel.IMergeHelper<TInputOutput>.Execute() at System.Linq.Parallel.MergeExecutor`1.Execute[TKey](PartitionedStream`2 partitions, Boolean ignoreOutput, ParallelMergeOptions options, TaskScheduler taskScheduler, Boolean isOrdered, CancellationState cancellationState, Int32 queryId) at System.Linq.Parallel.PartitionedStreamMerger`1.Receive[TKey](PartitionedStream`2 partitionedStream) at System.Linq.Parallel.AnyAllSearchOperator`1.WrapPartitionedStream[TKey](PartitionedStream`2 inputStream, IPartitionedStreamRecipient`1 recipient, BooleanpreferStriping, QuerySettings settings) at System.Linq.Parallel.UnaryQueryOperator`2.UnaryQueryOperatorResults.ChildResultsRecipient.Receive[TKey](PartitionedStream`2 inputStream) at System.Linq.Parallel.ScanQueryOperator`1.ScanEnumerableQueryOperatorResults.GivePartitionedStream(IPartitionedStreamRecipient`1 recipient) at System.Linq.Parallel.UnaryQueryOperator`2.UnaryQueryOperatorResults.GivePartitionedStream(IPartitionedStreamRecipient`1 recipient) at System.Linq.Parallel.QueryOperator`1.GetOpenedEnumerator(Nullable`1 mergeOptions, Boolean suppressOrder, Boolean forEffect, QuerySettings querySettings) at System.Linq.Parallel.QueryOpeningEnumerator`1.OpenQuery() at System.Linq.Parallel.QueryOpeningEnumerator`1.MoveNext() at System.Linq.Parallel.AnyAllSearchOperator`1.Aggregate() at System.Linq.ParallelEnumerable.Any[TSource](ParallelQuery`1 source, Func`2 predicate) at PFact.Program.Main(String[] args) in d:\myprojects\PFact\PFact\Program.cs:line 34 Any help would be appreciated. Thanks!

  Read the article

• Freezes (not crashes) with GCD, blocks and Core Data

  - by Lukasz

  I have recently rewritten my Core Data driven database controller to use Grand Central Dispatch to manage fetching and importing in the background. Controller can operate on 2 NSManagedContext's: NSManagedObjectContext *mainMoc instance variable for main thread. this contexts is used only by quick access for UI by main thread or by dipatch_get_main_queue() global queue. NSManagedObjectContext *bgMoc for background tasks (importing and fetching data for NSFetchedresultsController for tables). This background tasks are fired ONLY by user defined queue: dispatch_queue_t bgQueue (instance variable in database controller object). Fetching data for tables is done in background to not block user UI when bigger or more complicated predicates are performed. Example fetching code for NSFetchedResultsController in my table view controllers: -(void)fetchData{ dispatch_async([CDdb db].bgQueue, ^{ NSError *error = nil; [[self.fetchedResultsController fetchRequest] setPredicate:self.predicate]; if (self.fetchedResultsController && ![self.fetchedResultsController performFetch:&error]) { NSSLog(@"Unresolved error in fetchData %@", error); } if (!initial_fetch_attampted)initial_fetch_attampted = YES; fetching = NO; dispatch_async(dispatch_get_main_queue(), ^{ [self.table reloadData]; [self.table scrollRectToVisible:CGRectMake(0, 0, 100, 20) animated:YES]; }); }); } // end of fetchData function bgMoc merges with mainMoc on save using NSManagedObjectContextDidSaveNotification: - (void)bgMocDidSave:(NSNotification *)saveNotification { // CDdb - bgMoc didsave - merging changes with main mainMoc dispatch_async(dispatch_get_main_queue(), ^{ [self.mainMoc mergeChangesFromContextDidSaveNotification:saveNotification]; // Extra notification for some other, potentially interested clients [[NSNotificationCenter defaultCenter] postNotificationName:DATABASE_SAVED_WITH_CHANGES object:saveNotification]; }); } - (void)mainMocDidSave:(NSNotification *)saveNotification { // CDdb - main mainMoc didSave - merging changes with bgMoc dispatch_async(self.bgQueue, ^{ [self.bgMoc mergeChangesFromContextDidSaveNotification:saveNotification]; }); } NSfetchedResultsController delegate has only one method implemented (for simplicity): - (void)controllerDidChangeContent:(NSFetchedResultsController *)controller { dispatch_async(dispatch_get_main_queue(), ^{ [self fetchData]; }); } This way I am trying to follow Apple recommendation for Core Data: 1 NSManagedObjectContext per thread. I know this pattern is not completely clean for at last 2 reasons: bgQueue not necessarily fires the same thread after suspension but since it is serial, it should not matter much (there is never 2 threads trying access bgMoc NSManagedObjectContext dedicated to it). Sometimes table view data source methods will ask NSFetchedResultsController for info from bgMoc (since fetch is done on bgQueue) like sections count, fetched objects in section count, etc.... Event with this flaws this approach works pretty well of the 95% of application running time until ... AND HERE GOES MY QUESTION: Sometimes, very randomly application freezes but not crashes. It does not response on any touch and the only way to get it back to live is to restart it completely (switching back to and from background does not help). No exception is thrown and nothing is printed to the console (I have Breakpoints set for all exception in Xcode). I have tried to debug it using Instruments (time profiles especially) to see if there is something hard going on on main thread but nothing is showing up. I am aware that GCD and Core Data are the main suspects here, but I have no idea how to track / debug this. Let me point out, that this also happens when I dispatch all the tasks to the queues asynchronously only (using dispatch_async everywhere). This makes me think it is not just standard deadlock. Is there any possibility or hints of how could I get more info what is going on? Some extra debug flags, Instruments magical tricks or build setting etc... Any suggestions on what could be the cause are very much appreciated as well as (or) pointers to how to implement background fetching for NSFetchedResultsController and background importing in better way.

  Read the article

• Can I constrain a template parameter class to implement the interfaces that are supported by other?

  - by K. Georgiev

  The name is a little blurry, so here's the situation: I'm writing code to use some 'trajectories'. The trajectories are an abstract thing, so I describe them with different interfaces. So I have a code as this: namespace Trajectories { public interface IInitial < Atom > { Atom Initial { get; set; } } public interface ICurrent < Atom > { Atom Current { get; set; } } public interface IPrevious < Atom > { Atom Previous { get; set; } } public interface ICount < Atom > { int Count { get; } } public interface IManualCount < Atom > : ICount < Atom > { int Count { get; set; } } ... } Every concrete implementation of a trajectory will implement some of the above interfaces. Here's a concrete implementation of a trajectory: public class SimpleTrajectory < Atom > : IInitial < Atom >, ICurrent < Atom >, ICount < Atom > { // ICount public int Count { get; private set; } // IInitial private Atom initial; public Atom Initial { get { return initial; } set { initial = current = value; Count = 1; } } // ICurrent private Atom current; public Atom Current { get { return current; } set { current = value; Count++; } } } Now, I want to be able to deduce things about the trajectories, so, for example I want to support predicates about different properties of some trajectory: namespace Conditions { public interface ICondition &lt Atom, Trajectory &gt { bool Test(ref Trajectory t); } public class CountLessThan &lt Atom, Trajectory &gt : ICondition &lt Atom, Trajectory &gt where Trajectory : Trajectories.ICount &lt Atom &gt { public int Value { get; set; } public CountLessThan() { } public bool Test(ref Trajectory t) { return t.Count &lt Value; } } public class CurrentNormLessThan &lt Trajectory &gt : ICondition &lt Complex, Trajectory &gt where Trajectory : Trajectories.ICurrent &lt Complex &gt { public double Value { get; set; } public CurrentNormLessThan() { } public bool Test(ref Trajectory t) { return t.Current.Norm() &lt Value; } } } Now, here's the question: What if I wanted to implement AND predicate? It would be something like this: public class And &lt Atom, CondA, TrajectoryA, CondB, TrajectoryB, Trajectory &gt : ICondition &lt Atom, Trajectory &gt where CondA : ICondition &lt Atom, TrajectoryA &gt where TrajectoryA : // Some interfaces where CondB : ICondition &lt Atom, TrajectoryB &gt where TrajectoryB : // Some interfaces where Trajectory : // MUST IMPLEMENT THE INTERFACES FOR TrajectoryA AND THE INTERFACES FOR TrajectoryB { public CondA A { get; set; } public CondB B { get; set; } public bool Test(ref Trajectory t){ return A.Test(t) && B.Test(t); } } How can I say: support only these trajectories, for which the arguments of AND are ok? So I can be able to write: var vand = new CountLessThan(32) & new CurrentNormLessThan(4.0); I think if I create an orevall interface for every subset of interfaces, I could be able to do it, but it will become quite ugly.

  Read the article

• Broken Multithreading With Core Data

  - by spamguy

  This is a better-focused version of an earlier question that touches upon an entirely different subject from before. I am working on a Cocoa Core Data application with multiple threads. There is a Song and Artist; every Song has an Artist relation. There is a delegate code file not cited here; it more or less looks like the template XCode generates. I am far better working with the former technology than the latter, and any multithreading capability came from a Core Data template. When I'm doing all my ManagedObjectContext work in one method, I am fine. When I put fetch-or-insert-then-return-object work into a separate method, the application halts (but does not crash) at the new method's return statement, seen below. The new method even gets its own MOC to be safe, and it has not helped any. The result is one addition to Song and a halt after generating an Artist. I get no errors or exceptions, and I don't know why. I've debugged out the wazoo. My theory is that any errors occurring are in another thread, and the one I'm watching is waiting on something forever. What did I do wrong with getArtistObject: , and how can I fix it? Thanks. - (void)main { NSInteger songCount = 1; NSManagedObjectContext *moc = [[NSManagedObjectContext alloc] init]; [moc setPersistentStoreCoordinator:[[self delegate] persistentStoreCoordinator]]; [[NSNotificationCenter defaultCenter] addObserver:self selector:@selector(contextDidSave:) name:NSManagedObjectContextDidSaveNotification object:moc]; /* songDict generated here */ for (id key in songDict) { NSManagedObject *song = [NSEntityDescription insertNewObjectForEntityForName:@"Song" inManagedObjectContext:moc]; [song setValue:[songDictItem objectForKey:@"Name"] forKey:@"title"]; [song setValue:[self getArtistObject:(NSString *) [songDictItem objectForKey:@"Artist"]] forKey:@"artist"]; [songDictItem release]; songCount++; } NSError *error; if (![moc save:&error]) [NSApp presentError:error]; [[NSNotificationCenter defaultCenter] removeObserver:self name:NSManagedObjectContextDidSaveNotification object:moc]; [moc release], moc = nil; [[self delegate] importDone]; } - (NSManagedObject*) getArtistObject:(NSString*)theArtist { NSError *error = nil; NSManagedObjectContext *moc = [[NSManagedObjectContext alloc] init]; [moc setPersistentStoreCoordinator:[[self delegate] persistentStoreCoordinator]]; [[NSNotificationCenter defaultCenter] addObserver:self selector:@selector(contextDidSave:) name:NSManagedObjectContextDidSaveNotification object:moc]; NSFetchRequest *fetch = [[[NSFetchRequest alloc] init] autorelease]; NSEntityDescription *entityDescription = [NSEntityDescription entityForName:@"Artist" inManagedObjectContext:moc]; [fetch setEntity:entityDescription]; // object to be returned NSManagedObject *artistObject = [[NSManagedObject alloc] initWithEntity:entityDescription insertIntoManagedObjectContext:moc]; // set predicate (artist name) NSPredicate *pred = [NSPredicate predicateWithFormat:[NSString stringWithFormat:@"name = \"%@\"", theArtist]]; [fetch setPredicate:pred]; NSArray *response = [moc executeFetchRequest:fetch error:&error]; if (error) [NSApp presentError:error]; if ([response count] == 0) // empty resultset --> no artists with this name { [artistObject setValue:theArtist forKey:@"name"]; NSLog(@"%@ not found. Adding.", theArtist); return artistObject; } else return [response objectAtIndex:0]; } @end

  Read the article

• iPhone App Crashes when merging managed object contexts

  - by DVG

  Short Version: Using two managed object contexts, and while the context is saving to the store the application bombs when I attempt to merge the two contexts and reload the table view. Long Version: Okay, so my application is set up as thus. 3 view controllers, all table views. Platforms View Controller - Games View Controller (Predicated upon platform selection) - Add Game View Controller I ran into a problem when Games View Controller was bombing when adding a new entry to the context, because the fetched results contorller wanted to update the view for something that didn't match the predicate. As a solution, I rebuilt the Add Controller to use a second NSManagedObject Context, called adding context, following the design pattern in the Core Data Books example. My Games List View Controller is a delegate for the add controller, to handle all the saving, so my addButtonPressed method looks like this - (IBAction) addButtonPressed: (id) sender { AddGameTableViewController *addGameVC = [[AddGameTableViewController alloc] initWithNibName:@"AddGameTableViewController" bundle:nil]; NSManagedObjectContext *aAddingContext = [[NSManagedObjectContext alloc] init]; self.addingContext = aAddingContext; [aAddingContext release]; [addingContext setPersistentStoreCoordinator:[[gameResultsController managedObjectContext] persistentStoreCoordinator]]; addGameVC.context = addingContext; addGameVC.delegate = self; addGameVC.newGame = (Game *)[NSEntityDescription insertNewObjectForEntityForName:@"Game" inManagedObjectContext:addingContext]; UINavigationController *addNavCon = [[UINavigationController alloc] initWithRootViewController:addGameVC]; [self presentModalViewController:addNavCon animated:YES]; [addGameVC release]; [addNavCon release]; } There is also a delegate method which handles the saving. This all works swimmingly. The issue is getting the table view controller in the GameListViewController to update itself. Per the example, an observer is added to watch for the second context to be saved, and then to merge the addingContext with the primary one. So I have: - (void)addViewController:(AddGameTableViewController *)controller didFinishWithSave:(BOOL)save { if (save) { NSNotificationCenter *dnc = [NSNotificationCenter defaultCenter]; [dnc addObserver:self selector:@selector(addControllerContextDidSave:) name:NSManagedObjectContextDidSaveNotification object:addingContext]; //snip! Context Save Code [dnc removeObserver:self name:NSManagedObjectContextDidSaveNotification object:addingContext]; } self.addingContext = nil; [self dismissModalViewControllerAnimated:YES]; } - (void)addControllerContextDidSave:(NSNotification*)saveNotification { NSManagedObjectContext *myContext = [gameResultsController managedObjectContext]; [myContext mergeChangesFromContextDidSaveNotification:saveNotification]; } So now, what happens is after save is pressed, the application hangs for a moment and then crashes. The save is processed, as the new game is present when I relaunch the application, and the application seems to be flowing as appropriate, but it bombs out for reasons that are beyond my understanding. NSLog of the saveNotification spits out this: NSConcreteNotification 0x3b557f0 {name = NSManagingContextDidSaveChangesNotification; object = <NSManagedObjectContext: 0x3b4bb90>; userInfo = { inserted = {( <Game: 0x3b4f510> (entity: Game; id: 0x3b614e0 <x-coredata://13168366-B8E7-41C8-B384-BAF14A5E08D9/Game/p2> ; data: { name = "Final Fantasy XIII"; platform = 0x3b66910 <x-coredata://13168366-B8E7-41C8-B384-BAF14A5E08D9/Platform/p20>; }) )}; updated = {( <Platform: 0x3b67650> (entity: Platform; id: 0x3b66910 <x-coredata://13168366-B8E7-41C8-B384-BAF14A5E08D9/Platform/p20> ; data: { games = ( 0x3b614e0 <x-coredata://13168366-B8E7-41C8-B384-BAF14A5E08D9/Game/p2>, 0x603a530 <x-coredata://13168366-B8E7-41C8-B384-BAF14A5E08D9/Game/p1> ); name = "Xbox 360"; }) )}; }} I've tried both a simple [self.tableView reloadData]; and the more complicated multi-method table updating structure in the Core Data Books example. Both produce the same result.

  Read the article

• JPA2 adding referential contraint to table complicates criteria query with lazy fetch, need advice

  - by Quaternion

  Following is a lot of writing for what I feel is a pretty simple issue. Root of issue is my ignorance, not looking so much for code but advice. Table: Ininvhst (Inventory-schema inventory history) column ihtran (inventory history transfer code) using an old entity mapping I have: @Basic(optional = false) @Column(name = "IHTRAN") private String ihtran; ihtran is really a foreign key to table Intrnmst ("Inventory Transfer Master" which contains a list of "transfer codes"). This was not expressed in the database so placed a referential constraint on Ininvhst re-generating JPA2 entity classes produced: @JoinColumn(name = "IHTRAN", referencedColumnName = "TMCODE", nullable = false) @ManyToOne(optional = false) private Intrnmst intrnmst; Now previously I was using JPA2 to select the records/(Ininvhst entities) from the Ininvhst table where "ihtran" was one of a set of values. I used in.value() to do this... here is a snippet: cq = cb.createQuery(Ininvhst.class); ... In in = cb.in(transactionType); //Get in expression for transacton types for (String s : transactionTypes) { //has a value in = in.value(s);//check if the strings we are looking for exist in the transfer master } predicateList.add(in); My issue is that the Ininvhst used to contain a string called ihtran but now it contains Ininvhst... So I now need a path expression: this.predicateList = new ArrayList<Predicate>(); if (transactionTypes != null && transactionTypes.size() > 0) { //list of strings has some values Path<Intrnmst> intrnmst = root.get(Ininvhst_.intrnmst); //get transfermaster from Ininvhst Path<String> transactionType = intrnmst.get(Intrnmst_.tmcode); //get transaction types from transfer master In<String> in = cb.in(transactionType); //Get in expression for transacton types for (String s : transactionTypes) { //has a value in = in.value(s);//check if the strings we are looking for exist in the transfer master } predicateList.add(in); } Can I add ihtran back into the entity along with a join column that is both references "IHTRAN"? Or should I use a projection to somehow return Ininvhst along with the ihtran string which is now part of the Intrnmst entity. Or should I use a projection to return Ininvhst and somehow limit Intrnmst just just the ihtran string. Further information: I am using the resulting list of selected Ininvhst objects in a web application, the class which contains the list of Ininvhst objects is transformed into a json object. There are probably quite a few serialization methods that would navigate the object graph the problem is that my current fetch strategy is lazy so it hits the join entity (Intrnmst intrnmst) and there is no Entity Manager available at that point. At this point I have prevented the object from serializing the join column but now I am missing a critical piece of data. I think I've said too much but not knowing enough I don't know what you JPA experts need. What I would like is my original object to have both a string object and be able to join on the same column (ihtran) and have it as a string too, but if this isn't possible or advisable I want to hear what I should do and why. Pseudo code/English is more than fine.

  Read the article

• Option Trading: Getting the most out of the event session options

  - by extended_events

  You can control different aspects of how an event session behaves by setting the event session options as part of the CREATE EVENT SESSION DDL. The default settings for the event session options are designed to handle most of the common event collection situations so I generally recommend that you just use the defaults. Like everything in the real world though, there are going to be a handful of “special cases” that require something different. This post focuses on identifying the special cases and the correct use of the options to accommodate those cases. There is a reason it’s called Default The default session options specify a total event buffer size of 4 MB with a 30 second latency. Translating this into human terms; this means that our default behavior is that the system will start processing events from the event buffer when we reach about 1.3 MB of events or after 30 seconds, which ever comes first. Aside: What’s up with the 1.3 MB, I thought you said the buffer was 4 MB?The Extended Events engine takes the total buffer size specified by MAX_MEMORY (4MB by default) and divides it into 3 equally sized buffers. This is done so that a session can be publishing events to one buffer while other buffers are being processed. There are always at least three buffers; how to get more than three is covered later. Using this configuration, the Extended Events engine can “keep up” with most event sessions on standard workloads. Why is this? The fact is that most events are small, really small; on the order of a couple hundred bytes. Even when you start considering events that carry dynamically sized data (eg. binary, text, etc.) or adding actions that collect additional data, the total size of the event is still likely to be pretty small. This means that each buffer can likely hold thousands of events before it has to be processed. When the event buffers are finally processed there is an economy of scale achieved since most targets support bulk processing of the events so they are processed at the buffer level rather than the individual event level. When all this is working together it’s more likely that a full buffer will be processed and put back into the ready queue before the remaining buffers (remember, there are at least three) are full. I know what you’re going to say: “My server is exceptional! My workload is so massive it defies categorization!” OK, maybe you weren’t going to say that exactly, but you were probably thinking it. The point is that there are situations that won’t be covered by the Default, but that’s a good place to start and this post assumes you’ve started there so that you have something to look at in order to determine if you do have a special case that needs different settings. So let’s get to the special cases… What event just fired?! How about now?! Now?! If you believe the commercial adage from Heinz Ketchup (Heinz Slow Good Ketchup ad on You Tube), some things are worth the wait. This is not a belief held by most DBAs, particularly DBAs who are looking for an answer to a troubleshooting question fast. If you’re one of these anxious DBAs, or maybe just a Program Manager doing a demo, then 30 seconds might be longer than you’re comfortable waiting. If you find yourself in this situation then consider changing the MAX_DISPATCH_LATENCY option for your event session. This option will force the event buffers to be processed based on your time schedule. This option only makes sense for the asynchronous targets since those are the ones where we allow events to build up in the event buffer – if you’re using one of the synchronous targets this option isn’t relevant. Avoid forgotten events by increasing your memory Have you ever had one of those days where you keep forgetting things? That can happen in Extended Events too; we call it dropped events. In order to optimizes for server performance and help ensure that the Extended Events doesn’t block the server if to drop events that can’t be published to a buffer because the buffer is full. You can determine if events are being dropped from a session by querying the dm_xe_sessions DMV and looking at the dropped_event_count field. Aside: Should you care if you’re dropping events?Maybe not – think about why you’re collecting data in the first place and whether you’re really going to miss a few dropped events. For example, if you’re collecting query duration stats over thousands of executions of a query it won’t make a huge difference to miss a couple executions. Use your best judgment. If you find that your session is dropping events it means that the event buffer is not large enough to handle the volume of events that are being published. There are two ways to address this problem. First, you could collect fewer events – examine you session to see if you are over collecting. Do you need all the actions you’ve specified? Could you apply a predicate to be more specific about when you fire the event? Assuming the session is defined correctly, the next option is to change the MAX_MEMORY option to a larger number. Picking the right event buffer size might take some trial and error, but a good place to start is with the number of dropped events compared to the number you’ve collected. Aside: There are three different behaviors for dropping events that you specify using the EVENT_RETENTION_MODE option. The default is to allow single event loss and you should stick with this setting since it is the best choice for keeping the impact on server performance low.You’ll be tempted to use the setting to not lose any events (NO_EVENT_LOSS) – resist this urge since it can result in blocking on the server. If you’re worried that you’re losing events you should be increasing your event buffer memory as described in this section. Some events are too big to fail A less common reason for dropping an event is when an event is so large that it can’t fit into the event buffer. Even though most events are going to be small, you might find a condition that occasionally generates a very large event. You can determine if your session is dropping large events by looking at the dm_xe_sessions DMV once again, this time check the largest_event_dropped_size. If this value is larger than the size of your event buffer [remember, the size of your event buffer, by default, is max_memory / 3] then you need a large event buffer. To specify a large event buffer you set the MAX_EVENT_SIZE option to a value large enough to fit the largest event dropped based on data from the DMV. When you set this option the Extended Events engine will create two buffers of this size to accommodate these large events. As an added bonus (no extra charge) the large event buffer will also be used to store normal events in the cases where the normal event buffers are all full and waiting to be processed. (Note: This is just a side-effect, not the intended use. If you’re dropping many normal events then you should increase your normal event buffer size.) Partitioning: moving your events to a sub-division Earlier I alluded to the fact that you can configure your event session to use more than the standard three event buffers – this is called partitioning and is controlled by the MEMORY_PARTITION_MODE option. The result of setting this option is fairly easy to explain, but knowing when to use it is a bit more art than science. First the science… You can configure partitioning in three ways: None, Per NUMA Node & Per CPU. This specifies the location where sets of event buffers are created with fairly obvious implication. There are rules we follow for sub-dividing the total memory (specified by MAX_MEMORY) between all the event buffers that are specific to the mode used: None: 3 buffers (fixed)Node: 3 * number_of_nodesCPU: 2.5 * number_of_cpus Here are some examples of what this means for different Node/CPU counts: Configuration None Node CPU 2 CPUs, 1 Node 3 buffers 3 buffers 5 buffers 6 CPUs, 2 Node 3 buffers 6 buffers 15 buffers 40 CPUs, 5 Nodes 3 buffers 15 buffers 100 buffers   Aside: Buffer size on multi-processor computersAs the number of Nodes or CPUs increases, the size of the event buffer gets smaller because the total memory is sub-divided into more pieces. The defaults will hold up to this for a while since each buffer set is holding events only from the Node or CPU that it is associated with, but at some point the buffers will get too small and you’ll either see events being dropped or you’ll get an error when you create your session because you’re below the minimum buffer size. Increase the MAX_MEMORY setting to an appropriate number for the configuration. The most likely reason to start partitioning is going to be related to performance. If you notice that running an event session is impacting the performance of your server beyond a reasonably expected level [Yes, there is a reasonably expected level of work required to collect events.] then partitioning might be an answer. Before you partition you might want to check a few other things: Is your event retention set to NO_EVENT_LOSS and causing blocking? (I told you not to do this.) Consider changing your event loss mode or increasing memory. Are you over collecting and causing more work than necessary? Consider adding predicates to events or removing unnecessary events and actions from your session. Are you writing the file target to the same slow disk that you use for TempDB and your other high activity databases? <kidding> <not really> It’s always worth considering the end to end picture – if you’re writing events to a file you can be impacted by I/O, network; all the usual stuff. Assuming you’ve ruled out the obvious (and not so obvious) issues, there are performance conditions that will be addressed by partitioning. For example, it’s possible to have a successful event session (eg. no dropped events) but still see a performance impact because you have many CPUs all attempting to write to the same free buffer and having to wait in line to finish their work. This is a case where partitioning would relieve the contention between the different CPUs and likely reduce the performance impact cause by the event session. There is no DMV you can check to find these conditions – sorry – that’s where the art comes in. This is  largely a matter of experimentation. On the bright side you probably won’t need to to worry about this level of detail all that often. The performance impact of Extended Events is significantly lower than what you may be used to with SQL Trace. You will likely only care about the impact if you are trying to set up a long running event session that will be part of your everyday workload – sessions used for short term troubleshooting will likely fall into the “reasonably expected impact” category. Hey buddy – I think you forgot something OK, there are two options I didn’t cover: STARTUP_STATE & TRACK_CAUSALITY. If you want your event sessions to start automatically when the server starts, set the STARTUP_STATE option to ON. (Now there is only one option I didn’t cover.) I’m going to leave causality for another post since it’s not really related to session behavior, it’s more about event analysis. - Mike Share this post: email it! | bookmark it! | digg it! | reddit! | kick it! | live it!

  Read the article

• SQL SERVER – Concurrency Basics – Guest Post by Vinod Kumar

  - by pinaldave

  This guest post is by Vinod Kumar. Vinod Kumar has worked with SQL Server extensively since joining the industry over a decade ago. Working on various versions from SQL Server 7.0, Oracle 7.3 and other database technologies – he now works with the Microsoft Technology Center (MTC) as a Technology Architect. Let us read the blog post in Vinod’s own voice. Learning is always fun when it comes to SQL Server and learning the basics again can be more fun. I did write about Transaction Logs and recovery over my blogs and the concept of simplifying the basics is a challenge. In the real world we always see checks and queues for a process – say railway reservation, banks, customer supports etc there is a process of line and queue to facilitate everyone. Shorter the queue higher is the efficiency of system (a.k.a higher is the concurrency). Every database does implement this using checks like locking, blocking mechanisms and they implement the standards in a way to facilitate higher concurrency. In this post, let us talk about the topic of Concurrency and what are the various aspects that one needs to know about concurrency inside SQL Server. Let us learn the concepts as one-liners: Concurrency can be defined as the ability of multiple processes to access or change shared data at the same time. The greater the number of concurrent user processes that can be active without interfering with each other, the greater the concurrency of the database system. Concurrency is reduced when a process that is changing data prevents other processes from reading that data or when a process that is reading data prevents other processes from changing that data. Concurrency is also affected when multiple processes are attempting to change the same data simultaneously. Two approaches to managing concurrent data access: Optimistic Concurrency Model Pessimistic Concurrency Model Concurrency Models Pessimistic Concurrency Default behavior: acquire locks to block access to data that another process is using. Assumes that enough data modification operations are in the system that any given read operation is likely affected by a data modification made by another user (assumes conflicts will occur). Avoids conflicts by acquiring a lock on data being read so no other processes can modify that data. Also acquires locks on data being modified so no other processes can access the data for either reading or modifying. Readers block writer, writers block readers and writers. Optimistic Concurrency Assumes that there are sufficiently few conflicting data modification operations in the system that any single transaction is unlikely to modify data that another transaction is modifying. Default behavior of optimistic concurrency is to use row versioning to allow data readers to see the state of the data before the modification occurs. Older versions of the data are saved so a process reading data can see the data as it was when the process started reading and not affected by any changes being made to that data. Processes modifying the data is unaffected by processes reading the data because the reader is accessing a saved version of the data rows. Readers do not block writers and writers do not block readers, but, writers can and will block writers. Transaction Processing A transaction is the basic unit of work in SQL Server. Transaction consists of SQL commands that read and update the database but the update is not considered final until a COMMIT command is issued (at least for an explicit transaction: marked with a BEGIN TRAN and the end is marked by a COMMIT TRAN or ROLLBACK TRAN). Transactions must exhibit all the ACID properties of a transaction. ACID Properties Transaction processing must guarantee the consistency and recoverability of SQL Server databases. Ensures all transactions are performed as a single unit of work regardless of hardware or system failure. A – Atomicity C – Consistency I – Isolation D- Durability Atomicity: Each transaction is treated as all or nothing – it either commits or aborts. Consistency: ensures that a transaction won’t allow the system to arrive at an incorrect logical state – the data must always be logically correct.  Consistency is honored even in the event of a system failure. Isolation: separates concurrent transactions from the updates of other incomplete transactions. SQL Server accomplishes isolation among transactions by locking data or creating row versions. Durability: After a transaction commits, the durability property ensures that the effects of the transaction persist even if a system failure occurs. If a system failure occurs while a transaction is in progress, the transaction is completely undone, leaving no partial effects on data. Transaction Dependencies In addition to supporting all four ACID properties, a transaction might exhibit few other behaviors (known as dependency problems or consistency problems). Lost Updates: Occur when two processes read the same data and both manipulate the data, changing its value and then both try to update the original data to the new value. The second process might overwrite the first update completely. Dirty Reads: Occurs when a process reads uncommitted data. If one process has changed data but not yet committed the change, another process reading the data will read it in an inconsistent state. Non-repeatable Reads: A read is non-repeatable if a process might get different values when reading the same data in two reads within the same transaction. This can happen when another process changes the data in between the reads that the first process is doing. Phantoms: Occurs when membership in a set changes. It occurs if two SELECT operations using the same predicate in the same transaction return a different number of rows. Isolation Levels SQL Server supports 5 isolation levels that control the behavior of read operations. Read Uncommitted All behaviors except for lost updates are possible. Implemented by allowing the read operations to not take any locks, and because of this, it won’t be blocked by conflicting locks acquired by other processes. The process can read data that another process has modified but not yet committed. When using the read uncommitted isolation level and scanning an entire table, SQL Server can decide to do an allocation order scan (in page-number order) instead of a logical order scan (following page pointers). If another process doing concurrent operations changes data and move rows to a new location in the table, the allocation order scan can end up reading the same row twice. Also can happen if you have read a row before it is updated and then an update moves the row to a higher page number than your scan encounters later. Performing an allocation order scan under Read Uncommitted can cause you to miss a row completely – can happen when a row on a high page number that hasn’t been read yet is updated and moved to a lower page number that has already been read. Read Committed Two varieties of read committed isolation: optimistic and pessimistic (default). Ensures that a read never reads data that another application hasn’t committed. If another transaction is updating data and has exclusive locks on data, your transaction will have to wait for the locks to be released. Your transaction must put share locks on data that are visited, which means that data might be unavailable for others to use. A share lock doesn’t prevent others from reading but prevents them from updating. Read committed (snapshot) ensures that an operation never reads uncommitted data, but not by forcing other processes to wait. SQL Server generates a version of the changed row with its previous committed values. Data being changed is still locked but other processes can see the previous versions of the data as it was before the update operation began. Repeatable Read This is a Pessimistic isolation level. Ensures that if a transaction revisits data or a query is reissued the data doesn’t change. That is, issuing the same query twice within a transaction cannot pickup any changes to data values made by another user’s transaction because no changes can be made by other transactions. However, this does allow phantom rows to appear. Preventing non-repeatable read is a desirable safeguard but cost is that all shared locks in a transaction must be held until the completion of the transaction. Snapshot Snapshot Isolation (SI) is an optimistic isolation level. Allows for processes to read older versions of committed data if the current version is locked. Difference between snapshot and read committed has to do with how old the older versions have to be. It’s possible to have two transactions executing simultaneously that give us a result that is not possible in any serial execution. Serializable This is the strongest of the pessimistic isolation level. Adds to repeatable read isolation level by ensuring that if a query is reissued rows were not added in the interim, i.e, phantoms do not appear. Preventing phantoms is another desirable safeguard, but cost of this extra safeguard is similar to that of repeatable read – all shared locks in a transaction must be held until the transaction completes. In addition serializable isolation level requires that you lock data that has been read but also data that doesn’t exist. Ex: if a SELECT returned no rows, you want it to return no. rows when the query is reissued. This is implemented in SQL Server by a special kind of lock called the key-range lock. Key-range locks require that there be an index on the column that defines the range of values. If there is no index on the column, serializable isolation requires a table lock. Gets its name from the fact that running multiple serializable transactions at the same time is equivalent of running them one at a time. Now that we understand the basics of what concurrency is, the subsequent blog posts will try to bring out the basics around locking, blocking, deadlocks because they are the fundamental blocks that make concurrency possible. Now if you are with me – let us continue learning for SQL Server Locking Basics. Reference: Pinal Dave (http://blog.sqlauthority.com) Filed under: PostADay, SQL, SQL Authority, SQL Performance, SQL Query, SQL Server, SQL Tips and Tricks, T SQL, Technology Tagged: Concurrency

  Read the article

• Option Trading: Getting the most out of the event session options

  - by extended_events

  You can control different aspects of how an event session behaves by setting the event session options as part of the CREATE EVENT SESSION DDL. The default settings for the event session options are designed to handle most of the common event collection situations so I generally recommend that you just use the defaults. Like everything in the real world though, there are going to be a handful of “special cases” that require something different. This post focuses on identifying the special cases and the correct use of the options to accommodate those cases. There is a reason it’s called Default The default session options specify a total event buffer size of 4 MB with a 30 second latency. Translating this into human terms; this means that our default behavior is that the system will start processing events from the event buffer when we reach about 1.3 MB of events or after 30 seconds, which ever comes first. Aside: What’s up with the 1.3 MB, I thought you said the buffer was 4 MB?The Extended Events engine takes the total buffer size specified by MAX_MEMORY (4MB by default) and divides it into 3 equally sized buffers. This is done so that a session can be publishing events to one buffer while other buffers are being processed. There are always at least three buffers; how to get more than three is covered later. Using this configuration, the Extended Events engine can “keep up” with most event sessions on standard workloads. Why is this? The fact is that most events are small, really small; on the order of a couple hundred bytes. Even when you start considering events that carry dynamically sized data (eg. binary, text, etc.) or adding actions that collect additional data, the total size of the event is still likely to be pretty small. This means that each buffer can likely hold thousands of events before it has to be processed. When the event buffers are finally processed there is an economy of scale achieved since most targets support bulk processing of the events so they are processed at the buffer level rather than the individual event level. When all this is working together it’s more likely that a full buffer will be processed and put back into the ready queue before the remaining buffers (remember, there are at least three) are full. I know what you’re going to say: “My server is exceptional! My workload is so massive it defies categorization!” OK, maybe you weren’t going to say that exactly, but you were probably thinking it. The point is that there are situations that won’t be covered by the Default, but that’s a good place to start and this post assumes you’ve started there so that you have something to look at in order to determine if you do have a special case that needs different settings. So let’s get to the special cases… What event just fired?! How about now?! Now?! If you believe the commercial adage from Heinz Ketchup (Heinz Slow Good Ketchup ad on You Tube), some things are worth the wait. This is not a belief held by most DBAs, particularly DBAs who are looking for an answer to a troubleshooting question fast. If you’re one of these anxious DBAs, or maybe just a Program Manager doing a demo, then 30 seconds might be longer than you’re comfortable waiting. If you find yourself in this situation then consider changing the MAX_DISPATCH_LATENCY option for your event session. This option will force the event buffers to be processed based on your time schedule. This option only makes sense for the asynchronous targets since those are the ones where we allow events to build up in the event buffer – if you’re using one of the synchronous targets this option isn’t relevant. Avoid forgotten events by increasing your memory Have you ever had one of those days where you keep forgetting things? That can happen in Extended Events too; we call it dropped events. In order to optimizes for server performance and help ensure that the Extended Events doesn’t block the server if to drop events that can’t be published to a buffer because the buffer is full. You can determine if events are being dropped from a session by querying the dm_xe_sessions DMV and looking at the dropped_event_count field. Aside: Should you care if you’re dropping events?Maybe not – think about why you’re collecting data in the first place and whether you’re really going to miss a few dropped events. For example, if you’re collecting query duration stats over thousands of executions of a query it won’t make a huge difference to miss a couple executions. Use your best judgment. If you find that your session is dropping events it means that the event buffer is not large enough to handle the volume of events that are being published. There are two ways to address this problem. First, you could collect fewer events – examine you session to see if you are over collecting. Do you need all the actions you’ve specified? Could you apply a predicate to be more specific about when you fire the event? Assuming the session is defined correctly, the next option is to change the MAX_MEMORY option to a larger number. Picking the right event buffer size might take some trial and error, but a good place to start is with the number of dropped events compared to the number you’ve collected. Aside: There are three different behaviors for dropping events that you specify using the EVENT_RETENTION_MODE option. The default is to allow single event loss and you should stick with this setting since it is the best choice for keeping the impact on server performance low.You’ll be tempted to use the setting to not lose any events (NO_EVENT_LOSS) – resist this urge since it can result in blocking on the server. If you’re worried that you’re losing events you should be increasing your event buffer memory as described in this section. Some events are too big to fail A less common reason for dropping an event is when an event is so large that it can’t fit into the event buffer. Even though most events are going to be small, you might find a condition that occasionally generates a very large event. You can determine if your session is dropping large events by looking at the dm_xe_sessions DMV once again, this time check the largest_event_dropped_size. If this value is larger than the size of your event buffer [remember, the size of your event buffer, by default, is max_memory / 3] then you need a large event buffer. To specify a large event buffer you set the MAX_EVENT_SIZE option to a value large enough to fit the largest event dropped based on data from the DMV. When you set this option the Extended Events engine will create two buffers of this size to accommodate these large events. As an added bonus (no extra charge) the large event buffer will also be used to store normal events in the cases where the normal event buffers are all full and waiting to be processed. (Note: This is just a side-effect, not the intended use. If you’re dropping many normal events then you should increase your normal event buffer size.) Partitioning: moving your events to a sub-division Earlier I alluded to the fact that you can configure your event session to use more than the standard three event buffers – this is called partitioning and is controlled by the MEMORY_PARTITION_MODE option. The result of setting this option is fairly easy to explain, but knowing when to use it is a bit more art than science. First the science… You can configure partitioning in three ways: None, Per NUMA Node & Per CPU. This specifies the location where sets of event buffers are created with fairly obvious implication. There are rules we follow for sub-dividing the total memory (specified by MAX_MEMORY) between all the event buffers that are specific to the mode used: None: 3 buffers (fixed)Node: 3 * number_of_nodesCPU: 2.5 * number_of_cpus Here are some examples of what this means for different Node/CPU counts: Configuration None Node CPU 2 CPUs, 1 Node 3 buffers 3 buffers 5 buffers 6 CPUs, 2 Node 3 buffers 6 buffers 15 buffers 40 CPUs, 5 Nodes 3 buffers 15 buffers 100 buffers   Aside: Buffer size on multi-processor computersAs the number of Nodes or CPUs increases, the size of the event buffer gets smaller because the total memory is sub-divided into more pieces. The defaults will hold up to this for a while since each buffer set is holding events only from the Node or CPU that it is associated with, but at some point the buffers will get too small and you’ll either see events being dropped or you’ll get an error when you create your session because you’re below the minimum buffer size. Increase the MAX_MEMORY setting to an appropriate number for the configuration. The most likely reason to start partitioning is going to be related to performance. If you notice that running an event session is impacting the performance of your server beyond a reasonably expected level [Yes, there is a reasonably expected level of work required to collect events.] then partitioning might be an answer. Before you partition you might want to check a few other things: Is your event retention set to NO_EVENT_LOSS and causing blocking? (I told you not to do this.) Consider changing your event loss mode or increasing memory. Are you over collecting and causing more work than necessary? Consider adding predicates to events or removing unnecessary events and actions from your session. Are you writing the file target to the same slow disk that you use for TempDB and your other high activity databases? <kidding> <not really> It’s always worth considering the end to end picture – if you’re writing events to a file you can be impacted by I/O, network; all the usual stuff. Assuming you’ve ruled out the obvious (and not so obvious) issues, there are performance conditions that will be addressed by partitioning. For example, it’s possible to have a successful event session (eg. no dropped events) but still see a performance impact because you have many CPUs all attempting to write to the same free buffer and having to wait in line to finish their work. This is a case where partitioning would relieve the contention between the different CPUs and likely reduce the performance impact cause by the event session. There is no DMV you can check to find these conditions – sorry – that’s where the art comes in. This is  largely a matter of experimentation. On the bright side you probably won’t need to to worry about this level of detail all that often. The performance impact of Extended Events is significantly lower than what you may be used to with SQL Trace. You will likely only care about the impact if you are trying to set up a long running event session that will be part of your everyday workload – sessions used for short term troubleshooting will likely fall into the “reasonably expected impact” category. Hey buddy – I think you forgot something OK, there are two options I didn’t cover: STARTUP_STATE & TRACK_CAUSALITY. If you want your event sessions to start automatically when the server starts, set the STARTUP_STATE option to ON. (Now there is only one option I didn’t cover.) I’m going to leave causality for another post since it’s not really related to session behavior, it’s more about event analysis. - Mike Share this post: email it! | bookmark it! | digg it! | reddit! | kick it! | live it!

  Read the article

< Previous Page | 12 13 14 15 16 17  | Next Page >