test data - Page 520 - Developer IT

nginx, should all files served by owned by www-data?

- by Blankman

With nginx, should all files being served be owned by www-data? (assuming nginx is running under the context of www-data). Or is root ok?

Cloud storage provider lost my data. How to back up next time?

What do you do when cloud storage fails you? First, some background. A popular cloud storage provider (rhymes with Booger Link) damaged a bunch of my data. Getting it back was an uphill battle with all the usual accusations that it was my fault, etc. Finally I got the data back. Yes, I can back this up with evidence. Idiotically, I stayed with them, so I totally get that the rest of this is on me. The problem had been with a shared folder that works with all 12 computers my business and family use with the service. We'll call that folder the Tragic Briefcase. It is a sort of global folder that's publicly visible to all computers on the service. It's our main repository. Today I decided to deal with some residual effects of the Crash of '11. Part of the damage they did was that in just one of my computers (my primary, of course) all the documents in the Tragic Briefcase were duplicated in the Windows My Documents folder. I finally started deleting them. But guess what. Though they appeared to be duplicated in the file system, removing them from My Documents on the primary PC caused them to disappear from the Tragic Briefcase too. They efficiently disappeared from all the other computers' Tragic Briefcases as well. So now, 21 gigs of files are gone, and of course I don't know which ones. I want to avoid this in the future. Apart from using a different storage provider, the bigger picture is this: how do I back up my cloud data? A complete backup every week or so from web to local storage would cause me to exceed my ISP's bandwidth. Do I need to back up each of my 12 PCs locally? I do use Backupify for my primary Google Docs, but I have been storing taxes, confidential documents, Photoshop source, video source files, and so on using the web service. So it's a lot of data, but I need to keep it safe. Backup locally would also mean 2 backup drives or some kind of RAID per PC, right, because you can't trust a single point of failure? Assuming I move to DropBox or something of its ilk, what is the best way to make sure that if the next cloud storage provider messes up I can restore?

Read the article

"Test to measure your ability to follow directions and solve complex problems in a neat and orderly manner." [closed]

- by Matt

Use the table of symbol substitutions when answering the problem below: Circle = 0 Dot = 1 Line = 2 Triangle = 3 Square = 4 Pentagon = 5 Hexagon = 6 Cross = 7 Heart = 8 Smiley Face = 9 Use the following special rule when answering the problem below: If ever a square is next to a cross during long multiplication, the square shall be treated as a triangle. Problem: Show your work in doing long multiplication of Pentagon Pentagon Nonagon by Line Square Octagon. Show your work using symbols not numbers.

Read the article

How can Icheck how much data has been transfered in a particular remote computer in my LAN?

- by oDx

I am the system administrator. I am suspecting some unwanted data transfers in some of the computers under my lan. This also effects the overall performance of the network. Is there anyway by which i can check the transfered data over a particular interval. I am not supposed to install any applications on any other computers than mine. I have the router access. I can use tools like ettercap dsniff wireshark etc.

Read the article

how to connect and select data from a JDBC over RMI database in perl?

- by ericslaw

I have a 3rd party application that exposes in-memory data structures via a JDBC api over RMI. How can I get data out of that database api without having to run squirrel or similar GUI application? Perl solution preferred, but java solution acceptable.

Read the article

How can I check how much data has been transfered in a particular remote computer in my LAN?

- by oDx

I am the system administrator. I am suspecting some unwanted data transfers in some of the computers under my lan. This also effects the overall performance of the network. Is there anyway by which i can check the transfered data over a particular interval. I am not supposed to install any applications on any other computers than mine. I have the router access. I can use tools like ettercap dsniff wireshark etc.

Read the article

What is the best way/tool to analyze raw data(network stats) from Simulation?

- by user90500

After running a simulation(using a simulator(QualNet)) of a simulated network I end up with ip stats stored in a database, I then extract the data to a csv file So now I have 750mb of raw network stats(time stamp, packet id, source ip, source port, protocol, etc). What are the common ways of analyzing large amounts of data like above, if you want to know things like packet loss, throughput, delay, congestion, etc.

Read the article

PHP SimpleXML recursive function to list children and attibutes

- by Phill Pafford

I need some help on the SimpleXML calls for a recursive function that lists the elements name and attributes. Making a XML config file system but each script will have it's own config file as well as a new naming convention. So what I need is an easy way to map out all the elements that have attributes, so like in example 1 I need a simple way to call all the processes but I don't know how to do this without hard coding the elements name is the function call. Is there a way to recursively call a function to match a child element name? I did see the xpath functionality but I don't see how to use this for attributes. Any ideas? Also does the XML in the examples look correct? can I structure my XML like this? Example 1: <application> <processes> <process id="123" name="run batch A" /> <process id="122" name="run batch B" /> <process id="129" name="run batch C" /> </processes> <connections> <databases> <database usr="test" pss="test" hst="test" dbn="test" /> </databases> <shells> <ssh usr="test" pss="test" hst="test-2" /> <ssh usr="test" pss="test" hst="test-1" /> </shells> </connections> </application> Example 2: <config> <queues> <queue id="1" name="test" /> <queue id="2" name="production" /> <queue id="3" name="error" /> </queues> </config> Pseudo code: // Would return matching process id getProcess($process_id) { return the process attributes as array that are in the XML } // Would return matching DBN (database name) getDatabase($database_name) { return the database attributes as array that are in the XML } // Would return matching SSH Host getSSHHost($ssh_host) { return the ssh attributes as array that are in the XML } // Would return matching SSH User getSSHUser($ssh_user) { return the ssh attributes as array that are in the XML } // Would return matching Queue getQueue($queue_id) { return the queue attributes as array that are in the XML } EDIT: Can I pass two parms? on the first method you have suggested @Gordon public function findProcessById($id, $name) { $attr = false; $el = $this->xml->xpath("//process[@id='$id']"); // How do I also filter by the name? if($el && count($el) === 1) { $attr = (array) $el[0]->attributes(); $attr = $attr['@attributes']; } return $attr; }

Read the article

XSL unique values per node

- by Nathan

ok i have this xml <roots> <root> <name>first</name> <item type='test'><something>A</something></item> <item type='test'><something>B</something></item> <item type='test'><something>C</something></item> <item type='test'><something>A</something></item> <item type='other'><something>A</something></item> <item type='test'><something>B</something></item> <item type='other'><something>D</something></item> </root> <root> <name>second</name> <item type='test'><something>E</something></item> <item type='test'><something>B</something></item> <item type='test'><something>F</something></item> <item type='test'><something>A</something></item> <item type='other'><something>A</something></item> <item type='test'><something>B</something></item> <item type='other'><something>D</something></item> </root> </roots> now i need to get the unique values of each root node so far i have <?xml version="1.0" encoding="ISO-8859-1"?> <xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"> <xsl:output indent="yes" method="text"/> <xsl:key name="item-by-value" match="something" use="."/> <xsl:key name="rootkey" match="root" use="name"/> <xsl:template match="/"> <xsl:for-each select="key('rootkey','second')"> <xsl:for-each select="item/something"> <xsl:if test="generate-id() = generate-id(key('item-by-value', normalize-space(.)))"> <xsl:value-of select="."/> </xsl:if> </xsl:for-each> </xsl:for-each> </xsl:template> </xsl:stylesheet> if i use "First" as the key to get only the first root i get a good result ABCD how ever if i use "second" i only get EF but i need the result to be ABDFE

Read the article

file.createNewFile() creates files with last-modified time before actual creation time

- by Kaleb Pederson

I'm using JPoller to detect changes to files in a specific directory, but it's missing files because they end up with a timestamp earlier than their actual creation time. Here's how I test: public static void main(String [] files) { for (String file : files) { File f = new File(file); if (f.exists()) { System.err.println(file + " exists"); continue; } try { // find out the current time, I would hope to assume that the last-modified // time on the file will definitely be later than this System.out.println("-----------------------------------------"); long time = System.currentTimeMillis(); // create the file System.out.println("Creating " + file + " at " + time); f.createNewFile(); // let's see what the timestamp actually is (I've only seen it <time) System.out.println(file + " was last modified at: " + f.lastModified()); // well, ok, what if I explicitly set it to time? f.setLastModified(time); System.out.println("Updated modified time on " + file + " to " + time + " with actual " + f.lastModified()); } catch (IOException e) { System.err.println("Unable to create file"); } } } And here's what I get for output: ----------------------------------------- Creating test.7 at 1272324597956 test.7 was last modified at: 1272324597000 Updated modified time on test.7 to 1272324597956 with actual 1272324597000 ----------------------------------------- Creating test.8 at 1272324597957 test.8 was last modified at: 1272324597000 Updated modified time on test.8 to 1272324597957 with actual 1272324597000 ----------------------------------------- Creating test.9 at 1272324597957 test.9 was last modified at: 1272324597000 Updated modified time on test.9 to 1272324597957 with actual 1272324597000 The result is a race condition: JPoller records time of last check as xyz...123 File created at xyz...456 File last-modified timestamp actually reads xyz...000 JPoller looks for new/updated files with timestamp greater than xyz...123 JPoller ignores newly added file because xyz...000 is less than xyz...123 I pull my hair out for a while I tried digging into the code but both lastModified() and createNewFile() eventually resolve to native calls so I'm left with little information. For test.9, I lose 957 milliseconds. What kind of accuracy can I expect? Are my results going to vary by operating system or file system? Suggested workarounds? NOTE: I'm currently running Linux with an XFS filesystem. I wrote a quick program in C and the stat system call shows st_mtime as truncate(xyz...000/1000).

Read the article

Yet another Ant + JUnit classpath problem

- by user337591

Hi, I'm developing an Eclipse SWT application using Eclipse. There are also some JUnit 4 tests, which test some DAO's. But when I try to run the tests via an ant build, all of the tests fail, because the test classes aren't found. Google brought up about a million of people who all have the same problem, but none of their solutions seem to work for me -.- . These are the contents of my build.xml file: <property name="test.reports" value="./test/reports" /> <property name="classes" value="build" /> <path id="project.classpath"> <pathelement location="${classes}" /> </path> <target name="testreport"> <mkdir dir="${test.reports}" /> <junit fork="yes" printsummary="no" haltonfailure="no"> <batchtest fork="yes" todir="${test.reports}" > <fileset dir="${classes}"> <include name="**/Test*.class" /> </fileset> </batchtest> <formatter type="xml" /> <classpath refid="project.classpath" /> </junit> <junitreport todir="${test.reports}"> <fileset dir="${test.reports}"> <include name="TEST-*.xml" /> </fileset> <report todir="${test.reports}" /> </junitreport> </target> The test classes are in the build-directory together with the application classes, although they are in some subfolders according to their packages. Maybe this is important too: At first Ant complained that JUnit wasn't in its classpath, but since I put it there (with the eclipse configuration editor) it complains about JUnit being in its classpath twice. WARNING: multiple versions of ant detected in path for junit [junit] jar:file:C:/Users/as df/Documents/eclipse/plugins/org.apache.ant_1.7.1.v20090120-1145/lib/ant.jar!/org/apache/tools/ant/Project.class [junit] and jar:file:/C:/Users/as%20df/Documents/eclipse/plugins/org.apache.ant_1.7.1.v20090120-1145/lib/ant.jar!/org/apache/tools/ant/Project.class I've tried specifying each and every subdirectory, each and every class file, I've tried filesets and filelists, nothing seems to work. Thanks for your help, I've been sitting for hours on this thing now...

Read the article

Loading jar file using JCL(JarClassLoader ) : classpath in manifest is ignored ..

- by Xinus

I am trying to load jar file using JCL using following code FileInputStream fis = new FileInputStream(new File( "C:\\Users\\sunils\\glassfish-tests\\working\\test.jar") ); JarClassLoader jc = new JarClassLoader( ); jc.add(fis); Class main = jc.loadClass( "highmark.test.Main" ); String[] str={}; main.getMethod("test").invoke(null);//.getDeclaredMethod("main",String[].class).invoke(null,str); fis.close(); But when I try to run this program I get Exception as Exception in thread "main" java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at Main.main(Main.java:21) Caused by: java.lang.RuntimeException: Embedded startup not found, classpath is probably incomplete at org.glassfish.api.embedded.Server.<init>(Server.java:292) at org.glassfish.api.embedded.Server.<init>(Server.java:75) at org.glassfish.api.embedded.Server$Builder.build(Server.java:185) at org.glassfish.api.embedded.Server$Builder.build(Server.java:167) at highmark.test.Main.test(Main.java:33) ... 5 more According to this it is not able to locate class, But when I run the jar file explicitly it runs fine. It seems like JCL is ignoring other classes present in the jar file, MANIFEST.MF file in jar file shows: Manifest-Version: 1.0 Class-Path: . Main-Class: highmark.test.Main It seems to be ignoring Class-Path: . , This jar file runs fine when I run it using Java explicitly, This is just a test, in reality this jar file is coming as a InputStream and it cannot be stored in filesystem, How can I overcome this problem , Is there any workaround ? Thanks for any help . UNDATE: Here is a jar Main class : package highmark.test; import org.glassfish.api.embedded.*; import java.io.*; import org.glassfish.api.deployment.*; import com.sun.enterprise.universal.io.FileUtils; public class Main { public static void main(String[] args) throws IOException, LifecycleException, ClassNotFoundException { test(); } public static void test() throws IOException, LifecycleException, ClassNotFoundException{ Server.Builder builder = new Server.Builder("test"); Server server = builder.build(); server.createPort(8080); ContainerBuilder containerBuilder = server.createConfig(ContainerBuilder.Type.web); server.addContainer(containerBuilder); server.start(); File war=new File("C:\\Users\\sunils\\maventests\\simple-webapp\\target\\simple-webapp.war");//(File) inputStream.readObject(); EmbeddedDeployer deployer = server.getDeployer(); DeployCommandParameters params = new DeployCommandParameters(); params.contextroot = "simple"; deployer.deploy(war, params); } }

Read the article

Unit testing class in a web service in .net

- by Dan Bailiff

After some digging here, I took the advice in this thread: http://stackoverflow.com/questions/371961/how-to-unit-test-c-web-service-with-visual-studio-2008 I've created a separate class and my web service class is just a wrapper for that one. The problem is that when I try to create a unit test project in VS2008, it insists on creating a unit test that acts like I'm testing the web service calls instead of the class I specified. I can't get to the class I'm trying to test. I have a web service "subscription_api.asmx". The code behind is "subscription_api.cs" which contains the web method wrapper calls to the real code at "subscription.cs". I would expect to be able to do the following: [TestMethod()] public void GetSystemStatusTest() { subscription sub = new subscription(); XmlNode node = sub.GetSystemStatusTest(); Assert.IsNotNull(node); } But instead I get this mess which is autogenerated from VS'08: /// <summary> ///A test for GetSystemStatus ///</summary> // TODO: Ensure that the UrlToTest attribute specifies a URL to an ASP.NET page (for example, // http://.../Default.aspx). This is necessary for the unit test to be executed on the web server, // whether you are testing a page, web service, or a WCF service. [TestMethod()] [HostType("ASP.NET")] [AspNetDevelopmentServerHost("C:\\CVSROOT\\rnr\\pro\\product\\wms\\ss\\subscription_api", "/subscription_api")] [UrlToTest("http://localhost/subscription_api")] public void GetSystemStatusTest() { subscription_Accessor target = new subscription_Accessor(); // TODO: Initialize to an appropriate value XmlNode expected = null; // TODO: Initialize to an appropriate value XmlNode actual; actual = target.GetSystemStatus(); Assert.AreEqual(expected, actual); Assert.Inconclusive("Verify the correctness of this test method."); } Additionally, there is a "subscription_api.accessor" in the Test References folder. When I try this: [TestMethod()] public void GetSystemStatusTest2() { subscription_Accessor sub = new subscription_Accessor(); XmlNode node = sub.GetSystemStatus(); Assert.IsNotNull(node); } I get an error: Test method subscription_api.Test.subscriptionTest.GetSystemStatusTest2 threw exception: System.TypeInitializationException: The type initializer for 'subscription_Accessor' threw an exception. ---> System.ArgumentNullException: Value cannot be null. I'm really new to unit testing and feel lost. How can I create a unit test just for my subscription class in "subscription.cs" without testing the web service? Am I limited to testing within the same project (I hope not)? Do I have to put the target class in its own project outside of the web service project?

Read the article

How to return this XML-RPC response in an array using PHP?

- by mind.blank

I'm trying to put together a WordPress plugin and I want to grab a list of all categories (of other WordPress blogs) via XML-RPC. I have the following code and it looks like it works so far: function get_categories($rpcurl,$username,$password){ $rpcurl2 = $rpcurl."/xmlrpc.php"; $params = array(0,$username,$password,true); $request = xmlrpc_encode_request('metaWeblog.getCategories',$params); $ch = curl_init(); curl_setopt($ch, CURLOPT_URL, $rpcurl2); curl_setopt($ch, CURLOPT_HEADER, false); curl_setopt($ch, CURLOPT_HTTPHEADER, array("Content-Type: text/xml")); curl_setopt($ch, CURLOPT_RETURNTRANSFER, true); curl_setopt($ch, CURLOPT_TIMEOUT, 10); curl_setopt($ch, CURLOPT_POSTFIELDS, $request); $results = curl_exec($ch); $res = xmlrpc_decode($results); curl_close($ch); return $res; } If I use $res I get the following string as the response: Array If I use $results then I get: categoryId17 parentId0 descriptionTest categoryDescription categoryNameTest htmlUrlhttp://test.yoursite.com/?cat=17 rssUrlhttp://test.yoursite.com/?feed=rss2&cat=17 categoryId1 parentId0 descriptionUncategorized categoryDescription categoryNameUncategorized htmlUrlhttp://test.yoursite.com/?cat=1 rssUrlhttp://test.yoursite.com/?feed=rss2&cat=1 I need to pull out the names after description so Uncategorized and Test in this case. It's my first time coding in PHP. I got these results by echoing them to the page, so not sure if they get changed in that process or not... By the way I modified the above code from one that posts to a WordPress blog remotely so maybe I haven't set some of the options correctly? With var_dump($res) I get: array(2) { [0]=> array(7) { ["categoryId"]=> string(2) "17" ["parentId"]=> string(1) "0" ["description"]=> string(4) "Test" ["categoryDescription"]=> string(0) "" ["categoryName"]=> string(4) "Test" ["htmlUrl"]=> string(40) "http://test.youreventwebsite.com/?cat=17" ["rssUrl"]=> string(54) "http://test.youreventwebsite.com/?feed=rss2&cat=17" } [1]=> array(7) { ["categoryId"]=> string(1) "1" ["parentId"]=> string(1) "0" ["description"]=> string(13) "Uncategorized" ["categoryDescription"]=> string(0) "" ["categoryName"]=> string(13) "Uncategorized" ["htmlUrl"]=> string(39) "http://test.youreventwebsite.com/?cat=1" ["rssUrl"]=> string(53) "http://test.youreventwebsite.com/?feed=rss2&cat=1" } }

Read the article

Pure Server-Side Filtering with RadGridView and WCF RIA Services

Those of you who are familiar with WCF RIA Services know that the DomainDataSource control provides a FilterDescriptors collection that enables you to filter data returned by the query on the server. We have been using this DomainDataSource feature in our RIA Services with DomainDataSource online example for almost an year now. In the example, we are listening for RadGridViews Filtering event in order to intercept any filtering that is performed on the client and translate it to something that the DomainDataSource will understand, in this case a System.Windows.Data.FilterDescriptor being added or removed from its FilterDescriptors collection. Think of RadGridView.FilterDescriptors as client-side filtering and of DomainDataSource.FilterDescriptors as server-side filtering. We no longer need the client-side one. With the introduction of the Custom Filtering Controls feature many new possibilities have opened. With these custom controls we no longer need to do any filtering on the client. I have prepared a very small project that demonstrates how to filter solely on the server by using a custom filtering control. As I have already mentioned filtering on the server is done through the FilterDescriptors collection of the DomainDataSource control. This collection holds instances of type System.Windows.Data.FilterDescriptor. The FilterDescriptor has three important properties: PropertyPath: Specifies the name of the property that we want to filter on (the left operand). Operator: Specifies the type of comparison to use when filtering. An instance of FilterOperator Enumeration. Value: The value to compare with (the right operand). An instance of the Parameter Class. By adding filters, you can specify that only entities which meet the condition in the filter are loaded from the domain context. In case you are not familiar with these concepts you might find Brad Abrams blog interesting. Now, our requirements are to create some kind of UI that will manipulate the DomainDataSource.FilterDescriptors collection. When it comes to collections, my first choice of course would be RadGridView. If you are not familiar with the Custom Filtering Controls concept I would strongly recommend getting acquainted with my step-by-step tutorial Custom Filtering with RadGridView for Silverlight and checking the online example out. I have created a simple custom filtering control that contains a RadGridView and several buttons. This control is aware of the DomainDataSource instance, since it is operating on its FilterDescriptors collection. In fact, the RadGridView that is inside it is bound to this collection. In order to display filters that are relevant for the current column only, I have applied a filter to the grid. This filter is a Telerik.Windows.Data.FilterDescriptor and is used to filter the little grid inside the custom control. It should not be confused with the DomainDataSource.FilterDescriptors collection that RadGridView is actually bound to. These are the RIA filters. Additionally, I have added several other features. For example, if you have specified a DataFormatString on your original column, the Value column inside the custom control will pick it up and format the filter values accordingly. Also, I have transferred the data type of the column that you are filtering to the Value column of the custom control. This will help the little RadGridView determine what kind of editor to show up when you begin edit, for example a date picker for DateTime columns. Finally, I have added four buttons two of them can be used to add or remove filters and the other two will communicate the changes you have made to the server. Here is the full source code of the DomainDataSourceFilteringControl. The XAML: <UserControl x:Class="PureServerSideFiltering.DomainDataSourceFilteringControl" xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation" xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml" xmlns:telerikGrid="clr-namespace:Telerik.Windows.Controls;assembly=Telerik.Windows.Controls.GridView" xmlns:telerik="clr-namespace:Telerik.Windows.Controls;assembly=Telerik.Windows.Controls" Width="300"> <Border x:Name="LayoutRoot" BorderThickness="1" BorderBrush="#FF8A929E" Padding="5" Background="#FFDFE2E5"> <Grid> <Grid.RowDefinitions> <RowDefinition Height="Auto"/> <RowDefinition Height="150"/> <RowDefinition Height="Auto"/> </Grid.RowDefinitions> <StackPanel Grid.Row="0" Margin="2" Orientation="Horizontal" HorizontalAlignment="Center"> <telerik:RadButton Name="addFilterButton" Click="OnAddFilterButtonClick" Content="Add Filter" Margin="2" Width="96"/> <telerik:RadButton Name="removeFilterButton" Click="OnRemoveFilterButtonClick" Content="Remove Filter" Margin="2" Width="96"/> </StackPanel> <telerikGrid:RadGridView Name="filtersGrid" Grid.Row="1" Margin="2" ItemsSource="{Binding FilterDescriptors}" AddingNewDataItem="OnFilterGridAddingNewDataItem" ColumnWidth="*" ShowGroupPanel="False" AutoGenerateColumns="False" CanUserResizeColumns="False" CanUserReorderColumns="False" CanUserFreezeColumns="False" RowIndicatorVisibility="Collapsed" IsFilteringAllowed="False" CanUserSortColumns="False"> <telerikGrid:RadGridView.Columns> <telerikGrid:GridViewComboBoxColumn DataMemberBinding="{Binding Operator}" UniqueName="Operator"/> <telerikGrid:GridViewDataColumn Header="Value" DataMemberBinding="{Binding Value.Value}" UniqueName="Value"/> </telerikGrid:RadGridView.Columns> </telerikGrid:RadGridView> <StackPanel Grid.Row="2" Margin="2" Orientation="Horizontal" HorizontalAlignment="Center"> <telerik:RadButton Name="filterButton" Click="OnApplyFiltersButtonClick" Content="Apply Filters" Margin="2" Width="96"/> <telerik:RadButton Name="clearButton" Click="OnClearFiltersButtonClick" Content="Clear Filters" Margin="2" Width="96"/> </StackPanel> </Grid> </Border> </UserControl> And the code-behind: using System; using System.Collections.Generic; using System.Linq; using System.Net; using System.Windows; using System.Windows.Controls; using System.Windows.Documents; using System.Windows.Input; using System.Windows.Media; using System.Windows.Media.Animation; using System.Windows.Shapes; using Telerik.Windows.Controls.GridView; using System.Windows.Data; using Telerik.Windows.Controls; using Telerik.Windows.Data; namespace PureServerSideFiltering { /// <summary> /// A custom filtering control capable of filtering purely server-side. /// </summary> public partial class DomainDataSourceFilteringControl : UserControl, IFilteringControl { // The main player here. DomainDataSource domainDataSource; // This is the name of the property that this column displays. private string dataMemberName; // This is the type of the property that this column displays. private Type dataMemberType; /// <summary> /// Identifies the <see cref="IsActive"/> dependency property. /// </summary> /// <remarks> /// The state of the filtering funnel (i.e. full or empty) is bound to this property. /// </remarks> public static readonly DependencyProperty IsActiveProperty = DependencyProperty.Register( "IsActive", typeof(bool), typeof(DomainDataSourceFilteringControl), new PropertyMetadata(false)); /// <summary> /// Gets or sets a value indicating whether the filtering is active. /// </summary> /// <remarks> /// Set this to true if you want to lit-up the filtering funnel. /// </remarks> public bool IsActive { get { return (bool)GetValue(IsActiveProperty); } set { SetValue(IsActiveProperty, value); } } /// <summary> /// Gets or sets the domain data source. /// We need this in order to work on its FilterDescriptors collection. /// </summary> /// <value>The domain data source.</value> public DomainDataSource DomainDataSource { get { return this.domainDataSource; } set { this.domainDataSource = value; } } public System.Windows.Data.FilterDescriptorCollection FilterDescriptors { get { return this.DomainDataSource.FilterDescriptors; } } public DomainDataSourceFilteringControl() { InitializeComponent(); } public void Prepare(GridViewBoundColumnBase column) { this.LayoutRoot.DataContext = this; if (this.DomainDataSource == null) { // Sorry, but we need a DomainDataSource. Can't do anything without it. return; } // This is the name of the property that this column displays. this.dataMemberName = column.GetDataMemberName(); // This is the type of the property that this column displays. // We need this in order to see which FilterOperators to feed to the combo-box column. this.dataMemberType = column.DataType; // We will use our magic Type extension method to see which operators are applicable for // this data type. You can go to the extension method body and see what it does. ((GridViewComboBoxColumn)this.filtersGrid.Columns["Operator"]).ItemsSource = this.dataMemberType.ApplicableFilterOperators(); // This is very nice as well. We will tell the Value column its data type. In this way // RadGridView will pick up the best editor according to the data type. For example, // if the data type of the value is DateTime, you will be editing it with a DatePicker. // Nice! ((GridViewDataColumn)this.filtersGrid.Columns["Value"]).DataType = this.dataMemberType; // Yet another nice feature. We will transfer the original DataFormatString (if any) to // the Value column. In this way if you have specified a DataFormatString for the original // column, you will see all filter values formatted accordingly. ((GridViewDataColumn)this.filtersGrid.Columns["Value"]).DataFormatString = column.DataFormatString; // This is important. Since our little filtersGrid will be bound to the entire collection // of this.domainDataSource.FilterDescriptors, we need to set a Telerik filter on the // grid so that it will display FilterDescriptor which are relevane to this column ONLY! Telerik.Windows.Data.FilterDescriptor columnFilter = new Telerik.Windows.Data.FilterDescriptor("PropertyPath" , Telerik.Windows.Data.FilterOperator.IsEqualTo , this.dataMemberName); this.filtersGrid.FilterDescriptors.Add(columnFilter); // We want to listen for this in order to activate and de-activate the UI funnel. this.filtersGrid.Items.CollectionChanged += this.OnFilterGridItemsCollectionChanged; } /// <summary> // Since the DomainDataSource is a little bit picky about adding uninitialized FilterDescriptors // to its collection, we will prepare each new instance with some default values and then // the user can change them later. Go to the event handler to see how we do this. /// </summary> void OnFilterGridAddingNewDataItem(object sender, GridViewAddingNewEventArgs e) { // We need to initialize the new instance with some values and let the user go on from here. System.Windows.Data.FilterDescriptor newFilter = new System.Windows.Data.FilterDescriptor(); // This is a must. It should know what member it is filtering on. newFilter.PropertyPath = this.dataMemberName; // Initialize it with one of the allowed operators. // TypeExtensions.ApplicableFilterOperators method for more info. newFilter.Operator = this.dataMemberType.ApplicableFilterOperators().First(); if (this.dataMemberType == typeof(DateTime)) { newFilter.Value.Value = DateTime.Now; } else if (this.dataMemberType == typeof(string)) { newFilter.Value.Value = "<enter text>"; } else if (this.dataMemberType.IsValueType) { // We need something non-null for all value types. newFilter.Value.Value = Activator.CreateInstance(this.dataMemberType); } // Let the user edit the new filter any way he/she likes. e.NewObject = newFilter; } void OnFilterGridItemsCollectionChanged(object sender, System.Collections.Specialized.NotifyCollectionChangedEventArgs e) { // We are active only if we have any filters define. In this case the filtering funnel will lit-up. this.IsActive = this.filtersGrid.Items.Count > 0; } private void OnApplyFiltersButtonClick(object sender, RoutedEventArgs e) { if (this.DomainDataSource.IsLoadingData) { return; } // Comment this if you want the popup to stay open after the button is clicked. this.ClosePopup(); // Since this.domainDataSource.AutoLoad is false, this will take into // account all filtering changes that the user has made since the last // Load() and pull the new data to the client. this.DomainDataSource.Load(); } private void OnClearFiltersButtonClick(object sender, RoutedEventArgs e) { if (this.DomainDataSource.IsLoadingData) { return; } // We want to remove ONLY those filters from the DomainDataSource // that this control is responsible for. this.DomainDataSource.FilterDescriptors .Where(fd => fd.PropertyPath == this.dataMemberName) // Only "our" filters. .ToList() .ForEach(fd => this.DomainDataSource.FilterDescriptors.Remove(fd)); // Bye-bye! // Comment this if you want the popup to stay open after the button is clicked. this.ClosePopup(); // After we did our housekeeping, get the new data to the client. this.DomainDataSource.Load(); } private void OnAddFilterButtonClick(object sender, RoutedEventArgs e) { if (this.DomainDataSource.IsLoadingData) { return; } // Let the user enter his/or her requirements for a new filter. this.filtersGrid.BeginInsert(); this.filtersGrid.UpdateLayout(); } private void OnRemoveFilterButtonClick(object sender, RoutedEventArgs e) { if (this.DomainDataSource.IsLoadingData) { return; } // Find the currently selected filter and destroy it. System.Windows.Data.FilterDescriptor filterToRemove = this.filtersGrid.SelectedItem as System.Windows.Data.FilterDescriptor; if (filterToRemove != null && this.DomainDataSource.FilterDescriptors.Contains(filterToRemove)) { this.DomainDataSource.FilterDescriptors.Remove(filterToRemove); } } private void ClosePopup() { System.Windows.Controls.Primitives.Popup popup = this.ParentOfType<System.Windows.Controls.Primitives.Popup>(); if (popup != null) { popup.IsOpen = false; } } } } Finally, we need to tell RadGridViews Columns to use this custom control instead of the default one. Here is how to do it: using System; using System.Collections.Generic; using System.Linq; using System.Net; using System.Windows; using System.Windows.Controls; using System.Windows.Documents; using System.Windows.Input; using System.Windows.Media; using System.Windows.Media.Animation; using System.Windows.Shapes; using System.Windows.Data; using Telerik.Windows.Data; using Telerik.Windows.Controls; using Telerik.Windows.Controls.GridView; namespace PureServerSideFiltering { public partial class MainPage : UserControl { public MainPage() { InitializeComponent(); this.grid.AutoGeneratingColumn += this.OnGridAutoGeneratingColumn; // Uncomment this if you want the DomainDataSource to start pre-filtered. // You will notice how our custom filtering controls will correctly read this information, // populate their UI with the respective filters and lit-up the funnel to indicate that // filtering is active. Go ahead and try it. this.employeesDataSource.FilterDescriptors.Add(new System.Windows.Data.FilterDescriptor("Title", System.Windows.Data.FilterOperator.Contains, "Assistant")); this.employeesDataSource.FilterDescriptors.Add(new System.Windows.Data.FilterDescriptor("HireDate", System.Windows.Data.FilterOperator.IsGreaterThan, new DateTime(1998, 12, 31))); this.employeesDataSource.FilterDescriptors.Add(new System.Windows.Data.FilterDescriptor("HireDate", System.Windows.Data.FilterOperator.IsLessThanOrEqualTo, new DateTime(1999, 12, 31))); this.employeesDataSource.Load(); } /// <summary> /// First of all, we will need to replace the default filtering control /// of each column with out custom filtering control DomainDataSourceFilteringControl /// </summary> private void OnGridAutoGeneratingColumn(object sender, GridViewAutoGeneratingColumnEventArgs e) { GridViewBoundColumnBase dataColumn = e.Column as GridViewBoundColumnBase; if (dataColumn != null) { // We do not like ugly dates. if (dataColumn.DataType == typeof(DateTime)) { dataColumn.DataFormatString = "{0:d}"; // Short date pattern. // Notice how this format will be later transferred to the Value column // of the grid that we have inside the DomainDataSourceFilteringControl. } // Replace the default filtering control with our. dataColumn.FilteringControl = new DomainDataSourceFilteringControl() { // Let the control know about the DDS, after all it will work directly on it. DomainDataSource = this.employeesDataSource }; // Finally, lit-up the filtering funnel through the IsActive dependency property // in case there are some filters on the DDS that match our column member. string dataMemberName = dataColumn.GetDataMemberName(); dataColumn.FilteringControl.IsActive = this.employeesDataSource.FilterDescriptors .Where(fd => fd.PropertyPath == dataMemberName) .Count() > 0; } } } } The best part is that we are not only writing filters for the DomainDataSource we can read and load them. If the DomainDataSource has some pre-existing filters (like I have created in the code above), our control will read them and will populate its UI accordingly. Even the filtering funnel will light-up! Remember, the funnel is controlled by the IsActive property of our control. While this is just a basic implementation, the source code is absolutely yours and you can take it from here and extend it to match your specific business requirements. Below the main grid there is another debug grid. With its help you can monitor what filter descriptors are added and removed to the domain data source. Download Source Code. (You will have to have the AdventureWorks sample database installed on the default SQLExpress instance in order to run it.) Enjoy!Did you know that DotNetSlackers also publishes .net articles written by top known .net Authors? We already have over 80 articles in several categories including Silverlight. Take a look: here.

Read the article

OSI Model

- by kaleidoscope

The Open System Interconnection Reference Model (OSI Reference Model or OSI Model) is an abstract description for layered communications and computer network protocol design. In its most basic form, it divides network architecture into seven layers which, from top to bottom, are the Application, Presentation, Session, Transport, Network, Data Link, and Physical Layers. It is therefore often referred to as the OSI Seven Layer Model. A layer is a collection of conceptually similar functions that provide services to the layer above it and receives service from the layer below it. Description of OSI layers: Layer 1: Physical Layer · Defines the electrical and physical specifications for devices. In particular, it defines the relationship between a device and a physical medium. · Establishment and termination of a connection to a communications medium. · Participation in the process whereby the communication resources are effectively shared among multiple users. · Modulation or conversion between the representation of digital data in user equipment and the corresponding signals transmitted over a communications channel. Layer 2: Data Link Layer · Provides the functional and procedural means to transfer data between network entities. · Detect and possibly correct errors that may occur in the Physical Layer. The error check is performed using Frame Check Sequence (FCS). · Addresses is then sought to see if it needs to process the rest of the frame itself or whether to pass it on to another host. · The Layer is divided into two sub layers: The Media Access Control (MAC) layer and the Logical Link Control (LLC) layer. · MAC sub layer controls how a computer on the network gains access to the data and permission to transmit it. · LLC layer controls frame synchronization, flow control and error checking. Layer 3: Network Layer · Provides the functional and procedural means of transferring variable length data sequences from a source to a destination via one or more networks. · Performs network routing functions, and might also perform fragmentation and reassembly, and report delivery errors. · Network Layer Routers operate at this layer—sending data throughout the extended network and making the Internet possible. Layer 4: Transport Layer · Provides transparent transfer of data between end users, providing reliable data transfer services to the upper layers. · Controls the reliability of a given link through flow control, segmentation/de-segmentation, and error control. · Transport Layer can keep track of the segments and retransmit those that fail. Layer 5: Session Layer · Controls the dialogues (connections) between computers. · Establishes, manages and terminates the connections between the local and remote application. · Provides for full-duplex, half-duplex, or simplex operation, and establishes checkpointing, adjournment, termination, and restart procedures. · Implemented explicitly in application environments that use remote procedure calls. Layer 6: Presentation Layer · Establishes a context between Application Layer entities, in which the higher-layer entities can use different syntax and semantics, as long as the presentation service understands both and the mapping between them. The presentation service data units are then encapsulated into Session Protocol data units, and moved down the stack. · Provides independence from differences in data representation (e.g., encryption) by translating from application to network format, and vice versa. The presentation layer works to transform data into the form that the application layer can accept. This layer formats and encrypts data to be sent across a network, providing freedom from compatibility problems. It is sometimes called the syntax layer. Layer 7: Application Layer · This layer interacts with software applications that implement a communicating component. · Identifies communication partners, determines resource availability, and synchronizes communication. o When identifying communication partners, the application layer determines the identity and availability of communication partners for an application with data to transmit. o When determining resource availability, the application layer must decide whether sufficient network or the requested communication exists. o In synchronizing communication, all communication between applications requires cooperation that is managed by the application layer. Technorati Tags: Kunal,OSI,Networking

Read the article

SQL SERVER – An Efficiency Tool to Compare and Synchronize SQL Server Databases

- by Pinal Dave

There is no need to reinvent the wheel if it is already invented and if the wheel is already available at ease, there is no need to wait to grab it. Here is the similar situation. I came across a very interesting situation and I had to look for an efficient tool which can make my life easier and solve my business problem. Here is the scenario. One of the developers had deleted few rows from the very important mapping table of our development server (thankfully, it was not the production server). Though it was a development server, the entire development team had to stop working as the application started to crash on every page. Think about the lost of manpower and efficiency which we started to loose. Pretty much every department had to stop working as our internal development application stopped working. Thankfully, we even take a backup of our development server and we had access to full backup of the entire database at 6 AM morning. We do not take as a frequent backup of development server as production server (naturally!). Even though we had a full backup, the solution was not to restore the database. Think about it, there were plenty of the other operations since the last good full backup and if we restore a full backup, we will pretty much overwrite on the top of the work done by developers since morning. Now, as restoring the full backup was not an option we decided to restore the same database on another server. Once we had restored our database to another server, the challenge was to compare the table from where the database was deleted. The mapping table from where the data were deleted contained over 5000 rows and it was humanly impossible to compare both the tables manually. Finally we decided to use efficiency tool dbForge Data Compare for SQL Server from DevArt. dbForge Data Compare for SQL Server is a powerful, fast and easy to use SQL compare tool, capable of using native SQL Server backups as metadata source. (FYI we Downloaded dbForge Data Compare) Once we discovered the product, we immediately downloaded the product and installed on our development server. After we installed the product, we were greeted with the following screen. We clicked on the New Data Comparision to start our new comparison project. It brought up following screen. Here is the best part of the product, we just had to enter our database connection username and password along with source and destination details and we are done. The entire process is very simple and self intuiting. The best part was that for the source, we can either select database or even backup. This was indeed fantastic feature. Think about this, if you have a very big database, it will take long time to restore on the server. Once it is restored, you will be able to work with it. However, when you are working with dbForge Data Compare it will accept database backup as your source or destination. Once I click on the execute it brought up following screen where it displayed an excellent summary of the data compare. It has dedicated tabs for the what is changing in what table as well had details of the changed data. The best part is that, once we had reviewed the change. We click on the Synchronize button in the menu bar and it brought up following screen. You can see that the screen has very simple straight forward but very powerful features. You can generate a script to synchronize from target to source or even from source to target. Additionally, the database is a very complicated world and there are extensive options to configure various database options on the next screen. We also have the option to either generate script or directly execute the script to target server. I like to play on the safe side and I generated the script for my synchronization and later on after review I deployed the scripts on the server. Well, my team and we were able to get going from our disaster in less than 10 minutes. There were few people in our team were indeed disappointed as they were thinking of going home early that day but in less than 10 minutes they had to get back to work. There are so many other features in dbForge Data Compare for SQL Server, I am already planning to make this product company wide recommended product for Data Compare tool. Hats off to the team who have build this product. Here are few of the features salient features of the dbForge Data Compare for SQL Server Perform SQL Server database comparison to detect changes Compare SQL Server backups with live databases Analyze data differences between two databases Synchronize two databases that went out of sync Restore data of a particular table from the backup Generate data comparison reports in Excel and HTML formats Copy look-up data from development database to production Automate routine data synchronization tasks with command-line interface Go Ahead and Download the dbForge Data Compare for SQL Server right away. It is always a good idea to get familiar with the important tools before hand instead of learning it under pressure of disaster. Reference: Pinal Dave (http://blog.sqlauthority.com) Filed under: PostADay, SQL, SQL Authority, SQL Query, SQL Server, SQL Tips and Tricks, SQL Utility, T SQL, Technology

Read the article

Organization & Architecture UNISA Studies – Chap 5

- by MarkPearl

Learning Outcomes Describe the operation of a memory cell Explain the difference between DRAM and SRAM Discuss the different types of ROM Explain the concepts of a hard failure and a soft error respectively Describe SDRAM organization Semiconductor Main Memory The two traditional forms of RAM used in computers are DRAM and SRAM DRAM (Dynamic RAM) Divided into two technologies… Dynamic Static Dynamic RAM is made with cells that store data as charge on capacitors. The presence or absence of charge in a capacitor is interpreted as a binary 1 or 0. Because capacitors have natural tendency to discharge, dynamic RAM requires periodic charge refreshing to maintain data storage. The term dynamic refers to the tendency of the stored charge to leak away, even with power continuously applied. Although the DRAM cell is used to store a single bit (0 or 1), it is essentially an analogue device. The capacitor can store any charge value within a range, a threshold value determines whether the charge is interpreted as a 1 or 0. SRAM (Static RAM) SRAM is a digital device that uses the same logic elements used in the processor. In SRAM, binary values are stored using traditional flip flop logic configurations. SRAM will hold its data as along as power is supplied to it. Unlike DRAM, no refresh is required to retain data. SRAM vs. DRAM DRAM is simpler and smaller than SRAM. Thus it is more dense and less expensive than SRAM. The cost of the refreshing circuitry for DRAM needs to be considered, but if the machine requires a large amount of memory, DRAM turns out to be cheaper than SRAM. SRAMS are somewhat faster than DRAM, thus SRAM is generally used for cache memory and DRAM is used for main memory. Types of ROM Read Only Memory (ROM) contains a permanent pattern of data that cannot be changed. ROM is non volatile meaning no power source is required to maintain the bit values in memory. While it is possible to read a ROM, it is not possible to write new data into it. An important application of ROM is microprogramming, other applications include library subroutines for frequently wanted functions, System programs, Function tables. A ROM is created like any other integrated circuit chip, with the data actually wired into the chip as part of the fabrication process. To reduce costs of fabrication, we have PROMS. PROMS are… Written only once Non-volatile Written after fabrication Another variation of ROM is the read-mostly memory, which is useful for applications in which read operations are far more frequent than write operations, but for which non volatile storage is required. There are three common forms of read-mostly memory, namely… EPROM EEPROM Flash memory Error Correction Semiconductor memory is subject to errors, which can be classed into two categories… Hard failure – Permanent physical defect so that the memory cell or cells cannot reliably store data Soft failure – Random error that alters the contents of one or more memory cells without damaging the memory (common cause includes power supply issues, etc.) Most modern main memory systems include logic for both detecting and correcting errors. Error detection works as follows… When data is to be read into memory, a calculation is performed on the data to produce a code Both the code and the data are stored When the previously stored word is read out, the code is used to detect and possibly correct errors The error checking provides one of 3 possible results… No errors are detected – the fetched data bits are sent out An error is detected, and it is possible to correct the error. The data bits plus error correction bits are fed into a corrector, which produces a corrected set of bits to be sent out An error is detected, but it is not possible to correct it. This condition is reported Hamming Code See wiki for detailed explanation. We will probably need to know how to do a hemming code – refer to the textbook (pg. 188 – 189) Advanced DRAM organization One of the most critical system bottlenecks when using high-performance processors is the interface to main memory. This interface is the most important pathway in the entire computer system. The basic building block of main memory remains the DRAM chip. In recent years a number of enhancements to the basic DRAM architecture have been explored, and some of these are now on the market including… SDRAM (Synchronous DRAM) DDR-DRAM RDRAM SDRAM (Synchronous DRAM) SDRAM exchanges data with the processor synchronized to an external clock signal and running at the full speed of the processor/memory bus without imposing wait states. SDRAM employs a burst mode to eliminate the address setup time and row and column line precharge time after the first access In burst mode a series of data bits can be clocked out rapidly after the first bit has been accessed SDRAM has a multiple bank internal architecture that improves opportunities for on chip parallelism SDRAM performs best when it is transferring large blocks of data serially There is now an enhanced version of SDRAM known as double data rate SDRAM or DDR-SDRAM that overcomes the once-per-cycle limitation of SDRAM

Read the article

How to write simple code using TDD [migrated]

- by adeel41

Me and my colleagues do a small TDD-Kata practice everyday for 30 minutes. For reference this is the link for the excercise http://osherove.com/tdd-kata-1/ The objective is to write better code using TDD. This is my code which I've written public class Calculator { public int Add( string numbers ) { const string commaSeparator = ","; int result = 0; if ( !String.IsNullOrEmpty( numbers ) ) result = numbers.Contains( commaSeparator ) ? AddMultipleNumbers( GetNumbers( commaSeparator, numbers ) ) : ConvertToNumber( numbers ); return result; } private int AddMultipleNumbers( IEnumerable getNumbers ) { return getNumbers.Sum(); } private IEnumerable GetNumbers( string separator, string numbers ) { var allNumbers = numbers .Replace( "\n", separator ) .Split( new string[] { separator }, StringSplitOptions.RemoveEmptyEntries ); return allNumbers.Select( ConvertToNumber ); } private int ConvertToNumber( string number ) { return Convert.ToInt32( number ); } } and the tests for this class are [TestFixture] public class CalculatorTests { private int ArrangeAct( string numbers ) { var calculator = new Calculator(); return calculator.Add( numbers ); } [Test] public void Add_WhenEmptyString_Returns0() { Assert.AreEqual( 0, ArrangeAct( String.Empty ) ); } [Test] [Sequential] public void Add_When1Number_ReturnNumber( [Values( "1", "56" )] string number, [Values( 1, 56 )] int expected ) { Assert.AreEqual( expected, ArrangeAct( number ) ); } [Test] public void Add_When2Numbers_AddThem() { Assert.AreEqual( 3, ArrangeAct( "1,2" ) ); } [Test] public void Add_WhenMoreThan2Numbers_AddThemAll() { Assert.AreEqual( 6, ArrangeAct( "1,2,3" ) ); } [Test] public void Add_SeparatorIsNewLine_AddThem() { Assert.AreEqual( 6, ArrangeAct( @"1 2,3" ) ); } } Now I'll paste code which they have written public class StringCalculator { private const char Separator = ','; public int Add( string numbers ) { const int defaultValue = 0; if ( ShouldReturnDefaultValue( numbers ) ) return defaultValue; return ConvertNumbers( numbers ); } private int ConvertNumbers( string numbers ) { var numberParts = GetNumberParts( numbers ); return numberParts.Select( ConvertSingleNumber ).Sum(); } private string[] GetNumberParts( string numbers ) { return numbers.Split( Separator ); } private int ConvertSingleNumber( string numbers ) { return Convert.ToInt32( numbers ); } private bool ShouldReturnDefaultValue( string numbers ) { return String.IsNullOrEmpty( numbers ); } } and the tests [TestFixture] public class StringCalculatorTests { [Test] public void Add_EmptyString_Returns0() { ArrangeActAndAssert( String.Empty, 0 ); } [Test] [TestCase( "1", 1 )] [TestCase( "2", 2 )] public void Add_WithOneNumber_ReturnsThatNumber( string numberText, int expected ) { ArrangeActAndAssert( numberText, expected ); } [Test] [TestCase( "1,2", 3 )] [TestCase( "3,4", 7 )] public void Add_WithTwoNumbers_ReturnsSum( string numbers, int expected ) { ArrangeActAndAssert( numbers, expected ); } [Test] public void Add_WithThreeNumbers_ReturnsSum() { ArrangeActAndAssert( "1,2,3", 6 ); } private void ArrangeActAndAssert( string numbers, int expected ) { var calculator = new StringCalculator(); var result = calculator.Add( numbers ); Assert.AreEqual( expected, result ); } } Now the question is which one is better? My point here is that we do not need so many small methods initially because StringCalculator has no sub classes and secondly the code itself is so simple that we don't need to break it up too much that it gets confusing after having so many small methods. Their point is that code should read like english and also its better if they can break it up earlier than doing refactoring later and third when they will do refactoring it would be much easier to move these methods quite easily into separate classes. My point of view against is that we never made a decision that code is difficult to understand so why we are breaking it up so early. So I need a third person's opinion to understand which option is much better.

Read the article

My collection of favourite TFS utilities

- by Aaron Kowall

So, you’re in charge of your company or team’s Team Foundation Server. Wish it was easier to manage, administer, extend? Well, here are a few utilities that I highly recommend looking at. I’ve recently had need to rebuild my laptop and upgrade my local TFS environment to TFS 2012 Update 1. This gave me cause to enumerate some of the utilities I like to have on hand. One of the reasons I love to use TFS on projects is that it’s basically a complete ALM toolkit. Everything from Task Management, Version Control, Build Management, Test Management, Metrics and Reporting are all there ‘in the box’. However, no matter how complete a product set it, there are always ways to make it better. Here are a list of utilities and libraries that are pretty generally useful. this is not intended to be an exhaustive list of TFS extensions but rather a set that I recommend you look at. There are many more out there that may be applicable in one scenario or another. This set of tools should work with TFS 2012 or 2010 if you grab the right version. Most of these tools (and more) are available from the Visual Studio Gallery or CodePlex. General TFS Power Tools – This is ‘the’ collection of utilities and extensions delivered by the Product Group. Highly recommended from here are the Best Practice Analyzer for ensuring your TFS implementation is healthy and the Team Foundation Server Backups to ensure your TFS databases are backed up correctly. TFS Administrators Toolkit – helps make updates to work item types and reports across many team projects. Also provides visibility of disk usage by finding large files in version control or test attachments to assist in managing storage utilization. Version Control Git-TF - a set of cross-platform, command line tools that facilitate sharing of changes between TFS and Git. These tools allow a developer to use a local Git repository, and configure it to share changes with a TFS server. Great for all Git lovers who must integrate into a TFS repository. Testing TFS 2012 Tester Power Tool – A utility for bulk copying test cases which assists in an approach for managing test cases across multiple releases. A little plug that this utility was written and maintained by Anna Russo of Imaginet where I also work. Test Scribe - A documentation power tool designed to construct documents directly from the TFS for test plan and test run artifacts for the purpose of discussion, reporting etc. Reporting Community TFS Report Extensions - a single repository of SQL Server Reporting Services report for Team Foundation 2010 (and above). Check out the Test Plan Status report by Imaginet’s Steve St. Jean. Very valuable for your test managers. Builds TFS Build Manager – A great utility if you are build manager over a complex build environment with many TFS build definitions. Community TFS Build Extensions – contains many custom build activities. Current release binaries are for TFS 2010 but many of the activities can be recompiled for use with TFS 2012. While compiling this list, I was surprised by the number of TFS utilities and extensions I no longer use/need in TFS 2012 because of the great work by the TFS team addressing many gaps since the 2010 release. Are there any utilities you depend on that I’ve missed? I’d love to hear about them in the comments!

Read the article

Efficiently separating Read/Compute/Write steps for concurrent processing of entities in Entity/Component systems

- by TravisG

Setup I have an entity-component architecture where Entities can have a set of attributes (which are pure data with no behavior) and there exist systems that run the entity logic which act on that data. Essentially, in somewhat pseudo-code: Entity { id; map<id_type, Attribute> attributes; } System { update(); vector<Entity> entities; } A system that just moves along all entities at a constant rate might be MovementSystem extends System { update() { for each entity in entities position = entity.attributes["position"]; position += vec3(1,1,1); } } Essentially, I'm trying to parallelise update() as efficiently as possible. This can be done by running entire systems in parallel, or by giving each update() of one system a couple of components so different threads can execute the update of the same system, but for a different subset of entities registered with that system. Problem In reality, these systems sometimes require that entities interact(/read/write data from/to) each other, sometimes within the same system (e.g. an AI system that reads state from other entities surrounding the current processed entity), but sometimes between different systems that depend on each other (i.e. a movement system that requires data from a system that processes user input). Now, when trying to parallelize the update phases of entity/component systems, the phases in which data (components/attributes) from Entities are read and used to compute something, and the phase where the modified data is written back to entities need to be separated in order to avoid data races. Otherwise the only way (not taking into account just "critical section"ing everything) to avoid them is to serialize parts of the update process that depend on other parts. This seems ugly. To me it would seem more elegant to be able to (ideally) have all processing running in parallel, where a system may read data from all entities as it wishes, but doesn't write modifications to that data back until some later point. The fact that this is even possible is based on the assumption that modification write-backs are usually very small in complexity, and don't require much performance, whereas computations are very expensive (relatively). So the overhead added by a delayed-write phase might be evened out by more efficient updating of entities (by having threads work more % of the time instead of waiting). A concrete example of this might be a system that updates physics. The system needs to both read and write a lot of data to and from entities. Optimally, there would be a system in place where all available threads update a subset of all entities registered with the physics system. In the case of the physics system this isn't trivially possible because of race conditions. So without a workaround, we would have to find other systems to run in parallel (which don't modify the same data as the physics system), other wise the remaining threads are waiting and wasting time. However, that has disadvantages Practically, the L3 cache is pretty much always better utilized when updating a large system with multiple threads, as opposed to multiple systems at once, which all act on different sets of data. Finding and assembling other systems to run in parallel can be extremely time consuming to design well enough to optimize performance. Sometimes, it might even not be possible at all because a system just depends on data that is touched by all other systems. Solution? In my thinking, a possible solution would be a system where reading/updating and writing of data is separated, so that in one expensive phase, systems only read data and compute what they need to compute, and then in a separate, performance-wise cheap, write phase, attributes of entities that needed to be modified are finally written back to the entities. The Question How might such a system be implemented to achieve optimal performance, as well as making programmer life easier? What are the implementation details of such a system and what might have to be changed in the existing EC-architecture to accommodate this solution?

Read the article

Metro: Introduction to the WinJS ListView Control

- by Stephen.Walther

The goal of this blog entry is to provide a quick introduction to the ListView control – just the bare minimum that you need to know to start using the control. When building Metro style applications using JavaScript, the ListView control is the primary control that you use for displaying lists of items. For example, if you are building a product catalog app, then you can use the ListView control to display the list of products. The ListView control supports several advanced features that I plan to discuss in future blog entries. For example, you can group the items in a ListView, you can create master/details views with a ListView, and you can efficiently work with large sets of items with a ListView. In this blog entry, we’ll keep things simple and focus on displaying a list of products. There are three things that you need to do in order to display a list of items with a ListView: Create a data source Create an Item Template Declare the ListView Creating the ListView Data Source The first step is to create (or retrieve) the data that you want to display with the ListView. In most scenarios, you will want to bind a ListView to a WinJS.Binding.List object. The nice thing about the WinJS.Binding.List object is that it enables you to take a standard JavaScript array and convert the array into something that can be bound to the ListView. It doesn’t matter where the JavaScript array comes from. It could be a static array that you declare or you could retrieve the array as the result of an Ajax call to a remote server. The following JavaScript file – named products.js – contains a list of products which can be bound to a ListView. (function () { "use strict"; var products = new WinJS.Binding.List([ { name: "Milk", price: 2.44 }, { name: "Oranges", price: 1.99 }, { name: "Wine", price: 8.55 }, { name: "Apples", price: 2.44 }, { name: "Steak", price: 1.99 }, { name: "Eggs", price: 2.44 }, { name: "Mushrooms", price: 1.99 }, { name: "Yogurt", price: 2.44 }, { name: "Soup", price: 1.99 }, { name: "Cereal", price: 2.44 }, { name: "Pepsi", price: 1.99 } ]); WinJS.Namespace.define("ListViewDemos", { products: products }); })(); The products variable represents a WinJS.Binding.List object. This object is initialized with a plain-old JavaScript array which represents an array of products. To avoid polluting the global namespace, the code above uses the module pattern and exposes the products using a namespace. The list of products is exposed to the world as ListViewDemos.products. To learn more about the module pattern and namespaces in WinJS, see my earlier blog entry: http://stephenwalther.com/blog/archive/2012/02/22/metro-namespaces-and-modules.aspx Creating the ListView Item Template The ListView control does not know how to render anything. It doesn’t know how you want each list item to appear. To get the ListView control to render something useful, you must create an Item Template. Here’s what our template for rendering an individual product looks like: <div id="productTemplate" data-win-control="WinJS.Binding.Template"> <div class="product"> <span data-win-bind="innerText:name"></span> <span data-win-bind="innerText:price"></span> </div> </div> This template displays the product name and price from the data source. Normally, you will declare your template in the same file as you declare the ListView control. In our case, both the template and ListView are declared in the default.html file. To learn more about templates, see my earlier blog entry: http://stephenwalther.com/blog/archive/2012/02/27/metro-using-templates.aspx Declaring the ListView The final step is to declare the ListView control in a page. Here’s the markup for declaring a ListView: <div data-win-control="WinJS.UI.ListView" data-win-options="{ itemDataSource:ListViewDemos.products.dataSource, itemTemplate:select('#productTemplate') }"> </div> You declare a ListView by adding the data-win-control to an HTML DIV tag. The data-win-options attribute is used to set two properties of the ListView. The ListView is associated with its data source with the itemDataSource property. Notice that the data source is ListViewDemos.products.dataSource and not just ListViewDemos.products. You need to associate the ListView with the dataSoure property. The ListView is associated with its item template with the help of the itemTemplate property. The ID of the item template — #productTemplate – is used to select the template from the page. Here’s what the complete version of the default.html page looks like: <!DOCTYPE html> <html> <head> <meta charset="utf-8"> <title>ListViewDemos</title>  <link href="//Microsoft.WinJS.0.6/css/ui-dark.css" rel="stylesheet"> <script src="//Microsoft.WinJS.0.6/js/base.js"></script> <script src="//Microsoft.WinJS.0.6/js/ui.js"></script>  <link href="/css/default.css" rel="stylesheet"> <script src="/js/default.js"></script> <script src="/js/products.js" type="text/javascript"></script> <style type="text/css"> .product { width: 200px; height: 100px; border: white solid 1px; } </style> </head> <body> <div id="productTemplate" data-win-control="WinJS.Binding.Template"> <div class="product"> <span data-win-bind="innerText:name"></span> <span data-win-bind="innerText:price"></span> </div> </div> <div data-win-control="WinJS.UI.ListView" data-win-options="{ itemDataSource:ListViewDemos.products.dataSource, itemTemplate:select('#productTemplate') }"> </div> </body> </html> Notice that the page above includes a reference to the products.js file: <script src=”/js/products.js” type=”text/javascript”></script> The page above also contains a Template control which contains the ListView item template. Finally, the page includes the declaration of the ListView control. Summary The goal of this blog entry was to describe the minimal set of steps which you must complete to use the WinJS ListView control to display a simple list of items. You learned how to create a data source, declare an item template, and declare a ListView control.

Read the article

The last MVVM you'll ever need?

- by Nuri Halperin

As my MVC projects mature and grow, the need to have some omnipresent, ambient model properties quickly emerge. The application no longer has only one dynamic pieced of data on the page: A sidebar with a shopping cart, some news flash on the side – pretty common stuff. The rub is that a controller is invoked in context of a single intended request. The rest of the data, even though it could be just as dynamic, is expected to appear on it's own. There are many solutions to this scenario. MVVM prescribes creating elaborate objects which expose your new data as a property on some uber-object with more properties exposing the "side show" ambient data. The reason I don't love this approach is because it forces fairly acute awareness of the view, and soon enough you have many MVVM objects laying around, and views have to start doing null-checks in order to ensure you really supplied all the values before binding to them. Ick. Just as unattractive is the ViewData dictionary. It's not strongly typed, and in both this and the MVVM approach someone has to populate these properties – n'est pas? Where does that live? With MVC2, we get the formerly-futures feature Html.RenderAction(). The feature allows you plant a line in a view, of the format: <% Html.RenderAction("SessionInterest", "Session"); %> While this syntax looks very clean, I can't help being bothered by it. MVC was touting a very strong separation of concerns, the Model taking on the role of the business logic, the controller handling route and performing minimal view-choosing operations and the views strictly focused on rendering out angled-bracket tags. The RenderAction() syntax has the view calling some controller and invoking it inline with it's runtime rendering. This – to my taste – embeds too much knowledge of controllers into the view's code – which was allegedly forbidden. The one way flow "Controller Receive Data –> Controller invoke Model –> Controller select view –> Controller Hand data to view" now gets a "View calls controller and gets it's own data" which is not so one-way anymore. Ick. I toyed with some other solutions a bit, including some base controllers, special view classes etc. My current favorite though is making use of the ExpandoObject and dynamic features with C# 4.0. If you follow Phil Haack or read a bit from David Heyden you can see the general picture emerging. The game changer is that using the new dynamic syntax, one can sprout properties on an object and make use of them in the view. Well that beats having a bunch of uni-purpose MVVM's any day! Rather than statically exposed properties, we'll just use the capability of adding members at runtime. Armed with new ideas and syntax, I went to work: First, I created a factory method to enrich the focuse object: public static class ModelExtension { public static dynamic Decorate(this Controller controller, object mainValue) { dynamic result = new ExpandoObject(); result.Value = mainValue; result.SessionInterest = CodeCampBL.SessoinInterest(); result.TagUsage = CodeCampBL.TagUsage(); return result; } } This gives me a nice fluent way to have the controller add the rest of the ambient "side show" items (SessionInterest, TagUsage in this demo) and expose them all as the Model: public ActionResult Index() { var data = SyndicationBL.Refresh(TWEET_SOURCE_URL); dynamic result = this.Decorate(data); return View(result); } So now what remains is that my view knows to expect a dynamic object (rather than statically typed) so that the ASP.NET page compiler won't barf: <%@ Page Language="C#" Title="Ambient Demo" MasterPageFile="~/Views/Shared/Ambient.Master" Inherits="System.Web.Mvc.ViewPage<dynamic>" %> Notice the generic ViewPage<dynamic>. It doesn't work otherwise. In the page itself, Model.Value property contains the main data returned from the controller. The nice thing about this, is that the master page (Ambient.Master) also inherits from the generic ViewMasterPage<dynamic>. So rather than the page worrying about all this ambient stuff, the side bars and panels for ambient data all reside in a master page, and can be rendered using the RenderPartial() syntax: <% Html.RenderPartial("TagCloud", Model.SessionInterest as Dictionary<string, int>); %> Note here that a cast is necessary. This is because although dynamic is magic, it can't figure out what type this property is, and wants you to give it a type so its binder can figure out the right property to bind to at runtime. I use as, you can cast if you like. So there we go – no violation of MVC, no explosion of MVVM models and voila – right? Well, I could not let this go without a tweak or two more. The first thing to improve, is that some views may not need all the properties. In that case, it would be a waste of resources to populate every property. The solution to this is simple: rather than exposing properties, I change d the factory method to expose lambdas - Func<T> really. So only if and when a view accesses a member of the dynamic object does it load the data. public static class ModelExtension { // take two.. lazy loading! public static dynamic LazyDecorate(this Controller c, object mainValue) { dynamic result = new ExpandoObject(); result.Value = mainValue; result.SessionInterest = new Func<Dictionary<string, int>>(() => CodeCampBL.SessoinInterest()); result.TagUsage = new Func<Dictionary<string, int>>(() => CodeCampBL.TagUsage()); return result; } } Now that lazy loading is in place, there's really no reason not to hook up all and any possible ambient property. Go nuts! Add them all in – they won't get invoked unless used. This now requires changing the signature of usage on the ambient properties methods –adding some parenthesis to the master view: <% Html.RenderPartial("TagCloud", Model.SessionInterest() as Dictionary<string, int>); %> And, of course, the controller needs to call LazyDecorate() rather than the old Decorate(). The final touch is to introduce a convenience method to the my Controller class , so that the tedium of calling Decorate() everywhere goes away. This is done quite simply by adding a bunch of methods, matching View(object), View(string,object) signatures of the Controller class: public ActionResult Index() { var data = SyndicationBL.Refresh(TWEET_SOURCE_URL); return AmbientView(data); } //these methods can reside in a base controller for the solution: public ViewResult AmbientView(dynamic data) { dynamic result = ModelExtension.LazyDecorate(this, data); return View(result); } public ViewResult AmbientView(string viewName, dynamic data) { dynamic result = ModelExtension.LazyDecorate(this, data); return View(viewName, result); } The call to AmbientView now replaces any call the View() that requires the ambient data. DRY sattisfied, lazy loading and no need to replace core pieces of the MVC pipeline. I call this a good MVC day. Enjoy!

Read the article

Metro: Introduction to the WinJS ListView Control

- by Stephen.Walther

The goal of this blog entry is to provide a quick introduction to the ListView control – just the bare minimum that you need to know to start using the control. When building Metro style applications using JavaScript, the ListView control is the primary control that you use for displaying lists of items. For example, if you are building a product catalog app, then you can use the ListView control to display the list of products. The ListView control supports several advanced features that I plan to discuss in future blog entries. For example, you can group the items in a ListView, you can create master/details views with a ListView, and you can efficiently work with large sets of items with a ListView. In this blog entry, we’ll keep things simple and focus on displaying a list of products. There are three things that you need to do in order to display a list of items with a ListView: Create a data source Create an Item Template Declare the ListView Creating the ListView Data Source The first step is to create (or retrieve) the data that you want to display with the ListView. In most scenarios, you will want to bind a ListView to a WinJS.Binding.List object. The nice thing about the WinJS.Binding.List object is that it enables you to take a standard JavaScript array and convert the array into something that can be bound to the ListView. It doesn’t matter where the JavaScript array comes from. It could be a static array that you declare or you could retrieve the array as the result of an Ajax call to a remote server. The following JavaScript file – named products.js – contains a list of products which can be bound to a ListView. (function () { "use strict"; var products = new WinJS.Binding.List([ { name: "Milk", price: 2.44 }, { name: "Oranges", price: 1.99 }, { name: "Wine", price: 8.55 }, { name: "Apples", price: 2.44 }, { name: "Steak", price: 1.99 }, { name: "Eggs", price: 2.44 }, { name: "Mushrooms", price: 1.99 }, { name: "Yogurt", price: 2.44 }, { name: "Soup", price: 1.99 }, { name: "Cereal", price: 2.44 }, { name: "Pepsi", price: 1.99 } ]); WinJS.Namespace.define("ListViewDemos", { products: products }); })(); The products variable represents a WinJS.Binding.List object. This object is initialized with a plain-old JavaScript array which represents an array of products. To avoid polluting the global namespace, the code above uses the module pattern and exposes the products using a namespace. The list of products is exposed to the world as ListViewDemos.products. To learn more about the module pattern and namespaces in WinJS, see my earlier blog entry: http://stephenwalther.com/blog/archive/2012/02/22/metro-namespaces-and-modules.aspx Creating the ListView Item Template The ListView control does not know how to render anything. It doesn’t know how you want each list item to appear. To get the ListView control to render something useful, you must create an Item Template. Here’s what our template for rendering an individual product looks like: <div id="productTemplate" data-win-control="WinJS.Binding.Template"> <div class="product"> <span data-win-bind="innerText:name"></span> <span data-win-bind="innerText:price"></span> </div> </div> This template displays the product name and price from the data source. Normally, you will declare your template in the same file as you declare the ListView control. In our case, both the template and ListView are declared in the default.html file. To learn more about templates, see my earlier blog entry: http://stephenwalther.com/blog/archive/2012/02/27/metro-using-templates.aspx Declaring the ListView The final step is to declare the ListView control in a page. Here’s the markup for declaring a ListView: <div data-win-control="WinJS.UI.ListView" data-win-options="{ itemDataSource:ListViewDemos.products.dataSource, itemTemplate:select('#productTemplate') }"> </div> You declare a ListView by adding the data-win-control to an HTML DIV tag. The data-win-options attribute is used to set two properties of the ListView. The ListView is associated with its data source with the itemDataSource property. Notice that the data source is ListViewDemos.products.dataSource and not just ListViewDemos.products. You need to associate the ListView with the dataSoure property. The ListView is associated with its item template with the help of the itemTemplate property. The ID of the item template — #productTemplate – is used to select the template from the page. Here’s what the complete version of the default.html page looks like: <!DOCTYPE html> <html> <head> <meta charset="utf-8"> <title>ListViewDemos</title>  <link href="//Microsoft.WinJS.0.6/css/ui-dark.css" rel="stylesheet"> <script src="//Microsoft.WinJS.0.6/js/base.js"></script> <script src="//Microsoft.WinJS.0.6/js/ui.js"></script>  <link href="/css/default.css" rel="stylesheet"> <script src="/js/default.js"></script> <script src="/js/products.js" type="text/javascript"></script> <style type="text/css"> .product { width: 200px; height: 100px; border: white solid 1px; } </style> </head> <body> <div id="productTemplate" data-win-control="WinJS.Binding.Template"> <div class="product"> <span data-win-bind="innerText:name"></span> <span data-win-bind="innerText:price"></span> </div> </div> <div data-win-control="WinJS.UI.ListView" data-win-options="{ itemDataSource:ListViewDemos.products.dataSource, itemTemplate:select('#productTemplate') }"> </div> </body> </html> Notice that the page above includes a reference to the products.js file: <script src=”/js/products.js” type=”text/javascript”></script> The page above also contains a Template control which contains the ListView item template. Finally, the page includes the declaration of the ListView control. Summary The goal of this blog entry was to describe the minimal set of steps which you must complete to use the WinJS ListView control to display a simple list of items. You learned how to create a data source, declare an item template, and declare a ListView control.

Read the article

The blocking nature of aggregates

- by Rob Farley

I wrote a post recently about how query tuning isn’t just about how quickly the query runs – that if you have something (such as SSIS) that is consuming your data (and probably introducing a bottleneck), then it might be more important to have a query which focuses on getting the first bit of data out. You can read that post here. In particular, we looked at two operators that could be used to ensure that a query returns only Distinct rows. and The Sort operator pulls in all the data, sorts it (discarding duplicates), and then pushes out the remaining rows. The Hash Match operator performs a Hashing function on each row as it comes in, and then looks to see if it’s created a Hash it’s seen before. If not, it pushes the row out. The Sort method is quicker, but has to wait until it’s gathered all the data before it can do the sort, and therefore blocks the data flow. But that was my last post. This one’s a bit different. This post is going to look at how Aggregate functions work, which ties nicely into this month’s T-SQL Tuesday. I’ve frequently explained about the fact that DISTINCT and GROUP BY are essentially the same function, although DISTINCT is the poorer cousin because you have less control over it, and you can’t apply aggregate functions. Just like the operators used for Distinct, there are different flavours of Aggregate operators – coming in blocking and non-blocking varieties. The example I like to use to explain this is a pile of playing cards. If I’m handed a pile of cards and asked to count how many cards there are in each suit, it’s going to help if the cards are already ordered. Suppose I’m playing a game of Bridge, I can easily glance at my hand and count how many there are in each suit, because I keep the pile of cards in order. Moving from left to right, I could tell you I have four Hearts in my hand, even before I’ve got to the end. By telling you that I have four Hearts as soon as I know, I demonstrate the principle of a non-blocking operation. This is known as a Stream Aggregate operation. It requires input which is sorted by whichever columns the grouping is on, and it will release a row as soon as the group changes – when I encounter a Spade, I know I don’t have any more Hearts in my hand. Alternatively, if the pile of cards are not sorted, I won’t know how many Hearts I have until I’ve looked through all the cards. In fact, to count them, I basically need to put them into little piles, and when I’ve finished making all those piles, I can count how many there are in each. Because I don’t know any of the final numbers until I’ve seen all the cards, this is blocking. This performs the aggregate function using a Hash Match. Observant readers will remember this from my Distinct example. You might remember that my earlier Hash Match operation – used for Distinct Flow – wasn’t blocking. But this one is. They’re essentially doing a similar operation, applying a Hash function to some data and seeing if the set of values have been seen before, but before, it needs more information than the mere existence of a new set of values, it needs to consider how many of them there are. A lot is dependent here on whether the data coming out of the source is sorted or not, and this is largely determined by the indexes that are being used. If you look in the Properties of an Index Scan, you’ll be able to see whether the order of the data is required by the plan. A property called Ordered will demonstrate this. In this particular example, the second plan is significantly faster, but is dependent on having ordered data. In fact, if I force a Stream Aggregate on unordered data (which I’m doing by telling it to use a different index), a Sort operation is needed, which makes my plan a lot slower. This is all very straight-forward stuff, and information that most people are fully aware of. I’m sure you’ve all read my good friend Paul White (@sql_kiwi)’s post on how the Query Optimizer chooses which type of aggregate function to apply. But let’s take a look at SQL Server Integration Services. SSIS gives us a Aggregate transformation for use in Data Flow Tasks, but it’s described as Blocking. The definitive article on Performance Tuning SSIS uses Sort and Aggregate as examples of Blocking Transformations. I’ve just shown you that Aggregate operations used by the Query Optimizer are not always blocking, but that the SSIS Aggregate component is an example of a blocking transformation. But is it always the case? After all, there are plenty of SSIS Performance Tuning talks out there that describe the value of sorted data in Data Flow Tasks, describing the IsSorted property that can be set through the Advanced Editor of your Source component. And so I set about testing the Aggregate transformation in SSIS, to prove for sure whether providing Sorted data would let the Aggregate transform behave like a Stream Aggregate. (Of course, I knew the answer already, but it helps to be able to demonstrate these things). A query that will produce a million rows in order was in order. Let me rephrase. I used a query which produced the numbers from 1 to 1000000, in a single field, ordered. The IsSorted flag was set on the source output, with the only column as SortKey 1. Performing an Aggregate function over this (counting the number of rows per distinct number) should produce an additional column with 1 in it. If this were being done in T-SQL, the ordered data would allow a Stream Aggregate to be used. In fact, if the Query Optimizer saw that the field had a Unique Index on it, it would be able to skip the Aggregate function completely, and just insert the value 1. This is a shortcut I wouldn’t be expecting from SSIS, but certainly the Stream behaviour would be nice. Unfortunately, it’s not the case. As you can see from the screenshots above, the data is pouring into the Aggregate function, and not being released until all million rows have been seen. It’s not doing a Stream Aggregate at all. This is expected behaviour. (I put that in bold, because I want you to realise this.) An SSIS transformation is a piece of code that runs. It’s a physical operation. When you write T-SQL and ask for an aggregation to be done, it’s a logical operation. The physical operation is either a Stream Aggregate or a Hash Match. In SSIS, you’re telling the system that you want a generic Aggregation, that will have to work with whatever data is passed in. I’m not saying that it wouldn’t be possible to make a sometimes-blocking aggregation component in SSIS. A Custom Component could be created which could detect whether the SortKeys columns of the input matched the Grouping columns of the Aggregation, and either call the blocking code or the non-blocking code as appropriate. One day I’ll make one of those, and publish it on my blog. I’ve done it before with a Script Component, but as Script components are single-use, I was able to handle the data knowing everything about my data flow already. As per my previous post – there are a lot of aspects in which tuning SSIS and tuning execution plans use similar concepts. In both situations, it really helps to have a feel for what’s going on behind the scenes. Considering whether an operation is blocking or not is extremely relevant to performance, and that it’s not always obvious from the surface. In a future post, I’ll show the impact of blocking v non-blocking and synchronous v asynchronous components in SSIS, using some of LobsterPot’s Script Components and Custom Components as examples. When I get that sorted, I’ll make a Stream Aggregate component available for download.

Search Results

Search found 77599 results on 3104 pages for 'test data'.

Page 520/3104 | < Previous Page | 516 517 518 519 520 521 522 523 524 525 526 527 | Next Page >

- by Blankman

- by tomcam

- by Matt

- by oDx

- by ericslaw

- by oDx

- by user90500

- by Phill Pafford

- by Nathan

- by Kaleb Pederson

- by user337591

- by Xinus

- by Dan Bailiff

- by mind.blank

- by kaleidoscope

- by Pinal Dave

- by MarkPearl

- by adeel41

- by Aaron Kowall

- by TravisG

- by Stephen.Walther

- by Nuri Halperin

- by Stephen.Walther

- by Rob Farley

< Previous Page | 516 517 518 519 520 521 522 523 524 525 526 527 | Next Page >