reversed - Page 8 - Developer IT

Integration Patterns with Azure Service Bus Relay, Part 1: Exposing the on-premise service

- by Elton Stoneman

We're in the process of delivering an enabling project to expose on-premise WCF services securely to Internet consumers. The Azure Service Bus Relay is doing the clever stuff, we register our on-premise service with Azure, consumers call into our .servicebus.windows.net namespace, and their requests are relayed and serviced on-premise. In theory it's all wonderfully simple; by using the relay we get lots of protocol options, free HTTPS and load balancing, and by integrating to ACS we get plenty of security options. Part of our delivery is a suite of sample consumers for the service - .NET, jQuery, PHP - and this set of posts will cover setting up the service and the consumers. Part 1: Exposing the on-premise service In theory, this is ultra-straightforward. In practice, and on a dev laptop it is - but in a corporate network with firewalls and proxies, it isn't, so we'll walkthrough some of the pitfalls. Note that I'm using the "old" Azure portal which will soon be out of date, but the new shiny portal should have the same steps available and be easier to use. We start with a simple WCF service which takes a string as input, reverses the string and returns it. The Part 1 version of the code is on GitHub here: on GitHub here: IPASBR Part 1. Configuring Azure Service Bus Start by logging into the Azure portal and registering a Service Bus namespace which will be our endpoint in the cloud. Give it a globally unique name, set it up somewhere near you (if you’re in Europe, remember Europe (North) is Ireland, and Europe (West) is the Netherlands), and enable ACS integration by ticking "Access Control" as a service: Authenticating and authorizing to ACS When we try to register our on-premise service as a listener for the Service Bus endpoint, we need to supply credentials, which means only trusted service providers can act as listeners. We can use the default "owner" credentials, but that has admin permissions so a dedicated service account is better (Neil Mackenzie has a good post On Not Using owner with the Azure AppFabric Service Bus with lots of permission details). Click on "Access Control Service" for the namespace, navigate to Service Identities and add a new one. Give the new account a sensible name and description: Let ACS generate a symmetric key for you (this will be the shared secret we use in the on-premise service to authenticate as a listener), but be sure to set the expiration date to something usable. The portal defaults to expiring new identities after 1 year - but when your year is up *your identity will expire without warning* and everything will stop working. In production, you'll need governance to manage identity expiration and a process to make sure you renew identities and roll new keys regularly. The new service identity needs to be authorized to listen on the service bus endpoint. This is done through claim mapping in ACS - we'll set up a rule that says if the nameidentifier in the input claims has the value serviceProvider, in the output we'll have an action claim with the value Listen. In the ACS portal you'll see that there is already a Relying Party Application set up for ServiceBus, which has a Default rule group. Edit the rule group and click Add to add this new rule: The values to use are: Issuer: Access Control Service Input claim type: http://schemas.xmlsoap.org/ws/2005/05/identity/claims/nameidentifier Input claim value: serviceProvider Output claim type: net.windows.servicebus.action Output claim value: Listen When your service namespace and identity are set up, open the Part 1 solution and put your own namespace, service identity name and secret key into the file AzureConnectionDetails.xml in Solution Items, e.g: <azure namespace="sixeyed-ipasbr">  <service identityName="serviceProvider" symmetricKey="nuR2tHhlrTCqf4YwjT2RA2BZ/+xa23euaRJNLh1a/V4="/> </azure> Build the solution, and the T4 template will generate the Web.config for the service project with your Azure details in the transportClientEndpointBehavior: <behavior name="SharedSecret"> <transportClientEndpointBehavior credentialType="SharedSecret"> <clientCredentials> <sharedSecret issuerName="serviceProvider" issuerSecret="nuR2tHhlrTCqf4YwjT2RA2BZ/+xa23euaRJNLh1a/V4="/> </clientCredentials> </transportClientEndpointBehavior> </behavior> , and your service namespace in the Azure endpoint:  <endpoint address="sb://sixeyed-ipasbr.servicebus.windows.net/net" binding="netTcpRelayBinding" contract="Sixeyed.Ipasbr.Services.IFormatService" behaviorConfiguration="SharedSecret"> </endpoint> The sample project is hosted in IIS, but it won't register with Azure until the service is activated. Typically you'd install AppFabric 1.1 for Widnows Server and set the service to auto-start in IIS, but for dev just navigate to the local REST URL, which will activate the service and register it with Azure. Testing the service locally As well as an Azure endpoint, the service has a WebHttpBinding for local REST access:  <endpoint address="rest" binding="webHttpBinding" behaviorConfiguration="RESTBehavior" contract="Sixeyed.Ipasbr.Services.IFormatService" /> Build the service, then navigate to: http://localhost/Sixeyed.Ipasbr.Services/FormatService.svc/rest/reverse?string=abc123 - and you should see the reversed string response: If your network allows it, you'll get the expected response as before, but in the background your service will also be listening in the cloud. Good stuff! Who needs network security? Onto the next post for consuming the service with the netTcpRelayBinding. Setting up network access to Azure But, if you get an error, it's because your network is secured and it's doing something to stop the relay working. The Service Bus relay bindings try to use direct TCP connections to Azure, so if ports 9350-9354 are available *outbound*, then the relay will run through them. If not, the binding steps down to standard HTTP, and issues a CONNECT across port 443 or 80 to set up a tunnel for the relay. If your network security guys are doing their job, the first option will be blocked by the firewall, and the second option will be blocked by the proxy, so you'll get this error: System.ServiceModel.CommunicationException: Unable to reach sixeyed-ipasbr.servicebus.windows.net via TCP (9351, 9352) or HTTP (80, 443) - and that will probably be the start of lots of discussions. Network guys don't really like giving servers special permissions for the web proxy, and they really don't like opening ports, so they'll need to be convinced about this. The resolution in our case was to put up a dedicated box in a DMZ, tinker with the firewall and the proxy until we got a relay connection working, then run some traffic which the the network guys monitored to do a security assessment afterwards. Along the way we hit a few more issues, diagnosed mainly with Fiddler and Wireshark: System.Net.ProtocolViolationException: Chunked encoding upload is not supported on the HTTP/1.0 protocol - this means the TCP ports are not available, so Azure tries to relay messaging traffic across HTTP. The service can access the endpoint, but the proxy is downgrading traffic to HTTP 1.0, which does not support tunneling, so Azure can’t make its connection. We were using the Squid proxy, version 2.6. The Squid project is incrementally adding HTTP 1.1 support, but there's no definitive list of what's supported in what version (here are some hints). System.ServiceModel.Security.SecurityNegotiationException: The X.509 certificate CN=servicebus.windows.net chain building failed. The certificate that was used has a trust chain that cannot be verified. Replace the certificate or change the certificateValidationMode. The evocation function was unable to check revocation because the revocation server was offline. - by this point we'd given up on the HTTP proxy and opened the TCP ports. We got this error when the relay binding does it's authentication hop to ACS. The messaging traffic is TCP, but the control traffic still goes over HTTP, and as part of the ACS authentication the process checks with a revocation server to see if Microsoft’s ACS cert is still valid, so the proxy still needs some clearance. The service account (the IIS app pool identity) needs access to: www.public-trust.com mscrl.microsoft.com We still got this error periodically with different accounts running the app pool. We fixed that by ensuring the machine-wide proxy settings are set up, so every account uses the correct proxy: netsh winhttp set proxy proxy-server="http://proxy.x.y.z" - and you might need to run this to clear out your credential cache: certutil -urlcache * delete If your network guys end up grudgingly opening ports, they can restrict connections to the IP address range for your chosen Azure datacentre, which might make them happier - see Windows Azure Datacenter IP Ranges. After all that you've hopefully got an on-premise service listening in the cloud, which you can consume from pretty much any technology.

Read the article

Oracle Flashback Technologies - Overview

- by Sridhar_R-Oracle

Oracle Flashback Technologies - IntroductionIn his May 29th 2014 blog, my colleague Joe Meeks introduced Oracle Maximum Availability Architecture (MAA) and discussed both planned and unplanned outages. Let’s take a closer look at unplanned outages. These can be caused by physical failures (e.g., server, storage, network, file deletion, physical corruption, site failures) or by logical failures – cases where all components and files are physically available, but data is incorrect or corrupt. These logical failures are usually caused by human errors or application logic errors. This blog series focuses on these logical errors – what causes them and how to address and recover from them using Oracle Database Flashback. In this introductory blog post, I’ll provide an overview of the Oracle Database Flashback technologies and will discuss the features in detail in future blog posts. Let’s get started. We are all human beings (unless a machine is reading this), and making mistakes is a part of what we do…often what we do best! We “fat finger”, we spill drinks on keyboards, unplug the wrong cables, etc. In addition, many of us, in our lives as DBAs or developers, must have observed, caused, or corrected one or more of the following unpleasant events: Accidentally updated a table with wrong values !! Performed a batch update that went wrong - due to logical errors in the code !! Dropped a table !! How do DBAs typically recover from these types of errors? First, data needs to be restored and recovered to the point-in-time when the error occurred (incomplete or point-in-time recovery). Moreover, depending on the type of fault, it’s possible that some services – or even the entire database – would have to be taken down during the recovery process.Apart from error conditions, there are other questions that need to be addressed as part of the investigation. For example, what did the data look like in the morning, prior to the error? What were the various changes to the row(s) between two timestamps? Who performed the transaction and how can it be reversed? Oracle Database includes built-in Flashback technologies, with features that address these challenges and questions, and enable you to perform faster, easier, and convenient recovery from logical corruptions. HistoryFlashback Query, the first Flashback Technology, was introduced in Oracle 9i. It provides a simple, powerful and completely non-disruptive mechanism for data verification and recovery from logical errors, and enables users to view the state of data at a previous point in time.Flashback Technologies were further enhanced in Oracle 10g, to provide fast, easy recovery at the database, table, row, and even at a transaction level.Oracle Database 11g introduced an innovative method to manage and query long-term historical data with Flashback Data Archive. The 11g release also introduced Flashback Transaction, which provides an easy, one-step operation to back out a transaction. Oracle Database versions 11.2.0.2 and beyond further enhanced the performance of these features. Note that all the features listed here work without requiring any kind of restore operation.In addition, Flashback features are fully supported with the new multi-tenant capabilities introduced with Oracle Database 12c, Flashback Features Oracle Flashback Database enables point-in-time-recovery of the entire database without requiring a traditional restore and recovery operation. It rewinds the entire database to a specified point in time in the past by undoing all the changes that were made since that time.Oracle Flashback Table enables an entire table or a set of tables to be recovered to a point in time in the past.Oracle Flashback Drop enables accidentally dropped tables and all dependent objects to be restored.Oracle Flashback Query enables data to be viewed at a point-in-time in the past. This feature can be used to view and reconstruct data that was lost due to unintentional change(s) or deletion(s). This feature can also be used to build self-service error correction into applications, empowering end-users to undo and correct their errors.Oracle Flashback Version Query offers the ability to query the historical changes to data between two points in time or system change numbers (SCN) Oracle Flashback Transaction Query enables changes to be examined at the transaction level. This capability can be used to diagnose problems, perform analysis, audit transactions, and even revert the transaction by undoing SQLOracle Flashback Transaction is a procedure used to back-out a transaction and its dependent transactions.Flashback technologies eliminate the need for a traditional restore and recovery process to fix logical corruptions or make enquiries. Using these technologies, you can recover from the error in the same amount of time it took to generate the error. All the Flashback features can be accessed either via SQL command line (or) via Enterprise Manager. Most of the Flashback technologies depend on the available UNDO to retrieve older data. The following table describes the various Flashback technologies: their purpose, dependencies and situations where each individual technology can be used. Example Syntax Error investigation related:The purpose is to investigate what went wrong and what the values were at certain points in timeFlashback Queries ( select .. as of SCN | Timestamp ) - Helps to see the value of a row/set of rows at a point in timeFlashback Version Queries ( select .. versions between SCN | Timestamp and SCN | Timestamp) - Helps determine how the value evolved between certain SCNs or between timestamps Flashback Transaction Queries (select .. XID=) - Helps to understand how the transaction caused the changes.Error correction related:The purpose is to fix the error and correct the problems,Flashback Table (flashback table .. to SCN | Timestamp) - To rewind the table to a particular timestamp or SCN to reverse unwanted updates Flashback Drop (flashback table .. to before drop ) - To undrop or undelete a table Flashback Database (flashback database to SCN | Restore Point ) - This is the rewind button for Oracle databases. You can revert the entire database to a particular point in time. It is a fast way to perform a PITR (point-in-time recovery). Flashback Transaction (DBMS_FLASHBACK.TRANSACTION_BACKOUT(XID..)) - To reverse a transaction and its related transactions Advanced use cases Flashback technology is integrated into Oracle Recovery Manager (RMAN) and Oracle Data Guard. So, apart from the basic use cases mentioned above, the following use cases are addressed using Oracle Flashback. Block Media recovery by RMAN - to perform block level recovery Snapshot Standby - where the standby is temporarily converted to a read/write environment for testing, backup, or migration purposes Re-instate old primary in a Data Guard environment – this avoids the need to restore an old backup and perform a recovery to make it a new standby. Guaranteed Restore Points - to bring back the entire database to an older point-in-time in a guaranteed way. and so on..I hope this introductory overview helps you understand how Flashback features can be used to investigate and recover from logical errors. As mentioned earlier, I will take a deeper-dive into to some of the critical Flashback features in my upcoming blogs and address common use cases.

Read the article

DRY and SRP

- by Timothy Klenke

Originally posted on: http://geekswithblogs.net/TimothyK/archive/2014/06/11/dry-and-srp.aspxKent Beck’s XP Simplicity Rules (aka Four Rules of Simple Design) are a prioritized list of rules that when applied to your code generally yield a great design. As you’ll see from the above link the list has slightly evolved over time. I find today they are usually listed as: All Tests Pass Don’t Repeat Yourself (DRY) Express Intent Minimalistic These are prioritized. If your code doesn’t work (rule 1) then everything else is forfeit. Go back to rule one and get the code working before worrying about anything else. Over the years the community have debated whether the priority of rules 2 and 3 should be reversed. Some say a little duplication in the code is OK as long as it helps express intent. I’ve debated it myself. This recent post got me thinking about this again, hence this post. I don’t think it is fair to compare “Expressing Intent” against “DRY”. This is a comparison of apples to oranges. “Expressing Intent” is a principal of code quality. “Repeating Yourself” is a code smell. A code smell is merely an indicator that there might be something wrong with the code. It takes further investigation to determine if a violation of an underlying principal of code quality has actually occurred. For example “using nouns for method names”, “using verbs for property names”, or “using Booleans for parameters” are all code smells that indicate that code probably isn’t doing a good job at expressing intent. They are usually very good indicators. But what principle is the code smell of Duplication pointing to and how good of an indicator is it? Duplication in the code base is bad for a couple reasons. If you need to make a change and that needs to be made in a number of locations it is difficult to know if you have caught all of them. This can lead to bugs if/when one of those locations is overlooked. By refactoring the code to remove all duplication there will be left with only one place to change, thereby eliminating this problem. With most projects the code becomes the single source of truth for a project. If a production code base is inconsistent with a five year old requirements or design document the production code that people are currently living with is usually declared as the current reality (or truth). Requirement or design documents at this age in a project life cycle are usually of little value. Although comparing production code to external documentation is usually straight forward, duplication within the code base muddles this declaration of truth. When code is duplicated small discrepancies will creep in between the two copies over time. The question then becomes which copy is correct? As different factions debate how the software should work, trust in the software and the team behind it erodes. The code smell of Duplication points to a violation of the “Single Source of Truth” principle. Let me define that as: A stakeholder’s requirement for a software change should never cause more than one class to change. Violation of the Single Source of Truth principle will always result in duplication in the code. However, the inverse is not always true. Duplication in the code does not necessarily indicate that there is a violation of the Single Source of Truth principle. To illustrate this, let’s look at a retail system where the system will (1) send a transaction to a bank and (2) print a receipt for the customer. Although these are two separate features of the system, they are closely related. The reason for printing the receipt is usually to provide an audit trail back to the bank transaction. Both features use the same data: amount charged, account number, transaction date, customer name, retail store name, and etcetera. Because both features use much of the same data, there is likely to be a lot of duplication between them. This duplication can be removed by making both features use the same data access layer. Then start coming the divergent requirements. The receipt stakeholder wants a change so that the account number has the last few digits masked out to protect the customer’s privacy. That can be solve with a small IF statement whilst still eliminating all duplication in the system. Then the bank wants to take a picture of the customer as well as capture their signature and/or PIN number for enhanced security. Then the receipt owner wants to pull data from a completely different system to report the customer’s loyalty program point total. After a while you realize that the two stakeholders have somewhat similar, but ultimately different responsibilities. They have their own reasons for pulling the data access layer in different directions. Then it dawns on you, the Single Responsibility Principle: There should never be more than one reason for a class to change. In this example we have two stakeholders giving two separate reasons for the data access class to change. It is clear violation of the Single Responsibility Principle. That’s a problem because it can often lead the project owner pitting the two stakeholders against each other in a vein attempt to get them to work out a mutual single source of truth. But that doesn’t exist. There are two completely valid truths that the developers need to support. How is this to be supported and honour the Single Responsibility Principle? The solution is to duplicate the data access layer and let each stakeholder control their own copy. The Single Source of Truth and Single Responsibility Principles are very closely related. SST tells you when to remove duplication; SRP tells you when to introduce it. They may seem to be fighting each other, but really they are not. The key is to clearly identify the different responsibilities (or sources of truth) over a system. Sometimes there is a single person with that responsibility, other times there are many. This can be especially difficult if the same person has dual responsibilities. They might not even realize they are wearing multiple hats. In my opinion Single Source of Truth should be listed as the second rule of simple design with Express Intent at number three. Investigation of the DRY code smell should yield to the proper application SST, without violating SRP. When necessary leave duplication in the system and let the class names express the different people that are responsible for controlling them. Knowing all the people with responsibilities over a system is the higher priority because you’ll need to know this before you can express it. Although it may be a code smell when there is duplication in the code, it does not necessarily mean that the coder has chosen to be expressive over DRY or that the code is bad.

Read the article

Integration Patterns with Azure Service Bus Relay, Part 3: Anonymous partial-trust consumer

- by Elton Stoneman

This is the third in the IPASBR series, see also: Integration Patterns with Azure Service Bus Relay, Part 1: Exposing the on-premise service Integration Patterns with Azure Service Bus Relay, Part 2: Anonymous full-trust .NET consumer As the patterns get further from the simple .NET full-trust consumer, all that changes is the communication protocol and the authentication mechanism. In Part 3 the scenario is that we still have a secure .NET environment consuming our service, so we can store shared keys securely, but the runtime environment is locked down so we can't use Microsoft.ServiceBus to get the nice WCF relay bindings. To support this we will expose a RESTful endpoint through the Azure Service Bus, and require the consumer to send a security token with each HTTP service request. Pattern applicability This is a good fit for scenarios where: the runtime environment is secure enough to keep shared secrets the consumer can execute custom code, including building HTTP requests with custom headers the consumer cannot use the Azure SDK assemblies the service may need to know who is consuming it the service does not need to know who the end-user is Note there isn't actually a .NET requirement here. By exposing the service in a REST endpoint, anything that can talk HTTP can be a consumer. We'll authenticate through ACS which also gives us REST endpoints, so the service is still accessed securely. Our real-world example would be a hosted cloud app, where we we have enough room in the app's customisation to keep the shared secret somewhere safe and to hook in some HTTP calls. We will be flowing an identity through to the on-premise service now, but it will be the service identity given to the consuming app - the end user's identity isn't flown through yet. In this post, we’ll consume the service from Part 1 in ASP.NET using the WebHttpRelayBinding. The code for Part 3 (+ Part 1) is on GitHub here: IPASBR Part 3. Authenticating and authorizing with ACS We'll follow the previous examples and add a new service identity for the namespace in ACS, so we can separate permissions for different consumers (see walkthrough in Part 1). I've named the identity partialTrustConsumer. We’ll be authenticating against ACS with an explicit HTTP call, so we need a password credential rather than a symmetric key – for a nice secure option, generate a symmetric key, copy to the clipboard, then change type to password and paste in the key: We then need to do the same as in Part 2 , add a rule to map the incoming identity claim to an outgoing authorization claim that allows the identity to send messages to Service Bus: Issuer: Access Control Service Input claim type: http://schemas.xmlsoap.org/ws/2005/05/identity/claims/nameidentifier Input claim value: partialTrustConsumer Output claim type: net.windows.servicebus.action Output claim value: Send As with Part 2, this sets up a service identity which can send messages into Service Bus, but cannot register itself as a listener, or manage the namespace. RESTfully exposing the on-premise service through Azure Service Bus Relay The part 3 sample code is ready to go, just put your Azure details into Solution Items\AzureConnectionDetails.xml and “Run Custom Tool” on the .tt files. But to do it yourself is very simple. We already have a WebGet attribute in the service for locally making REST calls, so we are just going to add a new endpoint which uses the WebHttpRelayBinding to relay that service through Azure. It's as easy as adding this endpoint to Web.config for the service: <endpoint address="https://sixeyed-ipasbr.servicebus.windows.net/rest" binding="webHttpRelayBinding" contract="Sixeyed.Ipasbr.Services.IFormatService" behaviorConfiguration="SharedSecret"> </endpoint> - and adding the webHttp attribute in your endpoint behavior: <behavior name="SharedSecret"> <webHttp/> <transportClientEndpointBehavior credentialType="SharedSecret"> <clientCredentials> <sharedSecret issuerName="serviceProvider" issuerSecret="gl0xaVmlebKKJUAnpripKhr8YnLf9Neaf6LR53N8uGs="/> </clientCredentials> </transportClientEndpointBehavior> </behavior> Where's my WSDL? The metadata story for REST is a bit less automated. In our local webHttp endpoint we've enabled WCF's built-in help, so if you navigate to: http://localhost/Sixeyed.Ipasbr.Services/FormatService.svc/rest/help - you'll see the uri format for making a GET request to the service. The format is the same over Azure, so this is where you'll be connecting: https://[your-namespace].servicebus.windows.net/rest/reverse?string=abc123 Build the service with the new endpoint, open that in a browser and you'll get an XML version of an HTTP status code - a 401 with an error message stating that you haven’t provided an authorization header: <?xml version="1.0"?><Error><Code>401</Code><Detail>MissingToken: The request contains no authorization header..TrackingId:4cb53408-646b-4163-87b9-bc2b20cdfb75_5,TimeStamp:10/3/2012 8:34:07 PM</Detail></Error> By default, the setup of your Service Bus endpoint as a relying party in ACS expects a Simple Web Token to be presented with each service request, and in the browser we're not passing one, so we can't access the service. Note that this request doesn't get anywhere near your on-premise service, Service Bus only relays requests once they've got the necessary approval from ACS. Why didn't the consumer need to get ACS authorization in Part 2? It did, but it was all done behind the scenes in the NetTcpRelayBinding. By specifying our Shared Secret credentials in the consumer, the service call is preceded by a check on ACS to see that the identity provided is a) valid, and b) allowed access to our Service Bus endpoint. By making manual HTTP requests, we need to take care of that ACS check ourselves now. We do that with a simple WebClient call to the ACS endpoint of our service; passing the shared secret credentials, we will get back an SWT: var values = new System.Collections.Specialized.NameValueCollection(); values.Add("wrap_name", "partialTrustConsumer"); //service identity name values.Add("wrap_password", "suCei7AzdXY9toVH+S47C4TVyXO/UUFzu0zZiSCp64Y="); //service identity password values.Add("wrap_scope", "http://sixeyed-ipasbr.servicebus.windows.net/"); //this is the realm of the RP in ACS var acsClient = new WebClient(); var responseBytes = acsClient.UploadValues("https://sixeyed-ipasbr-sb.accesscontrol.windows.net/WRAPv0.9/", "POST", values); rawToken = System.Text.Encoding.UTF8.GetString(responseBytes); With a little manipulation, we then attach the SWT to subsequent REST calls in the authorization header; the token contains the Send claim returned from ACS, so we will be authorized to send messages into Service Bus. Running the sample Navigate to http://localhost:2028/Sixeyed.Ipasbr.WebHttpClient/Default.cshtml, enter a string and hit Go! - your string will be reversed by your on-premise service, routed through Azure: Using shared secret client credentials in this way means ACS is the identity provider for your service, and the claim which allows Send access to Service Bus is consumed by Service Bus. None of the authentication details make it through to your service, so your service is not aware who the consumer is (MSDN calls this "anonymous authentication").

Read the article

Getting Started with jqChart for ASP.NET Web Forms

- by jqChart

Official Site | Samples | Download | Documentation | Forum | Twitter Introduction jqChart takes advantages of HTML5 Canvas to deliver high performance client-side charts and graphs across browsers (IE 6+, Firefox, Chrome, Opera, Safari) and devices, including iOS and Android mobile devices. Some of the key features are: High performance rendering. Animaitons. Scrolling/Zoooming. Support for unlimited number of data series and data points. Support for unlimited number of chart axes. True DateTime Axis. Logarithmic and Reversed axis scale. Large set of chart types - Bar, Column, Pie, Line, Spline, Area, Scatter, Bubble, Radar, Polar. Financial Charts - Stock Chart and Candlestick Chart. The different chart types can be easily combined. System Requirements Browser Support jqChart supports all major browsers: Internet Explorer - 6+ Firefox Google Chrome Opera Safari jQuery version support jQuery JavaScript framework is required. We recommend using the latest official stable version of the jQuery library. Visual Studio Support jqChart for ASP.NET does not require using Visual Studio. You can use your favourite code editor. Still, the product has been tested with several versions of Visual Studio .NET and you can find the list of supported versions below: Visual Studio 2008 Visual Studio 2010 Visual Studio 2012 ASP.NET Web Forms support Supported version - ASP.NET Web Forms 3.5, 4.0 and 4.5 Installation Download and unzip the contents of the archive to any convenient location. The package contains the following folders: [bin] - Contains the assembly DLLs of the product (JQChart.Web.dll) for WebForms 3.5, 4.0 and 4.5. This is the assembly that you can reference directly in your web project (or better yet, add it to your ToolBox and then drag & drop it from there). [js] - The javascript files of jqChart and jqRangeSlider (and the needed libraries). You need to include them in your ASPX page, in order to gain the client side functionality of the chart. The first file is "jquery-1.5.1.min.js" - this is the official jQuery library. jqChart is built upon jQuery library version 1.4.3. The second file you need is the "excanvas.js" javascript file. It is used from the versions of IE, which dosn't support canvas graphics. The third is the jqChart javascript code itself, located in "jquery.jqChart.min.js". The last one is the jqRangeSlider javascript, located in "jquery.jqRangeSlider.min.js". It is used when the chart zooming is enabled. [css] - Contains the Css files that the jqChart and the jqRangeSlider need. [samples] - Contains some examples that use the jqChart. For full list of samples plese visit - jqChart for ASP.NET Samples. [themes] - Contains the themes shipped with the products. It is used from the jqRangeSlider. Since jqRangeSlider supports jQuery UI Themeroller, any theme compatible with jQuery UI ThemeRoller will work for jqRangeSlider as well. You can download any additional themes directly from jQuery UI's ThemeRoller site available here: http://jqueryui.com/themeroller/ or reference them from Microsoft's / Google's CDN. <link rel="stylesheet" type="text/css" media="screen" href="http://ajax.aspnetcdn.com/ajax/jquery.ui/1.8.21/themes/smoothness/jquery-ui.css" /> The final result you will have in an ASPX page containing jqChart would be something similar to that (assuming you have copied the [js] to the Script folder and [css] to Content folder of your ASP.NET site respectively). <%@ Page Language="C#" AutoEventWireup="true" CodeBehind="Default.aspx.cs" Inherits="samples_cs.Default" %> <%@ Register Assembly="JQChart.Web" Namespace="JQChart.Web.UI.WebControls" TagPrefix="jqChart" %> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> <html > <head runat="server"> <title>jqChart ASP.NET Sample</title> <link rel="stylesheet" type="text/css" href="~/Content/jquery.jqChart.css" /> <link rel="stylesheet" type="text/css" href="~/Content/jquery.jqRangeSlider.css" /> <link rel="stylesheet" type="text/css" href="~/Content/themes/smoothness/jquery-ui-1.8.21.css" /> <script src="<% = ResolveUrl("~/Scripts/jquery-1.5.1.min.js") %>" type="text/javascript"></script> <script src="<% = ResolveUrl("~/Scripts/jquery.jqRangeSlider.min.js") %>" type="text/javascript"></script> <script src="<% = ResolveUrl("~/Scripts/jquery.jqChart.min.js") %>" type="text/javascript"></script>  </head> <body> <form id="form1" runat="server"> <asp:ObjectDataSource ID="ObjectDataSource1" runat="server" SelectMethod="GetData" TypeName="SamplesBrowser.Models.ChartData"></asp:ObjectDataSource> <jqChart:Chart ID="Chart1" Width="500px" Height="300px" runat="server" DataSourceID="ObjectDataSource1"> <Title Text="Chart Title"></Title> <Animation Enabled="True" Duration="00:00:01" /> <Axes> <jqChart:CategoryAxis Location="Bottom" ZoomEnabled="true"> </jqChart:CategoryAxis> </Axes> <Series> <jqChart:ColumnSeries XValuesField="Label" YValuesField="Value1" Title="Column"> </jqChart:ColumnSeries> <jqChart:LineSeries XValuesField="Label" YValuesField="Value2" Title="Line"> </jqChart:LineSeries> </Series> </jqChart:Chart> </form> </body> </html> Official Site | Samples | Download | Documentation | Forum | Twitter

Read the article

So…is it a Seek or a Scan?

- by Paul White

You’re probably most familiar with the terms ‘Seek’ and ‘Scan’ from the graphical plans produced by SQL Server Management Studio (SSMS). The image to the left shows the most common ones, with the three types of scan at the top, followed by four types of seek. You might look to the SSMS tool-tip descriptions to explain the differences between them: Not hugely helpful are they? Both mention scans and ranges (nothing about seeks) and the Index Seek description implies that it will not scan the index entirely (which isn’t necessarily true). Recall also yesterday’s post where we saw two Clustered Index Seek operations doing very different things. The first Seek performed 63 single-row seeking operations; and the second performed a ‘Range Scan’ (more on those later in this post). I hope you agree that those were two very different operations, and perhaps you are wondering why there aren’t different graphical plan icons for Range Scans and Seeks? I have often wondered about that, and the first person to mention it after yesterday’s post was Erin Stellato (twitter | blog): Before we go on to make sense of all this, let’s look at another example of how SQL Server confusingly mixes the terms ‘Scan’ and ‘Seek’ in different contexts. The diagram below shows a very simple heap table with two columns, one of which is the non-clustered Primary Key, and the other has a non-unique non-clustered index defined on it. The right hand side of the diagram shows a simple query, it’s associated query plan, and a couple of extracts from the SSMS tool-tip and Properties windows. Notice the ‘scan direction’ entry in the Properties window snippet. Is this a seek or a scan? The different references to Scans and Seeks are even more pronounced in the XML plan output that the graphical plan is based on. This fragment is what lies behind the single Index Seek icon shown above: You’ll find the same confusing references to Seeks and Scans throughout the product and its documentation. Making Sense of Seeks Let’s forget all about scans for a moment, and think purely about seeks. Loosely speaking, a seek is the process of navigating an index B-tree to find a particular index record, most often at the leaf level. A seek starts at the root and navigates down through the levels of the index to find the point of interest: Singleton Lookups The simplest sort of seek predicate performs this traversal to find (at most) a single record. This is the case when we search for a single value using a unique index and an equality predicate. It should be readily apparent that this type of search will either find one record, or none at all. This operation is known as a singleton lookup. Given the example table from before, the following query is an example of a singleton lookup seek: Sadly, there’s nothing in the graphical plan or XML output to show that this is a singleton lookup – you have to infer it from the fact that this is a single-value equality seek on a unique index. The other common examples of a singleton lookup are bookmark lookups – both the RID and Key Lookup forms are singleton lookups (an RID lookup finds a single record in a heap from the unique row locator, and a Key Lookup does much the same thing on a clustered table). If you happen to run your query with STATISTICS IO ON, you will notice that ‘Scan Count’ is always zero for a singleton lookup. Range Scans The other type of seek predicate is a ‘seek plus range scan’, which I will refer to simply as a range scan. The seek operation makes an initial descent into the index structure to find the first leaf row that qualifies, and then performs a range scan (either backwards or forwards in the index) until it reaches the end of the scan range. The ability of a range scan to proceed in either direction comes about because index pages at the same level are connected by a doubly-linked list – each page has a pointer to the previous page (in logical key order) as well as a pointer to the following page. The doubly-linked list is represented by the green and red dotted arrows in the index diagram presented earlier. One subtle (but important) point is that the notion of a ‘forward’ or ‘backward’ scan applies to the logical key order defined when the index was built. In the present case, the non-clustered primary key index was created as follows: CREATE TABLE dbo.Example ( key_col INTEGER NOT NULL, data INTEGER NOT NULL, CONSTRAINT [PK dbo.Example key_col] PRIMARY KEY NONCLUSTERED (key_col ASC) ) ; Notice that the primary key index specifies an ascending sort order for the single key column. This means that a forward scan of the index will retrieve keys in ascending order, while a backward scan would retrieve keys in descending key order. If the index had been created instead on key_col DESC, a forward scan would retrieve keys in descending order, and a backward scan would return keys in ascending order. A range scan seek predicate may have a Start condition, an End condition, or both. Where one is missing, the scan starts (or ends) at one extreme end of the index, depending on the scan direction. Some examples might help clarify that: the following diagram shows four queries, each of which performs a single seek against a column holding every integer from 1 to 100 inclusive. The results from each query are shown in the blue columns, and relevant attributes from the Properties window appear on the right: Query 1 specifies that all key_col values less than 5 should be returned in ascending order. The query plan achieves this by seeking to the start of the index leaf (there is no explicit starting value) and scanning forward until the End condition (key_col < 5) is no longer satisfied (SQL Server knows it can stop looking as soon as it finds a key_col value that isn’t less than 5 because all later index entries are guaranteed to sort higher). Query 2 asks for key_col values greater than 95, in descending order. SQL Server returns these results by seeking to the end of the index, and scanning backwards (in descending key order) until it comes across a row that isn’t greater than 95. Sharp-eyed readers may notice that the end-of-scan condition is shown as a Start range value. This is a bug in the XML show plan which bubbles up to the Properties window – when a backward scan is performed, the roles of the Start and End values are reversed, but the plan does not reflect that. Oh well. Query 3 looks for key_col values that are greater than or equal to 10, and less than 15, in ascending order. This time, SQL Server seeks to the first index record that matches the Start condition (key_col >= 10) and then scans forward through the leaf pages until the End condition (key_col < 15) is no longer met. Query 4 performs much the same sort of operation as Query 3, but requests the output in descending order. Again, we have to mentally reverse the Start and End conditions because of the bug, but otherwise the process is the same as always: SQL Server finds the highest-sorting record that meets the condition ‘key_col < 25’ and scans backward until ‘key_col >= 20’ is no longer true. One final point to note: seek operations always have the Ordered: True attribute. This means that the operator always produces rows in a sorted order, either ascending or descending depending on how the index was defined, and whether the scan part of the operation is forward or backward. You cannot rely on this sort order in your queries of course (you must always specify an ORDER BY clause if order is important) but SQL Server can make use of the sort order internally. In the four queries above, the query optimizer was able to avoid an explicit Sort operator to honour the ORDER BY clause, for example. Multiple Seek Predicates As we saw yesterday, a single index seek plan operator can contain one or more seek predicates. These seek predicates can either be all singleton seeks or all range scans – SQL Server does not mix them. For example, you might expect the following query to contain two seek predicates, a singleton seek to find the single record in the unique index where key_col = 10, and a range scan to find the key_col values between 15 and 20: SELECT key_col FROM dbo.Example WHERE key_col = 10 OR key_col BETWEEN 15 AND 20 ORDER BY key_col ASC ; In fact, SQL Server transforms the singleton seek (key_col = 10) to the equivalent range scan, Start:[key_col >= 10], End:[key_col <= 10]. This allows both range scans to be evaluated by a single seek operator. To be clear, this query results in two range scans: one from 10 to 10, and one from 15 to 20. Final Thoughts That’s it for today – tomorrow we’ll look at monitoring singleton lookups and range scans, and I’ll show you a seek on a heap table. Yes, a seek. On a heap. Not an index! If you would like to run the queries in this post for yourself, there’s a script below. Thanks for reading! IF OBJECT_ID(N'dbo.Example', N'U') IS NOT NULL BEGIN DROP TABLE dbo.Example; END ; -- Test table is a heap -- Non-clustered primary key on 'key_col' CREATE TABLE dbo.Example ( key_col INTEGER NOT NULL, data INTEGER NOT NULL, CONSTRAINT [PK dbo.Example key_col] PRIMARY KEY NONCLUSTERED (key_col) ) ; -- Non-unique non-clustered index on the 'data' column CREATE NONCLUSTERED INDEX [IX dbo.Example data] ON dbo.Example (data) ; -- Add 100 rows INSERT dbo.Example WITH (TABLOCKX) ( key_col, data ) SELECT key_col = V.number, data = V.number FROM master.dbo.spt_values AS V WHERE V.[type] = N'P' AND V.number BETWEEN 1 AND 100 ; -- ================ -- Singleton lookup -- ================ ; -- Single value equality seek in a unique index -- Scan count = 0 when STATISTIS IO is ON -- Check the XML SHOWPLAN SELECT E.key_col FROM dbo.Example AS E WHERE E.key_col = 32 ; -- =========== -- Range Scans -- =========== ; -- Query 1 SELECT E.key_col FROM dbo.Example AS E WHERE E.key_col <= 5 ORDER BY E.key_col ASC ; -- Query 2 SELECT E.key_col FROM dbo.Example AS E WHERE E.key_col > 95 ORDER BY E.key_col DESC ; -- Query 3 SELECT E.key_col FROM dbo.Example AS E WHERE E.key_col >= 10 AND E.key_col < 15 ORDER BY E.key_col ASC ; -- Query 4 SELECT E.key_col FROM dbo.Example AS E WHERE E.key_col >= 20 AND E.key_col < 25 ORDER BY E.key_col DESC ; -- Final query (singleton + range = 2 range scans) SELECT E.key_col FROM dbo.Example AS E WHERE E.key_col = 10 OR E.key_col BETWEEN 15 AND 20 ORDER BY E.key_col ASC ; -- === TIDY UP === DROP TABLE dbo.Example; © 2011 Paul White email: [email protected] twitter: @SQL_Kiwi

Read the article

Performance Enhancement in Full-Text Search Query

- by Calvin Sun

Ever since its first release, we are continuing consolidating and developing InnoDB Full-Text Search feature. There is one recent improvement that worth blogging about. It is an effort with MySQL Optimizer team that simplifies some common queries’ Query Plans and dramatically shorted the query time. I will describe the issue, our solution and the end result by some performance numbers to demonstrate our efforts in continuing enhancement the Full-Text Search capability. The Issue: As we had discussed in previous Blogs, InnoDB implements Full-Text index as reversed auxiliary tables. The query once parsed will be reinterpreted into several queries into related auxiliary tables and then results are merged and consolidated to come up with the final result. So at the end of the query, we’ll have all matching records on hand, sorted by their ranking or by their Doc IDs. Unfortunately, MySQL’s optimizer and query processing had been initially designed for MyISAM Full-Text index, and sometimes did not fully utilize the complete result package from InnoDB. Here are a couple examples: Case 1: Query result ordered by Rank with only top N results: mysql> SELECT FTS_DOC_ID, MATCH (title, body) AGAINST ('database') AS SCORE FROM articles ORDER BY score DESC LIMIT 1; In this query, user tries to retrieve a single record with highest ranking. It should have a quick answer once we have all the matching documents on hand, especially if there are ranked. However, before this change, MySQL would almost retrieve rankings for almost every row in the table, sort them and them come with the top rank result. This whole retrieve and sort is quite unnecessary given the InnoDB already have the answer. In a real life case, user could have millions of rows, so in the old scheme, it would retrieve millions of rows' ranking and sort them, even if our FTS already found there are two 3 matched rows. Apparently, the million ranking retrieve is done in vain. In above case, it should just ask for 3 matched rows' ranking, all other rows' ranking are 0. If it want the top ranking, then it can just get the first record from our already sorted result. Case 2: Select Count(*) on matching records: mysql> SELECT COUNT(*) FROM articles WHERE MATCH (title,body) AGAINST ('database' IN NATURAL LANGUAGE MODE); In this case, InnoDB search can find matching rows quickly and will have all matching rows. However, before our change, in the old scheme, every row in the table was requested by MySQL one by one, just to check whether its ranking is larger than 0, and later comes up a count. In fact, there is no need for MySQL to fetch all rows, instead InnoDB already had all the matching records. The only thing need is to call an InnoDB API to retrieve the count The difference can be huge. Following query output shows how big the difference can be: mysql> select count(*) from searchindex_inno where match(si_title, si_text) against ('people') +----------+ | count(*) | +----------+ | 666877 | +----------+ 1 row in set (16 min 17.37 sec) So the query took almost 16 minutes. Let’s see how long the InnoDB can come up the result. In InnoDB, you can obtain extra diagnostic printout by turning on “innodb_ft_enable_diag_print”, this will print out extra query info: Error log: keynr=2, 'people' NL search Total docs: 10954826 Total words: 0 UNION: Searching: 'people' Processing time: 2 secs: row(s) 666877: error: 10 ft_init() ft_init_ext() keynr=2, 'people' NL search Total docs: 10954826 Total words: 0 UNION: Searching: 'people' Processing time: 3 secs: row(s) 666877: error: 10 Output shows it only took InnoDB only 3 seconds to get the result, while the whole query took 16 minutes to finish. So large amount of time has been wasted on the un-needed row fetching. The Solution: The solution is obvious. MySQL can skip some of its steps, optimize its plan and obtain useful information directly from InnoDB. Some of savings from doing this include: 1) Avoid redundant sorting. Since InnoDB already sorted the result according to ranking. MySQL Query Processing layer does not need to sort to get top matching results. 2) Avoid row by row fetching to get the matching count. InnoDB provides all the matching records. All those not in the result list should all have ranking of 0, and no need to be retrieved. And InnoDB has a count of total matching records on hand. No need to recount. 3) Covered index scan. InnoDB results always contains the matching records' Document ID and their ranking. So if only the Document ID and ranking is needed, there is no need to go to user table to fetch the record itself. 4) Narrow the search result early, reduce the user table access. If the user wants to get top N matching records, we do not need to fetch all matching records from user table. We should be able to first select TOP N matching DOC IDs, and then only fetch corresponding records with these Doc IDs. Performance Results and comparison with MyISAM The result by this change is very obvious. I includes six testing result performed by Alexander Rubin just to demonstrate how fast the InnoDB query now becomes when comparing MyISAM Full-Text Search. These tests are base on the English Wikipedia data of 5.4 Million rows and approximately 16G table. The test was performed on a machine with 1 CPU Dual Core, SSD drive, 8G of RAM and InnoDB_buffer_pool is set to 8 GB. Table 1: SELECT with LIMIT CLAUSE mysql> SELECT si_title, match(si_title, si_text) against('family') as rel FROM si WHERE match(si_title, si_text) against('family') ORDER BY rel desc LIMIT 10; InnoDB MyISAM Times Faster Time for the query 1.63 sec 3 min 26.31 sec 127 You can see for this particular query (retrieve top 10 records), InnoDB Full-Text Search is now approximately 127 times faster than MyISAM. Table 2: SELECT COUNT QUERY mysql>select count(*) from si where match(si_title, si_text) against('family‘); +----------+ | count(*) | +----------+ | 293955 | +----------+ InnoDB MyISAM Times Faster Time for the query 1.35 sec 28 min 59.59 sec 1289 In this particular case, where there are 293k matching results, InnoDB took only 1.35 second to get all of them, while take MyISAM almost half an hour, that is about 1289 times faster!. Table 3: SELECT ID with ORDER BY and LIMIT CLAUSE for selected terms mysql> SELECT <ID>, match(si_title, si_text) against(<TERM>) as rel FROM si_<TB> WHERE match(si_title, si_text) against (<TERM>) ORDER BY rel desc LIMIT 10; Term InnoDB (time to execute) MyISAM(time to execute) Times Faster family 0.5 sec 5.05 sec 10.1 family film 0.95 sec 25.39 sec 26.7 Pizza restaurant orange county California 0.93 sec 32.03 sec 34.4 President united states of America 2.5 sec 36.98 sec 14.8 Table 4: SELECT title and text with ORDER BY and LIMIT CLAUSE for selected terms mysql> SELECT <ID>, si_title, si_text, ... as rel FROM si_<TB> WHERE match(si_title, si_text) against (<TERM>) ORDER BY rel desc LIMIT 10; Term InnoDB (time to execute) MyISAM(time to execute) Times Faster family 0.61 sec 41.65 sec 68.3 family film 1.15 sec 47.17 sec 41.0 Pizza restaurant orange county california 1.03 sec 48.2 sec 46.8 President united states of america 2.49 sec 44.61 sec 17.9 Table 5: SELECT ID with ORDER BY and LIMIT CLAUSE for selected terms mysql> SELECT <ID>, match(si_title, si_text) against(<TERM>) as rel FROM si_<TB> WHERE match(si_title, si_text) against (<TERM>) ORDER BY rel desc LIMIT 10; Term InnoDB (time to execute) MyISAM(time to execute) Times Faster family 0.5 sec 5.05 sec 10.1 family film 0.95 sec 25.39 sec 26.7 Pizza restaurant orange county califormia 0.93 sec 32.03 sec 34.4 President united states of america 2.5 sec 36.98 sec 14.8 Table 6: SELECT COUNT(*) mysql> SELECT count(*) FROM si_<TB> WHERE match(si_title, si_text) against (<TERM>) LIMIT 10; Term InnoDB (time to execute) MyISAM(time to execute) Times Faster family 0.47 sec 82 sec 174.5 family film 0.83 sec 131 sec 157.8 Pizza restaurant orange county califormia 0.74 sec 106 sec 143.2 President united states of america 1.96 sec 220 sec 112.2 Again, table 3 to table 6 all showing InnoDB consistently outperform MyISAM in these queries by a large margin. It becomes obvious the InnoDB has great advantage over MyISAM in handling large data search. Summary: These results demonstrate the great performance we could achieve by making MySQL optimizer and InnoDB Full-Text Search more tightly coupled. I think there are still many cases that InnoDB’s result info have not been fully taken advantage of, which means we still have great room to improve. And we will continuously explore the area, and get more dramatic results for InnoDB full-text searches. Jimmy Yang, September 29, 2012

Read the article

DataTrigger inside ControlTemplate doesn't update

- by kennethkryger

I have a ListBox that is bound to a list of CustomerViewModel-objects, that each has two dependency properties: - Name (string) - Description (string) - IsVisible (bool) (the IsVisible property is True by default and is reversed via the ToggleVisibility Command on the CustomerViewModel) I would like to display the Name and Description to the right of a Border-control, that is has a Transparent background when the IsVisible property is True and Green when the False. My problem is that the DataTrigger part of the code below doesn't work the way I want, because the Setter-part isn't triggered when the IsVisible is changed. What am I doing wrong? Here's my code: <UserControl.Resources> <Style x:Key="ListBoxStyle" TargetType="{x:Type ListBox}"> <Setter Property="Margin" Value="-1,-1,0,0" /> <Setter Property="BorderThickness" Value="0" /> <Setter Property="Background" Value="Transparent" /> <Setter Property="ItemContainerStyle" Value="{DynamicResource ListboxItemStyle}" /> <Setter Property="ScrollViewer.HorizontalScrollBarVisibility" Value="Disabled" /> </Style> <Style x:Key="ListboxItemStyle" TargetType="{x:Type ListBoxItem}"> <Setter Property="Background" Value="Transparent" /> <Setter Property="FocusVisualStyle" Value="{x:Null}" /> <Setter Property="Template"> <Setter.Value> <ControlTemplate TargetType="{x:Type ListBoxItem}"> <Grid> <Border x:Name="border" Background="{TemplateBinding Background}" BorderBrush="#FFD4D6D5" BorderThickness="0,0,0,1"> <Grid Height="70" Margin="0,0,10,0"> <Grid.RowDefinitions> <RowDefinition Height="10" /> <RowDefinition Height="Auto" /> <RowDefinition /> <RowDefinition Height="10" /> </Grid.RowDefinitions> <Grid.ColumnDefinitions> <ColumnDefinition Width="Auto" /> <ColumnDefinition /> </Grid.ColumnDefinitions> <Border x:Name="visibilityColumn" Grid.Row="0" Grid.Column="0" Grid.RowSpan="4" Background="Transparent" Width="4" Margin="0,0,4,0" /> <TextBlock x:Name="customerName" Grid.Row="1" Grid.Column="1" Foreground="#FF191919" FontWeight="Bold" Text="{Binding Name}" VerticalAlignment="Top" /> <TextBlock Grid.Row="2" Grid.Column="1" VerticalAlignment="Stretch" Text="{Binding Description}" TextWrapping="Wrap" Foreground="#FFB4B4B4" TextTrimming="CharacterEllipsis" /> </Grid> <Border.ContextMenu> <ContextMenu> <MenuItem Header="Edit..." /> <MenuItem Header="Visible" IsCheckable="True" IsChecked="{Binding IsVisible}" Command="{Binding ToggleVisibility}"/> </ContextMenu> </Border.ContextMenu> </Border> </Grid> <ControlTemplate.Triggers> <Trigger Property="IsMouseOver" Value="True"> <Setter Property="Background" Value="#FFEEEEEE" /> </Trigger> <Trigger Property="IsSelected" Value="True"> <Setter Property="Background" Value="#FFF5F5F5" /> <Setter TargetName="customerName" Property="Foreground" Value="Green" /> </Trigger> <DataTrigger Binding="{Binding IsVisible}" Value="False">  <Setter Property="Background" Value="Green" /> </DataTrigger> </ControlTemplate.Triggers> </ControlTemplate> </Setter.Value> </Setter> </Style> </UserControl.Resources> <ListBox Style="{StaticResource ListBoxStyle}" ItemsSource="{Binding CustomerViewModels}" />

Read the article

Incremental PCA

- by smichak

Hi, Lately, I've been looking into an implementation of an incremental PCA algorithm in python - I couldn't find something that would meet my needs so I did some reading and implemented an algorithm I found in some paper. Here is the module's code - the relevant paper on which it is based is mentioned in the module's documentation. I would appreciate any feedback from people who are interested in this. Micha #!/usr/bin/env python """ Incremental PCA calculation module. Based on P.Hall, D. Marshall and R. Martin "Incremental Eigenalysis for Classification" which appeared in British Machine Vision Conference, volume 1, pages 286-295, September 1998. Principal components are updated sequentially as new observations are introduced. Each new observation (x) is projected on the eigenspace spanned by the current principal components (U) and the residual vector (r = x - U(U.T*x)) is used as a new principal component (U' = [U r]). The new principal components are then rotated by a rotation matrix (R) whose columns are the eigenvectors of the transformed covariance matrix (D=U'.T*C*U) to yield p + 1 principal components. From those, only the first p are selected. """ __author__ = "Micha Kalfon" import numpy as np _ZERO_THRESHOLD = 1e-9 # Everything below this is zero class IPCA(object): """Incremental PCA calculation object. General Parameters: m - Number of variables per observation n - Number of observations p - Dimension to which the data should be reduced """ def __init__(self, m, p): """Creates an incremental PCA object for m-dimensional observations in order to reduce them to a p-dimensional subspace. @param m: Number of variables per observation. @param p: Number of principle components. @return: An IPCA object. """ self._m = float(m) self._n = 0.0 self._p = float(p) self._mean = np.matrix(np.zeros((m , 1), dtype=np.float64)) self._covariance = np.matrix(np.zeros((m, m), dtype=np.float64)) self._eigenvectors = np.matrix(np.zeros((m, p), dtype=np.float64)) self._eigenvalues = np.matrix(np.zeros((1, p), dtype=np.float64)) def update(self, x): """Updates with a new observation vector x. @param x: Next observation as a column vector (m x 1). """ m = self._m n = self._n p = self._p mean = self._mean C = self._covariance U = self._eigenvectors E = self._eigenvalues if type(x) is not np.matrix or x.shape != (m, 1): raise TypeError('Input is not a matrix (%d, 1)' % int(m)) # Update covariance matrix and mean vector and centralize input around # new mean oldmean = mean mean = (n*mean + x) / (n + 1.0) C = (n*C + x*x.T + n*oldmean*oldmean.T - (n+1)*mean*mean.T) / (n + 1.0) x -= mean # Project new input on current p-dimensional subspace and calculate # the normalized residual vector g = U.T*x r = x - (U*g) r = (r / np.linalg.norm(r)) if not _is_zero(r) else np.zeros_like(r) # Extend the transformation matrix with the residual vector and find # the rotation matrix by solving the eigenproblem DR=RE U = np.concatenate((U, r), 1) D = U.T*C*U (E, R) = np.linalg.eigh(D) # Sort eigenvalues and eigenvectors from largest to smallest to get the # rotation matrix R sorter = list(reversed(E.argsort(0))) E = E[sorter] R = R[:,sorter] # Apply the rotation matrix U = U*R # Select only p largest eigenvectors and values and update state self._n += 1.0 self._mean = mean self._covariance = C self._eigenvectors = U[:, 0:p] self._eigenvalues = E[0:p] @property def components(self): """Returns a matrix with the current principal components as columns. """ return self._eigenvectors @property def variances(self): """Returns a list with the appropriate variance along each principal component. """ return self._eigenvalues def _is_zero(x): """Return a boolean indicating whether the given vector is a zero vector up to a threshold. """ return np.fabs(x).min() < _ZERO_THRESHOLD if __name__ == '__main__': import sys def pca_svd(X): X = X - X.mean(0).repeat(X.shape[0], 0) [_, _, V] = np.linalg.svd(X) return V N = 1000 obs = np.matrix([np.random.normal(size=10) for _ in xrange(N)]) V = pca_svd(obs) print V[0:2] pca = IPCA(obs.shape[1], 2) for i in xrange(obs.shape[0]): x = obs[i,:].transpose() pca.update(x) U = pca.components print U

Read the article

Adding to database with multiple text boxes

- by kira423

What I am trying to do with this script is allow users to update a url for their websites, and since each user isn't going to have the same amount of websites is is hard for me to just add $_POST['website'] for each of these. Here is the script <?php include("config.php"); include("header.php"); include("functions.php"); if(!isset($_SESSION['username']) && !isset($_SESSION['password'])){ header("Location: pubs.php"); } $getmember = mysql_query("SELECT * FROM `publishers` WHERE username = '".$_SESSION['username']."'"); $info = mysql_fetch_array($getmember); $getsites = mysql_query("SELECT * FROM `websites` WHERE publisher = '".$info['username']."'"); $postback = $_POST['website']; $webname = $_POST['webid']; if($_POST['submit']){ var_dump( $_POST['website'] ); $update = mysql_query("UPDATE `websites` SET `postback` = '$postback' WHERE name = '$webname'"); } print" <div id='center'> <span id='tools_lander'><a href='export.php'>Export Campaigns</a></span> <div id='calendar_holder'> <h3>Please define a postback for each of your websites below. The following variables should be used when creating your postback.<br /> cid = Campaign ID<br /> sid = Sub ID<br /> rate = Campaign Rate<br /> status = Status of Lead. 1 means payable 2 mean reversed<br /> A sample postback URL would be <br /> http://www.example.com/postback.php?cid=#cid&sid=#sid&rate=#rate&status=#status</h3> <table class='balances' align='center'> <form method='POST' action=''>"; while($website = mysql_fetch_array($getsites)){ print" <tr> <input type ='hidden' name='webid' value='".$website['id']."' /> <td style='font-weight:bold;'>".$website['name']."'s Postback:</td> <td><input type='text' style='width:400px;' name='website[]' value='".$website['postback']."' /></td> </tr>"; } print" <td style='float:right;position:relative;left:150px;'><input type='submit' name='submit' style='font-size:15px;height:30px;width:100px;' value='Submit' /></td> </form> </table> </div>"; include("footer.php"); ?> What I am attempting to do insert the what is inputted in the text boxes to their corresponding websites, and I cannot think of any other way to do it, and this obviously does not works and returns a notice stating Array to string conversion If there is a more logical way to do this please let me know.

Read the article

Help with refactoring PHP code

- by Richard Knop

I had some troubles implementing Lawler's algorithm but thanks to SO and a bounty of 200 reputation I finally managed to write a working implementation: http://stackoverflow.com/questions/2466928/lawlers-algorithm-implementation-assistance I feel like I'm using too many variables and loops there though so I am trying to refactor the code. It should be simpler and shorter yet remain readable. Does it make sense to make a class for this? Any advice or even help with refactoring this piece of code is welcomed: <?php /* * @name Lawler's algorithm PHP implementation * @desc This algorithm calculates an optimal schedule of jobs to be * processed on a single machine (in reversed order) while taking * into consideration any precedence constraints. * @author Richard Knop * */ $jobs = array(1 => array('processingTime' => 2, 'dueDate' => 3), 2 => array('processingTime' => 3, 'dueDate' => 15), 3 => array('processingTime' => 4, 'dueDate' => 9), 4 => array('processingTime' => 3, 'dueDate' => 16), 5 => array('processingTime' => 5, 'dueDate' => 12), 6 => array('processingTime' => 7, 'dueDate' => 20), 7 => array('processingTime' => 5, 'dueDate' => 27), 8 => array('processingTime' => 6, 'dueDate' => 40), 9 => array('processingTime' => 3, 'dueDate' => 10)); // precedence constrainst, i.e job 2 must be completed before job 5 etc $successors = array(2=>5, 7=>9); $n = count($jobs); $optimalSchedule = array(); for ($i = $n; $i >= 1; $i--) { // jobs not required to precede any other job $arr = array(); foreach ($jobs as $k => $v) { if (false === array_key_exists($k, $successors)) { $arr[] = $k; } } // calculate total processing time $totalProcessingTime = 0; foreach ($jobs as $k => $v) { if (true === array_key_exists($k, $arr)) { $totalProcessingTime += $v['processingTime']; } } // find the job that will go to the end of the optimal schedule array $min = null; $x = 0; $lastKey = null; foreach($arr as $k) { $x = $totalProcessingTime - $jobs[$k]['dueDate']; if (null === $min || $x < $min) { $min = $x; $lastKey = $k; } } // add the job to the optimal schedule array $optimalSchedule[$lastKey] = $jobs[$lastKey]; // remove job from the jobs array unset($jobs[$lastKey]); // remove precedence constraint from the successors array if needed if (true === in_array($lastKey, $successors)) { foreach ($successors as $k => $v) { if ($lastKey === $v) { unset($successors[$k]); } } } } // reverse the optimal schedule array and preserve keys $optimalSchedule = array_reverse($optimalSchedule, true); // add tardiness to the array $i = 0; foreach ($optimalSchedule as $k => $v) { $optimalSchedule[$k]['tardiness'] = 0; $j = 0; foreach ($optimalSchedule as $k2 => $v2) { if ($j <= $i) { $optimalSchedule[$k]['tardiness'] += $v2['processingTime']; } $j++; } $i++; } echo '<pre>'; print_r($optimalSchedule); echo '</pre>';

Read the article

Hibernate - strange order of native SQL parameters

- by Xorty

Hello, I am trying to use native MySQL's MD5 crypto func, so I defined custom insert in my mapping file. <hibernate-mapping package="tutorial"> <class name="com.xorty.mailclient.client.domain.User" table="user"> <id name="login" type="string" column="login"></id> <property name="password"> <column name="password" /> </property> <sql-insert>INSERT INTO user (login,password) VALUES ( ?, MD5(?) )</sql-insert> </class> </hibernate-mapping> Then I create User (pretty simple POJO with just 2 Strings - login and password) and try to persist it. session.beginTransaction(); // we have no such user in here yet User junitUser = (User) session.load(User.class, "junit_user"); assert (null == junitUser); // insert new user junitUser = new User(); junitUser.setLogin("junit_user"); junitUser.setPassword("junitpass"); session.save(junitUser); session.getTransaction().commit(); What actually happens? User is created, but with reversed parameters order. He has login "junitpass" and "junit_user" is MD5 encrypted and stored as password. What did I wrong? Thanks EDIT: ADDING POJO class package com.xorty.mailclient.client.domain; import java.io.Serializable; /** * POJO class representing user. * @author MisoV * @version 0.1 */ public class User implements Serializable { /** * Generated UID */ private static final long serialVersionUID = -969127095912324468L; private String login; private String password; /** * @return login */ public String getLogin() { return login; } /** * @return password */ public String getPassword() { return password; } /** * @param login the login to set */ public void setLogin(String login) { this.login = login; } /** * @param password the password to set */ public void setPassword(String password) { this.password = password; } /** * @see java.lang.Object#toString() * @return login */ @Override public String toString() { return login; } /** * Creates new User. * @param login User's login. * @param password User's password. */ public User(String login, String password) { setLogin(login); setPassword(password); } /** * Default constructor */ public User() { } /** * @return hashCode * @see java.lang.Object#hashCode() */ @Override public int hashCode() { final int prime = 31; int result = 1; result = prime * result + ((null == login) ? 0 : login.hashCode()); result = prime * result + ((null == password) ? 0 : password.hashCode()); return result; } /** * @param obj Compared object * @return True, if objects are same. Else false. * @see java.lang.Object#equals(java.lang.Object) */ @Override public boolean equals(Object obj) { if (this == obj) { return true; } if (obj == null) { return false; } if (!(obj instanceof User)) { return false; } User other = (User) obj; if (login == null) { if (other.login != null) { return false; } } else if (!login.equals(other.login)) { return false; } if (password == null) { if (other.password != null) { return false; } } else if (!password.equals(other.password)) { return false; } return true; } }

Read the article

The Changing Face of PASS

- by Bill Graziano

I’m starting my sixth year on the PASS Board. I served two years as the Program Director, two years as the Vice-President of Marketing and I’m starting my second year as the Executive Vice-President of Finance. There’s a pretty good chance that if PASS has done something you don’t like or is doing something you don’t like, that I’m involved in one way or another. Andy Leonard asked in a comment on his blog if the Board had ever reversed itself based on community input. He asserted that it hadn’t. I disagree. I’m not going to try and list all the changes we make inside portfolios based on feedback from and meetings with the community. I’m going to focus on major governance issues since I was elected to the Board. Management Company The first big change was our management company. Our old management company had a standard approach to running a non-profit. It worked well when PASS was launched. Having a ready-made structure and process to run the organization enabled the organization to grow quickly. As time went on we were limited in some of the things we wanted to do. The more involved you were with PASS, the more you saw these limitations. Key volunteers were regularly providing feedback that they wanted certain changes that were difficult for us to accomplish. The Board at that time wanted changes that were difficult or impossible to accomplish under that structure. This was not a simple change. Imagine a $2.5 million dollar company letting all its employees go on a Friday and starting with a new staff on Monday. We also had a very narrow window to accomplish that so that we wouldn’t affect the Summit – our only source of revenue. We spent the year after the change rebuilding processes and putting on the Summit in Denver. That’s a concrete example of a huge change that PASS made to better serve its members. And it was a change that many in the community were telling us we needed to make. Financials We heard regularly from our members that they wanted our financials posted. Today on our web site you can find audited financials going back to 2004. We publish our budget at the start of each year. If you ask a question about the financials on the PASS site I do my best to answer it. I’m also trying to do a better job answering financial questions posted in other locations. (And yes, I know I owe a few of you some blog posts.) That’s another concrete example of a change that our members asked for that the Board agreed was a good decision. Minutes When I started on the Board the meeting minutes were very limited. The minutes from a two day Board meeting might fit on one page. I think we did the bare minimum we were legally required to do. Today Board meeting minutes run from 5 to 12 pages and go into incredible detail on what we talk about. There are certain topics that are NDA but where possible we try to list the topic we discussed but that the actual discussion was under NDA. We also publish the agenda of Board meetings ahead of time. This is another specific example where input from the community influenced the decision. It was certainly easier to have limited minutes but I think the extra effort helps our members understand what’s going on. Board Q&A At the 2009 Summit the Board held its first public Q&A with our members. We’d always been available individually to answer questions. There’s a benefit to getting us all in one room and asking the really hard questions to watch us squirm. We learn what questions we don’t have good answers for. We get to see how many people in the crowd look interested in the various questions and answers. I don’t recall the genesis of how this came about. I’m fairly certain there was some community pressure though. Board Votes Until last November, the Board only reported the vote totals and not how individual Board members voted. That was one of the topics at a great lunch I had with Tim Mitchell and Kendal van Dyke at the Summit. That was also the topic of the first question asked at the Board Q&A by Kendal. Kendal expressed his opposition to to anonymous votes clearly and passionately and without trying to paint anyone into a corner. Less than 24 hours later the PASS Board voted to make individual votes public unless the topic was under NDA. That’s another area where the Board decided to change based on feedback from our members. Summit Location While this isn’t actually a governance issue it is one of the more public decisions we make that has taken some public criticism. There is a significant portion of our members that want the Summit near them. There is a significant portion of our members that like the Summit in Seattle. There is a significant portion of our members that think it should move around the country. I was one that felt strongly that there were significant, tangible benefits to our attendees to being in Seattle every year. I’m also one that has been swayed by some very compelling arguments that we need to have at least one outside Seattle and then revisit the decision. I can’t tell you how the Board will vote but I know the opinion of our members weighs heavily on the decision. Elections And that brings us to the grand-daddy of all governance issues. My thesis for this blog post is that the PASS Board has implemented policy changes in response to member feedback. It isn’t to defend or criticize our election process. It’s just to say that is has been under going continuous change since I’ve been on the Board. I ran for the Board in the fall of 2005. I don’t know much about what happened before then. I was actively volunteering for PASS for four years prior to that as a chapter leader and on the program committee. I don’t recall any complaints about elections but that doesn’t mean they didn’t occur. The questions from the Nominating Committee (NomCom) were trivial and the selection process rudimentary (For example, “Tell us about your accomplishments”). I don’t even remember who I ran against or how many other people ran. I ran for the VP of Marketing in the fall of 2007. I don’t recall any significant changes the Board made in the election process for that election. I think a lot of the changes in 2007 came from us asking the management company to work on the election process. I was expecting a similar set of puff ball questions from my previous election. Boy, was I in for a shock. The NomCom had found a much better set of questions and really made the interview portion difficult. The questions were much more behavioral in nature. I’d already written about my vision for PASS and my goals. They wanted to know how I handled adversity, how I handled criticism, how I handled conflict, how I handled troublesome volunteers, how I motivated people and how I responded to motivation. And many, many other things. They grilled me for over an hour. I’ve done a fair bit of technical sales in my time. I feel I speak well under pressure addressing pointed questions. This interview intentionally put me under pressure. In addition to wanting to know about my interpersonal skills, my work experience, my volunteer experience and my supervisory experience they wanted to see how I’d do under pressure. They wanted to see who would respond under pressure and who wouldn’t. It was a bit of a shock. That was the first big change I remember in the election process. I know there were other improvements around the process but none of them stick in my mind quite like the unexpected hour-long grilling. The next big change I remember was after the 2009 elections. Andy Warren was unhappy with the election process and wanted to make some changes. He worked with Hannes at HQ and they came up with a better set of processes. I think Andy moved PASS in the right direction. Nonetheless, after the 2010 election even more people were very publicly clamoring for changes to our election process. In August of 2010 we had a choice to make. There were numerous bloggers criticizing the Board and our upcoming election. The easy change would be to announce that we were changing the process in a way that would satisfy our critics. I believe that a knee-jerk response to criticism is seldom correct. Instead the Board spent August and September and October and November listening to the community. I visited two SQLSaturdays and asked questions of everyone I could. I attended chapter meetings and asked questions of as many people as they’d let me. At Summit I made it a point to introduce myself to strangers and ask them about the election. At every breakfast I’d sit down at a table full of strangers and ask about the election. I’m happy to say that I left most tables arguing about the election. Most days I managed to get 2 or 3 breakfasts in. I spent less time talking to people that had already written about the election. They were already expressing their opinion. I wanted to talk to people that hadn’t spoken up. I wanted to know what the silent majority thought. The Board all attended the Q&A session where our members expressed their concerns about a variety of issues including the election. The PASS Board also chose to create the Election Review Committee. We wanted people from the community that had been involved with PASS to look at our election process with fresh eyes while listening to what the community had to say and give us some advice on how we could improve the process. I’m a part of this as is Andy Warren. None of the other members are on the Board. I’ve sat in numerous calls and interviews with this group and attended an open meeting at the Summit. We asked anyone that wanted to discuss the election to come speak with us. The ERC held an open meeting at the Summit and invited anyone to attend. There are forums on the ERC web site where we’ve invited people to participate. The ERC has reached to key people involved in recent elections. The years that I haven’t mentioned also saw minor improvements in the election process. Off the top of my head I don’t recall what exact changes were made each year. Specifically since the 2010 election we’ve gone out of our way to seek input from the community about the process. I’m not sure what more we could have done to invite feedback from the community. I think to say that we haven’t “fixed” the election process isn’t a fair criticism at this time. We haven’t rushed any changes through the process. If you don’t see any changes in our election process in July or August then I think it’s fair to criticize us for ignoring the community or ask for an explanation for what we’ve done. In Summary Andy’s main point was that the PASS Board hasn’t changed in response to our members wishes. I think I’ve shown that time and time again the PASS Board has changed in response to what our members want. There are only two outstanding issues: Summit location and elections. The 2013 Summit location hasn’t been decided yet. Our work on the elections is also in progress. And at every step in the election review we’ve gone out of our way to listen to the community and incorporate their feedback on the process. I also hope I’m not encouraging everyone that wants some change in the organization to organize a “blog rush” against the Board. We take public suggestions very seriously but we also take the time to evaluate those suggestions and learn what the rest of our members think and make a measured decision.

Read the article

My computer freezes irregularly

- by Manhim

My computer started to freeze at irregular times for 3 weeks now. Please note that this question change with each things that i try. (For additional details) What happens My computer freezes, the video stops. (No graphic glitches, it just stops) Sound keeps playing up to some time (Usually 10-30 seconds) then stops playing. Sometimes, randomly, the screen on my G-15 keyboard flickers and I see caracters not at the right places. Usually happens for about 1-2 seconds and a bit before my computer freezes. I have to keep the power button pressed for 4 seconds to shut my computer down. I still hear my hard drives and fans working. Sometimes it works with no problems for a full day, some other times it just keeps freezing each time I restart my computer and I have to leave it for the rest of the day. Sometimes my mouse freezes for a fraction of a second (Like 0.01 to 0.2 seconds) quite randomly, usually before it freezes. No errors spotted by the "Action center" unlike when I had problems with my last video card on this system (Driver errors). My G-15 LCD screen also freezes. Sometimes my G-15 LCD screen flickers and caracters gets caried around temporary under heavy load. Now, most of the times, the BIOS hard disks boot order gets reversed for some reason and I have to put it to the right one and save each times I boot. (Might be unrelated, not sure, but it first started yesterday) Sometimes the BIOS doesn't detect my 750GB hard drive plugged in SATA1. What I did so far I have had similar problems in the past and I had changed my hard drive (It was faulty), so I tested my software RAID-0 array and it was faulty so I changed it. (I reinstalled Windows 7 with this part). I also tested with unplugging my secondary hard drive. My CPU was running at about 100 degree Celsius, I removed the dust between the fans and the heatsink and it's now between 45-55. I ran a CPU stress-test and it didn't freeze during the tests (using Prime95 on all cores) Ran a memory test (using memtest86+) for a single pass and there were no errors. Ran a GPU stress test with ati-tools and furmark and it didn't freeze during the tests. (No artefacts either) I had troubles with my graphic card when I got it, but I think that it got fixed with a driver update. I checked the voltages in my BIOS setup and they all seemed ok (±0.2 I think). I have ran on the computer without problems with Fedora 15 on an external hard drive (Appart that it couldn't load Gnome 3 and was reverting to Gnome 2, didn't want to install drivers since I use it on multiple computers) I used it to backup my files from the raid array to my 1TB hard drive for the reinstallation of Windows. (So the crashes only happenned on Windows) [The external hard drive is plugged directly on a SATA port] I contacted EVGA (My graphic card vendor) and pointed them on this question, I'm looking for an answer. Ran sensors on Fedora 15 and got this output: http://pastebin.com/0BHJnAvu Ran 6 short different CPU stress test on Fedora 15 (Haven't found any complete stress testers for Linux) and it didn't crash. Changed the thermal paste to some Artic Silver 5 for my CPU and stress tested the CPU, temperature was at 50 idle, then 64 highest and slowly went down to 62 during the test. Ran some stress testing with a temporary graphic card and it went ok. Ran furmark stress test with my original graphic card and it freezed again. GPU had a temp of 74C, a CPU temp of 58C and a mobo temp of 40C or 45C (Dunno which one it is from SpeedFan). Ran a furmark stress test and a CPU stress test at the same time, results: http://pastebin.com/2t6PLpdJ I have been using my computer without stressing it for about 2 hours now and no crashes yet. I also have disabled the AMD Cool'n'quiet function on the BIOS for a more regular power to the CPU. When I ran Furmark without C'n'q my computer didn't freeze but I had a "Driver Kernel Error" that have recovered (And Furmark crashed) all that while running a CPU stress test. The computer eventually frozed without me being at it, but this time my screen just went on sleep and I couldn't wake it. Using the stability tester in nTune my computer freezed again (In the same manner as before). I notived that Speedfan gives me a -12V of -16.97V and a -5V of -8.78V. I wonder if these numbers are reliable and if they are good or bad. I have swapped my G-15 with another basic USB keyboard (HP) and I have ran furmark for about 10 minutes with a CPU stability test running each 60 seconds for 30 seconds and my computer haven't crashed yet. Ran some more extended tests without my G-15 and it freezed like it usually do. Removed the nForce Hard disk controler. Disabled command queuing in the NVIDIA nForce SATA Controller for both port 0 and port 1 (Errors from the logs) Used CPUID HwMonitor, here are the voltages: http://pastebin.com/dfM7p4jV Changed some configurations in the motherboard BIOS: Disabled PEG Link Mode, Changed AI Tuning to Standard, Disabled the 1394 Controller, Disabled HD Audio, Disabled JMicron RAID controller and Disabled SATA Raid. When it happens When I play video games (Mostly) When I play flash games (Second most) When I'm looking at my desktop background (It rarely happens when I have a window open, but it does, sometimes) When my Graphic card and my CPU are stressed. Sometimes when my Graphic card is stressed. Never happenned while stressing only the CPU. Sometimes when my CPU is stressed. Specs Windows Seven x64 Home Premium Motherboard: M2N-SLI Deluxe CPU: AMD Phenom 9950 x2 @ 2.6GHz Memory: Kingston 4x2GB Dual Channel (Pretty basic memory sticks) Hard drives: Was 2x250GB (Western digital caviar) in raid-0 + 1TB (WD caviar black), I replaced the raid array with a 750GB (WD caviar black) [Yes I removed the array from the raid configurations] 750W Power supply No overcloking. Ever. There have been some power-downs like 4-5 weeks ago, but the problem didn't start immediately after. (I wasn't home, so my computer got shut-down) Event logs (Warnings, errors and critical errors) for the last 24 hours: http://pastebin.com/Bvvk31T7 My current to-try list Reinstall the drivers and software 1 by 1 and do extensive stress testing between each. Update the BIOS firmware to the most recent stable one. Change my motherboard. Status updates Keeping only the last 3 (28/06 04pm) More stress testing and still pass the tests. (28/06 03pm) Been stress testing for 10 minute straight now and 5 minutes with both CPU and GPU being stressed at the same time. (28/06 03pm) Stress-testing right now, so far no problems. A little hope Tests with Furmark and Prime95. Testing Windows bare-bone: 30 Minutes stress, no freeze. Installing an Anti-virus and some software, restarting computer. Testing with Anti-virus and some software (No drivers installed): 30 Minutes stress, no freeze. Installing audio drivers, restarting computer. Testing with the audio drivers: 30 Minutes stress, no freeze. Installing the latest graphic drivers from EVGA's website (without 3d vision since I don't use it), restarting computer. Testing with the graphic drivers: 30 Minutes stress, no freeze. Configuring Windows to my liking and installing more softwares. In this situation, how can I successfully pin-point the current hardware problem? (If it's a hardware problem) Because I don't really have the budget to just forget and replace everything. I also don't really have hardware to test-replace current hardware.

Read the article

How do I implement a remove by index method for a singly linked list in Java?

- by Lars Flyger

Hi, I'm a student in a programming class, and I need some help with this code I've written. So far I've written an entire linked list class (seen below), yet for some reason the "removeByIndex" method won't work. I can't seem to figure out why, the logic seems sound to me. Is there some problem I don't know about? public class List<T> { //private sub-class Link private class Link { private T value; private Link next; //constructors of Link: public Link (T val) { this.value = val; this.next = null; } public Link (T val, Link next) { this.value = val; this.next = next; } @SuppressWarnings("unused") public T getValue() { return value; } } private static final Exception NoSuchElementException = null; private static final Exception IndexOutOfBoundsException = null; private Link chain = null; //constructors of List: public List() { this.chain = null; } //methods of List: /** * Preconditions: none * Postconditions: returns true if list is empty */ public boolean isEmpty() { return this.chain == null; } /** * Preconditions: none * Postconditions: A new Link is added via add-aux * @param element */ public void add(T element) { this.add_aux(element, this.chain); } /** * Preconditions: none * Postconditions: A new Link is added to the current chain * @param element * @param chain */ private void add_aux(T element, Link chain) { if (chain == null) { //if chain is null set chain to a new Link with a value of //element this.chain = new Link(element); } else if (chain.next != null) { //if chain.next is not null, go to next item in chain and //try //to add element add_aux(element, chain.next); } else { //if chain.next is null, set chain.next equal to a new Link //with value of element. chain.next = new Link(element); } } /** * Preconditions: none * Postconditions: returns the link at the defined index via nthlink_aux * @param index * @return */ private Link nthLink (int index) { return nthLink_aux(index, this.chain); } /** * Preconditions: none * Postconditions: returns the link at the defined index in the specified *chain * @param i * @param c * @return */ private Link nthLink_aux (int i, Link c) { if (i == 0) { return c; } else return nthLink_aux(i-1, c.next); } /** * Preconditions: the specified element is present in the list * Postconditions: the specified element is removed from the list * @param element * @throws Exception */ public void removeElement(T element) throws Exception { if (chain == null) { throw NoSuchElementException; } //while chain's next is not null and the value of chain.next is not //equal to element, //set chain equal to chain.next //use this iteration to go through the linked list. else while ((chain.next != null) && !(chain.next.value.equals(element))){ Link testlink = chain.next; if (testlink.next.value.equals(element)) { //if chain.next is equal to element, bypass the //element. chain.next.next = chain.next.next.next; } else if (testlink.next == null) { throw NoSuchElementException; } } } /** * Preconditions: none * Postsconditions: the Link at the specified index is removed * @param index * @throws Exception */ public void removeByIndex(int index) throws Exception { if (index == 0) { //if index is 0, set chain equal to chain.next chain = chain.next; } else if (index > 0) { Link target = nthLink(index); while (target != null) { if (target.next != null) { target = target.next; } //if target.next is null, set target to null else { target = null; } } return; } else throw IndexOutOfBoundsException; } /** * Preconditions: none * Postconditions: the specified link's value is printed * @param link */ public void printLink (Link link) { if(link != null) { System.out.println(link.value.toString()); } } /** * Preconditions: none * Postconditions: all of the links' values in the list are printed. */ public void print() { //copy chain to a new variable Link head = this.chain; //while head is not null while (!(head == null)) { //print the current link this.printLink(head); //set head equal to the next link head = head.next; } } /** * Preconditions: none * Postconditions: The chain is set to null */ public void clear() { this.chain = null; } /** * Preconditions: none * Postconditions: Places the defined link at the defined index of the list * @param index * @param val */ public void splice(int index, T val) { //create a new link with value equal to val Link spliced = new Link(val); if (index <= 0) { //copy chain Link copy = chain; //set chain equal to spliced chain = spliced; //set chain.next equal to copy chain.next = copy; } else if (index > 0) { //create a target link equal to the link before the index Link target = nthLink(index - 1); //set the target's next equal to a new link with a next //equal to the target's old next target.next = new Link(val, target.next); } } /** * Preconditions: none * Postconditions: Check to see if element is in the list, returns true * if it is and false if it isn't * @param element * @return */ public boolean Search(T element) { if (chain == null) { //return false if chain is null return false; } //while chain's next is not null and the value of chain.next is not //equal to element, //set chain equal to chain.next //use this iteration to go through the linked list. else while ((chain.next != null) && !(chain.next.value.equals(element))) { Link testlink = chain.next; if (testlink.next.value.equals(element)) { //if chain.next is equal to element, return true return true; } else if (testlink.next == null) { return false; } } return false; } /** * Preconditions: none * Postconditions: order of the links in the list is reversed. */ public void reverse() { //copy chain Link current = chain; //set chain equal to null chain = null; while (current != null) { Link save = current; current = current.next; save.next = chain; chain = save; } } }'

Read the article

Hibernate exception

- by Mark

Hi all, im new to hibernate! i have followed the netbeans tutorial on creating a hibernate enabled application. after sucessfully creating a database in mysql workbench i reversed engineered the pojos etc and then tried to run a simple query(from Course) and got the following org.hibernate.MappingException: An association from the table coursemodule refers to an unmapped class: DAL.Module at org.hibernate.cfg.Configuration.secondPassCompileForeignKeys(Configuration.java:1252) at org.hibernate.cfg.Configuration.secondPassCompile(Configuration.java:1170) at org.hibernate.cfg.AnnotationConfiguration.secondPassCompile(AnnotationConfiguration.java:324) at org.hibernate.cfg.Configuration.buildSessionFactory(Configuration.java:1286) at org.hibernate.cfg.AnnotationConfiguration.buildSessionFactory(AnnotationConfiguration.java:859) heres the generated class for Course package DAL; // Generated 02-May-2010 16:41:16 by Hibernate Tools 3.2.1.GA import java.util.HashSet; import java.util.Set; /** * Course generated by hbm2java */ public class Course implements java.io.Serializable { private int id; private String name; private Set<Module> modules = new HashSet<Module>(0); public Course() { } public Course(int id, String name) { this.id = id; this.name = name; } public Course(int id, String name, Set<Module> modules) { this.id = id; this.name = name; this.modules = modules; } public int getId() { return this.id; } public void setId(int id) { this.id = id; } public String getName() { return this.name; } public void setName(String name) { this.name = name; } public Set<Module> getModules() { return this.modules; } public void setModules(Set<Module> modules) { this.modules = modules; } } and its config file course.hbm.xml <?xml version="1.0"?> <!DOCTYPE hibernate-mapping PUBLIC "-//Hibernate/Hibernate Mapping DTD 3.0//EN" "http://hibernate.sourceforge.net/hibernate-mapping-3.0.dtd">  <hibernate-mapping> <class name="DAL.Course" table="course" catalog="walkthrough"> <id name="id" type="int"> <column name="id" /> <generator class="assigned" /> </id> <property name="name" type="string"> <column name="name" not-null="true" /> </property> <set name="modules" inverse="false" table="coursemodule"> <key> <column name="courseId" not-null="true" unique="true" /> </key> <many-to-many entity-name="DAL.Module"> <column name="moduleId" not-null="true" unique="true" /> </many-to-many> </set> </class> </hibernate-mapping> hibernate.reveng.xml <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE hibernate-reverse-engineering PUBLIC "-//Hibernate/Hibernate Reverse Engineering DTD 3.0//EN" "http://hibernate.sourceforge.net/hibernate-reverse-engineering-3.0.dtd"> <hibernate-reverse-engineering> <schema-selection match-catalog="Walkthrough"/> <table-filter match-name="walkthrough"/> <table-filter match-name="course"/> <table-filter match-name="module"/> <table-filter match-name="studentmodule"/> <table-filter match-name="attendee"/> <table-filter match-name="student"/> <table-filter match-name="coursemodule"/> <table-filter match-name="session"/> <table-filter match-name="test"/> </hibernate-reverse-engineering> hibernate.cfg.xml <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE hibernate-configuration PUBLIC "-//Hibernate/Hibernate Configuration DTD 3.0//EN" "http://hibernate.sourceforge.net/hibernate-configuration-3.0.dtd"> <hibernate-configuration> <session-factory> <property name="hibernate.dialect">org.hibernate.dialect.MySQLDialect</property> <property name="hibernate.connection.driver_class">com.mysql.jdbc.Driver</property> <property name="hibernate.connection.url">jdbc:mysql://localhost:3306/Walkthrough</property> <property name="hibernate.connection.username">root</property> <property name="hibernate.connection.password">password</property> <property name="hibernate.show_sql">true</property> <property name="hibernate.current_session_context_class">thread</property> <mapping resource="DAL/Student.hbm.xml"/> <mapping resource="DAL/Walkthrough.hbm.xml"/> <mapping resource="DAL/Test.hbm.xml"/> <mapping resource="DAL/Module.hbm.xml"/> <mapping resource="DAL/Session.hbm.xml"/> <mapping resource="DAL/Course.hbm.xml"/> </session-factory> </hibernate-configuration> any ideas on why im getting this exception? ps. test is just a table with an id in it and is not related to anything. running "from Test" works

Read the article

Fibre channel long distance woes

- by Marki

I need a fresh pair of eyes. We're using a 15km fibre optic line across which fibrechannel and 10GbE is multiplexed (passive optical CWDM). For FC we have long distance lasers suitable up to 40km (Skylane SFCxx0404F0D). The multiplexer is limited by the SFPs which can do max. 4Gb fibrechannel. The FC switch is a Brocade 5000 series. The respective wavelengths are 1550,1570,1590 and 1610nm for FC and 1530nm for 10GbE. The problem is the 4GbFC fabrics are almost never clean. Sometimes they are for a while even with a lot of traffic on them. Then they may suddenly start producing errors (RX CRC, RX encoding, RX disparity, ...) even with only marginal traffic on them. I am attaching some error and traffic graphs. Errors are currently in the order of 50-100 errors per 5 minutes when with 1Gb/s traffic. Optics Here is the power output of one port summarized (collected using sfpshow on different switches) SITE-A units=uW (microwatt) SITE-B ********************************************** FAB1 SW1 TX 1234.3 RX 49.1 SW3 1550nm (ko) RX 95.2 TX 1175.6 FAB2 SW2 TX 1422.0 RX 104.6 SW4 1610nm (ok) RX 54.3 TX 1468.4 What I find curious at this point is the asymmetry in the power levels. While SW2 transmits with 1422uW which SW4 receives with 104uW, SW2 only receives the SW4 signal with similar original power only with 54uW. Vice versa for SW1-3. Anyway the SFPs have RX sensitivity down to -18dBm (ca. 20uW) so in any case it should be fine... But nothing is. Some SFPs have been diagnosed as malfunctioning by the manufacturer (the 1550nm ones shown above with "ko"). The 1610nm ones apparently are ok, they have been tested using a traffic generator. The leased line has also been tested more than once. All is within tolerances. I'm awaiting the replacements but for some reason I don't believe it will make things better as the apparently good ones don't produce ZERO errors either. Earlier there was active equipment involved (some kind of 4GFC retimer) before putting the signal on the line. No idea why. That equipment was eliminated because of the problems so we now only have: the long distance laser in the switch, (new) 10m LC-SC monomode cable to the mux (for each fabric), the leased line, the same thing but reversed on the other side of the link. FC switches Here is a port config from the Brocade portcfgshow (it's like that on both sides, obviously) Area Number: 0 Speed Level: 4G Fill Word(On Active) 0(Idle-Idle) Fill Word(Current) 0(Idle-Idle) AL_PA Offset 13: OFF Trunk Port ON Long Distance LS VC Link Init OFF Desired Distance 32 Km Reserved Buffers 70 Locked L_Port OFF Locked G_Port OFF Disabled E_Port OFF Locked E_Port OFF ISL R_RDY Mode OFF RSCN Suppressed OFF Persistent Disable OFF LOS TOV enable OFF NPIV capability ON QOS E_Port OFF Port Auto Disable: OFF Rate Limit OFF EX Port OFF Mirror Port OFF Credit Recovery ON F_Port Buffers OFF Fault Delay: 0(R_A_TOV) NPIV PP Limit: 126 CSCTL mode: OFF Forcing the links to 2GbFC produces no errors, but we bought 4GbFC and we want 4GbFC. I don't know where to look anymore. Any ideas what to try next or how to proceed? If we can't make 4GbFC work reliably I wonder what the people working with 8 or 16 do... I don't assume that "a few errors here and there" are acceptable. Oh and BTW we are in contact with everyone of the manufacturers (FC switch, MUX, SFPs, ...) Except for the SFPs to be changed (some have been changed before) nobody has a clue. Brocade SAN Health says the fabric is ok. MUX, well, it's passive, it's only a prism, nature at it's best. Any shots in the dark? APPENDIX: Answers to your questions @Chopper3: This is the second generation of Brocades exhibiting the problem. Before we had 5000s, now we have 5100s. In the beginning when we still had the active MUX we rented a longdistance laser once to put it into the switch directly in order to make tests for a day, during that day of course it was clean. But as I said, sometimes it's clean just like that. And sometimes it's not. Alternative switches would mean to rebuild the entire SAN with those only to test. Alternative SFPs, well they're hard to come by just like that. @longneck: The line is rented. It's a dark fibre (9um monomode) so there's noone else on it. Sure there are splices. I can't go and look but I have to trust they have been done correctly. As I said the line has been checked and rechecked (using an optical time-domain reflectometer). Obviously you don't have all this equipment yourself because it's way too expensive. @mdpc: What would be the "wrong" type of cable according to you? Up to the switch everything is monomode, yes. The connectors are the correct ones too. Yeah I know there are the green ones where the fibre is cut off at a certain angle etc. But we have the correct ones for all that I know. Progress Report #1 We have had two fabrics (=2x2 switches) with Brocade 5100s with FabricOS 6.4.1 and two fabrics (another 2x4 switches) on FabricOS 7.0.2. On the longdistance ISLs (one in each fabric) it turned out that with FOS 6.4.1 setting it to long distance issues warnings about the VC Init setting and consequently the fill word. But those are only warnings. FOS 7.0.2 requires you to do modifications to VCI and the fillword for long distance links. Setting FOS 6.4.1 to the LS (long-distance static distance) setting with wrong VCI and fillword setting made the whole fabric inoperational (stuck in an SCN loop, use fabriclog -s to see, you don't see it anywhere else, no port error counters or anything increasing). Currently I'm giving the one fabric with the IMHO more correct settings a beating and it seems to do fine, whereas the other one without much traffic still has errors here and there. In short: We have eliminated the active part of the MUX (the FC retimer). We are putting the long distance SFPs into the end equipment themselves. Just to be sure we bought new monomode cables to connect the end equipment to the remaining passive part of the MUX. We are now trying out several long distance configs. It's almost black magic. Everything that happens is mostly empirical, noone seems to have a clue what are the exact reasons to do something. ("We have tried this, and it didn't work, then we tried that and it worked, so we stuck with that." But noone really seems to know why.) I'll keep you updated. Progress Report #2 We got the new lasers for one of the fabrics on warranty. It's ultra clean even on 4GbFC. They're transmitting with roughly 2mW (3dBm) whereas the others are only at 1.5mW (1.5dBm) although that should really be enough. The other fabric (where the lasers are apparently ok) still produces one or two CRCs infrequently. Using sfpshow the SFP producing the actual RX errors shows Status/Ctrl: 0x82 Alarm flags[0,1] = 0x5, 0x40 Warn Flags[0,1] = 0x5, 0x40 Now I'll have to find out what that means. Not sure if it was there before. Well I'll first clear my head with a week of vacation. 8-)

Read the article

PHP 'Years' array

- by J M 4

I am trying to create an array for years which i will use in the DOB year piece of a form I am building. Currently, I know there are two ways to handle the issue but I don't really care for either: 1) Range: I know I can create a year array using the following <?php $year = range(1910,date("Y")); $_SESSION['years_arr'] = $year; ?> the problem with Point 1 is two fold: a) my function call shows the first year as 'selected' instead of "Year" as I have as option="0", and b) I want the years reversed so 2010 is the first in the least and shown decreasing. My function call is: PHP <?php function showOptionsDrop($array, $active, $echo=true){ $string = ''; foreach($array as $k => $v){ $s = ($active == $k)? ' selected="selected"' : ''; $string .= '<option value="'.$k.'"'.$s.'>'.$v.'</option>'."\n"; } if($echo) echo $string; else return $string; } ?> HTML <table> <tr> <td>State:</td> <td><select name="F1State"><option value="0">Choose a year</option><?php showOptionsDrop($_SESSION['years_arr'], null, true); ?></select> </td> </tr> </table> 2) Long Array I know i can physically create an array with years listed out but this takes up a lot of space and time if I ever want to go back and modify. ex: PHP $years = array('1900'=>"1900", '1901'=>"1901", '1902'=>"1902", '1903'=>"1903", '1904'=>"1904", '1905'=>"1905", '1906'=>"1906", '1907'=>"1907", '1908'=>"1908", '1909'=>"1909", '1910'=>"1910", '1911'=>"1911", '1912'=>"1912", '1913'=>"1913", '1914'=>"1914", '1915'=>"1915", '1916'=>"1916", '1917'=>"1917", '1918'=>"1918", '1919'=>"1919", '1920'=>"1920", '1921'=>"1921", '1922'=>"1922", '1923'=>"1923", '1924'=>"1924", '1925'=>"1925", '1926'=>"1926", '1927'=>"1927", '1928'=>"1928", '1929'=>"1929", '1930'=>"1930", '1931'=>"1931", '1932'=>"1932", '1933'=>"1933", '1934'=>"1934", '1935'=>"1935", '1936'=>"1936", '1937'=>"1937", '1938'=>"1938", '1939'=>"1939", '1940'=>"1940", '1941'=>"1941", '1942'=>"1942", '1943'=>"1943", '1944'=>"1944", '1945'=>"1945", '1946'=>"1946", '1947'=>"1947", '1948'=>"1948", '1949'=>"1949", '1950'=>"1950", '1951'=>"1951", '1952'=>"1952", '1953'=>"1953", '1954'=>"1954", '1955'=>"1955", '1956'=>"1956", '1957'=>"1957", '1958'=>"1958", '1959'=>"1959", '1960'=>"1960", '1961'=>"1961", '1962'=>"1962", '1963'=>"1963", '1964'=>"1964", '1965'=>"1965", '1966'=>"1966", '1967'=>"1967", '1968'=>"1968", '1969'=>"1969", '1970'=>"1970", '1971'=>"1971", '1972'=>"1972", '1973'=>"1973", '1974'=>"1974", '1975'=>"1975", '1976'=>"1976", '1977'=>"1977", '1978'=>"1978", '1979'=>"1979", '1980'=>"1980", '1981'=>"1981", '1982'=>"1982", '1983'=>"1983", '1984'=>"1984", '1985'=>"1985", '1986'=>"1986", '1987'=>"1987", '1988'=>"1988", '1989'=>"1989", '1990'=>"1990", '1991'=>"1991", '1992'=>"1992", '1993'=>"1993", '1994'=>"1994", '1995'=>"1995", '1996'=>"1996", '1997'=>"1997", '1998'=>"1998", '1999'=>"1999", '2000'=>"2000", '2001'=>"2001", '2002'=>"2002", '2003'=>"2003", '2004'=>"2004", '2005'=>"2005", '2006'=>"2006", '2007'=>"2007", '2008'=>"2008", '2009'=>"2009", '2010'=>"2010"); $_SESSION['years_arr'] = $years_arr; Does anybody have a recommended idea how to work - or just how to simply modify my existing code? Thank you!

Read the article

Incorrect output on changing sequence of declarations

- by max

Writing C++ code to implement Sutherland-Hodgeman polygon clipping. This order of declaration of these 2 statements gives correct output, reverse does not. int numberOfVertices = 5; Point pointList[] = { {50,50}, {200,300}, {310,110}, {130,90}, {70,40} }; I am passing the polygon vertex set to clippers in order - LEFT, RIGHT, TOP, BOTTOM. The exact error which comes when the declarations are reversed is that the bottom clipper, produces an empty set of vertices so no polygon is displayed after clipping. Correct: Incorrent: Confirmed by outputting the number of vertices produced after each pass: Correct: Incorrect: What is the reason for this error? Code: #include <iostream> #include <GL/glut.h> #define MAXVERTICES 10 #define LEFT 0 #define RIGHT 1 #define TOP 2 #define BOTTOM 3 using namespace std; /* Clipping window */ struct Window { double xmin; double xmax; double ymin; double ymax; }; struct Point { double x; double y; }; /* If I interchange these two lines, the code doesn't work. */ /**************/ int numberOfVertices = 5; Point pointList[] = { {50,50}, {200,300}, {310,110}, {130,90}, {70,40} }; /**************/ const Window w = { 100, 400, 60, 200 }; /* Checks whether a point is inside or outside a window side */ int isInside(Point p, int side) { switch(side) { case LEFT: return p.x >= w.xmin; case RIGHT: return p.x <= w.xmax; case TOP: return p.y <= w.ymax; case BOTTOM: return p.y >= w.ymin; } } /* Calculates intersection of a segment and a window side */ Point intersection(Point p1, Point p2, int side) { Point temp; double slope, intercept; bool infinite; /* Find slope and intercept of segment, taking care of inf slope */ if(p2.x - p1.x != 0) { slope = (p2.y - p1.y) / (p2.x - p1.x); infinite = false; } else { infinite = true; } intercept = p1.y - p1.x * slope; /* Calculate intersections */ switch(side) { case LEFT: temp.x = w.xmin; temp.y = temp.x * slope + intercept; break; case RIGHT: temp.x = w.xmax; temp.y = temp.x * slope + intercept; break; case TOP: temp.y = w.ymax; temp.x = infinite ? p1.x : (temp.y - intercept) / slope; break; case BOTTOM: temp.y = w.ymin; temp.x = infinite ? p1.x : (temp.y - intercept) / slope; break; } return temp; } /* Clips polygon against a side, updating the point list (called once for each side) */ void clipAgainstSide(int sideToClip) { int i, j=0; Point s,p; Point outputList[MAXVERTICES]; /* Main algorithm */ s = pointList[numberOfVertices-1]; for(i=0 ; i<numberOfVertices ; i++) { p = pointList[i]; if(isInside(p, sideToClip)) { /* p inside */ if(!isInside(s, sideToClip)) { /* p inside, s outside */ outputList[j] = intersection(p, s, sideToClip); j++; } outputList[j] = p; j++; } else if(isInside(s, sideToClip)) { /* s inside, p outside */ outputList[j] = intersection(s, p, sideToClip); j++; } s = p; } /* Updating number of points and point list */ numberOfVertices = j; /* ERROR: In last call with BOTTOM argument, numberOfVertices becomes 0 */ /* all earlier 3 calls have correct output */ cout<<numberOfVertices<<endl; for(i=0 ; i<numberOfVertices ; i++) { pointList[i] = outputList[i]; } } void SutherlandHodgemanPolygonClip() { clipAgainstSide(LEFT); clipAgainstSide(RIGHT); clipAgainstSide(TOP); clipAgainstSide(BOTTOM); } void init() { glClearColor(1,1,1,0); glMatrixMode(GL_PROJECTION); gluOrtho2D(0,1000,0,500); } void display() { glClear(GL_COLOR_BUFFER_BIT); /* Displaying ORIGINAL box and polygon */ glColor3f(0,0,1); glBegin(GL_LINE_LOOP); glVertex2i(w.xmin, w.ymin); glVertex2i(w.xmin, w.ymax); glVertex2i(w.xmax, w.ymax); glVertex2i(w.xmax, w.ymin); glEnd(); glColor3f(1,0,0); glBegin(GL_LINE_LOOP); for(int i=0 ; i<numberOfVertices ; i++) { glVertex2i(pointList[i].x, pointList[i].y); } glEnd(); /* Clipping */ SutherlandHodgemanPolygonClip(); /* Displaying CLIPPED box and polygon, 500px right */ glColor3f(0,0,1); glBegin(GL_LINE_LOOP); glVertex2i(w.xmin+500, w.ymin); glVertex2i(w.xmin+500, w.ymax); glVertex2i(w.xmax+500, w.ymax); glVertex2i(w.xmax+500, w.ymin); glEnd(); glColor3f(1,0,0); glBegin(GL_LINE_LOOP); for(int i=0 ; i<numberOfVertices ; i++) { glVertex2i(pointList[i].x+500, pointList[i].y); } glEnd(); glFlush(); } int main(int argc, char** argv) { glutInit(&argc, argv); glutInitDisplayMode(GLUT_SINGLE | GLUT_RGB); glutInitWindowSize(1000,500); glutCreateWindow("Sutherland-Hodgeman polygon clipping"); init(); glutDisplayFunc(display); glutMainLoop(); return 0; }

Read the article

Conceal packet loss in PCM stream

- by ZeroDefect

I am looking to use 'Packet Loss Concealment' to conceal lost PCM frames in an audio stream. Unfortunately, I cannot find a library that is accessible without all the licensing restrictions and code bloat (...up for some suggestions though). I have located some GPL code written by Steve Underwood for the Asterisk project which implements PLC. There are several limitations; although, as Steve suggests in his code, his algorithm can be applied to different streams with a bit of work. Currently, the code works with 8kHz 16-bit signed mono streams. Variations of the code can be found through a simple search of Google Code Search. My hope is that I can adapt the code to work with other streams. Initially, the goal is to adjust the algorithm for 8+ kHz, 16-bit signed, multichannel audio (all in a C++ environment). Eventually, I'm looking to make the code available under the GPL license in hopes that it could be of benefit to others... Attached is the code below with my efforts. The code includes a main function that will "drop" a number of frames with a given probability. Unfortunately, the code does not quite work as expected. I'm receiving EXC_BAD_ACCESS when running in gdb, but I don't get a trace from gdb when using 'bt' command. Clearly, I'm trampimg on memory some where but not sure exactly where. When I comment out the *amdf_pitch* function, the code runs without crashing... int main (int argc, char *argv[]) { std::ifstream fin("C:\\cc32kHz.pcm"); if(!fin.is_open()) { std::cout << "Failed to open input file" << std::endl; return 1; } std::ofstream fout_repaired("C:\\cc32kHz_repaired.pcm"); if(!fout_repaired.is_open()) { std::cout << "Failed to open output repaired file" << std::endl; return 1; } std::ofstream fout_lossy("C:\\cc32kHz_lossy.pcm"); if(!fout_lossy.is_open()) { std::cout << "Failed to open output repaired file" << std::endl; return 1; } audio::PcmConcealer Concealer; Concealer.Init(1, 16, 32000); //Generate random numbers; srand( time(NULL) ); int value = 0; int probability = 5; while(!fin.eof()) { char arr[2]; fin.read(arr, 2); //Generate's random number; value = rand() % 100 + 1; if(value <= probability) { char blank[2] = {0x00, 0x00}; fout_lossy.write(blank, 2); //Fill in data; Concealer.Fill((int16_t *)blank, 1); fout_repaired.write(blank, 2); } else { //Write data to file; fout_repaired.write(arr, 2); fout_lossy.write(arr, 2); Concealer.Receive((int16_t *)arr, 1); } } fin.close(); fout_repaired.close(); fout_lossy.close(); return 0; } PcmConcealer.hpp /* * Code adapted from Steve Underwood of the Asterisk Project. This code inherits * the same licensing restrictions as the Asterisk Project. */ #ifndef __PCMCONCEALER_HPP__ #define __PCMCONCEALER_HPP__ /** 1. What does it do? The packet loss concealment module provides a suitable synthetic fill-in signal, to minimise the audible effect of lost packets in VoIP applications. It is not tied to any particular codec, and could be used with almost any codec which does not specify its own procedure for packet loss concealment. Where a codec specific concealment procedure exists, the algorithm is usually built around knowledge of the characteristics of the particular codec. It will, therefore, generally give better results for that particular codec than this generic concealer will. 2. How does it work? While good packets are being received, the plc_rx() routine keeps a record of the trailing section of the known speech signal. If a packet is missed, plc_fillin() is called to produce a synthetic replacement for the real speech signal. The average mean difference function (AMDF) is applied to the last known good signal, to determine its effective pitch. Based on this, the last pitch period of signal is saved. Essentially, this cycle of speech will be repeated over and over until the real speech resumes. However, several refinements are needed to obtain smooth pleasant sounding results. - The two ends of the stored cycle of speech will not always fit together smoothly. This can cause roughness, or even clicks, at the joins between cycles. To soften this, the 1/4 pitch period of real speech preceeding the cycle to be repeated is blended with the last 1/4 pitch period of the cycle to be repeated, using an overlap-add (OLA) technique (i.e. in total, the last 5/4 pitch periods of real speech are used). - The start of the synthetic speech will not always fit together smoothly with the tail of real speech passed on before the erasure was identified. Ideally, we would like to modify the last 1/4 pitch period of the real speech, to blend it into the synthetic speech. However, it is too late for that. We could have delayed the real speech a little, but that would require more buffer manipulation, and hurt the efficiency of the no-lost-packets case (which we hope is the dominant case). Instead we use a degenerate form of OLA to modify the start of the synthetic data. The last 1/4 pitch period of real speech is time reversed, and OLA is used to blend it with the first 1/4 pitch period of synthetic speech. The result seems quite acceptable. - As we progress into the erasure, the chances of the synthetic signal being anything like correct steadily fall. Therefore, the volume of the synthesized signal is made to decay linearly, such that after 50ms of missing audio it is reduced to silence. - When real speech resumes, an extra 1/4 pitch period of sythetic speech is blended with the start of the real speech. If the erasure is small, this smoothes the transition. If the erasure is long, and the synthetic signal has faded to zero, the blending softens the start up of the real signal, avoiding a kind of "click" or "pop" effect that might occur with a sudden onset. 3. How do I use it? Before audio is processed, call plc_init() to create an instance of the packet loss concealer. For each received audio packet that is acceptable (i.e. not including those being dropped for being too late) call plc_rx() to record the content of the packet. Note this may modify the packet a little after a period of packet loss, to blend real synthetic data smoothly. When a real packet is not available in time, call plc_fillin() to create a sythetic substitute. That's it! */ /*! Minimum allowed pitch (66 Hz) */ #define PLC_PITCH_MIN(SAMPLE_RATE) ((double)(SAMPLE_RATE) / 66.6) /*! Maximum allowed pitch (200 Hz) */ #define PLC_PITCH_MAX(SAMPLE_RATE) ((SAMPLE_RATE) / 200) /*! Maximum pitch OLA window */ //#define PLC_PITCH_OVERLAP_MAX(SAMPLE_RATE) ((PLC_PITCH_MIN(SAMPLE_RATE)) >> 2) /*! The length over which the AMDF function looks for similarity (20 ms) */ #define CORRELATION_SPAN(SAMPLE_RATE) ((20 * (SAMPLE_RATE)) / 1000) /*! History buffer length. The buffer must also be at leat 1.25 times PLC_PITCH_MIN, but that is much smaller than the buffer needs to be for the pitch assessment. */ //#define PLC_HISTORY_LEN(SAMPLE_RATE) ((CORRELATION_SPAN(SAMPLE_RATE)) + (PLC_PITCH_MIN(SAMPLE_RATE))) namespace audio { typedef struct { /*! Consecutive erased samples */ int missing_samples; /*! Current offset into pitch period */ int pitch_offset; /*! Pitch estimate */ int pitch; /*! Buffer for a cycle of speech */ float *pitchbuf;//[PLC_PITCH_MIN]; /*! History buffer */ short *history;//[PLC_HISTORY_LEN]; /*! Current pointer into the history buffer */ int buf_ptr; } plc_state_t; class PcmConcealer { public: PcmConcealer(); ~PcmConcealer(); void Init(int channels, int bit_depth, int sample_rate); //Process a block of received audio samples. int Receive(short amp[], int frames); //Fill-in a block of missing audio samples. int Fill(short amp[], int frames); void Destroy(); private: int amdf_pitch(int min_pitch, int max_pitch, short amp[], int channel_index, int frames); void save_history(plc_state_t *s, short *buf, int channel_index, int frames); void normalise_history(plc_state_t *s); /** Holds the states of each of the channels **/ std::vector< plc_state_t * > ChannelStates; int plc_pitch_min; int plc_pitch_max; int plc_pitch_overlap_max; int correlation_span; int plc_history_len; int channel_count; int sample_rate; bool Initialized; }; } #endif PcmConcealer.cpp /* * Code adapted from Steve Underwood of the Asterisk Project. This code inherits * the same licensing restrictions as the Asterisk Project. */ #include "audio/PcmConcealer.hpp" /* We do a straight line fade to zero volume in 50ms when we are filling in for missing data. */ #define ATTENUATION_INCREMENT 0.0025 /* Attenuation per sample */ #if !defined(INT16_MAX) #define INT16_MAX (32767) #define INT16_MIN (-32767-1) #endif #ifdef WIN32 inline double rint(double x) { return floor(x + 0.5); } #endif inline short fsaturate(double damp) { if (damp > 32767.0) return INT16_MAX; if (damp < -32768.0) return INT16_MIN; return (short)rint(damp); } namespace audio { PcmConcealer::PcmConcealer() : Initialized(false) { } PcmConcealer::~PcmConcealer() { Destroy(); } void PcmConcealer::Init(int channels, int bit_depth, int sample_rate) { if(Initialized) return; if(channels <= 0 || bit_depth != 16) return; Initialized = true; channel_count = channels; this->sample_rate = sample_rate; ////////////// double min = PLC_PITCH_MIN(sample_rate); int imin = (int)min; double max = PLC_PITCH_MAX(sample_rate); int imax = (int)max; plc_pitch_min = imin; plc_pitch_max = imax; plc_pitch_overlap_max = (plc_pitch_min >> 2); correlation_span = CORRELATION_SPAN(sample_rate); plc_history_len = correlation_span + plc_pitch_min; ////////////// for(int i = 0; i < channel_count; i ++) { plc_state_t *t = new plc_state_t; memset(t, 0, sizeof(plc_state_t)); t->pitchbuf = new float[plc_pitch_min]; t->history = new short[plc_history_len]; ChannelStates.push_back(t); } } void PcmConcealer::Destroy() { if(!Initialized) return; while(ChannelStates.size()) { plc_state_t *s = ChannelStates.at(0); if(s) { if(s->history) delete s->history; if(s->pitchbuf) delete s->pitchbuf; memset(s, 0, sizeof(plc_state_t)); delete s; } ChannelStates.erase(ChannelStates.begin()); } ChannelStates.clear(); Initialized = false; } //Process a block of received audio samples. int PcmConcealer::Receive(short amp[], int frames) { if(!Initialized) return 0; int j = 0; for(int k = 0; k < ChannelStates.size(); k++) { int i; int overlap_len; int pitch_overlap; float old_step; float new_step; float old_weight; float new_weight; float gain; plc_state_t *s = ChannelStates.at(k); if (s->missing_samples) { /* Although we have a real signal, we need to smooth it to fit well with the synthetic signal we used for the previous block */ /* The start of the real data is overlapped with the next 1/4 cycle of the synthetic data. */ pitch_overlap = s->pitch >> 2; if (pitch_overlap > frames) pitch_overlap = frames; gain = 1.0 - s->missing_samples * ATTENUATION_INCREMENT; if (gain < 0.0) gain = 0.0; new_step = 1.0/pitch_overlap; old_step = new_step*gain; new_weight = new_step; old_weight = (1.0 - new_step)*gain; for (i = 0; i < pitch_overlap; i++) { int index = (i * channel_count) + j; amp[index] = fsaturate(old_weight * s->pitchbuf[s->pitch_offset] + new_weight * amp[index]); if (++s->pitch_offset >= s->pitch) s->pitch_offset = 0; new_weight += new_step; old_weight -= old_step; if (old_weight < 0.0) old_weight = 0.0; } s->missing_samples = 0; } save_history(s, amp, j, frames); j++; } return frames; } //Fill-in a block of missing audio samples. int PcmConcealer::Fill(short amp[], int frames) { if(!Initialized) return 0; int j =0; for(int k = 0; k < ChannelStates.size(); k++) { short *tmp = new short[plc_pitch_overlap_max]; int i; int pitch_overlap; float old_step; float new_step; float old_weight; float new_weight; float gain; short *orig_amp; int orig_len; orig_amp = amp; orig_len = frames; plc_state_t *s = ChannelStates.at(k); if (s->missing_samples == 0) { // As the gap in real speech starts we need to assess the last known pitch, //and prepare the synthetic data we will use for fill-in normalise_history(s); s->pitch = amdf_pitch(plc_pitch_min, plc_pitch_max, s->history + plc_history_len - correlation_span - plc_pitch_min, j, correlation_span); // We overlap a 1/4 wavelength pitch_overlap = s->pitch >> 2; // Cook up a single cycle of pitch, using a single of the real signal with 1/4 //cycle OLA'ed to make the ends join up nicely // The first 3/4 of the cycle is a simple copy for (i = 0; i < s->pitch - pitch_overlap; i++) s->pitchbuf[i] = s->history[plc_history_len - s->pitch + i]; // The last 1/4 of the cycle is overlapped with the end of the previous cycle new_step = 1.0/pitch_overlap; new_weight = new_step; for ( ; i < s->pitch; i++) { s->pitchbuf[i] = s->history[plc_history_len - s->pitch + i]*(1.0 - new_weight) + s->history[plc_history_len - 2*s->pitch + i]*new_weight; new_weight += new_step; } // We should now be ready to fill in the gap with repeated, decaying cycles // of what is in pitchbuf // We need to OLA the first 1/4 wavelength of the synthetic data, to smooth // it into the previous real data. To avoid the need to introduce a delay // in the stream, reverse the last 1/4 wavelength, and OLA with that. gain = 1.0; new_step = 1.0/pitch_overlap; old_step = new_step; new_weight = new_step; old_weight = 1.0 - new_step; for (i = 0; i < pitch_overlap; i++) { int index = (i * channel_count) + j; amp[index] = fsaturate(old_weight * s->history[plc_history_len - 1 - i] + new_weight * s->pitchbuf[i]); new_weight += new_step; old_weight -= old_step; if (old_weight < 0.0) old_weight = 0.0; } s->pitch_offset = i; } else { gain = 1.0 - s->missing_samples*ATTENUATION_INCREMENT; i = 0; } for ( ; gain > 0.0 && i < frames; i++) { int index = (i * channel_count) + j; amp[index] = s->pitchbuf[s->pitch_offset]*gain; gain -= ATTENUATION_INCREMENT; if (++s->pitch_offset >= s->pitch) s->pitch_offset = 0; } for ( ; i < frames; i++) { int index = (i * channel_count) + j; amp[i] = 0; } s->missing_samples += orig_len; save_history(s, amp, j, frames); delete [] tmp; j++; } return frames; } void PcmConcealer::save_history(plc_state_t *s, short *buf, int channel_index, int frames) { if (frames >= plc_history_len) { /* Just keep the last part of the new data, starting at the beginning of the buffer */ //memcpy(s->history, buf + len - plc_history_len, sizeof(short)*plc_history_len); int frames_to_copy = plc_history_len; for(int i = 0; i < frames_to_copy; i ++) { int index = (channel_count * (i + frames - plc_history_len)) + channel_index; s->history[i] = buf[index]; } s->buf_ptr = 0; return; } if (s->buf_ptr + frames > plc_history_len) { /* Wraps around - must break into two sections */ //memcpy(s->history + s->buf_ptr, buf, sizeof(short)*(plc_history_len - s->buf_ptr)); short *hist_ptr = s->history + s->buf_ptr; int frames_to_copy = plc_history_len - s->buf_ptr; for(int i = 0; i < frames_to_copy; i ++) { int index = (channel_count * i) + channel_index; hist_ptr[i] = buf[index]; } frames -= (plc_history_len - s->buf_ptr); //memcpy(s->history, buf + (plc_history_len - s->buf_ptr), sizeof(short)*len); frames_to_copy = frames; for(int i = 0; i < frames_to_copy; i ++) { int index = (channel_count * (i + (plc_history_len - s->buf_ptr))) + channel_index; s->history[i] = buf[index]; } s->buf_ptr = frames; return; } /* Can use just one section */ //memcpy(s->history + s->buf_ptr, buf, sizeof(short)*len); short *hist_ptr = s->history + s->buf_ptr; int frames_to_copy = frames; for(int i = 0; i < frames_to_copy; i ++) { int index = (channel_count * i) + channel_index; hist_ptr[i] = buf[index]; } s->buf_ptr += frames; } void PcmConcealer::normalise_history(plc_state_t *s) { short *tmp = new short[plc_history_len]; if (s->buf_ptr == 0) return; memcpy(tmp, s->history, sizeof(short)*s->buf_ptr); memcpy(s->history, s->history + s->buf_ptr, sizeof(short)*(plc_history_len - s->buf_ptr)); memcpy(s->history + plc_history_len - s->buf_ptr, tmp, sizeof(short)*s->buf_ptr); s->buf_ptr = 0; delete [] tmp; } int PcmConcealer::amdf_pitch(int min_pitch, int max_pitch, short amp[], int channel_index, int frames) { int i; int j; int acc; int min_acc; int pitch; pitch = min_pitch; min_acc = INT_MAX; for (i = max_pitch; i <= min_pitch; i++) { acc = 0; for (j = 0; j < frames; j++) { int index1 = (channel_count * (i+j)) + channel_index; int index2 = (channel_count * j) + channel_index; //std::cout << "Index 1: " << index1 << ", Index 2: " << index2 << std::endl; acc += abs(amp[index1] - amp[index2]); } if (acc < min_acc) { min_acc = acc; pitch = i; } } std::cout << "Pitch: " << pitch << std::endl; return pitch; } } P.S. - I must confess that digital audio is not my forte...

Search Results

Search found 195 results on 8 pages for 'reversed'.

Page 8/8 | < Previous Page | 4 5 6 7 8

- by Elton Stoneman

- by Sridhar_R-Oracle

- by Timothy Klenke

- by Elton Stoneman

- by jqChart

- by Paul White

- by Calvin Sun

- by kennethkryger

- by smichak

- by kira423

- by Richard Knop

- by Xorty

- by Bill Graziano

- by Manhim

- by Lars Flyger

- by Mark

- by Marki

- by J M 4

- by max

- by ZeroDefect

< Previous Page | 4 5 6 7 8