primary key - Page 235 - Developer IT

Value [...] not a valid choice, django-updown

- by tamara

I am trying to implemet django-updown https://github.com/weluse/django-updown. When I try to add vote trough the admin panel it says Value 1 not a valid choice. This is the models.py from the application: _SCORE_TYPE_CHOICES = ( ('-1', 'DISLIKE'), ('1', 'LIKE'), ) SCORE_TYPES = dict((value, key) for key, value in _SCORE_TYPE_CHOICES) class Vote(models.Model): content_type = models.ForeignKey(ContentType, related_name="updown_votes") object_id = models.PositiveIntegerField() key = models.CharField(max_length=32) score = models.SmallIntegerField(choices=_SCORE_TYPE_CHOICES) user = models.ForeignKey(User, blank=True, null=True, related_name="updown_votes") ip_address = models.IPAddressField() date_added = models.DateTimeField(default=datetime.datetime.now, editable=False) date_changed = models.DateTimeField(default=datetime.datetime.now, editable=False) Do you have an idea what could be wrong?

Read the article

Mysql Constraint problem

- by Bramjam

this is my table /* oefenreeks leerplan */ CREATE TABLE leerplan_oefenreeks ( leerplan_oefenreeks_id INT PRIMARY KEY AUTO_INCREMENT NOT NULL, leerplan_id INT NOT NULL, oefenreeks_id INT NOT NULL, plaats INT NOT NULL ); /* fk */ ALTER TABLE leerplan_oefenreeks ADD CONSTRAINT fk_leerp_oefenr_leerplan FOREIGN KEY(leerplan_id) REFERENCES leerplan (leerplan_id) ON DELETE CASCADE; ALTER TABLE leerplan_oefenreeks ADD CONSTRAINT fk_leerp_oefenr_oefenreeks FOREIGN KEY(oefenreeks_id) REFERENCES oefenreeks (oefenreeks_id) ON DELETE CASCADE; /* unique s *//*when I execute the nexline, my fk_leerp_oefenr_leerplan constraint vanishes/disappears*/ ALTER TABLE leerplan_oefenreeks ADD CONSTRAINT un_leerp_oefenr UNIQUE(leerplan_id, oefenreeks_id); ALTER TABLE leerplan_oefenreeks ADD CONSTRAINT un_leerp_oefenr_plaats UNIQUE(leerplan_id, plaats); when I go and check only 3 constraints exist (fk_leerp_oefenr_leerplan is gone) I don't understand why this happens, plz tell me (if you need more info just ask ;)

Read the article

Similar Sub-Classes

- by praks5432

Lets say I have a class A that is fairly simple like this - public class A{ private int randomField = 0; protected int key; protected double dmg; } Now I want to write a number of sub-classes that inherit the protected fields and only differ based on the initial values that are assigned to those fields - for example, if I wrote two subclasses B and C, the only difference between those two sub-classes would be that the values key and dmg would have different values. They would share a method, set, which would be exactly the same, in that it would affect the same variable. I find when I'm writing these sub-classes I'm repeating myself, as I just change the constructor to set different initial values to key and dmg, and simply copy and paste the set method. Is there a 'good' way to do this?

Read the article

Picking encryption cipher for mcrypt

- by Autolycus

I have few questions about this code: <?php $iv_size = mcrypt_get_iv_size(MCRYPT_RIJNDAEL_256, MCRYPT_MODE_ECB); $iv = mcrypt_create_iv($iv_size, MCRYPT_RAND); $key = "This is a very secret key"; $text = file_get_contents('path/to/your/file'); echo strlen($text) . "\n"; $crypttext = mcrypt_encrypt(MCRYPT_RIJNDAEL_256, $key, $text, MCRYPT_MODE_ECB, $iv); file_put_contents('path/to/your/file', $crypttext); ?> It encrypts the file just fine, however it adds additional nulls at the end, so if I encrypt: a test string is this one and here is a new line once decrypted becomes: a test string is this one and here is a new line 000000000000000 What's going on? Second, is MCRYPT_RIJNDAEL_256 compatible with AES-128? Finally, how would I let another party decrypt a file I've encrypted? They would need to know which encryption was used and I am not sure what to tell them.

Read the article

add a from to backup routine

- by Gerard Flynn

hi how do you put a process bar and button onto this code i have class and want to add a gui on to the code using System; using System.Collections.Generic; using System.ComponentModel; using System.Data; using System.Drawing; using System.Text; using System.Windows.Forms; using System.Data.SqlClient; using System.IO; using System.Threading; using Tamir.SharpSsh; using System.Security.Cryptography; using ICSharpCode.SharpZipLib.Checksums; using ICSharpCode.SharpZipLib.Zip; using ICSharpCode.SharpZipLib.GZip; namespace backup { public partial class Form1 : Form { public Form1() { InitializeComponent(); } /// <summary> /// Summary description for Class1. /// </summary> public class Backup { private string dbName; private string dbUsername; private string dbPassword; private static string baseDir; private string backupName; private static bool isBackup; private string keyString; private string ivString; private string[] backupDirs = new string[0]; private string[] excludeDirs = new string[0]; private ZipOutputStream zipOutputStream; private string backupFile; private string zipFile; private string encryptedFile; static void Main() { Backup.Log("BackupUtility loaded"); try { new Backup(); if (!isBackup) MessageBox.Show("Restore complete"); } catch (Exception e) { Backup.Log(e.ToString()); if (!isBackup) MessageBox.Show("Error restoring!\r\n" + e.Message); } } private void LoadAppSettings() { this.backupName = System.Configuration.ConfigurationSettings.AppSettings["BackupName"].ToString(); this.dbName = System.Configuration.ConfigurationSettings.AppSettings["DBName"].ToString(); this.dbUsername = System.Configuration.ConfigurationSettings.AppSettings["DBUsername"].ToString(); this.dbPassword = System.Configuration.ConfigurationSettings.AppSettings["DBPassword"].ToString(); //default to using where we are executing this assembly from Backup.baseDir = System.Reflection.Assembly.GetExecutingAssembly().Location.Substring(0, System.Reflection.Assembly.GetExecutingAssembly().Location.LastIndexOf("\\")) + "\\"; Backup.isBackup = bool.Parse(System.Configuration.ConfigurationSettings.AppSettings["IsBackup"].ToString()); this.keyString = System.Configuration.ConfigurationSettings.AppSettings["KeyString"].ToString(); this.ivString = System.Configuration.ConfigurationSettings.AppSettings["IVString"].ToString(); this.backupDirs = GetSetting("BackupDirs", ','); this.excludeDirs = GetSetting("ExcludeDirs", ','); } private string[] GetSetting(string settingName, char delimiter) { if (System.Configuration.ConfigurationSettings.AppSettings[settingName] != null) { string settingVal = System.Configuration.ConfigurationSettings.AppSettings[settingName].ToString(); if (settingVal.Length > 0) return settingVal.Split(delimiter); } return new string[0]; } public Backup() { this.LoadAppSettings(); if (isBackup) this.DoBackup(); else this.DoRestore(); Log("Finished"); } private void DoRestore() { System.Windows.Forms.OpenFileDialog fileDialog = new System.Windows.Forms.OpenFileDialog(); fileDialog.Title = "Choose .encrypted file"; fileDialog.Filter = "Encrypted files (*.encrypted)|*.encrypted|All files (*.*)|*.*"; fileDialog.InitialDirectory = Backup.baseDir; if (fileDialog.ShowDialog() == System.Windows.Forms.DialogResult.OK) { //string encryptedFile = GetFileName("encrypted"); string encryptedFile = fileDialog.FileName; string decryptedFile = this.GetDecryptedFilename(encryptedFile); //string originalFile = GetFileName("original"); this.Decrypt(encryptedFile, decryptedFile); //this.UnzipFile(decryptedFile, originalFile); } } //use the same filename as the backup except replace ".encrypted" with ".decrypted.zip" private string GetDecryptedFilename(string encryptedFile) { string name = encryptedFile.Substring(0, encryptedFile.LastIndexOf(".")); name += ".decrypted.zip"; return name; } private void DoBackup() { this.backupFile = GetFileName("bak"); this.zipFile = GetFileName("zip"); this.encryptedFile = GetFileName("encrypted"); this.DeleteFiles(); this.zipOutputStream = new ZipOutputStream(File.Create(zipFile)); try { //backup database first if (this.dbName.Length > 0) { this.BackupDB(backupFile); this.ZipFile(backupFile, this.GetName(backupFile)); } //zip any directories specified in config file this.ZipUserSpecifiedFilesAndDirectories(this.backupDirs); } finally { this.zipOutputStream.Finish(); this.zipOutputStream.Close(); } this.Encrypt(zipFile, encryptedFile); this.SCPFile(encryptedFile); this.DeleteFiles(); } /// <summary> /// Deletes any files created by the backup process, namely the DB backup file, /// the zip of all files backuped up, and the encrypred zip file /// </summary> private void DeleteFiles() { File.Delete(this.backupFile); File.Delete(this.zipFile); ///File.Delete(this.encryptedFile); } private void ZipUserSpecifiedFilesAndDirectories(string[] fileNames) { foreach (string fileName in fileNames) { string name = fileName.Trim(); if (name.Length > 0) { Log("Zipping " + name); this.ZipFile(name, this.GetNameFromDir(name)); } } } private void SCPFile(string inputPath) { string sshServer = System.Configuration.ConfigurationSettings.AppSettings["SSHServer"].ToString(); string sshUsername = System.Configuration.ConfigurationSettings.AppSettings["SSHUsername"].ToString(); string sshPassword = System.Configuration.ConfigurationSettings.AppSettings["SSHPassword"].ToString(); if (sshServer.Length > 0 && sshUsername.Length > 0 && sshPassword.Length > 0) { Scp scp = new Scp(sshServer, sshUsername, sshPassword); //Copy a file from local machine to remote SSH server scp.Connect(); Log("Connected to " + sshServer); //scp.Put(inputPath, "/home/wal/temp.txt"); scp.Put(inputPath, GetName(inputPath)); scp.Close(); } else { Log("Not SCP as missing login details"); } } private string GetName(string inputPath) { FileInfo info = new FileInfo(inputPath); return info.Name; } private string GetNameFromDir(string inputPath) { DirectoryInfo info = new DirectoryInfo(inputPath); return info.Name; } private static void Log(string msg) { try { string toLog = DateTime.Now.ToString() + ": " + msg; System.Diagnostics.Debug.WriteLine(toLog); System.IO.FileStream fs = new System.IO.FileStream(baseDir + "app.log", System.IO.FileMode.OpenOrCreate, System.IO.FileAccess.ReadWrite); System.IO.StreamWriter m_streamWriter = new System.IO.StreamWriter(fs); m_streamWriter.BaseStream.Seek(0, System.IO.SeekOrigin.End); m_streamWriter.WriteLine(toLog); m_streamWriter.Flush(); m_streamWriter.Close(); fs.Close(); } catch (Exception e) { Console.WriteLine(e.ToString()); } } private byte[] GetFileBytes(string path) { FileStream stream = new FileStream(path, FileMode.Open); byte[] bytes = new byte[stream.Length]; stream.Read(bytes, 0, bytes.Length); stream.Close(); return bytes; } private void WriteFileBytes(byte[] bytes, string path) { FileStream stream = new FileStream(path, FileMode.Create); stream.Write(bytes, 0, bytes.Length); stream.Close(); } private void UnzipFile(string inputPath, string outputPath) { ZipInputStream zis = new ZipInputStream(File.OpenRead(inputPath)); ZipEntry theEntry = zis.GetNextEntry(); FileStream streamWriter = File.Create(outputPath); int size = 2048; byte[] data = new byte[2048]; while (true) { size = zis.Read(data, 0, data.Length); if (size > 0) { streamWriter.Write(data, 0, size); } else { break; } } streamWriter.Close(); zis.Close(); } private bool ExcludeDir(string dirName) { foreach (string excludeDir in this.excludeDirs) { if (dirName == excludeDir) return true; } return false; } private void ZipFile(string inputPath, string zipName) { FileAttributes fa = File.GetAttributes(inputPath); if ((fa & FileAttributes.Directory) != 0) { string dirName = zipName + "/"; ZipEntry entry1 = new ZipEntry(dirName); this.zipOutputStream.PutNextEntry(entry1); string[] subDirs = Directory.GetDirectories(inputPath); //create directories first foreach (string subDir in subDirs) { DirectoryInfo info = new DirectoryInfo(subDir); string name = info.Name; if (this.ExcludeDir(name)) Log("Excluding " + dirName + name); else this.ZipFile(subDir, dirName + name); } //then store files string[] fileNames = Directory.GetFiles(inputPath); foreach (string fileName in fileNames) { FileInfo info = new FileInfo(fileName); string name = info.Name; this.ZipFile(fileName, dirName + name); } } else { Crc32 crc = new Crc32(); this.zipOutputStream.SetLevel(6); // 0 - store only to 9 - means best compression FileStream fs = null; try { fs = File.OpenRead(inputPath); } catch (IOException ioEx) { Log("WARNING! " + ioEx.Message);//might be in use, skip file in this case } if (fs != null) { byte[] buffer = new byte[fs.Length]; fs.Read(buffer, 0, buffer.Length); ZipEntry entry = new ZipEntry(zipName); entry.DateTime = DateTime.Now; // set Size and the crc, because the information // about the size and crc should be stored in the header // if it is not set it is automatically written in the footer. // (in this case size == crc == -1 in the header) // Some ZIP programs have problems with zip files that don't store // the size and crc in the header. entry.Size = fs.Length; fs.Close(); crc.Reset(); crc.Update(buffer); entry.Crc = crc.Value; this.zipOutputStream.PutNextEntry(entry); this.zipOutputStream.Write(buffer, 0, buffer.Length); } } } private void Encrypt(string inputPath, string outputPath) { RijndaelManaged rijndaelManaged = new RijndaelManaged(); byte[] encrypted; byte[] toEncrypt; //Create a new key and initialization vector. //myRijndael.GenerateKey(); //myRijndael.GenerateIV(); /*des.GenerateKey(); des.GenerateIV(); string temp1 = Convert.ToBase64String(des.Key); string temp2 = Convert.ToBase64String(des.IV);*/ //Get the key and IV. byte[] key = Convert.FromBase64String(keyString); byte[] IV = Convert.FromBase64String(ivString); //Get an encryptor. ICryptoTransform encryptor = rijndaelManaged.CreateEncryptor(key, IV); //Encrypt the data. MemoryStream msEncrypt = new MemoryStream(); CryptoStream csEncrypt = new CryptoStream(msEncrypt, encryptor, CryptoStreamMode.Write); //Convert the data to a byte array. toEncrypt = this.GetFileBytes(inputPath); //Write all data to the crypto stream and flush it. csEncrypt.Write(toEncrypt, 0, toEncrypt.Length); csEncrypt.FlushFinalBlock(); //Get encrypted array of bytes. encrypted = msEncrypt.ToArray(); WriteFileBytes(encrypted, outputPath); } private void Decrypt(string inputPath, string outputPath) { RijndaelManaged myRijndael = new RijndaelManaged(); //DES des = new DESCryptoServiceProvider(); byte[] key = Convert.FromBase64String(keyString); byte[] IV = Convert.FromBase64String(ivString); byte[] encrypted = this.GetFileBytes(inputPath); byte[] fromEncrypt; //Get a decryptor that uses the same key and IV as the encryptor. ICryptoTransform decryptor = myRijndael.CreateDecryptor(key, IV); //Now decrypt the previously encrypted message using the decryptor // obtained in the above step. MemoryStream msDecrypt = new MemoryStream(encrypted); CryptoStream csDecrypt = new CryptoStream(msDecrypt, decryptor, CryptoStreamMode.Read); fromEncrypt = new byte[encrypted.Length]; //Read the data out of the crypto stream. int bytesRead = csDecrypt.Read(fromEncrypt, 0, fromEncrypt.Length); byte[] readBytes = new byte[bytesRead]; Array.Copy(fromEncrypt, 0, readBytes, 0, bytesRead); this.WriteFileBytes(readBytes, outputPath); } private string GetFileName(string extension) { return baseDir + backupName + "_" + DateTime.Now.ToString("yyyyMMdd") + "." + extension; } private void BackupDB(string backupPath) { string sql = @"DECLARE @Date VARCHAR(300), @Dir VARCHAR(4000) --Get today date SET @Date = CONVERT(VARCHAR, GETDATE(), 112) --Set the directory where the back up file is stored SET @Dir = '"; sql += backupPath; sql += @"' --create a 'device' to write to first EXEC sp_addumpdevice 'disk', 'temp_device', @Dir --now do the backup BACKUP DATABASE " + this.dbName; sql += @" TO temp_device WITH FORMAT --Drop the device EXEC sp_dropdevice 'temp_device' "; //Console.WriteLine("sql="+sql); Backup.Log("Starting backup of " + this.dbName); ExecuteSQL(sql); } /// <summary> /// Executes the specified SQL /// Returns true if no errors were encountered during execution /// </summary> /// <param name="procedureName"></param> private void ExecuteSQL(string sql) { SqlConnection conn = new SqlConnection(this.GetDBConnectString()); try { SqlCommand comm = new SqlCommand(sql, conn); conn.Open(); comm.ExecuteNonQuery(); } finally { conn.Close(); } } private string GetDBConnectString() { StringBuilder builder = new StringBuilder(); builder.Append("Data Source=127.0.0.1; User ID="); builder.Append(this.dbUsername); builder.Append("; Password="); builder.Append(this.dbPassword); builder.Append("; Initial Catalog="); builder.Append(this.dbName); builder.Append(";Connect Timeout=30"); return builder.ToString(); } } } }

Read the article

AngularJS databinding

- by user3652865

How can I add multiple values to one object in an Array. I am having Environment and Cluster, I am able to assign multiple clusters to one environment. Now I want to add application name to this environment and cluster pair. I am having page called "Add Application". Here I am using select menu to for environment and Cluster. My first question is, when I select environment then want to show only those clusters which are assigned to that environment name. And assign application name to that pair. Also should be able to edit the Application field. I am using environmentServices and clusterServices to store updated data. link of JSFiddle: http://jsfiddle.net/avinashMaddy/J2KLK/5/ Please if someone can help me in this. Below is my code: <div class="maincontent" ng-controller="manageApplicationController"> <div class="article"> <form> <section>  <div class="col-md-4"> <label>Environment:</label> <select ng-model="newApp.selectedEnvironment" class="form-control" ng-options="environment.name for environment in environments"> <option value='' disabled style='display:none;'> Select Environment </option> </select> <span> <select ng-switch-when="true" disabled ng-model="newApp.selectedEnvironment" class="form-control" ng-options="environment.name for environment in environments"> <option value='' disabled style='display:none;'> Select Environment </option> </select> </span> </div>  <div class="col-md-4"> <label>Cluster:</label> <span ng-switch on="newApp.showCancel"> <select ng-switch-default ng-model="newApp.selectedCluster" class="form-control" ng-options="cluster for cluster in clusters"> <option value='' disabled style='display:none;'> Select Environment </option> </select> <select ng-switch-when="true" disabled ng-model="newApp.selectedCluster" class="form-control" ng-options="cluster for cluster in clusters"> <option value='' disabled style='display:none;'> Select Environment </option> </select> </span> </div>  <div class="col-md-4"> <label>Application Name:</label> <input type="text" class="form-control" name="applicationName" placeholder="Application" ng-model="app.name" required> <br/> <input type="hidden" ng-model="app.id" /> </div> </section>  <section class="col-md-12"> <button type="button" class="btn btn-default pull-right" ng-click="saveNewApplicatons()">Save</button> </section> </form> </div>  <div class="article"> <table class="table table-bordered table-striped"> <tr> <th colspan="6"> <div class="pull-left">Cluster Info</div> </th> </tr> <tr> <th>Environment</th> <th>Cluster</th> <th>Application</th> <th>Edit</th> <th>Header Ifo</th> </tr> <tr ng-repeat="app in applications"> <td>{{app.environment}}</td> <td>{{app.cluster}}</td> <td>{{app.name}}</td> <td> <a href="" ng-click="edit(app.id)" title="Edit">edit</span></a> | <a href="" ng-click="remove(app.id)" title="Delete">delete</a> </td> <td>  <script type="text/ng-template" id="addHederInfo.html"> <div class="modal-header"> <h3>Add Header Info</h3> </div> <div class="modal-body"> <input type="text" class="form-control" name="eName" placeholder="Header Key" ng-model="$parent.header.key"> <br/> <input type="text" class="form-control" name="eName" placeholder="Header Value" ng-model="$parent.header.value"> <br /> <input type="hidden" ng-model="header.id" /> <section> <div class="pull-right"> <button class="btn btn-primary" ng-click="saveHeader()">Add</button> <button class="btn btn-warning" ng-click="cancel()">Close</button> </div> </section> </div> <div class="modal-footer"> <h3>Existing Header Info for </h3> <table class="table table-bordered table-striped"> <tr> <th>Header Key</th> <th>Header Vlaue</th> </tr> <tr ng-repeat="header in headers"> <td>{{header.key}}</td> <td>{{header.value}}</td> </tr> </table> </div> </script>  <script type="text/ng-template" id="editHederInfo.html"> <div class="modal-header"> <h3>Edit Header Info</h3> </div> <div class="modal-body"> <input type="text" class="form-control" name="eName" placeholder="Header Key" ng-model="$parent.header.key"> <br/> <input type="text" class="form-control" name="eName" placeholder="Header Value" ng-model="$parent.header.value"> <br /> <input type="hidden" ng-model="header.id" /> <section> <div class="pull-right"> <button class="btn btn-primary" ng-click="saveHeader()">Update</button> <button class="btn btn-warning" ng-click="cancel()">Close</button> </div> </section> </div> <div class="modal-footer"> <h3>Existing Header Info for</h3> <table class="table table-bordered table-striped"> <tr> <th>Header Key</th> <th>Header Vlaue</th> <th>Edit</th> </tr> <tr ng-repeat="header in headers"> <td>{{header.key}}</td> <td>{{header.value}}</td> <td> <a href="" ng-click="editHeader(header.id)" title="Edit"><span class="glyphicon glyphicon-edit" ></span></a> | <a href="" ng-click="removeHeader(header.id)" title="Edit"><span class="glyphicon glyphicon-trash"></span></a> </td> </tr> </table> </div> </script>   <a href="" ng-click="addInfo()">Add</a> | <a href="" ng-click="editInfo()">Edit</a> </td> </tr> </table> </div> </div> Controller.js: var apsApp = angular.module('apsApp', []); apsApp.service('clusterService', function(){ var clusters=[]; //simply returns the environment list this.list = function () { return clusters; }; }); apsApp.service('environmentService', function(){ var environments=[ {name :'DEV',}, {name:'PROD',}, {name:'QA',}, {name:'Linux_Dev',} ]; //simply returns the environment list this.list = function () { return environments; }; apsApp.controller('manageApplicationController', function ($scope, environmentService, clusterService) { var uid = 0; $scope.environments= environmentService.list(); $scope.clusters= clusterService.list(); $scope.newApp = {}; $scope.newApp.selectedEnvironment = $scope.environments[0]; $scope.newApp.selectedCluster = $scope.clusters[0]; $scope.newApp.buttonLabel = 'Save'; $scope.newApp.showCancel = false; /*$scope.applications=[ {'name': 'Enterprice App Store' }, {'name': 'UsageGateway'}, {'name': 'Click 2 Fill'}, {'name': 'ATT SmartWiFi'} ];*/ //add new application $scope.saveNewApplicatons = function() { if ($scope.select.id == undefined) { //if this is new application, add it in applications array $scope.clusters.push({ id: uid++, cluster: $scope.newApp.cluster, environment: $scope.newApp.selectedEnvironment }); } else { $scope.clusters[$scope.select.id].cluster = $scope.select.cluster; $scope.newApp.id = undefined; $scope.newApp.buttonLabel = 'Save Cluster'; $scope.newApp.showCancel = false; }; //clear the add appplicaitons form $scope.newApp.selectedEnvironment = $scope.environments[0]; }; //delete application $scope.remove = function (id) { //search app with given id and delete it for (i in $scope.clusters) { if ($scope.clusters[i].id == id) { confirm("This Cluster will get deleted permanently"); $scope.clusters.splice(i, 1); $scope.clust = {}; } } }; $scope.cancelUpdate = function () { $scope.newApp.buttonLabel = 'Save Cluster'; $scope.newApp.showCancel = false; $scope.newApp.id = undefined; $scope.newApp.cluster = ""; $scope.newApp.selectedEnvironment = $scope.environments[0]; }; });

Read the article

Handling keyboard and mouse input (Win API)

- by Deluxe

There is a number of ways to catch mouse or keyboard under Windows. So I tried some of them, but every of them has some advantages and drawbacks. I want to ask you: Which method do use? I've tried these: WM_KEYDOWN/WM_KEYUP - Main disadvantage is that, I can't distinguish between left and right-handed keys like ALT, CONTROL or SHIFT. GetKeyboardState - This solves problem of first method, but there is new one. When I get that the Right-ALT key is pressed, I also get that the Left-Control key is down. This behaviour happens only when using localized keyboard layout (Czech - CS). WM_INPUT (Raw Input) - This method also doesn't distinguish left and right-handed keys (if I can remember) and for mouse movement sometimes generates message with zero delta values of mouse position.

Read the article

ASP.NET Podcast Show #148 - ASP.NET WebForms to build a Mobile Web Application

- by Wallym

Check the podcast site for the original url. This is the video and source code for an ASP.NET WebForms app that I wrote that is optimized for the iPhone and mobile environments. Subscribe to everything. Subscribe to WMV. Subscribe to M4V for iPhone/iPad. Subscribe to MP3. Download WMV. Download M4V for iPhone/iPad. Download MP3. Link to iWebKit. Source Code: <%@ Page Title="MapSplore" Language="C#" MasterPageFile="iPhoneMaster.master" AutoEventWireup="true" CodeFile="Default.aspx.cs" Inherits="AT_iPhone_Default" %> <asp:Content ID="Content1" ContentPlaceHolderID="head" Runat="Server"></asp:Content><asp:Content ID="Content2" ContentPlaceHolderID="Content" Runat="Server" ClientIDMode="Static"> <asp:ScriptManager ID="sm" runat="server" EnablePartialRendering="true" EnableHistory="false" EnableCdn="true" /> <script type="text/javascript" src="http://maps.google.com/maps/api/js?sensor=true"></script> <script language="javascript" type="text/javascript">  </script> <asp:HiddenField ID="Latitude" runat="server" /> <asp:HiddenField ID="Longitude" runat="server" /> <script type="text/javascript" src="http://ajax.googleapis.com/ajax/libs/jquery/1/jquery.min.js%22%3E%3C/script> <script language="javascript" type="text/javascript"> $(document).ready(function () { GetLocation(); setupMapDiv(); setupPlaceIveBeen(); }); function GetLocation() { if (navigator.geolocation != null) { navigator.geolocation.getCurrentPosition(getData); } else { var mess = document.getElementById('<%=Message.ClientID %>'); mess.innerHTML = "Sorry, your browser does not support geolocation. " + "Try the latest version of Safari on the iPhone, Android browser, or the latest version of FireFox."; } } function UpdateLocation_Click() { GetLocation(); } function getData(position) { var latitude = position.coords.latitude; var longitude = position.coords.longitude; var hdLat = document.getElementById('<%=Latitude.ClientID %>'); var hdLon = document.getElementById('<%=Longitude.ClientID %>'); hdLat.value = latitude; hdLon.value = longitude; } </script> <asp:Label ID="Message" runat="server" /> <asp:UpdatePanel ID="upl" runat="server"> <ContentTemplate> <asp:Panel ID="pnlStart" runat="server" Visible="true"> <div id="topbar"> <div id="title">MapSplore</div> </div> <div id="content"> <ul class="pageitem"> <li class="menu"> <asp:LinkButton ID="lbLocalDeals" runat="server" onclick="lbLocalDeals_Click"> <asp:Image ID="imLocalDeals" runat="server" ImageUrl="~/Images/ArtFavor_Money_Bag_Icon.png" Height="30" /> <span class="name">Local Deals.</span> <span class="arrow"></span> </asp:LinkButton> </li> <li class="menu"> <asp:LinkButton ID="lbLocalPlaces" runat="server" onclick="lbLocalPlaces_Click"> <asp:Image ID="imLocalPlaces" runat="server" ImageUrl="~/Images/Andy_Houses_on_the_horizon_-_Starburst_remix.png" Height="30" /> <span class="name">Local Places.</span> <span class="arrow"></span> </asp:LinkButton> </li> <li class="menu"> <asp:LinkButton ID="lbWhereIveBeen" runat="server" onclick="lbWhereIveBeen_Click"> <asp:Image ID="imImHere" runat="server" ImageUrl="~/Images/ryanlerch_flagpole.png" Height="30" /> <span class="name">I've been here.</span> <span class="arrow"></span> </asp:LinkButton> </li> <li class="menu"> <asp:LinkButton ID="lbMyStats" runat="server"> <asp:Image ID="imMyStats" runat="server" ImageUrl="~/Images/Anonymous_Spreadsheet.png" Height="30" /> <span class="name">My Stats.</span> <span class="arrow"></span> </asp:LinkButton> </li> <li class="menu"> <asp:LinkButton ID="lbAddAPlace" runat="server" onclick="lbAddAPlace_Click"> <asp:Image ID="imAddAPlace" runat="server" ImageUrl="~/Images/jean_victor_balin_add.png" Height="30" /> <span class="name">Add a Place.</span> <span class="arrow"></span> </asp:LinkButton> </li> <li class="button"> <input type="button" value="Update Your Current Location" onclick="UpdateLocation_Click()"> </li> </ul> </div> </asp:Panel> <div> <asp:Panel ID="pnlCoupons" runat="server" Visible="false"> <div id="topbar"> <div id="title">MapSplore</div> <div id="leftbutton"> <asp:LinkButton runat="server" Text="Return" ID="ReturnFromDeals" OnClick="ReturnFromDeals_Click" /></div></div> <div class="content"> <asp:ListView ID="lvCoupons" runat="server"> <LayoutTemplate> <ul class="pageitem" runat="server"> <asp:PlaceHolder ID="itemPlaceholder" runat="server" /> </ul> </LayoutTemplate> <ItemTemplate> <li class="menu"> <asp:LinkButton ID="lbBusiness" runat="server" Text='<%#Eval("Place.Name") %>' OnClick="lbBusiness_Click"> <span class="comment"> <asp:Label ID="lblAddress" runat="server" Text='<%#Eval("Place.Address1") %>' /> <asp:Label ID="lblDis" runat="server" Text='<%# Convert.ToString(Convert.ToInt32(Eval("Place.Distance"))) + " meters" %>' CssClass="smallText" /> <asp:HiddenField ID="hdPlaceId" runat="server" Value='<%#Eval("PlaceId") %>' /> <asp:HiddenField ID="hdGeoPromotionId" runat="server" Value='<%#Eval("GeoPromotionId") %>' /> </span> <span class="arrow"></span> </asp:LinkButton></li></ItemTemplate></asp:ListView><asp:GridView ID="gvCoupons" runat="server" AutoGenerateColumns="false"> <HeaderStyle BackColor="Silver" /> <AlternatingRowStyle BackColor="Wheat" /> <Columns> <asp:TemplateField AccessibleHeaderText="Business" HeaderText="Business"> <ItemTemplate> <asp:Image ID="imPlaceType" runat="server" Text='<%#Eval("Type") %>' ImageUrl='<%#Eval("Image") %>' /> <asp:LinkButton ID="lbBusiness" runat="server" Text='<%#Eval("Name") %>' OnClick="lbBusiness_Click" /> <asp:LinkButton ID="lblAddress" runat="server" Text='<%#Eval("Address1") %>' CssClass="smallText" /> <asp:Label ID="lblDis" runat="server" Text='<%# Convert.ToString(Convert.ToInt32(Eval("Distance"))) + " meters" %>' CssClass="smallText" /> <asp:HiddenField ID="hdPlaceId" runat="server" Value='<%#Eval("PlaceId") %>' /> <asp:HiddenField ID="hdGeoPromotionId" runat="server" Value='<%#Eval("GeoPromotionId") %>' /> <asp:Label ID="lblInfo" runat="server" Visible="false" /> </ItemTemplate> </asp:TemplateField> </Columns> </asp:GridView> </div> </asp:Panel> <asp:Panel ID="pnlPlaces" runat="server" Visible="false"> <div id="topbar"> <div id="title"> MapSplore</div><div id="leftbutton"> <asp:LinkButton runat="server" Text="Return" ID="ReturnFromPlaces" OnClick="ReturnFromPlaces_Click" /></div></div> <div id="content"> <asp:ListView ID="lvPlaces" runat="server"> <LayoutTemplate> <ul id="ulPlaces" class="pageitem" runat="server"> <asp:PlaceHolder ID="itemPlaceholder" runat="server" /> <li class="menu"> <asp:LinkButton ID="lbNotListed" runat="server" CssClass="name" OnClick="lbNotListed_Click"> Place not listed <span class="arrow"></span> </asp:LinkButton> </li> </ul> </LayoutTemplate> <ItemTemplate> <li class="menu"> <asp:LinkButton ID="lbImHere" runat="server" CssClass="name" OnClick="lbImHere_Click"> <%#DisplayName(Eval("Name")) %>  <%# Convert.ToString(Convert.ToInt32(Eval("Distance"))) + " meters" %> <asp:HiddenField ID="hdPlaceId" runat="server" Value='<%#Eval("PlaceId") %>' /> <span class="arrow"></span> </asp:LinkButton></li></ItemTemplate></asp:ListView> </div> </asp:Panel> <asp:Panel ID="pnlImHereNow" runat="server" Visible="false"> <div id="topbar"> <div id="title"> MapSplore</div><div id="leftbutton"> <asp:LinkButton runat="server" Text="Places" ID="lbImHereNowReturn" OnClick="lbImHereNowReturn_Click" /></div></div> <div id="rightbutton"> <asp:LinkButton runat="server" Text="Beginning" ID="lbBackToBeginning" OnClick="lbBackToBeginning_Click" /> </div> <div id="content"> <ul class="pageitem"> <asp:HiddenField ID="hdPlaceId" runat="server" /> <asp:HiddenField ID="hdPlaceLat" runat="server" /> <asp:HiddenField ID="hdPlaceLon" runat="server" /> <asp:HiddenField ID="hdPlaceTitle" runat="server" /> <asp:Button ID="btnImHereNow" runat="server" Text="I'm here" OnClick="btnImHereNow_Click" /> <asp:Label ID="lblPlaceTitle" runat="server" /><br /> <asp:TextBox ID="txtWhatsHappening" runat="server" TextMode="MultiLine" Rows="2" style="width:300px" /><br /> <div id="divImHere" style="width:300px; height:300px"></div> </div> </ul> </asp:Panel> <asp:Panel runat="server" ID="pnlIveBeenHere" Visible="false"> <div id="topbar"> <div id="title"> Where I've been</div><div id="leftbutton"> <asp:LinkButton ID="lbIveBeenHereBack" runat="server" Text="Back" OnClick="lbIveBeenHereBack_Click" /></div></div> <div id="content"> <asp:ListView ID="lvWhereIveBeen" runat="server"> <LayoutTemplate> <ul id="ulWhereIveBeen" class="pageitem" runat="server"> <asp:PlaceHolder ID="itemPlaceholder" runat="server" /> </ul> </LayoutTemplate> <ItemTemplate> <li class="menu" runat="server"> <asp:LinkButton ID="lbPlaceIveBeen" runat="server" OnClick="lbPlaceIveBeen_Click" CssClass="name"> <asp:Label ID="lblPlace" runat="server" Text='<%#Eval("PlaceName") %>' /> at <asp:Label ID="lblTime" runat="server" Text='<%#Eval("ATTime") %>' CssClass="content" /> <asp:HiddenField ID="hdATID" runat="server" Value='<%#Eval("ATID") %>' /> <span class="arrow"></span> </asp:LinkButton> </li> </ItemTemplate> </asp:ListView> </div> </asp:Panel> <asp:Panel runat="server" ID="pnlPlaceIveBeen" Visible="false"> <div id="topbar"> <div id="title"> I've been here </div> <div id="leftbutton"> <asp:LinkButton ID="lbPlaceIveBeenBack" runat="server" Text="Back" OnClick="lbPlaceIveBeenBack_Click" /> </div> <div id="rightbutton"> <asp:LinkButton ID="lbPlaceIveBeenBeginning" runat="server" Text="Beginning" OnClick="lbPlaceIveBeenBeginning_Click" /> </div> </div> <div id="content"> <ul class="pageitem"> <li> <asp:HiddenField ID="hdPlaceIveBeenPlaceId" runat="server" /> <asp:HiddenField ID="hdPlaceIveBeenLatitude" runat="server" /> <asp:HiddenField ID="hdPlaceIveBeenLongitude" runat="server" /> <asp:Label ID="lblPlaceIveBeenName" runat="server" /><br /> <asp:Label ID="lblPlaceIveBeenAddress" runat="server" /><br /> <asp:Label ID="lblPlaceIveBeenCity" runat="server" />, <asp:Label ID="lblPlaceIveBeenState" runat="server" /> <asp:Label ID="lblPlaceIveBeenZipCode" runat="server" /><br /> <asp:Label ID="lblPlaceIveBeenCountry" runat="server" /><br /> <div id="divPlaceIveBeen" style="width:300px; height:300px"></div> </li> </ul> </div> </asp:Panel> <asp:Panel ID="pnlAddPlace" runat="server" Visible="false"> <div id="topbar"><div id="title">MapSplore</div><div id="leftbutton"><asp:LinkButton ID="lbAddPlaceReturn" runat="server" Text="Back" OnClick="lbAddPlaceReturn_Click" /></div><div id="rightnav"></div></div><div id="content"> <ul class="pageitem"> <li id="liPlaceAddMessage" runat="server" visible="false"> <asp:Label ID="PlaceAddMessage" runat="server" /> </li> <li class="bigfield"> <asp:TextBox ID="txtPlaceName" runat="server" placeholder="Name of Establishment" /> </li> <li class="bigfield"> <asp:TextBox ID="txtAddress1" runat="server" placeholder="Address 1" /> </li> <li class="bigfield"> <asp:TextBox ID="txtCity" runat="server" placeholder="City" /> </li> <li class="select"> <asp:DropDownList ID="ddlProvince" runat="server" placeholder="Select State" /> <span class="arrow"></span> </li> <li class="bigfield"> <asp:TextBox ID="txtZipCode" runat="server" placeholder="Zip Code" /> </li> <li class="select"> <asp:DropDownList ID="ddlCountry" runat="server" onselectedindexchanged="ddlCountry_SelectedIndexChanged" /> <span class="arrow"></span> </li> <li class="bigfield"> <asp:TextBox ID="txtPhoneNumber" runat="server" placeholder="Phone Number" /> </li> <li class="checkbox"> <span class="name">You Here Now:</span> <asp:CheckBox ID="cbYouHereNow" runat="server" Checked="true" /> </li> <li class="button"> <asp:Button ID="btnAdd" runat="server" Text="Add Place" onclick="btnAdd_Click" /> </li> </ul></div> </asp:Panel> <asp:Panel ID="pnlImHere" runat="server" Visible="false"> <asp:TextBox ID="txtImHere" runat="server" TextMode="MultiLine" Rows="3" Columns="40" /><br /> <asp:DropDownList ID="ddlPlace" runat="server" /><br /> <asp:Button ID="btnHere" runat="server" Text="Tell Everyone I'm Here" onclick="btnHere_Click" /><br /> </asp:Panel> </div> </ContentTemplate> </asp:UpdatePanel> </asp:Content> Code Behind .cs file: using System;using System.Collections.Generic;using System.Linq;using System.Web;using System.Web.Security;using System.Web.UI;using System.Web.UI.HtmlControls;using System.Web.UI.WebControls;using LocationDataModel; public partial class AT_iPhone_Default : ViewStatePage{ private iPhoneDevice ipd; protected void Page_Load(object sender, EventArgs e) { LocationDataEntities lde = new LocationDataEntities(); if (!Page.IsPostBack) { var Countries = from c in lde.Countries select c; foreach (Country co in Countries) { ddlCountry.Items.Add(new ListItem(co.Name, co.CountryId.ToString())); } ddlCountry_SelectedIndexChanged(ddlCountry, null); if (AppleIPhone.IsIPad()) ipd = iPhoneDevice.iPad; if (AppleIPhone.IsIPhone()) ipd = iPhoneDevice.iPhone; if (AppleIPhone.IsIPodTouch()) ipd = iPhoneDevice.iPodTouch; } } protected void btnPlaces_Click(object sender, EventArgs e) { } protected void btnAdd_Click(object sender, EventArgs e) { bool blImHere = cbYouHereNow.Checked; string Place = txtPlaceName.Text, Address1 = txtAddress1.Text, City = txtCity.Text, ZipCode = txtZipCode.Text, PhoneNumber = txtPhoneNumber.Text, ProvinceId = ddlProvince.SelectedItem.Value, CountryId = ddlCountry.SelectedItem.Value; int iProvinceId, iCountryId; double dLatitude, dLongitude; DataAccess da = new DataAccess(); if ((!String.IsNullOrEmpty(ProvinceId)) && (!String.IsNullOrEmpty(CountryId))) { iProvinceId = Convert.ToInt32(ProvinceId); iCountryId = Convert.ToInt32(CountryId); if (blImHere) { dLatitude = Convert.ToDouble(Latitude.Value); dLongitude = Convert.ToDouble(Longitude.Value); da.StorePlace(Place, Address1, String.Empty, City, iProvinceId, ZipCode, iCountryId, PhoneNumber, dLatitude, dLongitude); } else { da.StorePlace(Place, Address1, String.Empty, City, iProvinceId, ZipCode, iCountryId, PhoneNumber); } liPlaceAddMessage.Visible = true; PlaceAddMessage.Text = "Awesome, your place has been added. Add Another!"; txtPlaceName.Text = String.Empty; txtAddress1.Text = String.Empty; txtCity.Text = String.Empty; ddlProvince.SelectedIndex = -1; txtZipCode.Text = String.Empty; txtPhoneNumber.Text = String.Empty; } else { liPlaceAddMessage.Visible = true; PlaceAddMessage.Text = "Please select a State and a Country."; } } protected void ddlCountry_SelectedIndexChanged(object sender, EventArgs e) { string CountryId = ddlCountry.SelectedItem.Value; if (!String.IsNullOrEmpty(CountryId)) { int iCountryId = Convert.ToInt32(CountryId); LocationDataModel.LocationDataEntities lde = new LocationDataModel.LocationDataEntities(); var prov = from p in lde.Provinces where p.CountryId == iCountryId orderby p.ProvinceName select p; ddlProvince.Items.Add(String.Empty); foreach (Province pr in prov) { ddlProvince.Items.Add(new ListItem(pr.ProvinceName, pr.ProvinceId.ToString())); } } else { ddlProvince.Items.Clear(); } } protected void btnImHere_Click(object sender, EventArgs e) { int i = 0; DataAccess da = new DataAccess(); double Lat = Convert.ToDouble(Latitude.Value), Lon = Convert.ToDouble(Longitude.Value); List<Place> lp = da.NearByLocations(Lat, Lon); foreach (Place p in lp) { ListItem li = new ListItem(p.Name, p.PlaceId.ToString()); if (i == 0) { li.Selected = true; } ddlPlace.Items.Add(li); i++; } pnlAddPlace.Visible = false; pnlImHere.Visible = true; } protected void lbImHere_Click(object sender, EventArgs e) { string UserName = Membership.GetUser().UserName; ListViewItem lvi = (ListViewItem)(((LinkButton)sender).Parent); HiddenField hd = (HiddenField)lvi.FindControl("hdPlaceId"); long PlaceId = Convert.ToInt64(hd.Value); double dLatitude = Convert.ToDouble(Latitude.Value); double dLongitude = Convert.ToDouble(Longitude.Value); DataAccess da = new DataAccess(); Place pl = da.GetPlace(PlaceId); pnlImHereNow.Visible = true; pnlPlaces.Visible = false; hdPlaceId.Value = PlaceId.ToString(); hdPlaceLat.Value = pl.Latitude.ToString(); hdPlaceLon.Value = pl.Longitude.ToString(); hdPlaceTitle.Value = pl.Name; lblPlaceTitle.Text = pl.Name; } protected void btnHere_Click(object sender, EventArgs e) { string UserName = Membership.GetUser().UserName; string WhatsH = txtImHere.Text; long PlaceId = Convert.ToInt64(ddlPlace.SelectedValue); double dLatitude = Convert.ToDouble(Latitude.Value); double dLongitude = Convert.ToDouble(Longitude.Value); DataAccess da = new DataAccess(); da.StoreUserAT(UserName, PlaceId, WhatsH, dLatitude, dLongitude); } protected void btnLocalCoupons_Click(object sender, EventArgs e) { double dLatitude = Convert.ToDouble(Latitude.Value); double dLongitude = Convert.ToDouble(Longitude.Value); DataAccess da = new DataAccess(); } protected void lbBusiness_Click(object sender, EventArgs e) { string UserName = Membership.GetUser().UserName; GridViewRow gvr = (GridViewRow)(((LinkButton)sender).Parent.Parent); HiddenField hd = (HiddenField)gvr.FindControl("hdPlaceId"); string sPlaceId = hd.Value; Int64 PlaceId; if (!String.IsNullOrEmpty(sPlaceId)) { PlaceId = Convert.ToInt64(sPlaceId); } } protected void lbLocalDeals_Click(object sender, EventArgs e) { double dLatitude = Convert.ToDouble(Latitude.Value); double dLongitude = Convert.ToDouble(Longitude.Value); DataAccess da = new DataAccess(); pnlCoupons.Visible = true; pnlStart.Visible = false; List<GeoPromotion> lgp = da.NearByDeals(dLatitude, dLongitude); lvCoupons.DataSource = lgp; lvCoupons.DataBind(); } protected void lbLocalPlaces_Click(object sender, EventArgs e) { DataAccess da = new DataAccess(); double Lat = Convert.ToDouble(Latitude.Value); double Lon = Convert.ToDouble(Longitude.Value); List<LocationDataModel.Place> places = da.NearByLocations(Lat, Lon); lvPlaces.DataSource = places; lvPlaces.SelectedIndex = -1; lvPlaces.DataBind(); pnlPlaces.Visible = true; pnlStart.Visible = false; } protected void ReturnFromPlaces_Click(object sender, EventArgs e) { pnlPlaces.Visible = false; pnlStart.Visible = true; } protected void ReturnFromDeals_Click(object sender, EventArgs e) { pnlCoupons.Visible = false; pnlStart.Visible = true; } protected void btnImHereNow_Click(object sender, EventArgs e) { long PlaceId = Convert.ToInt32(hdPlaceId.Value); string UserName = Membership.GetUser().UserName; string WhatsHappening = txtWhatsHappening.Text; double UserLat = Convert.ToDouble(Latitude.Value); double UserLon = Convert.ToDouble(Longitude.Value); DataAccess da = new DataAccess(); da.StoreUserAT(UserName, PlaceId, WhatsHappening, UserLat, UserLon); } protected void lbImHereNowReturn_Click(object sender, EventArgs e) { pnlImHereNow.Visible = false; pnlPlaces.Visible = true; } protected void lbBackToBeginning_Click(object sender, EventArgs e) { pnlStart.Visible = true; pnlImHereNow.Visible = false; } protected void lbWhereIveBeen_Click(object sender, EventArgs e) { string UserName = Membership.GetUser().UserName; pnlStart.Visible = false; pnlIveBeenHere.Visible = true; DataAccess da = new DataAccess(); lvWhereIveBeen.DataSource = da.UserATs(UserName, 0, 15); lvWhereIveBeen.DataBind(); } protected void lbIveBeenHereBack_Click(object sender, EventArgs e) { pnlIveBeenHere.Visible = false; pnlStart.Visible = true; } protected void lbPlaceIveBeen_Click(object sender, EventArgs e) { LinkButton lb = (LinkButton)sender; ListViewItem lvi = (ListViewItem)lb.Parent.Parent; HiddenField hdATID = (HiddenField)lvi.FindControl("hdATID"); Int64 ATID = Convert.ToInt64(hdATID.Value); DataAccess da = new DataAccess(); pnlIveBeenHere.Visible = false; pnlPlaceIveBeen.Visible = true; var plac = da.GetPlaceViaATID(ATID); hdPlaceIveBeenPlaceId.Value = plac.PlaceId.ToString(); hdPlaceIveBeenLatitude.Value = plac.Latitude.ToString(); hdPlaceIveBeenLongitude.Value = plac.Longitude.ToString(); lblPlaceIveBeenName.Text = plac.Name; lblPlaceIveBeenAddress.Text = plac.Address1; lblPlaceIveBeenCity.Text = plac.City; lblPlaceIveBeenState.Text = plac.Province.ProvinceName; lblPlaceIveBeenZipCode.Text = plac.ZipCode; lblPlaceIveBeenCountry.Text = plac.Country.Name; } protected void lbNotListed_Click(object sender, EventArgs e) { SetupAddPoint(); pnlPlaces.Visible = false; } protected void lbAddAPlace_Click(object sender, EventArgs e) { SetupAddPoint(); } private void SetupAddPoint() { double lat = Convert.ToDouble(Latitude.Value); double lon = Convert.ToDouble(Longitude.Value); DataAccess da = new DataAccess(); var zip = da.WhereAmIAt(lat, lon); if (zip.Count > 0) { var z0 = zip[0]; txtCity.Text = z0.City; txtZipCode.Text = z0.ZipCode; ddlProvince.ClearSelection(); if (z0.ProvinceId.HasValue == true) { foreach (ListItem li in ddlProvince.Items) { if (li.Value == z0.ProvinceId.Value.ToString()) { li.Selected = true; break; } } } } pnlAddPlace.Visible = true; pnlStart.Visible = false; } protected void lbAddPlaceReturn_Click(object sender, EventArgs e) { pnlAddPlace.Visible = false; pnlStart.Visible = true; liPlaceAddMessage.Visible = false; PlaceAddMessage.Text = String.Empty; } protected void lbPlaceIveBeenBack_Click(object sender, EventArgs e) { pnlIveBeenHere.Visible = true; pnlPlaceIveBeen.Visible = false; } protected void lbPlaceIveBeenBeginning_Click(object sender, EventArgs e) { pnlPlaceIveBeen.Visible = false; pnlStart.Visible = true; } protected string DisplayName(object val) { string strVal = Convert.ToString(val); if (AppleIPhone.IsIPad()) { ipd = iPhoneDevice.iPad; } if (AppleIPhone.IsIPhone()) { ipd = iPhoneDevice.iPhone; } if (AppleIPhone.IsIPodTouch()) { ipd = iPhoneDevice.iPodTouch; } return (iPhoneHelper.DisplayContentOnMenu(strVal, ipd)); }} iPhoneHelper.cs file: using System;using System.Collections.Generic;using System.Linq;using System.Web; public enum iPhoneDevice{ iPhone, iPodTouch, iPad}/// <summary>/// Summary description for iPhoneHelper/// </summary>/// public class iPhoneHelper{ public iPhoneHelper() { // // TODO: Add constructor logic here // } // This code is stupid in retrospect. Use css to solve this problem public static string DisplayContentOnMenu(string val, iPhoneDevice ipd) { string Return = val; string Elipsis = "..."; int iPadMaxLength = 30; int iPhoneMaxLength = 15; if (ipd == iPhoneDevice.iPad) { if (Return.Length > iPadMaxLength) { Return = Return.Substring(0, iPadMaxLength - Elipsis.Length) + Elipsis; } } else { if (Return.Length > iPhoneMaxLength) { Return = Return.Substring(0, iPhoneMaxLength - Elipsis.Length) + Elipsis; } } return (Return); }} Source code for the ViewStatePage: using System;using System.Data;using System.Data.SqlClient;using System.Configuration;using System.Web;using System.Web.Security;using System.Web.UI;using System.Web.UI.WebControls;using System.Web.UI.WebControls.WebParts;using System.Web.UI.HtmlControls; /// <summary>/// Summary description for BasePage/// </summary>#region Base class for a page.public class ViewStatePage : System.Web.UI.Page{ PageStatePersisterToDatabase myPageStatePersister; public ViewStatePage() : base() { myPageStatePersister = new PageStatePersisterToDatabase(this); } protected override PageStatePersister PageStatePersister { get { return myPageStatePersister; } } }#endregion #region This class will override the page persistence to store page state in a database.public class PageStatePersisterToDatabase : PageStatePersister{ private string ViewStateKeyField = "__VIEWSTATE_KEY"; private string _exNoConnectionStringFound = "No Database Configuration information is in the web.config."; public PageStatePersisterToDatabase(Page page) : base(page) { } public override void Load() { // Get the cache key from the web form data System.Int64 key = Convert.ToInt64(Page.Request.Params[ViewStateKeyField]); Pair state = this.LoadState(key); // Abort if cache object is not of type Pair if (state == null) throw new ApplicationException("Missing valid " + ViewStateKeyField); // Set view state and control state ViewState = state.First; ControlState = state.Second; } public override void Save() { // No processing needed if no states available if (ViewState == null && ControlState != null) return; System.Int64 key; IStateFormatter formatter = this.StateFormatter; Pair statePair = new Pair(ViewState, ControlState); // Serialize the statePair object to a string. string serializedState = formatter.Serialize(statePair); // Save the ViewState and get a unique identifier back. key = SaveState(serializedState); // Register hidden field to store cache key in // Page.ClientScript does not work properly with Atlas. //Page.ClientScript.RegisterHiddenField(ViewStateKeyField, key.ToString()); ScriptManager.RegisterHiddenField(this.Page, ViewStateKeyField, key.ToString()); } private System.Int64 SaveState(string PageState) { System.Int64 i64Key = 0; string strConn = String.Empty, strProvider = String.Empty; string strSql = "insert into tblPageState ( SerializedState ) values ( '" + SqlEscape(PageState) + "');select scope_identity();"; SqlConnection sqlCn; SqlCommand sqlCm; try { GetDBConnectionString(ref strConn, ref strProvider); sqlCn = new SqlConnection(strConn); sqlCm = new SqlCommand(strSql, sqlCn); sqlCn.Open(); i64Key = Convert.ToInt64(sqlCm.ExecuteScalar()); if (sqlCn.State != ConnectionState.Closed) { sqlCn.Close(); } sqlCn.Dispose(); sqlCm.Dispose(); } finally { sqlCn = null; sqlCm = null; } return i64Key; } private Pair LoadState(System.Int64 iKey) { string strConn = String.Empty, strProvider = String.Empty, SerializedState = String.Empty, strMinutesInPast = GetMinutesInPastToDelete(); Pair PageState; string strSql = "select SerializedState from tblPageState where tblPageStateID=" + iKey.ToString() + ";" + "delete from tblPageState where DateUpdated<DateAdd(mi, " + strMinutesInPast + ", getdate());"; SqlConnection sqlCn; SqlCommand sqlCm; try { GetDBConnectionString(ref strConn, ref strProvider); sqlCn = new SqlConnection(strConn); sqlCm = new SqlCommand(strSql, sqlCn); sqlCn.Open(); SerializedState = Convert.ToString(sqlCm.ExecuteScalar()); IStateFormatter formatter = this.StateFormatter; if ((null == SerializedState) || (String.Empty == SerializedState)) { throw (new ApplicationException("No ViewState records were returned.")); } // Deserilize returns the Pair object that is serialized in // the Save method. PageState = (Pair)formatter.Deserialize(SerializedState); if (sqlCn.State != ConnectionState.Closed) { sqlCn.Close(); } sqlCn.Dispose(); sqlCm.Dispose(); } finally { sqlCn = null; sqlCm = null; } return PageState; } private string SqlEscape(string Val) { string ReturnVal = String.Empty; if (null != Val) { ReturnVal = Val.Replace("'", "''"); } return (ReturnVal); } private void GetDBConnectionString(ref string ConnectionStringValue, ref string ProviderNameValue) { if (System.Configuration.ConfigurationManager.ConnectionStrings.Count > 0) { ConnectionStringValue = System.Configuration.ConfigurationManager.ConnectionStrings["ApplicationServices"].ConnectionString; ProviderNameValue = System.Configuration.ConfigurationManager.ConnectionStrings["ApplicationServices"].ProviderName; } else { throw new ConfigurationErrorsException(_exNoConnectionStringFound); } } private string GetMinutesInPastToDelete() { string strReturn = "-60"; if (null != System.Configuration.ConfigurationManager.AppSettings["MinutesInPastToDeletePageState"]) { strReturn = System.Configuration.ConfigurationManager.AppSettings["MinutesInPastToDeletePageState"].ToString(); } return (strReturn); }}#endregion AppleiPhone.cs file: using System;using System.Collections.Generic;using System.Linq;using System.Web; /// <summary>/// Summary description for AppleIPhone/// </summary>public class AppleIPhone{ public AppleIPhone() { // // TODO: Add constructor logic here // } static public bool IsIPhoneOS() { return (IsIPad() || IsIPhone() || IsIPodTouch()); } static public bool IsIPhone() { return IsTest("iPhone"); } static public bool IsIPodTouch() { return IsTest("iPod"); } static public bool IsIPad() { return IsTest("iPad"); } static private bool IsTest(string Agent) { bool bl = false; string ua = HttpContext.Current.Request.UserAgent.ToLower(); try { bl = ua.Contains(Agent.ToLower()); } catch { } return (bl); }} Master page .cs: using System;using System.Collections.Generic;using System.Linq;using System.Web;using System.Web.UI;using System.Web.UI.HtmlControls;using System.Web.UI.WebControls; public partial class MasterPages_iPhoneMaster : System.Web.UI.MasterPage{ protected void Page_Load(object sender, EventArgs e) { HtmlHead head = Page.Header; HtmlMeta meta = new HtmlMeta(); if (AppleIPhone.IsIPad() == true) { meta.Content = "width=400,user-scalable=no"; head.Controls.Add(meta); } else { meta.Content = "width=device-width, user-scalable=no"; meta.Attributes.Add("name", "viewport"); } meta.Attributes.Add("name", "viewport"); head.Controls.Add(meta); HtmlLink cssLink = new HtmlLink(); HtmlGenericControl script = new HtmlGenericControl("script"); script.Attributes.Add("type", "text/javascript"); script.Attributes.Add("src", ResolveUrl("~/Scripts/iWebKit/javascript/functions.js")); head.Controls.Add(script); cssLink.Attributes.Add("rel", "stylesheet"); cssLink.Attributes.Add("href", ResolveUrl("~/Scripts/iWebKit/css/style.css") ); cssLink.Attributes.Add("type", "text/css"); head.Controls.Add(cssLink); HtmlGenericControl jsLink = new HtmlGenericControl("script"); //jsLink.Attributes.Add("type", "text/javascript"); //jsLink.Attributes.Add("src", ResolveUrl("~/Scripts/jquery-1.4.1.min.js") ); //head.Controls.Add(jsLink); HtmlLink appleIcon = new HtmlLink(); appleIcon.Attributes.Add("rel", "apple-touch-icon"); appleIcon.Attributes.Add("href", ResolveUrl("~/apple-touch-icon.png")); HtmlMeta appleMobileWebAppStatusBarStyle = new HtmlMeta(); appleMobileWebAppStatusBarStyle.Attributes.Add("name", "apple-mobile-web-app-status-bar-style"); appleMobileWebAppStatusBarStyle.Attributes.Add("content", "black"); head.Controls.Add(appleMobileWebAppStatusBarStyle); } internal string FindPath(string Location) { string Url = Server.MapPath(Location); return (Url); }}

Read the article

Send regular keyboard samples OR keyboard state changes over network

- by Ciaran

Building a multi player asteroids game where ships compete with each other. Using UDP. Wanted to minimize traffic sent to server. Which would you do: Send periodic keyboard state samples every from client every to match server physics update rate e.g. 50 times per second. Highly resilient to packet loss and other reliabilty problems. Out of date packets disacarded by server. Generates a lot of unnuecessary traffic. Only send keyboard state when it changes (key up, key down). Radically less traffic sent from client to server. However, UDP can lose packets without you being informed. So the latter method could result in the vital packet never being resent unless I detect and resend this in a timely manner.

Read the article

How to Use Windows’ Advanced Search Features: Everything You Need to Know

- by Chris Hoffman

You should never have to hunt down a lost file on modern versions of Windows — just perform a quick search. You don’t even have to wait for a cartoon dog to find your files, like on Windows XP. The Windows search indexer is constantly running in the background to make quick local searches possible. This enables the kind of powerful search features you’d use on Google or Bing — but for your local files. Controlling the Indexer By default, the Windows search indexer watches everything under your user folder — that’s C:\Users\NAME. It reads all these files, creating an index of their names, contents, and other metadata. Whenever they change, it notices and updates its index. The index allows you to quickly find a file based on the data in the index. For example, if you want to find files that contain the word “beluga,” you can perform a search for “beluga” and you’ll get a very quick response as Windows looks up the word in its search index. If Windows didn’t use an index, you’d have to sit and wait as Windows opened every file on your hard drive, looked to see if the file contained the word “beluga,” and moved on. Most people shouldn’t have to modify this indexing behavior. However, if you store your important files in other folders — maybe you store your important data a separate partition or drive, such as at D:\Data — you may want to add these folders to your index. You can also choose which types of files you want to index, force Windows to rebuild the index entirely, pause the indexing process so it won’t use any system resources, or move the index to another location to save space on your system drive. To open the Indexing Options window, tap the Windows key on your keyboard, type “index”, and click the Indexing Options shortcut that appears. Use the Modify button to control the folders that Windows indexes or the Advanced button to control other options. To prevent Windows from indexing entirely, click the Modify button and uncheck all the included locations. You could also disable the search indexer entirely from the Programs and Features window. Searching for Files You can search for files right from your Start menu on Windows 7 or Start screen on Windows 8. Just tap the Windows key and perform a search. If you wanted to find files related to Windows, you could perform a search for “Windows.” Windows would show you files that are named Windows or contain the word Windows. From here, you can just click a file to open it. On Windows 7, files are mixed with other types of search results. On Windows 8 or 8.1, you can choose to search only for files. If you want to perform a search without leaving the desktop in Windows 8.1, press Windows Key + S to open a search sidebar. You can also initiate searches directly from Windows Explorer — that’s File Explorer on Windows 8. Just use the search box at the top-right of the window. Windows will search the location you’ve browsed to. For example, if you’re looking for a file related to Windows and know it’s somewhere in your Documents library, open the Documents library and search for Windows. Using Advanced Search Operators On Windows 7, you’ll notice that you can add “search filters” form the search box, allowing you to search by size, date modified, file type, authors, and other metadata. On Windows 8, these options are available from the Search Tools tab on the ribbon. These filters allow you to narrow your search results. If you’re a geek, you can use Windows’ Advanced Query Syntax to perform advanced searches from anywhere, including the Start menu or Start screen. Want to search for “windows,” but only bring up documents that don’t mention Microsoft? Search for “windows -microsoft”. Want to search for all pictures of penguins on your computer, whether they’re PNGs, JPEGs, or any other type of picture file? Search for “penguin kind:picture”. We’ve looked at Windows’ advanced search operators before, so check out our in-depth guide for more information. The Advanced Query Syntax gives you access to options that aren’t available in the graphical interface. Creating Saved Searches Windows allows you to take searches you’ve made and save them as a file. You can then quickly perform the search later by double-clicking the file. The file functions almost like a virtual folder that contains the files you specify. For example, let’s say you wanted to create a saved search that shows you all the new files created in your indexed folders within the last week. You could perform a search for “datecreated:this week”, then click the Save search button on the toolbar or ribbon. You’d have a new virtual folder you could quickly check to see your recent files. One of the best things about Windows search is that it’s available entirely from the keyboard. Just press the Windows key, start typing the name of the file or program you want to open, and press Enter to quickly open it. Windows 8 made this much more obnoxious with its non-unified search, but unified search is finally returning with Windows 8.1.

Read the article

Rendering ASP.NET Script References into the Html Header

- by Rick Strahl

One thing that I’ve come to appreciate in control development in ASP.NET that use JavaScript is the ability to have more control over script and script include placement than ASP.NET provides natively. Specifically in ASP.NET you can use either the ClientScriptManager or ScriptManager to embed scripts and script references into pages via code. This works reasonably well, but the script references that get generated are generated into the HTML body and there’s very little operational control for placement of scripts. If you have multiple controls or several of the same control that need to place the same scripts onto the page it’s not difficult to end up with scripts that render in the wrong order and stop working correctly. This is especially critical if you load script libraries with dependencies either via resources or even if you are rendering referenced to CDN resources. Natively ASP.NET provides a host of methods that help embedding scripts into the page via either Page.ClientScript or the ASP.NET ScriptManager control (both with slightly different syntax): RegisterClientScriptBlock Renders a script block at the top of the HTML body and should be used for embedding callable functions/classes. RegisterStartupScript Renders a script block just prior to the </form> tag and should be used to for embedding code that should execute when the page is first loaded. Not recommended – use jQuery.ready() or equivalent load time routines. RegisterClientScriptInclude Embeds a reference to a script from a url into the page. RegisterClientScriptResource Embeds a reference to a Script from a resource file generating a long resource file string All 4 of these methods render their <script> tags into the HTML body. The script blocks give you a little bit of control by having a ‘top’ and ‘bottom’ of the document location which gives you some flexibility over script placement and precedence. Script includes and resource url unfortunately do not even get that much control – references are simply rendered into the page in the order of declaration. The ASP.NET ScriptManager control facilitates this task a little bit with the abililty to specify scripts in code and the ability to programmatically check what scripts have already been registered, but it doesn’t provide any more control over the script rendering process itself. Further the ScriptManager is a bear to deal with generically because generic code has to always check and see if it is actually present. Some time ago I posted a ClientScriptProxy class that helps with managing the latter process of sending script references either to ClientScript or ScriptManager if it’s available. Since I last posted about this there have been a number of improvements in this API, one of which is the ability to control placement of scripts and script includes in the page which I think is rather important and a missing feature in the ASP.NET native functionality. Handling ScriptRenderModes One of the big enhancements that I’ve come to rely on is the ability of the various script rendering functions described above to support rendering in multiple locations: /// <summary> /// Determines how scripts are included into the page /// </summary> public enum ScriptRenderModes { /// <summary> /// Inherits the setting from the control or from the ClientScript.DefaultScriptRenderMode /// </summary> Inherit, /// Renders the script include at the location of the control /// </summary> Inline, /// <summary> /// Renders the script include into the bottom of the header of the page /// </summary> Header, /// <summary> /// Renders the script include into the top of the header of the page /// </summary> HeaderTop, /// <summary> /// Uses ClientScript or ScriptManager to embed the script include to /// provide standard ASP.NET style rendering in the HTML body. /// </summary> Script, /// <summary> /// Renders script at the bottom of the page before the last Page.Controls /// literal control. Note this may result in unexpected behavior /// if /body and /html are not the last thing in the markup page. /// </summary> BottomOfPage } This enum is then applied to the various Register functions to allow more control over where scripts actually show up. Why is this useful? For me I often render scripts out of control resources and these scripts often include things like a JavaScript Library (jquery) and a few plug-ins. The order in which these can be loaded is critical so that jQuery.js always loads before any plug-in for example. Typically I end up with a general script layout like this: Core Libraries- HeaderTop Plug-ins: Header ScriptBlocks: Header or Script depending on other dependencies There’s also an option to render scripts and CSS at the very bottom of the page before the last Page control on the page which can be useful for speeding up page load when lots of scripts are loaded. The API syntax of the ClientScriptProxy methods is closely compatible with ScriptManager’s using static methods and control references to gain access to the page and embedding scripts. For example, to render some script into the current page in the header: // Create script block in header ClientScriptProxy.Current.RegisterClientScriptBlock(this, typeof(ControlResources), "hello_function", "function helloWorld() { alert('hello'); }", true, ScriptRenderModes.Header); // Same again - shouldn't be rendered because it's the same id ClientScriptProxy.Current.RegisterClientScriptBlock(this, typeof(ControlResources), "hello_function", "function helloWorld() { alert('hello'); }", true, ScriptRenderModes.Header); // Create a second script block in header ClientScriptProxy.Current.RegisterClientScriptBlock(this, typeof(ControlResources), "hello_function2", "function helloWorld2() { alert('hello2'); }", true, ScriptRenderModes.Header); // This just calls ClientScript and renders into bottom of document ClientScriptProxy.Current.RegisterStartupScript(this,typeof(ControlResources), "call_hello", "helloWorld();helloWorld2();", true); which generates: <html xmlns="http://www.w3.org/1999/xhtml" > <head><title> </title> <script type="text/javascript"> function helloWorld() { alert('hello'); } </script> <script type="text/javascript"> function helloWorld2() { alert('hello2'); } </script> </head> <body> … <script type="text/javascript"> //<![CDATA[ helloWorld();helloWorld2();//]]> </script> </form> </body> </html> Note that the scripts are generated into the header rather than the body except for the last script block which is the call to RegisterStartupScript. In general I wouldn’t recommend using RegisterStartupScript – ever. It’s a much better practice to use a script base load event to handle ‘startup’ code that should fire when the page first loads. So instead of the code above I’d actually recommend doing: ClientScriptProxy.Current.RegisterClientScriptBlock(this, typeof(ControlResources), "call_hello", "$().ready( function() { alert('hello2'); });", true, ScriptRenderModes.Header); assuming you’re using jQuery on the page. For script includes from a Url the following demonstrates how to embed scripts into the header. This example injects a jQuery and jQuery.UI script reference from the Google CDN then checks each with a script block to ensure that it has loaded and if not loads it from a server local location: // load jquery from CDN ClientScriptProxy.Current.RegisterClientScriptInclude(this, typeof(ControlResources), "http://ajax.googleapis.com/ajax/libs/jquery/1.3.2/jquery.min.js", ScriptRenderModes.HeaderTop); // check if jquery loaded - if it didn't we're not online string scriptCheck = @"if (typeof jQuery != 'object') document.write(unescape(""%3Cscript src='{0}' type='text/javascript'%3E%3C/script%3E""));"; string jQueryUrl = ClientScriptProxy.Current.GetWebResourceUrl(this, typeof(ControlResources), ControlResources.JQUERY_SCRIPT_RESOURCE); ClientScriptProxy.Current.RegisterClientScriptBlock(this, typeof(ControlResources), "jquery_register", string.Format(scriptCheck,jQueryUrl),true, ScriptRenderModes.HeaderTop); // Load jquery-ui from cdn ClientScriptProxy.Current.RegisterClientScriptInclude(this, typeof(ControlResources), "http://ajax.googleapis.com/ajax/libs/jqueryui/1.7.2/jquery-ui.min.js", ScriptRenderModes.Header); // check if we need to load from local string jQueryUiUrl = ResolveUrl("~/scripts/jquery-ui-custom.min.js"); ClientScriptProxy.Current.RegisterClientScriptBlock(this, typeof(ControlResources), "jqueryui_register", string.Format(scriptCheck, jQueryUiUrl), true, ScriptRenderModes.Header); // Create script block in header ClientScriptProxy.Current.RegisterClientScriptBlock(this, typeof(ControlResources), "hello_function", "$().ready( function() { alert('hello'); });", true, ScriptRenderModes.Header); which in turn generates this HTML: <html xmlns="http://www.w3.org/1999/xhtml" > <head> <script src="http://ajax.googleapis.com/ajax/libs/jquery/1.3.2/jquery.min.js" type="text/javascript"></script> <script type="text/javascript"> if (typeof jQuery != 'object') document.write(unescape("%3Cscript src='/WestWindWebToolkitWeb/WebResource.axd?d=DIykvYhJ_oXCr-TA_dr35i4AayJoV1mgnQAQGPaZsoPM2LCdvoD3cIsRRitHKlKJfV5K_jQvylK7tsqO3lQIFw2&t=633979863959332352' type='text/javascript'%3E%3C/script%3E")); </script> <title> </title> <script src="http://ajax.googleapis.com/ajax/libs/jqueryui/1.7.2/jquery-ui.min.js" type="text/javascript"></script> <script type="text/javascript"> if (typeof jQuery != 'object') document.write(unescape("%3Cscript src='/WestWindWebToolkitWeb/scripts/jquery-ui-custom.min.js' type='text/javascript'%3E%3C/script%3E")); </script> <script type="text/javascript"> $().ready(function() { alert('hello'); }); </script> </head> <body> …</body> </html> As you can see there’s a bit more control in this process as you can inject both script includes and script blocks into the document at the top or bottom of the header, plus if necessary at the usual body locations. This is quite useful especially if you create custom server controls that interoperate with script and have certain dependencies. The above is a good example of a useful switchable routine where you can switch where scripts load from by default – the above pulls from Google CDN but a configuration switch may automatically switch to pull from the local development copies if your doing development for example. How does it work? As mentioned the ClientScriptProxy object mimicks many of the ScriptManager script related methods and so provides close API compatibility with it although it contains many additional overloads that enhance functionality. It does however work against ScriptManager if it’s available on the page, or Page.ClientScript if it’s not so it provides a single unified frontend to script access. There are however many overloads of the original SM methods like the above to provide additional functionality. The implementation of script header rendering is pretty straight forward – as long as a server header (ie. it has to have runat=”server” set) is available. Otherwise these routines fall back to using the default document level insertions of ScriptManager/ClientScript. Given that there is a server header it’s relatively easy to generate the script tags and code and append them to the header either at the top or bottom. I suspect Microsoft didn’t provide header rendering functionality precisely because a runat=”server” header is not required by ASP.NET so behavior would be slightly unpredictable. That’s not really a problem for a custom implementation however. Here’s the RegisterClientScriptBlock implementation that takes a ScriptRenderModes parameter to allow header rendering: /// <summary> /// Renders client script block with the option of rendering the script block in /// the Html header /// /// For this to work Header must be defined as runat="server" /// </summary> /// <param name="control">any control that instance typically page</param> /// <param name="type">Type that identifies this rendering</param> /// <param name="key">unique script block id</param> /// <param name="script">The script code to render</param> /// <param name="addScriptTags">Ignored for header rendering used for all other insertions</param> /// <param name="renderMode">Where the block is rendered</param> public void RegisterClientScriptBlock(Control control, Type type, string key, string script, bool addScriptTags, ScriptRenderModes renderMode) { if (renderMode == ScriptRenderModes.Inherit) renderMode = DefaultScriptRenderMode; if (control.Page.Header == null || renderMode != ScriptRenderModes.HeaderTop && renderMode != ScriptRenderModes.Header && renderMode != ScriptRenderModes.BottomOfPage) { RegisterClientScriptBlock(control, type, key, script, addScriptTags); return; } // No dupes - ref script include only once const string identifier = "scriptblock_"; if (HttpContext.Current.Items.Contains(identifier + key)) return; HttpContext.Current.Items.Add(identifier + key, string.Empty); StringBuilder sb = new StringBuilder(); // Embed in header sb.AppendLine("\r\n<script type=\"text/javascript\">"); sb.AppendLine(script); sb.AppendLine("</script>"); int? index = HttpContext.Current.Items["__ScriptResourceIndex"] as int?; if (index == null) index = 0; if (renderMode == ScriptRenderModes.HeaderTop) { control.Page.Header.Controls.AddAt(index.Value, new LiteralControl(sb.ToString())); index++; } else if(renderMode == ScriptRenderModes.Header) control.Page.Header.Controls.Add(new LiteralControl(sb.ToString())); else if (renderMode == ScriptRenderModes.BottomOfPage) control.Page.Controls.AddAt(control.Page.Controls.Count-1,new LiteralControl(sb.ToString())); HttpContext.Current.Items["__ScriptResourceIndex"] = index; } Note that the routine has to keep track of items inserted by id so that if the same item is added again with the same key it won’t generate two script entries. Additionally the code has to keep track of how many insertions have been made at the top of the document so that entries are added in the proper order. The RegisterScriptInclude method is similar but there’s some additional logic in here to deal with script file references and ClientScriptProxy’s (optional) custom resource handler that provides script compression /// <summary> /// Registers a client script reference into the page with the option to specify /// the script location in the page /// </summary> /// <param name="control">Any control instance - typically page</param> /// <param name="type">Type that acts as qualifier (uniqueness)</param> /// <param name="url">the Url to the script resource</param> /// <param name="ScriptRenderModes">Determines where the script is rendered</param> public void RegisterClientScriptInclude(Control control, Type type, string url, ScriptRenderModes renderMode) { const string STR_ScriptResourceIndex = "__ScriptResourceIndex"; if (string.IsNullOrEmpty(url)) return; if (renderMode == ScriptRenderModes.Inherit) renderMode = DefaultScriptRenderMode; // Extract just the script filename string fileId = null; // Check resource IDs and try to match to mapped file resources // Used to allow scripts not to be loaded more than once whether // embedded manually (script tag) or via resources with ClientScriptProxy if (url.Contains(".axd?r=")) { string res = HttpUtility.UrlDecode( StringUtils.ExtractString(url, "?r=", "&", false, true) ); foreach (ScriptResourceAlias item in ScriptResourceAliases) { if (item.Resource == res) { fileId = item.Alias + ".js"; break; } } if (fileId == null) fileId = url.ToLower(); } else fileId = Path.GetFileName(url).ToLower(); // No dupes - ref script include only once const string identifier = "script_"; if (HttpContext.Current.Items.Contains( identifier + fileId ) ) return; HttpContext.Current.Items.Add(identifier + fileId, string.Empty); // just use script manager or ClientScriptManager if (control.Page.Header == null || renderMode == ScriptRenderModes.Script || renderMode == ScriptRenderModes.Inline) { RegisterClientScriptInclude(control, type,url, url); return; } // Retrieve script index in header int? index = HttpContext.Current.Items[STR_ScriptResourceIndex] as int?; if (index == null) index = 0; StringBuilder sb = new StringBuilder(256); url = WebUtils.ResolveUrl(url); // Embed in header sb.AppendLine("\r\n<script src=\"" + url + "\" type=\"text/javascript\"></script>"); if (renderMode == ScriptRenderModes.HeaderTop) { control.Page.Header.Controls.AddAt(index.Value, new LiteralControl(sb.ToString())); index++; } else if (renderMode == ScriptRenderModes.Header) control.Page.Header.Controls.Add(new LiteralControl(sb.ToString())); else if (renderMode == ScriptRenderModes.BottomOfPage) control.Page.Controls.AddAt(control.Page.Controls.Count-1, new LiteralControl(sb.ToString())); HttpContext.Current.Items[STR_ScriptResourceIndex] = index; } There’s a little more code here that deals with cleaning up the passed in Url and also some custom handling of script resources that run through the ScriptCompressionModule – any script resources loaded in this fashion are automatically cached based on the resource id. Raw urls extract just the filename from the URL and cache based on that. All of this to avoid doubling up of scripts if called multiple times by multiple instances of the same control for example or several controls that all load the same resources/includes. Finally RegisterClientScriptResource utilizes the previous method to wrap the WebResourceUrl as well as some custom functionality for the resource compression module: /// <summary> /// Returns a WebResource or ScriptResource URL for script resources that are to be /// embedded as script includes. /// </summary> /// <param name="control">Any control</param> /// <param name="type">A type in assembly where resources are located</param> /// <param name="resourceName">Name of the resource to load</param> /// <param name="renderMode">Determines where in the document the link is rendered</param> public void RegisterClientScriptResource(Control control, Type type, string resourceName, ScriptRenderModes renderMode) { string resourceUrl = GetClientScriptResourceUrl(control, type, resourceName); RegisterClientScriptInclude(control, type, resourceUrl, renderMode); } /// <summary> /// Works like GetWebResourceUrl but can be used with javascript resources /// to allow using of resource compression (if the module is loaded). /// </summary> /// <param name="control"></param> /// <param name="type"></param> /// <param name="resourceName"></param> /// <returns></returns> public string GetClientScriptResourceUrl(Control control, Type type, string resourceName) { #if IncludeScriptCompressionModuleSupport // If wwScriptCompression Module through Web.config is loaded use it to compress // script resources by using wcSC.axd Url the module intercepts if (ScriptCompressionModule.ScriptCompressionModuleActive) { string url = "~/wwSC.axd?r=" + HttpUtility.UrlEncode(resourceName); if (type.Assembly != GetType().Assembly) url += "&t=" + HttpUtility.UrlEncode(type.FullName); return WebUtils.ResolveUrl(url); } #endif return control.Page.ClientScript.GetWebResourceUrl(type, resourceName); } This code merely retrieves the resource URL and then simply calls back to RegisterClientScriptInclude with the URL to be embedded which means there’s nothing specific to deal with other than the custom compression module logic which is nice and easy. What else is there in ClientScriptProxy? ClientscriptProxy also provides a few other useful services beyond what I’ve already covered here: Transparent ScriptManager and ClientScript calls ClientScriptProxy includes a host of routines that help figure out whether a script manager is available or not and all functions in this class call the appropriate object – ScriptManager or ClientScript – that is available in the current page to ensure that scripts get embedded into pages properly. This is especially useful for control development where controls have no control over the scripting environment in place on the page. RegisterCssLink and RegisterCssResource Much like the script embedding functions these two methods allow embedding of CSS links. CSS links are appended to the header or to a form declared with runat=”server”. LoadControlScript Is a high level resource loading routine that can be used to easily switch between different script linking modes. It supports loading from a WebResource, a url or not loading anything at all. This is very useful if you build controls that deal with specification of resource urls/ids in a standard way. Check out the full Code You can check out the full code to the ClientScriptProxyClass here: ClientScriptProxy.cs ClientScriptProxy Documentation (class reference) Note that the ClientScriptProxy has a few dependencies in the West Wind Web Toolkit of which it is part of. ControlResources holds a few standard constants and script resource links and the ScriptCompressionModule which is referenced in a few of the script inclusion methods. There’s also another useful ScriptContainer companion control to the ClientScriptProxy that allows scripts to be placed onto the page’s markup including the ability to specify the script location and script minification options. You can find all the dependencies in the West Wind Web Toolkit repository: West Wind Web Toolkit Repository West Wind Web Toolkit Home Page© Rick Strahl, West Wind Technologies, 2005-2010Posted in ASP.NET JavaScript

Read the article

C# 5 Async, Part 1: Simplifying Asynchrony – That for which we await

- by Reed

Today’s announcement at PDC of the future directions C# is taking excite me greatly. The new Visual Studio Async CTP is amazing. Asynchronous code – code which frustrates and demoralizes even the most advanced of developers, is taking a huge leap forward in terms of usability. This is handled by building on the Task functionality in .NET 4, as well as the addition of two new keywords being added to the C# language: async and await. This core of the new asynchronous functionality is built upon three key features. First is the Task functionality in .NET 4, and based on Task and Task<TResult>. While Task was intended to be the primary means of asynchronous programming with .NET 4, the .NET Framework was still based mainly on the Asynchronous Pattern and the Event-based Asynchronous Pattern. The .NET Framework added functionality and guidance for wrapping existing APIs into a Task based API, but the framework itself didn’t really adopt Task or Task<TResult> in any meaningful way. The CTP shows that, going forward, this is changing. One of the three key new features coming in C# is actually a .NET Framework feature. Nearly every asynchronous API in the .NET Framework has been wrapped into a new, Task-based method calls. In the CTP, this is done via as external assembly (AsyncCtpLibrary.dll) which uses Extension Methods to wrap the existing APIs. However, going forward, this will be handled directly within the Framework. This will have a unifying effect throughout the .NET Framework. This is the first building block of the new features for asynchronous programming: Going forward, all asynchronous operations will work via a method that returns Task or Task<TResult> The second key feature is the new async contextual keyword being added to the language. The async keyword is used to declare an asynchronous function, which is a method that either returns void, a Task, or a Task<T>. Inside the asynchronous function, there must be at least one await expression. This is a new C# keyword (await) that is used to automatically take a series of statements and break it up to potentially use discontinuous evaluation. This is done by using await on any expression that evaluates to a Task or Task<T>. For example, suppose we want to download a webpage as a string. There is a new method added to WebClient: Task<string> WebClient.DownloadStringTaskAsync(Uri). Since this returns a Task<string> we can use it within an asynchronous function. Suppose, for example, that we wanted to do something similar to my asynchronous Task example – download a web page asynchronously and check to see if it supports XHTML 1.0, then report this into a TextBox. This could be done like so: private async void button1_Click(object sender, RoutedEventArgs e) { string url = "http://reedcopsey.com"; string content = await new WebClient().DownloadStringTaskAsync(url); this.textBox1.Text = string.Format("Page {0} supports XHTML 1.0: {1}", url, content.Contains("XHTML 1.0")); } .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } Let’s walk through what’s happening here, step by step. By adding the async contextual keyword to the method definition, we are able to use the await keyword on our WebClient.DownloadStringTaskAsync method call. When the user clicks this button, the new method (Task<string> WebClient.DownloadStringTaskAsync(string)) is called, which returns a Task<string>. By adding the await keyword, the runtime will call this method that returns Task<string>, and execution will return to the caller at this point. This means that our UI is not blocked while the webpage is downloaded. Instead, the UI thread will “await” at this point, and let the WebClient do it’s thing asynchronously. When the WebClient finishes downloading the string, the user interface’s synchronization context will automatically be used to “pick up” where it left off, and the Task<string> returned from DownloadStringTaskAsync is automatically unwrapped and set into the content variable. At this point, we can use that and set our text box content. There are a couple of key points here: Asynchronous functions are declared with the async keyword, and contain one or more await expressions In addition to the obvious benefits of shorter, simpler code – there are some subtle but tremendous benefits in this approach. When the execution of this asynchronous function continues after the first await statement, the initial synchronization context is used to continue the execution of this function. That means that we don’t have to explicitly marshal the call that sets textbox1.Text back to the UI thread – it’s handled automatically by the language and framework! Exception handling around asynchronous method calls also just works. I’d recommend every C# developer take a look at the documentation on the new Asynchronous Programming for C# and Visual Basic page, download the Visual Studio Async CTP, and try it out.

Read the article

More Linux Tips and Tricks

we learn how easy it is to configure OpenSSH to use secure public key authentication instead of a password login.

Read the article

Interview with Tomas Ulin at the MySQL Innovation Day

- by Monica Kumar

MySQL Innovation Day held on June 5, 2012 was a great event for the MySQL engineers, users and customers to gather, share and network. I was able to get a few minutes with Tomas Ulin, Vice President of MySQL Engineering at Oracle, to ask him some questions. Here are the highlights of my interview with Tomas. Monica: This was the first MySQL Innovation Day, correct? Why now, what was the strategy behind hosting this kind of event? Tomas: In the last year, we have rolled out an incredible number of MySQL events worldwide – some targeted at developers that are new to MySQL and others for the MySQL savvy. At the MySQL Innovation Day, our first event of this kind,, we had a number of our key engineers presenting lightning talks delivering previews of key new features as well as discussing roadmap. Our goal is to keep an open dialogue with the MySQL community. In fact, we are hosting a two-day conference, another first, for the MySQL community called MySQL Connect on Sept. 29-30 in San Francisco. If you attended the MySQL Innovation Day and liked what we did, you are going to love MySQL Connect. We’ll have a lot more of our engineers and many users and community members presenting hour long sessions and hands on labs. Our engineers will be presenting new MySQL features as well offer previews of upcoming enhancements. Monica: What's the big take-away from today's MySQL Innovation Day? Tomas: I hope the most important takeaway for attendees was to see that Oracle has been driving, and continues to drive MySQL innovation with a steady stream of new great GA and Development Milestone releases. Monica: What were attendees most interested in? What feedback did they have? Tomas: Feedback from attendees was incredibly positive and encouraging. In particular, they liked the interaction with the MySQL engineers and were also excited about the new early access features in MySQL 5.6 and MySQL Cluster 7.3. In addition, sessions delivered by MySQL users like Facebook, Pinterest and Twitter were very well received. For example, Pinterest talked about using MySQL to scale from 0 to billions of page views/month, Twitter talked about “Scaling twitter with MySQL” and Facebook discussed the many options to implement MySQL master failover solutions. The presentations are already available for download while some of the session videos will be made available on the MySQL Innovation Day web page shortly. Monica: How would you distinguish the use of MySQL vs. Oracle Database? What key factors should customers consider? Tomas: MySQL and Oracle Database complement each other. They are very different products, best suited to different use cases. Customers can choose world-class solutions from Oracle to fulfill a variety of needs. MySQL is a great choice for enterprise web-based, custom and embedded apps. Oracle Database is the leading choice for enterprise packaged applications such as ERP, CRM as well as high-end data warehousing and business intelligence applications. Monica: What are the highlights of the current MySQL 5.6 Development Milestone Release and early access features for MySQL Cluster 7.3? Tomas: MySQL 5.6 development milestone release builds on MySQL 5.5 by improving: Optimizer for better Performance, Scalability Performance Schema for better instrumentation InnoDB for better transactional throughput Replication for higher availability, data integrity NoSQL options for more flexibility We announced some new early access features in MySQL 5.6, including binary log group commit. We also announced early access features in MySQL Cluster 7.3 including support for foreign key constraints. Monica: How do people get these releases? Tomas: You can access development milestone releases by going to: http://dev.mysql.com/downloads/mysqlThen select the “Development Release” tab. The MySQL Cluster 7.3 and other early access features can be downloaded at: http://labs.mysql.com Monica: What's coming up next for MySQL? Tomas: Our development team is working in overdrive, cranking out new features with community feedback. Don’t miss the MySQL Connect conference being held in San Francisco on Sept. 29 and 30th. My team and I will be there. I hope you can join us! Monica: Thank you for your time, Tomas. I look forward to seeing you at the MySQL Connect conference. To our followers, I hope you found this interview informative. I welcome your comments. Please stay tuned here for more updates on MySQL. Note: Monica Kumar is Senior Director of product marketing for Linux, Virtualization and MySQL at Oracle.

Read the article

Toorcon 15 (2013)

- by danx

The Toorcon gang (senior staff): h1kari (founder), nfiltr8, and Geo Introduction to Toorcon 15 (2013) A Tale of One Software Bypass of MS Windows 8 Secure Boot Breaching SSL, One Byte at a Time Running at 99%: Surviving an Application DoS Security Response in the Age of Mass Customized Attacks x86 Rewriting: Defeating RoP and other Shinanighans Clowntown Express: interesting bugs and running a bug bounty program Active Fingerprinting of Encrypted VPNs Making Attacks Go Backwards Mask Your Checksums—The Gorry Details Adventures with weird machines thirty years after "Reflections on Trusting Trust" Introduction to Toorcon 15 (2013) Toorcon 15 is the 15th annual security conference held in San Diego. I've attended about a third of them and blogged about previous conferences I attended here starting in 2003. As always, I've only summarized the talks I attended and interested me enough to write about them. Be aware that I may have misrepresented the speaker's remarks and that they are not my remarks or opinion, or those of my employer, so don't quote me or them. Those seeking further details may contact the speakers directly or use The Google. For some talks, I have a URL for further information. A Tale of One Software Bypass of MS Windows 8 Secure Boot Andrew Furtak and Oleksandr Bazhaniuk Yuri Bulygin, Oleksandr ("Alex") Bazhaniuk, and (not present) Andrew Furtak Yuri and Alex talked about UEFI and Bootkits and bypassing MS Windows 8 Secure Boot, with vendor recommendations. They previously gave this talk at the BlackHat 2013 conference. MS Windows 8 Secure Boot Overview UEFI (Unified Extensible Firmware Interface) is interface between hardware and OS. UEFI is processor and architecture independent. Malware can replace bootloader (bootx64.efi, bootmgfw.efi). Once replaced can modify kernel. Trivial to replace bootloader. Today many legacy bootkits—UEFI replaces them most of them. MS Windows 8 Secure Boot verifies everything you load, either through signatures or hashes. UEFI firmware relies on secure update (with signed update). You would think Secure Boot would rely on ROM (such as used for phones0, but you can't do that for PCs—PCs use writable memory with signatures DXE core verifies the UEFI boat loader(s) OS Loader (winload.efi, winresume.efi) verifies the OS kernel A chain of trust is established with a root key (Platform Key, PK), which is a cert belonging to the platform vendor. Key Exchange Keys (KEKs) verify an "authorized" database (db), and "forbidden" database (dbx). X.509 certs with SHA-1/SHA-256 hashes. Keys are stored in non-volatile (NV) flash-based NVRAM. Boot Services (BS) allow adding/deleting keys (can't be accessed once OS starts—which uses Run-Time (RT)). Root cert uses RSA-2048 public keys and PKCS#7 format signatures. SecureBoot — enable disable image signature checks SetupMode — update keys, self-signed keys, and secure boot variables CustomMode — allows updating keys Secure Boot policy settings are: always execute, never execute, allow execute on security violation, defer execute on security violation, deny execute on security violation, query user on security violation Attacking MS Windows 8 Secure Boot Secure Boot does NOT protect from physical access. Can disable from console. Each BIOS vendor implements Secure Boot differently. There are several platform and BIOS vendors. It becomes a "zoo" of implementations—which can be taken advantage of. Secure Boot is secure only when all vendors implement it correctly. Allow only UEFI firmware signed updates protect UEFI firmware from direct modification in flash memory protect FW update components program SPI controller securely protect secure boot policy settings in nvram protect runtime api disable compatibility support module which allows unsigned legacy Can corrupt the Platform Key (PK) EFI root certificate variable in SPI flash. If PK is not found, FW enters setup mode wich secure boot turned off. Can also exploit TPM in a similar manner. One is not supposed to be able to directly modify the PK in SPI flash from the OS though. But they found a bug that they can exploit from User Mode (undisclosed) and demoed the exploit. It loaded and ran their own bootkit. The exploit requires a reboot. Multiple vendors are vulnerable. They will disclose this exploit to vendors in the future. Recommendations: allow only signed updates protect UEFI fw in ROM protect EFI variable store in ROM Breaching SSL, One Byte at a Time Yoel Gluck and Angelo Prado Angelo Prado and Yoel Gluck, Salesforce.com CRIME is software that performs a "compression oracle attack." This is possible because the SSL protocol doesn't hide length, and because SSL compresses the header. CRIME requests with every possible character and measures the ciphertext length. Look for the plaintext which compresses the most and looks for the cookie one byte-at-a-time. SSL Compression uses LZ77 to reduce redundancy. Huffman coding replaces common byte sequences with shorter codes. US CERT thinks the SSL compression problem is fixed, but it isn't. They convinced CERT that it wasn't fixed and they issued a CVE. BREACH, breachattrack.com BREACH exploits the SSL response body (Accept-Encoding response, Content-Encoding). It takes advantage of the fact that the response is not compressed. BREACH uses gzip and needs fairly "stable" pages that are static for ~30 seconds. It needs attacker-supplied content (say from a web form or added to a URL parameter). BREACH listens to a session's requests and responses, then inserts extra requests and responses. Eventually, BREACH guesses a session's secret key. Can use compression to guess contents one byte at-a-time. For example, "Supersecret SupersecreX" (a wrong guess) compresses 10 bytes, and "Supersecret Supersecret" (a correct guess) compresses 11 bytes, so it can find each character by guessing every character. To start the guess, BREACH needs at least three known initial characters in the response sequence. Compression length then "leaks" information. Some roadblocks include no winners (all guesses wrong) or too many winners (multiple possibilities that compress the same). The solutions include: lookahead (guess 2 or 3 characters at-a-time instead of 1 character). Expensive rollback to last known conflict check compression ratio can brute-force first 3 "bootstrap" characters, if needed (expensive) block ciphers hide exact plain text length. Solution is to align response in advance to block size Mitigations length: use variable padding secrets: dynamic CSRF tokens per request secret: change over time separate secret to input-less servlets Future work eiter understand DEFLATE/GZIP HTTPS extensions Running at 99%: Surviving an Application DoS Ryan Huber Ryan Huber, Risk I/O Ryan first discussed various ways to do a denial of service (DoS) attack against web services. One usual method is to find a slow web page and do several wgets. Or download large files. Apache is not well suited at handling a large number of connections, but one can put something in front of it Can use Apache alternatives, such as nginx How to identify malicious hosts short, sudden web requests user-agent is obvious (curl, python) same url requested repeatedly no web page referer (not normal) hidden links. hide a link and see if a bot gets it restricted access if not your geo IP (unless the website is global) missing common headers in request regular timing first seen IP at beginning of attack count requests per hosts (usually a very large number) Use of captcha can mitigate attacks, but you'll lose a lot of genuine users. Bouncer, goo.gl/c2vyEc and www.github.com/rawdigits/Bouncer Bouncer is software written by Ryan in netflow. Bouncer has a small, unobtrusive footprint and detects DoS attempts. It closes blacklisted sockets immediately (not nice about it, no proper close connection). Aggregator collects requests and controls your web proxies. Need NTP on the front end web servers for clean data for use by bouncer. Bouncer is also useful for a popularity storm ("Slashdotting") and scraper storms. Future features: gzip collection data, documentation, consumer library, multitask, logging destroyed connections. Takeaways: DoS mitigation is easier with a complete picture Bouncer designed to make it easier to detect and defend DoS—not a complete cure Security Response in the Age of Mass Customized Attacks Peleus Uhley and Karthik Raman Peleus Uhley and Karthik Raman, Adobe ASSET, blogs.adobe.com/asset/ Peleus and Karthik talked about response to mass-customized exploits. Attackers behave much like a business. "Mass customization" refers to concept discussed in the book Future Perfect by Stan Davis of Harvard Business School. Mass customization is differentiating a product for an individual customer, but at a mass production price. For example, the same individual with a debit card receives basically the same customized ATM experience around the world. Or designing your own PC from commodity parts. Exploit kits are another example of mass customization. The kits support multiple browsers and plugins, allows new modules. Exploit kits are cheap and customizable. Organized gangs use exploit kits. A group at Berkeley looked at 77,000 malicious websites (Grier et al., "Manufacturing Compromise: The Emergence of Exploit-as-a-Service", 2012). They found 10,000 distinct binaries among them, but derived from only a dozen or so exploit kits. Characteristics of Mass Malware: potent, resilient, relatively low cost Technical characteristics: multiple OS, multipe payloads, multiple scenarios, multiple languages, obfuscation Response time for 0-day exploits has gone down from ~40 days 5 years ago to about ~10 days now. So the drive with malware is towards mass customized exploits, to avoid detection There's plenty of evicence that exploit development has Project Manager bureaucracy. They infer from the malware edicts to: support all versions of reader support all versions of windows support all versions of flash support all browsers write large complex, difficult to main code (8750 lines of JavaScript for example Exploits have "loose coupling" of multipe versions of software (adobe), OS, and browser. This allows specific attacks against specific versions of multiple pieces of software. Also allows exploits of more obscure software/OS/browsers and obscure versions. Gave examples of exploits that exploited 2, 3, 6, or 14 separate bugs. However, these complete exploits are more likely to be buggy or fragile in themselves and easier to defeat. Future research includes normalizing malware and Javascript. Conclusion: The coming trend is that mass-malware with mass zero-day attacks will result in mass customization of attacks. x86 Rewriting: Defeating RoP and other Shinanighans Richard Wartell Richard Wartell The attack vector we are addressing here is: First some malware causes a buffer overflow. The malware has no program access, but input access and buffer overflow code onto stack Later the stack became non-executable. The workaround malware used was to write a bogus return address to the stack jumping to malware Later came ASLR (Address Space Layout Randomization) to randomize memory layout and make addresses non-deterministic. The workaround malware used was to jump t existing code segments in the program that can be used in bad ways "RoP" is Return-oriented Programming attacks. RoP attacks use your own code and write return address on stack to (existing) expoitable code found in program ("gadgets"). Pinkie Pie was paid $60K last year for a RoP attack. One solution is using anti-RoP compilers that compile source code with NO return instructions. ASLR does not randomize address space, just "gadgets". IPR/ILR ("Instruction Location Randomization") randomizes each instruction with a virtual machine. Richard's goal was to randomize a binary with no source code access. He created "STIR" (Self-Transofrming Instruction Relocation). STIR disassembles binary and operates on "basic blocks" of code. The STIR disassembler is conservative in what to disassemble. Each basic block is moved to a random location in memory. Next, STIR writes new code sections with copies of "basic blocks" of code in randomized locations. The old code is copied and rewritten with jumps to new code. the original code sections in the file is marked non-executible. STIR has better entropy than ASLR in location of code. Makes brute force attacks much harder. STIR runs on MS Windows (PEM) and Linux (ELF). It eliminated 99.96% or more "gadgets" (i.e., moved the address). Overhead usually 5-10% on MS Windows, about 1.5-4% on Linux (but some code actually runs faster!). The unique thing about STIR is it requires no source access and the modified binary fully works! Current work is to rewrite code to enforce security policies. For example, don't create a *.{exe,msi,bat} file. Or don't connect to the network after reading from the disk. Clowntown Express: interesting bugs and running a bug bounty program Collin Greene Collin Greene, Facebook Collin talked about Facebook's bug bounty program. Background at FB: FB has good security frameworks, such as security teams, external audits, and cc'ing on diffs. But there's lots of "deep, dark, forgotten" parts of legacy FB code. Collin gave several examples of bountied bugs. Some bounty submissions were on software purchased from a third-party (but bounty claimers don't know and don't care). We use security questions, as does everyone else, but they are basically insecure (often easily discoverable). Collin didn't expect many bugs from the bounty program, but they ended getting 20+ good bugs in first 24 hours and good submissions continue to come in. Bug bounties bring people in with different perspectives, and are paid only for success. Bug bounty is a better use of a fixed amount of time and money versus just code review or static code analysis. The Bounty program started July 2011 and paid out $1.5 million to date. 14% of the submissions have been high priority problems that needed to be fixed immediately. The best bugs come from a small % of submitters (as with everything else)—the top paid submitters are paid 6 figures a year. Spammers like to backstab competitors. The youngest sumitter was 13. Some submitters have been hired. Bug bounties also allows to see bugs that were missed by tools or reviews, allowing improvement in the process. Bug bounties might not work for traditional software companies where the product has release cycle or is not on Internet. Active Fingerprinting of Encrypted VPNs Anna Shubina Anna Shubina, Dartmouth Institute for Security, Technology, and Society (I missed the start of her talk because another track went overtime. But I have the DVD of the talk, so I'll expand later) IPsec leaves fingerprints. Using netcat, one can easily visually distinguish various crypto chaining modes just from packet timing on a chart (example, DES-CBC versus AES-CBC) One can tell a lot about VPNs just from ping roundtrips (such as what router is used) Delayed packets are not informative about a network, especially if far away from the network More needed to explore about how TCP works in real life with respect to timing Making Attacks Go Backwards Fuzzynop FuzzyNop, Mandiant This talk is not about threat attribution (finding who), product solutions, politics, or sales pitches. But who are making these malware threats? It's not a single person or group—they have diverse skill levels. There's a lot of fat-fingered fumblers out there. Always look for low-hanging fruit first: "hiding" malware in the temp, recycle, or root directories creation of unnamed scheduled tasks obvious names of files and syscalls ("ClearEventLog") uncleared event logs. Clearing event log in itself, and time of clearing, is a red flag and good first clue to look for on a suspect system Reverse engineering is hard. Disassembler use takes practice and skill. A popular tool is IDA Pro, but it takes multiple interactive iterations to get a clean disassembly. Key loggers are used a lot in targeted attacks. They are typically custom code or built in a backdoor. A big tip-off is that non-printable characters need to be printed out (such as "[Ctrl]" "[RightShift]") or time stamp printf strings. Look for these in files. Presence is not proof they are used. Absence is not proof they are not used. Java exploits. Can parse jar file with idxparser.py and decomile Java file. Java typially used to target tech companies. Backdoors are the main persistence mechanism (provided externally) for malware. Also malware typically needs command and control. Application of Artificial Intelligence in Ad-Hoc Static Code Analysis John Ashaman John Ashaman, Security Innovation Initially John tried to analyze open source files with open source static analysis tools, but these showed thousands of false positives. Also tried using grep, but tis fails to find anything even mildly complex. So next John decided to write his own tool. His approach was to first generate a call graph then analyze the graph. However, the problem is that making a call graph is really hard. For example, one problem is "evil" coding techniques, such as passing function pointer. First the tool generated an Abstract Syntax Tree (AST) with the nodes created from method declarations and edges created from method use. Then the tool generated a control flow graph with the goal to find a path through the AST (a maze) from source to sink. The algorithm is to look at adjacent nodes to see if any are "scary" (a vulnerability), using heuristics for search order. The tool, called "Scat" (Static Code Analysis Tool), currently looks for C# vulnerabilities and some simple PHP. Later, he plans to add more PHP, then JSP and Java. For more information see his posts in Security Innovation blog and NRefactory on GitHub. Mask Your Checksums—The Gorry Details Eric (XlogicX) Davisson Eric (XlogicX) Davisson Sometimes in emailing or posting TCP/IP packets to analyze problems, you may want to mask the IP address. But to do this correctly, you need to mask the checksum too, or you'll leak information about the IP. Problem reports found in stackoverflow.com, sans.org, and pastebin.org are usually not masked, but a few companies do care. If only the IP is masked, the IP may be guessed from checksum (that is, it leaks data). Other parts of packet may leak more data about the IP. TCP and IP checksums both refer to the same data, so can get more bits of information out of using both checksums than just using one checksum. Also, one can usually determine the OS from the TTL field and ports in a packet header. If we get hundreds of possible results (16x each masked nibble that is unknown), one can do other things to narrow the results, such as look at packet contents for domain or geo information. With hundreds of results, can import as CSV format into a spreadsheet. Can corelate with geo data and see where each possibility is located. Eric then demoed a real email report with a masked IP packet attached. Was able to find the exact IP address, given the geo and university of the sender. Point is if you're going to mask a packet, do it right. Eric wouldn't usually bother, but do it correctly if at all, to not create a false impression of security. Adventures with weird machines thirty years after "Reflections on Trusting Trust" Sergey Bratus Sergey Bratus, Dartmouth College (and Julian Bangert and Rebecca Shapiro, not present) "Reflections on Trusting Trust" refers to Ken Thompson's classic 1984 paper. "You can't trust code that you did not totally create yourself." There's invisible links in the chain-of-trust, such as "well-installed microcode bugs" or in the compiler, and other planted bugs. Thompson showed how a compiler can introduce and propagate bugs in unmodified source. But suppose if there's no bugs and you trust the author, can you trust the code? Hell No! There's too many factors—it's Babylonian in nature. Why not? Well, Input is not well-defined/recognized (code's assumptions about "checked" input will be violated (bug/vunerabiliy). For example, HTML is recursive, but Regex checking is not recursive. Input well-formed but so complex there's no telling what it does For example, ELF file parsing is complex and has multiple ways of parsing. Input is seen differently by different pieces of program or toolchain Any Input is a program input executes on input handlers (drives state changes & transitions) only a well-defined execution model can be trusted (regex/DFA, PDA, CFG) Input handler either is a "recognizer" for the inputs as a well-defined language (see langsec.org) or it's a "virtual machine" for inputs to drive into pwn-age ELF ABI (UNIX/Linux executible file format) case study. Problems can arise from these steps (without planting bugs): compiler linker loader ld.so/rtld relocator DWARF (debugger info) exceptions The problem is you can't really automatically analyze code (it's the "halting problem" and undecidable). Only solution is to freeze code and sign it. But you can't freeze everything! Can't freeze ASLR or loading—must have tables and metadata. Any sufficiently complex input data is the same as VM byte code Example, ELF relocation entries + dynamic symbols == a Turing Complete Machine (TM). @bxsays created a Turing machine in Linux from relocation data (not code) in an ELF file. For more information, see Rebecca "bx" Shapiro's presentation from last year's Toorcon, "Programming Weird Machines with ELF Metadata" @bxsays did same thing with Mach-O bytecode Or a DWARF exception handling data .eh_frame + glibc == Turning Machine X86 MMU (IDT, GDT, TSS): used address translation to create a Turning Machine. Page handler reads and writes (on page fault) memory. Uses a page table, which can be used as Turning Machine byte code. Example on Github using this TM that will fly a glider across the screen Next Sergey talked about "Parser Differentials". That having one input format, but two parsers, will create confusion and opportunity for exploitation. For example, CSRs are parsed during creation by cert requestor and again by another parser at the CA. Another example is ELF—several parsers in OS tool chain, which are all different. Can have two different Program Headers (PHDRs) because ld.so parses multiple PHDRs. The second PHDR can completely transform the executable. This is described in paper in the first issue of International Journal of PoC. Conclusions trusting computers not only about bugs! Bugs are part of a problem, but no by far all of it complex data formats means bugs no "chain of trust" in Babylon! (that is, with parser differentials) we need to squeeze complexity out of data until data stops being "code equivalent" Further information See and langsec.org. USENIX WOOT 2013 (Workshop on Offensive Technologies) for "weird machines" papers and videos.

Read the article

Force.com presents Database.com SQL Azure/Amazon RDS unfazed

- by Sarang

At the DreamForce 2010 event in San Francisco Force.com unveiled their next big thing in the Fat SaaS portfolio "Database.com". I am still wondering how would they would've shelled out for that domain name. Now why would a already established SaaS player foray into a key building block like Database? Potentially allowing enterprises to build apps that do not utilize the Force.com stack! One key reason is being seen as the Fat SaaS player with evey trick in the SaaS space under his belt. You want CRM come hither, want a custom development PaaS like solution welcome home (VMForce), want all your apps to talk to a cloud DB and minimize latency by having it reside closer to you cloud apps? You've come to the right place sire! Other is potentially killing foray of smaller DB players like Oracle (Not surprisingly, the Database.com offering is a highly customized and scalable Oracle database) from entering the lucrative SaaS db marketplace. The feature set promised looks great out of the box for someone who likes to visualize cool new architectures. The ground realities are certainly going to be a lot different considering the SOAP/REST style access patterns in lieu of the comfortable old shoe of SQL. Microsoft suffered heavily with SDS (SQL Data Services) offering in early 2009 and had to pull the plug on the product only to reintroduce as a simple SQL Server in the cloud, SQL Windows Azure. Though MSFT is playing cool by providing OData semantics to work with SQL Windows Azure satisfying atleast some needs of the Web-Style to a DB. The other features like Social data models including Profiles, Status updates, feeds seem interesting as well. (Although I beleive social is just one of the aspects of large scale collaborative computing). All these features start "Free" for devs its a good news but the good news stops here. The overall pricing model of $ per Users per Transactions / Month is highly disproportionate compared to Amazon RDS (Based on MySQL) or SQL Windows Azure (Based on MSSQL). Roger Jennigs of Oakleaf did an interesting comparo based on 3, 10, 100, 500 users and it turns out that Database.com going by current understanding is way too expensive for the services on offer. The offering may not impact the decision for DotNet shops mulling their cloud stategy or even some Java/MySQL shops thinking about Amazon RDS, however for enterprises having already invested in other force.com offerings this could be a very important piece in the cloud strategy jigsaw. One which would address a key cloud DB issue of "Latency" for them at least it will help having the DB in the neighborhood. The tooling and "SQL like" access provider drivers (Think ODBC/JDBC) will be available later this year. Progress Software has already announced their JDBC driver stack for Database.com. It remains to be seen how effective the overall solutions proves to be in the longer run but for starts its a important decision towards consolidating Force.com's already strong positioning in the SaaS space. As always contrasting views are welcome! :)

Read the article

ESB Toolkit 2.0 EndPointConfig (HTTPS with WCF-BasicHttp and the ESB Toolkit 2.0)

- by Andy Morrison

Earlier this week I had an ESB endpoint (Off-Ramp in ESB parlance) that I was sending to over http using WCF-BasicHttp. I needed to switch the protocol to https: which I did by changing my UDDI Binding over to https: No problem from a management perspective; however, when I tried to run the process I saw this exception: Event Type: Error Event Source: BizTalk Server 2009 Event Category: BizTalk Server 2009 Event ID: 5754 Date: 3/10/2010 Time: 2:58:23 PM User: N/A Computer: XXXXXXXXX Description: A message sent to adapter "WCF-BasicHttp" on send port "SPDynamic.XXX.SR" with URI "https://XXXXXXXXX.com/XXXXXXX/whatever.asmx" is suspended. Error details: System.ArgumentException: The provided URI scheme 'https' is invalid; expected 'http'. Parameter name: via at System.ServiceModel.Channels.TransportChannelFactory`1.ValidateScheme(Uri via) at System.ServiceModel.Channels.HttpChannelFactory.ValidateCreateChannelParameters(EndpointAddress remoteAddress, Uri via) at System.ServiceModel.Channels.HttpChannelFactory.OnCreateChannel(EndpointAddress remoteAddress, Uri via) at System.ServiceModel.Channels.ChannelFactoryBase`1.InternalCreateChannel(EndpointAddress address, Uri via) at System.ServiceModel.Channels.ChannelFactoryBase`1.CreateChannel(EndpointAddress address, Uri via) at System.ServiceModel.Channels.ServiceChannelFactory.ServiceChannelFactoryOverRequest.CreateInnerChannelBinder(EndpointAddress to, Uri via) at System.ServiceModel.Channels.ServiceChannelFactory.CreateServiceChannel(EndpointAddress address, Uri via) at System.ServiceModel.Channels.ServiceChannelFactory.CreateChannel(Type channelType, EndpointAddress address, Uri via) at System.ServiceModel.ChannelFactory`1.CreateChannel(EndpointAddress address, Uri via) at System.ServiceModel.ChannelFactory`1.CreateChannel() at Microsoft.BizTalk.Adapter.Wcf.Runtime.WcfClient`2.GetChannel[TChannel](IBaseMessage bizTalkMessage, ChannelFactory`1& cachedFactory) at Microsoft.BizTalk.Adapter.Wcf.Runtime.WcfClient`2.SendMessage(IBaseMessage bizTalkMessage) MessageId: {1170F4ED-550F-4F7E-B0E0-1EE92A25AB10} InstanceID: {1640C6C6-CA9C-4746-AEB0-584FDF7BB61E} I knew from a previous experience that I likely needed to set the SecurityMode setting for my Send Port. But how do you do this for a Dynamic port (which I was using since this is an ESB solution)? Within the UDDI portal you have to add an additional Instance Info to your Binding named: EndPointConfig Then you have to set its value to: SecurityMode=Transport Like this: The EndPointConfig is how the ESB Toolkit 2.0 provides extensibility for the various transports. To see what the key-value pair options are for a given transport, open up an itinerary and change one of your resolvers to a “static” resolver by setting the “Resolver Implementation” to Static. Then select a “Transport Name” ”, for instance to WCF-BasicHttp. At this point you can then click on the “EndPoint Configuration” property for to see an adapter/ramp specific properties dialog (key-value pairs.) Here’s the dialog that popped up for WCF-BasicHttp: I simply set the SecurityMode to Transport. Please note that you will get different properties within the window depending on the Transport Name you select for the resolver. When you are done with your settings, export the itinerary to disk and find that xml; then find that resolver’s xml within that file. It will look like endpointConfig=SecurityMode=Transport in this case. Note that if you set additional properties you will have additional key-value pairs after endpointConfig= Copy that string and paste it into the UDDI portal for you Binding’s EndPointConfig Instance Info value.

Read the article

Translating with Google Translate without API and C# Code

- by Rick Strahl

Some time back I created a data base driven ASP.NET Resource Provider along with some tools that make it easy to edit ASP.NET resources interactively in a Web application. One of the small helper features of the interactive resource admin tool is the ability to do simple translations using both Google Translate and Babelfish. Here's what this looks like in the resource administration form: When a resource is displayed, the user can click a Translate button and it will show the current resource text and then lets you set the source and target languages to translate. The Go button fires the translation for both Google and Babelfish and displays them - pressing use then changes the language of the resource to the target language and sets the resource value to the newly translated value. It's a nice and quick way to get a quick translation going. Ch… Ch… Changes Originally, both implementations basically did some screen scraping of the interactive Web sites and retrieved translated text out of result HTML. Screen scraping is always kind of an iffy proposition as content can be changed easily, but surprisingly that code worked for many years without fail. Recently however, Google at least changed their input pages to use AJAX callbacks and the page updates no longer worked the same way. End result: The Google translate code was broken. Now, Google does have an official API that you can access, but the API is being deprecated and you actually need to have an API key. Since I have public samples that people can download the API key is an issue if I want people to have the samples work out of the box - the only way I could even do this is by sharing my API key (not allowed). However, after a bit of spelunking and playing around with the public site however I found that Google's interactive translate page actually makes callbacks using plain public access without an API key. By intercepting some of those AJAX calls and calling them directly from code I was able to get translation back up and working with minimal fuss, by parsing out the JSON these AJAX calls return. I don't think this particular Warning: This is hacky code, but after a fair bit of testing I found this to work very well with all sorts of languages and accented and escaped text etc. as long as you stick to small blocks of translated text. I thought I'd share it in case anybody else had been relying on a screen scraping mechanism like I did and needed a non-API based replacement. Here's the code: /// <summary> /// Translates a string into another language using Google's translate API JSON calls. /// <seealso>Class TranslationServices</seealso> /// </summary> /// <param name="Text">Text to translate. Should be a single word or sentence.</param> /// <param name="FromCulture"> /// Two letter culture (en of en-us, fr of fr-ca, de of de-ch) /// </param> /// <param name="ToCulture"> /// Two letter culture (as for FromCulture) /// </param> public string TranslateGoogle(string text, string fromCulture, string toCulture) { fromCulture = fromCulture.ToLower(); toCulture = toCulture.ToLower(); // normalize the culture in case something like en-us was passed // retrieve only en since Google doesn't support sub-locales string[] tokens = fromCulture.Split('-'); if (tokens.Length > 1) fromCulture = tokens[0]; // normalize ToCulture tokens = toCulture.Split('-'); if (tokens.Length > 1) toCulture = tokens[0]; string url = string.Format(@"http://translate.google.com/translate_a/t?client=j&text={0}&hl=en&sl={1}&tl={2}", HttpUtility.UrlEncode(text),fromCulture,toCulture); // Retrieve Translation with HTTP GET call string html = null; try { WebClient web = new WebClient(); // MUST add a known browser user agent or else response encoding doen't return UTF-8 (WTF Google?) web.Headers.Add(HttpRequestHeader.UserAgent, "Mozilla/5.0"); web.Headers.Add(HttpRequestHeader.AcceptCharset, "UTF-8"); // Make sure we have response encoding to UTF-8 web.Encoding = Encoding.UTF8; html = web.DownloadString(url); } catch (Exception ex) { this.ErrorMessage = Westwind.Globalization.Resources.Resources.ConnectionFailed + ": " + ex.GetBaseException().Message; return null; } // Extract out trans":"...[Extracted]...","from the JSON string string result = Regex.Match(html, "trans\":(\".*?\"),\"", RegexOptions.IgnoreCase).Groups[1].Value; if (string.IsNullOrEmpty(result)) { this.ErrorMessage = Westwind.Globalization.Resources.Resources.InvalidSearchResult; return null; } //return WebUtils.DecodeJsString(result); // Result is a JavaScript string so we need to deserialize it properly JavaScriptSerializer ser = new JavaScriptSerializer(); return ser.Deserialize(result, typeof(string)) as string; } To use the code is straightforward enough - simply provide a string to translate and a pair of two letter source and target languages: string result = service.TranslateGoogle("Life is great and one is spoiled when it goes on and on and on", "en", "de"); TestContext.WriteLine(result); How it works The code to translate is fairly straightforward. It basically uses the URL I snagged from the Google Translate Web Page slightly changed to return a JSON result (&client=j) instead of the funky nested PHP style JSON array that the default returns. The JSON result returned looks like this: {"sentences":[{"trans":"Das Leben ist großartig und man wird verwöhnt, wenn es weiter und weiter und weiter geht","orig":"Life is great and one is spoiled when it goes on and on and on","translit":"","src_translit":""}],"src":"en","server_time":24} I use WebClient to make an HTTP GET call to retrieve the JSON data and strip out part of the full JSON response that contains the actual translated text. Since this is a JSON response I need to deserialize the JSON string in case it's encoded (for upper/lower ASCII chars or quotes etc.). Couple of odd things to note in this code: First note that a valid user agent string must be passed (or at least one starting with a common browser identification - I use Mozilla/5.0). Without this Google doesn't encode the result with UTF-8, but instead uses a ISO encoding that .NET can't easily decode. Google seems to ignore the character set header and use the user agent instead which is - odd to say the least. The other is that the code returns a full JSON response. Rather than use the full response and decode it into a custom type that matches Google's result object, I just strip out the translated text. Yeah I know that's hacky but avoids an extra type and firing up the JavaScript deserializer. My internal version uses a small DecodeJsString() method to decode Javascript without the overhead of a full JSON parser. It's obviously not rocket science but as mentioned above what's nice about it is that it works without an Google API key. I can't vouch on how many translates you can do before there are cut offs but in my limited testing running a few stress tests on a Web server under load I didn't run into any problems. Limitations There are some restrictions with this: It only works on single words or single sentences - multiple sentences (delimited by .) are cut off at the ".". There is also a length limitation which appears to happen at around 220 characters or so. While that may not sound like much for typical word or phrase translations this this is plenty of length. Use with a grain of salt - Google seems to be trying to limit their exposure to usage of the Translate APIs so this code might break in the future, but for now at least it works. FWIW, I also found that Google's translation is not as good as Babelfish, especially for contextual content like sentences. Google is faster, but Babelfish tends to give better translations. This is why in my translation tool I show both Google and Babelfish values retrieved. You can check out the code for this in the West Wind West Wind Web Toolkit's TranslationService.cs file which contains both the Google and Babelfish translation code pieces. Ironically the Babelfish code has been working forever using screen scraping and continues to work just fine today. I think it's a good idea to have multiple translation providers in case one is down or changes its format, hence the dual display in my translation form above. I hope this has been helpful to some of you - I've actually had many small uses for this code in a number of applications and it's sweet to have a simple routine that performs these operations for me easily. Resources Live Localization Sample Localization Resource Provider Administration form that includes options to translate text using Google and Babelfish interactively. TranslationService.cs The full source code in the West Wind West Wind Web Toolkit's Globalization library that contains the translation code. © Rick Strahl, West Wind Technologies, 2005-2011Posted in CSharp HTTP Tweet (function() { var po = document.createElement('script'); po.type = 'text/javascript'; po.async = true; po.src = 'https://apis.google.com/js/plusone.js'; var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(po, s); })();

Read the article

Continuous Integration for SQL Server Part II – Integration Testing

- by Ben Rees

My previous post, on setting up Continuous Integration for SQL Server databases using GitHub, Bamboo and Red Gate’s tools, covered the first two parts of a simple Database Continuous Delivery process: Putting your database in to a source control system, and, Running a continuous integration process, each time changes are checked in. However there is, of course, a lot more to to Continuous Delivery than that. Specifically, in addition to the above: Putting some actual integration tests in to the CI process (otherwise, they don’t really do much, do they!?), Deploying the database changes with a managed, automated approach, Monitoring what you’ve just put live, to make sure you haven’t broken anything. This post will detail how to set up a very simple pipeline for implementing the first of these (continuous integration testing). NB: A lot of the setup in this post is built on top of the configuration from before, so it might be difficult to implement this post without running through part I first. There’ll then be a third post on automated database deployment followed by a final post dealing with the last item – monitoring changes on the live system. In the previous post, I used a mixture of Red Gate products and other 3rd party software – GitHub and Atlassian Bamboo specifically. This was partly because I believe most people work in an heterogeneous environment, using software from different vendors to suit their purposes and I wanted to show how this could work for this process. For example, you could easily substitute Atlassian’s BitBucket or Stash for GitHub, depending on your needs, or use an alternative CI server such as TeamCity, TFS or Jenkins. However, in this, post, I’ll be mostly using Red Gate products only (other than tSQLt). I would do this, firstly because I work for Red Gate. However, I also think that in the area of Database Delivery processes, nobody else has the offerings to implement this process fully – so I didn’t have any choice! Background on Continuous Delivery For me, a great source of information on what makes a proper Continuous Delivery process is the Jez Humble and David Farley classic: Continuous Delivery – Reliable Software Releases through Build, Test, and Deployment Automation This book is not of course, primarily about databases, and the process I outline here and in the previous article is a gross simplification of what Jez and David describe (not least because it’s that much harder for databases!). However, a lot of the principles that they describe can be equally applied to database development and, I would argue, should be. As I say however, what I describe here is a very simple version of what would be required for a full production process. A couple of useful resources on handling some of these complexities can be found in the following two references: Refactoring Databases – Evolutionary Database Design, by Scott J Ambler and Pramod J. Sadalage Versioning Databases – Branching and Merging, by Scott Allen In particular, I don’t deal at all with the issues of multiple branches and merging of those branches, an issue made particularly acute by the use of GitHub. The other point worth making is that, in the words of Martin Fowler: Continuous Delivery is about keeping your application in a state where it is always able to deploy into production. I.e. we are not talking about continuously delivery updates to the production database every time someone checks in an amendment to a stored procedure. That is possible (and what Martin calls Continuous Deployment). However, again, that’s more than I describe in this article. And I doubt I need to remind DBAs or Developers to Proceed with Caution! Integration Testing Back to something practical. The next stage, building on our set up from the previous article, is to add in some integration tests to the process. As I say, the CI process, though interesting, isn’t enormously useful without some sort of test process running. For this we’ll use the tSQLt framework, an open source framework designed specifically for running SQL Server tests. tSQLt is part of Red Gate’s SQL Test found on http://www.red-gate.com/products/sql-development/sql-test/ or can be downloaded separately from www.tsqlt.org - though I’ll provide a step-by-step guide below for setting this up. Getting tSQLt set up via SQL Test Click on the link http://www.red-gate.com/products/sql-development/sql-test/ and click on the blue Download button to download the Red Gate SQL Test product, if not already installed. Follow the install process for SQL Test to install the SQL Server Management Studio (SSMS) plugin on to your machine, if not already installed. Open SSMS. You should now see SQL Test under the Tools menu: Clicking this link will give you the basic SQL Test dialogue: As yet, though we’ve installed the SQL Test product we haven’t yet installed the tSQLt test framework on to any particular database. To do this, we need to add our RedGateApp database using this dialogue, by clicking on the + Add Database to SQL Test… link, selecting the RedGateApp database and clicking the Add Database link: In the next screen, SQL Test describes what will be installed on the database for the tSQLt framework. Also in this dialogue, uncheck the “Add SQL Cop tests” option (shown below). SQL Cop is a great set of pre-defined tests that work within the tSQLt framework to check the general health of your SQL Server database. However, we won’t be using them in this particular simple example: Once you’ve clicked on the OK button, the changes described in the dialogue will be made to your database. Some of these are shown in the left-hand-side below: We’ve now installed the framework. However, we haven’t actually created any tests, so this will be the next step. But, before we proceed, we’ve made an update to our database so should, again check this in to source control, adding comments as required: Also worth a quick check that your build still runs with the new additions!: (And a quick check of the RedGateAppCI database shows that the changes have been made). Creating and Testing a Unit Test There are, of course, a lot of very interesting unit tests that you could and should set up for a database. The great thing about the tSQLt framework is that you can write these in SQL. The example I’m going to use here is pretty Mickey Mouse – our database table is going to include some email addresses as reference data and I want to check whether these are all in a correct email format. Nothing clever but it illustrates the process and hopefully shows the method by which more interesting tests could be set up. Adding Reference Data to our Database To start, I want to add some reference data to my database, and have this source controlled (as well as the schema). First of all I need to add some data in to my solitary table – this can be done a number of ways, but I’ll do this in SSMS for simplicity: I then add some reference data to my table: Currently this reference data just exists in the database. For proper integration testing, this needs to form part of the source-controlled version of the database – and so needs to be added to the Git repository. This can be done via SQL Source Control, though first a Primary Key needs to be added to the table. Right click the table, select Design, then right-click on the first “id” row. Then click on “Set Primary Key”: NB: once this change is made, click Save to save the change to the table. Then, to source control this reference data, right click on the table (dbo.Email) and selecting the following option: In the next screen, link the data in the Email table, by selecting it from the list and clicking “save and close”: We should at this point re-commit the changes (both the addition of the Primary Key, and the data) to the Git repo. NB: From here on, I won’t show screenshots for the GitHub side of things – it’s the same each time: whenever a change is made in SQL Source Control and committed to your local folder, you then need to sync this in the GitHub Windows client (as this is where the build server, Bamboo is taking it from). An interesting point to note here, when these changes are committed in SQL Source Control (right-click database and select “Commit Changes to Source Control..”): The display gives a warning about possibly needing a migration script for the “Add Primary Key” step of the changes. This isn’t actually necessary in this case, but this mechanism would allow you to create override scripts to replace the default change scripts created by the SQL Compare engine (which runs underneath SQL Source Control). Ignoring this message (!), we add a comment and commit the changes to Git. I then sync these, run a build (or the build gets run automatically), and check that the data is being deployed over to the target RedGateAppCI database: Creating and Running the Test As I mention, the test I’m going to use here is a very simple one - are the email addresses in my reference table valid? This isn’t of course, a full test of email validation (I expect the email addresses I’ve chosen here aren’t really the those of the Fab Four) – but just a very basic check of format used. I’ve taken the relevant SQL from this Stack Overflow article. In SSMS select “SQL Test” from the Tools menu, then click on + New Test: In the next screen, give your new test a name, and also enter a name in the Test Class box (test classes are schemas that help you keep things organised). Also check that the database in which the test is going to be created is correct – RedGateApp in this example: Click “Create Test”. After closing a couple of subsequent dialogues, you’ll see a dummy script for the test, that needs filling in: We now need to define the SQL for our test. As mentioned before, tSQLt allows you to write your unit tests in T-SQL, and the code I’m going to use here is as below. This needs to be copied and pasted in to the query window, to replace the default given by tSQLt: – Basic email check test ALTER PROCEDURE [MyChecks].[test Check Email Addresses] AS BEGIN SET NOCOUNT ON Declare @Output VarChar(max) Set @Output = ” SELECT @Output = @Output + Email +Char(13) + Char(10) FROM dbo.Email WHERE email NOT LIKE ‘%_@__%.__%’ If @Output > ” Begin Set @Output = Char(13) + Char(10) + @Output EXEC tSQLt.Fail@Output End END; Once this script is entered, hit execute to add the Stored Procedure to the database. Before committing the test to source control, it’s worth just checking that it works! For a positive test, click on “SQL Test” from the Tools menu, then click Run Tests. You should see output like the following: - a green tick to indicate success! But of course, what we also need to do is test that this is actually doing something by showing a failed test. Edit one of the email addresses in your table to an incorrect format: Now, re-run the same SQL Test as before and you’ll see the following: Great – we now know that our test is really doing something! You’ll also see a useful error message at the bottom of SSMS: (leave the email address as invalid for now, for the next steps). The next stage is to check this new test in to source control again, by right-clicking on the database and checking in the changes with a commit message (and not forgetting to sync in the GitHub client): Checking that the Tests are Running as Integration Tests After the changes above are made, and after a build has run on Bamboo (manual or automatic), looking at the Stored Procedures for the RedGateAppCI, the SPROC for the new test has been moved over to the database. However this is not exactly what we were after. We didn’t want to just copy objects from one database to another, but actually run the tests as part of the build/integration test process. I.e. we’re continuously checking any changes we make (in this case, to the reference data emails), to ensure we’re not breaking a test that we’ve set up. The behaviour we want to see is that, if we check in static data that is incorrect (as we did in step 9 above) and we have the tSQLt test set up, then our build in Bamboo should fail. However, re-running the build shows the following: - sadly, a successful build! To make sure the tSQLt tests are run as part of the integration test, we need to amend a switch in the Red Gate CI config file. First, navigate to file sqlCI.targets in your working folder: Edit this document, make the following change, save the document, then commit and sync this change in the GitHub client:    <enableTsqlt>true</enableTsqlt> Now, if we re-run the build in Bamboo (NB: I’ve moved to a new server here, hence different address and build number): - superb, a broken build!! The error message isn’t great here, so to get more detailed info, click on the full build log link on this page (below the fold). The interesting part of the log shown is towards the bottom. Pulling out this part: 21-Jun-2013 11:35:19 Build FAILED. 21-Jun-2013 11:35:19 21-Jun-2013 11:35:19 "C:\Users\Administrator\bamboo-home\xml-data\build-dir\RGA-RGP-JOB1\sqlCI.proj" (default target) (1) -> 21-Jun-2013 11:35:19 (sqlCI target) -> 21-Jun-2013 11:35:19 EXEC : sqlCI error occurred: RedGate.Deploy.SqlServerDbPackage.Shared.Exceptions.InvalidSqlException: Test Case Summary: 1 test case(s) executed, 0 succeeded, 1 failed, 0 errored. [C:\Users\Administrator\bamboo-home\xml-data\build-dir\RGA-RGP-JOB1\sqlCI.proj] 21-Jun-2013 11:35:19 EXEC : sqlCI error occurred: [MyChecks].[test Check Email Addresses] failed: [C:\Users\Administrator\bamboo-home\xml-data\build-dir\RGA-RGP-JOB1\sqlCI.proj] 21-Jun-2013 11:35:19 EXEC : sqlCI error occurred: ringo.starr@beatles [C:\Users\Administrator\bamboo-home\xml-data\build-dir\RGA-RGP-JOB1\sqlCI.proj] 21-Jun-2013 11:35:19 EXEC : sqlCI error occurred: [C:\Users\Administrator\bamboo-home\xml-data\build-dir\RGA-RGP-JOB1\sqlCI.proj] 21-Jun-2013 11:35:19 EXEC : sqlCI error occurred: +----------------------+ [C:\Users\Administrator\bamboo-home\xml-data\build-dir\RGA-RGP-JOB1\sqlCI.proj] 21-Jun-2013 11:35:19 EXEC : sqlCI error occurred: |Test Execution Summary| [C:\Users\Administrator\bamboo-home\xml-data\build-dir\RGA-RGP-JOB1\sqlCI.proj] As a final check, we should make sure that, if we now fix this error, the build succeeds. So in SSMS, I’m going to correct the invalid email address, then check this change in to SQL Source Control (with a comment), commit to GitHub, and re-run the build: This should have fixed the build: It worked! Summary This has been a very quick run through the implementation of CI for databases, including tSQLt tests to test whether your database updates are working. The next post in this series will focus on automated deployment – we’ve tested our database changes, how can we now deploy these to target sites?

Read the article

How to find and fix performance problems in ORM powered applications

- by FransBouma

Once in a while we get requests about how to fix performance problems with our framework. As it comes down to following the same steps and looking into the same things every single time, I decided to write a blogpost about it instead, so more people can learn from this and solve performance problems in their O/R mapper powered applications. In some parts it's focused on LLBLGen Pro but it's also usable for other O/R mapping frameworks, as the vast majority of performance problems in O/R mapper powered applications are not specific for a certain O/R mapper framework. Too often, the developer looks at the wrong part of the application, trying to fix what isn't a problem in that part, and getting frustrated that 'things are so slow with <insert your favorite framework X here>'. I'm in the O/R mapper business for a long time now (almost 10 years, full time) and as it's a small world, we O/R mapper developers know almost all tricks to pull off by now: we all know what to do to make task ABC faster and what compromises (because there are almost always compromises) to deal with if we decide to make ABC faster that way. Some O/R mapper frameworks are faster in X, others in Y, but you can be sure the difference is mainly a result of a compromise some developers are willing to deal with and others aren't. That's why the O/R mapper frameworks on the market today are different in many ways, even though they all fetch and save entities from and to a database. I'm not suggesting there's no room for improvement in today's O/R mapper frameworks, there always is, but it's not a matter of 'the slowness of the application is caused by the O/R mapper' anymore. Perhaps query generation can be optimized a bit here, row materialization can be optimized a bit there, but it's mainly coming down to milliseconds. Still worth it if you're a framework developer, but it's not much compared to the time spend inside databases and in user code: if a complete fetch takes 40ms or 50ms (from call to entity object collection), it won't make a difference for your application as that 10ms difference won't be noticed. That's why it's very important to find the real locations of the problems so developers can fix them properly and don't get frustrated because their quest to get a fast, performing application failed. Performance tuning basics and rules Finding and fixing performance problems in any application is a strict procedure with four prescribed steps: isolate, analyze, interpret and fix, in that order. It's key that you don't skip a step nor make assumptions: these steps help you find the reason of a problem which seems to be there, and how to fix it or leave it as-is. Skipping a step, or when you assume things will be bad/slow without doing analysis will lead to the path of premature optimization and won't actually solve your problems, only create new ones. The most important rule of finding and fixing performance problems in software is that you have to understand what 'performance problem' actually means. Most developers will say "when a piece of software / code is slow, you have a performance problem". But is that actually the case? If I write a Linq query which will aggregate, group and sort 5 million rows from several tables to produce a resultset of 10 rows, it might take more than a couple of milliseconds before that resultset is ready to be consumed by other logic. If I solely look at the Linq query, the code consuming the resultset of the 10 rows and then look at the time it takes to complete the whole procedure, it will appear to me to be slow: all that time taken to produce and consume 10 rows? But if you look closer, if you analyze and interpret the situation, you'll see it does a tremendous amount of work, and in that light it might even be extremely fast. With every performance problem you encounter, always do realize that what you're trying to solve is perhaps not a technical problem at all, but a perception problem. The second most important rule you have to understand is based on the old saying "Penny wise, Pound Foolish": the part which takes e.g. 5% of the total time T for a given task isn't worth optimizing if you have another part which takes a much larger part of the total time T for that same given task. Optimizing parts which are relatively insignificant for the total time taken is not going to bring you better results overall, even if you totally optimize that part away. This is the core reason why analysis of the complete set of application parts which participate in a given task is key to being successful in solving performance problems: No analysis -> no problem -> no solution. One warning up front: hunting for performance will always include making compromises. Fast software can be made maintainable, but if you want to squeeze as much performance out of your software, you will inevitably be faced with the dilemma of compromising one or more from the group {readability, maintainability, features} for the extra performance you think you'll gain. It's then up to you to decide whether it's worth it. In almost all cases it's not. The reason for this is simple: the vast majority of performance problems can be solved by implementing the proper algorithms, the ones with proven Big O-characteristics so you know the performance you'll get plus you know the algorithm will work. The time taken by the algorithm implementing code is inevitable: you already implemented the best algorithm. You might find some optimizations on the technical level but in general these are minor. Let's look at the four steps to see how they guide us through the quest to find and fix performance problems. Isolate The first thing you need to do is to isolate the areas in your application which are assumed to be slow. For example, if your application is a web application and a given page is taking several seconds or even minutes to load, it's a good candidate to check out. It's important to start with the isolate step because it allows you to focus on a single code path per area with a clear begin and end and ignore the rest. The rest of the steps are taken per identified problematic area. Keep in mind that isolation focuses on tasks in an application, not code snippets. A task is something that's started in your application by either another task or the user, or another program, and has a beginning and an end. You can see a task as a piece of functionality offered by your application. Analyze Once you've determined the problem areas, you have to perform analysis on the code paths of each area, to see where the performance problems occur and which areas are not the problem. This is a multi-layered effort: an application which uses an O/R mapper typically consists of multiple parts: there's likely some kind of interface (web, webservice, windows etc.), a part which controls the interface and business logic, the O/R mapper part and the RDBMS, all connected with either a network or inter-process connections provided by the OS or other means. Each of these parts, including the connectivity plumbing, eat up a part of the total time it takes to complete a task, e.g. load a webpage with all orders of a given customer X. To understand which parts participate in the task / area we're investigating and how much they contribute to the total time taken to complete the task, analysis of each participating task is essential. Start with the code you wrote which starts the task, analyze the code and track the path it follows through your application. What does the code do along the way, verify whether it's correct or not. Analyze whether you have implemented the right algorithms in your code for this particular area. Remember we're looking at one area at a time, which means we're ignoring all other code paths, just the code path of the current problematic area, from begin to end and back. Don't dig in and start optimizing at the code level just yet. We're just analyzing. If your analysis reveals big architectural stupidity, it's perhaps a good idea to rethink the architecture at this point. For the rest, we're analyzing which means we collect data about what could be wrong, for each participating part of the complete application. Reviewing the code you wrote is a good tool to get deeper understanding of what is going on for a given task but ultimately it lacks precision and overview what really happens: humans aren't good code interpreters, computers are. We therefore need to utilize tools to get deeper understanding about which parts contribute how much time to the total task, triggered by which other parts and for example how many times are they called. There are two different kind of tools which are necessary: .NET profilers and O/R mapper / RDBMS profilers. .NET profiling .NET profilers (e.g. dotTrace by JetBrains or Ants by Red Gate software) show exactly which pieces of code are called, how many times they're called, and the time it took to run that piece of code, at the method level and sometimes even at the line level. The .NET profilers are essential tools for understanding whether the time taken to complete a given task / area in your application is consumed by .NET code, where exactly in your code, the path to that code, how many times that code was called by other code and thus reveals where hotspots are located: the areas where a solution can be found. Importantly, they also reveal which areas can be left alone: remember our penny wise pound foolish saying: if a profiler reveals that a group of methods are fast, or don't contribute much to the total time taken for a given task, ignore them. Even if the code in them is perhaps complex and looks like a candidate for optimization: you can work all day on that, it won't matter. As we're focusing on a single area of the application, it's best to start profiling right before you actually activate the task/area. Most .NET profilers support this by starting the application without starting the profiling procedure just yet. You navigate to the particular part which is slow, start profiling in the profiler, in your application you perform the actions which are considered slow, and afterwards you get a snapshot in the profiler. The snapshot contains the data collected by the profiler during the slow action, so most data is produced by code in the area to investigate. This is important, because it allows you to stay focused on a single area. O/R mapper and RDBMS profiling .NET profilers give you a good insight in the .NET side of things, but not in the RDBMS side of the application. As this article is about O/R mapper powered applications, we're also looking at databases, and the software making it possible to consume the database in your application: the O/R mapper. To understand which parts of the O/R mapper and database participate how much to the total time taken for task T, we need different tools. There are two kind of tools focusing on O/R mappers and database performance profiling: O/R mapper profilers and RDBMS profilers. For O/R mapper profilers, you can look at LLBLGen Prof by hibernating rhinos or the Linq to Sql/LLBLGen Pro profiler by Huagati. Hibernating rhinos also have profilers for other O/R mappers like NHibernate (NHProf) and Entity Framework (EFProf) and work the same as LLBLGen Prof. For RDBMS profilers, you have to look whether the RDBMS vendor has a profiler. For example for SQL Server, the profiler is shipped with SQL Server, for Oracle it's build into the RDBMS, however there are also 3rd party tools. Which tool you're using isn't really important, what's important is that you get insight in which queries are executed during the task / area we're currently focused on and how long they took. Here, the O/R mapper profilers have an advantage as they collect the time it took to execute the query from the application's perspective so they also collect the time it took to transport data across the network. This is important because a query which returns a massive resultset or a resultset with large blob/clob/ntext/image fields takes more time to get transported across the network than a small resultset and a database profiler doesn't take this into account most of the time. Another tool to use in this case, which is more low level and not all O/R mappers support it (though LLBLGen Pro and NHibernate as well do) is tracing: most O/R mappers offer some form of tracing or logging system which you can use to collect the SQL generated and executed and often also other activity behind the scenes. While tracing can produce a tremendous amount of data in some cases, it also gives insight in what's going on. Interpret After we've completed the analysis step it's time to look at the data we've collected. We've done code reviews to see whether we've done anything stupid and which parts actually take place and if the proper algorithms have been implemented. We've done .NET profiling to see which parts are choke points and how much time they contribute to the total time taken to complete the task we're investigating. We've performed O/R mapper profiling and RDBMS profiling to see which queries were executed during the task, how many queries were generated and executed and how long they took to complete, including network transportation. All this data reveals two things: which parts are big contributors to the total time taken and which parts are irrelevant. Both aspects are very important. The parts which are irrelevant (i.e. don't contribute significantly to the total time taken) can be ignored from now on, we won't look at them. The parts which contribute a lot to the total time taken are important to look at. We now have to first look at the .NET profiler results, to see whether the time taken is consumed in our own code, in .NET framework code, in the O/R mapper itself or somewhere else. For example if most of the time is consumed by DbCommand.ExecuteReader, the time it took to complete the task is depending on the time the data is fetched from the database. If there was just 1 query executed, according to tracing or O/R mapper profilers / RDBMS profilers, check whether that query is optimal, uses indexes or has to deal with a lot of data. Interpret means that you follow the path from begin to end through the data collected and determine where, along the path, the most time is contributed. It also means that you have to check whether this was expected or is totally unexpected. My previous example of the 10 row resultset of a query which groups millions of rows will likely reveal that a long time is spend inside the database and almost no time is spend in the .NET code, meaning the RDBMS part contributes the most to the total time taken, the rest is compared to that time, irrelevant. Considering the vastness of the source data set, it's expected this will take some time. However, does it need tweaking? Perhaps all possible tweaks are already in place. In the interpret step you then have to decide that further action in this area is necessary or not, based on what the analysis results show: if the analysis results were unexpected and in the area where the most time is contributed to the total time taken is room for improvement, action should be taken. If not, you can only accept the situation and move on. In all cases, document your decision together with the analysis you've done. If you decide that the perceived performance problem is actually expected due to the nature of the task performed, it's essential that in the future when someone else looks at the application and starts asking questions you can answer them properly and new analysis is only necessary if situations changed. Fix After interpreting the analysis results you've concluded that some areas need adjustment. This is the fix step: you're actively correcting the performance problem with proper action targeted at the real cause. In many cases related to O/R mapper powered applications it means you'll use different features of the O/R mapper to achieve the same goal, or apply optimizations at the RDBMS level. It could also mean you apply caching inside your application (compromise memory consumption over performance) to avoid unnecessary re-querying data and re-consuming the results. After applying a change, it's key you re-do the analysis and interpretation steps: compare the results and expectations with what you had before, to see whether your actions had any effect or whether it moved the problem to a different part of the application. Don't fall into the trap to do partly analysis: do the full analysis again: .NET profiling and O/R mapper / RDBMS profiling. It might very well be that the changes you've made make one part faster but another part significantly slower, in such a way that the overall problem hasn't changed at all. Performance tuning is dealing with compromises and making choices: to use one feature over the other, to accept a higher memory footprint, to go away from the strict-OO path and execute queries directly onto the RDBMS, these are choices and compromises which will cross your path if you want to fix performance problems with respect to O/R mappers or data-access and databases in general. In most cases it's not a big issue: alternatives are often good choices too and the compromises aren't that hard to deal with. What is important is that you document why you made a choice, a compromise: which analysis data, which interpretation led you to the choice made. This is key for good maintainability in the years to come. Most common performance problems with O/R mappers Below is an incomplete list of common performance problems related to data-access / O/R mappers / RDBMS code. It will help you with fixing the hotspots you found in the interpretation step. SELECT N+1: (Lazy-loading specific). Lazy loading triggered performance bottlenecks. Consider a list of Orders bound to a grid. You have a Field mapped onto a related field in Order, Customer.CompanyName. Showing this column in the grid will make the grid fetch (indirectly) for each row the Customer row. This means you'll get for the single list not 1 query (for the orders) but 1+(the number of orders shown) queries. To solve this: use eager loading using a prefetch path to fetch the customers with the orders. SELECT N+1 is easy to spot with an O/R mapper profiler or RDBMS profiler: if you see a lot of identical queries executed at once, you have this problem. Prefetch paths using many path nodes or sorting, or limiting. Eager loading problem. Prefetch paths can help with performance, but as 1 query is fetched per node, it can be the number of data fetched in a child node is bigger than you think. Also consider that data in every node is merged on the client within the parent. This is fast, but it also can take some time if you fetch massive amounts of entities. If you keep fetches small, you can use tuning parameters like the ParameterizedPrefetchPathThreshold setting to get more optimal queries. Deep inheritance hierarchies of type Target Per Entity/Type. If you use inheritance of type Target per Entity / Type (each type in the inheritance hierarchy is mapped onto its own table/view), fetches will join subtype- and supertype tables in many cases, which can lead to a lot of performance problems if the hierarchy has many types. With this problem, keep inheritance to a minimum if possible, or switch to a hierarchy of type Target Per Hierarchy, which means all entities in the inheritance hierarchy are mapped onto the same table/view. Of course this has its own set of drawbacks, but it's a compromise you might want to take. Fetching massive amounts of data by fetching large lists of entities. LLBLGen Pro supports paging (and limiting the # of rows returned), which is often key to process through large sets of data. Use paging on the RDBMS if possible (so a query is executed which returns only the rows in the page requested). When using paging in a web application, be sure that you switch server-side paging on on the datasourcecontrol used. In this case, paging on the grid alone is not enough: this can lead to fetching a lot of data which is then loaded into the grid and paged there. Keep note that analyzing queries for paging could lead to the false assumption that paging doesn't occur, e.g. when the query contains a field of type ntext/image/clob/blob and DISTINCT can't be applied while it should have (e.g. due to a join): the datareader will do DISTINCT filtering on the client. this is a little slower but it does perform paging functionality on the data-reader so it won't fetch all rows even if the query suggests it does. Fetch massive amounts of data because blob/clob/ntext/image fields aren't excluded. LLBLGen Pro supports field exclusion for queries. You can exclude fields (also in prefetch paths) per query to avoid fetching all fields of an entity, e.g. when you don't need them for the logic consuming the resultset. Excluding fields can greatly reduce the amount of time spend on data-transport across the network. Use this optimization if you see that there's a big difference between query execution time on the RDBMS and the time reported by the .NET profiler for the ExecuteReader method call. Doing client-side aggregates/scalar calculations by consuming a lot of data. If possible, try to formulate a scalar query or group by query using the projection system or GetScalar functionality of LLBLGen Pro to do data consumption on the RDBMS server. It's far more efficient to process data on the RDBMS server than to first load it all in memory, then traverse the data in-memory to calculate a value. Using .ToList() constructs inside linq queries. It might be you use .ToList() somewhere in a Linq query which makes the query be run partially in-memory. Example: var q = from c in metaData.Customers.ToList() where c.Country=="Norway" select c; This will actually fetch all customers in-memory and do an in-memory filtering, as the linq query is defined on an IEnumerable<T>, and not on the IQueryable<T>. Linq is nice, but it can often be a bit unclear where some parts of a Linq query might run. Fetching all entities to delete into memory first. To delete a set of entities it's rather inefficient to first fetch them all into memory and then delete them one by one. It's more efficient to execute a DELETE FROM ... WHERE query on the database directly to delete the entities in one go. LLBLGen Pro supports this feature, and so do some other O/R mappers. It's not always possible to do this operation in the context of an O/R mapper however: if an O/R mapper relies on a cache, these kind of operations are likely not supported because they make it impossible to track whether an entity is actually removed from the DB and thus can be removed from the cache. Fetching all entities to update with an expression into memory first. Similar to the previous point: it is more efficient to update a set of entities directly with a single UPDATE query using an expression instead of fetching the entities into memory first and then updating the entities in a loop, and afterwards saving them. It might however be a compromise you don't want to take as it is working around the idea of having an object graph in memory which is manipulated and instead makes the code fully aware there's a RDBMS somewhere. Conclusion Performance tuning is almost always about compromises and making choices. It's also about knowing where to look and how the systems in play behave and should behave. The four steps I provided should help you stay focused on the real problem and lead you towards the solution. Knowing how to optimally use the systems participating in your own code (.NET framework, O/R mapper, RDBMS, network/services) is key for success as well as knowing what's going on inside the application you built. I hope you'll find this guide useful in tracking down performance problems and dealing with them in a useful way.

Read the article

Software Architecture and Software Architecture Evaluation

How many of us have worked at places where the concept of software architecture was ridiculed for wasting time and money? Even more ridiculous to them was the concept of evaluating software architecture. I think the next time that I am in this situation again, and I hope that I never am I will have to push for this methodology in the software development life cycle. I have spent way too many hours/days/months/years working poorly architected systems or systems that were just built ADHOC. This in software development must stop. I can understand why systems get like this due to overzealous sales staff, demanding management that wants everything yesterday, and project managers asking if things are done yet before the project has even started. But seriously, some time must be spent designing the applications that we write along with evaluating the architecture so that it will integrate will within the existing systems of an origination. If placed in this situation again, I will strive to gain buying from key players within the business, for example: Senior Software Engineers\Developers, Software Architects, Project Managers, Software Quality Assurance, Technical Services, Operations, and Finance in order for this idea to succeed with upper management. In order to convince these key players I will have to show them the benefits of architecture and even more benefits of evaluating software architecture on a system wide level. Benefits of Software Architecture Evaluation Places Stakeholders in the Same Room to Communicate Ensures Delivery of Detailed Quality Goals Prioritizes Conflicting Goals Requires Clear Explication Improves the Quality of Documentation Discovers Opportunities for Cross-Project Reuse Improves Architecture Practices Once I had key player buy in then and only then would I approach upper management about my plan regarding implementing the concept of software architecture and using evaluation to ensure that the software being designed is the proper architecture for the project. In addition to the benefits listed above I would also show upper management how much time is being wasted by not doing these evaluations. For example, if project X cost us Y amount, then why do we have several implementations in various forms of X and how much money and time could we have saved if we just reused the existing code base to give each system the same functionality that was already created? After this, I would mention what would happen if we had 50 instances of this situation? Then I would show them how the software architecture evaluation process would have prevented this and that the optimization could have leveraged its existing code base to increase the speed and quality of its development. References:Carnegie Mellon Software Engineering Institute (2011). Architecture Tradeoff Analysis Method from http://www.sei.cmu.edu/architecture/tools/evaluate/atam.cfm

Read the article

Not All “Viruses” Are Viruses: 10 Malware Terms Explained

- by Chris Hoffman

Most people seem to call every type of malware a “virus”, but that isn’t technically accurate. You’ve probably heard of many more terms beyond virus: malware, worm, Trojan, rootkit, keylogger, spyware, and more. But what do all these terms mean? These terms aren’t just used by geeks. They make their way into even mainstream news stories about the latest web security problems and tech scares. Understanding them will help you understand the dangers your\ hear about. Malware The word “malware” is short for “malicious software.” Many people use the word “virus” to indicate any type of harmful software, but a virus is actually just a specific type of malware. The word “malware” encompasses all harmful software, including all the ones listed below. Virus Let’s start with viruses. A virus is a type of malware that copies itself by infecting other files, just as viruses in the real world infect biological cells and use those biological cells to reproduce copies of themselves. A virus can do many different things — watch in the background and steal your passwords, display advertisements, or just crash your computer — but the key thing that makes it a virus is how it spreads. When you run a virus, it will infect programs on your computer. When you run the program on another computer, the virus will infect programs on that computer, and so on. For example, a virus might infect program files on a USB stick. When the programs on that USB stick are run on another computer, the virus runs on the other computer and infects more program files. The virus will continue to spread in this way. Worm A worm is similar to a virus, but it spreads a different way. Rather than infecting files and relying on human activity to move those files around and run them on different systems, a worm spreads over computer networks on its own accord. For example, the Blaster and Sasser worms spread very quickly in the days of Windows XP because Windows XP did not come properly secured and exposed system services to the Internet. The worm accessed these system services over the Internet, exploited a vulnerability, and infected the computer. The worm then used the new infected computer to continue replicating itself. Such worms are less common now that Windows is properly firewalled by default, but worms can also spread in other ways — for example, by mass-emailing themselves to every email address in an effected user’s address book. Like a virus, a worm can do any number of other harmful things once it infects a computer. The key thing that makes it a worm is simply how it spreads copies of itself. Trojan (or Trojan Horse) A Trojan horse, or Trojan, is a type of malware that disguises itself as a legitimate file. When you download and run the program, the Trojan horse will run in the background, allowing third-parties to access your computer. Trojans can do this for any number of reasons — to monitor activity on your computer, to join your computer to a botnet. Trojans may also be used to open the floodgates and download many other types of malware onto your computer. The key thing that makes this type of malware a Trojan is how it arrives. It pretends to be a useful program and, when run, it hides in the background and gives malicious people access to your computer. It isn’t obsessed with copying itself into other files or spreading over the network, as viruses and worms are. For example, a piece of pirated software on an unscrupulous website may actually contain a Trojan. Spyware Spyware is a type of malicious software that spies on you without your knowledge. It collects a variety of different types of data, depending on the piece of spyware. Different types of malware can function as spyware — there may be malicious spyware included in Trojans that spies on your keystrokes to steal financial data, for example. More “legitimate” spyware may be bundled along with free software and simply monitor your web browsing habits, uploading this data to advertising servers so the software’s creator can make money from selling their knowledge of your activities. Adware Adware often comes along with spyware. It’s any type of software that displays advertising on your computer. Programs that display advertisements inside the program itself aren’t generally classified as malware. The kind of “adware” that’s particularly malicious is the kind that abuses its access to your system to display ads when it shouldn’t. For example, a piece of harmful adware may cause pop-up advertisements to appear on your computer when you’re not doing anything else. Or, adware may inject additional advertising into other web pages as you browse the web. Adware is often combined with spyware — a piece of malware may monitor your browsing habits and use them to serve you more targeted ads. Adware is more “socially acceptable” than other types of malware on Windows and you may see adware bundled with legitimate programs. For example, some people consider the Ask Toolbar included with Oracle’s Java software adware. Keylogger A keylogger is a type of malware that runs in the background, recording every key stroke you make. These keystrokes can include usernames, passwords, credit card numbers, and other sensitive data. The keylogger then, most likely, uploads these keystrokes to a malicious server, where it can be analyzed and people can pick out the useful passwords and credit card numbers. Other types of malware can act as keyloggers. A virus, worm, or Trojan may function as a keylogger, for example. Keyloggers may also be installed for monitoring purposes by businesses or even jealous spouses. Botnet, Bot A botnet is a large network of computers that are under the botnet creator’s control. Each computer functions as a “bot” because it’s infected with a specific piece of malware. Once the bot software infects the computer, ir will connect to some sort of control server and wait for instructions from the botnet’s creator. For example, a botnet may be used to initiate a DDoS (distributed denial of service) attack. Every computer in the botnet will be told to bombard a specific website or server with requests at once, and such millions or requests can cause a server to become unresponsive or crash. Botnet creators may sell access to their botnets, allowing other malicious individuals to use large botnets to do their dirty work. Rootkit A rootkit is a type of malware designed to burrow deep into your computer, avoiding detection by security programs and users. For example, a rootkit might load before most of Windows, burying itself deep into the system and modifying system functions so that security programs can’t detect it. A rootkit might hide itself completely, preventing itself from showing up in the Windows task manager. The key thing that makes a type of malware a rootkit is that it’s stealthy and focused on hiding itself once it arrives. Ransomware Ransomware is a fairly new type of malware. It holds your computer or files hostage and demands a ransom payment. Some ransomware may simply pop up a box asking for money before you can continue using your computer. Such prompts are easily defeated with antivirus software. More harmful malware like CryptoLocker literally encrypts your files and demands a payment before you can access them. Such types of malware are dangerous, especially if you don’t have backups. Most malware these days is produced for profit, and ransomware is a good example of that. Ransomware doesn’t want to crash your computer and delete your files just to cause you trouble. It wants to take something hostage and get a quick payment from you. So why is it called “antivirus software,” anyway? Well, most people continue to consider the word “virus” synonymous with malware as a whole. Antivirus software doesn’t just protect against viruses, but against all types of malware. It may be more accurately referred to as “antimalware” or “security” software. Image Credit: Marcelo Alves on Flickr, Tama Leaver on Flickr, Szilard Mihaly on Flickr

Read the article

No Wi-Fi after system reboot

- by ILya

Something strange is happening... I've installed a Wi-Fi card into my Ubuntu Server 11.04 machine. To configure it I do the following: sudo vi /etc/network/interfaces add: iface wlan0 inet dhcp wpa-driver wext wpa-ssid "Sweet Home" wpa-ap-scan 1 wpa-proto WPA wpa-pairwise TKIP wpa-group TKIP wpa-key-mgmt WPA-PSK wpa-psk <A KEY> auto wlan0 then: $ sudo /etc/init.d/networking restart * Running /etc/init.d/networking restart is deprecated because it may not enable again some interfaces * Reconfiguring network interfaces... ssh stop/waiting ssh start/running, process 1522 ssh stop/waiting ssh start/running, process 1590 And my machine successfully gets an ip to my wireless adapter. But after reboot it doesn't get any ip in wireless network. To fix it I run /etc/init.d/networking restart again and all is fine again - it gets an ip. I understand that I simply should add it to my startup scripts to make it work properly, but maybe there is a better way to configure it?

Read the article

Web Development Trends: Mobile First, Data-Oriented Development, and Single Page Applications

- by dwahlin

I recently had the opportunity to give a keynote talk at an Intel conference about key trends in the world of Web development that I feel teams should be taking into account with projects. It was a lot of fun and I had the opportunity to talk with a lot of different people about projects they’re working on. There are a million things that could be covered for this type of talk (HTML5 anyone?) but I only had 60 minutes and couldn’t possibly cover them all so I decided to focus on 3 key areas: mobile, data-oriented development, and SPAs. The talk was geared toward introducing people (many who weren’t Web developers) to topics such as mobile first development (demos showed a few tools to help here), responsive design techniques, data binding techniques that can simplify code, and Single Page Application (SPA) benefits. Links to code demos shown during the presentation can be found at the end of the slide deck. Web Development Trends - What's New in the World of Web Development by Dan Wahlin

Read the article

The ugly evolution of running a background operation in the context of an ASP.NET app

- by Jeff

If you’re one of the two people who has followed my blog for many years, you know that I’ve been going at POP Forums now for over almost 15 years. Publishing it as an open source app has been a big help because it helps me understand how people want to use it, and having it translated to six languages is pretty sweet. Despite this warm and fuzzy group hug, there has been an ugly hack hiding in there for years. One of the things we find ourselves wanting to do is hide some kind of regular process inside of an ASP.NET application that runs periodically. The motivation for this has always been that a lot of people simply don’t have a choice, because they’re running the app on shared hosting, or don’t otherwise have access to a box that can run some kind of regular background service. In POP Forums, I “solved” this problem years ago by hiding some static timers in an HttpModule. Truthfully, this works well as long as you don’t run multiple instances of the app, which in the cloud world, is always a possibility. With the arrival of WebJobs in Azure, I’m going to solve this problem. This post isn’t about that. The other little hacky problem that I “solved” was spawning a background thread to queue emails to subscribed users of the forum. This evolved quite a bit over the years, starting with a long running page to mail users in real-time, when I had only a few hundred. By the time it got into the thousands, or tens of thousands, I needed a better way. What I did is launched a new thread that read all of the user data in, then wrote a queued email to the database (as in, the entire body of the email, every time), with the properly formatted opt-out link. It was super inefficient, but it worked. Then I moved my biggest site using it, CoasterBuzz, to an Azure Website, and it stopped working. So let’s start with the first stupid thing I was doing. The new thread was simply created with delegate code inline. As best I can tell, Azure Websites are more aggressive about garbage collection, because that thread didn’t queue even one message. When the calling server response went out of scope, so went the magic background thread. Duh, all I had to do was move the thread to a private static variable in the class. That’s the way I was able to keep stuff running from the HttpModule. (And yes, I know this is still prone to failure, particularly if the app recycles. For as infrequently as it’s used, I have not, however, experienced this.) It was still failing, but this time I wasn’t sure why. It would queue a few dozen messages, then die. Running in Azure, I had to turn on the application logging and FTP in to see what was going on. That led me to a helper method I was using as delegate to build the unsubscribe links. The idea here is that I didn’t want yet another config entry to describe the base URL, appended with the right path that would match the routing table. No, I wanted the app to figure it out for you, so I came up with this little thing: public static string FullUrlHelper(this Controller controller, string actionName, string controllerName, object routeValues = null) { var helper = new UrlHelper(controller.Request.RequestContext); var requestUrl = controller.Request.Url; if (requestUrl == null) return String.Empty; var url = requestUrl.Scheme + "://"; url += requestUrl.Host; url += (requestUrl.Port != 80 ? ":" + requestUrl.Port : ""); url += helper.Action(actionName, controllerName, routeValues); return url; } And yes, that should have been done with a string builder. This is useful for sending out the email verification messages, too. As clever as I thought I was with this, I was using a delegate in the admin controller to format these unsubscribe links for tens of thousands of users. I passed that delegate into a service class that did the email work: Func<User, string> unsubscribeLinkGenerator = user => this.FullUrlHelper("Unsubscribe", AccountController.Name, new { id = user.UserID, key = _profileService.GetUnsubscribeHash(user) }); _mailingListService.MailUsers(subject, body, htmlBody, unsubscribeLinkGenerator); Cool, right? Actually, not so much. If you look back at the helper, this delegate then will depend on the controller context to learn the routing and format for the URL. As you might have guessed, those things were turning null after a few dozen formatted links, when the original request to the admin controller went away. That this wasn’t already happening on my dedicated server is surprising, but again, I understand why the Azure environment might be eager to reclaim a thread after servicing the request. It’s already inefficient that I’m building the entire email for every user, but going back to check the routing table for the right link every time isn’t a win either. I put together a little hack to look up one generic URL, and use that as the basis for a string format. If you’re wondering why I didn’t just use the curly braces up front, it’s because they get URL formatted: var baseString = this.FullUrlHelper("Unsubscribe", AccountController.Name, new { id = "--id--", key = "--key--" }); baseString = baseString.Replace("--id--", "{0}").Replace("--key--", "{1}"); Func unsubscribeLinkGenerator = user => String.Format(baseString, user.UserID, _profileService.GetUnsubscribeHash(user)); _mailingListService.MailUsers(subject, body, htmlBody, unsubscribeLinkGenerator); And wouldn’t you know it, the new solution works just fine. It’s still kind of hacky and inefficient, but it will work until this somehow breaks too.

Search Results

Search found 21589 results on 864 pages for 'primary key'.

Page 235/864 | < Previous Page | 231 232 233 234 235 236 237 238 239 240 241 242 | Next Page >

- by tamara

- by Bramjam

- by praks5432

- by Autolycus

- by Gerard Flynn

- by user3652865

- by Deluxe

- by Wallym

- by Ciaran

- by Chris Hoffman

- by Rick Strahl

- by Reed

- by Monica Kumar

- by danx

- by Sarang

- by Andy Morrison

- by Rick Strahl

- by Ben Rees

- by FransBouma

- by Chris Hoffman

- by ILya

- by dwahlin

- by Jeff

< Previous Page | 231 232 233 234 235 236 237 238 239 240 241 242 | Next Page >