matthew underwood - Page 29

Base de Datos Oracle, su mejor opción para reducir costos de IT

- by Ivan Hassig

Por Victoria Cadavid Sr. Sales Cosultant Oracle Direct Uno de los principales desafíos en la administración de centros de datos es la reducción de costos de operación. A medida que las compañías crecen y los proveedores de tecnología ofrecen soluciones cada vez más robustas, conservar el equilibrio entre desempeño, soporte al negocio y gestión del Costo Total de Propiedad es un desafío cada vez mayor para los Gerentes de Tecnología y para los Administradores de Centros de Datos. Las estrategias más comunes para conseguir reducción en los costos de administración de Centros de Datos y en la gestión de Tecnología de una organización en general, se enfocan en la mejora del desempeño de las aplicaciones, reducción del costo de administración y adquisición de hardware, reducción de los costos de almacenamiento, aumento de la productividad en la administración de las Bases de Datos y mejora en la atención de requerimientos y prestación de servicios de mesa de ayuda, sin embargo, las estrategias de reducción de costos deben contemplar también la reducción de costos asociados a pérdida y robo de información, cumplimiento regulatorio, generación de valor y continuidad del negocio, que comúnmente se conciben como iniciativas aisladas que no siempre se adelantan con el ánimo de apoyar la reducción de costos. Una iniciativa integral de reducción de costos de TI, debe contemplar cada uno de los factores que generan costo y pueden ser optimizados. En este artículo queremos abordar la reducción de costos de tecnología a partir de la adopción del que según los expertos es el motor de Base de Datos # del mercado.Durante años, la base de datos Oracle ha sido reconocida por su velocidad, confiabilidad, seguridad y capacidad para soportar cargas de datos tanto de aplicaciones altamente transaccionales, como de Bodegas de datos e incluso análisis de Big Data , ofreciendo alto desempeño y facilidades de administración, sin embrago, cuando pensamos en proyectos de reducción de costos de IT, además de la capacidad para soportar aplicaciones (incluso aplicaciones altamente transaccionales) con alto desempeño, pensamos en procesos de automatización, optimización de recursos, consolidación, virtualización e incluso alternativas más cómodas de licenciamiento. La Base de Datos Oracle está diseñada para proveer todas las capacidades que un área de tecnología necesita para reducir costos, adaptándose a los diferentes escenarios de negocio y a las capacidades y características de cada organización.Es así, como además del motor de Base de Datos, Oracle ofrece una serie de soluciones para optimizar la administración de la información a través de mecanismos de optimización del uso del storage, continuidad del Negocio, consolidación de infraestructura, seguridad y administración automática, que propenden por un mejor uso de los recursos de tecnología, ofrecen opciones avanzadas de configuración y direccionan la reducción de los tiempos de las tareas operativas más comunes. Una de las opciones de la base de datos que se pueden provechar para reducir costos de hardware es Oracle Real Application Clusters. Esta solución de clustering permite que varios servidores (incluso servidores de bajo costo) trabajen en conjunto para soportar Grids o Nubes Privadas de Bases de Datos, proporcionando los beneficios de la consolidación de infraestructura, los esquemas de alta disponibilidad, rápido desempeño y escalabilidad por demanda, haciendo que el aprovisionamiento, el mantenimiento de las bases de datos y la adición de nuevos nodos se lleve e cabo de una forma más rápida y con menos riesgo, además de apalancar las inversiones en servidores de menor costo. Otra de las soluciones que promueven la reducción de costos de Tecnología es Oracle In-Memory Database Cache que permite almacenar y procesar datos en la memoria de las aplicaciones, permitiendo el máximo aprovechamiento de los recursos de procesamiento de la capa media, lo que cobra mucho valor en escenarios de alta transaccionalidad. De este modo se saca el mayor provecho de los recursos de procesamiento evitando crecimiento innecesario en recursos de hardware. Otra de las formas de evitar inversiones innecesarias en hardware, aprovechando los recursos existentes, incluso en escenarios de alto crecimiento de los volúmenes de información es la compresión de los datos. Oracle Advanced Compression permite comprimir hasta 4 veces los diferentes tipos de datos, mejorando la capacidad de almacenamiento, sin comprometer el desempeño de las aplicaciones. Desde el lado del almacenamiento también se pueden conseguir reducciones importantes de los costos de IT. En este escenario, la tecnología propia de la base de Datos Oracle ofrece capacidades de Administración Automática del Almacenamiento que no solo permiten una distribución óptima de los datos en los discos físicos para garantizar el máximo desempeño, sino que facilitan el aprovisionamiento y la remoción de discos defectuosos y ofrecen balanceo y mirroring, garantizando el uso máximo de cada uno de los dispositivos y la disponibilidad de los datos. Otra de las soluciones que facilitan la administración del almacenamiento es Oracle Partitioning, una opción de la Base de Datos que permite dividir grandes tablas en estructuras más pequeñas. Esta aproximación facilita la administración del ciclo de vida de la información y permite por ejemplo, separar los datos históricos (que generalmente se convierten en información de solo lectura y no tienen un alto volumen de consulta) y enviarlos a un almacenamiento de bajo costos, conservando la data activa en dispositivos de almacenamiento más ágiles. Adicionalmente, Oracle Partitioning facilita la administración de las bases de datos que tienen un gran volumen de registros y mejora el desempeño de la base de datos gracias a la posibilidad de optimizar las consultas haciendo uso únicamente de las particiones relevantes de una tabla o índice en el proceso de búsqueda. Otros factores adicionales, que pueden generar costos innecesarios a los departamentos de Tecnología son: La pérdida, corrupción o robo de datos y la falta de disponibilidad de las aplicaciones para dar soporte al negocio. Para evitar este tipo de situaciones que pueden acarrear multas y pérdida de negocios y de dinero, Oracle ofrece soluciones que permiten proteger y auditar la base de datos, recuperar la información en caso de corrupción o ejecución de acciones que comprometan la integridad de la información y soluciones que permitan garantizar que la información de las aplicaciones tenga una disponibilidad de 7x24. Ya hablamos de los beneficios de Oracle RAC, para facilitar los procesos de Consolidación y mejorar el desempeño de las aplicaciones, sin embrago esta solución, es sumamente útil en escenarios dónde las organizaciones de quieren garantizar una alta disponibilidad de la información, ante fallo de los servidores o en eventos de desconexión planeada para realizar labores de mantenimiento. Además de Oracle RAC, existen soluciones como Oracle Data Guard y Active Data Guard que permiten replicar de forma automática las bases de datos hacia un centro de datos de contingencia, permitiendo una recuperación inmediata ante eventos que deshabiliten por completo un centro de datos. Además de lo anterior, Active Data Guard, permite aprovechar la base de datos de contingencia para realizar labores de consulta, mejorando el desempeño de las aplicaciones. Desde el punto de vista de mejora en la seguridad, Oracle cuenta con soluciones como Advanced security que permite encriptar los datos y los canales a través de los cueles se comparte la información, Total Recall, que permite visualizar los cambios realizados a la base de datos en un momento determinado del tiempo, para evitar pérdida y corrupción de datos, Database Vault que permite restringir el acceso de los usuarios privilegiados a información confidencial, Audit Vault, que permite verificar quién hizo qué y cuándo dentro de las bases de datos de una organización y Oracle Data Masking que permite enmascarar los datos para garantizar la protección de la información sensible y el cumplimiento de las políticas y normas relacionadas con protección de información confidencial, por ejemplo, mientras las aplicaciones pasan del ambiente de desarrollo al ambiente de producción. Como mencionamos en un comienzo, las iniciativas de reducción de costos de tecnología deben apalancarse en estrategias que contemplen los diferentes factores que puedan generar sobre costos, los factores de riesgo que puedan acarrear costos no previsto, el aprovechamiento de los recursos actuales, para evitar inversiones innecesarias y los factores de optimización que permitan el máximo aprovechamiento de las inversiones actuales. Como vimos, todas estas iniciativas pueden ser abordadas haciendo uso de la tecnología de Oracle a nivel de Base de Datos, lo más importante es detectar los puntos críticos a nivel de riesgo, diagnosticar las proporción en que están siendo aprovechados los recursos actuales y definir las prioridades de la organización y del área de IT, para así dar inicio a todas aquellas iniciativas que de forma gradual, van a evitar sobrecostos e inversiones innecesarias, proporcionando un mayor apoyo al negocio y un impacto significativo en la productividad de la organización. Más información http://www.oracle.com/lad/products/database/index.html?ssSourceSiteId=otnes 1Fuente: Market Share: All Software Markets, Worldwide 2011 by Colleen Graham, Joanne Correia, David Coyle, Fabrizio Biscotti, Matthew Cheung, Ruggero Contu, Yanna Dharmasthira, Tom Eid, Chad Eschinger, Bianca Granetto, Hai Hong Swinehart, Sharon Mertz, Chris Pang, Asheesh Raina, Dan Sommer, Bhavish Sood, Marianne D'Aquila, Laurie Wurster and Jie Zhang. - March 29, 2012 2Big Data: Información recopilada desde fuentes no tradicionales como blogs, redes sociales, email, sensores, fotografías, grabaciones en video, etc. que normalmente se encuentran de forma no estructurada y en un gran volumen

Read the article

Customize the SimpleMembership in ASP.NET MVC 4.0

- by thangchung

As we know, .NET 4.5 have come up to us, and come along with a lot of new interesting features as well. Visual Studio 2012 was also introduced some days ago. They made us feel very happy with cool improvement along with us. Performance when loading code editor is very good at the moment (immediate after click on the solution). I explore some of cool features at these days. Some of them like Json.NET integrated in ASP.NET MVC 4.0, improvement on asynchronous action, new lightweight theme on Visual Studio, supporting very good on mobile development, improvement on authentication… I reviewed them, and found out that in this version of .NET Microsoft was not only developed new feature that suggest from community but also focused on improvement performance of existing features or components. Besides that, they also opened source more projects, like Entity Framework, Reactive Extensions, ASP.NET Web Stack… At the moment, I feel Microsoft want to open source more and more their projects. Today, I am going to dive in deep on new SimpleMembership model. It is really good because in this security model, Microsoft actually focus on development needs. As we know, in the past, they introduce some of provider supplied for coding security like MembershipProvider, RoleProvider… I don’t need to talk but everyone that have ever used it know that they were actually hard to use, and not easy to maintain and unit testing. Why? Because every time you inherit it, you need to override all methods inside it. Some people try to abstract it by introduce more method with virtual keyword, and try to implement basic behavior, so in the subclass we only need to override the method that need for their business. But to me, it’s only the way to work around. ASP.NET team and Web Matrix knew about it, so they built the new features based on existing components on .NET framework. And one of component that comes to us is SimpleMembership and SimpleRole. They implemented the Façade pattern on the top of those, and called it is WebSecurity. In the web, we can call WebSecurity anywhere we want, and make a call to inside wrapper of it. I read a lot of them on web blog, on technical news, on MSDN as well. Matthew Osborn had an excellent article about it at his blog. Jon Galloway had an article like this at here. He analyzed why old membership provider not fixed well to ASP.NET MVC and how to get over it. Those are very good to me. It introduced to me about how to doing SimpleMembership on it, how to doing it on new ASP.NET MVC web application. But one thing, those didn’t tell me was how to doing it on existing security model (that mean we already had Users and Roles on legacy system, and how we can integrate it to this system), that’s a reason I will introduce it today. I have spent couples of hours to see what’s inside this, and try to make one example to clarify my concern. And it’s lucky that I can make it working well.The first thing, we need to create new ASP.NET MVC application on Visual Studio 2012. We need to choose Internet type for this web application. ASP.NET MVC actually creates all needs components for the basic membership and basic role. The cool feature is DoNetOpenAuth come along with it that means we can log-in using facebook, twitter or Windows Live if you want. But it’s only for LocalDb, so we need to change it to fix with existing database model on SQL Server. The next step we have to make SimpleMembership can understand which database we use and show it which column need to point to for the ID and UserName. I really like this feature because SimpleMembership on need to know about the ID and UserName, and they don’t care about rest of it. I assume that we have an existing database model like So we will point it in code like The codes for it, we put on InitializeSimpleMembershipAttribute like [AttributeUsage(AttributeTargets.Class | AttributeTargets.Method, AllowMultiple = false, Inherited = true)] public sealed class InitializeSimpleMembershipAttribute : ActionFilterAttribute { private static SimpleMembershipInitializer _initializer; private static object _initializerLock = new object(); private static bool _isInitialized; public override void OnActionExecuting(ActionExecutingContext filterContext) { // Ensure ASP.NET Simple Membership is initialized only once per app start LazyInitializer.EnsureInitialized(ref _initializer, ref _isInitialized, ref _initializerLock); } private class SimpleMembershipInitializer { public SimpleMembershipInitializer() { try { WebSecurity.InitializeDatabaseConnection("DefaultDb", "User", "Id", "UserName", autoCreateTables: true); } catch (Exception ex) { throw new InvalidOperationException("The ASP.NET Simple Membership database could not be initialized. For more information, please see http://go.microsoft.com/fwlink/?LinkId=256588", ex); } } } }And decorating it in the AccountController as below [Authorize] [InitializeSimpleMembership] public class AccountController : ControllerIn this case, assuming that we need to override the ValidateUser to point this to existing User database table, and validate it. We have to add one more class like public class CustomAdminMembershipProvider : SimpleMembershipProvider { // TODO: will do a better way private const string SELECT_ALL_USER_SCRIPT = "select * from [dbo].[User]private where UserName = '{0}'"; private readonly IEncrypting _encryptor; private readonly SimpleSecurityContext _simpleSecurityContext; public CustomAdminMembershipProvider(SimpleSecurityContext simpleSecurityContext) : this(new Encryptor(), new SimpleSecurityContext("DefaultDb")) { } public CustomAdminMembershipProvider(IEncrypting encryptor, SimpleSecurityContext simpleSecurityContext) { _encryptor = encryptor; _simpleSecurityContext = simpleSecurityContext; } public override bool ValidateUser(string username, string password) { if (string.IsNullOrEmpty(username)) { throw new ArgumentException("Argument cannot be null or empty", "username"); } if (string.IsNullOrEmpty(password)) { throw new ArgumentException("Argument cannot be null or empty", "password"); } var hash = _encryptor.Encode(password); using (_simpleSecurityContext) { var users = _simpleSecurityContext.Users.SqlQuery( string.Format(SELECT_ALL_USER_SCRIPT, username)); if (users == null && !users.Any()) { return false; } return users.FirstOrDefault().Password == hash; } } }SimpleSecurityDataContext at here public class SimpleSecurityContext : DbContext { public DbSet<User> Users { get; set; } public SimpleSecurityContext(string connStringName) : base(connStringName) { this.Configuration.LazyLoadingEnabled = true; this.Configuration.ProxyCreationEnabled = false; } protected override void OnModelCreating(DbModelBuilder modelBuilder) { base.OnModelCreating(modelBuilder); modelBuilder.Configurations.Add(new UserMapping()); } }And Mapping for User as below public class UserMapping : EntityMappingBase<User> { public UserMapping() { this.Property(x => x.UserName); this.Property(x => x.DisplayName); this.Property(x => x.Password); this.Property(x => x.Email); this.ToTable("User"); } }One important thing, you need to modify the web.config to point to our customize SimpleMembership <membership defaultProvider="AdminMemberProvider" userIsOnlineTimeWindow="15"> <providers> <clear/> <add name="AdminMemberProvider" type="CIK.News.Web.Infras.Security.CustomAdminMembershipProvider, CIK.News.Web.Infras" /> </providers> </membership> <roleManager enabled="false"> <providers> <clear /> <add name="AdminRoleProvider" type="CIK.News.Web.Infras.Security.AdminRoleProvider, CIK.News.Web.Infras" /> </providers> </roleManager>The good thing at here is we don’t need to modify the code on AccountController. We only need to modify on SimpleMembership and Simple Role (if need). Now build all solutions, run it. We should see a screen like thisIf I login to Twitter button at the bottom of this page, we will be transfer to twitter authentication pageYou have to waiting for a moment Afterwards it will transfer you back to your admin screenYou can find all source codes at my MSDN code. I will really happy if you guys feel free to put some comments as below. It will be helpful to improvement my code in the future. Thank for all your readings.

Read the article

Loading any MVC page fails with the error "An item with the same key has already been added."

- by MajorRefactoring

I am having an intermittent issue that is appearing on one server only, and is causing all MVC pages to fail to load with the error "An item with the same key has already been added." Restarting the application pool fixes the issue, but until then, loading any mvc page throws the following exception: Event code: 3005 Event message: An unhandled exception has occurred. Event time: 10/11/2012 08:09:24 Event time (UTC): 10/11/2012 08:09:24 Event ID: d76264aedc4241d4bce9247692510466 Event sequence: 6407 Event occurrence: 30 Event detail code: 0 Application information: Application domain: /LM/W3SVC/21/ROOT-2-129969647741292058 Trust level: Full Application Virtual Path: / Application Path: d:\websites\SiteAndAppPoolName\ Machine name: UKSERVER Process information: Process ID: 6156 Process name: w3wp.exe Account name: IIS APPPOOL\SiteAndAppPoolName Exception information: Exception type: ArgumentException Exception message: An item with the same key has already been added. Server stack trace: at System.Collections.Generic.Dictionary`2.Insert(TKey key, TValue value, Boolean add) at System.Linq.Enumerable.ToDictionary[TSource,TKey,TElement](IEnumerable`1 source, Func`2 keySelector, Func`2 elementSelector, IEqualityComparer`1 comparer) at System.Web.WebPages.Scope.WebConfigScopeDictionary.<>c__DisplayClass4.<.ctor>b__0() at System.Lazy`1.CreateValue() Exception rethrown at [0]: at System.Lazy`1.get_Value() at System.Web.WebPages.Scope.WebConfigScopeDictionary.TryGetValue(Object key, Object& value) at System.Web.Mvc.ViewContext.ScopeGet[TValue](IDictionary`2 scope, String name, TValue defaultValue) at System.Web.Mvc.ViewContext.ScopeCache.Get(IDictionary`2 scope, HttpContextBase httpContext) at System.Web.Mvc.ViewContext.GetClientValidationEnabled(IDictionary`2 scope, HttpContextBase httpContext) at System.Web.Mvc.Html.FormExtensions.FormHelper(HtmlHelper htmlHelper, String formAction, FormMethod method, IDictionary`2 htmlAttributes) at System.Web.Mvc.Html.FormExtensions.BeginForm(HtmlHelper htmlHelper, String actionName, String controllerName) at ASP._Page_Views_Dashboard_Functions_BookingQuickLookup_cshtml.Execute() in d:\Websites\SiteAndAppPoolName\Views\Dashboard\Functions\BookingQuickLookup.cshtml:line 3 at System.Web.WebPages.WebPageBase.ExecutePageHierarchy() at System.Web.Mvc.WebViewPage.ExecutePageHierarchy() at System.Web.WebPages.WebPageBase.ExecutePageHierarchy(WebPageContext pageContext, TextWriter writer, WebPageRenderingBase startPage) at System.Web.Mvc.Html.PartialExtensions.Partial(HtmlHelper htmlHelper, String partialViewName, Object model, ViewDataDictionary viewData) at ASP._Page_Views_Dashboard_Functions_cshtml.Execute() in d:\Websites\SiteAndAppPoolName\Views\Dashboard\Functions.cshtml:line 5 at System.Web.WebPages.WebPageBase.ExecutePageHierarchy() at System.Web.Mvc.WebViewPage.ExecutePageHierarchy() at System.Web.WebPages.WebPageBase.ExecutePageHierarchy(WebPageContext pageContext, TextWriter writer, WebPageRenderingBase startPage) at System.Web.Mvc.Html.RenderPartialExtensions.RenderPartial(HtmlHelper htmlHelper, String partialViewName, Object model) at ASP._Page_Views_Dashboard_Index_cshtml.Execute() in d:\Websites\SiteAndAppPoolName\Views\Dashboard\Index.cshtml:line 9 at System.Web.WebPages.WebPageBase.ExecutePageHierarchy() at System.Web.Mvc.WebViewPage.ExecutePageHierarchy() at System.Web.WebPages.WebPageBase.ExecutePageHierarchy(WebPageContext pageContext, TextWriter writer, WebPageRenderingBase startPage) at System.Web.Mvc.ViewResultBase.ExecuteResult(ControllerContext context) at System.Web.Mvc.ControllerActionInvoker.<>c__DisplayClass1c.<InvokeActionResultWithFilters>b__19() at System.Web.Mvc.ControllerActionInvoker.InvokeActionResultFilter(IResultFilter filter, ResultExecutingContext preContext, Func`1 continuation) at System.Web.Mvc.ControllerActionInvoker.InvokeActionResultFilter(IResultFilter filter, ResultExecutingContext preContext, Func`1 continuation) at System.Web.Mvc.ControllerActionInvoker.InvokeActionResultWithFilters(ControllerContext controllerContext, IList`1 filters, ActionResult actionResult) at System.Web.Mvc.ControllerActionInvoker.InvokeAction(ControllerContext controllerContext, String actionName) at System.Web.Mvc.Controller.ExecuteCore() at System.Web.Mvc.ControllerBase.Execute(RequestContext requestContext) at System.Web.Mvc.MvcHandler.<>c__DisplayClass6.<>c__DisplayClassb.<BeginProcessRequest>b__5() at System.Web.Mvc.Async.AsyncResultWrapper.<>c__DisplayClass1.<MakeVoidDelegate>b__0() at System.Web.Mvc.MvcHandler.<>c__DisplayClasse.<EndProcessRequest>b__d() at System.Web.HttpApplication.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute() at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously) Request information: Request URL: http://SiteAndAppPoolName.spawtz.com/Dashboard Request path: /Dashboard User host address: 86.164.135.41 User: Is authenticated: False Authentication Type: Thread account name: IIS APPPOOL\SiteAndAppPoolName Thread information: Thread ID: 17 Thread account name: IIS APPPOOL\SiteAndAppPoolName Is impersonating: False Stack trace: at System.Lazy`1.get_Value() at System.Web.WebPages.Scope.WebConfigScopeDictionary.TryGetValue(Object key, Object& value) at System.Web.Mvc.ViewContext.ScopeGet[TValue](IDictionary`2 scope, String name, TValue defaultValue) at System.Web.Mvc.ViewContext.ScopeCache.Get(IDictionary`2 scope, HttpContextBase httpContext) at System.Web.Mvc.ViewContext.GetClientValidationEnabled(IDictionary`2 scope, HttpContextBase httpContext) at System.Web.Mvc.Html.FormExtensions.FormHelper(HtmlHelper htmlHelper, String formAction, FormMethod method, IDictionary`2 htmlAttributes) at System.Web.Mvc.Html.FormExtensions.BeginForm(HtmlHelper htmlHelper, String actionName, String controllerName) at ASP._Page_Views_Dashboard_Functions_BookingQuickLookup_cshtml.Execute() in d:\Websites\SiteAndAppPoolName\Views\Dashboard\Functions\BookingQuickLookup.cshtml:line 3 at System.Web.WebPages.WebPageBase.ExecutePageHierarchy() at System.Web.Mvc.WebViewPage.ExecutePageHierarchy() at System.Web.WebPages.WebPageBase.ExecutePageHierarchy(WebPageContext pageContext, TextWriter writer, WebPageRenderingBase startPage) at System.Web.Mvc.Html.PartialExtensions.Partial(HtmlHelper htmlHelper, String partialViewName, Object model, ViewDataDictionary viewData) at ASP._Page_Views_Dashboard_Functions_cshtml.Execute() in d:\Websites\SiteAndAppPoolName\Views\Dashboard\Functions.cshtml:line 5 at System.Web.WebPages.WebPageBase.ExecutePageHierarchy() at System.Web.Mvc.WebViewPage.ExecutePageHierarchy() at System.Web.WebPages.WebPageBase.ExecutePageHierarchy(WebPageContext pageContext, TextWriter writer, WebPageRenderingBase startPage) at System.Web.Mvc.Html.RenderPartialExtensions.RenderPartial(HtmlHelper htmlHelper, String partialViewName, Object model) at ASP._Page_Views_Dashboard_Index_cshtml.Execute() in d:\Websites\SiteAndAppPoolName\Views\Dashboard\Index.cshtml:line 9 at System.Web.WebPages.WebPageBase.ExecutePageHierarchy() at System.Web.Mvc.WebViewPage.ExecutePageHierarchy() at System.Web.WebPages.WebPageBase.ExecutePageHierarchy(WebPageContext pageContext, TextWriter writer, WebPageRenderingBase startPage) at System.Web.Mvc.ViewResultBase.ExecuteResult(ControllerContext context) at System.Web.Mvc.ControllerActionInvoker.<>c__DisplayClass1c.<InvokeActionResultWithFilters>b__19() at System.Web.Mvc.ControllerActionInvoker.InvokeActionResultFilter(IResultFilter filter, ResultExecutingContext preContext, Func`1 continuation) at System.Web.Mvc.ControllerActionInvoker.InvokeActionResultFilter(IResultFilter filter, ResultExecutingContext preContext, Func`1 continuation) at System.Web.Mvc.ControllerActionInvoker.InvokeActionResultWithFilters(ControllerContext controllerContext, IList`1 filters, ActionResult actionResult) at System.Web.Mvc.ControllerActionInvoker.InvokeAction(ControllerContext controllerContext, String actionName) at System.Web.Mvc.Controller.ExecuteCore() at System.Web.Mvc.ControllerBase.Execute(RequestContext requestContext) at System.Web.Mvc.MvcHandler.<>c__DisplayClass6.<>c__DisplayClassb.<BeginProcessRequest>b__5() at System.Web.Mvc.Async.AsyncResultWrapper.<>c__DisplayClass1.<MakeVoidDelegate>b__0() at System.Web.Mvc.MvcHandler.<>c__DisplayClasse.<EndProcessRequest>b__d() at System.Web.HttpApplication.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute() at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously) Custom event details: As mentioned, it's every MVC action that throws this error until the app pool is restarted, and the error seems to be occurring in System.Web.WebPages.Scope.WebConfigScopeDictionary.TryGetValue(Object key, Object& value) Has anyone seen this issue before? It's only happening on this server, on any of the app pools on the server (not confined to this one) and an app pool restart sorts it. Any help much appreciated. Cheers, Matthew

Read the article

Oracle bleibt auch 2011 Spitzenreiter im Bereich Datenbanken

- by Anne Manke

Mit der Veröffentlichung der aktuellen Ausgabe "Market Share: All Software Markets, Worldwide 2011" bestätigt das weltweit führende Marktanalyseunternehmen Gartner Oracle's Marktführerschaft im Bereich der Relationellen Datenbank Management Systeme (RDBMS). Oracle konnte innerhalb des letzten Jahres seinen Abstand zu seinen Marktbegleitern im Bereich der RDBMS mit einem stabilen Wachstum von 18% sogar ausbauen: der Marktanteil stieg im Jahr 2010 von 48,2% auf 48,8% im Jahr 2011. Damit ist der Abstand zu Oracle's stärkstem Verfolger IBM auf 28,6%. Normal 0 false false false EN-US X-NONE X-NONE MicrosoftInternetExplorer4 /* Style Definitions */ table.MsoNormalTable {mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-priority:99; mso-style-qformat:yes; mso-style-parent:""; mso-padding-alt:0cm 5.4pt 0cm 5.4pt; mso-para-margin-top:0cm; mso-para-margin-right:0cm; mso-para-margin-bottom:12.0pt; mso-para-margin-left:0cm; mso-pagination:widow-orphan; font-size:11.0pt; font-family:"Calibri","sans-serif"; mso-ascii-font-family:Calibri; mso-ascii-theme-font:minor-latin; mso-fareast-font-family:"Times New Roman"; mso-fareast-theme-font:minor-fareast; mso-hansi-font-family:Calibri; mso-hansi-theme-font:minor-latin; mso-bidi-font-family:"Times New Roman"; mso-bidi-theme-font:minor-bidi;} table.MsoTableLightListAccent2 {mso-style-name:"Light List - Accent 2"; mso-tstyle-rowband-size:1; mso-tstyle-colband-size:1; mso-style-priority:61; mso-style-unhide:no; border:solid #C0504D 1.0pt; mso-border-themecolor:accent2; mso-padding-alt:0cm 5.4pt 0cm 5.4pt; mso-para-margin:0cm; mso-para-margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:11.0pt; font-family:"Calibri","sans-serif"; mso-ascii-font-family:Calibri; mso-ascii-theme-font:minor-latin; mso-hansi-font-family:Calibri; mso-hansi-theme-font:minor-latin; mso-bidi-font-family:"Times New Roman"; mso-bidi-theme-font:minor-bidi;} table.MsoTableLightListAccent2FirstRow {mso-style-name:"Light List - Accent 2"; mso-table-condition:first-row; mso-style-priority:61; mso-style-unhide:no; mso-tstyle-shading:#C0504D; mso-tstyle-shading-themecolor:accent2; mso-para-margin-top:0cm; mso-para-margin-bottom:0cm; mso-para-margin-bottom:.0001pt; line-height:normal; color:white; mso-themecolor:background1; mso-ansi-font-weight:bold; mso-bidi-font-weight:bold;} table.MsoTableLightListAccent2LastRow {mso-style-name:"Light List - Accent 2"; mso-table-condition:last-row; mso-style-priority:61; mso-style-unhide:no; mso-tstyle-border-top:2.25pt double #C0504D; mso-tstyle-border-top-themecolor:accent2; mso-tstyle-border-left:1.0pt solid #C0504D; mso-tstyle-border-left-themecolor:accent2; mso-tstyle-border-bottom:1.0pt solid #C0504D; mso-tstyle-border-bottom-themecolor:accent2; mso-tstyle-border-right:1.0pt solid #C0504D; mso-tstyle-border-right-themecolor:accent2; mso-para-margin-top:0cm; mso-para-margin-bottom:0cm; mso-para-margin-bottom:.0001pt; line-height:normal; mso-ansi-font-weight:bold; mso-bidi-font-weight:bold;} table.MsoTableLightListAccent2FirstCol {mso-style-name:"Light List - Accent 2"; mso-table-condition:first-column; mso-style-priority:61; mso-style-unhide:no; mso-ansi-font-weight:bold; mso-bidi-font-weight:bold;} table.MsoTableLightListAccent2LastCol {mso-style-name:"Light List - Accent 2"; mso-table-condition:last-column; mso-style-priority:61; mso-style-unhide:no; mso-ansi-font-weight:bold; mso-bidi-font-weight:bold;} table.MsoTableLightListAccent2OddColumn {mso-style-name:"Light List - Accent 2"; mso-table-condition:odd-column; mso-style-priority:61; mso-style-unhide:no; mso-tstyle-border-top:1.0pt solid #C0504D; mso-tstyle-border-top-themecolor:accent2; mso-tstyle-border-left:1.0pt solid #C0504D; mso-tstyle-border-left-themecolor:accent2; mso-tstyle-border-bottom:1.0pt solid #C0504D; mso-tstyle-border-bottom-themecolor:accent2; mso-tstyle-border-right:1.0pt solid #C0504D; mso-tstyle-border-right-themecolor:accent2;} table.MsoTableLightListAccent2OddRow {mso-style-name:"Light List - Accent 2"; mso-table-condition:odd-row; mso-style-priority:61; mso-style-unhide:no; mso-tstyle-border-top:1.0pt solid #C0504D; mso-tstyle-border-top-themecolor:accent2; mso-tstyle-border-left:1.0pt solid #C0504D; mso-tstyle-border-left-themecolor:accent2; mso-tstyle-border-bottom:1.0pt solid #C0504D; mso-tstyle-border-bottom-themecolor:accent2; mso-tstyle-border-right:1.0pt solid #C0504D; mso-tstyle-border-right-themecolor:accent2;} Revenue 2010 ($USM) Revenue 2011 ($USM) Growth 2010 Growth 2011 Share 2010 Share 2011 Oracle 9,990.5 11,787.0 10.9% 18.0% 48.2% 48.8% IBM 4,300.4 4,870.4 5.4% 13.3% 20.7% 20.2% Microsoft 3,641.2 4,098.9 10.1% 12.6% 17.6% 17.0% SAP/Sybase 744.4 1,101.1 12.8% 47.9% 3.6% 4.6% Teradata 754.7 882.3 16.9% 16.9% 3.6% 3.7% Source: Gartner’s “Market Share: All Software Markets, Worldwide 2011,” March 29, 2012, By Colleen Graham, Joanne Correia, David Coyle, Fabrizio Biscotti, Matthew Cheung, Ruggero Contu, Yanna Dharmasthira, Tom Eid, Chad Eschinger, Bianca Granetto, Hai Hong Swinehart, Sharon Mertz, Chris Pang, Asheesh Raina, Dan Sommer, Bhavish Sood, Marianne D'Aquila, Laurie Wurster and Jie Normal 0 false false false EN-US X-NONE X-NONE MicrosoftInternetExplorer4 /* Style Definitions */ table.MsoNormalTable {mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-priority:99; mso-style-qformat:yes; mso-style-parent:""; mso-padding-alt:0cm 5.4pt 0cm 5.4pt; mso-para-margin-top:0cm; mso-para-margin-right:0cm; mso-para-margin-bottom:12.0pt; mso-para-margin-left:0cm; mso-pagination:widow-orphan; font-size:11.0pt; font-family:"Calibri","sans-serif"; mso-ascii-font-family:Calibri; mso-ascii-theme-font:minor-latin; mso-fareast-font-family:"Times New Roman"; mso-fareast-theme-font:minor-fareast; mso-hansi-font-family:Calibri; mso-hansi-theme-font:minor-latin; mso-bidi-font-family:"Times New Roman"; mso-bidi-theme-font:minor-bidi;} table.MsoTableLightListAccent2 {mso-style-name:"Light List - Accent 2"; mso-tstyle-rowband-size:1; mso-tstyle-colband-size:1; mso-style-priority:61; mso-style-unhide:no; border:solid #C0504D 1.0pt; mso-border-themecolor:accent2; mso-padding-alt:0cm 5.4pt 0cm 5.4pt; mso-para-margin:0cm; mso-para-margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:11.0pt; font-family:"Calibri","sans-serif"; mso-ascii-font-family:Calibri; mso-ascii-theme-font:minor-latin; mso-hansi-font-family:Calibri; mso-hansi-theme-font:minor-latin; mso-bidi-font-family:"Times New Roman"; mso-bidi-theme-font:minor-bidi;} table.MsoTableLightListAccent2FirstRow {mso-style-name:"Light List - Accent 2"; mso-table-condition:first-row; mso-style-priority:61; mso-style-unhide:no; mso-tstyle-shading:#C0504D; mso-tstyle-shading-themecolor:accent2; mso-para-margin-top:0cm; mso-para-margin-bottom:0cm; mso-para-margin-bottom:.0001pt; line-height:normal; color:white; mso-themecolor:background1; mso-ansi-font-weight:bold; mso-bidi-font-weight:bold;} table.MsoTableLightListAccent2LastRow {mso-style-name:"Light List - Accent 2"; mso-table-condition:last-row; mso-style-priority:61; mso-style-unhide:no; mso-tstyle-border-top:2.25pt double #C0504D; mso-tstyle-border-top-themecolor:accent2; mso-tstyle-border-left:1.0pt solid #C0504D; mso-tstyle-border-left-themecolor:accent2; mso-tstyle-border-bottom:1.0pt solid #C0504D; mso-tstyle-border-bottom-themecolor:accent2; mso-tstyle-border-right:1.0pt solid #C0504D; mso-tstyle-border-right-themecolor:accent2; mso-para-margin-top:0cm; mso-para-margin-bottom:0cm; mso-para-margin-bottom:.0001pt; line-height:normal; mso-ansi-font-weight:bold; mso-bidi-font-weight:bold;} table.MsoTableLightListAccent2FirstCol {mso-style-name:"Light List - Accent 2"; mso-table-condition:first-column; mso-style-priority:61; mso-style-unhide:no; mso-ansi-font-weight:bold; mso-bidi-font-weight:bold;} table.MsoTableLightListAccent2LastCol {mso-style-name:"Light List - Accent 2"; mso-table-condition:last-column; mso-style-priority:61; mso-style-unhide:no; mso-ansi-font-weight:bold; mso-bidi-font-weight:bold;} table.MsoTableLightListAccent2OddColumn {mso-style-name:"Light List - Accent 2"; mso-table-condition:odd-column; mso-style-priority:61; mso-style-unhide:no; mso-tstyle-border-top:1.0pt solid #C0504D; mso-tstyle-border-top-themecolor:accent2; mso-tstyle-border-left:1.0pt solid #C0504D; mso-tstyle-border-left-themecolor:accent2; mso-tstyle-border-bottom:1.0pt solid #C0504D; mso-tstyle-border-bottom-themecolor:accent2; mso-tstyle-border-right:1.0pt solid #C0504D; mso-tstyle-border-right-themecolor:accent2;} table.MsoTableLightListAccent2OddRow {mso-style-name:"Light List - Accent 2"; mso-table-condition:odd-row; mso-style-priority:61; mso-style-unhide:no; mso-tstyle-border-top:1.0pt solid #C0504D; mso-tstyle-border-top-themecolor:accent2; mso-tstyle-border-left:1.0pt solid #C0504D; mso-tstyle-border-left-themecolor:accent2; mso-tstyle-border-bottom:1.0pt solid #C0504D; mso-tstyle-border-bottom-themecolor:accent2; mso-tstyle-border-right:1.0pt solid #C0504D; mso-tstyle-border-right-themecolor:accent2;} Normal 0 false false false EN-US X-NONE X-NONE MicrosoftInternetExplorer4 /* Style Definitions */ table.MsoNormalTable {mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-priority:99; mso-style-qformat:yes; mso-style-parent:""; mso-padding-alt:0cm 5.4pt 0cm 5.4pt; mso-para-margin-top:0cm; mso-para-margin-right:0cm; mso-para-margin-bottom:12.0pt; mso-para-margin-left:0cm; mso-pagination:widow-orphan; font-size:11.0pt; font-family:"Calibri","sans-serif"; mso-ascii-font-family:Calibri; mso-ascii-theme-font:minor-latin; mso-fareast-font-family:"Times New Roman"; mso-fareast-theme-font:minor-fareast; mso-hansi-font-family:Calibri; mso-hansi-theme-font:minor-latin; mso-bidi-font-family:"Times New Roman"; mso-bidi-theme-font:minor-bidi;} table.MsoTableLightListAccent2 {mso-style-name:"Light List - Accent 2"; mso-tstyle-rowband-size:1; mso-tstyle-colband-size:1; mso-style-priority:61; mso-style-unhide:no; border:solid #C0504D 1.0pt; mso-border-themecolor:accent2; mso-padding-alt:0cm 5.4pt 0cm 5.4pt; mso-para-margin:0cm; mso-para-margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:11.0pt; font-family:"Calibri","sans-serif"; mso-ascii-font-family:Calibri; mso-ascii-theme-font:minor-latin; mso-hansi-font-family:Calibri; mso-hansi-theme-font:minor-latin; mso-bidi-font-family:"Times New Roman"; mso-bidi-theme-font:minor-bidi;} table.MsoTableLightListAccent2FirstRow {mso-style-name:"Light List - Accent 2"; mso-table-condition:first-row; mso-style-priority:61; mso-style-unhide:no; mso-tstyle-shading:#C0504D; mso-tstyle-shading-themecolor:accent2; mso-para-margin-top:0cm; mso-para-margin-bottom:0cm; mso-para-margin-bottom:.0001pt; line-height:normal; color:white; mso-themecolor:background1; mso-ansi-font-weight:bold; mso-bidi-font-weight:bold;} table.MsoTableLightListAccent2LastRow {mso-style-name:"Light List - Accent 2"; mso-table-condition:last-row; mso-style-priority:61; mso-style-unhide:no; mso-tstyle-border-top:2.25pt double #C0504D; mso-tstyle-border-top-themecolor:accent2; mso-tstyle-border-left:1.0pt solid #C0504D; mso-tstyle-border-left-themecolor:accent2; mso-tstyle-border-bottom:1.0pt solid #C0504D; mso-tstyle-border-bottom-themecolor:accent2; mso-tstyle-border-right:1.0pt solid #C0504D; mso-tstyle-border-right-themecolor:accent2; mso-para-margin-top:0cm; mso-para-margin-bottom:0cm; mso-para-margin-bottom:.0001pt; line-height:normal; mso-ansi-font-weight:bold; mso-bidi-font-weight:bold;} table.MsoTableLightListAccent2FirstCol {mso-style-name:"Light List - Accent 2"; mso-table-condition:first-column; mso-style-priority:61; mso-style-unhide:no; mso-ansi-font-weight:bold; mso-bidi-font-weight:bold;} table.MsoTableLightListAccent2LastCol {mso-style-name:"Light List - Accent 2"; mso-table-condition:last-column; mso-style-priority:61; mso-style-unhide:no; mso-ansi-font-weight:bold; mso-bidi-font-weight:bold;} table.MsoTableLightListAccent2OddColumn {mso-style-name:"Light List - Accent 2"; mso-table-condition:odd-column; mso-style-priority:61; mso-style-unhide:no; mso-tstyle-border-top:1.0pt solid #C0504D; mso-tstyle-border-top-themecolor:accent2; mso-tstyle-border-left:1.0pt solid #C0504D; mso-tstyle-border-left-themecolor:accent2; mso-tstyle-border-bottom:1.0pt solid #C0504D; mso-tstyle-border-bottom-themecolor:accent2; mso-tstyle-border-right:1.0pt solid #C0504D; mso-tstyle-border-right-themecolor:accent2;} table.MsoTableLightListAccent2OddRow {mso-style-name:"Light List - Accent 2"; mso-table-condition:odd-row; mso-style-priority:61; mso-style-unhide:no; mso-tstyle-border-top:1.0pt solid #C0504D; mso-tstyle-border-top-themecolor:accent2; mso-tstyle-border-left:1.0pt solid #C0504D; mso-tstyle-border-left-themecolor:accent2; mso-tstyle-border-bottom:1.0pt solid #C0504D; mso-tstyle-border-bottom-themecolor:accent2; mso-tstyle-border-right:1.0pt solid #C0504D; mso-tstyle-border-right-themecolor:accent2;}

Read the article

High Load mysql on Debian server stops every day. Why?

- by Oleg Abrazhaev

I have Debian server with 32 gb memory. And there is apache2, memcached and nginx on this server. Memory load always on maximum. Only 500m free. Most memory leak do MySql. Apache only 70 clients configured, other services small memory usage. When mysql use all memory it stops. And nothing works, need mysql reboot. Mysql configured use maximum 24 gb memory. I have hight weight InnoDB bases. (400000 rows, 30 gb). And on server multithread daemon, that makes many inserts in this tables, thats why InnoDB. There is my mysql config. [mysqld] # # * Basic Settings # default-time-zone = "+04:00" user = mysql pid-file = /var/run/mysqld/mysqld.pid socket = /var/run/mysqld/mysqld.sock port = 3306 basedir = /usr datadir = /var/lib/mysql tmpdir = /tmp language = /usr/share/mysql/english skip-external-locking default-time-zone='Europe/Moscow' # # Instead of skip-networking the default is now to listen only on # localhost which is more compatible and is not less secure. # # * Fine Tuning # #low_priority_updates = 1 concurrent_insert = ALWAYS wait_timeout = 600 interactive_timeout = 600 #normal key_buffer_size = 2024M #key_buffer_size = 1512M #70% hot cache key_cache_division_limit= 70 #16-32 max_allowed_packet = 32M #1-16M thread_stack = 8M #40-50 thread_cache_size = 50 #orderby groupby sort sort_buffer_size = 64M #same myisam_sort_buffer_size = 400M #temp table creates when group_by tmp_table_size = 3000M #tables in memory max_heap_table_size = 3000M #on disk open_files_limit = 10000 table_cache = 10000 join_buffer_size = 5M # This replaces the startup script and checks MyISAM tables if needed # the first time they are touched myisam-recover = BACKUP #myisam_use_mmap = 1 max_connections = 200 thread_concurrency = 8 # # * Query Cache Configuration # #more ignored query_cache_limit = 50M query_cache_size = 210M #on query cache query_cache_type = 1 # # * Logging and Replication # # Both location gets rotated by the cronjob. # Be aware that this log type is a performance killer. #log = /var/log/mysql/mysql.log # # Error logging goes to syslog. This is a Debian improvement :) # # Here you can see queries with especially long duration log_slow_queries = /var/log/mysql/mysql-slow.log long_query_time = 1 log-queries-not-using-indexes # # The following can be used as easy to replay backup logs or for replication. # note: if you are setting up a replication slave, see README.Debian about # other settings you may need to change. #server-id = 1 #log_bin = /var/log/mysql/mysql-bin.log server-id = 1 log-bin = /var/lib/mysql/mysql-bin #replicate-do-db = gate log-bin-index = /var/lib/mysql/mysql-bin.index log-error = /var/lib/mysql/mysql-bin.err relay-log = /var/lib/mysql/relay-bin relay-log-info-file = /var/lib/mysql/relay-bin.info relay-log-index = /var/lib/mysql/relay-bin.index binlog_do_db = 24avia expire_logs_days = 10 max_binlog_size = 100M read_buffer_size = 4024288 innodb_buffer_pool_size = 5000M innodb_flush_log_at_trx_commit = 2 innodb_thread_concurrency = 8 table_definition_cache = 2000 group_concat_max_len = 16M #binlog_do_db = gate #binlog_ignore_db = include_database_name # # * BerkeleyDB # # Using BerkeleyDB is now discouraged as its support will cease in 5.1.12. #skip-bdb # # * InnoDB # # InnoDB is enabled by default with a 10MB datafile in /var/lib/mysql/. # Read the manual for more InnoDB related options. There are many! # You might want to disable InnoDB to shrink the mysqld process by circa 100MB. #skip-innodb # # * Security Features # # Read the manual, too, if you want chroot! # chroot = /var/lib/mysql/ # # For generating SSL certificates I recommend the OpenSSL GUI "tinyca". # # ssl-ca=/etc/mysql/cacert.pem # ssl-cert=/etc/mysql/server-cert.pem # ssl-key=/etc/mysql/server-key.pem [mysqldump] quick quote-names max_allowed_packet = 500M [mysql] #no-auto-rehash # faster start of mysql but no tab completition [isamchk] key_buffer = 32M key_buffer_size = 512M # # * NDB Cluster # # See /usr/share/doc/mysql-server-*/README.Debian for more information. # # The following configuration is read by the NDB Data Nodes (ndbd processes) # not from the NDB Management Nodes (ndb_mgmd processes). # # [MYSQL_CLUSTER] # ndb-connectstring=127.0.0.1 # # * IMPORTANT: Additional settings that can override those from this file! # The files must end with '.cnf', otherwise they'll be ignored. # !includedir /etc/mysql/conf.d/ Please, help me make it stable. Memory used /etc/mysql # free total used free shared buffers cached Mem: 32930800 32766424 164376 0 139208 23829196 -/+ buffers/cache: 8798020 24132780 Swap: 33553328 44660 33508668 Maybe my problem not in memory, but MySQL stops every day. As you can see, cache memory free 24 gb. Thank to Michael Hampton? for correction. Load overage on server 3.5. Maybe hdd or another problem? Maybe my config not optimal for 30gb InnoDB ? I'm already try mysqltuner and tunung-primer.sh , but they marked all green. Mysqltuner output mysqltuner >> MySQLTuner 1.0.1 - Major Hayden <[email protected]> >> Bug reports, feature requests, and downloads at http://mysqltuner.com/ >> Run with '--help' for additional options and output filtering -------- General Statistics -------------------------------------------------- [--] Skipped version check for MySQLTuner script [OK] Currently running supported MySQL version 5.5.24-9-log [OK] Operating on 64-bit architecture -------- Storage Engine Statistics ------------------------------------------- [--] Status: -Archive -BDB -Federated +InnoDB -ISAM -NDBCluster [--] Data in MyISAM tables: 112G (Tables: 1528) [--] Data in InnoDB tables: 39G (Tables: 340) [--] Data in PERFORMANCE_SCHEMA tables: 0B (Tables: 17) [!!] Total fragmented tables: 344 -------- Performance Metrics ------------------------------------------------- [--] Up for: 8h 18m 33s (14M q [478.333 qps], 259K conn, TX: 9B, RX: 5B) [--] Reads / Writes: 84% / 16% [--] Total buffers: 10.5G global + 81.1M per thread (200 max threads) [OK] Maximum possible memory usage: 26.3G (83% of installed RAM) [OK] Slow queries: 1% (259K/14M) [!!] Highest connection usage: 100% (201/200) [OK] Key buffer size / total MyISAM indexes: 1.5G/5.6G [OK] Key buffer hit rate: 100.0% (6B cached / 1M reads) [OK] Query cache efficiency: 74.3% (8M cached / 11M selects) [OK] Query cache prunes per day: 0 [OK] Sorts requiring temporary tables: 0% (0 temp sorts / 247K sorts) [!!] Joins performed without indexes: 106025 [!!] Temporary tables created on disk: 49% (351K on disk / 715K total) [OK] Thread cache hit rate: 99% (249 created / 259K connections) [!!] Table cache hit rate: 15% (2K open / 13K opened) [OK] Open file limit used: 15% (3K/20K) [OK] Table locks acquired immediately: 99% (4M immediate / 4M locks) [!!] InnoDB data size / buffer pool: 39.4G/5.9G -------- Recommendations ----------------------------------------------------- General recommendations: Run OPTIMIZE TABLE to defragment tables for better performance MySQL started within last 24 hours - recommendations may be inaccurate Reduce or eliminate persistent connections to reduce connection usage Adjust your join queries to always utilize indexes Temporary table size is already large - reduce result set size Reduce your SELECT DISTINCT queries without LIMIT clauses Increase table_cache gradually to avoid file descriptor limits Variables to adjust: max_connections (> 200) wait_timeout (< 600) interactive_timeout (< 600) join_buffer_size (> 5.0M, or always use indexes with joins) table_cache (> 10000) innodb_buffer_pool_size (>= 39G) Mysql primer output -- MYSQL PERFORMANCE TUNING PRIMER -- - By: Matthew Montgomery - MySQL Version 5.5.24-9-log x86_64 Uptime = 0 days 8 hrs 20 min 50 sec Avg. qps = 478 Total Questions = 14369568 Threads Connected = 16 Warning: Server has not been running for at least 48hrs. It may not be safe to use these recommendations To find out more information on how each of these runtime variables effects performance visit: http://dev.mysql.com/doc/refman/5.5/en/server-system-variables.html Visit http://www.mysql.com/products/enterprise/advisors.html for info about MySQL's Enterprise Monitoring and Advisory Service SLOW QUERIES The slow query log is enabled. Current long_query_time = 1.000000 sec. You have 260626 out of 14369701 that take longer than 1.000000 sec. to complete Your long_query_time seems to be fine BINARY UPDATE LOG The binary update log is enabled Binlog sync is not enabled, you could loose binlog records during a server crash WORKER THREADS Current thread_cache_size = 50 Current threads_cached = 45 Current threads_per_sec = 0 Historic threads_per_sec = 0 Your thread_cache_size is fine MAX CONNECTIONS Current max_connections = 200 Current threads_connected = 11 Historic max_used_connections = 201 The number of used connections is 100% of the configured maximum. You should raise max_connections INNODB STATUS Current InnoDB index space = 214 M Current InnoDB data space = 39.40 G Current InnoDB buffer pool free = 0 % Current innodb_buffer_pool_size = 5.85 G Depending on how much space your innodb indexes take up it may be safe to increase this value to up to 2 / 3 of total system memory MEMORY USAGE Max Memory Ever Allocated : 23.46 G Configured Max Per-thread Buffers : 15.84 G Configured Max Global Buffers : 7.54 G Configured Max Memory Limit : 23.39 G Physical Memory : 31.40 G Max memory limit seem to be within acceptable norms KEY BUFFER Current MyISAM index space = 5.61 G Current key_buffer_size = 1.47 G Key cache miss rate is 1 : 5578 Key buffer free ratio = 77 % Your key_buffer_size seems to be fine QUERY CACHE Query cache is enabled Current query_cache_size = 200 M Current query_cache_used = 101 M Current query_cache_limit = 50 M Current Query cache Memory fill ratio = 50.59 % Current query_cache_min_res_unit = 4 K MySQL won't cache query results that are larger than query_cache_limit in size SORT OPERATIONS Current sort_buffer_size = 64 M Current read_rnd_buffer_size = 256 K Sort buffer seems to be fine JOINS Current join_buffer_size = 5.00 M You have had 106606 queries where a join could not use an index properly You have had 8 joins without keys that check for key usage after each row join_buffer_size >= 4 M This is not advised You should enable "log-queries-not-using-indexes" Then look for non indexed joins in the slow query log. OPEN FILES LIMIT Current open_files_limit = 20210 files The open_files_limit should typically be set to at least 2x-3x that of table_cache if you have heavy MyISAM usage. Your open_files_limit value seems to be fine TABLE CACHE Current table_open_cache = 10000 tables Current table_definition_cache = 2000 tables You have a total of 1910 tables You have 2151 open tables. The table_cache value seems to be fine TEMP TABLES Current max_heap_table_size = 2.92 G Current tmp_table_size = 2.92 G Of 366426 temp tables, 49% were created on disk Perhaps you should increase your tmp_table_size and/or max_heap_table_size to reduce the number of disk-based temporary tables Note! BLOB and TEXT columns are not allow in memory tables. If you are using these columns raising these values might not impact your ratio of on disk temp tables. TABLE SCANS Current read_buffer_size = 3 M Current table scan ratio = 2846 : 1 read_buffer_size seems to be fine TABLE LOCKING Current Lock Wait ratio = 1 : 185 You may benefit from selective use of InnoDB. If you have long running SELECT's against MyISAM tables and perform frequent updates consider setting 'low_priority_updates=1'

Read the article

How do you convert a parent-child (adjacency) table to a nested set using PHP and MySQL?

- by mrbinky3000

I've spent the last few hours trying to find the solution to this question online. I've found plenty of examples on how to convert from nested set to adjacency... but few that go the other way around. The examples I have found either don't work or use MySQL procedures. Unfortunately, I can't use procedures for this project. I need a pure PHP solution. I have a table that uses the adjacency model below: id parent_id category 1 0 ROOT_NODE 2 1 Books 3 1 CD's 4 1 Magazines 5 2 Books/Hardcover 6 2 Books/Large Format 7 4 Magazines/Vintage And I would like to convert it to a Nested Set table below: id left right category 1 1 14 Root Node 2 2 7 Books 3 3 4 Books/Hardcover 4 5 6 Books/Large Format 5 8 9 CD's 6 10 13 Magazines 7 11 12 Magazines/Vintage Here is an image of what I need: I have a function, based on the pseudo code from this forum post (http://www.sitepoint.com/forums/showthread.php?t=320444) but it doesn't work. I get multiple rows that have the same value for left. This should not happen. <?php /** -- -- Table structure for table `adjacent_table` -- CREATE TABLE IF NOT EXISTS `adjacent_table` ( `id` int(11) NOT NULL AUTO_INCREMENT, `father_id` int(11) DEFAULT NULL, `category` varchar(128) DEFAULT NULL, PRIMARY KEY (`id`) ) ENGINE=MyISAM DEFAULT CHARSET=latin1 AUTO_INCREMENT=8 ; -- -- Dumping data for table `adjacent_table` -- INSERT INTO `adjacent_table` (`id`, `father_id`, `category`) VALUES (1, 0, 'ROOT'), (2, 1, 'Books'), (3, 1, 'CD''s'), (4, 1, 'Magazines'), (5, 2, 'Hard Cover'), (6, 2, 'Large Format'), (7, 4, 'Vintage'); -- -- Table structure for table `nested_table` -- CREATE TABLE IF NOT EXISTS `nested_table` ( `id` int(11) NOT NULL AUTO_INCREMENT, `lft` int(11) DEFAULT NULL, `rgt` int(11) DEFAULT NULL, `category` varchar(128) DEFAULT NULL, PRIMARY KEY (`id`) ) ENGINE=MyISAM DEFAULT CHARSET=latin1 AUTO_INCREMENT=1 ; */ mysql_connect('localhost','USER','PASSWORD') or die(mysql_error()); mysql_select_db('DATABASE') or die(mysql_error()); adjacent_to_nested(0); /** * adjacent_to_nested * * Reads a "adjacent model" table and converts it to a "Nested Set" table. * @param integer $i_id Should be the id of the "root node" in the adjacent table; * @param integer $i_left Should only be used on recursive calls. Holds the current value for lft */ function adjacent_to_nested($i_id, $i_left = 0) { // the right value of this node is the left value + 1 $i_right = $i_left + 1; // get all children of this node $a_children = get_source_children($i_id); foreach ($a_children as $a) { // recursive execution of this function for each child of this node // $i_right is the current right value, which is incremented by the // import_from_dc_link_category method $i_right = adjacent_to_nested($a['id'], $i_right); // insert stuff into the our new "Nested Sets" table $s_query = " INSERT INTO `nested_table` (`id`, `lft`, `rgt`, `category`) VALUES( NULL, '".$i_left."', '".$i_right."', '".mysql_real_escape_string($a['category'])."' ) "; if (!mysql_query($s_query)) { echo "<pre>$s_query</pre>\n"; throw new Exception(mysql_error()); } echo "<p>$s_query</p>\n"; // get the newly created row id $i_new_nested_id = mysql_insert_id(); } return $i_right + 1; } /** * get_source_children * * Examines the "adjacent" table and finds all the immediate children of a node * @param integer $i_id The unique id for a node in the adjacent_table table * @return array Returns an array of results or an empty array if no results. */ function get_source_children($i_id) { $a_return = array(); $s_query = "SELECT * FROM `adjacent_table` WHERE `father_id` = '".$i_id."'"; if (!$i_result = mysql_query($s_query)) { echo "<pre>$s_query</pre>\n"; throw new Exception(mysql_error()); } if (mysql_num_rows($i_result) > 0) { while($a = mysql_fetch_assoc($i_result)) { $a_return[] = $a; } } return $a_return; } ?> This is the output of the above script. INSERT INTO nested_table (id, lft, rgt, category) VALUES( NULL, '2', '5', 'Hard Cover' ) INSERT INTO nested_table (id, lft, rgt, category) VALUES( NULL, '2', '7', 'Large Format' ) INSERT INTO nested_table (id, lft, rgt, category) VALUES( NULL, '1', '8', 'Books' ) INSERT INTO nested_table (id, lft, rgt, category) VALUES( NULL, '1', '10', 'CD\'s' ) INSERT INTO nested_table (id, lft, rgt, category) VALUES( NULL, '10', '13', 'Vintage' ) INSERT INTO nested_table (id, lft, rgt, category) VALUES( NULL, '1', '14', 'Magazines' ) INSERT INTO nested_table (id, lft, rgt, category) VALUES( NULL, '0', '15', 'ROOT' ) As you can see, there are multiple rows sharing the lft value of "1" same goes for "2" In a nested-set, the values for left and right must be unique. Here is an example of how to manually number the left and right ID's in a nested set: UPDATE - PROBLEM SOLVED First off, I had mistakenly believed that the source table (the one in adjacent-lists format) needed to be altered to include a source node. This is not the case. Secondly, I found a cached page on BING (of all places) with a class that does the trick. I've altered it for PHP5 and converted the original author's mysql related bits to basic PHP. He was using some DB class. You can convert them to your own database abstraction class later if you want. Obviously, if your "source table" has other columns that you want to move to the nested set table, you will have to adjust the write method in the class below. Hopefully this will save someone else from the same problems in the future. <?php /** -- -- Table structure for table `adjacent_table` -- DROP TABLE IF EXISTS `adjacent_table`; CREATE TABLE IF NOT EXISTS `adjacent_table` ( `id` int(11) NOT NULL AUTO_INCREMENT, `father_id` int(11) DEFAULT NULL, `category` varchar(128) DEFAULT NULL, PRIMARY KEY (`id`) ) ENGINE=MyISAM DEFAULT CHARSET=latin1 AUTO_INCREMENT=8 ; -- -- Dumping data for table `adjacent_table` -- INSERT INTO `adjacent_table` (`id`, `father_id`, `category`) VALUES (1, 0, 'Books'), (2, 0, 'CD''s'), (3, 0, 'Magazines'), (4, 1, 'Hard Cover'), (5, 1, 'Large Format'), (6, 3, 'Vintage'); -- -- Table structure for table `nested_table` -- DROP TABLE IF EXISTS `nested_table`; CREATE TABLE IF NOT EXISTS `nested_table` ( `lft` int(11) NOT NULL DEFAULT '0', `rgt` int(11) DEFAULT NULL, `id` int(11) DEFAULT NULL, `category` varchar(128) DEFAULT NULL, PRIMARY KEY (`lft`), UNIQUE KEY `id` (`id`), UNIQUE KEY `rgt` (`rgt`) ) ENGINE=MyISAM DEFAULT CHARSET=latin1; */ /** * @class tree_transformer * @author Paul Houle, Matthew Toledo * @created 2008-11-04 * @url http://gen5.info/q/2008/11/04/nested-sets-php-verb-objects-and-noun-objects/ */ class tree_transformer { private $i_count; private $a_link; public function __construct($a_link) { if(!is_array($a_link)) throw new Exception("First parameter should be an array. Instead, it was type '".gettype($a_link)."'"); $this->i_count = 1; $this->a_link= $a_link; } public function traverse($i_id) { $i_lft = $this->i_count; $this->i_count++; $a_kid = $this->get_children($i_id); if ($a_kid) { foreach($a_kid as $a_child) { $this->traverse($a_child); } } $i_rgt=$this->i_count; $this->i_count++; $this->write($i_lft,$i_rgt,$i_id); } private function get_children($i_id) { return $this->a_link[$i_id]; } private function write($i_lft,$i_rgt,$i_id) { // fetch the source column $s_query = "SELECT * FROM `adjacent_table` WHERE `id` = '".$i_id."'"; if (!$i_result = mysql_query($s_query)) { echo "<pre>$s_query</pre>\n"; throw new Exception(mysql_error()); } $a_source = array(); if (mysql_num_rows($i_result)) { $a_source = mysql_fetch_assoc($i_result); } // root node? label it unless already labeled in source table if (1 == $i_lft && empty($a_source['category'])) { $a_source['category'] = 'ROOT'; } // insert into the new nested tree table // use mysql_real_escape_string because one value "CD's" has a single ' $s_query = " INSERT INTO `nested_table` (`id`,`lft`,`rgt`,`category`) VALUES ( '".$i_id."', '".$i_lft."', '".$i_rgt."', '".mysql_real_escape_string($a_source['category'])."' ) "; if (!$i_result = mysql_query($s_query)) { echo "<pre>$s_query</pre>\n"; throw new Exception(mysql_error()); } else { // success: provide feedback echo "<p>$s_query</p>\n"; } } } mysql_connect('localhost','USER','PASSWORD') or die(mysql_error()); mysql_select_db('DATABASE') or die(mysql_error()); // build a complete copy of the adjacency table in ram $s_query = "SELECT `id`,`father_id` FROM `adjacent_table`"; $i_result = mysql_query($s_query); $a_rows = array(); while ($a_rows[] = mysql_fetch_assoc($i_result)); $a_link = array(); foreach($a_rows as $a_row) { $i_father_id = $a_row['father_id']; $i_child_id = $a_row['id']; if (!array_key_exists($i_father_id,$a_link)) { $a_link[$i_father_id]=array(); } $a_link[$i_father_id][]=$i_child_id; } $o_tree_transformer = new tree_transformer($a_link); $o_tree_transformer->traverse(0); ?>

Developer IT