Search Results

Search found 754 results on 31 pages for 'aggregate'.

Page 24/31 | < Previous Page | 20 21 22 23 24 25 26 27 28 29 30 31  | Next Page >

• Scalable / Parallel Large Graph Analysis Library?

  - by Joel Hoff

  I am looking for good recommendations for scalable and/or parallel large graph analysis libraries in various languages. The problems I am working on involve significant computational analysis of graphs/networks with 1-100 million nodes and 10 million to 1+ billion edges. The largest SMP computer I am using has 256 GB memory, but I also have access to an HPC cluster with 1000 cores, 2 TB aggregate memory, and MPI for communication. I am primarily looking for scalable, high-performance graph libraries that could be used in either single or multi-threaded scenarios, but parallel analysis libraries based on MPI or a similar protocol for communication and/or distributed memory are also of interest for high-end problems. Target programming languages include C++, C, Java, and Python. My research to-date has come up with the following possible solutions for these languages: C++ -- The most viable solutions appear to be the Boost Graph Library and Parallel Boost Graph Library. I have looked briefly at MTGL, but it is currently slanted more toward massively multithreaded hardware architectures like the Cray XMT. C - igraph and SNAP (Small-world Network Analysis and Partitioning); latter uses OpenMP for parallelism on SMP systems. Java - I have found no parallel libraries here yet, but JGraphT and perhaps JUNG are leading contenders in the non-parallel space. Python - igraph and NetworkX look like the most solid options, though neither is parallel. There used to be Python bindings for BGL, but these are now unsupported; last release in 2005 looks stale now. Other topics here on SO that I've looked at have discussed graph libraries in C++, Java, Python, and other languages. However, none of these topics focused significantly on scalability. Does anyone have recommendations they can offer based on experience with any of the above or other library packages when applied to large graph analysis problems? Performance, scalability, and code stability/maturity are my primary concerns. Most of the specialized algorithms will be developed by my team with the exception of any graph-oriented parallel communication or distributed memory frameworks (where the graph state is distributed across a cluster).

  Read the article

• SQL SERVER 2008 JOIN hints

  - by Nai

  Hi all, Recently, I was trying to optimise this query UPDATE Analytics SET UserID = x.UserID FROM Analytics z INNER JOIN UserDetail x ON x.UserGUID = z.UserGUID Estimated execution plan show 57% on the Table Update and 40% on a Hash Match (Aggregate). I did some snooping around and came across the topic of JOIN hints. So I added a LOOP hint to my inner join and WA-ZHAM! The new execution plan shows 38% on the Table Update and 58% on an Index Seek. So I was about to start applying LOOP hints to all my queries until prudence got the better of me. After some googling, I realised that JOIN hints are not very well covered in BOL. Therefore... Can someone please tell me why applying LOOP hints to all my queries is a bad idea. I read somewhere that a LOOP JOIN is default JOIN method for query optimiser but couldn't verify the validity of the statement? When are JOIN hints used? When the sh*t hits the fan and ghost busters ain't in town? What's the difference between LOOP, HASH and MERGE hints? BOL states that MERGE seems to be the slowest but what is the application of each hint? Thanks for your time and help people! I'm running SQL Server 2008 BTW. The statistics mentioned above are ESTIMATED execution plans.

  Read the article

• LINQ compiled query DataBind issue

  - by Brian

  Hello All, I have a pretty extensive reporting page that uses LINQ. There is one main function that returns an IQueryable object. I then further filter / aggregate the returned query depending on the report the user needs. I changed this function to a compiled query, it worked great, and the speed increase was astonishing. The only problem comes when i want to databind the results. I am databinding to a standard asp.net GridView and it works fine, no problems. I am also databinding to an asp.net chart control, this is where my page is throwing an error. this works well: GridView gv = new GridView(); gv.DataSource = gridSource; But this does not: Series s1 = new Series("Series One"); s1.Points.DataBindXY(gridSource, "Month", gridSource, "Success"); The error i receive is this: System.NotSupportedException Specified method is not supported When i look into my gridSource var at run time i see this using a typical linq query: SELECT [t33].[value2] AS [Year], [t33].[value22] AS [Month], [t33].[value3] AS [Calls]...... I see this after i change the query to compiled: {System.Linq.OrderedEnumerable<<>f__AnonymousType15<string,int,int,int,int,int,int,int>,string>} This is obviously the reason why the databindxy is no longer working, but i am not sure how to get around it. Any help would be appreciated! Thanks

  Read the article

• Adding Table Columns to a Group by clause - Ruby on Rails - Postgresql

  - by bgadoci

  I am trying to use Heroku and apparently Postgresql is a lot more strict than SQL for aggregate functions. When I am pushing to Heroku I am getting an error stating the below. On another question I asked I received some guidance that said I should just add the columns to my group by clause and I am not sure how to do that. See the full error below and the PostsControll#index. SELECT posts.*, count(*) as vote_total FROM "posts" INNER JOIN "votes" ON votes.post_id = posts.id GROUP BY votes.post_id ORDER BY created_at DESC LIMIT 5 OFFSET 0): PostsController def index @tag_counts = Tag.count(:group => :tag_name, :order => 'count_all DESC', :limit => 20) conditions, joins = {}, :votes @ugtag_counts = Ugtag.count(:group => :ugctag_name, :order => 'count_all DESC', :limit => 20) conditions, joins = {}, :votes @vote_counts = Vote.count(:group => :post_title, :order => 'count_all DESC', :limit => 20) conditions, joins = {}, :votes unless(params[:tag_name] || "").empty? conditions = ["tags.tag_name = ? ", params[:tag_name]] joins = [:tags, :votes] end @posts=Post.paginate( :select => "posts.*, count(*) as vote_total", :joins => joins, :conditions=> conditions, :group => "votes.post_id", :order => "created_at DESC", :page => params[:page], :per_page => 5) @popular_posts=Post.paginate( :select => "posts.*, count(*) as vote_total", :joins => joins, :conditions=> conditions, :group => "votes.post_id", :order => "vote_total DESC", :page => params[:page], :per_page => 3) respond_to do |format| format.html # index.html.erb format.xml { render :xml => @posts } format.json { render :json => @posts } format.atom end end

  Read the article

• Data Transfer Objects VS Domain/ActiveRecord Entities in the View in RoR

  - by leypascua

  I'm coming from a .NET background, where it is a practice to not bind domain/entity models directly to the view in not-so-basic CRUD-ish applications where the view does not directly project entity fields as-is. I'm wondering what's the practice in RoR, where the default persistence mechanism is ActiveRecord. I would assert that presentation-related info should not be leaked to the entities, not sure though if this is how real RoR heads would do it. If DTOs/model per view is the approach, how will you do it in Rails? Your thoughts? EDIT: Some examples: - A view shows a list of invoices, with the number of unique items in one column. - A list of credit card accounts, where possibly fraudulent transactions were executed. For that, the UI needs to show this row in red. For both scenarios, The lists don't show all of the fields of the entities, just a few to show in the list (like invoice #, transaction date, name of the account, the amount of the transaction) For the invoice example, The invoice entity doesn't have a field "No. of line items" mapped on it. The database has not been denormalized for perf reasons and it will be computed during query time using aggregate functions. For the credit card accounts example, surely the card transaction entity doesn't have a "Show-in-red" or "IsFraudulent" invariant. Yes it may be a business rule, but for this example, that is a presentation concern, so I would like to keep it out of my domain model.

  Read the article

• C# - Help with LINQ

  - by cmaduro

  I need to check if a certain property exists within a class. Please refer to the LINQ query in question. For the life of me I cannot make the compiler happy. class Program { static void Main(string[] args) { ModuleManager m = new ModuleManager(); IModule module = m.FindModuleForView(typeof(HomeView)); Console.WriteLine(module.GetType().ToString()); Console.ReadLine(); } } public class ModuleManager { [ImportMany] public IEnumerable<Lazy<IModule>> Modules { get; set; } [ImportMany] public IEnumerable<Lazy<View>> Views { get; set; } public ModuleManager() { //An aggregate catalog that combines multiple catalogs var catalog = new AggregateCatalog(); //Adds all the parts found in the same assembly as the Program class catalog.Catalogs.Add(new AssemblyCatalog(typeof(Program).Assembly)); //Create the CompositionContainer with the parts in the catalog _container = new CompositionContainer(catalog); //Fill the imports of this object try { this._container.ComposeParts(this); } catch (CompositionException compositionException) { Console.WriteLine(compositionException.ToString()); } } public IModule FindModuleForView(Type view) { //THIS IS THE PROBLEM var module = from m in Modules where ( from p in m.Value.GetType().GetProperties() where p.GetType().Equals(view) select p ) select m; } public CompositionContainer _container { get; set; } } public interface IModule { } [Export] public class HomeModule : IModule { public HomeModule() { } [Export] public HomeView MyHomeView { get { return new HomeView(); } set { } } } public class HomeView : View { } public class View { }

  Read the article

• Database warehoue design: fact tables and dimension tables

  - by morpheous

  I am building a poor man's data warehouse using a RDBMS. I have identified the key 'attributes' to be recorded as: sex (true/false) demographic classification (A, B, C etc) place of birth date of birth weight (recorded daily): The fact that is being recorded My requirements are to be able to run 'OLAP' queries that allow me to: 'slice and dice' 'drill up/down' the data and generally, be able to view the data from different perspectives After reading up on this topic area, the general consensus seems to be that this is best implemented using dimension tables rather than normalized tables. Assuming that this assertion is true (i.e. the solution is best implemented using fact and dimension tables), I would like to see some help in the design of these tables. 'Natural' (or obvious) dimensions are: Date dimension Geographical location Which have hierarchical attributes. However, I am struggling with how to model the following fields: sex (true/false) demographic classification (A, B, C etc) The reason I am struggling with these fields is that: They have no obvious hierarchical attributes which will aid aggregation (AFAIA) - which suggest they should be in a fact table They are mostly static or very rarely change - which suggests they should be in a dimension table. Maybe the heuristic I am using above is too crude? I will give some examples on the type of analysis I would like to carryout on the data warehouse - hopefully that will clarify things further. I would like to aggregate and analyze the data by sex and demographic classification - e.g. answer questions like: How does male and female weights compare across different demographic classifications? Which demographic classification (male AND female), show the most increase in weight this quarter. etc. Can anyone clarify whether sex and demographic classification are part of the fact table, or whether they are (as I suspect) dimension tables.? Also assuming they are dimension tables, could someone elaborate on the table structures (i.e. the fields)? The 'obvious' schema: CREATE TABLE sex_type (is_male int); CREATE TABLE demographic_category (id int, name varchar(4)); may not be the correct one.

  Read the article

• ASP.NET: aggregating validators in a user control

  - by orsogufo

  I am developing a web application where I would like to perform a set of validations on a certain field (an account name in the specific case). I need to check that the value is not empty, matches a certain pattern and is not already used. I tried to create a UserControl that aggregates a RequiredFieldValidator, a RegexValidator and a CustomValidator, then I created a ControlToValidate property like this: public partial class AccountNameValidator : System.Web.UI.UserControl { public string ControlToValidate { get { return ViewState["ControlToValidate"] as string; } set { ViewState["ControlToValidate"] = value; AccountNameRequiredFieldValidator.ControlToValidate = value; AccountNameRegexValidator.ControlToValidate = value; AccountNameUniqueValidator.ControlToValidate = value; } } } However, if I insert the control on a page and set ControlToValidate to some control ID, when the page loads I get an error that says Unable to find control id 'AccountName' referenced by the 'ControlToValidate' property of 'AccountNameRequiredFieldValidator', which makes me think that the controls inside my UserControl cannot resolve correctly the controls in the parent page. So, I have two questions: 1) Is it possible to have validator controls inside a UserControl validate a control in the parent page? 2) Is it correct and good practice to "aggregate" multiple validator controls in a UserControl? If not, what is the standard way to proceed?

  Read the article

• SQL Query - group by more than one column, but distinct

  - by Ranhiru

  I have a bidding table, as follows: SellID INT FOREIGN KEY REFERENCES SellItem(SellID), CusID INT FOREIGN KEY REFERENCES Customer(CusID), Amount FLOAT NOT NULL, BidTime DATETIME DEFAULT getdate() Now in my website I need to show the user the current bids; only the highest bid but without repeating the same user. SELECT CusID, Max(Amount) FROM Bid WHERE SellID = 10 GROUP BY CusID ORDER BY Max(Amount) DESC This is the best I have achieved so far. This gives the CusID of each user with the maximum bid and it is ordered ascending. But I need to get the BidTime for each result as well. When I try to put the BidTime in to the query: SELECT CusID, Max(Amount), BidTime FROM Bid WHERE SellID = 10 GROUP BY CusID ORDER BY Max(Amount) DESC I am told that "Column 'Bid.BidTime' is invalid in the select list because it is not contained in either an aggregate function or the GROUP BY clause." Thus I tried: SELECT CusID, Max(Amount), BidTime FROM Bid WHERE SellID = 10 GROUP BY CusID, BidTime ORDER BY Max(Amount) DESC But this returns all the rows. No distinction. Any suggestions on solving this issue?

  Read the article

• Application Architect vs. Systems Architect vs. Enterprise Architect?

  - by iaman00b

  So many buzzwords. Not sure if I need to start playing BS Bingo or not. And I'm not trying to be cynical. But I've heard many people with these various titles. There never seems to be a clear delineation between the three. Or there's a lot of domain crossover between the three. Actually, another I've seen while looking around here on Stackoverflow has been "Solutions Architect" as well. But that one doesn't seem to be so prevalent in other places. There are questions here and there with vague answers. But I'd like definative answers to this. Please assume I'm still relatively new to software stuff and that I'm trying to map out a career path. Oh, and please be gentle folks; this most definitely is not a duplicate question. Neither is it an aggregate. So kindly leave it alone. Xp

  Read the article

• Expression Tree with Property Inheritance causes an argument exception

  - by Adam Driscoll

  Following this post: link text I'm trying to create an expression tree that references the property of a property. My code looks like this: public interface IFoo { void X {get;set;} } public interface IBar : IFoo { void Y {get;set;} } public interface IFooBarContainer { IBar Bar {get;set;} } public class Filterer { //Where T = "IFooBarContainer" public IQueryable<T> Filter<T>(IEnumerable<T> collection) { var argument = Expression.Parameter(typeof (T), "item"); //... //where propertyName = "IBar.X"; PropertyOfProperty(argument, propertyName); } private static MemberExpression PropertyOfProperty(Expression expr, string propertyName) { return propertyName.Split('.').Aggregate<string, MemberExpression>(null, (current, property) => Expression.Property(current ?? expr, property)); } } I receive the exception: System.ArgumentException: Instance property 'X' is not defined for type 'IBar' ReSharper turned the code in the link above into the condensed statement in my example. Both forms of the method returned the same error. If I reference IBar.Y the method does not fail.

  Read the article

• Aggregating / Collecting AJAX requests

  - by Ganesh Shankar

  I have situation where a user can manipulate a large set of data (presented in a table) by using a bunch of filters represented as checkboxes. The page is AJAXed up so the user doesn't have to wait for a full page refresh every time they click a filter. The way it's currently implemented is by having an event handler watch all the checkboxes and request filtered data from the server when a click event is triggered. This works fine. However, there is a usability & performance issue with doing it this way. For example, if a user clicks 6 checkboxes, 6 AJAX requests are triggered and they all come back at various intervals causing the page to be updated 6 times. This will most probably annoy the user and seems rather inefficient. I want to put some kind of timeout on the event handler to do something like this: "Wait for 1 second and if there are no more filters clicked trigger the AJAX request". However, at the moment I've only been able to delay all 6 requests by 1 second. I'm not sure how to aggregate / collect the filter info into 1 AJAX request. Any suggestions would be greatly appreciated!

  Read the article

• Application log aggregation, management and notifications...

  - by Matthew Savage

  I'm wondering what everyone is using for logging, log management and log aggregation on their systems. I am working in a company which uses .NET for all it's applications and all systems are Windows based. Currently each application looks after its own logging and notifications of failures (e.g. if app A fails it will send out its own 'call for help' to an admin). While this current practice works its a bit hacky and hard to manage. I've been trying to find some options for making this work better and I've come up with the following: log4net & Chainsaw (ah, if it works). Logging via log4net or another framework into a central database & rolling our own management tool. Logging to the Windows event log and using MOM or System Center Operations Manager to aggregate and manage each of these servers & their apps. A hand-rolled solution to suck all the log files into one point and work some magic across them. Essentially what we are after is something which can pull log entries all together and allow for some analytics to be run across them, plus use a kind of event based system to, for example, send out a warning email when there have been 30+ warning level logs for an application in the last x minutes. So is there anything I've missed, or something someone else can suggest?

  Read the article

• Algorithm To Select Most Popular Places from Database

  - by Russell C.

  We have a website that contains a database of places. For each place our users are able to take one of the follow actions which we record: VIEW - View it's profile RATING - Rate it on a scale of 1-5 stars REVIEW - Review it COMPLETED - Mark that they've been there WISH LIST - Mark that they want to go there FAVORITE - Mark that it's one of their favorites In our database table of places each place contains a count of the number of times each action above was taken as well as the average rating given by users. views ratings avg_rating completed wishlist favorite What we want to be able to do is generate lists of the top places using the above information. Ideally, we would want to be able to generate this list using a relatively simple SQL query without needing to do any legwork to calculate additional fields or stack rank places against one another. That being said, since we only have about 50,000 places we could run a nightly cron job to calculate some fields such as rankings on different categories if it would make a meaningful difference in the overall results of our top places. I'd appreciate if you could make some suggestions on how we should think about bubbling the best places to the top, which criteria we should weight more heavily, and given that information - suggest what the MySQL query would need to look like in order to select the top 10 places. One thing to note is that at this time we are less concerned with the recency of a place being popular - meaning that looking at the aggregate information is fine and that more recent data doesn't need to be weighted more heavily. Thanks in advance for your help & advice!

  Read the article

• How to build a JSON response by combining @foo.to_json(options) and @bars.to_json(options) in Rails

  - by smotchkkiss

  First, the desired result I have User and Item models. I'd like to build a JSON response that looks like this: { "user": {"username":"Bob!","foo":"whatever","bar":"hello!"}, "items": [ {"id":1, "name":"one", "zim":"planet", "gir":"earth"}, {"id":2, "name":"two", "zim":"planet", "gir":"mars"} ] } However, my User and Item model have more attributes than just those. I found a way to get this to work, but beware, it's not pretty... Please help... My hacks home_controller.rb class HomeController < ApplicationController def observe respond_to do |format| format.js { render :json => Observation.new(current_user, @items).to_json } end end end observation.rb # NOTE: this is not a subclass of ActiveRecord::Base # this class just serves as a container to aggregate all "observable" objects class Observation attr_accessor :user, :items def initialize(user, items) self.user = user self.items = items end # The JSON needs to be decoded before it's sent to the `to_json` method in the home_controller otherwise the JSON will be escaped... # What a mess! def to_json { :user => ActiveSupport::JSON.decode(user.to_json(:only => :username, :methods => [:foo, :bar])), :items => ActiveSupport::JSON.decode(auctions.to_json(:only => [:id, :name], :methods => [:zim, :gir])) } end end

  Read the article

• How can I merge multiple Compass Resources into one, with one score?

  - by Brent Fisher

  I am trying to integrate compass into my platform using the JDBC ResultSetToResourceMapping. What I want to do is set it up so that I could have multiple result set mappings, tied to one Resource, that produces one result, with one score, and a merged score. I have tried to trick Compass into doing this by mapping the same id across them, even though the property fields are different, but it just ends up giving me separate hits for each. E.g. I have the following Data Model, Cases and Comments. One case might have several comments. Say I search for a term that appears in multiple comments. Right now, I a hit for each one, each with a different score. Is there a way that I could merge or aggregate those hits into one hit? Say, instead of Score Entity ID Snippets 100.0% Case 3558 ... The fox jumped over the lazy dog ... 60.0% Case 3558 ... In Alabama today, three jumping turtles were ... 25.0% Case 3558 ... Three jumpers fled the scene... I get Score Entity ID Snippets 100.0% Case 3558 The fox jumped over the lazy dog ...In Alabama today, three jumping turtles were ... Three jumpers fled the scene... Where the latter score is an aggregated score.

  Read the article

• JPA2 Criteria API creates invalid SQL when using groupBy

  - by Stephan

  JPA2 with the Criteria API seems to generate invalid SQL for PostgreSQL. For this code: Root<DBObjectAccessCounter> from = query.from(DBObjectAccessCounter.class); Path<DBObject> object = from.get(DBObjectAccessCounter_.object); Expression<Long> sum = builder.sumAsLong(from.get(DBObjectAccessCounter_.count)); query.multiselect(object, sum).groupBy(object); I get the following exception: ERROR: column "dbobject1_.id" must appear in the GROUP BY clause or be used in an aggregate function The generated SQL is: select dbobjectac0_.object_id as col_0_0_, sum(dbobjectac0_.count) as col_1_0_, dbobject1_.id as id1001_, dbobject1_.name as name1013_, dbobject1_.lastChanged as lastChan2_1013_, dbobject1_.type_id as type3_1013_ from DBObjectAccessCounter dbobjectac0_ inner join DBObject dbobject1_ on dbobjectac0_.object_id=dbobject1_.id group by dbobjectac0_.object_id Obviously, the first item of the select statement (dbobjectac0_.object_id) does not match the group by clause. Simplified example It does not even work for this simple example: Root<DBObjectAccessCounter> from = query.from(DBObjectAccessCounter.class); Path<DBObject> object = from.get(DBObjectAccessCounter_.object); query.select(object).groupBy(object); which returns select dbobject1_.id as id924_, dbobject1_.name as name933_, dbobject1_.lastChanged as lastChan2_933_, dbobject1_.type_id as type3_933_ from DBObjectAccessCounter dbobjectac0_ inner join DBObject dbobject1_ on dbobjectac0_.object_id=dbobject1_.id group by dbobjectac0_.object_id Does anyone know how to fix this?

  Read the article

• F# List SelectMany

  - by Tuomas Hietanen

  This is quite simple question but I didn't find an answer: Is there any Seq/List operation in F# to match the LINQ SelectMany? I know I can use System.Linq in F# if I want to. I know I can make a recursive method and use F# Computation Expressions (and make even more powerful things). But if I try to prove that F# List operations are more powerful than LINQ... .Where = List.filter .Select = List.map .Aggregate = List.fold ... In C# SelectMany usage syntax is pretty simple: var flattenedList = from i in items1 from j in items2 select ... Is there any easy direct match, List.flatten, List.bind or something like that? SelectMany has a couple of signatures, but the most complex one seems to be: IEnumerable<TResult> SelectMany<TSource, TCollection, TResult>( this IEnumerable<TSource> source, Func<TSource, IEnumerable<TCollection>> collectionSelector, Func<TSource, TCollection, TResult> resultSelector ); In F# terms this would be: ('a -> 'b list) -> ('a -> 'b -> 'c) -> 'a list -> 'c list

  Read the article

• Java - SwingWorker - problem

  - by Yatendra Goel

  I am developing a Java Desktop Application. This app executes the same task public class MyTask implements Callable<MyObject> { in multiple thread simultaneously. Now, when a user clicks on a "start" button, I have created a SwingWorker myWorker and have executed it. Now, this myWorker creates multiple instances of MyTask and submits them to an ExecutorService. Each MyTask instance has a loop and generates an intermediate result at every iteration. Now, I want to collect these intermediate results from each MyTask instances as soon as they are generated. Then after collecting these intermediate results from every MyTask instance, I want to publish it through SwingWorker.publish(MyObject) so that the progress is shown on the EDT. Q1. How can I implement this? Should I make MyTask subclass of SwingWorker instead of Callable to get intermediate results also, because I think that Callable only returns final result. Q2. If the answer of Q1. is yes, then can you give me a small example to show how can I get those intermediate results and aggregate them and then publish them from main SwingWorker? Q3. If I can't use SwingWorker in this situation, then how can I implement this?

  Read the article

• Migrating a Large amount of data from old publishing site to new site

  - by tommizzle

  Hi, I am currently in the process of creating a new news/publishing site on the Movable Type platform. There are around 20 or so sites with 20,000+ rows of data to be moved/aggregated to ~8 sites (we have a number of location specific sites and are going to aggregate the content from these into 1 single site for each niche). We have discussed how to do this and came to the conclusion that it would probably be better to hire somebody to do it (I could probably do it, but i'm limited on time and am sure that a specialist would be more efficient). So my questions to you guys are: 1) What kind of skill set should we look for in an applicant? 2) There will be a large amount of input from our side... is getting somebody to work remotely out of the question? 3) How long would a task like this traditionally take (I know this question is very subjective, but an estimation would be awesome)? 4) Do you have any recommendations for firms who would be able to take on a large task like this? Thanks in advance, Tom

  Read the article

• Would this method work to scale out SQL queries?

  - by David

  I have a database containing a single huge table. At the moment a query can take anything from 10 to 20 minutes and I need that to go down to 10 seconds. I have spent months trying different products like GridSQL. GridSQL works fine, but is using its own parser which does not have all the needed features. I have also optimized my database in various ways without getting the speedup I need. I have a theory on how one could scale out queries, meaning that I utilize several nodes to run a single query in parallel. The idea is to take an incoming SQL query and simply run it exactly like it is on all the nodes. When the results are returned to a coordinator node, the same query is run on the union of the resultsets. I realize that an aggregate function like average need to be rewritten into a count and sum to the nodes and that the coordinator divides the sum of the sums with the sum of the counts to get the average. What kinds of problems could not easily be solved using this model. I believe one issue would be the count distinct function. Edit: I am getting so many nice suggestions, but none have addressed the method.

  Read the article

• Rails Active Record Mysql find query HAVING clause

  - by meetraghu28

  Is there a way to use the HAVING clause in some other way without using group by. I am using rails and following is a sample sccenario of the problem that i am facing. In rails you can use the Model.find(:all,:select,conditions,:group) function to get data. In this query i can specify a having clause in the :group param. But what if i dont have a group by clause but want to have a having clause in the result set. Ex: Lets take a query select sum(x) as a,b,c from y where "some_conditions" group by b,c; This query has a sum() aggregation on one of the fields. No if there is nothing to aggregate then my result should be an empty set. But mysql return a NULL row. So this problem can be solved by using select sum(x) as a,b from y where "some_conditions" group by b having a NOT NULL; but what happens in case i dont have a group by clause?? a query like below select sum(x) as a,b from y where "some_conditions"; so how to specify that sum(x) should not be NULL? Any solution that would return an empty set in this case instead of a NULL row will help and also that solution should be doable in rails. We can use subqueries to get this condition working with sumthin like this select * from ((select sum(x) as b FROM y where "some_condition") as subq) where subq.b is not null; but is there a better way to do this thru sql/rails ??

  Read the article

• CQRS and email notification

  - by t0PPy

  Reading up on CQRS there is a lot of talk of email notification - i'm wondering where to get the data from. Imagine a senario where one user invites other users to an event. To inform a user that he has been invitet to an event, he is send an email. The concrete mecanics might go like this: A "CreateEvent" command with an associated collection of user to invite, is received by the server. A new meeting aggregate is created and a method "InviteUser" is called for each user that is to be invited. Each time a user is invited to an event, a domain event "UserWasInvitedToEvent" is raised. An email notification sender picks up the domain event and sends out the notification email. Now my question is this: Where do i go for information to include in the email? Say i want to include a description of the event as well as the users name. Since this is CQRS i can't get it thru my domain model; All the properties of the domain objects are private! Should i then query the write side? Or maybe move email notification to a different service entirely? Any thoughts will be much appriciated!

  Read the article

• Need guidelines for optimizing WebGL performance by minimizing shader changes

  - by brainjam

  I'm trying to get an idea of the practicality of WebGL for rendering large architectural interior scenes, consisting of 100K's of triangles. These triangles are distributed over many objects, and there are many materials in the scene. On the other hand, there are no moving parts. And the materials tend to be fairly simple, mostly based on texture maps. There is a lot of texture map sharing .. for example all the chairs in scene will share a common map. There is also some multitexturing - up to three textures overlaid in a material. I've been doing a little experimentation and reading, and gather that frequently switching materials during a rendering pass will slow things down. For example, a scene with 200K triangles will have significant performance differences, depending on whether there are 10 or 1000 objects, assuming that each time an object is displayed a new material is set up. So it seems that if performance is important the scene should be sorted by materials so as to minimize material switching. What I'm looking for is guidelines on how to think of the overhead of various state changes, and where do I get the biggest bang for the buck. For example, what are the relative performance costs of, say, gl.useProgram(), gl.uniformMatrix4fv(), gl.drawElements() should I try to write ubershaders to minimize shader switching? should I try to aggregate geometry to minimize the number of gl.drawElements() calls I realize that mileage may vary depending on browser, OS, and graphics hardware. And I'm also not looking for heroic measures. Just some guidelines from people who have already had some experience in making scenes fast. I'll add that while I've had some experience with fixed-pipeline OpenGL programming in the past, I'm rather new to the WebGL/OpenGL ES 2.0 way of doing things.

  Read the article

• Recommend me an architecture for this Facebook application

  - by andybaird

  Firstly, this question is subjective. There is not a right answer for this question and it really depends on what works for you. I'm hoping to use this thread as a breeding ground for ideas. I hope this is acceptable in this medium. I'm working on building a Facebook app that will be replacing an already popular app that gets ~50k hits a day. The original app is using a very typical LAMP setup with help from some Zend libraries for database layer extraction. For the most part the app worked well, except to solve a lot of issues I ended up fragmenting tables to speed things up. As a result, I couldn't do a lot of things with the app that I wanted to (namely any processing using aggregate data that needed to be returned quickly) So I'm starting to design plans for the next version of this application, and I have a whole bunch of new and cool features that I know would choke my current setup. I'm looking for technological recommendations of data storage methods that scale well. The database does not necessarily need to be relational, simple key/value storage would suffice (although at present time I know little to nothing about KV stores) What's your recommendation? How would you tackle this? I'd like to take a completely free approach to this -- although I am most familiar and comfortable using PHP, I want to leave all technical options open.

  Read the article

< Previous Page | 20 21 22 23 24 25 26 27 28 29 30 31  | Next Page >