Search Results

Search found 16914 results on 677 pages for 'single threaded'.

Page 627/677 | < Previous Page | 623 624 625 626 627 628 629 630 631 632 633 634  | Next Page >

• Optimising speeds in HDF5 using Pytables

  - by Sree Aurovindh

  The problem is with respect to the writing speed of the computer (10 * 32 bit machine) and the postgresql query performance.I will explain the scenario in detail. I have data about 80 Gb (along with approprite database indexes in place). I am trying to read it from Postgresql database and writing it into HDF5 using Pytables.I have 1 table and 5 variable arrays in one hdf5 file.The implementation of Hdf5 is not multithreaded or enabled for symmetric multi processing.I have rented about 10 computers for a day and trying to write them inorder to speed up my data handling. As for as the postgresql table is concerned the overall record size is 140 million and I have 5 primary- foreign key referring tables.I am not using joins as it is not scalable So for a single lookup i do 6 lookup without joins and write them into hdf5 format. For each lookup i do 6 inserts into each of the table and its corresponding arrays. The queries are really simple select * from x.train where tr_id=1 (primary key & indexed) select q_t from x.qt where q_id=2 (non-primary key but indexed) (similarly five queries) Each computer writes two hdf5 files and hence the total count comes around 20 files. Some Calculations and statistics: Total number of records : 14,37,00,000 Total number of records per file : 143700000/20 =71,85,000 The total number of records in each file : 71,85,000 * 5 = 3,59,25,000 Current Postgresql database config : My current Machine : 8GB RAM with i7 2nd generation Processor. I made changes to the following to postgresql configuration file : shared_buffers : 2 GB effective_cache_size : 4 GB Note on current performance: I have run it for about ten hours and the performance is as follows: The total number of records written for each file is about 6,21,000 * 5 = 31,05,000 The bottle neck is that i can only rent it for 10 hours per day (overnight) and if it processes in this speed it will take about 11 days which is too high for my experiments. Please suggest me on how to improve. Questions: 1. Should i use Symmetric multi processing on those desktops(it has 2 cores with about 2 GB of RAM).In that case what is suggested or prefereable? 2. If i change my postgresql configuration file and increase the RAM will it enhance my process. 3. Should i use multi threading.. In that case any links or pointers would be of great help Thanks Sree aurovindh V

  Read the article

• How to perform duplicate key validation using entlib (or DataAnnotations), MVC, and Repository pattern

  - by olivehour

  I have a set of ASP.NET 4 projects that culminate in an MVC (3 RC2) app. The solution uses Unity and EntLib Validation for cross-cutting dependency injection and validation. Both are working great for injecting repository and service layer implementations. However, I can't figure out how to do duplicate key validation. For example, when a user registers, we want to make sure they don't pick a UserID that someone else is already using. For this type of validation, the validating object must have a repository reference... or some other way to get an IQueryable / IEnumerable reference to check against other rows already in the DB. What I have is a UserMetadata class that has all of the property setters and getters for a user, along with all of the appropriate DataAnnotations and EntLib Validation attributes. There is also a UserEntity class implemented using EF4 POCO Entity Generator templates. The UserEntity depends on UserMetadata, because it has a MetadataTypeAttribute. I also have a UserViewModel class that has the same exact MetadataType attribute. This way, I can apply the same validation rules, via attributes, to both the entity and viewmodel. There are no concrete references to the Repository classes whatsoever. All repositories are injected using Unity. There is also a service layer that gets dependency injection. In the MVC project, service layer implementation classes are injected into the Controller classes (the controller classes only contain service layer interface references). Unity then injects the Repository implementations into the service layer classes (service classes also only contain interface references). I've experimented with the DataAnnotations CustomValidationAttribute in the metadata class. The problem with this is the validation method must be static, and the method cannot instantiate a repository implementation directly. My repository interface is IRepository, and I have only one single repository implementation class defined as EntityRepository for all domain objects. To instantiate a repository explicitly I would need to say new EntityRepository(), which would result in a circular dependency graph: UserMetadata [depends on] DuplicateUserIDValidator [depends on] UserEntity [depends on] UserMetadata. I've also tried creating a custom EntLib Validator along with a custom validation attribute. Here I don't have the same problem with a static method. I think I could get this to work if I could just figure out how to make Unity inject my EntityRepository into the validator class... which I can't. Right now, all of the validation code is in my Metadata class library, since that's where the custom validation attribute would go. Any ideas on how to perform validations that need to check against the current repository state? Can Unity be used to inject a dependency into a lower-layer class library?

  Read the article

• JSON or YAML encoding in GWT/Java on both client and server

  - by KennethJ

  I'm looking for a super simple JSON or YAML library (not particularly bothered which one) written in Java, and can be used in both GWT on the client, and in its original Java form on the server. What I'm trying to do is this: I have my models, which are shared between the client and the server, and these are the primary source of data interchange. I want to design the web service in between to be as simple as possible, and decided to take the RESTful approach. My problem is that I know our application will grow substantially in the future, and writing all the getters, setters, serialization, factories, etc. by hand fills me with absolute dread. So in order to avoid it, I decided to implement annotations to keep track of attributes on the models. The reason I can't just serialize everything directly, using GWT's own one, or one which works through reflection, is because we need a certain amount of logic going on in the serialization process. I.e. whether references to other models get serialized during the serialization of the original model, or whether an ID is just passed, and general simple things like that. I've then written an annotation processor to preprocess my shared models and generate an implementing class with all the getters, setters, serialization, lazy-loading, etc. To make a long story short, I need some type of simple YAML or JSON library, which allows me to encode and decode manually, so I can generate this code through my annotation processor. I have had a look around the interwebs, but every single one I ran into supported some reflection which, while all fine and dandy, make it pretty much useless for GWT. And in the case of GWT's own JSON library, it uses JSNI for speed purposes, making it useless server side. One solution I did think about involved writing writing two sets of serialization methods on the models, one for the client and one for the server, but I'd rather not do that. Also, I'm pretty new to GWT, and even though I have done a lot of Java, it was back in the 1.2 days, so it's a bit rusty. So if you think I'm going about this problem completely the wrong way, I'm open to suggestions.

  Read the article

• Selecting the contents of an ASP.NET TextBox in an UpdatePanel after a partial page postback

  - by Scott Mitchell

  I am having problems selecting the text within a TextBox in an UpdatePanel. Consider a very simple page that contains a single UpdatePanel. Within that UpdatePanel there are two Web controls: A DropDownList with three statically-defined list items, whose AutoPostBack property is set to True, and A TextBox Web control The DropDownList has a server-side event handler for its SelectedIndexChanged event, and in that event handler there's two lines of code: TextBox1.Text = "Whatever"; ScriptManager.RegisterStartupScript(this, this.GetType(), "Select-" + TextBox1.ClientID, string.Format("document.getElementById('{0}').select();", TextBox1.ClientID), true); The idea is that whenever a user chooses and item from the DropDownList there is a partial page postback, at which point the TextBox's Text property is set and selected (via the injected JavaScript). Unfortunately, this doesn't work as-is. (I have also tried putting the script in the pageLoad function with no luck, as in: ScriptManager.RegisterStartupScript(..., "function pageLoad() { ... my script ... }");) What happens is the code runs, but something else on the page receives focus at the conclusion of the partial page postback, causing the TextBox's text to be unselected. I can "fix" this by using JavaScript's setTimeout to delay the execution of my JavaScript code. For instance, if I update the emitted JavaScript to the following: setTimeout("document.getElementById('{0}').select();", 111); It "works." I put works in quotes because it works for this simple page on my computer. In a more complex page on a slower computer with more markup getting passed between the client and server on the partial page postback, I have to up the timeout to over a second to get it to work. I would hope that there is a more foolproof way to achieve this. Rather than saying, "Delay for X milliseconds," it would be ideal to say, "Run this when you're not going to steal the focus." What's perplexing is that the .Focus() method works beautifully. That is, if I scrap my JavaScript and replace it with a call to TextBox1.Focus(); then the TextBox receives focus (although the text is not selected). I've examined the contents of MicrosoftAjaxWebForms.js and see that the focus is set after the registered scripts run, but I'm my JavaScript skills are not strong enough to decode what all is happening here and why the selected text is unselected between the time it is selected and the end of the partial page postback. I've also tried using Firebug's JavaScript debugger and see that when my script runs the TextBox's text is selected. As I continue to step through it the text remains selected, but then after stepping off the last line of script (apparently) it all of the sudden gets unselected. Any ideas? I am pulling my hair out. Thanks in advance...

  Read the article

• Large number of UPDATE queries slowing down page

  - by Bryan Lewis

  I am reading and validating large fixed-width text files (range from 10-50K lines) that are submitted via our ASP.net website (coded in VB.Net). I do an initial scan of the file to check for basic issues (line length, etc). Then I import each row into a MS SQL table. Each DB rows basically consists of a record_ID (Primary, auto-incrementing) and about 50 varchar fields. After the insert is done, I run a validation function on the file that checks each field in each row based on a bunch of criteria (trimmed length, isnumeric, range checks, etc). If it finds an error in any field, it inserts a record into the Errors table, which has an error_ID, the record_ID and an error message. In addition, if the field fails in a particular way, I have to do a "reset" on that field. A reset might consist of blanking the entire field, or simply replacing the value with another value (e.g. replacing the string with a new one that has all illegals chars taken out). I have a 5,000 line test file. The upload, initial check, and import takes about 5-6 seconds. The detailed error check and insert into the Errors table takes about 5-8 seconds (this file has about 1200 errors in it). However, the "resets" part takes about 40-45 seconds for 750 fields that need to be reset. When I comment out the resets function (returning immediately without actually calling the UPDATE stored proc), the process is very fast. With the resets turned on, the pages take 50 seconds to return. My UPDATE stored proc is using some recommended code from http://sommarskog.se/dynamic_sql.html, whereby it uses CASE instead of dynamic SQL: UPDATE dbo.Records SET dbo.Records.file_ID = CASE @field_name WHEN 'file_ID' THEN @field_value ELSE file_ID END, . . (all 50 varchar field CASE statements here) . WHERE dbo.Records.record_ID = @record_ID Is there any way I can help my performance here. Can I somehow group all of these UPDATE calls into a single transaction? Should I be reworking the UPDATE query somehow? Or is it just sheer quantity of 750+ UPDATEs and things are just slow (it's a quad proc server with 8GB ram). Any suggestions appreciated.

  Read the article

• Implementing a logging library in .NET with a database as the storage medium

  - by Dave

  I'm just starting to work on a logging library that everyone can use to keep track of any sort of system information while the user is running our application. The simplest example so far is to track Info, Warnings, and Errors. I want all plugins to be able to use this feature, but since each developer might have a different idea of what's important to report, I want to keep this as generic as possible. In the C++ world, I would normally use something like a stl::pair<string,string> to act as a key value pair structure, and have a stl::list of these to act as a "row" in the log. The log cache would then be a list<list<pair<string,string>>> (ugh!). This way, the developers can use a const string key like INFO, WARNING, ERROR to have a consistent naming for a column in the database (for SELECTing specific types of information). I'd like the database to be able to deal with any number of distinct column names. For example, John might have an INFO row with a column called USER, and Bill might have an INFO row with a column called FILENAME. I want the log viewer to be able to display all information, and if one report doesn't have a value for INFO / FILENAME, those fields should just appear blank. So one option is to use List<List<KeyValuePair<String,String>>, and the another is to have the log library consumer somehow "register" its schema, and then have the database do an ALTER TABLE to handle this situation. Yet another idea is to have a table that's just for key value pairs, with a foreign key that maps the key value pairs back to the original log entry. I obviously don't want logging to bog down the system, so I only lock the log cache to make a copy of the data (and remove the already-copied data), then a background thread will dump the information to the database. My specific questions regarding this are: Do you see any performance issues? In other words, have you ever tried something like this and found that certain things just don't work well in practice? Is there a more .NETish way to implement the key value pairs, other than List<List<KeyValuePair<String,String>>>? Even if there is a way to do #2 better, is the ALTER TABLE idea I proposed above a Bad Thing? Would you recommend multiple databases over a single one? I don't yet have an idea of how frequently the log would get written to, but we ideally would like to have lots of low level information. Perhaps there should be a DB with a fixed schema only for the low level stuff, and then another DB that's more flexible for reporting information back to users.

  Read the article

• Move options between multiple lists

  - by Martha

  We currently have a form with the standard multi-select functionality of "here are the available options, here are the selected options, here are some buttons to move stuff back and forth." However, the client now wants the ability to not just select certain items, but to also categorize them. For example, given a list of books, they want to not just select the ones they own, but also the ones they've read, the ones they would like to read, and the ones they've heard about. (All examples fictional.) Thankfully, a selected item can only be in one category at a time. I can find many examples of moving items between listboxes, but not a single one for moving items between multiple listboxes. To add to the complication, the form needs to have two sets of list+categories, e.g. a list of movies that need to be categorized in addition to the aforementioned books. EDIT: Having now actually sat down to try to code the non-javascripty bits, I need to revise my question, because I realized that multiple select lists won't really work from the "how do I inform the server about all this lovely new information" standpoint. So the html code is now a pseudo-listbox, i.e. an unordered list (ul) displayed in a box with a scrollbar, and each list item (<li>) has a set of five radio buttons (unselected/own/read/like/heard). My task is still roughly the same: how to take this one list and make it easy to categorize the items, in such a way that the user can tell at a glance what is in what category. (The pseudo-listbox has some of the same disadvantages as a multi-select listbox, namely it's hard to tell what's selected if the list is long enough to scroll.) The dream solution would be a drag-and-drop type thing, but at this point even buttons would be OK. Another modification (a good one) is that the client has revised the lists, so the longest is now "only" 62 items long (instead of the many hundreds they had before). The categories will still mostly contain zero, one, or two selected items, possibly a couple more if the user was overzealous. As far as OS and stuff, the site is in classic asp (quit snickering!), the server-side code is VBScript, and so far we've avoided the various Javascript libraries by the simple expedient of almost never using client-side scripting. This one form for this one client is currently the big exception. Give 'em an inch and they want a mile... Oh, and I have to add: I suck at Javascript, or really at any C-descendant language. Curly braces give me hives. I'd really, really like something I can just copy & paste into my page, maybe tweak some variable names, and never look at it again. A girl can dream, can't she? :) [existing code deleted because it's largely irrelevant.]

  Read the article

• Why does my Perl regular expression only find the last occurrence?

  - by scharan

  I have the following input to a Perl script and I wish to get the first occurrence of NAME="..." strings in each of the <table>...</table> structures. The entire file is read into a single string and the regex acts on that input. However, the regex always returns the last occurrence of NAME="..." strings. Can anyone explain what is going on and how this can be fixed? Input file: ADSDF <TABLE> NAME="ORDERSAA" line1 line2 NAME="ORDERSA" line3 NAME="ORDERSAB" </TABLE> <TABLE> line1 line2 NAME="ORDERSB" line3 </TABLE> <TABLE> line1 line2 NAME="ORDERSC" line3 </TABLE> <TABLE> line1 line2 NAME="ORDERSD" line3 line3 line3 </TABLE> <TABLE> line1 line2 NAME="QUOTES2" line3 NAME="QUOTES3" NAME="QUOTES4" line3 NAME="QUOTES5" line3 </TABLE> <TABLE> line1 line2 NAME="QUOTES6" NAME="QUOTES7" NAME="QUOTES8" NAME="QUOTES9" line3 line3 </TABLE> <TABLE> NAME="MyName IsKhan" </TABLE> Perl Code starts here: use warnings; use strict; my $nameRegExp = '(<table>((NAME="(.+)")|(.*|\n))*</table>)'; sub extractNames($$){ my ($ifh, $ofh) = @_; my $fullFile; read ($ifh, $fullFile, 1024);#Hardcoded to read just 1024 bytes. while( $fullFile =~ m#$nameRegExp#gi){ print "found: ".$4."\n"; } } sub main(){ if( ($#ARGV + 1 )!= 1){ die("Usage: extractNames infile\n"); } my $infileName = $ARGV[0]; my $outfileName = $ARGV[1]; open my $inFile, "<$infileName" or die("Could not open log file $infileName"); my $outFile; #open my $outFile, ">$outfileName" or die("Could not open log file $outfileName"); extractNames( $inFile, $outFile ); close( $inFile ); #close( $outFile ); } #call main();

  Read the article

• CakePHP: How can I change this find call to include all records that do not exist in the associated

  - by Stephen

  I have a few tables with the following relationships: Company hasMany Jobs, Employees, and Trucks, Users I've got all my foreign keys set up properly, along with the tables' Models, Controllers, and Views. Originally, the Jobs table had a boolean field called "assigned". The following find operation (from the JobsController) successfully returns all employees, all trucks, and any jobs that are not assigned and fall on a certain day for a single company (without returning users by utilizing the containable behavior): $this->set('resources', $this->Job->Company->find('first', array( 'conditions' => array( 'Company.id' => $company_id ), 'contain' => array( 'Employee', 'Truck', 'Job' => array( 'conditions' => array( 'Job.assigned' => false, 'Job.pickup_date' => date('Y-m-d', strtotime('Today')); ) ) ) ))); Now, since writing this code, I decided to do a lot more with the job assignments. So I've created a new model "Assignment" that belongsTo Truck and belongsTo Job. I've added the hasMany Assignments to both the Truck model and the Jobs Model. I have both foreign keys in the assignments table, along with some other assignment fields. Now, I'm trying to get the same information above, only instead of checking the assigned field from the job table, I want to check the assignments table to ensure that the job does not exist there. I can no longer use the containable behavior if I'm going to use the "joins" feature of the find method due to mysql errors (according to the cookbook). But, the following query returns all jobs, even if they fall on different days. $this->set('resources', $this->Job->Company->find('first', array( 'joins' => array( array( 'table' => 'employees', 'alias' => 'Employee', 'type' => 'LEFT', 'conditions' => array( 'Company.id = Employee.company_id' ) ), array( 'table' => 'trucks', 'alias' => 'Truck', 'type' => 'LEFT', 'conditions' => array( 'Company.id = Truck.company_id' ) ), array( 'table' => 'jobs', 'alias' => 'Job', 'type' => 'LEFT', 'conditions' => array( 'Company.id = Job.company_id' ) ), array( 'table' => 'assignments', 'alias' => 'Assignment', 'type' => 'LEFT', 'conditions' => array( 'Job.id = Assignment.job_id' ) ) ), 'conditions' => array( 'Job.pickup_date' => $day, 'Company.id' => $company_id, 'Assignment.job_id IS NULL' ) )));

  Read the article

• CellTable + AsyncListViewAdapter<T>, stuck at 'loading' when paging

  - by Jaroslav Záruba

  Hi I'm trying to make CellTable, AsyncListViewAdapter<T> and SimplePager<T> working together. I managed to display my data, but whenever I click either "go to start" or "go to end" button I get that 'loading' indicator which stays there till the end of days. ( AsyncListViewAdapter<T>.onRangeChanged doesn't even get called this time.) As I have only single row in my data those two buttons should be (and appear to be) disabled. I went through the code-snippets in this thread, but I can't see nothing suspicions in what I've been doing. Is there some obvious answer to a rookie mistake? I shrinked my variable names, hopefully it won't wrap too much. protected class MyAsyncAdapter extends AsyncListViewAdapter<DTO> { @Override protected void onRangeChanged(ListView<DTO> v) { // doesn't get called on go2start/go2end :( Range r = v.getRange(); fetchData(r.getStart(), r.getLength()); } } private void addTable() { // table: CellTable<DTO> table = new CellTable<DTO>(10); table.addColumn(new Column<DTO, String>(new TextCell()) { @Override public String getValue(DTO namespace) { return namespace.getName(); } }, "Name"); // pager: SimplePager<DTO> pager = new SimplePager<DTO>(table); table.setPager(pager); adapter = new MyAsyncAdapter(); adapter.addView(table); // does not make any difference: // adapter.updateDataSize(0, false); // adapter.updateDataSize(10, true); VerticalPanel vPanel = new VerticalPanel(); vPanel.add(table); vPanel.add(pager); RootLayoutPanel.get().add(vPanel); } // success-handler of my fetching AsyncCallback @Override public void onSuccess(List<DTO> data) { // AsyncCallback<List<DTO>> has start field adapter.updateViewData(start, data.size(), data); if(data.size() < length) adapter.updateDataSize(start + data.size(), true); } Regards J. Záruba

  Read the article

• Save HashMap data into SQLite

  - by Matthew

  I'm Trying to save data from Json into SQLite. For now I keep the data from Json into HashMap. I already search it, and there's said use the ContentValues. But I still don't get it how to use it. I try looking at this question save data to SQLite from json object using Hashmap in Android, but it doesn't help a lot. Is there any option that I can use to save the data from HashMap into SQLite? Here's My code. MainHellobali.java // Hashmap for ListView ArrayList<HashMap<String, String>> all_itemList; @Override protected void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); setContentView(R.layout.activity_main_helloballi); all_itemList = new ArrayList<HashMap<String, String>>(); // Calling async task to get json new getAllItem().execute(); } private class getAllItem extends AsyncTask<Void, Void, Void> { @Override protected Void doInBackground(Void... arg0) { // Creating service handler class instance ServiceHandler sh = new ServiceHandler(); // Making a request to url and getting response String jsonStr = sh.makeServiceCall(url, ServiceHandler.GET); Log.d("Response: ", "> " + jsonStr); if (jsonStr != null) { try { all_item = new JSONArray(jsonStr); // looping through All Contacts for (int i = 0; i < all_item.length(); i++) { JSONObject c = all_item.getJSONObject(i); String item_id = c.getString(TAG_ITEM_ID); String category_name = c.getString(TAG_CATEGORY_NAME); String item_name = c.getString(TAG_ITEM_NAME); // tmp hashmap for single contact HashMap<String, String> allItem = new HashMap<String, String>(); // adding each child node to HashMap key => value allItem.put(TAG_ITEM_ID, item_id); allItem.put(TAG_CATEGORY_NAME, category_name); allItem.put(TAG_ITEM_NAME, item_name); // adding contact to contact list all_itemList.add(allItem); } } catch (JSONException e) { e.printStackTrace(); } } else { Log.e("ServiceHandler", "Couldn't get any data from the url"); } return null; } } I have DatabasehHandler.java and AllItem.java too. I can put it in here if its necessary. Thanks before ** Add Edited Code ** // looping through All Contacts for (int i = 0; i < all_item.length(); i++) { JSONObject c = all_item.getJSONObject(i); String item_id = c.getString(TAG_ITEM_ID); String category_name = c.getString(TAG_CATEGORY_NAME); String item_name = c.getString(TAG_ITEM_NAME); DatabaseHandler databaseHandler = new DatabaseHandler(this); //error here "The Constructor DatabaseHandler(MainHellobali.getAllItem) is undefined }

  Read the article

• Simplifying a four-dimensional rule table in Matlab: addressing rows and columns of each dimension

  - by Cate

  Hi all. I'm currently trying to automatically generate a set of fuzzy rules for a set of observations which contain four values for each observation, where each observation will correspond to a state (a good example is with Fisher's Iris Data). In Matlab I am creating a four dimensional rule table where a single cell (a,b,c,d) will contain the corresponding state. To reduce the table I am following the Hong and Lee method of row and column similarity checking but I am having difficulty understanding how to address the third and fourth dimensions' rows and columns. From the method it is my understanding that each dimension is addressed individually and if the rule is true, the table is simplified. The rules for merging are as follows: If all cells in adjacent columns or rows are the same. If two cells are the same or if either of them is empty in adjacent columns or rows and at least one cell in both is not empty. If all cells in a column or row are empty and if cells in its two adjacent columns or rows are the same, merge the three. If all cells in a column or row are empty and if cells in its two adjacent columns or rows are the same or either of them is empty, merge the three. If all cells in a column or row are empty and if all the non-empty cells in the column or row to its left have the same region, and all the non-empty cells in the column or row to its right have the same region, but one different from the previously mentioned region, merge these three columns into two parts. Now for the confusing bit. Simply checking if the entire row/column is the same as the adjacent (rule 1) seems simple enough: if (a,:,:,:) == (a+1,:,:,:) (:,b,:,:) == (:,b+1,:,:) (:,:,c,:) == (:,:,c+1,:) (:,:,:,d) == (:,:,:,d+1) is this correct? but to check if the elements in the row/column match, or either is zero (rules 2 and 4), I am a bit lost. Would it be something along these lines: for a = 1:20 for i = 1:length(b) if (a+1,i,:,:) == (a,i,:,:) ... else if (a+1,i,:,:) == 0 ... else if (a,i,:,:) == 0 etc. and for the third and fourth dimensions: for c = 1:20 for i = 1:length(a) if (i,:,c,:) == (i,:,c+1,:) ... else if (i,:,c+1,:) == 0 ... else if (i,:,c,:) == 0 etc. for d = 1:20 for i = 1:length(a) if (i,:,:,d) == (i,:,:,d+1) ... else if (i,:,:,d+1) == 0 ... else if (i,:,:,d) == 0 etc. even any help with four dimensional arrays would be useful as I'm so confused by the thought of more than three! I would advise you look at the paper to understand my meaning - they themselves have used the Iris data but only given an example with a 2D table. Thanks in advance, hopefully!

  Read the article

• D3 fisheye on width on bar chart

  - by Dexter Tan

  i have been trying to create a vertical bar chart with a d3 fisheye cartesian distortion with only the x-axis being distorted. I have succeeded in distorting the x position of the vertical bars on mouseover with the following code: var maxMag = d3.max(dataset, function(d) { return d.value[10]; }); var minDate = d3.min(dataset, function(d) { return new Date(d.value[1], d.value[2]-1, d.value[3]).getTime(); }); var maxDate = d3.max(dataset, function(d) { return new Date(d.value[1], d.value[2]-1, d.value[3]).getTime(); }); var yScale = d3.scale.linear().domain([0, maxMag]).range([0, h]); var xScale = d3.fisheye.scale(d3.scale.linear).domain([minDate, maxDate]).range([0, w]).focus(w/2); var bar = svg.append("g") .attr("class", "bars") .selectAll(".bar") .data(dataset) .enter().append("rect") .attr("class", "bar") .attr("y", function(d) { return h - yScale(d.value[10]); }) .attr("width", w/dataset.length) .attr("height", function(d) { return yScale(d.value[10]); }) .attr("fill", function(d) { return (d.value[10] <= 6? "yellow" : "orange" ); }) .call(position); // Positions the bars based on data. function position(bar) { bar.attr("x", function(d) { var date = null; if (d.date != null) { date = d.date; } else { date = new Date(d.value[1],0,1); if (d.value[2] != null) date.setMonth(d.value[2]-1); if (d.value[3] != null) date.setMonth(d.value[3]); d.date = date; } return xScale(date.getTime()); }); } svg.on("mousemove", function() { var mouse = d3.mouse(this); xScale.distortion(2.5).focus(mouse[0]); bar.call(position); }); However at this point, applying fisheye on the width remains a mystery to me. I have tried several methods like using a fisheye scale for width however it does not work as expected. What i wish to do is to have the width of a bar expand on mouseover, the same way a single vertical bar is singled out on mouseover with the cartesian distortion. Any clues or help will be much appreciated! edit: http://dexter.xumx.me to view the visualisation i am talking about for easier understanding!

  Read the article

• Detecting abuse for post rating system

  - by Steven smethurst

  I am using a wordpress plugin called "GD Star Rating" to allow my users to vote on stories that I post to one of my websites. http://everydayfiction.com/ Recently we have been having a lot of abuse of the system. Stories that have obviously been voted up artificially. "GD Star Rating" creates some detailed logs when a user votes on a story. Including; IP, Time of vote, and user_adgent, ect.. For example this story has 181 votes with an average of 5.7 http://www.everydayfiction.com/snowman-by-shaun-simon/ Most other stories only get around ~40 votes each day. At first I thought that the story got on to a social bookmarking site Digg, Stumbleupon ect... but after checking the logs I found that this story is getting the same amount of traffic that a normal story gets ~2k-3k. I checked if all the votes for this perpendicular story where coming from a the same IP address. I could see this happening if a user was at a school's computer lab using all their lab computers to vote up this story. Not one duplicate IP address in the log for this story. SELECT ip, COUNT(*) as count FROM wp_gdsr_votes_log WHERE id=3932 GROUP BY (ip ) ORDER BY count DESC Next I thought that a use might be using a proxy to vote up a story. I checked this by grouping all the browser user_agent together to see if there a single browser voting in a perpendicular way. At most 7 users where using a similar browser but voted sporadically (1-5), no evidence of wrong doing. SELECT user_agent, COUNT(*) as count FROM wp_gdsr_votes_log WHERE id=3932 GROUP BY ( user_agent) ORDER BY count DESC I check was to see if all the votes came in at a once. Maybe someone has a really interesting bot that can change the user_adgent and uses proxies, ect... At most 5 votes came with in 2 mins of each other. It doesn't seem to be any regularity on how people vote (IE a 5 vote does not come in once a min) SELECT * FROM wp_gdsr_votes_log WHERE id =3932 AND vote=5 ORDER BY wp_gdsr_votes_log.voted DESC The obvious solution to this problem is to force people to login before they are allowed to vote. But I would prefer to not have to go down that route unless it is absolutely necessary. I'm looking for suggestions on things to test for to detect the abuse.

  Read the article

• Continuous Form, how to add/update records with external connection

  - by Mohgeroth

  EDIT After some more research I found that I cannot use a continuous form with an unbound form since it can only reference a single record at a time. Given that I've altered my question... I have a sample form that pulls out data to enter into a table as an intermediary. Initially the form is unbound and I open connections to two main recordsets. I set the listbox's recordset equal to one of them and the forms recordset equal to the other. The problem is that I cannot add records or update existing ones. Attempting to key into the fields does nothing almost as if the field was locked (Which it is not). Settings of the recordsets are OpenKeyset and LockPessimistic. Tables are not linked, they come from an outside access database seperate from this project and must remain that way. I am using an adodb connection to get the data. Could the separation of the data from the project be causing this? Sample Code from the Form Option Compare Database Option Explicit Private conn As CRobbers_Connections Private exception As CError_Trapping Private mClient_Translations As ADODB.Recordset Private mUnmatched_Clients As ADODB.Recordset Private mExcluded_Clients As ADODB.Recordset //Construction Private Sub Form_Open(Cancel As Integer) Set conn = New CRobbers_Connections Set exception = New CError_Trapping Set mClient_Translations = New ADODB.Recordset Set mUnmatched_Clients = New ADODB.Recordset Set mExcluded_Clients = New ADODB.Recordset mClient_Translations.Open "SELECT * FROM Client_Translation" _ , conn.RBRS_Conn, adOpenKeyset, adLockPessimistic mUnmatched_Clients.Open "SELECT DISTINCT(a.Client) as Client" _ & " FROM Master_Projections a " _ & " WHERE Client NOT IN ( " _ & " SELECT DISTINCT ClientID " _ & " FROM Client_Translation);" _ , conn.RBRS_Conn, adOpenKeyset, adLockPessimistic mExcluded_Clients.Open "SELECT * FROM Clients_Excluded" _ , conn.RBRS_Conn, adOpenKeyset, adLockPessimistic End Sub //Add new record to the client translations Private Sub cmdAddNew_Click() If lstUnconfirmed <> "" Then AddRecord End If End Sub Private Function AddRecord() With mClient_Translations .AddNew .Fields("ClientID") = Me.lstUnconfirmed .Fields("ClientAbbr") = Me.txtTmpShort .Fields("ClientName") = Me.txtTmpLong .Update End With UpdateRecords End Function Private Function UpdateRecords() Me.lstUnconfirmed.Requery End Function //Load events (After construction) Private Sub Form_Load() Set lstUnconfirmed.Recordset = mUnmatched_Clients //Link recordset into listbox Set Me.Recordset = mClient_Translations End Sub //Destruction method Private Sub Form_Close() Set conn = Nothing Set exception = Nothing Set lstUnconfirmed.Recordset = Nothing Set Me.Recordset = Nothing Set mUnmatched_Clients = Nothing Set mExcluded_Clients = Nothing Set mClient_Translations = Nothing End Sub

  Read the article

• Using mcrypt to pass data across a webservice is failing

  - by adam

  Hi I'm writing an error handler script which encrypts the error data (file, line, error, message etc) and passes the serialized array as a POST variable (using curl) to a script which then logs the error in a central db. I've tested my encrypt/decrypt functions in a single file and the data is encrypted and decrypted fine: define('KEY', 'abc'); define('CYPHER', 'blowfish'); define('MODE', 'cfb'); function encrypt($data) { $td = mcrypt_module_open(CYPHER, '', MODE, ''); $iv = mcrypt_create_iv(mcrypt_enc_get_iv_size($td), MCRYPT_RAND); mcrypt_generic_init($td, KEY, $iv); $crypttext = mcrypt_generic($td, $data); mcrypt_generic_deinit($td); return $iv.$crypttext; } function decrypt($data) { $td = mcrypt_module_open(CYPHER, '', MODE, ''); $ivsize = mcrypt_enc_get_iv_size($td); $iv = substr($data, 0, $ivsize); $data = substr($data, $ivsize); if ($iv) { mcrypt_generic_init($td, KEY, $iv); $data = mdecrypt_generic($td, $data); } return $data; } echo "<pre>"; $data = md5(''); echo "Data: $data\n"; $e = encrypt($data); echo "Encrypted: $e\n"; $d = decrypt($e); echo "Decrypted: $d\n"; Output: Data: d41d8cd98f00b204e9800998ecf8427e Encrypted: ê÷#¯KžViiÖŠŒÆÜ,ÑFÕUW£´Œt?†÷>c×åóéè+„N Decrypted: d41d8cd98f00b204e9800998ecf8427e The problem is, when I put the encrypt function in my transmit file (tx.php) and the decrypt in my recieve file (rx.php), the data is not fully decrypted (both files have the same set of constants for key, cypher and mode). Data before passing: a:4:{s:3:"err";i:1024;s:3:"msg";s:4:"Oops";s:4:"file";s:46:"/Applications/MAMP/htdocs/projects/txrx/tx.php";s:4:"line";i:80;} Data decrypted: Mª4:{s:3:"err";i:1024@7OYªç`^;g";s:4:"Oops";s:4:"file";sôÔ8F•Ópplications/MAMP/htdocs/projects/txrx/tx.php";s:4:"line";i:80;} Note the random characters in the middle. My curl is fairly simple: $ch = curl_init($url); curl_setopt($ch, CURLOPT_POST, true); curl_setopt($ch, CURLOPT_POSTFIELDS, 'data=' . $data); curl_setopt($ch, CURLOPT_RETURNTRANSFER, true); $output = curl_exec($ch); Things I suspect could be causing this: Encoding of the curl request Something to do with mcrypt padding missing bytes I've been staring at it too long and have missed something really really obvious If I turn off the crypt functions (so the transfer tx-rx is unencrypted) the data is received fine. Any and all help much appreciated! Thanks, Adam

  Read the article

• Problem in databinding a dictionary in ListView combo-box column.

  - by Ashish Ashu

  I have a listview of which itemsource is set to my custom collection, let's say MyCollection. The code below is not full code , it's just a code snippets to explain the problem. class Item : INotifyPropertyChanged { Options _options; public Options OptionProp { get { return _options; } set { _options = value; OnPropertyChanged ("OptionProp");} } string _Name; public string NameProp { get { return _Name; } set { _Name = value; OnPropertyChanged ("NameProp");} } } class Options : Dictionary<string,string> { public Options() { this.Clear(); this.Add("One" , "1" ); this.Add("Two" , "2" ); this.Add("Three" , "3" ); } } MyCollection in my viewModel class viewModel { ObservableCollection<Item> **MyCollection**; KeyValuePair<sting,string> **SelectedOption**; } The listview Item Source is set to my MyCollection. <ListView ItemSource = MyCollectoin> I Listview contains two columns of which I have defined a datatemplats in the listview. First column is a combo-box of which Itemsource is set to Options ( defined above ) Second column is a simple textblock to display Name. Problem 1. I have defined a datatemplate for first column in which I have a combo box , I have set the Itemsource =**MyCollection** and SelectedItem = SelectedOption of the combo-box. User can perform following operations in the listview: Add ( Add the row in the listview ) Move Up ( Move row up in the listview ) Move Down ( Move down the item in the listview ) .Now when I add the row in the listview , the combo-box selected index is always comes to -1 (first column). However the combo box contains options One, Two and Three. Also, I have initialized the SelectedOption to contain the first item, i:e One. problem 2. . Let suppose, I have added a single row in a listview and I have selected Option "one" in the combo box manually. Now when I perform Move Up or Move Down operations the selected index of a combo box is again set to -1. In the Move Up or Move Down operation , I am calling MoveUp and MoveDown methods of the Observable collection. Probelm 3 How to serialize the entire collection in XML. Since I can't serialize the Dictionary and KeyValue Pair. I have to restore the state of the listview.

  Read the article

• How to check for palindrome using Python logic

  - by DrOnline

  My background is only a 6 month college class in basic C/C++, and I'm trying to convert to Python. I may be talking nonsense, but it seems to me C, at least at my level, is very for-loop intensive. I solve most problems with these loops. And it seems to me the biggest mistake people do when going from C to Python is trying to implement C logic using Python, which makes things run slowly, and it's just not making the most of the language. I see on this website: http://hyperpolyglot.org/scripting (serach for "c-style for", that Python doesn't have C-style for loops. Might be outdated, but I interpret it to mean Python has its own methods for this. I've tried looking around, I can't find much up to date (Python 3) advice for this. How can I solve a palindrome challenge in Python, without using the for loop? I've done this in C in class, but I want to do it in Python, on a personal basis. The problem is from the Euler Project, great site btw. def isPalindrome(n): lst = [int(n) for n in str(n)] l=len(lst) if l==0 || l==1: return True elif len(lst)%2==0: for k in range (l) ##### else: while (k<=((l-1)/2)): if (list[]): ##### for i in range (999, 100, -1): for j in range (999,100, -1): if isPalindrome(i*j): print(i*j) break I'm missing a lot of code here. The five hashes are just reminders for myself. Concrete questions: 1) In C, I would make a for loop comparing index 0 to index max, and then index 0+1 with max-1, until something something. How to best do this in Python? 2) My for loop (in in range (999, 100, -1), is this a bad way to do it in Python? 3) Does anybody have any good advice, or good websites or resources for people in my position? I'm not a programmer, I don't aspire to be one, I just want to learn enough so that when I write my bachelor's degree thesis (electrical engineering), I don't have to simultaneously LEARN an applicable programming language while trying to obtain good results in the project. "How to go from basic C to great application of Python", that sort of thing. 4) Any specific bits of code to make a great solution to this problem would also be appreciated, I need to learn good algorithms.. I am envisioning 3 situations. If the value is zero or single digit, if it is of odd length, and if it is of even length. I was planning to write for loops... PS: The problem is: Find the highest value product of two 3 digit integers that is also a palindrome.

  Read the article

• Reducer getting fewer records than expected

  - by sathishs

  We have a scenario of generating unique key for every single row in a file. we have a timestamp column but the are multiple rows available for a same timestamp in few scenarios. We decided unique values to be timestamp appended with their respective count as mentioned in the below program. Mapper will just emit the timestamp as key and the entire row as its value, and in reducer the key is generated. Problem is Map outputs about 236 rows, of which only 230 records are fed as an input for reducer which outputs the same 230 records. public class UniqueKeyGenerator extends Configured implements Tool { private static final String SEPERATOR = "\t"; private static final int TIME_INDEX = 10; private static final String COUNT_FORMAT_DIGITS = "%010d"; public static class Map extends Mapper<LongWritable, Text, Text, Text> { @Override protected void map(LongWritable key, Text row, Context context) throws IOException, InterruptedException { String input = row.toString(); String[] vals = input.split(SEPERATOR); if (vals != null && vals.length >= TIME_INDEX) { context.write(new Text(vals[TIME_INDEX - 1]), row); } } } public static class Reduce extends Reducer<Text, Text, NullWritable, Text> { @Override protected void reduce(Text eventTimeKey, Iterable<Text> timeGroupedRows, Context context) throws IOException, InterruptedException { int cnt = 1; final String eventTime = eventTimeKey.toString(); for (Text val : timeGroupedRows) { final String res = SEPERATOR.concat(getDate( Long.valueOf(eventTime)).concat( String.format(COUNT_FORMAT_DIGITS, cnt))); val.append(res.getBytes(), 0, res.length()); cnt++; context.write(NullWritable.get(), val); } } } public static String getDate(long time) { SimpleDateFormat utcSdf = new SimpleDateFormat("yyyyMMddhhmmss"); utcSdf.setTimeZone(TimeZone.getTimeZone("America/Los_Angeles")); return utcSdf.format(new Date(time)); } public int run(String[] args) throws Exception { conf(args); return 0; } public static void main(String[] args) throws Exception { conf(args); } private static void conf(String[] args) throws IOException, InterruptedException, ClassNotFoundException { Configuration conf = new Configuration(); Job job = new Job(conf, "uniquekeygen"); job.setJarByClass(UniqueKeyGenerator.class); job.setOutputKeyClass(Text.class); job.setOutputValueClass(Text.class); job.setMapperClass(Map.class); job.setReducerClass(Reduce.class); job.setInputFormatClass(TextInputFormat.class); job.setOutputFormatClass(TextOutputFormat.class); // job.setNumReduceTasks(400); FileInputFormat.addInputPath(job, new Path(args[0])); FileOutputFormat.setOutputPath(job, new Path(args[1])); job.waitForCompletion(true); } } It is consistent for higher no of lines and the difference is as huge as 208969 records for an input of 20855982 lines. what might be the reason for reduced inputs to reducer?

  Read the article

• Windows Question: RunOnce/Second Boot Issues [closed]

  - by Greg

  Moved to Super User: Windows Question: RunOnce/Second Boot Issues I am attempting to create a Windows XP SP3 image that will run my application on Second Boot. Here is the intended workflow. 1) Run Image Prep Utility (I wrote) on windows to add my runonce entries and clean a few things up. 2) Reboot to ghost, make image file. 3) Package into my ISO and distribute. 4) System will be imaged by user. 5) On first boot, I have about 5 things that run, one of which includes a driver updater (I wrote) for my own specific devices. 6) One of the entries inside of HKCU/../runonce is a reg file, which adds another key to HKLM/../runonce. This is how second boot is acquired. 7) As a result of the driver updater, user is prompted to reboot. 8) My application is then launched from HKLM/../runonce on second boot. This workflow works perfectly, except for a select few legacy systems that contain devices that cause the add hardware wizard to pop up. When the add hardware wizard pops up is when I begin to see problems. It's important to note, that if I manually inspect the registry after the add hardware wizard pops up, it appears as I would expect, with all the first boot scripts having run, and it's sitting in a state I would correctly expect it to be in for a second boot scenario. The problem comes when I click next on the add hardware wizard, it seems to re-run the single entry I've added, and re-executes the runonce scripts. (only one script now as it's already executed and cleared out the initial entries). This causes my application to open as if it were a second boot, only when next is clicked on the add hardware wizard. If I click cancel, and reboot, then it also works as expected. I don't care as much about other solutions, because I could design a system that doesn't fully rely on Microsoft's registry. I simply can't find any information as to WHY this is happening. I believe this is some type of Microsoft issue that's presenting itself as a result of an overstretched image that's expected to support too many legacy platforms, but any help that can be provided would be appreciated. Thanks,

  Read the article

• Updating DetailViewController from RootController

  - by Stefano Salmaso

  I'm trying to create an iPad application with a similar user interface to Apple's Mail application, i.e: RootView controller (table view) on the left hand side of the split view for navigation with a multiple view hierarchy. When a table cell is selected a new table view is pushed on the left hand side The new view on the left side can update the detail view. I can accomplish both tasks BUT NOT TOGETHER. I mean I can make a multi-level table view in the RootController.(HERE you can find the working source code). Or I can make a single-level table view in the RootController which can update the detailViewController (here there is the source code:http://www.megaupload.com/?d=D6L0463G). Can anyone tell me how to make a multi-level table in the RootController which can update a detailViewController? There is more source code at the link but below is the method in which I presume I have to declare a new detailViewController (which has to be put in the UISplitViewController): - (void)tableView:(UITableView *)TableView didSelectRowAtIndexPath:(NSIndexPath *)indexPath { NSDictionary *dictionary = [self.tableDataSource objectAtIndex:indexPath.row]; //Get the children of the present item. NSArray *Children = [dictionary objectForKey:@"Children"]; // if([Children count] == 0) { /* Create and configure a new detail view controller appropriate for the selection. */ NSUInteger row = indexPath.row; UIViewController <SubstitutableDetailViewController> *detailViewController = nil; if (row == 0) { FirstDetailViewController *newDetailViewController = [[FirstDetailViewController alloc]initWithNibName:@"FirstDetailView" bundle:nil]; detailViewController = newDetailViewController; } if (row == 1) { SecondDetailViewController *newDetailViewController = [[SecondDetailViewController alloc]initWithNibName:@"SecondDetailView" bundle:nil]; detailViewController = newDetailViewController; } // Update the split view controller's view controllers array. NSArray *viewControllers = [[NSArray alloc] initWithObjects:self.navigationController, detailViewController, nil]; splitViewController.viewControllers = viewControllers//nothing happens..... [viewControllers release];// } else { //Prepare to tableview. RootViewController *rvController = [[RootViewController alloc]initWithNibName:@"RootViewController" bundle:[NSBundle mainBundle]]; //Increment the Current View rvController.current_level += 1; //Set the title; rvController.current_title = [dictionary objectForKey:@"Title"]; //Push the new table view on the stack [self.navigationController pushViewController:rvController animated:YES]; rvController.tableDataSource = Children; [rvController.tableView reloadData]; //without this instrucion,items won't be loaded inside the second level of the table [rvController release]; } }

  Read the article

• Cepstral Analysis for pitch detection

  - by Ohmu

  Hi! I'm looking to extract pitches from a sound signal. Someone on IRC just explain to me how taking a double FFT achieves this. Specifically: take FFT take log of square of absolute value (can be done with lookup table) take another FFT take absolute value I am attempting this using vDSP I can't understand how I didn't come across this technique earlier. I did a lot of hunting and asking questions; several weeks worth. More to the point, I can't understand why I didn't think of it. I am attempting to achieve this with vDSP library. it looks as though it has functions to handle all of these tasks. However, I'm wondering about the accuracy of the final result. I have previously used a technique which scours the frequency bins of a single FFT for local maxima. when it encounters one, it uses a cunning technique (the change in phase since the last FFT) to more accurately place the actual peak within the bin. I am worried that this precision will be lost with this technique I'm presenting here. I guess the technique could be used after the second FFT to get the fundamental accurately. But it kind of looks like the information is lost in step 2. as this is a potentially tricky process, could someone with some experience just look over what I'm doing and check it for sanity? also, I've heard there is an alternative technique involving fitting a quadratic over neighbouring bins. Is this of comparable accuracy? if so, I would favour it, as it doesn't involve remembering bin phases. so questions: does this approach makes sense? Can it be improved? I'm a bit worried about And the log square component; there seems to be a vDSP function to do exactly that: vDSP_vdbcon however, there is no indication it precalculates a log-table -- I assume it doesn't, as the FFT function requires an explicit pre-calculation function to be called and passed into it. and this function doesn't. Is there some danger of harmonics being picked up? is there any cunning way of making vDSP pull out the maxima, biggest first? Can anyone point me towards some research or literature on this technique? the main question: is it accurate enough? Can the accuracy be improved? I have just been told by an expert that the accuracy IS INDEED not sufficient. Is this the end of the line? Pi PS I get SO annoyed (npi) when I want to create tags, but cannot. :| I have suggested to the maintainers that SO keep track of attempted tags, but I'm sure I was ignored. we need tags for vDSP, accelerate framework, cepstral analysis

  Read the article

• Are there known problems with >= and <= and the eval function in JS?

  - by Augier

  I am currently writing a JS rules engine which at one point needs to evaluate boolean expressions using the eval() function. Firstly I construct an equation as such: var equation = "relation.relatedTrigger.previousValue" + " " + relation.operator + " " + "relation.value"; relation.relatedTrigger.previousValue is the value I want to compare. relation.operator is the operator (either "==", "!=", <=, "<", "", ="). relation.value is the value I want to compare with. I then simply pass this string to the eval function and it returns true or false as such: return eval(equation); This works absolutely fine (with words and numbers) or all of the operators except for = and <=. E.g. When evaluating the equation: relation.relatedTrigger.previousValue <= 100 It returns true when previousValue = 0,1,10,100 & all negative numbers but false for everything in between. I would greatly appreciate the help of anyone to either answer my question or to help me find an alternative solution. Regards, Augier. P.S. I don't need a speech on the insecurities of the eval() function. Any value given to relation.relatedTrigger.previousValue is predefined. edit: Here is the full function: function evaluateRelation(relation) { console.log("Evaluating relation") var currentValue; //if multiple values if(relation.value.indexOf(";") != -1) { var values = relation.value.split(";"); for (x in values) { var equation = "relation.relatedTrigger.previousValue" + " " + relation.operator + " " + "values[x]"; currentValue = eval(equation); if (currentValue) return true; } return false; } //if single value else { //Evaluate the relation and get boolean var equation = "relation.relatedTrigger.previousValue" + " " + relation.operator + " " + "relation.value"; console.log("relation.relatedTrigger.previousValue " + relation.relatedTrigger.previousValue); console.log(equation); return eval(equation); } } Answer: Provided by KennyTM below. A string comparison doesn't work. Converting to a numerical was needed.

  Read the article

• SQL Server CTE referred in self joins slow

  - by Kharlos Dominguez

  Hello, I have written a table-valued UDF that starts by a CTE to return a subset of the rows from a large table. There are several joins in the CTE. A couple of inner and one left join to other tables, which don't contain a lot of rows. The CTE has a where clause that returns the rows within a date range, in order to return only the rows needed. I'm then referencing this CTE in 4 self left joins, in order to build subtotals using different criterias. The query is quite complex but here is a simplified pseudo-version of it WITH DataCTE as ( SELECT [columns] FROM table INNER JOIN table2 ON [...] INNER JOIN table3 ON [...] LEFT JOIN table3 ON [...] ) SELECT [aggregates_columns of each subset] FROM DataCTE Main LEFT JOIN DataCTE BananasSubset ON [...] AND Product = 'Bananas' AND Quality = 100 LEFT JOIN DataCTE DamagedBananasSubset ON [...] AND Product = 'Bananas' AND Quality < 20 LEFT JOIN DataCTE MangosSubset ON [...] GROUP BY [ I have the feeling that SQL Server gets confused and calls the CTE for each self join, which seems confirmed by looking at the execution plan, although I confess not being an expert at reading those. I would have assumed SQL Server to be smart enough to only perform the data retrieval from the CTE only once, rather than do it several times. I have tried the same approach but rather than using a CTE to get the subset of the data, I used the same select query as in the CTE, but made it output to a temp table instead. The version referring the CTE version takes 40 seconds. The version referring the temp table takes between 1 and 2 seconds. Why isn't SQL Server smart enough to keep the CTE results in memory? I like CTEs, especially in this case as my UDF is a table-valued one, so it allowed me to keep everything in a single statement. To use a temp table, I would need to write a multi-statement table valued UDF, which I find a slightly less elegant solution. Did some of you had this kind of performance issues with CTE, and if so, how did you get them sorted? Thanks, Kharlos

  Read the article

• Web SITE publishing, dynamic compilation, smoke & mirrors

  - by tbehunin

  When you publish a web SITE in Visual Studio, in the dialog box that follows, you are given an option to "Allow this precompiled site to be updatable". According to MSDN, checking this option "specifies that all program code is compiled into assemblies, but that .aspx files (including single-file ASP.NET Web pages) are copied as-is to the target folder". With this option checked, you can update existing .aspx files as well as add new ones without any issue. When a page, that has either been updated or newly created, is requested, the page gets dynamically compiled at run-time and is then processed and returned to the user. If, on the other hand, you didn't check that checkbox during the publish phase, the .aspx files get compiled, along with the code-behind and App_Code files in separate assemblies. The .aspx files are then completely overwritten with a line of text that says: This is a marker file generated by the precompilation tool, and should not be deleted! You obviously can't edit an existing page in this scenario. If you were to ADD a new .aspx file to this site, you would get a .Net run-time error saying that the file hasn't been precompiled. With that background, my questions are these: Something must be able to determine that this website was published to be updatable (allow dynamic compilation) or not. If it was published as updatable, it must also be able to determine whether a file was changed or added, so it can do a dynamic compile. Who makes those determinations? IIS? ASP.NET worker process? HOW does it make those determinations? If I had the same website published in both of those scenarios, could I make a visual determination that one is updatable and the other is not? Is there some bit I can look at in the assemblies using Reflector to make that determination myself? In addition to answering those questions, what also might be helpful would be information on the process flow from when a resource is requested to when it starts being processed, not necessarily the ASP.NET Page Lifecycle, but what happens BEFORE ASP.Net worker process starts processing the page and firing off events. The dynamic compilation appears to be smoke and mirrors. Can someone demystify this for me?

  Read the article

< Previous Page | 623 624 625 626 627 628 629 630 631 632 633 634  | Next Page >