Search Results

Search found 5942 results on 238 pages for 'total starnger'.

Page 36/238 | < Previous Page | 32 33 34 35 36 37 38 39 40 41 42 43  | Next Page >

  • Help on Website response time KPI parameters

    - by geeth
    I am working on improving website performance. Here are the list of key performance indicators I am looking at for each page Total Bytes downloaded Number of requests DNS look up time FirstByte Download time DOM content load time Total load time Is there any optimum value for each KPI to indicate website performance? Please help me in this regard.

    Read the article

  • Parsing or Extracting the content of html table.

    - by Harikrishna
    Can I parse the html tables by giving only column name ? Like only those data should be extracted from the table which matches those column names I give. Like for example I have table of column names like serial no., name, address, phone no,total Rs.. And I want to extract the information about only name, phone no and total Rs.. Then how can I do it?

    Read the article

  • Visualize Classifier Error Weka

    - by user1780592
    Hye there i have a have datasets where this data i have test it on weka with J48 classifier It give me an output = 87.2611% Total of instances = 157 Correctly Instances = 137 Incorrectly instance = 20 Then i have do a visualize classifier error on my data. However my result have been decrease to: New result = 85.4015% Correctly Instances = 117 Incorrectly instances = 20 Total of instances = 137 Is there any reason for that? Should my result become much better after i do the visualize classifier error?

    Read the article

  • how to pass an id number string to this class

    - by Phil
    I'm very much a vb person, but have had to use this id number class in c#. I got it from http://www.codingsanity.com/idnumber.htm : using System; using System.Text.RegularExpressions; namespace Utilities.SouthAfrica { /// <summary> /// Represents a South African Identity Number. /// valid number = 7707215230080 /// invalid test number = 1234567891234 /// /// </summary> [Serializable()] public class IdentityNumber { #region Enumerations /// <summary> /// Indicates a gender. /// </summary> public enum PersonGender { Female = 0, Male = 5 } public enum PersonCitizenship { SouthAfrican = 0, Foreign = 1 } #endregion #region Declarations static Regex _expression; Match _match; const string _IDExpression = @"(?<Year>[0-9][0-9])(?<Month>([0][1-9])|([1][0-2]))(?<Day>([0-2][0-9])|([3][0-1]))(?<Gender>[0-9])(?<Series>[0-9]{3})(?<Citizenship>[0-9])(?<Uniform>[0-9])(?<Control>[0-9])"; #endregion #region Constuctors /// <summary> /// Sets up the shared objects for ID validation. /// </summary> static IdentityNumber() { _expression = new Regex(_IDExpression, RegexOptions.Compiled | RegexOptions.Singleline); } /// <summary> /// Creates the ID number from a string. /// </summary> /// <param name="IDNumber">The string ID number.</param> public IdentityNumber(string IDNumber) { _match = _expression.Match(IDNumber.Trim()); } #endregion #region Properties /// <summary> /// Indicates the date of birth encoded in the ID Number. /// </summary> /// <exception cref="System.ArgumentException">Thrown if the ID Number is not usable.</exception> public DateTime DateOfBirth { get { if(IsUsable == false) { throw new ArgumentException("ID Number is unusable!", "IDNumber"); } int year = int.Parse(_match.Groups["Year"].Value); // NOTE: Do not optimize by moving these to static, otherwise the calculation may be incorrect // over year changes, especially century changes. int currentCentury = int.Parse(DateTime.Now.Year.ToString().Substring(0, 2) + "00"); int lastCentury = currentCentury - 100; int currentYear = int.Parse(DateTime.Now.Year.ToString().Substring(2, 2)); // If the year is after or at the current YY, then add last century to it, otherwise add // this century. // TODO: YY -> YYYY logic needs thinking about if(year > currentYear) { year += lastCentury; } else { year += currentCentury; } return new DateTime(year, int.Parse(_match.Groups["Month"].Value), int.Parse(_match.Groups["Day"].Value)); } } /// <summary> /// Indicates the gender for the ID number. /// </summary> /// <exception cref="System.ArgumentException">Thrown if the ID Number is not usable.</exception> public PersonGender Gender { get { if(IsUsable == false) { throw new ArgumentException("ID Number is unusable!", "IDNumber"); } int gender = int.Parse(_match.Groups["Gender"].Value); if(gender < (int) PersonGender.Male) { return PersonGender.Female; } else { return PersonGender.Male; } } } /// <summary> /// Indicates the citizenship for the ID number. /// </summary> /// <exception cref="System.ArgumentException">Thrown if the ID Number is not usable.</exception> public PersonCitizenship Citizenship { get { if(IsUsable == false) { throw new ArgumentException("ID Number is unusable!", "IDNumber"); } return (PersonCitizenship) Enum.Parse(typeof(PersonCitizenship), _match.Groups["Citizenship"].Value); } } /// <summary> /// Indicates if the IDNumber is usable or not. /// </summary> public bool IsUsable { get { return _match.Success; } } /// <summary> /// Indicates if the IDNumber is valid or not. /// </summary> public bool IsValid { get { if(IsUsable == true) { // Calculate total A by adding the figures in the odd positions i.e. the first, third, fifth, // seventh, ninth and eleventh digits. int a = int.Parse(_match.Value.Substring(0, 1)) + int.Parse(_match.Value.Substring(2, 1)) + int.Parse(_match.Value.Substring(4, 1)) + int.Parse(_match.Value.Substring(6, 1)) + int.Parse(_match.Value.Substring(8, 1)) + int.Parse(_match.Value.Substring(10, 1)); // Calculate total B by taking the even figures of the number as a whole number, and then // multiplying that number by 2, and then add the individual figures together. int b = int.Parse(_match.Value.Substring(1, 1) + _match.Value.Substring(3, 1) + _match.Value.Substring(5, 1) + _match.Value.Substring(7, 1) + _match.Value.Substring(9, 1) + _match.Value.Substring(11, 1)); b *= 2; string bString = b.ToString(); b = 0; for(int index = 0; index < bString.Length; index++) { b += int.Parse(bString.Substring(index, 1)); } // Calculate total C by adding total A to total B. int c = a + b; // The control-figure can now be determined by subtracting the ones in figure C from 10. string cString = c.ToString() ; cString = cString.Substring(cString.Length - 1, 1) ; int control = 0; // Where the total C is a multiple of 10, the control figure will be 0. if(cString != "0") { control = 10 - int.Parse(cString.Substring(cString.Length - 1, 1)); } if(_match.Groups["Control"].Value == control.ToString()) { return true; } } return false; } } #endregion } } Here is the code from my default.aspx.cs page: using System; using System.Collections.Generic; using System.Linq; using System.Web; using System.Web.UI; using System.Web.UI.WebControls; using Utilities.Southafrica; <- this is the one i added to public partial class _Default : System.Web.UI.Page { protected void Page_Load(object sender, EventArgs e) { var someNumber = new IdentityNumber("123456"); <- gives error } } Can someone please tell the syntax for how I pass an id number to the class? Thanks

    Read the article

  • array data taking XML value is not taking css value in as3

    - by Sagar S. Ranpise
    I have xml structure all data comes here inside CDATA I am using css file in it to format text and classes are mentioned in xml Below is the code which shows data but does not format with CSS. Thanks in advance! var myXML:XML = new XML(); var myURLLoader:URLLoader = new URLLoader(); var myURLRequest:URLRequest = new URLRequest("test.xml"); myURLLoader.load(myURLRequest); //////////////For CSS/////////////// var myCSS:StyleSheet = new StyleSheet(); var myCSSURLLoader:URLLoader = new URLLoader(); var myCSSURLRequest:URLRequest = new URLRequest("test.css"); myCSSURLLoader.load(myCSSURLRequest); myCSSURLLoader.addEventListener(Event.COMPLETE,processXML); var i:int; var textHeight:int = 0; var textPadding:int = 10; var txtName:TextField = new TextField(); var myMov:MovieClip = new MovieClip(); var myMovGroup:MovieClip = new MovieClip(); var myArray:Array = new Array(); function processXML(e:Event):void { myXML = new XML(myURLLoader.data); trace(myXML.person.length()); var total:int = myXML.person.length(); trace("total" + total); for(i=0; i<total; i++) { myArray.push({name: myXML.person[i].name.toString()}); trace(myArray[i].name); } processCSS(); } function processCSS():void { myCSS.parseCSS(myCSSURLLoader.data); for(i=0; i<myXML.person.length(); i++) { myMov.addChild(textConvertion(myArray[i].name)); myMov.y = textHeight; textHeight += myMov.height + textPadding; trace("Text: "+myXML.person[i].name); myMovGroup.addChild(myMov); } this.addChild(myMovGroup); } function textConvertion(textConverted:String) { var tc:TextField = new TextField(); tc.htmlText = textConverted; tc.multiline = true; tc.wordWrap = true; tc.autoSize = TextFieldAutoSize.LEFT; tc.selectable = true; tc.y = textHeight; textHeight += tc.height + textPadding; tc.styleSheet = myCSS; return tc; }

    Read the article

  • how to pass an id number string to this class (asp.net, c#)

    - by Phil
    I'm very much a vb person, but have had to use this id number class in c#. I got it from http://www.codingsanity.com/idnumber.htm : using System; using System.Text.RegularExpressions; namespace Utilities.SouthAfrica { /// <summary> /// Represents a South African Identity Number. /// valid number = 7707215230080 /// invalid test number = 1234567891234 /// /// </summary> [Serializable()] public class IdentityNumber { #region Enumerations /// <summary> /// Indicates a gender. /// </summary> public enum PersonGender { Female = 0, Male = 5 } public enum PersonCitizenship { SouthAfrican = 0, Foreign = 1 } #endregion #region Declarations static Regex _expression; Match _match; const string _IDExpression = @"(?<Year>[0-9][0-9])(?<Month>([0][1-9])|([1][0-2]))(?<Day>([0-2][0-9])|([3][0-1]))(?<Gender>[0-9])(?<Series>[0-9]{3})(?<Citizenship>[0-9])(?<Uniform>[0-9])(?<Control>[0-9])"; #endregion #region Constuctors /// <summary> /// Sets up the shared objects for ID validation. /// </summary> static IdentityNumber() { _expression = new Regex(_IDExpression, RegexOptions.Compiled | RegexOptions.Singleline); } /// <summary> /// Creates the ID number from a string. /// </summary> /// <param name="IDNumber">The string ID number.</param> public IdentityNumber(string IDNumber) { _match = _expression.Match(IDNumber.Trim()); } #endregion #region Properties /// <summary> /// Indicates the date of birth encoded in the ID Number. /// </summary> /// <exception cref="System.ArgumentException">Thrown if the ID Number is not usable.</exception> public DateTime DateOfBirth { get { if(IsUsable == false) { throw new ArgumentException("ID Number is unusable!", "IDNumber"); } int year = int.Parse(_match.Groups["Year"].Value); // NOTE: Do not optimize by moving these to static, otherwise the calculation may be incorrect // over year changes, especially century changes. int currentCentury = int.Parse(DateTime.Now.Year.ToString().Substring(0, 2) + "00"); int lastCentury = currentCentury - 100; int currentYear = int.Parse(DateTime.Now.Year.ToString().Substring(2, 2)); // If the year is after or at the current YY, then add last century to it, otherwise add // this century. // TODO: YY -> YYYY logic needs thinking about if(year > currentYear) { year += lastCentury; } else { year += currentCentury; } return new DateTime(year, int.Parse(_match.Groups["Month"].Value), int.Parse(_match.Groups["Day"].Value)); } } /// <summary> /// Indicates the gender for the ID number. /// </summary> /// <exception cref="System.ArgumentException">Thrown if the ID Number is not usable.</exception> public PersonGender Gender { get { if(IsUsable == false) { throw new ArgumentException("ID Number is unusable!", "IDNumber"); } int gender = int.Parse(_match.Groups["Gender"].Value); if(gender < (int) PersonGender.Male) { return PersonGender.Female; } else { return PersonGender.Male; } } } /// <summary> /// Indicates the citizenship for the ID number. /// </summary> /// <exception cref="System.ArgumentException">Thrown if the ID Number is not usable.</exception> public PersonCitizenship Citizenship { get { if(IsUsable == false) { throw new ArgumentException("ID Number is unusable!", "IDNumber"); } return (PersonCitizenship) Enum.Parse(typeof(PersonCitizenship), _match.Groups["Citizenship"].Value); } } /// <summary> /// Indicates if the IDNumber is usable or not. /// </summary> public bool IsUsable { get { return _match.Success; } } /// <summary> /// Indicates if the IDNumber is valid or not. /// </summary> public bool IsValid { get { if(IsUsable == true) { // Calculate total A by adding the figures in the odd positions i.e. the first, third, fifth, // seventh, ninth and eleventh digits. int a = int.Parse(_match.Value.Substring(0, 1)) + int.Parse(_match.Value.Substring(2, 1)) + int.Parse(_match.Value.Substring(4, 1)) + int.Parse(_match.Value.Substring(6, 1)) + int.Parse(_match.Value.Substring(8, 1)) + int.Parse(_match.Value.Substring(10, 1)); // Calculate total B by taking the even figures of the number as a whole number, and then // multiplying that number by 2, and then add the individual figures together. int b = int.Parse(_match.Value.Substring(1, 1) + _match.Value.Substring(3, 1) + _match.Value.Substring(5, 1) + _match.Value.Substring(7, 1) + _match.Value.Substring(9, 1) + _match.Value.Substring(11, 1)); b *= 2; string bString = b.ToString(); b = 0; for(int index = 0; index < bString.Length; index++) { b += int.Parse(bString.Substring(index, 1)); } // Calculate total C by adding total A to total B. int c = a + b; // The control-figure can now be determined by subtracting the ones in figure C from 10. string cString = c.ToString() ; cString = cString.Substring(cString.Length - 1, 1) ; int control = 0; // Where the total C is a multiple of 10, the control figure will be 0. if(cString != "0") { control = 10 - int.Parse(cString.Substring(cString.Length - 1, 1)); } if(_match.Groups["Control"].Value == control.ToString()) { return true; } } return false; } } #endregion } } Can someone please tell the syntax for how I pass an id number to the class? Thanks

    Read the article

  • Getting memory usage from 'ps aux' output with awk

    - by JustB
    Hello everybody, I have to solve an exercise using awk. Basically I need to retrieve from 'ps aux' command the total of memory usage for each user and format like this: User Total%Mem user1 3.4% user2 1.5% and so on. The problem I can't seem to solve is: how do I know how many users are logged in? And how can I make a different sum for each one of them? Thank you :)

    Read the article

  • How to get rank based on SUM's?

    - by Kenan
    I have comments table where everything is stored, and i have to SUM everything and add BEST ANSWER*10. I need rank for whole list, and how to show rank for specified user/ID. Here is the SQL: SELECT m.member_id AS member_id, (SUM(c.vote_value) + SUM(c.best)*10) AS total FROM comments c LEFT JOIN members m ON c.author_id = m.member_id GROUP BY c.author_id ORDER BY total DESC LIMIT {$sql_start}, 20

    Read the article

  • Help ---- SQL Script

    - by Vinoj Nambiar
    Store No Store Name Region Division Q10(response) Q21(response) 2345       ABC              North Test              1                       5 2345                            North Test              6                       3 2345       ABC              North Test              4                       6 1st calculation 1 ) Engaged(%) = Response Greater than 4.5 3 (total response greater than 4.5) / 6 (total count) * 100 = 50% Store No Store Name Region Division Q10 Q21 2345             ABC      North Test           1       5 2345             ABC      North Test           6       3 2345            ABC       North Test           4       6 2) not engaged (%) = Response less than 2 1 (total response less than 2) / 6 (total count) * 100 = 16.66% I should be able to get the table like this Store No Store Name Region Division Engaged(%) Disengaged(%) 2345            ABC     North Test                 50                 16.66

    Read the article

  • Can I do this in one Mysql query?

    - by bsandrabr
    Hi I have a table with two columns: column A column B 1 2 1 2 2 1 I want to return total of ones = 3 total of twos = 3 The best I can come up with is two queries like so: SELECT sum( CASE WHEN columnA =1 THEN 1 ELSE 0 END ) + sum(CASE WHEN columnB =1 THEN 1 ELSE 0 END ) SELECT sum( CASE WHEN columnA =2 THEN 1 ELSE 0 END ) + sum(CASE WHEN columnB =2 THEN 1 ELSE 0 END ) Can this be done in one query? Thanks

    Read the article

  • How to compare two floating-point values in shell script

    - by Reem
    I had to do a division in shell script and the best way was: result1=`echo "scale=3; ($var1 / $total) * 100"| bc -l` result2=`echo "scale=3; ($var2 / $total) * 100"| bc -l` but I want to compare the values of $result1 and $result2 Using if test $result1 -lt $result2 or if [ $result1 -gt $result2 ] didn't work :( Any idea how to do that?

    Read the article

  • Converting a certain SQL query into relational algebra

    - by Fumler
    Just doing an assignment for my database course and I just want to double check that I've correctly wrapped my head around relational algebra. The SQL query: SELECT dato, SUM(pris*antall) AS total FROM produkt, ordre WHERE ordre.varenr = produkt.varenr GROUP BY dato HAVING total >= 10000 The relational algebra: stotal >= 10000( ?R(dato, total)( sordre.varenr = produkt.varenr( datoISUM(pris*antall(produkt x ordre)))) Is this the correct way of doing it?

    Read the article

  • Which is faster??

    - by kaki
    is opening a large file once reading it completely once to list faster (or) opening smaller files whose total sum of size is equal to large file and loading smaller file into list manupalating one by one faster? which is faster?? is the difference is time large enough to impact my program?? total time difference of lesser then of 30 sec is negligible for me

    Read the article

  • Drawing single pixel in Quartz

    - by wwrob
    I have an array of CGPoints, and I'd like to fill the whole screen with colours, the colour of each pixel depending on the total distance to each of the points in the array. The natural way to do this is to, for each pixel, compute the total distance, and turn that into a colour. Questions follow: 1) How can I colour a single pixel in Quartz? I've been thinking of making 1 by 1 rectangles. 2) Are there better, more efficient ways to achieve this effect?

    Read the article

  • AspxPivotGrid Group column header

    - by eugene4968
    I want to be able to add a parent column header to each pair of columns shown below. So I want it to look like this: Grand Total OnHand Available Anyone know a way to do this? I know you can group columns together but I'm wondering how I can apply a label to those groups. Alternatively, if theres no way to do that, I could just override the "Grand Total" display text. The only problem is I can't get it aligned above its correct columns.

    Read the article

  • How to compare two "not integer" values in shell script

    - by Reem
    I had to do a division in shell script and the best way was: result1=`echo "scale=3; ($var1 / $total) * 100"| bc -l` result2=`echo "scale=3; ($var2 / $total) * 100"| bc -l` but I want to compare the values of $result1 and $result2 Using if test $result1 -lt $result2 or if [ $result1 -gt $result2 ] didn't work :( Any idea how to do that?

    Read the article

  • How to split records per hour in order to display them as a chart?

    - by Axel
    Hi, I have an SQL table like this : sales(product,timestamp) I want to display a chart using Open Flash Chart but i don't know how to get the total sales per hour within the last 12 hours. ( the timestamp column is the sale date ) By example i will end up with an array like this : array(12,5,8,6,10,35,7,23,4,5,2,16) every number is the total sales in each hour. Note: i want to use php or only mysql for this. Thanks

    Read the article

  • Need help parsing HTML with a regex in python

    - by laspal
    Hi, My string is mystring = "<tr><td><span class='para'><b>Total Amount : </b>INR (Indian Rupees) 100.00</span></td></tr>" My problem here is I have to search and get the total amount test = re.search("(Indian Rupees)(\d{2})(?:\D|$)", mystring) but my test give me None. How can I get the values and values can be 10.00, 100.00, 1000.00 Thanks

    Read the article

  • Basic data alignment question

    - by Broken Logic
    I've been playing around to see how my computer works under the hood. What I'm interested in is seeing is what happens on the stack inside a function. To do this I've written the following toy program: #include <stdio.h> void __cdecl Test1(char a, unsigned long long b, char c) { char c1; unsigned long long b1; char a1; c1 = 'b'; b1 = 4; a1 = 'r'; printf("%d %d - %d - %d %d Total: %d\n", (long)&b1 - (long)&a1, (long)&c1 - (long)&b1, (long)&a - (long)&c1, (long)&b - (long)&a, (long)&c - (long)&b, (long)&c - (long)&a1 ); }; struct TestStruct { char a; unsigned long long b; char c; }; void __cdecl Test2(char a, unsigned long long b, char c) { TestStruct locals; locals.a = 'b'; locals.b = 4; locals.c = 'r'; printf("%d %d - %d - %d %d Total: %d\n", (long)&locals.b - (long)&locals.a, (long)&locals.c - (long)&locals.b, (long)&a - (long)&locals.c, (long)&b - (long)&a, (long)&c - (long)&b, (long)&c - (long)&locals.a ); }; int main() { Test1('f', 0, 'o'); Test2('f', 0, 'o'); return 0; } And this spits out the following: 9 19 - 13 - 4 8 Total: 53 8 8 - 24 - 4 8 Total: 52 The function args are well behaved but as the calling convention is specified, I'd expect this. But the local variables are a bit wonky. My question is, why wouldn't these be the same? The second call seems to produce a more compact and better aligned stack. Looking at the ASM is unenlightening (at least to me), as the variable addresses are still aliased there. So I guess this is really a question about the assembler itself allocates the stack to local variables. I realise that any specific answer is likely to be platform specific. I'm more interested in a general explanation unless this quirk really is platform specific. For the record though, I'm compiling with VS2010 on a 64bit Intel machine.

    Read the article

  • Flash: How to preload upcoming SWF while current one plays

    - by pthesis
    I have a Flash slideshow that plays SWFs listed in an XML file. I would like to have the upcoming SWF load while the current one displays. I've tried all sorts of combinations of LoadMovie and LoadMovieNum, including creating an empty movie clip, but there's something I'm just not getting. Right now, after making the first round through all the files, it transitions smoothly from slide to slide, but I'd like for it to preload so that it transitions without the "Loading..." screen the first time around. It can be viewed here: slideshow Where should I put the LoadMovie line to load the next file (image[p+1]), and how should it look? function loadXML(loaded) { if (loaded) { xmlNode = this.firstChild; image = []; description = []; total = xmlNode.childNodes.length; for (i=0; i<total; i++) { image[i] = xmlNode.childNodes[i].childNodes[0].firstChild.nodeValue; description[i] = xmlNode.childNodes[i].childNodes[1].firstChild.nodeValue; } firstImage(); } else { content = "file not loaded!"; } } xmlData = new XML(); xmlData.ignoreWhite = true; xmlData.onLoad = loadXML; xmlData.load("xmlfile.xml"); ///////////////////////////////////// back_btn.onRelease = function () { backImage(); }; next_btn.onRelease = function () { nextImage(); }; p = 0; function nextImage() { if (p<(total-1)) { p++; trace(this); _root.mc_loadfile.loadMovie (image[p]); _root.movie_name.text = image[p]; next_btn._visible = true; back_btn._visible = true; if (getBytesLoaded() == getBytesTotal()) slideshow(); } else if (p == (total-1)) { p = 0; firstImage(); } } function backImage() { clearInterval(myInterval); if (p>0) { --p; _root.mc_loadfile.loadMovie (image[p]); _root.movie_name.text = image[p]; next_btn._visible = true; if (p != 0) { back_btn._visible = true; } else { back_btn._visible = false; } slideshow(); } } I'd appreciate any help.

    Read the article

  • Silverlight 4: Binding to a calculation of control properties

    - by Rich.Carpenter
    What I would like to do is pretty simple. Given textboxes for ItemPrice, Tax and Total, I need the text value for Total to be bound to ItemPrice + Tax and the Tax value to display ItemPrice * taxRate. Could someone offer a brief explanation as to how this would be accomplished or point me to an appropriate example? I see property binding examples all over the place, but none that show binding to a calculation of the properties of two controls.

    Read the article

  • Can MySQL reasonably perform queries on billions of rows?

    - by haxney
    I am planning on storing scans from a mass spectrometer in a MySQL database and would like to know whether storing and analyzing this amount of data is remotely feasible. I know performance varies wildly depending on the environment, but I'm looking for the rough order of magnitude: will queries take 5 days or 5 milliseconds? Input format Each input file contains a single run of the spectrometer; each run is comprised of a set of scans, and each scan has an ordered array of datapoints. There is a bit of metadata, but the majority of the file is comprised of arrays 32- or 64-bit ints or floats. Host system |----------------+-------------------------------| | OS | Windows 2008 64-bit | | MySQL version | 5.5.24 (x86_64) | | CPU | 2x Xeon E5420 (8 cores total) | | RAM | 8GB | | SSD filesystem | 500 GiB | | HDD RAID | 12 TiB | |----------------+-------------------------------| There are some other services running on the server using negligible processor time. File statistics |------------------+--------------| | number of files | ~16,000 | | total size | 1.3 TiB | | min size | 0 bytes | | max size | 12 GiB | | mean | 800 MiB | | median | 500 MiB | | total datapoints | ~200 billion | |------------------+--------------| The total number of datapoints is a very rough estimate. Proposed schema I'm planning on doing things "right" (i.e. normalizing the data like crazy) and so would have a runs table, a spectra table with a foreign key to runs, and a datapoints table with a foreign key to spectra. The 200 Billion datapoint question I am going to be analyzing across multiple spectra and possibly even multiple runs, resulting in queries which could touch millions of rows. Assuming I index everything properly (which is a topic for another question) and am not trying to shuffle hundreds of MiB across the network, is it remotely plausible for MySQL to handle this? UPDATE: additional info The scan data will be coming from files in the XML-based mzML format. The meat of this format is in the <binaryDataArrayList> elements where the data is stored. Each scan produces = 2 <binaryDataArray> elements which, taken together, form a 2-dimensional (or more) array of the form [[123.456, 234.567, ...], ...]. These data are write-once, so update performance and transaction safety are not concerns. My naïve plan for a database schema is: runs table | column name | type | |-------------+-------------| | id | PRIMARY KEY | | start_time | TIMESTAMP | | name | VARCHAR | |-------------+-------------| spectra table | column name | type | |----------------+-------------| | id | PRIMARY KEY | | name | VARCHAR | | index | INT | | spectrum_type | INT | | representation | INT | | run_id | FOREIGN KEY | |----------------+-------------| datapoints table | column name | type | |-------------+-------------| | id | PRIMARY KEY | | spectrum_id | FOREIGN KEY | | mz | DOUBLE | | num_counts | DOUBLE | | index | INT | |-------------+-------------| Is this reasonable?

    Read the article

  • Combining FileStream and MemoryStream to avoid disk accesses/paging while receiving gigabytes of data?

    - by w128
    I'm receiving a file as a stream of byte[] data packets (total size isn't known in advance) that I need to store somewhere before processing it immediately after it's been received (I can't do the processing on the fly). Total received file size can vary from as small as 10 KB to over 4 GB. One option for storing the received data is to use a MemoryStream, i.e. a sequence of MemoryStream.Write(bufferReceived, 0, count) calls to store the received packets. This is very simple, but obviously will result in out of memory exception for large files. An alternative option is to use a FileStream, i.e. FileStream.Write(bufferReceived, 0, count). This way, no out of memory exceptions will occur, but what I'm unsure about is bad performance due to disk writes (which I don't want to occur as long as plenty of memory is still available) - I'd like to avoid disk access as much as possible, but I don't know of a way to control this. I did some testing and most of the time, there seems to be little performance difference between say 10 000 consecutive calls of MemoryStream.Write() vs FileStream.Write(), but a lot seems to depend on buffer size and the total amount of data in question (i.e the number of writes). Obviously, MemoryStream size reallocation is also a factor. Does it make sense to use a combination of MemoryStream and FileStream, i.e. write to memory stream by default, but once the total amount of data received is over e.g. 500 MB, write it to FileStream; then, read in chunks from both streams for processing the received data (first process 500 MB from the MemoryStream, dispose it, then read from FileStream)? Another solution is to use a custom memory stream implementation that doesn't require continuous address space for internal array allocation (i.e. a linked list of memory streams); this way, at least on 64-bit environments, out of memory exceptions should no longer be an issue. Con: extra work, more room for mistakes. So how do FileStream vs MemoryStream read/writes behave in terms of disk access and memory caching, i.e. data size/performance balance. I would expect that as long as enough RAM is available, FileStream would internally read/write from memory (cache) anyway, and virtual memory would take care of the rest. But I don't know how often FileStream will explicitly access a disk when being written to. Any help would be appreciated.

    Read the article

< Previous Page | 32 33 34 35 36 37 38 39 40 41 42 43  | Next Page >