Search Results

Search found 65558 results on 2623 pages for 'large data'.

Page 103/2623 | < Previous Page | 99 100 101 102 103 104 105 106 107 108 109 110  | Next Page >

  • Adding line with text between pattern and next occurence of the same pattern in bash

    - by kasper
    I am writing a bash script that modifies a file that looks like this: --- usr1 --- data data data data data data data data data data data data --- usr2 --- data data data data data data data data --- usr3 --- data data data data --- endline --- One question is: How to add next user line --- usrn --- after last user data lines? Second one is: How to delete specific user data lines (data lines and --- userx ---) i.e. I would like to delete usr2 with all his data set. It must work on bash 2.05 :) and I think it will use awk or sed, but I'm not sure.

    Read the article

  • How to Load Data from Text File in MySQL?

    - by Taz
    I have this file availableHere Fields are separated by space and newline starts at new line. I am using this command to load data mysql> LOAD DATA LOCAL INFILE 'C:\\Documents and Settings\\Scan\\My Documents\\Downloads\\images_en.nt\\Sample4.txt' INTO TABLE NER.Images FIELDS TERMINATED BY' ' LINES TERMINATED BY '\n';' Only one row is loaded and if I execute again No row is loaded. Where is the problem? Data can be modified to reformat if required.

    Read the article

  • cc1plus: error: include: Value too large for defined data type when compiling with g++

    - by Android
    I am making a project that should compile on Windows and Linux. I have made the project in Visual Studio and then made a makefile for linux. I created all the files in Windows with VS. It compiles and runs perfectly in VS but when I run the makefile and it runs g++ I get $ g++ -c -I include -o obj/Linux_x86/Server.obj src/Server.cpp cc1plus: error: include: Value too large for defined data type cc1plus: error: src/Server.cpp: Value too large for defined data type The code is nothing more than a Hello World atm. I just wanted to make sure that everything was working before I started development. I have tried searching but to no avail. Any help would be appreciated.

    Read the article

  • Compressing large text data before storing into db?

    - by Steel Plume
    Hello, I have application which retrieves many large log files from a system LAN. Currently I put all log files on Postgresql, the table has a column type TEXT and I don't plan any search on this text column because I use another external process which nightly retrieves all files and scans for sensitive pattern. So the column value could be also a BLOB or a CLOB, but now my question is the following, the database has already its compression system, but could I improve this compression manually like with common compressor utilities? And above all WHAT IF I manually pre-compress the large file and then I put as binary into the data table, is it unuseful as database system provides its internal compression?

    Read the article

  • Designing DAL in .NET to be "data-source independent" and not just "database independent" ?

    - by Munish Goyal
    How to design such flexible DAL (specifically in .NET) ? What interfaces .NET provides and what should be done on my own ? Its a greenfield project starting with SQL Server as data source but in future, parts of it will move to different NoSQL type of datastores. Also, we may need to experiment with lot of different datastores (like some data may have to go with Cassandra, some with RDBMS, some to other DHT etc.) Therefore easily switchable access layer will be needed. All i know right now is the 'data' and 'operations needed on that data'.

    Read the article

  • large images on toolbar

    - by cagi
    Hi to all. Is it possible to have a 32x32 images on toolbar while keeping 16x16 in menu. I created toolbar resource (IDR_TOOLBAR1) and set it's button size as 32x32 and load it: if (!m_wndToolBar.CreateEx(this,...) || !m_wndToolBar.LoadToolBar(IDR_TOOLBAR1 )) { ... } When started toolbar correctly shows large btn images but menu is also enlarged showing 32x32 item images!? How to have menu images as before (small 16x16) My app info: SDI, VS2008 Thanks in advance! P.S. I have noticed an option check box "Large Images" in quick customization dialog under options tab. The dialog is enabled by default in my project. When selecting that option images buttons enlarge instantly, the problem is the new sized images is created from small ones and quality is poor. It would be better if new images list is loaded? Once again thanks in advance!

    Read the article

  • Resources to learn about engineering aspects of data analytics (OLAP, warehousing, ETL, etc.)

    - by JT
    I'm a math/stats guy, interested in learning more about the engineering aspects of "data analytics" (this may be an overly broad term, this is a case of "I don't know what I don't know", so I'm not sure how to be more specific). I'm fine with manipulating and analyzing the data once it's already stored somewhere and I can access it, and I'm fine with writing scripts and SQL queries (and have a general knowledge of things like normalization). What I don't know is the whole engineering process of capturing and storing the data. For example, terms I've heard thrown about that I only vaguely understand the meaning of include: - OLAP, OLTP - Data warehousing - ETL - ??? What's a good book (or any other resource) to learn about these kinds of things? What are things I should know about database design (normalization seems kinda "obvious" to me, something I would have done even before I knew the term -- is there anything else?)? In other words, for jobs falling under the umbrella term of "analytics engineer", what kinds of things should I know?

    Read the article

  • Generate and merge data with python multiprocessing

    - by Bobby
    I have a list of starting data. I want to apply a function to the starting data that creates a few pieces of new data for each element in the starting data. Some pieces of the new data are the same and I want to remove them. The sequential version is essentially: def create_new_data_for(datum): """make a list of new data from some old datum""" return [datum.modified_copy(k) for k in datum.k_list] data = [some list of data] #some data to start with #generate a list of new data from the old data, we'll reduce it next newdata = [] for d in data: newdata.extend(create_new_data_for(d)) #now reduce the data under ".matches(other)" reduced = [] for d in newdata: for seen in reduced: if d.matches(seen): break #so we haven't seen anything like d yet seen.append(d) #now reduced is finished and is what we want! I want to speed this up with multiprocessing. I was thinking that I could use a multiprocessing.Queue for the generation. Each process would just put the stuff it creates on, and when the processes are reducing the data, they can just get the data from the Queue. But I'm not sure how to have the different process loop over reduced and modify it without any race conditions or other issues. What is the best way to do this safely? or is there a different way to accomplish this goal better?

    Read the article

  • Project design / FS layout for large django projects

    - by rcreswick
    What is the best way to layout a large django project? The tutuorials provide simple instructions for setting up apps, models, and views, but there is less information about how apps and projects should be broken down, how much sharing is allowable/necessary between apps in a typical project (obviously that is largely dependent on the project) and how/where general templates should be kept. Does anyone have examples, suggestions, and explanations as to why a certain project layout is better than another? I am particularly interested in the incorporation of large numbers of unit tests (2-5x the size of the actual code base) and string externalization / templates.

    Read the article

  • Is there a lightweight multipart/form-data parser in C or C++?

    - by Hongli
    I'm looking at integrating multipart form-data parsing in a web server module so that I can relieve backend web applications (often written in dynamic languages) from parsing the multipart data themselves. The multipart grammar (RFC 2046) looks non-trivial and if I implement it by hand a lot of things can go wrong. Is there already a good, lightweight multipart/form-data parser written in C or C++? I'm looking for one with no external dependencies other than the C or C++ standard library. I don't need email attachment handling or buffered I/O classes or a portability runtime or whatever, just multipart/form-data parsing. Things that I've considered: GMime - depends on glib, so no go. libapreq - too large, depends on APR, badly documented, no unit tests. I've also looked at writing a parser with Ragel, but I can't figure out how to do it because the grammar is not static: the boundary can change arbitrarily.

    Read the article

  • How to read a file with variable multi-row data in Python

    - by dr.bunsen
    I have a file that is about 100Mb that looks like this: #meta data 1 skadjflaskdjfasljdfalskdjfl sdkfjhasdlkgjhsdlkjghlaskdj asdhfk #meta data 2 jflaksdjflaksjdflkjasdlfjas ldaksjflkdsajlkdfj #meta data 3 alsdkjflasdjkfglalaskdjf This file contains one row of meta data that corresponds to several, variable length data containing only alpha-numeric characters. What is the best way to read this data into a simple list like this: data = [[#meta data 1, skadjflaskdjfasljdfalskdjflsdkfjhasdlkgjhsdlkjghlaskdjasdhfk], [#meta data 2, jflaksdjflaksjdflkjasdlfjasldaksjflkdsajlkdfj], [#meta data 3, alsdkjflasdjkfglalaskdjf]] My initial idea was to use the read() method to read the whole file into memory and then use regular expressions to parse the data into the desired format. Is there a better more pythonic way? All metadata lines start with an octothorpe and all data lines are all alpha-numeric. Thanks!

    Read the article

  • Finding Errant Output to System.out in Large Java Program

    - by SvrGuy
    Hi, We have a large java code base [~1 M Lines]. Buried (somewhere) in the code base is some old debug output to System.out that we want to remove (its cluttering things up). The problem is: out code base is so large that we can't easily find where the output is coming from. What we want is a way to see where System.out.println is getting called from (like a stack trace from an exception or some such). Its not suitable to debugging -- the errant output is coming from some errant thread somewhere etc. Any ideas on how to track the source of this errant output down? PS: 99.99% of calls to System.out are legit, and we have thousands of them, so simply searching the code base for System.out calls is not a solution!

    Read the article

  • how to split a very large database on sql server

    - by ken jackson
    I have a 90 GB SQL Server database that I want to make more manageable. It stores stock data from 50+ different stocks from 2009 and 2010, and each stock is a separate table. Some tables have hundreds of millions of rows, and other have just a few million. What I want to do is somehow split the database, so that I don't have a single database file that is 90 GB. What I want is to be able to somehow magically split all the tables so that I can backup the 2009 data once and not have to keep on including it in the backup every time I backup the entire database, however, I would like the 2009 data to be included whenever I do a query. Is partitioning the database the way to go? Will it do the above for me, or will I need some other solution? I research partitioning, but I wasn't sure if that would solve all my problems. I wasn't able to find anything that would tell me whether or not it would migrate prexisting data, or whether it only worked for newly inserted data. Any help or pointers would be much appreciated. Thanks in advance, Ken

    Read the article

  • Why use hashing to create pathnames for large collections of files?

    - by Stephen
    Hi, I noticed a number of cases where an application or database stored collections of files/blobs using a has to determine the path and filename. I believe the intended outcome is a situation where the path never gets too deep, or the folders ever get too full - too many files (or folders) in a folder making for slower access. EDIT: Examples are often Digital libraries or repositories, though the simplest example I can think of (that can be installed in about 30s) is the Zotero document/citation database. Why do this? EDIT: thanks Mat for the answer - does this technique of using a hash to create a file path have a name? Is it a pattern? I'd like to read more, but have failed to find anything in the ACM Digital Library

    Read the article

  • Tool to diagonalize large matrices

    - by Xodarap
    I want to compute a diffusion kernel, which involves taking exp(b*A) where A is a large matrix. In order to play with values of b, I'd like to diagonalize A (so that exp(A) runs quickly). My matrix is about 25k x 25k, but is very sparse - only about 60k values are non-zero. Matlab's "eigs" function runs of out memory, as does octave's "eig" and R's "eigen." Is there a tool to find the decomposition of large, sparse matrices? Dunno if this is relevant, but A is an adjacency matrix, so it's symmetric, and it is full rank.

    Read the article

  • Looking for easy way to analyze var_dump (PHP) on large objects

    - by sdek
    I know (PHP's) var_dump is supposed to be "human readable" and all, but analyzing large objects is just a pain in the neck. I am struggling to make sense of a few of the large objects that are being passed around in a script that we are running. (I know that using xdebug with and IDE is a good idea, but I have not been able to get xdebug to run on this project for some reason - several days lost, ugh). Any ideas on how I can easily digest the contents of a really big var_dump? Any ideas are welcome... Although I am hoping that there is something similar to Thomas Frank's JSON tool (where you just put some code in and it gives a nice graphical representation).

    Read the article

  • Java regex replace multiple file paths in a large String

    - by Joe Goble
    So a Regex pro I am not, and I'm looking for a good way to do this. I have a large string which contains a variable number <img> tags. I need to change the path on all of these images to images/. The large string also contains other stuff not just these img's. <img src='http://server.com/stuff1/img1.jpg' /> <img src='http://server.com/stuff2/img2.png' /> Replacing the server name with a ReplaceAll() I could do, it's the variable path in the middle I'm clueless on how to include. It doesn't necessarily need to be a regex, but looping through the entire string just seems wasteful.

    Read the article

  • Javascript: Passing large objects or strings between function considered a bad practice

    - by Mr. Smee
    Is it considered a bad practice to pass around a large string or object (lets say from an ajax response) between functions? Would it be beneficial in any way save the response in a variable and keep reusing that variable? So in the code it would be something like this: var response; $.post(url, function(resp){ response = resp; }) function doSomething() { // do something with the response here } vs $.post(url, function(resp){ doSomething(resp); }) function doSomething(resp) { // do something with the resp here } Assume resp is a large object or string and it can be passed around between multiple functions.

    Read the article

  • html5 uploader + jquery drag & drop: how to store file data with FormData?

    - by lauthiamkok
    I am making a html5 drag and drop uploader with jquery, below is my code so far, the problem is that I get an empty array without any data. Is this line incorrect to store the file data - fd.append('file', $thisfile);? $('#div').on( 'dragover', function(e) { e.preventDefault(); e.stopPropagation(); } ); $('#div').on( 'dragenter', function(e) { e.preventDefault(); e.stopPropagation(); } ); $('#div').on( 'drop', function(e){ if(e.originalEvent.dataTransfer){ if(e.originalEvent.dataTransfer.files.length) { e.preventDefault(); e.stopPropagation(); // The file list. var fileList = e.originalEvent.dataTransfer.files; //console.log(fileList); // Loop the ajax post. for (var i = 0; i < fileList.length; i++) { var $thisfile = fileList[i]; console.log($thisfile); // HTML5 form data object. var fd = new FormData(); //console.log(fd); fd.append('file', $thisfile); /* var file = {name: fileList[i].name, type: fileList[i].type, size:fileList[i].size}; $.each(file, function(key, value) { fd.append('file['+key+']', value); }) */ $.ajax({ url: "upload.php", type: "POST", data: fd, processData: false, contentType: false, success: function(response) { // .. do something }, error: function(jqXHR, textStatus, errorMessage) { console.log(errorMessage); // Optional } }); } /*UPLOAD FILES HERE*/ upload(e.originalEvent.dataTransfer.files); } } } ); function upload(files){ console.log('Upload '+files.length+' File(s).'); }; then if I use another method is that to make the file data into an array inside the jquery code, var file = {name: fileList[i].name, type: fileList[i].type, size:fileList[i].size}; $.each(file, function(key, value) { fd.append('file['+key+']', value); }); but where is the tmp_name data inside e.originalEvent.dataTransfer.files[i]? php, print_r($_POST); $uploaddir = './uploads/'; $file = $uploaddir . basename($_POST['file']['name']); if (move_uploaded_file($_POST['file']['tmp_name'], $file)) { echo "success"; } else { echo "error"; } as you can see that tmp_name is needed to upload the file via php... html, <div id="div">Drop here</div>

    Read the article

  • Is there any reason why someone would want to create an Core Data model programmatically?

    - by mystify
    I wonder in which cases it would be good to make an NSManagedObjectModel completely programmatically, with NSEntityDescription instances and all this stuff. I'm that kind of person who prefers to code programmatically, rejecting Interface Builder. But when it comes to Core Data, I have a hard time figuring out why I should kill my time NOT using the nice Xcode Data Modeler tool. And since data models are stuck to a given state (except when you want to do some ugly migration operations where thinks probably go wrong and users get mad, really mad), I see no big sense in a data model that's made programmatically for the purpose of changing it all the time. Did I miss something?

    Read the article

  • MySQL Cluster data nodes - slow SELECTs

    - by Boyan Georgiev
    Hi to all. First off, I'm new to MySQL Cluster. This is my pain: I've managed to setup a MySQL Cluster with two data nodes, two SQL nodes and one management server. Everything works pretty well, except the following: my data nodes are spread across an intranet link which incurs latency into communications between the data nodes. Apparently, due to MySQL Cluster's internal partitioning schemes, when my PHP application pulls data from the cluster via SELECT queries, parts of the data are pulled from both data nodes. This makes the page appear onscreen REALLY slowly. If I bring one data node offline, the data can only be pulled from that single remaining data node, and thus, the final result (HTML output) appears on the screen in a very timely fashion. So, my question is this: can the data nodes/cluster be told to pull data from partitions stored only on a particular data node?

    Read the article

  • Downloading large files with AFNetworking

    - by goodfella
    I'm trying to implement downloading of a large file and show to user current progress, but block in: -[AFURLConnectionOperation setDownloadProgressBlock:] returns incorrect bytesRead and totalBytesRead values (they are smaller than they should be). For example: If I have a 90MB file and when it downloads completely, latest block invocation in setDownloadProgressBlock: gives me totalBytesRead value about 30MB. On other side, if file is 2MB large, latest block invocation gives correct totalBytesRead 2MB value. AFNetworking is updated to the latest version from github. If AFNetworking can't do it correctly, what solution can I use? Edit: I've determined that even if file is not downloaded completely (and this happens every time with relatively big file) AFNetworking calls success block in: -[AFHTTPRequestOperation setCompletionBlockWithSuccess:failure] I asked a similar question here about this situation, but didn't get any answers. I can check in code downloaded and real file sizes, but AFNetworking has no API for continuation of partial download.

    Read the article

  • large test data for knapsack problem

    - by user347918
    i am researcher student. I am searching large data for knapsack problem. I wanted test my algorithm for knapsack problem. But i couldn't find large data. I need data has 1000 item and capacity is no matter. The point is item as much as huge it's good for my algorithm. Is there any huge data available in internet. Does anybody know please guys i need urgent.

    Read the article

  • How to structure a Visual Studio project for the data access layer

    - by Akk
    I currently have a project that uses various DB access technologies mainly for showcasing or for demos. Currently we have: Namespace App.Data (App.Data.dll) Folder NHibernate Folder EntityFramework Folder LinqToSql The above structure is ok as we only use Sql Server as the DB. But going forward we will be including Oracle, MySql etc. So what would be a better structure with this in mind? I thought about: Namespace App.Data.SqlServer (App.Data.SqlServer.dll) Folder NHibernate Folder EntityFramework Folder LinqToSql Or would it just be better to have separate assemblies for each database and access technology?: Namespace App.Data.SqlServer.NHibernate (App.Data.SqlServer.NHibernate.dll) Namespace App.Data.SqlServer.EntityFramework(App.Data.SqlServer.EntityFramework.dll) Namespace App.Data.Oracle.NHibernate (App.Data.Oracle.NHibernate.dll) Namespace App.Data.MySql.NHibernate (App.Data.MySql.Oracle.dll)

    Read the article

< Previous Page | 99 100 101 102 103 104 105 106 107 108 109 110  | Next Page >