Search Results

Search found 70567 results on 2823 pages for 'binary file'.

Page 5/2823 | < Previous Page | 1 2 3 4 5 6 7 8 9 10 11 12  | Next Page >

  • Any efficient way to read datas from large binary file?

    - by limi
    Hi, I need to handle tens of Gigabytes data in one binary file. Each record in the data file is variable length. So the file is like: <len1><data1><len2><data2>..........<lenN><dataN> The data contains integer, pointer, double value and so on. I found python can not even handle this situation. There is no problem if I read the whole file in memory. It's fast. But it seems the struct package is not good at performance. It almost stuck on unpack the bytes. Any help is appreciated. Thanks.

    Read the article

  • C# - parse content away from structure in a binary file

    - by Jeff Godfrey
    Using C#, I need to read a packed binary file created using FORTRAN. The file is stored in an "Unformatted Sequential" format as described here (about half-way down the page in the "Unformatted Sequential Files" section): http://www.tacc.utexas.edu/services/userguides/intel8/fc/f_ug1/pggfmsp.htm As you can see from the URL, the file is organized into "chunks" of 130 bytes or less and includes 2 length bytes (inserted by the FORTRAN compiler) surrounding each chunk. So, I need to find an efficient way to parse the actual file payload away from the compiler-inserted formatting. Once I've extracted the actual payload from the file, I'll then need to parse it up into its varying data types. That'll be the next exercise. My first thoughts are to slurp up the entire file into a byte array using File.ReadAllBytes. Then, just iterate through the bytes, skipping the formatting and transferring the actual data to a second byte array. In the end, that second byte array should contain the actual file contents minus all the formatting, which I'd then need to go back through to get what I need. As I'm fairly new to C#, I thought there might be a better, more accepted way of tackling this. Also, in case it's helpful, these files could be fairly large (say 30MB), though most will be much smaller...

    Read the article

  • parse content away from structure in a binary file

    - by Jeff Godfrey
    Using C#, I need to read a packed binary file created using FORTRAN. The file is stored in an "Unformatted Sequential" format as described here (about half-way down the page in the "Unformatted Sequential Files" section): http://www.tacc.utexas.edu/services/userguides/intel8/fc/f_ug1/pggfmsp.htm As you can see from the URL, the file is organized into "chunks" of 130 bytes or less and includes 2 length bytes (inserted by the FORTRAN compiler) surrounding each chunk. So, I need to find an efficient way to parse the actual file payload away from the compiler-inserted formatting. Once I've extracted the actual payload from the file, I'll then need to parse it up into its varying data types. That'll be the next exercise. My first thoughts are to slurp up the entire file into a byte array using File.ReadAllBytes. Then, just iterate through the bytes, skipping the formatting and transferring the actual data to a second byte array. In the end, that second byte array should contain the actual file contents minus all the formatting, which I'd then need to go back through to get what I need. As I'm fairly new to C#, I thought there might be a better, more accepted way of tackling this. Also, in case it's helpful, these files could be fairly large (say 30MB), though most will be much smaller...

    Read the article

  • --log-slave-updates is OFF but some updates are still logged to the slave binary log?

    - by quanta
    MySQL version 5.5.14 According to the document, by the default, slave does not log to its binary log any updates that are received from a master server. Here are my config. on the slave: # egrep 'bin|slave' /etc/my.cnf relay-log=mysqld-relay-bin log-bin = /var/log/mysql/mysql-bin binlog-format=MIXED sync_binlog = 1 log-bin-trust-function-creators = 1 mysql> show global variables like 'log_slave%'; +-------------------+-------+ | Variable_name | Value | +-------------------+-------+ | log_slave_updates | OFF | +-------------------+-------+ 1 row in set (0.01 sec) mysql> select @@log_slave_updates; +---------------------+ | @@log_slave_updates | +---------------------+ | 0 | +---------------------+ 1 row in set (0.00 sec) but slave still logs the some changes to its binary logs, let's see the file size: -rw-rw---- 1 mysql mysql 37M Apr 1 01:00 /var/log/mysql/mysql-bin.001256 -rw-rw---- 1 mysql mysql 25M Apr 2 01:00 /var/log/mysql/mysql-bin.001257 -rw-rw---- 1 mysql mysql 46M Apr 3 01:00 /var/log/mysql/mysql-bin.001258 -rw-rw---- 1 mysql mysql 115M Apr 4 01:00 /var/log/mysql/mysql-bin.001259 -rw-rw---- 1 mysql mysql 105M Apr 4 18:54 /var/log/mysql/mysql-bin.001260 and the sample query when reading these binary files with mysqlbinlog utility: #120404 19:08:57 server id 3 end_log_pos 110324763 Query thread_id=382435 exec_time=0 error_code=0 SET TIMESTAMP=1333541337/*!*/; INSERT INTO norep_SplitValues VALUES ( NAME_CONST('cur_string',_utf8'118212' COLLATE 'utf8_general_ci')) /*!*/; # at 110324763 Did I miss something? Reply to @RolandoMySQLDBA: If replication brought this over, then the same query has to be in the relay logs. Please go find the relay log that has the INSERT query with the same TIMESTAMP (1333541337). There is no such query with the same TIMESTAMP in the relay logs. If you cannot find it in the relay logs, then look and see if Infobright is posting the INSERT query. In that instance, the INSERT should be recorded in the binary logs of the Slave. Looking more deeply into the binary logs, I see that almost of the queries are CREATE/INSERT/UPDATE/DROP "temporary" tables, something like this: # at 123873315 #120405 0:42:04 server id 3 end_log_pos 123873618 Query thread_id=395373 exec_time=0 error_code=0 SET TIMESTAMP=1333561324/*!*/; SET @@session.pseudo_thread_id=395373/*!*/; CREATE TEMPORARY TABLE `norep_tmpcampaign` ( `campaignid` INTEGER(11) NOT NULL DEFAULT '0' , `status` INTEGER(11) NOT NULL DEFAULT '0' , `updated` DATETIME, KEY `campaignid` (`campaignid`) )ENGINE=MEMORY /*!*/; # at 123873618 #120405 0:42:04 server id 3 end_log_pos 123873755 Query thread_id=395373 exec_time=0 error_code=0 SET TIMESTAMP=1333561324/*!*/; DROP TABLE IF EXISTS `norep_tmpcampaign1` /* generated by server */ "temporary" here means that they are dropped after calculation is done. I also tells the slave not to replicate any statement matches the norep_ wildcard pattern: replicate-wild-ignore-table=%.norep_% But there is an exception table in the binary logs: # at 123828094 #120405 0:37:21 server id 3 end_log_pos 123828495 Query thread_id=395209 exec_time=0 error_code=0 SET TIMESTAMP=1333561041/*!*/; INSERT INTO sessions (SessionId, ApplicationName, Created, Expires, LockDate, LockId, Timeout, Locked, SessionItems, Fla gs) Values('pgv2exo4y4vo4ccz44vwznu0', '/', '2012-04-05 00:37:21', '2012-04-05 00:57:21', '2012-04-05 00:37:21', 0, 20, 0, 'AwAAAP////8IdXNlcm5hbWUGdXNlcmlkCHBlcm1pdGlkAgAAAAQAAAAGAAAAAQABAAEA', 0) /*!*/; Description: mysql> desc reportingdb.sessions; +-----------------+------------------+------+-----+---------------------+-------+ | Field | Type | Null | Key | Default | Extra | +-----------------+------------------+------+-----+---------------------+-------+ | SessionId | varchar(64) | NO | PRI | | | | ApplicationName | varchar(255) | NO | | | | | Created | timestamp | NO | | 0000-00-00 00:00:00 | | | Expires | timestamp | NO | | 0000-00-00 00:00:00 | | | LockDate | timestamp | NO | | 0000-00-00 00:00:00 | | | LockId | int(11) unsigned | NO | | NULL | | | Timeout | int(11) unsigned | NO | | NULL | | | Locked | bit(1) | NO | | NULL | | | SessionItems | varchar(255) | YES | | NULL | | | Flags | int(11) | NO | | NULL | | +-----------------+------------------+------+-----+---------------------+-------+ I'm sure all these queries are posting by MySQL, not Infobright: $ mysql-ib -u root -p Enter password: Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 48971 Server version: 5.1.40 build number (revision)=IB_4.0.5_r15240_15370(ice) (static) Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> select * from information_schema.tables where table_name='sessions'; Empty set (0.02 sec) I've been trying some INSERT/UPDATE queries with testing tables on the master, it is copied to the relay logs, not binary logs on slave: # at 311664029 #120405 0:15:23 server id 1 end_log_pos 311664006 Query thread_id=10458250 exec_time=0 error_code=0 use testuser/*!*/; SET TIMESTAMP=1333559723/*!*/; update users set email2='[email protected]' where id=22 /*!*/; Pay attention to the server id, in the relay logs, server id is master's (1) and in the binary log, server id is slave's (3 in this case). Reply to @RolandoMySQLDBA: Thu Apr 5 10:06:00 ICT 2012 Run CREATE DATABASE quantatest; on the Master now, please. Tell me if CREATE DATABASE quantatest; showed up in the Slave's Binary Logs. As I said above: I've been trying some INSERT/UPDATE queries with testing tables on the master, it is copied to the relay logs, not binary logs and you can guess, IO thread copied it to the relay logs, not binary logs. #120405 10:07:25 server id 1 end_log_pos 347573819 Query thread_id=10480775 exec_time=0 error_code=0 SET TIMESTAMP=1333595245/*!*/; /*!\C latin1 *//*!*/; SET @@session.character_set_client=8,@@session.collation_connection=8,@@session.collation_server=8/*!*/; create database quantatest /*!*/; The question must probably change to: why some update queries still logged to the slave binary logs althrough --log-slave-updates is disabled? Where they come from? Here are few last: /*!*/; # at 27492197 #120405 10:12:45 server id 3 end_log_pos 27492370 Query thread_id=410353 exec_time=0 error_code=0 SET TIMESTAMP=1333595565/*!*/; CREATE TEMPORARY TABLE norep_SplitValues ( value VARCHAR(1000) NOT NULL ) ENGINE=MEMORY /*!*/; # at 27492370 #120405 10:12:45 server id 3 end_log_pos 27492445 Query thread_id=410353 exec_time=0 error_code=0 SET TIMESTAMP=1333595565/*!*/; BEGIN /*!*/; # at 27492445 #120405 10:12:45 server id 3 end_log_pos 27492619 Query thread_id=410353 exec_time=0 error_code=0 SET TIMESTAMP=1333595565/*!*/; INSERT INTO norep_SplitValues VALUES ( NAME_CONST('cur_string',_utf8'119577' COLLATE 'utf8_general_ci')) /*!*/; # at 27492619 #120405 10:12:45 server id 3 end_log_pos 27492695 Query thread_id=410353 exec_time=0 error_code=0 SET TIMESTAMP=1333595565/*!*/; COMMIT /*!*/; # at 27492918 #120405 10:12:46 server id 3 end_log_pos 27493115 Query thread_id=410353 exec_time=0 error_code=0 SET TIMESTAMP=1333595566/*!*/; SELECT `reportingdb`.`selfserving_get_locationad`(_utf8'3' COLLATE 'utf8_general_ci',_utf8'' COLLATE 'utf8_general_ci') /*!*/; # at 27493199 #120405 10:12:46 server id 3 end_log_pos 27493353 Query thread_id=410353 exec_time=0 error_code=0 SET TIMESTAMP=1333595566/*!*/; /*!\C utf8 *//*!*/; SET @@session.character_set_client=33,@@session.collation_connection=33,@@session.collation_server=8/*!*/; DROP TEMPORARY TABLE IF EXISTS `norep_SplitValues` /* generated by server */ /*!*/;

    Read the article

  • How to compare mp3, flac audio data in a file, ignoring header data (ID3 tag) etc.?

    - by Rob
    I've backed up some audio files up in 2 places and added ID3 tags into one backup but not the other, since time has passed my own memory has faded on whether the backups are actually the same, but now one has ID3 data and the other doesn't, basic binary compare will fail and inspection will be cumbersome. Is there a tool to compare just the audio data (not the header, ID3) in mp3s, flac files, and other files using header data such as ID3. started a thread on beyond compare here: http://www.scootersoftware.com/vbulletin/showthread.php?t=7413 would consider other comparison software that does this task

    Read the article

  • Using Haskell's Parsec to parse binary files?

    - by me2
    Parsec is designed to parse textual information, but it occurs to me that Parsec could also be suitable to do binary file format parsing for complex formats that involve conditional segments, out-of-order segments, etc. Is there an ability to do this or a similar, alternative package that does this? If not, what is the best way in Haskell to parse binary file formats?

    Read the article

  • floating point hex octal binary

    - by workinprogress
    Hi, I am working on a calculator that allows you to perform calculations past the decimal point in octal, hexadecimal, binary, and of course decimal. I am having trouble though finding a way to convert floating point decimal numbers to floating point hexadecimal, octal, binary and vice versa. The plan is to do all the math in decimal and then convert the result into the appropriate number system. Any help, ideas or examples would be appreciated. Thanks!

    Read the article

  • Skip List vs. Binary Tree

    - by Claudiu
    I recently came across the data structure known as a Skip list. They seem to have very similar behavior to a binary search tree... my question is - why would you ever want to use a skip list over a binary search tree?

    Read the article

  • Matlab - binary vector with high concentration of 1s (or 0s)

    - by JohnIdol
    What's the best way to generate a number X of random binary vectors of size N with concentration of 1s (or, simmetrically, of 0s) that spans from very low to very high? Using randint or unidrnd (as in this question) will generate binary vectors with uniform distributions, which is not what I need in this case. Any help appreciated!

    Read the article

  • Translate binary joke to ASCII

    - by Nix
    I found this cartoon is a binary joke and have been trying to translate! I got the binary from the GIF, but I can't find out how to translate to ASCII... 10001 11011 11 1011 01 01001 11011 0111011 01 100101 10 01111 01 100101 10 01111 01 01001 11011 0111011 01 10001 11011 11 1011 01 10001 11011 11 1011 01 01001 11011 0111011 01 100101 10 01111 01 100101 10 01111 01 01001 11011 0111011 01 1001 11011 11 1011 01 01101 Any help translating would be great.

    Read the article

  • What is validating a binary search tree?

    - by dotnetdev
    I read on here of an exercise in interviews known as validating a binary search tree. How exactly does this work? What would one be looking for in validating a binary search tree? I have written a basic search tree, but never heard of this concept. Thanks

    Read the article

  • Binary socket and policy file in Flex

    - by Daniil
    Hi, I'm trying to evaluate whether Flex can access binary sockets. Seems that there's a class calles Socket (flex.net package). The requirement is that Flex will connect to a server serving binary data. It will then subscribe to data and receive the feed which it will interpret and display as a chart. I've never worked with Flex, my experience lies with Java, so everything is new to me. So I'm trying to quickly set something simple up. The Java server expects the following: DataInputStream in = ..... byte cmd = in.readByte(); int size = in.readByte(); byte[] buf = new byte[size]; in.readFully(buf); ... do some stuff and send binary data in something like out.writeByte(1); out.writeInt(10000); ... etc... Flex, needs to connect to a localhost:6666 do the handshake and read data. I got something like this: try { var socket:Socket = new Socket(); socket.connect('192.168.110.1', 9999); Alert.show('Connected.'); socket.writeByte(108); // 'l' socket.writeByte(115); // 's' socket.writeByte(4); socket.writeMultiByte('HHHH', 'ISO-8859-1'); socket.flush(); } catch (err:Error) { Alert.show(err.message + ": " + err.toString()); } The first thing is that Flex does a <policy-file-request/>. I've modified the server to respond with: <?xml version="1.0"?> <!DOCTYPE cross-domain-policy SYSTEM "/xml/dtds/cross-domain-policy.dtd"> <cross-domain-policy> <site-control permitted-cross-domain-policies="master-only"/> <allow-access-from domain="192.168.110.1" to-ports="*" /> </cross-domain-policy> After that - EOFException happens on the server and that's it. So the question is, am I approaching whole streaming data issue wrong when it comes to Flex? Am I sending the policy file wrong? Unfortunately, I can't seem to find a good solid example of how to do it. It seems to me that Flex can do binary Client-Server application, but I personally lack some basic knowledge when doing it. I'm using Flex 3.5 in IntelliJ IDEA IDE. Any help is appreciated. Thank you!

    Read the article

  • REST service with binary data

    - by user179437
    Hi I want to create Restful service which can accept binary data. I've implemented javax.xml.ws.Provider interface, but i can't get content of request. If I use javax.xml.ws.Dispatch then its send only XML data, but I need transfer binary data. Please give some solution, but I don't prefer to use JAX-RS or Restlets. Thanks.

    Read the article

  • Examples of attoparsec in parsing binary file formats?

    - by me2
    Previously attoparsec was suggested to me for parsing complex binary file formats. While I can find examples of attoparsec parsing HTTP, which is essentially text based, I cannot find an example parsing actual binary, for example, a TCP packet, or image file, or mp3. Can someone post some code or pointer to some code which does this using attoparsec?

    Read the article

  • VBScript - image to binary

    - by countnazgul
    Hi to all, i'm not a VB programmer but i need a vbscript that convert image file (from local disk) to be converted to binary data and the passed to webservice. I realize how to pass data to webservice but i can't find how to convert the image file to binary data. I spend a lot of time to find some kind of solution but with no luck. Can somebody help me? Thanks!

    Read the article

  • How to get the binary data from a download link

    - by jacob
    Hi all, I need to get the binary data of a download link. When I run this link in the browser it always starts download manager. Rather I would like to copy that binary and display it on teh browser. How is this possible in any language. Objective c or c# preferred. Thanks

    Read the article

  • Question on Binary Search Trees.

    - by AGeek
    Hi, I was thinking of implementing a binary search trees. I have implemented some very basic operations such as search, insert, delete. Please share your experiences as to what all other operations i could perform on binary search trees, and some real time operations(basic) that is needed every time for any given situation.. I hope my question was clear.. Thanks.

    Read the article

  • How can i declare a Binary type

    - by hamza
    i need to work with a binary number i just write : const x = 00010000 ; & it doesn't work i know that i can use an hexadecimal number that have the same value as 00010000 but for more knowledge i want to know if there is a type in C++ for binary numbers & if there isn't ,is there another solution for my Problem ? thanks

    Read the article

  • get a range of objects with binary search

    - by Behrooz
    I have some data like this: ID Value 1 AAA 1 ABC 2 dasd 2 dsfdsf 2 dsfsd 3 df 3 dwqef they are objects(not plain text). and i want to get all objects with the ID = 2. I can do a binary binary search and get the index 3,but how can i get (2 and 4) is there any efficient algorithm? the real problem has lists with about one Million items. any language except bf and lisp can help.

    Read the article

  • how to init binary buffer in python

    - by ace
    so, I read from DB binary field i.e. 'field1' to var Buf1, and then do something like: unpack_from('I', Buf1, 0) so, all is ok. but question is how can I ini Buf1 without going to DB? I can get value from DB manually and init my var statically, but how? in DB field 'field1' I see something like '0x7B0500000100000064000000B80100006'. and how can I init valid binary buffer from it?

    Read the article

  • VB6: Slow Binary Write?

    - by Tom the Junglist
    Wondering why a particular binary write operation in VB is so slow. The function reads a Byte array from memory and dumps it into a file like this: Open Destination For Binary Access Write As #1 Dim startP, endP As Long startP = BinaryStart endP = UBound(ReadBuf) - 1 Dim i as Integer For i = startP To endP DoEvents Put #1, (i - BinaryStart) + 1, ReadBuf(i) Next Close #1 For two megabytes on a slower system, this can take up to a minute. Can anyone tell me why this is so slow?

    Read the article

  • Is this the right strategy to convert an in-level order binary tree to a doubly linked list?

    - by Ankit Soni
    So I recently came across this question - Make a function that converts a in-level-order binary tree into a doubly linked list. Apparently, it's a common interview question. This is the strategy I had - Simply do a pre-order traversal of the tree, and instead of returning a node, return a list of nodes, in the order in which you traverse them. i.e return a list, and append the current node to the list at each point. For the base case, return the node itself when you are at a leaf. so you would say left = recursive_function(node.left) right = recursive_function(node.right) return(left.append(node.data)).append(right);` Is this the right approach?

    Read the article

< Previous Page | 1 2 3 4 5 6 7 8 9 10 11 12  | Next Page >