Write asynchronously to file in perl
Posted
by Stefhen
on Stack Overflow
See other posts from Stack Overflow
or by Stefhen
Published on 2010-04-12T00:32:01Z
Indexed on
2010/04/12
0:43 UTC
Read the original article
Hit count: 525
Basically I would like to:
- Read a large amount of data from the network into an array into memory.
- Asynchronously write this array data, running it thru bzip2 before it hits the disk.
repeat..
Is this possible? If this is possible, I know that I will have to somehow read the next pass of data into a different array as the AIO docs say that this array must not be altered before the async write is complete. I would like to background all of my writes to disk in order as the bzip2 pass is going to take much longer than the network read.
Is this doable? Below is a simple example of what I think is needed, but this just reads a file into array @a for testing.
use warnings;
use strict;
use EV;
use IO::AIO;
use Compress::Bzip2;
use FileHandle;
use Fcntl;
my @a;
print "loading to array...\n";
while(<>) {
$a[$. - 1] = $_;
}
print "array loaded...\n";
my $aio_w = EV::io IO::AIO::poll_fileno, EV::WRITE, \&IO::AIO::poll_cb;
aio_open "./out", O_WRONLY || O_NONBLOCK, 0, sub {
my $fh = shift or die "error while opening: $!\n";
aio_write $fh, undef, undef, $a, -1, sub {
$_[0] > 0 or die "error: $!\n";
EV::unloop;
};
};
EV::loop EV::LOOP_NONBLOCK;
© Stack Overflow or respective owner