Does aria2 support write small files in batch?

Posted by Jon on Super User See other posts from Super User or by Jon
Published on 2014-05-28T01:52:39Z Indexed on 2014/05/28 3:34 UTC
Read the original article Hit count: 243

Filed under:

download

|

wget

I'm using aria2 to download 8 million jpg from flickr. Each image is about 100KB. I got a list of urls of these images in a txt file, the format is:

http://farm2.staticflickr.com/1070/1151334893_5a8e7f77f4.jpg

I'm wondering whether aria2 support writing small files in batch? Say write 100 image to disk when all of them are download in the memory, not just write every single file when the download is finished. Because I think writing in batch will better protect my hard disk.

Or do you have other software or opensource code to recommend?

© Super User or respective owner

Related posts about download

Slow Browsing/Direct Download, but Fast Bittorrent Download

as seen on Super User - Search for 'Super User'
I'm using Windows XP SP2. I have a 1 MB connection via a SpeedTouch 585, and my internet speed registers at 0.3 MB, with a maximum download of 30kbps. Not to mention a terrible ping at 500-1500. On the other hand, I get full speed in uTorrent - a bittorrent program - reaching up to 100 kbps; the way… >>> More
Download the other half of the half downloaded video and download managers for Linux

as seen on Super User - Search for 'Super User'
I tried downloading several videos last night on Chrome but most of them are downloaded incompletely. Is there a software that reads the flv or ogg files and downloads the rest of the file from the specified url? Downloading several GBs all over again gains me nothing, just adds to Global warming… >>> More
Download manager that can continue download

as seen on Stack Overflow - Search for 'Stack Overflow'
I have a file half-downloaded. Do you know a download manager that can continue the download without loading the part I already loaded? Is this possible with HTTP? Thanks >>> More
Tracking download of non-html (like pdf) downloads with jQuery and Google Analytics

as seen on Developer IT - Search for 'Developer IT'
Hi folks, it’s been quite calm at Developer IT’s this summer since we were all involved in other projects, but we are slowly comming back. In this post, we will present a simple way of tracking files download with Google Analytics with the help of jQuery. We work for a client that offers… >>> More
Direct Links To Download Microsoft Office 2007 Products From Microsoft Download Servers

as seen on Tech Dreams - Search for 'Tech Dreams'
Downloading installers of Microsoft Office 2007 products from Microsoft Office website is not an easy task. To download any of the Microsoft Office 2007 products – you need to sign-in using Windows Live/Passport/Hotmail account, fill a big profile form and accept their terms & conditions… >>> More

Related posts about wget

Make wget not download files larger than X size

as seen on Super User - Search for 'Super User'
Okay, I give up. How do I size limit which files are downloaded, like say I don't want any files bigger than 2 MB? >>> More
How to start using Wget?

as seen on Super User - Search for 'Super User'
Please, forgive me for asking this question. Usually I would try to learn thisngs myself first before bothering others, but my situation is urgent - if I don't act now and don't download all my family pictures from this website, it will be closed in about two weeks from now and I will loose all of… >>> More
wget mirroring, subdomains and directories and cookies

as seen on Server Fault - Search for 'Server Fault'
Hi all, I have an account on a web page that is now "full" (ie I have used up all my allocated space) and I would like to make a mirror of that site. wget seems like the thing to use. The problem is that I would only like to mirror the sites the lie within this directory http://user.domain.com/room/2324343/transcript/… >>> More
How can I install things in Linux with *no yum* and *no wget*?

as seen on Super User - Search for 'Super User'
I'm a newbie to Linux (that mainly uses Windows and Mac OS X) needing some advice. I was trying to install git on a Linux machine today, and encountered some problems: Not knowing the version of the installed OS, I've opened the /proc/version file which said: Linux version 2.6.9-42.0.2.ELsmp (bhcompile@ls20-bc1-13… >>> More
Getting wget to dowload only files with specific name patterns

as seen on Stack Overflow - Search for 'Stack Overflow'
I want to use wget to DL some files. I want to DL only files whose name that fit a certain pattern, e.g. ???.txt and not any other *.txt files. Can this be done with wget? I could only find a way to --accept/--reject files based on the extension. Thanks! >>> More