Recursive FTP directory listing in shell/bash with a single session (using cURL or ftp)

Posted by Timo on Stack Overflow See other posts from Stack Overflow or by Timo
Published on 2010-04-10T17:57:08Z Indexed on 2010/04/10 18:03 UTC
Read the original article Hit count: 343

Filed under:

shell

|

bash

|

ftp

|

curl

|

session

I am writing a little shellscript that needs to go through all folders and files on an ftp server (recursively). So far everything works fine using cURL - but it's pretty slow, becuase cURL starts a new session for every command. So for 500 directories, cURL preforms 500 logins.

Does anybody know, whether I can stay logged in using cURL (this would be my favourite solution) or how I can use ftp with only one session in a shell script?

I know how to execute a set of ftp commands and retrieve the response, but for the recursive listing, it has to be a little more dynamic...

Thanks for your help!

© Stack Overflow or respective owner

Related posts about shell

How to restrict the users' shell allowing to execute shell programs

as seen on Server Fault - Search for 'Server Fault'
Is it possible to prevent any user to not use commands like ls, rm and other system commands which could harm the system. But the users should be able to execute shell programs. >>> More
Shell extension installation not recognized by Windows 7 64-bit shell

as seen on Stack Overflow - Search for 'Stack Overflow'
I have a Copy Hook Handler shell extension that I'm trying to install on Windows 7 64-bit. The shell extension DLL is compiled in two separate versions for 32-bit and 64-bit Windows. The DLL implements DLLRegisterServer which adds the necessary registry entries. After adding the registry entries… >>> More
Running shell commands without a shell window

as seen on Stack Overflow - Search for 'Stack Overflow'
With either subprocess.call or subprocess.Popen, executing a shell command makes a shell window quicky appear and disappear. How can I run the shell command without the shell window? >>> More
Why can't I reinstall MySQL?

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
I've been looking all around the Internet for an answer but didn't find anything. I hope you can help me now. I have a server with MySQL. From one day to another, MySQL didn't let me enter with my root password anymore (accsess denied for user 'root'@'localhost' using password: 'YES'). So I tried… >>> More
Bash/shell script - shell output redirection inside a function

as seen on Stack Overflow - Search for 'Stack Overflow'
function grabSourceFile { cd /tmp/lmpsource wget $1 > $LOG baseName=$(basename $1) tar -xvf $baseName > $LOG cd $baseName } When I call this function The captured output is not going to the log file. The output redirection works fine until I call the… >>> More

Related posts about bash

launching a program from bash causes bash to go to new prompt

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
When I run a program from the console, e.g. me@box:~$ firefox I expect the console to log error messages (I think this is std out or std err?) and other items from the program, firefox in this case. But today I notice that bash just opens the program and goes to a new prompt, e.g. me@box:~$… >>> More
How to debug a .bash_profile

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
I was updating my .bash_profile, and unfortunetly I made a few updates and now I am getting: env: bash: No such file or directory env: bash: No such file or directory env: bash: No such file or directory env: bash: No such file or directory env: bash: No such file or directory -bash: tar: command… >>> More
Every command fails with "command not found" after changing .bash_profile?

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
I was updating my .bash_profile, and unfortunetly I made a few updates and now I am getting: env: bash: No such file or directory env: bash: No such file or directory env: bash: No such file or directory env: bash: No such file or directory env: bash: No such file or directory -bash: tar: command… >>> More
Is there any fundamental difference between piping in mac and linux?

as seen on Super User - Search for 'Super User'
ps -e | grep bash sample output from a linux machine: 1128 pts/14 00:00:00 bash 7491 pts/7 00:00:00 bash 12651 pts/14 00:00:00 bash 16145 pts/2 00:00:00 bash sample output from a mac machine: 58352 ttys000 0:00.09 login -pfl username /bin/bash -c exec -la bash /bin/bash 58353 ttys000… >>> More
why is $0 set to -bash?

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
First login process name seems to be set to "-bash", but if I subshell then it becomes "bash". for example: root@nowere:~# echo $0 -bash root@nowere:~# bash root@nowere:~# echo $0 bash -bash is causing some scripts to fail, such as . /usr/share/debconf/confmodule exec /usr/share/debconf/frontend… >>> More