FtpTail – Monitor the tail of a file via FTP

ftptail prints the last lines of a file stored on an FTP server and then monitors it for changes. (Similar to the Unix tail command)

I wanted to monitor the access logs for hamstersoup.com Unfortunately I can't run any scripts on the server, I can only access the files via FTP and I don't want to waste bandwidth downloading a 100KB log file every few minutes. My script ftptail solves the problem:

Example

ftptail -f -p passwd.txt will@hamstersoup.com/access.log

Command Line Options

Usage: ftptail [OPTIONS] user@host/file
Print the last 10 lines of file on an FTP server.

Options:
   -n, --lines=N           Print the last N lines of the file (default:10)
   -f, --follow            Keep FTP connection open and append as file grows
   -s, --sleep-interval=S  Sleep S seconds between updates    (default:200)
   -p, --password-file=P   Use the password stored in P to login
   -v, --verbose           Dump info about FTP connection to STDERR

Feedback

If you download ftptail and have problems or suggestions then leave a comment.
I'm surprised that I couldn't find another utility that does this. Maybe my Google-Fu failed me, please let me know about any existing tools.

Hope somebody finds this useful.

.

Advertisements

15 Responses to FtpTail – Monitor the tail of a file via FTP

  1. Sven says:

    hi will,

    thanks for that great idea & the obvious name “ftptail”.
    I encountered some problems when using it on our proftpd as it handles SIZE very strictly (see: http://www.castaglia.org/proftpd/doc/contrib/ProFTPD-mini-HOWTO-ASCII.html).

    So I added

    my $type = $ftp->binary;

    before every

    my $size = $ftp->size

    Remaining Problems which could be worked around using another Hostname and Username:
    – ftptail cannot handle Subdomains as FTP-Hostname
    – ftptail cannot handle Username with “-” in it

    As this was my first ever Self-Fix on a Perlskript I better leave these issues alone. 🙂

    That said, I have a question:
    As we provide webhosting without Shell-Access I’d like to offer it to our advanced customers. Would that be OK for you ?

    Cheers & many thanks for ftptail

    Sven | http://blog.bad-voegelsen.de

  2. Will Moffat says:

    Hi Sven,

    Thanks for the positive feedback and bug fix.
    FtpTail was only a quick hack to solve a problem I had, that explains the host and user name bugs. I’ll try and fix that problem too.

    Feel free to use it as you like.
    regards,
    –Will

  3. Sven says:

    hi will,
    works ok now with subdomain, user with dash unchecked till now. main functionality: rockin.
    thanks for your quick fix.
    sven

  4. James Baker says:

    I know that this post is quite old, but I am not a programmer and would like some help.I use ftptail to access an ftp logfile for my gameserver.

    Basically, I have a bot (bigbrotherbot.com) that is a series of python scripts that read the logfile, and communicate with the gameserver via rcon. It warns and bans for stuff like swearing, spamming, recruiting, etc. However, the normal version needs to be run on the same server as the game. I use ftptail to make the logfile local. It works fine, except for one irratating error:
    Every second (the sleep interval I have set) it gets the last 10 lines. If there is not any new lines, it writes a period. bigbrotherbot records this as an error, giving me a syntax error every second.
    Since the syntax errors are compuslory for this level of logging (it records important error messages as well), is there any way you could remove the recording of the period at every update, or make it an option?

    Also, a hint for those wanting to save the output from this: after the commands, insert >[outputfilename] 2>&1 This saves it to the outputfilename.

  5. Will Moffat says:

    Hi James, I don’t quite understand why you need to redirect STDERR to your file as well. But that’s not important.

    If you look through the perl code you will see two ways that I print to standard error:
    warn “some info”
    print STDERR “more info”

    Just delete anything you don’t want to appear. For example, line 177
    print STDERR ".";

    Hope that solves your problem.
    –Will

  6. James Baker says:

    Oh, I thought I was doing something wrong. Is the period stderr?
    I’ll try straightforward >out, and see how that goes.
    If it works, thanks for the heads up.

  7. […] The script is thoroughly explained on his blog entry. […]

  8. ngbakes says:

    Hi,
    I’d just like to ask, as you know perl, and scripting, and I don’t etc etc
    do you know of a way I could add ftptail start ftptail status commands?

  9. Jim says:

    Gold friggin’ start for you! I have a webhost that doesn’t rotate it’s weblogs or allow SSH. This is absolutely perfect and is working like a champ. Thank you very very much!

  10. Moorglade says:

    Marvelous! I was about to try and hack something together to do exactly this, but thought I’d do a google search first. First hit, and works like a charm.

    You’ve just saved me several hours of fighting with perl – thanks 🙂

  11. Thanks for the great script. I used it to ftptail a log which had exploded to a gigabyte on a site.
    Some hosts allow multiple domains/sub-domains and give user names like ‘subdomain@domain’ while the ftp host is still ‘subdomain’ making the argument to the script look like subdomain@domain@subdomain (or anotherDomain@originalDomain@anotherDomain). The split at the first @ doesn’t work in that case. A quick change to support that could be the diff pasted below. It adds a -u or –user-name= option. Usage changes to
    will@hamstersoup.com/robots.txt
    OR
    -u will hamstersoup.com/robots.txt
    Diff:
    20a21
    > -u, –user-name=U Use the user name U to login (Use if you have an @ in it)
    67a69
    > my $userNameCL=””;
    70a73
    > ‘u|user-name=s’ => \$userNameCL,
    75,76c78
    < my $url = shift @ARGV || &usage('You must specify a URL to tail. Example: will@hamstersoup.com/robots.txt');
    my $url = shift @ARGV || &usage(‘You must specify a URL to tail. Example: will@hamstersoup.com/robots.txt OR -u will hamstersoup.com/robots.txt’);
    77a80,85
    > if($userNameCL) {
    > $url =~ /([^\/]+)\/(.+)/ || die “Malformed URL? Cannot extract host/file from $url. user=[$userNameCL] host=[$2] file=[$3]\n”;
    > ($username,$host,$file)=($userNameCL,$1,$2);
    > }
    > else {
    > $url =~ /([^@]+)\@([^\/]+)\/(.+)/ || die “Malformed URL? Cannot extract user\@host/file from $url. user=[$1] host=[$2] file=[$3]\n”;
    78a87
    > }

  12. Will Moffat says:

    Hi Pranav, thanks for the patch. Do you use GitHub? I was planning to move ftptail there so it’s easier for others to improve it.

  13. tsiru says:

    Script can’t connect when password is given this way:
    ftptail [OPTIONS] user:password@host/file

  14. John Dyer says:

    Hey Will, I couldnt find this on github so I went ahead and put it up under my account, giving you full credit of course! Let me know if you want me to take it down, or move it.

    https://github.com/krumpt/ftptail

    Thanks again, this tool was great!

    -John

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: