Article 21274 of best.www:
From: inwap@shell3.ba.best.com (Joe Smith)
Newsgroups: best.www
Subject: Re: How to download log files automatically?
Date: 6 Mar 1998 01:47:04 -0800
Organization: Chez INWAP (people, computers, cats)

In article <6do53b$t4e$1@nntp2.ba.best.com>,
Joe Landau <jrl@versaform.com> wrote:
>What would be the best way to download log files daily or weekly, 
>automatically?  And rename them in the process so they're identifiable...

I can help you with the renaming part.  Execute these commands from a
telnet session to set things up.

shell% mkdir bin
shell% lynx -source http://www.inwap.com/mybin/http-log >bin/http-log
shell% chmod +x bin/http-log
shell% crontab -l >temp || echo creating new crontab
shell% echo "15 3,7 * * * bin/http-log" >>temp
shell% crontab temp
shell% rm temp

This will copy the httpd_access log files from the public_html directory
to a directory called log1w in your HOME directory.  In the process, they
will be renamed to have a Y2K-complient name.  For instance, I've got:

shell3% ls -l log1w
total 4302
-rw-------  1 inwap  best   281812 Feb 27 03:51 19980227.acc.gz
-rw-------  1 inwap  best      438 Feb 27 03:15 19980227.err
-rw-------  1 inwap  best   333039 Feb 28 03:50 19980228.acc.gz
-rw-------  1 inwap  best     2614 Feb 28 03:14 19980228.err
-rw-------  1 inwap  best   339123 Mar  1 08:54 19980301.acc.gz
-rw-------  1 inwap  best      288 Mar  1 03:15 19980301.err
-rw-------  1 inwap  best   277414 Mar  2 03:50 19980302.acc.gz
-rw-------  1 inwap  best      218 Mar  2 03:15 19980302.err
-rw-------  1 inwap  best   285726 Mar  3 03:52 19980303.acc.gz
-rw-------  1 inwap  best       97 Mar  3 03:15 19980303.err
-rw-------  1 inwap  best   329507 Mar  4 04:17 19980304.acc.gz
-rw-------  1 inwap  best      476 Mar  4 03:15 19980304.err
-rw-------  1 inwap  best   286221 Mar  5 03:50 19980305.acc.gz
-rw-------  1 inwap  best      132 Mar  5 03:15 19980305.err

The log files will accumulate until you delete them manually.
You can downlod the log files once a week, once a month, whenever.

=====================================================================

I have another script that renames older log files into directories
called log1w, log2w, log3w, log4w and OldLogs.  (The first four directories
have one week's worth of log files each.)  This script also creates a log
summary (plain text, no barcharts).  To use it, do this:

shell% lynx -source http://www.inwap.com/mybin/http-daily   >bin/http-daily
shell% lynx -source http://www.inwap.com/mybin/http-sum     >bin/http-sum
shell% lynx -source http://www.inwap.com/mybin/http-sum.ini >bin/http-sum.ini
shell% chmod +x bin/http-daily bin/http-sum
shell% crontab -e
  # edit the file so that it now has three lines:
  15 03 * * * bin/http-log
  45 07 * * * bin/http-daily
  00 09 * * * find OldLogs -mtime +90 -exec rm {} \;
You will need to edit bin/http-sum.ini to have your web site name(s).

FYI: The perl script that summarizes the log file runs with no modifications
on Windows-95 as well.  The Sambar server (www.sambar.com) runs under
Win95/WinNT, is compatible with Best's web server, and produces NCSA
common log format files.
	-Joe

-- 
INWAP.COM is Joe and Sally Smith, and our cat.
See http://www.inwap.com/ for "ReBoot", PDP-10.