Apache配置與應用之AWstats日志分析實驗流程

Apache配置與應用之AWstats日志分析實驗流程

實驗環境：centos7操作系統、一臺win10虛擬機操作系統，需要軟件包：

bind.x86_64 32:9.11.4-9.P2.el7 httpd.x86_64 0:2.4.6-90.el7.centos（這里為了方便使用yum直接安裝）

awstats-7.6.tar.gz（需要自己準備，鏈接：https://pan.baidu.com/s/1apCEUdFoAoUTVeWwYyJaow
提取碼：wt6u ）

具體操作流程如下：

1）解壓縮awstats軟件包

tar zxvf awstats-7.6.tar.gz -C /usr/local

ls /usr/local/

2）安裝bind和httpd服務配置bind相關文件

yum install -y bind httpd

[root@localhost tools]# vim /etc/named.conf 
[root@localhost tools]# head -21 /etc/named.conf  | tail 
options {
        listen-on port 53 { any; };
        listen-on-v6 port 53 { ::1; };
        directory       "/var/named";
        dump-file       "/var/named/data/cache_dump.db";
        statistics-file "/var/named/data/named_stats.txt";
        memstatistics-file "/var/named/data/named_mem_stats.txt";
        recursing-file  "/var/named/data/named.recursing";
        secroots-file   "/var/named/data/named.secroots";
        allow-query     { any; };
[root@localhost tools]# vim /etc/named.rfc1912.zones 
[root@localhost tools]# head -29 /etc/named.rfc1912.zones  | tail -5
zone "lcc.com" IN {
        type master;
        file "lcc.com.zone";
        allow-update { none; };
};
[root@localhost tools]# cd /var/named/
[root@localhost named]# ls
data  dynamic  named.ca  named.empty  named.localhost  named.loopback  slaves
[root@localhost named]# cp -p named.localhost lcc.com.zone
[root@localhost named]# vim lcc.com.zone 
[root@localhost named]# cat lcc.com.zone 
$TTL 1D
@       IN SOA  @ rname.invalid. (
                                        0       ; serial
                                        1D      ; refresh
                                        1H      ; retry
                                        1W      ; expire
                                        3H )    ; minimum
        NS      @
        A       127.0.0.1
www IN  A       192.168.68.145
[root@localhost named]# systemctl stop firewalld.service 
[root@localhost named]# setenforce 0
[root@localhost named]# systemctl start named
[root@localhost named]# systemctl start httpd
[root@localhost named]# netstat -antp | grep httpd
tcp6       0      0 :::80                   :::*                    LISTEN      88534/httpd         
[root@localhost named]# netstat -antp | grep named
tcp        0      0 192.168.68.145:53       0.0.0.0:*               LISTEN      88258/named         
tcp        0      0 127.0.0.1:53            0.0.0.0:*               LISTEN      88258/named         
tcp        0      0 127.0.0.1:953           0.0.0.0:*               LISTEN      88258/named         
tcp6       0      0 ::1:53                  :::*                    LISTEN      88258/named         
tcp6       0      0 ::1:953                 :::*                    LISTEN      88258/named  

在win10的虛擬機上測試一下是否可以成功解析方才設置的域名

3）為要統計的站點建立配置文件

指定httpd主配置文件的路徑——》設置日志類型——》為指定web站點創建配置文件

[root@localhost tar]# cd /usr/local/awstats-7.6/tools/
[root@localhost tools]# ls
awstats_buildstaticpages.pl  logresolvemerge.pl
awstats_configure.pl         maillogconvert.pl
awstats_exportlib.pl         nginx
awstats_updateall.pl         urlaliasbuilder.pl
dolibarr                     webmin
geoip_generator.pl           xslt
httpd_conf
[root@localhost tools]# ./awstats_configure.pl 

----- AWStats awstats_configure 1.0 (build 20140126) (c) Laurent Destailleur -----
This tool will help you to configure AWStats to analyze statistics for
one web server. You can try to use it to let it do all that is possible
in AWStats setup, however following the step by step manual setup
documentation (docs/index.html) is often a better idea. Above all if:
- You are not an administrator user,
- You want to analyze downloaded log files without web server,
- You want to analyze mail or ftp log files instead of web log files,
- You need to analyze load balanced servers log files,
- You want to 'understand' all possible ways to use AWStats...
Read the AWStats documentation (docs/index.html).

-----> Running OS detected: Linux, BSD or Unix
Warning: AWStats standard directory on Linux OS is '/usr/local/awstats'.
If you want to use standard directory, you should first move all content
of AWStats distribution from current directory:
/usr/local/awstats-7.6
to standard directory:
/usr/local/awstats
And then, run configure.pl from this location.
Do you want to continue setup from this NON standard directory [yN] ? y   //輸入y

-----> Check for web server install

Enter full config file path of your Web server.
Example: /etc/httpd/httpd.conf
Example: /usr/local/apache2/conf/httpd.conf
Example: c:\Program files\apache group\apache\conf\httpd.conf
Config file path ('none' to skip web server setup):
> /etc/httpd/conf/httpd.conf       //HTTPd的主配置文件的路徑

-----> Check and complete web server config file '/etc/httpd/conf/httpd.conf'
  All AWStats directives are already present.

-----> Update model config file '/usr/local/awstats-7.6/wwwroot/cgi-bin/awstats.model.conf'
  File awstats.model.conf updated.

-----> Need to create a new config file ?
Do you want me to build a new AWStats config/profile
file (required if first install) [y/N] ? y             //輸入y

-----> Define config file name to create
What is the name of your web site or profile analysis ?
Example: www.mysite.com
Example: demo
Your web site, virtual server or profile name:
> www.lcc.com                                       //自己設置的域名

-----> Define config file path
In which directory do you plan to store your config file(s) ?
Default: /etc/awstats
Directory path to store config file(s) (Enter for default):
>                                                                      //默認下面全都回車即可

-----> Create config file '/etc/awstats/awstats.www.lcc.com.conf'
 Config file /etc/awstats/awstats.www.lcc.com.conf created.

-----> Add update process inside a scheduler
Sorry, configure.pl does not support automatic add to cron yet.
You can do it manually by adding the following command to your cron:  //執行日志分析的方法（兩種）
/usr/local/awstats-7.6/wwwroot/cgi-bin/awstats.pl -update -config=www.lcc.com
Or if you have several config files and prefer having only one command:
/usr/local/awstats-7.6/tools/awstats_updateall.pl now      //會報錯，解決方法在文末
Press ENTER to continue...  

A SIMPLE config file has been created: /etc/awstats/awstats.www.lcc.com.conf
You should have a look inside to check and change manually main parameters.
You can then manually update your statistics for 'www.lcc.com' with command:
> perl awstats.pl -update -config=www.lcc.com
You can also read your statistics for 'www.lcc.com' with URL:
> http://localhost/awstats/awstats.pl?config=www.lcc.com   //這里就是我們要測試的地址，需要將localhost換為www.lcc.com

Press ENTER to finish...

[root@localhost tools]# systemctl restart httpd

4）修改Apache 2.4以上版本自動生成的awstats訪問權限

[root@localhost tools]# vim /etc/httpd/conf/httpd.conf 
[root@localhost tools]# tail /etc/httpd/conf/httpd.conf 
# This is to permit URL access to scripts/files in AWStats directory.
#
<Directory "/usr/local/awstats-7.6/wwwroot">
    Options None
    AllowOverride None
    #Order allow,deny            //注釋掉
    #Allow from all             //注釋掉
    Require all granted         //添加（所有權限）
</Directory>

5）修改站點統計配置文件

[root@localhost tools]# vim /etc/awstats/awstats.www.lcc.com.conf 
[root@localhost tools]# head -50 /etc/awstats/awstats.www.lcc.com.conf  | tail -1
LogFile="/var/log/httpd/access_log"
[root@localhost tools]# head -220 /etc/awstats/awstats.www.lcc.com.conf  | tail -1
DirData="/var/lib/awstats"
[root@localhost tools]# cd /var/lib/               //創建上面路徑的文件夾awstats
[root@localhost lib]# mkdir awstats

6）執行日志分析

//在上面的第三步過程中是給出方法來執行日志分析的方式的，而且也可以通過執行周期性計劃任務來方便管理日志
//每執行一次該命令，刷新訪問頁面就會對相關數據進行統計顯示出來
[root@localhost tools]# /usr/local/awstats-7.6/wwwroot/cgi-bin/awstats.pl -update -config=www.lcc.com
Create/Update database for config "/etc/awstats/awstats.www.lcc.com.conf" by AWStats version 7.6 (build 20161204)
From data in log file "/var/log/httpd/access_log"...
Phase 1 : First bypass old records, searching new record...
Searching new records from beginning of log file...
Jumped lines in file: 0
Parsed lines in file: 0
 Found 0 dropped records,
 Found 0 comments,
 Found 0 blank records,
 Found 0 corrupted records,
 Found 0 old records,
 Found 0 new qualified records.

為了方便管理，那么我們可以使用周期性任務計劃來管理執行有興趣可以使用crontab -e 命令來創建周期性任務計劃表

小結

本次實驗是為了實現使用awstats進行日志分析，過程并不復雜，但是需要注意的地方還是需要留意的，對應之處本人稍作了一些注釋。

當然該實驗有兩個問題，這邊仔細講述一下：

1.使用http://www.lcc.com/awstats/awstats.pl?config=www.lcc.com 進行訪問不容易讓人記住，所有需要優化一下

[root@localhost html]#  cd /usr/local/awstats-7.6/tools/
[root@localhost tools]# cd -
/var/www/html
[root@localhost html]# vim aws.html
[root@localhost html]# cat aws.html 
<html>
<head>
<meta http-equiv=refresh content="0;
url=http://www.lcc.com/awstats/awstats.pl?config=www.lcc.com">
</head>
<body>
</body>
</html>

這樣設置之后可以使用http://www.lcc.com/aws.html進行訪問

2.在執行日志分析命令時使用第二種方法會遇到如下error

[root@localhost tools]# /usr/local/awstats-7.6/tools/awstats_updateall.pl now
Error: Can't find AWStats program ('awstats.pl').
Use -awstatsprog option to solve this.

這個原因是因為未指定awstats.pl的絕對路徑，根據如下操作即可

[root@localhost tools]# find /usr/local/ -name awstats.pl
/usr/local/awstats-7.6/wwwroot/cgi-bin/awstats.pl
[root@localhost tools]# cd ../wwwroot/cgi-bin/
[root@localhost cgi-bin]# ls
awdownloadcsv.pl  awredir.pl  awstats.model.conf  awstats.pl  lang  lib  plugins
[root@localhost tools]# /usr/local/awstats-7.6/tools/awstats_updateall.pl now -awstatsprog="../wwwroot/cgi-bin/awstats.pl"
Running '"../wwwroot/cgi-bin/awstats.pl" -update -config=www.lcc.com -configdir="/etc/awstats"' to update config www.lcc.com
Create/Update database for config "/etc/awstats/awstats.www.lcc.com.conf" by AWStats version 7.6 (build 20161204)
From data in log file "/var/log/httpd/access_log"...
Phase 1 : First bypass old records, searching new record...
Direct access after last parsed record (after line 101)
Jumped lines in file: 101
 Found 101 already parsed records.
Parsed lines in file: 236
 Found 0 dropped records,
 Found 0 comments,
 Found 0 blank records,
 Found 2 corrupted records,
 Found 0 old records,
 Found 234 new qualified records.