大数据技术之Hadoop(十一)——网站流量日志数据分析系统

目录

素材：

一、模块开发——数据预处理

1、分析预处理的数据

2、实现数据的预处理

（1）创建Maven项目，添加相关依赖

（2）创建JavaBean对象，封装日志记录

（3）创建MapReduce程序，执行数据预处理

 二、模块开发——数据仓库开发

1、上传文件

2、实现数据仓库

三、模块开发——数据分析

 四、模块开发——数据导出

五、模块开发——日志分析系统报表展示

1、搭建日志分析系统

（1）创建项目，添加相关依赖

（2）编写配置文件

2、实现报表功能展示

（1）创建持久化类

（2）实现DAO层

（3）实现Service层

（4）实现Controller层

（5）实现页面功能

3、系统功能模块展示

​编辑

参考书籍：

素材：

http://链接: https://pan.baidu.com/s/1aNxbVjNq1z1A-oOIYy_E4Q?pwd=gjpt 提取码: gjpthttp://链接: https://pan.baidu.com/s/1aNxbVjNq1z1A-oOIYy_E4Q?pwd=gjpt 提取码: gjpt

一、模块开发——数据预处理

1、分析预处理的数据

        在收集的日志文件中，通常情况下，不能直接将日志文件进行数据分析，这是因为日志文件中有许多不合法的数据(比如日志数据在网络传输过程中发送数据丢失)。

        在数据预处理阶段,主要目的就是对收集的原始数据进行清洗和筛选，因此使用MapReduce 技术就可以轻松实现。在实际开发中，数据预处理过程通常不会直接将不合法的数据直接删除，而是对每条数据添加标识字段，从而避免其他业务使用时丢失数据。

        另外,此次数据预处理只是清洗和筛选不合法的数据信息，会读取每行日志文件数据并最终输出一条数据，不会进行其他操作，因此在使用MapReduce技术进行处理过程中，只会涉及 Map 阶段，不会涉及Reduce 阶段。在默认情况下，ReduceTask值为1，因此在主运行函数中，需要设置 Job.setNumReduceTasks(0)。

2、实现数据的预处理

（1）创建Maven项目，添加相关依赖

pom.xml文件配置如下：

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>cn.itcast</groupId>
    <artifactId>HadoopDataReport</artifactId>
    <version>1.0-SNAPSHOT</version>

    <dependencies>
        <dependency>
            <groupId>org.apache.hadoop</groupId>
            <artifactId>hadoop-common</artifactId>
            <version>2.10.1</version>
        </dependency>
        <dependency>
            <groupId>org.apache.hadoop</groupId>
            <artifactId>hadoop-client</artifactId>
            <version>2.10.1</version>
        </dependency>
        <dependency>
            <groupId>org.apache.hadoop</groupId>
            <artifactId>hadoop-hdfs</artifactId>
            <version>2.10.1</version>
        </dependency>
        <dependency>
            <groupId>org.apache.hadoop</groupId>
            <artifactId>hadoop-mapreduce-client-core</artifactId>
            <version>2.10.1</version>
        </dependency>

    </dependencies>


</project>

（2）创建JavaBean对象，封装日志记录

        收集的日志数据中，每一行代表一条日志记录，并且包含有多个用空格分隔的字段信息，为了方便后续数据处理，创建一个 JavaBean 对象对每条数据进行封装。

WebLogBean.java

package cn.itcast.weblog.bean;

import org.apache.hadoop.io.Writable;

import java.io.DataInput;
import java.io.DataOutput;
import java.io.IOException;

/**
 * 对接外部数据的层，表结构定义最好跟外部数据源保持一致
 * 同时实现序列号，方便网络数据传播
 */
public class WebLogBean implements Writable {
    private boolean valid = true;   //标记数据是否合法
    private String remote_addr;     //访客IP地址
    private String remote_user;     //记录访客用户信息，忽略属性“-”
    private String time_local;      //记录访问时间与时区
    private String request;         //记录请求的URL
    private String status;          //请求状态
    private String body_bytes_sent; //记录发送给客户端文件主体内容大小
    private String http_referer;    //记录从哪个页面链接访问过来的
    private String http_user_agent; //记录客户浏览器的相关信息

    //设置 WebLogBean 进行字段数据封装
    public void setBean(boolean valid, String remote_addr, String remote_user,
                        String time_local, String request, String status,
                        String body_bytes_sent,
                        String http_referer, String http_user_agent) {
        this.valid = valid;
        this.remote_addr = remote_addr;
        this.remote_user = remote_user;
        this.time_local = time_local;
        this.request = request;
        this.status = status;
        this.body_bytes_sent = body_bytes_sent;
        this.http_referer = http_referer;
        this.http_user_agent = http_user_agent;
    }

    public boolean isValid() {
        return valid;
    }

    public void setValid(boolean valid) {
        this.valid = valid;
    }

    public String getRemote_addr() {
        return remote_addr;
    }

    public void setRemote_addr(String remote_addr) {
        this.remote_addr = remote_addr;
    }

    public String getRemote_user() {
        return remote_user;
    }

    public void setRemote_user(String remote_user) {
        this.remote_user = remote_user;
    }

    public String getTime_local() {
        return time_local;
    }

    public void setTime_local(String time_local) {
        this.time_local = time_local;
    }

    public String getRequest() {
        return request;
    }

    public void setRequest(String request) {
        this.request = request;
    }

    public String getStatus() {
        return status;
    }

    public void setStatus(String status) {
        this.status = status;
    }

    public String getBody_bytes_sent() {
        return body_bytes_sent;
    }

    public void setBody_bytes_sent(String body_bytes_sent) {
        this.body_bytes_sent = body_bytes_sent;
    }

    public String getHttp_referer() {
        return http_referer;
    }

    public void setHttp_referer(String http_referer) {
        this.http_referer = http_referer;
    }

    public String getHttp_user_agent() {
        return http_user_agent;
    }

    public void setHttp_user_agent(String http_user_agent) {
        this.http_user_agent = http_user_agent;
    }

    //重写 toString() 方法，使用 Hive 默认分隔符进行分隔，为后续导入 Hive 表提供便利
    @Override
    public String toString() {
        StringBuilder sb = new StringBuilder();
        sb.append(valid);

        sb.append("\001").append(this.getRemote_addr());
        sb.append("\001").append(this.getRemote_user());
        sb.append("\001").append(this.getTime_local());
        sb.append("\001").append(this.getRequest());
        sb.append("\001").append(this.getStatus());
        sb.append("\001").append(this.getBody_bytes_sent());
        sb.append("\001").append(this.getHttp_referer());
        sb.append("\001").append(this.getHttp_user_agent());
        return sb.toString();
    }

    //序列化方法

    @Override
    public void readFields(DataInput dataInput) throws IOException {
        this.valid = dataInput.readBoolean();
        this.remote_addr = dataInput.readUTF();
        this.remote_user = dataInput.readUTF();
        this.time_local = dataInput.readUTF();
        this.request = dataInput.readUTF();
        this.status = dataInput.readUTF();
        this.body_bytes_sent = dataInput.readUTF();
        this.http_referer = dataInput.readUTF();
        this.http_user_agent = dataInput.readUTF();
    }

    // 反序列化方法（注意与序列化方法顺序保持一致）

    @Override
    public void write(DataOutput dataOutput) throws IOException {
        dataOutput.writeBoolean(this.valid);
        dataOutput.writeUTF(this.remote_addr);
        dataOutput.writeUTF(this.remote_user);
        dataOutput.writeUTF(this.time_local);
        dataOutput.writeUTF(this.request);
        dataOutput.writeUTF(this.status);
        dataOutput.writeUTF(this.body_bytes_sent);
        dataOutput.writeUTF(this.http_referer);
        dataOutput.writeUTF(this.http_user_agent);
    }
}

（3）创建MapReduce程序，执行数据预处理

        创建 JavaBean 实体类后，接下来开始编写 MapReduce 程序，进行数据预处理。

WebLogPreProcess.java

package cn.itcast.weblog.preproces;

import cn.itcast.weblog.bean.WebLogBean;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.LongWritable;
import org.apache.hadoop.io.NullWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.Job;
import org.apache.hadoop.mapreduce.Mapper;
import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;
import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;
import java.io.IOException;
import java.util.HashSet;
import java.util.Set;

/**
 *  日志数据处理：数据清洗、日期格式转换、缺失字段填充默认值、字段添加合法标记
 */
public class WebLogPreProcess {
    public static void main(String[] args) throws Exception{
        Configuration conf = new Configuration();
        Job job = Job.getInstance(conf);
        job.setJarByClass(WebLogPreProcess.class);
        job.setMapperClass(WebLogPreProcessMapper.class);
        job.setOutputKeyClass(Text.class);
        job.setOutputValueClass(NullWritable.class);
        //此次案例测试数据不是非常大，所以使用本地路径
        //（实际情况会对 HDFS 上存储的文件进行处理）
        FileInputFormat.setInputPaths(job, new Path("/home/huanganchi/Hadoop/实训项目/HadoopDemo/textHadoop/weblog/input"));
        FileOutputFormat.setOutputPath(job, new Path("/home/huanganchi/Hadoop/实训项目/HadoopDemo/textHadoop/weblog/output"));
        //将 ReduceTask 属设置为 0，不需要 Reduce 阶段
        job.setNumReduceTasks(0);
        boolean res = job.waitForCompletion(true);
        System.exit(res ? 0 : 1);
    }
    //mapreduce 程序 map 阶段
    public static class WebLogPreProcessMapper extends Mapper<LongWritable, Text, Text, NullWritable> {
        //用来存储网站 URL 分类数据
        Set<String> pages = new HashSet<String>();
        Text k = new Text();
        NullWritable v = NullWritable.get();
        /**
         * 设置初始化方法， 用来表示用户请求的是合法数据
         */
        @Override
        protected void setup(Context context) throws IOException, InterruptedException {
            pages.add("/about");
            pages.add("/black-ip-list");
            pages.add("/cassendra-cluster/");
            pages.add("/finance-rhive-repurchase");
            pages.add("/hadoop-family-roadmad");
            pages.add("/hadoop-hive-intro/");
            pages.add("/hadoop-zookeeper-intro");
            pages.add("/hadoop-mahout-roadmap");
        }
        /**
         * 重写 map()方法，对每行记录重新解析转换并输出
         */
        @Override
        protected void map(LongWritable key, Text value, Context context) throws IOException, InterruptedException {
            //获取一行数据
            String line = value.toString();
            //调用解析类 WebLogParser 解析日志数据，最后封装为 WebLogBean 对象
            WebLogBean webLogBean = WebLogParser.parser(line);
            if (webLogBean != null) {
                //过滤 js/图片/css 等静态资源

                WebLogParser.filtStaticResource(webLogBean, pages);
                k.set(webLogBean.toString());
                context.write(k, v);
            }
        }
    }
}

WebLogParser.java

package cn.itcast.weblog.preproces;

import cn.itcast.weblog.bean.WebLogBean;
import java.text.ParseException;
import java.text.SimpleDateFormat;
import java.util.Locale;
import java.util.Set;

public class WebLogParser {
    //定义时间格式
    public static SimpleDateFormat df1 = new SimpleDateFormat("dd/MMM/yyyy:HH:mm:ss", Locale.US);
    public static SimpleDateFormat df2 = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss", Locale.US);

    /**
     * 根据采集的数据字段信息进行解析封装
     */
    public static WebLogBean parser(String line) {
        WebLogBean webLogBean = new WebLogBean();
        //把一行数据以空格字符切割并存入数组arr中
        String[] arr = line.split(" ");
        //如果数组长度小于等于11，说明这条数据不完整，因此可以忽略这条数据
        if (arr.length > 11) {
            //满足条件的数据逐个赋值给webLogBean对象
            webLogBean.setRemote_addr(arr[0]);
            webLogBean.setRemote_user(arr[1]);
            String time_local = formatDate(arr[3].substring(1));
            if (null == time_local || "".equals(time_local)) time_local = "-invalid_time-";
            webLogBean.setTime_local(time_local);
            webLogBean.setRequest(arr[6]);
            webLogBean.setStatus(arr[8]);
            webLogBean.setBody_bytes_sent(arr[9]);
            webLogBean.setHttp_referer(arr[10]);
            //如果useragent元素较多，拼接useragent
            if (arr.length > 12) {
                StringBuilder sb = new StringBuilder();
                for (int i = 11; i < arr.length; i++) {
                    sb.append(arr[i]);
                }
                webLogBean.setHttp_user_agent(sb.toString());
            } else {
                webLogBean.setHttp_user_agent(arr[11]);
            }
            //大于 400，HTTP 错误
            if (Integer.parseInt(webLogBean.getStatus()) >= 400) {
                webLogBean.setValid(false);
            }
            if ("-invalid_time-".equals(webLogBean.getTime_local())) {
                webLogBean.setValid(false);
            }
        } else {
            webLogBean = null;
        }
        return webLogBean;
    }
    //对请求路径资源是否合法进行标记
    public static void filtStaticResource(WebLogBean bean, Set<String> pages) {
        if (!pages.contains(bean.getRequest())) {
            bean.setValid(false);
        }
    }
    //格式化时间方法
    public static String formatDate(String time_local) {
        try {
            return df2.format(df1.parse(time_local));
        } catch (ParseException e) {
            return null;
        }
    }
}

运行结果：

 二、模块开发——数据仓库开发

1、上传文件

在启动了Hadoop的Linux系统root目录下创建目录weblog，并将预处理产生的结果文件上传到 weblog 目录下。

cd
mkdir weblog
cd weblog
执行 rz 文件上传命令

在HDFS上创建目录，用于存放预处理过的数据，并上传数据到HDFS。

hadoop fs -mkdir -p /weblog/preprocessed
hadoop fs -put part-m-00000 /weblog/preprocessed

2、实现数据仓库

启动Hive数据仓库，执行以下操作：

--创建数据仓库
DROP DATABASE IF EXISTS weblog;
CREATE DATABASE weblog;

USE weblog;

--创建表
CREATE TABLE ods_weblog_origin (
	valid string,	--有效标志
	remote_addr string, --来源IP
	remote_user string,	--用户标志
	time_local string,	--访问完整时间
	request string,		--请求的URL
	status string,		--响应码
	body_bytes_sent string,	--传输字节数
	http_referer string,	--来源URL
	http_user_agent string	--客户终端标志
	)
	partitioned by (datestr string)
	row format delimited fields terminated by '\001';

--导入数据
load data inpath '/weblog/preprocessed' overwrite into table ods_weblog_origin partition(datestr='20130918');

--生成明细表
--1. 创建明细表 ods_weblog_detwail 
CREATE TABLE ods_weblog_detwail (
	valid 		string,	--有效标志
	remote_addr	string, --来源IP
	remote_user	string,	--用户标志
	time_local 	string,	--访问完整时间
	daystr 		string,	--访问日期
	timestr 	string,	--访问时间
	month 	 	string,	--访问月
	day 	 	string,	--访问日
	hour 	 	string,	--访问时
	request 	string,	--请求的URL
	status 		string,		--响应码
	body_bytes_sent string,	--传输字节数
	http_referer 	string,	--来源URL
	ref_host 		string,	--来源的host
	ref_path		string,	--来源路径
	ref_query		string,	--来源参数query
	ref_query_id	string,	--来源参数query的值
	http_user_agent string	--客户终端标志
	)
	partitioned by (datestr string);
	
--2. 创建临时中间表 t_ods_tmp_referurl
CREATE TABLE t_ods_tmp_referurl as 	SELECT a.*, b.* 
	FROM ods_weblog_origin a LATERAL VIEW
	parse_url_tuple(regexp_replace(http_referer, "\"", ""),'HOST', 'PATH', 'QUERY', 'QUERY:id') b 
	as host, path, query, query_id;
	
--3. 创建临时中间表 t_ods_tmp_detail
CREATE TABLE t_ods_tmp_detail as 
	SELECT b.*, substring(time_local, 0, 10) as daystr,  
	substring(time_local, 12) as tmstr,
	substring(time_local, 6, 2) as month,
	substring(time_local, 9, 2) as day,
	substring(time_local, 11, 3) as hour
	FROM t_ods_tmp_referurl b;
	
--4. 修改默认动态分区参数
set hive.exec.dynamic.partition=true;
set hive.exec.dynamic.partition.mode=nonstrict;

--5. 向 ods_weblog_detwail 表中加载数据
insert overwrite table ods_weblog_detwail partition(datestr)
SELECT DISTINCT otd.valid, otd.remote_addr, otd.remote_user, 
otd.time_local, otd.daystr, otd.tmstr, otd.month, otd.day, otd.hour,
otr.request, otr.status, otr.body_bytes_sent,
otr.http_referer, otr.host, otr.path,
otr.query, otr.query_id, otr.http_user_agent, otd.daystr
FROM t_ods_tmp_detail as otd, t_ods_tmp_referurl as otr
WHERE otd.remote_addr = otr.remote_addr
AND otd.time_local = otr.time_local
AND otd.body_bytes_sent = otr.body_bytes_sent
AND otd.request = otr.request;

三、模块开发——数据分析

--数据分析
--流量分析
--创建每日访问量表dw_pvs_everyday
CREATE TABLE IF NOT EXISTS dw_pvs_everyday(pvs bigint, month string, day string);

--从宽表 ods_weblog_detwail 获取每日访问量数据并插入维度表 dw_pvs_everyday
INSERT INTO TABLE dw_pvs_everyday
SELECT COUNT(*) AS pvs, owd.month AS month, owd.day AS day 
FROM ods_weblog_detwail owd GROUP BY owd.month, owd.day;

--人均浏览量分析
--创建维度表dw_avgpv_user_everyday
CREATE TABLE  IF NOT EXISTS dw_avgpv_user_everyday (day string, avgpv string);

--从宽表 ods_weblog_detwail 获取相关数据并插入维度表 dw_avgpv_user_everyday
INSERT INTO TABLE dw_avgpv_user_everyday
SELECT '2013-09-18', SUM(b.pvs)/COUNT(b.remote_addr) FROM
(SELECT remote_addr, COUNT(1) AS pvs FROM ods_weblog_detwail WHERE
datestr = '2013-09-18' GROUP by remote_addr) b;

INSERT INTO TABLE dw_avgpv_user_everyday
SELECT '2013-09-19', SUM(b.pvs)/COUNT(b.remote_addr) FROM
(SELECT remote_addr, COUNT(1) AS pvs FROM ods_weblog_detwail WHERE
datestr = '2013-09-19' GROUP by remote_addr) b;

INSERT INTO TABLE dw_avgpv_user_everyday
SELECT '2013-09-20', SUM(b.pvs)/COUNT(b.remote_addr) FROM
(SELECT remote_addr, COUNT(1) AS pvs FROM ods_weblog_detwail WHERE
datestr = '2013-09-20' GROUP by remote_addr) b;

INSERT INTO TABLE dw_avgpv_user_everyday
SELECT '2013-09-21', SUM(b.pvs)/COUNT(b.remote_addr) FROM
(SELECT remote_addr, COUNT(1) AS pvs FROM ods_weblog_detwail WHERE
datestr = '2013-09-21' GROUP by remote_addr) b;

INSERT INTO TABLE dw_avgpv_user_everyday
SELECT '2013-09-22', SUM(b.pvs)/COUNT(b.remote_addr) FROM
(SELECT remote_addr, COUNT(1) AS pvs FROM ods_weblog_detwail WHERE
datestr = '2013-09-22' GROUP by remote_addr) b;

INSERT INTO TABLE dw_avgpv_user_everyday
SELECT '2013-09-23', SUM(b.pvs)/COUNT(b.remote_addr) FROM
(SELECT remote_addr, COUNT(1) AS pvs FROM ods_weblog_detwail WHERE
datestr = '2013-09-23' GROUP by remote_addr) b;

INSERT INTO TABLE dw_avgpv_user_everyday
SELECT '2013-09-24', SUM(b.pvs)/COUNT(b.remote_addr) FROM
(SELECT remote_addr, COUNT(1) AS pvs FROM ods_weblog_detwail WHERE
datestr = '2013-09-24' GROUP by remote_addr) b;

 四、模块开发——数据导出

1. 创建 MySql 数据库和表
--数据导出
--创建数据仓库
DROP DATABASE IF EXISTS sqoopdb;
CREATE DATABASE sqoopdb;

USE sqoopdb;

--创建表
CREATE TABLE t_avgpv_num (
dateStr VARCHAR(255) DEFAULT NULL,
avgPvNum DECIMAL(6,2) DEFAULT NULL
) ENGINE=MyISAM DEFAULT CHARSET=utf8;


2. 执行数据导出命令
sqoop export \
--connect jdbc:mysql://hadoop01.bgd01:3306/sqoopdb \
--username root \
--password 123456 \
--table t_avgpv_num \
--columns "dateStr,avgPvNum" \
--fields-terminated-by '\001' \
--export-dir /user/hive/warehouse/weblog.db/dw_avgpv_user_everyday

五、模块开发——日志分析系统报表展示

1、搭建日志分析系统

（1）创建项目，添加相关依赖

        创建一个Java Web 框架的Maven工程

pom.xml

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>cn.itcast</groupId>
    <artifactId>Weblog</artifactId>
    <version>1.0-SNAPSHOT</version>
    <packaging>war</packaging>
    <dependencies>
        <!-- Spring -->
        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-context</artifactId>
            <version>4.2.4.RELEASE</version>
        </dependency>
        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-beans</artifactId>
            <version>4.2.4.RELEASE</version>
        </dependency>
        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-webmvc</artifactId>
            <version>4.2.4.RELEASE</version>
        </dependency>
        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-jdbc</artifactId>
            <version>4.2.4.RELEASE</version>
        </dependency>
        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-aspects</artifactId>
            <version>4.2.4.RELEASE</version>
        </dependency>
        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-jms</artifactId>
            <version>4.2.4.RELEASE</version>
        </dependency>
        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-context-support</artifactId>
            <version>4.2.4.RELEASE</version>
        </dependency>
        <!-- Mybatis -->
        <dependency>
            <groupId>org.mybatis</groupId>
            <artifactId>mybatis</artifactId>
            <version>3.2.8</version>
        </dependency>
        <dependency>
            <groupId>org.mybatis</groupId>
            <artifactId>mybatis-spring</artifactId>
            <version>1.2.2</version>
        </dependency>
        <dependency>
            <groupId>com.github.miemiedev</groupId>
            <artifactId>mybatis-paginator</artifactId>
            <version>1.2.15</version>
        </dependency>
        <!-- MySql -->
        <dependency>
            <groupId>mysql</groupId>
            <artifactId>mysql-connector-java</artifactId>
            <version>5.1.32</version>
        </dependency>
        <!-- 连接池 -->
        <dependency>
            <groupId>com.alibaba</groupId>
            <artifactId>druid</artifactId>
            <version>1.0.9</version>
        </dependency>
        <!-- JSP相关 -->
        <dependency>
            <groupId>jstl</groupId>
            <artifactId>jstl</artifactId>
            <version>1.2</version>
        </dependency>
        <dependency>
            <groupId>javax.servlet</groupId>
            <artifactId>servlet-api</artifactId>
            <version>2.5</version>
            <scope>provided</scope>
        </dependency>
        <dependency>
            <groupId>javax.servlet</groupId>
            <artifactId>jsp-api</artifactId>
            <version>2.0</version>
            <scope>provided</scope>
        </dependency>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.12</version>
        </dependency>
        <dependency>
            <groupId>com.fasterxml.jackson.core</groupId>
            <artifactId>jackson-databind</artifactId>
            <version>2.4.2</version>
        </dependency>
    </dependencies>

    <build>
        <finalName>${project.artifactId}</finalName>
        <resources>
            <resource>
                <directory>src/main/java</directory>
                <includes>
                    <include>**/*.properties</include>
                    <include>**/*.xml</include>
                </includes>
                <filtering>false</filtering>
            </resource>
            <resource>
                <directory>src/main/resources</directory>
                <includes>
                    <include>**/*.properties</include>
                    <include>**/*.xml</include>
                </includes>
                <filtering>false</filtering>
            </resource>
        </resources>

        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.2</version>
                <configuration>
                    <source>1.8</source>
                    <target>1.8</target>
                    <encoding>UTF-8</encoding>
                </configuration>
            </plugin>

            <!-- 配置Tomcat插件 -->
            <plugin>
                <groupId>org.apache.tomcat.maven</groupId>
                <artifactId>tomcat7-maven-plugin</artifactId>
                <version>2.2</version>
                <configuration>
                    <path>/</path>
                    <port>8080</port>
                </configuration>
            </plugin>
        </plugins>
    </build>
    <properties>
        <maven.compiler.source>8</maven.compiler.source>
        <maven.compiler.target>8</maven.compiler.target>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    </properties>

</project>

（2）编写配置文件

applicationContext.xml

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
	xmlns:context="http://www.springframework.org/schema/context" xmlns:p="http://www.springframework.org/schema/beans"
	xmlns:aop="http://www.springframework.org/schema/aop" xmlns:tx="http://www.springframework.org/schema/tx"
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-4.2.xsd
	http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-4.2.xsd
	http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop-4.2.xsd http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx-4.2.xsd
	http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util-4.2.xsd">

	<!-- 数据库连接池 -->
	<!-- 加载配置文件 -->
	<context:property-placeholder location="classpath:properties/db.properties" />
	<!-- 数据库连接池 -->
	<bean id="dataSource" class="com.alibaba.druid.pool.DruidDataSource"
		destroy-method="close">
		<property name="url" value="${jdbc.url}" />
		<property name="username" value="${jdbc.username}" />
		<property name="password" value="${jdbc.password}" />
		<property name="driverClassName" value="${jdbc.driver}" />
		<property name="maxActive" value="10" />
		<property name="minIdle" value="5" />
	</bean>
	<!-- 让spring管理sqlsessionfactory使用mybatis和spring整合包中的 -->
	<bean id="sqlSessionFactory" class="org.mybatis.spring.SqlSessionFactoryBean">
		<!-- 数据库连接池 -->
		<property name="dataSource" ref="dataSource" />
		<!-- 加载mybatis的全局配置文件 -->
		<property name="configLocation" value="classpath:mybatis/SqlMapConfig.xml" />
	</bean>
	<!-- 使用扫描包的形式来创建mapper代理对象 -->
	<bean class="org.mybatis.spring.mapper.MapperScannerConfigurer">
		<property name="basePackage" value="cn.itcast.mapper" />
	</bean>

	<!-- 事务管理器 -->
	<bean id="transactionManager"
		class="org.springframework.jdbc.datasource.DataSourceTransactionManager">
		<!-- 数据源 -->
		<property name="dataSource" ref="dataSource" />
	</bean>
	<!-- 通知 -->
	<tx:advice id="txAdvice" transaction-manager="transactionManager">
		<tx:attributes>
			<!-- 传播行为 -->
			<tx:method name="save*" propagation="REQUIRED" />
			<tx:method name="insert*" propagation="REQUIRED" />
			<tx:method name="add*" propagation="REQUIRED" />
			<tx:method name="create*" propagation="REQUIRED" />
			<tx:method name="delete*" propagation="REQUIRED" />
			<tx:method name="update*" propagation="REQUIRED" />
			<tx:method name="find*" propagation="SUPPORTS" read-only="true" />
			<tx:method name="select*" propagation="SUPPORTS" read-only="true" />
			<tx:method name="get*" propagation="SUPPORTS" read-only="true" />
		</tx:attributes>
	</tx:advice>
	<!-- 切面 -->
	<aop:config>
		<aop:advisor advice-ref="txAdvice"
			pointcut="execution(* cn.itcast.service..*.*(..))" />
	</aop:config>

	<!-- 配置包扫描器，扫描所有带@Service注解的类 -->
	<context:component-scan base-package="cn.itcast.service" />

</beans>

db.properties

jdbc.driver=com.mysql.jdbc.Driver
jdbc.url=jdbc:mysql://192.168.8.201:3306/sqoopdb?characterEncoding=utf-8
jdbc.username=root
jdbc.password=123456

springmvc.xml

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:p="http://www.springframework.org/schema/beans"
	xmlns:context="http://www.springframework.org/schema/context"
    xmlns:mvc="http://www.springframework.org/schema/mvc"
	xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-4.2.xsd
        http://www.springframework.org/schema/mvc http://www.springframework.org/schema/mvc/spring-mvc-4.2.xsd
        http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-4.2.xsd">

	<!-- 扫描指定包路径 使路径当中的@controller注解生效 -->
	<context:component-scan base-package="cn.itcast.controller" />
	<!-- mvc的注解驱动  -->
	<mvc:annotation-driven />
	
	<!-- 视图解析器 -->
	<bean
		class="org.springframework.web.servlet.view.InternalResourceViewResolver">
		<property name="prefix" value="/WEB-INF/jsp/" />
		<property name="suffix" value=".jsp" />
	</bean>
	
	<!-- 配置资源映射 -->
	<mvc:resources location="/css/" mapping="/css/**"/>
	<mvc:resources location="/js/" mapping="/js/**"/>
	<mvc:resources location="/echarts/" mapping="/echarts/**"/>
	<mvc:resources location="/assets/" mapping="/assets/**"/>
	<mvc:resources location="/img/" mapping="/img/**"/>
</beans>

web.xml

<?xml version="1.0" encoding="UTF-8"?>
<web-app xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://java.sun.com/xml/ns/javaee" xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd" version="2.5">
  <display-name>Weblog</display-name>
  
  <welcome-file-list>
		<welcome-file>index.html</welcome-file>
	</welcome-file-list>

	<!-- 加载spring容器 -->
	<context-param>
		<param-name>contextConfigLocation</param-name>
		<param-value>classpath:spring/applicationContext.xml</param-value>
	</context-param>
	<listener>
		<listener-class>org.springframework.web.context.ContextLoaderListener</listener-class>
	</listener>

	<!-- 解决post乱码 -->
	<filter>
		<filter-name>CharacterEncodingFilter</filter-name>
		<filter-class>org.springframework.web.filter.CharacterEncodingFilter</filter-class>
		<init-param>
			<param-name>encoding</param-name>
			<param-value>utf-8</param-value>
		</init-param>
	</filter>
	<filter-mapping>
		<filter-name>CharacterEncodingFilter</filter-name>
		<url-pattern>/*</url-pattern>
	</filter-mapping>

	<!-- 配置springmvc的前端控制器 -->
	<servlet>
		<servlet-name>data-report</servlet-name>
		<servlet-class>org.springframework.web.servlet.DispatcherServlet</servlet-class>
		<init-param>
			<param-name>contextConfigLocation</param-name>
			<param-value>classpath:spring/springmvc.xml</param-value>
		</init-param>
		<load-on-startup>1</load-on-startup>
	</servlet>
	
	<!-- 拦截所有请求 jsp除外 -->
	<servlet-mapping>
		<servlet-name>data-report</servlet-name>
		<url-pattern>/</url-pattern>
	</servlet-mapping>

	<!-- 全局错误页面 -->
	<error-page>
		<error-code>404</error-code>
		<location>/WEB-INF/jsp/404.jsp</location>
	</error-page>
  
</web-app>

 SqlMapConfig.xml

<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE configuration PUBLIC "-//mybatis.org//DTD Config 3.0//EN" "http://mybatis.org/dtd/mybatis-3-config.dtd">
<configuration>
</configuration>

2、实现报表功能展示

（1）创建持久化类

TAvgpvNum.java

package cn.itcast.pojo;

import java.math.BigDecimal;

public class TAvgpvNum {
	private String datestr;// 日期
	private BigDecimal avgpvnum;// 平均PV数量

	public String getDatestr() {
		return datestr;
	}

	public void setDatestr(String datestr) {
		this.datestr = datestr == null ? null : datestr.trim();
	}

	public BigDecimal getAvgpvnum() {
		return avgpvnum;
	}

	public void setAvgpvnum(BigDecimal avgpvnum) {
		this.avgpvnum = avgpvnum;
	}
}

AvgToPageBean.java

package cn.itcast.pojo;

public class AvgToPageBean {

    private String[] dates;

    private double[] data;

    public String[] getDates() {
        return dates;
    }

    public void setDates(String[] dates) {
        this.dates = dates;
    }

    public double[] getData() {
        return data;
    }

    public void setData(double[] data) {
        this.data = data;
    }

}

（2）实现DAO层

TAvgpvNumMapper.java

package cn.itcast.mapper;

import java.util.List;

import cn.itcast.pojo.TAvgpvNum;

public interface TAvgpvNumMapper {

	public List<TAvgpvNum> selectByDate(String startDate, String endDate);

}

TAvgpvNumMapper.xml

<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE mapper PUBLIC "-//mybatis.org//DTD Mapper 3.0//EN" "http://mybatis.org/dtd/mybatis-3-mapper.dtd" >
<mapper namespace="cn.itcast.mapper.TAvgpvNumMapper">

	<select id="selectByDate" resultType="cn.itcast.pojo.TAvgpvNum"
		parameterType="String">
		select *
		from t_avgpv_num
		where dateStr between #{0} and #{1} order by dateStr asc;
	</select>

</mapper>

（3）实现Service层

AvgPvService.java

package cn.itcast.service;

public interface AvgPvService {

    //根据日期查询数据
	public String getAvgPvNumByDates(String startDate, String endDate);

}

AvgPvServiceImpl.java

package cn.itcast.service.impl;

import java.util.List;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;

import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.ObjectMapper;

import cn.itcast.mapper.TAvgpvNumMapper;
import cn.itcast.pojo.AvgToPageBean;
import cn.itcast.pojo.TAvgpvNum;
import cn.itcast.service.AvgPvService;

@Service
public class AvgPvServiceImpl implements AvgPvService {

	@Autowired
	private TAvgpvNumMapper mapper;

	@Override
	public String getAvgPvNumByDates(String startDate, String endDate) {
		
		//调用查询方法
		List<TAvgpvNum> lists = mapper.selectByDate(startDate, endDate);
		// 数组大小
		int size = 7;
		//保存日期数据
		String[] dates = new String[size];
		//保存人均浏览页面数据
		double[] datas = new double[size];
		int i = 0;
		for (TAvgpvNum tAvgpvNum : lists) {
			dates[i] = tAvgpvNum.getDatestr();
			datas[i] = tAvgpvNum.getAvgpvnum().doubleValue();
			i++;
		}
		//定义AvgToPageBean对象，用于前台页面展示
		AvgToPageBean bean = new AvgToPageBean();
		bean.setDates(dates);
		bean.setData(datas);
		//Jackson提供的类，用于把对象转换成Json字符串
		ObjectMapper om = new ObjectMapper();
		String beanJson = null;
		try {
			beanJson = om.writeValueAsString(bean);
		} catch (JsonProcessingException e) {
			e.printStackTrace();
		}
		//返回Json格式的对象
		return beanJson;
	}

}

（4）实现Controller层

IndexController.java

package cn.itcast.controller;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.ResponseBody;

import cn.itcast.service.AvgPvService;


@Controller
public class IndexController {

	@Autowired
	private AvgPvService pvService;

	@RequestMapping("/index")
	public String showIndex() {
		return "index";
	}

	@RequestMapping(value = "/avgPvNum", produces = "application/json;charset=UTF-8")
	@ResponseBody
	public String getChart() {
		System.out.println("获取平均pv数据..");
		String data = pvService.getAvgPvNumByDates("2013-09-18", "2013-09-24");
		return data;
	}

}

（5）实现页面功能

index.jsp

<%@ page language="java" contentType="text/html; charset=UTF-8"
	pageEncoding="UTF-8"%>
<!DOCTYPE html>
<html lang="en">
<head>
<!-- BEGIN META -->
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="description" content="">
<meta name="author" content="Olive Enterprise">
<!-- END META -->
<script src="/js/jquery-1.11.3.min.js"></script>
<script src="/js/echarts.min.js"></script>
<script src="/js/china.js"></script>

<!-- BEGIN SHORTCUT ICON -->
<link rel="shortcut icon" href="/img/favicon.ico">
<!-- END SHORTCUT ICON -->
<title>流量运营分析 - pinyougou®itcast全站数据平台</title>
<!-- BEGIN STYLESHEET-->
<link href="/css/bootstrap.min.css" rel="stylesheet">
<!-- BOOTSTRAP CSS -->
<link href="/css/bootstrap-reset.css" rel="stylesheet">
<!-- BOOTSTRAP CSS -->
<link href="/assets/font-awesome/css/font-awesome.css" rel="stylesheet">
<!-- FONT AWESOME ICON CSS -->
<link href="/css/style.css" rel="stylesheet">
<!-- THEME BASIC CSS -->
<link href="/css/style-responsive.css" rel="stylesheet">
<!-- THEME RESPONSIVE CSS -->
<link href="/assets/morris.js-0.4.3/morris.css" rel="stylesheet">
<!-- MORRIS CHART CSS -->
<!--dashboard calendar-->
<link href="/css/clndr.css" rel="stylesheet">
<!-- CALENDER CSS -->
<!--[if lt IE 9]>
		<script src="js/html5shiv.js">
		</script>
		<script src="js/respond.min.js">
		</script>
		<![endif]-->
<!-- END STYLESHEET-->

</head>
<body>
	<!-- BEGIN SECTION -->
	<section id="container">
		<!-- BEGIN HEADER -->
		<header class="header white-bg">
			<!-- SIDEBAR TOGGLE BUTTON -->
			<div class="sidebar-toggle-box">
				<div data-placement="right" class="fa fa-bars tooltips"></div>
			</div>
			<!-- SIDEBAR TOGGLE BUTTON  END-->
			<a href="index.html" class="logo"> ITCAST <span> PINYOUGOU
			</span><span> 全站流量运营分析 </span>
			</a>
			<!-- START HEADER  NAV -->

			<nav class="nav notify-row" id="top_menu">

				<ul class="nav top-menu">
					<!-- START NOTIFY TASK BAR -->

					<li class="dropdown"><a data-toggle="dropdown"
						class="dropdown-toggle" href="#"> <i class="fa fa-tasks">
						</i> <span class="badge bg-success"> 6 </span>
					</a>

						<ul class="dropdown-menu extended tasks-bar">
							<li class="notify-arrow notify-arrow-blue"></li>
							<li>
								<p class="blue">You have 6 pending tasks</p>
							</li>
							<li><a href="#">
									<div class="task-info">
										<div class="desc">Dashboard v1.3</div>
										<div class="percent">40%</div>
									</div>
									<div class="progress progress-striped">
										<div class="progress-bar progress-bar-success set-40"
											role="progressbar" aria-valuenow="40" aria-valuemin="0"
											aria-valuemax="100">
											<span class="sr-only"> 40% Complete (success) </span>
										</div>
									</div>
							</a></li>
							<li><a href="#">
									<div class="task-info">
										<div class="desc">Database Update</div>
										<div class="percent">60%</div>
									</div>
									<div class="progress progress-striped">
										<div class="progress-bar progress-bar-warning set-60"
											role="progressbar" aria-valuenow="60" aria-valuemin="0"
											aria-valuemax="100">
											<span class="sr-only"> 60% Complete (warning) </span>
										</div>
									</div>
							</a></li>
							<li><a href="#">
									<div class="task-info">
										<div class="desc">Iphone Development</div>
										<div class="percent">87%</div>
									</div>
									<div class="progress progress-striped">
										<div class="progress-bar progress-bar-info set-87"
											role="progressbar" aria-valuenow="20" aria-valuemin="0"
											aria-valuemax="100">
											<span class="sr-only"> 87% Complete </span>
										</div>
									</div>
							</a></li>
							<li><a href="#">
									<div class="task-info">
										<div class="desc">Mobile App</div>
										<div class="percent">33%</div>
									</div>
									<div class="progress progress-striped">
										<div class="progress-bar progress-bar-danger set-33"
											role="progressbar" aria-valuenow="80" aria-valuemin="0"
											aria-valuemax="100">
											<span class="sr-only"> 33% Complete (danger) </span>
										</div>
									</div>
							</a></li>
							<li><a href="#">
									<div class="task-info">
										<div class="desc">Dashboard v1.3</div>
										<div class="percent">45%</div>
									</div>
									<div class="progress progress-striped active">
										<div class="progress-bar set-45" role="progressbar"
											aria-valuenow="45" aria-valuemin="0" aria-valuemax="100">
											<span class="sr-only"> 45% Complete </span>
										</div>

									</div>
							</a></li>
							<li class="external"><a href="#"> See All Tasks </a></li>
						</ul></li>
					<!-- END NOTIFY TASK BAR -->

					<!-- START NOTIFY INBOX BAR -->

					<li id="header_inbox_bar" class="dropdown"><a
						data-toggle="dropdown" class="dropdown-toggle" href="#"> <i
							class="fa fa-envelope-o"> </i> <span class="badge bg-important">
								5 </span>
					</a>
						<ul class="dropdown-menu extended inbox">
							<li class="notify-arrow notify-arrow-blue"></li>
							<li>
								<p class="blue">You have 5 new messages</p>
							</li>
							<li><a href="#"> <span class="photo"> <img
										alt="avatar" src="./img/avatar-mini.jpg">
								</span> <span class="subject"> <span class="from">
											Chintan Pandya </span> <span class="time"> Just now </span>
								</span> <span class="message"> Hello, this is an example msg. </span>
							</a></li>
							<li><a href="#"> <span class="photo"> <img
										alt="avatar" src="./img/avatar-mini2.jpg">
								</span> <span class="subject"> <span class="from"> Parth
											Jani </span> <span class="time"> 10 mins </span>
								</span> <span class="message"> Hi, Bro how are you ? </span>
							</a></li>
							<li><a href="#"> <span class="photo"> <img
										alt="avatar" src="./img/avatar-mini3.jpg">
								</span> <span class="subject"> <span class="from"> Jay
											Bardolia </span> <span class="time"> 3 hrs </span>
								</span> <span class="message"> This is awesome dashboard. </span>
							</a></li>
							<li><a href="#"> <span class="photo"> <img
										alt="avatar" src="./img/avatar-mini4.jpg">
								</span> <span class="subject"> <span class="from"> ADMIN
											BLACK </span> <span class="time"> Just now </span>
								</span> <span class="message"> Hello, this is metrolab </span>
							</a></li>
							<li><a href="#"> See all messages </a></li>
						</ul></li>
					<!-- END NOTIFY INBOX BAR -->

					<!-- START NOTIFY NOTIFICATION BAR -->

					<li id="header_notification_bar" class="dropdown"><a
						data-toggle="dropdown" class="dropdown-toggle" href="#"> <i
							class="fa fa-bell-o"> </i> <span class="badge bg-warning">
								7 </span>
					</a>
						<ul class="dropdown-menu extended notification">
							<li class="notify-arrow notify-arrow-blue"></li>
							<li>
								<p class="blue">You have 7 new notifications</p>
							</li>
							<li><a href="#"> <span class="label label-danger">
										<i class="fa fa-bolt"> </i>
								</span> Server #3 overloaded. <span class="small italic"> 34
										mins </span>
							</a></li>
							<li><a href="#"> <span class="label label-warning">
										<i class="fa fa-bell"> </i>
								</span> Server #10 not respoding. <span class="small italic"> 1
										Hours </span>
							</a></li>
							<li><a href="#"> <span class="label label-danger">
										<i class="fa fa-bolt"> </i>
								</span> Database overloaded 24%. <span class="small italic"> 4
										hrs </span>
							</a></li>
							<li><a href="#"> <span class="label label-success">
										<i class="fa fa-plus"> </i>
								</span> New user registered. <span class="small italic"> Just
										now </span>
							</a></li>
							<li><a href="#"> <span class="label label-primary">
										<i class="fa fa-bullhorn"> </i>
								</span> Application error. <span class="small italic"> 10 mins </span>
							</a></li>
							<li><a href="#"> See all notifications </a></li>
						</ul></li>
					<!-- END NOTIFY NOTIFICATION BAR -->

				</ul>


			</nav>
			<!-- END HEADER NAV -->

			<!-- START USER LOGIN DROPDOWN  -->

			<div class="top-nav ">
				<ul class="nav pull-right top-menu">
					<li><input type="text" class="form-control search"
						placeholder="Search"></li>
					<li class="dropdown"><a data-toggle="dropdown"
						class="dropdown-toggle" href="#"> <img alt=""
							src="img/avatar1_small.jpg"> <span class="username">
								ADMIN BLACK </span> <b class="caret"> </b>
					</a>
						<ul class="dropdown-menu extended logout">
							<li class="log-arrow-up"></li>
							<li><a href="#"> <i class=" fa fa-suitcase"> </i>
									Profile
							</a></li>
							<li><a href="#"> <i class="fa fa-cog"> </i> Settings
							</a></li>
							<li><a href="#"> <i class="fa fa-bell-o"> </i>
									Notification
							</a></li>
							<li><a href="login.html"> <i class="fa fa-key"> </i> Log
									Out
							</a></li>
						</ul></li>
				</ul>
			</div>
			<!-- END USER LOGIN DROPDOWN  -->
		</header>
		<!-- END HEADER -->
		<!-- BEGIN SIDEBAR -->
		<aside>
			<div id="sidebar" class="nav-collapse">
				<ul class="sidebar-menu" id="nav-accordion">
					<li><a href="index.html" class="active"> <i
							class="fa fa-dashboard"> </i> <span> 流量概况 </span>
					</a></li>


					<li class="sub-menu"><a href="javascript:;"> <i
							class="fa fa-book"> </i> <span> 流量分析 </span>
					</a>
						<ul class="sub">
							<li><a href="flow-trend.html"> 趋势分析 </a></li>
							<li><a href="flow-contract.html"> 对比分析 </a></li>
							<li><a href="flow-online.html"> 当前在线 </a></li>
							<li><a href="flow-detail.html"> 访问明细 </a></li>
						</ul></li>

					<li class="sub-menu"><a href="javascript:;"> <i
							class="fa fa-th"> </i> <span> 来源分析 </span>
					</a>
						<ul class="sub">
							<li><a href="#"> 来源分类 </a></li>
							<li><a href="#"> 搜索引擎 </a></li>
							<li><a href="#"> 搜索词 </a></li>
							<li><a href="#"> 来路域名 </a></li>
							<li><a href="#"> 来路页面 </a></li>
						</ul></li>

					<li class="sub-menu"><a href="javascript:;"> <i
							class="fa fa-book"> </i> <span> 受访分析 </span>
					</a>
						<ul class="sub">
							<li><a href="#"> 受访分析 </a></li>
							<li><a href="#"> 受访域名 </a></li>
							<li><a href="#"> 受访页面 </a></li>
							<li><a href="#"> 受访升降榜 </a></li>
							<li><a href="#"> 热点图 </a></li>
						</ul></li>

					<li class="sub-menu"><a href="javascript:;"> <i
							class="fa fa-th"> </i> <span> 访客分析 </span>
					</a>
						<ul class="sub">
							<li><a href="basic_table.html"> 地区/运营商 </a></li>
							<li><a href="responsive_table.html">终端详情 </a></li>
							<li><a href="dynamic_table.html"> 新老访客</a></li>
							<li><a href="advanced_table.html"> 忠诚度 </a></li>
							<li><a href="editable_table.html"> 活跃度 </a></li>
						</ul></li>
				</ul>
			</div>
		</aside>
		<!-- END SIDEBAR -->


		<!-- BEGIN MAIN CONTENT -->
		<section id="main-content">
			<!-- BEGIN WRAPPER  -->
			<section class="wrapper">

				<!-- 流量概况表 开始 -->
				<div class="row">
					<div class="col-lg-12 col-sm-12">
						<section class="panel">
							<div class="panel-body">
								<!-- 为ECharts准备一个具备大小（宽高）的Dom -->
								<div>
									<span><h3>流量信息概况</h3></span>
								</div>
								<div id="trendtable" style="width: 90%;">
									<table id='pv_table' class="table table-bordered" margin=auto>
										<thead style='font-weight: bold'>
											<tr>
												<td>日期</td>
												<td>浏览次数（PV）</td>
												<td>独立访客(UV)</td>
												<td>IP</td>
												<td>新独立访客</td>
												<td>访问次数</td>
											</tr>
										</thead>
										<tbody>
											<tr>
												<td>1001</td>
												<td>4702</td>
												<td>3096</td>
												<td>2880</td>
												<td>2506</td>
												<td>3773</td>
											</tr>
											<tr>
												<td>1002</td>
												<td>7528</td>
												<td>4860</td>
												<td>4435</td>
												<td>4209</td>
												<td>5975</td>
											</tr>
											<tr>
												<td>1003</td>
												<td>7286</td>
												<td>4741</td>
												<td>4409</td>
												<td>4026</td>
												<td>5817</td>
											</tr>
										</tbody>
									</table>
								</div>
							</div>
						</section>
					</div>
				</div>
				<!-- 流量概况表 结束-->


				<!-- BEGIN ROW  最近7天日平均PV量-->
				<div class="row">
					<div class="col-lg-6 col-sm-6">
						<section class="panel">
							<div class="panel-body">
								<div id="main1" style="width: 100%; height: 400px;"></div>
								<script type="text/javascript">
									$(document)
											.ready(
													function() {
														var myChart = echarts
																.init(document
																		.getElementById('main1'));
														// 显示标题，图例和空的坐标轴
														myChart
																.setOption({
																	title : {
																		text : '最近7天日平均PV量',
																		subtext : '动态数据'
																	},
																	tooltip : {},
																	legend : {
																		data : [ '日平均PV量' ]
																	},
																	xAxis : {
																		data : []
																	},
																	yAxis : {},
																	series : [ {
																		name : '日平均PV量',
																		type : 'bar',
																		data : []
																	} ]
																});
														//loading 动画
														myChart.showLoading();
														// 异步加载数据
														$.get('http://localhost:8080/avgPvNum').done(function(data) {
															//填入数据
															myChart.setOption({
																xAxis : {
																							data : data.dates
																						},
																						series : [ {
																							// 根据名字对应到相应的系列
																							name : 'PV量',
																							data : data.data
																						} ]
																					});
																			//数据加载完成后再调用 hideLoading 方法隐藏加载动画
																			myChart.hideLoading();
																		});
													});
								</script>

							</div>
						</section>
					</div>
					<div class="col-lg-6 col-sm-6">
						<section class="panel">
							<div class="panel-body">

								<div id="main2" style="width: 600px; height: 400px;"
									align="center"></div>
								<script type="text/javascript">
									// 基于准备好的dom，初始化echarts实例
									var myChart = echarts.init(document
											.getElementById('main2'));

									myChart
											.setOption({
												series : [ {
													name : '访问来源',
													type : 'pie',
													radius : '55%',
													data : [ {
														value : 235,
														name : '视频广告'
													}, {
														value : 274,
														name : '联盟广告'
													}, {
														value : 310,
														name : '邮件营销'
													}, {
														value : 335,
														name : '直接访问'
													}, {
														value : 400,
														name : '搜索引擎'
													} ]
												} ],
												itemStyle : {
													normal : {
														// 阴影的大小
														shadowBlur : 200,
														// 阴影水平方向上的偏移
														shadowOffsetX : 0,
														// 阴影垂直方向上的偏移
														shadowOffsetY : 0,
														// 阴影颜色
														shadowColor : 'rgba(0, 0, 0, 0.5)'
													}
												}
											})
								</script>
							</div>
						</section>
					</div>

				</div>
				<!-- END ROW  -->

				<!-- 折线图 row 开始 -->
				<div class="row">
					<div class="col-lg-12 col-sm-12">
						<section class="panel">
							<div class="panel-body">
								<div id="main3" style="width: 100%; height: 500px;"></div>
								<script type="text/javascript">
									// 基于准备好的dom，初始化echarts实例
									var myChart3 = echarts.init(document
											.getElementById('main3'));
									option = {
										title : {
											text : '访客来源地区分布图',
											subtext : '模拟数据',
											x : 'center'
										},
										tooltip : {
											trigger : 'item'
										},
										legend : {
											orient : 'vertical',
											x : 'left',
											data : [ '访客UV' ]
										},
										dataRange : {
											min : 0,
											max : 2500,
											x : 'left',
											y : 'bottom',
											text : [ '高', '低' ], // 文本，默认为数值文本
											calculable : true
										},
										toolbox : {
											show : true,
											orient : 'vertical',
											x : 'right',
											y : 'center',
											feature : {
												mark : {
													show : true
												},
												dataView : {
													show : true,
													readOnly : false
												},
												restore : {
													show : true
												},
												saveAsImage : {
													show : true
												}
											}
										},
										roamController : {
											show : true,
											x : 'right',
											mapTypeControl : {
												'china' : true
											}
										},
										series : [
												{
													name : '访客UV',
													type : 'map',
													mapType : 'china',
													roam : false,
													itemStyle : {
														normal : {
															label : {
																show : true
															}
														},
														emphasis : {
															label : {
																show : true
															}
														}
													},
													data : [
															{
																name : '北京',
																value : Math
																		.round(Math
																				.random() * 1000)
															},
															{
																name : '天津',
																value : Math
																		.round(Math
																				.random() * 1000)
															},
															{
																name : '上海',
																value : Math
																		.round(Math
																				.random() * 1000)
															},
															{
																name : '重庆',
																value : Math
																		.round(Math
																				.random() * 1000)
															},
															{
																name : '河北',
																value : Math
																		.round(Math
																				.random() * 1000)
															},
															{
																name : '河南',
																value : Math
																		.round(Math
																				.random() * 1000)
															},
															{
																name : '云南',
																value : Math
																		.round(Math
																				.random() * 1000)
															},
															{
																name : '辽宁',
																value : Math
																		.round(Math
																				.random() * 1000)
															},
															{
																name : '黑龙江',
																value : Math
																		.round(Math
																				.random() * 1000)
															},
															{
																name : '湖南',
																value : Math
																		.round(Math
																				.random() * 1000)
															},
															{
																name : '安徽',
																value : Math
																		.round(Math
																				.random() * 1000)
															},
															{
																name : '山东',
																value : Math
																		.round(Math
																				.random() * 1000)
															},
															{
																name : '新疆',
																value : Math
																		.round(Math
																				.random() * 1000)
															},
															{
																name : '江苏',
																value : Math
																		.round(Math
																				.random() * 1000)
															},
															{
																name : '浙江',
																value : Math
																		.round(Math
																				.random() * 1000)
															},
															{
																name : '江西',
																value : Math
																		.round(Math
																				.random() * 1000)
															},
															{
																name : '湖北',
																value : Math
																		.round(Math
																				.random() * 1000)
															},
															{
																name : '广西',
																value : Math
																		.round(Math
																				.random() * 1000)
															},
															{
																name : '甘肃',
																value : Math
																		.round(Math
																				.random() * 1000)
															},
															{
																name : '山西',
																value : Math
																		.round(Math
																				.random() * 1000)
															},
															{
																name : '内蒙古',
																value : Math
																		.round(Math
																				.random() * 1000)
															},
															{
																name : '陕西',
																value : Math
																		.round(Math
																				.random() * 1000)
															},
															{
																name : '吉林',
																value : Math
																		.round(Math
																				.random() * 1000)
															},
															{
																name : '福建',
																value : Math
																		.round(Math
																				.random() * 1000)
															},
															{
																name : '贵州',
																value : Math
																		.round(Math
																				.random() * 1000)
															},
															{
																name : '广东',
																value : Math
																		.round(Math
																				.random() * 1000)
															},
															{
																name : '青海',
																value : Math
																		.round(Math
																				.random() * 1000)
															},
															{
																name : '西藏',
																value : Math
																		.round(Math
																				.random() * 1000)
															},
															{
																name : '四川',
																value : Math
																		.round(Math
																				.random() * 1000)
															},
															{
																name : '宁夏',
																value : Math
																		.round(Math
																				.random() * 1000)
															},
															{
																name : '海南',
																value : Math
																		.round(Math
																				.random() * 1000)
															},
															{
																name : '台湾',
																value : Math
																		.round(Math
																				.random() * 1000)
															},
															{
																name : '香港',
																value : Math
																		.round(Math
																				.random() * 1000)
															},
															{
																name : '澳门',
																value : Math
																		.round(Math
																				.random() * 1000)
															} ]
												},

										]
									};

									myChart3.setOption(option);
								</script>
							</div>
						</section>
					</div>
				</div>


				<!-- 折线图 row 结束 -->
				<!-- 动态加载数据示例 开始-->
				<div class="row">
					<div class="col-lg-12 col-sm-12">
						<section class="panel">
							<div class="panel-body">
								<div id="main4" style="width: 100%; height: 400px;"></div>
								<script type="text/javascript">
									// 基于准备好的dom，初始化echarts实例
									var myChart4 = echarts.init(document
											.getElementById('main4'));

									// 指定图表的配置项和数据
									option = {
										title : {
											text : '近一周访客数量变化趋势',
											subtext : '动态数据'
										},
										tooltip : {
											trigger : 'axis'
										},
										legend : {
											data : [ '独立访客', '新独立访客' ]
										},
										toolbox : {
											show : true,
											feature : {
												mark : {
													show : true
												},
												dataView : {
													show : true,
													readOnly : false
												},
												magicType : {
													show : true,
													type : [ 'line', 'bar' ]
												},
												restore : {
													show : true
												},
												saveAsImage : {
													show : true
												}
											}
										},
										calculable : true,
										xAxis : [ {
											type : 'category',
											boundaryGap : false,
											data : []
										} ],
										yAxis : [ {
											type : 'value',
											axisLabel : {
												formatter : '{value} 人'
											}
										} ],
										series : [ {
											name : '独立访客',
											type : 'line',
											data : [],
											markPoint : {
												data : [ {
													type : 'max',
													name : '最大值'
												}, {
													type : 'min',
													name : '最小值'
												} ]
											},
											markLine : {
												data : [ {
													type : 'average',
													name : '平均值'
												} ]
											}
										}, {
											name : '新独立访客',
											type : 'line',
											data : [],
											markPoint : {
												data : [ {
													type : 'max',
													name : '最大值'
												}, {
													type : 'min',
													name : '最小值'
												} ]
											},
											markLine : {
												data : [ {
													type : 'average',
													name : '平均值'
												} ]
											}
										} ]
									};

									// 使用刚指定的配置项和数据显示图表。
									myChart4.setOption(option);
									myChart4.showLoading();
									// 异步加载数据
									$.get('http://localhost:8080/flowNum')
											.done(function(data) {
												// 填入数据
												myChart4.setOption({
													xAxis : {
														data : data.dates
													},
													series : [ {
														name : '独立访客',
														data : data.uvs
													}, {
														name : '新独立访客',
														data : data.new_uvs
													} ]
												});
												myChart4.hideLoading();
											});
								</script>
							</div>
						</section>
					</div>
				</div>
				<!-- 动态加载数据示例 结束-->


				<!-- BEGIN ROW  -->
				<div class="row">
					<div class="col-lg-6">
						<div class="panel">
							<div class="panel-body">
								<footer class="project-category">
									<ul>
										<li class="active">
											<h5>Project 1</h5>
											<div id="work-progress6"></div>
										</li>
										<li>
											<h5>Project 2</h5>
											<div id="work-progress7"></div>
										</li>
										<li>
											<h5>Project 3</h5>
											<div id="work-progress8"></div>
										</li>
									</ul>
									<h1>Projects accomplished</h1>
								</footer>
								<!-- END  FOOTER -->
							</div>
						</div>
					</div>
					<div class="col-lg-6">
						<div class="panel">
							<div class="panel-body">
								<div class="bio-chart">
									<input class="knob" data-width="100" data-height="100"
										data-displayPrevious=true data-thickness=".2" value="78"
										data-fgColor="#f9a3a3" data-bgColor="#e8e8e8">
									<h4 class="red">Profit</h4>
								</div>
								<div class="bio-chart">
									<input class="knob" data-width="100" data-height="100"
										data-displayPrevious=true data-thickness=".2" value="63"
										data-fgColor="#fcce54" data-bgColor="#e8e8e8">
									<h4 class="yellow">Expansion</h4>
								</div>
							</div>
						</div>
					</div>
				</div>
				<!-- END ROW  -->
			</section>
			<!-- END WRAPPER  -->
		</section>
		<!-- END MAIN CONTENT -->
		<!-- BEGIN FOOTER -->
		<footer class="site-footer">
			<div class="text-center">
				2017 &copy; PINYOUGOU by <a href="" target="_blank">
					www.itcast.cn </a> <a href="http://www.itcast.cn" class="go-top"> <i
					class="fa fa-angle-up"> </i>
				</a>
			</div>
		</footer>
		<!-- END  FOOTER -->
	</section>
	<!-- END SECTION -->
	<!-- BEGIN JS -->

	<!-- BASIC JQUERY 1.8.3 LIB. JS -->
	<script src="js/bootstrap.min.js"></script>
	<!-- BOOTSTRAP JS -->
	<script src="js/jquery.dcjqaccordion.2.7.js"></script>
	<!-- ACCORDIN JS -->
	<script src="js/jquery.scrollTo.min.js"></script>
	<!-- SCROLLTO JS -->
	<script src="js/jquery.nicescroll.js"></script>
	<!-- NICESCROLL JS -->
	<script src="js/respond.min.js"></script>
	<!-- RESPOND JS -->
	<script src="js/jquery.sparkline.js"></script>
	<!-- SPARKLINE JS -->
	<script src="js/sparkline-chart.js"></script>
	<!-- SPARKLINE CHART JS -->
	<script src="js/common-scripts.js"></script>
	<!-- BASIC COMMON JS -->
	<script src="js/count.js"></script>
	<!-- COUNT JS -->
	<!--Morris-->
	<script src="assets/morris.js-0.4.3/morris.min.js"></script>
	<!-- MORRIS JS -->
	<script src="assets/morris.js-0.4.3/raphael-min.js"></script>
	<!-- MORRIS  JS -->
	<script src="js/chart.js"></script>
	<!-- CHART JS -->
	<!--Calendar-->
	<script src="js/calendar/clndr.js"></script>
	<!-- CALENDER JS -->
	<script src="js/calendar/evnt.calendar.init.js"></script>
	<!-- CALENDER EVENT JS -->
	<script src="js/calendar/moment-2.2.1.js"></script>
	<!-- CALENDER MOMENT JS -->
	<!-- <script src="http://cdnjs.cloudflare.com/ajax/libs/underscore.js/1.5.2/underscore-min.js"></script> -->
	<!-- UNDERSCORE JS -->
	<script src="assets/jquery-knob/js/jquery.knob.js"></script>
	<!-- JQUERY KNOB JS -->
	<script>
		//knob
		$(".knob").knob();
	</script>

	<!-- END JS -->
</body>
</html>

3、系统功能模块展示

参考书籍：

《Hadoop大数据技术原理与应用》

 文章来源地址https://www.toymoban.com/news/detail-404965.html

到了这里，关于大数据技术之Hadoop(十一)——网站流量日志数据分析系统的文章就介绍完了。如果您还想了解更多内容，请在右上角搜索TOY模板网以前的文章或继续浏览下面的相关文章，希望大家以后多多支持TOY模板网！