物流实时数仓DWD层——1.准备工作

这篇具有很好参考价值的文章主要介绍了物流实时数仓DWD层——1.准备工作。希望对大家有所帮助。如果存在错误或未考虑完全的地方,请大家不吝赐教,您也可以点击"举报违法"按钮提交疑问。

目录

1.创建主程序——DwdOrderRelevantApp类

2.创建DWD层的事实表——来源于订单表和订单明细表

(1)创建订单表实体类

(2)创建订单明细表实体类

(3)创建交易域:下单事务事实表实体类,并整合(1)与(2),采用下单时间

(4)创建交易域:支付成功事务事实表实体类,并整合(1)与(2),采用支付时间和支付状态

(5)创建物流域:揽收(接单)事务事实表实体类,并整合(1)与(2),采用揽收时间

(6)创建物流域:发单事务事实表实体类,并整合(1)与(2),采用发货时间

(7)创建物流域:转运完成事务事实表实体类,并整合(1)与(2),采用转运完成时间

(8)创建物流域:派送成功事务事实表实体类,并整合(1)与(2),采用派送成功时间

(9)创建物流域:签收事务事实表实体类,并整合(1)与(2),采用签收时间

(10)创建交易域:取消运单事务事实表实体类,并整合(1)与(2),采用取消时间文章来源地址https://www.toymoban.com/news/detail-790276.html


1.创建主程序——DwdOrderRelevantApp类

package com.atguigu.tms.realtime.app.dwd;

/**
 * 订单相关事实表准备
 * 需要启动的进程
 *      zk、kafka、hdfs、OdsApp、DwdOrderRelevantApp
 **/

public class DwdOrderRelevantApp {

    public static void main(String[] args) throws Exception {
        // TODO 1.环境准备
        // 1.1 指定流处理环境以及检查点相关的设置
        // 1.2 设置并行度,和kafka主题的分区数保持一致
        
        // TODO 2.从kafka的tms_ods主题中读取
        // 2.1 声明消费的主题    
        // 2.2 创建消费者对象      
        // 2.3 消费数据  封装为流
        
        // TODO 3.筛选订单和订单明细数据
        
        // TODO 4.对流中的数据类型进行转换  jsonStr -> jsonObj
     
        // TODO 5.按照order_id进行分组
     
        // TODO 6.定义侧输出流标签:匿名内部类
        //  主流:下单
        //  侧输出流:支付成功、取消订单、揽收(接单)、发单、转运完成、派送成功、签署

        // TODO 7.分流
        
        // TODO 8.从主流中提取侧输出流
        
        // TODO 9.将不同流的数据写到kafka的不同主题中
     }
}

2.创建DWD层的事实表——来源于订单表和订单明细表

(1)创建订单表实体类
package com.atguigu.tms.realtime.beans;

import lombok.Data;

import java.math.BigDecimal;

/**
 * 订单实体类
 */
@Data
public class DwdOrderInfoOriginBean {
    // 编号(主键)
    String id;

    // 运单号
    String orderNo;

    // 运单状态
    String status;

    // 取件类型,1为网点自寄,2为上门取件
    String collectType;

    // 客户id
    String userId;

    // 收件人小区id
    String receiverComplexId;

    // 收件人省份id
    String receiverProvinceId;

    // 收件人城市id
    String receiverCityId;

    // 收件人区县id
    String receiverDistrictId;

    // 收件人姓名
    String receiverName;

    // 发件人小区id
    String senderComplexId;

    // 发件人省份id
    String senderProvinceId;

    // 发件人城市id
    String senderCityId;

    // 发件人区县id
    String senderDistrictId;

    // 发件人姓名
    String senderName;

    // 支付方式
    String paymentType;

    // 货物个数
    Integer cargoNum;

    // 金额
    BigDecimal amount;

    // 预计到达时间
    Long estimateArriveTime;

    // 距离,单位:公里
    BigDecimal distance;

    // 创建时间
    String createTime;

    // 更新时间
    String updateTime;

    // 是否删除
    String isDeleted;
}
(2)创建订单明细表实体类
package com.atguigu.tms.realtime.beans;

import lombok.Data;

import java.math.BigDecimal;

/**
 *订单货物明细实体类
 */
@Data
public class DwdOrderDetailOriginBean {
    // 编号(主键)
    String id;

    // 运单id
    String orderId;

    // 货物类型
    String cargoType;

    // 长cm
    Integer volumnLength;

    // 宽cm
    Integer volumnWidth;

    // 高cm
    Integer volumnHeight;

    // 重量 kg
    BigDecimal weight;

    // 创建时间
    String createTime;

    // 更新时间
    String updateTime;

    // 是否删除
    String isDeleted;
}
(3)创建交易域:下单事务事实表实体类,并整合(1)与(2),采用下单时间
package com.atguigu.tms.realtime.beans;

import com.atguigu.tms.realtime.utils.DateFormatUtil;
import lombok.Data;

import java.math.BigDecimal;

/**
 *交易域:下单事务事实表实体类
 */
@Data
public class DwdTradeOrderDetailBean {
    // 运单明细ID
    String id;

    // 运单id
    String orderId;

    // 货物类型
    String cargoType;

    // 长cm
    Integer volumeLength;

    // 宽cm
    Integer volumeWidth;

    // 高cm
    Integer volumeHeight;

    // 重量 kg
    BigDecimal weight;

    // 下单时间
    String orderTime;

    // 运单号
    String orderNo;

    // 运单状态
    String status;

    // 取件类型,1为网点自寄,2为上门取件
    String collectType;

    // 客户id
    String userId;

    // 收件人小区id
    String receiverComplexId;

    // 收件人省份id
    String receiverProvinceId;

    // 收件人城市id
    String receiverCityId;

    // 收件人区县id
    String receiverDistrictId;

    // 收件人姓名
    String receiverName;

    // 发件人小区id
    String senderComplexId;

    // 发件人省份id
    String senderProvinceId;

    // 发件人城市id
    String senderCityId;

    // 发件人区县id
    String senderDistrictId;

    // 发件人姓名
    String senderName;

    // 支付方式
    String paymentType;

    // 货物个数
    Integer cargoNum;

    // 金额
    BigDecimal amount;

    // 预计到达时间
    String estimateArriveTime;

    // 距离,单位:公里
    BigDecimal distance;

    // 时间戳
    Long ts;

    public void mergeBean(DwdOrderDetailOriginBean detailOriginBean, DwdOrderInfoOriginBean infoOriginBean) {
        // 合并原始明细字段
        this.id = detailOriginBean.id;
        this.orderId = detailOriginBean.orderId;
        this.cargoType = detailOriginBean.cargoType;
        this.volumeLength = detailOriginBean.volumnLength;
        this.volumeWidth = detailOriginBean.volumnWidth;
        this.volumeHeight = detailOriginBean.volumnHeight;
        this.weight = detailOriginBean.weight;
        this.orderTime =
            DateFormatUtil.toYmdHms(DateFormatUtil.toTs(
                detailOriginBean.createTime.replaceAll("T", " ")
                    .replaceAll("Z", ""), true)
                + 8 * 60 * 60 * 1000);
        this.ts = DateFormatUtil.toTs(
            detailOriginBean.createTime.replaceAll("T", " ")
                .replaceAll("Z", ""), true)
            + 8 * 60 * 60 * 1000;

        // 合并原始订单字段
        this.orderNo = infoOriginBean.orderNo;
        this.status = infoOriginBean.status;
        this.collectType = infoOriginBean.collectType;
        this.userId = infoOriginBean.userId;
        this.receiverComplexId = infoOriginBean.receiverComplexId;
        this.receiverProvinceId = infoOriginBean.receiverProvinceId;
        this.receiverCityId = infoOriginBean.receiverCityId;
        this.receiverDistrictId = infoOriginBean.receiverDistrictId;
        this.receiverName = infoOriginBean.receiverName;
        this.senderComplexId = infoOriginBean.senderComplexId;
        this.senderProvinceId = infoOriginBean.senderProvinceId;
        this.senderCityId = infoOriginBean.senderCityId;
        this.senderDistrictId = infoOriginBean.senderDistrictId;
        this.senderName = infoOriginBean.senderName;
        this.paymentType = infoOriginBean.paymentType;
        this.cargoNum = infoOriginBean.cargoNum;
        this.amount = infoOriginBean.amount;
        this.estimateArriveTime = DateFormatUtil.toYmdHms(
            infoOriginBean.estimateArriveTime - 8 * 60 * 60 * 1000);
        this.distance = infoOriginBean.distance;
    }
}
(4)创建交易域:支付成功事务事实表实体类,并整合(1)与(2),采用支付时间和支付状态
package com.atguigu.tms.realtime.beans;

import com.atguigu.tms.realtime.utils.DateFormatUtil;
import lombok.Data;

import java.math.BigDecimal;

/**
 *交易域:支付成功事务事实表实体类
 */
@Data
public class DwdTradePaySucDetailBean {
    // 运单明细ID
    String id;

    // 运单id
    String orderId;

    // 货物类型
    String cargoType;

    // 长cm
    Integer volumeLength;

    // 宽cm
    Integer volumeWidth;

    // 高cm
    Integer volumeHeight;

    // 重量 kg
    BigDecimal weight;

    // 支付时间
    String payTime;

    // 运单号
    String orderNo;

    // 运单状态
    String status;

    // 取件类型,1为网点自寄,2为上门取件
    String collectType;

    // 客户id
    String userId;

    // 收件人小区id
    String receiverComplexId;

    // 收件人省份id
    String receiverProvinceId;

    // 收件人城市id
    String receiverCityId;

    // 收件人区县id
    String receiverDistrictId;

    // 收件人姓名
    String receiverName;

    // 发件人小区id
    String senderComplexId;

    // 发件人省份id
    String senderProvinceId;

    // 发件人城市id
    String senderCityId;

    // 发件人区县id
    String senderDistrictId;

    // 发件人姓名
    String senderName;

    // 支付方式
    String paymentType;

    // 货物个数
    Integer cargoNum;

    // 金额
    BigDecimal amount;

    // 预计到达时间
    String estimateArriveTime;

    // 距离,单位:公里
    BigDecimal distance;

    // 时间戳
    Long ts;

    public void mergeBean(DwdOrderDetailOriginBean detailOriginBean, DwdOrderInfoOriginBean infoOriginBean) {
        // 合并原始明细字段
        this.id = detailOriginBean.id;
        this.orderId = detailOriginBean.orderId;
        this.cargoType = detailOriginBean.cargoType;
        this.volumeLength = detailOriginBean.volumnLength;
        this.volumeWidth = detailOriginBean.volumnWidth;
        this.volumeHeight = detailOriginBean.volumnHeight;
        this.weight = detailOriginBean.weight;

        // 合并原始订单字段
        this.orderNo = infoOriginBean.orderNo;
        this.status = infoOriginBean.status;
        this.collectType = infoOriginBean.collectType;
        this.userId = infoOriginBean.userId;
        this.receiverComplexId = infoOriginBean.receiverComplexId;
        this.receiverProvinceId = infoOriginBean.receiverProvinceId;
        this.receiverCityId = infoOriginBean.receiverCityId;
        this.receiverDistrictId = infoOriginBean.receiverDistrictId;
        this.receiverName = infoOriginBean.receiverName;
        this.senderComplexId = infoOriginBean.senderComplexId;
        this.senderProvinceId = infoOriginBean.senderProvinceId;
        this.senderCityId = infoOriginBean.senderCityId;
        this.senderDistrictId = infoOriginBean.senderDistrictId;
        this.senderName = infoOriginBean.senderName;
        this.paymentType = infoOriginBean.paymentType;
        this.cargoNum = infoOriginBean.cargoNum;
        this.amount = infoOriginBean.amount;
        this.estimateArriveTime = DateFormatUtil.toYmdHms(
            infoOriginBean.estimateArriveTime - 8 * 60 * 60 * 1000);
        this.distance = infoOriginBean.distance;

        this.payTime =
            DateFormatUtil.toYmdHms(DateFormatUtil.toTs(
                infoOriginBean.updateTime.replaceAll("T", " ")
                    .replaceAll("Z", ""), true)
                + 8 * 60 * 60 * 1000);
        this.ts = DateFormatUtil.toTs(
            infoOriginBean.updateTime.replaceAll("T", " ")
                .replaceAll("Z", ""), true)
            + 8 * 60 * 60 * 1000;
    }
}
(5)创建物流域:揽收(接单)事务事实表实体类,并整合(1)与(2),采用揽收时间
package com.atguigu.tms.realtime.beans;

import com.atguigu.tms.realtime.utils.DateFormatUtil;
import lombok.Data;

import java.math.BigDecimal;

/**
 *物流域:揽收(接单)事务事实表实体类
 */
@Data
public class DwdTransReceiveDetailBean {
    // 运单明细ID
    String id;

    // 运单id
    String orderId;

    // 货物类型
    String cargoType;

    // 长cm
    Integer volumeLength;

    // 宽cm
    Integer volumeWidth;

    // 高cm
    Integer volumeHeight;

    // 重量 kg
    BigDecimal weight;

    // 揽收时间
    String receiveTime;

    // 运单号
    String orderNo;

    // 运单状态
    String status;

    // 取件类型,1为网点自寄,2为上门取件
    String collectType;

    // 客户id
    String userId;

    // 收件人小区id
    String receiverComplexId;

    // 收件人省份id
    String receiverProvinceId;

    // 收件人城市id
    String receiverCityId;

    // 收件人区县id
    String receiverDistrictId;

    // 收件人姓名
    String receiverName;

    // 发件人小区id
    String senderComplexId;

    // 发件人省份id
    String senderProvinceId;

    // 发件人城市id
    String senderCityId;

    // 发件人区县id
    String senderDistrictId;

    // 发件人姓名
    String senderName;

    // 支付方式
    String paymentType;

    // 货物个数
    Integer cargoNum;

    // 金额
    BigDecimal amount;

    // 预计到达时间
    String estimateArriveTime;

    // 距离,单位:公里
    BigDecimal distance;

    // 时间戳
    Long ts;

    public void mergeBean(DwdOrderDetailOriginBean detailOriginBean, DwdOrderInfoOriginBean infoOriginBean) {
        // 合并原始明细字段
        this.id = detailOriginBean.id;
        this.orderId = detailOriginBean.orderId;
        this.cargoType = detailOriginBean.cargoType;
        this.volumeLength = detailOriginBean.volumnLength;
        this.volumeWidth = detailOriginBean.volumnWidth;
        this.volumeHeight = detailOriginBean.volumnHeight;
        this.weight = detailOriginBean.weight;

        // 合并原始订单字段
        this.orderNo = infoOriginBean.orderNo;
        this.status = infoOriginBean.status;
        this.collectType = infoOriginBean.collectType;
        this.userId = infoOriginBean.userId;
        this.receiverComplexId = infoOriginBean.receiverComplexId;
        this.receiverProvinceId = infoOriginBean.receiverProvinceId;
        this.receiverCityId = infoOriginBean.receiverCityId;
        this.receiverDistrictId = infoOriginBean.receiverDistrictId;
        this.receiverName = infoOriginBean.receiverName;
        this.senderComplexId = infoOriginBean.senderComplexId;
        this.senderProvinceId = infoOriginBean.senderProvinceId;
        this.senderCityId = infoOriginBean.senderCityId;
        this.senderDistrictId = infoOriginBean.senderDistrictId;
        this.senderName = infoOriginBean.senderName;
        this.paymentType = infoOriginBean.paymentType;
        this.cargoNum = infoOriginBean.cargoNum;
        this.amount = infoOriginBean.amount;
        this.estimateArriveTime = DateFormatUtil.toYmdHms(
            infoOriginBean.estimateArriveTime - 8 * 60 * 60 * 1000);
        this.distance = infoOriginBean.distance;
        this.receiveTime =
            DateFormatUtil.toYmdHms(DateFormatUtil.toTs(
                infoOriginBean.updateTime.replaceAll("T", " ")
                    .replaceAll("Z", ""), true)
                + 8 * 60 * 60 * 1000);
        this.ts = DateFormatUtil.toTs(
            infoOriginBean.updateTime.replaceAll("T", " ")
                .replaceAll("Z", ""), true)
            + 8 * 60 * 60 * 1000;
    }
}
(6)创建物流域:发单事务事实表实体类,并整合(1)与(2),采用发货时间
package com.atguigu.tms.realtime.beans;

import com.atguigu.tms.realtime.utils.DateFormatUtil;
import lombok.Data;

import java.math.BigDecimal;

/**
 *物流域:发单事务事实表实体类
 */
@Data
public class DwdTransDispatchDetailBean {
    // 运单明细ID
    String id;

    // 运单id
    String orderId;

    // 货物类型
    String cargoType;

    // 长cm
    Integer volumeLength;

    // 宽cm
    Integer volumeWidth;

    // 高cm
    Integer volumeHeight;

    // 重量 kg
    BigDecimal weight;

    // 发单时间
    String dispatchTime;

    // 运单号
    String orderNo;

    // 运单状态
    String status;

    // 取件类型,1为网点自寄,2为上门取件
    String collectType;

    // 客户id
    String userId;

    // 收件人小区id
    String receiverComplexId;

    // 收件人省份id
    String receiverProvinceId;

    // 收件人城市id
    String receiverCityId;

    // 收件人区县id
    String receiverDistrictId;

    // 收件人姓名
    String receiverName;

    // 发件人小区id
    String senderComplexId;

    // 发件人省份id
    String senderProvinceId;

    // 发件人城市id
    String senderCityId;

    // 发件人区县id
    String senderDistrictId;

    // 发件人姓名
    String senderName;

    // 支付方式
    String paymentType;

    // 货物个数
    Integer cargoNum;

    // 金额
    BigDecimal amount;

    // 预计到达时间
    String estimateArriveTime;

    // 距离,单位:公里
    BigDecimal distance;

    // 时间戳
    Long ts;

    public void mergeBean(DwdOrderDetailOriginBean detailOriginBean, DwdOrderInfoOriginBean infoOriginBean) {
        // 合并原始明细字段
        this.id = detailOriginBean.id;
        this.orderId = detailOriginBean.orderId;
        this.cargoType = detailOriginBean.cargoType;
        this.volumeLength = detailOriginBean.volumnLength;
        this.volumeWidth = detailOriginBean.volumnWidth;
        this.volumeHeight = detailOriginBean.volumnHeight;
        this.weight = detailOriginBean.weight;

        // 合并原始订单字段
        this.orderNo = infoOriginBean.orderNo;
        this.status = infoOriginBean.status;
        this.collectType = infoOriginBean.collectType;
        this.userId = infoOriginBean.userId;
        this.receiverComplexId = infoOriginBean.receiverComplexId;
        this.receiverProvinceId = infoOriginBean.receiverProvinceId;
        this.receiverCityId = infoOriginBean.receiverCityId;
        this.receiverDistrictId = infoOriginBean.receiverDistrictId;
        this.receiverName = infoOriginBean.receiverName;
        this.senderComplexId = infoOriginBean.senderComplexId;
        this.senderProvinceId = infoOriginBean.senderProvinceId;
        this.senderCityId = infoOriginBean.senderCityId;
        this.senderDistrictId = infoOriginBean.senderDistrictId;
        this.senderName = infoOriginBean.senderName;
        this.paymentType = infoOriginBean.paymentType;
        this.cargoNum = infoOriginBean.cargoNum;
        this.amount = infoOriginBean.amount;
        this.estimateArriveTime = DateFormatUtil.toYmdHms(
            infoOriginBean.estimateArriveTime - 8 * 60 * 60 * 1000);
        this.distance = infoOriginBean.distance;
        this.dispatchTime =
            DateFormatUtil.toYmdHms(DateFormatUtil.toTs(
                infoOriginBean.updateTime.replaceAll("T", " ")
                    .replaceAll("Z", ""), true)
                + 8 * 60 * 60 * 1000);
        this.ts = DateFormatUtil.toTs(
            infoOriginBean.updateTime.replaceAll("T", " ")
                .replaceAll("Z", ""), true)
            + 8 * 60 * 60 * 1000;
    }
}
(7)创建物流域:转运完成事务事实表实体类,并整合(1)与(2),采用转运完成时间
package com.atguigu.tms.realtime.beans;

import com.atguigu.tms.realtime.utils.DateFormatUtil;
import lombok.Data;

import java.math.BigDecimal;

/**
 *物流域:转运完成事务事实表实体类
 */
@Data
public class DwdTransBoundFinishDetailBean {
    // 运单明细ID
    String id;

    // 运单id
    String orderId;

    // 货物类型
    String cargoType;

    // 长cm
    Integer volumeLength;

    // 宽cm
    Integer volumeWidth;

    // 高cm
    Integer volumeHeight;

    // 重量 kg
    BigDecimal weight;

    // 转运完成时间
    String boundFinishTime;

    // 运单号
    String orderNo;

    // 运单状态
    String status;

    // 取件类型,1为网点自寄,2为上门取件
    String collectType;

    // 客户id
    String userId;

    // 收件人小区id
    String receiverComplexId;

    // 收件人省份id
    String receiverProvinceId;

    // 收件人城市id
    String receiverCityId;

    // 收件人区县id
    String receiverDistrictId;

    // 收件人姓名
    String receiverName;

    // 发件人小区id
    String senderComplexId;

    // 发件人省份id
    String senderProvinceId;

    // 发件人城市id
    String senderCityId;

    // 发件人区县id
    String senderDistrictId;

    // 发件人姓名
    String senderName;

    // 支付方式
    String paymentType;

    // 货物个数
    Integer cargoNum;

    // 金额
    BigDecimal amount;

    // 预计到达时间
    String estimateArriveTime;

    // 距离,单位:公里
    BigDecimal distance;

    // 时间戳
    Long ts;

    public void mergeBean(DwdOrderDetailOriginBean detailOriginBean, DwdOrderInfoOriginBean infoOriginBean) {
        // 合并原始明细字段
        this.id = detailOriginBean.id;
        this.orderId = detailOriginBean.orderId;
        this.cargoType = detailOriginBean.cargoType;
        this.volumeLength = detailOriginBean.volumnLength;
        this.volumeWidth = detailOriginBean.volumnWidth;
        this.volumeHeight = detailOriginBean.volumnHeight;
        this.weight = detailOriginBean.weight;

        // 合并原始订单字段
        this.orderNo = infoOriginBean.orderNo;
        this.status = infoOriginBean.status;
        this.collectType = infoOriginBean.collectType;
        this.userId = infoOriginBean.userId;
        this.receiverComplexId = infoOriginBean.receiverComplexId;
        this.receiverProvinceId = infoOriginBean.receiverProvinceId;
        this.receiverCityId = infoOriginBean.receiverCityId;
        this.receiverDistrictId = infoOriginBean.receiverDistrictId;
        this.receiverName = infoOriginBean.receiverName;
        this.senderComplexId = infoOriginBean.senderComplexId;
        this.senderProvinceId = infoOriginBean.senderProvinceId;
        this.senderCityId = infoOriginBean.senderCityId;
        this.senderDistrictId = infoOriginBean.senderDistrictId;
        this.senderName = infoOriginBean.senderName;
        this.paymentType = infoOriginBean.paymentType;
        this.cargoNum = infoOriginBean.cargoNum;
        this.amount = infoOriginBean.amount;
        this.estimateArriveTime = DateFormatUtil.toYmdHms(
            infoOriginBean.estimateArriveTime - 8 * 60 * 60 * 1000);
        this.distance = infoOriginBean.distance;
        this.boundFinishTime =
            DateFormatUtil.toYmdHms(DateFormatUtil.toTs(
                infoOriginBean.updateTime.replaceAll("T", " ")
                    .replaceAll("Z", ""), true)
                + 8 * 60 * 60 * 1000);
        this.ts = DateFormatUtil.toTs(
            infoOriginBean.updateTime.replaceAll("T", " ")
                .replaceAll("Z", ""), true)
            + 8 * 60 * 60 * 1000;
    }
}
(8)创建物流域:派送成功事务事实表实体类,并整合(1)与(2),采用派送成功时间
package com.atguigu.tms.realtime.beans;

import com.atguigu.tms.realtime.utils.DateFormatUtil;
import lombok.Data;

import java.math.BigDecimal;

/**
 *物流域:派送成功事务事实表实体类
 */
@Data
public class DwdTransDeliverSucDetailBean {
    // 运单明细ID
    String id;

    // 运单id
    String orderId;

    // 货物类型
    String cargoType;

    // 长cm
    Integer volumeLength;

    // 宽cm
    Integer volumeWidth;

    // 高cm
    Integer volumeHeight;

    // 重量 kg
    BigDecimal weight;

    // 派送成功时间
    String deliverTime;

    // 运单号
    String orderNo;

    // 运单状态
    String status;

    // 取件类型,1为网点自寄,2为上门取件
    String collectType;

    // 客户id
    String userId;

    // 收件人小区id
    String receiverComplexId;

    // 收件人省份id
    String receiverProvinceId;

    // 收件人城市id
    String receiverCityId;

    // 收件人区县id
    String receiverDistrictId;

    // 收件人姓名
    String receiverName;

    // 发件人小区id
    String senderComplexId;

    // 发件人省份id
    String senderProvinceId;

    // 发件人城市id
    String senderCityId;

    // 发件人区县id
    String senderDistrictId;

    // 发件人姓名
    String senderName;

    // 支付方式
    String paymentType;

    // 货物个数
    Integer cargoNum;

    // 金额
    BigDecimal amount;

    // 预计到达时间
    String estimateArriveTime;

    // 距离,单位:公里
    BigDecimal distance;

    // 时间戳
    Long ts;

    public void mergeBean(DwdOrderDetailOriginBean detailOriginBean, DwdOrderInfoOriginBean infoOriginBean) {
        // 合并原始明细字段
        this.id = detailOriginBean.id;
        this.orderId = detailOriginBean.orderId;
        this.cargoType = detailOriginBean.cargoType;
        this.volumeLength = detailOriginBean.volumnLength;
        this.volumeWidth = detailOriginBean.volumnWidth;
        this.volumeHeight = detailOriginBean.volumnHeight;
        this.weight = detailOriginBean.weight;

        // 合并原始订单字段
        this.orderNo = infoOriginBean.orderNo;
        this.status = infoOriginBean.status;
        this.collectType = infoOriginBean.collectType;
        this.userId = infoOriginBean.userId;
        this.receiverComplexId = infoOriginBean.receiverComplexId;
        this.receiverProvinceId = infoOriginBean.receiverProvinceId;
        this.receiverCityId = infoOriginBean.receiverCityId;
        this.receiverDistrictId = infoOriginBean.receiverDistrictId;
        this.receiverName = infoOriginBean.receiverName;
        this.senderComplexId = infoOriginBean.senderComplexId;
        this.senderProvinceId = infoOriginBean.senderProvinceId;
        this.senderCityId = infoOriginBean.senderCityId;
        this.senderDistrictId = infoOriginBean.senderDistrictId;
        this.senderName = infoOriginBean.senderName;
        this.paymentType = infoOriginBean.paymentType;
        this.cargoNum = infoOriginBean.cargoNum;
        this.amount = infoOriginBean.amount;
        this.estimateArriveTime = DateFormatUtil.toYmdHms(
            infoOriginBean.estimateArriveTime - 8 * 60 * 60 * 1000);
        this.distance = infoOriginBean.distance;
        this.deliverTime =
            DateFormatUtil.toYmdHms(DateFormatUtil.toTs(
                infoOriginBean.updateTime.replaceAll("T", " ")
                    .replaceAll("Z", ""), true)
                + 8 * 60 * 60 * 1000);
        this.ts = DateFormatUtil.toTs(
            infoOriginBean.updateTime.replaceAll("T", " ")
                .replaceAll("Z", ""), true)
            + 8 * 60 * 60 * 1000;
    }
}
(9)创建物流域:签收事务事实表实体类,并整合(1)与(2),采用签收时间
package com.atguigu.tms.realtime.beans;

import com.atguigu.tms.realtime.utils.DateFormatUtil;
import lombok.Data;

import java.math.BigDecimal;

/**
 * 物流域:签收事务事实表实体类
 */
@Data
public class DwdTransSignDetailBean {
    // 运单明细ID
    String id;

    // 运单id
    String orderId;

    // 货物类型
    String cargoType;

    // 长cm
    Integer volumeLength;

    // 宽cm
    Integer volumeWidth;

    // 高cm
    Integer volumeHeight;

    // 重量 kg
    BigDecimal weight;

    // 签收时间
    String signTime;

    // 运单号
    String orderNo;

    // 运单状态
    String status;

    // 取件类型,1为网点自寄,2为上门取件
    String collectType;

    // 客户id
    String userId;

    // 收件人小区id
    String receiverComplexId;

    // 收件人省份id
    String receiverProvinceId;

    // 收件人城市id
    String receiverCityId;

    // 收件人区县id
    String receiverDistrictId;

    // 收件人姓名
    String receiverName;

    // 发件人小区id
    String senderComplexId;

    // 发件人省份id
    String senderProvinceId;

    // 发件人城市id
    String senderCityId;

    // 发件人区县id
    String senderDistrictId;

    // 发件人姓名
    String senderName;

    // 支付方式
    String paymentType;

    // 货物个数
    Integer cargoNum;

    // 金额
    BigDecimal amount;

    // 预计到达时间
    String estimateArriveTime;

    // 距离,单位:公里
    BigDecimal distance;

    // 时间戳
    Long ts;

    public void mergeBean(DwdOrderDetailOriginBean detailOriginBean, DwdOrderInfoOriginBean infoOriginBean) {
        // 合并原始明细字段
        this.id = detailOriginBean.id;
        this.orderId = detailOriginBean.orderId;
        this.cargoType = detailOriginBean.cargoType;
        this.volumeLength = detailOriginBean.volumnLength;
        this.volumeWidth = detailOriginBean.volumnWidth;
        this.volumeHeight = detailOriginBean.volumnHeight;
        this.weight = detailOriginBean.weight;

        // 合并原始订单字段
        this.orderNo = infoOriginBean.orderNo;
        this.status = infoOriginBean.status;
        this.collectType = infoOriginBean.collectType;
        this.userId = infoOriginBean.userId;
        this.receiverComplexId = infoOriginBean.receiverComplexId;
        this.receiverProvinceId = infoOriginBean.receiverProvinceId;
        this.receiverCityId = infoOriginBean.receiverCityId;
        this.receiverDistrictId = infoOriginBean.receiverDistrictId;
        this.receiverName = infoOriginBean.receiverName;
        this.senderComplexId = infoOriginBean.senderComplexId;
        this.senderProvinceId = infoOriginBean.senderProvinceId;
        this.senderCityId = infoOriginBean.senderCityId;
        this.senderDistrictId = infoOriginBean.senderDistrictId;
        this.senderName = infoOriginBean.senderName;
        this.paymentType = infoOriginBean.paymentType;
        this.cargoNum = infoOriginBean.cargoNum;
        this.amount = infoOriginBean.amount;
        this.estimateArriveTime = DateFormatUtil.toYmdHms(
            infoOriginBean.estimateArriveTime - 8 * 60 * 60 * 1000);
        this.distance = infoOriginBean.distance;
        this.signTime =
            DateFormatUtil.toYmdHms(DateFormatUtil.toTs(
                infoOriginBean.updateTime.replaceAll("T", " ")
                    .replaceAll("Z", ""), true)
                + 8 * 60 * 60 * 1000);
        this.ts = DateFormatUtil.toTs(
            infoOriginBean.updateTime.replaceAll("T", " ")
                .replaceAll("Z", ""), true)
            + 8 * 60 * 60 * 1000;
    }
}
(10)创建交易域:取消运单事务事实表实体类,并整合(1)与(2),采用取消时间
package com.atguigu.tms.realtime.beans;

import com.atguigu.tms.realtime.utils.DateFormatUtil;
import lombok.Data;

import java.math.BigDecimal;

/**
 *交易域:取消运单事务事实表实体类
 */
@Data
public class DwdTradeCancelDetailBean {
    // 运单明细ID
    String id;

    // 运单id
    String orderId;

    // 货物类型
    String cargoType;

    // 长cm
    Integer volumeLength;

    // 宽cm
    Integer volumeWidth;

    // 高cm
    Integer volumeHeight;

    // 重量 kg
    BigDecimal weight;

    // 取消时间
    String cancelTime;

    // 运单号
    String orderNo;

    // 运单状态
    String status;

    // 取件类型,1为网点自寄,2为上门取件
    String collectType;

    // 客户id
    String userId;

    // 收件人小区id
    String receiverComplexId;

    // 收件人省份id
    String receiverProvinceId;

    // 收件人城市id
    String receiverCityId;

    // 收件人区县id
    String receiverDistrictId;

    // 收件人姓名
    String receiverName;

    // 发件人小区id
    String senderComplexId;

    // 发件人省份id
    String senderProvinceId;

    // 发件人城市id
    String senderCityId;

    // 发件人区县id
    String senderDistrictId;

    // 发件人姓名
    String senderName;

    // 支付方式
    String paymentType;

    // 货物个数
    Integer cargoNum;

    // 金额
    BigDecimal amount;

    // 预计到达时间
    String estimateArriveTime;

    // 距离,单位:公里
    BigDecimal distance;

    // 时间戳
    Long ts;

    public void mergeBean(DwdOrderDetailOriginBean detailOriginBean, DwdOrderInfoOriginBean infoOriginBean) {
        // 合并原始明细字段
        this.id = detailOriginBean.id;
        this.orderId = detailOriginBean.orderId;
        this.cargoType = detailOriginBean.cargoType;
        this.volumeLength = detailOriginBean.volumnLength;
        this.volumeWidth = detailOriginBean.volumnWidth;
        this.volumeHeight = detailOriginBean.volumnHeight;
        this.weight = detailOriginBean.weight;

        // 合并原始订单字段
        this.orderNo = infoOriginBean.orderNo;
        this.status = infoOriginBean.status;
        this.collectType = infoOriginBean.collectType;
        this.userId = infoOriginBean.userId;
        this.receiverComplexId = infoOriginBean.receiverComplexId;
        this.receiverProvinceId = infoOriginBean.receiverProvinceId;
        this.receiverCityId = infoOriginBean.receiverCityId;
        this.receiverDistrictId = infoOriginBean.receiverDistrictId;
        this.receiverName = infoOriginBean.receiverName;
        this.senderComplexId = infoOriginBean.senderComplexId;
        this.senderProvinceId = infoOriginBean.senderProvinceId;
        this.senderCityId = infoOriginBean.senderCityId;
        this.senderDistrictId = infoOriginBean.senderDistrictId;
        this.senderName = infoOriginBean.senderName;
        this.paymentType = infoOriginBean.paymentType;
        this.cargoNum = infoOriginBean.cargoNum;
        this.amount = infoOriginBean.amount;
        this.estimateArriveTime = DateFormatUtil.toYmdHms(
            infoOriginBean.estimateArriveTime - 8 * 60 * 60 * 1000);
        this.distance = infoOriginBean.distance;
        this.cancelTime =
            DateFormatUtil.toYmdHms(DateFormatUtil.toTs(
                infoOriginBean.updateTime.replaceAll("T", " ")
                    .replaceAll("Z", ""), true)
                + 8 * 60 * 60 * 1000);
        this.ts = DateFormatUtil.toTs(
            infoOriginBean.updateTime.replaceAll("T", " ")
                .replaceAll("Z", ""), true)
            + 8 * 60 * 60 * 1000;
    }
}

到了这里,关于物流实时数仓DWD层——1.准备工作的文章就介绍完了。如果您还想了解更多内容,请在右上角搜索TOY模板网以前的文章或继续浏览下面的相关文章,希望大家以后多多支持TOY模板网!

本文来自互联网用户投稿,该文观点仅代表作者本人,不代表本站立场。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如若转载,请注明出处: 如若内容造成侵权/违法违规/事实不符,请点击违法举报进行投诉反馈,一经查实,立即删除!

领支付宝红包 赞助服务器费用

相关文章

  • Flink实时电商数仓(八)

    主要任务:从kafka页面日志主题读取数据,统计 七日回流用户:之前活跃的用户,有一段时间不活跃了,之后又开始活跃,称为回流用户 当日独立用户数:同一个用户当天重复登录,只算作一个独立用户。 读取kafka页面主题数据 转换数据结构: String - JSONObject 过滤数据,u

    2024年02月03日
    浏览(38)
  • Flink+Doris 实时数仓

    Doris基本原理 Doris基本架构非常简单,只有FE(Frontend)、BE(Backend)两种角色,不依赖任何外部组件,对部署和运维非常友好。架构图如下 可以 看到Doris 的数仓架构十分简洁,不依赖 Hadoop 生态组件,构建及运维成本较低。 FE(Frontend)以 Java 语言为主,主要功能职责: 接收用户

    2024年02月07日
    浏览(47)
  • Flink电商实时数仓(四)

    业务数据:数据都是MySQL中的表格数据, 使用Flink SQL 处理 日志数据:分为page页面日志(页面信息,曝光信息,动作信息,报错信息)和启动日志(启动信息,报错信息),使用Flink Stream API处理 五种日志数据: “start”; 启动信息 “err”; 错误信息 “display”; 曝光信息 “ac

    2024年01月17日
    浏览(45)
  • 实时数仓|基于Flink1.11的SQL构建实时数仓探索实践

    实时数仓主要是为了解决传统数仓数据时效性低的问题,实时数仓通常会用在实时的 OLAP 分析、实时的数据看板、业务指标实时监控等场景。虽然关于实时数仓的架构及技术选型与传统的离线数仓会存在差异,但是关于数仓建设的基本方法论是一致的。本文会分享基于 Flink

    2024年02月16日
    浏览(43)
  • Flink实时电商数仓之Doris框架(七)

    大规模并行处理的分析型数据库产品。使用场景:一般先将原始数据经过清洗过滤转换后,再导入doris中使用。主要实现的功能有: 实时看板 面向企业内部分析师和管理者的报表 面向用户或者客户的高并发报表分析 即席查询 统一数仓构建:替换了原来由Spark, Hive,Kudu, Hba

    2024年02月03日
    浏览(45)
  • Flink实时数仓同步:拉链表实战详解

    在大数据领域,业务数据通常最初存储在关系型数据库,例如MySQL。然而,为了满足日常分析和报表等需求,大数据平台会采用多种不同的存储方式来容纳这些业务数据。这些存储方式包括离线仓库、实时仓库等,根据不同的业务需求和数据特性进行选择。 举例来说,假设业

    2024年01月20日
    浏览(48)
  • 大数据Flink(六十一):Flink流处理程序流程和项目准备

    文章目录 Flink流处理程序流程和项目准备 一、Flink流处理程序的一般流程

    2024年02月11日
    浏览(38)
  • Flink CDC和Flink SQL构建实时数仓Flink写入Doris

    软件环境 Flink1.13.3 Scala 2.12 doris 0.14 一、MySQL 开启binlog日志、创建用户 1.开启bin log MySQL 8.0默认开启了binlog,可以通过代码show variables like \\\"%log_bin%\\\";查询是否开启了,show variables like \\\"%server_id%\\\";查询服务器ID。 上图分别显示了bin long是否开启以及bin log所在的位置。 2.创建用户 C

    2024年02月02日
    浏览(76)
  • flink 实时数仓构建与开发[记录一些坑]

    1、业务库使用pg数据库, 业务数据可以改动任意时间段数据 2、监听采集业务库数据,实时捕捉业务库数据变更,同时实时变更目标表和报表数据 实时数据流图与分层设计说明 1、debezium采集pg库表数据同步到kafka 【kafka模式】 2、flink 消费kafka写入pg或kafka 【upset-kafka,新版k

    2024年02月16日
    浏览(37)
  • 美团买菜基于 Flink 的实时数仓建设

    美团买菜是美团自营生鲜零售平台,上面所有的商品都由美团亲自采购,并通过供应链物流体系,运输到距离用户 3km 范围内的服务站。用户从美团买菜平台下单后,商品会从服务站送到用户手中,最快 30 分钟内。 上图中,左侧的时间轴展示了美团买菜的发展历程,右侧展示

    2024年02月09日
    浏览(43)

觉得文章有用就打赏一下文章作者

支付宝扫一扫打赏

博客赞助

微信扫一扫打赏

请作者喝杯咖啡吧~博客赞助

支付宝扫一扫领取红包,优惠每天领

二维码1

领取红包

二维码2

领红包