📚第一章 前言
在前面对data-integration做了一些简单了解，从部署到应用，今天尝试把后端运行作业代码拎出来，去真正运行一下，只有实操之后才会有更深刻的认识，有些看着简单的功能，实操过程中会遇到很多问题，这个时候你的想法也会发生改变，所以很多时候为什么开发人员痛恨做产品的同事，因为嘴上说的简单，真正去做一下，你自己感受一下，就知道了！
- 基于Kettle开发的web版数据集成开源工具(data-integration)-部署篇
- 基于Kettle开发的web版数据集成开源工具(data-integration)-介绍篇
- 基于Kettle开发的web版数据集成开源工具(data-integration)-应用篇
data-integration可以理解为Kettle的web版本，详情可参照上面的文章，这里直接从代码层面开始
📚第二章 demo源码
直接在我之前个人项目上进行的韧小钊(纯个人项目，整理或者更新全靠一时兴起，如果对你有用，挺好，没用请PASS），现在想法就是后端实现运行作业的功能，至于保存作业，都是前端传给后台的，后台存的就是JSON串，所以后端改动应该不大，这里就是尝试先提取后端运行作业代码，直接运行前端传参，看能否成功。
📗pom.xml引入Kettle引擎核心文件
引入之后，启动项目报jar包冲突，则在下面文件中进行exclusion配置排除
<!-- Kettle核心包 20240106 by rxz-->
<dependency>
<groupId>pentaho-kettle</groupId>
<artifactId>kettle-engine</artifactId>
<version>${kettle.version}</version>
<exclusions>
<exclusion>
<artifactId>servlet-api</artifactId>
<groupId>javax.servlet</groupId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>pentaho-kettle</groupId>
<artifactId>kettle-core</artifactId>
<version>${kettle.version}</version>
<exclusions>
<exclusion>
<artifactId>servlet-api</artifactId>
<groupId>javax.servlet</groupId>
</exclusion>
</exclusions>
</dependency>
📗java源码
📕 controller
package com.renxiaozhao.api.controller;
import com.renxiaozhao.common.Result;
import com.renxiaozhao.service.inf.PdiUseDemoService;
import io.swagger.annotations.Api;
import io.swagger.annotations.ApiOperation;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;
/**
* kettle/pdi使用样例-控制层
*
* @author 韧小钊
*/
@RestController
@RequestMapping("/kettle")
@Api(tags = "KETTLE-PDI")
@Slf4j
public class PdiUseDemoController {
/**
* 服务对象
*/
@Autowired
private PdiUseDemoService pdiUseDemoService;
@PostMapping("/run")
@ApiOperation(value = "kettle作业运行", notes = "kettle作业运行")
public Result<Void> executeJob(@RequestParam String jobJson) throws Exception {
pdiUseDemoService.executeJob(jobJson);
return Result.success();
}
}
📕 service
这里就是对应基于Kettle开发的web版数据集成开源工具(data-integration)-应用篇中的后台服务流程章节的ProjectExecutorController.executeById，目的是研究Kettle引擎的具体使用，所以这里是去掉了各种封装，直接前端传参（作业JSON串），后端直接执行
package com.renxiaozhao.service.inf;
import com.baomidou.mybatisplus.extension.service.IService;
import com.renxiaozhao.bean.entity.SportEntity;
public interface PdiUseDemoService extends IService<SportEntity> {
//直接复制的原来代码，有些实体类暂时用不到，请先忽略
void executeJob(String jobJson) throws Exception;
}
package com.renxiaozhao.service.impl;
import com.baomidou.mybatisplus.extension.service.impl.ServiceImpl;
import com.renxiaozhao.bean.entity.SportEntity;
import com.renxiaozhao.dao.mapper.SportMapper;
import com.renxiaozhao.service.inf.PdiUseDemoService;
import com.renxiaozhao.service.util.JSONLinkedObject;
import com.renxiaozhao.service.util.XML;
import lombok.extern.slf4j.Slf4j;
import org.apache.commons.io.FileUtils;
import org.apache.commons.lang.StringEscapeUtils;
import org.pentaho.di.core.exception.KettleMissingPluginsException;
import org.pentaho.di.core.exception.KettleXMLException;
import org.pentaho.di.core.logging.LogLevel;
import org.pentaho.di.core.logging.LoggingObjectType;
import org.pentaho.di.core.logging.SimpleLoggingObject;
import org.pentaho.di.core.variables.Variables;
import org.pentaho.di.core.xml.XMLHandler;
import org.pentaho.di.trans.Trans;
import org.pentaho.di.trans.TransAdapter;
import org.pentaho.di.trans.TransExecutionConfiguration;
import org.pentaho.di.trans.TransMeta;
import org.springframework.stereotype.Service;
import org.w3c.dom.Document;
import java.io.File;
import java.io.IOException;
import java.util.HashMap;
import java.util.UUID;
@Slf4j
@Service
public class PdiUseDemoServiceImpl extends ServiceImpl<SportMapper,SportEntity> implements PdiUseDemoService {
@Override
public void executeJob(String jobJson) throws Exception {
execute(jobJson);
}
private void execute(String jobJson) throws Exception {
// 构建TransMeta 对象
TransMeta transMeta = buildTransMeta(jobJson);
TransExecutionConfiguration executionConfiguration = new TransExecutionConfiguration();
// 设置默认值以便运行配置可以正确设置
executionConfiguration.setExecutingLocally(true);
executionConfiguration.setExecutingRemotely(false);
executionConfiguration.setExecutingClustered(false);
// 不启用安全模式
executionConfiguration.setSafeModeEnabled(true);
executionConfiguration.getUsedVariables(transMeta);
executionConfiguration.setLogLevel(LogLevel.DEBUG);
// 默认设置本地引擎执行
executionConfiguration.setRunConfiguration("Pentaho local");
//设置命令参数
executionConfiguration.setVariables(new HashMap<>());
// 创建trans
Trans trans = new Trans(transMeta);
String spoonLogObjectId = UUID.randomUUID().toString();
SimpleLoggingObject spoonLoggingObject = new SimpleLoggingObject(Thread.currentThread().getName() + "-" + Thread.currentThread().getId()
, LoggingObjectType.SPOON, null);
spoonLoggingObject.setContainerObjectId(spoonLogObjectId);
spoonLoggingObject.setLogLevel(executionConfiguration.getLogLevel());
trans.setParent(spoonLoggingObject);
trans.setLogLevel(executionConfiguration.getLogLevel());
trans.setReplayDate(executionConfiguration.getReplayDate());
trans.setRepository(executionConfiguration.getRepository());
trans.setMonitored(false);
// 启动转换
trans.addTransListener(new TransAdapter() {
@Override
public void transFinished(Trans trans) {
log.info("项目执行完成");
}
});
trans.startThreads();
}
public TransMeta buildTransMeta(String jobJson) throws IOException, KettleXMLException, KettleMissingPluginsException {
Document document;
//json转xml
if (!jobJson.startsWith("<?xml")) {
// json转xml
jobJson = StringEscapeUtils.unescapeXml(jobJson);
jobJson = "<?xml version=\"1.0\" encoding=\"UTF-8\"?>" + XML.toString(new JSONLinkedObject(jobJson));
log.info("json转换成xml，转换后的xml:{}", jobJson);
}
// 写到临时目录
File outFile = new File("D:\\tmp\\test", "test.xml");
FileUtils.writeStringToFile(outFile, jobJson);
// 加载xml
document = XMLHandler.loadXMLString(jobJson);
TransMeta transMeta = new TransMeta();
transMeta.loadXML(
document.getDocumentElement(), outFile.getPath(), null, null, true, new Variables(),
(message, rememberText, rememberPropertyName) -> {
// Yes means: overwrite
return true;
});
if (transMeta.hasMissingPlugins()) {
log.info("【{}】缺少执行插件。", jobJson);
}
return transMeta;
}
}
📕 其它
代码已上传
📕 maven settings.xml
<?xml version="1.0" encoding="UTF-8"?>
<settings xmlns="http://maven.apache.org/SETTINGS/1.2.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.2.0 https://maven.apache.org/xsd/settings-1.2.0.xsd">
<localRepository>D:\soft\apache-maven-3.8.5\apache-maven-3.8.5\repository</localRepository>
<pluginGroups></pluginGroups>
<proxies></proxies>
<servers></servers>
<mirrors>
<mirror>
<id>pentaho-public</id>
<name>Pentaho Public Mirror</name>
<url>https://repo.orl.eng.hitachivantara.com/artifactory/pnt-mvn/</url>
<mirrorOf>*</mirrorOf>
</mirror>
<mirror>
<id>aliyunmaven</id>
<mirrorOf>*</mirrorOf>
<name>阿里云公共仓库</name>
<url>https://maven.aliyun.com/repository/public</url>
</mirror>
</mirrors>
<profiles></profiles>
</settings>
📗测试
启动服务，通过swagger测试
📕 测试文件
传参对应的就是dp_portal_project_file的project_file，详见基于Kettle开发的web版数据集成开源工具(data-integration)-应用篇
测试文件参考如下（直接使用的话，记得修改数据库信息）：
{
"transformation": {
"attributes": "",
"connection": [
{
"access": "Native",
"attributes": {
"attribute": [
{
"attribute": "Y",
"code": "SUPPORTS_BOOLEAN_DATA_TYPE"
},
{
"attribute": "Y",
"code": "SUPPORTS_TIMESTAMP_DATA_TYPE"
},
{
"attribute": "N",
"code": "QUOTE_ALL_FIELDS"
},
{
"attribute": "N",
"code": "FORCE_IDENTIFIERS_TO_LOWERCASE"
},
{
"attribute": "N",
"code": "FORCE_IDENTIFIERS_TO_UPPERCASE"
},
{
"attribute": "Y",
"code": "PRESERVE_RESERVED_WORD_CASE"
},
{
"attribute": "N",
"code": "STRICT_NUMBER_38_INTERPRETATION"
},
{
"attribute": "",
"code": "PREFERRED_SCHEMA_NAME"
}
]
},
"dataSourceId": "e2be98ed-9077-4802-8795-df45acb7b033",
"dataSourceName": "dataintegration_db",
"data_tablespace": "",
"database": "dataintegration_db",
"dsName": "dataintegration_db",
"index_tablespace": "",
"name": "e2be98ed-9077-4802-8795-df45acb7b033",
"password": "Encrypted 2be98afc86ae49d94a40dab508cc2fd89",
"port": "3306",
"server": "192.168.17.10",
"servername": "",
"type": "MYSQL",
"username": "stelladp"
}
],
"info": {
"capture_step_performance": "N",
"clusterschemas": "",
"created_date": "2020/02/05 13:34:55.102",
"created_user": "037de45a-e2ad-4d0a-943c-455db05161f4",
"dependencies": "",
"description": "",
"directory": "/home/admin",
"extended_description": "",
"feedback_shown": "Y",
"feedback_size": 50000,
"is_key_private": "N",
"key_for_session_key": "",
"log": "",
"maxdate": {
"connection": "",
"field": "",
"maxdiff": 0,
"offset": 0,
"table": ""
},
"modified_date": "2020/02/19 17:08:25.150",
"modified_user": "admin",
"name": "执行SQL",
"parameters": {
"parameter": [
]
},
"partitionschemas": "",
"shared_objects_file": "",
"size_rowset": 10000,
"slaveservers": "",
"sleep_time_empty": 50,
"sleep_time_full": 50,
"step_performance_capturing_delay": 1000,
"step_performance_capturing_size_limit": 100,
"trans_type": "Normal",
"trans_version": "",
"unique_connections": "N",
"using_thread_priorities": "Y"
},
"nodeList": {
"flowInfo": {
"Id": "fdf50d66dc7d",
"Name": "我的流程",
"Remark": ""
},
"lineList": [
],
"nodeList": [
{
"copy": 0,
"dataQuerySql": "",
"dataSourceId": "",
"datasourceId": "",
"errors": 0,
"icon": "iconfont icon-kongjian1",
"id": "ExecSQL6925c0883a9b",
"label": "执行SQL脚本",
"left": "341px",
"linesInput": 0,
"linesOutput": 0,
"linesRead": 0,
"linesRejected": 0,
"linesUpdated": 0,
"linesWritten": 0,
"pluginId": "ExecSQL",
"pluginOutput": "Y",
"priority": "-",
"projectError": 0,
"projectId": "",
"projectName": "",
"seconPluginType": "scripts",
"seconds": 0,
"speed": 0,
"status": false,
"statusDescription": "",
"stepExecutionStatu": "",
"stepExecutionStatus": "",
"stepMetaExecutionStatus": null,
"stepName": "",
"stepSql": "",
"top": "212px"
}
],
"statusListen": [
]
},
"notepads": "",
"order": {
"hop": [
]
},
"slave-step-copy-partition-distribution": "",
"slave_transformation": "N",
"step": [
{
"BindString": "",
"Connectpool": false,
"Excute_sql": "",
"GUI": {
"draw": "Y",
"xloc": "160",
"yloc": "224"
},
"arguments": {
"argument": [
]
},
"attributes": [
],
"bool": true,
"cluster": false,
"cluster_schema": [
],
"connecName": "",
"connect_size": "",
"connection": "e2be98ed-9077-4802-8795-df45acb7b033",
"copies": "1",
"custom_distribution": [
],
"defalut_mode": "",
"delete_field": "",
"descripe": "The default catalog of connections created by this pool.",
"description": [
],
"distribute": "Y",
"execute_each_row": "N",
"fields": {
"field": [
]
},
"free_space": "",
"header": "N",
"identifier": false,
"identifier_captial": false,
"identifier_lowercase": false,
"initFlag": true,
"insert_field": "",
"name": "执行SQL脚本",
"noempty": "N",
"oldStepName": "执行SQL脚本",
"outFields": [
],
"partitioning": {
"method": "none",
"schema_name": [
]
},
"preserve_case": true,
"quoteString": "N",
"read_field": "",
"remotesteps": {
"input": "\n ",
"output": "\n "
},
"replace_variables": "N",
"set_params": false,
"single_statement": "Y",
"sql": "select count(1) from dp_portal_role",
"stoponempty": "N",
"strict_number": false,
"time_stamp": true,
"type": "ExecSQL",
"update_field": ""
}
],
"step_error_handling": {
"error": [
]
}
}
}
📕 测试结果
报错了…😔详见问题二
⁉️问题记录
❓问题一：jar包冲突 - An attempt was made to call the method javax.servlet.ServletContext.setInitParameter(Ljava/lang/String;Ljava/lang/String;)Z but it does not exist. Its class, javax.servlet.ServletContext, is available from the following locations:
java.lang.BootstrapMethodError: java.lang.NoSuchMethodError: javax.servlet.ServletContext.setInitParameter(Ljava/lang/String;Ljava/lang/String;)Z
at org.springframework.boot.web.servlet.server.AbstractServletWebServerFactory.lambda$mergeInitializers$0(AbstractServletWebServerFactory.java:253)
at org.springframework.boot.web.embedded.jetty.ServletContextInitializerConfiguration.callInitializers(ServletContextInitializerConfiguration.java:66)
at org.springframework.boot.web.embedded.jetty.ServletContextInitializerConfiguration.configure(ServletContextInitializerConfiguration.java:55)
at org.eclipse.jetty.webapp.WebAppContext.configure(WebAppContext.java:517)
at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1454)
at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:852)
at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:278)
at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:545)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:138)
at org.eclipse.jetty.server.Server.start(Server.java:415)
at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:108)
at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113)
at org.eclipse.jetty.server.Server.doStart(Server.java:382)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.springframework.boot.web.embedded.jetty.JettyWebServer.initialize(JettyWebServer.java:108)
at org.springframework.boot.web.embedded.jetty.JettyWebServer.<init>(JettyWebServer.java:86)
at org.springframework.boot.web.embedded.jetty.JettyServletWebServerFactory.getJettyWebServer(JettyServletWebServerFactory.java:410)
at org.springframework.boot.web.embedded.jetty.JettyServletWebServerFactory.getWebServer(JettyServletWebServerFactory.java:153)
at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.createWebServer(ServletWebServerApplicationContext.java:181)
at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.onRefresh(ServletWebServerApplicationContext.java:154)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:543)
at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:142)
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:775)
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:397)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:316)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1260)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1248)
at com.renxiaozhao.api.ApiApplicationServer.main(ApiApplicationServer.java:21)
Caused by: java.lang.NoSuchMethodError: javax.servlet.ServletContext.setInitParameter(Ljava/lang/String;Ljava/lang/String;)Z
at java.lang.invoke.MethodHandleNatives.resolve(Native Method)
at java.lang.invoke.MemberName$Factory.resolve(MemberName.java:975)
at java.lang.invoke.MemberName$Factory.resolveOrFail(MemberName.java:1000)
at java.lang.invoke.MethodHandles$Lookup.resolveOrFail(MethodHandles.java:1394)
at java.lang.invoke.MethodHandles$Lookup.linkMethodHandleConstant(MethodHandles.java:1750)
at java.lang.invoke.MethodHandleNatives.linkMethodHandleConstant(MethodHandleNatives.java:477)
... 29 common frames omitted
2024-01-06 13:40:50.233 [main] INFO org.eclipse.jetty.server.AbstractConnector -
Started ServerConnector@697a0948{HTTP/1.1,[http/1.1]}{0.0.0.0:9080}
2024-01-06 13:40:50.242 [main] INFO org.eclipse.jetty.server.AbstractConnector -
Stopped ServerConnector@697a0948{HTTP/1.1,[http/1.1]}{0.0.0.0:9080}
2024-01-06 13:40:50.243 [main] INFO org.eclipse.jetty.server.handler.ContextHandler -
Stopped o.s.b.w.e.j.JettyEmbeddedWebAppContext@521441d5{application,/renxiaozhao,[file:///C:/Users/lenovo/AppData/Local/Temp/jetty-docbase.1734690716899826200.9080/, jar:file:/D:/soft/apache-maven-3.8.5/apache-maven-3.8.5/repository/io/springfox/springfox-swagger-ui/2.9.2/springfox-swagger-ui-2.9.2.jar!/META-INF/resources, jar:file:/D:/soft/apache-maven-3.8.5/apache-maven-3.8.5/repository/com/github/xiaoymin/knife4j-spring-ui/2.0.8/knife4j-spring-ui-2.0.8.jar!/META-INF/resources],UNAVAILABLE}
2024-01-06 13:40:50.247 [main] WARN o.s.b.w.s.c.AnnotationConfigServletWebServerApplicationContext -
Exception encountered during context initialization - cancelling refresh attempt: org.springframework.context.ApplicationContextException: Unable to start web server; nested exception is org.springframework.boot.web.server.WebServerException: Unable to start embedded Jetty web server
2024-01-06 13:40:50.261 [main] INFO o.s.b.a.l.ConditionEvaluationReportLoggingListener -
Error starting ApplicationContext. To display the conditions report re-run your application with 'debug' enabled.
2024-01-06 13:40:50.287 [main] ERROR o.s.b.diagnostics.LoggingFailureAnalysisReporter -
***************************
APPLICATION FAILED TO START
***************************
Description:
An attempt was made to call the method javax.servlet.ServletContext.setInitParameter(Ljava/lang/String;Ljava/lang/String;)Z but it does not exist. Its class, javax.servlet.ServletContext, is available from the following locations:
jar:file:/D:/soft/apache-maven-3.8.5/apache-maven-3.8.5/repository/javax/servlet/servlet-api/2.4/servlet-api-2.4.jar!/javax/servlet/ServletContext.class
jar:file:/D:/soft/apache-maven-3.8.5/apache-maven-3.8.5/repository/javax/servlet/javax.servlet-api/4.0.1/javax.servlet-api-4.0.1.jar!/javax/servlet/ServletContext.class
It was loaded from the following location:
file:/D:/soft/apache-maven-3.8.5/apache-maven-3.8.5/repository/javax/servlet/servlet-api/2.4/servlet-api-2.4.jar
Action:
Correct the classpath of your application so that it contains a single, compatible version of javax.servlet.ServletContext
Disconnected from the target VM, address: '127.0.0.1:50771', transport: 'socket'
Process finished with exit code 1
❗解决方式一：Kettle排除掉冲突jar（因为没引入Kettle之前是没问题的）
<kettle.version>9.2.0.0-290</kettle.version>
<dependency>
<groupId>pentaho-kettle</groupId>
<artifactId>kettle-engine</artifactId>
<version>${kettle.version}</version>
<exclusions>
<exclusion>
<artifactId>servlet-api</artifactId>
<groupId>javax.servlet</groupId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>pentaho-kettle</groupId>
<artifactId>kettle-core</artifactId>
<version>${kettle.version}</version>
<exclusions>
<exclusion>
<artifactId>servlet-api</artifactId>
<groupId>javax.servlet</groupId>
</exclusion>
</exclusions>
</dependency>
❕拓展：通过Maven Helper插件排除冲突jar包
使用IDEA工具，安装Maven Helper插件，进行冲突jar包排查，步骤如下：
- 安装插件
- 安装完成之后，打开
pom.xml文件，可以看到Dependency Analyzer
- 点击
Dependency Analyzer进行分析
- 尽量定位修改的位置进行排查（找到修改的
pom.xml进行定位）
冲突嘛（排除哪一方都行，根据报错排除对应的jar包，不行再试另一个jar包）
- 直接点击
Exclude，帮你自动生成排除配置
❓问题二：Database type not found!…database type with plugin id [Oracle] couldn't be found!
2024-01-05 16:54:41.190 [qtp1963906615-73] WARN org.eclipse.jetty.server.HttpChannel -
/renxiaozhao/kettle/run
org.springframework.web.util.NestedServletException: Request processing failed; nested exception is org.pentaho.di.core.exception.KettleXMLException:
错误从XML文件读取转换
Database type not found!
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1013)
at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:908)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:665)
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:882)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:750)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:867)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1623)
at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:99)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:92)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:93)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:200)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1588)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1345)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1557)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.server.Server.handle(Server.java:502)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:364)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.pentaho.di.core.exception.KettleXMLException:
错误从XML文件读取转换
Database type not found!
at org.pentaho.di.trans.TransMeta.loadXML(TransMeta.java:3460)
at com.renxiaozhao.service.impl.PdiUseDemoServiceImpl.buildTransMeta(PdiUseDemoServiceImpl.java:96)
at com.renxiaozhao.service.impl.PdiUseDemoServiceImpl.execute(PdiUseDemoServiceImpl.java:41)
at com.renxiaozhao.service.impl.PdiUseDemoServiceImpl.executeJob(PdiUseDemoServiceImpl.java:37)
at com.renxiaozhao.service.impl.PdiUseDemoServiceImpl$$FastClassBySpringCGLIB$$56121f70.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:684)
at com.renxiaozhao.service.impl.PdiUseDemoServiceImpl$$EnhancerBySpringCGLIB$$b8de3a8b.executeJob(<generated>)
at com.renxiaozhao.api.controller.PdiUseDemoController.executeJob(PdiUseDemoController.java:33)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:189)
at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:138)
at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:102)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:895)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:800)
at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1038)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:942)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1005)
... 47 common frames omitted
Caused by: java.lang.RuntimeException: Database type not found!
at org.pentaho.di.core.database.DatabaseMeta.setValues(DatabaseMeta.java:642)
at org.pentaho.di.core.database.DatabaseMeta.setDefault(DatabaseMeta.java:525)
at org.pentaho.di.core.database.DatabaseMeta.<init>(DatabaseMeta.java:516)
at org.pentaho.di.core.database.DatabaseMeta.<init>(DatabaseMeta.java:986)
at org.pentaho.di.trans.TransMeta.loadXML(TransMeta.java:3030)
... 68 common frames omitted
Caused by: org.pentaho.di.core.exception.KettleDatabaseException:
database type with plugin id [Oracle] couldn't be found!
at org.pentaho.di.core.database.DatabaseMeta.findDatabaseInterface(DatabaseMeta.java:592)
at org.pentaho.di.core.database.DatabaseMeta.getDatabaseInterface(DatabaseMeta.java:566)
at org.pentaho.di.core.database.DatabaseMeta.setValues(DatabaseMeta.java:640)
... 72 common frames omitted
❗解决思路一：该插件类型不存在，需要手动制作
检查对应配置文件kettle-database-types.xml，发现配置是存在的，所以这里报错不是类型不存在所致
<database-type id="ORACLE">
<description>Oracle</description>
<classname>org.pentaho.di.core.database.OracleDatabaseMeta</classname>
</database-type>
❗解决思路二：确保json转化的xml文件正确
❗解决思路三：正确加载kettle
现在怀疑就是kettle没有正确加载，但是如何排查，还是一头雾水，有知道的大佬欢迎告知，接下来再排查半小时，没结果就放弃了，大周六的，不要太卷！文章来源：https://www.toymoban.com/news/detail-782292.html
问题解决已更新：Kettle Local引擎使用记录（二）：问题记录及解决文章来源地址https://www.toymoban.com/news/detail-782292.html
到了这里，关于Kettle Local引擎使用记录（一）（基于Kettle web版数据集成开源工具data-integration源码）的文章就介绍完了。如果您还想了解更多内容，请在右上角搜索TOY模板网以前的文章或继续浏览下面的相关文章，希望大家以后多多支持TOY模板网！
