官方 https://github.com/alibaba/DataX 的程序没有打包clickhouse ,要自行下载打包成jar包放在plugin文件夹里,具体用法看官方文档
我只用到oracle 和 mssql 就放两个job.json配置文件:
oracle 导入时要注意编码:mandatoryEncoding 这个参数官网没有例子,我看源码才发现的
我的oracle生产环境是ISO8859-1的,如果不加这个参数导入clickhouse 中文会乱码,原因可能是操作系统是中文。
在windows 下用cmd 运行datax ,要 chcp 65001 才能 显示出不乱码的中文
{
"job": {
"setting": {
"speed": {
"byte": 1048576
},
"errorLimit": {
"record": 0,
"percentage": 0.02
}
},
"content": [
{
"reader": {
"name": "oraclereader",
"parameter": {
"username": "username",
"password": "password",
"mandatoryEncoding":"gb2312",
"connection": [
{
"jdbcUrl": ["jdbc:oracle:thin:@192.168.1.100:1521:orcl"],
"querySql":["select t1 from table where rownum <10 "]
}
],
}
},
"writer": {
"name": "clickhousewriter",
"parameter": {
"username": "default",
"password": "123456",
"column": ["*"],
"connection": [
{
"jdbcUrl": "jdbc:clickhouse://192.168.1.100:8123/default",
"table": ["test"]
}
],
"preSql": [],
"postSql": [],
"batchSize": 65536,
"batchByteSize": 134217728,
"dryRun": false,
"writeMode": "insert"
}
}
}
]
}
}
mssql 这个没有特别的:
{
"job": {
"setting": {
"speed": {
"byte": 1048576
},
"errorLimit": {
"record": 0,
"percentage": 0.02
}
},
"content": [
{
"reader": {
"name": "sqlserverreader",
"parameter": {
"username": "sa",
"password": "123456",
"mandatoryEncoding":"gb2312",
"connection": [
{
"jdbcUrl": ["jdbc:sqlserver://192.168.2.100:1433;DatabaseName=Database"],
"querySql":[" select t2 from table "]
}
]
}
},
"writer": {
"name": "clickhousewriter",
"parameter": {
"username": "default",
"password": "123456",
"column": ["*"],
"connection": [
{
"jdbcUrl": "jdbc:clickhouse://192.168.1.100:8123/default",
"table": [" "]
}
],
"preSql": [],
"postSql": [],
"batchSize": 65536,
"batchByteSize": 134217728,
"dryRun": false,
"writeMode": "insert"
}
}
}
]
}
}