Oracle Reader
Oracle Reader 插件用于从 Oracle 读取数据
配置样例
配置一个从Oracle数据库同步抽取数据到本地的作业:
json
{
"job": {
"setting": {
"speed": {
"byte": 1048576,
"channel": 1
}
},
"content": {
"reader": {
"name": "oraclereader",
"parameter": {
"username": "oracle",
"password": "password",
"column": [
"id",
"name"
],
"splitPk": "db_id",
"connection": {
"table": [
"table"
],
"jdbcUrl": "jdbc:oracle:thin:@127.0.0.1:5432/orcl"
}
}
},
"writer": {
"name": "streamwriter",
"parameter": {
"print": true
}
}
}
}
}参数说明
该插件基于 RDBMS Reader 实现，因此可以参考 RDBMS Reader 的所有配置项。
对 GEOMETRY 类型的支持
从 Addax 4.0.13 开始，实验性的支持 Oracle GEOMETRY 类型，该插件会把该类型的数据转为 JSON 数组字符串。
假定你有这样的的表和数据
sql
-- create demo table
CREATE TABLE spatial_data(id int, name varchar2(50), geo SDO_GEOMETRY);
-- insert some records
INSERT INTO HR.spatial_data (id, name, geo) VALUES (
1,
'Point 1',
MDSYS.SDO_GEOMETRY(
2001, -- 二维点类型
NULL,
MDSYS.SDO_POINT_TYPE(1, 1, NULL), -- X, Y 坐标为 1
NULL,
NULL
)
);
INSERT INTO HR.spatial_data (id, name, geo) VALUES (
2,
'Line 1',
MDSYS.SDO_GEOMETRY(
2002, -- 二维线类型
NULL,
NULL,
MDSYS.SDO_ELEM_INFO_ARRAY(1,2,1), -- 表示一个由两个点组成的线段
MDSYS.SDO_ORDINATE_ARRAY(1,1, 2,2) -- 线段的起点为 (1,1)，终点为 (2,2)
)
);
INSERT INTO HR.spatial_data (id, name, geo) VALUES (
3,
'Polygon 1',
MDSYS.SDO_GEOMETRY(
2003, -- 二维多边形类型
NULL,
NULL,
MDSYS.SDO_ELEM_INFO_ARRAY(1,1003,1), -- 表示一个由三个点组成的三角形
MDSYS.SDO_ORDINATE_ARRAY(1,1, 2,2, 3,1) -- 三角形的三个顶点分别为 (1,1)、(2,2) 和 (3,1)
)
);读取表该的数据的最后输出结果类似如下：
txt
___ _ _
/ _ \ | | | |
/ /_\ \ __| | __| | __ ___ __
| _ |/ _` |/ _` |/ _` \ \/ /
| | | | (_| | (_| | (_| |> <
\_| |_/\__,_|\__,_|\__,_/_/\_\
:: Addax version :: (v4.0.13-SNAPSHOT)
2023-04-24 22:45:03.270 [ main] INFO VMInfo - VMInfo# operatingSystem class => sun.management.OperatingSystemImpl
2023-04-24 22:45:03.285 [ main] INFO Engine -
{
"content":{
"reader":{
"name":"oraclereader",
"parameter":{
"column":[
"*"
],
"connection":[
{
"jdbcUrl":[
"jdbc:oracle:thin:@127.0.0.1:1521/XE"
],
"table":[
"HR.spatial_data"
]
}
],
"username":"oracle",
"password":"*****"
}
},
"writer":{
"name":"streamwriter",
"parameter":{
"print":true
}
}
},
"setting":{
"speed":{
"bytes":-1,
"channel":1
}
}
}
2023-04-24 22:45:03.305 [ main] INFO PerfTrace - PerfTrace traceId=job_-1, isEnable=false
2023-04-24 22:45:03.305 [ main] INFO JobContainer - Addax jobContainer starts job.
2023-04-24 22:45:03.306 [ main] INFO JobContainer - Set jobId = 0
2023-04-24 22:45:04.540 [ job-0] INFO OriginalConfPretreatmentUtil - Available jdbcUrl:jdbc:oracle:thin:@127.0.0.1:1521/XE.
2023-04-24 22:45:04.541 [ job-0] WARN OriginalConfPretreatmentUtil - 您的配置文件中的列配置存在一定的风险. 因为您未配置读取数据库表的列，当您的表字段个数、类型有变动时，可能影响任务正确性甚至会运行出错。请检查您的配置并作出修改.
2023-04-24 22:45:04.545 [ job-0] INFO JobContainer - Addax Reader.Job [oraclereader] do prepare work .
2023-04-24 22:45:04.546 [ job-0] INFO JobContainer - Addax Writer.Job [streamwriter] do prepare work .
2023-04-24 22:45:04.546 [ job-0] INFO JobContainer - Job set Channel-Number to 1 channel(s).
2023-04-24 22:45:04.548 [ job-0] INFO JobContainer - Addax Reader.Job [oraclereader] splits to [1] tasks.
2023-04-24 22:45:04.548 [ job-0] INFO JobContainer - Addax Writer.Job [streamwriter] splits to [1] tasks.
2023-04-24 22:45:04.569 [ job-0] INFO JobContainer - Scheduler starts [1] taskGroups.
2023-04-24 22:45:04.577 [ taskGroup-0] INFO TaskGroupContainer - taskGroupId=[0] start [1] channels for [1] tasks.
2023-04-24 22:45:04.579 [ taskGroup-0] INFO Channel - Channel set byte_speed_limit to -1, No bps activated.
2023-04-24 22:45:04.580 [ taskGroup-0] INFO Channel - Channel set record_speed_limit to -1, No tps activated.
2023-04-24 22:45:04.593 [0-0-0-reader] INFO CommonRdbmsReader$Task - Begin to read record by Sql: [select * from HR.spatial_data
] jdbcUrl:[jdbc:oracle:thin:@10.90.70.11:13521/XE].
2023-04-24 22:45:05.561 [0-0-0-reader] INFO CommonRdbmsReader$Task - Finished read record by Sql: [select * from HR.spatial_data
] jdbcUrl:[jdbc:oracle:thin:@10.90.70.11:13521/XE].
1 Point 1 [{"sdo_gtype":2001,"sdo_srid":0,"sdo_point":{"x":1.0,"y":1.0,"z":null}}]
2 Line 1 [{"sdo_gtype":2002,"sdo_srid":0,"sdo_point":{"x":null,"y":null,"z":null},"sdo_elem_info":[1,2,1],"sdo_ordinates":[1.0,1.0,2.0,2.0]}]
3 Polygon 1 [{"sdo_gtype":2003,"sdo_srid":0,"sdo_point":{"x":null,"y":null,"z":null},"sdo_elem_info":[1,1003,1],"sdo_ordinates":[1.0,1.0,2.0,2.0,3.0,1.0]}]
2023-04-24 22:45:07.589 [ job-0] INFO AbstractScheduler - Scheduler accomplished all tasks.
2023-04-24 22:45:07.589 [ job-0] INFO JobContainer - Addax Writer.Job [streamwriter] do post work.
2023-04-24 22:45:07.590 [ job-0] INFO JobContainer - Addax Reader.Job [oraclereader] do post work.
2023-04-24 22:45:07.592 [ job-0] INFO JobContainer - PerfTrace not enable!
2023-04-24 22:45:07.593 [ job-0] INFO StandAloneJobContainerCommunicator - Total 3 records, 372 bytes | Speed 124B/s, 1 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 0.000s | Percentage 100.00%
2023-04-24 22:45:07.594 [ job-0] INFO JobContainer -
任务启动时刻 : 2023-04-24 22:45:03
任务结束时刻 : 2023-04-24 22:45:07
任务总计耗时 : 4s
任务平均流量 : 124B/s
记录写入速度 : 1rec/s
读出记录总数 : 3
读写失败总数 : 0注意：该数据类型目前还处于实验支持阶段，作者对此数据类型的理解并不深刻，也未经过全面的测试，请勿直接在生产环境使用。