RDBMS Writer

RDBMS Writer plugin supports writing data to traditional RDBMS. This is a generic relational database writer plugin that can support more relational database writing by registering database drivers.

At the same time, RDBMS Writer is also the base class for other relational database writer plugins. The following writer plugins all depend on this plugin:

• Oracle Writer
• MySQL Writer
• PostgreSQL Writer
• ClickHouse Writer
• SQLServer Writer
• Access Writer
• Databend Writer

Note: If a dedicated database writer plugin is already provided, it is recommended to use the dedicated plugin. If the database you need to write to does not have a dedicated plugin, consider using this generic plugin. Before use, you need to perform the following operations to run normally, otherwise exceptions will occur.

Configure Driver

Suppose you need to write data to IBM DB2. Since no dedicated writer plugin is provided, we can use this plugin to implement it. Before use, you need to perform the following two operations:

1. Download the corresponding JDBC driver and copy it to the plugin/writer/rdbmswriter/libs directory
2. Modify the task configuration file, find the driver item, and fill in the correct JDBC driver name, such as DB2's driver name com.ibm.db2.jcc.DB2Driver

The following lists common databases and their corresponding driver names:

• Apache Impala: com.cloudera.impala.jdbc41.Driver
• Enterprise DB: com.edb.Driver
• PrestoDB: com.facebook.presto.jdbc.PrestoDriver
• IBM DB2: com.ibm.db2.jcc.DB2Driver
• MySQL: com.mysql.cj.jdbc.Driver
• Sybase Server: com.sybase.jdbc3.jdbc.SybDriver
• TDengine: com.taosdata.jdbc.TSDBDriver
• 达梦数据库: dm.jdbc.driver.DmDriver
• 星环Inceptor: io.transwarp.jdbc.InceptorDriver
• TrinoDB: io.trino.jdbc.TrinoDriver
• PrestoSQL: io.prestosql.jdbc.PrestoDriver
• Oracle DB: oracle.jdbc.OracleDriver
• PostgreSQL: org.postgresql.Drive

Configuration

Configure a job to write to RDBMS.

{
  "job": {
    "setting": {
      "speed": {
        "channel": 1,
        "bytes": -1
      }
    },
    "content": {
      "reader": {
        "name": "streamreader",
        "parameter": {
          "column": [
            {
              "value": "Addax",
              "type": "string"
            },
            {
              "value": 19880808,
              "type": "long"
            },
            {
              "value": "1988-08-08 08:08:08",
              "type": "date"
            },
            {
              "value": true,
              "type": "bool"
            },
            {
              "value": "test",
              "type": "bytes"
            }
          ],
          "sliceRecordCount": 1000
        }
      },
      "writer": {
        "name": "rdbmswriter",
        "parameter": {
          "username": "username",
          "password": "password",
          "driver": "dm.jdbc.driver.DmDriver",
          "column": [
            "*"
          ],
          "preSql": [
            "delete from XXX;"
          ],
          "connection": {
            "jdbcUrl": "jdbc:dm://ip:port/database",
            "table": [
              "table"
            ]
          }
        }
      }
    }
  }
}

Parameters

This plugin provides configuration for writing to relational databases. For detailed parameter descriptions, please refer to the original RDBMS Writer documentation.