Castled 源码解析 - connector 模块几个中间表定义

Castled 的connector利用了schema 以及中间表进行数据存储(包含的已经提交的,未提交的)
官方使用了一个属于snapshot(快照),对于数据的处理,官方使用了excep sql 函数,基于不同
时间的snapshot 利用excep 就可以知道数据的变动

几个提供的中间表

主要在ConnectorExecutionConstants 中定义

 
public class ConnectorExecutionConstants {
 
    public static final String CASTLED_CONTAINER = "castled";
    public static final String UNCOMMITTED_SNAPSHOT = "uncommitted_snapshot";
    public static final String COMMITTED_SNAPSHOT = "committed_snapshot";
    public static final String COMMITTED_SNAPSHOT_BACKUP = "committed_snapshot_bkp";
    public static final String FAILED_RECORDS = "failed_records";
 
    public static final Path WAREHOUSE_UNLOAD_DIR_PATH = Paths.get("warehouse_unloads");
    public static final Path FAILURE_RECORDS_DIR = Paths.get("pipeline_failed_records");
    public static final Path APP_UPLOADS_PATH = Paths.get("app_uploads");
   
    public static String getQualifiedCommittedSnapshot(String uuid) {
        return String.format("%s.%s_%s", CASTLED_CONTAINER, uuid, COMMITTED_SNAPSHOT);
    }
 
    public static String getFailedRecordsTable(String uuid) {
        return String.format("%s_%s", uuid, FAILED_RECORDS);
    }
 
    public static String getQualifiedUncommittedSnapshot(String uuid) {
        return String.format("%s.%s_%s", CASTLED_CONTAINER, uuid, UNCOMMITTED_SNAPSHOT);
    }
 
    public static String getCommittedSnapshot(String uuid) {
        return String.format("%s_%s", uuid, COMMITTED_SNAPSHOT);
    }
 
    public static String getUncommittedSnapshot(String uuid) {
        return String.format("%s_%s", uuid, UNCOMMITTED_SNAPSHOT);
    }
 
    public static String getQualifiedCommittedSnapshotBkp(String uuid) {
        return String.format("%s.%s_%s", CASTLED_CONTAINER, uuid, COMMITTED_SNAPSHOT_BACKUP);
    }
 
    public static String getCommittedSnapshotBackup(String uuid) {
        return String.format("%s_%s", uuid, COMMITTED_SNAPSHOT_BACKUP);
    }
 
}

说明

以上几个定义的中间表是比较重要的,如果运行Castled的话,就会看到类似的表在datawarehouse中,而且不同poller就是利用上边的定义进行数据处理的

参考资料

https://github.com/castledio/castled

上一篇:Castled 源码解析 - connector 模块WarehousePollContext 说明


下一篇:渗透测试之CFS三层靶机内网渗透