Email收到了这样的报错：
2016-12-07 02:52:22 WARNING OGG-01004 Aborted grouped transaction on 'MSP.USER_ACTIVITY_LIFETIME_AGG', Database error 1403 (OCI Error ORA-01403: no data found, SQL <UPDATE "MSP"."USER_ACTIVITY_LIFETIME_AGG" x SET x."USER_ACTIVITY_LIFETIME_ID" = :a0,x."NIKE_PLUS_USER_ID" = :a1,x."ACTIVITY_NUM" = :a3,x."FUEL_TOTAL" = :a4,x."STEP_TOTAL" = :a5,x."CALORIE_TOTAL" = :a6,x."DURATION_TOTAL" = :a7,x."DISTANCE_TOTAL" = :a8,x."WALK_TOTAL" = :a9,x."FUEL_ACTIVITY_NUM" = :a10,x."FUEL_DAILY_AVG" = :a11,x."BEST_ACTIVE_HOUR_START" = :a12,x."BEST_ACTIVE_MINUTE_START" = :a13,x."BEST_ACTIVE_HOUR_END" = :a14,x."BEST_ACTIVE_MINUTE_END" = :a15,x."FUEL_BEST_ACTIVE_HOUR" = :a16,x."FUEL_BEST_DAY" = :a17,x."FUEL_BEST_DAY_AMT" = :a18,x."FUEL_BEST_WEEK" = :a19,x."FUEL_BEST_WEEK_AMT" = :a20,x."FUEL_BEST_MONTH" = :a21,x."FUEL_BEST_MONTH_AMT" = :a22,x."CREATE_DATE" = :a23,x."UPDATE_DATE" = :a24,x."FUEL_BEST_HOUR_AMT" = :a25,x."CURRENT_STREAK" = :a27,x."CURRENT_STREAK_START_DATE" = :a28,x."MISS_STREAK" = :a29,x."MISS_STREAK_START_DATE" = :a30,x."LONGEST_STREAK" = :a31,x."LONGEST_STREAK_START_DATE" = :a32,x."LONGEST_MISS_STREAK" = :a33,x."LONGEST_MISS_STREAK_START_DATE" = :a34,x."DEVICE_ID" = :a35,x."FUEL_BEST_SUNDAY" = :a36,x."FUEL_BEST_SUNDAY_AMT" = :a37,x."FUEL_BEST_MONDAY" = :a38,x."FUEL_BEST_MONDAY_AMT" = :a39,x."FUEL_BEST_TUESDAY" = :a40,x."FUEL_BEST_TUESDAY_AMT" = :a41,x."FUEL_BEST_WEDNESDAY" = :a42,x."FUEL_BEST_WEDNESDAY_AMT" = :a43,x."FUEL_BEST_THURSDAY" = :a44,x."FUEL_BEST_THURSDAY_AMT" = :a45,x."FUEL_BEST_FRIDAY" = :a46,x."FUEL_BEST_FRIDAY_AMT" = :a47,x."FUEL_BEST_SATURDAY" = :a48,x."FUEL_BEST_SATURDAY_AMT" = :a49,x."ACTIVE_PERCENTAGE" = :a50,x."FUEL_BEST_YEAR" = :a51,x."FUEL_BEST_YEAR_AMT" = :a52,x."SUPER_GOAL_CNT" = :a53,x."TAG_IT_STREAK_CAREER" = :a54,x."TAG_IT_STREAK_CURRENT" = :a55,x."TAG_IT_TOTAL" = :a56,x."TAG_NOT_IT_TOTAL" = :a57,x."TAG_STREAK_CAREER" = :a58,x."TAG_STREAK_CURRENT" = :a59,x."ESP_SCORE_TOTAL" = :a60,x."ACTIVE_TIME_TOTAL" = :a61,x."STEP_CNT" = :a62,x."STEP_CNT_AVG" = :a63,x."STEP_WEEK_AVG" = :a64,x."STEP_CNT_MAX" = :a65,x."STEP_DATE_LONGEST" = :a66,x."FUEL_BEST_CURRENT_WEEK" = :a67,x."FUEL_BEST_CURRENT_WEEK_AMT" = :a68,x."START_DATE" = :a69,x."SMALL_HOURS_COUNT" = :a70,x."LATE_NIGHT_COUNT" = :a71,x."BRIGHT_AND_EARLY_COUNT" = :a72,x."MORNING_COUNT" = :a73,x."EARLY_MORNING_COUNT" = :a74,x."LATE_MORNING_COUNT" = :a75,x."DAYTIME_COUNT" = :a76,x."EARLY_AFTERNOON_COUNT" = :a77,x."LATE_AFTERNOON_COUNT" = :a78,x."EVENING_COUNT" = :a79,x."EARLY_EVENING_COUNT" = :a80,x."EARLY_NIGHT_COUNT" = :a81,x."MOST_CALORIES_BURNED" = :a82,x."LONGEST_ACTIVITY_DURATION" = :a83,x."FUEL_BEST_SESSION" = :a84,x."WORKOUTOFDAY_TOTAL" = :a85,x."SOCIAL_SHARE_TOTAL" = :a86,x."FUEL_BEST_SESSION_DATE" = :a87,x."LAST_ACTIVITY_TZ_OFFSET" = :a88,x."LAST_ACTIVITY_DST_OFFSET" = :a89,x."CALORIE_BEST_WEEK" = :a90,x."CALORIE_BEST_WEEK_AMT" = :a91,x."LONGEST_ACTIVITY_DURATION_DATE" = :a92,x."DURATION_BEST_WEEK" = :a93,x."DURATION_BEST_WEEK_AMT" = :a94,x."INTENSITY_BEST_AMT" = :a95,x."INTENSITY_BEST" = :a96,x."FUEL_SUNDAY_TOTAL" = :a97,x."FUEL_SUNDAY_TOTAL_COUNT" = :a98,x."FUEL_MONDAY_TOTAL" = :a99,x."FUEL_MONDAY_TOTAL_COUNT" = :a100,x."FUEL_TUESDAY_TOTAL" = :a101,x."FUEL_TUESDAY_TOTAL_COUNT" = :a102,x."FUEL_WEDNESDAY_TOTAL" = :a103,x."FUEL_WEDNESDAY_TOTAL_COUNT" = :a104,x."FUEL_THURSDAY_TOTAL" = :a105,x."FUEL_THURSDAY_TOTAL_COUNT" = :a106,x."FUEL_FRIDAY_TOTAL" = :a107,x."FUEL_FRIDAY_TOTAL_COUNT" = :a108,x."FUEL_SATURDAY_TOTAL" = :a109,x."FUEL_SATURDAY_TOTAL_COUNT" = :a110,x."STARS_BEST_DAY_AMT" = :a111,x."STARS_BEST_DAY_DATE" = :a112,x."MOST_FUEL_IN_SESSION" = :a113,x."MOST_FUEL_IN_SESSION_DATE" = :a114 WHERE x."UPM_USER_ID" = :b0 AND x."ACTIVITY_TYPE_ID" = :b1>).
2016-12-07 02:52:28 WARNING OGG-01154 SQL error 1403 mapping MSP.USER_ACTIVITY_LIFETIME_AGG to MSP.USER_ACTIVITY_LIFETIME_AGG OCI Error ORA-01403: no data found, SQL <UPDATE "MSP"."USER_ACTIVITY_LIFETIME_AGG" x SET x."USER_ACTIVITY_LIFETIME_ID" = :a0,x."NIKE_PLUS_USER_ID" = :a1,x."ACTIVITY_NUM" = :a3,x."FUEL_TOTAL" = :a4,x."STEP_TOTAL" = :a5,x."CALORIE_TOTAL" = :a6,x."DURATION_TOTAL" = :a7,x."DISTANCE_TOTAL" = :a8,x."WALK_TOTAL" = :a9,x."FUEL_ACTIVITY_NUM" = :a10,x."FUEL_DAILY_AVG" = :a11,x."BEST_ACTIVE_HOUR_START" = :a12,x."BEST_ACTIVE_MINUTE_START" = :a13,x."BEST_ACTIVE_HOUR_END" = :a14,x."BEST_ACTIVE_MINUTE_END" = :a15,x."FUEL_BEST_ACTIVE_HOUR" = :a16,x."FUEL_BEST_DAY" = :a17,x."FUEL_BEST_DAY_AMT" = :a18,x."FUEL_BEST_WEEK" = :a19,x."FUEL_BEST_WEEK_AMT" = :a20,x."FUEL_BEST_MONTH" = :a21,x."FUEL_BEST_MONTH_AMT" = :a22,x."CREATE_DATE" = :a23,x."UPDATE_DATE" = :a24,x."FUEL_BEST_HOUR_AMT" = :a25,x."CURRENT_STREAK" = :a27,x."CURRENT_STREAK_START_DATE" = :a28,x."MISS_STREAK" = :a29,x."MISS_STREAK_START_DATE" = :a30,x."LONGEST_STREAK" = :a31,x."LONGEST_STREAK_START_DATE" = :a32,x."LONGEST_MISS_STREAK" = :a33,x."LONGEST_MISS_STREAK_START_DATE" = :a34,x."DEVICE_ID" = :a35,x."FUEL_BEST_SUNDAY" = :a36,x."FUEL_BEST_SUNDAY_AMT" = :a37,x."FUEL_BEST_MONDAY" = :a38,x."FUEL_BEST_MONDAY_AMT" = :a39,x."FUEL_BEST_TUESDAY" = :a40,x."FUEL_BEST_TUESDAY_AMT" = :a41,x."FUEL_BEST_WEDNESDAY" = :a42,x."FUEL_BEST_WEDNESDAY_AMT" = :a43,x."FUEL_BEST_THURSDAY" = :a44,x."FUEL_BEST_THURSDAY_AMT" = :a45,x."FUEL_BEST_FRIDAY" = :a46,x."FUEL_BEST_FRIDAY_AMT" = :a47,x."FUEL_BEST_SATURDAY" = :a48,x."FUEL_BEST_SATURDAY_AMT" = :a49,x."ACTIVE_PERCENTAGE" = :a50,x."FUEL_BEST_YEAR" = :a51,x."FUEL_BEST_YEAR_AMT" = :a52,x."SUPER_GOAL_CNT" = :a53,x."TAG_IT_STREAK_CAREER" = :a54,x."TAG_IT_STREAK_CURRENT" = :a55,x."TAG_IT_TOTAL" = :a56,x."TAG_NOT_IT_TOTAL" = :a57,x."TAG_STREAK_CAREER" = :a58,x."TAG_STREAK_CURRENT" = :a59,x."ESP_SCORE_TOTAL" = :a60,x."ACTIVE_TIME_TOTAL" = :a61,x."STEP_CNT" = :a62,x."STEP_CNT_AVG" = :a63,x."STEP_WEEK_AVG" = :a64,x."STEP_CNT_MAX" = :a65,x."STEP_DATE_LONGEST" = :a66,x."FUEL_BEST_CURRENT_WEEK" = :a67,x."FUEL_BEST_CURRENT_WEEK_AMT" = :a68,x."START_DATE" = :a69,x."SMALL_HOURS_COUNT" = :a70,x."LATE_NIGHT_COUNT" = :a71,x."BRIGHT_AND_EARLY_COUNT" = :a72,x."MORNING_COUNT" = :a73,x."EARLY_MORNING_COUNT" = :a74,x."LATE_MORNING_COUNT" = :a75,x."DAYTIME_COUNT" = :a76,x."EARLY_AFTERNOON_COUNT" = :a77,x."LATE_AFTERNOON_COUNT" = :a78,x."EVENING_COUNT" = :a79,x."EARLY_EVENING_COUNT" = :a80,x."EARLY_NIGHT_COUNT" = :a81,x."MOST_CALORIES_BURNED" = :a82,x."LONGEST_ACTIVITY_DURATION" = :a83,x."FUEL_BEST_SESSION" = :a84,x."WORKOUTOFDAY_TOTAL" = :a85,x."SOCIAL_SHARE_TOTAL" = :a86,x."FUEL_BEST_SESSION_DATE" = :a87,x."LAST_ACTIVITY_TZ_OFFSET" = :a88,x."LAST_ACTIVITY_DST_OFFSET" = :a89,x."CALORIE_BEST_WEEK" = :a90,x."CALORIE_BEST_WEEK_AMT" = :a91,x."LONGEST_ACTIVITY_DURATION_DATE" = :a92,x."DURATION_BEST_WEEK" = :a93,x."DURATION_BEST_WEEK_AMT" = :a94,x."INTENSITY_BEST_AMT" = :a95,x."INTENSITY_BEST" = :a96,x."FUEL_SUNDAY_TOTAL" = :a97,x."FUEL_SUNDAY_TOTAL_COUNT" = :a98,x."FUEL_MONDAY_TOTAL" = :a99,x."FUEL_MONDAY_TOTAL_COUNT" = :a100,x."FUEL_TUESDAY_TOTAL" = :a101,x."FUEL_TUESDAY_TOTAL_COUNT" = :a102,x."FUEL_WEDNESDAY_TOTAL" = :a103,x."FUEL_WEDNESDAY_TOTAL_COUNT" = :a104,x."FUEL_THURSDAY_TOTAL" = :a105,x."FUEL_THURSDAY_TOTAL_COUNT" = :a106,x."FUEL_FRIDAY_TOTAL" = :a107,x."FUEL_FRIDAY_TOTAL_COUNT" = :a108,x."FUEL_SATURDAY_TOTAL" = :a109,x."FUEL_SATURDAY_TOTAL_COUNT" = :a110,x."STARS_BEST_DAY_AMT" = :a111,x."STARS_BEST_DAY_DATE" = :a112,x."MOST_FUEL_IN_SESSION" = :a113,x."MOST_FUEL_IN_SESSION_DATE" = :a114 WHERE x."UPM_USER_ID" = :b0 AND x."ACTIVITY_TYPE_ID" = :b1>.
2016-12-07 02:52:28 ERROR OGG-01296 Error mapping from MSP.USER_ACTIVITY_LIFETIME_AGG to MSP.USER_ACTIVITY_LIFETIME_AGG.
2016-12-07 02:52:28 ERROR OGG-01668 PROCESS ABENDING.
大致的看一下就是因为要update的数据没找到，然后abended了
平时这种报错看得太多了，加个HC参数(handlecollision) 然后等lag追上了或者等几分钟把HC参数注掉就是了
今天这个case也不例外
但是自己想去挖掘是哪条/哪几条记录造成update的no data found
先来看看HC参数的影响，
使用HANDLECOLLISIONS的几个场景：
1. target丢失delete记录(missing delete)，忽略该问题并不记录到discardfile
2. target丢失update记录(missing update)
- 更新的键值是主键=》 update转换成INSERT ，默认情况下插入记录不完整
- 更新的键值是非主键=》 忽略该问题并不记录到discardfile
3. 重复插入已存在的主键值到target表中，这将被replicat转换为UPDATE现有主键值的行的其他非主键列
另：该参数仅处理数据本身的Insert/Delete冲突，如果出现两端映射或其它结构性问题Replicat进程依然会abend，不能被忽略
此外对于主键的更新操作，若在target使用HANDLECOLLISIONS且该update丢失，在会转换为INSERT该主键的操作，注意默认情况下插入的记录不完整，FETCHOPTIONS FETCHPKUPDATECOLS将捕获完整的redo image镜像到trail中，这保证把primary key的更新通过HANDLECOLLISIONS转换为对target的一个完整记录的插入。
因为自己一开始遇到这报错，也就先加个HC参数把进程启起来再说了。
事后深入分析嘛---->如下：
看了HC关于update的说法，先看看表主键
SQL> select a.constraint_name, a.column_name
2 from dba_cons_columns a, dba_constraints b
3 where a.constraint_name = b.constraint_name
and b.constraint_type = 'P'
and a.table_name = 'USER_ACTIVITY_LIFETIME_AGG' 4 5 ;
no rows selected
没有主键！！！
那么就是说加了HC之后就忽略这个报错也不记到discard file里面去了。
那么就是说有数据丢失了！那几条数据也就不管了！（关于数据丢失的话其实得看数据库的数据要求严格与否！因为这里的库是个报表库，定期也有数据的refresh，所以对数据的精确度要求没太高）
看看一开始因为什么数据不存在造成的abended
之前自己都是在logdump里面乱看的，完全找不到很明确的record
经过一番的分析，进程abended的时候会记ggserr.log 也会去记discard file或者report file呀
应该去report、discard file里面找找有价值的信息
先把abended以后的info r4f_sp信息贴出来
2016-12-07 02:52:28 WARNING OGG-01154 SQL error 1403 mapping MSP.USER_ACTIVITY_LIFETIME_AGG to MSP.USER_ACTIVITY_LIFETIME_AGG OCI Error ORA-01403: no data found,
2016-12-07 02:52:28 WARNING OGG-01003 Repositioning to rba 709137304 in seqno 6889.
===================================
GGSCI (ora-bi-p-5.va2.b2c.nike.com) 6> info R4F_SP
REPLICAT R4F_SP Last Started 2016-12-03 18:16 Status ABENDED
Checkpoint Lag 00:00:08 (updated 00:05:57 ago)
Log Read Checkpoint File /gg/app/ggadmin/product/12.1.2.1.0/dirdat/replicate/pdsp/r4006889
2016-12-07 02:52:20.002353 RBA 709137304
=========================================
GGSCI (ora-bi-p-5.va2.b2c.nike.com) 29> sh ls -lrt /gg/app/ggadmin/product/12.1.2.1.0/dirdat/replicate/pdsp/r4*
.......（省略）
-rw-r-----. 1 ggadmin oinstall 1023999438 Dec 7 01:04 /gg/app/ggadmin/product/12.1.2.1.0/dirdat/replicate/pdsp/r4006888
-rw-r-----. 1 ggadmin oinstall 820886242 Dec 7 03:11 /gg/app/ggadmin/product/12.1.2.1.0/dirdat/replicate/pdsp/r4006889
==========================================
去report file找找信息：
***********************************************************************
* ** Run Time Statistics ** *
***********************************************************************
Last record for the last committed transaction is the following:
___________________________________________________________________
Trail name : /gg/app/ggadmin/product/12.1.2.1.0/dirdat/replicate/pdsp/r4006889
Hdr-Ind : E (x45) Partition : . (x04)
UndoFlag : . (x00) BeforeAfter: A (x41)
RecLength : 1619 (x0653) IO Time : 2016-12-07 02:52:20.002353
IOType : 15 (x0f) OrigNode : 255 (xff)
TransInd : . (x02) FormatType : R (x52)
SyskeyLen : 0 (x00) Incomplete : . (x00)
AuditRBA : 55760 AuditPos : 2023822924
Continued : N (x00) RecCount : 1 (x01)
2016-12-07 02:52:20.002353 FieldComp Len 1619 RBA 709135576
Name: MSP.USER_ACTIVITY_LIFETIME_AGG
___________________________________________________________________
Reading /gg/app/ggadmin/product/12.1.2.1.0/dirdat/replicate/pdsp/r4006889, current RBA 709153355, 11861808 records
Report at 2016-12-07 02:52:28 (activity since 2016-12-07 00:01:00)
From Table MSP.USER_ACTIVITY_LIFETIME_AGG to MSP.USER_ACTIVITY_LIFETIME_AGG:
# inserts: 6112
# updates: 450184
# deletes: 0
# discards: 1 因为process abended所以会记到discard file了（在未启用HC的情况下）（所以要求我们平时正常运行的goldengate一定不要加上HC参数，实在紧急处理数据问题、abended问题再去用，用完也记得及时注释掉）
Last log location read:
FILE: /gg/app/ggadmin/product/12.1.2.1.0/dirdat/replicate/pdsp/r4006889
SEQNO: 6889
RBA: 709153355（最后一次log读到这里，待会可以用logdump看看这个RBA的位置）
TIMESTAMP: 2016-12-07 02:52:20.002360
EOF: NO
READERR: 0
2016-12-07 02:52:28 ERROR OGG-01668 PROCESS ABENDING.
好的，report file就如上的有用的信息。那么接下来可以看看discard file看看是哪条记录！！！！
ggadmin@ora-bi-p-5:DCBIPRD5:/gg/app/ggadmin/product/12.1.2.1.0/dirrpt$less r4f_sp1.dsc
Oracle GoldenGate Delivery for Oracle process started, group R4F_SP discard file opened: 2016-12-03 18:16:22.461903
Current time: 2016-12-07 02:52:28
Discarded record from action ABEND on error 1403
OCI Error ORA-01403: no data found, SQL <UPDATE "MSP"."USER_ACTIVITY_LIFETIME_AGG" x SET x."USER_ACTIVITY_LIFETIME_ID" = :a0,x."NIKE_PLUS_USER_ID" = :a1,x."ACTIVITY_NUM" = :a3,x."FUEL_TOTAL" = :a4,x."STEP_TOTAL" = :a5,x."CALORIE_TOTAL" = :a6,x."DURATION_TOTAL" = :a7,x."DISTANCE_TOTAL" = :a8,x."WALK_TOTAL" = :a9,x."FUEL_ACTIVITY_NUM" = :a10,x."FUEL_DAILY_AVG" = :a11,x."BEST_ACTIVE_HOUR_START" = :a12,x."BEST_ACTIVE_MINUTE_START" = :a13,x."BEST_ACTIVE_HOUR_END" = :a14,x."BEST_ACTIVE_MINUTE_END" = :a15,x."FUEL_BEST_ACTIVE_HOUR" = :a16,x."FUEL_BEST_DAY" = :a17,x."FUEL_BEST_DAY_AMT" = :a18,x."FUEL_BEST_WEEK" = :a19,x."FUEL_BEST_WEEK_AMT" = :a20,x."FUEL_BEST_MONTH" = :a21,x."FUEL_BEST_MONTH_AMT" = :a22,x."CREATE_DATE" = :a23,x."UPDATE_DATE" = :a24,x."FUEL_BEST_HOUR_AMT" = :a25,x."CURRENT_STREAK" = :a27,x."CURRENT_STREAK_START_DATE" = :a28,x."MISS_STREAK" = :a29,x."MISS_STREAK_START_DATE" = :a30,x."LONGEST_STREAK" = :a31,x."LONGEST_STREAK_START_DATE" = :a32,x."LONGEST_MISS_STREAK" = :a33,x."LONGEST_MISS_STREAK_STAR
Aborting transaction on /gg/app/ggadmin/product/12.1.2.1.0/dirdat/replicate/pdsp/r4 beginning at seqno 6889 rba 709137304
error at seqno 6889 rba 709153355（这个RBA跟之前看report file时候log读到最后一个位置的RBA一样，已经很明显的说明了这个位置开始的记录必定是造成process abended的记录，不要暂时不要着急，慢慢来）
Problem replicating MSP.USER_ACTIVITY_LIFETIME_AGG to MSP.USER_ACTIVITY_LIFETIME_AGG
Record not found
Mapping problem with compressed update record (target format)...
*
USER_ACTIVITY_LIFETIME_ID = fa873844-39c5-4b9f-bee1-9dcbe76ece4e
000000: 66 61 38 37 33 38 34 34 2d 33 39 63 35 2d 34 62 |fa873844-39c5-4b|
000010: 39 66 2d 62 65 65 31 2d 39 64 63 62 65 37 36 65 |9f-bee1-9dcbe76e|
000020: 63 65 34 65 |ce4e |
NIKE_PLUS_USER_ID = 430995F13A1119D6E05330690C0ADA3B
000000: 34 33 30 39 39 35 46 31 33 41 31 31 31 39 44 36 |430995F13A1119D6|
000010: 45 30 35 33 33 30 36 39 30 43 30 41 44 41 33 42 |E05330690C0ADA3B|
UPM_USER_ID = 430995F13A1019D6E05330690C0ADA3B
000000: 34 33 30 39 39 35 46 31 33 41 31 30 31 39 44 36 |430995F13A1019D6|
000010: 45 30 35 33 33 30 36 39 30 43 30 41 44 41 33 42 |E05330690C0ADA3B|
后面的列的信息就不贴了，因为这张表有一百多列
看完这里，我就去查查记录在源库PDSP和目标库BI的数据信息
BI：目标库无记录
SQL> select USER_ACTIVITY_LIFETIME_ID,NIKE_PLUS_USER_ID from msp.USER_ACTIVITY_LIFETIME_AGG where UPM_USER_ID ='430995F13A1019D6E05330690C0ADA3B';
no rows selected
PDSP：源库有三条纪录
SQL> select USER_ACTIVITY_LIFETIME_ID,NIKE_PLUS_USER_ID from msp.USER_ACTIVITY_LIFETIME_AGG where UPM_USER_ID ='430995F13A1019D6E05330690C0ADA3B';
USER_ACTIVITY_LIFETIME_ID NIKE_PLUS_USER_ID
---------------------------------------------------------------------------------------------------------------------------
fa873844-39c5-4b9f-bee1-9dcbe76ece4e 430995F13A1119D6E05330690C0ADA3B
d8174109-5a5c-4ef4-a752-d30370b67f7c 430995F13A1119D6E05330690C0ADA3B
2cc196ae-4959-4d3e-a34d-713310aab95e 430995F13A1119D6E05330690C0ADA3B
看到这里于是明白了，结合这条update语句来看，update语句如下格式
UPDATE "MSP"."USER_ACTIVITY_LIFETIME_AGG" SET 表的各个列=xxxx WHERE "UPM_USER_ID" = :b0 AND "ACTIVITY_TYPE_ID" = :b1>.
但是UPM_USER_ID ='430995F13A1019D6E05330690C0ADA3B';这个在BI里面没记录呀（当然也可以根据ACTIVITY_TYPE_ID来找，或者两个一起找，结果一样的）
可以再去logdump里面看看trail file是不是这样的：
直接pos 709153355到这个位置来看
可以看到从709153355这个位置开始后面三条纪录正好是update所丢失的记录
所以这次的问题分析的很透彻，就是BI里面没这三条记录从而导致update不到造成abended，那么该怎么做呢？
个人觉得可以跳过这三条记录，然后手工insert这三条记录进去。（这三条记录直接往源库里面查下，然后转换成insert语句执行在目标库即可完事）
跳过的方法无非就是HANDLECOLLISION或者REPORT ERROR，不过report error好就好在会把这三条跳过去的记录记到discard file！
关于HC和report error的区别在后面另开一篇可以详细的讲下。
分析到这里，也结束了。
自己当时只是加了HC参数去起起来，然后等lag没了再去把HC注掉。其实这样在数据严格要求的环境下这样是不行的。
但是这里要求不太高。定期也有数据重新refresh的操作。
这里再发点加完HC参数起起来之后的信息状态，也可以佐证之前的分析：
可以看到stats看到的 Total update collisions 3.00
即说明了是三条记录，之前也说了加了HC参数有冲突的数据是不会记到discard file的，我去看了，是没有！
GGSCI (ora-bi-p-5.va2.b2c.nike.com) 34> stats R4F_SP
Sending STATS request to REPLICAT R4F_SP ...
Start of Statistics at 2016-12-07 03:18:16.
Replicating from MSP.USER_ACTIVITY_LIFETIME_AGG to MSP.USER_ACTIVITY_LIFETIME_AGG:
*** Total statistics since 2016-12-07 03:07:15 ***
Total inserts 719.00
Total updates 54583.00
Total deletes 0.00
Total discards 0.00
Total operations 55302.00
Total update collisions 3.00
*** Daily statistics since 2016-12-07 03:07:15 ***
Total inserts 719.00
Total updates 54583.00
Total deletes 0.00
Total discards 0.00
Total operations 55302.00
Total update collisions 3.00
*** Hourly statistics since 2016-12-07 03:07:15 ***
Total inserts 719.00
Total updates 54583.00
Total deletes 0.00
Total discards 0.00
Total operations 55302.00
Total update collisions 3.00
*** Latest statistics since 2016-12-07 03:07:15 ***
Total inserts 719.00
Total updates 54583.00
Total deletes 0.00
Total discards 0.00
Total operations 55302.00
Total update collisions 3.00
End of Statistics.