如何用ABAP代码的方式在短时间内批量生成大量订单数据用于性能测试

Package: CRMS4_TEST_DATA_GENERATION - Carsten's original report

(1) API: BAPI_MATERIAL_SAVEREPLICA
(2) BP report: CRMS4_HOME_CREATE_BPS
(3) Service Order: CRMS4_HOME_CREATE_ORDERS

涉及到的一些数据库表

(1) TVKOV: Org. Unit: Distribution Channels per Sales Organization (1) VKORG: Sales Organization (2) VTWEG: Distribution Channel

(2) T001: company code (1) BUKRS: SELECT FROM t001 FOR ALL ENTRIES IN lt_tvko WHERE bukrs = lt_tvko-bukrs (2) LAND1: country key

(3) TVKO: Organizational Unit: Sales Organizations (1) VKORG: Sales Organization (2) include SI_TVKO: Organizational Unit: Sales Organizations detail (3) bukrs: Company code of the sales organization

(4) T134: Material type (1) prod_type_code: determine material or service - 02

(5) TVTA: Organizational Unit: Sales Area(s)

(6) T137: Industries for materials

(7) TSTL: Taxes: Valid Tax Categories for Each Country (1) TALND: Country is relevant for taxes (2) TATYP: Tax category (sales tax, federal sales tax,...) char4

(8) MATERIAL_MAINTAIN_DARK (1) MAKT: long description (2) MARC: Plant Data for Material (3) MARD: Storage Location Data for Material (4) MFHM: Production Resource Tool (PRT) Fields in the Material Master (5) MARM: Units of Measure for Material (6) MEAN: International Article Numbers (EANs) for Material (7) MBEW: Material Valuation (8) STEU: For Taxes: Data Transfer

single order creation touched table:

(1) CRMS4D_BTX_H: shadow table containing guid, BUS type, object_id and process_type

(2) CRMS4D_SERV_H: Header of service transactions, a collection of many include structure for each extension / set

(3) CRMS4D_SCHEDLIN: Scheduled Lines Table

(4) CRMS4D_BTX_I: item shadow table

(5) CRMS4D_PARTNER: Customer Mgmt S4 DB Table for Business Partner Information

(6) CRMS4D_SERV_I: Items of service transactions

Initial performance data QGP/504

(1) Create single service order with single line item: 0.5 ~ 1 seconds

(2) Create single service order with random number of line item between 1 and 30: 2 ~ 3 seconds

(3) Create 100 service order with single line item: 16 seconds

(4) Create 100 service order with random number of line item between 1 and 30: 150 ~ 156 seconds

2018-01-15 Sync after new year holiday

Latest data size till 2018-01-15:

如何用ABAP代码的方式在短时间内批量生成大量订单数据用于性能测试

Step1: eliminate out-of-memory runtime error

In the very beginning my mass data generation report could not be running for a long time, because as time goes by, the allocated memory increases and finally ends up with an "out-of-memory" runtime dump.

如何用ABAP代码的方式在短时间内批量生成大量订单数据用于性能测试
如何用ABAP代码的方式在短时间内批量生成大量订单数据用于性能测试
For example, I have totally 1000 service order to create, and the package size is 100. When I have finished the creation for the first 900 service orders, the internal buffer for these 900 service orders should be cleaned up, or else there might be no enough memory for the last 100 order creation.
The internal buffer needs to be cleanup includes:

(1) product API buffer
(2) organization unit buffer
(3) partner buffer
(4) pricing buffer
(5) one order API buffer

One example of pricing buffer could be found below:

如何用ABAP代码的方式在短时间内批量生成大量订单数据用于性能测试
It holds some internal buffer which will not be released so as time goes by, these two buffer table turns bigger and bigger and finally eat up all memory.

如何用ABAP代码的方式在短时间内批量生成大量订单数据用于性能测试
Unfortunately the pricing API does not have any method currently to cleanup these two buffer table, so I have to use a workaround in QGP:

buffer cleanup:

如何用ABAP代码的方式在短时间内批量生成大量订单数据用于性能测试
Step2: figure out where the drop down list entries for Subject come from?

如何用ABAP代码的方式在短时间内批量生成大量订单数据用于性能测试

Step3: figure out subject data storage design
Suppose a given Service request has maintained this kind of category data, what exactly data is stored in database table?

如何用ABAP代码的方式在短时间内批量生成大量订单数据用于性能测试

2018-01-30 A very draft performance measurement

Scenario: search by header description
REPORT zorder_query.
PARAMETERS: descr TYPE crmd_orderadm_h-description OBLIGATORY DEFAULT 'Jerry',

        conta  TYPE char1 AS CHECKBOX DEFAULT abap_false,
        ttype  Type crmd_orderadm_h-process_type DEFAULT 'SRVO',
        ctype  type char1 AS CHECKBOX DEFAULT abap_false.

DATA: lt_selection_parameter TYPE genilt_selection_parameter_tab,

  ls_query_parameters    TYPE genilt_query_parameters,
  ls_selection_parameter TYPE genilt_selection_parameter.

DATA(lo_core) = cl_crm_bol_core=>get_instance( ).
lo_core->load_component_set( 'ONEORDER' ).

IF conta = abap_true.
ls_selection_parameter = VALUE #( attr_name = 'DESCRIPTION' sign = 'I' option = 'CP'
low = |{ descr }| ).
ELSE.
ls_selection_parameter = VALUE #( attr_name = 'DESCRIPTION' sign = 'I' option = 'EQ'
low = descr ).
ENDIF.

APPEND ls_selection_parameter TO lt_selection_parameter.

IF ctype = abap_true.
ls_selection_parameter = VALUE #( attr_name = 'PROCESS_TYPE' sign = 'I' option = 'EQ' low = ttype ).
APPEND ls_selection_parameter TO lt_selection_parameter.
ENDIF.

ls_query_parameters-max_hits = 100.

cl_crm_order_timer_home=>start( ).
TRY.

DATA(lo_collection) = lo_core->dquery(
    iv_query_name               = 'BTQSrvOrd'
    it_selection_parameters            = lt_selection_parameter
    is_query_parameters                = ls_query_parameters ).

CATCH cx_root INTO DATA(cx_root).

WRITE:/ cx_root->get_text( ).
RETURN.

ENDTRY.
cl_crm_order_timer_home=>stop( 'Search by Description' ).
WRITE:/ |Number of Service Orders found: { lo_collection->size( ) }| COLOR COL_NEGATIVE.

如何用ABAP代码的方式在短时间内批量生成大量订单数据用于性能测试

Result

around 16 seconds.
total records which match this criteria in DB: 65,660

如何用ABAP代码的方式在短时间内批量生成大量订单数据用于性能测试
DB data volume:

如何用ABAP代码的方式在短时间内批量生成大量订单数据用于性能测试
7.86 seconds for CDS view query:

如何用ABAP代码的方式在短时间内批量生成大量订单数据用于性能测试
如何用ABAP代码的方式在短时间内批量生成大量订单数据用于性能测试
CDS view: CRMS4_SERV_H_SEARCH

如何用ABAP代码的方式在短时间内批量生成大量订单数据用于性能测试
如何用ABAP代码的方式在短时间内批量生成大量订单数据用于性能测试
如何用ABAP代码的方式在短时间内批量生成大量订单数据用于性能测试
如何用ABAP代码的方式在短时间内批量生成大量订单数据用于性能测试

Test report used by Jerry in year 2017 for One order prototype performance test
You can find those report in QGS:ZCRMS4_ORDER_SEARCH_PARTY_NAME
Search service orders which contain specified business partner ID with given partner function

如何用ABAP代码的方式在短时间内批量生成大量订单数据用于性能测试

ZCRMS4_ORDER_SEARCH_PRODUCT_ID
Search service order which contain the line item product with given product id.

如何用ABAP代码的方式在短时间内批量生成大量订单数据用于性能测试

ZCRMS4_ORDER_SEARCH_PROD_SOLD
Search with combination of sold to party name and line item product id.

如何用ABAP代码的方式在短时间内批量生成大量订单数据用于性能测试

ZCRMS4_ORDER_SEARCH_SOLDTO_NAM
search by sold to party name.

如何用ABAP代码的方式在短时间内批量生成大量订单数据用于性能测试
Jerry will copy them to QGP soon.
Tip: Do SAT trace on applications which could not be launched by SAT

Extensible CDS view done by China team for reference
package and CDS view listed below:

I quickly go through your view CRMS4_SERV_H_SEARCH and it seems fine for me. I am not clear why it does not work for extensibility.

Please read this article knowledge article written by Jerry on 2016-12-8 to check whether it helps. Sorry for Chinese because it is written for colleagues in China. You can try to use Google translation.

Search criteria supported in prototype done in year 2017, for your reference

check area under BUS2000116 (CRMST_QUERY_SRVO_BTIL).

(1) The cell with green means those criteria is supported in our prototype.
(2) The cell with silver means in theory they will be very easy to be supported.
(3) The cell with red means they are not supported in prototype due to complexity.

如何用ABAP代码的方式在短时间内批量生成大量订单数据用于性能测试

2018-02-05 Sync about Order Header description field search performance issue

What Jerry has learned from HANA DB colleague

When I paste it and execute in HANA studio, it is executed statically. On the other hand when I execute my ABAP report, the SQL statement is executed dynamically. The word dynamically means the search parameter for SQL is not known by HANA engine until runtime.

如何用ABAP代码的方式在短时间内批量生成大量订单数据用于性能测试
and paste the statement arguments into HANA studio as well:
504,BUS2000116,BUS2000116,BUS2000140,BUS2000105,BUS2000137,BUS2000131,BUS2000146,BUS2000159,BUS2000153,BUS2000199,,Y,,%2017-12-21%,SRVO,100

如何用ABAP代码的方式在短时间内批量生成大量订单数据用于性能测试
this time we get the consistent performance result executed in HANA studio and in ABAP:

如何用ABAP代码的方式在短时间内批量生成大量订单数据用于性能测试

approach 1
What is SAP HANA Hint?
From SAP note 2142945
Hints in the context of this SAP Note are instructions for the SAP HANA database server.
Hints can influence the way how a database request is generated or processed.
They don't change the result set of the database requests.
Hints are typically used in order to optimize SAP HANA performance or memory consumption. They may be used temporarily for testing purposes or permanently.
before hint is applied

如何用ABAP代码的方式在短时间内批量生成大量订单数据用于性能测试
如何用ABAP代码的方式在短时间内批量生成大量订单数据用于性能测试
after hint is applied

如何用ABAP代码的方式在短时间内批量生成大量订单数据用于性能测试
Less than one second

如何用ABAP代码的方式在短时间内批量生成大量订单数据用于性能测试
What is hint no_cs_join?

如何用ABAP代码的方式在短时间内批量生成大量订单数据用于性能测试
如何用ABAP代码的方式在短时间内批量生成大量订单数据用于性能测试
Which problems and pitfalls exist in the area of hints?

如何用ABAP代码的方式在短时间内批量生成大量订单数据用于性能测试
Behavior in case of invalid hints

如何用ABAP代码的方式在短时间内批量生成大量订单数据用于性能测试
Fuzzy search in S4 Material search implementation

如何用ABAP代码的方式在短时间内批量生成大量订单数据用于性能测试
consumption view:

如何用ABAP代码的方式在短时间内批量生成大量订单数据用于性能测试
hint is also used in SADL query implementation.

如何用ABAP代码的方式在短时间内批量生成大量订单数据用于性能测试
如何用ABAP代码的方式在短时间内批量生成大量订单数据用于性能测试
incident 1880139488 is created for "SAP DBTech JDBC: [258]: insufficient privilege: Not authorized".
To discard Performance Project incidents from Rel1 Monitoring we approved in our today Product team meeting to enter in “Info” field of the incident: “S4CRM-200” please update this incident accordingly.

2018-02-13 sync meeting
Some test data:
product id: SANSUMGC7PRO000000
description: Jerry 2018-02-10 9:35PM
sold to party id: VIP000001
sold to past mc name1: Ali website 1 the involved CDS view is determined based on object type:

如何用ABAP代码的方式在短时间内批量生成大量订单数据用于性能测试
如何用ABAP代码的方式在短时间内批量生成大量订单数据用于性能测试
如何用ABAP代码的方式在短时间内批量生成大量订单数据用于性能测试
Search by product id

select from:
CRMS4_SERVHSRCH INNER JOIN CRMS4_SERVISRCH ON CRMS4_SERVISRCH~OBJECT_ID = CRMS4_SERVHSRCH~OBJECT_ID
where statement:

如何用ABAP代码的方式在短时间内批量生成大量订单数据用于性能测试
performance: 1.54 seconds

如何用ABAP代码的方式在短时间内批量生成大量订单数据用于性能测试
When I change to use another product id: ANDROID0000000001, Performance turns poor:

如何用ABAP代码的方式在短时间内批量生成大量订单数据用于性能测试
如何用ABAP代码的方式在短时间内批量生成大量订单数据用于性能测试
but only 0.2 second is spent on CDS view fetch?

如何用ABAP代码的方式在短时间内批量生成大量订单数据用于性能测试
Search by sold to party ID
select from:
CRMS4_SERVHSRCH INNER JOIN CRMS4D_PARTNER ON CRMS4D_PARTNER~OBJECT_ID = CRMS4_SERVHSRCH~OBJECT_ID

如何用ABAP代码的方式在短时间内批量生成大量订单数据用于性能测试
如何用ABAP代码的方式在短时间内批量生成大量订单数据用于性能测试
performance: 1.47 seconds

如何用ABAP代码的方式在短时间内批量生成大量订单数据用于性能测试
search by sold to party name

如何用ABAP代码的方式在短时间内批量生成大量订单数据用于性能测试
如何用ABAP代码的方式在短时间内批量生成大量订单数据用于性能测试
如何用ABAP代码的方式在短时间内批量生成大量订单数据用于性能测试
如何用ABAP代码的方式在短时间内批量生成大量订单数据用于性能测试
The second where - CRMS4_SERVHSRCH:

如何用ABAP代码的方式在短时间内批量生成大量订单数据用于性能测试
如何用ABAP代码的方式在短时间内批量生成大量订单数据用于性能测试
The third where - CRMS4_SERVISRCH:

如何用ABAP代码的方式在短时间内批量生成大量订单数据用于性能测试
One question here:
I have specified "Wang" as sold to party name, however according to the above screenshot, another String "HYBRIS" is used to fire the CDS view search.

如何用ABAP代码的方式在短时间内批量生成大量订单数据用于性能测试
till now the value is still "Wang":

如何用ABAP代码的方式在短时间内批量生成大量订单数据用于性能测试
如何用ABAP代码的方式在短时间内批量生成大量订单数据用于性能测试
如何用ABAP代码的方式在短时间内批量生成大量订单数据用于性能测试
如何用ABAP代码的方式在短时间内批量生成大量订单数据用于性能测试
pay attention to the parameter: iv_mc_name1

如何用ABAP代码的方式在短时间内批量生成大量订单数据用于性能测试
如何用ABAP代码的方式在短时间内批量生成大量订单数据用于性能测试
So HYBRIS is partner ID: finally the specified SOLD_TO_PARTY_NAME is converted to SOLD_TO_PARTY(ID) and searched against CRMS4_SERVISRCH.
performance: 1.28 seconds

如何用ABAP代码的方式在短时间内批量生成大量订单数据用于性能测试
Question from Harish

如何用ABAP代码的方式在短时间内批量生成大量订单数据用于性能测试
如何用ABAP代码的方式在短时间内批量生成大量订单数据用于性能测试
only 34 seconds are spent on DB?

如何用ABAP代码的方式在短时间内批量生成大量订单数据用于性能测试
如何用ABAP代码的方式在短时间内批量生成大量订单数据用于性能测试

For example, why crms4d_btx_h is called so many times?

如何用ABAP代码的方式在短时间内批量生成大量订单数据用于性能测试
check whether this get_category is really necessary if it is not made visible in UI configuration:

如何用ABAP代码的方式在短时间内批量生成大量订单数据用于性能测试
如何用ABAP代码的方式在短时间内批量生成大量订单数据用于性能测试

本文来自云栖社区合作伙伴“汪子熙”,了解相关信息可以关注微信公众号"汪子熙"。

上一篇:来吧,自己动手撸一个分布式ID生成器组件(下)


下一篇:Java利用hanlp完成语句相似度分析的案例详解