Skip to content

[fix](binlog) Preserve row binlog compaction policy - #68033

Open
luwei16 wants to merge 1 commit into
apache:masterfrom
luwei16:fix/DORIS-28878-binlog-compaction-policy
Open

luwei16 wants to merge 1 commit into
apache:masterfrom
luwei16:fix/DORIS-28878-binlog-compaction-policy

Conversation

@luwei16

@luwei16 luwei16 commented Sep 15, 2026

Copy link
Copy Markdown
Contributor

What problem does this PR solve?

Issue Number: None

Related PR: None

Problem Summary: Changing a cloud table compaction policy sent the same policy to every visible index, including hidden row-binlog tablets. Those tablets require the dedicated binlog policy, so the ALTER persisted and synchronized an incompatible policy. Exclude row-binlog indexes from compaction-policy updates while preserving their inclusion for other tablet metadata updates.

Release note

Preserve the dedicated binlog compaction policy on hidden row-binlog tablets when altering a cloud table compaction policy.

Check List (For Author)

  • Test: Unit Test
    • FE_UT_PARALLEL=1 bash run-fe-ut.sh --run CloudSchemaChangeHandlerTest
  • Behavior changed: Yes. Cloud compaction-policy ALTER no longer updates hidden row-binlog tablets.
  • Does this need documentation: No

### What problem does this PR solve?

Issue Number: None

Related PR: None

Problem Summary: Changing a cloud table compaction policy sent the same policy to every visible index, including hidden row-binlog tablets. Those tablets require the dedicated binlog policy, so the ALTER persisted and synchronized an incompatible policy. Exclude row-binlog indexes from compaction-policy updates while preserving their inclusion for other tablet metadata updates.

### Release note

Preserve the dedicated binlog compaction policy on hidden row-binlog tablets when altering a cloud table compaction policy.

### Check List (For Author)

- Test: Unit Test
    - FE_UT_PARALLEL=1 bash run-fe-ut.sh --run CloudSchemaChangeHandlerTest
- Behavior changed: Yes. Cloud compaction-policy ALTER no longer updates hidden row-binlog tablets.
- Does this need documentation: No
@luwei16
luwei16 requested a review from gavinchou as a code owner September 15, 2026 17:04
@hello-stephen

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@luwei16

luwei16 commented Sep 15, 2026

Copy link
Copy Markdown
Contributor Author

run buildall

@luwei16 luwei16 changed the title [fix](cloud) Preserve row binlog compaction policy [fix](binlog) Preserve row binlog compaction policy Sep 15, 2026
@hello-stephen

Copy link
Copy Markdown
Contributor

FE UT Coverage Report

Increment line coverage 100.00% (2/2) 🎉
Increment coverage report
Complete coverage report

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-H: Total hot run time: 16726 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 7f743dd43879ab49c9f0482e75fc0dfeba56e02e, data reload: false

------ Round 1 ----------------------------------
============================================
q1	17750	3029	3060	3029
q2	2092	259	225	225
q3	10237	875	514	514
q4	4676	283	200	200
q5	7645	570	383	383
q6	132	115	93	93
q7	519	499	385	385
q8	9233	854	833	833
q9	3425	2412	2393	2393
q10	6510	840	685	685
q11	393	194	182	182
q12	619	262	196	196
q13	18118	1508	1156	1156
q14	157	152	139	139
q15	q16	434	391	365	365
q17	1333	909	795	795
q18	3134	2264	2237	2237
q19	1250	902	795	795
q20	373	283	208	208
q21	5624	1684	1839	1684
q22	330	270	229	229
Total cold run time: 93984 ms
Total hot run time: 16726 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	3379	3331	3317	3317
q2	500	381	360	360
q3	2274	2281	2188	2188
q4	1196	1165	892	892
q5	2189	2111	2096	2096
q6	169	118	86	86
q7	1004	945	945	945
q8	1596	1389	1388	1388
q9	3149	3118	3092	3092
q10	1838	1806	1623	1623
q11	359	268	251	251
q12	453	431	342	342
q13	1487	1525	1148	1148
q14	168	172	164	164
q15	q16	396	401	364	364
q17	3658	3279	3247	3247
q18	4797	4425	4785	4425
q19	859	867	897	867
q20	1007	952	844	844
q21	3871	3129	3251	3129
q22	386	357	313	313
Total cold run time: 34735 ms
Total hot run time: 31081 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-DS: Total hot run time: 81821 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 7f743dd43879ab49c9f0482e75fc0dfeba56e02e, data reload: false

query5	4241	410	337	337
query6	412	144	128	128
query7	4916	383	228	228
query8	291	125	113	113
query9	8689	2882	2891	2882
query10	379	215	179	179
query11	5387	1054	915	915
query12	121	70	68	68
query13	1185	453	347	347
query14	6075	2200	2090	2090
query14_1	1986	1961	1974	1961
query15	178	117	107	107
query16	922	387	355	355
query17	1045	455	363	363
query18	2382	332	237	237
query19	163	137	108	108
query20	96	74	73	73
query21	210	104	90	90
query22	5429	5226	5358	5226
query23	6619	6259	6186	6186
query23_1	6052	6186	6218	6186
query24	7272	1103	773	773
query24_1	786	804	801	801
query25	437	296	256	256
query26	1221	246	132	132
query27	2771	406	284	284
query28	4646	1490	1488	1488
query29	907	408	325	325
query30	253	154	133	133
query31	821	393	325	325
query32	142	74	72	72
query33	459	208	179	179
query34	989	816	484	484
query35	390	403	338	338
query36	562	553	539	539
query37	118	80	65	65
query38	1003	841	818	818
query39	506	490	470	470
query39_1	450	459	458	458
query40	202	94	76	76
query41	54	51	54	51
query42	75	72	70	70
query43	249	242	212	212
query44	992	527	528	527
query45	109	111	102	102
query46	756	865	530	530
query47	772	760	716	716
query48	308	308	220	220
query49	554	230	177	177
query50	759	271	193	193
query51	8287	7988	8160	7988
query52	68	72	61	61
query53	191	201	149	149
query54	204	168	157	157
query55	73	57	59	57
query56	220	168	186	168
query57	709	664	651	651
query58	195	184	175	175
query59	1212	1242	1120	1120
query60	250	232	155	155
query61	103	119	106	106
query62	363	202	182	182
query63	171	135	141	135
query64	2814	697	583	583
query65	1603	1636	1588	1588
query66	1905	261	201	201
query67	9634	9890	9610	9610
query68	2928	1240	722	722
query69	347	219	199	199
query70	666	626	615	615
query71	247	166	162	162
query72	2274	1618	1507	1507
query73	665	648	347	347
query74	1971	1235	1120	1120
query75	1161	1101	961	961
query76	2302	708	495	495
query77	252	259	216	216
query78	4051	3581	3138	3138
query79	2344	817	555	555
query80	1601	322	276	276
query81	492	151	136	136
query82	811	136	98	98
query83	305	215	188	188
query84	249	109	90	90
query85	795	328	276	276
query86	408	172	171	171
query87	1024	984	890	890
query88	2771	2104	2102	2102
query89	292	194	175	175
query90	1864	130	127	127
query91	130	118	106	106
query92	78	71	72	71
query93	1455	1091	640	640
query94	660	247	225	225
query95	535	250	301	250
query96	771	583	256	256
query97	1038	1076	1043	1043
query98	158	135	132	132
query99	428	345	304	304
Total cold run time: 177838 ms
Total hot run time: 81821 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
ClickBench: Total hot run time: 14.7 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 7f743dd43879ab49c9f0482e75fc0dfeba56e02e, data reload: false

query1	0.01	0.00	0.01
query2	0.08	0.04	0.03
query3	0.25	0.11	0.11
query4	1.60	0.10	0.10
query5	0.17	0.16	0.16
query6	1.27	0.68	0.69
query7	0.03	0.01	0.00
query8	0.05	0.03	0.03
query9	0.29	0.21	0.22
query10	0.35	0.34	0.35
query11	0.16	0.12	0.11
query12	0.15	0.13	0.11
query13	0.32	0.31	0.31
query14	0.45	0.45	0.45
query15	0.37	0.37	0.36
query16	0.22	0.24	0.24
query17	0.68	0.74	0.65
query18	0.17	0.16	0.17
query19	1.19	1.14	1.17
query20	0.01	0.02	0.02
query21	15.44	0.17	0.11
query22	5.06	0.04	0.04
query23	16.18	0.25	0.10
query24	2.98	0.34	0.27
query25	0.11	0.04	0.04
query26	0.73	0.17	0.14
query27	0.04	0.03	0.03
query28	3.63	0.53	0.27
query29	12.50	3.23	2.57
query30	0.26	0.11	0.12
query31	2.75	0.37	0.18
query32	3.53	0.32	0.24
query33	1.49	1.61	1.40
query34	15.41	2.20	1.77
query35	1.77	1.74	1.72
query36	0.45	0.29	0.28
query37	0.06	0.04	0.04
query38	0.04	0.03	0.02
query39	0.03	0.03	0.02
query40	0.12	0.07	0.07
query41	0.08	0.03	0.02
query42	0.03	0.02	0.03
query43	0.03	0.03	0.03
Total cold run time: 90.54 s
Total hot run time: 14.7 s

@luwei16

luwei16 commented Sep 16, 2026

Copy link
Copy Markdown
Contributor Author

/review

@github-actions

Copy link
Copy Markdown
Contributor

Codex automated review failed and did not complete.

Error: Codex completed, but no new pull request review was submitted for the current head SHA.
Workflow run: https://github.com/apache/doris/actions/runs/35050073912

Please inspect the workflow logs and rerun the review after the underlying issue is resolved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants