Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[improvement](log)Reduce stats log. #45900

Merged
merged 1 commit into from
Dec 25, 2024
Merged

[improvement](log)Reduce stats log. #45900

merged 1 commit into from
Dec 25, 2024
+3 −1

Conversation

Jibing-Li
Copy link
Contributor

@Jibing-Li Jibing-Li commented Dec 25, 2024 •
edited
Loading

What problem does this PR solve?

Reduce unnecessary stats log.

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen
Copy link
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@Jibing-Li Jibing-Li marked this pull request as ready for review December 25, 2024 02:34
@Jibing-Li
Copy link
Contributor Author

run buildall

1 similar comment
@Jibing-Li
Copy link
Contributor Author

run buildall

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Dec 25, 2024
@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions GitHub Actions
Copy link
Contributor

PR approved by anyone and no changes requested.

zfr9527
zfr9527 approved these changes Dec 25, 2024
View reviewed changes
Copy link
Contributor

@zfr9527 zfr9527 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@doris-robot
Copy link

TPC-H: Total hot run time: 33047 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 881bd25d5c307bb821ad93eb8bdf92d20518c2ee, data reload: false

------ Round 1 ----------------------------------
q1	17589	6292	6192	6192
q2	2050	327	176	176
q3	10385	1245	737	737
q4	10213	897	448	448
q5	7551	2248	2021	2021
q6	224	183	145	145
q7	900	748	626	626
q8	9239	1492	1301	1301
q9	5332	5076	4987	4987
q10	6853	2335	1848	1848
q11	484	281	253	253
q12	357	381	215	215
q13	17808	3869	2939	2939
q14	244	236	209	209
q15	628	511	504	504
q16	637	631	613	613
q17	581	885	319	319
q18	7095	6621	6392	6392
q19	2200	1051	597	597
q20	311	326	187	187
q21	2941	2308	2023	2023
q22	364	332	315	315
Total cold run time: 103986 ms
Total hot run time: 33047 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6516	6378	6375	6375
q2	252	335	244	244
q3	2248	2625	2295	2295
q4	1443	1865	1379	1379
q5	4358	4837	4894	4837
q6	198	177	143	143
q7	2170	1967	1832	1832
q8	2736	2814	2821	2814
q9	7356	7339	7348	7339
q10	3113	3326	2741	2741
q11	578	517	505	505
q12	730	740	629	629
q13	3410	3839	3116	3116
q14	284	298	297	297
q15	584	507	511	507
q16	664	733	646	646
q17	1275	1760	1254	1254
q18	7808	7403	7336	7336
q19	892	1195	1268	1195
q20	1987	2073	1961	1961
q21	5761	5341	4887	4887
q22	633	609	600	600
Total cold run time: 54996 ms
Total hot run time: 52932 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 196903 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 881bd25d5c307bb821ad93eb8bdf92d20518c2ee, data reload: false

query1	1314	946	959	946
query2	6522	2412	2342	2342
query3	11106	4899	4915	4899
query4	33098	23895	23547	23547
query5	4278	604	473	473
query6	280	198	214	198
query7	3983	497	301	301
query8	306	267	232	232
query9	9697	2757	2740	2740
query10	445	321	251	251
query11	17727	15281	15115	15115
query12	158	107	108	107
query13	1587	544	420	420
query14	9959	6865	7502	6865
query15	252	206	192	192
query16	8281	619	463	463
query17	1559	792	604	604
query18	2185	437	309	309
query19	217	179	159	159
query20	128	121	112	112
query21	211	136	108	108
query22	4765	4670	4485	4485
query23	35112	33458	33535	33458
query24	6556	2273	2298	2273
query25	465	472	387	387
query26	750	248	155	155
query27	2272	469	335	335
query28	5489	2494	2479	2479
query29	558	520	409	409
query30	208	191	154	154
query31	986	947	842	842
query32	82	60	68	60
query33	506	364	311	311
query34	775	865	536	536
query35	804	839	746	746
query36	1046	1049	972	972
query37	121	96	73	73
query38	4329	4169	4328	4169
query39	1526	1473	1451	1451
query40	205	121	100	100
query41	45	45	43	43
query42	123	101	102	101
query43	521	532	503	503
query44	1391	815	829	815
query45	184	179	174	174
query46	897	1059	659	659
query47	2014	2027	1920	1920
query48	396	407	335	335
query49	758	488	389	389
query50	629	740	399	399
query51	7356	7426	7319	7319
query52	97	99	88	88
query53	226	275	197	197
query54	488	512	444	444
query55	86	77	79	77
query56	261	256	233	233
query57	1232	1231	1167	1167
query58	236	236	226	226
query59	3222	3370	3084	3084
query60	276	253	244	244
query61	109	110	114	110
query62	889	814	764	764
query63	234	194	195	194
query64	3538	1066	657	657
query65	3314	3207	3297	3207
query66	856	424	310	310
query67	16247	15879	15590	15590
query68	9853	748	504	504
query69	487	308	257	257
query70	1216	1101	1151	1101
query71	445	284	249	249
query72	6249	3866	3859	3859
query73	660	739	366	366
query74	10327	9266	9133	9133
query75	4638	3136	2622	2622
query76	5426	1180	774	774
query77	1016	352	270	270
query78	10235	10238	9547	9547
query79	4628	852	575	575
query80	681	513	542	513
query81	478	274	230	230
query82	216	152	119	119
query83	203	163	139	139
query84	282	87	68	68
query85	752	376	306	306
query86	356	336	340	336
query87	4493	4401	4666	4401
query88	4044	2259	2219	2219
query89	430	318	287	287
query90	2125	192	189	189
query91	133	142	106	106
query92	66	57	53	53
query93	2779	829	514	514
query94	659	407	298	298
query95	329	269	250	250
query96	491	606	278	278
query97	2712	2847	2689	2689
query98	220	201	201	201
query99	1678	1616	1423	1423
Total cold run time: 303780 ms
Total hot run time: 196903 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.73 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 881bd25d5c307bb821ad93eb8bdf92d20518c2ee, data reload: false

query1	0.03	0.03	0.02
query2	0.07	0.04	0.03
query3	0.24	0.06	0.07
query4	1.63	0.10	0.10
query5	0.40	0.41	0.41
query6	1.19	0.65	0.65
query7	0.02	0.02	0.02
query8	0.04	0.03	0.03
query9	0.59	0.51	0.50
query10	0.54	0.56	0.54
query11	0.15	0.10	0.10
query12	0.14	0.12	0.11
query13	0.60	0.61	0.60
query14	2.84	2.85	2.71
query15	0.88	0.83	0.83
query16	0.39	0.38	0.38
query17	1.02	0.99	1.02
query18	0.24	0.21	0.20
query19	1.93	1.87	1.98
query20	0.01	0.01	0.02
query21	15.38	0.92	0.57
query22	0.76	0.81	0.59
query23	15.33	1.40	0.61
query24	3.34	0.42	1.83
query25	0.28	0.18	0.07
query26	0.28	0.14	0.14
query27	0.06	0.07	0.05
query28	13.72	1.50	1.04
query29	12.56	3.85	3.24
query30	0.25	0.09	0.07
query31	2.83	0.59	0.38
query32	3.23	0.56	0.46
query33	3.09	3.16	3.14
query34	16.67	5.17	4.51
query35	4.54	4.49	4.52
query36	0.65	0.49	0.48
query37	0.10	0.06	0.06
query38	0.05	0.04	0.04
query39	0.04	0.02	0.03
query40	0.16	0.13	0.13
query41	0.08	0.02	0.02
query42	0.04	0.03	0.02
query43	0.04	0.03	0.02
Total cold run time: 106.43 s
Total hot run time: 30.73 s

@Jibing-Li Jibing-Li merged commit 18a086e into apache:master Dec 25, 2024
30 of 32 checks passed
github-actions bot pushed a commit that referenced this pull request Dec 25, 2024
@Jibing-Li
[improvement](log)Reduce stats log. (#45900)
97916dc 
### What problem does this PR solve?

Reduce unnecessary stats log.

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

### Release note

None
@github-actions github-actions bot mentioned this pull request Dec 25, 2024
Merged
github-actions bot pushed a commit that referenced this pull request Dec 25, 2024
@Jibing-Li
[improvement](log)Reduce stats log. (#45900)
f7a5531 
### What problem does this PR solve?

Reduce unnecessary stats log.

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

### Release note

None
@github-actions github-actions bot mentioned this pull request Dec 25, 2024
Merged
Jibing-Li added a commit that referenced this pull request Dec 25, 2024
@github-actions @Jibing-Li
branch-3.0: [improvement](log)Reduce stats log. #45900 (#45915)
7470fbe 
Cherry-picked from #45900

Co-authored-by: James <[email protected]>
Jibing-Li added a commit that referenced this pull request Dec 25, 2024
@github-actions @Jibing-Li
branch-2.1: [improvement](log)Reduce stats log. #45900 (#45916)
a3ec862 
Cherry-picked from #45900

Co-authored-by: James <[email protected]>
@Jibing-Li Jibing-Li deleted the reducelog branch December 25, 2024 15:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@morrySnow morrySnow morrySnow approved these changes

@zfr9527 zfr9527 zfr9527 approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/2.1.8-merged dev/3.0.4-merged reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@Jibing-Li @hello-stephen @doris-robot @zfr9527 @morrySnow