Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

branch-3.0: [fix](csv reader) fix core dump when parsing csv with enclose #45485 #45888

Open
wants to merge 1 commit into
base: branch-3.0
Choose a base branch
from

Conversation

github-actions[bot]
Copy link
Contributor

Cherry-picked from #45485

@hello-stephen
Copy link
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@dataroaring dataroaring reopened this Dec 25, 2024
@hello-stephen
Copy link
Contributor

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 41250 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 95f004027aa878048a73705d184a4725b1709f22, data reload: false

------ Round 1 ----------------------------------
q1	17982	7506	7381	7381
q2	2060	167	188	167
q3	11152	1177	1197	1177
q4	10701	792	722	722
q5	7852	2882	2850	2850
q6	238	150	155	150
q7	975	643	630	630
q8	10124	1947	2036	1947
q9	6779	6474	6516	6474
q10	7237	2307	2311	2307
q11	476	258	262	258
q12	404	217	220	217
q13	18855	3019	3066	3019
q14	247	218	214	214
q15	568	536	515	515
q16	740	635	611	611
q17	984	548	509	509
q18	7297	6625	6651	6625
q19	3360	1056	1028	1028
q20	480	209	206	206
q21	3980	3237	3300	3237
q22	1130	1006	1027	1006
Total cold run time: 113621 ms
Total hot run time: 41250 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7425	7400	7330	7330
q2	340	245	236	236
q3	3067	2848	2899	2848
q4	2037	1747	1763	1747
q5	5613	5622	5703	5622
q6	227	144	145	144
q7	2158	1768	1761	1761
q8	3340	3351	3370	3351
q9	8875	8851	8719	8719
q10	3555	3471	3501	3471
q11	594	493	496	493
q12	807	622	601	601
q13	16416	3148	3156	3148
q14	303	274	267	267
q15	563	539	533	533
q16	694	659	657	657
q17	1814	1624	1590	1590
q18	8150	7918	7345	7345
q19	3429	1482	1482	1482
q20	2079	1902	1809	1809
q21	5742	5288	5143	5143
q22	1147	1053	1023	1023
Total cold run time: 78375 ms
Total hot run time: 59320 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 195700 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 95f004027aa878048a73705d184a4725b1709f22, data reload: false

query1	1250	926	945	926
query2	6235	2063	1977	1977
query3	10922	4236	4165	4165
query4	66733	28765	23437	23437
query5	5721	445	440	440
query6	501	172	183	172
query7	5974	316	317	316
query8	355	232	237	232
query9	9424	2694	2684	2684
query10	526	282	259	259
query11	17961	15249	15753	15249
query12	173	100	104	100
query13	1567	439	427	427
query14	11414	7635	6591	6591
query15	215	180	182	180
query16	7404	494	548	494
query17	1434	581	600	581
query18	1972	320	318	318
query19	235	158	154	154
query20	127	112	120	112
query21	74	44	44	44
query22	4543	4505	4590	4505
query23	34349	34003	34073	34003
query24	5944	2908	2977	2908
query25	557	425	454	425
query26	696	186	176	176
query27	1706	304	322	304
query28	4293	2524	2524	2524
query29	708	456	469	456
query30	259	163	167	163
query31	1017	833	834	833
query32	63	57	56	56
query33	479	297	285	285
query34	893	497	509	497
query35	835	734	741	734
query36	1087	968	941	941
query37	115	75	71	71
query38	4081	4051	3991	3991
query39	1537	1470	1484	1470
query40	153	86	86	86
query41	48	46	47	46
query42	109	96	96	96
query43	530	495	498	495
query44	1188	816	792	792
query45	183	174	166	166
query46	1148	750	771	750
query47	1968	1899	1863	1863
query48	476	357	391	357
query49	734	394	388	388
query50	856	410	414	410
query51	7236	7131	7154	7131
query52	100	90	89	89
query53	265	185	190	185
query54	552	469	452	452
query55	81	76	75	75
query56	262	236	232	232
query57	1170	1110	1109	1109
query58	215	210	207	207
query59	3157	2836	2914	2836
query60	277	264	251	251
query61	106	109	105	105
query62	764	660	648	648
query63	217	192	192	192
query64	1908	651	647	647
query65	3237	3196	3178	3178
query66	758	303	302	302
query67	15784	15248	15300	15248
query68	4659	547	542	542
query69	475	253	261	253
query70	1151	1129	1090	1090
query71	369	265	256	256
query72	6525	3869	3939	3869
query73	767	341	343	341
query74	9185	8895	8924	8895
query75	3278	2599	2654	2599
query76	2483	1069	1067	1067
query77	470	263	275	263
query78	10902	9739	9517	9517
query79	7942	599	598	598
query80	1873	439	429	429
query81	551	237	246	237
query82	1288	116	116	116
query83	262	143	144	143
query84	297	88	85	85
query85	1700	292	304	292
query86	480	302	288	288
query87	4412	4243	4314	4243
query88	5782	2408	2405	2405
query89	429	300	293	293
query90	2006	183	181	181
query91	179	145	144	144
query92	69	50	52	50
query93	6457	544	530	530
query94	881	300	281	281
query95	351	248	247	247
query96	648	283	282	282
query97	3328	3136	3185	3136
query98	230	204	207	204
query99	1591	1288	1306	1288
Total cold run time: 337881 ms
Total hot run time: 195700 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 33 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 95f004027aa878048a73705d184a4725b1709f22, data reload: false

query1	0.03	0.03	0.03
query2	0.06	0.03	0.03
query3	0.23	0.06	0.06
query4	1.63	0.10	0.11
query5	0.54	0.53	0.54
query6	1.13	0.72	0.73
query7	0.02	0.02	0.01
query8	0.04	0.02	0.05
query9	0.56	0.51	0.50
query10	0.55	0.55	0.56
query11	0.14	0.10	0.11
query12	0.13	0.11	0.11
query13	0.61	0.60	0.60
query14	3.09	3.04	2.93
query15	0.91	0.84	0.83
query16	0.37	0.37	0.39
query17	1.04	1.04	1.04
query18	0.22	0.20	0.21
query19	1.95	1.90	1.83
query20	0.01	0.01	0.01
query21	15.36	0.59	0.60
query22	2.44	3.17	2.12
query23	16.81	1.11	0.81
query24	3.10	0.85	1.20
query25	0.14	0.13	0.14
query26	0.46	0.14	0.14
query27	0.05	0.05	0.05
query28	10.64	1.11	1.07
query29	12.54	3.22	3.14
query30	0.25	0.06	0.05
query31	2.88	0.37	0.38
query32	3.27	0.46	0.48
query33	2.94	2.95	3.01
query34	17.02	4.49	4.50
query35	4.52	4.45	4.50
query36	0.66	0.49	0.47
query37	0.09	0.06	0.06
query38	0.04	0.03	0.04
query39	0.04	0.02	0.02
query40	0.15	0.11	0.12
query41	0.08	0.02	0.03
query42	0.03	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 106.81 s
Total hot run time: 33 s

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants