Journal of Jilin University(Engineering and Technology Edition) ›› 2025, Vol. 55 ›› Issue (9): 3007-3019.doi: 10.13229/j.cnki.jdxbgxb.20240344

Previous Articles    

Sampling method of software product line based on adaptive evolutionary algorithm

Dan-tong OUYANG1,2(),Zhe YUAN1,2,Li-ming ZHANG1,2   

  1. 1.College of Computer Science and Technology,Jilin University,Changchun 130012,China
    2.Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education,Jilin University,Changchun 130012,China
  • Received:2024-04-01 Online:2025-09-01 Published:2025-11-14

Abstract:

In order to further improve the diversity of the sample set, the DDHNSbS algorithm was proposed, in which the Diversity Mutation Strategy(DMS)not only preserves the diversity of the initial population, but also optimizes the effect of traditional genetic operators. Meanwhile, to make up for the limitations of novelty search, diversity detection(DD)was proposed to check and flip the assigned values of features with a certain probability. So as to further diversify the number of features selected in the sample set, probability-aware diversification via the Halton sequence(Halton-PaD)was achieved, and the generated initial configurations' uniformity was developed thanks to Halton-PaD. Experiments were carried out on 39 public SPLs, and the experimental results show that when DDHNSbS algorithm is compared with PaD-NSbS algorithm, DDHNSbS performs significantly better in 42.1% of the small SPLs, and has the same performance on other SPLs.

Key words: software product lines, diverse sampling, novelty search, probability-aware diversification, evolutionary algorithm

CLC Number: 

  • TP311

Fig.1

Example of DD"

Table 1

FMs used in the experiment"

FM|F||C||core||dead|
fiasco_17_102341 17876
lrzip206320
BerkeleyDBC181920
HiPAcc3110410
Jhipster4510470
E-shop290426300
Printers172310500
SmartHomev2.2608270
WebPortal436840
busybox_1_28_0998962120
Polly4010090
7z4421050
JavaGC3910570
VP94210490
axtls_2_1_49419040
uClibc-ng_1_0_292691 403140
toybox_0_7_5316106815
toybox5441 0204365
axTLS6842 1553381
fiasco1 6385 22849964
SPLOT-FM-50005 0009 419750
uClinux1 8502 46871 237
ref49551 2183 099050
adderII1 2763 206037
ecos-icse111 2443 146035
m5272c31 3233 297043
pati1 2483 266047
olpce22941 2743 881042
integrator_arm91 26750 606044
at91sam7sek1 3193 963074
se77x91 31949 937058
phycore229x1 3604 026089
freebsd-icse111 39662 163338
busybox-1.18.06 79617 836123 939
uClinux-config11 25431 637156 012
buildroot14 91045 6031006 895
freetz31 012102 70511714 445
2.6.28.6-icse116 888343 94458102
embtoolkit23 516180 5118396 561

Table 2

Comparison of DDHNSbS, NSbS and PaD-NSbS"

FM编号DDHNNSbSPaDNFM编号DDHNNSbSPaDN
163.00663.148?65.220?21357.001362.475?376.201?
22.0532.053?2.053?222.12518.621?9.992?
31.8241.824?1.824?235.6357.794?5.466?
44.0024.045?4.355?245.9328.152?5.868?
54.1794.179?4.195?255.8937.643?5.893?
61.1284.175?1.202?266.6248.902?6.581?
75.8415.888?5.891?275.6827.480?5.606?
80.0510.331?0.064?285.9627.542?5.964?
90.2680.399?0.289?296.1427.468?6.099?
104.0986.743?4.623?306.0938.701?6.044?
116.8136.813?6.813?316.2259.028?6.098?
1212.65012.650?12.650?326.1438.877?6.144?
137.6367.636?7.636?3318.85820.218?21.663?
148.5008.500?8.500?34279.247284.050?284.028?
153.6913.703?4.007?35399.488401.478?401.233?
1621.43221.724?23.859?36312.876322.055?321.407?
171.0783.150?1.299?371 349.7051 366.916?1 370.377?
188.2678.355?8.448?3845.59747.574?47.405?
1928.31828.332?28.591?392 169.5142 175.095?2 175.915?
2092.30793.106?94.124?

Fig.2

Number of the best spread of each algorithm"

Fig.3

Percentage of FM of each scale in the significantly increased FM"

Fig.4

Number of features selected for each sampler generation configuration of the WebPortal"

Fig.5

Increment of the continuous configuration generated by each sampler on WebPortal"

Fig.6

Distribution of configurations selected by DDHNSbS and M1 on busybox_1_28_0 and ref4955 in the behavior space"

Table 6

Comparison of DDHNSbS and M3"

FM编号DDHNSbSM3IR/%
163.00663.177?-0.27
22.0532.053?0.00
31.8241.824?0.00
44.0024.025?-0.57
54.1794.179?0.00
61.1281.142?-1.24
75.8415.854?-0.22
80.0510.053?-3.92
90.2680.264?1.49
104.0984.212?-2.78
116.8136.813?0.00
1212.65012.650?0.00
137.6367.636?0.00
148.5008.500?0.00
153.6913.693?-0.05
1621.43221.568?-0.63
171.0781.116?-3.53
188.2678.394?-1.54
1928.31828.311?0.02
2092.30792.300?0.01
21357.001357.762?-0.21
222.1252.380?-12.00
235.6356.236?-10.67
245.9326.660?-12.27
255.8936.558?-11.28
266.6247.165?-8.17
275.6826.261?-10.19
285.9626.728?-12.85
296.1426.762?-10.09
306.0936.689?-9.78
316.2256.933?-11.37
326.1436.809?-10.82
3318.85818.697?0.85
34279.247276.312?1.05
35399.488399.740?-0.06
36312.876317.214?-1.39
371 349.7051 351.619?-0.14
3845.59746.134?-1.18
392 169.5142 172.433?-0.13

Fig.7

Improvement rate of DDHNSbS compared to M3"

Fig.8

Improvement rate of DDHNSbS compared to M4"

Table 7

Comparison of DDHNSbS and M4"

FM编号DDHNSbSM4IR/%
163.00662.968?0.06
22.0532.053?0.00
31.8241.824?0.00
44.0024.031?-0.72
54.1794.179?0.00
61.1281.165?-3.28
75.8415.847?-0.10
80.0510.052?-1.96
90.2680.271?-1.12
104.0984.383?-6.95
116.8136.813?0.00
1212.65012.650?0.00
137.6367.636?0.00
148.5008.500?0.00
153.6913.700?-0.24
1621.43221.492?-0.28
171.0781.123?-4.17
188.2678.332?-0.79
1928.31828.421?-0.36
2092.30792.305?0.00
21357.001354.963?0.57
222.1252.281?-7.34
235.6356.156?-9.25
245.9326.468?-9.04
255.8936.429?-9.10
266.6247.118?-7.46
275.6826.197?-9.06
285.9626.498?-8.99
296.1426.599?-7.44
306.0936.667?-9.42
316.2256.802?-9.27
326.1436.667?-8.51
3318.85819.021?-0.86
34279.247274.058?1.86
35399.488401.03?-0.39
36312.876311.247?0.52
371 349.7051 349.448?0.02
3845.59746.994?-3.06
392 169.5142 168.406?0.05
[1] Clements P, Northrop L. Software Product Lines[M]. Boston: Addison-Wesley, 2002.
[2] Hou Y M, Ouyang D T, Tian X L, et al. Evolutionary many-objective satisfiability solver for configuring software product lines[J]. Applied Intelligence: The International Journal of Artificial Intelligence, Neural Networks, and Complex Problem-Solving Technologies, 2022, 52(9): 10650-10673.
[3] 孙宝凤, 任欣欣, 郑再思, 等. 考虑工人负荷的多目标流水车间优化调度[J]. 吉林大学学报: 工学版, 2021, 51(3): 900-909.
Sun Bao-feng, Ren Xin-xin, Zheng Zai-si, et al. Multi⁃objective flow shop optimal scheduling considering worker's load[J]. Journal of Jilin University (Engineering and Technology Edition), 2021, 51(3): 900-909.
[4] 马永杰, 陈敏. 基于卡尔曼滤波预测策略的动态多目标优化算法[J]. 吉林大学学报: 工学版, 2022, 52(6): 1442-1458.
Ma Yong-jie, Chen Min. Dynamic multi⁃objective optimization algorithm based on Kalman filter prediction strategy [J]. Journal of Jilin University (Engineering and Technology Edition), 2022, 52(6): 1442-1458.
[5] Pett T, Krieter S, Thüm T, et al. AutoSMP: an evaluation platform for sampling algorithms[C]∥Proceedings of the 25th ACM International Systems and Software Product Line Conference-Volume B, New York, NY, USA, 2021: 41-44.
[6] Devroey X, Perrouin G, Legay A, et al. Search-based similarity-driven behavioural SPL testing[C]∥Proceedings of the 10th International Workshop on Variability Modelling of Software-Intensive Systems, Salvador, Brazil, 2016: 89-96.
[7] Oh J, Gazzillo P, Batory D, et al. Scalable uniform sampling for real-world software product lines[R]. Austin: The Univ Texas at Austin, 2020.
[8] Dutra R, Laeufer K, Bachrach J, et al. Efficient sampling of SAT solutions for testing[C]∥Proceedings of the 40th International Conference on Software Engineering, Gothenburg, Sweden, 2018: 549-559.
[9] Plazar Q, Acher M, Perrouin G, et al. Uniform sampling of sat solutions for configurable systems: Are we there yet?[C]∥Proceedings of the 2019 12th IEEE Conference on Software Testing, Validation and Verification (ICST), Xi'an, China, 2019: 240-251.
[10] Heradio R, Fernandez-Amoros D, Galindo J A, et al. Uniform and scalable SAT-sampling for configurable systems[C]∥Proceedings of the 24th ACM Conference on Systems and Software Product Line, Montreal, Quebec, Canada, 2020: 1-11.
[11] Liebig J, Von Rhein A, Kästner C, et al. Scalable analysis of variable software[C]∥Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering, Saint Petersburg, Russia, 2013: 81-91.
[12] Medeiros F, Kästner C, Ribeiro M, et al. A comparison of 10 sampling algorithms for configurable systems[C]∥Proceedings of the 38th International Conference on Software Engineering, Austin, Texas, USA, 2016: 643-654.
[13] Melo J, Flesborg E, Brabrand C, et al. A quantitative analysis of variability warnings in linux[C]∥Proceedings of the 10th International Workshop on Variability Modelling of Software-intensive Systems, Salvador, Brazil, 2016: 3-8.
[14] Cohen M B, Dwyer M B, Shi J. Interaction testing of highly-configurable systems in the presence of constraints[C]∥Proceedings of the 2007 International Symposium on Software Testing and Analysis, London, England, United Kingdom, 2007: 129-139.
[15] Krieter S, Thüm T, Schulze S, et al. YASA: yet another sampling algorithm[C]∥Proceedings of the 14th International Working Conference on Variability Modelling of Software-Intensive Systems, Magdeburg, Germany, 2020: 1-10.
[16] Garvin B J, Cohen M B, Dwyer M B. Evaluating improvements to a meta-heuristic search for constrained interaction testing[J]. Empirical Software Engineering, 2011, 16(1): 61-102.
[17] Chakraborty S, Fremont D J, Meel K S, et al. On parallel scalable uniform SAT witness generation[C]∥Proceedings of the 21st International Conference on Tools and Algorithms for the Construction and Analysis of Systems, London, UK, 2015: 304-319.
[18] Ermon S, Gomes C P, Selman B. Uniform solution sampling using a constraint solver as an oracle[J/OL]. [2024-03-15].
[19] Kaltenecker C, Grebhahn A, Siegmund N, et al. Distance-based sampling of software configuration spaces[C]∥Proceedings of the 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE), Montreal, Quebec, Canada, 2019: 1084-1094.
[20] Henard C, Papadakis M, Perrouin G, et al. Looking for novelty in search-based software combinatorial explosion: using similarity to generate and prioritize t-wise test configurations for software product lines[J]. IEEE Transactions on Software Engineering, 2014, 40(7): 650-670.
[21] Henard C, Papadakis M, Harman M, et al. Combining multi-objective search and constraint solving for configuring large software product lines[C]∥2015 IEEE/ACM 37th IEEE International Conference on Software Engineering, Firenze, Italy, 2015: 517-528.
[22] Hierons R M, Li M, Liu X, et al. SIP: optimal product selection from feature models using many-objective evolutionary optimization[J]. ACM Transactions on Software Engineering and Methodology (TOSEM), 2016, 25(2): 1-39.
[23] Sayyad A S, Ingram J, Menzies T, et al. Scalable product line configuration: a straw to break the camel's back[C]∥2013 28th IEEE/ACM International Conference on Automated Software Engineering (ASE), Palo Alto, USA, 2013: 465-474.
[24] Xiang Y, Zhou Y R, Zheng Z B, et al. Configuring software product lines by combining many-objective optimization and SAT solvers[J]. ACM Transactions on Software Engineering and Methodology (TOSEM), 2018, 26(4): 1-46.
[25] Xiang Y, Huang H, Zhou Y R, et al. Search-based diverse sampling from real-world software product lines[C]∥Proc of the 44th International Conference on Software Engineering, Pittsburgh, USA, 2022: 1945-1957.
[26] Xiang Y, Yang X W, Huang H, et al. Sampling configurations from software product lines via probability-aware diversification and SAT solving[J]. Automated Software Engineering, 2022, 29(2): 1-45.
[27] 黄美发, 景晖, 钟艳如, 等. 基于拟随机序列的三维模型表面采样方法[J]. 计算机工程, 2008, 34(14): 263-265.
Huang Mei-fa, Jing Hui, Zhong Yan-ru, et al. Sampling method on 3D model surface based on quasi random sequence[J]. Computer Engineering, 2008, 34(14): 263-265.
[28] 周慧思, 欧阳丹彤, 田新亮, 等. 基于模型诊断的一种新编码方法[J]. 计算机研究与发展, 2023, 60(1): 95-102.
Zhou Hui-si, Ouyang Dan-tong, Tian Xin-liang, et al. A novel encoding for model-based diagnosis[J]. Journal of Computer Research and Development, 2023, 60(1): 95-102.
[29] Kang K C, Cohen S G, Hess J A, et al. Feature-oriented domain analysis (FODA) feasibility study[R]. Pittsburgh: Carnegie Mellon University, 1990.
[30] Maaranen H, Miettinen K, Mäkelä M M. Quasi-random initial population for genetic algorithms[J]. Computers & Mathematics with Applications, 2004, 47(12): 1885-1895.
[31] Keller A. Quasi-Monte Carlo image synthesis in a nutshell[C]∥Monte Carlo and Quasi-Monte Carlo Methods 2012, Sydney, Australia, 2013: 213-249.
[32] Luo C, Sun B Q, Qiao B, et al. LS-sampling: an effective local search based sampling approach for achieving high t-wise coverage[C]∥Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, Athens, Greece, 2021: 1081-1092.
[33] Baranov E, Legay A, Meel K S. Baital: an adaptive weighted sampling approach for improved t-wise coverage[J/OL]. [2024-03-15].
[34] Xiang Y, Huang H, Li M Q, et al. Looking for novelty in search-based software product line testing[J]. IEEE Transactions on Software Engineering, 2021, 48(7): 2317-2338.
[35] Arcuri A, Briand L. A practical guide for using statistical tests to assess randomized algorithms in software engineering[C]∥Proceedings of the 33rd International Conference on Software Engineering, Honolulu, Hawaii, USA, 2011: 1-10.
[1] Pei-ming SUN,Zhe WANG. Optimization method of party affairs activity scheduling based on directional differential evolution algorithm [J]. Journal of Jilin University(Engineering and Technology Edition), 2025, 55(8): 2761-2770.
[2] Yong-jie MA,Min CHEN. Dynamic multi⁃objective optimization algorithm based on Kalman filter prediction strategy [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(6): 1442-1458.
[3] Zhou-zhou LIU,Qian-yun ZHANG,Xin-hua MA,Han PENG. Compressed sensing signal reconstruction based on optimized discrete differential evolution algorithm [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(6): 2246-2252.
[4] Bing-hai ZHOU,Qiong WU. Balancing and bi⁃objective optimization of robotic assemble lines [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(2): 720-727.
[5] Lei JIANG,Ren-chu GUAN. Design of fuzzy comprehensive evaluation system for talent quality based on multi⁃objective evolutionary algorithm [J]. Journal of Jilin University(Engineering and Technology Edition), 2020, 50(5): 1856-1861.
[6] Lei CHEN,Jiang⁃feng WANG,Yuan⁃li GU,Xue⁃dong YAN. Multi⁃source traffic data fusion algorithm based onmind evolutionary algorithm optimization [J]. Journal of Jilin University(Engineering and Technology Edition), 2019, 49(3): 705-713.
[7] HU Guan-yu, QIAO Pei-li. High dimensional differential evolutionary algorithm based on cloud population for network security prediction [J]. 吉林大学学报(工学版), 2016, 46(2): 568-577.
[8] LI Gen,LI Wen-hui. Facial feature tracking based on mind evolutionary algorithm [J]. 吉林大学学报(工学版), 2015, 45(2): 606-612.
[9] LI Gen,LI Wen-hui. Face occlusion recognition based on MEBML [J]. 吉林大学学报(工学版), 2014, 44(5): 1410-1416.
[10] YOU Xiao-ming, LIU Sheng, WANG Yu-ming. Quantum-behaved network resource parallel allocation optimization model and application [J]. 吉林大学学报(工学版), 2012, 42(增刊1): 341-345.
[11] DING Hui, LI Hong-guang. Agent-based evolutionary approach towards solving constrained multi-objective optimization problems [J]. 吉林大学学报(工学版), 2011, 41(增刊1): 173-178.
[12] ZHAO Wen-hong1, WANG Wei1,2, WANG Yu-ping2, HAO Ming-qiang3 . ZHAO Wen-hong1, WANG Wei1,2, WANG Yu-ping2, HAO Ming-qiang3 [J]. 吉林大学学报(工学版), 2008, 38(04): 865-870.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!