电商网页数据, 爬虫技术, 网页数据爬取, 频繁项集, 树形结构, 自适应挖掘

To effectively mine and utilize web data, a correlation data adaptive mining algorithm is proposed with the support of web crawling technology. Web crawling technology is used to crawl e-commerce web pages, breaking through data acquisition limitations, storing the required data in a structured format in a database, and providing an orderly data foundation for subsequent processing. Based on sparse data stored in the database, the Apriori algorithm is applied according to the minimum support degree. Through pruning strategy, this algorithm can accurately find frequent itemsets in sparse data while reducing data processing, successfully solving the problem of traditional methods being difficult to obtain frequent itemsets due to data sparsity. Using a tree structure as a framework, the frequent itemsets obtained by the Apriori algorithm are used as nodes. By adaptively combining old and new nodes, a mining tree is constructed until there are no nodes to combine.The generated mining tree comprehensively presents data association relationships, thereby obtaining sufficient and accurate e-commerce webpage association data mining results. After verification on the web pages of large e-commerce websites, it is found that the proposed algorithm can ensure the integrity of web crawling, accurately obtain frequent itemsets targeting purchasing behavior, and precisely mine associated data in e-commerce web pages. While providing useful references for e-commerce platform operation, it also provides better decision support for merchants and consumers.

E-commerce webpage data, crawling technology, Web data crawling, frequent itemsets, tree structure, adaptive mining