Journal of Jilin University(Engineering and Technology Edition) ›› 2020, Vol. 50 ›› Issue (1): 289-296.doi: 10.13229/j.cnki.jdxbgxb20180902

Previous Articles    

Thangka image annotation based on ontology

Tie-jun WANG1,2(),Wei-lan WANG1,2()   

  1. 1. Key Laboratory of China's Ethnic Languages and Information Technology of Ministry of Education, Northwest Minzu University, Lanzhou 730030, China
    2. School of Mathematics and Computer Science, Northwest Minzu University, Lanzhou 730030, China
  • Received:2018-09-03 Online:2020-01-01 Published:2020-02-06
  • Contact: Wei-lan WANG E-mail:wtj@mail.lzjtu.cn;wangweilan@xbmu.edu.cn

RICH HTML

 5   

PDF (PC)

215

Abstract:

The existing image automatic annotation algorithm has very limited ability of automatic annotation and semantic description of Thangka image. This paper proposes a semantic annotation method of Thangka image based on domain knowledge ontology. First, the system of Thangka image annotation framework was constructed. Then in the early study of the Thangka image target recognition and classification, on the basis of combining Thangka domain knowledge ontology, two levels of Thangka image annotation were realized, which were local region automatic annotation and ontology?based global annotation. Experimental results prove that the proposed method is effective for Thangka image semantic annotation.

Key words: computer application, Thangka image, image annotation, domain ontology

CLC Number: 

  • TP391

Fig.1

Ontology based semantic tagging framework for Thangka images"

Fig.2

Headwear classification and recognition method"

Fig.3

Image extracted from headdress and its grayscale image"

Fig.4

Basic global threshold algorithm for segmented two valued images"

Table 1

Headwear recognition results"

识别算法头冠发髻僧帽总分类正确率
文献[20]方法96.694.176.589.1
基于频率谱参数和颜色特征参数的DDkNN方法88848084.0
基于频率谱参数+颜色特征参数+形状参数+纹理参数的DDkNN方法96929293.33

Fig.5

6 common gesture recognition"

Fig.6

Thangka domain ontology model"

Table 2

List of general inference rules"

关 系推理规则举例
常见人物关系（父子、母子、父女、母女、夫妻、兄弟、叔侄、爷孙、师徒、同门）

IF(A 有父亲 B) AND (B有父亲C) THEN (A 有孙子C)

IF(A 有儿子B) AND (B有母亲C) THEN (A 有妻子C)

IF(A 有兄弟B) AND (B有儿子C) THEN (A 有侄子C)

圣像特有关系（别名、原名、藏文名、梵文名、汉译名、意译名、音译名、化身、法身、报身）

IF(A 有别名B) AND (B有原名C) THEN (A 有原名C)

IF(A 有法身B) AND (B有别名C) THEN (A 有法身C)

IF(A 有报身B) AND (B有法身C) THEN (A 有法身C)

IF(A 有化身B) AND (B有报身C) THEN (A 有报身C)

Table 3

Example of combination feature recognition"

主 尊组合特征
不动明王忿怒相+一面二臂+三目+右手举般若剑+左手结期克印+半跪姿
大梵天坐骑天鹅+四面+左手持宝瓶+右手持轮
大黑天左手捧嘎巴拉碗+右手持金刚钺刀
喜金刚十六只手都持嘎巴拉碗
虚空藏菩萨寂静相+一面二臂+宝剑+蓝色身体
金刚亥母舞姿+猪首
狮面佛母狮头+蓝色人身+左手托嘎巴拉碗+右手结期克印+金刚钺刀+天杖

Fig.7

Image tagging process based on ontology"

Fig.8

Image tagging system of Thangka"

1 钱建军. 基于语义的唐卡图像标注与检索研究[D].兰州：西北民族大学中国民族信息技术研究院，2010.
Qian Jian-jun. A study on semantic-based thangka image annotation and retrieval[D]. Lanzhou:National Information Technology Research Institute，Northwest Minzu University，2010.
2 毕学慧, 刘华明, 王维兰. 基于头饰特征的唐卡标注和检索[J]. 计算机科学, 2014, 41(2): 312-316.
Bi Xue-hui, Liu Hua-ming, Wang Wei-lan. Thangka annotation and retrieval based on headdress features[J]. Computer Science, 2014, 41(2): 312-316.
3 鲍泓, 徐光美, 冯松鹤, 等. 自动图像标注技术研究进展[J]. 计算机科学, 2011, 38(7): 35-40.
Bao Hong, Xu Guang-mei, Feng Song-he. Advances in automatic image annotation[J]. Computer Science, 2011, 38(7): 35-40.
4 Zhang D, Islam M M, Lu G. A review on automatic image annotation techniques[J]. Pattern Recognition, 2012, 45(1): 346-362.
5 李志欣, 施智平, 张灿龙, 等. 混合生成式和判别式模型的图像自动标注[J]. 中国图象图形学报, 2015, 20(5): 687-699.
Li Zhi-xin, Shi Zhi-ping, Zhang Can-long, et al. Hybrid generative/discriminative model for automatic image annotation[J]. Journal of Image and Graphics, 2015, 20(5): 687-699.
6 Ding X, Li B, Xiong W, et al. Multi-Instance multi-label learning combining hierarchical context and its application to image annotation[J]. IEEE Transactions on Multimedia, 2016, 18(8): 1-12.
7 Zhou Z H, Zhang M L. Multi-label learning[M]∥ Sammut Claude, Webbv Geoffrey I. Encyclopedia of Machine Learning and Data Mining, Berlin: Springer, 2016:352-366.
8 Zhao M, Chow T W S, Zhang Z, et al. Automatic image annotation via compact graph based semi-supervised learning[J]. Knowledge-Based Systems, 2015, 76: 148-165.
9 史彩娟, 阮秋琦. 基于增强稀疏性特征选择的网络图像标注[J]. 软件学报, 2015, 26(7): 1800-1811.
Shi Cai-juan, Ruan Qiu-qi. Feature selection with enhanced sparsity for web image annotation[J]. Journal of Software, 2015, 26(7): 1800-1811.
10 Wang H, Liu S, Chia L T. Does ontology help in image retrieval？ a comparison between keyword, text ontology and multi-modality ontology approaches[C]∥ACM International Conference on Multimedia, Santa Barbara, USA, 2006: 109-112.
11 Srikanth M, Varner J, Bowden M, et al. Exploiting ontologies for automatic image annotation[C]∥Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval, Salvador, Brazil, 2005: 552-558.
12 Smailis C V, Iakovidis D K. Ontology-based automatic image annotation exploiting generalized qualitative spatial semantics[C]∥The 7th Hellenic Conference on Artificial Intelligence, Lamia, Greece, 2012: 299-306.
13 Singh P, Goudar R H, Rathore R, et al. Domain ontology based efficient image retrieval[C]∥The 7th International Conference on Intelligent Systems and Control, Coimbatore, Tamilnadu, India, 2013: 445-452.
14 王琦, 鲁东明. 基于构图分析的古代壁画相关度评价方法[J]. 浙江大学学报:工学版, 2012, 46(3): 21-30.
Wang Qi, Lu Dong-Ming. Composition analysis-based relevance ranking for ancient mural[J]. Journal of Zhejiang University(Engineering Science), 2012, 46(3): 21-30.
15 Wei Z, Luo X, Zhou F. Ontology based automatic image annotation using multi-class SVM[C]∥The 7th International Conference on Image and Graphics, Qingdao, China, 2013: 434-438.
16 Ismat A R, Md Z U, Masaki A. Ontology based supervised learning for image annotation[J]. IEICE Technical Report Image Engineering[J]. 2014, 114: 41-46.
17 Ferdinand D, Paul M, Stéphanie F, et al. Developing a knowledge base to support the annotation of ultrasound images of ectopic pregnancy[J]. Journal of Biomedical Semantics, 2017, 8(1): 4-17.
18 Hyun-Deok No, Kwang-won Seo, Dong-Hyuk Im. Semantic image annotation and retrieval in mobile environments[J]. Journal of Korea Multimedia Society, 2016, 19(8): 1498-1504.
19 Vinyals O, Toshev A, Bengio S, et al. Show and tell: a neural image caption generator[C]∥Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Boston, United States, 2015: 3156-3164.
20 Xu K, Ba J L, Kiros R, et al. Show, attend and tell: neural image caption generation with visual attention [C]∥The 32nd International Conference on Machine Learning, Lile, France, 2015: 2048-2057.
21 蒋树强, 闵巍庆, 王树徽. 面向智能交互的图像识别技术综述与展望[J]. 计算机研究与发展, 2016, 53(1): 113-122.
Jiang Shu-qiang, Min Wei-qing, Wang Shu-hui. Survey and prospect of intelligent interaction-oriented image recognition techniques[J]. Journal of Computer Research and Development, 2016, 53(1): 113-122.
22 Karpathy A, Li F F. Deep visual-semantic alignments for generating image descriptions[C]//IEEE Conference on Computer Vision and Pattern Recognition, Boston, United States, 2015: 3128-3137.
23 Fang H, Platt J C, Zitnick C L, et al. From captions to visual concepts and back[C]//IEEE Conference on Computer Vision and Pattern Recognition, Boston, United States, 2015: 1473-1482.
24 Borisyuk F, Gordo A, Sivakumar V. Rosetta: large scale system for text detection and recognition in images[C]//The 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, London, United Kingdom, 2018: 71-79.
25 王铁君, 王维兰. 基于多特征融合的唐卡图像法器识别方法[J]. 计算机工程, 2016, 42(3): 198-203, 207.
Wang Tie-jun, Wang Wei-lan. Religious ritual implement recognition method in Thangka image based on multi-feature fusion[J]. Computer Engineering, 2016, 42(3): 198-203, 207.
26 吴禄慎,常参参,王晓辉,等. 基于局部调节卷积神经网络的图像识别方法[J]. 计算机工程, 2018, 44(12): 240-246.
Wu Lu-shen,Chang Can-can,Wang Xiaohui,et al. Image recognition method based on local adjustment convolutional neural network[J]. Computer Engineering, 2018, 44(12): 240-246.
27 王铁君, 王维兰. 基于Jena的唐卡领域本体推理[J]. 吉林大学学报:工学版, 2016, 46(6): 2059-2066.
Wang Tie-jun, Wang Wei-lan. Thangka domain ontology reasoning based on Jena[J]. Journal of Jilin University(Engineering and Technology Edition), 2016, 46(6): 2059-2066.
28 王铁君, 王维兰. 唐卡领域本体研究与构建[J]. 吉林大学学报: 理学版, 2017, 55(2): 363-370.
Wang Tie-jun, Wang Wei-lan. Research and construction of Thangka domain ontology[J]. Journal of Jilin University (Science Edition), 2017, 55(2): 363-370.
[1] Xiao-dong ZHANG,Xiao-jun XIA,Hai-feng LYU,Xu-chao GONG,Meng-jia LIAN. Dynamic load balancing of physiological data flow in big data network parallel computing environment [J]. Journal of Jilin University(Engineering and Technology Edition), 2020, 50(1): 247-254.
[2] Xiao-hui WANG,Lu-shen WU,Hua-wei CHEN. Denoising of scattered point cloud data based on normal vector distance classification [J]. Journal of Jilin University(Engineering and Technology Edition), 2020, 50(1): 278-288.
[3] Jun-yi DENG,Yan-heng LIU,Shi FENG,Rong-cun ZHAO,Jian WANG. GSPN⁃based model to evaluate the performance and securi tytradeoff in Ad-hoc network [J]. Journal of Jilin University(Engineering and Technology Edition), 2020, 50(1): 255-261.
[4] Xiong-fei LI,Jing WANG,Xiao-li ZHANG,Tie-hu FAN. Multi-focus image fusion based on support vector machines and window gradient [J]. Journal of Jilin University(Engineering and Technology Edition), 2020, 50(1): 227-236.
[5] Hong-yan WANG,He-lei QIU,Jia ZHENG,Bing-nan PEI. Visual tracking method based on low⁃rank sparse representation under illumination change [J]. Journal of Jilin University(Engineering and Technology Edition), 2020, 50(1): 268-277.
[6] Xiang-jiu CHE,Hua-luo LIU,Qing-bin SHAO. Fabric defect recognition algorithm based onimproved Fast RCNN [J]. Journal of Jilin University(Engineering and Technology Edition), 2019, 49(6): 2038-2044.
[7] Bing-hai ZHOU,Qiong WU. Balancing and optimization of robotic assemble lines withtool and space constraint [J]. Journal of Jilin University(Engineering and Technology Edition), 2019, 49(6): 2069-2075.
[8] Hong-wei ZHAO,Peng WANG,Li-li FAN,Huang-shui HU,Ping-ping LIU. Similarity retention instance retrieval method [J]. Journal of Jilin University(Engineering and Technology Edition), 2019, 49(6): 2045-2050.
[9] Jun SHEN,Xiao ZHOU,Zu-qin JI. Implementation of service dynamic extended network and its node system model [J]. Journal of Jilin University(Engineering and Technology Edition), 2019, 49(6): 2058-2068.
[10] You ZHOU,Sen YANG,Da-lin LI,Chun-guo WU,Yan WANG,Kang-ping WANG. Acceleration platform for face detection and recognition based on field⁃programmable gate array [J]. Journal of Jilin University(Engineering and Technology Edition), 2019, 49(6): 2051-2057.
[11] Bin LI,Xu ZHOU,Fang MEI,Shuai-ning PAN. Location recommendation algorithm based on K-means and matrix factorization [J]. Journal of Jilin University(Engineering and Technology Edition), 2019, 49(5): 1653-1660.
[12] Xiong-fei LI,Lu SONG,Xiao-li ZHANG. Remote sensing image fusion based on cooperative empirical wavelet transform [J]. Journal of Jilin University(Engineering and Technology Edition), 2019, 49(4): 1307-1319.
[13] Yuan-ning LIU,Shuai LIU,Xiao-dong ZHU,Guang HUO,Tong DING,Kuo ZHANG,Xue JIANG,Shu-jun GUO,Qi-xian ZHANG. Iris secondary recognition based on decision particle swarm optimization and stable texture [J]. Journal of Jilin University(Engineering and Technology Edition), 2019, 49(4): 1329-1338.
[14] Bin LI,Guo⁃jun SHEN,Geng SUN,Ting⁃ting ZHENG. Improved chicken swarm optimization algorithm [J]. Journal of Jilin University(Engineering and Technology Edition), 2019, 49(4): 1339-1344.
[15] Yan-jun SUN,Xuan-jing SHEN,Hai-peng CHEN,Yong-zhe ZHAO. Recaptured image forensics algorithm based on local plane linear point [J]. Journal of Jilin University(Engineering and Technology Edition), 2019, 49(4): 1320-1328.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
Copyright © Journal of Jilin University(Engineering and Technology Edition), All Rights Reserved.
Tel: +86-431-85095297　Fax: +86-431-85094074　E-mail: xbgxb@jlu.edu.cn
Powered by Beijing Magtech Co. Ltd