In this blog, I would make a summary of 3D shape retrieval.
Sketch based 3D Shape Retrieval
- Wang F, Kang L, Li Y. Sketch-based 3d shape retrieval using convolutional neural networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015: 1875-1883.
 |
In almost all state of the art approaches, sketch based 3D shape retrieval amounts to finding the “best views” for 3D models and hand-crafting the right features for matching sketches and views.
This paper drastically reduce the number of views to noly two predefined directions for the whole dataset, assuming that the majority of models are upright. Then learn two Siamese Convolutional Neural Networks (CNNs), one for the views and one for the sketches.
The upright assumption appears to be strong, but it turns out to be sensible for 3D datasets.
Contributions
- They propose to learn feature representations for sketch based shape retrieval, which bypass the dilemma of best view selection;
- They adopt two Siamese Convolutional Neural Networks to successfully learn similarities in both the within-domain and the cross domain.
- They outputform all the state of the art methods on three large datasets significantly.
Learning high-level feature by deep belief networks for 3-D model retrieval and recognition
- Bu S, Liu Z, Han J, et al. Learning high-level feature by deep belief networks for 3-D model retrieval and recognition[J]. IEEE Transactions on Multimedia, 2014, 16(8): 2154-2167.
In this paper, the authors propose a multi-level 3D shape feature extraction framework by using deep learning. The low-level 3-D shape descriptors are first encoded into geometric bag-of-words, from which middle-level patterns are discovered to explore geometric relationships among words. After that, high-level shape features are learned via deep belief networks, which are more discriminative for the tasks of shape classification and retrieval.
 |
The authors get Scale-Invariant Heat Kernel Signature (SI-HKS) and Average Geodesic Distance (AGD) features to form low-level descriptors. In the next setp, bag-of-features are computed to represent to occurrence probability of geometric words to form Middle-level features. The disadvantage of bag-of-features is the fact that they only consider the occurrence distribution of the words and ignore the structural relationship between them, which decreases their dis- crimination. At last, deep belief networks (DBNs) are constructed to get the deep features.
Contributions
- The authors propose a deep learning framework for shape classifi- cation and retrieval. To the best of our knowledge, it is the first time to apply deep learning into 3D shape retrieval.
- Our work can extract higher level features from local descriptors, and these features have better discriminability and generalization against intra-class large geometrical variations.
Limitations
- Based on the theory of deep learning, using themiddle-level featuresmay cause the lower generaliza-tion, because during the middle-level features processing some information is lost, especially the local geometric connections.
- In the proposed method, the features for the deep learning input are global features. Although the authors adopt geodesic-sensitive BoFs to preserve the geometrical relationship of BoFs, the local connection information is still missing. Hence, the pro-posed method is difficult to apply to more sophisticated tasks such as segmentation, partial retrieval, andsymmetric detection.
DeepShape: Deep-Learned Shape Descriptor for 3D Shape Retrieval
- Xie J, Dai G, Zhu F, et al. DeepShape: Deep-Learned Shape Descriptor for 3D Shape Retrieval[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016.
 |
This paper proposed a high-level shape feature learning scheme to extract features that are insensitive to deformations via a novel discriminative deep auto-encoder. They extract multiscale shape distribution shape descriptor based on Heat Kernel Signature (HKS), and the multiscale shape distribution shape descriptor was used as the input of their auto-encoder network. Then, by imposing the Fisher discrimination criterion on the neurons in the hidden layer, they developed a novel discriminative deep auto-encoder for shape feature learning. At last, they concatenate all the hidden layers form multiple discriminative auto-encoders to form a shape descriptor for 3D shape matching and retrieval.
- Tabia H, Laga H, Picard D, et al. Covariance descriptors for 3D shape matching and retrieval[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2014: 4185-4192.
- Su H, Maji S, Kalogerakis E, et al. Multi-view convolutional neural networks for 3d shape recognition[C]//Proceedings of the IEEE international conference on computer vision. 2015: 945-953.
- Li B, Lu Y, Li C, et al. Shrec’14 track: extended large scale sketch-based 3D shape retrieval[C]//Eurographics workshop on 3D object retrieval. 2014, 2014.
- Savva M, Yu F, Su H, et al. SHREC’16 Track Large-Scale 3D Shape Retrieval from ShapeNet Core55[C]//Proceedings of the Eurographics Workshop on 3D Object Retrieval. 2016.
- Fang Y, Xie J, Dai G, et al. 3d deep shape descriptor[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015: 2319-2328.
- S. Bai; X. Bai; Z. Zhou; Z. Zhang; Q. Tian; L. J. Latecki, “GIFT: Towards Scalable 3D Shape Retrieval,” in IEEE Transactions on Multimedia , vol.PP, no.99, pp.1-1 doi: 10.1109/TMM.2017.2652071
- Wu Z, Song S, Khosla A, et al. 3d shapenets: A deep representation for volumetric shapes[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015: 1912-1920.
Their project site http://3dshapenets.cs.princeton.edu/
I have ran this code on the server in my lab, and it is necessary to run the code with ssh and Screen as well as run this code without matlab GUI. The details can be found in my blog Ubuntu ssh Screen