Main Interests
My research interests span the fields of computer vision and machine learning, predominantly on object recognition, visual tracking, and scene understanding.
Projects
Manufactured Object Identification in Non-Controlled Environment
Objective of this project is to build an object recognition prototype based on computer vision and machine learning. The clients need the utilization of the system in a natural way. The system needs to satisfy the following constraints:
1. Recognition of object model in an non-controlled environment. 2. Identification of object by recognition of reference number. 3. Utilization of tablet as input equipment. 4. Duration of recognition under 5 seconds. 5. Possible to update new products. |
Personal Movie Recommendation System
Creation of personal movie recommendation system for individual user.
Item-based Collaborative Filtering;
Item similarity estimation;
Movie rate prediction;
Personal movie recommendation.
Data: MovieLens.
100,000 ratings from 1000 users on 1700 movies.
Item-based Collaborative Filtering;
Item similarity estimation;
Movie rate prediction;
Personal movie recommendation.
Data: MovieLens.
100,000 ratings from 1000 users on 1700 movies.
Text Detection on Reflective Surfaces in Nature Scene
Object identification by reading the reference number, localizing the reference number is the essential task. Furthermore, the great success of smartphones and large demands in content-based image understanding have made text detection a crucial task. It is desirable to build practical system that are robust to detect text under various conditions. As the text contains large variation in language, font, color, scale and orientation in complex scene, the detection remains unsolved. The difficulties mainly come from two aspects: (1) the diversity of the texts and (2) the complexity of the backgrounds. On one hand, text is a high level concept but better defined than the generic objects; on the other hand, repeated patterns (such as windows and barriers) and random clutters (such as grasses and leaves) may be similar to texts, and thus lead to potential false positives. In our case, the presence of the reflection ruins the homogeneity of the text characteristics, weakens the text saliency on the surface. These make the detection of text on reflective surface even more complicated. Thus we aim to build a detection system which is able to deal with not only text in nature scene, but also the text on reflective surfaces. To the best of our knowledge, detection of text on reflective surface has not been studied.
Entire Reflective Object Surface Structure Understanding
Reflection from reflective surface has been a long-standing problem for object recognition, it brings negative effects on object’s color, texture and structural information. Because of that, it is not a trivial task to recognize the surface structure effected by the reflection, especially when the object is entirely reflective. Most of the time, reflection is considered as noise. In this paper, we propose a novel method for entire reflective object sub-segmentation by transforming the reflection motion into object surface label. Instead of considering the reflection as noise, our approach takes reflection as an advantage for understanding the surface structure of the entire reflective objects. The experimental results on specular and transparent objects show that the surface structures of the reflective objects can be revealed and the segmentation based on the surface structure outperforms the approaches in literature.
Segmentation Results
1st column: original image; 2rd column: ground-truth segmentation; 3rd column: graph based segmentation; 4th column: EM segmentation; 5th column: proposed method.
1st column: original image; 2rd column: ground-truth segmentation; 3rd column: graph based segmentation; 4th column: EM segmentation; 5th column: proposed method.
Recognition of Characters Engraved on Reflective Surfaces
The characters engraved on the objects carry important semantic information such as reference number and model ID. The recognition of these characters is one of most efficient way to identify the object-model. However, various manufactured objects are highly reflective. The high intensity variations due to the reflection phenomena make the character recognition task very challenging. In this paper, we present a framework for the recognition of engraved characters on reflective surfaces. Our approach adapts local geometric features to make the recognition scale invariant and less sensitive to the reflections. Then the classification performances of SVM classifiers are analysed with three decision boundaries accompanied by individual and combination of the features. Our main contribution lies on boosting the recognition performances by introducing two cascaded SVM model based on the previously analysed accuracy rates. Multiple evaluation results presented in this paper show that the proposed method outperforms single classifier based methods for the recognition of characters engraved on reflective surfaces. Moreover, a challenging dataset is also released for further research purpose.
Human Action Recognition in the Video with Recurrent Neural Networks
In recent years, automatic recognition of human actions has become a very active research area, and has many perspectives in a variety of domains, such as video surveillance, video retrieval and human behavior understanding. The goal of the human action recognition is to analyze automatically the actions and their context in the video data. In recent years, there has been a growing interest in approaches which automatically build complex descriptors calculated around detected salient points for human action recognition. In our work, we propose to replace the last step by the recognition of a succession of states and their localization in the video. In the other words, we aim to separate the sequences into different parts, which represent the localization and repetition of actions and also indicate the instants without any action.
Real-time cellphone IMEI number localization and recognition for user identification
This project aims to develop a novel anonymous loggin system. Since the IMEI number of mobile phone is unique, it can also be the signature of the phone owner. Instead of loggin some website by providing personal information, use the IMEI number as the ID is considered as a more secured way. The proposed system detects and recognizes the IMEI number of a mobile phone based on computer vision and Optical Character Recognition technique.
Ancient monuments 3D digital model reconstruction(invited talk), in collaboration with archaeologist
Infernal puzzle
- 1200+ pieces (few mm to some cm) of high quality marble with high quality letter engraving coming from several wall plates
- fragments are damaged both in shape and in color
- clearly, some parts are missing and some fragments do not belong to the same set of plates
- manual reconstruction is impossible
- from archaeology point of view
- from 3D data processing point of view
fragments reconstruction by assembling cloud points use Iterative Closest Point (ICP)
Object Reference Number Tracking
In this paper, we present an industry driven manufactured reflective object identification framework. The framework contains two processes: offline and online. The offline processing takes static image as input, the features based on parameterized contours are extracted for constructing the object template. The online processing takes video sequence as input, the same type of features are extracted and further used for object model recognition by template matching. Then the reference number of the matched object can be localized by the prior knowledge from the object template. Once the reference number is initially localized, all the parameterized contour segments of the target object are tracked. By combining the information of all tracked segments, the reference number is localized in the entire video. This framework allows us to naturally handle the reflective objects and accurately localize the reference number for further object identification.
Face Beautification with Guided Filter
Just for fun !
The guided filter is a technique for edge-aware image filtering. Because of its nice visual quality, fast speed, and ease of implementation, the guided filter has witnessed various applications in real products, such as image editing apps in phones and stereo reconstruction.
The 1st image: original; 2rd image: meitu xx; 3rd: guided filter.
The guided filter is a technique for edge-aware image filtering. Because of its nice visual quality, fast speed, and ease of implementation, the guided filter has witnessed various applications in real products, such as image editing apps in phones and stereo reconstruction.
The 1st image: original; 2rd image: meitu xx; 3rd: guided filter.
Publications
International Journal:
Q. Lu, O. Laligant, E. Fauvet, A. Zakhorava : ”Entire Reflective Object Structure Understanding,”
ELSEVIER: Pattern Recognition Letters, 2015. Accepted
Q. Lu, O. Laligant, E. Fauvet, A. Zakhorava : ”Reflective Manufactured Object Reference Localization,”
Journal of Electronic Imaging. Under review
International Conference:
Q. Lu, O. Laligant, E. Fauvet, A. Zakhorava : "Entire Reflective Object Structure Understanding based on reflection motion features", IEEE Proceeding: British Machine Vision Conference (BMVC’15), Swansea, UK, 2015.
Q. Lu, O. Laligant, E. Fauvet, A. Zakhorava : "Manufactured Object Sub-Segmentation",
IAPR Machine Vision and Applications (MVA’15), Tokyo, Japan, 2015.
Q. Lu, O. Laligant, E. Fauvet, A. Zakhorava : "Local Surface Curvature Analysis",
SPIE Proceeding: International Conference of Digital Image Processing (ICDIP’15), Los Angeles, USA, 2015.
Q. Lu, O. Laligant, E. Fauvet, A. Zakhorava : "Local Surface Orientation Analysis",
SPIE Proceding: Quality Control by Artificial Vision (QCAV’15), Le Creusot, France, 2015.
P. Phutane, Q. Lu, E. Fauvet, O. Laligant : "Recognition of Carved Character on Reflective Surfaces", Asian
Conference of Pattern Recognition (ACPR’15). Under review
Invited Talk:
E.Fauvet, A.Hostein, O.Laligant, Q.Lu, F.Truchetet, "CIL XIII,2657 and new technologies", Oxford, UK. 2015.
Thesis:
Master Thesis: "Temporal Action localization in videos with Recurrent Neural Networks",
University of Paris Descartes, 2012.
Ph.D. Thesis: "Manufactured Object Recognition under Non-Controlled Environment",
University of Burgundy, 2015
Q. Lu, O. Laligant, E. Fauvet, A. Zakhorava : ”Entire Reflective Object Structure Understanding,”
ELSEVIER: Pattern Recognition Letters, 2015. Accepted
Q. Lu, O. Laligant, E. Fauvet, A. Zakhorava : ”Reflective Manufactured Object Reference Localization,”
Journal of Electronic Imaging. Under review
International Conference:
Q. Lu, O. Laligant, E. Fauvet, A. Zakhorava : "Entire Reflective Object Structure Understanding based on reflection motion features", IEEE Proceeding: British Machine Vision Conference (BMVC’15), Swansea, UK, 2015.
Q. Lu, O. Laligant, E. Fauvet, A. Zakhorava : "Manufactured Object Sub-Segmentation",
IAPR Machine Vision and Applications (MVA’15), Tokyo, Japan, 2015.
Q. Lu, O. Laligant, E. Fauvet, A. Zakhorava : "Local Surface Curvature Analysis",
SPIE Proceeding: International Conference of Digital Image Processing (ICDIP’15), Los Angeles, USA, 2015.
Q. Lu, O. Laligant, E. Fauvet, A. Zakhorava : "Local Surface Orientation Analysis",
SPIE Proceding: Quality Control by Artificial Vision (QCAV’15), Le Creusot, France, 2015.
P. Phutane, Q. Lu, E. Fauvet, O. Laligant : "Recognition of Carved Character on Reflective Surfaces", Asian
Conference of Pattern Recognition (ACPR’15). Under review
Invited Talk:
E.Fauvet, A.Hostein, O.Laligant, Q.Lu, F.Truchetet, "CIL XIII,2657 and new technologies", Oxford, UK. 2015.
Thesis:
Master Thesis: "Temporal Action localization in videos with Recurrent Neural Networks",
University of Paris Descartes, 2012.
Ph.D. Thesis: "Manufactured Object Recognition under Non-Controlled Environment",
University of Burgundy, 2015