next up previous
Next: PhD Supervision Up: Developing Flexible Models for Previous: Image Annotation

Relationship to Original Proposal

In the original proposal I claimed that at the end of three years I would have achieve the following main milestones:

1.
Demonstrated building flexible projective models from sets of uncalibrated stereo pairs and using them for image search
2.
Described a system for building hierarchical flexible shape models, and demonstrate their advantages over `flat' models.
3.
Demonstrated new techniques of building models from small data sets
4.
Demonstrated building and using multi-scale flexible shape models
5.
Demonstrated search techniques which are more robust to occlusion
6.
Describe a framework for point-based and curve-based flexible models, and give quantitative results comparing the different approaches in different circumstances
7.
Demonstrated methods on data from real problems

The current status of these goals is as follows:

1.
Algorithms have been developed for building statistical models of the 3D shape of objects from pairs of uncalibrated images of examples of the object, and we have demonstrated image search using them. However, we have yet to develop algorithms for building 3D models from single images
2.
Some details of how to build and use hierarchical shape models have been developed, but the techniques have yet to be applied to a serious problem.
3.
We have developed techniques to add extra `artificial' shape variation to a shape model. This allows us to build models from small training sets (or even a single example) and to fit them to other examples. The extra variation takes the form of smooth elastic deformation. This produces models which are not as specific as pure statistical models, so can deform to unusual shapes. They should therefore be used with care, but are very useful in interactive search tools, in which the user can correct any mistakes.
4.
We have developed multi-scale appearance models, in which the texture model varies at each level of an image pyramid. Although they use the same shape model at each level, the texture part of the model deals with the variation at each scale [2].
5.
The Active Shape Model can deal with small amounts of occlusion well in its standard form. We have experimented with modifications to explicitly detect occluded parts, but so far the performance has been little different to that of the original ASM. I have recently incorporated a robust kernel into the calculation of the residual term during the AAM update step. The aim is to reduce the significance of sampled pixels which are significantly different from those expected, which are likely to be in occluded regions. Results are encouraging, but as yet no formal experiments have been done to assess the performance properly.
6.
I now believe that most 'curve-based' models (eg using B-splines, fourier or spherical harmonics) can be shown to be equivalent to a related point based model. Essentially the argument is that if a model describes the position of one or more curves, one can systematically place landmarks along these curves to generate a set of points. By manipulating the curves we manipulate the points, and vice-versa. Thus we could simply manipulate the points with a (linear or non-linear) shape model, reducing the need for an alternative curve-based model.
7.
The algorithms have been applied to numerous problems, including face tracking and recognition, industrial inspection problems and medical applications in two and three dimensions.

In practice, priorities have changed from those in the original proposal. My work tends to be driven by applications being tackled elsewhere in the department and the availability of relevant images for a particular problem. The modelling and search algorithms I have been working on are being used to tackle many different problems. These often generate interesting technical and theoretical problems, which provide fruitful subjects for research. For instance, the Active Appearance Model was developed in response to a need to fit appearance models to face images rapidly, but is applicable to many other problem domains. Similarly, work on the interactive markup tool was started in response for the need to label training examples quickly when new applications are being investigated.


next up previous
Next: PhD Supervision Up: Developing Flexible Models for Previous: Image Annotation
Tim Cootes
12/22/1998