We have developed a technique for building compact models of the shape and appearance of variable structures in 2-D images. The structures of interest may be flexible objects such as biological organs or the variability may come from the loose connections between rigid parts, such as in assemblies of industrial components. The models are derived from the statistics of labelled images containing examples of the objects. Each model consists of a flexible shape template (a Point Distribution Model - PDM ) describing how the relative locations of important points on the shapes can vary, and a statistical model of the expected grey-levels in a region around each point. Given a new image of an object, and an estimate of its position, we can fit an instance of a model to the object using an iterative optimisation technique. At every point of the model we search the image nearby for areas of grey-level landscape which look like those around the points in the training images. We can then update the current estimate of shape and pose parameters to give a better fit of the model to the image. Since we have a model of the allowed shapes we can apply constraints to ensure the model instance remains `legal'. This has the effect of only allowing these Active Shape Models to deform in ways exhibited in the training set, making them quite specific. We have recently developed a multi-resolution version of the search algorithm which allows faster and more accurate location of new image structures. We have also been developing model building techniques to deal with small training sets , where statistical models perform poorly, and new types of non-linear shape model which can compactly represent a wider variety of shape variation . A summary of the modelling and search techniques is given in the Recent Research Work report, with more details in [15,16,17,1,19,20,24].
These models have several advantages over the methods used by other authors. In particular they can be used to represent a wide variety of objects, both man-made and biological, the same techniques being applied in every case (they do not have to be `hand crafted' for each example). The models explicitly describe the variations in shape and, being compact linear representations, can be used efficiently in image search.
We have extended some of these techniques to model and search for structures in 3-D volume data  and image sequences .