VAMPIRE___Visual Active Memory Processes and Interactive REtrieval
VAMPIRE Events Publications Consortium Media archive
Intro Research Activities Scenario 'Mobile augmented reality' Scenario 'video annotations' Slideshow
object recognition and learningvisual trackingaction recognitionAR gearself localisationscene analysis
contextual analysisinteraction and augmented realitysystem integration


System integration in VAMPIRE is concerned with the realization of visual active memory applications, e.g. the mobile assistant in the augmented reality scenario as well as in the tennis annotation scenario. The main task for the system integration framework is to integrate several specific vision and scene interpretation modules with different temporal behaviour into a concise system architecture with high reactivity.

To solve those and other challenges arising from system integration in VAMPIRE from both conceptual and practical perspectives, the XCF SDK has been developed that supports researchers in developing systems build on the visual active memory concept.

The XCF Software Development Kit

The application scenarios impose several constraints on the software framework that had to be developed in VAMPIRE. System components should be able to exchange data with soft real-time performance and to memorise episodes, events and scenes. Given these requirements we focused on a shared repository and the distribution of algorithms. The whole XCF SDK consists of (1) the XML enabled Communication Framework (XCF) that allows to distribute components over several computing nodes, (2) the Active Memory XML Server and interface for coordination and data management, and (3) a supporting library named XMLTIO that supports users with an XPath-based API for simple XML processing. The complete integration software is licensed under the GPL and available for download at >>Sourceforge.

XML Communication

Since it is easy to learn, flexible and suited for abstract concept descriptions, we developed XML vocabularies for symbolic cognitive systems data (e.g., states, events, objects, etc.). The instance documents then contain directly the semantical information that is accessed and selected using the standardised XQuery/XPath mechanisms. Declarative, name-based selection of XML nodes with XPath expressions helps in building data types that can be extensible and in building systems that will not break if modifications occur.

Meta-information, e.g., data type specifications, are kept separate in corresponding XML schema files and are not encoded in the instance documents themselves. XML schemas are able to specify content models and ranges of allowed values in great detail. Providing fine grained sets of semantically grouped declarations in separate schemas with associated XML namespaces makes them reusable throughout different systems. Extensibility of data types is possible when rules for schema evolution are taken into account.

On top of the ICE communications engine XCF itself was developed to provide an easy to use middleware for building distributed object-oriented architectures that can efficiently exchange XML and referenced binary data (e.g., images). The referenced binary data structures are transmitted natively in a composite object together with the corresponding XML message similar to the recently proposed XML-binary Optimised Packaging (XOP) standard. This combines the flexibility of XML with the efficiency of low-level communication semantics for large amounts of binary data.

The XCF core itself features a pattern-based design and offers communication semantics like publisher/subscriber and (a)synchronous remote procedure calls/method invocations as well as event channels. All XCF objects and exposed methods can be dynamically registered at runtime.

Active Memory XML Server

To address the issue of data management, the active memory concept and implementation is applied for use in our system integration framework. On top of a native XML database library, an XCF-based client-server architecture was implemented. XML data like objects, actions or 3D locations and binary data like image patches are fed into the active memory server and can be retrieved via XPath statements. Thus, not only XML but also binary data can be shared by several system modules in parallel. For both kinds of data, powerful standard DBMS methods like insert, update, remove and query are exposed. Node selection and referral is based on XPath statements.

Coordination between the components is provided by a flexible event-notification mechanism. Event subscriptions can specify an XPath to narrow down documents of interest. Coordination is thus data-driven and not dependent on the presence of specific components. An easy-to-use client API is included in the XCF distribution. A prototypical part of an active memory system that exemplifies data flow and interaction between different components and the active memory is shown here:



Introspection and Compacting

System introspection is directly supported through an implementation of the interceptor pattern that helps in debugging and monitoring a running distributed system. Both, the active memory server as well as the XCF communication components are reflexive in that all kinds of data and process configurations are represented by a self-describing XML data model that can be analysed and changed on-the-fly.

An exemplary component that benefits directly from this, is the generic compacting filter for XML streams which we developed to reduce waste of system resources. An example for this is object recognition, where usually parts of the transmitted data may be false when the user looks around in the augmented reality scenario, other data may be redundant, e.g. when the user gazes at a static scene. In this situation object recognition will work well and send nearly identical object recognition results at a high frequency. Both, redundant and obviously false recognition results consume resources of the system in terms of network bandwidth and overall system performance. Therefore, a generic framework component based on reflection was developed that is able to compact processed information and detect relevant changes in the streams of XML information. This allows us to integrate existing modules even if their temporal behaviour is sub-optimal for the overall integrated system.

Besides this, system introspection at runtime is supported through generic framework tools (e.g. the MCI as shown in the figure below). Even more specific introspection is possible by an implementation of a distributed interceptor pattern in the active communication objects of XCF.


Rapid Prototyping

Though XCF is realized in C++ for performance reasons, there are also Matlab and Java bindings for rapid prototyping. It provides an easy to use basis for distributed processing in an asynchronous, decoupled fashion. Software tools to simulate active framework components as well as to use introspective features like e.g. intercepting a publisher or a server or to peek inside an active memory server are provided to help in debugging or evaluation of components during development.

Selected Publications