Generative AI

At the heart of generative AI are large language models (LLMs) with powerful transformer architectures. These models are trained on high-performance computers using extensive amounts of data and form the engine of modern AI systems, from retrieval-augmented generation (RAG) approaches to agentic systems.

Research at Fraunhofer IAIS focuses on the further development of fundamental AI technologies and the responsible design of large language models.

• AI-based curation and synthesis of high-quality training data: We are researching new methods in which (domain-specific) training data is selected and generated with the help of AI. This data can be used to make training more efficient and develop more powerful models.
• Improved reasoning skills: We are researching reasoning models that first break down complex tasks into understandable sub-steps and then solve them coherently, step by step.
• Knowledge distillation: We are exploring new approaches to transfer selected capabilities of large models into compact, energy-efficient models that can be operated locally.
• Domain-specific adaptation: We are researching methods for optimizing lean, specialized models in order to solve domain-specific tasks more precisely and resource-efficiently.
• Multi-agent systems: We research modular agent systems that dynamically combine different models depending on the task at hand and cooperatively solve complex problems.
• Multimodality: We conduct research on multimodal foundation models that can process text, images, audio, video, and structured data in various combinations. Our focus is on new methods that improve multimodal reasoning across multiple modalities.

Research into generative AI is closely linked to the Lamarr Institute's Foundation Model Group. An important milestone is the development of the Teuken models as part of the OpenGPT-X research project, which have set new standards through innovative tokenization and multilingual training data.

Our research aims to develop the next generation of AI systems that are more powerful, efficient, and adaptable. We are making great strides in the areas of data curation and synthesis, reasoning, knowledge distillation, domain-specific model adaptation, and dynamically composable multi-agent systems. This enables companies to deploy AI solutions that are optimally tailored to their individual requirements and can be seamlessly integrated into existing processes.