Extending Context and Exploring Multilingual Reasoning with Poro 2 Long

AMD Silo AI is releasing two new additions to the Poro 2 family: Poro 2 8B Long, which provides a significantly expanded context window, and Poro 2 8B Math Reasoning, a preview of a math-focused reasoning variant trained to solve mathematical problems in Finnish and English. Together, these releases and their training approach lay the foundation for future development of agentic workflows, enabling more dynamic and autonomous AI processes. 

Poro is a family of open-source, multilingual large language models (LLMs) optimized for Finnish language performance, developed by AMD Silo AI in partnership with TurkuNLP and trained on the LUMI supercomputer. The original Poro 2 models (8B and 70B parameters), released last year, were built on the Llama 3.1 8B and 70B base models using Continued Pretraining (CPT) to improve Finnish language competence. While Finnish serves as the demonstration language, the underlying methodology applies to any language: Poro exemplifies how base models can be adapted for multilingual use.  

Extending Context Window with 16x Increase in Tokens 

Poro 2 8B Long extends the model’s maximum context window from 8,192 tokens to 131,072 tokens, a 16x improvement over the original Poro 2 8B. This allows the model to process substantially more text in a single interaction. The expanded context window is especially valuable in Retrieval Augmented Generation (RAG) pipelines, where relevant documents are automatically retrieved and provided to the model as additional context when answering user questions. With a larger context window, the model can incorporate more retrieved information, supporting more comprehensive and well-informed responses.

Poro 2 8B Math Reasoning: Exploring Multilingual Mathematical Reasoning 

 Poro 2 8B Math Reasoning is an alternative version of Poro 2 8B Long that has been trained to reason about mathematical problems. As a preview release, it is not yet production-ready, but it represents a meaningful step forward. Many reasoning models today produce reasoning traces only in English or Chinese, while Poro 2 8B Math Reasoning can reason in both Finnish and English. This allows Finnish-speaking users to review reasoning traces in their native language, identify errors, and build confidence in results.   

Poro 3, expected later in the year, will build on this foundation as a more capable reasoning model with additional features. In the meantime, the Math Reasoning model, along with its associated code and training datasets, is released under an open-source license, making the resources available for researchers and developers to inspect, reproduce, and extend. Additionally, the release includes playbooks describing how the models were trained on AMD Instinct ^_TMGPU hardware, further expanding the community's access to practical model training techniques for multilingual AI development. 

Beyond these immediate capabilities, both the extended context window and the reasoning features introduced in this release lay out the groundwork for future agentic capabilities, enabling more dynamic and autonomous AI workflows in upcoming releases. 

Advancing Digital Sovereignty and European AI Capabilities 

AMD Silo AI develops open-source, multilingual language models and training resources for research and industry, work that includes the Poro family and initiatives such as Minerva. AMD Silo AI is a member of the OpenEuroLLM consortium, a pan‑European initiative of 20 universities, companies and EuroHPC centres building open, next‑generation language models for European languages. The Poro 2 8B Long release contributes to OpenEuroLLM’s mission to strengthen Europe’s digital sovereignty and competitiveness and to democratize access to high‑quality, multilingual AI. To learn more, visit the initiatives website.  

The released materials 

The Poro 2 8B Long and Poro 2 8B Math Reasoning models are available on HuggingFace. The updated playbooks and some of the training datasets are available on the AMD ROCm™ blog. 

• Poro 2 8B Long on Hugging Face
• Poro 2 8B Math Reasoning on Hugging Face
• The playbook for both Poro 2 8B Long and Poro 2 8B Math Reasoning on the AMD ROCm™ Blog. 

For more information on previous Poro releases, visit the AMD ROCm™ blog.