Project
Dataset of Olympiad Mathematics Problem
Russian-Language Dataset for Training, Fine-Tuning, and Evaluating LLMs
About project
About project
About project
Developing the mathematical capabilities of large language models (LLMs) requires high-quality training data. However, the Russian-language ecosystem lacks large-scale olympiad-level datasets that enable models to learn complex reasoning, understand mathematical context, and solve non-standard problems. Such datasets are valuable not only for evaluating mathematical proficiency but also for assessing general reasoning ability—the capacity to construct logical chains of thought and draw well-founded conclusions.
CHALLENGE
CHALLENGE
CHALLENGE
Our goal was to build a Russian-language adaptation of the AIME dataset while preserving the accuracy of mathematical notation, the structure of the original materials, and the quality required for modern LLM training and evaluation.
SOLUTION
SOLUTION
SOLUTION
We developed a complete pipeline for collecting, processing, translating, and adapting problems from the American Invitational Mathematics Examination (AIME).
As the primary source, we used the Art of Problem Solving (AoPS) archive, which contains AIME problems from 1983 to 2025. An automated Python pipeline extracted problem statements, solutions, answers, and metadata while preserving the original LaTeX mathematical notation and document structure.
To translate the corpus into Russian, we implemented an automated workflow using the DeepSeek Reasoner model via its API. Carefully engineered prompts ensured that mathematical expressions, formatting, and technical terminology were preserved.
In the final stage, the data was converted into a machine-readable format suitable for training and evaluating LLMs. The dataset was organized into a test set and few-shot examples and enriched with metadata, problem categories, and multiple solution approaches.
RESULTS
RESULTS
RESULTS
We created ruAIME—a Russian-language dataset of olympiad mathematics problems that includes the original English problems, Russian translations, solutions, and metadata.
The dataset can be used for:
- training and fine-tuning mathematical LLMs
- evaluating models' ability to perform complex mathematical reasoning
- developing AI-powered educational applications and intelligent tutoring systems
- conducting research in cross-lingual machine learning
To validate the dataset, we evaluated the DeepSeek Reasoner model using the lm-evaluation-harness framework. On the Russian version of the benchmark, the model achieved an accuracy of approximately 70%, demonstrating that the dataset is well-suited for evaluating the mathematical reasoning capabilities of modern LLMs.
IMPLEMENTATION
IMPLEMENTATION
IMPLEMENTATION
A key feature of the project was the development of a fully automated data preparation pipeline, including:
- parsing and extracting problems from the AoPS web archive
- processing HTML content while preserving LaTeX mathematical markup
- automatically extracting answers and complete solutions
- cleaning the data, detecting duplicates, and performing quality control
- translating the corpus using an LLM through an API
- structuring the data in Parquet and JSON formats
- preparing benchmark test sets for use with lm-evaluation-harness
We also performed a statistical analysis of the dataset, examining answer distributions, topic balance, and verifying the absence of significant anomalies and duplicate entries. The analysis confirmed that the corpus is both high-quality and representative, making it well suited for machine learning tasks.
