Yandex Open Source Open Source LLM Training Tool that Saves Up to 20% on GPU Resources

Yandex introduced YaFSDP, a new and open source method for training large language models (LLM). YaFSDP currently symbolizes the most publicly available effective method to improve GPU communication and reduce memory usage in LLM education. The method offers up to 26% speedup compared to FSDP, depending on the architecture and number of parameters. Reducing the training time of LLMs by using YaFSDP has the potential to save up to 20% in GPU resources.

With the aim of making a meaningful contribution to the development of the global AI community, Yandex has made YaFSDP available to LLM developers and AI enthusiasts around the world.

Serving as a senior developer at Yandex and part of the team behind YaFSDP Mikhail Khruschev, He said: â € œ At the moment, we are actively experimented in the form of a width of Yafsdp in the form of model architects and parameter dimensions. We are excited to share our advances in LLM education with the global ML community, contributing to increased accessibility and productivity for researchers and developers around the world.â€

Contributions of YaFSDP to Language Model Development Projects in Turkey

Various technology and financial institutions in Turkey are developing Turkish language models and carrying out important projects worldwide in this field. These big language models developed in Turkey can gain significant advantages with the YaFSDP method offered by Yandex. The GPU savings and training accelerations offered by YaFSDP can contribute to realizing these projects more efficiently and cost-effectively. In particular, saving up to 20% in GPU resources and achieving a speedup of up to 26% during language model training makes these projects both economical and operational. It can make it more sustainable.

Why should YaFSDP be preferred?

Education for LLMs is a time-consuming and resource-intensive process. Machine learning engineers and companies that want to develop their own LLMs are forced to spend a significant amount of time and GPU resources, and therefore money, on training these models. The larger the model, the more time and expense it takes to train it.

Yandex YaFSDP eliminates the inefficiency in GPU communication, making GPU interactions seamless and ensuring that the training uses only as much processing memory as necessary.

By optimizing learning speed and performance, YaFSDP helps AI developers around the world use less computing power and GPU resources when training their models. For example, in a preliminary training scenario involving a model with 70 billion parameters, using YaFSDP has the potential to save the equivalent of approximately 150 GPU resources. This works out to roughly 50% per month depending on the virtual GPU provider or platform. Savings of $500K to $1.5MIt means Â.

YaFSDP training efficiency

YaFSDP, an improved version of FSDP, performs better compared to the FSDP method in communication-heavy phases of LLM education, such as pre-training, alignment and fine-tuning. YaFSDP’s final speedup on Llama 2 and Llama 3 shows significant improvements in training speed, reaching 21% and 26% on Llama 2 70B and Llama 3 70B respectively turns out to be putting it.

Mikhail Khruschev said, “YaFSDP showed impressive results on models ranging from 13 to 70 billion parameters, with a particularly strong performance in the 30 to 70 billion range. “YaFSDP is the most suitable among the currently widely used open source models based on the LLaMA architecture,” he says.

YaFSDP is not the first open source tool from Yandex. The company has previously shared other tools that have become popular among the ML community:

• CatBoost, high-performance library for gradient boosting in decision trees.
• YTsaurus, big data platform for distributed storage and processing.
• AQLM is the most advanced tool for extreme compression of large language models, jointly developed by Yandex Research, HSE University, Skoltech, IST Austria and NeuralMagic. Advanced quantization algorithm.
• Developed in collaboration with Petals, Yandex Research, HSE University, University of Washington, Hugging Face, ENS Paris-Saclay and Yandex School of Data Analysis, the library is designed to simplify the training and fine-tuning process of LLMs.