Toolverse
All skills
Security

training-llms-megatron

by davila7

Trains large language models (2B-462B parameters) using NVIDIA Megatron-Core with advanced parallelism strategies. Use when training models u003e1B parameters, need maximum GPU efficiency (47% MFU on H100), or require tensor/pipeline/sequence/context/expert parallelism.

Installation

Pick a client and clone the repository into its skills directory.

Installation

Quick info

Author
davila7
Category
Security
GitHub repo

About this skill

Trains large language models (2B-462B parameters) using NVIDIA Megatron-Core with advanced parallelism strategies. Use when training models u003e1B parameters, need maximum GPU efficiency (47% MFU on H100), or require tensor/pipeline/sequence/context/expert parallelism. Production-ready framework used for Nemotron, LLaMA, DeepSeek.

How to use

  1. Zainstaluj Megatron-Core za pomocą Dockera (rekomendowane) — uruchom obraz nvcr.io/nvidia/pytorch:25.04-py3 z flagą --gpus all, lub zainstaluj pakiet bezpośrednio poleceniem pip install megatron-core.

  2. Przygotuj konfigurację paralelizmu na podstawie rozmiaru modelu — dla modelu 7B na 8 GPU użyj tensor parallel=1, pipeline parallel=1, data parallel=8; dla 70B na 64 GPU ustaw tensor parallel=4, pipeline parallel=4, data parallel=4.

  3. Skonfiguruj hiperparametry trenowania zgodnie z wybraną strategią paralelizmu i rozmiarem zestawu danych.

  4. Uruchom trenowanie rozproszone poleceniem torchrun --nproc_per_node=2 examples/run_simple_mcore_train_loop.py dla dwóch GPU, lub użyj gotowego skryptu ./examples/llama/train_llama3_8b_fp8.sh do trenowania modelu LLaMA-3 8B.

  5. Monitoruj metryki wydajności podczas trenowania, aby upewnić się, że osiągasz oczekiwaną efektywność obliczeniową (Model FLOP Utilization).

Related skills

accessibility-compliance

by wshobson

Implement WCAG 2.2 compliant interfaces with mobile accessibility, inclusive design patterns, and assistive technology support. Use when auditing accessibility, implementing ARIA patterns, building for screen readers, or ensuring inclusive user experiences.

Security
2173

ui-audit

by openclaw

AI skill for automated UI audits. Evaluate interfaces against proven UX principles for visual hierarchy, accessibility, cognitive load, navigation, and more. Based on Making UX Decisions by Tommy Geoco.

Security
1223

openapi-spec-generation

by wshobson

Generate and maintain OpenAPI 3.1 specifications from code, design-first specs, and validation patterns. Use when creating API documentation, generating SDKs, or ensuring API contract compliance.

Security
18109

python-expert

by Shubhamsaboo

Senior Python developer expertise for writing clean, efficient, and well-documented code.\nUse when: writing Python code, optimizing Python scripts, reviewing Python code for best practices,\ndebugging Python issues, implementing type hints, or when user mentions Python, PEP 8,

Security
2777

zendesk

by vm0-ai

Zendesk Support REST API for managing tickets, users, organizations, and support operations. Use this skill to create tickets, manage users, search, and automate customer support workflows.

Security
11100

solidity-security

by wshobson

Master smart contract security best practices to prevent common vulnerabilities and implement secure Solidity patterns. Use when writing smart contracts, auditing existing contracts, or implementing security measures for blockchain applications.

Security
10105