DeepSpeed支持yaml配置文件

[randydl]

这里提供一个简单的方案：

my_train.py

import sys
from omegaconf import OmegaConf
from llamafactory.train.tuner import run_exp


def parse_args():
    results = []
    for arg in sys.argv[1:]:
        if not arg.startswith('-') and arg.endswith('.yaml'):
            content = OmegaConf.load(arg)
            for key, value in content.items():
                results.append(f'--{key}={value}')
        else:
            results.append(arg)
    return results


if __name__ == '__main__':
    args = parse_args()
    sys.argv = [sys.argv[0]] + args
    run_exp()

my_run.sh

#!/bin/bash

deepspeed \
    --num_gpus 8 \
    --num_nodes 2 \
    --hostfile hostfile \
    --master_addr 10.252.32.12 \
    my_train.py "$@"

Run bellow code:

bash my_run.sh examples/train_lora/llama3_lora_sft_ds3.yaml