Xtuner 大模型单卡低成本微调之本地实战--医学问答

xtuner convert pth_to_hf internlm_chat_7b_qlora_medqa2019_e3.py ./work_dirs/internlm_chat_7b_qlora_medqa2019_e3/epoch_3.pth/mp_rank_00_model_states.pt ./任何使用xtuner可以查看这里

安装和使用

github 文末有

自定义微调

场景需求

基于 InternLM-chat-7B 模型，用 MedQA 数据集进行微调，将其往医学问答领域对齐。

真实数据预览

问题	答案
What are ketorolac eye drops?（什么是酮咯酸滴眼液？）	Ophthalmic ketorolac is used to treat itchy eyes caused by allergies. It also is used to treat swelling and redness (inflammation) that can occur after cataract surgery. Ketorolac is in a class of medications called nonsteroidal anti-inflammatory drugs (NSAIDs). It works by stopping the release of substances that cause allergy symptoms and inflammation.
What medicines raise blood sugar? （什么药物会升高血糖？）	Some medicines for conditions other than diabetes can raise your blood sugar level. This is a concern when you have diabetes. Make sure every doctor you see knows about all of the medicines, vitamins, or herbal supplements you take. This means anything you take with or without a prescription. Examples include: Barbiturates. Thiazide diuretics. Corticosteroids. Birth control pills (oral contraceptives) and progesterone. Catecholamines. Decongestants that contain beta-adrenergic agents, such as pseudoephedrine. The B vitamin niacin. The risk of high blood sugar from niacin lowers after you have taken it for a few months. The antipsychotic medicine olanzapine (Zyprexa).

数据准备

以 Medication QA 数据集为例

问题	药物类型	问题类型	回答	主题	URL
aaa	bbb	ccc	ddd	eee	fff

将数据转为 XTuner 的数据格式
目标格式：(.jsonL)

[{
   
    "conversation":[
        {
   
            "system": "xxx",
            "input": "xxx",
            "output": "xxx"
        }
    ]
},