Dataset formats for fine-tuning: chat, Alpaca, ShareGPT and which one you need
The field names each trainer expects, verified against its official documentation, and the details that break a file without telling you why.
By Brayan Pedraza10 min read
In short
Almost the whole ecosystem eats JSONL in one of four shapes: a text field for continued pretraining, a messages field with roles for chat (TRL, Unsloth and OpenAI fine-tuning), the prompt and completion fields, or Alpaca's three fields instruction, input and output, expected by Axolotl, LLaMA-Factory and Unsloth. Pick by the trainer you will use, not by what looks most complete: a file with the wrong field names fails validation and the error rarely says which field is the odd one.
The table that solves 90% of cases
| Shape | Fields | What it is for | Who eats it |
|---|---|---|---|
| Text | text | Continued pretraining on your material | TRL |
| Conversation | messages[{role, content}] | Supervised chat fine-tuning | TRL, Unsloth, OpenAI |
| Instruction | prompt, completion | Completion fine-tuning | TRL |
| Alpaca | instruction, input, output | Instruction fine-tuning | Axolotl, LLaMA-Factory, Unsloth |
| ShareGPT | conversations[{from, value}] | Public chat datasets | Many trainers |
| Preference | prompt, chosen, rejected | DPO and friends | TRL |
Source for the TRL row: its dataset formats page, which documents them as conversational and standard.
The details that break things
These are not in the tutorials and each costs an afternoon.
PRMTrainer uses plurals
Where the rest use completion and label, PRMTrainer expects completions and labels. A file valid for the rest of TRL fails here with no clear explanation.
OpenAI's DPO does not use TRL's names
The most expensive item on the list, because it looks like you already solved it. TRL uses prompt, chosen and rejected. OpenAI uses input — an object wrapping messages — plus preferred_output and non_preferred_output, and accepts a single turn only. An exporter assuming TRL produces files that fail OpenAI's validation.
OpenAI supports per-message weights
In supervised fine-tuning you can set weight: 0 on an assistant message so the model is not trained on it. Useful when an example holds one correct answer and another you want to keep as context but not imitate.
LLaMA-Factory needs an extra entry
The JSONL is not enough: the set must be registered in dataset_info.json with its columns and tags. If training starts and learns nothing, check there before blaming the model.
ShareGPT is deprecated as an option, not as a format
In Axolotl, type: sharegpt is deprecated. Today you declare type: chat_template with message_property_mappings. The data format is still perfectly usable; what changed is how you tell the trainer to interpret it.
Anthropic has no documented format of its own
As far as we could verify, there is no public Anthropic fine-tuning format. The only official readable one is AWS Bedrock's for Claude 3 Haiku: {"system": "...", "messages": [...]}, with a minimum of 32 records.
Which to choose, in three questions
Do you want to teach it new knowledge? Then you probably do not want fine-tuning. Fine-tuning teaches form: tone, structure, output format, domain vocabulary. For facts that change, RAG is cheaper, more updatable and more auditable. If you are fine-tuning anyway, use text for continued pretraining.
Do you want it to answer the way you would? Conversation format (messages). It is the most portable: TRL, Unsloth and OpenAI fine-tuning read the same file.
Are you using Axolotl, LLaMA-Factory or Unsloth with community recipes? Alpaca, because that is what their example configs expect and it saves you a translation step.
For RAG, none of the above
If the destination is a vector store rather than a trainer, the formats are different:
| Destination | Shape |
|---|---|
| LangChain | Document with page_content and metadata |
| LlamaIndex | TextNode with id_, text and metadata |
| Pinecone, Qdrant, pgvector, Chroma | id, text and metadata |
One detail that saves pain: make the id stable and derived from the content. Then re-indexing updates the node instead of duplicating it, and you can regenerate the index without wiping it first.
What can be generated without calling a model
There is one more format that almost nobody produces from documents and that needs no model: verifiable reward pairs for reinforcement learning.
DeepSeek-R1 (arXiv:2501.12948, §2.2.2) explains why it matters:
«for math problems with deterministic results, the model is required to provide the final answer in a specified format […] enabling reliable rule-based verification of correctness […] We do not apply the outcome or process neural reward model […] because we find that the neural reward model may suffer from reward hacking.»
AI2's reference set for this has messages, ground_truth and dataset. And GRPOTrainer only requires the prompt column: any extra column reaches the reward function as a named argument.
Practical conclusion: an RLVR dataset is JSONL with prompt and ground_truth, and a table in a PDF is already a set of facts with a single answer. «How much did the lab test cost?» has one answer, it is printed on the page, and checking it is comparing two strings.
Two quality rules to apply when generating them, or the dataset punishes the model for being right:
- If a row key repeats, that question is not emitted: it would have two answers.
- If there is a tie for a column's maximum or minimum, same.
How to export it
In the browser: PDF to AI data has the eight formats in a dropdown and shows the first lines before you download.
In the terminal:
npx sirdas dataset docs/*.pdf --format chat --split --hf -d dataset/
npx sirdas dataset docs/*.pdf --format alpaca -d ds/
npx sirdas rl invoices/*.pdf --lang en -o rl.jsonl
npx sirdas rl --reward-fn > reward.pyWith --hf you get train.jsonl, validation.jsonl, test.jsonl and the Hugging Face card. With --reward-fn, the Python reward function ready to paste into GRPOTrainer, using the same comparison criterion that generated the pairs.