Dataset for Llama, Mistral and local models
JSONL in TRL and Unsloth's format, ready to fine-tune a model on your own machine.
Sent to servers: 0 BNothing left your device
About this tool
If you are fine-tuning an open model on your machine or on a rented GPU, the documents should never pass through someone else's service just to become a text file. Here they do not.
The conversation format produces the "messages" field with its roles, which TRL, Unsloth and llama-factory load with no translation in between. The plain text format is for continued pretraining on your own material.
How it works
- 1
Drop the PDFs.
- 2
Pick the conversation format.
- 3
Download the JSONL and load it with load_dataset.
Questions
- Does it work for LoRA and QLoRA?
- Yes: the dataset format is independent of the fine-tuning technique. LoRA changes how weights are updated, not how data is read.
- What about DPO or preference data?
- We do not generate it, deliberately: a chosen/rejected pair requires producing two answers, which means calling a model with your documents. That would break the one thing we promise.
- Can I automate this for hundreds of files?
- Yes, with the CLI: npx sirdas dataset *.pdf --split -d ./ds. Same engine.