Can OpenAI models be fine-tuned?

You can adapt OpenAI’s models to your own data by using the fine-tuning API. Under the hood, fine-tuning is simply supervised learning on examples you provide, so you teach the model to prefer your style of prompts and completions instead of relying solely on its base knowledge.

At present, all of the GPT-3 family models—originally known as ada, babbage, curie and davinci, and today accessed as babbage-002 and davinci-002—can be fine-tuned, and this capacity remains supported for those variants. In addition, OpenAI now lets you fine-tune the GPT-3.5-turbo series. You prepare your training data as a JSONL file of prompt-completion pairs, upload it via the CLI or the Python client, and then invoke the fine-tuning endpoint, specifying your base model and files. Once training completes, you receive the name of a new “ft:” model which you can address just like any other in your Completions or Chat Completions calls  .

The fine-tuning workflow looks like this in Python: first you convert your examples into the JSONL format that the API expects, then you upload the file with

openai.File.create(file=open("mydata.jsonl"), purpose="fine-tune")

and then calling

openai.FineTune.create(model="gpt-3.5-turbo", training_file="file-XYZ", n_epochs=3)

Behind the scenes, OpenAI will train a copy of the base model on your data. When it finishes, you’ll receive a callback or you can poll the job status; the response includes your new model’s name, which you then use in subsequent requests to get outputs that reflect your custom examples.

Right now the only models you cannot fine-tune via the public API are GPT-4 and its variants. OpenAI has signaled that GPT-4 fine-tuning is on the roadmap but has not yet enabled it for general users. If your needs demand deeper customization of GPT-4 today, you can experiment with the ChatGPT “Custom GPT” builder in the developer settings, but that is a separate product from the API’s fine-tuning interface.

When you fine-tune, keep in mind that quality depends on data quality and quantity. Hundreds to thousands of well-chosen examples usually suffice for tone or formatting adjustments, while more complex behavior changes may require tens of thousands. Because the API charges both for training tokens and for usage of your fine-tuned model, you’ll also want to monitor costs: training the model costs a few cents per thousand tokens, and subsequent usage runs at rates comparable to the base model but with an extra fine-tuning premium.

You can fine-tune some of OpenAI’s deployed models today by preparing JSONL data, uploading it, and creating a fine-tune job. You’ll receive a custom model name in return, which you simply plug into your normal completion calls.