OpenAI’s text generation models are pre-trained on large datasets and can be further customized through a process called fine-tuning. Fine-tuning improves the model’s performance by training it with additional examples that may not fit into a single prompt. Once fine-tuned, the model generates higher-quality results with shorter prompts, reducing costs and enabling faster response times.
The fine-tuning process involves preparing and uploading training data, training a new fine-tuned model, evaluating the results, and iterating if necessary. Key benefits of fine-tuning include higher output quality, the ability to train with more examples, cost reduction through shorter prompts, and faster response times.
The overall fine-tuning process can be summarized in three steps: preparing and uploading training data, training a new fine-tuned model, and evaluating the results before iterating if necessary. After fine-tuning, the model is ready to use for generating content tailored to specific needs or requirements.
Common use cases
“Fine-tuning OpenAI text generation models can make them better for specific applications, but it requires a careful investment of time and effort.”
OpenAI
- Setting style, tone, and formatting: Fine-tuning can help adjust the model’s output to match a specific style, tone, or format.
- Improving output reliability: Fine-tuning can enhance the consistency and accuracy of the model’s responses.
- Handling complex prompts: Fine-tuning can enable the model to better follow complex prompts and generate appropriate responses.
- Addressing edge cases: Fine-tuning can help the model handle niche scenarios or specific requirements more effectively.
- Performing new tasks: Fine-tuning can help the model learn new skills and perform tasks that may be difficult to convey through prompts alone.
- Reducing costs and wait times: By generating higher-quality results with shorter prompts, fine-tuning can reduce costs and enable faster response times.
Model + data format
Fine-tuning is currently available for the following models:
- gpt-3.5-turbo-1106 (recommended)
- gpt-3.5-turbo-0613
- babbage-002
- davinci-002
- gpt-4-0613 (experimental)
“GPT-4 fine-tuning is in experimental access and eligible developers can request access via the fine-tuning UI.”
Open AI
Data form
- gpt-3.5-turbo
{“messages”: [{“role”: “system”, “content”: “Marv is a factual chatbot that is also sarcastic.”}, {“role”: “user”, “content”: “What’s the capital of France?”}, {“role”: “assistant”, “content”: “Paris, as if everyone doesn’t know that already.”}]}
{“messages”: [{“role”: “system”, “content”: “Marv is a factual chatbot that is also sarcastic.”}, {“role”: “user”, “content”: “Who wrote ‘Romeo and Juliet’?”}, {“role”: “assistant”, “content”: “Oh, just some guy named William Shakespeare. Ever heard of him?”}]}
{“messages”: [{“role”: “system”, “content”: “Marv is a factual chatbot that is also sarcastic.”}, {“role”: “user”, “content”: “How far is the Moon from Earth?”}, {“role”: “assistant”, “content”: “Around 384,400 kilometers. Give or take a few, like that really matters.”}]}
- babbage and davinci:
{“prompt”: “<prompt text>”, “completion”: “<ideal generated text>”}
{“prompt”: “<prompt text>”, “completion”: “<ideal generated text>”}
{“prompt”: “<prompt text>”, “completion”: “<ideal generated text>”}
“To fine-tune a model, you are required to provide at least 10 examples. We typically see clear improvements from fine-tuning on 50 to 100 training examples with gpt-3.5-turbo but the right number varies greatly based on the exact use case.”
Open AI
Commission
cost = 1kトークンあたりの基本コスト * 入力ファイルのトークン数 * 学習エポック数
- Check format + quote: Code
Token limits
The token limit for training examples varies depending on the chosen model’s context length. In the GPT-3.5 Turbo 1106 model, the maximum context length is 16,385 tokens, so each training example is limited to the same token count. For the GPT-3.5 Turbo 0613 model, the limit is 4,096 tokens per training example.
If examples exceed the default length, tokens are removed from the end to meet the maximum context length. To ensure that the entire training example fits within the context, it’s essential to keep the total token count in the message content below the specified limit.
Counting Tokens: Notebook
Example
In this example, we fine-tune the GPT-3.5 Turbo 0613 model using a dataset specifically designed to train the model for classifying customer messages into both primary and sub-categories. The primary goal of this fine-tuning is to enhance the model’s ability to accurately categorize customer messages. The model’s classification output is formatted as JSON.
Data
| Train | Validate | Test |
|---|---|---|
| 73 | 19 | 10 |
- Training data:
{“messages”: [{“role”: “user”, “content”: “How do I get help with accessibility in your branches?”}, {“role”: “assistant”, “content”: “{\”Top Category\”: \”Customer Service\”, \”Sub Category\”: \”Special Requests\”}”}]}
{“messages”: [{“role”: “user”, “content”: “I can\u2019t log into my mobile banking app.”}, {“role”: “assistant”, “content”: “{\”Top Category\”: \”Online and Mobile Banking\”, \”Sub Category\”: \”Technical Support\”}”}]}
{“messages”: [{“role”: “user”, “content”: “I suspect a fraudulent transaction on my account, what should I do?”}, {“role”: “assistant”, “content”: “{\”Top Category\”: \”Fraud and Security\”, \”Sub Category\”: \”Reporting Fraud\”}”}]}
…
- Validation data
{“messages”: [{“role”: “user”, “content”: “How do I set up autopay for my bills?”}, {“role”: “assistant”, “content”: “{\”Top Category\”: \”Payments and Transfers\”, \”Sub Category\”: \”Bill Payments\”}”}]}
{“messages”: [{“role”: “user”, “content”: “How do I give feedback on a recent interaction with a bank representative?”}, {“role”: “assistant”, “content”: “{\”Top Category\”: \”Customer Service\”, \”Sub Category\”: \”Feedback and Suggestions\”}”}]}
{“messages”: [{“role”: “user”, “content”: “I need a copy of my account statement for the last six months.”}, {“role”: “assistant”, “content”: “{\”Top Category\”: \”Account Management\”, \”Sub Category\”: \”Account Information\”}”}]}
…
- Test data
| Support Query | Top Category | Sub Category |
| Are there any fees for using online banking? | Fees and Charges | Understanding Fees |
| What types of home loans do you offer for first-time buyers? | Loans and Mortgages | Loan Application |
| How do I file a complaint about a recent issue? | Customer Service | Complaint Handling |
| Can I cancel a pending transfer? | Payments and Transfers | Wire Transfers |
| How do I open a new savings account? | Account Management | Account Opening |
| The ATM didn’t dispense cash but my account was charged. | Transaction Services | Failed Transactions |
| What are the steps to activate my new card? | Card Services | Card Activation/Deactivation |
| How do I download and install the bank’s mobile app? | Online and Mobile Banking | Technical Support |
| What are the bank’s security measures against fraud? | Fraud and Security | Security Measures |
| How do I open a fixed deposit account? | Savings and Investments | Account Types |
Tweak and evaluate
- Fine-tuning the process
- Result (limit output of 50 tokens)
| Support Query | Top Category | Sub Category | Prediction (Fine-tune) | Prediction (Origin) |
| Are there any fees for using online banking? | Fees and Charges | Understanding Fees | {“Top Category”: “Fees and Charges”, “Sub Category”: “Understanding Fees”} | The fees associated with online banking can vary depending on the bank or financial institution you are using. Some banks may offer online banking services for free, while others may charge fees for certain transactions or services. Common fees that may be associated with online banking include |
| What types of home loans do you offer for first-time buyers? | Loans and Mortgages | Loan Application | “Top Category”: “Loans and Mortgages”, “Sub Category”: “Loan Application”} | As an AI language model, I don’t offer home loans myself. However, I can provide you with information on common types of home loans that are often available for first-time buyers. These include: 1. Conventional Loans: These are traditional mortgage |
| How do I file a complaint about a recent issue? | Customer Service | Complaint Handling | {“Top Category”: “Customer Service”, “Sub Category”: “Feedback and Suggestions”} | To file a complaint about a recent issue, follow these steps: 1. Identify the appropriate organization or entity: Determine which organization or entity is responsible for addressing your complaint. This could be a company, government agency, educational institution, or any other relevant |
| Can I cancel a pending transfer? | Payments and Transfers | Wire Transfers | {“Top Category”: “Payments and Transfers”, “Sub Category”: “Wire Transfers”} | It depends on the specific circumstances and the policies of the institution or service you are using for the transfer. Generally, if the transfer is still pending, there may be a chance to cancel it. You should contact the institution or service provider involved in the |
| How do I open a new savings account? | Account Management | Account Opening | {“Top Category”: “Savings and Investments”, “Sub Category”: “Account Types”} | To open a new savings account, you can follow these steps:1. Research and compare different banks or financial institutions: Look for banks or credit unions that offer savings accounts with favorable interest rates, low fees, and convenient access.2. Choose the |
| The ATM didn’t dispense cash but my account was charged. | Transaction Services | Failed Transactions | {“Top Category”: “Transaction Services”, “Sub Category”: “Failed Transactions”} | I’m sorry to hear that. In such a situation, it is important to take the following steps: 1. Check your account balance: Confirm if the amount has been deducted from your account balance. You can do this by checking your account statement online |
| What are the steps to activate my new card? | Card Services | Card Activation/Deactivation | {“Top Category”: “Card Services”, “Sub Category”: “Card Activation”} | The steps to activate a new card may vary depending on the type of card and the issuing bank. However, here are some general steps that can help you activate your new card: 1. Read the instructions: Carefully review any instructions or information provided |
| How do I download and install the bank’s mobile app? | Online and Mobile Banking | Technical Support | {“Top Category”: “Online and Mobile Banking”, “Sub Category”: “Mobile Banking”} | To download and install a bank’s mobile app, follow these general steps: 1. Open the app store on your mobile device. If you have an iPhone, open the App Store, and if you have an Android device, open the Google Play Store |
| What are the bank’s security measures against fraud? | Fraud and Security | Security Measures | {“Top Category”: “Fraud and Security”, “Sub Category”: “Security Measures”} | Banks employ various security measures to protect against fraud. Some common security measures include: 1. Multi-factor authentication: Banks often require customers to provide multiple forms of identification, such as passwords, PINs, security questions, or biometric data (f |
| How do I open a fixed deposit account? | Savings and Investments | Account Types | {“Top Category”: “Savings and Investments”, “Sub Category”: “Account Types”} | To open a fixed deposit account, you can follow these steps: 1. Research and choose a bank: Look for banks that offer fixed deposit accounts with competitive interest rates and favorable terms. 2. Visit the bank branch: Go to the nearest branch of |
Conclusion
After fine-tuning, the model demonstrates significantly enhanced performance compared to the original version. The refined output is more concise, well-structured in JSON format, and closely aligns with the input data used during fine-tuning. Additionally, the model’s accuracy in classifying customer messages into specific categories has improved notably.
These results underscore the success of the fine-tuning process in achieving our primary objective of accurately categorizing customer messages. By utilizing the improved model through the GPT API, businesses can benefit from increased efficiency and cost-effectiveness.
While currently, fine-tuned models are only accessible via the API, future integrations with OpenCV and other tools may enable broader applications beyond API usage, potentially expanding the capabilities of AI assistants and other related systems.
