Instruction fine-tuning

https://www.coursera.org/learn/generative-ai-with-llms/lecture/exyNC/instruction-fine-tuning

Instruction fine-tuning - Week 2 | Coursera

In this lesson, you'll learn about methods to improve the performance of existing language models for specific use cases. You'll also explore key metrics for evaluating the performance of fine-tuned language models (LLMs). The focus is on fine-tuning LLMs with instruction prompts to make them more effective in carrying out specific tasks.

Instruction prompts are essential for guiding LLMs to understand and fulfill specific instructions or prompts accurately. While some LLMs can perform zero-shot inference, smaller models often struggle with these tasks. One-shot or few-shot inference, which involves including examples along with instructions, helps but has limitations.

Fine-tuning, a supervised learning process, offers a more effective solution. Unlike pre-training, where LLMs learn from vast unstructured text data, fine-tuning uses labeled examples (prompt-completion pairs) to update the model's weights. Instruction fine-tuning, in particular, enhances LLM performance on various tasks. It involves training the model with examples demonstrating how it should respond to specific instructions.

To perform instruction fine-tuning, you need to prepare your training data, create validation and test splits, and pass prompts to the LLM, which generates completions. These completions are compared to the expected responses in the training data, and loss is calculated using cross-entropy. The model's weights are updated via backpropagation over multiple epochs to improve task performance.

Validation and test data sets are used to measure the LLM's performance, providing validation and test accuracy scores. The fine-tuning process results in a new version of the base model, often referred to as an "instruct model," which excels at specific tasks. Instruction fine-tuning is the most common form of fine-tuning for LLMs today.

 

이 수업에서는 특정 사용 사례를 위해 기존 언어 모델의 성능을 향상시키는 방법을 배울 것입니다. 또한 세세한 조정된 언어 모델(LLM)의 성능을 평가하는 주요 지표를 살펴보겠습니다. 중점은 명령 프롬프트를 사용하여 LLM을 세세하게 조정하여 특정 작업을 보다 효과적으로 수행하도록 하는 데 있습니다.

명령 프롬프트는 LLM이 구체적인 지시 사항이나 프롬프트를 정확하게 이해하고 수행하도록 안내하는 데 필수적입니다. 일부 LLM은 제로 샷 추론을 수행할 수 있지만, 작은 모델은 이러한 작업에서 어려움을 겪는 경우가 많습니다. 원샷 또는 퓨샷 추론은 지시사항과 함께 예제를 포함하는 것이 도움이 되지만 제한 사항이 있습니다.

세세한 조정은 보다 효과적인 해결책을 제공합니다. 사전 훈련과는 달리, 세세한 조정은 레이블이 지정된 예제(프롬프트-완료 쌍)를 사용하여 모델의 가중치를 업데이트하는 지도 학습 프로세스입니다. 특히 명령어 세세한 조정은 다양한 작업에서 LLM의 성능을 향상시킵니다. 이것은 모델이 특정 지시에 대한 응답 방법을 학습하는 예제를 사용하여 모델을 훈련하는 것을 포함합니다.

명령어 세세한 조정을 수행하려면 훈련 데이터를 준비하고 검증 및 테스트 분할을 생성하고 프롬프트를 LLM에 전달하여 완료를 생성합니다. 이러한 완료는 훈련 데이터의 기대 응답과 비교되며 교차 엔트로피를 사용하여 손실이 계산됩니다. 모델의 가중치는 여러 에폭 동안 역전파를 통해 업데이트되어 작업 성능을 향상시킵니다.

검증 및 테스트 데이터 세트를 사용하여 LLM의 성능을 측정하며 검증 정확도 및 테스트 정확도 점수를 제공합니다. 세세한 조정 프로세스는 특정 작업에서 뛰어난 "명령어 모델"이라고도 하는 기본 모델의 새 버전을 생성합니다. 명령어 세세한 조정은 현재 LLM의 가장 일반적인 세세한 조정 형태입니다.