고정 헤더 영역
상세 컨텐츠
본문
https://www.youtube.com/watch?v=1pedAIvTWXk&list=PLo2EIpI_JMQvWfQndUesu0nPBAtZ9gP1o&index=9
Tokenizer
Raw text (This course is amazing!)
Input IDs [ 101, 2023, 2607, 2003, 6429, 999, 102] adding the start of the sentence, and end of the sentence.
Raw text -> Tokens -> Special tokens -> Input IDs.
Model
The AutoModel class loads a model without its pretraining head.
'Hugging Face Course' 카테고리의 다른 글
| The tokenization pipeline (0) | 2023.09.14 |
|---|---|
| What happens inside the pipeline function? (TensorFlow) (0) | 2023.09.13 |
| The Transformer architecture (0) | 2023.09.13 |
| What is Transfer Learning (0) | 2023.09.13 |
| The pipeline function (0) | 2023.09.13 |
