Taeyoon.Kim.DS

Amazon S3 Standard - General Purpose: Ideal for frequently accessed data with high availability and low latency. Suitable for big data analytics, mobile apps, and content distribution. Amazon S3 - Infrequent Access: Designed for less frequently accessed data that requires rapid access when needed. Used for disaster recovery and backups. Amazon S3 One Zone - Infrequent Access: Offers high durabil..

AWS Certified Machine Learning Specialty 2023. 9. 18. 16:24

회사마다 인프라 요구 사항이 다르다. 각 회사에 필요한 인프라는 개발하는 애플리켕션의 개수와 전문성 수준에 따라 다르다. * (스펙트럼의 한편) 일시적인 비즈니스 분석에 ML을 사용하는 회사 (분기별 계획 발표를 위해 신규 사용자 수를 예측하는데 ML을 사용) 이러한 경우에는 Jupyter notebook, Python, Pandas등 기본적인 도구만 사용. 간단한 ML use case하나만 있는 경우에도 infrastructure가 많이 필요하지 않다. * (스펙트럼의 반대편) 자율 주행 자동차는 정확도와 레이턴시에 대한 요구 사항이 있다. SWOOP에는 정확도, TP와 FP의 Trade off를 고려해야하고, FN (침해품이지만 비침해품으로 처리) 도 너무 많이 발생하면 안된다. 알고리즘이 24시간내..

데이터 과학 2023. 9. 17. 18:30

Trainer - 1. Model 2. Training/ Validation/ Test dataset 3. Tokenizer 4. Data Collator 5. Hyperparameters 6. Metrics Evaluation/ Prediction ! pip install datasets transformers[sentencepiece] from datasets import load_dataset from transformers import AutoTokenizer, DataCollatorWithPadding raw_datasets = load_dataset("glue", "mrpc") checkpoint = "bert-base-cased" tokenizer = AutoTokenizer.from_pre..

Hugging Face Course 2023. 9. 15. 22:13

from PIL import Image import glob from transformers import BlipProcessor, BlipForConditionalGeneration processor = BlipProcessor.from_pretrained("Salesforce/blip-image-captioning-base") model = BlipForConditionalGeneration.from_pretrained("Salesforce/blip-image-captioning-base")

Hugging Face Course 2023. 9. 14. 23:16

https://www.udemy.com/course/aws-machine-learning/learn/lecture/16666294#content In this Udemy course on AWS Certified Machine Learning Specialty, the instructor begins by introducing Amazon S3 (Simple Storage Service) and its key concepts. S3 allows you to store objects or files in buckets, and these buckets must have globally unique names. Objects within S3 are essentially files with unique ke..

AWS Certified Machine Learning Specialty 2023. 9. 14. 22:40

https://www.youtube.com/watch?v=Yffk5aydLzg&list=PLo2EIpI_JMQvWfQndUesu0nPBAtZ9gP1o&index=17 A tokenizer takes texts as inputs and outputs numbers the associated model can make sense of. from transformers import AutoTokenizer tokenizer = AutoTokenizer.from_pretrained("bert-base-uncased") inputs = tokenizer("Let's try to tokenize!") print(inputs["input-ids"] [101, 2292, 1005, 1055, 3046, 2000, 19..

Hugging Face Course 2023. 9. 14. 19:08

Raw text --> Input IDs --> Logits --> Predictions (math.sotfmax) return_tensors = "pt" or "tf" Instantiate a Transformers model (PyTorch) config file -> config class -> model config https://www.youtube.com/watch?v=AhChOFRegn4&list=PLo2EIpI_JMQvWfQndUesu0nPBAtZ9gP1o&index=11

Hugging Face Course 2023. 9. 13. 19:33

https://www.youtube.com/watch?v=1pedAIvTWXk&list=PLo2EIpI_JMQvWfQndUesu0nPBAtZ9gP1o&index=9 Tokenizer Raw text (This course is amazing!) Input IDs [ 101, 2023, 2607, 2003, 6429, 999, 102] adding the start of the sentence, and end of the sentence. Raw text -> Tokens -> Special tokens -> Input IDs. Model The AutoModel class loads a model without its pretraining head.

Hugging Face Course 2023. 9. 13. 19:22