openai gpt3 inference in inferentia2(neruon)

import openai

openai.api_key = 'mykey'

response = openai.ChatCompletion.create(
    model="gpt-3.5-turbo",
    messages=[
        {"role": "user", "content": "Hello, I'm a language model,"}
    ]
)
print(response['choices'][0]['message']['content'])

 

< setting>

AMI : Deep Learning AMI Neuron PyTorch 1.13 (Ubuntu 20.04) 20240102

인스턴스 : Inf2.xlarge

source /opt/aws_neuron_venv_pytorch/bin/activate

pip install transformers-neuronx --extra-index-url=https://pip.repos.neuron.amazonaws.com

python3 sample.py

 

 

**코드 참조 : https://platform.openai.com/docs/guides/rate-limits/error-mitigation?context=tier-free 

**코드 참조 2 : 챗지피티4

** openai에서 사용가능한 모델 : https://platform.openai.com/docs/models/overview

첨에 RateLimitError 에러가 나서 limit 확인

https://platform.openai.com/docs/guides/rate-limits/usage-tiers?context=tier-free

 

RateLimitError 해결해볼라고 14000원 결제도 함..