ChatGPT

I made a ChatGPT where you can give instructions into ChatGPT and it will follow.

I pretrained ChatGPT with data to generate the next token.

Then I finetuned ChatGPT to follow instructions.

I used Alpaca prompt style:

These are the answers given by ChatGPT:

These are the scores for testing my chatgpt with Llama 3:

Masked Multihead Attention

I made a masked Multihead Attention mechanism for LLMs.

This is the code:

They are combined the context vectors in the last dimension each learning something different from the input x.