Sheepy-T: A fully open-source instruction-tuned language model based on GPT-J running locally on iPhone 14. Reply for beta access via TestFlight.

Note that this video is running in real time, rather than being sped up 10x like the previous incarnation (

This is possible because it's built on top of a slightly smaller model, GPT-J, by @AiEleuther rather than LLaMA. Thus it can fit entirely in memory rather than being constantly paged in with mmap. On top of that, it is a fully open model without questionable provenance.

That said- it still only barely fits into memory- you need a device with at least 6GB of RAM. That means it works on all iPhone 14, and the Pro and Pro Max variants of iPhone 12 and 13. All other iPhones, the XS, XR, and mini are unsupported.

How snappy can you make this? Millisecond latency?

It can generate about 3 tokens a second at peak, but sometimes iOS has some background tasks which makes it run slower.

Joining the line for beta lol

interesting. would love to try it out

Roger keen for invite @jkoukides

Please let me try it. And is it possible to run it on an iPhone 7?