AI Chat: Local LLM & GGUF
AI Chat: Local LLM & GGUF
Keep your AI conversations in your own hands.
AI Chat is a local AI chat app that works even without an internet connection.
Conversations are processed on your device and are never sent to an external server.
Once a model is downloaded, you can keep chatting in Airplane Mode. Replies are generated on your device.
No account needed. What you type and what the AI replies never leave your device. We don't run any server that receives your conversations.
Search Hugging Face for open models in GGUF format, such as Gemma and Qwen, and download one that fits your device right in the app.
Streaming replies, with Markdown and code blocks shown cleanly.
Choose the model you run
Search and download open GGUF models from Hugging Face, right in the app
Server Mode runs your iPhone or iPad as an OpenAI-compatible API server, so other devices on the same network can use the loaded model.
Over Tailscale or your home network, connect to an OpenAI-compatible server you run yourself and use larger models (advanced, optional).
Getting Started
1. Choose your first model
When you open the app, the chat screen on the Local tab appears. If you haven't chosen a model yet, follow the on-screen guide to add one.
Open Settings (the gear icon at the top left) > AI Model > Add GGUF Model... to search for and download GGUF models from Hugging Face. If you're not sure what to choose, pick one from Settings > Supported Models. These models have been tested with the app, and their settings are already tuned.
Each model and quantization file is labeled from its estimated memory use:
- Runs: fits in your device's memory with room to spare
- Tight on memory: runs, but with little room left; other apps may become unstable
- Won't run: too large for your device's memory, so it can't be downloaded
Once the download finishes, the model loads automatically and you can start chatting.
2. Search for models
Type part of a model name (for example, "Qwen3.5" or "Gemma") in the search field of the model browser to search public GGUF repositories on Hugging Face, sorted by downloads. If a repository has several quantization files (such as Q4_K_M or Q8_0), choose one from the list.
Higher-bit quantizations give more accurate answers but need more storage and memory. If you're unsure, a Q4 file (such as Q4_K_M) is a good balance, and many models mark one as "Recommended".
3. Adjust how the AI responds
In Settings > AI Behavior, you can adjust:
- System Prompt: describe how you want the AI to behave. You can also start from Templates
- Temperature: a slider from 0.0 (precise, consistent) to 2.0 (creative)
- Context Size: 2K to 32K (up to 64K on devices with 12GB of memory). Larger sizes hold more conversation or document, but take longer before the reply starts
- Long-Context Priority (Memory Saver): switch between reading long documents at once and keeping a quick back-and-forth
- MTP / DFlash / Speculative Decoding: experimental features that run a small look-ahead model with the main model. They are off by default, and some combinations can be slower
4. Attach files
Tap the paperclip next to the message field to attach text files, PDFs, CSV, JSON, and more. The contents are passed to the AI. With supported models (Gemma 4 E2B / E4B, Nemotron 3.5 Bolt E4B, LFM3 Omni 2B), you can also attach photos, audio, and video. Attached files are kept in the Files app under On My iPhone > AI Chat, where you can view or delete them at any time.
5. Manage conversations
In the conversation list, you can search, pin conversations to the top, and swipe to delete. From the "..." menu in a conversation, you can export it as Markdown to open or share in other apps.
6. Connect to your own server (advanced)
You can connect to an OpenAI-compatible inference server you run at home or on your own computer (such as llama-server) and use models too large for the device alone.
Add a connection profile in Settings > Remote Inference and enter the server address over Tailscale or your local network. Examples of starting the server:
- http: start llama-server with --host 0.0.0.0 --port 8080. Traffic inside your Tailscale network (tailnet) is encrypted with WireGuard
- https: run tailscale serve --bg --https=443 http://127.0.0.1:8080 so that Tailscale handles TLS
- https (alternative): issue a certificate with tailscale cert <host>.ts.net and start llama-server with --ssl-key-file and --ssl-cert-file
After saving, use Test Connection to check that it works. You can save several servers as profiles and switch between them. Only Tailscale (*.ts.net), private IP, and .local hosts can be used.
7. Use this device as a server
Your iPhone or iPad can act as an OpenAI-compatible API server for other devices. Go to Settings > Server Mode > Open Server Mode and tap Start. Enter the address shown on screen (and the API key, if you set one) in the other device, such as another copy of AI Chat on its Remote tab or any OpenAI-compatible client, to use the model loaded on this device. A simple chat page is also available from a web browser on the same network.
8. Use on iPad
On iPad, the conversation list and the chat are shown side by side. When the window gets narrow, such as in Split View, the layout switches to a single column like on iPhone.
9. Troubleshooting
- A model shows "Won't run" and can't be selected
It is too large to run reliably in your device's memory. Try a smaller quantization (such as Q4) or a model with fewer parameters.
- Replies are slow or stop partway
Try a smaller Context Size, or turn off Long-Context Priority.
- Downloads are slow or don't progress
Use Wi-Fi. Models are hundreds of MB to several GB, so mobile data can be slow and use a lot of data.
AI Chat is a local AI chat app that works even without an internet connection.
Conversations are processed entirely on your iPhone or iPad. What you type, attach, and receive is never sent to an external server, and it keeps working in Airplane Mode. No cloud AI subscription required.
■ Three simple steps
1. Pick a model that fits your device (or just take the "Recommended" pick)
2. Download it (hundreds of MB to a few GB; Wi-Fi recommended)
3. Start chatting
■ Open models you choose
Search Hugging Face for open models in GGUF format (Gemma, Qwen, and more), download them in the app, and run them on your device. Each model is labeled "Runs", "Tight on memory", or "Won't run" for your device.
■ Supported Models (as of v1.0.24)
Tested models with tuned settings such as Temperature. Just pick one.
- Recommended: Google Gemma 4 (E2B / E4B), Alibaba Qwen3.5 4B / 0.8B
- Also supported: NVIDIA Nemotron 3.5 Bolt E4B / Nemotron 3 Nano 4B, Microsoft Phi-4 mini, IBM Granite 4.2 3B, Prism ML Bonsai 8B / 27B / Ternary Bonsai 2 9B / 4B / 27B, LiquidAI LFM3 Omni 2B / LFM3 1.7B / LFM2.5 2.6B / LFM2.5 1.2B JP, Shanghai AI Lab Agents-A1 4B, XHToken Spark-X2.6 4B / 1.7B / Spark-X2.5 4B / 1.7B, Mistral AI Ministral 3 3B, Hugging Face SmolLM3 3B, Google Gemma 3 1B, LLM-jp 3.1 1.8B, OpenBMB MiniCPM5 2B, MBZUAI IFM K2-Horizon 3.7B / 0.9B, Preferred Networks PLaMo 3 Translate 8B / 2B (translation)
You can also try any GGUF that isn't on the list.
■ Photos, audio, and video
With Gemma 4 E2B / E4B, Nemotron 3.5 Bolt E4B, and LFM3 Omni 2B, attach photos or audio and ask about them. For video, the audio is used, and LFM3 Omni 2B also reads the frames.
■ Draw images from text
Pick Z-Image Turbo (Alibaba Tongyi Lab) or FLUX.2 klein 4B (Black Forest Labs) to draw an image from what you write (8GB of memory or more). Use "Edit the last image" to refine it or "Draw another version" to redraw it. Images stay in the conversation and can be saved to Photos.
■ Fine-tune responses
- System prompt for personality and tone
- Temperature from 0.0 (precise) to 2.0 (creative)
- Context size from 2K to 32K (up to 64K with 12GB of memory). For large models, a safe length is chosen automatically and shown on screen
- Long-context priority or conversation priority
■ Speed-up methods (experimental)
Run a small look-ahead model with the main model: MTP, DFlash, or Speculative Decoding. All are off by default, and some combinations can be slower.
■ Designed for privacy
- History and attachments are stored encrypted on your device and included in your iPhone backup
- No account or sign-in
- No ads, no tracking
■ Turn your iPhone into an AI server
Server Mode runs your iPhone or iPad as an OpenAI-compatible API server. Other devices on the same network can use the loaded model, and a simple chat page works from any web browser. API key authentication is available.
■ Connect to your own server (advanced)
Connect over your home network or a VPN such as Tailscale to an OpenAI-compatible server you run (such as llama-server) to use models too large for the device. Only Tailscale, private IP, and .local hosts are allowed, so conversations never go to a public server by mistake.
■ Main features
- Streaming replies with Markdown and code blocks
- History with search, pinning, renaming, and Markdown export
- Attach text files and PDFs; long documents are trimmed only when they don't fit, and the app tells you
- Two-column layout on iPad, and a layout that follows the fold on iPhone Duo
■ Requirements
The models you can use depend on your device's memory.
- 12GB: 8B–9B class models and the 1.75-bit 27B
- 8GB (iPhone 15 Pro and later): most supported models
- 6GB: models up to about 2.5GB, such as Gemma 4 E2B
- 3–4GB: 1B–2B class models such as Qwen3.5 0.8B and LFM2.5 1.2B JP
Downloading models needs several GB of storage and an internet connection (only for the download). AI replies can contain mistakes or outdated information; check primary sources for anything important.
Privacy Policy:https://www.kystudio.jp/home/aichat/en/privacypolicy
Disclaimer — AI Chat
Established: August 21, 2026
Developer: Kazuhiro Yoshimitsu
1. About AI output
This app uses large language models (LLMs) to generate text automatically, and image generation models to create images from what you write. The generated content:
- may include incorrect, outdated, or nonexistent information (so-called "hallucinations")
- can differ for the same question depending on the model and settings you choose
- is not guaranteed to be accurate, complete, or up to date
- in the case of images, may differ from what you asked for, or may resemble real people, works, or trademarks
The app passes what you write to the image generation model as is, and has no mechanism of its own to check or restrict the content of generated images. What you generate and how you use it is your own decision and responsibility.
Use any information obtained through this app at your own judgment and risk, and check primary sources as needed.
2. Not professional advice
Responses from this app are not a substitute for medical, health, legal, tax, investment, or other professional advice. For decisions that require expertise, such as diagnosis, treatment, investment, or legal matters, always consult a qualified professional such as a doctor, lawyer, or tax accountant. This app is not a medical device and is not intended to diagnose, treat, or prevent any disease.
3. Use of generated content
To the extent permitted by law, the developer is not liable for damage arising from the use of text or images generated with this app, including sharing with third parties, business use, publication, or posting in public. This does not apply to damage caused by the developer's willful misconduct or gross negligence. The developer does not guarantee that generated content does not infringe the rights of third parties (such as copyright, trademark, or portrait rights); this includes images that resemble real people or the works or trademarks of others. Please check this yourself before use.
4. About on-device models
AI models downloadable through this app (open models such as Gemma and Qwen, and image generation models such as Z-Image Turbo and FLUX.2 klein) are distributed by their respective providers under their own licenses. The same applies to the supporting models downloaded together with an image generation model. Check each model's terms and license on its provider's page (each repository on Hugging Face). The developer does not guarantee the quality, performance, or output of any model.
5. Model performance and requirements
Response speed and quality (for image generation, the time it takes to draw and the quality of the image) depend heavily on the chosen model, the quantization format, and your device's performance (memory and chip). Labels such as "Runs" shown in the app are for reference only and do not guarantee smooth operation in every situation.
6. Remote inference
When you use the Remote feature to connect to a server you operate yourself, the developer is not involved in and is not responsible for that server's availability, responses, or security. You are responsible for managing the server you connect to.
7. Continuity of service
This app is developed and provided by an individual. Features may be added or changed, or the app may be discontinued in the future. Downloaded models are deleted when you uninstall the app. Conversation history, attachments, and generated images are included in your device backup (iCloud Backup or a backup on your computer), but they are lost if you uninstall the app without a backup. Back up or save important conversations and images as needed.
8. Your legal rights
Nothing in this disclaimer limits any right you have under the laws of the country or region where you live, including consumer protection laws, that cannot be excluded or limited. If any part of this disclaimer conflicts with such a law, that part applies only to the extent the law allows.
For users in Australia: this app comes with guarantees that cannot be excluded under the Australian Consumer Law, and this disclaimer does not limit your rights under those guarantees. The same applies to your rights under similar laws, such as South Africa's Consumer Protection Act.
9. Governing law
This disclaimer is governed by the laws of Japan, without limiting Article 8. If there is any difference between the English and Japanese versions, the Japanese version prevails.
Last updated: November 7, 2026
©2025 Kazuhiro Yoshimitsu