Cheatsheet: How to Run Local LLM with Ollama

What is Ollama?

• A lightweight, open-source framework for running large language models (LLMs) locally.
• Benefits: Privacy, customization, cost-effectiveness, and offline access.

---

System Requirements

• OS: Linux, macOS, or Windows.
• RAM: 16GB+ (32GB recommended for larger models).
• Storage: 20GB+ free space.
• GPU: Optional but recommended for faster performance.

---

Installation Steps

1. Download Ollama:
   • Visit https://ollama.com/download and download the latest version for your OS.
2. Install Dependencies:
   • Ensure Python 3.8+ and pip are installed.
3. Set Up Virtual Environment:
   • Use venv or conda to create an isolated environment.
4. Install Ollama:
   • Run pip install ollama in your terminal.

---

Popular Models to Try

• Gemma3: Lightweight and efficient for general tasks.
  • Command: ollama run gemma3
• Deepseek-R1: Specialized for research and data analysis.
  • Command: ollama run deepseek-r1
• Llama3.3: Balanced performance and resource usage.
  • Command: ollama run llama3.3

---

Running Your First LLM

1. Start the Ollama Server:
   • Run ollama serve in your terminal.
2. Interact with the Model:
   • Use commands like ollama run <model_name> to start a session.
   • Example: ollama run gemma3
3. Send Prompts:
   • Type your prompt directly in the terminal after starting the model.

---

Advanced Features

• Fine-Tuning Models:
  • Use your own dataset to fine-tune pre-trained models.
• Integration:
  • Integrate Ollama with apps using its REST API or Python SDK.

---

Troubleshooting Tips

• Performance Issues:
  • Enable GPU acceleration if available.
  • Use smaller models like Gemma3 for low-end machines.
• Debugging Errors:
  • Check Ollama logs for error messages.
  • Update dependencies with pip install --upgrade ollama.

---

FAQs

1. Can I run Ollama on a low-end machine?
   • Yes, but stick to smaller models like Gemma3.
2. Is Ollama free?
   • Yes, it’s open-source and free to use.
3. How do I update Ollama?
   • Run pip install --upgrade ollama.
4. Can I use Ollama commercially?
   • Yes, but check the model’s licensing terms.
5. Ollama vs. Cloud-Based LLMs?
   • Ollama runs locally for better privacy and control; cloud LLMs are hosted remotely and often require subscriptions.

---

Quick Commands Cheatsheet

Task	Command
Install Ollama	pip install ollama
Start Ollama Server	ollama serve
Run Gemma3	ollama run gemma3
Run Deepseek-R1	ollama run deepseek-r1
Run Llama3.3	ollama run llama3.3
Update Ollama	pip install --upgrade ollama

---

By following this cheatsheet, you’ll be able to set up and run local LLMs with Ollama in no time. Happy experimenting! 🚀