https://entorb.net//wiki/index.php?action=history&feed=atom&title=Local_private_LLMLocal private LLM - Revision history2026-05-06T10:24:37ZRevision history for this page on the wikiMediaWiki 1.43.1https://entorb.net//wiki/index.php?title=Local_private_LLM&diff=5254&oldid=prevTorben: /* Installation */2025-06-01T14:40:41Z<p><span class="autocomment">Installation</span></p>
<p><b>New page</b></p><div>== Motivation ==<br />
* Large language models (LLMs) are very good at analyzing (large) texts<br />
* There are multiple free-to-test providers in the internet<br />
* Sensitive private or confidential data (like diary/journal, medical records, etc.) should '''NOT''' be shared with any external service<br />
<br />
== Solution ==<br />
Run your own LLM<br />
<br />
== Installation ==<br />
I use the nice open-source tool [https://ollama.com Ollama] to manage and host the LLM.<br />
<br />
== Running ==<br />
Start in terminal/command window by passing a model (the llama3.2 was not working well for me, see below)<br />
ollama run llama3.2:latest<br />
<br />
A list of supported LLM models can be found [https://ollama.com/search here].<br />
<br />
For me the [https://ollama.com/library/qwen3 qwen3] model worked best, so far.<br />
<br />
Besides the model, you need to select a model size, that fits to your hardware.<br />
<br />
For me:<br />
MacBook Pro M1 16GB<br />
-> models up to 8b work, but with patience <br />
Windows PC with gaming graphics card (GPU) GeForce RTX 3060 12GB<br />
-> models up to 14b run very smooth<br />
<br />
To check if the model fits into you GPU memory, first run<br />
ollama run <model><br />
than open a second terminal window and run<br />
ollama ps<br />
to see where the model is stored.<br />
<br />
== Usage ==<br />
=== chat commands === <br />
/clear : clear current session<br />
/bye : exit<br />
<br />
=== command line commands === <br />
show currently running model(s) and where it is stored (RAM or GPU RAM)<br />
ollama ps<br />
<br />
stop model (automatically done after 5min inactivity)<br />
ollama stop llama3.2:latest<br />
<br />
delete model<br />
ollama rm llama3.2:latest<br />
<br />
===Remote access===<br />
Set env variable<br />
OLLAMA_HOST=192.168.0.123:11434<br />
<br />
===Local storage of the models ===<br />
[https://github.com/ollama/ollama/blob/main/docs/faq.md#where-are-models-stored where-are-models-stored]<br />
macOS: ~/.ollama/models <br />
Linux: /usr/share/ollama/.ollama/models<br />
Windows: C:\Users\%username%\.ollama\models<br />
change via env var OLLAMA_MODELS (ollama service restart needed)<br />
<br />
=== Links ===<br />
* [https://ollama.com/library/deepseek-r1 Deepseek Models]<br />
* [https://github.com/ollama/ollama/blob/main/docs/faq.md FAQ]</div>Torben