[{"element_type":"user_image","content":"https://cdn-learn.adafruit.com/user_assets/assets/000/002/353/original/5248866.jpg?1778202737","metadata":{}},{"element_type":"text","content":"\n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n        \u003cp\u003eMany of us have been there: working with Claude, ChatGPT, Copilot, etc. and \u003cstrong\u003eBOOM,\u003c/strong\u003e\u0026nbsp;the token allowance runs out and you have to wait a few hours for it to reset. All you want to do is finish your project.\u003c/p\u003e\n\u003cp\u003eWell, you are on your personal computer which can compute things. Perhaps your machine can even play some fairly recent games? If so, you have free compute right at your fingertips.\u003c/p\u003e\n      \n\n\n\n\n\n\n\n\n\n\n\n\n\n\n","metadata":{}},{"element_type":"alert","content":"\u003cp\u003eThis guide will help you set up a local large language model. It will not help you set up OpenClaw or other semi-autonomous \"AI\" software on your machine.\u003c/p\u003e","metadata":{"markdown":"This guide will help you set up a local large language model. It will not help you set up OpenClaw or other semi-autonomous \"AI\" software on your machine.","alert_type":"warning","icon":"info"}},{"element_type":"text","content":"\n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n        \u003cp\u003eSome of the AI models used by the big companies are available for free. This is true. The trouble is knowing which model to get, where to get it, what to run it with and how to tune the model to your machine. If this sounds complicated, it's less complicated than installing current games and likely takes up less space. But you'll want to follow this guide for some tips on how to do this.\u003c/p\u003e\n\u003cp\u003eThere are many models and many ways to run them. I'll show you some of what I have done as an example for you to do similar.\u003c/p\u003e\n      \n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n","metadata":{}},{"element_type":"alert","content":"\u003cp\u003eNote that, in the changing world of large language models (LLM), things can, um, change. This guide might need some tweaks from time to time. Hopefully the concepts will be general enough to help you get started.\u003c/p\u003e","metadata":{"markdown":"Note that, in the changing world of large language models (LLM), things can, um, change. This guide might need some tweaks from time to time. Hopefully the concepts will be general enough to help you get started.","alert_type":"primary","icon":"thinking"}},{"element_type":"text","content":"\n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n        \u003ch2\u003eWhat You'll Be Running On\u003c/h2\u003e\n      \n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n","metadata":{}},{"element_type":"text","content":"\n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n        \u003ch3\u003eHardware\u003c/h3\u003e\n      \n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n","metadata":{}},{"element_type":"text","content":"\n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n        \u003cp\u003eThis was one of the trickiest parts of my home computer journey. I have a fairly beefy machine but it's 6 years old. For what I do it's enough but for heavy modeling it's not the best. I have an AMD Threadripper 3960X 24 core/48 thread CPU and 128GB RAM and an nVidia RTX2080 Super GPU with 8GB VRAM. Models love GPUs and mine is dated with only 8GB VRAM. The 128GB system RAM is good but not shared with the GPU.\u003c/p\u003e\n\u003cp\u003eMy machine is not a top tier system like buying a $5000 dedicated machine or having an RTX5090 GPU. So running the best models is not an option, as even if they ran, the measure of success, tokens/second, would be very low.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eYou should start by gathering up your hardware specifications. Write down your CPU model number, your system RAM, and your graphics card information. If you only have integrated graphics, there \u003cspan style=\"text-decoration: underline;\"\u003emight\u003c/span\u003e me a chance, so note that.\u003c/p\u003e\n      \n\n\n\n\n\n\n\n\n\n\n\n\n","metadata":{}},{"element_type":"alert","content":"\u003cp\u003eThere are projects to help use hardware you might have idle, such as integrated graphics. You should look into this as a separate quest as it's changing rapidly.\u003c/p\u003e","metadata":{"markdown":"There are projects to help use hardware you might have idle, such as integrated graphics. You should look into this as a separate quest as it's changing rapidly.","alert_type":"info","icon":"plus"}},{"element_type":"text","content":"\n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n        \u003ch3\u003eThe Upgrade Conundrum\u003c/h3\u003e\n\u003cp\u003eAfter writing down your hardware specifications, do you already have regrets, thinking your machine might not run do what you want?\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eHere are some thoughts on that:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eLook for another machine in your house that might work better\u003c/li\u003e\n\u003cli\u003eUpgrade your current machine (which is expensive in today's market but likely possible)\u003c/li\u003e\n\u003cli\u003eBuy a new machine. This is beyond the scope of this note as I cannot possibly inform you on what hardware you need. Consider your requirements and budget, do some reading, and look to make an informed decision.\u003c/li\u003e\n\u003c/ul\u003e\n      \n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n","metadata":{}},{"element_type":"text","content":"\n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n        \u003ch3\u003eWhat Do You Want the LLM to Do?\u003c/h3\u003e\n\u003cp\u003eWhen looking for a model, it helps to know what you might want to do with it. Do you want help coding? Writing? Research? Using other programs with an LLM (\"Tool Use\"), computer vision? Write down what you need and what you'd like along with your system specifications.\u003c/p\u003e\n      \n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n","metadata":{}},{"element_type":"text","content":"\n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n        \u003ch2\u003eSoftware: Finding the Right Model Fit\u003c/h2\u003e\n      \n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n","metadata":{}},{"element_type":"text","content":"\n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n        \u003cp\u003eFinding the right model to run given the hardware I had was a a large roadblock for me. Then a colleague found the free website \u003ca href=\"https://www.canirun.ai/\" target=\"_blank\"\u003ecanirun.ai\u003c/a\u003e. You can enter the hardware parameters of your machine and get a list of models and how well they may perform on your hardware. Do note it is rather GPU-centric, although it does have Raspberry Pi computers in the hardware list if that's what you're looking for.\u003c/p\u003e\n\u003cp\u003eThis is likely not the only tool to help you choose, but it is the one standalone site I know of.\u003c/p\u003e\n\u003cp\u003eHere is output for my graphics card:\u003c/p\u003e\n      \n\n\n\n\n\n\n\n\n\n\n\n","metadata":{}},{"element_type":"user_image","content":"https://cdn-learn.adafruit.com/user_assets/assets/000/002/344/original/canirun.png.png?1778098001","metadata":{}},{"element_type":"text","content":"\n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n        \u003cp\u003eThe top chart shows, I believe, more current models and how they may run. At right is a confidence on their running well or not and to the left of that how many tokens per second it could generate.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eSo for my graphics card, only three more recent models might work well. Then there is the second table which has all models. You'll likely find many of these may run fine on your computer. The trade-off is they are likely older models or models trimmed down to fit small hardware budgets.\u003c/p\u003e\n\u003cp\u003eLooking at the models that will fit, check at the tags after the model name. They include:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eWrench/spanner: Use tools\u003c/li\u003e\n\u003cli\u003eStar - popular\u003c/li\u003e\n\u003cli\u003eLight bulb - thinking/reasoning\u003c/li\u003e\n\u003cli\u003eEye - vision\u003c/li\u003e\n\u003cli\u003eGraph - mixture of experts\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eFinally there is a box with the model's license. MIT or Apache is basically free to use. Other models might have conditions. Note that depending on your use case. You can click that tag to get additional information. For home use only, likely you're fine with any of them.\u003c/p\u003e\n      \n\n\n\n\n\n\n\n\n\n\n","metadata":{}},{"element_type":"text","content":"\n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n        \u003ch2\u003eMaking the Leap: How to Run Models\u003c/h2\u003e\n      \n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n","metadata":{}},{"element_type":"text","content":"\n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n        \u003cp\u003eThere are many ways to run models. I looked for a \u003cstrong\u003emodel interface\u003c/strong\u003e, sometimes called an inference engine, basically a front end. The ones I found:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003e\n\u003ca href=\"https://lmstudio.ai/\" target=\"_blank\"\u003eLM Studio\u003c/a\u003e (free for home and work use, \u003ca href=\"https://lmstudio.ai/app-terms\" target=\"_blank\"\u003eterms\u003c/a\u003e)\u003c/li\u003e\n\u003cli\u003e\n\u003ca href=\"https://ollama.com/\" target=\"_blank\"\u003eOllama\u003c/a\u003e (free but running multiple \"cloud models\" costs extra)\u0026nbsp;\u003c/li\u003e\n\u003cli\u003e\n\u003ca href=\"https://github.com/ggml-org/llama.cpp\" target=\"_blank\"\u003ellama.cpp\u003c/a\u003e (MIT license)\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eI chose LM Studio. It runs on mac, Windows, and Linux in a GUI. It seemed more straightforward and no cost, so I chose it. You can do your research on the one you might want to use.\u003c/p\u003e\n\u003cp\u003eI'm sure I see someone in the back asking if these are \"\u003cstrong\u003eharnesses\u003c/strong\u003e.\" No. The model interface is what you use to see the LLM. A downloaded file is the LLM itself. The harness is the scaffolding between the two that makes it able to remember, interact, and use agents. I'm sure there is a harness on my machine but when I use LM Studio, it handles things so I don't have to make harness connections (I think).\u003c/p\u003e\n\u003cp\u003eSee \u003ca href=\"https://www.youtube.com/shorts/rglBN2cuE7Q\" target=\"_blank\"\u003ethis video\u003c/a\u003e on what a harness is and how all the pieces relate to each other.\u003c/p\u003e\n\u003cp\u003e\u0026nbsp;\u003c/p\u003e\n      \n\n\n\n\n\n\n\n\n\n","metadata":{}},{"element_type":"text","content":"\n  \n  \n  \n  \n  \n  \n  \n  \n        \u003ch3\u003eUsing LM Studio\u003c/h3\u003e\n\u003cp\u003eHere is where I get more specific as LM Studio is the only model interface I have tried. If you want another interface (command line, Docker, etc.) you'll want to find a specific guide for that. You can always try LM Studio (as it's free) and go in another direction later.\u003c/p\u003e\n\u003cp\u003e\u003ca href=\"https://lmstudio.ai/download\" target=\"_blank\"\u003eDownload\u003c/a\u003e and install LM Studio for your machine and CPU architecture. When I started this guide, the latest version for Windows x64 was 0.4.12 (Build 1). They have great \u003ca href=\"https://lmstudio.ai/docs/app/basics\" target=\"_blank\"\u003eGetting Started\u003c/a\u003e documentation.\u003c/p\u003e\n\u003cp\u003eYou'll want to download one or more LLMs to your computer. Click on the Robot/Magnifying Glass icon at left (circled in red).\u003c/p\u003e\n      \n\n\n\n\n\n\n\n","metadata":{}},{"element_type":"user_image","content":"https://cdn-learn.adafruit.com/user_assets/assets/000/002/345/original/LM_Studio_select_model.png?1778100233","metadata":{}},{"element_type":"text","content":"\n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n        \u003cp\u003eAnd here is the screen that pops up:\u003c/p\u003e\n      \n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n","metadata":{}},{"element_type":"user_image","content":"https://cdn-learn.adafruit.com/user_assets/assets/000/002/346/original/LM_Studio_Pick_Models.png?1778100483","metadata":{}},{"element_type":"text","content":"\n  \n  \n  \n  \n        \u003cp\u003eSearch for the model(s) you were looking for on \u003ca href=\"https://www.canirun.ai/\" target=\"_blank\"\u003ecanirun.ai\u003c/a\u003e\u003cspan\u003e. Or look through the list. The same icons for model strengths are used except reasoning here is a green brain. The \"Best Match\" says \"Only include staff picks that can fit on this device. Based on an estimated model capacity of xxxx.xx GB\".\u0026nbsp;\u003c/span\u003e\u003c/p\u003e\n\u003cp\u003e\u003cspan\u003eDo not select such a big model that your hardware couldn't do well with it. On my 8GB GPU, I think models with more than 9B are likely too big.\u003c/span\u003e\u003c/p\u003e\n\u003cp\u003e\u003cspan\u003eYou can read about the models. When you click download, it will download the model data, which is usually several gigabytes.\u003c/span\u003e\u003c/p\u003e\n\u003cp\u003e\u003cspan\u003eAfter downloading, you can clock on the \"My LLMs\" icon circled below to see your models.\u003c/span\u003e\u003c/p\u003e\n      \n\n\n\n","metadata":{}},{"element_type":"user_image","content":"https://cdn-learn.adafruit.com/user_assets/assets/000/002/347/original/LM_Studio_Models_downloaded_circled.png?1778102222","metadata":{}},{"element_type":"text","content":"\n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n        \u003cp\u003eI have four models downloaded. I have been trying out the qwen 3.5 and 3.6 models. Gemma 4 and Trellis 2 are there also.\u003c/p\u003e\n\u003cp\u003eHow about trying one out! Click the \"Use in New Chat\" button circled below.\u003c/p\u003e\n      \n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n","metadata":{}},{"element_type":"user_image","content":"https://cdn-learn.adafruit.com/user_assets/assets/000/002/348/original/LM_Studio_use.png?1778102645","metadata":{}},{"element_type":"text","content":"\n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n        \u003cp\u003eI type \"Hello\" and start a conversation:\u003c/p\u003e\n      \n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n","metadata":{}},{"element_type":"user_image","content":"https://cdn-learn.adafruit.com/user_assets/assets/000/002/349/original/hello.png?1778103076","metadata":{}},{"element_type":"text","content":"\n  \n  \n  \n  \n  \n  \n  \n        \u003cp\u003eI have successfully used it that mode; to discuss CircuitPython coding and more. All with zero cloud costs.\u003c/p\u003e\n      \n\n\n\n\n\n\n","metadata":{}},{"element_type":"text","content":"\n  \n  \n  \n        \u003cp\u003eNote that it's best to use a new project for new topics and not switch in the middle of a conversation. I tried after starting with CircuitPython and every answer to the new line of questioning led to it wanting to use a solution to the previous line of questioning.\u003c/p\u003e\n      \n\n\n","metadata":{}},{"element_type":"text","content":"\n  \n  \n  \n  \n  \n        \u003ch2\u003eTuning Your Model\u003c/h2\u003e\n\u003cp\u003eIf you get an error, something about 4096 tokens or some such, you need to tune your model and you probably want to anyway to be sure the model's thinking time is kept to a minimum.\u003c/p\u003e\n\u003cp\u003eClick the My Models icon on the left then the Load button on the right. There will be three sliders:\u003c/p\u003e\n\u003col\u003e\n\u003cli\u003eContext Length - this is the cause of the error if you had one. You will need to try different values to see how to minimize thinking time. Definitely higher than a default 4096 unless you do not have a GPU. With K and V Cache Quantization Type F16 and checked, I get my best performance at Context Length at 11486 for my 8GB card.\u003c/li\u003e\n\u003cli\u003eGPU Offload - will allow the model to use your graphics card if you have one. Set it almost max, leave a bit for your computer to render your screens. I got my best performance at 30 (out of 32?).\u003c/li\u003e\n\u003cli\u003eCPU Thread Pool Size - even though it says thread (I have 48 threads) it seems to be limited to CPUs in your microprocessor. I have 24, so I set it at 21 to let it use most of the processing power. Bumping it up seemed to hurt performance some.\u003c/li\u003e\n\u003c/ol\u003e\n\u003cp\u003eThese will have a big impact on your model seeming to be as snappy as your cloud AI provider (if you have one) so spend some time in tuning to get good results for that model. If no setting helps, perhaps look to use a less taxing model to get better responses at the cost the model may have been trained on less data.\u003c/p\u003e\n      \n\n\n\n\n","metadata":{}},{"element_type":"user_image","content":"https://cdn-learn.adafruit.com/user_assets/assets/000/002/350/original/tune.png?1778103608","metadata":{}},{"element_type":"text","content":"\n  \n  \n        \u003cp\u003eI would change a setting, give it a greeting, see how many tokens per second were generated and how many tokens were used. Change one setting, give it a similar greeting, see the results, and iterated like that. Likely not the best.\u003c/p\u003e\n      \n\n","metadata":{}},{"element_type":"text","content":"\n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n        \u003ch2\u003eTool Use\u003c/h2\u003e\n\u003cp\u003eOut of the box, models will not likely be using tools to do local or remote operations. If you don't need that' you're golden. If you do want to use an LLM to do things, you'll need a tool package added in.\u003c/p\u003e\n\u003cp\u003eI found \u003ca href=\"https://lmstudio.ai/beledarian/beledarians-lm-studio-tools\" target=\"_blank\"\u003eBeledarians-lm-studio-tools\u003c/a\u003e. It adds a number of common tools for accessing your computer locally, fetching web content, and more. And there are security settings to enable, ask, or disable use of each one. Go to the page linked above and click the \"Run in LM Studio\" button.\u003c/p\u003e\n      \n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n","metadata":{}},{"element_type":"user_image","content":"https://cdn-learn.adafruit.com/user_assets/assets/000/002/352/original/tool_load.png?1778104503","metadata":{}},{"element_type":"text","content":"\n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n  \n        \u003cp\u003eThis will give you many more settings in the right toolbar:\u003c/p\u003e\n      \n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n","metadata":{}},{"element_type":"user_image","content":"https://cdn-learn.adafruit.com/user_assets/assets/000/002/351/original/tools.png?1778104306","metadata":{}},{"element_type":"text","content":"\n  \n        \u003cp\u003eYou'll want to go through the list to change from Ask to Allow or Deny depending on your trust in the model using tools responsibly.\u003c/p\u003e\n      \n","metadata":{}},{"element_type":"text","content":"\n        \u003ch2\u003eReference:\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003e\n\u003ca href=\"https://www.theregister.com/ai-and-ml/2026/05/11/yes-local-llms-are-ready-to-ease-the-compute-strain/5237451?shem=dsdf,sharefoc,agadiscoversdl,,sh/x/discover/m1/4\" target=\"_blank\"\u003eYes, local LLMs are ready to ease the compute strain\u003c/a\u003e 5/2026\u003c/li\u003e\n\u003cli\u003e\n\u003ca href=\"https://unsloth.ai/docs/basics/api\" target=\"_blank\"\u003eHow to use Unsloth as an API endpoint\u003c/a\u003e\u0026nbsp;\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"https://www.xda-developers.com/real-bottleneck-of-self-hosted-llm-stack-is-not-gpu/\" target=\"_blank\"\u003eAfter a year of self-hosting LLMs, I realized the real bottleneck isn’t the GPU\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"https://www.xda-developers.com/claude-code-with-a-local-llm-running-offline-is-the-hybrid-setup-i-didnt-know-i-needed/\" target=\"_blank\"\u003eClaude Code with a local LLM running offline is the hybrid setup I didn't know I needed\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\n\u003ca href=\"https://www.microcenter.com/site/mc-news/article/best-local-llms-8gb-16gb-32gb-memory-guide.aspx\" target=\"_blank\"\u003eRun AI Locally: The Best LLMs for 8GB, 16GB, 32GB Memory and Beyond\u003c/a\u003e 11/2025\u003cbr\u003e\u003cbr\u003e\n\u003c/li\u003e\n\u003c/ul\u003e\n      ","metadata":{}}]