Agentic applications represent a significant leap in AI, moving beyond simple chatbots to systems that can autonomously reason, plan, and execute multi-step tasks to achieve a defined goal. They are designed to operate with minimal human intervention, continuously learning and adapting over time.

I recently embarked on an interesting experiment with GitHub Copilot Agent that has given me some valuable insights into the evolving landscape of software development. I thought it would be worth sharing my experience and observations.

When it comes to running LLMs locally (or offline - i.e., not in the cloud), we have several tools to choose from. Coming from the “cloud-native” services, I was quite surprised by their popularity. However, after observing the rapid cloud cost accumulation, I decided to do a little research on the subject.

Probably the best article on the subject is https://getstream.io/blog/best-local-llm-tools/ - this is a great one-stop-shop to start your offline inferencing with open-source models.

Tools for Offline LLM Inferencing:

1. Ollama
2. Llamafile / Executable Linkable Format (ELF)
3. Jan
4. LM Studio
5. LLaMa.cpp