When it comes to running LLMs locally (or offline - i.e., not in the cloud), we have several tools to choose from. Coming from the “cloud-native” services, I was quite surprised by their popularity. However, after observing the rapid cloud cost accumulation, I decided to do a little research on the subject.

Probably the best article on the subject is https://getstream.io/blog/best-local-llm-tools/ - this is a great one-stop-shop to start your offline inferencing with open-source models.

Tools for Offline LLM Inferencing:

1. Ollama
2. Llamafile / Executable Linkable Format (ELF)
3. Jan
4. LM Studio
5. LLaMa.cpp

The Engineering Excellence is a measurable efficiency of the dev piline described in PR lifecycle

Sorry, this articale is not finished yet, but the core idea should be visible from here:

1. Commits in all PRs
   1. Commits to all PRs ratio
2. Updated Open PRs
3. Closed (not Merged) PRs
   1. Abandoned PRs ratio
4. Merged PRs
   1. Merged PRs Avg. Duration
   2. Merged PRs Additions (+lines)
   3. Merged PRs Deletions (-lines)
   4. Merged PRs Changed Files
   5. Merged PRs → main
   6. Merged PRs → main ratio
5. AutoCRQ Deployments
   1. AutoCRQ PRs Lead Time

PR lifecycle

In our group we follow simple yet effetive PR workflow: poly-repo setup with a simplified GitHub process. Basically we use main branch with automatic staging deployment and ability to promote a build to the production environment. The promotion script automatically creates a CRQ (a change record in ServiceNow), that we call “auto-CRQ” since it doesn’t requite manual steps (approvals), unless the site status is not green.