AI Repo: lyogavin/airllm

Skyler: Hey everyone, welcome back to Git News A I, the show where we dig through GitHub's hottest trending repositories so you don't have to. Big thanks to our sponsors today — Ada, which helps teams integrate A I into their existing workflows without the headaches, Ago Consulting, your go-to for silicon development consulting from A I all the way down to SoC design, and Zen Semiconductor, an umbrella company with a whole portfolio of silicon-industry companies, including Ada for deploying A I at scale in any business, and their Vect appliance which runs local L L Ms and vector search on dedicated hardware — seriously cool stuff. Today we've got five repos on the docket: we're talking about running 70-billion parameter models on a single 4 gig GPU, turning your entire codebase into a queryable knowledge graph, building real-time voice A I agents, a full-stack L L M observability platform, and a terminal-native coding agent built on DeepSeek. Let's get into it. Skyler: Alright, first up we have AirLLM by lyogavin, sitting at over 27,000 stars and climbing. The tagline alone should stop you in your tracks — 70 billion parameter L L M inference on a single 4 gigabyte GPU. If you've been in the M L space for more than five minutes, you know that running a 70B model typically requires something like 140 gigabytes of GPU memory, which means racks of expensive hardware most of us just don't have at home or in a small startup environment. AirLLM completely flips that script. Skyler: So how does it actually pull this off? The core idea is layer-wise inference combined with aggressive memory optimization. Instead of loading the entire model into GPU memory at once, AirLLM loads and processes one layer at a time, keeping the rest on disk or CPU memory and streaming it in as needed. You take a performance hit in terms of speed, absolutely, but you gain the ability to actually run the model in the first place, which is a pretty compelling trade-off when the alternative is not running it at all. Skyler: The project just dropped version 3.1.0, and the latest pull request is particularly exciting — they've added support for Kimi K3. If you haven't heard of Kimi K3, it's one of the newer frontier-class models, so the fact that AirLLM is already adding support shows the team is very much keeping pace with the rapidly moving model landscape. The library supports a wide range of architectures and integrates cleanly with HuggingFace model repos, so your existing workflow probably doesn't need much adjustment. Skyler: Who should absolutely check this out? If you're a researcher, a hobbyist, or a small team that wants to experiment with massive models without spinning up a cloud instance every time, this is genuinely transformative. It's also a fantastic educational tool — being able to poke around a 70B model locally changes how you think about these systems. Check the show notes for the link, because this one is a must-bookmark. Skyler: Next repo — and this one has an absolutely wild star count — is Graphify by Graphify Labs, sitting at over 101,000 stars. One hundred and one thousand. That is not a typo. This project turns your entire codebase, including documentation, SQL schemas, config files, and PDFs, into a queryable knowledge graph. Think of it less like a search engine and more like giving your A I assistant a deep, structured understanding of how your entire project actually fits together. Skyler: What makes Graphify really interesting from a technical standpoint is what it doesn't do. A lot of tools in this space rely on vector stores and semantic search, which can be fuzzy and sometimes frustratingly unpredictable. Graphify instead uses deterministic A S T parsing — that's Abstract Syntax Tree parsing — to build its knowledge graph locally. Every edge in the graph is explained, meaning you can trace exactly why the system thinks two pieces of code or documentation are related. No black boxes, no embeddings doing mysterious things behind the scenes. Skyler: It's designed as a skill — or plugin — for the major A I coding environments: Claude Code, Cursor, Codex, and Gemini C L I. So whatever your current A I-assisted development setup looks like, there's a good chance Graphify slots right in. You ask a question about your codebase, and instead of getting a vague similarity-based answer, you get a precise graph-traversal answer with lineage you can actually audit. Skyler: The recent activity has been quiet, which sometimes makes people nervous, but with over 100K stars and the breadth of what it supports, this feels more like a mature, stable project than an abandoned one. If you work on large, complex codebases where understanding cross-cutting concerns is genuinely painful — and honestly, who doesn't — Graphify is worth a serious look. Check the show notes for the link. Skyler: Okay, repo number three is LiveKit Agents, just under 12,000 stars, and this one is near and dear to my heart because real-time voice A I is having such a moment right now. LiveKit is a well-known open-source framework for real-time audio and video, and their Agents library extends that foundation specifically for building real-time voice A I agents. We're talking full duplex conversation, low latency, production-ready infrastructure for voice-driven A I applications. Skyler: The framework handles all the gnarly plumbing that makes real-time voice so hard to get right — things like interruption handling, voice activity detection, turn-taking logic, and streaming responses. If you've ever tried to build a voice bot from scratch and ended up in a nightmare of threading issues and audio buffer management, you know exactly what I'm talking about. LiveKit Agents abstracts all of that so you can focus on the actual agent behavior and personality. Skyler: Looking at the recent commits, there's a brand new Bland T T S plugin that just landed, adding text-to-speech support from Bland A I. And they've also done some nice developer experience work, making every example in the repo a UV workspace member, which if you're using UV for Python package management — and you should be, it's fantastic — means you can spin up any example with minimal friction. Skyler: The framework integrates with major L L M providers and supports multi-modal agents, so you're not just limited to audio — you can build agents that also process video. If you're building anything in the customer service, accessibility, or conversational A I space, LiveKit Agents is one of the most production-battle-tested options available. It's open source, it's actively developed, and the community is solid. Check the show notes for the link. Skyler: Alright, repo four is Opik by Comet M L, and this one is sitting at over 21,000 stars. Opik is a full observability and evaluation platform for L L M applications, and I want to be clear about what that means because "observability" gets thrown around a lot. Opik covers three distinct problem areas: debugging during development, systematic evaluation of your L L M's behavior, and production monitoring once you've shipped. Skyler: The debugging side gives you comprehensive tracing — so every time your L L M application runs, you can see exactly what prompts went in, what came out, how long each step took, and where things went sideways. If you've ever shipped a RAG system and then spent hours trying to figure out why it gave a weird answer to a specific query, having this kind of trace data is genuinely life-changing for your debugging workflow. Skyler: On the evaluation side, Opik offers automated evaluations, which means you can define test cases and metrics and run them continuously, almost like unit tests for your L L M behavior. This is huge for teams that are iterating fast on prompts and need some guardrail against regressions. It also plays nicely with agentic workflows, which are notoriously hard to evaluate because the execution paths are non-deterministic. Skyler: The recent commits show the team has been doing some solid backend work — there's a fix for a date partitioning bug in the traces system that could cause issues with far-future IDs, and documentation work on AssertJ assertion conventions, which tells me they're being disciplined about testing and developer documentation. That's always a good sign for a project you're going to depend on in production. If you're building anything with L L Ms professionally, Opik deserves a spot in your toolchain. Check the show notes for the link. Skyler: And our fifth and final repo today is DeepSeek Reasonix by esengine, at almost 30,000 stars. This is a DeepSeek-native A I coding agent built specifically for your terminal, and it's written in Go, which is a bit of a refreshing departure from the Python-everything world of A I tooling. The core pitch here is a terminal-native experience engineered specifically around prefix-cache stability, and that last part is key to understanding what makes this different. Skyler: Prefix caching is a technique where the L L M provider caches the computed state of repeated prompt prefixes, so if your system prompt or context stays the same across requests, you're not paying the full compute cost every time. DeepSeek has pretty aggressive prefix caching on their API, and Reasonix is explicitly designed to work with that — meaning it structures its context and requests in ways that maximize cache hits. The result is a much more economical and stable agent that you can genuinely leave running in the background. Skyler: The "leave it running" aspect is actually a design philosophy here, not just a casual feature. Most coding agents are designed for quick interactive sessions. Reasonix is designed for longer-horizon tasks where you set it going and come back to results. For the kind of refactoring jobs or code generation tasks that might take dozens of A P I calls, that cache stability translates into real cost savings and much more predictable behavior. Skyler: Recent activity shows two pull requests merged — one around local rewind reliability and one fixing NPM idempotency, both of which suggest the team is grinding through reliability and correctness improvements, which is exactly what you want to see in an agent you're going to trust with your codebase. If you're a Go developer or just someone who lives in the terminal and wants a coding agent that respects that environment, DeepSeek Reasonix is absolutely worth trying. Check the show notes for the link. Skyler: And that's a wrap on today's episode of Git News A I. Huge thank you to our sponsors — Ada for making A I integration actually practical for real teams, Ago Consulting for their incredible work in silicon development from A I systems down to SoC design, and Zen Semiconductor for pushing the frontier on local A I hardware with their Vect appliance and their broader portfolio of silicon-industry companies. These folks make this show possible and they're all doing genuinely interesting work, so go check them out. If you enjoyed today's episode, make sure you check out AI Special Report in the Ada podcast network — it's a fantastic show for staying current on the broader A I landscape and pairs really well with what we do here. And of course, subscribe wherever you get your podcasts, leave a review if you're feeling generous, and all the repo links from today's episode are in the show notes. We'll see you next time.