Posts

If you’re running inference, training, AI experiments or a lab locally on Apple silicon, you’ll likely want to be able to monitor your workloads. If you’re running your workloads on CPUs, you have the normal *top family of tools available like top, htop and btop. But for GPUs, you’ll need to reach for a different set of tools. The two that I’ve been testing are: PowerMetrics macmon PowerMetrics GPU Metrics The built-in tool on macOS is called PowerMetrics. This works, but there are a few problems. First off, you have to run it with sudo, which is pretty tedious. The other problem is that Apple hasn’t documented it very well, so I’ve had to make some assumptions in interpreting the data. ...

With the proliferation and widespread adoption of LLMs, I’ve heard a lot of people talking about how they “lie” recently. I’ll admit, I’ve felt the urge to reach for that word myself, but it’s not quite right for what’s actually happening. LLMs don’t lie as such: the more accurate term is hallucination. LLMs are built on what is known as a corpus, which is all of the data used to train the model. If the information you’re looking for isn’t in the corpus, then the LLM can’t by itself give you the answer. What tends to happen in modern frontier systems is that the orchestration layer and the model will work together to try and fetch the missing information from the internet or any other storage the system is aware of. If that fails, the LLM will do one of two things: ...

I have a lot of discussions about AI these days, in fact, it’s almost exclusively what I talk about in a work context. Some months ago, I found myself in a difficult position, in trying to wrangle with all of the different components that I was discussing. There is a rapidly growing and sprawling landscape to try and get your head around and I eventually had to sit down and try and model it in a simple, describable way. ...