Some Programmers Use AI (LLMs) Quite Differently

I see lots of otherwise smart people doing a few dozen manual prompts per day, by hand, and telling me they're not impressed with the current wave of AI.

They'll might say things like: AI's code doesn't reach 100% success rate expectation (whether for code correctness, speed, etc).

I rely on AI coding heavily and my expectations sky high, but I get good results and I'd like to share how / why:

First, let me say that I think asking a human to use an LLM to do a difficult task, is like asking a human to render a difficult 3D scene of a game using only his fingers on a calculator - very much possible! but very much not effective / not smart.

Small powerful LLM's like PHI can easily handle millions of separate small prompts (especially when you have a few 4080 GPU's)

The idea of me.. as a human.. using an LLM.. is just kind of ridiculous.. it conjures the same insane feelings of a monkey pushing buttons on a pocket calculator, your 4090 does math trillions of times per second with it's tens of thousands of tiny calculators so we all know the Idea of handing off originally-human-manual-tasks does work.

So Instead: I use my code to exploit the full power of my LLMs, (for me that's cpp controlling CURL communicating with an LLM serving responses thru LmStudio)

I use a basic loop which passes LLM written code into my project and calls msbuild. If the code compiles I let it run and compare it's output results to my desired expectations. If the result are identical I look at the time it spent in the algorithm. If that time is the best one yet I set it as the current champion. New code generated is asked to improve the implementation and is given the current champion as a refence in it's input prompt.

I've since "rewritten" my fastest Raytracers, Pathfinders, 3D mesh generators etc all with big performance improvements.

I've even had it implement novel new algorithms which I never actually wrote before by just giving it the unit tests and waiting for a brand new from scratch generation which passed. (mostly todo with instant 2D direct reachability, similar to L.O.S. grid acceleration)

I can just pick any algorithm now and leave my computer running all night to get reliably good speed ups by morning. (Only problem is I largely don't understand how any of my core tech actually works any more :D, just that it does and it's fast!)

I've been dealing with Amazon's business AI department recently and even their LLM experts tell me no one they know does this and that I should go back to just using manual IDE LLM UI code helpers lol!

Anyways, best luck this year, have fun guys!

Enjoy