TL;DR: Use import modernc.org/sqlite with no CGO deps and 100% database/sql interface compatibility at about a 20% cost in performance. Annoying name and source hosting, but oh well.
To CGO or not to CGO, that is the question. “It depends” is always the answer. If we do, we get some kick-ass speed and official compatibility with the database/sql standard Go interface. We also are immediately ready to write a compatible API in C directly. But, building for CGO is a pain in the ass.
The problem is that you really pay for performance when you drop the C library:
This post summarizes some basic ingestion and query benchmarks using mattn/go-sqlite3 and modernc.org/sqlite. tldr; the Go translation is between twice as slow and 10% slower for both small datasets and large, for INSERTs and SELECTs.
Another advantage of just accepting our CGO fate is that later I can incorporate other C things (more notable for Python stubbing than anything) and be ready for it.
Still, adding a CGO dependency to this single thing will force every Bonzai monolith that includes a command branch that uses this to use CGO even if it is just for that single branch. There is no way that is worth it. So the answer is NO CGO — ever. I don’t care how much a perf hit we take. The load is already distributed across different instances of keg directories. And, if it every came to it, we can replace the internal non-CGO API with one that is, no premature optimization.
By the way, looked at cvilsmeier/sqinn-go which is so bad it’s comical. Strong recommendation to stay the hell away from it, but you’ll get that sense immediately when you read “listed in some-dumb-ass’s” awesome-go list.”