Dekita

I built an offline invoice sorter for freelancers, then measured whether local inference is actually cheaper

devchallenge hf26challenge

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle

What I Built

InvoiceLens reads a folder of invoices and receipts, pulls the fields out of each one with a local open-weight model, sorts them into categories from a rule file you edit yourself, and writes a spreadsheet you can hand to an accountant.

It is built for one person: the freelancer who bills by the hour and files their own taxes. In September a client invoice arrives, in October a hosting bill, and in between a pile of mixed receipts. At tax time that pile has to become one file that says what came in, from whom, and which of it is a business cost.

Nothing leaves the machine. No API key, no account, no network call after the model is downloaded.

The challenge prompt lists ideas, and its first one is almost exactly this: "an invoice and receipt sorter for freelancers that runs locally, so client data never leaves their laptop." I took that literally and then spent most of the build arguing with it, because the obvious way to do this on a laptop does not work. More on that below.

What it does today, measured rather than claimed:

| Capability | Status | |---|---| | Text-based .pdf and .txt input | Works | | Field extraction (vendor, dates, amounts, line items) | Works | | Expense categorization from a YAML rule file | Works | | Deductibility flags | Works, rules are yours to edit | | CSV + JSON + Markdown output | Works | | Self-repair loop for malformed model replies | Works | | Scanned PDFs and images (OCR) | Not in v1 | | A GUI | No. CLI only. | | Tax advice | No. It sorts and flags. It does not decide your return. |

Demo

There is no deployed link, and that is the point rather than an omission: a tool whose entire argument is that your client data never leaves your laptop cannot demo by uploading your client data. So here is the real thing, run on three synthetic invoices on a 4-core i5-8350U:

$ python -m invoicelens scan fixtures --out report --rules rules/rules.example.yaml

InvoiceLens 0.1.0: local-only invoice triage
model : ~/.cache/invoicelens/models/gemma-2-2b-it-Q4_K_M.gguf (1708582752 bytes)
rules : rules/rules.example.yaml (7 categories)
files : 3

engine: llama.cpp, CPU only
[1/3] design-retainer-2026-09.txt
      vendor=NORTHWIND STUDIO date=2026-09-03 total=2400.0 USD
      category=professional_services deductible=True
      extraction: 1 attempt

[2/3] office-mixed-2026-09.txt
      vendor=MERIDIAN OFFICE SUPPLY date=2026-09-27 total=1124.82 USD
      category=office_supplies deductible=True
      extraction: 1 attempt

[3/3] saas-subscription-2026-10.txt
      vendor=GITHUB, INC. date=2026-10-02 total=4.0 USD
      category=software_subscriptions deductible=True
      extraction: 1 attempt

done in 223.3s: 3/3 extracted, 0 repaired, 0 failed

Three for three, first attempt, no repair prompts. The middle one is the interesting file: a mixed receipt with a standing desk, a monitor arm, paper, a chair, and two kilos of coffee beans, and the model got all five line items right. The third file is the one that shows the schema holding up: a $4 GitHub Team invoice whose second line is "Additional Actions minutes (2,000)", which the model correctly read as quantity 2000 at unit price 0 rather than as another $4 charge.

The spreadsheet that comes out:

file,vendor,invoice_number,invoice_date,due_date,currency,subtotal,tax,total,category,deductible,rule_note,extraction_attempts,extraction_repaired,error
design-retainer-2026-09.txt,NORTHWIND STUDIO,NS-2026-0917,2026-09-03,2026-10-03,USD,2400.0,0.0,2400.0,professional_services,True,Subcontractor and agency invoices are normally a business cost.,1,False,
office-mixed-2026-09.txt,MERIDIAN OFFICE SUPPLY,MOS-88921,2026-09-27,2026-10-27,USD,1041.5,83.32,1124.82,office_supplies,True,Office equipment may be deductible or may need to be capitalized. Check locally.,1,False,
saas-subscription-2026-10.txt,"GITHUB, INC.",GH-7714-2026,2026-10-02,,USD,4.0,0.0,4.0,software_subscriptions,True,Business software subscriptions are normally deductible.,1,False,

And the honest part, because the demo above hides it. That mixed receipt was mis-filed. The model extracted everything correctly, but the keyword rules classified the whole invoice as office_supplies, because two hits (office from the vendor name, monitor from a line item) beat meals_entertainment's one hit (coffee). The coffee landed in office supplies, where nobody will look at it again. It is in the README's limitations section, it is in the fixture, and fixing it needs either per-line-item classification or a model pass over the rules. Neither is in v1.

Code

{% github dekita-dev/invoicelens %}

MIT licensed, 45 passing tests that run with no model download, and a tools/smoke_engine.py that does a real end-to-end check when you do have the model.

How I Built It

The model is Gemma 2 2B IT, quantized to Q4_K_M, about 1.6 GB, run through the official llama.cpp release binary. InvoiceLens starts llama-server as a subprocess, waits for /health, and talks to it over HTTP on 127.0.0.1. The engine wrapper is roughly 40 lines of urllib against an OpenAI-compatible surface.

That shape is not what I started with, and the reason it changed is the most useful thing I can tell you about building on Windows.

llama-cpp-python is out. It publishes no win_amd64 wheels for any Python version, and this machine has no C++ compiler.

gpt4all looked like the answer and was not. It ships a py3-none-win_amd64 wheel with llama.cpp bundled in, which is exactly what you want. Three things kill it on Windows, each verified rather than assumed:

  1. The wheel contains only two backends, CUDA and Kompute/Vulkan. There is no

CPU backend DLL at all, and device="cpu" is a no-op: gpt4all.py:222 hard-codes backend = "kompute" on every non-macOS platform.

  1. The Kompute build aborts inside llama.cpp during model load. It does so

identically at llama.cpp:11912 for every combination of n_ctx in {2048, 4096, 8192} and n_gpu_layers in {0, 1, 12, 100}. That matrix rules out context size and GPU memory as causes, which is what let me stop guessing.

  1. The bundled llama.cpp is old. The architecture table inside the DLL contains

gemma2 but not gemma3, llama4 or qwen3, placing it around early 2025, before GGUF v3 was fully supported. A current Gemma 2 GGUF (version 3) aborts in llama_model_load as a result.

I want to be clear about step 3, because I got it wrong first. I assumed it was a CUDA problem, installed the CUDA runtime and cuBLAS wheels (which did fix a real LoadLibraryExW error 0x7e), and the assert did not move. Fixing a real bug did not fix the actual bug. The version mismatch was the cause, and the only way to see it was to read the architecture table out of the DLL.

So: the official release binary. Current, a real CPU backend that picks the right instruction-set variant at load time, and an HTTP surface that means no ctypes binding against a moving C ABI.

Two other things worth stealing from this build:

The parse error becomes the next prompt. The model is asked for a strict JSON object. If the reply does not parse, the exception text and the bad reply are fed back as the next prompt. This is deliberately not a regex fallback: the model still owns the extraction, and the retry only tells it what went wrong. Every run reports how many attempts each file needed, so you can see the failure rate instead of trusting a silent success.

Physical cores, not logical ones. Generation with a small quantized model is memory-bandwidth bound, so hyperthreads cost more than they return. Measured on this machine with tools/bench_backends.py, which reads the server's own timings block instead of dividing wall clock by a guessed token count:

| Threads | Prompt tok/s | Generation tok/s | |---|---|---| | 2 | 18.7 | 5.15 | | 4 | 23.0 | 7.44 | | 8 | 33.6 | 5.84 |

Two things fall out of the split. Prompt processing keeps scaling all the way to eight threads. Generation peaks at the four physical cores and then goes backwards: with hyperthreads on it runs 22% slower than with four. And because generation is roughly two thirds of a real invoice's time, four threads still wins overall even though eight would get through the prompt 46% faster. The default is now physical core count, detected per platform, capped at 8.

Then I tried the GPU, because the machine has three of them. The device matrix is not what the spec sheet suggests. llama-bench, 512-token prompt and 16 generated tokens, four threads, all 26 layers offloaded:

| Device | Prompt tok/s | Generation tok/s | |---|---|---| | CPU, 4 threads | 23.7 | 6.4 | | Intel UHD 620 (integrated) | 18.5 | 4.3 | | AMD RX 6600 | 1142.6 | 42.1 | | Quadro P500 | cannot allocate | - |

The integrated GPU is slower than the CPU, so "offload to the GPU" is not advice, it is a device-selection problem. Full offload to the RX 6600, measured on the real three-invoice workload with the benchmark that produced the thread table:

| | CPU (default) | RX 6600, 26 layers | |---|---|---| | Prompt tok/s | 23.0 | 604 | | Generation tok/s | 7.44 | 38.7 | | Cold start | 5.7 s | 8.3 s | | Three invoices, wall clock | 223.3 s | 26.1 s |

That is 8.6x end to end, and it survives the honesty check: every field the tool depends on came out bit-identical between the CPU run and the GPU run, from vendor and dates through all nine line items to deductibility, one attempt each, zero repairs. One optional free-text field did not survive: notes missed a line it had caught on CPU, reproducibly, across two GPU runs. Same model, different arithmetic, different answer on a field nobody should be trusting anyway. That caveat ships in the README next to the flag.

The flag is opt-in (--n-gpu-layers 26 --device vulkan1) and the default is untouched, because the fastest device is not something the tool can know for you: --list-devices exists precisely because device 0 on this box is the worst usable one.

How this differs from the neighbouring submission. Another entry in this challenge, LedgerVault AI, solves the same problem for the same person, and in two ways it is ahead of mine: it has a real UI, and it does OCR through tesseract.js in a WebAssembly worker, which means it reads scans and photos. InvoiceLens v1 does neither.

The trade goes the other way on setup. LedgerVault requires you to install Ollama separately and run llama3.2, qwen2.5, mistral or smollm2 through it. InvoiceLens has no external runtime at all beyond pip install -r requirements.txt: it drives llama-server from the official llama.cpp release as a subprocess, and it uses Gemma. So it is one subprocess to manage against one service to install, and a CLI against a browser app. Which of those is the better trade depends entirely on whether the person doing their taxes would rather run a command or open a page.

Why Does Open Innovation Matter?

I went into this expecting to write that running locally is cheaper. I measured it, and on this evidence it is not, and the article is better for saying so.

Token counts, server-reported rather than estimated:

| File | Prompt tokens | Completion tokens | Wall clock | |---|---|---|---| | design-retainer-2026-09.txt | 583 | 273 | 57.0 s | | office-mixed-2026-09.txt | 573 | 400 | 72.9 s | | saas-subscription-2026-10.txt | 508 | 258 | 50.4 s | | Total | 1664 | 931 | 180.3 s |

The token counts came out identical on a second, independent run. Worth knowing: tokenizing a fixed prompt is deterministic even though generation is not, so the cost side of this table is reproducible and the time side is not.

Priced against Gemini 2.5 Flash-Lite at $0.10 per million input and $0.40 per million output tokens (Google's published rate, page last updated 2026-10-14, the cheapest tier on that page), those 2595 tokens cost $0.000539, about $0.00018 per invoice.

Now the same run as electricity. At an assumed 45 W whole-machine draw for 180.3 seconds and $0.15/kWh, that is $0.000338. The hosted API is 1.59× the electricity. At 300 invoices a year the two differ by roughly 2 cents.

So the money argument is a wash, and anyone who tells you local inference is free is selling something. What you actually buy is not cost:

The data does not leave, and the cheap option specifically does not keep it private. Google's own pricing page states that on the free tier, inputs and outputs are used to improve Google's products; the paid tier does not. So the "just use the cheap API" path means a freelancer's client names, project names and amounts go to a third party, either for money or for training data. For a one-person business whose product is discretion, that is the whole argument.

No account, no key, no quota, no deprecation. Nothing to rotate, nothing to be rate-limited by, no model version pulled out from under you mid-quarter.

It works on a plane, which matters more than it sounds for anyone who does their books on a trip.

And the honest price of all that is time: one to one and a quarter minutes per invoice on four cores (180.3 s and 223.3 s on two runs of the same three invoices), against roughly two seconds for a hosted API. That is the real trade, and it is a good trade for a few hundred invoices and a bad one for five thousand. The README says so in those words, and it also says what the opt-in GPU flag does to the number: the same three invoices in 26.1 s on an RX 6600, about 9 seconds each.

One measured detail that surprised me: the system prompt is 986 characters, and the measured prompt for the first file was 583 tokens. Roughly 40% of what you send on every call is instructions, not invoice. That overhead is identical whether you run it locally or pay per token, so it is not an argument for either side. It is just the thing nobody tells you.

My Agent Session

This article was written with AI assistance. Every number in it was measured on the machine described in the demo above, and the scripts that produced them are linked below so you can rerun them.

This was built by an AI agent, and the interesting part of the session was not the code, it was the dead ends. Every runtime claim above was falsified with evidence before it was abandoned, and the full investigation is written down in the repository rather than in a chat log that will be gone next week:

The thread table above has a history worth telling, because it was wrong twice over and the second error is the part that matters. The first draft of this post carried a thread table headed "seconds per token" from a script that actually measured seconds per output line. Going back to reproduce the numbers found the worse problem: bench_threads.py could not produce a number at all. Its worker unpacked two command-line arguments and the caller passed one, so every run died with an IndexError inside the worker and printed an empty row that looked like a measurement. The mismatch had been sitting in the initial commit, which means the published table cannot have come from the script as committed. Both are fixed (66a6e9a, e044102), the table above is re-measured, and run-to-run spread is 10 to 20 percent: the shape of the curve is the finding, not the third digit.

The GPU section has its own list, and it is longer. Upstream llama.cpp changed the default of --n-gpu-layers from 0 to auto between the two builds this project uses, which means a flag you do not pass is a flag you do not control: the first "CPU control" run on the Vulkan build silently offloaded to the integrated GPU and reported a generation rate the CPU cannot physically reach, and that physical impossibility is the only reason it was caught. On that same build --n-gpu-layers 0 does not even mean CPU compute, llama-bench reports backend: Vulkan with the weights still in system RAM. And omitting --device is not "use device 0": it measured half the explicit-device rate and matched no single GPU on the machine. Each one is written up with its raw logs in docs/bench-vulkan.md.

I did not use DevRelay to save and embed the session, so the agent_session tag is not in this post. Installing it means putting a third-party MCP gateway on the machine, which was not a call I wanted to make on someone else's behalf; the links above cover the same ground and will not expire.

Prize Categories

Best Use of Gemma (Featured, $200). The whole extraction path is Gemma 2 2B IT, Q4_K_M, running locally through llama.cpp. The project would not work without it, and the model choice is also why the privacy argument holds: a 2B model that fits in 1.6 GB is what makes "runs on the laptop you already own" true rather than aspirational.

<!-- Thanks for participating! -->

← All writing