I build LLM agents and retrieval systems — and, the harder half of the job,
the evaluation that tells you whether they actually work.
For teams where being wrong is expensive: scientific and clinical work,
regulated data, and anywhere a confident wrong answer costs more than no
answer at all.
Four ways I take on work. Each one is tied to something in the next section
rather than to a keyword list.
Evaluation and calibration audit
2–3 weeks · fixed scope
The hardest question in an ML project is rarely "can we build it" — it is
"how do we know it is working, and how confident should we be". I take an
existing model or agent and build the apparatus that answers it: an
evaluation suite tied to the decision you are actually making, calibration
of the scores it produces, and a written recommendation you can act on.
My doctorate was on exactly this, and it is the part of the field most AI
consultancies skip.
Agent architectures, context and memory management, and RAG pipelines built
on documents that are messy in the ways real documents are. Evaluation wired
in from the first week rather than bolted on once something has already gone
wrong in production.
Gene therapy and proteomics, remote sensing, drug-discovery feature
selection. Domains where the data is expensive, the experiments are slow,
and the model's real job is to decide what to test next.
For teams with good engineers and nobody senior who has taken machine
learning to production before. Architecture review, hiring, mentoring, and
the unglamorous work of turning research code into something monitored,
tested and maintainable.
Designing recombinant AAV vectors means searching a combinatorial
sequence space that is far too large to assay exhaustively in the lab.
Approach
Generative models applied to the biomolecular engineering of gene
therapies: folding proteins and designing capsids, optimising promoters,
miniaturising ITRs, and engineering for antibody avoidance — so that
wet-lab budget goes to the variants most likely to work.
Outcome
The methods were written up and peer reviewed: Rabinowitz & Kalia,
“Fine tuning AAV: machine learning the capsid”,
Cell & Gene Therapy Insights 2023; 9(4), 785–798.
Retrieval systems rarely fail at the embedding step. They fail earlier,
on the unglamorous work of getting real Word and PDF documents into
clean, correctly segmented text.
Approach
Built and released two pieces of that layer: a Word document parser
producing structured JSON, and a classifier that labels extracted PDF
text chunks by type so downstream chunking can respect document
structure instead of splitting on character counts.
Stack
Python, unstructured, scikit-learn
Outcome
Both released as standalone open-source tools, so the ingestion layer is
something I bring to an engagement rather than something a client pays
to have rebuilt.
Ordinal classification and calibrated probabilities
Doctoral thesis · Department of Statistics, University of Oxford · supervised by Brian D. Ripley · 2011
Problem
Many prediction problems have ordered outcomes, and the useful
output is not a label but a calibrated probability. Standard classifiers
give neither.
Approach
Developed RankingSVM, an ordinal classifier using thermometer encoding
to capture non-linearity through support vector machines; paired it with
Bayesian proportional-odds logistic regression to estimate probabilities
consistent with observed market prices.
Evaluation
Introduced a non-parametric method for comparing ordinal classifiers by
treating evaluation as a betting problem — a model's predictive quality
is measured by what a stochastic-programming gambling strategy built on
it would actually earn. Proper scoring in the most literal sense.
Outcome
A thesis, three accompanying papers, and the evaluation method I still
use to decide whether a client's model is good enough to act on.
People who need answers from a database mostly cannot write SQL, and the
queries that matter are rarely the simple ones.
Approach
Proposed and prototyped semantic parsing of natural language into complex
SQL, working from the then-current research literature — the problem the
whole industry now reaches for an LLM to solve, approached before there
was one to reach for.
Outcome
A working demo the engineering team could try, and an internal machine
learning meet-up series that brought the wider group up to speed on the
field.
Object detection in satellite imagery
An object detector over multispectral LANDSAT scenes, locating sargassum
rafts on open water so Caribbean coastal operators get warning before a
bloom lands.
Code and notebooks
Anomaly detection without labels
Isolation forests and random cut forests compared on unlabelled, heavily
imbalanced data, including the streaming case where the model has to update
as the definition of normal moves.
CodeFeature selection experiments
Recognise your problem in any of that?
Half an hour is usually enough to tell whether I am the right person for it, and
I will say so either way.
Merged contributions to
PyTorch Geometric
(device-handling test decorators, packaging, documentation) and
einops.
About
I am Ravi Kalia. I read Applied Statistical Computing at Oxford before staying on for
a doctorate in statistical machine learning there, supervised by
Brian D. Ripley. The work
was on ordinal classification and on the awkward question of how you tell whether a
probabilistic model is any good — a strange but useful preparation for two decades of
applied machine learning.
Across those roles the work has spanned gene therapy and proteomics, remote sensing,
financial modelling and document-heavy NLP. I have spoken on machine learning at
PyCon, PyData, Google Developer Group and several universities across four
countries, and I write regularly at Synthetic
Musings.
What holds my attention right now: agent memory and context management, which I
think is where most of the real engineering in LLM systems currently sits;
explainability treated as a localization problem rather than a feature-attribution
one; and the perennial question of calibration — how much of a model's confidence
you should actually believe.
I take on a small number of engagements at a time, usually some mix of hands-on
build work and advisory. If you have a problem where the modelling is genuinely
uncertain — where it is not obvious whether the thing can be built, or how you
would know if it worked — that is the kind of work I most want to hear about.
Two ways in, and neither is better than the other. Put half an hour in my calendar,
or write first and tell me what you are up against — the form goes straight to my
inbox.
New writing on machine learning and agent systems, plus the occasional note on
consulting availability. No schedule, no marketing, unsubscribe in one click.
Genial Labs