Rajiv Shah - rajistics blog

Categories

All (36)

Adversarial (1)

Agentic (1)

Agentic AI (1)

Agents (4)

AI (8)

ai (1)

AI Engineering (3)

Annotated Talk (8)

Anomaly (1)

Basketball (1)

Classification (1)

Claude (1)

Coding Agents (3)

Conferences (2)

Conformal (1)

Data Preprocessing (1)

data science (1)

Dataframes (1)

Dataset (2)

Deep Learning (1)

Earthquake (2)

Evaluation (2)

Explainability (3)

featureselection (1)

Finetuning (1)

GenAI (1)

Generative AI (2)

Graph RAG (1)

H2O (1)

Harness Engineering (2)

Huggingface (2)

Interpretability (2)

Keras (2)

Label (1)

Leakage (2)

LLM (4)

machine learning (1)

Machine Learning (3)

MAPIE (1)

MCP (1)

MLOps (6)

MNIST (2)

Model Explanation (1)

NLP (4)

OpenAI (1)

OpenHands (1)

Optimization (1)

Outlier (1)

Productivity (1)

Quickdraw (1)

R (1)

RAG (2)

resources (1)

Retrieval (1)

RL (1)

RNN (2)

RSS (1)

SARCOS (1)

Semi-supervised (1)

Setfit (1)

Shiny (1)

snowflake (1)

Sparse (1)

sport (1)

Sport (1)

SportVu (1)

System Prompts (1)

Tensorflow (2)

Testing (2)

Text-to-SQL (1)

Transfer Learning (1)

XAI (2)

Xgboost (1)

Other Formats

Jupyter

GitNexus for GraphRAG

Testing GitNexus with OpenHands on VS Code

I have been skeptical of Graph RAG. I made a short video about that skepticism, and my issue was not that graphs are useless. The issue was the plumbing. For a lot of…

System Prompts Don’t Get Shorter. They Shift.

Claude Opus, Fable, and the harness layer around the model

A visual look at Claude system prompt evolution, and why the interesting question is where prompt complexity moves as models improve.

AI Engineer World’s Fair 2026: Talks Worth Watching

I sit in the middle of corn fields and watch the live stream. Thanks @aiDotEngineer and @swyx.

Harness Engineering: Why the System Around the Model Decides Agent Performance

Coding agents are forcing a shift in how we think about AI systems. A few years ago, many of us were using language models for short, stateless tasks. In this talk, Rajiv…

Reframing at AI Engineer Europe 2026

I’ve always been interested in how teams innovate when building AI. At AI Engineer Europe 2026, the talks that stayed with me weren’t the ones about bigger models or faster…

Taming the Algorithm: Moving from Algorithmic Feeds to Curating my own AI News

Annoyed by the “X is dead” or “have you seen this game-changer technology?” posts? With algorithmic feeds, this kind of content bubbles to the top on X and LinkedIn. I found…

Running Code and Failing Models

Machine learning is a glass cannon. When used correctly, it can be a truly transformative technology, but just a small oversight can cause it to become misleading and even…

A Practical Guide to Evaluating Generative AI Applications

Generative AI projects rarely fail because teams cannot get a model to produce a good demo. They fail because teams cannot reliably tell whether the system is improving…

RAG Retrieval Deep Dive: BM25, Embeddings, and the Power of Agentic Search

Watch the full video

My Picks for the Best AI/ML News Sources

A curated list of the best newsletters, podcasts, YouTube channels, and communities for staying up to date with AI and ML news.

Understanding Sparse Matrices through Interactive Visualizations

When working with machine learning models, preparing data properly is essential. One common preprocessing technique is one-hot encoding, which transforms categorical data…

Feature Selection Methods and Feature Selection Curves

How to Select the Best Features for Machine Learning!

Interpretable Machine Learning Models Simply Explained

Watch the full video

Spark of AI: How Transfer Learning Unlocked AI’s Potential

Watch the full video

Practical Lessons in Building Generative AI: RAG and Text to SQL

Watch the full video

Snowflake ML Intro Notebook - ML Forecasting

This notebook introduces several key features of Snowflake ML in the process of training a machine learning model for forecasting Chicago bus ridership.

Evaluation for Large Language Models (LLMs) and Generative AI - A Deep Dive

Watch the full video

Reasoning in Large Language Models

I was wowed by ChatGPT. While I understood tasks like text generation and summarization, something was different with ChatGPT. When I looked at the literature, I saw this…

Text style transfer in a spreadsheet using Hugging Face Inference Endpoints

We change our conversational style from informal to formal speech. We often do this without thinking when talking to our friends compared to addressing a judge. Computers…

Few shot text classification with SetFit

Data scientists often do not have large amounts of labeled data. This issue is even graver when dealing with problems with tens or hundreds of classes. The reality is very…

Getting predictions intervals with conformal inference

Data scientists often overstate the certainty of their predictions. I have had engineers laugh at my point predictions and point out several types of errors in my model that…

Explaining predictions from 🤗 transformer models

This post covers 3 easy-to-use 📦 packages to get started. You can also check out the Colab 📓 companion notebook at https://bit.ly/raj_explain and the Youtube 🎥 video for…

Dynamic Adversarial Data Collection

Are you looking for better training data for your models? Let me tell you about dynamic adversarial data collection!

Model Interpretability and Explainability for Machine Learning Models

Watch the full video

Stand Up for Best Practices

Source: Yuriy Guts selection from Shutterstock

Optimization Strategies

As a data scientist, you spend a lot of your time helping to make better decisions. You build predictive models to provide improved insights. You might be predicting whether…

Using Unlabeled Data to Label Data

Your boss hands you a pile of a 100,000 unlabeled images and asks you to categorize whether they are sandals, pants, boots, etc.

Using Google’s Quickdraw to create an MNIST style dataset!

For those running deep learning models, MNIST is ubiquotuous. This dataset of handwritten digits serves many purposes from benchmarking numerous algorithms (its referenced…

Deep Learning with R

For R users, there hasn’t been a production grade solution for deep learning (sorry MXNET). This post introduces the Keras interface for R and how it can be used to perform…

Building Worlds for Reinforcement Learning

OpenAI’s Gym places reinforcement learning into the masses. It comes with a wealth of environments from the classic cart pole, board games, Atari, and now the new Universe wh…

Taking an H2O Model to Production

One of the best feelings as a Data Scientist is when the model you have poured your heart and soul into, moves into production. Your model is now grown-up and you get to…

Using xgbfi for revealing feature interactions

Tree based methods excel in using feature or variable interactions. As a tree is built, it picks up on the interaction of features. For example, buying ice cream may not be…

Outlier App

I was recently trying various outlier detection algorithms. For me, the best way to understand an algorithm is to tinker with it. I wanted to share my recent work on a shiny…

RNN Addition (1st Grade)

Ever since I ran across RNNs, they have intrigued me with their ability to learn. The best background is Denny Britz’s tutorial, Karpathy’s totally accessible and fun post on …

SportVu Analysis

This post shares some of the code that I have created for analyzing NBA SportVu data.

Shiny front end for Tensorflow demo

I built a GUI front end for tensorflow from shiny, the code is available at Github. The shiny app allows trying different inputs, RNN cell types, and even optimizers. The…