Start with a simple story

Imagine you run a small restaurant, and you need a cook to prepare a dish.

Option 1: You call a restaurant across town, place the order, wait for delivery, and hope the roads are clear.

Option 2: You hire the cook and put them inside your own kitchen.

Option 2 is faster, simpler, and doesn't depend on anyone else's traffic.

That is the idea behind in-process ONNX Runtime in .NET. The AI model runs inside your own application instead of living on a separate server that you call over the network.

First, what is ONNX?

ONNX (Open Neural Network Exchange) is a universal file format for AI models. Think of it as the PDF of machine learning.

A PDF opens the same way everywhere, no matter which app created it. In the same way, a .onnx file can be created by many tools (ML.NET, PyTorch, TensorFlow, scikit-learn) and run in .NET, Java, C++, or JavaScript.

ONNX Runtime is the engine, built by Microsoft, that actually opens and runs that .onnx file quickly.

What does "in-process" mean?

In-process means your C# code loads the model into its own memory and calls it like any other function. There is no REST endpoint, no Docker container just for AI, and no "is the AI service down?" at 2 AM.

Let's build something (in 5 minutes, only C#)

We'll predict a house price from three inputs: area (sq ft), bedrooms, and age of the house.

The whole flow is: train in C# → save as ONNX → run with ONNX Runtime.

Step 1: Create the project

dotnet new console -n HouseAi cd HouseAi dotnet add package Microsoft.ML dotnet add package Microsoft.ML.OnnxConverter dotnet add package Microsoft.ML.OnnxRuntime

Step 2: Paste this into Program.cs

using Microsoft.ML; using Microsoft.ML.OnnxRuntime; using Microsoft.ML.OnnxRuntime.Tensors; // ---------- PART 1: Train a tiny model in C# ---------- var ml = new MLContext(seed: 1); // Make practice data: price depends on area, bedrooms and age of the house var random = new Random(42); var houses = Enumerable.Range(0, 300).Select(_ => { float area = random.Next(500, 2500); float bedrooms = random.Next(1, 6); float age = random.Next(0, 30); float price = area * 0.05f + bedrooms * 5f - age * 0.3f; // price in lakhs return new HouseData { Area = area, Bedrooms = bedrooms, Age = age, Price = price }; }).ToList(); var trainingData = ml.Data.LoadFromEnumerable(houses); var pipeline = ml.Transforms .Concatenate("Features", nameof(HouseData.Area), nameof(HouseData.Bedrooms), nameof(HouseData.Age)) .Append(ml.Transforms.NormalizeMinMax("Features")) .Append(ml.Regression.Trainers.Sdca(labelColumnName: nameof(HouseData.Price), maximumNumberOfIterations: 200)); var trainedModel = pipeline.Fit(trainingData); // ---------- PART 2: Save it as an ONNX file ---------- using (var stream = File.Create("house_price.onnx")) { ml.Model.ConvertToOnnx(trainedModel, trainingData, stream); } Console.WriteLine("Model saved as house_price.onnx"); // ---------- PART 3: Load it with ONNX Runtime and predict ---------- using var session = new InferenceSession("house_price.onnx"); // Peek inside the model so you can see what it expects foreach (var input in session.InputMetadata) Console.WriteLine($"Model input: {input.Key}"); // The values for the house we want to price var myHouse = new Dictionary<string, float> { ["Area"] = 1200, ["Bedrooms"] = 3, ["Age"] = 10 }; // Give the model one small "box of numbers" per input (shape [1,1]). // Any input we don't have a value for (like the Price label) is set to 0. var inputs = session.InputMetadata.Keys.Select(name => { float value = myHouse.TryGetValue(name, out var v) ? v : 0f; var tensor = new DenseTensor<float>(new[] { value }, new[] { 1, 1 }); return NamedOnnxValue.CreateFromTensor(name, tensor); }).ToList(); using var results = session.Run(inputs); float predicted = results.First(r => r.Name == "Score").AsEnumerable<float>().First(); Console.WriteLine($"Predicted price: {predicted:F1} lakhs"); // ---------- The shape of our data ---------- public class HouseData { public float Area { get; set; } public float Bedrooms { get; set; } public float Age { get; set; } public float Price { get; set; } }

Step 3: Run it

dotnet run

You'll see the model's inputs listed, followed by a predicted price. For this house, the true formula gives 1200 × 0.05 + 3 × 5 − 10 × 0.3 = 72 lakhs. The model's answer will be close to that but not exact, because it learns from examples instead of knowing the formula. That is the whole idea of machine learning.

Understanding the code (in plain words)

Part 3 is the in-process ONNX Runtime part. Parts 1 and 2 only exist to give you a model file to play with. In a real project you usually train once, keep the .onnx file, and ship only Part 3.

A few words worth knowing:

• InferenceSession is the loaded model. Think of it as opening the recipe card and getting the cook ready.
• DenseTensor is a fancy word for "a box of numbers." The model only understands numbers in boxes of a specific shape. Ours is [1, 1], meaning one value.
• NamedOnnxValue is a label on the box. The label (like "Area") must match the name the model expects.
• session.Run() is the moment the prediction happens.

Wait... where is the AI here?

Good question! Look at the prediction code. There is no pricing formula in it, no if area > 1000 rule, nothing like that.

Instead, during training the model looked at 300 example houses and figured out the pattern by itself: bigger area means higher price, more bedrooms means higher price, older house means lower price. That learned pattern is saved inside house_price.onnx.

When you give it a house it has never seen, it uses what it learned to make a good guess. That is the AI part: learning from examples instead of following rules a programmer wrote.

Use it properly in ASP.NET Core

The most common beginner mistake is loading the model on every request. Loading is slow. Running is fast.

Load once and reuse:

// Program.cs builder.Services.AddSingleton(new InferenceSession("house_price.onnx")); // Controller [ApiController] [Route("api/]controller[")] public class PriceController : ControllerBase { private readonly InferenceSession _session; public PriceController(InferenceSession session) => _session = session; [HttpGet] public IActionResult Get(float area, float bedrooms, float age) { var values = new Dictionary<string, float> { ["Area"] = area, ["Bedrooms"] = bedrooms, ["Age"] = age }; var inputs = _session.InputMetadata.Keys.Select(name => NamedOnnxValue.CreateFromTensor( name, new DenseTensor<float>(new[] { values.GetValueOrDefault(name) }, new[] { 1, 1 }) )).ToList(); using var results = _session.Run(inputs); var price = results.First(r => r.Name == "Score").AsEnumerable<float>().First(); return Ok(new { price }); } }

InferenceSession.Run is thread-safe, so one shared session can serve many requests at once.

Don't want to train anything? Use a ready-made model

You can skip training completely and download a pre-built .onnx model. Good places to look:

• ONNX Model Zoo, for example the MNIST handwritten-digit model
• Hugging Face, for example all-MiniLM-L6-v2 for text embeddings

Then your whole project is just dotnet add package Microsoft.ML.OnnxRuntime plus the InferenceSession code from Part 3.

Why developers love this approach

1. Fast. No network hop, so predictions take milliseconds.

2. Simple. One app, one deployment, no separate AI service.

3. Private. Your data never leaves your process.

4. Cheap. No per-call API fees and no extra servers.

Where would you use this in the real world?

• Spam or sentiment detection on user messages
• Recognizing objects in photos uploaded to your site
• Fraud scoring on every payment, instantly
• Text embeddings for search, running locally
• Small language models inside desktop or edge apps
• Support ticket classification inside an existing .NET admin portal

When should you NOT use in-process?

Be honest about the trade-offs:

• Your model is huge and needs a big GPU server, so a dedicated inference service makes more sense.
• Many different apps need to share one model, so one central service is easier to manage.
• Your data science team updates models daily and wants independent deployments.

For a lot of everyday apps, though, in-process is the simplest thing that works.

Wrapping up

Adding AI to a .NET app doesn't have to mean building a complicated backend. With ONNX Runtime you can:

1. Train a model (even in C# with ML.NET) or download one

2. Save it as a .onnx file

3. Load it with InferenceSession

4. Call Run() like any other C# method

Your AI becomes just another part of your application.

Happy coding!