C#

[Googlikhail]

Search before asking

• I have searched the HUB issues and discussions and found no similar questions.

Question

Hello, can you tell me how to properly pre-process a photo in C#? I have a webcam detection project and I use the NuGet packages Emgu.CV and Microsoft.ML.Transforms.Onnx. I made a code to test it on the image, but I saw that the model made the wrong output and built something like a “ladder” of detection squares, just like in the output version for the webcam project. There is very little information regarding new versions of yolo and C# and I hope for your help. After trying different options for correcting the error, I came to the conclusion that the model is receiving incorrectly preprocessed images - that’s why I’m asking a question regarding preprocessing.

`using System;
using System.Drawing;
using System.IO;
using Emgu.CV;
using Emgu.CV.CvEnum;
using Emgu.CV.Structure;
using Microsoft.ML;
using Microsoft.ML.Data;
using Microsoft.ML.Transforms.Onnx;

public class Program
{
static void Main(string[] args)
{
// Creating an ML.NET context
var mlContext = new MLContext();

    // Loading the YOLOv8n model from a file
    string modelPath = @"C:\ModelYolo\yolov8n.onnx";

    // Creating an image processing pipeline and using the ONNX model
    var pipeline = mlContext.Transforms.LoadImages(outputColumnName: "Image", imageFolder: null, inputColumnName: nameof(ImageData.ImagePath))
.Append(mlContext.Transforms.ResizeImages(outputColumnName: "images", imageWidth: 640, imageHeight: 480, inputColumnName: "Image"))
.Append(mlContext.Transforms.ExtractPixels(outputColumnName: "images", inputColumnName: "images"))
.Append(mlContext.Transforms.ApplyOnnxModel(modelFile: modelPath, outputColumnName: "output0", inputColumnName: "images"));

    // Training a model on empty data
    var model = pipeline.Fit(mlContext.Data.LoadFromEnumerable(new List<ImageData>()));

    // Creating a forecasting engine
    var predictionEngine = mlContext.Model.CreatePredictionEngine<ImageData, OnnxPrediction>(model);

    // Uploading an image
    var bitmap = new Bitmap(@"C:\ModelYolo\aHR0cDovL3d3dy5saXZlc2N.jpg");
    var imageBytes = ImageToByte(bitmap);

    // Converting Bitmap to Mat
    Mat frame = bitmap.ToImage<Bgr, byte>().Mat;

    // Making a forecast
    var prediction = predictionEngine.Predict(new ImageData { Image = imageBytes });

    // Processing forecast results
    for (int i = 0; i < prediction.PredictedLabels.Length / 84; i++)
    {
        var pred = prediction.PredictedLabels.Skip(i * 84).Take(84).ToArray();

        // Getting the coordinates of the rectangle
        var x1 = pred[0];
        var y1 = pred[1];
        var x2 = pred[2];
        var y2 = pred[3];

        // Draw a rectangle on the image
        CvInvoke.Rectangle(frame, new Rectangle((int)x1, (int)y1, (int)(x2 - x1), (int)(y2 - y1)), new MCvScalar(0, 0, 255), 2);
    }

    // Save the image

    frame.Save(@"C:\ModelYolo\cats_detected.jpg");
}

public static byte[] ImageToByte(Bitmap img)
{
    using (var stream = new MemoryStream())
    {
        img.Save(stream, System.Drawing.Imaging.ImageFormat.Png);
        return stream.ToArray();
    }
}

}

public class ImageData
{
public byte[] Image { get; set; }
public string ImagePath { get; set; }
}

public class OnnxPrediction
{
[ColumnName("output0")]
public float[] PredictedLabels { get; set; }
}`

Additional

No response

[github-actions]

👋 Hello @Googlikhail, thank you for raising an issue about Ultralytics HUB 🚀! Please visit our HUB Docs to learn more:

• Quickstart. Start training and deploying YOLO models with HUB in seconds.
• Datasets: Preparing and Uploading. Learn how to prepare and upload your datasets to HUB in YOLO format.
• Projects: Creating and Managing. Group your models into projects for improved organization.
• Models: Training and Exporting. Train YOLOv5 and YOLOv8 models on your custom datasets and export them to various formats for deployment.
• Integrations. Explore different integration options for your trained models, such as TensorFlow, ONNX, OpenVINO, CoreML, and PaddlePaddle.
• Ultralytics HUB App. Learn about the Ultralytics App for iOS and Android, which allows you to run models directly on your mobile device.
  • iOS. Learn about YOLO CoreML models accelerated on Apple's Neural Engine on iPhones and iPads.
  • Android. Explore TFLite acceleration on mobile devices.
• Inference API. Understand how to use the Inference API for running your trained models in the cloud to generate predictions.

If this is a 🐛 Bug Report, please provide screenshots and steps to reproduce your problem to help us get started working on a fix.

If this is a ❓ Question, please provide as much information as possible, including dataset, model, environment details etc. so that we might provide the most helpful response.

We try to respond to all issues as promptly as possible. Thank you for your patience!

[pderrenger]

@Googlikhail hello! 👋

It looks like your question revolves around the correct way to pre-process an image in C# for use with a YOLO model. From your code snippet and description of the issue, it seems like the image scaling and formatting might not be correctly matched with what the model expects. Here are a few things to consider:

1. Image Size and Aspect Ratio: Ensure that the image dimensions you're resizing to match the input size expected by the YOLO model without altering the aspect ratio in a way that distorts the image.

2. Normalization: YOLO models typically expect pixel values to be normalized. If your model expects pixel values in the range [0, 1] or [-1, 1], you'll need to apply this normalization after converting the image to pixels but before passing it to the model.

3. Color Ordering: Ensure the color channels of your input images match what the model was trained with. YOLO models usually work with RGB images, but sometimes image loading libraries default to BGR.

4. Debugging Pre-Processing: To debug if preprocessing is the issue, you might compare the input and output of your preprocessing steps with those from a known good implementation (e.g., in Python) on the same image to spot differences.

Since your case involves C#, aligning your preprocessing steps with these considerations might require specific adjustments based on the libraries (Emgu.CV and Microsoft.ML.Transforms.Onnx) you're using.

Consider revisiting step 3, related to the extraction and normalization of pixel values, ensuring that any necessary normalization is applied correctly according to your model's training.

Unfortunately, without diving deeper into the specifics of the ONNX model you are using and the exact preprocessing steps it was trained with, these suggestions are somewhat general. For more detailed guidance on preprocessing requirements for YOLO models and other specifics, the Ultralytics HUB Docs might offer additional insights: https://docs.ultralytics.com/hub.

Hope this helps! If you have more questions or need further clarification, feel free to ask.

[github-actions]

👋 Hello there! We wanted to give you a friendly reminder that this issue has not had any recent activity and may be closed soon, but don't worry - you can always reopen it if needed. If you still have any questions or concerns, please feel free to let us know how we can help.

For additional resources and information, please see the links below:

• Docs: https://docs.ultralytics.com
• HUB: https://hub.ultralytics.com
• Community: https://community.ultralytics.com

Feel free to inform us of any other issues you discover or feature requests that come to mind in the future. Pull Requests (PRs) are also always welcomed!

Thank you for your contributions to YOLO 🚀 and Vision AI ⭐