Fastest way to Draw RGB Bitmap

[maptz]

Hiya,

I wonder if someone can help me. I'm trying to do a little proof of concept.

I have an incoming video stream made up of RGB24 bitmaps. I'm receiving uncompressed byte arrays of known width and height.

I am trying to write a C# app that shows these frames as they come in with as little delay as possible.

I as looking at Silk.Net Direct3D and OpenGL as a way of displaying the images, but realised I was going to have to dive in deep. in 3d vertices etc, and felt like this was overkill.

Can anyone familiar with Silk.Net point me in the direction of the best and fastest way I can display and refresh 2D pixel arrays on a window as quickly as possible?

[Beyley]

fastest way is probably going to be GL or D3D11, just update a texture when you get the data, and draw a quad on screen with said texture

[maptz]

Thanks @Beyley . That's really helpful.

If anyone else is interested in how to do this with OpenGL relatively simply, I'd suggest following the OpenGL tutorials in the Silk.Net Github repo.

I was able to show an image pretty quickly by following the code here.

[maptz]

Hey @Beyley or anyone else,

I kind of got this working in OpenGL, and have come to a bottleneck. I'm certain it's because of how I'm setting up the display in OpenGL.

I wonder if someone can guide me through the next step. Essentially, I've got a simple app which starts off by pulling 50 bitmap images from somewhere and stores the bytes into an array. All this happens before my OpenGL window is started (in the call to SetupFrames).

Then I start the OpenGL window using Silk.Net and cycle through each image by using the SetTexture method every time the Render event is called.

I'm seeing a bit of stuttering which I'm sure shouldn't be happening for such a simple scenario.

I feel like switching out the actual Texture object is not the right thing to be doing (Can I write the next frame to a secondary buffer or something)?

 /* #region Private Static Fields */
    private static long _lastRender = 0;
    /* #endregion Private Static Fields */
    /* #region Public Static Fields */
    public static List<double> _lastFramesMs = new List<double>();
    /* #endregion Public Static Fields */
    /* #region Private Fields */
    private int _frameCount = 50;
    private int _frameIndex = 0;
    private FrameImage? [] _frames;
    private bool _hasLoaded;
    private BufferObject<uint> Ebo;
    private GL Gl;
    private readonly uint[] Indices = {0, 1, 3, 1, 2, 3};
    private Shader Shader;
    //Create a texture object.
    private Texture Texture;
    private VertexArrayObject<float, uint> Vao;
    private BufferObject<float> Vbo;
    // OpenGL has image origin in the bottom-left corner.
    private readonly float[] Vertices = {//X    Y      Z     U   V
    1.0f, 1.0f, 0.0f, 1f, 0f, 1.0f, -1.0f, 0.0f, 1f, 1f, -1.0f, -1.0f, 0.0f, 0f, 1f, -1.0f, 1.0f, 1.0f, 0f, 0f};
    private IWindow window;
    /* #endregion Private Fields */
/* #region Private Methods */
 private void KeyDown(IKeyboard arg1, Key arg2, int arg3)
    {
        if (arg2 == Key.Escape)
        {
            window.Close();
        }
    }

    private void OnClose()
    {
        Vbo.Dispose();
        Ebo.Dispose();
        Vao.Dispose();
        Shader.Dispose();
        //Remember to dispose the texture.
        Texture.Dispose();
    }

    private unsafe void OnLoad()
    {
        IInputContext input = window.CreateInput();
        for (int i = 0; i < input.Keyboards.Count; i++)
        {
            input.Keyboards[i].KeyDown += KeyDown;
        }

        Gl = GL.GetApi(window);
        Ebo = new BufferObject<uint>(Gl, Indices, BufferTargetARB.ElementArrayBuffer);
        Vbo = new BufferObject<float>(Gl, Vertices, BufferTargetARB.ArrayBuffer);
        Vao = new VertexArrayObject<float, uint>(Gl, Vbo, Ebo);
        Vao.VertexAttributePointer(0, 3, VertexAttribPointerType.Float, 5, 0);
        Vao.VertexAttributePointer(1, 2, VertexAttribPointerType.Float, 5, 3);
        Shader = new Shader(Gl, "shader.vert", "shader.frag");
        //Loading a texture.
        SetTexture();
        _hasLoaded = true;
    }

    private unsafe void OnRender(double obj)
    {
        var now = DateTime.Now.Ticks;
        var ticks = now - _lastRender;
        var ticksPerSeconds = 10_000_000;
        _lastRender = now;
        var msSinceLast = 1000.0 * ticks / (double)ticksPerSeconds;
        _lastFramesMs.Add(msSinceLast);
        if (_lastFramesMs.Count == 50)
        {
            var avgMSPerFrame = _lastFramesMs.Average();
            Console.WriteLine("Avg frame gap: " + msSinceLast);
            var fps = 1.0 / (avgMSPerFrame / 1000.0);
            Console.WriteLine("Avg fps: " + fps);
            _lastFramesMs.Clear();
        }

        SetTexture();
        Gl.Clear((uint)ClearBufferMask.ColorBufferBit);
        Vao.Bind();
        Shader.Use();
        //Bind a texture and and set the uTexture0 to use texture0.
        Texture.Bind(TextureUnit.Texture0);
        Shader.SetUniform("uTexture0", 0);
        Gl.DrawElements(PrimitiveType.Triangles, (uint)Indices.Length, DrawElementsType.UnsignedInt, null);
    }

    private void SetTexture()
    {
        //Console.WriteLine($"Showing frame {_frameIndex+1}");
        var frame = _frames[_frameIndex];
        var span = new Span<byte>(frame.Bytes);
        var newTexture = new Texture(Gl, span, 1920, 1080);
        var oldTexture = Texture;
        Texture = newTexture;
        _frameIndex = _frameIndex < _frameCount - 1 ? _frameIndex + 1 : 0;
        if (oldTexture != null)
            oldTexture.Dispose();
    }

    private FrameImage? [] SetupFrames()
    {
...
}


    /* #endregion Private Methods */
    /* #region Public Methods */
    public void Run()
    {
        _frames = SetupFrames();
        GC.Collect();
        var options = WindowOptions.Default;
        options.Size = new Vector2D<int>(1920, 1080);
        options.Title = "OpenGL";
        window = Window.Create(options);
        window.Load += OnLoad;
        window.Render += OnRender;
        window.Closing += OnClose;
        window.Run();
    }

[Beyley]

you likely want to be just re-setting the data on the same texture using glTexImage2D iirc, don't create a whole new texture object

[maptz]

Amazing @Beyley. That was precisely it!

Rather than creating a new Texture on every Render cycle, I just call the following method on the Texture class with the new bytes. I'm getting 60 fps playback rather than the stuttery < 30 fps playback I had before. Solved!

public unsafe void Update(Span<byte> data, uint width, uint height)
	{
		fixed (void* d = &data[0])
		{
			//Setting the data of a texture.
			_gl.TexImage2D(TextureTarget.Texture2D, 0, (int)InternalFormat.Rgb32f, width, height, 0, PixelFormat.Bgra, PixelType.UnsignedByte, d);
			SetParameters();
		}
	}