AVCaptureVideoDataOutput improvements
We have the AVCaptureVideoDataOutputMBS Class in MBS Xojo AVFoundation Plugin. This class allows you to get live data from the camera as CMSampleBufferMBS objects. Now this was a bit slow as we did all the processing on the main thread. But for doing something useful in 30fps, you need time on the main thread for your Xojo app and we need to move as much work as possible of the heavy work away from the main thread.So for next pre-release we cache properties in CVImageBufferMBS class. If you query NSImageMBS, JPEGData or CIImageMBS there, we will create them for you and remember the values, so in case you call the methods again, we have the values directly available and don't recreate them.
As conversion of the camera data (e.g. in YUV format) to RGB may be expensive, we want to do the conversion on the preemptive threads. So we added PrepareCIImage, PrepareJPEGData and PrepareNSImage properties to AVCaptureVideoDataOutputMBS class. If you set them, we will do the part of the work you requested on the preemptive thread and than pre-fill CMSampleBufferMBS and CVImageBufferMBS with the data, so query calls on the properties will be very quick.
This does not reduce the CPU time needed for the whole app, but moves more work to other threads, so you have more time on the main thread for your application doing something with the video.
If you plan to display video, please do not draw it yourself to a canvas, but use the AVCaptureVideoPreviewLayerMBS class or the AVCaptureViewControlMBS control (the control is 64-bit only).
If you know other places, where we could do processing on background thread and call event later on main thread with results, please let us know.