« Two months until the … | Home | MBS Xojo Plugins, ver… »

Improved threading for Xojo

While Xojo has some threading support, it could do better and today we think what would be needed for this.

Allow callbacks on other threads

Several modern APIs on all platforms may require you to provide a callback function, so you may just have a little global method called with a parameter. You remember the parameter and regularly check with a timer if the value was set. e.g. a code like this:

Sub MyCallback(granted as boolean) GlobalModule.isGranted = granted GlobalModule.gotResult = True End Sub

This is the most simple way to do a callback, which may receive on a non-Xojo thread from the OS or a library. The code generated by the compiler looks like this translated back into Xojo code:

Sub MyCallback2(granted as boolean) StackCheck Var currentException = GetCurrentException If currentException = Nil Then GlobalModule.isGranted = granted GlobalModule.gotResult = True End If If CurrentException <> Nil Then ReraiseException(CurrentException) UnlockObject(CurrentException) End If End Sub

The stack check causes problems since Xojo doesn't know the stack size for foreign threads and thus likes to raise the exception always. We add a line with "#Pragma StackOverflowChecking false" to disable stack checking in the callback function:

Sub MyCallback(granted as boolean) #Pragma StackOverflowChecking False GlobalModule.isGranted = granted GlobalModule.gotResult = True End Sub

And as we see in assembly, we get the stack check removed:

Sub MyCallback3(granted as boolean) GlobalModule.isGranted = granted GlobalModule.gotResult = True Var currentException = Nil If CurrentException <> Nil Then ReraiseException(CurrentException) UnlockObject(CurrentException) End If End Sub

Yes, assembly shows with standard optimization, that it sets a variable to nil and then checks whether it is nil. But with aggressive or moderate optimization, that is gone and you have:

Sub MyCallback(granted as boolean) GlobalModule.isGranted = granted GlobalModule.gotResult = True End Sub

This is perfectly thread safe already today, except in the debugger. Even if the nil check is not optimized away, the code is thread safe as ReraiseException and UnlockObject are never called. But still the debugger callbacks may cause problems, so don't try in debugger if possible. Or better have main thread do nothing while this runs to avoid crashes in the debugger. The method looks with debugger hooks like this:

Sub MyCallback(granted as boolean) RuntimeDebuggerRegisterFrame DebuggerHook GlobalModule.isGranted = granted DebuggerHook GlobalModule.gotResult = True DebuggerHook End Sub

The delegate itself for a global method has no overhead. The pointer you pass to the declare for the delegate is the pointer of your global function, so no problems there.

Number crunching

You can do some number crunching in your delegate. e.g. if you get it to run via declares on a preemptive thread. The first change we need is to add a BackgroundTasks pragma to avoid the IDE calling to background tasks.

Sub MyWorker(p as ptr, size as integer) #Pragma StackOverflowChecking False #Pragma BackgroundTasks false Dim u As Integer = size-1 For i As Integer = 0 To u p.Int8(i) = p.Int8(i) + 1 Next End Sub

If you only use pointers (not MemoryBlock objects), no strings, no objects and only simple functions like math functions, you may enjoy a nice performance increase right in Xojo. Like you can take one of the Xojo implementations for encryption and put it in here.

Do object locking thread safe

The next problem is object locking, nil object checking and array handling. Some of the functions there are thread safe without Xojo Inc. changing anything. But others need to be adapted. Let's take something like object locking. This function in the runtime increases the reference count of an object by one. While all modern CPUs have commands to increase a value by one and return the value in one atomic instruction, Xojo seems not to use that currently. See e.g. InterlockedIncrement function. Here is a snippet from our plugin code, where we use InterlockedDecrement to decrement the reference count. Our Release() function is thread-safe and it calls delete to run destructor and free memory in C++:

STDMETHODIMP_(ULONGMyClass::Release() 
{ 
    DebugMessage(__FUNCTION__);
    LONG NewRefCount = InterlockedDecrement(&RefCount); 
    if (NewRefCount == 0) { delete this; } 
    return NewRefCount; 
}

In Xojo the unlock object function is tricky as it not just has to decrease reference counter, but also run destructors. And these destructor (chains) may contain whatever code is there and may not be thread safe themselves.

If I'd have the job to code this, I would move destructor calls to main thread: When unlock is called in a preemptive thread it would store the to-be-released objects into a global array. Then on the app level there is a timer to check this array once a second for new entries and does the destructor call there. Of course this array needs to be locked with a mutex, but the function itself should be able to use atomic decrement and just to the extra destructor work if the value turns out zero. On the main thread, it would update the object list to remove the object and free it's memory. Alternative version could be to simply not call the destructor if the object is freed on the thread and just leak it. Or to run it and require the developer to make sure it is thread safe. For this you define for the developers, that code in preemptive threads must make sure you keep objects around and avoid destructors to run.

Same may be done for strings, but there it may be easier. Since there is no destructor, the runtime just needs to free the memory itself for the string structure and the data it points to.

We could and should make a request with Xojo Inc. to get things like 10 to 20 runtime functions around object and string creation and reference counting to be thread safe (see e.g. #71681). And to keep performance, I mean not to just add mutex everyone, but to prefer to use atomic operations. And then we could use objects and strings in preemptive threaded code.

This should include if possible debugger hook calls. Or alternatively a #pragma for thread safety. This #pragma could disable background tasks, stack checking, debugger calls and other things as needed and prevent you from calling methods not allowed in threads, e.g. which would have an attribute to either clearly state them as thread safe or unsafe to make a decision at compiling. And it may be good to run an automated process to find the methods checking for UI thread access and add an attribute to them to have compiler disallow their use in thread class.

Once the basic locking functions would be supported for code, you could use Xojo objects to store your pointers and numbers. And query pointer from memory block. You may still not be able to call most framework functions

Allow web framework thread safe

Once you would have an option to have a preemptive-thread class in Xojo to work, you would want to do more there. Like processing JSON, doing database requests or doing picture effects. And you may want to use it for the web framework to speed it up a lot. But that would require to walk through all the code and look for access to globals and wrap that in mutex.

But if that work could be done, the Xojo web app could run with maybe one main thread and 3 preemptive threads to do the processing of requests, which could be parallelized a lot in parts like JSON processing.

Thanks for reading. Let us know if you are interested in more threading code.

The biggest plugin in space...
27 02 23 - 13:30