Day 21 - Continuity Camera and Vision

Yesterday we saw how we can scan a wish list and use text recognition for Windows. Today we would like to look at a way of scanning a document and recognizing the text for Apple users. To do this, we first use the Continuity Camera functions in combination with the iPhone for scanning and then recognize the text with Vision.

The ContinuityCamera functions allow you to capture an image or scan a page by controlling your iPhone from your Mac and having it perform the scan or image capture. The requirement for this is that the devices are logged into iCloud with the same Apple ID, are on the same network, Bluetooth is enabled, the mobile device is running iOS 12 (or newer) and the computer is a Mac with macOS Mojave (or newer).

So let's start with our script, which will be quite small today despite its great functionality. The ContinuityCamera.Initialize function initializes Continuity Camera and makes it possible to use the following functions.

When we have finished the scan or image capture on the iOS device, a script is called in which we can continue working with the result provided by the device. We now want to set this script with ContinuityCamera.SetScriptTrigger. To do this, we first specify the file name of the database in which we find the script. Since we are already in the database, we use Get ( FileName ) here. Then comes the name of the script, in our case ContinuityCamera. Now that everything is prepared, we can call ContinuityCamera.Import, which starts the acquisition. In the parameters of this function we can then decide whether we want to scan a document with the mobile device, take a picture or draw a sketch. Here are the numerical values for this:

We select 1 for scanning a document.

Set Variable [ $r ; Value: MBS( "ContinuityCamera.Initialize" ) ]
Set Variable [ $r ; Value: MBS("ContinuityCamera.SetScriptTrigger"; Get ( FileName ); "ContinuityCamera") ]
Set Variable [ $r ; Value: MBS( "ContinuityCamera.Import"; 1 ) ]

Once this script has been started, we can scan our document with the cell phone. If we press Save, the scanned document is transferred to the computer and the ContinuityCamera script that we have just set with ContinuityCamera.SetScriptTrigger is called. Here we get the scanned document with ContinuityCamera.Result and can then process it further. Next, we want to recognize the text on this document using Vision from Apple. With Vision, we can recognize barcodes, assign images to a specific group or, as already mentioned, recognize the text. For text recognition, we use Vision.RecognizeText and pass the document to this function. We then receive the text that we can write in the provided field as a return.

Set Variable [ $Image ; Value: MBS("ContinuityCamera.Result") ]
Set Field [ Giftee::WishlistIn ; $Image ]
Set Variable [ $Text ; Value: MBS( "Vision.RecognizeText"; $Image ) ]
Set Field [ Giftee::Presents ; $Text ]

That's it for our door and I will see you again tomorrow for door number 22.