Vision
On-device image understanding
Works on the image bytes you pass in, so it needs no permission and no network. Images come from Image or from a share extension input. Rectangles are normalised 0 to 1 with the origin at the top left, not Vision's bottom left.
Methods
Vision.text(image, options)[Any]Every line of text found, as { text, confidence, x, y, width, height }.
- image Any required
From Image.file, Image.download, Canvas.render, or a share extension input.
- options Any required
- languages [String]
BCP-47 codes to prefer, like ["it-IT", "en-US"]. Left out, Vision decides.
- fast Boolean
Trades accuracy for speed. Worth it in a widget, where the budget is short.
- languages [String]
returns One entry per line, in the order Vision found them, not top to bottom.
Vision.barcodes(image)[Any]Barcodes and QR codes, as { payload, symbology, x, y, width, height }.
- image Any required
The image to scan.
returns symbology is the bare name, "QR" or "EAN13", not Vision's prefixed constant.
Vision.faces(image)[Any]Face rectangles, as { x, y, width, height, roll, yaw }.
- image Any required
The image to scan.
returns roll and yaw are radians, and 0 when Vision could not tell.
Examples
Text out of a screenshot
var shot = Image.file("/path/to/shot.png");
var lines = Vision.text(shot);
Script.setWidget(new Widget({
child: Column({
children: lines.slice(0, 4).map(function (l) { return Text(l.text); }),
}),
}));