GuidesChangelog
Guides

Photo Observations

InSkill can extract data from a picture you take in a step. Instead of asking the user to enter information, they can take a picture of a nameplate or operator screen and automatically get information from that picture. To do this, create an Observation processor by selecting Photo:

This will create an empty processor. Give it a name that describes what it does.

Now click Upload Image for an example of this picture

Finally, click the button to add variables to extract from the picture.

The variables tell InSkill where to save the data, and how to find data. This example will process a pump nameplate and put data into variables pump.model, pump.v, pump.hz, etc. The processor finds this data by looking for the target text such as "PUMP" and reading the text to the right of it. You can match some text and get the information in any direction, up, down, right or left.

The target is the text to look for. It needs to be exactly the same as in the picture, so "PUMP" will match where "Pump" will not. In this picture, "V~" may have a space or not, its hard to tell. Use ^ character to match if there's a space or not, so "V^~" will match if there's a space after the V or not.

The wildcard * will match any single character so if the target is "MX*" it will match MX followed by any character (MX1, MX8, MXp). This is different than the ^ which matches 0 or 1 spaces.

To use a photo processor, in an Observation step, if you choose Photo, the step will prompt the user to take a picture with the camera or to upload a picture. On that Observation you can also choose from existing Photo processors to run when the user's picture is uploaded.

Lots of nameplates or user screens have data with a title and the data, so the directions above make it easy to capture the information. But sometimes there is information with no title or target to find first. In these cases, there is an option to use a Regex. Regex is a text searching standard in programming that is very powerful but also confusing and difficult.

Some Regex resources are:

In the example above, the target KW actually finds "0.37 HP 0.5 ln" because that's to the right of it. So it uses a filter Regex to take the 0.37 part. The Regex is "/[.0-9]+ /" which must begin and end with the / forward slash. Inside the [] brackets it says match the . period and numbers from 0-9. Just that would match one of those characters, so the + means as many characters that match. This Regex matches any numbers including the period, but not including the space and the HP etc. You can use the filter Regex to remove any unwanted text

Updated about 1 year ago