QR Codes and Manifest

Overview

QR codes are a critical part of Manifest software as they are used as the central reference point for 3D instructions. There are a few things to remember and note about them so that we can best prepare ourselves and our customers for POC’s and larger scale rollouts.

General platform considerations

HoloLens2

This equipment has built-in software for recognizing QR codes in a very accurate way. Using HoloLens with QR codes typically allows for high performance tracking and accuracy in certain environments. Users can expect sub-inch accuracy for AR Notes within 20 feet or so of the QR code. After that, holograms tend to “drift” and require other methods to ensure accuracy.

If you notice that your holograms are drifting after a certain distance, you can employ “Alignment tags” to have your users rescan and ensure accuracy.

Best Practice from Microsoft

Quiet zone around QR codes

To be read correctly, QR codes require a margin around all sides of the code. This margin must not contain any printed content and should be four modules (a single black square in the code) wide.

The QR spec contains more information about quiet zones.

Lighting and backdrop

QR code detection quality is susceptible to varying illumination and backdrop.

Under normal lighting conditions, provide enough contrast for the black/white modules for better performance.
In extreme lighting conditions with bright lighting or dark backdrop, you can try to reduce and adjust contrast which could improve QR code detection rate. The white background within the QR code can be reduced from 255 downwards.

Size of QR codes

Windows Mixed Reality devices don’t work with QR codes with sides smaller than 5 cm. For QR codes with sides ranging from 5 cm to 10 cm, the device must be fairly close to detect the code. It may take longer to detect the code.

The exact time to detect codes depends not only on the size of the QR codes, but how far away you are from the code. Moving closer to the code will help offset issues with size.

Distance and angular position from the QR code

The tracking cameras can only detect a certain level of detail. For small codes (less than 10 cm along the sides) you must be fairly close. For a version 1 QR code varying from 10 cm to 25 cm in size, the minimum detection distance is between 0.15 meters and 0.5 meters. The detection distance for size increases linearly, but also depends on supported QR version or module size. The higher the version, the smaller the modules, which can only be detected from a closer position. You can also try micro-QR codes if you want the distance of detection to be longer. QR detection works with a range of angles += 45 deg to ensure we have proper resolution to detect the code.

Other detection considerations:

QR codes on curved surfaces aren’t supported.
In-plane orientation is supported. Out of plane should be <= +-45 looking straight on to have better detection.
The physical size of the QR code should have modules of at least 2/3 pixels. Note: Higher versions of QR codes will have smaller modules.

For the tradeoff relationship between distance and size of the QR code for optimal detection, see the following graph:

^{Figure 1: scanning speed versus QR code size}

iPad Pro

iPads and iOS in general do not have built in QR code scanners apart from the one built into the camera that is used for linking to URL’s. In Manifest, Taqtile uses ARKit in two different modes to scan QR codes with accuracy:

Image Recognition (current Manifest method)

The first method is using image recognition. This is a fast and fairly accurate way of recognizing and positioning AR content. There are some downsides, however. One is that slight differences in alignment can be detected when scanning an asset multiple times.

Plane finding + Image recognition (experimental option in iPad for Manifest 3.0)

The next (experimental) method uses the image recognition in conjunction with ARKit plane finding to attempt to get a more accurate and consistent scan. This works very well if the QR code is mounted on a large flat surface like a wall or side of a machine. If the area is small and has different depths around it, you may see erratic behavior. Some handling has been added to this option so that the best and most consistent experience is had when the user rocks back and forth a bit when scanning the QR Code. This will average some of the angles and generally produce more consistent and accurate results.

^{Figure 2: Inconsistent scanning}

iPad (non-LIDAR)

IPad models that do not have LIDAR (generally non-Pro designated models) can also use AR functionality. It should be understood however that the accuracy and speed of ARKit on these devices is generally less dependable than with a LIDAR equipped iPad model.

Printing QR codes

List differences in versions: iPad 3.0 pays attention to the size of the QR code.

56mm -> DEFAULT (unchangeable after 3.0)
46mm -> AFTER 3.0 Print at 82% Scale

How to determine scale if you have existing QR codes at a different size than 56mm:

In this example, if your existing QR code is 43mm and Manifest is generating PDF’s at 56mm, the calculation for scaling would be:

43/56 = .7679

Existing QR Size/Default printing size = Scaling

In the example, you should print your QR codes at 77% scale in order to maintain the 43mm code size.

^{Figure 3: “Print QR” in macOS printing options dialog “Print QR” in MacOS Printing options in macOS}

^{Figure 4: “Download PDF” MS Edge on MacOS}

^{Figure 5: Print dialog on Windows 11}

^{Figure 6: Change default setting to “100%” scale}

Considerations for Production use

How do I label and print for many assets?

In order for AR instructions to align, the QR code should be placed in the exact same location for every Asset. There are many ways to mount QR Codes:

Create a QR Code mounting template in Manifest for the AssetClass. This can be used with photo references and measurements to show the exact location to mount.

^{Figure 7: Template list for QR code placement}

^{Figure 8: Creating Job for QR Code setup}

QRCodeplacementtemplateinsidemanifest - Community

^{Figure 9: QR Code placement template inside Manifest}

Create the SOP for QR Code placement inside existing equipment setup / breakdown plans for your work area. Exact placement instructions can be added to other existing setup procedures so that existing and new equipment are setup in the same way.
If you are using only HoloLens and 3D platforms, you can use Touch Alignment. For more on Touch Alignment, view here.

How do I mount the QR Code in the proper location on my equipment?

There are many considerations when thinking about where to place a QR Code on an Asset. Some are obvious, some less so.

Don’t put the QR code in a hard to reach location.
Remember what types of procedures are most used on the equipment and try to put the code in a centrally located area.
Locating a flat surface with several inches of whitespace around it will improve your results.
Height of the code has some implications for accuracy and ease of use. The average height of a male worldwide is 5’6” and a woman is 5’2”. In the US the numbers are higher. It is good to mount the code at a height where the person scanning the code is comfortable and can look straight ahead to scan the code.

What do I do if I have to move the QR code location on an asset?

This is a destructive maneuver in that Manifest does not support moving all of the AR content with the QR code. If you do this, you will lose the spatial authoring and will require a user with Author role to re author the spatial notes and steps and leaderlines in your instructions. For this reason it’s important that you make the right choices about placement from the beginning.

What do I do about drift?

Drift is a common issue with AR experiences. Drift occurs when the initial scan point moves significantly from it’s original location after it has been scanned. Much of your success in regards to issues with drift have to do with the actual hardware solutions themselves. The HoloLens 2 does a great job of keeping content where you would expect it to be. Full headsets tend to be better at controlling drift as they are scanning full environments.

To deal with drift where alignment is critical, you can use Alignment tags in Manifest. Every Asset has 2 options for tags. One is the main marker and the other is an alignment tag. You can attach one alignment tag per step in a template. If steps occur a significant distance from the original QR code or you are experiencing consistent drift, consider adding alignment tags to your equipment at critical locations.

Does the lighting condition matter?

In a word “yes”. Some devices like the HoloLens2 use an IR scanner for identifying and scanning QR codes. IR is very sensitive to light conditions. In the case of HoloLens, the same cameras are used for hand tracking, so lighting can also affect hand functions like Touch Alignment and Hand Notes.

Low light levels can also be a problem. They can be worked around by using a flashlight for scanning to QR code, but quite often in very low light a headset or even an iPad will lose tracking. In the opposite extreme, too bright light levels like direct sunlight can create problems with scanning.

Glare on the tag itself will make problems with lighting worse. Be careful not to laminate or print your tags on a material that is glossy.

Does the angle of the scan matter?

When possible, you should try to be directly in front of the QR code, looking straight ahead and without a tilt. That will always produce the best scanning results.