CareAR App - Version 24.01

With release 24.01 of the CareAR App, we introduce the following:

• AR Measurement Tool
• Enhanced AR Hints
• AR Element support in an Experience
• Forms support in an Experience
• Change audio encoding to be wideband audio
• Enhanced low bandwidth mode support
• Improved automatic connectivity test
• Various client UX enhancements

New Features and Capabilities

AR MEASUREMENT TOOL

Several of our customers have found use cases where the ability to capture a linear measurement during an Assist session would be useful. One use case is the measurement of a wall’s height to help plan the correct ladder to bring to a site location.

With release 24.01, we introduce the AR Measurement Tool in the CareAR mobile app tool strip which allows a streaming Assist collaborator to:

• Activate measurement mode (which changes the streaming collaborator's view and presents an indicator on the top of the screen for all other participants)
• Create a simple measurement by setting start and ending points of the distance measured
• Connect an additional measurement to an existing measurement to create a compound measurement
• Set preferred units of measure (inches or centimeters)
• Take a snapshot of the measurements captured
• Copy the measurement value of a segment to the clipboard to make it available to paste to an application outside the CareAR app
• Undo the measurement (or series of measurements that comprise connected segments)

Once the streaming user exits measurement mode with the measurement intact, it will be converted to an anchored AR annotation and will be composited (by the streaming participant’s app) into the session to be viewable by all session participants. Exiting measurement mode will trigger removal of the measurement mode indicator on all other participants’ view.

Should the streaming user wish to make an additional measurement, they can simply re-enter measurement mode and use the measurement tool.

The AR measurement tool is supported in both collaborative mode (two or more users) and solo mode.

For all users in the Assist session who are NOT the streaming collaborator (namely, the host and any additional participants), the AR measurement tool will be greyed out in the Assist tool strip. If one of the non-streaming participants taps the tool button, a popup will be displayed to indicate that the tool is not available to non-streaming participants.

AR Hints

As a prerequisite to the AR measurement action, the streaming collaborator will need to collect sufficient cloud points to execute the measurement properly. This is where the enhanced AR hints will help guide the user to collect sufficient surface data. The measurement mode tool button will be greyed out and non-functional until sufficient 3D mapping data (cloud points) are collected.

Optimal Use Cases

The following table provides guidance of use cases that are recommended and not recommended when using the AR measurement tool.

Limitations and Restrictions

The AR measurement tool is provided to allow the user to take rough measurements using the CareAR mobile app. It is not designed to be a precision measurement instrument and uses the device’s available resources to calculate an approximate measurement.

The measurement mode will not apply to streams originating from the Assist browser app, the Assist smart glasses app, the drone app, or from a paused video feed.

ENHANCED AR HINTS

We updated our AR hints feature used in an Assist session to improve the feedback to the streaming user and to the host/other collaborators. With this update, we’ve made AR hints permanently enabled.

Streaming User's Experience

• The initial AR coaching hint that is overlaid and presented to the user has been updated. During initialization, the vertical Assist tool bar is hidden from the user.
• Once sufficient surface information is collected, the app displays the tool bar to the streaming user with all the AR annotation tools ready and available for use.
• The app simultaneously indicates to the streaming user that AR is ready to use and presents the small green AR indicator to the user in the upper left corner of the screen.
• If the AR status is amber, the app will display the reason for the amber state. The reasons may be one of the following:
  
  • lacking surface detection
  • insufficient light
  • excessive motion
  • too close

Note that if the user has achieved "green status" once and then transitions to "amber status," we will not change the presentation of the vertical tool bar and all AR annotation tools will be available. A user will be able to use each of the AR annotation tools in this state, even if there isn't a surface to affix an annotation to; it is expected that the app will make a best effort to place the annotation tool at the depth that seems appropriate.

The user will continue to see white crosshatches throughout the process to indicate the collection of surface information.

Host User’s and Collaborator User’s Experience

• Once the streaming user activates the camera, the host user (and all additional collaborators, if there are 3 or more users in the session) will see the small AR indicator in the upper left corner of the CareAR screen. Until sufficient surface information is collected, the indicator will be amber.
• While the AR indicator is amber, AR related annotation tools will not be available to the host or the other collaborators. The tool button corresponding to a given AR annotation tool will be greyed out. (This preserves existing functionality.)

If the AR indicator is amber and the host or additional collaborator clicks or taps the indicator, then a popup will be presented indicating "AR is initializing by streamer"

Once the streaming user has gathered sufficient surface information, the host's AR indicator will turn green and full AR annotation from all the related tools will be made available.

Note that if the user has achieved "green status" once and then transitions to "amber status," we will not change the presentation of the vertical tool bar and all AR annotation tools will be available. The host will be able to use each of the AR annotation tools in this state, even if there isn't a surface to affix an annotation to; it is expected that the app will make a best effort to place the annotation tool at the depth that seems appropriate.

Configuring AR Hints

With this enhanced update, AR Hints are always enabled for the streaming user and no configuration is required. Prior to release 24.01, there was a configurable switch in the settings menu that was defaulted to off.

AR ELEMENT SUPPORT IN AN EXPERIENCE

In release 24.01 of the Experience Builder (made available in late February 2024), we introduced a new AR element capability that allows you users to view a 3D object in augmented reality anchored on a surface in real space. The release of this capability was supported in the web based Instruct app.

With release 24.01 of the CareAR app, we now support the in-app loading and running of experiences with an AR element.

FORMS ELEMENT SUPPORT IN AN EXPERIENCE

Also, in release 24.01 of the Experience Builder (made available in late February 2024), we introduced a new Form element capability that allows you to embed forms into a page that can collect information from the user. You can have form submissions emailed to a configured email address or sent via webhook to a configured HTTPs endpoint. You can also optionally save to session history. The release of this capability was supported in the web based Instruct app.

With release 24.01 of the CareAR app, we now support the in-app loading and running of experiences that include one or more Forms element(s).

CHANGE AUDIO ENCODING TO BE WIDEBAND AUDIO

The audio encoding during an Assist session has been updated to use wideband audio, enabling an improved live audio experience as well as higher fidelity audio in Assist recordings.

ENHANCED LOW BANDWIDTH MODE SUPPORT

With enhancements available within newer versions with some of the underlying SDK we have been able to introduce further enhancements to low-bandwidth mode functionality.

IMPROVED AUTOMATIC CONNECTIVITY TEST

With release 23.02 (and earlier releases), we used an ICMP-based ping to confirm service reachability. For many enterprise IT departments, ICMP traffic is frowned upon and is commonly classified as a security threat. Because of this, we disabled the ICMP-based ping in release 23.03.

In this release, we introduce a REST API-based service reachability test, thereby satisfying the enterprise IT policy of not using ICMP.

CLIENT UX ENHANCEMENTS

Adjusted navigation behaviors of "Left Arrow" and the "X" when within an Experience

Left Arrow Navigation: As a user navigates their way through an experience, the "Left arrow" is intended to take user to the previous experience page. On the first page, if the user taps the "left arrow" then it takes the user to the app's home screen.

The behavior we adjusted is, instead of taking the user to the home screen (when on the first page), we take the user to the screen where the user originally launched the experience. For example, as a user if I am in the History screen and tap on an experience, then if I choose to tap the left arrow on the first page, then the app should take me to the screen where I originally loaded the experience (and not the home screen.

"X" Navigation: If the user taps the "X" from within an Experience, then the app should then take the user the screen where the user originally launched the experience.

Removal of Tour Guide

The Tour Guide as you have seen in release 23.03 and earlier has been removed from the app in this version in preparation for a brand new refreshed version in an upcoming future release. (The Tour Guide was formerly located under the Help and Feedback selection of the “hamburger menu.”)

Improved generic error handling for the desktop app

As an ongoing effort to improve the usability of the desktop app, a revised screen was created for generic errors that may be encountered. (This would be experienced when there are not context-specific error handling screens.)