With release 24.02 of the CareAR App, we announce the following:

• Screen sharing
• Scan functionality
• UX enhancements

New Features and Capabilities

SCREEN SHARING

In the context of remote support and remote solve, being able to share one's desktop or mobile device screen is helpful for users to share information and communicate context.

Common use cases for screen sharing include desktop support and various ITSM flows, mobile configuration assistance, enhanced troubleshooting, and collaboration centered around a shared document that supports problem resolution (e.g., a particular page in a PDF manual, wiring diagram, or low-level schematic).

CareAR’s new screen sharing feature introduces the ability for a CareAR Assist host or participant (i.e., a collaborator) to share the screen of their device into an Assist session.

They can share:

• The entire screen of the desktop, mobile phone, or tablet; or
• The screen of a specific application (desktop)

Desktop users with multiple screens will be able to select which screen to share. Desktop-based users will have more options than mobile phone/tablet users due to the richer window and screen management that the desktop device affords.

Assist session participants who are not collaborators, but are using the CareAR native mobile app, will have a mechanism that allows them to request to share into the session. By tapping the Share tool, they will transmit a request to the host, effectively making a request for collaborator status (in the same way that a non-collaborator participant would request of the host permission to stream video in the passing the ball feature).

Guest users who join an Assist session using the CareAR Assist for Browser app or CareAR Assist for Smartglasses app can observe the shared screen in the session.

Participants in the screen share who have joined using the CareAR app (for mobile or desktop) can use the laser pointer to call out areas of interest, the snapshot tool to take a “screen grab”, and (for hosts only) the Video recording tool to make recordings.

Minimum requirements: the host of the CareAR Assist session must run release 24.02 of the CareAR application (desktop or mobile).

Screen Share Policy and Permission

The security model for the screen share feature is simple: the host of the session controls who is allowed to share. Non-host users who wish to share their screen must request permission AND the host must grant permission. The host may choose to decline the request.

For more strict controls over screen share access, the role-based permissions table (on Admin portal > Users > Roles) has been extended to include an "Allow Screen Share" column. For screen share as a feature to be permitted to be used on the CareAR desktop/mobile app, the role-based permissions table must have that role/feature checked. If it is not checked, then the feature will not be available to that user role. This aligns with the other features that are in the role-based permissions table (e.g., Allow Recording, Allow Snapshot). Using this tool, you can effectively disable screen share for all users in the tenant, if that is the preferred policy of the organization.

Commercial Considerations

Screen sharing is included free-of-charge for all CareAR users in all account plans and types.

Limitations

The screen share capability in an Assist session requires that the host be running version 24.02 of the CareAR application.

Note that users who have joined an Assist session using the CareAR Assist for Browser web app will not be able to share their screen. However, the browser-based user can view a shared screen when a host or collaborator has shared from the desktop or CareAR native mobile app.

The CareAR screen share feature, by design, does not permit remote control of another user’s screen or device.

Users who share an application that includes audio (e.g., streaming a video) will not be able to share the audio into the session.

SCAN FUNCTIONALITY

To support the CareAR Visual Verification, the CareAR app now supports a scan mode. This is used during the machine learning model creation process. Content creators and administrators can use the CareAR app to scan an object for the purposes of generating a data set that can be used to train an ML model.

The scan functionality is used during the initial stages of ML model creation. Refer to the documentation on the CareAR help center (https://help.carear.com) for complete instructions to use the Visual Verification feature.

Requirements

To use the scan functionality within the CareAR mobile app, there are both plan and device requirements.

Plan Requirements

Tenants must have one of the following to use the visual verification and scan feature.

• Platform plan
• AI Services add-on

Device Requirements

To use the scan functionality your device must meet the following requirements.

• iOS 17 or later
• iPhone 12 Pro, iPad Pro 2021, and later models
  NOTE: Android is not supported

ML Model Creation Permission

Only users with Administrator and Content Creator roles will have access to the Create ML model option from the home screen of the CareAR app. Users with roles other than Administrator and Content Creator will not see the Create ML model button on the home screen.

NOTE: If users have a role other than Administrator or Content Creator, they will not see the Create ML model button.

Perform a Scan

To perform a scan users will tap on “Create ML model” then either select an existing project or create a new project. The view from the camera will be presented on screen. To begin the scan simply tap on the scan button. Scanning will begin immediately as indicated with a counter that shows how many images have been captured. You can capture up to a maximum of 200. Once 200 images have been captured or you stop the scan, the images will be processed, and a 3D mesh will be created from the captured images. This process can take some time, but once complete you will see a 3D mesh to review. If acceptable, click on SAVE SCAN and the data will be uploaded to the CareAR platform.

Once the scan data has been uploaded to the CareAR platform the next step is to login to the CareAR platform and label your data. The user will get an email notification when the uploaded data is ready for labeling. Once logged into the CareAR portal, go to AI Services -> Object Detection -> PROJECT NAME and then click on the scan name to label the data.

UX ENHANCEMENTS

Several UX enhancements have been included in release 24.02.

Input Box Design

The input box design was revised to improve symmetry and spacing. This was implemented for the sign-in screens, the profile screen, and the session invitation screen. See the examples below:

Change ‘X’ Icon in AR Measurement Tool

In 24.01 of the CareAR app, the new AR Measurement tool provided a means to exit the measurement mode with an ‘X’ located at the top right corner. We adjusted the user interface so that ‘X’ simply exits the measurement mode and does not preserve any measurements taken as annotations. There’s a button on the bottom center that has a checkmark; by tapping that button, you 1) preserve the measurement as an annotation, and 2) exit the measurement mode.

AR Measurement Tool – Landscape Mode

With release 24.02, we’ve added the ability to use the AR Measurement tool in landscape mode.

Removal of Visual Feedback During 3D Map Generation

Prior to release 24.02, when a user streams video into an Assist session using the CareAR app, the user will see small white “cross hatches” to illustrate the fact that mapping data is collected. This visual feedback is a leftover artifact of the early AR application implementations, where it was common to provide some type of visual feedback to the user.

The market now has many AR applications, and many users are now familiar with how to use them. Further, the AR hints enhancement that CareAR delivered in release 24.01 helps to guide the user (and the host) how to best collect the 3D world space information, making the need for the cross hatches unnecessary.

Based on feedback from our customers, we’ve elected to remove the small white cross hatches entirely from the AR training phase. This helps when detailed snapshots and video recordings are created and used as business process artifacts.

Adding a Searched Experience as a Favorite

For users who are members of a tenant with many Instruct Experiences, the search tool is helpful to quickly find an Experience. We enhanced the CareAR app to let a user mark an Experience returned in the search results as a Favorite. Similarly, a user can remove the Favorite tag from an Experience that was delivered in a set of search results.

Send Logs

The CareAR mobile app now provides a Send Logs capability in the settings menu for those rare cases where CareAR (or our partners) need help troubleshooting an issue. The user, if requested to share logs, can do so by tapping the “hamburger menu” settings icon to expose the top level settings menu. From there, simply tap Send logs and the user is prompted through the necessary steps.