In order to add on-device collections to your Android app, it is necessary to:
- create an on-device collection in CraftAR;
- generate a collection bundle for your SDK version;
- download the bundle of that collection from CraftAR and add it to the app;
- you are ready to go and start using the on-device collection with the SDK
Once the collection is added to the device, you can start using the On-device Image Recognition SDK to perform visual search queries on the device. You can find more details for the first step in the tutorial about how to create an on-device collection in CraftAR. And for the second and third steps, please read the tutorial about how to Manage on-device collections for the Android SDKs.
An Image Recognition app using the native On-device Android Image Recognition SDK can be implemented by the following two steps. First, setup the on-device collection. Second, set up the page and the camera capture and run the on-device image recognition to get the results for each item that is recognized.
1. Set up the on-device collection in your app
Loading collections into memory
Once the collection is added to the local database, we can set the collection that will be used for on-device image recognition using the CraftAROnDeviceIR class.
Using the CraftAROnDeviceIR.setCollection() method and passing the collection, the plugin will load the collection to the On-device Image Recognition module and set it active for Image recognition.
|
1
2
3
4
5
6
7
|
CraftAROnDeviceIR.setCollection(collection, function (collection) {
// Success callback
}, function (error) {
// Error callback
}, function(progressFloat) {
// ProgressCallback
});
|
Once you get the success callback, the on-device Image Recognition module is ready to receive images and produce recognition results.
Loading collections can take a few seconds, depending on the amount of images to load. The Plugin provides progress feedback for this process as well.
This process can be done in any page in your application. You just need to make sure you have access to the cordova.js script.
2 Setup the Plugin in your project
Once you have setup the CraftAR Cordova Plugin into your Cordova project, it’s time to implement the views that will allow to perform the Image Recognition on-device.
1. Start the camera capture in a new view
First of all, make sure the cordova.js script is included in your index.html start page.
|
1
|
[crayon-618c389b3281d668097900 inline="true" ]<script src="cordova.js"></script>
|
[/crayon]
After that, We need to start a new view. This view will allow the Plugin to open the camera in the background and display our UI as a transparent overlay.
|
1
2
3
|
document.addEventListener("deviceready", function () {
CraftARSDK.startView(null, null, {"loadUrl" : "your_search_view.html"});
}, false);
|
2. Start the video capture
To be able to access the Plugin JS interface, the capture page needs to import the cordova.js script. Then you can listen for the ‘deviceReady’ event and use the SDK to start the camera capture.
|
1
|
[crayon-618c389b32820167330454 inline="true" ]<script src="cordova.js"></script>
|
[/crayon]
Once the device is ready, you will have to start the capture using CraftAR SDK:
|
1
2
3
4
5
6
7
8
|
document.addEventListener("deviceready", function () {
startCraftAR();
}, false);
function startCraftAR() {
CraftARSDK.onStartCapture(didStartCapture);
CraftARSDK.startCapture();
}
|
3. Prepare the SDK to process searches
The CraftARSDK class also manages the search processes by sending search events to the SearchController, in this case the CraftAROnDeviceIR instance. The CraftAROnDeviceIR instance performs visual searches in the collection of images that is previously set for this app. You can find more details with regards to how to set the bundles in the separate tutorial about how to manage collection bundles with the On-device Image Recognition SDK.
Set the CraftAROnDeviceIR class as the searchController. The searchController is the object that will manage the singleShotSearch(), startFinder() and stopFinder() calls of the CraftARSDK. We will also set the Callbacks for the search responses.
We can do this after getting the callback we set for when the capture is ready:
|
1
2
3
4
5
6
|
function didStartCapture() {
CraftARSDK.searchController = CraftAROnDeviceIR.searchController;
CraftAROnDeviceIR.onSearchResults(getResults);
CraftAROnDeviceIR.onSearchError(onError);
setuOnDeviceRecognition();
}
|
2. Implementing the searches and parsing the results
Once you are ready with the previous steps, it’s time to add code to start scanning the real world.
Using the CraftARSDK class we can search the on-device image database in two modes: Finder Mode or Single Shot Mode.
Option A. Use Single Shot Mode to take a single picture
Call singleShotSearch to perform a search with a single image. By calling this method, the SDK triggers the still image capture from the camera and forwards the captured image to the search controller (in this case, the CraftAROnDeviceIR instance)
A common approach consists in triggering the search when the user performs an action on the UI (for example, clicking on a button). Ensure that you have initialized everything before allowing the user to perform the singleShotSearch, otherwise it will fail.
|
1
|
CraftARSDK.singleShotSearch();
|
The response to a query triggers the callback passed to onSearchResults() with the results available in an array that is ready to parse. If the query failed, the callback passed to onSearchError() will be triggered. A query can fail for multiple reasons, but the most common one is when the query image does not have enough details. If you point to a completely uniform surface (i.e a white wall), you will receive this error.
|
1
2
3
4
5
6
7
8
|
function getResults(results) {
if (results.length > 0) {
var item = results[0].item;
} else {
alert("Nothing found!");
}
CraftARSDK.camera.restartCapture();
}
|
When the singleShotSearch() method is called, the camera is frozen until restartCapture() is called.
Option B. Use Finder Mode for continuous scanning
Call startFinder to start searching continuously without user intervention. This method forwards the camera frames to the search controller (in this case, the CraftAROnDeviceIR instance)
|
1
|
CraftARSDK.startFinder();
|
For every frame that is processed, the SDK generates a query and searches for matches in the local collection. The response to a query produces a callback to searchResults with the results available in an array that is ready to parse.
|
1
2
3
4
5
|
function getResults(results) {
if (results.length > 0) {
var item = results[0].item;
}
}
|