- Augmented Reality SDK v4+
- Cloud Image Recognition SDK v3.0+
- On-device Image Recognition SDK v1.1+
Are you still using an older version? Previous versions of the SDKs will not receive updates anymore. If you need help transitioning to the newer version, you can get support through our support channel.
An Image Recognition app using the Android native SDK can be implemented following two steps. First, you need to set up the CraftARActivity and then you can run image recognition and parse the results for each item that is recognised.
If you want to see an example that implements Cloud Image Recognition using the Augmented Reality SDK, take a look at the open source samples available in our Github repository: https://github.com/Catchoom/craftar-example-android
Set up the SDK in your CraftARActivity
Once you have set up the CraftARSDK into your Android project, it’s time to implement the CraftARActivity that will show the experience.
The following is an example for an app that uses CraftAR’s Cloud Image Recognition.
1. Create a CraftARCameraView in your layout. This is the view where the camera frames will be drawn.
|
1
2
3
4
5
6
7
8
9
10
11
|
<RelativeLayout xmlns:android="http://schemas.android.com/apk/res/android"
xmlns:tools="http://schemas.android.com/tools"
android:layout_width="match_parent"
android:layout_height="match_parent"
tools:context=".MyCraftARActivity">
<com.craftar.CraftARCameraView
android:layout_width="match_parent"
android:layout_height="match_parent"
</com.craftar.CraftARCameraView>
</RelativeLayout>
|
2. Make your Activity extend from CraftARActivity
You will have to implement the methods onPostCreate(), onPreviewStarted(), and onCameraOpenFailed(). Obtain the CraftARSDK instance and start the camera capture:
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
|
public class MyCraftARActivity extends CraftARActivity{
@Override
public void onPostCreate() {
View mainLayout= (View) getLayoutInflater().inflate(R.layout.activity_main, null);
setContentView(mainLayout);
mCraftARSDK = CraftARSDK.Instance();
/**
* Initialise the SDK with your Application Context before doing any other
* operation with any other module. You can do this from any Activity
* (for example the Splash Activity); */
mCraftARSDK.init(getApplicationContext());
mCraftARSDK.startCapture((CraftARActivity)this); //Pass a reference to the Activity
...
}
@Override
public void onPreviewStarted(int frameWidth, int frameHeight) {
// startCapture() was completed successfully, and the Camera Preview has been started
}
@Override
public void onCameraOpenFailed(){
// startCapture() failed, so the Camera Preview could not be started
}
}
|
3. Obtain and configure the CraftARCloudRecognition instance
Once the CraftARSDK is initalised, you can get an instance of the CraftARCloudRecognition. To perform image recognition, you will have to implement the SetCollectionListener and CraftARSearchResponseHandler interfaces. For simplicity, we will implement them in the same Activity, so everything is in place. Call setCollection() to configure the SDK to your with the collection pointed by your collection token.
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
|
public class MyCraftARActivity extends CraftARActivity implements
CraftARSearchResponseHandler, SetCollectionListener{
@Override
public void onPostCreate() {
...
//Get the Cloud Image Recognition instance
mCloudIR = CraftARCloudRecognition.Instance();
/* Set this class as the one to receive search responses. */
mCloudIR.setCraftARSearchResponseHandler((CraftARSearchResponseHandler)this);
/**
* Set the Search controller from the Cloud IR class.
* The Cloud IR class knows how to perform visual searches in the CraftAR CRS
* Service. The searchController is a helper class that receives the camera frames
* and pictures from the SDK and manages the Single shot and the Finder mode searches.*/
mCraftARSDK.setSearchController(mCloudIR.getSearchController());
/**
* Set the collection we want to search with the COLLECTION_TOKEN.
* When the collection is ready, the collectionReady callback will be triggered.
*/
mCloudIR.setCollection("your_collection_token", (SetCollectionListener)this);
}
...
@Override
public void collectionReady() {
//setCollection() was completed successfully, and the token is valid.
}
@Override
public void setCollectionFailed(CraftARError craftARError){
//setCollection could not be completed. Check the error for more details
}
@Override
public void searchResults(ArrayList<CraftARResult> results, long searchTime, int requestCode){
//Returns the search results for an Image Recognition request
}
@Override
public void searchFailed(CraftARError craftARError, int requestCode){
//An image recognition request failed, check the error for more details
}
}
|
4. Do the Image Recognition request and parse the results
The CraftARSDK class manages the search modes (the Finder Mode or Single Shot Mode) and forwards the events to the assigned SearchController. When a response is found, the CraftARCloudRecognition instance will notify the CraftARSearchResponseHandler (in this case our CraftARActivity) with the search results.
You can scan in the two modes described below
Option A. Use Single Shot Mode to take a single picture
You can call singleShotSearch to perform a search with a single image. This method sends a single query to CraftAR’s Cloud Image Recognition. The response to that query triggers searchResults. You could for instance call the following function once the collectionReady callback has been received:
|
1
2
3
4
|
void mySearchFunction() {
//Trigger this function when clicking on a button, for example.
mCraftARSDK.singleShotSearch();
}
|
and then the implementation of searchResults() should parse the results and restart the camera.
|
1
2
3
4
5
6
7
8
9
10
11
12
|
@Override
public void searchResults(ArrayList<CraftARResult> results, long searchTime, int requestCode) {
//Call restartCapture if you have called singleShotSearch(), to restart the camera.
mCraftARSDK.getCamera().restartCapture();
if(results.size()==0){
Log.d(TAG,"Nothing found");
}else{
CraftARResult result = results.get(0); //We get only the first result
CraftARItem item = result.getItem();
Log.d(TAG,"Found item "+item.getItemName());
}
}
|
Option B. Use Finder Mode for continuous scanning
You can call startFinder to start searching continuously without user intervention. This method sends queries at a controllable rate to CraftAR’s Cloud Image Recognition. For every query, the response triggers searchResults.
You could for instance call the following function once the collectionReady callback has been received:
|
1
2
3
4
|
void mySearchFunction() {
//Trigger this function when clicking on a button, for example.
mCraftARSDK.startFinder();
}
|
and then the implementation of searchResults() should parse the results.
|
1
2
3
4
5
6
7
8
9
10
11
12
|
@Override
public void searchResults(ArrayList<CraftARResult> results, long searchTime, int requestCode) {
//Call stopFinder() to stop sending queries
if(results.size()==0){
Log.d(TAG,"Nothing found");
}else{
mCraftARSDK.stopFinder()
CraftARResult result = results.get(0); //We get only the first result
CraftARItem item = result.getItem();
Log.d(TAG,"Found item "+item.getItemName());
}
}
|