OpenCV For Mobile Devices Using Android Studio

Background

OpenCV was known as a computer vision library for 15 years. Ever increasing open source community have extensively made use of OpenCV without any hesitation and contribute back to it’s growth as well. Ever since its inception OpenCV has extended their supports to wide range of platforms and the recent adoption is Android mobile platforms.

There are two main IDEs supported at the moment for Android development of OpenCV.

  1. Eclipse with CDT plugin, Android SDK, NDK and ADT pluginThere are plethora of tutorial on Android Development using Eclipse including official pages of OpenCV. This and this would make you through in development of OpenCV using Eclipse. OpenCV java wrapping can also be built for Eclipse.
  2. Second option is using Android Studio (AS). AS gets better with time as lots of community effort is put forward though initially it had significant bottlenecks.

Which is better? Though it is hard to answer, recently Android community have announced that their support for Eclipse is ended and now they are focusing on AS. Detailed answers can be found in this.

The purpose of this blog is to make Android developers’ life easy in adopting with the development of OpenCV using AS. Let’s get start.

Pre-requisites

Download OpenCV and unzip (at say <opencv-source>), if you want to code in C++ and use native interface for Android. In this case, you can install OpenCV for Windows using this. You can still build OpenCV4Android from source. You can simply ignore this, if you prefer pre-built OpenCV4Android libraries instead.

  1. Download and unzip OpenCV4Android (at say <opencv4android>)
  2. Download and install Android Studio

android_studio_splashscreen-780x585

 

Run AS

  1. Launch it and follow the instructions to build a new project as given in this.
  2. You can have a testing device (such as an Android mobile device) or an emulator in order to test your application.To install HAXM or it’s latest, go to ‘Tools’ -> ‘Android’ -> ‘SDK Manager’. In the opening window, select ‘Appearance & Behaviour’ -> ‘System Settings’ -> ‘Android SDK’ and select ‘SDK Tools’ tab. Check the ‘Intel x86 Emulator Accelerator …’ radio button and click ‘Apply’ button to install HAXM for Android Studio. Then run ‘intelhaxm-android.exe’ file, following the steps given in this.
  3. If you are having “haxm is not working and emulator runs in emulation mode” error when you run the application on emulator, the HAXM should be re-installed. This is because AVD cannot have higher memory limit than the HAXM. You can adjust memory limit of HAXM by re-installing HAXM using the instructions given in 2).
    Or otherwise, if you need to change AVD memory, follow the above link’s 3rd best answer    in ‘Tools’ -> ‘Android’ -> ‘AVD Manager’.

It is advised to use an Android mobile device for testing instead of the emulator as it eats up lots of memory in the system.

SDK manager

Install SDK tools using SDK Manager

OpenCV with AS

After having experimented a sample code above, lets take an OpenCV example. Sample projects can be found in ‘<opencv4android>\samples’ folder.

  1. Copy the ‘res’ folder of a sample project (say in ‘<opencv4android>\OpenCV-android-sdk\samples\color-blob-detection’) and replace it in the project you just created above (say ‘opencvsample’) in the workspace location (‘<AndroidStudioPorjects>\opencvsample\app\src\main’)
    After replacing ‘res’ folder, make sure you retain ‘mipmap*’ folders and ‘colors’, ‘dimens’, and ‘styles’ files in ‘values’ folder.
  2. Copy ‘src’ folder in ‘<opencv4android>\samples\color-blob-detection’ and paste inside the ‘<AndroidStudioPorjects>\opencvsample\app\src\main’ after removing ‘java’ folder resides in this location. Rename the ‘src’ folder as ‘java’.

Now you will have to import modules and libraries for this project as given below.

Import OpenCV Modules and Libraries

Import opencv.java modules by File->New->Import Modules and provide  ‘<opencv4android>\OpenCV-android-sdk\sdk\java’ (Instead we can just copy these modules from sdk path and add the library path in settings.gradle)

Import module

Specify SDK path to import OpenCV for Android

if there is a Gradle synchronizing error, change the ‘compileSdkVersion’ and others as follows in build.gradle file in openCVLibraryXXX (imported module) folder.

Change build gradle

SDK version should be matched with the one we have in the system

And add as a dependency to the ‘app’ module in File->’Project Structure’

Add dependencies.PNG

Add OpenCV library as a dependency to the application

Copy libraries of OpenCV from ‘<opencv4android>\OpenCV-android-sdk\sdk\native\libs’ into a newly created folder (jniLibs) in side the ‘app\src\main’ folder. (Otherwise you will have an error saying “OpenCV Manager package was not found! Try to install it?”)

If ‘namespace opencv is not bound’ shows for opencv:show_fps in layout/*.xml file, add ‘xmlns:opencv=”http://schemas.android.com/apk/res-auto&#8217; at the beginning.

Camera Configurations

This is an implementation of ‘<opencv4android>\samples\color-blob-detection’ in AS. When you run this application using emulator; since the camera is needed here; there can be times that the application will not run smoothly as some configuration on camera is needed.

Add web cam on the emulator using AVD Manager. Please note that, add only one (front or back) as shown

web cam back

Add web camera on AVD

Add web cam permission on AndroidManifest.xml file

cam perm on manifest

Add permission for camera to be used in the application

Check the AndroidManifest.xml file for names whether those are different as we are just copying samples on the built project (on mine, the package name, android:name, had to be changed).

You should add web cam permission on AndroidManifest.xml file to enable permission option in android devices. After that you can grant permission on the device itself.

Note that, in recent Android versions, you will have to grant permission explicitly as shown below unlike in the previous versions. Otherwise your application will be stopped unexpectedly saying ‘It seems that your device does not support camera (or it is locked). Application will be closed.’ For that go to Menu -> Settings -> Apps -> <My_Application> and click ‘Permission’. Then in the opening window, enable ‘Camera’.

Phone Application

Tap Permissions and enable Camera

Using An External Android Device For Testing

To use a device instead of emulator in AS, follow the instructions given in this. Sometimes the device might not get detected in Windows environment as USB drivers needed to detect them. Install required drivers using this.

Select Run -> Run ‘app’. In the ‘Device Chooser’ window, select ‘Choose a running device’. If it shows that the device is offline even after it is detected, enable ‘USB debugging’ option in your device. It should be enabled in order to work with the device.

In the recent versions of Android, you might not see ‘Developer options’ as it comes as a hidden feature. Go to Settings -> General -> About device. tap the ‘Build number’ section 7 times. This will show-up ‘Developer options’. You will find ‘USB debugging’ under ‘Developer options’.

When you run the application, you might observe that the camera is rotated 90 degrees. There is a solution to solve this. But this time it is “OpenCV Error: Insufficient memory ( Failed to …” error. To overcome this, I came up with these modifications.

mRgba = inputFrame.rgba();
Size sizeTemp = mRgba.size();
Core.transpose(mRgba, mRgba);
Core.flip(mRgba, mRgba, 1);
Imgproc.resize(mRgba, mRgba, sizeTemp);

This is the end of this blog and hope you have gained something out of this. Should you require any clarifications, feel free to comment below.

3 thoughts on “OpenCV For Mobile Devices Using Android Studio

  1. Pingback: Building OpenCV For Android From Source | You, Myself and Community

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s