Building OpenCV & OpenCV Extra Modules For Android From Source

Background

In my last blog, I described how to set up Android Studio (AS) to work with OpenCV4Android. Initially Android supported ADT plugin in Eclipse. If you are an Eclipse user in Android development, build OpenCV java wrapper for java or simply build OpenCV for C++ API following the respective blogs. You will need standalone SDK for Android in this case. However recently Android announced that their support for ADT plugin in Eclipse has ended and requested all the developments in Eclipse to shift to Android Studio environment.

OpenCV4Android has two forms as follows.

  1. Using OpenCV’s native interface. This would be a little hectic task for a fresh OpenCV starter. You will have to learn NDK stuffs but promising when it comes to performance optimization. Isn’t it nice to code in C++ and make work on a mobile, just forgetting the interconnections?
  2. Using Java-API. This is easier than the native interface. You just need to import OpenCV classes and move on. Not to forget, computations are performed at native level and hence their is a cost in comparably high JNI calls. On the other hand, Java has its own advantage and so does this method.

Both the methods need OpenCV4Android distribution either pre-built one or building from the source. OpenCV4Android pre-built libraries has only modules in OpenCV main repository and if you need more than that, you need a pure building from the source. Though you can build java wrapper for OpenCV using this, the ‘opencv-xxx.jar’ file does not contain android module. Even after you include android in the jar file, you will have to manually configure the dependency setup for AS. Therefore it is again advised to go for a building from the source for Android. Let’s start!

Pre-requisites

  1. Download OpenCV and unzip (say at <opencv-source>)
  2. Download OpenCV extra modules and unzip (say at <opencv-contrib-source>)
  3. Download Android NDK (say at <ndk-dir>)
  4. Download CMake (to say <cmake-dir>) and MinGW (to say <mingw-dir>)
  5. Install Android Studio

 

Configuration

Go to ‘<opencv-source>/platforms’ and create a folder named ‘android_arm’

Run ‘<cmake-dir>/bin/cmake-gui’ set paths as follows.

source and release dir.PNG

Set paths of source and building directories

Press ‘Add Entry’ button in the cmake-gui, add ‘ANDROID_NDK’ as a cmake ‘path’ variable and provide the value as <ndk-dir> (in my case path is ‘C:\android-ndk-r10e’).
Add ‘ANDROID_NDK_HOST_X64’ too and check that.

Set ‘CMAKE_TOOLCHAIN_FILE’ as ‘<opencv-source>/platforms/android/android.toolchain.cmake’.

Press ‘Configure’. Choose ‘MinGW Makefile’ as the compiler and press ‘Finish’. See  whether above configurations are properly set as in the below figures. Keep other settings as it is.

Android settings cmake.PNG

Android Variables in CMake

cmake settings cmake toolchain.PNG

CMake Toolchain Variables in CMake

Building From Source

Go to <mingw-dir>/msys/1.0 and run ‘msys’ bash file.

Navigate to ‘<opencv-source>/platforms/android_arm’ path and run ‘mingw32-make’ command. After completing this, run ‘mingw32-make install’.

java_for_android_3.PNG

OpenCV for Android is built using MinGW

Now you have successfully built OpenCV4Android in ‘<opencv-source>/platforms/android_arm/install/sdk’

Then, import built modules as follows (detailed step-by-step explanation is given in the previous blog)

  1. Launch Android Studio and create a new project selecting File -> New -> New Project…
  2. Go to File -> New -> Import Module… and provide  ‘<opencv-source>/platforms/android_arm/install/sdk/java’. Click ‘Next’ and ‘Finish’.
  3. If there is a Gradle synchronizing error, change the ‘compileSdkVersion’ and others as follows in build.gradle file in openCVLibraryXXX (imported module) folder.
  4. Then add the imported library as a dependency to the ‘app’ module in File->’Project Structure’.
  5. Copy libraries of OpenCV from ‘<opencv-source>/platforms/android_arm/install/sdk/native/libs’ into a newly created folder (jniLibs) in side the ‘app\src\main’ folder.

Additionally; in AS; after you import the module you might experience a gradle build error as follows.

unspecified on project app resolves to an APK archive which is not supported as a compilation dependency.

Therefore modify lines in build.gradle file of imported module as follows.

  1. Replace ‘apply plugin: com.android.application’ as, ‘apply plugin: com.android.library’
  2. Remove the line ‘applicationId “org.opencv”‘

 

OpenCV Extra Modules for Android Studio

Below are the steps to build OpenCV’s extra modules for Android Studio.

Go to each of the extra modules you need (say ‘<opencv-contrib-source>/modules/extra_1’) and open ‘CMakeLists.txt’ file. Find the line containing ‘ocv_define_module’ option.

ocv_define_module(extra_1 opencv some other modules)

Add ‘java’ modules at the end of the parenthesis.

ocv_define_module(extra_1 opencv some other modules java)

If there is no ‘WRAP’ found in the list, add ‘WRAP’ before ‘java’

ocv_define_module(extra_1 opencv some other modules WRAP java)

Open ‘cmake-gui’ and select ‘OPENCV_EXTRA_MODULES_PATH’ under ‘OPENCV’ category and insert ‘<opencv-contrib-source>/modules’

And keep the above ‘cmake-gui’ settings as it is. At run, extra module ‘opencv_bioinspired’ might fail and therefore uncheck that before the configuration in cmake-gui.

Again follow the steps given under sub topic ‘Building From Source’ explained earlier in this blog.

Final Word

Even after all, in your OpenCV 3+; though all the settings, modules and dependencies are set and imported; you might feel something missing after seeing applications launch on the Android device stops unexpectedly as follows.

Package not found

OpenCV Manager package was not found! Try to install it?

This is because initial OpenCV module call points to ‘opencv_java’ instead of ‘opencv_java3’.

To change this, go to openCVLibraryXXX (imported module) -> src -> main -> java -> org -> opencv -> android -> StaticHelper.java.

Find the line,

result &= loadLibrary(“opencv_java”);

and change this as,

result &= loadLibrary(“opencv_java3”);

 

This is the end of this blog and hope you found this useful. Why not you comment below if you think some necessary parts are missing or on something I have done wrong. Good day!

 

OpenCV For Mobile Devices Using Android Studio

Background

OpenCV was known as a computer vision library for 15 years. Ever increasing open source community have extensively made use of OpenCV without any hesitation and contribute back to it’s growth as well. Ever since its inception OpenCV has extended their supports to wide range of platforms and the recent adoption is Android mobile platforms.

There are two main IDEs supported at the moment for Android development of OpenCV.

  1. Eclipse with CDT plugin, Android SDK, NDK and ADT pluginThere are plethora of tutorial on Android Development using Eclipse including official pages of OpenCV. This and this would make you through in development of OpenCV using Eclipse. OpenCV java wrapping can also be built for Eclipse.
  2. Second option is using Android Studio (AS). AS gets better with time as lots of community effort is put forward though initially it had significant bottlenecks.

Which is better? Though it is hard to answer, recently Android community have announced that their support for Eclipse is ended and now they are focusing on AS. Detailed answers can be found in this.

The purpose of this blog is to make Android developers’ life easy in adopting with the development of OpenCV using AS. Let’s get start.

Pre-requisites

Download OpenCV and unzip (at say <opencv-source>), if you want to code in C++ and use native interface for Android. In this case, you can install OpenCV for Windows using this. You can still build OpenCV4Android from source. You can simply ignore this, if you prefer pre-built OpenCV4Android libraries instead.

  1. Download and unzip OpenCV4Android (at say <opencv4android>)
  2. Download and install Android Studio

android_studio_splashscreen-780x585

 

Run AS

  1. Launch it and follow the instructions to build a new project as given in this.
  2. You can have a testing device (such as an Android mobile device) or an emulator in order to test your application.To install HAXM or it’s latest, go to ‘Tools’ -> ‘Android’ -> ‘SDK Manager’. In the opening window, select ‘Appearance & Behaviour’ -> ‘System Settings’ -> ‘Android SDK’ and select ‘SDK Tools’ tab. Check the ‘Intel x86 Emulator Accelerator …’ radio button and click ‘Apply’ button to install HAXM for Android Studio. Then run ‘intelhaxm-android.exe’ file, following the steps given in this.
  3. If you are having “haxm is not working and emulator runs in emulation mode” error when you run the application on emulator, the HAXM should be re-installed. This is because AVD cannot have higher memory limit than the HAXM. You can adjust memory limit of HAXM by re-installing HAXM using the instructions given in 2).
    Or otherwise, if you need to change AVD memory, follow the above link’s 3rd best answer    in ‘Tools’ -> ‘Android’ -> ‘AVD Manager’.

It is advised to use an Android mobile device for testing instead of the emulator as it eats up lots of memory in the system.

SDK manager

Install SDK tools using SDK Manager

OpenCV with AS

After having experimented a sample code above, lets take an OpenCV example. Sample projects can be found in ‘<opencv4android>\samples’ folder.

  1. Copy the ‘res’ folder of a sample project (say in ‘<opencv4android>\OpenCV-android-sdk\samples\color-blob-detection’) and replace it in the project you just created above (say ‘opencvsample’) in the workspace location (‘<AndroidStudioPorjects>\opencvsample\app\src\main’)
    After replacing ‘res’ folder, make sure you retain ‘mipmap*’ folders and ‘colors’, ‘dimens’, and ‘styles’ files in ‘values’ folder.
  2. Copy ‘src’ folder in ‘<opencv4android>\samples\color-blob-detection’ and paste inside the ‘<AndroidStudioPorjects>\opencvsample\app\src\main’ after removing ‘java’ folder resides in this location. Rename the ‘src’ folder as ‘java’.

Now you will have to import modules and libraries for this project as given below.

Import OpenCV Modules and Libraries

Import opencv.java modules by File->New->Import Modules and provide  ‘<opencv4android>\OpenCV-android-sdk\sdk\java’ (Instead we can just copy these modules from sdk path and add the library path in settings.gradle)

Import module

Specify SDK path to import OpenCV for Android

if there is a Gradle synchronizing error, change the ‘compileSdkVersion’ and others as follows in build.gradle file in openCVLibraryXXX (imported module) folder.

Change build gradle

SDK version should be matched with the one we have in the system

And add as a dependency to the ‘app’ module in File->’Project Structure’

Add dependencies.PNG

Add OpenCV library as a dependency to the application

Copy libraries of OpenCV from ‘<opencv4android>\OpenCV-android-sdk\sdk\native\libs’ into a newly created folder (jniLibs) in side the ‘app\src\main’ folder. (Otherwise you will have an error saying “OpenCV Manager package was not found! Try to install it?”)

If ‘namespace opencv is not bound’ shows for opencv:show_fps in layout/*.xml file, add ‘xmlns:opencv=”http://schemas.android.com/apk/res-auto&#8217; at the beginning.

Camera Configurations

This is an implementation of ‘<opencv4android>\samples\color-blob-detection’ in AS. When you run this application using emulator; since the camera is needed here; there can be times that the application will not run smoothly as some configuration on camera is needed.

Add web cam on the emulator using AVD Manager. Please note that, add only one (front or back) as shown

web cam back

Add web camera on AVD

Add web cam permission on AndroidManifest.xml file

cam perm on manifest

Add permission for camera to be used in the application

Check the AndroidManifest.xml file for names whether those are different as we are just copying samples on the built project (on mine, the package name, android:name, had to be changed).

You should add web cam permission on AndroidManifest.xml file to enable permission option in android devices. After that you can grant permission on the device itself.

Note that, in recent Android versions, you will have to grant permission explicitly as shown below unlike in the previous versions. Otherwise your application will be stopped unexpectedly saying ‘It seems that your device does not support camera (or it is locked). Application will be closed.’ For that go to Menu -> Settings -> Apps -> <My_Application> and click ‘Permission’. Then in the opening window, enable ‘Camera’.

Phone Application

Tap Permissions and enable Camera

Using An External Android Device For Testing

To use a device instead of emulator in AS, follow the instructions given in this. Sometimes the device might not get detected in Windows environment as USB drivers needed to detect them. Install required drivers using this.

Select Run -> Run ‘app’. In the ‘Device Chooser’ window, select ‘Choose a running device’. If it shows that the device is offline even after it is detected, enable ‘USB debugging’ option in your device. It should be enabled in order to work with the device.

In the recent versions of Android, you might not see ‘Developer options’ as it comes as a hidden feature. Go to Settings -> General -> About device. tap the ‘Build number’ section 7 times. This will show-up ‘Developer options’. You will find ‘USB debugging’ under ‘Developer options’.

When you run the application, you might observe that the camera is rotated 90 degrees. There is a solution to solve this. But this time it is “OpenCV Error: Insufficient memory ( Failed to …” error. To overcome this, I came up with these modifications.

mRgba = inputFrame.rgba();
Size sizeTemp = mRgba.size();
Core.transpose(mRgba, mRgba);
Core.flip(mRgba, mRgba, 1);
Imgproc.resize(mRgba, mRgba, sizeTemp);

This is the end of this blog and hope you have gained something out of this. Should you require any clarifications, feel free to comment below.

Building Java Wrapper For OpenCV

In my last post, I explained how OpenCV is built on Windows environment using MinGW. Here I will explain how OpenCV for Java can be built using MinGW.

Java has a wide range of proven coding practices such as high level abstraction, easy memory management and more. This is an endeavour of the OpenCV community to bring best of both worlds where C++ is used at the development level and Java (not to mention python) at the implementation level. And this has opened up more audience levels after OpenCV adapted Java and python languages.

It is of course easy to download OpenCV pre-built libraries for java. But pre-built files are only available for Windows and for other OSes, it is required to build from source. Further, the java bindings are only available for main modules in the opencv main repo and not for opencv extra modules, at least as of now.

What if you need opencv extra modules in java wrappers. Then you are left with only option. Build it from the source! This blog covers most of the part related to Windows environment, still you would be able to grasp the main steps involving in the process on any OSes.

Pre-requisites

Java path

Figure 1: Set System Paths for Ant & Java

  1. Download python 2.6+
  2. Download jdk 6+ as java binding for OpenCV needs it.
  3. Set java path as shown in the Figure 1.
    1. Set ‘JAVA_HOME’ variable under ‘User variables’
    2. Set ‘Path’ variable under ‘System variables’
  4. Download Ant (say at ‘ant-dir’)
  5. Set ant path variables (refer Figure 1) referring this
  6. Download CMake
  7. Download MinGW and set path

CMake

Follow cmake variable settings under CMake Configuration in the previous blog. Additionally uncheck ‘BUILD_SHARED_LIBS’ under ‘BUILD’ category as shown in the Figure 2.

No shared

Figure 2: Uncheck shared library option

Once you ‘Configure’ (please select MinGW Makefiles and click’Finish’), you will see a new option, ‘BUILD_FAT_JAVA_LIB’ which specifies that a java wrapper will be created for all OpenCV libraries enabled.

For me, whatever the reason (if you know, please point me out) the set paths for ant, java and python had no effect on cmake and they were not picked at the configuration. You can check it after ‘Configure’ in the generating window whether Java wrappers: is set to YES or NO. If it’s YES, you are through and simply ignore the following steps and just click ‘Configure’. But for me it was NO. Therefore I had to manually set paths using cmake-gui.

Ant path

Figure 3: Set Ant path in cmake

Give the path ‘<ant-dir>\bin\ant.bat’ for ‘ANT_EXECUTABLE’ under ‘Ungroped Entries’ as shown in the Figure 3. (Don’t worry on the other un-set paths under ‘Ungrouped Entries’ as those are not required under this topic)

Set java paths in cmake-gui as shown in the Figure 4.

Then ‘Configure’, you will see ‘BUILD_opencv_java’ is checked under BUILD category.

 

Java path cmake

Figure 4: Set Java path in cmake

MinGW

Follow the steps given under MinGW Compilation in the previous blog.

Note: Strangely mingw32-make has invoked ‘jre’ path instead of ‘jdk’ though I have set paths in both system and cmake-gui correctly. It searches for tools.jar in ‘<jre-dir>/lib’ folder while it is of course resides in ‘<jdk-dir>/lib’. I copied tools.jar and pasted inside ‘<jre-dir>/lib’ to proceed with the building process. (I know this is not the recommended way of doing this. But for the moment it worked for me. If you know a better way of doing this, please point me)

OpenCV Extra Modules

Follow the steps given under OpenCV Extra Modules in the previous blog.

Though the above steps will include extra modules in ‘opencv_java’ library, it will not bind java libraries with .jar file. For this you will have to specify in each of your preferred extra modules that you need java binding at .jar file.

E.g.: If you need ‘text’ module in ‘opencv_contrib’,

  1. Open file <opencv_contrib_path>\modules\text\CMakeLists.txt> and search for the line containing ‘ocv_define_module’ option.
  2. Add ‘java’ at the end of the module list as

ocv_define_module(text opencv_ml … WRAP python java)

If the ‘ocv_define_module’ does not contain ‘WRAP’, then include that before ‘java’. Otherwise you will probably have an error saying ‘cannot find -ljava’.

E.g.: ‘text’ module contains ‘WRAP’ while ‘adas’ does not. Therefore add ‘WRAP java’ as follows in <opencv_contrib_path>\modules\adas\CMakeLists.txt>

ocv_define_module(adas opencv_xobjdetect WRAP java)

 

Using Java Wrappers in Eclipse

The above steps will generate two important files inside ‘<opencv-build>\install\java’. One is ‘libopencv_java300.dll’ static library containing all opencv built libraries. The second is ‘opencv-xxx.jar’ executable file which can now be used as the java binding interface by Eclipse. Please refer this site to configure Eclipse environment in order to use ‘opencv-xxx.jar’ for java wrapping.

 

This is the end of this blog and hope you enjoyed it. It would be great if you can point me mistakes, if you have tackled any. Thank you.

 

Building OpenCV Using MinGW on Windows

Background

OpenCV is a rich computer vision open-source library initially maintained by Intel Inc. and now governed by itseez. Development of OpenCV is generally based on C and C++ before it is implicitly said that C API is almost deprecated. By the way python and Java wrappers are also available to make use of rich functionalities and efficient coding style in respective languages. Further, OpenCV supports cross platforms ranging from PCs to Mobile devices while supporting various GPU acceleration optimizations such as CUDA and OpenCL as well as TBB, etc.

opencv-logo

The main objective of this blog is to present the way in which OpenCV 3.0.0 is built on Windows environment. Though there are so many articles, blogs and forums covering this basic topic, I saw a lack of integration of building OpenCV with MinGW for platforms (Eclipse, Netbeans, etc.) other than Visual Studio. If you also have faced the same issue then the below steps are for you.

Download

  1. Download or clone OpenCV and unzip to a desired destination (say ‘opencv-source’).
  2. Download OpenCV Extra Modules (as say ‘opencv_contrib’)
  3. Download cmake binaries and unzip into a desired location (say ‘cmake-binary’).
  4. Download and Install MinGW as a fresher (say at ‘mingw-dir’)
  5. Set PATH of MinGW (otherwise you would have an error saying ‘libgmp-10.dll is missing’)

CMake Configuration

96px-cmake-svg

  1. Run ‘cmake-gui’ in <cmake-binary>/bin
  2. Enter <opencv-source> path in the ‘Where is the source code:’ text box.
  3. Enter the newly created build directory’s path (say ‘opencv-build’) in ‘Where to build the binaries:’ text box.
  4. Check both ‘Grouped’ and ‘Advanced’ radio buttons in order to navigate options available in cmake-gui easily.
  5. For the moment disable IPP by unchecking the WITH_IPP under the WITH category (otherwise ‘cannot find -lRunTmChk’ error is expected when building OpenCV)
  6. Click ‘Configure’. In the popping window, select MinGW option under ‘Specify the generator for this project’ and click ‘Finish’.
  7. Wait until the configuration is done and then click ‘Generate’.Now you have properly configured OpenCV and check whether you have cvconfig.h inside the ‘opencv-build’ folder.
cmake-gui

CMake GUI configuration setup

MinGW Compilation

  1. Before building the project it is advised to uncomment ‘add_extra_compiler_option(-Werror=non-virtual-dtor)’ option in ‘<opencv-source>\cmake\OpenCVCompilerOptions.cmake’ file in order to avoid errors related to ‘[-Werror=non-virtual-dtor]’ while mingw building process.
  2. Go to <mingw-dir>/msys/1.0 and run ‘msys’ bash file.
  3. Navigate to <opencv-build> path and run ‘mingw32-make’ command.
  4. Run ‘mingw32-make install’ after build is successful.After successful building and installing of OpenCV go to <opencv-build> folder and copy generated files to respective destinations as follows.
  5. Copy two folders in ‘<opencv-build>\install\include’ to ‘<mingw-dir>\include’
  6. Copy files in ‘<opencv-build>\install\x86\mingw\bin’ to ‘C:\Windows\SysWOW64’ if your system is 64-bit, otherwise ‘C:\Windows\System32’
  7. Copy files in ‘<opencv-build>\install\x86\mingw\lib’ to ‘<mingw-dir>\lib’
msys

OpenCV modules are compiled using MinGW

OpenCV Extra Modules

Additionally if you need extra opencv modules such as Text, Face, etc. modules categorized under separate repository follow the below steps.

extra module

Specify OpenCV Extra Modules path

  1. Unzip ‘opencv_contrib’ modules zip file at a desired path (say ‘opencv_contrib_path’).
  2. Again open ‘cmake-gui’ and select ‘OPENCV_EXTRA_MODULES_PATH’ under ‘OPENCV’ category and insert ‘<opencv_contrib_path>\modules’
  3. For the moment deselect ‘opencv_bioinspired’ module under the ‘BUILD’ category as it crashes unexpectedly.
  4. It seems some API mismatch in the system generates ‘<opencv_contrib_path>\opencv_contrib-master\modules\ximgproc\src\spar
    se_match_interpolators.cpp:171:52: error: ‘const class cv::_InputArray’ has no m
    ember named ‘isVector” error. Therefore try to ‘Configure’ and ‘Generate’ after commenting out the lines as follows.    CV_Assert( !from_points.empty() && //from_points.isVector() &&
    !to_points  .empty() && //to_points  .isVector() &&

 

These are the ways and resolutions I have made on the way to the successful OpenCV run on my Windows environment. If this helps you, I am quite happy with that or if you think I have done something silly, well, please point it out.