Deep learning-based image processing embedded on C#

Deep learning-based image processing embedded on C# .NET

Cheska Abarro1, Angela Caliwag2, Donguk Kwon, Wansu Lim3 Department of Aeronautics, Mechanical and Electronic Convergence Engineering

Kumoh National Institute of Technology, Gumi, South Korea 1cheskaabarro@kumoh.ac.kr, 2a_caliwag@kumoh.ac.kr, 3wansu.lim@kumoh.ac.kr

Abstract--Python is widely used for its large open source library for image processing due to its growing popularity as a scientific programming language and free availability of many state-of-the-art image processing tools in the ecosystem. Meanwhile, C# .NET is known for its flexibility in different platforms. It is also one of the commonly used object-oriented programming language. In this paper, we combined the best of python's image processing and the flexibility of C# application by using Dynamic Link Library (dll). We developed a simple C# windows form application that takes an image as an input, sends it to python for image processing using OpenCV with applied MTCNN and displaying it back as an output. By using this method, we are able to get the best of both programming languages.

Keywords--dll, c# .net, python, image processing, Open Source Computer Vision Library (OpenCV), multi-task cascaded convolutional networks (MTCNN)

I. INTRODUCTION

We live in a time where images contain lots of information which is sometimes difficult to obtain. This is why image processing has become a highly valuable skill, applicable in many cases. Python is widely used for its large open source library for image processing due to its growing popularity as a scientific programming language and free availability of many state-of-the-art image processing tools in the ecosystem. One of the best and commonly used open source library available in python is the Open Source Computer Vision Library (OpenCV). To give an example, [1] stated that in some social networks, such as Facebook, faces are automatically detected but in other cases, there are applications that allow us to alter the original colors of an image by applying different filters.

While applying different filters using OpenCV may be helpful, applying multi-task cascaded convolutional networks (MTCNN) to a processed image can enhance the image to be more of a data because MTCNN support both face detection and alignment for multiple faces [2][3]. Thus, making the image more a significant part of a data in some application like in thermal camera images and such.

But all of that said, Python is dynamically typed. This means that you don't declare the type of variable while writing the code which led to run-time error. Meanwhile, C# .NET may lack in terms of image processing but is widely known for its flexibility in different platforms. It is also one of the commonly used object-oriented programming language. Python's flexibility when it comes to data manipulation is one of the great factors but Python's large open library source and flexibility in image processing is the prime candidate why we want to embed our python code to our .NET application. This interoperability opens up possibilities to use each language that is best suited for without having to convert the whole application into one or another.

In this paper, we combined the best of python's image processing and the flexibility of C# application using Dynamic Link Library (dll). We developed a simple C# windows form application that takes an image as an input, sends it to python for image processing using OpenCV and apply MTCNN then Python pass it back to C# to display it as an output. By using this method, we are able to get the best of both programming languages.

II. SOFTWARE DESIGN The following are the software platforms needed.

A. Windows Form Application using C# .NET C# is an object-oriented programming language that can

run on different platforms such as windows, android, apple, etc. While C# is a programming language, .NET is the framework that the language is built on.

We created a simple C# .NET windows application that takes an image as an input. In this application, takes an image as an input and process it using Dynamic Link Library (dll)based python. Using the overall system flow shown in Fig. 1, after choosing an image, the C# will convert the image to base64. Base64 will be sent to Python using Dynamic Link Library(dll) for processing.

B. Image Processing using Python OpenCV Python is widely used for its large open source library for

image processing. The most commonly used open source library for image processing in python is Open Source Computer Vision Library (OpenCV). OpenCV is a library of Python bindings designed to solved computer vision problems. OpenCV is a popular vision library in the development of computer vision applications. It consists of a large number of C functions and a small number of C++ classes and integrates a large number of general algorithms for digital image processing, which can quickly solve digital image processing problems [4]. We embed the python in our C# application for durability and flexibility in terms of processing image. As shown in Fig. 2, after receiving base64 as an input from C#, the python will convert the base64 to image in able to process the image using OpenCV in python.

Fig. 1. Overall system flow of embedded C# and Python.

Fig. 2. Step by step procedure of overall system flow of embedded C# and Python.

C. Multi-Task Cascaded Convolutional Networks (MTCNN) Multi-Task Cascaded Convolution Networks (MTCNN)

[5] is a popular CNN-based cascade face detector and one of the state-of-the-art methods of the face detection dataset [2]. [2] MTCNN is the first work that exploits the relevance between face detection and alignment tasks, which can simultaneously detect face and facial landmarks' localization and enhance their accuracy. MTCNN is an algorithm consists of three stages, which detects the bounding boxes of faces in an image along with their 5-point face landmarks. We use MTCNN as part of our trial and error stage for image processing.

Fig. 3. The colored image (left) is the original image from C# Windows. The gray scaled image (right) is the image processed by Python.

D. Dynamic Link Library (DLL) Dynamic Link Library (DLL) is Microsoft's

implementation of the shared library concept. DLL can contain code, data, and resources in any combination. Although DLL can export data, it is generally used only by its functions. DLLs provide a way to modularize applications so that their functionality can be updated and reused more easily. DLLs also help reduce memory overhead when several applications use the same functionality at the same time, because although each application receives its own copy of the DLL data, the applications share the DLL code. DLL is usually compiled in C++ but we have compiled our DLL using C# .NET. DLL file can be open or imported in other programming languages like C#, C++ and Java.

By using method of combining two programming languages and platform using Dynamic Link Library (dll), we get the best of python in terms of image processing by acquiring its huge open source library for image processing and also acquiring the flexibility of C# .NET application.

III. SIMULATIONS RESULTS AND DISCUSSION

To test our method, we used the windows form application created using C# .Net as part of trial and error to illustrate that Python and C# DLL-based embedding is successful. In our windows form application, we consider an image as an input. The C# will convert the image to base64 string which will then be pass to python. When Python received the base64 string, python will convert it to image. After converting it to image, python will process the image to grayscale. Then, MTCNN will be applied. When the image processing and MTCNN applying is done, python will convert the processed image to base64 string and return it to C#. C# windows form application will then display the processed output as image. In Fig. 3, the colored image is the image input by C#. After passing it to python for processing, the colored image will be displayed as gray scaled with MTCNN applied.

IV. CONCLUSION AND FUTURE WORKS

We have simulated the Python and C# DLL-based embedding by developing a windows from application that takes an image as an input and be processed by Python. The simulated method can be furthered studied and applied for deep learning and other open source image processing libraries available for the said programming languages. This method can also be used if either Python and C# limits your coding features. By using this method, we have simulated that we can acquire the best of both programming languages without converting the whole code. Future works may refer to the application of the simulated method to real time video. This paper only simulated with images. Researchers can also further study the use of dll not just in Python and C# but also in other programming languages such as C++/C, Java, etc.

ACKNOWLEDGMENT

This work was supported by the Ministry of SMEs and Startups, S. Korea (S2829065, S3010704), and by the National Research Foundation of Korea (2020R1A4A101777511).

REFERENCES

[1] Jose Sigut, Miguel Castro, Rafael Arnay, and Marta Sigut "OpenCV Basics: A Mobile Application to Support the Teaching of Computer Vsion Concepts" IEEE Transactions on Education, Vol. 63, No:4, November 2020

[2] Huiyu Mo , Leibo Liu ,Wenping Zhu , Qiang Li, Hong Liu, Shouyi Yin, and Shaojun Wei "A Multi-Task Hardwired Accelerator for Face Detection and Alignment" IEEE Transactions on Circuits and Systems for Video Technology, Vol. 30, No:11, November 2020

[3] Xiaochao Li, Zhenjie Yang and Hongwei Wu "Face Detection Based on Receptive Field Multi-Task Cascaded Convlutional Nueral Networks" IEEE Access Digital Object Identifier 10.1109/ACCESS.2020.3023782 October 5, 2020

[4] Jie Huang, Anmin Huang and Liming Wang "Intelligent Video Surveillance of Tourist Attractions Based on Virtual Reality Technology" IEEE Access Digital Object Identifier 10.1109/ACCESS.2020.3020637 September 11, 2020

[5] Baosheng Yu, and Dacheng Tao "Anchor Cascade for Efficient Face Detection" IEEE Transaction on Image Processing, Vol. 28, No:5, May 2019

 ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download