Table of Contents



635647700Computer Vision: Human Motion Detection12 November 2017Jackie DavisMichael LedetAdvisor: Cris KoutsougerasCollege of Computer Science and Industrial TechnologySoutheastern Louisiana University500 W University Ave, Hammond, LA 70402Table of ContentsAbstract............................................................................................................................................2Introduction…………………………………………………………………………………….….3Objective………… ……………………………………………………………………………..4-5Progress………………………………………………………………………………………....6-8Deliverables………………………………………………………………………………..…..9-10References …………………………………………………………………………………….....11AbstractThe concept for our project was to design a device that detects human presence using a small, inexpensive microcontroller with a camera and open-source computer vision libraries. The purpose of this project is to incorporate concepts from previous courses and to demonstrate skills gained at Southeastern Louisiana University.IntroductionSurveillance is becoming a more popular topic every day. Thinking about the protection of ourselves, our families, and our valuables is a daily occurrence for most of us. However, security options for home and office are not always affordable or convenient. Most of these security cameras need to have a human to watch the feed to detect any prohibited object or person. In contrast, automated systems can run up to thousands of dollars after equipment cost, installation and monthly fees for monitoring service charges. With the development of open-source computer vision libraries, it is possible to build a human-motion detection device on a budget cost. This is exactly what our solution does. Our design achieves human-motion detection for a fraction of the cost of most advanced security cameras. Utilizing the Raspberry Pi Zero model in junction with OpenCV, an open-source library for computer vision functions, allowed us to create a very small device that can detect human presence and automatically notify you. To complete this project, our team had to overcome a wide variety of challenges. It is of great importance to note that all our knowledge in Computer Vision concepts and Linux commands are self-taught. Additionally, all the environments that were used were all new to us including: the operating system on the Pi Zero, using the OpenCV library, and using Python for programming. Gaining knowledge of Linux systems, including commands for operation and control, was necessary to complete this project. This was the primary requirement for working in the operating system on the Raspberry Pi Zero: Debian 8 (commonly referred to as “Jessie”). Additionally, to use the OpenCV library, we had to learn the fundamentals of Computer puter vision is a very specific field of computing in which devices are made to process and analyze digital images and videos. In short, these devices are trying to automate tasks that the human visual system can accomplish. Some examples of this are video tracking and object recognition. In our case, we focused on learning about human-motion detection. ObjectiveThe objective is to make an optical, motion detecting device that is small, efficient, and cost-effective, using a single-chip microcontroller. The purpose of this project is to use the knowledge we have gained to create a low-cost motion detection device that will detect human presence. The components of our project are a Raspberry Pi Zero W, a Raspberry Pi camera module, and a 16GB microSD card that serves as the mounting point for the Zero’s operating system. After running into several issues regarding various operating systems, our team eventually settles on using Debian 8: Jessie Lite. Jessie is one version behind the most recent version of the Debian operating system. The lite designates that this edition of Jessie contains no graphical interface and is controlled purely through the command line interface. The key to creating this project was using OpenCV, an open-source computer vision library. This library has many different features regarding real-time computer vision applications, such as: facial recognition, object identification, motion tracking, and augmented reality. OpenCV is written in C++, but has bindings in Python, Java, and MATLAB.376237511709400037623753999865Figure 1. Raspberry Pi Zero W, Camera module and microSD card. This is the base of our system.Figure 1. Raspberry Pi Zero W, Camera module and microSD card. This is the base of our system.As far as responsibilities are concerned, we have divided the work equally amongst ourselves. Jacaqueta will oversee practical interfacing of the raspberry pi zero and learning its capabilities. She will also oversee capturing test images using the camera module with and without OpenCV. Michael will oversee deciding on the optical sensor for the raspberry pi, and learning the capabilities and limitations of the camera. He will also oversee installing all the dependencies for the OpenCV library compilation. Lastly, he will oversee setting up the live feed for the camera. Most of the other responsibilities will be shared. As a team, we came to a decision on what microcontroller to use. Ultimately, it was decided that using the Raspberry Pi Zero W in junction with OpenCV was the best option to complete the project and this decision was made unanimously. Furthermore, interfacing with the optical sensor was done by both team members. Obviously, we both needed to know how the Raspberry Pi Zero W works in unison with the Pi camera, since this is the root of our entire project. Together we could successfully install OpenCV, program for motion detection, and optimize our system. Progression30480003166745Figure 2. Raspberry Pi Zero W modelFigure 2. Raspberry Pi Zero W model3048000914400After laying out the concept of our project, we needed to decide on the materials to use. The first objective was to decide on what microprocessor and optical sensor we wanted to use. It was important that we decided on these two simultaneously, because they needed to be compatible with each other. After a great deal of time researching all our options, we had narrowed it down to three, which all had similar features. These features are embedded in the underlying concept of our project: small, cost effective, and sufficient. Originally, we decided to go with the Adafruit Trinket-Mini and an Adafruit mini-spy cam, mainly because they were the most cost efficient for the project. However, they did not have the computational power we needed to process images. Ultimately, we decided on the Raspberry Pi Zero W (figure 1) due to its higher processing power and adaptability.The Raspberry Pi Zero W, as with all Raspberry Pi systems, was running a lot of programs and background processes we did not have use for. We uninstalled the Debian 9: Stretch operating system and install the older Debian 8: Jessie Lite operating system. Jessie Lite only runs command line, python 2.7 or 3, and wireless network connectivity. This cut the amount of RAM used to process basic functions by 82%. The Pi Zero W has a total weight of 0.3 ounces and small dimensions of 2.6 in x 1.2 in x 0.2 in. It also has 512MB of RAM and single-core 1GHz processor chip costing only $10.00. Ultimately, for being such a small component, this micro-controller really packages a lot of power and features. In comparison to the much bigger Raspberry Pi 3, the Pi Zero W was an excellent choice for this project. The Pi 3, costing $35.00, does not have the built in wireless networking capabilities that the Pi Zero W has. It weighs 1.58 ounces and has dimensions of 3.4 in x 2.2 in x 0.7 in and is about three times the size of the Pi Zero W. The Pi 3 has a quad-core 1.2 GHz processor chip and 1 GB of RAM. It has HDMI and USB hook-ups whereas the Pi Zero W has mini-HDMI and micro-USB hook-ups. The Wi-Fi adapter for the Pi 3 costs anywhere from $5.00 to $25.00. Both microcontrollers have a micro-SD card slot, micro-USB power source, and a videoCore IV GPU.There are several reasons why this microcontroller was chosen over other potential candidates. Firstly, the controller is simple in design and smaller than most of the other microcontrollers we evaluated, however, it was significantly bigger than the Trinket-Mini we originally planned to use. This controller had enough functionality to work within the parameters of our concept design. It also had enough computing power to deal with the image analysis, which is the root of our project, yet the controller is not so powerful that it would be considered overkill. Furthermore, this specific controller is very compatible with the optical sensor that was chosen. The dimensions of the casing for the sensor is 0.98 in x 0.90 in x 0.35 in. The total weight of the sensor is 3.4 grams. It has a high-resolution module with up to 1080p video quality and up to 3280 x 2464 pixels for photos costing $21.00 bringing our overall project cost to $31.00. The purpose for this optical sensor was to record video to be analyzed using the OpenCV installed on our Pi Zero W. Overall, and like our chosen microcontroller, this camera has a decent variety of options for such a small package.34353502419350Figure 3. Raspberry Pi Camera v2.1Figure 3. Raspberry Pi Camera v2.13425825622300There are several reasons why we ultimately decided to go with this optical sensor as opposed to the other potential candidates. Firstly, it was the smallest and the most flexible. It can be arranged in a variety of different ways and it can fit into a very small space which is perfect for the parameters of our project. Also, it is very simplistic which is good for us because it typically equates to maximum control through our own program design. It uses JPEG, which is a common and simple file format, to store its photos as well as raw video feed. This makes it great for our team because we will be able to design a program, with much more available resources as compared to if the optical sensor used a different format. Next, we were ready to begin working with the Zero and Pi camera. The first thing we needed to do on the zero was to expand the filesystem to use all the available memory on the microSD card. This was done by accessing the main configuration menu via the command: sudo raspi-config. In this command, sudo is our elevated privileges, like “run as administrator” in windows. Next, we enable the pi camera module which is located within the “interfacing options” of the configuration menu. From here, we wanted to test the camera module before proceeding any further. Using the built-in commands for the camera module on the raspberry pi, we could capture some images. The command that was used for image capture is: sudo raspistill -o image01.jpg, where image01.jpg is the name of the picture file. The picture file format is jpeg for this process. Before expanding the filesystem, however, we had to format the SD card and wipe any data from it that was not related to our raspberry pi or python. To complete this, we needed a micro-SD to SD card converter in order to connect the micro-SD card to a computer. From there, we the standard SD card editor provided by Windows to format the SD card to FAT32. Then, we used a program called Windows Flash Tool to write the .image file containing the Jessie Lite OS to the SD card for use on the Pi Zero W. Once the OS was installed onto the SD card, we booted the Pi Zero W and installed all the updates and upgrades we were missing using the commands “sudo apt-get install update” and “sudo apt-get install upgrade”, respectively, into the command line.The next objective was to begin prepping the system for the OpenCV installation. OpenCV has a ton of dependencies, which means the setup must be very precise or failure will ensue. The required packages for OpenCV are as follows:GCC 4.4.x or laterCMake 2.6 or higherGitGTK+2.x or higher, including headers (libgtk2.0-dev)pkg-configPython 2.6 or later and Numpy 1.5 or later with developer packages (python-dev, python-numpy)ffmpeg or libav development packages: libavcodec-dev, libavformat-dev, libswscale-dev[optional] libtbb2 libtbb-dev[optional] libdc1394 2.x[optional] libjpeg-dev, libpng-dev, libtiff-dev, libjasper-dev, libdc1394-22-devIt is worth noting that these can be installed using the basic sudo apt-get update/upgrade commands available by default on the raspberry pi. Additionally, it is equally important to recognize that all this software is open-source and free for anyone to use for educational and personal usage. We can start at the top by talking about GCC. GCC is the GNU Compiler Collection. A compiler is software that allows the source code that was written by a programmer, to be translated into assembly or machine language. The GCC collection contains compiler for C, C++, Objective C, Fortran, Java, and Ada programming languages. In our case, this is necessary because OpenCV was design in the C++ language. Therefore, we will need this to compile the library, as well as the scripts we write for OpenCV. The next primary dependency for OpenCV is CMake. CMake is a cross-platform open-source software that handles the building process. The key feature of CMake is its ability to build a directory tree outside of the source tree. This means that we can delete builds without removing the source files. CMake will be used in junction with GCC creating a great starting point for installing and running OpenCV. The only required dependency for CMake is a C++ compiler, which is GCC in our case. Git is primarily used as a source code manager in software development, but it can be used to keep track of variations within any file system. GTK+, or the GIMP Toolkit, is a multi-platform toolkit used for creating graphical interfaces. Pkg-config is used for querying the system for installed libraries. It essentially tells the user what is on the system and if all dependencies are present. There are many different versions of Python, but OpenCV requires 2.6, or a newer version, to be installed. Python is a high-level programming language that is widely used for a variety of applications. People like to use it because it is user-friendly and uses a syntax that can accomplish multiple tasks within just a few lines. It is more efficient that the Java or C++ languages in this way. Python is the language in which our scripts for OpenCV programs will be written. Ultimately, we decided to go with Python 2.7 because it was much more lightweight than newer systems, yet it had the capabilities that we needed. Creating a lightweight environment was of critical importance because OpenCV is process intensive. It is a big job for the Pi Zero just to compile and install OpenCV. Similarly, we decided to install OpenCV 3.1 instead of 3.3.0 because the difference was significant in terms of size, yet the newer version did not contain anything extra that we needed. After covering all the dependencies for OpenCV, we were ready to begin installing them on our machine in order to prep for the OpenCV installation. We began by installing the developer tools: CMake, Git, and pkg-config. Then, we installed the image I/O packages that we thought we might use: libjpeg-dev, libtiff5-dev, libjasper-dev,libpng12-dev. These are all libraries that allows us to capture and manipulate different image formats. Likewise, we needed to install I/O packages for video: libavcodec-dev, libavformat-dev, libswscale-dev, libv4l-dev, libxvidcore-dev, libx264-dev. Next, we installed our GTK development library: libgtk2.0-dev. As stated previously, this is used for OpenCV’s GUI interface. Then, the Python 2.7 was installed. At this point, we were ready to grab the OpenCV source code from GitHub using:wget -O opencv.zip opencv.zipThen, we began to set up Python to be used for the OpenCV build. This required the installation of pip, a python package manager:wget python get-pip.pyOnce this was done, we needed to set up to create a virtual environment for our images and videos to be processed and fed. This Python virtual environment was installed to keep the dependencies in separate places by creating independent Python environments for each one: sudo pip install virtualenv virtualenvwrappersudo rm -rf ~/.cache/pipWe needed to update our main file, ~/.profile, to include all the libraries and files for the virtual environments as shown:# virtualenv and virtualenvwrapperexport WORKON_HOME=$HOME/.virtualenvssource /usr/local/bin/virtualenvwrapper.shecho -e "\n# virtualenv and virtualenvwrapper" >> ~/.profileecho "export WORKON_HOME=$HOME/.virtualenvs" >> ~/.profileecho "source /usr/local/bin/virtualenvwrapper.sh" >> ~/.profileNext, we reloaded the main file with the command source ~/.profile to make sure we were within it.Now, we would create the virtual computer vision environment by entering:mkvirtualenv cv -p python2.7If you see the (cv) preceding your prompt statements, you are now working in the cv virtual environment. If not, enter the following:source ~/.profileworkon cvSince we were now fully working in our cv virtual environment, we needed to install a Python dependency called NumPy which is the fundamental package for scientific computation in Python. We only entered the line:pip install numpyFrom here, we were in the final stages of the OpenCV installation where we installed the OpenCV. Up until this point, we were install dependencies that would support the OpenCV installation. We needed to make sure we were in the cv virtual environment (workon cv) otherwise the system would install incorrectly and crash. The process was as follows:cd ~/opencv-3.1.0/mkdir buildcd buildmake -D CMAKE_BUILD_TYPE=RELEASE \ -D CMAKE_INSTALL_PREFIX=/usr/local \ -D INSTALL_PYTHON_EXAMPLES=OFF \ -D OPENCV_EXTRA_MODULES_PATH=~/opencv_contrib-3.1.0/modules \ -D BUILD_EXAMPLES=ON ..The installation took about five to six hours, so time and patience was key during this portion of the installation. The very last step in the installation was to create a symbolic link with the OpenCV bindings and our cv virtual environment for Python 2.7 and then to test that it is installed correctly:cd ~/.virtualenvs/cv/lib/python2.7/site-packages/ln -s /usr/local/lib/python2.7/site-packages/cv2.so cv2.soAfter this, we exited the rebooted the system and entered the following code to test to make sure our OpenCV download was complete and correctly installed:316230020002500source ~/.profile workon cvpython>>> import cv2>>> cv2.__version__'3.1.0'>>>317182593345Figure 5. Confirming OpenCV installationFigure 5. Confirming OpenCV installationIt took three tries to complete the installation. We got a fail error at 82%, twice after nineteen hours, which stopped the installation. After running into several issues regarding various operating systems, our team eventually settles on using Debian 8: Jessie Lite. Jessie is one version behind the most recent version of the Debian operating system. The lite designates that this edition of Jessie contains no graphical interface and is controlled purely through the command line interface. Figure 6. Installing OpenCVHowever, this solution was not apparent at first. In fact, we went through about a month worth of trails before figuring out the solution to our problems. One of our failed trials included trying to VisualGDB, a cross-platform compiler. In this effort, we had planned to install the VisualGDB add-ons within Visual Studios. The goal was to compile OpenCV 3.1.0 on a standard windows operating system and then place the binaries onto the Pi Zero. After several fruitless tries in this environment, we decided to try other solutions. Another one of our failed trials includes trying to compile various versions of OpenCV on the original operating system: Stretch. The idea here was to free up as much of the system as possible to devote for the compilation of OpenCV. This meant minimizing all our other processes and freeing up memory. These trials were of course failures though because the variable that we were overlooking was the operating system itself. However, after searching for many potential solutions, our team came to realize that running a lighter version of the operating system could be the potential solution. This is how we stumbled across Debian 8: Jessie Lite.344805010541000It is critically important to note that this error that we were running into was during the compilation step, and not the installation step. This essentially allowed for the operating system to be less taxing on the physical system, in comparison to a heavier operating system like Debian 9: Stretch. With the removal of the GUI, in the lite versions of the operating systems, we could minimize the footprint of our entire system. The “lite” version of Debian comes as barebones as possible in terms of software. This allows us to start with a clean-slate in an operating system with a minimized footprint. 3457575521335Figure 7. Successfully compiling OpenCVFigure 7. Successfully compiling OpenCVDeliverablesDescriptionStartFinishResponsibilityStatusFinish Research and decide on a specific microcontrollerAugustSeptember Jacaqueta, MichaelCompletePractical interfacing with Raspberry Pi Zero; learning its capabilities and limitationsAugustSeptemberJacaquetaCompleteDecide on Optical Sensor for Motion DetectionAugustSeptember MichaelCompleteInterfacing with Optical SensorSeptemberOctoberJacaqueta, MichaelCompleteSuccessfully capture images using Raspberry Pi Zero and CameraSeptemberOctober JacaquetaCompleteInstall dependencies and prep system for OpenCV compilationSeptemberOctoberMichaelCompleteSuccessfully complete installation of OpenCVOctoberNovemberJacaqueta, MichaelCompleteTest Camera interfacing options through OpenCVOctoberNovemberJacaquetaCompleteEstablish live feed for video streamOctoberNovemberMichaelCompleteProgramming for Motion DetectionNovemberDecemberJacaqueta, Michael40%Finalize code and optimize systemNovemberDecemberJacaqueta, Michael60%ReferencesAdafruit: 2.7: Requirements for OpenCV: Pi Forums: Pi Model Comparison: Documentation: ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download