Paper Title (use style: paper title)



INTERACTIVE HOLOGRAM: TRANSFORMING EDUCATION1Minal Acharya, 2Priti Chouhan, 3Saipranavi Kundaram, 4Asmita Deshmukh1,2,3B.E., 4Asst. ProfessorDept. of Computer Engineering, K.C. College of Engineering & Management Studies & Research, Thane, Maharashtra, India1minalacharya4598@, 2chouhanpriti1407@, 3saipranavikundaram8@, 4asmita.deshmukh@kccemsr.edu.in Abstract— The field of education has had a drastic change when it comes to teaching. However, no matter how much one tries, it is difficult to explain many concepts in the field of science, geography, etc., without a diagrammatic view. But even with these diagrams, learning is much easier to understand when the actual object is present right in front of student’s eyes. Making this possible is not feasible for every concept. That’s when hologram steps in. To engross students in the process of learning, we make it interactive. This paper describes a way to build such an interactive hologram and apply it in the of education. However, its applications are wide -from museums to zoos to planetariums. The designing process consists of several steps that include image duplication, object detection, hand detection and speech synthesis, which collectively make the hologram interactive. A holographic image is built by placing an upright prism on a screen using Pepper’s Ghost technique. For object detection, it uses TensorFlow Object Detection API on top of Keras to train models. The object detection algorithm used here is YoloV3. Furthermore, an object detected in hologram by object detection API is made interactive by integrating trained hand recognition models. The detection of the user’s hand is used as a cue to convert the detected object text into voice responses. This paper aims to enhance the way of teaching and encourage students to learn more with fun.Index Terms—3D view, Interactive, Hologram, Educational tool, hand Detection, object detection, speech synthesis, machine learning, Human Computer Interaction, Computer Vision________________________________________________________________________________________________________Introduction Children and teens have grown up with technology, easily available to them. With technology, we have always tried to improve on representations, visuals, graphics, etc. to send a clearer message. Like when blackboards were aided with charts and handmade models. Then this was further improved with the use of animations and projector. Now, holographs have the potential to replace these projectors. The hologram allows the viewer to get a 360° view of any entity. This makes understanding easier and provides an advanced level of representation. Making it interactive ensures the involvement of the user, the targeted users being students. With the addition of a new dimension to the traditional blackboard or projected images, a deeper level of understanding can be achieved. In spite of its advantages, the impact of 3-dimensional representation on the learning process is still not explored. We all have surely seen the holograms sci-fi movies use to give off a futuristic impression. However, we no longer believe we are far off from achieving this technology. That is, precisely, what this paper aims to do. The concept of the interactive hologram has many applications in different fields. For example, in museums, where the need of a guide can be eliminated using an interactive hologram. This can be used in business models and plans that require 3D representation for better understanding. It can be deployed in corporate presentations as well. Even in planetariums, this concept can be used to display outer space entities. This paper discusses designing an interactive 3D hologram from a sole image. The next section describes the prior work done on interactive hologram using different technologies. Later, we describe the building process of the interactive hologram and its achievement in transforming education. Lastly, the result and analysis of the implemented work and its future implications are discussed.Literature Survey The hologram has been progressively added into every field to increase the effectiveness in understanding so that things get simple. The idea of an interactive hologram was introduced as a supplementary educational tool. So far, the most prominent work is done using an SVM model to detect the position of a hand, detect the color on that position and match it with preset data. For example, on a globe, if the hand position detects red color and red is color code for Asia, it will synthesize ‘Asia’ as output[8]. To understand which detection algorithm was the best, it was necessary to understand optimizing conditions for each one. Ultimately, YoloV3 was the most apt choice. It is a regression-based algorithm and it takes a full image in one evaluation. Since, feature extraction is not a part of this paper yet, with YoloV3, the accuracy levels were easy to attain. Compared to state-of-the-art detection systems, YoloV3 makes more localization errors but is less likely to predict false detections where nothing exists. It increases the accuracy of object detection[14]. Earlier, classification-based models, region-based classification, or sliding windows were used to detect objects. Only high scoring regions of the image are considered for detection as they were time time-consuming. After determining the position of the objects detected, the user sends it as a text string to pyttsx [13]. Object Detection is widely used in different applications such as detecting vehicles, face detection, autonomous vehicles, etc. Firstly, training the model on train images and evaluation of model using test performances is done. Next, images are labelled and these labels are stored into the record file format. After the generation of file format, a label map with a unique id is created. Once everything is initialized, training of the model based on system configuration is done simultaneously after each phase loss is reported. As the training method progresses, it gets lower[12]. MethodologyProject SetupThe setup consists of a laptop which has the trained object detection model. An upside-down prism is placed on the screen of this laptop which displays the holographic projection of an image. A webcam is used to detect a hand that triggers speech synthesis and converts detected objects from text to speech.Figure 1. Model Set-upWorkingThe user uploads an image to the front end of the system. This uploaded image parallelly goes to an OpenCV program and in the back end, it goes to the object detector that is trained on Tensorflow on top of Keras. The OpenCV program converts a normal image (Fig.1(a)) into a duplicated image (Fig.1(b)) that will be used for displaying the hologram. In the backend, Tensorflow uses the uploaded image to perform object detection and saves the output in a string. Until the user does not place his or her hand in the frame, this output remains as it is. However, the moment the user places his or her hand in the specified frame of web camera, output of object detection is converted into speech format. This is how we make it interactive. There are several detection algorithms that can be used. For example, faster R CNN, Single Shot Detector, Yolo V3, etc. However, this particular project uses Yolo V3 for very obvious reasons. Firstly, the speed of Yolo V3 is relatively faster than R CNN and SSD, but it is known to have some overlapping predictions. Also, this model affords to lose some accuracy in exchange for speed because there is no feature extraction from detected objects. However, in future if the model is used to extract features from detected objects, then we might have to lose some speed. For example, if we want to do feature extraction, then we should focus more on getting higher accuracy as opposed to faster detection. However, if we want to take videos as input along with images, then we will have to focus more on the speed than the accuracy. Implementation The following results are of an input image with ‘Earth’ as the target object. Just like this, the model can be run on many other input images like Saturn, brain, lungs, apple, orange, etc. The intention was to cover up the basic curriculum of kindergarten. In future, we plan on expanding this model to higher grades while increasing the accuracy and focusing on feature extraction as well, since the images are likely to be more intricate and complicated.The dataset to train this model was collected from Google’s dataset library. An approximate time analysis -OperationTime taken(in seconds)/Image ?Object Detection0.5744Hand Detection0.3562Speech synthesis0.3117Table 1AssumptionsImage used is stored locallyEncoding/Decoding technique used - base64python version - 3libraries installed - cv2, numpy, datetimemodules installed - base64, requests, jsonTable 2Figure 2. Input Image given to a holoquad.py code as well as to the object detection folderFigure 3. Conversion of input image to holographic image which will show us holographic projection when the prism is kept in middleFigure 4. Holographic display of an imageFigure 5. Object is detected of an input imageFigure 6. When the hand is detected the text, output is converted into speech outputOnce a hand is detected, the output which is stored in a string format gets converted into speech format using python’s pyttsx3 library. In future, we would integrate gesture recognition to automate the control of image operations such as zoom in,zoom out. rotate, slide, etc.Conclusion The aim of the project was to make the hologram interactive which will assist in understanding concepts while making the learning process fun. The hologram was created through a simple Pepper Ghost’s technique. This paper made the hologram interactive by using object detection with the Tensor flow and YOLOv3 algorithm, hand detection using computer vision, and speech synthesis using python’s pyttsx3 library. The paper also implemented OpenCV’s grab cut and duplication operations to achieve an image (Fig.3) required for holographic display. Students in schools can use this interactive hologram concept to explore various objects with their hand movements. It makes education much more engaging and interesting. However, interactive hologram still has some limitations. The display of hologram using Pepper Ghost’s technique requires a low light environment in order to display hologram clearly. However, if the lighting approaches a darker setting, the hand detection accuracy goes down as most of the web cameras right now do not have a standard set for low light imaging. The annotated text of a detected object will convert into voice response only when the user’s hand is in a specified frame of the web camera, The YoloV3 algorithm will lose some speed if it goes for feature extraction of an object. The future work of this project is to increase the accuracy and precision of the hand in lower light settings. We could also integrate gesture recognition to automate the control of image parameters. It is also possible to scrape out a few lines on the detected object using web crawling and synthesize that into speech. This project is a small-scale implementation but can be broadened into many fields. The technology of interactive hologram can be applicable in many fields such as in museums, to explain a business model and plans, for corporate presentations, etc. For a long time, interactive holograms were a fictional concept that portrayed a futuristic world. However, unlike flying cars, the world of computer vision is not far off from achieving this technology.ReferencesJiono Mahfud, Takafumi Matsumaru, “Interactive aerial projection of 3D hologram object” 2016 IEEE International Conference on Robotics and Biomimetics (ROBIO)Abdullah H. Awad, Faten F. Kharbat, “The first design of a smart hologram for teaching” 2018 Advances in Science and Engineering Technology International Conferences (ASET)Fabrizia Bovier; Giuseppe Caggianese; Giuseppe De Pietro; Luigi Gallo; Pietro Neroni , “An Interactive 3D Holographic Pyramid for Museum Exhibition” 2016 12th International Conference on Signal-Image Technology & Internet-Based Systems (SITIS)Chan Vei Siang; Muhammad Ismail Mat Isham; Farhan Mohamed; Yusman Azimi Yusoff; “Interactive Holographic Application using Augmented Reality EduCard and 3D Holographic Pyramid for Interactive and Immersive Learning”Fares Jalled, “Object Detection Using Image Processing”,2016 Moscow Institute of Physics & Technology, Department of Radio Engineering & Cybernetics Ilia Voronkov, Moscow Institute of Physics & Technology, Department of Radio Engineering & CyberneticsMd. T. Akhtar1, S. T. Razi, K. N. Jaman, A. Azimusshan, Md. A. Sohel, “Fast and Real Life Object Detection System Using Simple Webcam” 2018 1 Cyber Patrol Cell, Kolkata Police Directorate, Kolkata, India IT, Logicoy Software Technology, Bangalore, India Dept of CSE, Aliah University, Kolkata, Indian Staffing & Recruitment, Test Yantra Software Solutions, Bangalore, Indian IT, Tata Consultancy Services, Kolkata, IndiaStanley Bileschi and Lior Wolf, “A Unified System For Object Detection, Texture Recognition, and Context Analysis Based on the Standard Model Feature Set” The Center for Biological and Computational Learning,Massachusetts Institute of Technology, Cambridge, MA 02138.Zeenat AlKassim, “Building an Interactive 3D Hologram by SVM: Revolutionizing Educational Systems” 2017 Electrical Engineering Department, United Arab Emirates University AlAin, UAE., B. N. K., & Sasikala, T. “Object Detection and Count of Objects in Image using TensorFlow Object Detection API”. 2019 International Conference on Smart Systems and Inventive Technology (ICSSIT). Redmon, Santosh Divvala, Ross Girshick, Ali Farhadi University of Washington “You Only Look Once: Unified, Real-Time Object Detection”. IEEE Conference on Computer Vision and Pattern Recognition (CVPR). ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download