Introduction .com



Contents TOC \o "1-2" \h \z \u Introduction PAGEREF _Toc22629864 \h 2Use Case PAGEREF _Toc22629865 \h 2Steps Required PAGEREF _Toc22629866 \h 2Current Project Design PAGEREF _Toc22629867 \h 3Step 1: PAGEREF _Toc22629868 \h 3Step 3: PAGEREF _Toc22629869 \h 3Step 4: PAGEREF _Toc22629870 \h 3Azure Tools Used PAGEREF _Toc22629871 \h 3MS Azure Tools PAGEREF _Toc22629872 \h 4Tool 1: Azure Functions Serverless compute PAGEREF _Toc22629873 \h 4Tool 2: Video Indexer PAGEREF _Toc22629874 \h 5Tool 3: Custom Vision (Cognitive Services) PAGEREF _Toc22629875 \h 6Tool 4: Machine Learning Service/Studio PAGEREF _Toc22629876 \h 7Tool 5: Databricks PAGEREF _Toc22629877 \h 8Tool 6: Windows Data Science Virtual Machine PAGEREF _Toc22629878 \h 9Tool 7: Storage Explorer and BLOB storage PAGEREF _Toc22629879 \h 10Tool 8: Logic Apps PAGEREF _Toc22629880 \h 12Tool 9: Data lake Storage PAGEREF _Toc22629881 \h 13Google Cloud Services PAGEREF _Toc22629882 \h 14AutoML Video Intelligence PAGEREF _Toc22629883 \h 14GCS Costs PAGEREF _Toc22629884 \h 14Summary PAGEREF _Toc22629885 \h 15IntroductionSchool rugby coaches have matches recorded and use the recordings to playback and study events in the matches. These allow the coaches to feedback to the players good plays, improvements, errors, etc. and assists with coaching.Use CaseA user records a game. The recording is submitted to our client and the recording is reviewed and specific events, e.g. Kick offs, scums and line outs, are tagged and their start and end time recorded. Currently this is done manually by a human which has time, cost and scalability implications.We see this as a potential use case for a machine learning model. The user will upload the recording to a service (possibly via a web app), based on the MS Azure cloud platform. The service will process the recording and produce a report with the same output as the human generated form. Potentially this service will be more cost effective than the manual version and will be more easily scalable.Steps RequiredConvert the video to images (frames) with a timestamp metadata, options:A third-party online video-jpg converterA locally run C# windows form appA Php/JScript function on a web appA serverless function which is triggered when a video is uploaded to storageUpload the images to MS Azure storageAnalyze and tag the imagesFind the first image in a tagged sequenceFind the last image in the sequenceOutput a list of the first images and last images - the tag and the associated timestampCurrent Project DesignStep 1: I have built a C# Windows App which can be installed on any local windows computer.This allows the user to select a video saved on their computer and uploads the video to an Azure storage Account.Step 2:A serverless Azure Function App is triggered by the arrival of the video in the blob. The Function generates images from a video frame at a rate of 1 per second of video. The images are saved in another blob store.The original video is deleted to save storage costs.Step 3:A second Azure serverless function is triggered by the Logic App on completion of step 2.This function sends the images to the Custom Vision project and the image is classified.This function also builds an Xml file with the predictions from Custom Vision and store this in a Documents Blob.The images are deleted.Step 4:The Xml documented is sent to my Email address by the final action of the logic App.Azure Tools UsedLogic AppFunction AppCustom VisionBlob storageMS Azure ToolsThe information in this report was obtained from the Microsoft Azure website. CITATION Mic19 \l 1033 (Microsoft, n.d.)Tool 1: Azure Functions Serverless computeDescriptionA fully managed service which frees us from provisioning servers to run code. The code is executed based on an event, e.g. a video upload to storage, the code executes, e.g. convert the video to images.Charges are based on the time the code executes.A potential solution to convert videos to static images and to analyze the images with custom Vision.RelationshipsCan respond to and be triggered various events, e.g. a video uploaded to a storage “bucket”.LanguageThe code is in the language of your choice.PricingPriceFree Grant (per month)Execution Time (Resource Consumption)$0.000016/GB-s400 000 GB-sTotal Executions$0.20 per million1 millionResource consumption is calculated in GB-s, by multiplying average memory size used by the time in milliseconds that the code takes to execute.Tool 2: Video IndexerDescriptionUses media Artificial Intelligence (AI) technology to extract insights and content discovery from videos.Includes:Object identification: Automatically identify and label objects (for example: scrums. lineouts and kickoffs) when they appearShot detection: Detect when a shot starts and ends, based on visual analysisSearch: Video Indexer enables extraction of human understandable, time-stamped metadata and allows building search indexes based on that metadataUnfortunately, it is not customisable to enable labeling of custom objects.Not useful for this project due to limited labeling.RelationshipsNo integration or relationship to other Tools, essentially a standalone solution.Language and User Interface (UI)Web based UI which allows user to upload a video from a URL or a locally stored video. The app then indexes the videoPricingNo upfront costs, pricing is per minute of content ($015/min).Establish ArchitectureN/A as we cannot customize to our labelling requirements.Tool 3: Custom Vision (Cognitive Services)DescriptionAzure Custom Vision is a service that lets you build, deploy, and improve your own image classifiers without writing code. An image classifier is an AI service that applies labels to images, according to their visual characteristics. Custom Vision allows you to determine the labels to apply.Custom vision uses user submitted labeled images with and without the required characteristic to train the algorithm.Features:Image classification: labels an imageObject detection: the co-ordinates of the labeled image (great for multiple similar objects in one image)Custom vision labels images that are uploaded, so we don’t need a labeled dataset to train the model.A potential tool to use to label the images.RelationshipsNoneLanguage and UIThe Custom Vision Service is available as a set of native SDKs as well as through a web-based interface on the Custom Vision home page.PricingInstanceTransactions per second (TPS)FeaturesPriceStandard10 TPSUpload and prediction transactionsUp to 100 projects$3.024 per 1,000 transactions Training$30.239 per compute hourImage StorageUp to 6 MB each $1.059 per 1,000 images Tool 4: Machine Learning Service/StudioDescriptionSimplified machine learning with powerful, no-code, automated machine learning capabilities and open-source supportRapidly build and deploy machine learning models using tools that meet your needs across skill levels, from no-code to code-first experiences. Use a visual drag-and-drop interface, a hosted notebook environment, or automated machine learning. Accelerate model development with automated feature engineering, algorithm selection, and hyperparameter sweeping.A possible solution to the “Motion” detection.RelationshipsNoneLanguage and User Interface (UI)Uses a visual drop and drag interface with no coding requirements or PricingPricing depends on the computer instance chosen, starting from $0.077/hour for a basic machine.Tool 5: DatabricksDescriptionThis is an Apache Spark based analytics platform which can be used for image classification. It integrates with the Machine Learning service.Supports TensorFlow and scikit-learn.This appears to be more useful for analyzing data in tables, so not a recommended solution.RelationshipsIntegrates with Machine learning service.Numerous other Azure services, such as Blob storage.Language and UIAzure Databricks supports Python, Scala, R, Java, and SQL.PricingDatabricks DBU (Databricks Units) consumption and computing consumption.Price depends on the specific compute instance.Starting from $0.692/hour for a DS3 v2 instance with 4 vCPU and 14 GiB RAM.The compute is cluster based and scales out for the workload. Importantly the cluster can be set to turn off and shutdown automatically when not in use, saving costs.Tool 6: Windows Data Science Virtual MachineDescriptionA Windows base Virtual machine (VM) that is preinstalled and configured with the tools needed for data Analytics and machine learning.Tools such as Ananconda Python, Cognitive Toolkit, Azure machine Learning service and Tensorflow are already preinstalled configured on the VM.Useful to develop and build the machine learning applications and models which would support the production application.This could be further explored as a platform to run a “motion” detecting CNN/RNN.RelationshipsIntegrates with all the Azure services.UIUses remote desktop to access the VM.PricingThere are no software charges for this data science VM image. You pay only the Azure usage fees. They depend on the size of the virtual machine you provision. The minimum recommended VM for this purpose would be an ND6 GPU based VM with enough vCPU and RAM to handle machine learning workloads. At $1,819.89 per month.Tool 7: Storage Explorer and BLOB storageDescriptionAzure Blob storage is Microsoft's object storage solution for the cloud.Blob storage is designed for:Serving images or documents directly to a browser.Storing files for distributed access.Streaming video and audio.Writing to log files.Storing data for backup and restore, disaster recovery, and archiving.Storing data for analysis by an on-premises or Azure-hosted service.This could be used for uploading and storing the videos as well as the generated frames and the labeled output after classification.RelationshipsAccessible from other Azure services.Language and UIStorage explore is a UI for managing and accessing the storage containers.APIs and libraries are available for the common languagesPricingTotal cost of block blob storage depends on: Volume of data stored per month. Quantity and types of operations performed, along with any data transfer costs. Data redundancy option selected.Data storage pricesAll prices are per GB, per month.PremiumHotCoolArchiveFirst 50 terabyte (TB) / month $0.15 per GB $0.0184 per GB $0.01 per GB $0.00099 per GBNext 450 TB / Month $0.15 per GB $0.0177 per GB $0.01 per GB$0.00099 per GBOver 500 TB / Month $0.15 per GB $0.0170 per GB $0.01 per GB$0.00099 per GBOperations and data transfer pricesPremiumHotCoolArchiveWrite operations (per 10,000)$0.0175$0.05$0.10$0.10List and Create Container Operations (per 10,000)$0.05$0.05$0.05$0.05Read operations (per 10,000)$0.0014$0.004$0.001$5Data Retrieval (per GB)FreeFree $0.01$0.02Data Write (per GB)FreeFree Free Free The hot tier appears to be the recommended option for our type of storage requirements. Premium is for high performance with high IO operation and cool would be too slow to access.Tool 8: Logic AppsDescriptionAn integration tool which allow us to create a work flow on Azure. This tool automates the workflow of a service.This allows us to connect all our processes together and controls when each process will run.An ideal tool for controlling the flow through the App.RelationshipsIntegrates with all azure services.Language and UIThis is an Azure portal UI which requires no code writing.PricingA consumption pricing plan, so we only pay for what’s used.Pricing is based on action, each step in the process is an action, in our case the first action is a trigger which starts the logic App, followed by a series of 5 actions.Price per executionAction$0.000028This is an ideal tool to manage the flow from video upload to delivering an Xml document.Tool 9: Data lake StorageDescriptionBuilt on Blob storage, this storage solution is optimized for running analytic workloads on data, its optimized for fast i/o making analytic workloads run faster and thus reducing costs of analysis.It differs from blob storage as it can store in a hierarchical folder manner similar to a standard computers file storage.This could be used for uploading and storing the videos, with each video in its own folder. As well as the generated frames and the labeled output after classification.RelationshipsAccessible from other Azure services.Language and UIStorage Explorer is a UI for managing and accessing the storage containers.PricingData storage pricesAll prices are per GB, per month.HotCoolArchiveFirst 50 terabyte (TB) / month $0.0184 per GB $0.01 per GB $0.002 per GBNext 450 TB / Month $0.0177 per GB $0.01 per GB$0.002 per GBOver 500 TB / Month $0.0169 per GB $0.01 per GB$0.002 per GBOperations and data transfer pricesHotCoolArchiveWrite operations (per 10,000)$0.065$0.13$0.13List and Create Container Operations (per 10,000)$0.065$0.065$0.065Read operations (per 10,000)$0.0052$0.013$6.50Data Retrieval (per GB)Free $0.01$0.02Data Write (per GB)Free Free Free The hot tier would be our required storage type.Google Cloud ServicesDuring investigation of the concept of “Motion” and analyzing events in a rugby match with associated start and end times, I found the Google Cloud.Google Cloud Services has all the functionality available in Microsoft Azure that is discussed in this document.They have an additional service, AutoML Video Intelligence, which would be an ideal fit for this project.A discussion with Google support suggests that this is an excellent solution to our project.AutoML Video IntelligenceThe Google documentation specifically discusses that this service can be used to review soccer videos and highlight actions such as goal, foul and penalty, a very similar task to what we need for analyzing rugby videos.This service allows uploading and labeling of videos which will then train a model to classify video segments.They recommend between 200 and 1000 video examples per event, depending on the number of events and how distinctive an event is. This would be the costliest step in terms of time taken to scan a rugby videos and produce enough video examples of each event.GCS Costs GCS has a trial with 40 free hours for training and 5 free hours for prediction.Subsequent training costs approximately $27 per hour and prediction $0.42 per hour.Prediction time is roughly equal to the total video length.SummaryMicrosoft Azure Cloud has the services required to enable the prediction of an event based on classification of an Image by a Custom Vision model.Unfortunately, this platform proved to be unable to easily handle the scale required to analyze a full-length rugby video using the steps that were built for the Proof of Concept and will require an alternate approach to succeed.Google Cloud Services has a Video Intelligence service which on initial inspection appears to be an ideal solution for this project and I would recommend further exploration of this service going forward. ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download