1 Executive Summary - Virginia Tech

The WARC contained 43,748 HTML responses, so we can assume that it takes on average about 43 minutes / 43,748 responses = 59ms per response. Given that there are 3.08 billion pages in the February 2017 Index, we can compute that processing the entire month would take about 3.08 billion x 59ms per response = 50,478 hours of m4.2xlarge compute time. ................
................