OPA Reporting Design Document - National Archives



National Archives and Records AdministrationNational Archives Catalog (The Catalog)NARA Catalog Reporting Design– Catalog Perspective –Status-FinalVersion 1.7June 11, 2015National Archives & Records AdministrationNARA Catalog Reporting DesignArchana BallurMadhu KoneniRhea MandavilliVersion 1.7Contract Number GS-35F-0541UOrder Number NAMA-13-F-0120June 11, 2015Contents TOC \o "2-3" \h \z \t "Heading 1,1" 1Overview PAGEREF _Toc384981046 \h 22Reporting Technology PAGEREF _Toc384981047 \h 32.1Data Sources PAGEREF _Toc384981048 \h 32.2Log Files PAGEREF _Toc384981049 \h 42.3Database PAGEREF _Toc384981050 \h 42.4Report Format PAGEREF _Toc384981051 \h 52.5Roles and Permissions (14.34) PAGEREF _Toc384981052 \h 52.6System Admin Configuration PAGEREF _Toc384981053 \h 62.6.1Configure Top N Count PAGEREF _Toc384981054 \h 62.6.2Configure Time Based Reports PAGEREF _Toc384981055 \h 63Access Reports PAGEREF _Toc384981056 \h 73.1Number of queries- Daily/Weekly PAGEREF _Toc384981057 \h 73.2Top 1000 Search Terms PAGEREF _Toc384981058 \h 83.3Top 1000 Most Accessed Files PAGEREF _Toc384981059 \h 93.4Simple Search Access Reports PAGEREF _Toc384981060 \h 113.5Advanced Search Access Reports PAGEREF _Toc384981061 \h 124Contributions Reports PAGEREF _Toc384981062 \h 144.1No. of User Contributions broken down by type of contribution PAGEREF _Toc384981063 \h 144.2No. Of Registered Users who contributed by Contribution Type PAGEREF _Toc384981064 \h 154.3Top 100 records with most contributions PAGEREF _Toc384981065 \h 174.4Reasons for removing contributions PAGEREF _Toc384981066 \h 194.5Reasons for Restoring contributions PAGEREF _Toc384981067 \h 204.6Other User Contribution Reports PAGEREF _Toc384981068 \h 214.6.1Average number of contributions for the top 1000 contributors by contribution type. PAGEREF _Toc384981069 \h 214.6.2Top 100 contributors with their username and total number of contributions (excluding NARA users) [Req: 14.8.4] PAGEREF _Toc384981070 \h 224.6.3Top 1000 tags contributed by users and the total number of links that the tags have to records in OPA [Req: 14.8.5] PAGEREF _Toc384981071 \h 235API - Reports PAGEREF _Toc384981072 \h 255.1Number of Queries Generated through the API PAGEREF _Toc384981073 \h 255.2Characteristics of Queries made through API PAGEREF _Toc384981074 \h 265.3Number of times exports are generated through API PAGEREF _Toc384981075 \h 265.4Total no. of Bytes served via the API PAGEREF _Toc384981076 \h 275.5No. of User contributions via the API PAGEREF _Toc384981077 \h 285.6No. of Unique contributors via the API PAGEREF _Toc384981078 \h 296Export Reports PAGEREF _Toc384981079 \h 316.1No. of times records are exported PAGEREF _Toc384981080 \h 316.2Average number of Exports per User PAGEREF _Toc384981081 \h 337Saved Lists - Reports PAGEREF _Toc384981082 \h 348Server Space Reports PAGEREF _Toc384981083 \h 359Shares – Reports PAGEREF _Toc384981084 \h 369.1No. of Shares made by Users PAGEREF _Toc384981085 \h 369.2No. of Shares per User PAGEREF _Toc384981086 \h 379.3Records with most no. of shares PAGEREF _Toc384981087 \h 389.4Top 100 Shares by Destination PAGEREF _Toc384981088 \h 399.5No. of times Copy URL is accessed PAGEREF _Toc384981089 \h 4010Digital Analytics – Reports PAGEREF _Toc384981090 \h 4110.1Tabbed Groupings Access PAGEREF _Toc384981091 \h 4110.2Advanced Search – Access PAGEREF _Toc384981092 \h 4210.3Policy & Help Link – Access PAGEREF _Toc384981093 \h 4211Statistics – Report PAGEREF _Toc384981094 \h 4412Requirements INDEX PAGEREF _Toc384981095 \h 45Version ControlVersionDateAuthorSummary Description0.12014-02-19Rhea MandavilliInitial Outline and structure to incorporate all requirements1.02014-03-10Archana Ballur,Madhu KoneniFirst Version1.22014-04-02Archana BallurAdded Diagram to Section 2.1 Data Sources to show the various sources input to the Reporting System1.32014-04-11Archana BallurUpdated based on Requirements Spreadsheet dated Apr 41.42014-07-28Altán CabalUpdate based on requested log format changes1.52014-11-14Brandon StahlRemoved “Confidential to Search Technologies” text from footer1.62014-11-24Brandon StahlReplaced url with url1.72015-06-11Kristy MartinChanged branding for system name throughout document.OverviewThis document is a detailed description of the the National Archives Catalog reporting system and covers the following aspects:Technologies used to generate reports Sources used to generate reports SQL/Pseudo queries that can be used to extract the information required for the reportsSample ReportsRoles and Permissions of Authorized UsersSystem Configuration OptionsFor the purpose of organizing this document, the reports are classified into the following categories:Access Reports – Related to various access statistics. Contributions Reports - Related to user contributions.API reports – Related to all the API requests from public users through systems other than the Catalog.Server Space Reports – Related to Server space information.Export Reports – Related to statistics on downloads/exports of Catalog documents/objects/contributions etc.Saved Lists Reports – Related to the “My Lists” created by users.Shares Reports – Related to shares on public social networking sites like Twitter, Facebook, etc.Digital Analytics Reports – Reports which can be obtained using Google AnalyticsStatistics Reports – Related to the statistics of Catalog descriptions.Reporting TechnologyIt is assumed that Splunk is the reporting tool used for the Catalog reporting purposes. It is currently being evaluated by NARA as part of “Analysis of Alternatives”. Splunk Enterprise transforms machine data into real-time operational intelligence. It enables organizations to monitor, search, analyze, visualize and act on the massive streams of machine data generated by the websites, applications, servers and networks. Customers use Splunk software to improve analysis of log data to better manage their business. Splunk software automatically indexes all of the data, including structured, unstructured and complex multi-line application log data, enabling you to search on all of the data without need for custom connectors and without the scalability limitations inherent in traditional solutions. Once the data is in Splunk, one can quickly search, report and diagnose operations. Splunk can also integrate with relational database systems such as MySQL. Splunk allows users to create a wide variety of reports. These reports can be requested on demand and can also be scheduled.Data Sources Data for Reports will be obtained either from various log files on the server or from the relational database.The following diagram captures the Reporting system and the various sources of the log files and database tables.Log FilesIn Catalog production, the application will be logging all the various events and actions including errors and other information that is needed to generate the reports based on the requirements. Tools like Splunk can process huge amount of this log data structuring and indexing the data. One can query it to obtain information from the logs for a given time range and it also offers various functions which can be used to further filter and clean up the returned results.Data source and methods to extract report details from log files will be detailed for each report in various sections below.DatabaseThe information stored in the database can be used to generate some of the reports. Tools like Splunk have DB Connectors using which data from database can be fed to Splunk and Splunk can be configured to generate these reports. Splunk accepts SQL queries and reports can be configured with the SQL statements. These reports can be scheduled and also requested on demand. Report Format [Req: 14.7 XE "14.7" ] All Catalog reports are Microsoft Excel compatible. The report names will have the pattern “Report_Name_YYYY.MM.DD.HH:MM:SS”. For example: The report for the top 1000 search terms that resulted in zero results will be named “Top_Searches_ZeroResults_2014.03.05.00:00:00”All the reports will have the following template. The sections 4 to 12 will have complete examples.Report NameRun Date<date on which the report was run>Reporting Period<start date of the reporting period> - <end date of the reporting period><Report Content goes here>Figure SEQ Figure \* ARABIC 1 Report in Excel FormatRoles and Permissions (14.34)System administrator can create user accounts in Splunk defining their roles and permissions. From the requirements, there are two types of users identified as Authorized Users – System Administrators and Reporters. Following table captures the different types of Authorized Users and the permissions they will be granted in Splunk to generate, save, print and modify reporting period.Generate Reports [Req: 14.1 XE "14.1" ]System Administrator can generate report [Req: 14.1.1 XE "14.1.1" ]Reporter can generate report [Req: 14.1.2 XE "14.1.2" ]Save Reports [Req: 14.2 XE "14.2" ]System Administrator can save report [Req: 14.2.1 XE "14.2.1" ]Reporter can save report [Req: 14.2.2 XE "14.2.2" ]Print Reports [Req: 14.3 XE "14.3" ]System Administrator can save report [Req: 14.3.1 XE "14.3.1" ]Reporter can save report [Req: 14.3.2 XE "14.3.2" ]Modify Reporting Period for all system reports [Req: 14.5 XE "14.5" ]System Administrator to enter the start/end dates for all system reports. [Req: 14.5.1 XE "14.5.1" ]Reporter to enter the start/end dates for all system reports. [Req: 14.5.2 XE "14.5.2" ]System Admin Configuration System administrator will be able to configure the reporting settings for various parameters that are listed in the requirements below.Configure Top N CountThe system will provide the capability for a System Administrator to configure all reports to report on the top 10, top 100, or top 1000 statistics specific to that report. [Req: 14.16 XE "14.16" ]The system shall provide the capability for a System Administrator to configure all reports to report on the top 10 statistics specific to that report. [Req: 14.16.1 XE "14.16.1" ]The system shall provide the capability for a System Administrator to configure all reports to report on the top 100 statistics specific to that report. [Req: 14.16.2 XE "14.16.2" ]The system shall provide the capability for a System Administrator to configure all reports to report on the top 1000 statistics specific to that report. [Req: 14.16.3 XE "14.16.3" ]Configure Time Based ReportsSystem administrator will have the capability to make modifications to the reporting period to allow them to view reports for data for specific periods of time. The ability to modify reporting start and end dates shall provide system administrator the flexibility to tailor reports to specific data needs.The Catalog system shall provide the capability for a System Administrator to configure time-based reports to provide totals for weekly, monthly, yearly, and since inception. [Req: 14.15 XE "14.15" ]The system shall provide the capability for a System Administrator to configure time-based reports to provide weekly totals. [Req: 14.15.1 XE "14.15.1" ]The system shall provide the capability for a System Administrator to configure time-based reports to provide monthly totals. [Req: 14.15.2 XE "14.15.2" ]The system shall provide the capability for a System Administrator to configure time-based reports to provide yearly totals. [Req: 14.15.3 XE "14.15.3" ]The system shall provide the capability for a System Administrator to configure time-based reports to provide totals since inception. [Req: 14.15.4 XE "14.15.4" ]Access ReportsThe system will automatically generate the following scheduled access reports [Req: 14.4.6 XE "14.4.6" ]. It will send an email notification to System administrator when the report is ready [Req: 14.4.1 XE "14.4.1" ].Number of queries- Daily/WeeklyThe system will generate the following scheduled weekly access reports on each Monday at 12:01 AM [Req: 14.4.3 XE "14.4.3" ].Report TypeDaily number of queries [Req: 14.4.3.1 XE "14.4.3.1" ] Weekly Number of Queries for the Last Quarter [Req: 14.4.3.2 XE "14.4.3.2" ]Source: This report can be generated using the Application Server log files which are input to Splunk. Splunk will be configured to extract the required information from the logs. The following log format will be used for logging in the Catalog application. This is only an initial version and the format may change.<date-timestamp> MainType=USAGE Controller=<controller-Name> Type=WebApp, Query= {query parameters} <date-timestamp> MainType=USAGE Controller=<controller-Name> Type=API, Query= {query parameters}Sample Logs:2014-03-04 15:28:27,088 MainType=INFO Controller=SearchController Type=WebApp, Query= {q=Truman&source=authorities&type=person&filter=year: 1930}2014-03-04 15:29:20,233 MainType=INFO Controller=SearchController Type=API, Query= {q=Obama}Splunk Configuration:For these reports, Splunk will be configured to identify the keys “Type=WebApp” or “Type=API” and “Query={q=” and filter the queries by the date range. Sample Report:Catalog Access Report - Weekly number of Queries for the last quarterRun Date02/16/2013Reporting Period 4/1/2012 - 6/30/2012WeekNumber of Queries4/01/2012 - 04/07/2012200004/08/2012 - 04/14/2012300004/15/2012 - 04/21/2012250004/22/2012 - 04/28/2012280004/29/2012 - 05/05/20122400Top 1000 Search TermsReport ContentTop 1000 search terms in the last 1 month [Req: 14.4.3.3 XE "14.4.3.3" ]Top 1000 Search Terms in the last 1 year [Req: 14.4.3.4 XE "14.4.3.4" ]Top 1000 search terms since inception – monthly report [Req: 14.4.3.5 XE "14.4.3.5" ]Top 1000 search terms that return zero results – monthly report [Req: 14.4.3.6 XE "14.4.3.6" ]Source: This report can be generated using the Application Server log files which are input to Splunk. Splunk will be configured to extract the required information from the logs. The following log format will be used for logging in the Catalog application. This is only an initial version and the format may change.<date-timestamp> MainType=USAGE <controller-Name> Type=WebApp, Query= {query parameters} <date-timestamp> MainType=USAGE <controller-Name> Type=API, Query= {query parameters}Sample Logs:2014-03-04 15:28:27 MainType=INFO SearchController Type=OPA, Query= {q=Truman&source=authorities&type=person&filter=year: 1930}Splunk Configuration:For these reports, Splunk will be configured to identify the keys “Type=WebApp” or “Type=API” and “Query={q=” and filter the queries by the date range. Extract the q=<query term> from the Query String and group the results by query term to get the top search terms.Sample Report:Catalog Access Report - Top 1,000 Search Terms - YearlyRun Date02/16/2013Reporting Period 02/16/2012 - 02/16/2013??Search TermFrequencywar419602benjamin franklin2dd2142declaration of independence2immigration records2industrial revolution2Izzo2Top 1000 Most Accessed Files The system will generate a weekly Top 1000 Most Accessed Files Report on each Monday at 12:01 AM. [Req: 14.4.4 XE "14.4.4" ]The system will generate a weekly Top 1000 Most Accessed Files Report since Inception on each Monday at 12:01 AM. [Req: 14.4.5 XE "14.4.5" ]Report TypeTop 1000 Most Accessed: Descriptions with Digital Objects in the last 1 week. [Req: 14.4.4.1 XE "14.4.4.1" ]Top 1000 Most Accessed: Descriptions without Digital Objects in the last 1 week. [Req: 14.4.4.2 XE "14.4.4.2" ]Top 1000 Most Accessed: Authority Records in the last 1 week. [Req: 14.4.4.3 XE "14.4.4.3" ]Top 1000 Most Accessed webpages (, Presidential Libraries) in the last 1 week [Req: 14.4.4.4 XE "14.4.4.4" ]Top 1000 Most Accessed: Descriptions with Digital Objects since inception. [Req: 14.4.5.1 XE "14.4.5.1" ]Top 1000 Most Accessed: Descriptions without Digital Objects since inception. [Req: 14.4.5.2 XE "14.4.5.2" ]Top 1000 Most Accessed: Authority Records since inception. [Req: 14.4.5.3 XE "14.4.5.3" ]Top 1000 Most Accessed webpages (, Presidential Libraries) since inception[Req: 14.4.5.4 XE "14.4.5.4" XE "1.20.1" ]Source: This report can be generated using the Application Server log files which are input to Splunk. Splunk will be configured to extract the required information from the logs. The following log format will be used for logging in the Catalog application. This is only an initial version and the format may change.<date-timestamp> MainType=INFO Controller=<controller-Name> Type=WebApp, Action=ViewFullResults, Source=<source-name, NaId=<naid>, ObjectId=<objectId><date-timestamp> MainType=INFO Controller=<controller-Name> Type=API, Action=ViewFullResults, Source=<source-name, NaId=<naid>, ObjectId=<objectId>Source will be one of OnlineHoldings, DescriptionsOnly, WebPages and AuthoritiesSample Logs:2014-01-04 17:28:27 MainType=INFO Controller=FullResultsController Type=WebApp, Action=ViewFullResults, Source=DescriptionsOnly Naid=123452014-03-04 15:28:27 MainType=INFO Controller=FullResultsController Type=API, Action=ViewFullResults, Source=WebPages Naid=167261Splunk Configuration:For these reports, Splunk will be configured to identify these keys (“Type=WebApp” or “Type=API”), “Action=ViewFullResults” and filter the queries by the date range. Group by Naid to get the count as to how many times each record has been accessed. Splunk can return the top n of the results.Sample Report:Catalog Report - NARA's 1000 Most Accessed: Descriptions without Digital Objects - WeeklyRun Date02/16/2013Reporting Period 01/20/2013 - 01/26/2013??File NameFrequency Search Access ReportsThe system shall generate a report that displays the number of times the filter options are accessed[Req: 14.24 XE "14.24" ]Source: This report can be generated using the Application Server log files which are input to Splunk. Splunk will be configured to extract the required information from the logs. The following log format will be used for logging in the Catalog application. This is only an initial version and the format may change.<date-timestamp> MainType=USAGE Controller=<ControllerName> Type=WebApp, Query= {q=Truman&facet.year=2010}Sample Logs:2014-01-04 17:28:27 MainType=USAGE Controller=BriefResultsController Type=WebApp, Query= {q=Truman&facet.fields=year,oldScope,level}Splunk Configuration:For these reports, Splunk will be configured to identify these keys “Type=WebApp”, “Query= contains (facet)” and filter the queries by the daterange. Extract the facet.fields parameters from the Query String and then group by facet.fields to get the no. of times each of the filter option was accessed.Sample Report:Catalog Report - No. of times the filter options was accessed from Simple Search ScreenRun Date02/16/2013Reporting Period 01/20/2013 - 01/26/2013??FilterNo. of TimesData source - 123Level of Description - Collection34File Format - ASCII Text1Advanced Search Access ReportsThe system shall generate a report on the type of filters that are applied to advanced searches, and the frequency of each. [Req: 14.25 XE "14.25" ]Source: This report can be generated using the Application Server log files which are input to Splunk. Splunk will be configured to extract the required information from the logs. The following log format will be used for logging in the Catalog application. This is only an initial version and the format may change.<date-timestamp> MainType=USAGE Controller=<ControllerName> - Type=WebApp, SearchType=Advanced, Query={q=Truman& f.filtername= filtervalue}Sample Logs:2014-01-04 17:28:27 MainType=USAGE Controller=BriefResultsController Type=WebApp, Query= {q=Truman& f.inclusiveStart= range (1900, 1999)}Splunk Configuration:For these reports, Splunk will be configured to identify these keys “Type=WebApp”, “Query= contains (f.)” and filter the queries by the daterange. Extract the f.filtername parameters from the Query String and then group by f.filtername to get the no. of times each of the filter option was accessed.Sample Report:Catalog Report - No. of times the filter options was accessed from Advanced Search ScreenRun Date02/16/2013Reporting Period 01/20/2013 - 01/26/2013??FilterNo. of TimesData source - 123Level of Description - Collection34File Format - ASCII Text1Contributions ReportsThe system will generate the following scheduled reports to capture user contributed data. [Req: 14.8 XE "14.8" ].It will send an email notification to System administrator when the report is ready [Req: 14.4.1 XE "14.4.1" ].No. of User Contributions broken down by type of contributionThe system will generate a report on the total number of user contributions broken down by type of contribution (translations, transcriptions, comments, tags). [Req: 14.8.1 XE "14.8.1" ]The system will also generate a report on the cumulative number of user contributions made across all types of contribution (translations, transcriptions, comments, and tags). [Req: 14.8.1.1 XE "14.8.1.1" ]Source: The source for these reports will be the Catalog Database. Splunk will be configured to communicate with the MySQL database. Refer to the Application Server Design document for the database schema.Query:This is the initial query as per the schema at the moment. This is subject to change based on the schema updates.SELECT TR_count.Transcriptions, TL_count.Translations, C_ments, T_count.TagsFROM( SELECT count(*) AS TranscriptionsFROM annotations_transcriptions WHERE annotation_ts >= NOW() - INTERVAL 1 WEEK ) AS TR_count,(SELECT COUNT(*) AS TranslationsFROM annotations_translationsWHERE annotation_ts >= NOW() - INTERVAL 1 WEEK ) AS TL_count,(SELECT COUNT(*) AS CommentsFROM annotations_commentsWHERE annotation_ts >= NOW() - INTERVAL 1 WEEK ) AS C_count,(SELECT COUNT(*) AS TagsFROM annotations_tagsWHERE annotation_ts >= NOW() - INTERVAL 1 WEEK ) AS T_countFor the cumulative count, the select statement would be:SELECT SUM (TR_count.Transcriptions + TL_count.Translations + C_ments + T_count.Tags) AS CUMULATIVEFROM( SELECT count(*) AS TranscriptionsFROM annotations_transcriptions WHERE annotation_ts >= NOW() - INTERVAL 1 WEEK ) AS TR_count,(SELECT COUNT(*) AS TranslationsFROM annotations_translationsWHERE annotation_ts >= NOW() - INTERVAL 1 WEEK ) AS TL_count,(SELECT COUNT(*) AS CommentsFROM annotations_commentsWHERE annotation_ts >= NOW() - INTERVAL 1 WEEK ) AS C_count,(SELECT COUNT(*) AS TagsFROM annotations_tagsWHERE annotation_ts >= NOW() - INTERVAL 1 WEEK ) AS T_countSample Report: Catalog Report - No. of user contributions broken down by type of contributionRun Date02/16/2013Reporting Period 01/20/2013 - 01/26/2013??Contribution TypeTotalTags100Comments162Transcriptions123Translations12??All397No. Of Registered Users who contributed by Contribution TypeThe system will generate a report on the total number of register users who have contributed data, broken down by type of contribution (translations, transcriptions, comments, tags), with totals for weekly, monthly, yearly, and since inception, as well as cumulative totals across all types and time periods. [Req: 14.8.2 XE "14.8.2" , 14.8.2.1 XE "14.8.2.1" ]Source: The source for these reports will be the Catalog Database. Splunk will be configured to communicate with the MySQL database. Refer to the Application Server Design document for the database schema.Query:This is subject to change based on the schema updates.SELECT TR_count.Transcriptions, TL_count.Translations, C_ments, T_count.TagsFROM ( SELECT COUNT(DISTINCT account_id) AS Transcriptions FROM annotations_transcriptions WHERE annotation_ts >= NOW() - INTERVAL 1 WEEK ) AS TR_count, (SELECT COUNT(DISTINCT account_id) AS Translations FROM annotations_translations WHERE annotation_ts >= NOW() - INTERVAL 1 WEEK ) AS TL_count, (SELECT COUNT(DISTINCT account_id) AS Comments FROM annotations_comments WHERE annotation_ts >= NOW() - INTERVAL 1 WEEK ) AS C_count, (SELECT COUNT(DISTINCT account_id) AS Tags FROM annotations_tags WHERE annotation_ts >= NOW() - INTERVAL 1 WEEK ) AS T_countFor the cumulative count, replace the select statement with the following:SELECT SUM (TR_count.Transcriptions + TL_count.Translations + C_ments + T_count.Tags) AS CUMULATIVEFROM ( SELECT COUNT(DISTINCT account_id) AS Transcriptions FROM annotations_transcriptions WHERE annotation_ts >= NOW() - INTERVAL 1 WEEK ) AS TR_count, (SELECT COUNT(DISTINCT account_id) AS Translations FROM annotations_translations WHERE annotation_ts >= NOW() - INTERVAL 1 WEEK ) AS TL_count, (SELECT COUNT(DISTINCT account_id) AS Comments FROM annotations_comments WHERE annotation_ts >= NOW() - INTERVAL 1 WEEK ) AS C_count, (SELECT COUNT(DISTINCT account_id) AS Tags FROM annotations_tags WHERE annotation_ts >= NOW() - INTERVAL 1 WEEK ) AS T_countSample Report: Catalog User Contributions Total - Registers Users (Including NARA staff) - Since InceptionRun Date02/16/2013Reporting Period01/01/2012 - 02/16/2013?Contribution TypeTotalTags1200Comments3500Transcriptions600Translations700??All6000??Total Number of users contributed2020Average contributions per user2.97029703Top 100 records with most contributionsReport TypeTop 100 records (descriptions, authorities, and digital objects) that contain the most comments contributed by users. [Req: 14.8.6 XE "14.8.6" ]Top 100 records(digital objects) that contain the most translations contributed by users [Req: 14.8.7 XE "14.8.7" ]Top 100 records (digital objects) that contain translations with the most edits, and the total number of edits made to the translations. [Req: 14.8.8 XE "14.8.8" ]Top 100 records (digital objects) that contain transcriptions with the most edits, the average number of edits made to the transcriptions, and the total number of edits per transcription. [Req: 14.8.9 XE "14.8.9" ]Source: The source for these reports will be the Catalog Database. Splunk will be configured to communicate with the MySQL database. Refer to the Application Server Design document for the database schema.Query:For Report - Top 100 records that contain the most translations contributed by users, following pseudo query will be used.SELECT AC.opa_id, OT.opa_title, count(*)FROM annotations_translations AC, opa_titles OTWHERE AC.opa_id = OT.opa_idGROUP BY AC.opa_id, OT.opa_titleLIMIT 100Sample Report:Catalog user contributions - Top 100 Records with most commentsRun Date?02/16/2013Reporting Period01/01/2013 – 02/16/2013NAIDTypeTitlePageComments300321DescriptionThe Final Rolls of Citizens and Freedmen of the Five Civilized Tribes in Indian Territory?72300320Digital ObjectIndex to the Final Rolls of Citizens and Freedmen of the Five Civilized Tribes in Indian Territory1 of 570535413DescriptionWe can do it!, ca?502745164Digital ObjectThe Indian School Journal1 of 10457226539DescriptionLetter from George McGovern to Harry S. Truman?457283870DescriptionFall 1973: 17-34-2: What did Truman Say About the CIA?, by Benjamin F. Onate?402745164Digital ObjectThe Indian School Journal2 of 5302745164Digital ObjectThe Indian School Journal3 of 520Reasons for removing contributions[Req: 14.8.10 XE "14.8.10" ] The system will generate a report on the reasons that user-contributed data has been removed from the Catalog, broken down by type of contribution (translations, transcriptions, comments, tags), and the total number of contributions removed for each reason.Source: The source for these reports will be the Catalog Database. Splunk will be configured to communicate with the MySQL database. Refer to the Application Server Design document for the database schema.Query: SELECT AL.annotation_type, count(*), AR.reasonFROM annotations_log AL, accounts_reasons ARWHERE AL.reason_id = AR.id AND AL.action = “REMOVE”GROUP BY annotation_type, reasonSample Report:Catalog user contributions - Reasons for removing contributionsRun Date02/16/2013?Contribution TypeReasonCountTagsVANDALISM100CommentsVANDALISM200TranscriptionsVANDALISM10TranslationsVANDALISM10TagsSPAM50CommentsSPAM100TranscriptionsSPAM5TranslationsSPAM5TagsFOUL LANGUAGE100CommentsFOUL LANGUAGE100TranscriptionsFOUL LANGUAGE0TranslationsFOUL LANGUAGE0TagsOTHER200CommentsOTHER100TranscriptionsOTHER20TranslationsOTHER10???AllAll1010Reasons for Restoring contributions[Req: 14.8.11 XE "14.8.11" ] Reasons for restoring user contributed data by contribution type, total number of contributions restored for each reasonSource:The source for these reports will be the Catalog Database. Splunk will be configured to communicate with the MySQL database. Refer to the Application Server Design document for the database schema.Query:The following is the pseudo query:SELECT AL.annotation_type, count(*), AR.reasonFROM annotations_log AL, accounts_reasons ARWHERE AL.reason_id = AR.id AND AL.action = “RESTORE”GROUP BY annotation_type, reason_idSample Report:Catalog user contributions - Reasons for restoring contributionsRun Date02/16/2013?Contribution TypeReasonCountTagsACCIDENTAL REMOVAL50CommentsACCIDENTAL REMOVAL100TranscriptionsACCIDENTAL REMOVAL5TranslationsACCIDENTAL REMOVAL5TagsREQUESTED BY USER40CommentsREQUESTED BY USER100TranscriptionsREQUESTED BY USER5TranslationsREQUESTED BY USER5TagsOTHER100CommentsOTHER100TranscriptionsOTHER0TranslationsOTHER0???AllAll510Other User Contribution ReportsAverage number of contributions for the top 1000 contributors by contribution type.The system will generate a report on the average number of user contributions for the top 1,000 contributors, broken down by type of contribution (translation, transcription, comments, and tags).[Req: 14.8.3 XE "14.8.3" ]Source:The source for these reports will be the Catalog Database. Splunk will be configured to communicate with the MySQL database. Refer to the Application Server Design document for the database schema.Query: The following is the pseudo query:SELECT annotation_type, count(*) FROM annotation_log WHERE account_id IN (SELECT TOP 1000 account_id FROM annotation_log GROUP BY account_id) GROUP BY annotation_type;This will give the counts for each annotation type for the top 1000 contributors.To get the average for each type, divide the counts by 1000.Sample Report:Catalog user contributions – Average number of contributions for top 1000 contributorsRun Date02/16/2013Reporting Period01/01/2013 - 02/16/2013Contribution TypeAverageTags1000Comments1500Transcriptions500Translations400All3300Top 100 contributors with their username and total number of contributions (excluding NARA users) [Req: 14.8.4 XE "14.8.4" ] The system will generate a report on the top 100 contributors including their username and total number of contributions, excluding users registered with a NARA email address.[Req: 14.8.4 XE "14.8.4" ]Source: The source for these reports will be the Catalog Database. Splunk will be configured to communicate with the MySQL database. Refer to the Application Server Design document for the database schema.Query:CREATE VIEW contributions_by_userid AS(SELECT account_id, count(*) count FROMannotations_transcriptions GROUP BY account_id) UNION (SELECT account_id, count(*) FROM annotations_translations GROUP BY account_id)UNION(SELECT account_id, count(*) FROM annotations_comments GROUP BY account_id)UNION(SELECT account_id, count(*) FROM annotations_tags GROUP BY account_id)SELECT account_id, SUM(count) FROM contributions_by_useridGROUP BY account_idSample Report:Catalog Report – Top 100 Contributors and their Total no. of contributions(Excluding NARA Staff)Run Date02/16/2013Reporting Period01/01/2013 – 02/16/2013User NameAllmkoneni240aballur205pnelson179sdugan165lleu130Top 1000 tags contributed by users and the total number of links that the tags have to records in the system [Req: 14.8.5 XE "14.8.5" ] Source:The source for these reports will be the Catalog Database. Splunk will be configured to communicate with the MySQL database. Refer to the Application Server Design document for the database schema.Query:SELECT annotation, count(*) as no_of_recordsFROM annotations_tagsGROUP BY annotationORDER BY no_of_records DESCSample Report:Catalog Report – Top 1000 Tags and Total no. of links tags have to records in the CatalogRun Date02/16/2013Reporting Period02/9/2013 – 02/16/2013TagsTotal no. of record tagged Independence24Washington DC21China Agreement15Coal deals3Gold Industry1API - ReportsThe following section describes the various API reports that need to be generated.Number of Queries Generated through the APIThe system will generate a report on the number of queries generated through the API [Req: 14.26 XE "14.26" ].Source: This report can be generated using the Application Server log files which are input to Splunk. Splunk will be configured to extract the required information from the logs. The following log format will be used for logging in the Catalog application. This is only an initial version and the format may change.<date-timestamp> MainType=USAGE <controller-Name> Type=API, Query= {query parameters} Sample Logs:2014-03-04 15:28:27 MainType=INFO FullResultsController Type=API, Query= {holdings/7226539/description&format=JSON}2014-03-04 15:28:27 MainType=INFO SearchController Type=API, Query= {q=Truman&type=description}SPLUNK configuration:For these reports, Splunk will be configured to identify the keys “Type=API”, “Query”, filter by the date range.Sample Report: Catalog Report - No. of queries generated through the API in the last 1 weekRun Date02/16/2013Reporting Period02/9/2013 - 02/16/2013No. of queries generated through the API in last 1 week212Characteristics of Queries made through APIThe system shall generate a report that indicates the characteristics of queries made through the use of the API, including queries that caused errors, speed on query responses, and a listing of the most common fields queried [Req: 14.27 XE "14.27" ].Source:This report can be generated using the Application Server log files which are input to Splunk. Splunk will be configured to extract the required information from the logs. The following log format will be used for logging in the Catalog application. This is only an initial version and the format may change.<date timestamp> ERROR <Controller-Name> Type=API respTime= <response time> query= {query parameters} Sample Logs:2014-03-04 15:29:12 ERROR SearchController Type=API, results=0, query = {q=LGBTQ}Splunk Configuration:For these reports, Splunk will be configured to identify these keys (“Type=API”)Sample Report: Catalog Report – Characteristics of queries made via the APIRun Date02/16/2013Reporting Period02/16/2011 - 02/16/2013Query ERRORResponse Timeq=Truman&source=authoryYES2msq=Truman&source=authoritiesNO5msNumber of times exports are generated through APIThe system will generate a report on the number of exports generated through the API [Req: 14.28 XE "14.28" ].Source: This report can be generated using the Application Server log files as input to Splunk. Splunk will be configured to extract the required information from the logs and generate a report. The following log format will be used for all the logging in the Catalog application. This is only an initial version and the format may change.<date-timestamp> MainType=INFO <controller-Name> Type=WebApp, Action=export,All exports generated though the API will be logged as Type=API and Action=export.SPLUNK configuration:For these reports, Splunk will be configured to identify these keys “Type=API and Action=export”. Sample Report: Catalog Report - No. of times exports are generated through the APIRun Date02/16/2013Reporting Period02/9/2013 - 02/16/2013No. of times exports generated through the API181Total no. of Bytes served via the APIThe system will generate a report on the number of total bytes served via the API. [Req: 14.29 XE "14.29" ] Source:This report can be generated using the Application Server log files as input to Splunk. Splunk will be configured to extract the required information from the logs and generate a report. The following log format will be used for all the logging in the Catalog application. This is only an initial version and the format may change.<date timestamp> ERROR <Controller-Name> Type=API respTime= <response time> query= {query parameters} , BytesServed = 1290192817For every query response, a log will be made and it will log the amount of data being transferred.Splunk Configuration:For these reports, Splunk will be configured to identify these keys “Type=API”, “BytesServed”Sample Report:Catalog Report - No. of Bytes Served via the APIRun Date02/16/2013Reporting Period02/9/2013 - 02/16/2013No. of Bytes Served via the API10.1 GBNo. of User contributions via the APIThe system will generate a report on the number of user contributions (translations, transcriptions, comments, tags) via the API. [Req: 14.30 XE "14.30" ]Source:This report can be generated using the Application Server log files which are input to Splunk. Splunk will be configured to extract the required information from the logs. The following log format will be used for logging in the Catalog application. This is only an initial version and the format may change.<date timestamp> MainType=USAGE Controller=<Controller-Name> Type={API, WebApp}, Action=save, AnnotationType=<annotation type>, Naid=<naid> , Object=<object no if any>, Username=<username>Sample Logs:2014-03-04 15:28:27,054 MainType=USAGE Controller=gov.nara.opa.api.controller.annotation.tags.CreateTagController Type=API, Action=save, AnnotationType= Transcription, Naid= 12345, Object=1, Username=jsmithSPLUNK configuration:For these reports, Splunk will be configured to identify these keys “Type=API, contains (INFO), Action=save, contains (AnnotationType) and filter by date range. Then, group by contribution Type to get the counts.Sample Report: Catalog Report - No. of user contributions via the API made in the last 1 weekRun Date02/16/2013Reporting Period02/9/2013 - 02/16/2013Contribution TypeNo. of User Contributions via APITranscriptions 12Translations7Comments109Tags191No. of Unique contributors via the APIThe system will generate a report on the number of unique contributors via the API. [Req: 14.31 XE "14.31" ]Source:This report can be generated using the Application Server log files which are input to Splunk. Splunk will be configured to extract the required information from the logs. The following log format will be used for logging in the Catalog application. This is only an initial version and the format may change.<date timestamp> MainType=USAGE Controller=<Controller-Name> Type=API, Action=save, AnnotationType=<annotation type>, Naid=<naid>, Object=<object no if any>, Username=<username>Sample Logs:2014-03-04 15:28:27,889 MainType=USAGE Controller=CreateTagController Type=API, Action=save, AnnotationType= Transcription, Naid= 12345, Object=1, Username=jsmithSPLUNK configuration:For these reports, Splunk will be configured to identify these keys “Type=API, “MainType=INFO”, Action=save, “AnnotationType” and filter by date range.Extract the results as stated above and apply filter to get distinct usernames. Additional Note: Splunk as a way to remove duplicates. For more info - Report:Catalog Report - No. of unique contributors via the API made in the last 1 weekRun Date02/16/2013Reporting Period02/9/2013 - 02/16/2013No. of unique contributors via the API made in last 1 week12Export ReportsNo. of times records are exported The system will generate a report on the number of times records are exported from the system, and the characteristics of the export. [Req: 14.9 XE "14.9" ]The various characteristics that will be included in the report are:Number of records exported- Type of formats exported- Type of results exported (e.g., brief vs. full results)- Total file size- Type of export (normal, bulk, API)- Citizen contributed data included in export (tags, comments, transcriptions, translations)- Type of image setting (thumbnails, only metadata, metadata and thumbnails)The system will generate a report on the number of times records are exported from the system, broken down by type of export format. [Req: 14.10 XE "14.10" ]The system will generate a report on the number of times that records are exported from the system. [Req: 14.11 XE "14.11" ]Source:This report can be generated using the Application Server log files which are input to Splunk. Splunk will be configured to extract the required information from the logs. The following log format will be used for logging in the Catalog application. This is only an initial version and the format may change.<date-timestamp> MainType=INFO <controller-Name> Type=WebApp, Action=export, Format=<format>,TotalRecords= <no of records exported>, ExportOptions = {brief, full, thumbnails, tags, comments, transcriptions, translations}, username: <username>Sample Logs:2014-03-04 15:28:27 MainType=INFO Controller-Name Type=API, Action=export, Format=XML, TotalRecords=500,ExportOptions={thumbnails }, username:jsmith2013-01-15 15:28:27 MainType=INFO Controller-Name Type=API, Action=export, Format=XML, TotalRecords=500,ExportOptions={thumbnails, comments, tags}, username:jsmithSPLUNK configuration:For these reports, Splunk will be configured to identify these keys “Type=API”, “MainType=INFO”, “Action=export” and filter by date range.Group by exported format type and get the count as to no. of times records are exported in CSV format, no. of times records are exported in PDF format etc.Sample Report:Catalog Report- No. of times records are exported from the system, broken down by export formatRun Date02/16/2013Reporting Period01/20/2013 - 01/26/2013??File FormatFrequencyXML45CSV54JSON12PDF90Catalog Report- No. of times records are exported from the system in the last 1 weekRun Date02/16/2013Reporting Period02/9/2013 - 02/16/2013??DateFrequency02/09/20134502/10/20135402/11/20131202/12/20139002/13/20134502/14/20133202/15/20131102/16/201387Average number of Exports per UserThe system will generate a report on the average number of exports per user. [Req: 14.12 XE "14.12" ] Source: This report can be generated using the Application Server log files which are input to Splunk. Splunk will be configured to extract the required information from the logs. The following log format will be used for logging in the Catalog application. This is only an initial version and the format may change.<date-timestamp> MainType=INFO <controller-Name> Type=API, Action=export, Format=<format>,TotalRecords= <no of records exported>, ExportOptions = {thumbnails, tags, comments, transcriptions, translations}, username: <username>Sample Logs:2014-03-04 15:28:27 MainType=INFO Controller-Name Type=API, Action=export, Format=XML, TotalRecords=500,ExportOptions={thumbnails }, username:jsmithSPLUNK configuration:For these reports, Splunk will be configured to identify these keys “Type=API”, contains (MainType=INFO), Action=exportThen, group by username to get the count of no. of exports per user and use Splunk’s avg function to get the average.Sample Report:Catalog Report- Average no. of exports per userRun Date02/16/2013Reporting Period01/20/2013 - 01/26/2013??UsernameAverage no. of exportsjsmith32mia6112amartin11kelizabeth51Saved Lists - ReportsThe system will generate a report on the number of saved lists created by registered users. [Req: 14.17 XE "14.17" ]Following Reports to be GeneratedNo. of saved lists created weekly. [Req: 14.17.1 XE "14.17.1" ]No. of saved lists created monthly. [Req: 14.17.2 XE "14.17.2" ]No. of saved lists created yearly. [Req: 14.17.3 XE "14.17.3" ]No. of saved lists created since inception. [Req: 14.17.4 XE "14.17.4" ]Source:The source for these reports will be the Catalog Database. Splunk will be configured to communicate with the MySQL database. Refer to the Application Server Design document for the database schema.Query:SELECT account_id, COUNT(* )FROM accounts_lists GROUP BY account_idSample Report:Catalog Report- No. of saved lists created by registered userRun Date02/16/2013Reporting Period01/20/2013 - 01/26/2013??UsernameNo. of lists createdjsmith32mia6112amartin11kelizabeth51Server Space ReportsThe system will generate a monthly Catalog Server Space Report on the first Monday of a month [Req: 14.4.2 XE "14.4.2" ]Source: The information can be obtained from the Catalog servers. A script will be scheduled to run on the servers to extract the information required to generate this report. Sample Report: Catalog Server Space ReportRun Date 02/16/2013Reporting Period01/01/2013 - 01/31/2013Total Space300000Total Used Space30000Space Reserved for Maintenance10000Total Available/Usable Space260000??All sizes rounded up to the nearest GB?Shares – ReportsNo. of Shares made by UsersRelease INFO: Not in R1The system will generate a report on the number of shares made by users [Req: 14.18 XE "14.18" ].Following Reports to be GeneratedNumber of shares made in the last 1 week [Req: 14.18.1 XE "14.18.1" ]Number of shares made in the last 1 month [Req: 14.18.2 XE "14.18.2" ]Number of shares made in the last 1 year [Req: 14.18.3 XE "14.18.3" ]Number of shares made since inception [Req: 14.18.4 XE "14.18.1" ]Source: Front end will log the information related to Shares such as Username, URL (Results Page), Record (for content detail pages), and Share Destination. This is only an initial version and the format may change.<date-timestamp> MainType=INFO <Controller-Name> - Username=<username>, SharedTo=<twitter/email/etc>, Url=<complete url>, RecordId= <naid>Sample Logs:2014-03-04 15:28:27 MainType=INFO LogController – username= jsmith, SharedTo=twitter, url= 12:23:12 MainType=INFO LogController – username= jsmith, SharedTo=twitter, RecordId=desc-12345Splunk Configuration:For these reports, Splunk will be configured to identify the keys - ”MainType=INFO”, “sharedTo”Filter the above results by using different time ranges to get the weekly, monthly, yearly and since inception reports. Sample Report: Catalog Report - No. of shares made in last 1 weekRun Date02/16/2013Reporting Period 02/9/2013 - 02/16/2013??No. of shares made in last 1 week78??No. of Shares per UserRelease INFO: Not in R1The system will generate a report that captures the number of shares per user. [Req: 14.19 XE "14.19" ]Source: Front end will log the information related to Shares such as Username, URL (Results Page), Record (for content detail pages), and Share Destination. This is only an initial version and the format may change.<date-timestamp> MainType=INFO <Controller-Name> - username=<username>, sharedTo=<twitter/email/etc>, Url=<complete url>, RecordId=<naid>Sample Logs:2014-03-04 15:28:27 MainType=INFO LogController– Username= jsmith, SharedTo=twitter, Url= Configuration:For these reports, Splunk will be configured to identify the keys - ”MainType=INFO”, “SharedTo”Group the returned results by username to get the count of no. of shares made per user.Sample Report: Catalog Report - No. of shares per userRun Date02/16/2013Reporting Period 02/9/2013 - 02/16/2013??UsernameNo. of sharesPublic56Jsmith34kelizabeth1223Rdann6Records with most no. of sharesRelease INFO: Not in R1The system will generate a report that captures the records with the most number of shares. [Req: 14.20 XE "14.20" ]Source: Front end will log the information related to Shares such as Username, URL (Results Page), Record (for content detail pages), and Share Destination. This is only an initial version and the format may change.<date-timestamp> MainType=INFO <Controller-Name> - Username=<username>, SharedTo=<twitter/email/etc>, recordId=<naid>Sample Logs:2014-03-04 15:28:27 MainType=INFO LogController– Username= jsmith, SharedTo=twitter, RecordId=672514Splunk Configuration:For these reports, Splunk will be configured to identify the keys - ”MainType=INFO”, “SharedTo”, and “RecordId”Group by Naid to get the count of no. of times this record was shared.Sample Report: Catalog Report - Records with most no. of sharesRun Date02/16/2013Reporting Period 02/9/2013 - 02/16/2013??Record – NAIDNo. of sharesdesc-1234556desc-87711134desc-45311223desc-9182616Top 100 Shares by Destination Release INFO: Not in R1The system will generate a report on the top 100 share destinations on social media platforms, broken down by the destination (e.g., Twitter) and the total number of shares for that destination. [Req: 14.21 XE "14.21" ]Source: Front end will log the information related to Shares such as Username, URL (Results Page), Record (for content detail pages), and Share Destination. This is only an initial version and the format may change.<date-timestamp> MainType=INFO <Controller-Name> - Username=<username>, SharedTo=<twitter/email/etc>, Url=<complete url>, RecordId=<naid>Sample Logs:2014-03-04 15:28:27 MainType=INFO LogController – Username= jsmith, SharedTo=twitter, Url= 15:28:27 MainType=INFO LogController – Username= jsmith, SharedTo=twitter, RecordId=12345Splunk Configuration:For these reports, Splunk will be configured to identify the keys- “MainType=INFO”, “SharedTo”Group by SharedTo field to get the count of no. of times records were shared to this destination.Sample Report: Catalog Report - Top 100 Share Destinations and no. of shares to that destinationRun Date02/16/2013Reporting Period 02/9/2013 - 02/16/2013??Share DestinationNo. of sharesGmail56Email34Facebook23Twitter6No. of times Copy URL is accessedRelease INFO: Not in R1The system shall generate a report that displays the number of times the copy URL function is accessed. [Req: 14.22 XE "14.22" ]Source: Front end will log the information related to Shares such as Username, URL (Results Page), Record (for content detail pages), and Share Destination. This is only an initial version and the format may change.<date-timestamp> MainType=INFO <Javascript Filename> <JavaScript Function Name> - Username=<username>, SharedTo=CopyUrl, Url=<complete url>, NaId=<naid>Sample Logs:2014-03-04 15:28:27 MainType=INFO LogController – Username= jsmith, SharedTo=”CopyUrl”, Url= Configuration:For these reports, Splunk will be configured to identify the keys- “MainType=INFO”, “SharedTo=CopyUrl”. Count the no. of results to get the no. of times copy URL was accessed.Sample Report: Catalog Report - No. of times Copy URL is accessedRun Date02/16/2013Reporting Period 02/9/2013 - 02/16/2013??No. of times Copy URL was accessed56Digital Analytics – ReportsThe system will implement a digital analytics program. [Req: 14.13 XE "14.13" ]"Implementation of a digital analytics program” - The gov't wide contract is for Google Analytics. URL for information on code to implement: Groupings AccessThe system will generate a report on the total number of times that the tabbed groupings are accessed in the brief search results display. [Req: 14.14 XE "14.14" ]Following Reports to be GeneratedTotal no. of times the tabbed groupings are accessed in the brief search results display in the last 1 week. [Req: 14.14.1 XE "14.14.1" ]Total no. of times the tabbed groupings are accessed in the brief search results display in the last one month. [Req: 14.14.2 XE "14.14.2" ]Total no. of times the tabbed groupings are accessed in the brief search results display in the last one year. [Req: 14.14.3 XE "14.14.3" ]Source:The information for this report will be obtained from the Digital Analytics provided by Google. The logs from Google Analytics will be used to generate information required for this report. Exact Google Analytics Log Format will be available during the development phase when it is integrated into the website and logs are generated on the development server. The report content i.e. Frequency of accessing the tabs in the brief results display can be obtained from Google Analytics log files. Google Analytics captures the all the clicks and events on a webpage.Sample Report:Catalog Report - Tabbed Groupings AccessRun Date02/16/2013Reporting Period 02/9/2013 - 02/16/2013??Tab nameNo. of times accessedAll455Available Online212Web Pages211Documents114Images321Video122Advanced Search – AccessThe system will generate a report that displays the number of times the Advanced Search Screen is accessed. [Req: 14.23 XE "14.23" ]Source:The information for this report will be obtained from the Digital Analytics provided by Google. The logs from Google Analytics will be used to generate information required for this report. Exact Google Analytics Log Format will be available during the development phase when it is integrated into the website and logs are generated on the development server. The report content i.e. Frequency of accessing the tabs in the brief results display can be obtained from Google Analytics log files. Google Analytics captures the all the clicks and events on a webpage.Sample Report:Catalog Report - Advanced Search Screen AccessRun Date02/16/2013Reporting Period 02/9/2013 - 02/16/2013??No. of Advanced Search Screen was accessed78Policy & Help Link – AccessRelease INFO: Not in R1The system will generate a report on the number of times the policy link is accessed. [Req: 14.32 XE "14.32" ]Following Reports to be GeneratedDisplay the number of times the policy link is accessed weekly. [Req: 14.32.1 XE "14.32.1" ]Display the number of times the policy link is accessed monthly. [Req: 14.32.2 XE "14.32.2" ]Display the number of times the policy link is accessed yearly. [Req: 14.32.3 XE "14.32.3" ]The system shall generate a report on the number of times the help link is accessed. [Req: 14.34 XE "14.34" ]Source:The information for this report will be obtained from the Digital Analytics provided by Google. The logs from Google Analytics will be used to generate information required for this report. Exact Google Analytics Log Format will be available during the development phase when it is integrated into the website and logs are generated on the development server. The report content i.e. Frequency of accessing the tabs in the brief results display can be obtained from Google Analytics log files. Google Analytics captures the all the clicks and events on a webpage.Sample Report:Catalog Report - No. of times Policy Link was accessed - in the last 1 weekRun Date02/16/2013Reporting Period 02/9/2013 - 02/16/2013??No. of times Policy link was accessed43Statistics – ReportThis refers to the Statistics page which shows the amount of data per level of description.The system will provide the capability for a System Administrator to view the amount of data per level of description (i.e. Collection, Record Group, File Unit, Series, and Item). [Req: 14.6 XE "14.6" ]The system will provide the capability to select a level of description (i.e. Collection, Record Group, Series, File Unit and Item) in order to preview data. [Req: 14.6.1 XE "14.6.1" ]The system will provide the capability for a System Administrator to view the amount of data per level of description (i.e. Collection, Record Group, File Unit, Series, and Item). [Req: 14.6.2 XE "14.6.2" ]The system will provide the capability for a Reporter to view the amount of data per level of description (i.e. Collection, Record Group, File Unit, Series, and Item). [Req: 14.6.3 XE "14.6.3" ]Statistics will be a report available in the form of a webpage and will be part of the Catalog website and can be accessed by Public users also. Statistics link will be available in the header block on all pages of the Catalog website. More details about Statistics webpage, the look and feel as well as the contents of the page is detailed in Section 12 of the Catalog UI Design Document.The above two reports can be generated using the ARC data indexed in SOLR. This has already been implemented in the prototype. Requirements INDEX INDEX \e "" \c "2" \z "1033" 1.20.11014.1514.1.1514.1.2514.103114.113114.123314.134114.144114.14.14114.14.24114.14.34114.15614.15.1614.15.2614.15.3614.15.4614.16614.16.1614.16.2614.16.3614.173414.17.13414.17.23414.17.33414.17.43414.183614.18.13614.18.23614.18.33614.193714.2514.2.1514.2.2514.203814.213914.224014.234214.241114.251214.262514.272614.282614.292714.3514.3.1514.3.2614.302814.312914.324214.32.14314.32.24314.32.34314.344314.4.17, 1414.4.23514.4.3714.4.3.1714.4.3.2714.4.3.3814.4.3.4814.4.3.5814.4.3.6814.4.4914.4.4.1914.4.4.2914.4.4.3914.4.4.41014.4.5914.4.5.11014.4.5.21014.4.5.31014.4.5.41014.4.6714.5614.5.1614.5.2614.64414.6.14414.6.24414.6.34414.7514.81414.8.11414.8.1.11414.8.101914.8.112014.8.21514.8.2.11514.8.32114.8.42214.8.52314.8.61714.8.71714.8.81714.8.91714.931Database Reports Technical DetailsDB App:Download the SplunkDB Connect from the below instructions for installing the app:Log into Splunk EnterpriseOn the Apps Menu, click Manage AppsClick Install App from fileIn the upload app window, click “Choose File”Locate the .tar.gz file you just downloaded, and then click open or choose. Click upload.Click Restart Splunk, and then confirm that you want to restartTo install apps and add-ons directly into Splunk Enterprise:Put the downloaded file in $SPLUNK_HOME/etc/apps directoryUntar and unzip your apps or add-on, using a tool like tar –xvf (on linux/unix) or WinZip (Windows)Restart SplunkAfter installing the app, set it up by clicking on setup:Set Java HomeClick SaveInstall JDBC driverCopy the MySql driver mysql-connector-java-5.1.18.jar to the $SPLUNK_HOME/etc/apps/dbx/bin/lib directory.Restart Splunk (Settings -> Server Controls -> Restart)Create DB connection DB inputs ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download