Operational Research and Evaluation🔗

The National Center for Spectator Sports Safety and Security (NCS⁴) conducts operational research to help make better decisions and/or evaluation on technologies. At the NCS⁴ scientists, engineers, and industry stakeholders with the requisite expertise work with solution providers to research capability development, management and assurance and identify relevant research questions to evaluate claimed capabilities and to test such competencies. Researchers then validate the utility of a given technological solution, communicate results to providers, and produce a variety of peer-reviewed scientific outputs to disseminate findings. A scientific research brief for the project will also be maintained on the NCS⁴ website indefinitely in a repository for safety and security technological solutions indefinitely thus generating greater credibility and visibility among various stakeholders in the sports and entertainment security industry over the long term.

Operational Research/Evaluation Process:

  1. Design: The NCS⁴ lead scientist/principal investigator, in combination with members of the technology solution provider, develop a series of research questions that relate specifically to the capabilities of a given product. The experiment and evaluation are then designed to best address these competencies.

  2. Internal Review Board (IRB): Depending on the nature of the research design studies to include human research subjects, either actively recruited, passively participate, or unknowingly involved, must first undergo an assessment by the Institutional Review Board. In other instances, where only anonymized secondary data may be involved this step will not be necessary.

  3. Team: The operational research and evaluation team are identified based on the objectives and requisite skills detailed during the design phase and the availability of scientists, engineers, and industry stakeholders who meet the standards required to assist in the research and evaluation. Once established a brief that details the team composition is provided to the solution provider.

  4. Costs: Cost is calculated by a combination of factors including, but not limited to, the duration of the research, location, and team composition. Upon determining cost, a scope of work and services agreement is submitted through the appropriate channel within the USM for review, approval and signature.

  5. Begin Experiment: For projects that include an experiment, the operational deployment (real world or simulated) of the product at a predetermined location for a sufficient amount of time to collect the necessary data for analysis is put in place.

  6. Data Collection: In the case of projects that include an experiment, data collection efforts that seek to answer research questions established during the design will be initiated at the start of the experiment and ongoing for the duration. In the case of those projects that are strictly evaluations, whereby secondary data analysis is necessary to assess capability, solution providers will provide the data in an agreed upon format.

  7. Data Analysis: The data collected will be utilized to best address the series of research questions set out in the design and so no particular analytical technique will be preferred over another, but rather will be selected if it provides best method for answering a particular question(s). Techniques such as mathematical modeling, statistical analysis, mathematical optimization, spatiotemporal modeling, econometric modeling, and cost-benefit analyses can and will be used when and where appropriate.

  8. Draft Research Brief: The NCS⁴ lead scientist/principal investigator will generate a research brief that includes an overview of the research conducted including a description of the study area, how the data were collected and used, what the results were and what they mean. It may also further highlight safety and security gaps that may generate ideas for new solutions and technologies.

  9. Peer-reviewed Conference Papers: In an effort to further disseminate knowledge about the technological solution members of the research and evaluation team will endeavor to produce peer-reviewed conference papers for IEEE and other technical professional organizations dedicated to advancing technology for the benefit of humanity.

  10. Peer-reviewed Journal Articles: In an effort to further disseminate knowledge about the technological solution members of the research and evaluation team will endeavor to produce peer-reviewed journal articles for a wide array of disciplines that include, but are not limited to, operational research, resilience engineering, safety science, security, criminology, economics, and geography.

  11. Quality Control: A draft research brief is first sent to the operational research and evaluation team for review. Upon revision the research brief is then reviewed internally by a member of the NCS⁴ leadership team.

  12. Publish: The NCS⁴ lead scientist/principal investigator will provide the final research brief to the solution provider. Copies of the research brief will be available to the solution provider for further distribution and maintained on the NCS⁴ website in perpetuity.

For more information, please contact Dr. Justin Kurland at Justin.Kurland@usm.edu or (601) 266-5022.