Health Case Study - Centers for Medicare and Medicaid Services (CMS)

The CMS Certification & Accreditation (C&A) Program is a critical component of the entire CMS Integrated IT Investment & System Lifecycle “Framework.” As part of this program, all CMS systems must undergo a Security Test & Evaluation (ST&E) prior to accreditation or reaccreditation. The ST&E is conducted as part of the C&A process for a new system before it is placed into an operational state, when a significant change has been made to an existing system, or at least every three years. The ST&E assists in verifying implementation of appropriate management, operational, and technical security controls for the information system.

CMS engaged Thomas & Herbert Consulting LLC to perform these ST&Es of Enterprise Data Centers (EDC) as part of the CMS C&A program. To perform security testing or analyses at CMS, the testing team consisted of independent third-party individual(s) responsible for developing and executing test procedures. To be considered independent, the testing team did not have a stake in the development, maintenance, or documentation of the system to be tested. CMS ST&Es are one component of an overall information security program to ensure that CMS applications, systems, and supporting infrastructure operate securely and resist attacks that attempt to compromise sensitive information and/or cause harm to CMS data.

The assessment team used the security testing methodology approved by CMS to conduct these ST&Es. The scope per ST&E varied based on specific EDC C&A needs. At any point, it may have included all or some of the following components: vulnerability scanning of infrastructure components; mainframe assessment; or verification of an EDC’s implementation of security controls for compliance with CMS’ Accepted Risk Safeguard (ARS), policy for Information Security Program (PISP), and Core Security Requirement (CSR). The testing team performed ST&Es of applications and/or infrastructures located at CMS data centers throughout the country. The testing team reviewed available application documentation, including but not limited to requirements documents, Risk Assessments (RA), SSPs, Contingency Plans (CP) and design documents, prior to designing and conducting the test of an application. The testing team was familiar with CMS and other Federal security policies, procedures, standards and laws pertaining to this activity. All work was completed in accordance with the CMS IS Testing Approach. A comprehensive security testing report was created after each ST&E in accordance with the CMS IS Reporting Standard. The testing team provided on-going post-test support to clarify findings, make recommendations, review Corrective Action Plans (CAP), and validate the corrective action as necessary. Each ST&E typically included the following:

  • Development of an overall project plan for each task. The development of the project plan was based upon a CMS-established date for performance of the on-site tests for each system. When a system was identified for testing, CMS supplied the Test Team with appropriate documentation to include, but not limited to requirements documents, RAs, SSPs, CPs, and design documents for each system identified based on the type and the scope of the test
  • Development of the Requirements Document based on the type and scope of the test
  • Development of the Rules of Engagement Document based on the type and the scope of the test
  • Development of a Test Plan and Test Scripts in accordance with the CMS IS Testing Approach
  • Determination of whether the operational, managerial and technical controls of the systems were implemented correctly to protect the Confidentiality, Integrity and Availability (CIA) of the information processed by those systems in accordance with CMS policies and procedures
  • Discovery of the design, implementation and operational flaws that could have violated CMS IS Policies, Standards, Procedures and Guidelines; Business Partners System Security Manual (BPSSM); CMS Internet Architecture; HIPAA policies and other IS Standards (where applicable)
  • Assessment of the susceptibility of the systems tested to insider, Intranet, Internet and network based attacks
  • Assessment of the consistency between the system documentation and its implementation
  • Assurance that all working papers for each test were complete and legible
  • Collection, summarization and analysis of all test results in accordance with CMS IS Testing Approach and CMS IS Reporting Standard
  • Reviewed Corrective Action Plans (CAP) material and closure documentation and provided guidance as to their adequacy

All testing performed utilized the approach in CMS IS Testing Approach and CMS IS Reporting Standard, which is based on the NIST SP 800-53 and 800-53A. Within the framework of this testing activity, the testing team furnished all the necessary services, qualified personnel, material, equipment, hardware, software, facilities, and other related services, not otherwise provided by the Government, to meet ST&E procurement objectives. Due to the high sensitivity of this information, the testing team ensured that all information was protected from unauthorized access and secured according to the CMS IS policies and standards as defined in the CMS Policy for the Information Security Program (PISP) and the CMS Acceptable Risk Safeguards (ARS).

T&H brought innovations and showed our resourcefulness through:

  • The ST&E testing team used not only off-the-shelf testing tools to perform all of its system tests on client sites, but also created some of its own non-standard scripts that were used on a regular basis to evaluate the security of a given system
  • T&H testers did not take a given answer at its face value during the testing interviews. Achievement of satisfactory sign-off required verification and validation of answers provided through review of supporting documentation
  • T&H testers were extremely flexible and agile. The Change Control Board established by T&H immediately assessed impact and feasibility of any scope changes. The T&H Team worked with the client team at every step of the way to ensure testing proceeds according to plan and met or exceeded client expectations