CMS Pub. 100-17, ch. 117_systems_security, § 3.21

(Rev. 15.1; Issued: 07-17-25; Effective: 02-28-25; Implementation: 08-18-25)

There are operational and security focused tools that are changing the paradigm to increase staff

efficiency and look at issues in a different way. While many of the new tools have useful new

features, FISMA systems are required to follow the MAC ARS or demonstrate how the intent of the

MAC ARS is being met. If an AI based tool is planned for use, then the security team needs to

evaluate and document how the tool implements the MAC ARS. In addition, the following points

need to be considered with implementing and maintaining an AI based solution.

• Potential issues with AI – There is often no audit trail, or supporting data, to show how the

software arrived at its conclusion. Depending on the nature of a decision, supporting

documentation may be needed to demonstrate how the decision was made.

• Periodic validation is required – Policies and procedures for periodic validation will need to

be documented to make certain the tool is operating as intended and no security “gaps”

exist. These will need to include instructions for recreating the results. If it is impossible to

recreate the AI results with 100% accuracy, then tolerances need to be documented.

• Periodic assessment is required – Certain data may be used to initially seed the AI, but as

conditions change, additional data may need to be added, or some data may need to be

removed or modified. As changes are made, associated policies and procedures may need

to be updated.

• AI account management – AI tools may bring complexity with accounts needed to operate

effectively. Management should treat any account, even those used for AI, with the same

security requirements as their other user, service, and administrative accounts.

• AI external connections – AI tools should be evaluated to determine if the tool operation or

the data being analyzed is being sent outside of the organization-controlled network (e.g.

cloud repository). If so, CMS should be consulted prior to implementation.

• In the event that the AI tool being implemented cannot align exactly to part of a MAC ARS

control, management should evaluate if the tool has addressed the risk of the requirement. If

the tool addresses the risk but the implementation is different than what the MAC ARS

identifies, this should be documented within the organizations SSPP and policies. If the tool

does not address the risk, then management may need to determine if additional control

implementations are needed to fully address that MAC ARS control. MACs should consult

with CMS if a technical limitation is encountered.

Additional information about AI at CMS can be found here: https://ai.cms.gov/.