NYDFS Issues Proposed Guidance on AI and External Consumer Data Use

SUMMARY

First, insurers within the scope of the Circular Letter should carefully consider NYDFS’ guidance when complying with applicable New York statutes and regulations, including those cited in the Circular Letter. However, even if formally issued by NYDFS, the Circular Letter does not have the force of law. Although NYDFS is requesting feedback before presumably formally issuing the Circular Letter, such input is not equivalent to the comprehensive analysis, debate and process required to enact legislation or promulgate a regulation.

Second, the Circular Letter applies only to authorized insurers using ECDIS and AIS for underwriting or pricing.

Third, the Circular Letter states that NYDFS will examine an insurer’s use of ECDIS and AIS under Sections 308 and 309 of the New York Insurance Law and the guidance in the Circular Letter.

Lastly, in addition to addressing authorized insurers’ use of ECDIS and AIS, the Circular Letter also seeks to clarify the disclosure requirements in NYDFS’ 2019 Circular Letter No. 1, which was issued to authorized life insurers using expedited, accelerated or algorithmic underwriting.

NYDFS GUIDANCE IN THE PROPOSED CIRCULAR LETTER

The Circular Letter states that insurers should establish governance processes, policies and procedures to ensure ECDIS and AIS use complies with applicable New York law. The Circular Letter reminds insurers that NYDFS “expects that insurers use of emerging technologies such as artificial intelligence will be conducted in a manner that complies with all applicable federal and state laws, rules, and regulations.”

The Circular Letter recognizes the importance of industry innovation, stating, “[t]he use of [ECDIS] and [AIS] can both benefit insurers and consumers alike by simplifying and expediting insurance underwriting and pricing processes, and potentially result in more accurate underwriting and pricing of insurance.” NYDFS states that it expects insurers “who utilize such technologies establish a proper governance and risk management framework to mitigate the potential harm to consumers and comply with all relevant legal obligations” and to address the following:

• Fairness Principles. “An insurer should not use ECDIS or AIS for underwriting or pricing purposes unless the insurer can establish that the data source or model, as applicable, does not use and is not based in any way on any class protected pursuant to Insurance Law Article 26…” or in a way that such “use would result in or permit any unfair discrimination or otherwise violate [insurance laws] or any regulations.” To show compliance, the insurer should be able to demonstrate:
  • Data validation in accordance with “generally accepted actuarial standards of practice and are based on actual or reasonably anticipated experience, including, but not limited to, statistical studies, predictive modeling, and risk assessments”
  • Proxy assessments that demonstrate that the ECDIS does “not serve as a proxy for any protected classes that may result in unfair or unlawful discrimination”
  • Comprehensive underwriting and pricing assessments that demonstrate that “ECDIS or AIS does not collect or use criteria that would constitute unfair or unlawful discrimination or an unfair trade practice,” including the evaluation of any use of third-party software, data, services or products to ensure they meet the insurer’s compliance obligations, and should address:
    • “[W]hether the use of ECDIS or AIS produces disproportionate adverse effects in underwriting and/or pricing on similarly situated insureds, or insureds of a protected class”
    • “[I]f there is prima facie showing of such a disproportionate adverse effect, further assessing whether there is a legitimate, lawful, and fair explanation or rationale for the differential effect on similarly situated insureds,” or if there is a finding of prima facie evidence, the insurer should alter the way it uses ECDIS or AIS to account for the differential effect
    • “[I]f a legitimate, lawful, and fair explanation or rationale can account for the differential effect, further conducting and appropriately documenting a search and analysis for a less discriminatory alternative variable(s) or methodology that would reasonably meet the insurer’s legitimate business needs.”
  • Document the processes and reasoning behind its testing methodologies and analysis for unfair or unlawful discrimination commensurate with the insurer’s use of ECDIS and AIS and the complexity and materiality of such ECDIS and AIS
  • Test regularly and frequently to confirm the insurer meets such compliance obligations
  • Perform a quantitative assessment that “use[s] multiple statistical metrics in evaluating data and model outputs to ensure a comprehensive understanding” and compliance, including adverse impact ratios, denial odds ratios, marginal effects, standardized mean differences, Z-tests and T-Tests, and drivers of disparity
  • Perform a qualitative assessment of unfair or unlawful discrimination, which includes “[being] able to explain, at all times, how the insurer’s AIS operates and to articulate the intuitive logical relationship between ECDIS and other model variables with an insured or potential insured individual’s risk”
• Governance and Risk Management. The Circular Letter reminds insurers of their obligations under 11 NYCRR § 90.2 “to have a corporate governance framework that is appropriate for the nature, scale, and complexity of the insurer,” which includes the following:
  • Board and senior management oversight
  • Policies, procedures and documentation including, but not limited to:
    • “[A]n up-to-date inventory of all AIS implemented for use, under development for implementation, or recently retired”
    • “[A] description of how each AIS operates, including any ECDIS or other inputs and their sources”
    • “[A] description of the process for tracking changes of an insurer’s use of ECDIS and AIS over time”
    • “[A] description of testing conducted to periodically assess the output of AIS models”
    • “[A] description of data lifecycle management process”
  • A consumer complaint response and resolution process
  • Risk management and internal controls, including an audit function
  • Third-party vendor due diligence and compliance, ensuring that the insurer’s use of third-party software, data, services and products comply with these requirements and that appropriate contractual terms and monitoring are in place
• Transparency and Disclosure. The Circular Letter states “[a]s discussed in Circular Letter No. 1 (2019), transparency is an important consideration in the use of ECDIS to underwrite and price insurance,” so the insurer will need to continue to comply with its obligations for the disclosure of the use of certain ECDIS and AIS in the underwriting and pricing process while clarifying what types of data elements or use of data should be disclosed to a consumer as it relates to underwriting and pricing. “The failure to adequately disclose the material elements of an AIS, and the external data sources upon which it relies, to a consumer may constitute an unfair trade practice under Insurance Law Article 24.”

Further, where an insurer is using ECDIS or AIS and there is a declination, limitation, rate differential or other adverse underwriting decision, “the reason or reasons provided to the insured or potential insured, or a medical professional designee, should include details about all information upon which the insurer based any declination, limitation, rate differential, or other adverse underwriting decision, including the specific source of the information upon which the insurer based its adverse underwriting or pricing decision.” The notice shall specifically disclose “(i) whether the insurer uses AIS in its underwriting or pricing process, (ii) whether the insurer uses data about the person obtained from external vendors, and (iii) that such person has the right to request information about the specific data that resulted in the underwriting or pricing decision, including contact information for making such request.”

TAKEAWAYS/COMMENTARY

The Circular Letter, as with the NAIC Model Bulletin, is a reminder to insurers of their obligations under existing laws when using AI, including those regulating unlawful/unfair discrimination, corporate governance and risk management frameworks, underwriting and pricing, actuarial, disclosure and related requirements. NYDFS unsurprisingly intends to be at the forefront of the race to establish guardrails around the use of AI in insurance and had previously signaled its intention to issue this guidance.

The Circular Letter combines a principles-based approach (See NAIC Model Bulletin) with certain aspects of the Colorado legislation when requiring an insurer to conduct detailed testing to prove that its use of ECDIS and AIS does not permit unlawful discrimination.

The Circular Letter’s emphasis on data set documentation and the actuarial validity of data is consistent with emerging best practices that promote deliberate reflection about how data sets might affect AI systems. Often, harm caused by AI can be traced back to the characteristics of underlying data sets. Additionally, the Circular Letter’s emphasis on risk and impact assessments to determine whether the use of ECDIS or AIS produces adverse or discriminatory effects aligns with guidance in the National Institute of Standards and Technology’s AI Risk Management Framework, a voluntary guidance document intended to guide organizations to increase the trustworthiness of AI systems.

The documentation, testing and ongoing monitoring expectations in the Circular Letter may prove challenging to operationalize for in-scope insurers that have not yet developed an internal AI governance program. As a practical next step, an insurer could conduct an inventory of current AIS and ECDIS use or development within its organization. Each AIS, for example, may require varying types of quantitative and qualitative testing depending on its context of use and intended outputs.

It is unclear how NYDFS may enforce the guidance in the Circular Letter once issued. The guidance does not have the force of law but suggests insurers using ECDIS and AIS take comprehensive action subject to NYDFS examination. Licensed insurers that currently use or plan to deploy AIS and ECDIS should carefully review the Circular Letter and are invited to submit comments to NYDFS by March 17, 2024.

For more insights on what’s ahead for AI in 2024, including general guidance on AI governance and risk management considerations, please see our Special Report, AI and the Next Frontier: Understanding What’s Ahead in 2024.

If you want to discuss the Circular Letter more in-depth or have any questions, please reach out to the authors or your regular McDermott lawyer.