On July 8, the Colorado Department of Insurance (CO DOI) piloted its third stakeholder meeting, during which CO DOI continued to outline the underwriting practices of life insurers as part of the required rulemaking process to implement SB-169 ( as codified in Colorado Statutes section 10-3-1104.9). CO DOI discussed its proposed upcoming study and call for data on life insurers to gather information that will inform the development of Colorado’s regulation. Additionally, ACLI presented its “ACLI Proposed Algorithmic Accountability Regulation” at the meeting.
- Industry survey and data call
During the July 8 stakeholder meeting, CO DOI announced that it will issue a survey and call for data to a sample of life insurance companies. The CO DOI proposed that the top 10 operators with the highest number of lives covered be asked to complete the survey and data call. The CO DOI requested comments on their proposed study and data call until July 22, 2022, and expects to publish the study and data call in mid-August. The CO DOI indicated that it would hold another stakeholder meeting before the study and call for data were published, but also stated that it may be possible to address the comments without requiring a meeting.
The CO DOI believes that the survey and call for data will provide it with an industry perspective:
- Current ECDIS usage practices and algorithms.
- Views of the various components of SB-169.
The proposed survey will gather information regarding the company’s practices regarding unfair discrimination management and testing processes. The survey questions seek to determine what insurers are doing to assess whether their use of algorithms or predictive models using ECDIS results in a disproportionately negative result for protected classes that goes beyond reasonable correlations with underlying underwriting practices. Specifically, the survey sought information in relation to four categories as follows:
- Processes and procedures around unfair discriminationincluding (i) whether there are any specific external data sources or types of external data that the insurer does not use, as a matter of internal company policy, and (ii) which of the external data sources, predictive models or underwriting practices of the insurer currently are tested for unfair discrimination.
- Evaluations of external data sources – ie. inputs to algorithms or predictive modelsincluding how the insurer decides whether an ECDIS should be tested, what deviation is tested, what testing methodology is used and what analysis is used to evaluate an ECDIS.
- Evaluating an ensemble of predictive modelsincluding what results are considered when testing predictive models, what bias is tested, what testing methodology is used to determine race or other demographic information, what analysis is used to evaluate the results, such as whether there is a disproportionately negative result for certain demographic groups and what measures are taken to mitigate problems once they are detected.
- Documenting actual cases of control and testing procedureswhich seeks actual documentation of the management and testing procedures the insurer has conducted and information as to whether or not there has been a finding of unfair discrimination.
In addition to the proposed study, CO DOI proposes to initiate a call for data to determine whether insurers’ data would be suitable for stress testing to determine whether unfair discrimination exists in the use of ECDIS. The proposed data call seeks information on:
- Application data for products that use ECDIS in one or more forecasting algorithms/models
The proposed call for data seeks specific information on applications received over a specified time period with a minimum of 10,000 applications for life insurance products that use models that include ECDIS in the underwriting process.
- List of products that do NOT use ECDIS in at least one forecasting algorithm/modelas well as a brief explanation of what data/information is being used and why it is outside the scope of the law.
Birney Birnbaum suggested that CO DOI consider trimming the sails by limiting survey and recall data to lifetime or lifetime, as these are the products sold in underserved markets at the highest risk of unfair discrimination.
The CO DOI emphasized that it will not use the information from the survey or data call to subject an insurer to any enforcement action.
- Algorithmic Accountability Regulation proposed by ACLI
As the stakeholder meeting began, Gillian Froment, a former Ohio insurance director, presented the principles-based ACLI Proposed Algorithmic Accountability Regulation (ACLI Proposal). ACLI’s proposal adopts a risk management framework that addresses unfair insurance discrimination based on race. The risk management framework requires insurers to:
(a) develop and maintain an internal management process regarding the use of ECDIS in insurance to reduce the risk of unfair discrimination;
(b) prepare a report documenting the insurer’s implementation of its internal management process in relation to the use of ECDIS in insurance to reduce the risk of unfair discrimination;
(c) conduct and maintain impact assessments (IAs) to determine whether the use of ECDIS in an underwriting algorithm or predictive model results in unfair discrimination based on race; and
(d) produce an IA report documenting the insurer’s analysis of the use of ECDIS in its algorithms and predictive models in a two-part structure.
Under the principles-based approach, each insurer will determine the most appropriate title and course of testing to determine whether the insurer’s use of ECDIS results in unfair discrimination based on race that goes beyond a reasonable correlation with underlying insurance practice. Consumer Representative Birney Birnbaum criticized this approach, saying it would require the CO DOI to review multiple testing methods because they would vary by insurer. Colorado Commissioner Conway expressed concern that ACLI’s proposal sails a little overboard because he disagrees with insurers determining the appropriate method or threshold to test for unfair discrimination. He suggested Colorado’s regulation chart the course for required testing.
The ACLI proposal’s two-part reporting structure will direct insurers to first document their internal management process, testing, methodologies and results (Part 1) and then summarize previous documentation and provide certification that they have completed testing in accordance with the regulation (Part 2). ACLI’s proposal provides for the insurer to submit both Part 1 and Part 2 in the early years and if the insurer’s use of ECDIS has not changed from year to year, the insurer may simply submit Part 2 annually and make Part 1 available to the CO DOI upon request. The Commissioner expressed concern that, given the speed at which the industry is innovating and developing in the use of ECDIS, there will always be changes to Part 1.
The Commissioner’s comments on the ACLI proposal, the proposed study, and the data call make it clear that the CO DOI has a different tack in mind than the ACLI proposal. We will continue to watch the wind direction to see how insurers should raise their sails.