Using MNsure should have been easy for Becky Fink.
As a local clinic director, she'd been trained by the new health insurance exchange on how to help people sign up online.
But the site never worked for her even though she "tried, tried, many, many times, many times."
It wasn't her fault.
Fink was among thousands of other Minnesotans who saw MNsure's cheery Paul Bunyan TV ads and tried to use the site once it went live in October. Some had success, but others are still trying to get insurance or have given up all together.
But behind MNsure's upbeat façade was a swamp of management failures and tech glitches that would cripple the more than $100 million website.
MNsure leaders blamed tight deadlines and evolving federal requirements for the website's malfunctions.
However, internal MNsure documents and interviews with insurance company officials, county workers and other stakeholders reveal a more complicated story.
People interviewed by MPR News say MNsure's top officials, including former executive director April Todd-Malmlov, were insular and stingy with information and that they ignored advice for improvement. They say MNsure staff was disorganized and often didn't have answers to their questions. And an early decision by the state to buy software that was still under development compounded the problems, adding to what one recent report called a "crisis mode" inside MNsure.
Moreover, the state never tested the website on consumers before it went live.
State officials had expected MNsure to enroll up to 1.3 million people for insurance by 2016, but a recent review by consulting firm Optum found MNsure is unlikely to meet those goals.
MNsure's new leadership says things are working better now. The paper trail, though, shows many problems were evident early on.
NOT FOCUSED ON TECHNOLOGY
The Affordable Care Act, which authorized Minnesota and other states to build an insurance marketplace, had barely been on the books for five months in the fall 2010 when then- Republican Gov. Tim Pawlenty effectively rejected a $1 million planning grant.
When Gov. Mark Dayton took office in 2011 he charged ahead, in part by setting up an exchange task force.
Problems, though, were already taking root.
The group wasn't thinking deeply enough about the technological nuts and bolts of the project, and that the same was true of the state employees leading the effort, said task force member Dannette Coleman, chief for individual and family business for the Medica health plan.
"This is really a technology project," she said. "One of the challenges is the [MNsure] leadership viewed this from a health policy standpoint and not from an IT technology implementation standpoint, and I think that drove a lot of the challenges along the way."
Insurance company executives also complained that the Legislature denied the industry a seat on the agency's board over concerns about a conflict of interest, so tough questions about the site's technological infrastructure weren't asked early or frequently enough.
When MNsure went live, its technical flaws emerged quickly. Some people were locked out of their applications, while others struggled to find out if they were eligible for financial assistance. MNsure didn't play well with certain web browsers, and many users were confronted with frozen screens.
MNsure officials would have known about many of these problems if they had tested the site with consumers prior to Oct. 1.
But, as agency officials rushed to make their deadline, they say they didn't have time to do what IT consultant Michael Krigsman calls a cardinal rule of technology development.
Testing "is one of those things that's so foundational, it doesn't even deserve a 'Why?'" said Krigsman, who specializes in preventing IT projects from failing. "It's like, 'Why do we need to breathe the air?'"
OFF-THE-SHELF AND UNTESTED
In 2011, Minnesota counties were helping the Department of Human Services vet software for a massive overhaul of the state's outdated system that calculates someone's eligibility for Medicaid or other public assistance.
County workers process Medical Assistance applications and MNsure was supposed to automate the application system.
Ramsey County Community Human Services director Monty Martin said software developed by IBM Curam stood out among the products the counties analyzed, partly because other states used it and it was "off-the-shelf."
In other words it wouldn't have to be built from scratch, and could be reconfigured to fit Minnesota's specific needs.
Martin said the state and counties were anxious to avoid another HealthMatch, an expensive and failed IT program under the Pawlenty administration that was meant to improve eligibility calculations.
"We in the county had reached an independent conclusion that commercial off-the-shelf products were the direction we would need to go generally in government because development of our own software pieces was much too expensive and hard to maintain," Martin said.
But what the state ultimately licensed from Curam was a product that was untested.
And even though Curam's existing products appeared to work well for other states, one major consulting firm seeking to win the MNsure contract was leery.
CONTRACT NEGOTIATIONS STALL
By the spring of 2012, the state still needed someone to build MNsure and was back at square one in the hiring process.
The state's initial approach of using multiple contractors to build different parts of the site proved too expensive and also took up a lot of valuable time. So MNsure officials returned to Virginia-based Maximus and New York-based Deloitte, two companies that had made bids to serve as the project's primary contractor.
Negotiations began with Deloitte, but the firm's price tag was $20 million over the state's budget, said MNsure legal counsel Mike Turpin.
Deloitte also didn't want to be responsible for the IBM Curam software, Turpin said. The firm insisted that IBM have its own contract with the state, effectively ensuring that Deloitte wasn't responsible for the Curam product.
Turpin said he wasn't sure why Deloitte resisted using the Curam software, but that "they certainly said there was a level of risk they weren't willing to accept, that they felt more comfortable with their own mix of [software] they presented."
Deloitte's concerns foreshadowed some of the troubles to come once MNsure went live. In December, Gov. Dayton sent the head of IBM a laundry list of problems stemming from the Curam software and demanded a fix.
Deloitte declined requests to talk further about the negotiations, and Curam has said repeatedly that it is working with the state to fix those issues.
As contract negotiations dragged on Turpin said the project was losing time to make headway.
"Every week, every month that extended where we were in negotiations meant that we were not doing work on the ground to actually build the technology," Turpin said.
MNsure was "a three-year project," said Tom Baden, the Department of Human Services IT leader for MNsure. But by the time Maximus was on board, the state had less than a year-and-a-half to get MNsure up and running. Maximus' job was to oversee the construction of the website.
FEDS CHANGE THE TIMETABLE
In early 2013, the state had to make a major adjustment. The federal government announced that MNsure had only a few months to make sure the site could perform 70 functions.
The news forced the state to move immediately from planning the site's operational processes to building software, Baden said. That also prompted the state to take the lead role on the project from Maximus.
"What [the process planning] wasn't allowing us to accomplish was the development of the system," Baden said. "We had to focus like a laser to get the system built."
He said it was a positive turning point for the project.
"We really got narrowly focused and the [vendor] teams really ignited and started working together," Baden said.
Baden says his team put in a heroic effort in the following months. But internal documents show the project was in trouble well before the Oct. 1 deadline.
Last spring, the first of three progress reports commissioned by MNsure found five out of nine quality assurance tasks were labeled unsatisfactory.
Six months later, by the time the site was live, four of the five were still unsatisfactory.
The documents are revealing, said Krigsman, the IT specialist who reviewed the reports for MPR News.
"The quality assurance is really screwed up," he said.
In general, the reports show that "there are lots and lots of problems related to how the project is being managed," Krigsman said.
The reports also show that MNsure officials launched the website well aware there were problems with the Curam software.
A browser problem would eventually prove especially vexing for some users even after the website's launch.
In fact, MNsure officials knew there were problems with the Curam software as early as May 2013, Baden said.
"I call them the famous list of 55," Baden said. "We had 55 issues with Curam."
Starting then, Baden said one of IBM's top officials visited St. Paul regularly to whittle down the list. While the list of Curam problems fluctuated over the summer, Baden said some issues remained when MNsure launched.
A month ago, a MNsure review conducted by consulting firm Optum found that the Curam software had 108 defects - more than double any other vendor.
DESPERATELY SEEKING DATA
MNsure has so far paid nearly $400,000 for three independent progress reports.
But MNsure board chairman Brian Beutner said he was only vaguely familiar with the reports when asked about them last month. He said he wasn't aware of the problems outlined in the reports, either.
"Why did I not know ... management was a concern? I'm not seeing the reports," Beutner said. "It's not being brought up, and part of that is you rely on management to make the judgment call."
The board became legally responsible for MNsure in August 2013, but started meeting as early as May.
Beutner, however, said the board got scant details about what was going on behind the scenes at MNsure, getting most of their information orally at board meetings from Todd-Malmlov, who is known for being highly intelligent but tight-lipped.
Todd-Malmlov resigned in December amid mounting problems with the site. She did not reply to requests to be interviewed for this story.
Bad information from MNsure led to an embarrassment for the Department of Human Services.
The department told the Office of Legislative Auditor that MNsure's software had fixed a long standing problem with certain eligibility determinations. Weeks later, Commissioner Lucinda Jesson had to acknowledge the problem had not been fixed.
Organizations with a stake in MNsure's success say they had trouble communicating with the agency's top brass, too, and it was often difficult to decipher who was in charge of day-to-day operations aside from Todd-Malmlov.
As the summer of 2013 approached, insurance companies were getting nervous that MNsure was running out of time to test the system. Industry executives wanted to craft contingency plans with MNsure, but their suggestions fell on deaf ears for months.
"Anytime you have to build something this complex and ... this difficult, you test, you test again and then you go back and test again," said Medica's Coleman. "We really ran out of runway. So [insurers] were working with folks at MNsure to ask for contingency planning and unfortunately we didn't get to that point as early as we would've liked to."
Internal emails show insurance companies were also seeking information about documents called 834s - electronic files that provide insurers the information they need to enroll people in their health plans.
"Please, please, please," wrote an official from UCare to MNsure on July 24, 2013, about two months before the site went live. "A sample of a MNsure 834."
The email was signed "Desperately Seeking Data."
Beutner, MNsure's board chairman, said he regrets the agency wasn't more forthcoming.
"I think if I could point to one of the largest failures of MNsure, it's been a communication failure," he said. "It's been managing the expectations of what was actually being built, when it was going to be delivered."
RAISING THE ALARM
Despite all the warning signs, Beutner said he doesn't think anyone suggested delaying the site's launch to ensure it would work properly.
"The federal government didn't give [a delay] as an option," Beutner said. "It was very clear from everybody involved that October 1 was a drop-dead date -- we're going to be up and running," Beutner said.
Even as problems piled up throughout the fall, MNsure's former top official, April Todd-Malmlov downplayed the issues.
"At this stage, I think the website is doing a very good job," Todd-Malmlov told MPR News three weeks after the site went live. "Does that mean it has everything that we ultimately wanted to have? No."
Not only was the depth of the website's problems hidden from public view, MNsure kept running ads on TV and radio encouraging people to enroll in a health plan. Getting enough people to sign up for insurance is crucial for MNsure, not only for its mission but because the agency's funding will eventually be tied to the amount of money consumers pay in premiums.
MNsure has so far enrolled 90,000 people in health insurance. But at that rate, the agency will suffer a budget shortfall by 2015.
And it's unclear how many of those new policy holders have been enrolled by hand. Right now, the system is cobbled together by manual workarounds that are sucking resources at MNsure, county offices, clinics and at insurance companies selling policies on the site.
In the coming weeks, MNsure officials will announce a plan to right the troubled site. Interim CEO Scott Leitz, who replaced Todd-Malmlov, has said making sure people get coverage is his top priority between now and the end of open enrollment in March.
In the meantime, some people continue to sign-up on paper.
That's how Fink, executive director of the Nucleus Clinic in Coon Rapids, spent her New Year's Eve, the last day to sign up for a policy that would take effect Jan. 1. She called four people she'd tried and failed to enroll through MNsure's website.
"They all came in and I gave them all coffee and we made copies of the paper applications," she said. "We got them all done and we faxed them in."
MNsure's performance has been disappointing for IT leader Tom Baden.
But there certainly wasn't a lack of commitment among MNsure's staff, he said.
"I watched people work themselves to the bone over weekends, holidays, literally put their lives on hold for this," Baden said. "I admire them for that effort."