Appeal from the Circuit Court of St. Clair County. No. 94-MR-221 Honorable Robert L. Craig, Judge, presiding.
The opinion of the court was delivered by: Justice Hopkins
Petitioner, Protestant Memorial Medical Center, Inc., doing business as Memorial Hospital (the Hospital), sought administrative review in the trial court of the decision of the respondents, the Department of Public Aid (Department) and Robert W. Wright, the Director of Public Aid, that the Hospital was overpaid $52,505.77 for outpatient services under the Illinois Medical Assistance program (Medicaid program) and that the Department's sample used in its audit was statistically valid. The court determined that the Department failed to prove that the sample it selected for extrapolation purposes was statistically valid. The court held that the Department was entitled to only $3,169.59 because the Department used an invalid sampling and extrapolation procedure. The Department appeals.
On appeal, the Department contends (1) that the Hospital was not entitled to appeal the Department's decision to the circuit court as the Hospital had "invited" the Department's decision, thereby creating the error, and (2) that the circuit court's determination that the Department used an invalid sampling and extrapolation procedure was against the manifest weight of the evidence. We affirm for the reasons set forth below.
The Hospital is a participating service provider for the Medicaid program, a medical assistance program administered and monitored by the Department. See 305 ILCS 5/5-1 et seq. (West 1996). The Department's rules and regulations require it to audit service providers and recover excess payments made under the program. 89 Ill. Adm. Code §140.30 (Supp. 1987). Pursuant to its rules, the Department conducted an audit of all outpatient services rendered to Medicaid recipients by the Hospital during the period of September 1, 1986, to February 29, 1988. Following two reaudits, the Department determined that the Hospital received $110,477.82 in overpayments, and the Department notified the Hospital that the Department planned to recoup this amount. To determine the amount of overpayment, the Department used a sample and extrapolation method, which is allowed under the Department's rules and regulations. 89 Ill. Adm. Code §140.30(b) (Supp. 1987).
The Hospital disagreed with the Department's findings, and a hearing was held before an administrative law Judge (ALJ) on 46 separate days over a three-year period. At that hearing, the Hospital disputed the Department's determination of the amount of overpayment for specific emergency services rendered and also challenged the validity of the Department's sample used to extrapolate the amount of overpayments. Evidence regarding the sampling and extrapolation method used by the Department in its audit was presented through the testimony of three experts.
Dr. John Nosari, dean of the School of Business and Management at Sangamon State University, testified for the Department as an expert in auditing. Nosari explained the Department's sample and extrapolation method. Nosari defined the Department's "universe" as the number of recipients who received outpatient treatment by the Hospital during the audit period, i.e., 6,134 recipients. Nosari stated that recipients were selected as the universe because medical services are provided to the recipient and the purpose of the audit is to determine if the services and the billing of the services were in accordance with the Department's policies. For the universe, 16,958 services were rendered by the Hospital.
Nosari stated that "a random number generator computer program" selected the Department's sample of 367 recipients from the universe. These sample recipients received 1,410 services. Nosari also stated that random selection is important because each item in the population or universe has an equal chance of being selected, thereby assuring that the sample is representative. Nosari agreed that it is important to relate the universe selected to the purpose of the audit and that different aspects of a universe have different characteristics. Nosari did not believe there is any method for testing the randomness of a sample, and he was unaware of the Wald-Wolfowitz Runs Test (WW test).
Nosari explained that the variation of a universe's characteristics determines the spread of the sample from the mean. Nosari agreed that if a sample is not representative of a universe, that sample is not reliable and the results of the sample cannot be projected to the entire universe. Nosari admitted that services could be used as the unit for the universe and that using a stratified sample would extrapolate to the universe with more accuracy; he stated, however, that the Department's computer is not programmed for selecting services as the universe and sample.
Following the selection of the sample, auditors reviewed the files of the sample recipients to determine if the services rendered were appropriate and were billed properly under the Department's policies. If the auditors disagreed with the Hospital's billing for a particular service, a discrepancy was noted. The primary area where discrepancies arose was in emergency room services, because billing for these services varied greatly depending on whether the Department believed the service to be a true emergency under its guidelines. Nosari explained that the average discrepancy for the sample recipients, defined as the sample mean, was found to be $18.05 per recipient. The sample mean was then extrapolated to the entire universe, i.e., 6,134 multiplied by $18.05, amounting to an overpayment to the Hospital of $110,718.87.
Dr. Ik-Whan Kwon, chairman of the Management and Decision Services (a division of the School of Business) at St. Louis University, testified for the Hospital as an expert in statistics. Kwon reviewed the Department's audit data and determined that the Department should have used stratified services, i.e., grouping services into the three categories of institutional, general medical, and pharmacy, as the unit of analysis for the universe and the sample, rather than recipient. Kwon based his opinion on the fact that the audit evaluated the measurement of the payment based upon the services received and not upon the recipient. Kwon explained that the unit of analysis is important because any statistical information based on the unit of analysis will be applied to the entire universe. After the unit of analysis is selected, then the size of the sample, the sampling methodologies, the statistical tools to be employed, and the statistical techniques to be used are selected.
Kwon also explained that simple random sampling, which was used here, is an acceptable statistical method, but it is appropriate only when the unit of analysis is homogeneous, i.e., when each recipient received a similar amount of either services or payments. If the unit of analysis is not homogeneous, then simple random sampling of the recipients results in grossly exaggerated variations and the randomness creates a biased result. Additionally, Kwon stated that a computerized random sample does not guarantee randomness, as randomness is determined by whether a sample is selected in the fashion that the population is distributed.
Kwon explained that if the sample is not proportionate to the universe, then the sample is not representative of the universe. Kwon found that the proportions of the three categories of services in the Department's sample were not proportionate to those of the universe. Kwon performed a computerized WW test on the Department's sample and population. Kwon stated that the WW test can be used to determine if a sample is randomly selected from a known population. Kwon testified that the results of the WW test revealed that the Department's sample was not truly random and therefore was not representative of the universe. Similarly, the Chi Square test Kwon performed on the Department's data revealed that the sample did not reflect the universe. Kwon concluded that if a sample is not random, efficient, and representative of the universe, or if any of these three criteria are violated, then a sample is invalid and extrapolation is useless.
Dr. Donald Roberts, a professor of mathematics at the University of Illinois, testified for the Department as an expert in statistical auditing. Roberts stated that the Department applied valid statistical principles to the audit, i.e., the Department followed all of the proper steps. Roberts disagreed with Kwon's assessment that the sample was invalid, and in Roberts' opinion, the method used by the Department was valid, so the results were also valid. Roberts believed that Kwon confused efficiency with validity. Roberts agreed with Kwon about the criteria for efficient testing, but Roberts stated that just because one method was more efficient did not mean that another method was invalid. Roberts thought that the validity of a sample was essential, but he did not believe that representativeness was essential. Roberts ...