General
The drafting of the CPA exam multiple choice questions is outsourced to third party companies. These drafts are then reviewed and approved by AICPA volunteers who serve on one or more content committees. On any given CPA exam that is administered at a Prometric facility, approximately 15% of the multiple choice questions are designated as “pre-test” questions. These questions are inserted into the candidate’s exam in order to develop good data on possible new questions. These pre-test questions do not count for or against the candidate’s final score.
Yes. The format of the CPA exam and the pool of multiple choice questions and simulations is the same in every state.
The CPA exam continues to test knowledge and skills that have been deemed to be important for entry-level CPAs in order to protect the public. However, under the computer-based format, the exam now focuses more on skills that are important for entry-level practice, such as research and communication, which were difficult to assess in a paper-based environment.
The AICPA believes that the cost of taking the exam is fair for the following reasons:
- The cost of the CPA exam is still in the middle range of other professional licensing exams.
- There has been no indication from candidates that the cost of taking the exam is a barrier to becoming a CPA.
The survey results from Prometric test takers and other focus groups are compiled and evaluated by the entire “enterprise” (AICPA, NASBA, and Prometric). Several changes have been made because of the feedback from CPA exam applicants. These changes include, among others, the following:
-
- NASBA has upgraded their phone system for better customer assistance and response times to exam applicants.
- AICPA has changed the timing of the release of candidate scores. Originally, the scores were released once during the testing window. Now the exam scores are released twice during the testing window. For example, during the January,-February-March testing window, scores would be released at the end of February and March.
The AICPA does interact with exam review companies, but provides them with no additional information than what is already in the public domain. AICPA representatives make sure that review companies have the latest, correct information. Periodic face to face meetings and phone Q&A sessions are conducted to facilitate dialogue.
Additionally, the AICPA believes that many do offer sufficient materials for an applicant to pass the CPA exam. However, it does not evaluate or endorse any review program. It is a personal preference whether CPA candidates choose to study with a CPA exam review company.
CPAs at the manager level are encouraged to volunteer for AICPA committees. CPA candidates are encouraged to thoroughly answer the Prometric surveys at the end of the exam. On a general note, the AICPA conducts focus groups and other market research with CPAs and CPA candidates. A recent focus group research project was just completed with eight different groups in four states.
Grading
The multiple choice portions of the exam are graded by computer programs. The exam is scored using Item Response Theory (IRT). According the AICPA website: “IRT is a well-established psychometric approach to scoring used by licensing and certification examinations that administer many different test forms. Based on the large amounts of data that are collected in pretesting, the difficulty level as well as other statistical characteristics of examination questions are known and taken into account in scoring.” Test-takers can rest assured that the pre-test questions they answer and the data that is subsequently collected are put to good use in future CPA exams. Pre-test questions are assigned a difficulty level depending on how well test-takers score on them.
The written portion of the exam is graded by both computer programs and humans. The sophisticated computer programs use advanced algorithms to search for certain words and word combinations that will assist in the scoring of the exam. Every written exam is graded by at least one person though to ensure the accuracy of the computer grading.
Tests are graded in a timely fashion once taken. Because there is no “curve” it is not necessary to wait until all of the tests have been taken. Candidate scores are not determined directly by the scores of their peers during a particular testing window. Please refer to the discussion above regarding Item Response Theory.
NASBA publishes detailed test result data in a publication entitled “Candidate Performance on the Uniform CPA Exam.”
Computer-based Exam
One of the primary goals of the new computer-based exam is to implement realism into the CPA exam. According to Mr. Johnson, the new exam is responsive to this goal in testng important skills to protect the public and using technology that is used by CPAs.
Some additional areas that may be considered as a source for future test questions (depending on the results of the practice analysis) may include:
-
- International component of tax, audit and accounting
- Enterprise risk management
- Impact of PCAOB
- Fraud
- Any skill or issue that is important for entry-level accountants to know in order to protect the public.
The AICPA will continue to make sure that the exam has high fidelity with the practice of accounting in the real world. Through practice analysis surveys the AICPA will get feedback about what is important to the profession.
As discussed above, all test questions are classified according to their content and difficulty level before they are administered on an operational test. This classification is based upon how well candidates perform on them when they are pretest questions. These questions are then administered to the candidate using a “multistage adaptive test delivery model.” Here is an excerpt from http://www.cpa-exam.org/ that explains this testing model: A multistage adaptive test delivery model is used for Auditing and Attestation (AUD), Financial Accounting and Reporting (FAR), and Regulation (REG) multiple-choice testlets. This means that the first testlet presented to the candidate is at a level of moderate difficulty. Subsequent testlets – at the same or slightly more difficult level – are then chosen automatically based on the examinee’s performance on the previous testlet.
There are multiple testlets and routings, based on candidate performance, that are administered each window. Questions in each testlet are determined ahead of time. The individual questions within each testlet do not change. The adaptive portion of the exam is the testlet itself.
As stated above, for Audit, Financial Accounting and Reporting, and Regulation, the first testlet is composed of moderate difficulty questions. The second testlet is responsive to your performance on the first testlet. Finally, the third testlet is responsive to performance on combined results of both the first and second testlets.
The value proposition of a multi-stage adaptive test is that adaptive tests target groups of testing items at a person’s particular proficiency level. Thereby, this form of testing gives a better idea of where a candidate’s ability lies.
From a lay perspective, for example, the theory behind adaptive testing can be better understood through the following. In an oversimplified example, if you wanted to determine a person’s ability to do two-column addition using adaptive testing, you could determine the person’s ability by giving them a series of single-column, two-column and three-column addition problems. This would allow for efficiently determining the persons ability.
CONCLUSION
The Professional Accounting Society of America (PASA) would like to thank Gregory Johnson and the AICPA for their cooperation. Their commitment to providing up to date and accurate information to CPA candidates is admirable.
