Certification Development
Job Analysis
A Job Analysis is conducted to identify and validate the knowledge and skills which will be measured by the examination. The results of the Job Analysis will serve as the basis for the examination. The Job Analysis is conducted with a third-party psychometric partner and includes a scientific survey of industry professionals, certificants, and others engaged in HVAC system cleaning. The analysis focuses on the knowledge, skills, and abilities of a competent candidate.
The Certification Council has determined that a Job Analysis will be conducted every 5 years. This determination is based on the frequency of substantive changes in the field and compliance with standards for professional certification programs.
Examination Specifications
The content for the examination is determined based on the recommended content outline and content area weights developed following each Job Analysis. It's important to note that the content areas or domains of NADCA's certification exams are not represented equally through the test form regarding the number of questions as well as the point value of each question within any given domain.
Item Writing and Review
Item writing and review activities are typically conducted every two years. Item writers and reviewers are required to complete item development training prior to writing and/or reviewing exam items. Following the training session, item developers are granted access to the online item development portal. All item development activities will be facilitated and monitored by the psychometric vendor.
Once items have been written by trained Subject Matter Experts (SMEs), they will be reviewed by an SME panel and the psychometric testing consultant.
Examination Assembly & Approval
Draft examination forms are assembled by the psychometric consultant to meet the exam specifications and reviewed and approved by an SME committee appointed by the Certification Council. The draft form(s) will meet the requirements of the test specifications with respect to content and weighting.
Establishing a Passing Point
The passing point for the exam is established using a criterion-reference technique. Qualified and trained SMEs participate in determining the passing point working with the psychometric consultant. The process used to determine the passing point is documented as part of a cut score study report. A standard setting study is conducted following completion of each Job Analysis study.
The final SME-recommended passing point from this process is approved by the Certification Council. Any modifications to the recommended passing point are documented.
Specific methodologies for establishing the passing score for exams and ensuring test forms are equivalent are selected and documented by the psychometric consultant based on compliance with national accreditation standards.
Equating
Each assembled form must meet the requirements of the exam specifications and exhibit form-level statistical characteristics within range of the tolerance threshold. In addition to conformance to the form-level statistical characteristics for the base form, each assembled form must exhibit a cut score equivalent to the base form.
Test Analysis Reports
Performance statistics are calculated and retained for each exam. Test analysis are conducted by the Certification Council’s selected psychometrician.
If statistical or other technical analysis reveals testing irregularities, the Certification Council may cancel or invalidate exam scores and offer a retest attempt to affected candidates. If further investigation of results reveals that disciplinary action may be warranted, the Certification Council will complete the investigation and review following the process outlined in the Disciplinary Policy.
Documentation
All examination development activities (including job analysis, item development, test form assembly, cut score studies, equating, and test/item analysis) will be documented in formal exam development reports by the contracted testing agency.
If you are interested in learning more about the development of NADCA's certifications or serving as an SME, please contact NADCA's Executive Director, Caitlin McWilliams at cmcwilliams@nadca.com.