Survey research increasingly relies on automatic tools to classify open-ended occupational data. With the rise of machine learning (ML) and large language models (LLMs), this task is shifting from post-survey coding to live classification during interviews. Respondents actively evaluate algorithmic suggestions, creating a joint human–machine decision process where data quality cannot be assessed by accuracy metrics alone. Drawing on human–computer interaction and survey methodology, we propose the COARSE (Classification Outcomes and Respondent Engagement) evaluation framework, which distinguishes five outcomes: accurate classification, misclassification, omission error, commission error, and appropriate rejection. Evidence from a representative German survey deploying an interactive ML-based instrument (OccuCoDe) shows that respondents cannot reliably safeguard data quality when algorithms fail. Instead of rejecting poor suggestions, they often settle for “good enough” answers. When correct options are present, errors reflect misjudgments of subtle distinctions. When respondents reject machine suggestions, they report higher task difficulty afterward, especially when valid options were overlooked. These results show that data errors extend to human cognition and survey interaction, going beyond established machine learning metrics for algorithmic decision-making. As humans and machines collaborate in the survey-answering process, the COARSE framework offers a new lens to evaluate data quality and improve automated coding systems.