> The particular VAM system used to estimate this metric may be flawed or even completely broken.
> The particular VAM system used to estimate this metric may be flawed or even completely broken. But at the point in the article this pull quote is located, that argument had not been made. > My biggest problem with the article is it doesn't describe alternate methods of teacher evaluation. VAM may be flawed, but how does it compare to the other methods? If we accept that teacher evaluation based on student performance is necessary (is it? I have no idea!), what's a better way to do it? The main point I took from the article is that the VAM approach is a black box. It is data driven and we have some reason to suspect it should be useful, but we simply have no way to audit that on specific cases. Combine that with the fact that this particular teacher is exemplary, yet got an abysmal score, we are faced with the worst possible result: Someone that should get a great score received a terrible one, and we don't know why. So we can't figure out what went wrong or how to fix it. This isn't a new issue. Statistical models used to be more human-understandable, then over several decades neural networks and big data and so forth have become popular. They often achieve good results, but often remain partially or entirely black boxes. It makes sense to use a black box method to recognize handwriting, for example, if it does well on average. It's fine if it messes up on some samples. But teachers deserve to be treated fairly, each and every single one. That's the tension here - better methods on average tend to be less auditable on individual cases.
Posted by azakai on Nov 4, 2014, 1:40 AM
Tags
Related issues
Raise real-world problems, find people facing the same issue, and discuss solutions.9,000 issues0 comments
reliability (EdTech)
I frankly find the situation around UB puzzling.
I think it depends a little on how and where you work.
Self modifying has some sneaky failure modes with modern CPUs.
> Ban phones from class.
Coolify v5.x
Are you talking about a technical problem, or a conceptual problem?
This is not uncommon in the more blue-collar businesses.
> I wonder though, do people not strace programs any more?
> The particular VAM system used to estimate this metric may be flawed or even completely broken.
> The particular VAM system used to estimate this metric may be flawed or even completely broken. But at the point in the article this pull quote is located, that argument had not been made. > My biggest problem with the article is it doesn't describe alternate methods of teacher evaluation. VAM may be flawed, but how does it compare to the other methods? If we accept that teacher evaluation based on student performance is necessary (is it? I have no idea!), what's a better way to do it? The main point I took from the article is that the VAM approach is a black box. It is data driven and we have some reason to suspect it should be useful, but we simply have no way to audit that on specific cases. Combine that with the fact that this particular teacher is exemplary, yet got an abysmal score, we are faced with the worst possible result: Someone that should get a great score received a terrible one, and we don't know why. So we can't figure out what went wrong or how to fix it. This isn't a new issue. Statistical models used to be more human-understandable, then over several decades neural networks and big data and so forth have become popular. They often achieve good results, but often remain partially or entirely black boxes. It makes sense to use a black box method to recognize handwriting, for example, if it does well on average. It's fine if it messes up on some samples. But teachers deserve to be treated fairly, each and every single one. That's the tension here - better methods on average tend to be less auditable on individual cases.
Comments (0)
No comments yet. Start the discussion.