« veep talk and the Indian diaspora | Main | The way they were »

August 16, 2020

Comments

It’s a hard problem made worse by an incompetent Department of Education.
(The Secretary of State Gavi Williamson is a running joke.)

France, for example, has dealt with it by awarding average grades and funding an extra 10,000 university places:
https://www.theguardian.com/education/2020/aug/12/school-exams-covid-what-could-uk-have-learned-from-eu

Teachers spent a considerable amount of time and effort producing predicted games for their students (at the request of the DoE) for use in coming up with the national estimates. That work was essentially thrown out*, and the graces produced by an arbitrary algorithm.
Clearly in any year, a significant number of students will fail to attain their predicted grades. What seems to have happened here is that those failures were applied arbitrarily, in order to achieve a similar overall result to a normal year.
To make it worse, the arbitrary marking down was skewed by school results history, so if you were a bright kid in a poor comprehensive, you were far more likely to be screwed (and also less likely to have sat mock exams which you might have been able to rely on).

* Except for those for classes of very small size.
Which increased the relative advantage of private schools, which tend to have smaller class sizes, and teach a broader range of less common subjects at A Level.

It’s hard to be accurate about the clusterfnck, as it is still unfolding.

https://www.theguardian.com/education/2020/aug/15/controversial-exams-algorithm-to-set-97-of-gcse-results
.... The government’s response to the debacle was thrown into further confusion on Saturday night after Ofqual announced students could use coursework to challenge A-level grades, only to withdraw the policy hours later. In a brief statement, Ofqual said the policy was “being reviewed” by its board and that further information would be released “in due course”, without explaining what elements of the policy had caused a problem....

What makes it worse is the short available time between the awarding od grades and allocation of university places. A lot of these kids will either miss the chance of a place at their chosen university, of have to wait a year.

Places are (always?) allocated by the results of tests called A-levels.

Well up to a point.
If a university really wants you (or if they’re desperate to fill a course), they can make an unconditional offer - about 8% od places last year.

Also FWIW, OFQAL - the exam standards regulator - has for some time been held in utter contempt by teachers I know.

For anyone with a higher boredom threshold, and more time on their hands than me, this is Ofqual’s several hundred page apologia for the mess:
https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/909368/6656-1_Awarding_GCSE__AS__A_level__advanced_extension_awards_and_extended_project_qualifications_in_summer_2020_-_interim_report.pdf

Nigel, thanks for the info, though not sure if I'm ready to go thru the pdf just yet.

I did find this

https://ffteducationdatalab.org.uk/2020/08/a-level-results-2020-how-have-grades-been-calculated/

Well up to a point. If a university really wants you (or if they’re desperate to fill a course), they can make an unconditional offer - about 8% od places last year.

I don't know if it's changed, but in my youth an "unconditional offer" meant two Es at "A" level, which was in those days the lowest possible pass grade in two subjects. And to attend university, you used to have to have at least 2 "A" levels and 5 "O" levels, unless special dispensation for the latter was obtained, for example for dyslexia.

But now, in any case, there is a rival system (still not used by most) for getting into UK universities, the IB or International Baccalaureate. I don't know what's happened to the IB's grades in the pandemic.

I haven't finished this yet, but it was sent by a very reliable friend, and is a 25 tweet thread by a Professor of EU Law at Bristol University. It is apparently rather enlightening, at least about the way universities are reacting to the current mess.

https://twitter.com/syrpis/status/1294634399749865474

It’s a good thread, GFTNC.
What it doesn’t really highlight is that universities can face very stiff financial penalties from government if they take on more students than their official numbers cap agreed with government.
https://wonkhe.com/blogs/student-number-controls-are-back-with-deadly-penalties-for-over-recruitment/

So the system forces universities to guess at the number of offers they need to make to fill their numbers of places, makes those conditional offers legally irrevocable, and at the moment threatens large penalties if it turns out universities have guessed badly.

GftNC, thanks, especially for that twitter thread. It seems like the offers have been reduced to 'firm choices', so the numbers they accept are probably not the 2x, but perhaps a slight increase? So if I were king, I'd be having schools integrate online teaching as a way of dealing with infrastructure.

Of course, this would also entail hiring more teachers rather than simply stuffing extra bodies into zoom rooms, but this would be a good place to put money into, especially to deal with what will surely be a covid recession.

students choose the subjects with an eye to the university they want to attend and the subject they want to study (no changing majors multiple times like I did)

I think this may be the part that I find most appalling. You expect someone in their late teens to know what subject they want to focus on? When there are university subjects, at least in the US, which secondary schools not only don't expose students to, they never even get mentioned! I certainly never heard the term Mechanical Engineering in school. Let alone Anthropology.

And also, it seems, what career they want to pursue for a lifetime. Would have been tough on me. When I was in high school, computers were barely a thing outside a few military research groups (think Grace Hopper). Even when I was an undergraduate, there was a grand total one one (1!) computer software class available (FORTRAN programming).

It all seems designed to minimize innovation. Not optimum in today's world.

Yes, I have always thought we were forced to specialise too early (at 16, in most cases, when choosing A levels, thus already separating out the arts from the sciences), and that the US system is better in that way (albeit our batchelor degrees are three years, whereas yours are I believe four). The breadth of education available to a US liberal arts student (or even science students as I understand it) at a really good university makes for impressively widely educated people. And I observe that here at ObWi in practice. Snow's The Two Cultures applies much more in the UK than the US it seems to me. But I do think the "really good university" bit is key.

The breadth of education available to a US liberal arts student (or even science students as I understand it) at a really good university makes for impressively widely educated people.

What's impressive is that nearly equal breadth of choice is available even at quite mediocre colleges here. (Though not, admittedly, at diploma mills.) And while a science (or engineering) major may have a lot of required courses, there is still opportunity, and usually a requirement**, to take several liberal arts classes.

Note also that it is not uncommon for a student to start out with a STEM major and switch to the liberal arts before graduation. Switching the other way is much rarer, primarily because of the need for lots of math classes that STEM students do in their first 2 years, but liberal arts students typically don't.

** The usual term of art is "breadth requirements," and pretty much every major at every university has them. My impression is the universities in the UK have a level of specialization that we don't see here until grad school.

wj, I agree with much of what you say, particularly about the breadth requirements and so on for the STEM majors. But as for the "quite mediocre colleges" point, the equal breadth of choice of subject might be available, but I am not so sure about the level of the teaching etc. Out of interest, how would you establish what is a "diploma mill"? Is it just "college" v "university", or is there something else?

"diploma mill" is a business which is set up to get someone a degree -- so if a job requires "a college degree" (as depressingly many do for no obvious reason) they will have one. Any education which results is entirely incidental. Great money spinner for the owners, however.

Here the distinction between a college and a university is mostly that a university will encompass multiple colleges. That is, it may have a College of Letters & Science, a College of Engineering, a College of Agriculture, a Business School (for MBAs), a Law School, etc.** All those colleges represent some level of specialization, even though their breadth requirements often cross the boundaries between them.

A college (sometimes officially named an "institute") will typically be more specialized. There are some which are independent of any university, in addition to those which are part of one. But it's far more an issue of size, and somewhat of specialization, than of quality. For example, MIT and CalTech are colleges, but nobody doubts the quality of their teaching.

** I admit to cribbing from what my own alma mater has.

I'd actually argue that the best education to produce a well rounded college graduate in the US comes from the small, private, secular* liberal arts colleges. These are also, ironically, the most endangered colleges in the US.

If I had my druthers, no one would be able to declare a major until after they complete their lower division studies and the lower division courses would all be highly interdisciplinary and project based. First year classes would be pass/fail. Students could begin to specialize in their junior year, but cross-disciplinary work would still be highly encouraged.

Never happen in a million years. Too much institutional power baked into the structure of the schools and disciplines.

I still don't know how you would know what was a "diploma mill", since presumably it would have to be calling itself either a college, or university, or maybe institute, if it was giving out degrees!

But as for the rest of your comment, I am astonished. I had understood (even from a recent google) that generally, US colleges only taught undergraduates for batchelor degrees, and conferred no graduate degrees, nor undertook any postgrad research activities, unlike universities. Well, you live and learn.

*secular in my 1:26 not being a hard division, but rather including those colleges that may have some denominational tie (historical or ongoing) but without any orthodoxy requirements for their faculty or students.

The US tends to use "university" and "college" interchangeably. Though there are plenty of colleges that offer some form of graduate degree.

The diploma mills are mostly in the for-profit sector and market themselves to working adults. They tend to concentrate heavily on credentialing degrees.

If I had my druthers, no one would be able to declare a major until after they complete their lower division studies and the lower division courses would all be highly interdisciplinary and project based. First year classes would be pass/fail. Students could begin to specialize in their junior year, but cross-disciplinary work would still be highly encouraged.

I definitely see the attraction. (After all, I had a liberal arts degree to go with my engineering degree. It was my life.) But your plan only works for STEM majors if those lower division classes include calculus classes. For which non-STEM folks have no use. But without them, it would take more like 3 years of upper division classes to finish a STEM major.

Or, I suppose, you could have everybody get a liberal arts degree first, and only then tackle STEM subjects if they wish. But that would also require either getting back to minimal cost colleges** or racking up even more college debt.

It also occurs to me that, if we are going to push out entry to the workforce another 2-4 years, we are also going to want to look at pushing retirement ages out as well. Maybe not to the point, relative to life expectancy, we were when Social Security was enacted. But well past 70.

** Which, admittedly, I think we ought to do anyway.

Liberal arts includes mathematics for a reason. I'm all in favor of STEAM, as opposed to STEM. Integrate the mathematics into the others.

Shift the focus in the "advanced math" courses away from the calculational aspects and onto the mathematical thinking side of things. Teach calc and the others through theory and application and let those not so numerically inclined use software to do the calculations so long as they understand and appreciate what is being done and why and can think through the mathematical problem in a productive manner.

My life would be so much easier if the mathematically challenged but bureaucracy attuned assessment gurus in charge of curricular development had an ounce of logic and critical thinking backing up their self-justifying quant. Make them work through Euclid before they are allowed to do anything that puts them in power.

wj, It occurs to me that moving the retirement age older is going the wrong way. While unemployment rates have hovered just under 4 % the covid has pushed them to just under 14% for those over 55. The reality of age is in our society means that nber could never recover, as it is much harder to find a job after 55.

In addition, we keep talking about the potential declining need for labor and high unemployment for millennials, both exacerbated if you raise the retirement age.

IMO, the delay getting into the workforce, plus a reduced retirement age is the only rational way to reduce the workforce, raise wages that allows taxes to support the minimal cost education and increased retirees. It is a plan we could adjust as automation and specialization limit the available options for low education jobs.

I did poorly in calc in college, but that was largely a product of pedagogical approach rather than of a lack of mathematical ability on my part. My dear friend who is an applied maths Ph.D. assures me that he would have been able to get me through any and all required math with a more conceptual. less methodological approach.

Offer both types of classes and let the students choose which is productive. Let them take their time and make the lower division math classes stretch classes. You take them until you either get through the whole thing and qualify for specialist classes or you do well enough for general purposes (without access to specialist classes that require that math). You should still be able to work on projects that require higher math, you just need to work cross-disciplinarily with someone who can make sure of the numbers.

Same can be said of communication skills, FWIW. I have tons of students who can calculate like crazy, but can't abstract or think critically. The logic and inductive thinking I force them through makes them not just better writers, but better engineers.

For example, MIT and CalTech are colleges, but nobody doubts the quality of their teaching.

Sorry wj, but if this is an attempt to clarify to a non-American the American usage of "college" vs "university" by the assertion that MIT is a college and not a university, it's no wonder GftNC is getting more rather than less confused.

From Wikipedia:

Massachusetts Institute of Technology is a private research university in Cambridge, Massachusetts. The institute is a land-grant, sea-grant, and space-grant university, with an urban campus that extends more than a mile alongside the Charles River...Founded in 1861 in response to the increasing industrialization of the United States, MIT adopted a European polytechnic university model and stressed laboratory instruction in applied science and engineering....

Also:

The California Institute of Technology is a private research university in Pasadena, California...

I think this US News explanation" of college vs university in USian terms is pretty good. Basically, in my mind, and the article more or less corroborates this, a college doesn't give graduate degrees, a university does.

Second nous's comments about non-secular liberal arts colleges. There are three within an hour of where I live: Colby, Bates, and Bowdoin. Undergrads only, very very high quality education, also expensive, but in varying degrees they give a fair amount of help to kids who need it. Bates has something I wish had been there for me: an orientation for kids who are the first in their families to go to college. Or, to put it a different way, who continue their education beyond high school. (Another nuance of American usage.)

And as another little linguistic entertainment, the Wiki on MIT shifts seamlessly from calling it a university to calling it "the institute." That was the most common term used when I was a student and later when I worked there. Or, to be cheeky, "the 'tute."

P.S. Meant to add, I don't have any light to shed on "diploma mills." There are plenty of ranking entities that would give the interested aspiring college student a clue. ("College student" = post high school person........) ("Non-traditional student" = someone who goes to college later in life) (Or maybe I'm outdated even on that.)

Colleges that do give out graduate degrees tend to offer some form of terminal masters or MBA. These degrees attract professionals and teachers looking to move up a salary scale that requires an advanced degree. They tend not to offer any doctoral programs.

Cross-posting a bit, sorry, I've got distractions.

But I also want to echo wj's and nous's reference to "breadth requirements."

A gazillion years ago when I was an undergrad, MIT required a minimum of one "humanities" course per semester, on average, which ended up being 20-25% of your coursework. Conversely -- speaking as someone who ended up majoring in humanities ("Course XXIB-2, Humanities and Science with a Concentration in Literature and a Minor in History") -- a minimum of about 1/3 of the course work had to be in science/math/engineering areas (the acronym "STEM" hadn't been invented yet), no matter what you majored in. There was a "humanities requirement" and a "science distribution requirement" -- the latter being at least three courses, selected from three different science/math departments. Plus you had to take at least one lab-based class.

I haven't kept up, but I don't think it's all that different today. The point being: even at one of the most "polarized [paralyzed, as the wags said in 1970] around science and engineering" universities in the world, as an undergrad you still had to have some breadth to your class selection.

It occurs to me that moving the retirement age older is going the wrong way. . . .

IMO, the delay getting into the workforce, plus a reduced retirement age is the only rational way to reduce the workforce

I understand your point. However I note that, for a great many people, their job is, to a great extent, not just their social life but the focus for their life. When they retire, not unlike people who end up unemployed, they are somewhat lost. Which has a variety of negative effects.

It has occurred to me that one reason why women live longer than men may be that they tend not to end up "out of work." In that meals still need to be prepared, laundry done, and housecleaning. For good or ill, women are the ones who are expected to still be doing those things, regardless of age. A man who retires, unless he has or finds a hobby, tends to end up just sitting around. A woman, in contrast, has stuff to do regardless. It may not be fulfilling work, but it might be it provides something important.

"Breadth" as implemented in big universities in the US has taken the inevitable consumerist turn and consists mostly of cheeky, lite approaches to the subject, heavy on pop culture. The good courses do this with a deep engagement that actually brings the topic alive. The poor versions either pander superficially for the sake of enrollment numbers or fail entirely at the outreach side of their commission and reinforce the idea that the subject is only for specialists.

This is what happens when a) the regents and administrators that run things are mostly drawn from business backgrounds and treat education as a service industry and b) budgets are controlled by disciplinary specialization that gathers resources into distinct "silos."

I'm not a bitter cross-disciplinarian. No, not at all.

Janie, Thanks for that US News link. I confess I was foundering a bit, trying to work up an explanation on the fly. Perhaps that will give GftNC more insight.

I'm not a bitter cross-disciplinarian. No, not at all.

I sympathize. I'm far removed from all the practical aspects now, but it's a shame how higher education has "evolved." Along with a lot of other shames, it's fair to say.

One more comment: unless it's another thing I'm out of touch / outdated about, in the US, if someone talks about "going to college," that means almost anything post high school; it has nothing to do with the distinction between a college and a university. No one says "I'm going to university." To me, that would immediately identify someone as British, or heavily shaped by British usage.

lj -- your use of "uni" bemuses me for the same reason. Even in the OP, which is framed as a set of musings and questions about the UK situation from someone more familiar with the US, you use "uni." Is that the equivalent of Japanese usage, or what?

Not only does "going to college" encompass both both colleges and universities. It also includes attending a 2 year community college / junior college -- whether one is using it for lower division courses in anticipation of transferring to a 4 year college, or just intending to stop after two years.

Thus when I say that all but two of my high school graduating class went to college the next year, that includes the plurality (maybe 1/3?) who were at community college.

My "college" experience involved one year of a narrowly focused private engineering school (very bad fit), two years of a small liberal arts college (that offers two MA degrees), one term each at two community colleges (refreshing needed transfer credits), and two R1 state universities (BA, MA/Ph.D. respectively). There are huge cultural differences across that spread.

nous --

1) Thus illustrating why it's almost impossible to explain the vocab. ;-)

2) Fun times when you have to have transcripts, right? Or is there now a way to collect them all from one depository?

Thanks for all the explanations, wj, Janie and nous, and yes that US News link was particularly helpful. Now I come to think of it, I wish I had tried for a US liberal arts college, or at any rate done US style further education, I think it would have suited me much better than UK style. I can't now remember why I didn't suggest it, I think my parents might have gone for it. Ah well, many decades too late.

By the way, "uni" seems to have taken over here in recent years. I don't like it, we never said it when I was young, it was always "university".

2) Fun times when you have to have transcripts, right? Or is there now a way to collect them all from one depository?

Don't know what that process looks like now with registrars using digital records and requests being much easier to arrange. Last time I needed transcripts from all of them it was still a phone-and-limited-Internet prospect requiring at least a couple of days to get all the particulars together for each institution (student ID #s from a decade+ earlier, paying off any outstanding fees, getting cashier checks, etc. across three states).

That was between community college 2 and R1 prime. Grad school (R1.2) was happy to accept R1's transfer info for the transcript and just contacted the registrar at each of the others to ensure that there were no red flags in my records.

I'd bet that today it would just be one day of hitting the registrar's page at each and filling out the forms.

All of it looks easier in retrospect after a few years spent on the academic job market. The smaller the institution, the more they demand just to complete an application.

I don't miss that, either.

Fun stuff, thanks all. In answer to Janie's question, while the majority of foreign teachers here are US, there is a large minority of UKians and I picked up uni from them I think. Plus it is easier to type.

But the fact is that Japanese higher ed, which pre-war was built on the German gymnasium model, but post-war imitated the US model (and in some cases was forced to imitate), but with little regard to the function of higher education, simply adopting the form, has created a situation where I just grab at whatever term is handy.

Ironically, I think there is a relation between the state that UK higher ed is in now and previous experiments with the sector. The so called Binary Divide
https://en.wikipedia.org/wiki/Binary_Divide

created a two tiered system which allowed the rapid expansion of higher ed thru polytecnics while maintaining the 'brand' of universities. In 1992, that divide was erased and the polytechnics were converted into universities as part of Thatcherite reforms. This ping-ponging allowed the elite universities to maintain their claim on funding while having the UK look a lot better on comparisons of the number of university educated in the population.

Up to a point, lj.
Some of the newer universities are quite successful both financially and academically.

The student funding model (and the much higher fees from overseas students) have up until now produces surpluses for rapid expansion for the successful ones - while the less successful have struggled..

There are huge disparities in research funding, but even there, the technology departments have made headway, particularly with money from industry.

The pandemic has left many of the weaker ones in financial dire straits, though.

Latest developments make the exam situation sound even worse than I thought.

A-level grades 'drop below three-year average', new analysis suggests
https://www.bbc.co.uk/news/education-53799860

This is abysmal:
...The Sixth Form Colleges Association (SFCA) said it looked at 65,000 exam entries in 41 subjects from sixth form colleges and found that grades were 20% lower than historic performances for similar students in those colleges....

Some of the newer universities are quite successful both financially and academically.

Thanks Nigel, I'm curious what your take is on that. Here in Japan, there is a very tiered system of former Imperial Universities, elite private schools, lower tier national universities, prefectural universities (brought in after WW2 by American reformers who wanted to replicate the state university system in the US) and private universities. It's not airtight, but generally holds. Is a similar set of tiers seen/perceived in the UK?

There is, but not such a variety of providers, I think.

And some of the better managed/more entrepreneurial new universities have succeeded in carving out their own niches.
The proliferation of new scientific/engineering specialisms has made it much easier to set up competitive departments and attract students than it might have been a few decades back.

This post explains what's fundamentally wrong with the algorithm used.

A gazillion years ago when I was an undergrad, MIT required a minimum of one "humanities" course per semester, on average, which ended up being 20-25% of your coursework.

was the same at RIT in the late 80s. they use the quarter system, not semesters, but every quarter you had to take at least one liberal arts class. ends up being about 1/4 of the class time. the liberal arts classes were definitely easier than the tech classes, though.

"Going to 'uni'" is a much pricklier proposition in Japan than the UK, I believe, but it is a popular choice for urchins.

This post explains what's fundamentally wrong with the algorithm used.

Sounds persuasive.

What it doesn't mention, though, is that the teacher predicted grades were not made on the same basis as previous years.

As I understand it, in a normal year, the estimated grade is to help universities in making their offers to students. It's an optimistic view (the best result the student is capable of in that subject) rather than a hard prediction, so it is not in the least surprising that it appears statistically poor at predicting actual outcomes.
But that is not what teachers were asked to do this year - they were asked for their best prediction of actual outcomes had the students at the exam.

What it also doesn't highlight is that teachers were refused the option of ranking students equally in a particular subject. So for any given pair, or trio of students performing similarly, the discrimination between the two/three is almost certain to be utterly arbitrary.

a much pricklier proposition in Japan than the UK, I believe, but it is a popular choice for urchins

Now that kind of uni I can enthusiastically go for!

And, after a week of confusion, a U-turn.
https://www.bbc.co.uk/news/uk-53810655

One of my cheats as an EE major was to take Intro to Logic, which was a philosophy class, to satisfy one of my Hum/Soc (short for humanities/social sciences in Rutgers engineering parlance, pronounced "HUM-sock") requirements. Boolean algebra with different symbols, more or less.

There was an appropriately binary aspect to that class in that there were two kinds of students - those who got it, for whom it was very easy, and those who didn't, for whom it was very difficult.

A gazillion years ago when I was an undergrad, MIT required a minimum of one "humanities" course per semester, on average, which ended up being 20-25% of your coursework

At the same time, I think, I was an undergraduate at Vanderbilt, where breadth was broader. The requirement for a major was 30 semester hours, out of 122 required for a degree, and no more than 36 could be counted towards that 122.

So you had to take stuff outside your area, and there were several specific "distribution requirements" you had to meet as well - foreign language, social science, science, etc.

This scheme was, as I recall, fairly common at universities at the time, according to reports from friends.

To add to the college/university confusion, Vanderbilt, like many schools, is and was a university, made up of colleges - Arts and Sciences, Engineering, etc, and of course has doctoral programs.

One of my cheats as an EE major was to take Intro to Logic, which was a philosophy class,

A good friend, mathematically talented, did something similar.

As a freshman he signed up for an advanced undergraduate/graduate class in symbolic logic in the philosophy department.

He was the only undergraduate in a class of seven or eight, and was initially intimidated, but soon reported to me that the graduate students all "sat around and smoked pipes and talked about Nietzsche," but none of them could do the problems, which weren't that hard.

One of my cheats as an EE major was to take Intro to Logic, which was a philosophy class,

At my undergraduate school, if your major required calculus or higher, you couldn't take Intro to Logic for credit. Instead I took metaphysics (of which I remember nothing) and ethics (which was quite useful).

He was the only undergraduate in a class of seven or eight, and was initially intimidated, but soon reported to me that the graduate students all "sat around and smoked pipes and talked about Nietzsche," but none of them could do the problems, which weren't that hard.

Thanks for that, byomtov. I'm still laughing over it.

https://www.theguardian.com/education/2020/aug/21/ofqual-exams-algorithm-why-did-it-fail-make-grade-a-levels

An analysis of the equation for determining grades.

A short summary of that article (to the extent I understood it):
If you're a good pupil from an elite background (school), you're good. At least, good enough.
If you're a great pupil, but from a big, non-elite school? Well tough, you get classified by your (presumptive) class, not you accomplishments.

Shorter still: preserve the elite. Period.

Not that American universities don't have similar problems -- think "legacy" admissions at Ivy League schools. But that equation makes it particularly blatant.

My own plan for college admissions:


Set two thresholds for SAT/ACT/GPA:
above the "high threshold" -> 100% admit
below the "low threshold" -> 0% admit
in between: random selection. The point of the thresholds is that "below the low threshold
is unlikely to survive to graduation; above the high threshold is damn certain to survive
to graduation".

Sure, if you want to throw some 'legacies' into the "100% admit" pool, fine. The system
is fair, race-/class-/gender-neutral, and super easy to administer.

Did I mention that I like random number generators? I might still have a 1600bpi magtape of hardware-generated random numbers from ye olden dayes. No tape drive to read 'em, though.

At minimum, need something to address those who went with a 2 year college initially. Some folks just aren't ready for college at 18, but by 20 they are fine.

I also have an issue with the SAT/ACT. Two issues, actually.
First, there's no recognition that some people are "test sophisticated". That is, they can think like a test creator, and thus can often spot a correct answer in a multiple choice test, even when they know little or nothing about the subject.
Second, those tests evaluate (sort of) what you know. That is, what facts you have in memory. Rather than recognizing that, especially today but even half a century ago when I took them, facts can be looked up fairly easily. What you need for college is a) knowing how to track down those facts, and b) knowing how to reason using those facts.

With admissions tests you are at the mercy of how easily the scoring can be automated. Scoring can only be automated by choosing simple, clear cut problems to test that require less judgment and interpretation.

Grading comprehension tests costs more because the work is slow and requires the use of scorers with enough expertise to make judgment calls (and other scorers to make judgment calls about those judgment calls). A good test of comprehension is going to be irreducibly more expensive than the automated test for this reason. When cost is a major driving factor, you usually end up with a watered down product that purports to show more than it actually does.

I think we'd be better off without all the ranking and admissions dog show. Too many institutional resources go into creating the impression of "eliteness" and not enough into actual education. Strip out the bling and aim to create a transformative experience.

It’s not just test sophistication. It’s also how you react emotionally to testing. I was always a good test taker, because I was stimulated without being too nervous. The one time I recall doing more poorly than I would have expected was because I wasn’t motivated enough to sit through the test. I actually ran out of time and randomly filled in Bs, Cs, and Ds for the last however-many questions in the last minute. Normally, I would finish that sort of test with time to spare.

Some people who know the material well enough to do well simply can’t think under the conditions of standardized testing, where you have to demonstrate some degree of knowledge, reasoning, and interpretation on broad subject matter in a particular place, at a particular time, within a particular time, and in a particular way. It’s an odd contrivance when you step back and think about it.

I'm interested in people who give unanticipated answers, like the kid (I forget how old, but not very) I recently read about who answered "What is half of twelve?" with "twe".

Also, of course, the famous Niels Bohr barometer story, which turns out (possibly) not to actually be about Niels Bohr. I know Bohr's granddaughter slightly, I will ask her if she knows if it was him the next time she is in the UK.

https://en.wikipedia.org/wiki/Barometer_question

An amusing take on the differences between British and American education.

"English and American education is VERY different. As someone who was schooled the English way but married an American teacher, I take a look at those differences in this video."
Lost in the Pond: 8 Ways British and American Education Systems Are Very Different (YouTube)

I went straight for dropping the barometer and timing it’s fall.

Dropping science like Galileo dropped the orange...

[ rare Beastie Boys trifecta now in play ]

With admissions tests you are at the mercy of how easily the scoring can be automated.

I have read that software for scoring essays is getting quite good. OTOH, all of the best programs seem to be deep-learning AIs, so there's the problem of not knowing exactly what criteria the optimization has solved for. As an online acquaintance says, "Deep-learning AI was invented to drive mathematicians crazy."

" I actually ran out of time and randomly filled in Bs, Cs, and Ds for the last however-many questions in the last minute. "

Not sure about "most" such tests, but at least for the SAT and similar clones, if you can eliminate ONE of the possible answers, then a random choice of the other answers gets you a net gain.

A #2 pencil has 6 sides. I put 'dots' on the sides, so that I could 'roll a die' to randomly select an answer among the ones that I hadn't eliminated, saving time to use on the questions that I had a better shot at getting correct.

How geeky is that? SUPER-geeky.

Yes, I like random numbers, why do you ask?

at least for the SAT and similar clones, if you can eliminate ONE of the possible answers, then a random choice of the other answers gets you a net gain.

And as I recall, for most of the questions at least two, and sometimes 3 of the available answers can be readily eliminated. Those who are "test sophisticated" figure that out and answer pretty much everything. Those who aren't, skip the ones where they don't know the answer.

Obviously anybody can be taught that simple fact. But only if their high school teachers think to tell them.

The last time I took the GRE -- for some reason the local university where I wanted to take classes wouldn't accept my almost 30-year-old previous scores -- you had to take it on a computer rather than on paper. The software had eliminated one of the common test-taking skills from my youth: on the first pass through the test, answer only the questions where you know the answer. Then attack the ones that are harder for you. The software would not advance until I answered the current question.

I enjoyed the math testing. None of the problems were hard, but they "invited" you to answer the wrong question. You might still get the proper answer, but you would have wasted a bunch of time.

I have read that software for scoring essays is getting quite good. OTOH, all of the best programs seem to be deep-learning AIs, so there's the problem of not knowing exactly what criteria the optimization has solved for.

I've got access to a few of these via my institution. As a writing teacher I don't find any of them particularly useful. They work mostly like a Chinese Box. They are quite good on a formal level for grammar and usage. Beyond that, they reward the use of formulaic constructions that are often at odds with the intent or purpose of the writing. I've seen a lot of tidy, vapid, high machine scoring papers and I've got a lot of students who write messy, insightful, low machine scoring papers.

Yes, essays can be automated, but only if you want standardized students.

My most brilliant students, by and large, were mostly students who had failed their writing placement exams because they had tried to put a twelve page thought into a two page test with an interesting style and came up short as a result.

Best use of essay scoring software, so far as I can tell, is to give students access to them, but only after they have already put their essay through at least three developmental drafts to get the whole thing worked out. The machine can then help with grammar, transitions, and usage. It does jack all for critical thinking and becoming a more reader-conscious writer.

The last time I took the GRE -- for some reason the local university where I wanted to take classes wouldn't accept my almost 30-year-old previous scores -- you had to take it on a computer rather than on paper. The software had eliminated one of the common test-taking skills from my youth: on the first pass through the test, answer only the questions where you know the answer. Then attack the ones that are harder for you. The software would not advance until I answered the current question.

When I last took the GRE in 2004 the overall score was dependent on the results of the first handful of questions, which were used as rangefinders. Get it right, the next question would be more difficult - get it wrong, the next question would be less difficult. Once they found what they thought was the level of the student they would work to find the limits of that range.

Problem was, if you missed the first question then it took several questions to get back to that higher level again, so an initial miss could artificially limit the top bound of your score if you got unlucky.

I was lucky and nailed it for verbal. Did only mediocre on the written, trying to give one of those 12 page answers with 2 page typing speed and far too much parsing of synonyms for precision and connotation.

When I last took the GRE in 2004 the overall score was dependent on the results of the first handful of questions, which were used as rangefinders.

I noticed that in the vocabulary stuff. I didn't notice it in the math, but there was clearly a cap on level of difficulty there: everything was below calculus, no matter that you were getting all of the answers right. Which made that whole part a gigantic waste for the classes I wanted to take, all of which required calculus and differential equations. And one class where having a semester of real analysis was very useful.

The comments to this entry are closed.

Blog powered by Typepad