Thoughts on verification and validation from Judith Curry. With the usual nonsense about “denialists” in comments.
16 thoughts on “Building Confidence In Climate Models”
Comments are closed.
Thoughts on verification and validation from Judith Curry. With the usual nonsense about “denialists” in comments.
Comments are closed.
A lot of BS about not having to V&V the models and trying to gloss over the fact that they cannot reproduce even closely past climate data. I really like the part where she tries to justify by saying that all the models used tend to agree and have been vetted by that paragon of unbiased judgement the IPSCC.
Judith Curry is full of it. She’s a committed warmer. This latest stand of hers is just . an attempt to obfuscate and still claim the warmies are right. Utter crap. She needs to find another field of work. I hear MacDonalds have positions from time to time.
Look, Curry is obviously carrying water for the Warmists (currying favor with them?), but the underlying point shouldn’t be lost. V&V for the models would be valuable, and if it were methodologically valid, would at least give us a common ground for debate.
I don’t endorse the AGW model as it is, but whether one agrees with Curry’s suppositions or not, this is a good opportunity to raise the level of debate on both sides. She may be backing the wrong team (as it were), but at least she is willing to establish some rules and live by them…a big improvement over many in that camp…
One of the fundamental problems with the “hindcasting” and other attempts to validate the models is the -relative- paucity and low skill of the older surface data. It keeps getting lumped and labeled as “The Instrumental Period.” The satellite period (1978+) uses sensors that at least have a shot of measuring “The temperature of a gridcell.” Then the instrumental error is propagated, even though a randomly situated (0.1C error) thermometer has no business being expected to properly measure the gridcell temperature to that same 0.1C.
Well, I appreciate what Judith Curry has to say here. I think that she is not a “warmist” but in fact has annoyed the “consensus” AGW community by her refusal to say everything is peachy on the science side.
And the V&V issue is one of the reasons I am still a skeptic. We would not fly a manned aircraft or launch vehicle designed with codes that have been as sloppily validated as the current crop of GCMs. The climatology community has really got to come up to the standards of computational physics and aeronautical/nuclear engineering as far as these codes go.
As an indicator of how clueless the IPCC is, even in the latest report (AR4), they compare the historical record to the average retrodiction of different climate models. In other words, none of the models they are using is correct, but they can come close to simulating the historical record by averaging over a lot of runs of different models. I am astonished that anyone with any training in computational physics would do this.
“Airplanes are designed using models that are inadequate in their ability to simulate turbulent flow.”
More specifically, CFD is unable to *predict* the onset and structure of turbulence, even in the case of a two-dimensional unsteady flow over a wing section. It *can* be used to “simulate” turbulent flow once the turbulence model is back-fitted to test data. But the CFD modeler knows that all he’s doing is curve-fitting, albeit in a manner that is so abstract that only he really understands it.
Prior to anchoring a CFD simulation with test data, one can force a solution to go turbulent and let it evolve. The result will almost certainly not correspond to the real situation, and it will not become more accurate with time. The important thing to realize, however, is that its accuracy is completely unknown prior to anchoring with test data.
A global climate model involves three-dimensional turbulent, mixed compressible and incompressible, multiphase, unsteady flow. It is a problem so many orders of magnitude more complex than the simple 2-D unsteady (incompressible) wing that it would be impossible to state that we even understand it. Worse, the most important forcing function, the energy input from the sun, is not completely understood.
Ms. Curry uses a lot of technical-sounding language to mask all of this, and imply that the global climate models may be coarse, but are useful. But when one is predicting a 1 in 286 change over a period of 100 years, it is useful to first know whether the algebraic sign is correct. Given that far simpler CFD models, prior to anchoring, are in error by +/-10% or more, it is impossible to state that a noise-level change in an un-anchored complex model is representative of reality.
I would submit that no amount of V&V would allow a model to be used to predict to a certainty of less than +/- 0.5%. The prediction of a 1K change out of 286 K average is +0.35%. She’s attempting to con us, here. I’m not buying it.
Suppose you had some model that predicted when the next meteor shower was going to happen.. and you claim that the meteor showers are getting more frequent and by next century there will be one happening every afternoon. As a result, you suggest that the world is going to end.
If I were to ask why you think the world is going to end, and you were to argue the accuracy of your model, I would be an *idiot* to engage you in discussion.
I wish climate change doomsayers would answer the bloody question: why should I care what your model predicts?
I don’t see why climate prediction needs to be taken down to the level of weather prediction.
We should be able to get a rough estimate of the climate of any planet with just a few variables – the average absorptivity/emmisivity of the atmospheric gasses, the absorptivity/emmissivity profile of the surface matter. The solar irradiance at distance.
Then you integrate the absorbed profile, the emitted profile and subtract. That’s all it should take, because that is what we are trying to measure – the climate, the average attractor about which all chaotic weather gravitates.
The specifics of the weather are so complicated as to be nearly impossible to measure.
So it sounds like you guys aren’t very confident in current climate models. Do you think it is a problem that current climate models, you know the stuff that certain large portions of society want to base global decisions on, can’t instill even a little bit of confidence in you? While it’s not the fundamental problem, the fact that people can make predictions with these models without serious followup on whether the predictions are right or not, is a serious one in its own right.
As it turns out, I consider Curry sufficiently genuine to post a couple of my more or less standard recommendations: use prediction markets and/or prizes to reward those who make good predictions and build public trust in whatever predictive models survive the process.
“‘We should be able to get a rough estimate of the climate of any planet with just a few variables – the average absorptivity/emmisivity of the atmospheric gasses, the absorptivity/emmissivity profile of the surface matter. The solar irradiance at distance.”
The deal is that you have convective (and latent heat or phase change) transport of a lot of energy across latitudes — atmospheric winds, ocean currents. Energy deposited at equatorial latitudes moves poleward through fluid transport where it gets reradiated.
Also, the albedo changes with cloud cover. It is a harder problem than people let on.
Of course there is a great deal of energy transport within the earth – but at the end of the day, at static temperature, energy in equals energy out. You should be able to draw a box around the earth and get the average without any great difficulty.
Of course there is a great deal of energy transport within the earth – but at the end of the day, at static temperature, energy in equals energy out. You should be able to draw a box around the earth and get the average without any great difficulty.
The average is slightly above zero. There’s a small net outflow from the Earth cooling off (and technically an even smaller net inflow from tidal friction, meteorites, neutrinos and other oddities). It’s not enough to tell you what the mean global surface temperature is.
It’s not what the model predicts, or how well it predicts it, that actually matters. The model says something like “global temperatures will raise by 5 percent in the next 50 years”. Ok, great, what’s that *mean*? Answer the question.. back it up with *evidence*.
Curry’s pretty generous. For at least two widely used GCMs, ECHO-2 and CCSM3–there is absolutely nothing on record that could be called “validation.” In fact, I’m reluctant to show any deference to any of the (very) few attempts have been made at the problem. The issue isn’t simply limited to testing the models, but stretches from end to end–from control set data to the test “frameworks” themselves.
If the Earth were a perfect sphere with homogeneous emissivity and reflectivity, then yeah, all you’d need is the crunch basic exchange. Of course, that’s not the case. Earth’s atmosphere and surface is highly anisotropic, resulting in countless exchanges that do not combine to yield analytic solutions. So you’re stuck with numerical methods on a huge space of variables.
Not sure what you’re saying here.
ASEI sez: You should be able to draw a box around the earth and get the average without any great difficulty.
Energy throughput and a steady-state approximation are not in general sufficient to establish the temperature of a system.