Not Rolling Dice

I’ve commented in the past (even recently) that risk estimates of continuing to fly the Shuttle are overblown. There are good arguments to retire the system, but the risk of losing the crew isn’t one of them, both because they aren’t as high as people are saying, and because losing another crew wouldn’t be the end of the world. As I’ve said repeatedly, if we’re not willing to risk human lives on spaceflight, then it’s probably not worth doing. Anyway, Dick Covey, former astronaut and head of USA, apparently agrees with me (at least about the risk numbers):

The often-quoted PRA numbers do not factor in the continuous improvement in the vehicle and operations — of which there have been numerous and significant changes — or the quality and performance of the team that makes it work.

PRA estimates alone should never be used to reach a go/no-go determination on flying one, two or 10 more missions. PRA is intended primarily to provide an analytical yardstick for making sound engineering decisions about the development of a system and whether incremental changes in a system would improve or degrade relative safety.

Applying statistical probability techniques to the space shuttle PRA number to determine the risk of flying multiple missions implies a randomness in safe shuttle operations that does not exist, and belies the real approach to risk identification and management that defines the current space shuttle program.

The shuttle currently operates at the highest level of safety in its history. It is not without risk, but that risk is better understood and mitigated now than at any time in shuttle history.

Absolutely. The Shuttle has never been safer than it is today. Mike Griffin has just been using the PRA numbers to scare Congress into retiring the system so he could free up the funds for the Scotty rocket.

And this nonsense about needing “recertification” (whatever that means — it was never “certified” in the first place) in 2010 is just that. The CAIB never really provided any basis for this date. It’s an arbitrary one that just happened to coincide with the planned completion of ISS, so it seemed like a good marker for the decision as to whether or not to continue to program. We don’t really know if the vehicles need an OMDP, or mini or nano OMDP. We would just have to continue to inspect as we fly.

28 thoughts on “Not Rolling Dice”

  1. Yeah, Rand, you and Dick know a lot more about reliability than people who actually do that kind of work, which, no doubt, is exactly why rockets continue to kill crew every hundred launches more or less.

  2. Absolutely. The Shuttle has never been safer than it is today.

    I don’t buy it.First, there’s the matter of feedback. As far as I can tell, there’s going to be perhaps 20 flights of the Shuttle from here on out. We already have a bit over 120 launches of the Shuttle plus considerable testing data on the components of the Shuttle. Despite the testing data, that’s not a lot of data on which to base a claim that the Shuttle will be safer than a 1 in 77 failure rate no matter how good the risk analysis techniques are.

    For example, I don’t think NASA has a good understanding of the risk of tile failure in the thermal protection system of the Shuttle. There’s been only one failure (Columbia accident, of course), and they never took images of the damage prior to reentry. The current tile repair techniques are untested in a real life situation where an astronaut can easily cause more damage than they repair. IMHO, it’s still open whether the repair tile option improves overall safety of the vehicle. Clearly there will be scenarios where it’s pretty obvious that the tile damage is not survivable and yet repairable. But there are scenarios where the tile damage might cause lethal reentry damage. Then someone has to weigh whether that risk is worse than the risk of attempting repair. To be blunt, NASA can’t know enough to know where to draw that line.

    Second, NASA doesn’t have a lot of experience with aging space vehicles since the Shuttles are the first to have this problem. I don’t see models of aging based on airplanes applying to the Shuttles. The operating environment is very different. The materials are unusual. The maintenance cycle is completely different.

    Third, the Shuttle may be safer now, but that’s not 2011-2015. The Shuttles will be a few years older and perhaps more important, they will launch only twice a year. In 2009, there will be six launches of the Shuttle. Three times as much work to keep maintenance crews honed.

    My take is that a very low flight rate over five years and older Shuttles will mean a worse than average chance of LOC. Griffin’s estimate does not seem too high to me.

  3. Another thing, what makes the Shuttles (excluding the Columbia) safer now than they were at the start of 1993 (1992 was a good year for the Shuttle with 8 launches)? What’s the difference that makes them safer now? They had fewer launches (52 as opposed to 124 now) under their belt, but still had a few of the old Saturn V people, Looks to me like a lot of the experience is no longer there and the vehicles are 16 years older.

  4. Big rockets have never had more than one 9 of reliability, and I don’t think they ever will have better than that. The expendible portions of shuttle are always operating in the early portions of the bathtub curve despite the fact that we functionally test the hardware to get some portions off the steepest part of that curve. Now days the reusable portions of the shuttles are getting to the back end of that curve. It’s not just the structure, the shuttle orbiters are full of Kapton insulated wiring which is a known hazard that only gets worse with time. Sure some things get better over time, but not everything.

    The only way we will ever break into the multiple 9’s of reliability with launch vehicles is if we start building them with reusable first stages capable of soft, dry landings. That was Boeing’s primary rationale behind this configuration. It was designed around reliability from the start. It was by then obvious from the shuttle program that reusing the part of the vehicle exposed to the harsh environment of orbit and reentry was resulted in excessive rework and extremely low launch rates. Also vertical take off vehicles (which rockets are) have notoriously low reliability rates because they rely solely on their engines for lift, and engines are never as reliable as static structure.

  5. Big rockets have never had more than one 9 of reliability, and I don’t think they ever will have better than that.

    The X-15 demonstrated two 9’s of reliability 40 years ago, and the Shuttle has demonstrated nearly two 9’s, unless you think both programs had tens of accidents that the government has covered up.

    Please tell us which large rockets are only 90% reliable.

    That was Boeing’s primary rationale behind this configuration. It was designed around reliability from the start.

    Well, that configuration was designed around a scramjet engine that didn’t exist then and still doesn’t today. Marketing hype about the “reliability” of scramjet engines is just that.

    It was by then obvious from the shuttle program that reusing the part of the vehicle exposed to the harsh environment of orbit and reentry was resulted in excessive rework and extremely low launch rates.

    The fact that something is obvious to you does not necessarily make it true. For example,

    Also vertical take off vehicles (which rockets are) have notoriously low reliability rates because they rely solely on their engines for lift, and engines are never as reliable as static structure.

    Not all rockets are vertical takeoff vehicles.

    Additionally, engine-out capability for VTOL rockets is quite feasible given a sufficient number of engines. The DC-X, which flew in the 1990’s, was very nearly capable of making an engine-out landing. The limiting factor was the design of the landing gear.

  6. Edward, the composites found on the leading edges. The entire control system is unique though that isn’t a “material”. I see I’m wrong about a good portion of the internal structure.

    The forward fuselage is constructed of conventional 2024 aluminum alloy skin-stringer panels, frames and bulkheads.

    Thought the alloys used were funkier. There are two different engine systems on the orbiter, the SSMEs and the orbital maneuvering system. Nozzles use high temperature materials not normally found on planes. And they’re probably using an odd alloy for the cryo plumbing associated with the SSMEs.

  7. Edward, the composites found on the leading edges.

    Carbon-carbon, like the brakes on the stealth fighter. Like brakes, they’re replaceable so the age argument doesn’t apply. NASA could replace the leading edges after every flight, and the costs would still be picoscopic compared to any proposed “Shuttle replacement.”

    There are two different engine systems on the orbiter, the SSMEs and the orbital maneuvering system. Nozzles use high temperature materials not normally found on planes.

    Actually, three engine systems — SSME, OMS, and RCS. Hm. You don’t think jet engines use high-temperature materials?

    “The inner surface of each combustion chamber, as well as the inner surface of each nozzle, is cooled by liquid hydrogen flowing through brazed stainless steel tube-wall coolant passages.”

    Stainless steel is used on airplanes like the MiG-25.

    http://www.thomasnet.com/products/stainless-steel-aircraft-aerospace-95963898-1.html

    Besides, the SSMEs have been repeatedly replaced during the life of the Shuttle (and I’m sure the OMS and RCS thrusters have been as well).

  8. I don’t think the 2010 date was intended as a “you’ve got to do it in 2010” but as a “you should really have done this already and if you can’t get it done before 2010 you should stop using the Shuttles”.

    Since the point was lost on NASA the Shuttles are already operating past their sell-by date and in fact have been the whole time because they were never certified. How is that an argument for continuing to use them?

    And have any of the Shuttle frames been thoroughly and completely (that means 100%) checked for stress fatigue and cracking? (Big modern commercial planes go through that after a certain number of flights don’t they?).

    Anyone have a link to a source if the answer is yes?

    If cutting corners is ok then isn’t that one less reason to oppose Ares I? If spending billions on the continued operation of the STS without a certification is ok then hasn’t yet another reason for opposing Ares I disappeared?

    Not that I’m arguing for Ares I, certainly not; it’s an argument against both.

  9. I don’t see the age of the shuttles as being a major problem here. Almost all of the components, including a vast portion of the wiring, has been replaced on all of the shuttles multiple times during their service lifetime. Flight controls, engines, life support equipment… the whole works has been replaced from time to time. Perhaps not after every launch, but with all of the flights these machines have seen it certainly has been a part of the regular service regimen.

    There are C-130 planes that certainly are about as complex as the Shuttles, that have been seeing much harsher landing environments…. going into war zones for crying out loud…. that have been in service for over 30+ years. The basic design is now over 50 years old.

    Age of aircraft isn’t the issue, although a healthy service record is important. Is anybody disputing that the shuttles get some of the best service reviews of any kind of vehicle currently in operation in the USA?

    The shuttles are certainly at the best performance and the safest that they have ever been throughout their lifetime at the moment. All of the “bugs” and problems that needed to be found have been found to a certain extent. There certainly is a flight history to compare to and recurring issues are being dealt with. The “recertification” issue is a red herring and those who quote that as rationale for retiring the shuttle know this as well. It certainly isn’t a bureaucratic issue being brought up by the FAA.

    The argument about retiring the shuttle is purely an economic issue, not a technical one. Is it cheaper to retire the shuttles and go with something that is cheaper to operate? Also, can 90%+ of the missions the shuttle is currently being used for or planned to be used for be done with another system?

    One thing that is being lost with the shuttle retirement is the capability of returning large object from space in an enclosed (aka the shuttle cargo bay) environment that is protected from re-entry heating. I don’t know what missions would require that sort of capability, although it has been used successfully in the past. Bringing up astronauts + supplies for doing a task like the Hubble repair missions is something that the shuttle is also uniquely qualified for, and that neither the Ares I nor the Ares V are going to be able to easily replace.

  10. “The “recertification” issue is a red herring and those who quote that as rationale for retiring the shuttle know this as well.”

    Are you saying the CAIB intentionally lied in their findings? I certainly don’t know that it’s a red herring, far from it.

    To everyone who’s arguing that things are good enough: if there already is enough data and knowledge then certification would be a breeze wouldn’t it? If so why would it be a contentious issue in the first place? Why not just get it done? Isn’t the issue instead that it would require a lot of additional work that would cost billions?

    Which way is it? Which part of your argument is false? Can’t have it both ways at the same time.

  11. Are you saying the CAIB intentionally lied in their findings?

    Do you think the CAIB was infallible?

    Have you ever looked at the people who served on the CAIB? It was stacked with physics and political science professors. The chairmen of the board was a surface warfare officer with bureaucratic experience serving on various government commissions, not a pilot or an aerospace engineer. The only astronaut on the board was a backseater, not a pilot.

    This was not the FAA, NTSB, or a military accident investigation board. It was a typical political commission with members who had no experience in aircraft accident investigation.

    What leads you to believe that most of the board members even understood the “recertification issue”?

    if there already is enough data and knowledge then certification would be a breeze wouldn’t it?

    Insufficient data. It’s impossible to answer that question until you tell us what you think certification means.

    You say NASA should “certify” the Shuttles as if it’s obvious what that means. It isn’t. There is no “certificate” NASA could qualify for just by doing some more work.

    The Administrator of NASA could fill out a piece of paper and call it a “certificate,” but presumably you want more than that. If you think it’s necessary for NASA to do additional work on the Shuttles to make them safe to fly after 2010, why didn’t the CAIB order that work done after the Challenger accident?

  12. Edward Wright…

    No I’m not saying the CAIB is infallible. My original question still stands: does Robert Horning think the CAIB intentionally lied in their findings? Because that is in effect what he said.

    And argue their findings instead of applying personal attacks on the CAIB board (and no the CAIB board didn’t do all the work themselves, you do know what sitting on a board means right? No I shouldn’t assume that you do).

    You write:
    “What leads you to believe that most of the board members even understood the “recertification issue”?”

    and then later say you don’t know what is meant by certification ^_^

    Ok certification means getting empirical data (not from theoretical models) on all the actual components in use, identifying wear and tear, and checking that they still confirm to the specifications for their use including margins.

    You’re opposing something you say you don’t understand what means while at the same time arguing that it’s not needed because what’s already being done is good enough? That’s completely retarded and I’ll do you the favor of telling you.

    Then you manage to write:
    “If you think it’s necessary for NASA to do additional work on the Shuttles to make them safe to fly after 2010, why didn’t the CAIB order that work done after the Challenger accident?”

    Hello? CAIB couldn’t “order” squat: they gave their findings, conclusions, and recommendations.

    And guess what one of those recommendations was? No forget that, don’t guess I’ll make sure to tell you that it was to (re-)certify all components no later than 2010.

  13. Ok certification means getting empirical data (not from theoretical models) on all the actual components in use, identifying wear and tear, and checking that they still confirm to the specifications for their use including margins.

    NASA does that after every flight. Once again, what is it you think NASA should do that they *don’t* do today.

    And why is it necessary to do that in 2010 but not in 2009?

    And guess what one of those recommendations was? No forget that, don’t guess I’ll make sure to tell you that it was to (re-)certify all components no later than 2010.

    That’s nice. Did they explain how NASA was to “(re-)certify all components”? And why such “(re-)certification” was not unnecessary for safe operation before 2010 but suddenly became necessary on January 1, 2010? Which somehow changed by executive fiat to December 31, 2010?

  14. Edward, I think was partly in error about my comments on “unusual materials”. It’s just not as bad as I thought. But it’s absurd to say a material use is nothing special because it is used (possibly in a very different form) somewhere else. For example, carbon composites used on the leading edge of the Shuttle are used in a far different manner than in brakes. The construction process is probably far different as well.

    And the nozzles burn at a somewhat higher temperature than jet engines and for far shorter times. I don’t think they use the same materials.

    To continue my original complaint. I bet there’s a lot of unique software and hardware contained within the Shuttle. For example, nothing else uses the SSMEs. Even though these get replaced on a regular basis, the crew to maintain them is slowly becoming less experienced with handling them. It’s going to get a lot worse when the flight rate drops to two a year. And it looks to me like some of the contractors are going to get a lot harder to reach when there’s no longer a SSME production chain (which might have already been discontinued).

  15. The Shuttle is crap but still is the best manned launch vehicle there is. Period. Orion won’t be ready in time – if ever.

    I consider it moronic to retire the Shuttle before any alternatives exist. As moronic as it was failing to use perfectly good Saturn V launchers and turning them into museum pieces.

    As a heavy launch vehicle for anything but space station components, crew, or Hubble servicing Shuttle has failed: the weight of the manned and reusable portions cuts into usable payload, even with the high ISP engines it has. It is even debatable if it is really necessary for station building. The Russians managed to build Mir without a Shuttle: they just used heavy Proton launches and automated docking. No reason the same couldn’t be done by the USA with a Delta-4 or Atlas-5 Heavy, except existing ISS components were designed and built with the Shuttle in consideration.

    Still, I consider winged re-entry vehicles a technological dead end which will become more obvious as time progresses. Only Earth has an atmosphere suitable for using them. The low-G places we are interested in going to (Moon, Deimos, Phobos, Mars, asteroids) do not. A VTVL vehicle, like the lunar lander module, would be more suitable for such environments. So count me among the people who were behind Delta Clipper.

  16. The Shuttle is crap but still is the best manned launch vehicle there is. Period.

    Depends on the metric. Soyuz looks pretty good to me. Price per astronaut is much better than the Shuttle and it has a similar reliability.

    Still, I consider winged re-entry vehicles a technological dead end which will become more obvious as time progresses. Only Earth has an atmosphere suitable for using them.

    Even if winged reentry vehicles are only useful on Earth, that fact alone is far more than enough to preclude them from being “dead ends” for a long time to come.

  17. Edward Wright wrote:
    “NASA does that after every flight.”

    So you say. Why do you then oppose (re-)certification? You already claim they have the necessary data!

    Edward Wright wrote:
    “Once again, what is it you think NASA should do that they *don’t* do today. “

    I’ve already given a specific example way up above in an earlier post and asked if anyone could document that the frames are checked thoroughly (then a certain someone came along and started spewing nonsense without reading or comprehending anything at all).

    For example in the case of ultrasonic and electromagnetic NDT it is my understanding that the relevant methods for a complete check of the frames would require the complete disassembly of the Shuttles. Is there any sufficient NDT that does not require such disassembly? Has it been performed? As far as I know the answer is no and no.

    I’ve also said 100%. Since you, Edward Wright, is claiming that the there’s nothing additional to check can you please provide a source saying the entirety of the Shuttles are thoroughly checked? That means 100%. Can you also please explain the discrepancy in the argument made by those who say everything is currently checked well enough while simultaneously saying either that (re-)certification would be too costly or that it’s not needed or that the Shuttles are not certified?

    Because while you might fail to realize it all those positions are self-contradictory.

    And have someone else explain to you what “no later than” means because I’m fed up with giving you English tuition for free.

    FFS

    And if you have no real answers to any of the above why not save us all the waste instead of playing word games? I’ll even say please.

  18. HH, no one, not NASA, not the CAIB, has ever defined what “certification” means, let alone “recertification.” The Shuttle was never certified (other than every Flight Readiness Review for every flight), and there is no need for “recertification.” The CAIB didn’t “lie” when they called for this, but they weren’t very clear on it, either, even in their own minds. All they meant was that it shouldn’t be flown any longer than absolutely necessary (and they considered ISS completion absolutely necessary), and that if we wanted to fly it longer than that, such a decision shouldn’t be taken lightly.

  19. Should they have to? Wasn’t the Shuttles built to specifications? That’s the certification criteria right there.

  20. OK, you’re unfamiliar with the meaning of the word “certification” (which is a legal term of art). No, the Shuttle has never met its specifications (e.g., 65,000 lbs to LEO), and never will, but that’s a separate issue from certification, the process for which is completely undefined.

  21. Come on, every single part that makes up each Shuttle has to have been built to engineering specifications right? I’m not talking about lofty aims or lugubrious “man-rating” but the actual nuts and bolts (the CAIB was in part formed to increase Shuttle safety and thus this is a very natural and key topic) which is the meaning of the word as seen used all across the world in all businesses that do engineering.

    The same meaning of the word certification that is used in airplanes and automobiles (and just about anything else built) and which you either do in-house or send to a classification society like for example DNV or a multitude of others. Or both. And often with a third party (or several) doing their own certification review in addition (hello government).

    Unless one lives in China of course. Or works at NASA it seems ^_^

    I really don’t see why one would attribute any other meaning to the word in this context or claim that none exists: as a lowly example the certifications stamps on your electronic equipment aren’t based on aims but a check of the engineering itself: as long as the engineering works out with respect to applicable standards the aim itself doesn’t matter squat to the certification process (only to the users).

    I don’t think you don’t know this and that makes me scratch my head.

    But let’s say you’re right, how then do you explain the hue and cry over the massive costs of (re-)certification? How then does one even explain that people (like for example anonymous.space and I believe also you yourself about a year back) are talking about massive billion dollar range costs for it in the first place? What are they basing such numbers upon if your current argument is correct?

    I apologize if this is all yet another example of Americans not speaking English and having given entirely different meanings to words but I very much doubt that to be the case since this thread is the first time ever I’ve seen anyone having a problem with the (rather obvious) meaning of the word.

    And by the way if the CAIB was so fuzzy around the edges (not hard to believe at all) then how come you’re sure they used “certification” as a legalistic code in the way you claim? Where can one read the definition of the “legal term of art” you refer to?

  22. Certification is very well defined for aviation. You can go look up in a book what is required, per FAA procedures. Such a book has never been written for the Shuttle, and it’s not a simple matter of transferring the procedures from aviation, because Shuttle has many systems that don’t even exist in an aircraft, with no experience of how long they can really safely go without refurbishment or replacement (one of the reasons that it would be extremely premature to put a certification process on the space transport industry). It is not an aircraft, except for a brief period of its mission, and it remains an experimental system.

    Estimates of what “recertification” would cost for Shuttle are based on the costs of doing a full OMDP for whichever of the orbiters (Discovery I think) is due for one, and perhaps a lesser one for Endeavor (which is a newer vehicle, and again, where that term isn’t well defined, though I suppose that it could be sort of equivalent to a D check). But no one has ever discussed “certification” of Shuttles, as far as I know, prior to the CAIB, and the CAIB had no special insight into what would be involved in it, other than what they gathered by talking to NASA personnel, who probably had given it little thought. The fact remains that the 2010 date was driven by need to complete ISS, and had nothing to do with when the Shuttles were “due” for “recertification.”

  23. Your reply amazes me. I do realize that I’m beating a dead horse but I (and everybody else) should continue doing that as long as people try to operate space transportation systems upon the carcass.

    The Shuttle components were manufactured to specifications.

    Those specifications were whatever NASA deemed sufficient.

    Certification obviously means ensuring that the Shuttle components still meet those specifications and requirements (including any later changes) for every part of every Shuttle.

    This is not being done in full according to every source I have. No one has come forward with sourced information to the contrary.

    This issue is dead simple yet the replies are a buffet of avoiding the topic and arguments made, obfuscation, nonsense, repeating or introducing small pieces of information I would hope would be obvious to most interested bystanders with some knowledge (including me) and in general adding absolutely nothing at all.

    In other words you are obviously and most likely consciously arguing against common good practice and minimum standards.

    I find that to be on the level of “NASA O ring managers” and completely disgusting and unjustifiable. Worse; by now I don’t expect this reply to result in anything at all except more of the same.

    I’m somewhat shocked and sad by this and this will be my last words here, bye.

  24. Certification obviously means ensuring that the Shuttle components still meet those specifications and requirements (including any later changes) for every part of every Shuttle.

    No, that is not what “certification” means (at least for aviation), “obviously” or otherwise. No matter how much you want it to mean that, it doesn’t. The word for that is “verification.” There is no established procedure to certify a Shuttle Orbiter, regardless of how upset that reality makes you.

    In other words you are obviously and most likely consciously arguing against common good practice and minimum standards.

    No one is arguing, or has argued against that. But that’s not certification, either. Words really do mean things.

    The reason that we insist on not misusing the word “certification” is because of the potentially dire implications it would have for the fledgling space transport industry should the FAA take it into its head that spacecraft require it. It would likely strangle it in the cradle.

  25. The Shuttle never has nor never will be certified as the word is defined. For that matter, because it is a government vehicle, it doesn’t have to be certified. Military aircraft aren’t certified either unless they hope to sell civilian versions (such as the L-500 version of the C-130).

    As for the complexity of the C-130 verses the Shuttle mentioned above, the Shuttle is a far more complicated vehicle. In addition to the flight controls, the Shuttle has a reaction control system controlled by a complex fly-by-wire system. There are also two separate propulsion systems, APUs, fuel cells, life support systems far more complicated than that required on an aircraft. The C-130 is a very rugged airplane with decades of service behind it. If it were anywhere as complicated as a Shuttle, it would require far more people to maintain and operate than is actually the case.

Comments are closed.