Re: "Practices and Principles" to death

in reply to "Practices and Principles" to death

One designs tests in order to guarantee certain levels of quality for their products and I can see how important a high level of quality must be implemented on projects building satellites. Should anything go wrong and the product is lost, or the ratio of the cost of making repairs compared to the cost of preventing points of failure being to high.

Given the amount of processes you said to have, there will be a risk management office and a quality management office. I'm guessing the risk management office, among other things, will do some calculations on the Earned Monetary Value for situations where strategies of avoiding or mitigating risks are compared to doing nothing. Based on that, doing a lot of tests may be seen as being really cheap compared to failure events.

Documenting situations for what went wrong is of extreme importance for making sure it won't happen again. When doing a new project one will not forget to take some measures on matters that may be similar to previous ones. So 'lessons learned' from previous projects helps in improving quality and reducing risks for new projects. Also in preventing occurrences where money would have to be spent to implement contingency plans or workarounds on problems.

The objective of having all those processes is to lower the cost of projects. Strategies for quality have, of course, associated costs. Is it worth to incur in such costs? Well, it depends when compared to the costs incurred when having your products fail.

One necessary process for the project is to revise the documentation and identifying what is applicable to the current project, as there may be cost and time consuming activities that bring no value.

All those processes are invaluable as they help on identifying total project cost to present to your clients. Either they accept them or they don't. But you know what it will cost to the company to do something for the client. And you can decide whether the project is an opportunity or a financial disaster for the company.

Comment on Re: "Practices and Principles" to death

Replies are listed 'Best First'.
Re^2: "Practices and Principles" to death by ack (Deacon) on Mar 01, 2008 at 03:29 UTC
I think I understand what you're talking about. At the roots of what I think you're saying, I would say that I have no particular contest. Though I interpret what you're saying as being one of trying to get me to consider the issue from the 'buyer's' (or customer's) point of view: I am the buyer/customer of the systems and my job, as Chief Systems Engineer is to advise our Program Managers who hold the checkbooks, on what and how we expect our suppliers to produce the systems we are responsible for. I am not meaning to be at all critical of what you were saying; I do think your points are largely correct. I just want to be sure that you understand that it seemed like you may have presumed that I work for a supplier and might not appreciate the customer's point of view. But to the excellent points you presented, my problem is that all of those analyses and evaluations and considerations that you spoke of actually rarely occur. Often because we can't find anyone that can actually figure out what went wrong since when we loose an asset it isn't around for us to do a post mortem on...it's either in the ocean, or blown to bits when destroyed during launch, or is floating around in space). In addition, it seems, we have too few 'experts' to do all the things that you pointed out. So, failures happen; but way too frequently noone trys to (or can) really get to the 'root cause'. Instead, tiger teams of people try to imagine what 'might' have happened, and then try to determine what testing would have proven or disproven that such a hypothesized cause would occur. Those tests then become: first, required tests for all future projects and second, folklore that noone seems to remember why they were even dreamed up. For what it's worth, the same pathology shows up in most every aspect of our systems productions (concept formation, system requirements production and derivation, system design and production, testing, and on-orbit operations). My focus was on testing because, for our particular work, it is the area most malignantly infected with the pathology: the tendancy to substitute 'do what we always do' for 'thinking'. With respect to your comments about 'cost-benefits' and those types things...they are, for me, certainly nobel goals and SHOULD be what is being considered. But I know of no real case where anyone has done any such analyses. They seem to just try to reason that such analyses SHOULD validate the mass of processes, practices, and policies...but noone (to my knowledge) seems to ever do so. What I see is that they follow their 'SHOULD validate' with a very loud proclamations that almost quote what you wrote. In your case I believe it was meant with true constructive observation; the proclamation litanies, on the other hand, seem all too often to be just a smoke screen (hence, I suppose, the cause for part of my initial reaction to what you had written). In fact, when I talked about what we've (over the past 10 years or so) hae been doing to try to change it (i.e., to try 'thinking' instead of just 'doing it because that's what we've always done' has resulted in producing our last 10 satellites with demonstrated reliability...based upon number of systems fielded compared to the number that have failed...that matches or exceeds the last 10 to 12 produced in the 'traditional way'. And we have produced them for total costs that are between 1/3 and 1/4 of the 'traditional' systems' costs and we have produced them on on a schedule that is getting quicker (rather than the 'traditionalists' whose schedules are creaping towards longer) typically 2-3 years compared to the 5-7 years of the 'traditional ones.' So based upon that evidence (which I would, of course, consider somewhat anecdotal...I think I misspelled that, sorry...since the analyses that you suggested have not been done on our systems either) I would have to say that at face value the analyses you suggested would likely end up arguing against the 'traditional' approach. Of course your observation that in my business the loss of a system is very high...many tens of millions of dollars...is absolutely true; so you are definitely correct in saying that the amount, focus, and considerations for testing are heavily influenced by that consideration. But at the end of the day, no matter what the costs involved, I think we should still be working to be sure that we put 'thought' into our testing and be frugal but responsible and accountable for ensuring that the testing that we do is what is really needed...not just because thats-how-we've-always-done-it. That's my thesis...my story...and I'm sticking to it.;) ack Albuquerque, NM	[reply]

Replies are listed 'Best First'.

Re^2: "Practices and Principles" to death
by ack (Deacon) on Mar 01, 2008 at 03:29 UTC

I think I understand what you're talking about. At the roots of what I think you're saying, I would say that I have no particular contest.

Though I interpret what you're saying as being one of trying to get me to consider the issue from the 'buyer's' (or customer's) point of view: I am the buyer/customer of the systems and my job, as Chief Systems Engineer is to advise our Program Managers who hold the checkbooks, on what and how we expect our suppliers to produce the systems we are responsible for. I am not meaning to be at all critical of what you were saying; I do think your points are largely correct. I just want to be sure that you understand that it seemed like you may have presumed that I work for a supplier and might not appreciate the customer's point of view.

But to the excellent points you presented, my problem is that all of those analyses and evaluations and considerations that you spoke of actually rarely occur. Often because we can't find anyone that can actually figure out what went wrong since when we loose an asset it isn't around for us to do a post mortem on...it's either in the ocean, or blown to bits when destroyed during launch, or is floating around in space). In addition, it seems, we have too few 'experts' to do all the things that you pointed out.

So, failures happen; but way too frequently noone trys to (or can) really get to the 'root cause'. Instead, tiger teams of people try to imagine what 'might' have happened, and then try to determine what testing would have proven or disproven that such a hypothesized cause would occur. Those tests then become: first, required tests for all future projects and second, folklore that noone seems to remember why they were even dreamed up.

For what it's worth, the same pathology shows up in most every aspect of our systems productions (concept formation, system requirements production and derivation, system design and production, testing, and on-orbit operations). My focus was on testing because, for our particular work, it is the area most malignantly infected with the pathology: the tendancy to substitute 'do what we always do' for 'thinking'.

With respect to your comments about 'cost-benefits' and those types things...they are, for me, certainly nobel goals and SHOULD be what is being considered. But I know of no real case where anyone has done any such analyses. They seem to just try to reason that such analyses SHOULD validate the mass of processes, practices, and policies...but noone (to my knowledge) seems to ever do so. What I see is that they follow their 'SHOULD validate' with a very loud proclamations that almost quote what you wrote. In your case I believe it was meant with true constructive observation; the proclamation litanies, on the other hand, seem all too often to be just a smoke screen (hence, I suppose, the cause for part of my initial reaction to what you had written).

In fact, when I talked about what we've (over the past 10 years or so) hae been doing to try to change it (i.e., to try 'thinking' instead of just 'doing it because that's what we've always done' has resulted in producing our last 10 satellites with demonstrated reliability...based upon number of systems fielded compared to the number that have failed...that matches or exceeds the last 10 to 12 produced in the 'traditional way'. And we have produced them for total costs that are between 1/3 and 1/4 of the 'traditional' systems' costs and we have produced them on on a schedule that is getting quicker (rather than the 'traditionalists' whose schedules are creaping towards longer) typically 2-3 years compared to the 5-7 years of the 'traditional ones.'

So based upon that evidence (which I would, of course, consider somewhat anecdotal...I think I misspelled that, sorry...since the analyses that you suggested have not been done on our systems either) I would have to say that at face value the analyses you suggested would likely end up arguing against the 'traditional' approach.

Of course your observation that in my business the loss of a system is very high...many tens of millions of dollars...is absolutely true; so you are definitely correct in saying that the amount, focus, and considerations for testing are heavily influenced by that consideration.

But at the end of the day, no matter what the costs involved, I think we should still be working to be sure that we put 'thought' into our testing and be frugal but responsible and accountable for ensuring that the testing that we do is what is really needed...not just because thats-how-we've-always-done-it.

That's my thesis...my story...and I'm sticking to it.;)

ack Albuquerque, NM

[reply]

In Section Meditations