Hi,

Many of you will have been following an interesting thread [1] that started on the development-process list and then spread to this one. That thread is starting to grow unmanageable in scope, so, as John MacGregor suggests [2], here is my attempt to explore one or two of the topics in a new thread.

Background
----------

Lukas made a familiar assertion ("Safety is a system property, not a property of software"), which Oscar found to be unhelpful; a wide-ranging discussion followed, which introduced a number of other terms and ideas. The difficulty involved in defining such terms, Lukas noted today [3], is a worthy goal for ELISA, but one that he wished to avoid in the Tools and Code Improvements group, which has other goals.

I'd like to share my perspective on this topic in an effort to clear up some possible misunderstandings, and to begin the work of defining a model (formally speaking, an ontology) to help frame our discussions. I have some prior experience with this sort of thing [4], and have no illusions about the difficulties involved, but I'm prepared to try :-)

Objectives
----------

As a first step, I think it may be useful to distinguish between four objectives with respect to safety, which, in the absence of an agreed set of terms, we can characterise using the following questions:

#1: What do we mean by 'safe' in a particular context, and what criteria
must be satisfied to achieve and maintain this desired state?

#2: How can we ensure that a given system implementation satisfies the
criteria defined in #1?

#3: How can we be confident that the criteria identified in #1 and the
measures identified in #2 are sufficient?

#4: How can we be confident in the processes and tools that we use to
achieve objectives #1, #2 and #3?

I realise that this has already introduced a number of terms (context, criteria, measures, confident, sufficient, processes) that we might need to define more precisely, but please bear with me for now :-)

I believe that it is useful to make these distinctions, because some of our discussions traverse several objectives, and a proposition or a technique that might apply to one of these will not necessarily be helpful when applied to another.

The safety standards that I'm familiar with (IEC 61508 and ISO 26262) seem to be primarily concerned with objectives #2, #3 and #4, but they also list techniques for elaborating #1, and identify some general criteria that are typically applied for this objective.

As Oscar notes, these standards also permit us to break down (decompose) systems into components, and to examine the distinct roles that software and hardware components play in the safety of a system. We should also consider the tools that we use to create or refine these elements, to examine how they contribute to objectives #2 and #3, and how #4 applies to them.

Part of this motivation here is the desire to have reusable components (especially software components) and tools, with clearly-defined and widely-applicable safety-relevant characteristics, which we can feel confident about using for different contexts and systems. There is explicit support for this at a system component level in ISO 262626, in the form of the 'Software Element out of Context" (SEooC) concept.

"Safety is a system property"
---------------------------

This assertion is associated with a school of thought that we might label the 'system theoretic safety perspective', which has been notably articulated by Nancy Leveson:

“Because safety is an emergent property, it is not possible to take a single system component, like a software module or a single human action, in isolation and assess its safety. A component that is perfectly safe in one system or in one environment may not be when used in another.”

-- Engineering a Safer World: Systems Thinking Applied to Safety [5]

This assertion might at first seem to be incompatible with a desire for reusable components. In my opinion, however, it does not mean that we can *only* undertake safety tasks in reference to a complete and specific system. Rather, it asserts that we cannot perform a *complete* safety analysis for a given component *unless* we also consider the system context of that component.

Furthermore, this perspective applies primarily to objective #1 (although it also has some bearing on #3), which means that there are many tools and techniques relating to safety whose merits can be examined without explicit reference to a system context.

However, in order to make reasoned arguments about safety with respect to a software component, we must be able to answer (at least) the following questions:

1) What system (or kind of system) we are considering, and in what
environment(s) is it intended to operate?
2) What is this system's intended purpose, and how does the software
contribute to it?
3) What do we mean by 'safe' in the context of the system?
4) How is the software relevant to achieving or maintaining this
desired state?

We might examine how the properties of a given piece of software might contribute to the safety of a system, or examine how those properties might represent a threat to its safe operation, but without defining the context that we have considered - and what 'safe' means in that context, we can't reasonably assert that the software is 'safe'.

We should always attempt to answer these questions, even if our answers are unsatisfactory (e.g. "We have not considered the impact of any environmental factors on the operation of the system") and provisional. If we fail to consider and describe the context in which our software will operate - or if we base our reasoning on an assumed context that we have not described - then flaws in that reasoning may go unnoticed, with possibly dangerous consequences.

A system context might be concrete and complete (e.g. a specific integration of hardware and software as part of a product), partial (e.g. only describing the other components with which our component interacts) or an abstraction (e.g. a set of assumptions and constraints for a given category of system). But without it we cannot make meaningful statements about what may be considered safe (objective #1).

Regards,

Paul


[1] https://lists.elisa.tech/g/development-process/topic/80309250#762
[2] https://lists.elisa.tech/g/devel/message/1381
[3] https://lists.elisa.tech/g/development-process/message/792
[4] https://gitlab.com/trustable/documents/-/blob/master/ontology/elements.md
[5] https://direct.mit.edu/books/book/2908/Engineering-a-Safer-WorldSystems-Thinking-Applied