Hi Lukas,

Yes, but there is an interface between the downstream integrator and the kernel
community (between 1. and 2.&3.). Is a DIA needed for that or something else?
(See discussion above).

If we resolved the one point above, we have enough material to document this email
thread in report form, and see where it fits into the description of a reference
process.

I think we're d'accord for this thread. I might want to revisit exactly what a DIA is, but
I'd foreseen that for the 5th veil.. where the DIA is standing in all its beauty in front
of us....

Hopefully I'll remember it then. On to the 3rd veil, although I'm not sure that it makes sense
before the workshop.

Cheers

John

Mit freundlichen Grüßen / Best regards

John MacGregor

Safety, Security and Privacy (CR/AEX4)
Robert Bosch GmbH | Postfach 10 60 50 | 70049 Stuttgart | GERMANY | www.bosch.com
Tel. +49 711 811-42995 | Mobil +49 151 543 09433 | John.MacGregor@...

Sitz: Stuttgart, Registergericht: Amtsgericht Stuttgart, HRB 14000;
Aufsichtsratsvorsitzender: Franz Fehrenbach; Geschäftsführung: Dr. Volkmar Denner,
Prof. Dr. Stefan Asenkerschbaumer, Dr. Michael Bolle, Dr. Christian Fischer, Dr. Stefan Hartung,
Dr. Markus Heyn, Harald Kröger, Christoph Kübel, Rolf Najork, Uwe Raschke, Peter Tyroller

-----Original Message-----
From: Lukas.Bulwahn@... <Lukas.Bulwahn@...>
Sent: Freitag, 15. Mai 2020 07:48
To: MacGregor John (CR/ADT4) <John.MacGregor@...>
Cc: development-process@...
Subject: Re: [development-process] "Pseudo-DIA" between the Linux Kernel
Development Community and its Users - 2/5 We're talking about an Operating
System here

I agree. At the time the system developer decides to use the
operating system, the operating system developer has already finished
testing. The process is more like the selection process for a
hardware component. The customer can assess the testing that the
supplier has already done and do its own testing to evaluate alternative

components.

Yes.

This process isn't particularly foreseen in the safety standards. That
being said, we're talking about the general functionality in this
e-mail, not the safety functionality.

Yes, I agree. There is a step of interpretation from what the safety standards
expects vs. how the whole development and integration act.

What 26262 says, in 20262-8 Clause 12 is that the organization should
prepare a specification for the component, including the requirements
on the component. The organization must then provide evidence the
requirements have been tested to an adequate coverage level and that
the component is valid for its intended use by verification according to clause 9.

The process for deriving the specification and requirements is not
described and I assume that the component should be treated like any
component created by bespoke development, i.e. in the architecture and
design phases of 20262-6.

Nicolas Mc Guire addressed this in his 61508-based approach,
especially noting that when other factors are relatively equal, one
criteria for selection between alternative components could be their
certifiabliity; their systematic capability. That's one of the uses for Annex QR.

That is the implicit DIA here. Then the operating system supplier
executes an own "development process" down to the implementation
(with

his own V model process).

Either we misunderstand each other or I disagree. At the point that
the customer uses the operating system, as in using it in its
architecture and design phase, the operating system already exists and
should be completely implemented.

I think the issue is that:

We both acknowledge that there is an own development process down to the
implementation and a (partial) one for verification & validation.
The main result of that process is the kernel source code. But, we expect another
"supporting result" wrt. quality assurance/management that describes which
activities have been performed how well. Let us call that a "process quality report"
for now.

Side note: Of course, at the moment, this process quality report does not exist for
the kernel as a structured and collective description (you would need to know and
search various sources etc. to get something in that direction).

But let us assume it would exist:

I think you now make thoughts along those lines: the process quality report would
describe its scope of covered activities and hence, the downstream integrator would
simply need to close all further gaps after understanding the scope. (So, no Pseudo-
DIA required.) I was thinking along those lines: There is one document, a Pseudo-
DIA, that defines the interface between the kernel community and downstream
integrator (for any previous work the community has done), and with that at hand,
we have a clear understanding what need to described in a process quality report, as
the scope is defined now.

Does this resolve the misunderstanding?

I am happy to say no DIA is involved, but the process quality report has (amongst
many other content) a defined scope according to a certain development process
description. That would cover the concern I was trying to resolve with this Pseudo-
DIA here at this process interface between community and downstream integrator.

The only DIA I see here, is the DIA I discussed below, where the user
community issues a generic DIA that the operating system developer can
consider during further development. Considering that I said that the
OS has already been developed in this scenario, the OS has been
developed based on the requirements, including any customer
requirements, that the OS developer gathered in its requirement phase.

Okay. I will try to resolve that below.

And longer question:
So, does the operating system supplier or the system developer test
the operating system?
(I have an assumption here and there is again a certain split of the
activities in this
area.)

Both do testing. The operating system supplier can, of course,
integrate the OS in its test harnesses, or representative test user
systems, before release. The system developer does acceptance and
validation testing in its development process.

But where is this assumption documented? If that assumption is crucial for quality
assurance, which I believe it is, this point would need to a common agreement and
details need documentation. A Pseudo-DIA could offer a structured form of
documenting that.

I hope that everyone agree that this point needs to cover by our discussion and
presentation in some way. Anyone disagrees?

I think a process description should cover these insights above from
you, and the assumed responsibilities here.

(If this is part of a DIA or some even more basic overview document
on the process is secondary to me; we will figure it out while
constructing.)

It is clear though (even if we do not have the description) that we
certainly need to have the common understanding, e.g., as you
described the overall process above, and one core aspect of it is
"who does what". Otherwise, this mapping of ISO clauses/QM standards
(ASPICE and such) to described activities is hardly to understand.

Off the top of my head, I don't know what the implications are for
supplier management. I could imagine that acceptance test development
is covered there as well.

Maybe that is where the testing of the downstream integrator side lands. Just that
you know: The split of testing is tricky, that why I did not like that example for the
initial email on pseudo DIA.

A discussion of a proper description and mapping on its own, let us remember and
shift that to a later point. As I said, it is tricky point for a process perspective.

As we see in this discussion, already on the higher levels, we
assume multiple "V- model like development process" happening in
some interweaved and temporally decoupled way.

Like I said, I think that component development has finished before
the system developer starts. What is happening, especially with
Linux, is that, after a version is released, the version continues to
be updated with bug-fixes. That, for me, is part of the maintenance
process, the quality of which could be a criteria in the component selection

process.

Documenting that as if it is "one V-model process" will probably
distort the whole process assessment and that has impact on results
(at least, in comprehensibility, and maybe even in the risk assessment).

There is of course, the possibility that the system developer
needs custom OS functionality, a new feature (or, to use Chris
Temple's terminology, a

chunk).

In which case, if the system developer can afford it, either party
can develop the feature or enhancement from scratch. Consider
what Google has done with off-tree development of Linux
functionality for Android, which they sometimes got retroactively accepted

upstream.

In this case, Google would be making a DIA with itself: between
its Android and Linux organisations, which is foreseen in the
definition of a

DIA.

Agree. I consider this a completely new and different development
process, that needs a DIA in itself.

An organization making a DIA with itself is the default. Usually,
the organization using Linux will act internally and as part of the
community. That does not invalidate the idea of a DIA. Still
activities in the community are different than the activities that
are done internally, and in the end, the quality statement comes
from understanding how well both activities (internally, and in the
community

with others) work, and how well they "work together" (interact, are
aligned, etc.).

While a system developer must choose an operating system from the
flavours and versions that are available at the time, what ELISA
is actually looking at is the interaction between the user
community and the Linux development community. The user community
has the possibility to specify its expectations on future
operating system
development: a DIA for future functionality in general rather than
for a specific feature development effort. The operating system
developers then have the possibility of considering the needs of
the user

community when adding or modifying OS features.

Agree. That is the upfront development to the development process above.

Summarizing, we identified three "development processes":

1. Understanding the system needs and selecting and configuring the
operating system appropriately. Result: kernel build configuration

2. Previous development process of the operating system (without any
involvement of the own organization)
Result: source code (relevant here to the system integrator is the
source code for kernel build configuration above)

3. Specific development process of the operating system (with
involvement of the own organization and a clear goal towards a
specific quality level)
Result: New features specifically driven by extensions of 1.

Again, 2. And 3. Intersect but it help in the discussion to separate those.

Do you agree to "all" above?

Yes, there are these 3 development processes, whereby 1) does not
involve operating system development would not involve a DIA between
an individual user and the OS developer.

Yes, but there is an interface between the downstream integrator and the kernel
community (between 1. and 2.&3.). Is a DIA needed for that or something else?
(See discussion above).

If we resolved the one point above, we have enough material to document this email
thread in report form, and see where it fits into the description of a reference
process.

Lukas