1. Devel
  2. Topics

New WG proposals: Safety Engineering Process, Linux Developers

elana.copperman@...
 

I have also updated the subject line, to reflect the proposal for the WG split.


From: devel@... <devel@...> on behalf of Elana Copperman <Elana.Copperman@...>
Sent: Tuesday, September 14, 2021 5:40 PM
To: Paul Albertella <paul.albertella@...>; devel@... <devel@...>
Subject: Re: [ELISA Technical Community] New WG proposal: Safety Engineering Process
 
Thanks, Paul, for your clear summary of previous discussions.  
My added inputs are included below Paul's text.  The proposed name for the new WG remains TBC, I am open to suggestions.

As noted by Paul, this has been extensively discussed in the recent Dev Process WG weekly calls.
My expectation is to announce this new format in the upcoming ELC/LSS conference, as well as in the 
next ELISA workshop, thus inviting new members to join and to contribute. 

Regards
Elana

From: devel@... <devel@...> on behalf of Paul Albertella <paul.albertella@...>
Sent: Tuesday, September 14, 2021 4:05 PM
To: devel@... <devel@...>
Subject: [ELISA Technical Community] New WG proposal: Safety Engineering Process
 
Hi,

The Development Process working group has been discussing proposals for
the evolution of the group. The WG has historically attempted to cover a
number of goals in a single context, which has occasionally created
confusion; to address this, we propose to split into two separate
working groups, each with a clearer focus.

Elana Copperman and I have proposed mission statements and planned
activities for these two new groups, which have been discussed in the
Dev Process WG weekly calls.

Having reached some consensus about a way forward in that forum, I'd now
like to share my proposal (see below) with the wider ELISA community for
comment, before seeking to formalise the group at the TSC. I hope that
Elana will share details of her proposal in due course.

Regards,

Paul

---------------------------------------------------------------------

Proposed working group name
---------------------------

Safety Engineering Process

Proposed WG chair
-----------------

Paul Albertella

Proposed meeting schedule
-------------------------

Weekly on Thursday, in the same slot as the current Dev process call

Proposed mission statement
--------------------------

This working group aims to examine safety-related claims that we might
like to make about Linux as part of a system, and to explore how we can
gather and present evidence to support such claims as part of a safety
engineering process.

We are interested in two broad classes of claims and evidence:

a) Those relating to development practices for Linux as a whole; for
example, the peer review process for patches, the testing performed on a
kernel stable branch when preparing a release, or the testing performed
by a system integrator for a product that incorporates Linux

b) Those relating to specific properties or behaviour of Linux; for
example, features that we can enable or disable in a kernel config, the
(inferred) design of a subsystem, the characteristics of a driver, or
tests that can verify aspects of a given feature

For (b), we will focus on engineering process aspects (construction,
verification, integration, use and maintenance) relating to the feature
or property, rather than technical details of its design/implementation.

In both cases we may examine practices or processes that are undertaken
by the Linux kernel community, as well as communities or organisations
that consume Linux (e.g. AGL, LTP), or we may identify practices or
processes from other sources that might be used by organisations who
want to use Linux to build systems with safety requirements.

Planned activities
------------------

Our objectives are to:

* Define claims that we would like to make about Linux and/or the
   processes used to develop it or integrate it as part of a product
* Identify evidence that we might use to support these claims
* Identify strategies or tools that we can use to gather, generate or
   reformulate evidence (perhaps in collaboration with the Tools WG)
* Examine the evidence gathered and document our findings
* Share and peer-review our findings in an elisa.tech Github repo

Possible claims might be proposed by another WG, perhaps in relation to
a specific use case, technique or safety argument, or we may select our
own. Claims that we examine may not all be safety claims, per se; some
might relate to software quality, for example, or functionality that
does not have an explicit safety role, since both of these may be needed
to support safety arguments.

We will use contributors’ knowledge of specific safety standards to
inform our work where appropriate - for example to determine what
criteria to apply for a given class of evidence - but specifying how a
given claim and its supporting evidence may be used as part of a
certification process should be considered out of scope.

Where we do not have consensus on results or conclusions, we will
document the different perspectives and/or open questions. We will
manage the material produced in an elisa.tech Github repo and
collectively review it in GitHub PRs, inviting review or input from the
rest of the ELISA community where appropriate.

As part of this work, we also plan to write a summary of the results and
conclusions of the work done by the Development Process WG, which will
also be made available in an elisa.tech Github repo. This is intended to
provide background for new participants and other WGs, as well as a
baseline / conceptual framework for the work of the new group.

Collaboration
-------------

The group expects to collaborate with the existing working groups and
with the new group proposed by Elana, both to identify or discuss claims
and evidence for analysis, and to address wider questions relating to
safety engineering processes. Where we work with another group on given
topics, we may jointly present the results at an Elisa workshop.

*********************************************************************************************

Proposed working group name
---------------------------

Linux Developers WG

Proposed WG chair
-----------------

Elana Copperman

Proposed meeting schedule
-------------------------

Weekly on Mondays, pending availability of participants

Proposed mission statement
-------------------------

  • To identify existing Linux Kernel features which may be leveraged for use in safety critical systems.  

  • For example, 

    1. Mechanisms for protections of various memory types.  

    2. Dynamic analysis for multi-threaded systems. 

    3. Kernel profiling using ebpf-based tools. 

    4. Isolation techniques and FFI (Freedom From Interference)

    5. Safety extensions to Linux drivers.


  • To bring together kernel developers and producers of safety critical systems to demonstrate use of 

  • such features in real systems, and to learn from these experiences together as a community.


  • To propose enhancements to such features and to work as a community to design / implement / 

    • deploy kernel patches.  Such patches should help to make those features more amenable 

    • for use in safety critical systems.


  • To recommend tools or processes to be provided by other ELISA WGs so that those patches 

  • and features can be used by designers and integrators producing safety critical systems. 


Planned activities
------------------
The scope of this WG does not in any way include safety qualification or any safety claims on how the 
integrator can or should use these features or patches.  The only claims that would be made are a 
description of the feature and its functional impact. 
Evidence to support deployment of any feature in a safety-critical system may be clarified by work 
products produced by the other ELISA WGs, focusing on collaboration as outlined by Paul above.

The work group will invite developers of safety critical systems to identify and to enhance existing 
kernel features with elements that may be used effectively to support safety claims.
Lukas Bulwahn
 

Proposed working group name
---------------------------

Linux Developers WG

Proposed WG chair
-----------------

Elana Copperman

Proposed meeting schedule
-------------------------

Weekly on Mondays, pending availability of participants

Proposed mission statement
-------------------------

To identify existing Linux Kernel features which may be leveraged for use in safety critical systems.

For example,

Mechanisms for protections of various memory types.

Dynamic analysis for multi-threaded systems.

Kernel profiling using ebpf-based tools.

Isolation techniques and FFI (Freedom From Interference)

Safety extensions to Linux drivers.


To bring together kernel developers and producers of safety critical systems to demonstrate use of

such features in real systems, and to learn from these experiences together as a community.


To propose enhancements to such features and to work as a community to design / implement /

deploy kernel patches. Such patches should help to make those features more amenable

for use in safety critical systems.


To recommend tools or processes to be provided by other ELISA WGs so that those patches

and features can be used by designers and integrators producing safety critical systems.


Planned activities
------------------
The scope of this WG does not in any way include safety qualification or any safety claims on how the
integrator can or should use these features or patches. The only claims that would be made are a
description of the feature and its functional impact.
Evidence to support deployment of any feature in a safety-critical system may be clarified by work
products produced by the other ELISA WGs, focusing on collaboration as outlined by Paul above.

The work group will invite developers of safety critical systems to identify and to enhance existing
kernel features with elements that may be used effectively to support safety claims.
I am a big fan of this working group and I think it deserves some exploration.

The key challenge I currently see is that "developers of safety
critical systems" (which---by historic incident---have a significant
different background on the complexity of software they have been
developing and handling, and the style it has been handled) will
attempt "to enhance existing kernel features" (which have been
developed in a very different manner and evolve in a different
manner).

I feel that I simply do not have the kernel development experience
with the few basic patches I submitted and bandwidth to contribute
here with critical development contributions. I hope that there are
others that have the needed kernel development knowledge and
experience to quickly start feature development, and that this does
just end up creating a wish list of ambitious kernel features that are
impossible to implement.

I hope there are a few developers interested in joining that working
group and wish all of you joining that group a great learning
experience, good luck and I am looking forward to some results on the
linux-kernel mailing list.

Of course, kernel feature development and documentation of kernel
features will unaviodingly continue on the corresponding main channels
as well. I hope to see you there soon.

I would not worry too much about linking it to the mission statement
or the name here; I think it is important to start exploring and see
who is interested in joining and what the group can then really do.


Lukas
Lukas Bulwahn
 

On Wed, Sep 15, 2021 at 6:04 PM Lukas Bulwahn <lukas.bulwahn@...> wrote:

Proposed working group name
---------------------------

Linux Developers WG

Proposed WG chair
-----------------

Elana Copperman

Proposed meeting schedule
-------------------------

Weekly on Mondays, pending availability of participants

Proposed mission statement
-------------------------

To identify existing Linux Kernel features which may be leveraged for use in safety critical systems.

For example,

Mechanisms for protections of various memory types.

Dynamic analysis for multi-threaded systems.

Kernel profiling using ebpf-based tools.

Isolation techniques and FFI (Freedom From Interference)

Safety extensions to Linux drivers.


To bring together kernel developers and producers of safety critical systems to demonstrate use of

such features in real systems, and to learn from these experiences together as a community.


To propose enhancements to such features and to work as a community to design / implement /

deploy kernel patches. Such patches should help to make those features more amenable

for use in safety critical systems.


To recommend tools or processes to be provided by other ELISA WGs so that those patches

and features can be used by designers and integrators producing safety critical systems.


Planned activities
------------------
The scope of this WG does not in any way include safety qualification or any safety claims on how the
integrator can or should use these features or patches. The only claims that would be made are a
description of the feature and its functional impact.
Evidence to support deployment of any feature in a safety-critical system may be clarified by work
products produced by the other ELISA WGs, focusing on collaboration as outlined by Paul above.

The work group will invite developers of safety critical systems to identify and to enhance existing
kernel features with elements that may be used effectively to support safety claims.
I am a big fan of this working group and I think it deserves some exploration.

The key challenge I currently see is that "developers of safety
critical systems" (which---by historic incident---have a significant
different background on the complexity of software they have been
developing and handling, and the style it has been handled) will
attempt "to enhance existing kernel features" (which have been
developed in a very different manner and evolve in a different
manner).

I feel that I simply do not have the kernel development experience
with the few basic patches I submitted and bandwidth to contribute
here with critical development contributions. I hope that there are
others that have the needed kernel development knowledge and
experience to quickly start feature development, and that this does
just end up creating a wish list of ambitious kernel features that are
impossible to implement.
Upps... missed a critical word in the sentence above:

...and that this does NOT just end up creating a wish list of
ambitious kernel features that are impossible to implement.

Note to myself: Read before sending, not afterwards...

Lukas
elana.copperman@...
 

Thanks for comments.
2 quick notes:
1. Still waiting for additional ideas on relevant names - perhaps a prize to the winning name?
2. Lukas, some of my original text was unfortunately omitted. All of the suggested ideas are working features, not academic exercises or "ambitious ideas that are impossible to implement."
The call is to those who actually design and develop production systems.
The key challenge is to provide a supportive environment so that the open source community can benefit.
Regards
Elana

-----Original Message-----
From: Lukas Bulwahn <lukas.bulwahn@...>
Sent: Wednesday, September 15, 2021 7:07 PM
To: Elana Copperman <Elana.Copperman@...>
Cc: Paul Albertella <paul.albertella@...>; devel@...
Subject: Re: [ELISA Technical Community] New WG proposals: Safety Engineering Process, Linux Developers

On Wed, Sep 15, 2021 at 6:04 PM Lukas Bulwahn <lukas.bulwahn@...> wrote:

Proposed working group name
---------------------------

Linux Developers WG

Proposed WG chair
-----------------

Elana Copperman

Proposed meeting schedule
-------------------------

Weekly on Mondays, pending availability of participants

Proposed mission statement
-------------------------

To identify existing Linux Kernel features which may be leveraged for use in safety critical systems.

For example,

Mechanisms for protections of various memory types.

Dynamic analysis for multi-threaded systems.

Kernel profiling using ebpf-based tools.

Isolation techniques and FFI (Freedom From Interference)

Safety extensions to Linux drivers.


To bring together kernel developers and producers of safety critical
systems to demonstrate use of

such features in real systems, and to learn from these experiences together as a community.


To propose enhancements to such features and to work as a community
to design / implement /

deploy kernel patches. Such patches should help to make those
features more amenable

for use in safety critical systems.


To recommend tools or processes to be provided by other ELISA WGs so
that those patches

and features can be used by designers and integrators producing safety critical systems.


Planned activities
------------------
The scope of this WG does not in any way include safety
qualification or any safety claims on how the integrator can or
should use these features or patches. The only claims that would be made are a description of the feature and its functional impact.
Evidence to support deployment of any feature in a safety-critical
system may be clarified by work products produced by the other ELISA WGs, focusing on collaboration as outlined by Paul above.

The work group will invite developers of safety critical systems to
identify and to enhance existing kernel features with elements that may be used effectively to support safety claims.
I am a big fan of this working group and I think it deserves some exploration.

The key challenge I currently see is that "developers of safety
critical systems" (which---by historic incident---have a significant
different background on the complexity of software they have been
developing and handling, and the style it has been handled) will
attempt "to enhance existing kernel features" (which have been
developed in a very different manner and evolve in a different
manner).

I feel that I simply do not have the kernel development experience
with the few basic patches I submitted and bandwidth to contribute
here with critical development contributions. I hope that there are
others that have the needed kernel development knowledge and
experience to quickly start feature development, and that this does
just end up creating a wish list of ambitious kernel features that are
impossible to implement.
Upps... missed a critical word in the sentence above:

...and that this does NOT just end up creating a wish list of ambitious kernel features that are impossible to implement.

Note to myself: Read before sending, not afterwards...

Lukas
Lukas Bulwahn
 

On Sat, Sep 18, 2021 at 10:24 PM Elana Copperman
<Elana.Copperman@...> wrote:

Thanks for comments.
2 quick notes:
1. Still waiting for additional ideas on relevant names - perhaps a prize to the winning name?
I am all in for prizes! That is a good way to see some nice suggestions...

2. Lukas, some of my original text was unfortunately omitted. All of the suggested ideas are working features, not academic exercises or "ambitious ideas that are impossible to implement."
The call is to those who actually design and develop production systems.
The key challenge is to provide a supportive environment so that the open source community can benefit.
Great! You are addressing my concern. So, I think this can be a
successful WG, and we should kick it off and get started.

I agree with your assessment of the key challenge.
In that regard, providing a supportive environment, I think one of the
first points might be to understand:

- What is currently blocking those who actually design and develop
production systems to contribute the features? knowledge gap?
organisational aspects? some missing alignment?
- What does the forum on the linux-kernel mailing list and related
development channel already offer to provide a supportive environment
for those who actually design and develop production systems?
- What is missing in the linux-kernel community to provide a
supportive environment for those who actually design and develop
production systems?
- How can we address those missing aspects to provide a supportive
environment for those who actually design and develop production
systems?

I currently have only a rough understanding of the blockers from
informal discussions among developers and some managers, but I think
the topics that are blocking contributions are widely spread, and
partly deeply rooted, such as technical knowledge gap or challenging
strategic considerations, such as developing a proper open-source
strategy that includes investments in upstream work.

I would be interested to see if we could create a---potentially
anonymized (under Chatham House Rules))---list of blockers for those
who actually design and develop production systems to effectively
contribute.

Lukas

Regards
Elana


-----Original Message-----
From: Lukas Bulwahn <lukas.bulwahn@...>
Sent: Wednesday, September 15, 2021 7:07 PM
To: Elana Copperman <Elana.Copperman@...>
Cc: Paul Albertella <paul.albertella@...>; devel@...
Subject: Re: [ELISA Technical Community] New WG proposals: Safety Engineering Process, Linux Developers

On Wed, Sep 15, 2021 at 6:04 PM Lukas Bulwahn <lukas.bulwahn@...> wrote:

Proposed working group name
---------------------------

Linux Developers WG

Proposed WG chair
-----------------

Elana Copperman

Proposed meeting schedule
-------------------------

Weekly on Mondays, pending availability of participants

Proposed mission statement
-------------------------

To identify existing Linux Kernel features which may be leveraged for use in safety critical systems.

For example,

Mechanisms for protections of various memory types.

Dynamic analysis for multi-threaded systems.

Kernel profiling using ebpf-based tools.

Isolation techniques and FFI (Freedom From Interference)

Safety extensions to Linux drivers.


To bring together kernel developers and producers of safety critical
systems to demonstrate use of

such features in real systems, and to learn from these experiences together as a community.


To propose enhancements to such features and to work as a community
to design / implement /

deploy kernel patches. Such patches should help to make those
features more amenable

for use in safety critical systems.


To recommend tools or processes to be provided by other ELISA WGs so
that those patches

and features can be used by designers and integrators producing safety critical systems.


Planned activities
------------------
The scope of this WG does not in any way include safety
qualification or any safety claims on how the integrator can or
should use these features or patches. The only claims that would be made are a description of the feature and its functional impact.
Evidence to support deployment of any feature in a safety-critical
system may be clarified by work products produced by the other ELISA WGs, focusing on collaboration as outlined by Paul above.

The work group will invite developers of safety critical systems to
identify and to enhance existing kernel features with elements that may be used effectively to support safety claims.
I am a big fan of this working group and I think it deserves some exploration.

The key challenge I currently see is that "developers of safety
critical systems" (which---by historic incident---have a significant
different background on the complexity of software they have been
developing and handling, and the style it has been handled) will
attempt "to enhance existing kernel features" (which have been
developed in a very different manner and evolve in a different
manner).

I feel that I simply do not have the kernel development experience
with the few basic patches I submitted and bandwidth to contribute
here with critical development contributions. I hope that there are
others that have the needed kernel development knowledge and
experience to quickly start feature development, and that this does
just end up creating a wish list of ambitious kernel features that are
impossible to implement.
Upps... missed a critical word in the sentence above:

...and that this does NOT just end up creating a wish list of ambitious kernel features that are impossible to implement.

Note to myself: Read before sending, not afterwards...

Lukas
elana.copperman@...
 

Thanks, Lukas.  See replies in-line below.


From: devel@... <devel@...> on behalf of Lukas Bulwahn <lukas.bulwahn@...>
Sent: Monday, September 20, 2021 12:05 PM
To: Elana Copperman <Elana.Copperman@...>
Cc: Paul Albertella <paul.albertella@...>; devel@... <devel@...>
Subject: Re: [ELISA Technical Community] New WG proposals: Safety Engineering Process, Linux Developers
 
On Sat, Sep 18, 2021 at 10:24 PM Elana Copperman
<Elana.Copperman@...> wrote:
>
> Thanks for comments.
> 2 quick notes:
> 1. Still waiting for additional ideas on relevant names - perhaps a prize to the winning name?

I am all in for prizes! That is a good way to see some nice suggestions...
[EC] Well, let's see some suggestions coming in ...

> 2. Lukas, some of my original text was unfortunately omitted.  All of the suggested ideas are working features, not academic exercises or "ambitious ideas that are impossible to implement."
> The call is to those who actually design and develop production systems.
> The key challenge is to provide a supportive environment so that the open source community can benefit.

Great! You are addressing my concern. So, I think this can be a
successful WG, and we should kick it off and get started.
[EC] We will discuss at the TSC meeting on 29.9

I agree with your assessment of the key challenge.
In that regard, providing a supportive environment, I think one of the
first points might be to understand:

- What is currently blocking those who actually design and develop
production systems to contribute the features? knowledge gap?
organisational aspects? some missing alignment?
[EC} Blocking by safety assessment.  It is straightforward to qualify proprietary (not open-source) safety mechanisms in user applications.  Kernel patches, however, are currently impossible to qualify.  This is a major issue which I hope Paul can address.

- What does the forum on the linux-kernel mailing list and related
development channel already offer to provide a supportive environment
for those who actually design and develop production systems?
[EC] Nothing.  Hopefully the new WG with Paul's leadership will make some breakthroughs.

- What is missing in the linux-kernel community to provide a
supportive environment for those who actually design and develop
production systems?
[EC] A forum for collaboration with safety experts.  Collaboration means listening to each other and working together.  Not an easy challenge. 

- How can we address those missing aspects to provide a supportive
environment for those who actually design and develop production
systems?
[EC] that's what Paul and I are trying to do.  🙂

I currently have only a rough understanding of the blockers from
informal discussions among developers and some managers, but I think
the topics that are blocking contributions are widely spread, and
partly deeply rooted, such as technical knowledge gap or challenging
strategic considerations, such as developing a proper open-source
strategy that includes investments in upstream work.

I would be interested to see if we could create a---potentially
anonymized (under Chatham House Rules))---list of blockers for those
who actually design and develop production systems to effectively
contribute.
[EC] You are welcome to get this started.

Lukas

> Regards
> Elana
>
>
> -----Original Message-----
> From: Lukas Bulwahn <lukas.bulwahn@...>
> Sent: Wednesday, September 15, 2021 7:07 PM
> To: Elana Copperman <Elana.Copperman@...>
> Cc: Paul Albertella <paul.albertella@...>; devel@...
> Subject: Re: [ELISA Technical Community] New WG proposals: Safety Engineering Process, Linux Developers
>
> On Wed, Sep 15, 2021 at 6:04 PM Lukas Bulwahn <lukas.bulwahn@...> wrote:
> >
> > > Proposed working group name
> > > ---------------------------
> > >
> > > Linux Developers WG
> > >
> > > Proposed WG chair
> > > -----------------
> > >
> > > Elana Copperman
> > >
> > > Proposed meeting schedule
> > > -------------------------
> > >
> > > Weekly on Mondays, pending availability of participants
> > >
> > > Proposed mission statement
> > > -------------------------
> > >
> > > To identify existing Linux Kernel features which may be leveraged for use in safety critical systems.
> > >
> > > For example,
> > >
> > > Mechanisms for protections of various memory types.
> > >
> > > Dynamic analysis for multi-threaded systems.
> > >
> > > Kernel profiling using ebpf-based tools.
> > >
> > > Isolation techniques and FFI (Freedom From Interference)
> > >
> > > Safety extensions to Linux drivers.
> > >
> > >
> > > To bring together kernel developers and producers of safety critical
> > > systems to demonstrate use of
> > >
> > > such features in real systems, and to learn from these experiences together as a community.
> > >
> > >
> > > To propose enhancements to such features and to work as a community
> > > to design / implement /
> > >
> > > deploy kernel patches.  Such patches should help to make those
> > > features more amenable
> > >
> > > for use in safety critical systems.
> > >
> > >
> > > To recommend tools or processes to be provided by other ELISA WGs so
> > > that those patches
> > >
> > > and features can be used by designers and integrators producing safety critical systems.
> > >
> > >
> > > Planned activities
> > > ------------------
> > > The scope of this WG does not in any way include safety
> > > qualification or any safety claims on how the integrator can or
> > > should use these features or patches.  The only claims that would be made are a description of the feature and its functional impact.
> > > Evidence to support deployment of any feature in a safety-critical
> > > system may be clarified by work products produced by the other ELISA WGs, focusing on collaboration as outlined by Paul above.
> > >
> > > The work group will invite developers of safety critical systems to
> > > identify and to enhance existing kernel features with elements that may be used effectively to support safety claims.
> > >
> >
> > I am a big fan of this working group and I think it deserves some exploration.
> >
> > The key challenge I currently see is that "developers of safety
> > critical systems" (which---by historic incident---have a significant
> > different background on the complexity of software they have been
> > developing and handling, and the style it has been handled) will
> > attempt "to enhance existing kernel features" (which have been
> > developed in a very different manner and evolve in a different
> > manner).
> >
> > I feel that I simply do not have the kernel development experience
> > with the few basic patches I submitted and bandwidth to contribute
> > here with critical development contributions. I hope that there are
> > others that have the needed kernel development knowledge and
> > experience to quickly start feature development, and that this does
> > just end up creating a wish list of ambitious kernel features that are
> > impossible to implement.
> >
>
> Upps... missed a critical word in the sentence above:
>
> ...and that this does NOT just end up creating a wish list of ambitious kernel features that are impossible to implement.
>
> Note to myself: Read before sending, not afterwards...
>
> Lukas
Lukas Bulwahn
 

I agree with your assessment of the key challenge.
In that regard, providing a supportive environment, I think one of the
first points might be to understand:

- What is currently blocking those who actually design and develop
production systems to contribute the features? knowledge gap?
organisational aspects? some missing alignment?
[EC} Blocking by safety assessment. It is straightforward to qualify proprietary (not open-source) safety mechanisms in user applications. Kernel patches, however, are currently impossible to qualify. This is a major issue which I hope Paul can address.
My question was really looking for more practical answers here.
E.g., just for the purpose of illustration: Although I understand what
kind of changes to the memory management system might help verifying
or validating critical property (relevant for safety systems), I
cannot implement and submit anything relevant to the kernel's memory
management because I personally do not have the necessary in-depth
knowledge for memory management.

Your hypothesis "Kernel patches are currently impossible to qualify"
needs to be due to certain reasons; I would really like to understand
why this is currently impossible? Is it due to lack of knowledge about
the contributor and its development process? Is it due to lack of
expertise in understanding and judging the existing code? etc.

- What does the forum on the linux-kernel mailing list and related
development channel already offer to provide a supportive environment
for those who actually design and develop production systems?
[EC] Nothing. Hopefully the new WG with Paul's leadership will make some breakthroughs.
Again, I would like to understand this more specifically:

Stating that the linux-kernel mailing list provides nothing, seems a
bit harsh. For the code improvement group, we identified that the
linux-kernel mailing list provides a way for obtaining feedback and
integrating our improvements (and that we do not need any separate
channels, communities or networks); we just lack partly the
skill/comprehension and partly the time to assess all bug
findings/issues we identified, but for a few simple issues, each
participant could submit an improvement and get it integrated with the
help of the linux-kernel mailing list and the established maintainers.

Can you not integrate the features you developed through the
linux-kernel mailing list? Is that broken for the code you would like
to modify and extend? Why are these features rejected?

- What is missing in the linux-kernel community to provide a
supportive environment for those who actually design and develop
production systems?
[EC] A forum for collaboration with safety experts. Collaboration means listening to each other and working together. Not an easy challenge.

- How can we address those missing aspects to provide a supportive
environment for those who actually design and develop production
systems?
[EC] that's what Paul and I are trying to do. 🙂

I currently have only a rough understanding of the blockers from
informal discussions among developers and some managers, but I think
the topics that are blocking contributions are widely spread, and
partly deeply rooted, such as technical knowledge gap or challenging
strategic considerations, such as developing a proper open-source
strategy that includes investments in upstream work.

I would be interested to see if we could create a---potentially
anonymized (under Chatham House Rules))---list of blockers for those
who actually design and develop production systems to effectively
contribute.
[EC] You are welcome to get this started.
Let us see if there are others that are willing to share their experiences.


Lukas
elana.copperman@...
 

Thanks, Lukas, for your comments.  See brief replies below; we should continue this discussion next week, on my return from holiday.

From: Lukas Bulwahn <lukas.bulwahn@...>
Sent: Monday, September 20, 2021 10:00 PM
To: Elana Copperman <Elana.Copperman@...>
Cc: devel@... <devel@...>; Paul Albertella <paul.albertella@...>
Subject: Re: [ELISA Technical Community] New WG proposals: Safety Engineering Process, Linux Developers
 
> I agree with your assessment of the key challenge.
> In that regard, providing a supportive environment, I think one of the
> first points might be to understand:
>
> - What is currently blocking those who actually design and develop
> production systems to contribute the features? knowledge gap?
> organisational aspects? some missing alignment?
> [EC} Blocking by safety assessment.  It is straightforward to qualify proprietary (not open-source) safety mechanisms in user applications.  Kernel patches, however, are currently impossible to qualify.  This is a major issue which I hope Paul can address.
>

My question was really looking for more practical answers here.
E.g., just for the purpose of illustration: Although I understand what
kind of changes to the memory management system might help verifying
or validating critical property (relevant for safety systems), I
cannot implement and submit anything relevant to the kernel's memory
management because I personally do not have the necessary in-depth
knowledge for memory management.

Your hypothesis "Kernel patches are currently impossible to qualify"
needs to be due to certain reasons; I would really like to understand
why this is currently impossible? Is it due to lack of knowledge about
the contributor and its development process? Is it due to lack of
expertise in understanding and judging the existing code? etc.
>>>>  Not at all.  It is the basic mistrust of Linux which is manifested in ELISA as well. 
>>>>  We are seeing in the Architecture WG what a long process would be needed for a trivial use case.
>>>>  Qualifying a complex real-life production use case for release as a kernel patch would incur unreasonable cost, effort and resources.

> - What does the forum on the linux-kernel mailing list and related
> development channel already offer to provide a supportive environment
> for those who actually design and develop production systems?
> [EC] Nothing.  Hopefully the new WG with Paul's leadership will make some breakthroughs.
>

Again, I would like to understand this more specifically:

Stating that the linux-kernel mailing list provides nothing, seems a
bit harsh. 
>>>> Where was this statement made?  

For the code improvement group, we identified that the
linux-kernel mailing list provides a way for obtaining feedback and
integrating our improvements (and that we do not need any separate
channels, communities or networks); we just lack partly the
skill/comprehension and partly the time to assess all bug
findings/issues we identified, but for a few simple issues, each
participant could submit an improvement and get it integrated with the
help of the linux-kernel mailing list and the established maintainers.

Can you not integrate the features you developed through the
linux-kernel mailing list? Is that broken for the code you would like
to modify and extend? Why are these features rejected?
>>>> ??? Who developed features through the Linux kernel mailing list?

> - What is missing in the linux-kernel community to provide a
> supportive environment for those who actually design and develop
> production systems?
> [EC] A forum for collaboration with safety experts.  Collaboration means listening to each other and working together.  Not an easy challenge.
>
> - How can we address those missing aspects to provide a supportive
> environment for those who actually design and develop production
> systems?
> [EC] that's what Paul and I are trying to do.  🙂
>
> I currently have only a rough understanding of the blockers from
> informal discussions among developers and some managers, but I think
> the topics that are blocking contributions are widely spread, and
> partly deeply rooted, such as technical knowledge gap or challenging
> strategic considerations, such as developing a proper open-source
> strategy that includes investments in upstream work.
>
> I would be interested to see if we could create a---potentially
> anonymized (under Chatham House Rules))---list of blockers for those
> who actually design and develop production systems to effectively
> contribute.
> [EC] You are welcome to get this started.
>

Let us see if there are others that are willing to share their experiences.


Lukas
Lukas Bulwahn
 

On Tue, Sep 21, 2021 at 10:32 PM Elana Copperman
<Elana.Copperman@...> wrote:

Thanks, Lukas, for your comments. See brief replies below; we should continue this discussion next week, on my return from holiday.
Elana, enjoy your vacation! This email thread can wait another few
days until we continue.

Lukas
elana.copperman@...
 

Back from holiday, taking this up once again.

 

From: devel@... <devel@...> On Behalf Of Elana Copperman
Sent: Tuesday, September 21, 2021 11:33 PM
To: Lukas Bulwahn <lukas.bulwahn@...>
Cc: devel@...; Paul Albertella <paul.albertella@...>
Subject: Re: [ELISA Technical Community] New WG proposals: Safety Engineering Process, Linux Developers

 

Thanks, Lukas, for your comments.  See brief replies below; we should continue this discussion next week, on my return from holiday.

 

From: Lukas Bulwahn <lukas.bulwahn@...>
Sent: Monday, September 20, 2021 10:00 PM
To: Elana Copperman <Elana.Copperman@...>
Cc: devel@... <devel@...>; Paul Albertella <paul.albertella@...>
Subject: Re: [ELISA Technical Community] New WG proposals: Safety Engineering Process, Linux Developers

 

> I agree with your assessment of the key challenge.
> In that regard, providing a supportive environment, I think one of the
> first points might be to understand:
>
> - What is currently blocking those who actually design and develop
> production systems to contribute the features? knowledge gap?
> organisational aspects? some missing alignment?
> [EC} Blocking by safety assessment.  It is straightforward to qualify proprietary (not open-source) safety mechanisms in user applications.  Kernel patches, however, are currently impossible to qualify.  This is a major issue which I hope Paul can address.
>

My question was really looking for more practical answers here.
E.g., just for the purpose of illustration: Although I understand what
kind of changes to the memory management system might help verifying
or validating critical property (relevant for safety systems), I
cannot implement and submit anything relevant to the kernel's memory
management because I personally do not have the necessary in-depth
knowledge for memory management.

Your hypothesis "Kernel patches are currently impossible to qualify"
needs to be due to certain reasons; I would really like to understand
why this is currently impossible? Is it due to lack of knowledge about
the contributor and its development process? Is it due to lack of
expertise in understanding and judging the existing code? etc.

>>>>  Not at all.  It is the basic mistrust of Linux which is manifested in ELISA as well. 

>>>>  We are seeing in the Architecture WG what a long process would be needed for a trivial use case.

>>>>  Qualifying a complex real-life production use case for release as a kernel patch would incur unreasonable cost, effort and resources.

> - What does the forum on the linux-kernel mailing list and related
> development channel already offer to provide a supportive environment
> for those who actually design and develop production systems?
> [EC] Nothing.  Hopefully the new WG with Paul's leadership will make some breakthroughs.
>

Again, I would like to understand this more specifically:

Stating that the linux-kernel mailing list provides nothing, seems a
bit harsh. 

>>>> Where was this statement made?  

 

For the code improvement group, we identified that the
linux-kernel mailing list provides a way for obtaining feedback and
integrating our improvements (and that we do not need any separate
channels, communities or networks); we just lack partly the
skill/comprehension and partly the time to assess all bug
findings/issues we identified, but for a few simple issues, each
participant could submit an improvement and get it integrated with the
help of the linux-kernel mailing list and the established maintainers.

Can you not integrate the features you developed through the
linux-kernel mailing list? Is that broken for the code you would like
to modify and extend? Why are these features rejected?
>>>> ??? Who developed features through the Linux kernel mailing list?

>>>> The features to be developed can certainly be introduced via the Linux kernel mailing list.

>>>> However, the expectation is that they will not be accepted by safety assessors if introduced in this way.

>>>> Instead, they are introduced as proprietary user space applications, and then much more readily approved.

>>>> We need to find an appropriate way to introduce open source Linux-based features which are accepted.


> - What is missing in the linux-kernel community to provide a
> supportive environment for those who actually design and develop
> production systems?
> [EC] A forum for collaboration with safety experts.  Collaboration means listening to each other and working together.  Not an easy challenge.
>
> - How can we address those missing aspects to provide a supportive
> environment for those who actually design and develop production
> systems?
> [EC] that's what Paul and I are trying to do.  🙂
>
> I currently have only a rough understanding of the blockers from
> informal discussions among developers and some managers, but I think
> the topics that are blocking contributions are widely spread, and
> partly deeply rooted, such as technical knowledge gap or challenging
> strategic considerations, such as developing a proper open-source
> strategy that includes investments in upstream work.
>
> I would be interested to see if we could create a---potentially
> anonymized (under Chatham House Rules))---list of blockers for those
> who actually design and develop production systems to effectively
> contribute.
> [EC] You are welcome to get this started.
>

Let us see if there are others that are willing to share their experiences.


Lukas
elana.copperman@...
 

Still waiting for concrete suggestions for the new WG.  @Lukas Bulwahn – no one to take the prize?

Currently the only proposal is "Linux Developers WG", which we all agreed is not ideal.

Some additional suggestions:

  1. Linux features for safety
  2. Linux features amenable for safety
  3. Safety engineering features
  4. Linux-based elements for safety-critical systems

(none of which is ideal, but as a basis for discussion).

Thanks

Elana

 

From: devel@... <devel@...> On Behalf Of Elana Copperman
Sent: Monday, September 20, 2021 3:01 PM
To: Lukas Bulwahn <lukas.bulwahn@...>
Cc: Paul Albertella <paul.albertella@...>; devel@...
Subject: Re: [ELISA Technical Community] New WG proposals: Safety Engineering Process, Linux Developers

 

Thanks, Lukas.  See replies in-line below.

 

 

From: devel@... <devel@...> on behalf of Lukas Bulwahn <lukas.bulwahn@...>
Sent: Monday, September 20, 2021 12:05 PM
To: Elana Copperman <Elana.Copperman@...>
Cc: Paul Albertella <paul.albertella@...>; devel@... <devel@...>
Subject: Re: [ELISA Technical Community] New WG proposals: Safety Engineering Process, Linux Developers

 

On Sat, Sep 18, 2021 at 10:24 PM Elana Copperman
<Elana.Copperman@...> wrote:
>
> Thanks for comments.
> 2 quick notes:
> 1. Still waiting for additional ideas on relevant names - perhaps a prize to the winning name?

I am all in for prizes! That is a good way to see some nice suggestions...

[EC] Well, let's see some suggestions coming in ...


 
Lukas Bulwahn
 

- How can we address those missing aspects to provide a supportive
environment for those who actually design and develop production
systems?
[EC] that's what Paul and I are trying to do. 🙂

I currently have only a rough understanding of the blockers from
informal discussions among developers and some managers, but I think
the topics that are blocking contributions are widely spread, and
partly deeply rooted, such as technical knowledge gap or challenging
strategic considerations, such as developing a proper open-source
strategy that includes investments in upstream work.

I would be interested to see if we could create a---potentially
anonymized (under Chatham House Rules))---list of blockers for those
who actually design and develop production systems to effectively
contribute.
[EC] You are welcome to get this started.
Let us see if there are others that are willing to share their experiences.
I will create a set of survey questions to collect some feedback on
that in a structured way.

Lukas
Lukas Bulwahn
 

(snip)


For the code improvement group, we identified that the
linux-kernel mailing list provides a way for obtaining feedback and
integrating our improvements (and that we do not need any separate
channels, communities or networks); we just lack partly the
skill/comprehension and partly the time to assess all bug
findings/issues we identified, but for a few simple issues, each
participant could submit an improvement and get it integrated with the
help of the linux-kernel mailing list and the established maintainers.

Can you not integrate the features you developed through the
linux-kernel mailing list? Is that broken for the code you would like
to modify and extend? Why are these features rejected?
??? Who developed features through the Linux kernel mailing list?
The features to be developed can certainly be introduced via the Linux kernel mailing list.
However, the expectation is that they will not be accepted by safety assessors if introduced in this way.
Instead, they are introduced as proprietary user space applications, and then much more readily approved.
We need to find an appropriate way to introduce open source Linux-based features which are accepted.
I am sorry. I am confused and I do not understand.

Let us assume:
A new feature for the kernel, i.e., a feature extending the kernel's
functionality, has been developed in a reasonably systematic way with
sufficient documentation to argue sufficient understanding of the
functionality and its contribution to a specific safety claim.

Side question: Is it already unrealistic or difficult to meet this
assumption above? Is there a gap to develop in a reasonably systematic
way? Is there a knowledge/process gap to create documentation of the
functionality? Is it difficult to formulate the specific safety claim
of interest and argue the contribution?

If that assumption above is met, is such a feature and its related
patches accepted by safety assessors?

I assume all those aspects and questions are so far unrelated to
integrating patches in the kernel through the kernel mailing list.

Next, then, these patches and documentation are sent to the
linux-kernel mailing list. Then, they are integrated into the kernel
repository, then the source code is released and distributed. Then, it
is compiled by somebody, put into a system and used for some purpose.

At which point in these steps, does the feature switch from being
acceptable for safety assessor to not being acceptable to safety
assessor? What happens in that step such that the feature is not
acceptable?

Lukas

(snip)
elana.copperman@...
 

Lukas, this discussion does not seem to be appropriate for the mailing list, and if there is sufficient interest - we can use tomorrow's call for this purpose. @Paul Albertella?

In brief, the process which you describe below gets stuck because Linux is not qualified. To accept a kernel patch as a basis for safety claims, given that such a patch inevitably uses existing kernel features, our experience has been that you would need a fully qualified Linux kernel to support the kernel patch development process.
Elana

-----Original Message-----
From: Lukas Bulwahn <lukas.bulwahn@...>
Sent: Wednesday, September 29, 2021 9:34 AM
To: Elana Copperman <Elana.Copperman@...>
Cc: devel@...; Paul Albertella <paul.albertella@...>
Subject: Re: [ELISA Technical Community] New WG proposals: Safety Engineering Process, Linux Developers

(snip)


For the code improvement group, we identified that the linux-kernel
mailing list provides a way for obtaining feedback and integrating our
improvements (and that we do not need any separate channels,
communities or networks); we just lack partly the skill/comprehension
and partly the time to assess all bug findings/issues we identified,
but for a few simple issues, each participant could submit an
improvement and get it integrated with the help of the linux-kernel
mailing list and the established maintainers.

Can you not integrate the features you developed through the
linux-kernel mailing list? Is that broken for the code you would like
to modify and extend? Why are these features rejected?
??? Who developed features through the Linux kernel mailing list?
The features to be developed can certainly be introduced via the Linux kernel mailing list.
However, the expectation is that they will not be accepted by safety assessors if introduced in this way.
Instead, they are introduced as proprietary user space applications, and then much more readily approved.
We need to find an appropriate way to introduce open source Linux-based features which are accepted.
I am sorry. I am confused and I do not understand.

Let us assume:
A new feature for the kernel, i.e., a feature extending the kernel's functionality, has been developed in a reasonably systematic way with sufficient documentation to argue sufficient understanding of the functionality and its contribution to a specific safety claim.

Side question: Is it already unrealistic or difficult to meet this assumption above? Is there a gap to develop in a reasonably systematic way? Is there a knowledge/process gap to create documentation of the functionality? Is it difficult to formulate the specific safety claim of interest and argue the contribution?

If that assumption above is met, is such a feature and its related patches accepted by safety assessors?

I assume all those aspects and questions are so far unrelated to integrating patches in the kernel through the kernel mailing list.

Next, then, these patches and documentation are sent to the linux-kernel mailing list. Then, they are integrated into the kernel repository, then the source code is released and distributed. Then, it is compiled by somebody, put into a system and used for some purpose.

At which point in these steps, does the feature switch from being acceptable for safety assessor to not being acceptable to safety assessor? What happens in that step such that the feature is not acceptable?

Lukas

(snip)
Lukas Bulwahn
 

On Wed, Sep 29, 2021 at 8:38 AM Elana Copperman
<Elana.Copperman@...> wrote:

Lukas, this discussion does not seem to be appropriate for the mailing list, and if there is sufficient interest - we can use tomorrow's call for this purpose. @Paul Albertella?

In brief, the process which you describe below gets stuck because Linux is not qualified. To accept a kernel patch as a basis for safety claims, given that such a patch inevitably uses existing kernel features, our experience has been that you would need a fully qualified Linux kernel to support the kernel patch development process.
Okay, thanks. Now, I understand why we get stuck. Of course, if we
come to the conclusion that a fully qualified Linux kernel (whatever
that means; I have no clear semantics for such a term; qualification
always happens in the context of a (assumed) system) is required for
any feature we may develop, then we really might be stuck.

I resolved that question for the scope that was required for the
feature to be integrated. By employing general software engineering
arguments, I could limit that to a clearly manageable scope of kernel
functionality. If there is interest in how that is resolved, I would
suggest just looking at a specific feature and reconsider the
challenge we encountered that leads to the point above.

I am not a big fan of meetings; email is often the better means of
communication for technical discussions. Only, seldomly, a meeting
really adds value to a discussion of in-depth considerations.

Lukas
elana.copperman@...
 

We can start with cgroups and namespaces, focusing on the questions raised by Bruce and Priyanka in their presentation last week.
Emails would work if you would leave the original full discussion, it is very difficult to follow this censored flow.

-----Original Message-----
From: Lukas Bulwahn <lukas.bulwahn@...>
Sent: Wednesday, September 29, 2021 11:51 AM
To: Elana Copperman <Elana.Copperman@...>
Cc: Paul Albertella <paul.albertella@...>; devel@...
Subject: Re: [ELISA Technical Community] New WG proposals: Safety Engineering Process, Linux Developers

On Wed, Sep 29, 2021 at 8:38 AM Elana Copperman <Elana.Copperman@...> wrote:

Lukas, this discussion does not seem to be appropriate for the mailing list, and if there is sufficient interest - we can use tomorrow's call for this purpose. @Paul Albertella?

In brief, the process which you describe below gets stuck because Linux is not qualified. To accept a kernel patch as a basis for safety claims, given that such a patch inevitably uses existing kernel features, our experience has been that you would need a fully qualified Linux kernel to support the kernel patch development process.
Okay, thanks. Now, I understand why we get stuck. Of course, if we come to the conclusion that a fully qualified Linux kernel (whatever that means; I have no clear semantics for such a term; qualification always happens in the context of a (assumed) system) is required for any feature we may develop, then we really might be stuck.

I resolved that question for the scope that was required for the feature to be integrated. By employing general software engineering arguments, I could limit that to a clearly manageable scope of kernel functionality. If there is interest in how that is resolved, I would suggest just looking at a specific feature and reconsider the challenge we encountered that leads to the point above.

I am not a big fan of meetings; email is often the better means of communication for technical discussions. Only, seldomly, a meeting really adds value to a discussion of in-depth considerations.

Lukas
Lukas Bulwahn
 

On Wed, Sep 29, 2021 at 10:58 AM Elana Copperman
<Elana.Copperman@...> wrote:

We can start with cgroups and namespaces, focusing on the questions raised by Bruce and Priyanka in their presentation last week.
Emails would work if you would leave the original full discussion, it is very difficult to follow this censored flow.
Agree. That is a really good idea. We need to establish proper
recommendations and policies for well-structured email-based
discussions. I will reach out to Shuah for the TSC meeting today to
have a smaller group describe recommendations and policies for
well-structured email-based discussions.

Then, we document those for the overall group exchanging emails, and
potentially even enforce certain policies for email discussions, such
as already done on linux-kernel@..., and continuously
pointed out by some technical stewards/mentors on similar lists.

Lukas


-----Original Message-----
From: Lukas Bulwahn <lukas.bulwahn@...>
Sent: Wednesday, September 29, 2021 11:51 AM
To: Elana Copperman <Elana.Copperman@...>
Cc: Paul Albertella <paul.albertella@...>; devel@...
Subject: Re: [ELISA Technical Community] New WG proposals: Safety Engineering Process, Linux Developers

On Wed, Sep 29, 2021 at 8:38 AM Elana Copperman <Elana.Copperman@...> wrote:

Lukas, this discussion does not seem to be appropriate for the mailing list, and if there is sufficient interest - we can use tomorrow's call for this purpose. @Paul Albertella?

In brief, the process which you describe below gets stuck because Linux is not qualified. To accept a kernel patch as a basis for safety claims, given that such a patch inevitably uses existing kernel features, our experience has been that you would need a fully qualified Linux kernel to support the kernel patch development process.
Okay, thanks. Now, I understand why we get stuck. Of course, if we come to the conclusion that a fully qualified Linux kernel (whatever that means; I have no clear semantics for such a term; qualification always happens in the context of a (assumed) system) is required for any feature we may develop, then we really might be stuck.

I resolved that question for the scope that was required for the feature to be integrated. By employing general software engineering arguments, I could limit that to a clearly manageable scope of kernel functionality. If there is interest in how that is resolved, I would suggest just looking at a specific feature and reconsider the challenge we encountered that leads to the point above.

I am not a big fan of meetings; email is often the better means of communication for technical discussions. Only, seldomly, a meeting really adds value to a discussion of in-depth considerations.

Lukas

1 - 17 of 17