Hi Lukas,

Thanks very much for your supportive words and thoughtful feedback!

Responses inline below.

On 14/09/2021 17:36, Lukas Bulwahn wrote:

My preference on naming, hence, would be on:

- Evidence WG
- Claims on Open-Source Software WG
- Open-Source Engineering Process WG

The name is important, just for newcomers and outsiders not to set the
wrong expectations.

Agreed!

The consensus in the last Dev Process WG meeting was that the name
needed to have the word 'safety' in it (as with the Safety Architecture
WG) to make it clear to participants that safety is explicitly in scope
for our discussions!

Participants expressed a concern that 'Evidence' and 'Claims', while
descriptive of the intended *approach* for the group, do not clearly
express its intended *scope* (Claims about what? Evidence of what?).
'Engineering Process' was therefore proposed as the best overall
description of this scope.

However, I believe that the intention with the proposed name was to
focus on the role of 'Engineering Processes in Safety', as opposed to
'Processes for Safety Engineering', so the name may need a rethink.

Let's discuss this in tomorrow's WG meeting.

(Linguistic sidebar: The English language is renowned for its
imprecision and ambiguity when building compound descriptors in this
way, because it lacks clear grammar rules to aid parsing!)

I do not know if LTP (it stands for Linux Test Project, right?) is a
community or organisation; I would really just think LTP stands for
the test suite itself, i.e., the source code in the repository.

I mentioned LTP because it is both a consumer of Linux and a source of
of potentially relevant engineering process material (test suites) that
relate to it. If we wanted to analyse that material, and the processes
around it, then we could gather evidence from the project's mailing
list, Github project, and documentation (see
http://linux-test-project.github.io/).

However, we wouldn't *necessarily* plan to use evidence relating to the
work of LTP as a community/organisation to directly support our claims
about Linux. We might instead consider how the LTP's processes, and the
materials that it generates and maintains, might potentially be re-used
by an organisation incorporating Linux as part of a product, to generate
supporting evidence as part of *its* engineering processes.

I wrote:

As part of this work, we also plan to write a summary of the results and
conclusions of the work done by the Development Process WG, which will
also be made available in an elisa.tech Github repo. This is intended to
provide background for new participants and other WGs, as well as a
baseline / conceptual framework for the work of the new group.

Lukas wrote:

I think that is quite important; it might also be a separate group
(the continuation of the development process WG; those that have
invested and learned something in that group) but certainly should
happen. I would be very interested in the overall summary, results and
conclusions. I got lost on the way on what the consensus and lessons
learned really were in the course of all the discussions.

I certainly agree that this will require contributions and review by the
members of the Dev Process WG who were involved in the historical
discussions, but I think it makes sense for the new working group to
'own' it as an activity.

Note that I'm *not* proposing to devote WG sessions to authoring or
reviewing these materials. The goal would be to identify topics that we
would like to cover, ask for volunteers to write them up and submit the
material in a GitHub PR, and reviewers to read and comment on these.
We'll then regularly review progress in WG meetings and decide when
contributions are ready for for 'publication' (merge into mainline).

If this activity come to dominate the new WG discussions (or we don't
find time to do it!) then we might alternatively plan to have a monthly
working session devoted to this.

My preference on naming, hence, would be on:

- Evidence WG
- Claims on Open-Source Software WG
- Open-Source Engineering Process WG

The name is important, just for newcomers and outsiders not to set the
wrong expectations.

Agreed!
The consensus in the last Dev Process WG meeting was that the name needed to have the word 'safety' in it (as with the Safety Architecture WG) to make it clear to participants that safety is explicitly in scope for our discussions!
Participants expressed a concern that 'Evidence' and 'Claims', while descriptive of the intended *approach* for the group, do not clearly express its intended *scope* (Claims about what? Evidence of what?). 'Engineering Process' was therefore proposed as the best overall description of this scope.
However, I believe that the intention with the proposed name was to focus on the role of 'Engineering Processes in Safety', as opposed to 'Processes for Safety Engineering', so the name may need a rethink.
Let's discuss this in tomorrow's WG meeting.

My preference on naming, hence, would be on:
- Evidence WG
- Claims on Open-Source Software WG
- Open-Source Engineering Process WG
The name is important, just for newcomers and outsiders not to set the
wrong expectations.

I do not know if LTP (it stands for Linux Test Project, right?) is a
community or organisation; I would really just think LTP stands for
the test suite itself, i.e., the source code in the repository.

As part of this work, we also plan to write a summary of the results and
conclusions of the work done by the Development Process WG, which will
also be made available in an elisa.tech Github repo. This is intended to
provide background for new participants and other WGs, as well as a
baseline / conceptual framework for the work of the new group.

I think that is quite important; it might also be a separate group
(the continuation of the development process WG; those that have
invested and learned something in that group) but certainly should
happen. I would be very interested in the overall summary, results and
conclusions. I got lost on the way on what the consensus and lessons
learned really were in the course of all the discussions.

Hi,

The Development Process working group has been discussing proposals for
the evolution of the group. The WG has historically attempted to cover a
number of goals in a single context, which has occasionally created
confusion; to address this, we propose to split into two separate
working groups, each with a clearer focus.

Elana Copperman and I have proposed mission statements and planned
activities for these two new groups, which have been discussed in the
Dev Process WG weekly calls.

Having reached some consensus about a way forward in that forum, I'd now
like to share my proposal (see below) with the wider ELISA community for
comment, before seeking to formalise the group at the TSC. I hope that
Elana will share details of her proposal in due course.

Regards,

Paul

---------------------------------------------------------------------

Proposed working group name
---------------------------

Safety Engineering Process

Proposed WG chair
-----------------

Paul Albertella

Proposed meeting schedule
-------------------------

Weekly on Thursday, in the same slot as the current Dev process call

Proposed mission statement
--------------------------

This working group aims to examine safety-related claims that we might
like to make about Linux as part of a system, and to explore how we can
gather and present evidence to support such claims as part of a safety
engineering process.

We are interested in two broad classes of claims and evidence:

a) Those relating to development practices for Linux as a whole; for
example, the peer review process for patches, the testing performed on a
kernel stable branch when preparing a release, or the testing performed
by a system integrator for a product that incorporates Linux

b) Those relating to specific properties or behaviour of Linux; for
example, features that we can enable or disable in a kernel config, the
(inferred) design of a subsystem, the characteristics of a driver, or
tests that can verify aspects of a given feature

For (b), we will focus on engineering process aspects (construction,
verification, integration, use and maintenance) relating to the feature
or property, rather than technical details of its design/implementation.

In both cases we may examine practices or processes that are undertaken
by the Linux kernel community, as well as communities or organisations
that consume Linux (e.g. AGL, LTP), or we may identify practices or
processes from other sources that might be used by organisations who
want to use Linux to build systems with safety requirements.

Planned activities
------------------

Our objectives are to:

* Define claims that we would like to make about Linux and/or the
processes used to develop it or integrate it as part of a product
* Identify evidence that we might use to support these claims
* Identify strategies or tools that we can use to gather, generate or
reformulate evidence (perhaps in collaboration with the Tools WG)

* Examine the evidence gathered and document our findings
* Share and peer-review our findings in an elisa.tech Github repo

Possible claims might be proposed by another WG, perhaps in relation to
a specific use case, technique or safety argument, or we may select our
own. Claims that we examine may not all be safety claims, per se; some
might relate to software quality, for example, or functionality that
does not have an explicit safety role, since both of these may be needed
to support safety arguments.

We will use contributors’ knowledge of specific safety standards to
inform our work where appropriate - for example to determine what
criteria to apply for a given class of evidence - but specifying how a
given claim and its supporting evidence may be used as part of a
certification process should be considered out of scope.

Where we do not have consensus on results or conclusions, we will
document the different perspectives and/or open questions. We will
manage the material produced in an elisa.tech Github repo and
collectively review it in GitHub PRs, inviting review or input from the
rest of the ELISA community where appropriate.

As part of this work, we also plan to write a summary of the results and
conclusions of the work done by the Development Process WG, which will
also be made available in an elisa.tech Github repo. This is intended to
provide background for new participants and other WGs, as well as a
baseline / conceptual framework for the work of the new group.

Collaboration
-------------

The group expects to collaborate with the existing working groups and
with the new group proposed by Elana, both to identify or discuss claims
and evidence for analysis, and to address wider questions relating to
safety engineering processes. Where we work with another group on given
topics, we may jointly present the results at an Elisa workshop.