Linux and ioctls
|
Elana C
Hi, To add to yesterday's discussion – see https://www.kernel.org/doc/html/latest/process/botching-up-ioctls.html This is basic common sense advice for any usage of ioctls. There is a nice summary of well know pitfalls to avoid (some of which we could check using static analysis tools). And the caveat that any usage of ioctls should be checked against preferred Linux options such as sysfs or debugfs.
Hardware dependent features such as the ioctl syscall, devmem, etc. should be avoided altogether whenever possible in safety critical applications, for reasons of safety as well as security. And the only really necessary usage should be in driver code, not safety critical management code. Regards Elana
|
|
|
|
Robert Krutsch
Hi Elena, The API of the watchdog is ioctl based in this case. Even if we would have a user-space API on top, as all complex drivers have we would still end up at the same ioctl as the interface between user space and kernel-space. What can be done is to say: "our watchdog is to be considered as a module out-of-tree and we assume someone else provides us with a watchdog module that fulfils the FuSa needs". From our req. we can deduce requirements for the watchdog module and stop there. This is a simplification of scope and would take out some parts of the analysis for this App but it would not solve the issues of someone that solves the problem for real. They would need to analyze the watchdog module and deal with the ioctl or rewrite this module altogether as a user-space driver. If that sounds familiar , it is basically what you have to do when you port to QNX... BR, Robert
On Wed, Mar 10, 2021 at 9:29 AM Elana C <elana.copperman@...> wrote:
|
|
|
|
Elana C
Thanks, Robert, for your comment. This is all fine and true. And as noted, at the end of the day, the drivers end up calling the ioctl system interface. But then my question ends up being, for qualification of the watchdog subsystem – will we need to qualify each driver and its implementation to support each specific hardware interface and each specific system call which accesses that hardware interface?
I am only concerned that this an endless saga. We can do this (as you say) "for real", and spend weeks digging in to each low level system call called by each driver. Moreover, if you check the header files for the watchdog core and dev utilities, you can readily see that the watchdog (as "simple" as it seems …) touches on many other complex subsystems – such as the file system and the memory management subsystem. Where does "for real" end? And more generically – why is this definition of "for real" in the use case of Linux (with its ~30 million lines of code …) so much more stringent than the "normal" definition of "for real" in safety qualification of non-open source commercial software? In the latter case, safety requirements are defined, implemented and validated on a much higher level. Regards Elana
From: safety-architecture@... <safety-architecture@...>
On Behalf Of Robert Krutsch
Sent: Wednesday, March 10, 2021 12:49 PM To: safety-architecture@... Subject: Re: [ELISA Safety Architecture WG] Linux and ioctls
Hi Elena,
The API of the watchdog is ioctl based in this case. Even if we would have a user-space API on top, as all complex drivers have we would still end up at the same ioctl as the interface between user space and kernel-space.
What can be done is to say: "our watchdog is to be considered as a module out-of-tree and we assume someone else provides us with a watchdog module that fulfils the FuSa needs". From our req. we can deduce requirements for the watchdog module and stop there.
This is a simplification of scope and would take out some parts of the analysis for this App but it would not solve the issues of someone that solves the problem for real. They would need to analyze the watchdog module and deal with the ioctl or rewrite this module altogether as a user-space driver. If that sounds familiar , it is basically what you have to do when you port to QNX...
BR, Robert
On Wed, Mar 10, 2021 at 9:29 AM Elana C <elana.copperman@...> wrote:
|
|
|
|
Paoloni, Gabriele <gabriele.paoloni@...>
Hi Elana, all
WRT the watchdog subsystem I believe that the analysis should be focused on HW independent functions. I.e.: # The WatchDog Timer Driver Core. obj-$(CONFIG_WATCHDOG_CORE) += watchdog.o
watchdog-objs += watchdog_core.o watchdog_dev.o
watchdog-$(CONFIG_WATCHDOG_PRETIMEOUT_GOV) += watchdog_pretimeout.o
obj-$(CONFIG_WATCHDOG_PRETIMEOUT_GOV_NOOP) += pretimeout_noop.o obj-$(CONFIG_WATCHDOG_PRETIMEOUT_GOV_PANIC) += pretimeout_panic.o from Note that at this stage I still don’t know if the settings CONFIG_WATCHDOG_PRETIMEOUT_GOV * are applicable (to be figured out…)
Everything below this point in the Makefile is HW specific and I don’t think should be considered in the analysis (or at least it is lower priority) Thanks Gab
From: safety-architecture@... <safety-architecture@...>
On Behalf Of Elana C
Sent: Wednesday, March 10, 2021 12:01 PM To: Robert Krutsch <krutsch@...>; safety-architecture@... Subject: Re: [ELISA Safety Architecture WG] Linux and ioctls
Thanks, Robert, for your comment. This is all fine and true. And as noted, at the end of the day, the drivers end up calling the ioctl system interface. But then my question ends up being, for qualification of the watchdog subsystem – will we need to qualify each driver and its implementation to support each specific hardware interface and each specific system call which accesses that hardware interface?
I am only concerned that this an endless saga. We can do this (as you say) "for real", and spend weeks digging in to each low level system call called by each driver. Moreover, if you check the header files for the watchdog core and dev utilities, you can readily see that the watchdog (as "simple" as it seems …) touches on many other complex subsystems – such as the file system and the memory management subsystem. Where does "for real" end? And more generically – why is this definition of "for real" in the use case of Linux (with its ~30 million lines of code …) so much more stringent than the "normal" definition of "for real" in safety qualification of non-open source commercial software? In the latter case, safety requirements are defined, implemented and validated on a much higher level. Regards Elana
From:
safety-architecture@... <safety-architecture@...>
On Behalf Of Robert Krutsch
Hi Elena,
The API of the watchdog is ioctl based in this case. Even if we would have a user-space API on top, as all complex drivers have we would still end up at the same ioctl as the interface between user space and kernel-space.
What can be done is to say: "our watchdog is to be considered as a module out-of-tree and we assume someone else provides us with a watchdog module that fulfils the FuSa needs". From our req. we can deduce requirements for the watchdog module and stop there.
This is a simplification of scope and would take out some parts of the analysis for this App but it would not solve the issues of someone that solves the problem for real. They would need to analyze the watchdog module and deal with the ioctl or rewrite this module altogether as a user-space driver. If that sounds familiar , it is basically what you have to do when you port to QNX...
BR, Robert
On Wed, Mar 10, 2021 at 9:29 AM Elana C <elana.copperman@...> wrote:
--------------------------------------------------------------------- This e-mail and any attachments may contain confidential material for
|
|
|
|
Christopher Temple
Elena, Robert, this is a good discussion and getting to the details of the challenge – thanks!
Your statements are in line with my thinking around effort explosion.
To prevent this it makes sense to keep the safety argumentation away from the deep plumbing wherever possible in practical systems. As presented in Lyon eGas, for example, resorts to an external watchdog.
We had also discussed the benefits of an external watchdog in the context of the tell-tale use case. At the time there were voices advocating for doing it with a watchdog via Linux to understand where the challenges are as part of the investigation.
(Since the watchdog subsystem is part of the safety mechanism it would need more than qualification.)
As part of the investigation this is helpful insight that the project is obtaining.
I’d be interested in Lukas’s opinion.
Best regards Chris
From: safety-architecture@... <safety-architecture@...>
On Behalf Of Elana C via lists.elisa.tech
Sent: Mittwoch, 10. März 2021 12:01 To: Robert Krutsch <krutsch@...>; safety-architecture@... Subject: Re: [ELISA Safety Architecture WG] Linux and ioctls
Thanks, Robert, for your comment. This is all fine and true. And as noted, at the end of the day, the drivers end up calling the ioctl system interface. But then my question ends up being, for qualification of the watchdog subsystem – will we need to qualify each driver and its implementation to support each specific hardware interface and each specific system call which accesses that hardware interface?
I am only concerned that this an endless saga. We can do this (as you say) "for real", and spend weeks digging in to each low level system call called by each driver. Moreover, if you check the header files for the watchdog core and dev utilities, you can readily see that the watchdog (as "simple" as it seems …) touches on many other complex subsystems – such as the file system and the memory management subsystem. Where does "for real" end? And more generically – why is this definition of "for real" in the use case of Linux (with its ~30 million lines of code …) so much more stringent than the "normal" definition of "for real" in safety qualification of non-open source commercial software? In the latter case, safety requirements are defined, implemented and validated on a much higher level. Regards Elana
From:
safety-architecture@... <safety-architecture@...>
On Behalf Of Robert Krutsch
Hi Elena,
The API of the watchdog is ioctl based in this case. Even if we would have a user-space API on top, as all complex drivers have we would still end up at the same ioctl as the interface between user space and kernel-space.
What can be done is to say: "our watchdog is to be considered as a module out-of-tree and we assume someone else provides us with a watchdog module that fulfils the FuSa needs". From our req. we can deduce requirements for the watchdog module and stop there.
This is a simplification of scope and would take out some parts of the analysis for this App but it would not solve the issues of someone that solves the problem for real. They would need to analyze the watchdog module and deal with the ioctl or rewrite this module altogether as a user-space driver. If that sounds familiar , it is basically what you have to do when you port to QNX...
BR, Robert
On Wed, Mar 10, 2021 at 9:29 AM Elana C <elana.copperman@...> wrote:
|
|
|
|
Elana C
Fully agreed. At this point – awareness is fundamentally important, but I doubt we can yet make any final decisions, where we draw the lines.
Thanks, Gab, for sharing the makefile – this helps to highlight the different system layers. But the makefile only buries the original questions in a high level makefile.
Each of these end up calling other subsystems, including ioctls 😊. And that's before we even touch on support for watchdogs in multitasking environments. Or specific kernel configurations which impact watchdog behavior. So we are still left with our original question – how "deep" is "real" for safety qualification? At some point, we will need to define a model of "trust" for low-level infrastructure, based on intensive testing.
Anyhow, let's leave this email thread for additional opinions and feedback. For the Development Process WG, I have been "feeding" github with some drafts, and we can continue to discuss the details in tomorrow's meeting. https://github.com/elisa-tech/wg-development-process (you should ignore the svg documents for now, I am playing around with using Inkspace as an open source drawing tool – but that is a side issue.) Regards Elana
From: Christopher Temple <Christopher.Temple@...>
Sent: Wednesday, March 10, 2021 2:05 PM To: Elana Copperman <Elana.Copperman@...>; Robert Krutsch <krutsch@...>; safety-architecture@... Subject: RE: [ELISA Safety Architecture WG] Linux and ioctls
Elena, Robert, this is a good discussion and getting to the details of the challenge – thanks!
Your statements are in line with my thinking around effort explosion.
To prevent this it makes sense to keep the safety argumentation away from the deep plumbing wherever possible in practical systems. As presented in Lyon eGas, for example, resorts to an external watchdog.
We had also discussed the benefits of an external watchdog in the context of the tell-tale use case. At the time there were voices advocating for doing it with a watchdog via Linux to understand where the challenges are as part of the investigation.
(Since the watchdog subsystem is part of the safety mechanism it would need more than qualification.)
As part of the investigation this is helpful insight that the project is obtaining.
I’d be interested in Lukas’s opinion.
Best regards Chris
From:
safety-architecture@... <safety-architecture@...>
On Behalf Of Elana C via lists.elisa.tech
Thanks, Robert, for your comment. This is all fine and true. And as noted, at the end of the day, the drivers end up calling the ioctl system interface. But then my question ends up being, for qualification of the watchdog subsystem – will we need to qualify each driver and its implementation to support each specific hardware interface and each specific system call which accesses that hardware interface?
I am only concerned that this an endless saga. We can do this (as you say) "for real", and spend weeks digging in to each low level system call called by each driver. Moreover, if you check the header files for the watchdog core and dev utilities, you can readily see that the watchdog (as "simple" as it seems …) touches on many other complex subsystems – such as the file system and the memory management subsystem. Where does "for real" end? And more generically – why is this definition of "for real" in the use case of Linux (with its ~30 million lines of code …) so much more stringent than the "normal" definition of "for real" in safety qualification of non-open source commercial software? In the latter case, safety requirements are defined, implemented and validated on a much higher level. Regards Elana
From:
safety-architecture@... <safety-architecture@...>
On Behalf Of Robert Krutsch
Hi Elena,
The API of the watchdog is ioctl based in this case. Even if we would have a user-space API on top, as all complex drivers have we would still end up at the same ioctl as the interface between user space and kernel-space.
What can be done is to say: "our watchdog is to be considered as a module out-of-tree and we assume someone else provides us with a watchdog module that fulfils the FuSa needs". From our req. we can deduce requirements for the watchdog module and stop there.
This is a simplification of scope and would take out some parts of the analysis for this App but it would not solve the issues of someone that solves the problem for real. They would need to analyze the watchdog module and deal with the ioctl or rewrite this module altogether as a user-space driver. If that sounds familiar , it is basically what you have to do when you port to QNX...
BR, Robert
On Wed, Mar 10, 2021 at 9:29 AM Elana C <elana.copperman@...> wrote:
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
|
|
|
|
On Wed, Mar 10, 2021 at 1:05 PM Christopher Temple
<Christopher.Temple@...> wrote: Yes, this is a good and interesting first step. I need to catch up on the actual application's safety goal, which faults were identified, the exact intended functional behavior of the kernel watchdog, and which faults can effectively be mitigated and which not. So: step 0: describe the functional behaviors (the application, and the kernel watchdog) step 1 (risk/fault collection): Identify the faults of both functional behaviors (e.g., brainstorm and cluster them) step 2 (dependent faults): Look at the common faults of both; it is clear that the common faults are largely overlapping given a largely common context; at least from a purely enumerative consideration of faults, without considering the actual probability each fault has. This is quite tricky, e.g., just for illustration, compare and assess the probability of a compiler introducing a non-terminating loop vs. the probability of some developer doing that. Obviously that is relevant, as you then could ask "does it matter that two different compilers were responsible for compilation of application and kernel watchdog or does it matter that two different developers that were responsible for implementation of application and kernel watchdog?"; that leads to step 3... step 3 (complexity vs. mitigation decision): Determine if the added complexity of the kernel watchdog functionality is actually worth the remaining mitigation of faults (after assessment of individual risks and common dependent risks), or if other alternatives should be considered. I will try to dig into that, and then comment in more detail. If someone from the group could quickly point me to those documents that exist for step 0 (or even step 1), I will give it a read. Lukas
|
|
|