[ELISA Automotive WG] Kernel Source Code repo for the AGL demo


Jochen Kall
 

Hi Gab,

what Sudip said 😝
I don't think that scales very well if you have many patches though.
You can also change the Repo and Branch settings in the recipe to have it get the sources from wherever you want, even works with local repos, very useful while developing.

Ps: I played a bit with this tagging idea, and one of the things realized was that you should always add a new line for each tag and have nothing else but the tag in said line, that way git won't run into merge conflicts if something else is changed on the line in question.
Not sure how well that scales with the kernel, especially if stuff is renamed/moved around, but i'm hoping tags set up that way in a "tagging" branch can be kept in sync with the main branch by merging without too much overhead.

Best regards
Jochen

-----UrsprĂźngliche Nachricht-----
Von: Sudip Mukherjee <sudip.mukherjee@...>
Gesendet: Montag, 1. März 2021 15:16
An: Paoloni, Gabriele <gabriele.paoloni@...>; Jochen Kall
<Jochen.Kall@...>; automotive@...
Cc: safety-architecture@...; Dellosa, Stefano
<stefano.dellosa@...>
Betreff: Re: [ELISA Automotive WG] Kernel Source Code repo for the AGL
demo

Hi Gab,

On 01/03/2021 1:44 pm, Paoloni, Gabriele wrote:
Hi Jochen, all



StefanoD has now pushed the initial KSR mm file here
https://github.com/elisa-
tech/Safety_Architecture_WG/tree/master/AGL_c
luster_demo_use_case
<https://github.com/elisa-
tech/Safety_Architecture_WG/tree/master/AGL_
cluster_demo_use_case>

However as next step I would like to tag the Linux Kernel ‘entrypoints’
associated to the KSRs directly from the Kernel source repo used by
the AGL demo.

Do you actually know how to retrieve the Kernel source repo and
tag/branch from the AGL repo?
Repo = git://git.yoctoproject.org/linux-yocto.git
Branch = v5.4/standard/base

Note: the branch pointed above is using v5.4.101 but AGL is still at
v5.4.85 with SHA at 4f2b484a791fac88262922aa26ddd5ac3df9720f




Also for the future if we wanted to modify the Kernel, rebuild and use
an updated Kernel image how can we do that in meta-elisa?
Just adding a bbappend file with the patch will be enough.



--
Regards
Sudip


Paoloni, Gabriele <gabriele.paoloni@...>
 

Hi Jochen, Sudip

-----Original Message-----
From: safety-architecture@... <safety-
architecture@...> On Behalf Of Jochen Kall
Sent: Monday, March 1, 2021 3:56 PM
To: Sudip Mukherjee <sudip.mukherjee@...>; Paoloni,
Gabriele <gabriele.paoloni@...>; automotive@...
Cc: safety-architecture@...; Dellosa, Stefano
<stefano.dellosa@...>
Subject: Re: [ELISA Safety Architecture WG] [ELISA Automotive WG] Kernel
Source Code repo for the AGL demo

Hi Gab,

what Sudip said 😝
I don't think that scales very well if you have many patches though.
You can also change the Repo and Branch settings in the recipe to have it get
the sources from wherever you want, even works with local repos, very
useful while developing.
It sounds like a better approach to me


Ps: I played a bit with this tagging idea, and one of the things realized was
that you should always add a new line for each tag and have nothing else but
the tag in said line, that way git won't run into merge conflicts if something
else is changed on the line in question.
Not sure how well that scales with the kernel, especially if stuff is
renamed/moved around, but i'm hoping tags set up that way in a "tagging"
branch can be kept in sync with the main branch by merging without too
much overhead.
Mmmm I noticed that there is also another feature as in "add hyperlink"; so
I am wondering if an alternative could be to associate an hyperlink to the
Requirement. E.g.
https://github.com/torvalds/linux/blob/f40ddce88593482919761f74910f42f4b84c004b/init/main.c#L849
In the link above I am pointing a specific line of a specific file of a specific TAG.

What do you think?

Thanks
Gab


Best regards
Jochen

-----UrsprĂźngliche Nachricht-----
Von: Sudip Mukherjee <sudip.mukherjee@...>
Gesendet: Montag, 1. März 2021 15:16
An: Paoloni, Gabriele <gabriele.paoloni@...>; Jochen Kall
<Jochen.Kall@...>; automotive@...
Cc: safety-architecture@...; Dellosa, Stefano
<stefano.dellosa@...>
Betreff: Re: [ELISA Automotive WG] Kernel Source Code repo for the AGL
demo

Hi Gab,

On 01/03/2021 1:44 pm, Paoloni, Gabriele wrote:
Hi Jochen, all



StefanoD has now pushed the initial KSR mm file here
https://github.com/elisa-
tech/Safety_Architecture_WG/tree/master/AGL_c
luster_demo_use_case
<https://github.com/elisa-
tech/Safety_Architecture_WG/tree/master/AGL_
cluster_demo_use_case>

However as next step I would like to tag the Linux Kernel ‘entrypoints’
associated to the KSRs directly from the Kernel source repo used by
the AGL demo.

Do you actually know how to retrieve the Kernel source repo and
tag/branch from the AGL repo?
Repo = git://git.yoctoproject.org/linux-yocto.git
Branch = v5.4/standard/base

Note: the branch pointed above is using v5.4.101 but AGL is still at
v5.4.85 with SHA at 4f2b484a791fac88262922aa26ddd5ac3df9720f




Also for the future if we wanted to modify the Kernel, rebuild and use
an updated Kernel image how can we do that in meta-elisa?
Just adding a bbappend file with the patch will be enough.



--
Regards
Sudip


---------------------------------------------------------------------
INTEL CORPORATION ITALIA S.p.A. con unico socio
Sede: Milanofiori Palazzo E 4
CAP 20094 Assago (MI)
Capitale Sociale Euro 104.000,00 interamente versato
Partita I.V.A. e Codice Fiscale 04236760155
Repertorio Economico Amministrativo n. 997124
Registro delle Imprese di Milano nr. 183983/5281/33
Soggetta ad attivita' di direzione e coordinamento di
INTEL CORPORATION, USA

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.


Paoloni, Gabriele <gabriele.paoloni@...>
 

Hi Sudip

-----Original Message-----
From: Sudip Mukherjee <sudip.mukherjee@...>
Sent: Monday, March 1, 2021 6:26 PM
To: Paoloni, Gabriele <gabriele.paoloni@...>; Jochen Kall
<jochen.kall@...>; automotive@...
Cc: safety-architecture@...; Dellosa, Stefano
<stefano.dellosa@...>
Subject: Re: [ELISA Safety Architecture WG] [ELISA Automotive WG] Kernel
Source Code repo for the AGL demo



On 01/03/2021 4:12 pm, Paoloni, Gabriele wrote:
Hi Jochen, Sudip

-----Original Message-----
From: safety-architecture@... <safety-
architecture@...> On Behalf Of Jochen Kall
Sent: Monday, March 1, 2021 3:56 PM
To: Sudip Mukherjee <sudip.mukherjee@...>; Paoloni,
Gabriele <gabriele.paoloni@...>; automotive@...
Cc: safety-architecture@...; Dellosa, Stefano
<stefano.dellosa@...>
Subject: Re: [ELISA Safety Architecture WG] [ELISA Automotive WG]
Kernel
Source Code repo for the AGL demo

Hi Gab,

what Sudip said 😝
I don't think that scales very well if you have many patches though.
You can also change the Repo and Branch settings in the recipe to have it
get
the sources from wherever you want, even works with local repos, very
useful while developing.
It sounds like a better approach to me
But by doing so you are now bound to one specific version of the kernel
and then Elisa demo will not have the updated kernel like AGL or Yocto
will have. Unless, someone volunteers to maintain the repo and update
the branch as and when LTS kernel is released or Yocto updates their branch.
We actually do not want a moving kernel for our ELISA qualification (for fun)
activities.
We already discussed this in the Automotive WG some time ago and we said
that the LST used in Yocto should stick for a while with just few security and
bugfix updates.
I think we could eventually rebase time by time




Ps: I played a bit with this tagging idea, and one of the things realized was
that you should always add a new line for each tag and have nothing else
but
the tag in said line, that way git won't run into merge conflicts if something
else is changed on the line in question.
Not sure how well that scales with the kernel, especially if stuff is
renamed/moved around, but i'm hoping tags set up that way in a
"tagging"
branch can be kept in sync with the main branch by merging without too
much overhead.
Mmmm I noticed that there is also another feature as in "add hyperlink"; so
I am wondering if an alternative could be to associate an hyperlink to the
Requirement. E.g.
https://github.com/torvalds/linux/blob/f40ddce88593482919761f74910f42f4
b84c004b/init/main.c#L849

Please use:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/init/
main.c?id=f40ddce88593482919761f74910f42f4b84c004b#n849

github repo is not the official repo of the kernel.
Sure, it was just an example to propose the hyperlink in place of tagging inside
the code (that would imply spoiling it)

Thanks
Gab




--
Regards
Sudip
---------------------------------------------------------------------
INTEL CORPORATION ITALIA S.p.A. con unico socio
Sede: Milanofiori Palazzo E 4
CAP 20094 Assago (MI)
Capitale Sociale Euro 104.000,00 interamente versato
Partita I.V.A. e Codice Fiscale 04236760155
Repertorio Economico Amministrativo n. 997124
Registro delle Imprese di Milano nr. 183983/5281/33
Soggetta ad attivita' di direzione e coordinamento di
INTEL CORPORATION, USA

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.


Jochen Kall
 

Hi there,

No forking, that's exactly what we said we do not want to do, since that way we cut ourselves off from fixes, we did agree though to stick with the long term stable version Yocto is currently on.
Sudip is right, of course I did not mean to permanently make the recipe pull from a fork, just temporarily while you are working on something it's much more convenient than adding patches over and over, or maybe that's just me.
Shouldn't have brought it up, there is no realistic alternative to patching via .bpappend, but during development of a patch I'd definitively reroute to a fork.

Jochen

-----UrsprĂźngliche Nachricht-----
Von: Sudip Mukherjee <sudip.mukherjee@...>
Gesendet: Montag, 1. März 2021 19:51
An: Paoloni, Gabriele <gabriele.paoloni@...>; Jochen Kall
<Jochen.Kall@...>; automotive@...
Cc: safety-architecture@...; Dellosa, Stefano
<stefano.dellosa@...>
Betreff: Re: [ELISA Safety Architecture WG] [ELISA Automotive WG] Kernel
Source Code repo for the AGL demo



On 01/03/2021 5:32 pm, Paoloni, Gabriele wrote:
Hi Sudip
<snip>

Source Code repo for the AGL demo

Hi Gab,

what Sudip said 😝
I don't think that scales very well if you have many patches though.
You can also change the Repo and Branch settings in the recipe to
have it
get
the sources from wherever you want, even works with local repos,
very useful while developing.
It sounds like a better approach to me
But by doing so you are now bound to one specific version of the
kernel and then Elisa demo will not have the updated kernel like AGL
or Yocto will have. Unless, someone volunteers to maintain the repo
and update the branch as and when LTS kernel is released or Yocto
updates their branch.

We actually do not want a moving kernel for our ELISA qualification
(for fun) activities.
We already discussed this in the Automotive WG some time ago and we
said that the LST used in Yocto should stick for a while with just few
security and bugfix updates.
I think we could eventually rebase time by time
I think you have a wrong conception about fun. Anyways, I might have
missed that discussion. But, I am entirely failing to understand how will you
get the security and bug fixes if you keep your version fixed and do not
update to latest v5.4.y. If you fork the kernel branch now, Elisa demo will be
fixed at v5.4.85 but AGL will move to v5.4.101 and beyond as soon as they
update their manifest. So, AGL will get all the security and bug fixes but Elisa
demo will not.



--
Regards
Sudip


Jochen Kall
 

Hi Gab,

my thoughts about the tagging vs linking:

Ps: I played a bit with this tagging idea, and one of the things
realized was that you should always add a new line for each tag and
have nothing else but the tag in said line, that way git won't run
into merge conflicts if something else is changed on the line in question.
Not sure how well that scales with the kernel, especially if stuff is
renamed/moved around, but i'm hoping tags set up that way in a "tagging"
branch can be kept in sync with the main branch by merging without too
much overhead.
Mmmm I noticed that there is also another feature as in "add hyperlink"; so I
am wondering if an alternative could be to associate an hyperlink to the
Requirement. E.g.
https://github.com/torvalds/linux/blob/f40ddce88593482919761f74910f42f4
b84c004b/init/main.c#L849
In the link above I am pointing a specific line of a specific file of a specific TAG.

What do you think?
[Jochen Kall]
These kinds of links the safetyaddon already creates based on the requirement tags in the code of files under monitoring.
Creating them directly by hand will make it nasty to move it to new revisions, if the line that is tagged moves due to code changes.
I did a bit of experimenting with making a new branch for adding tags, then committing changes to the main branch and merging main into the tag branch to sync, and that worked quite well as long as tags got their own line, so git takes care oft he donkey work.
Of course one still has to check what git suggested and once a file is renamed or moved, manuall work is inevitable, but at least git does most of the heavy lifting that way.
With the manual links you'd have to check and change every single one of them every time the revision changes, not fun i reckon

Thanks
Gab
Best regards,
Jochen


Paoloni, Gabriele <gabriele.paoloni@...>
 

Hi guys

Sorry I didn't make myself clear. I meant that since I do not expect too many changes/fixes on a stable LTS release
I would expect the safety analyses to be 'mostly' valid as the LST evolve; so if there are exceptions we may consider
to 'rebase' the analyses as in fixing the parts that would not be valid anymore...
Maybe we can discuss this today...

BTW I append 'for fun' anytime I need to use a word that commonly implies a degree of safeness that we cannot and
do not want to claim so far (because we are still to early in exploration)

Thanks
Gab

-----Original Message-----
From: Sudip Mukherjee <sudip.mukherjee@...>
Sent: Tuesday, March 2, 2021 12:09 PM
To: Jochen Kall <jochen.kall@...>; Paoloni, Gabriele
<gabriele.paoloni@...>; automotive@...
Cc: safety-architecture@...; Dellosa, Stefano
<stefano.dellosa@...>
Subject: Re: [ELISA Safety Architecture WG] [ELISA Automotive WG] Kernel
Source Code repo for the AGL demo

Hi Jochen,

On 02/03/2021 10:07 am, Jochen Kall wrote:
Hi there,

No forking, that's exactly what we said we do not want to do, since that
way we cut ourselves off from fixes, we did agree though to stick with the
long term stable version Yocto is currently on.
Sudip is right, of course I did not mean to permanently make the recipe pull
from a fork, just temporarily while you are working on something it's much
more convenient than adding patches over and over, or maybe that's just
me.

No, not just you. :)
During development thats the correct way else the developer will spend
more time rebasing than actual development work. But after the
development is over and is ready to be merged to elisa repo then it
should be via bbappend and patches on top of the kernel that is used by
AGL.



--
Regards
Sudip
---------------------------------------------------------------------
INTEL CORPORATION ITALIA S.p.A. con unico socio
Sede: Milanofiori Palazzo E 4
CAP 20094 Assago (MI)
Capitale Sociale Euro 104.000,00 interamente versato
Partita I.V.A. e Codice Fiscale 04236760155
Repertorio Economico Amministrativo n. 997124
Registro delle Imprese di Milano nr. 183983/5281/33
Soggetta ad attivita' di direzione e coordinamento di
INTEL CORPORATION, USA

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.


Paoloni, Gabriele <gabriele.paoloni@...>
 

Hi Jochen

-----Original Message-----
From: automotive@... <automotive@...> On Behalf
Of Jochen Kall
Sent: Tuesday, March 2, 2021 11:18 AM
To: Paoloni, Gabriele <gabriele.paoloni@...>; Sudip Mukherjee
<sudip.mukherjee@...>; automotive@...
Cc: safety-architecture@...; Dellosa, Stefano
<stefano.dellosa@...>
Subject: Re: [ELISA Safety Architecture WG] [ELISA Automotive WG] Kernel
Source Code repo for the AGL demo

Hi Gab,

my thoughts about the tagging vs linking:

Ps: I played a bit with this tagging idea, and one of the things
realized was that you should always add a new line for each tag and
have nothing else but the tag in said line, that way git won't run
into merge conflicts if something else is changed on the line in question.
Not sure how well that scales with the kernel, especially if stuff is
renamed/moved around, but i'm hoping tags set up that way in a
"tagging"
branch can be kept in sync with the main branch by merging without too
much overhead.
Mmmm I noticed that there is also another feature as in "add hyperlink"; so
I
am wondering if an alternative could be to associate an hyperlink to the
Requirement. E.g.
https://github.com/torvalds/linux/blob/f40ddce88593482919761f74910f42f4
b84c004b/init/main.c#L849
In the link above I am pointing a specific line of a specific file of a specific
TAG.

What do you think?
[Jochen Kall]
These kinds of links the safetyaddon already creates based on the
requirement tags in the code of files under monitoring.
Creating them directly by hand will make it nasty to move it to new revisions,
if the line that is tagged moves due to code changes.
I did a bit of experimenting with making a new branch for adding tags, then
committing changes to the main branch and merging main into the tag branch
to sync, and that worked quite well as long as tags got their own line, so git
takes care oft he donkey work.
Of course one still has to check what git suggested and once a file is renamed
or moved, manuall work is inevitable, but at least git does most of the heavy
lifting that way.
With the manual links you'd have to check and change every single one of
them every time the revision changes, not fun i reckon
Right; in practice if I wanted to tag, for example, start_kernel() I could set:
https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/init/main.c?h=linux-5.4.y#n577
This link is going to be valid as long as this file is not touched by any commit.
Now looking at the commit log for main.c in the LTS I see that it's been touched
4 times along 2020 but quite a bit more along 2019
https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/log/init/main.c?h=linux-5.4.y
On the flip side tagging directly in the code would imply patching such LTS...where though?
We need to create a dedicated branch that we regularly rebase on top of the LTS; from
what you say we should think of the tag patches as part of the safety/requirements analysis
and rebasing them would be part of the safety analysis maintenance as opposed to updating
the hyperlink when needed....let's discuss about it today....I honestly dunno what is the best
option...

Thanks
Gab


Thanks
Gab
Best regards,
Jochen




---------------------------------------------------------------------
INTEL CORPORATION ITALIA S.p.A. con unico socio
Sede: Milanofiori Palazzo E 4
CAP 20094 Assago (MI)
Capitale Sociale Euro 104.000,00 interamente versato
Partita I.V.A. e Codice Fiscale 04236760155
Repertorio Economico Amministrativo n. 997124
Registro delle Imprese di Milano nr. 183983/5281/33
Soggetta ad attivita' di direzione e coordinamento di
INTEL CORPORATION, USA

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.


John MacGregor
 

Hi,

I got to thinking yesterday after my somewhat impromptu remarks.

What is the use case here?

As far as I can tell, Gab wants to be able to jump automatically from the Freeplane requirements tool to... the signature of the corresponding function in the source code.

It's not so clear to me if that's the source code in the Linux kernel tree or some other public place, the source code in the Automotive WG's repo or the source code in the local clone of either of the above...

Jochen's solution would instrument the local code, but it isn't clear (at least to me) how the tags can be found automatically, let alone in Freeplane.

For the "jumping out of Freeplane" part the two solutions I can think of are a) indeed using a hyperlink (although that could point to a file on the local file system) or b) maybe calling a shell script which calls an editor with some sort of reference to the signature. (b) seems to be possible, but Freeplane's documentation is a bit unilluminating.

With the hyperlink solution, we seem to be stuck with the problem of specifying the line number, although it might be possible to embed a bookmark in the html (source: <span id="xyz"/> link: http://url#xyz other sources <a href="http:url#xyz"> void xyz (void) </a> ).

With the shell script solution, there's the question of which editor(s) to support and how to get from the command line invocation to the code with the signature. I haven't found anything pretty yet.

For using public sources, I came across the bootlin exlixir cross/reference, where Gab's url is also specified with a line number [1], but at least there is a cross-reference for the signature [2] as well as a search facility.

I have the vague recollection that there were other public cross-reference sites, but I didn't find anything after a cursory search. Bootlin's site seems quite broad and up-to-date.

It's also possible to brew your own cross-reference with LXR[3], although it seems rather old.

Gnu published a list of resources[4] for cross-referencing, where the approach here would be to offer something that worked on the local code installation. There is a list of tools supporting ctags[4]. I also came across this interesting tutorial[5], which notes that the kernel has architecture-dependent code. This might cause ambiguous references (as we can see in [2]).

Any opinions, or does that just muddy the waters more?

Cheers

John


[1] https://elixir.bootlin.com/linux/latest/source/init/main.c#L849
[2] https://elixir.bootlin.com/linux/latest/A/ident/start_kernel
[3] linux cross-reference
[4] http://ctags.sourceforge.net/tools.html

On 02/03/2021 13:24, Paoloni, Gabriele wrote:
Hi Jochen

-----Original Message-----
From: automotive@... <automotive@...> On Behalf
Of Jochen Kall
Sent: Tuesday, March 2, 2021 11:18 AM
To: Paoloni, Gabriele <gabriele.paoloni@...>; Sudip Mukherjee
<sudip.mukherjee@...>; automotive@...
Cc: safety-architecture@...; Dellosa, Stefano
<stefano.dellosa@...>
Subject: Re: [ELISA Safety Architecture WG] [ELISA Automotive WG] Kernel
Source Code repo for the AGL demo

Hi Gab,

my thoughts about the tagging vs linking:

Ps: I played a bit with this tagging idea, and one of the things
realized was that you should always add a new line for each tag and
have nothing else but the tag in said line, that way git won't run
into merge conflicts if something else is changed on the line in question.
Not sure how well that scales with the kernel, especially if stuff is
renamed/moved around, but i'm hoping tags set up that way in a
"tagging"
branch can be kept in sync with the main branch by merging without too
much overhead.
Mmmm I noticed that there is also another feature as in "add hyperlink"; so
I
am wondering if an alternative could be to associate an hyperlink to the
Requirement. E.g.
https://github.com/torvalds/linux/blob/f40ddce88593482919761f74910f42f4
b84c004b/init/main.c#L849
In the link above I am pointing a specific line of a specific file of a specific
TAG.

What do you think?
[Jochen Kall]
These kinds of links the safetyaddon already creates based on the
requirement tags in the code of files under monitoring.
Creating them directly by hand will make it nasty to move it to new revisions,
if the line that is tagged moves due to code changes.
I did a bit of experimenting with making a new branch for adding tags, then
committing changes to the main branch and merging main into the tag branch
to sync, and that worked quite well as long as tags got their own line, so git
takes care oft he donkey work.
Of course one still has to check what git suggested and once a file is renamed
or moved, manuall work is inevitable, but at least git does most of the heavy
lifting that way.
With the manual links you'd have to check and change every single one of
them every time the revision changes, not fun i reckon
Right; in practice if I wanted to tag, for example, start_kernel() I could set:
https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/init/main.c?h=linux-5.4.y#n577
This link is going to be valid as long as this file is not touched by any commit.
Now looking at the commit log for main.c in the LTS I see that it's been touched
4 times along 2020 but quite a bit more along 2019
https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/log/init/main.c?h=linux-5.4.y
On the flip side tagging directly in the code would imply patching such LTS...where though?
We need to create a dedicated branch that we regularly rebase on top of the LTS; from
what you say we should think of the tag patches as part of the safety/requirements analysis
and rebasing them would be part of the safety analysis maintenance as opposed to updating
the hyperlink when needed....let's discuss about it today....I honestly dunno what is the best
option...
Thanks
Gab


Thanks
Gab
Best regards,
Jochen




---------------------------------------------------------------------
INTEL CORPORATION ITALIA S.p.A. con unico socio
Sede: Milanofiori Palazzo E 4
CAP 20094 Assago (MI)
Capitale Sociale Euro 104.000,00 interamente versato
Partita I.V.A. e Codice Fiscale 04236760155
Repertorio Economico Amministrativo n. 997124
Registro delle Imprese di Milano nr. 183983/5281/33
Soggetta ad attivita' di direzione e coordinamento di
INTEL CORPORATION, USA
This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.


Paoloni, Gabriele <gabriele.paoloni@...>
 

Hi John

Sorry for all this delay, it's been a crazy week

-----Original Message-----
From: safety-architecture@... <safety-
architecture@...> On Behalf Of John MacGregor
Sent: Wednesday, March 3, 2021 1:01 PM
To: Paoloni, Gabriele <gabriele.paoloni@...>; Jochen Kall
<jochen.kall@...>; Sudip Mukherjee
<sudip.mukherjee@...>; automotive@...
Cc: safety-architecture@...; Dellosa, Stefano
<stefano.dellosa@...>
Subject: Re: [ELISA Safety Architecture WG] [ELISA Automotive WG] Kernel
Source Code repo for the AGL demo

Hi,

I got to thinking yesterday after my somewhat impromptu remarks.

What is the use case here?

As far as I can tell, Gab wants to be able to jump automatically from
the Freeplane requirements tool to... the signature of the corresponding
function in the source code.

It's not so clear to me if that's the source code in the Linux kernel
tree or some other public place, the source code in the Automotive WG's
repo or the source code in the local clone of either of the above...

Jochen's solution would instrument the local code, but it isn't clear
(at least to me) how the tags can be found automatically, let alone in
Freeplane.
I think Jochen already presented how freeplane can link to the code tags.
@Jochen maybe you can chime in with more details...
Also you may have seen my other email where I tried to manually add
tags and simulate a re-base without much success...


For the "jumping out of Freeplane" part the two solutions I can think of
are a) indeed using a hyperlink (although that could point to a file on
the local file system) or b) maybe calling a shell script which calls an
editor with some sort of reference to the signature. (b) seems to be
possible, but Freeplane's documentation is a bit unilluminating.

With the hyperlink solution, we seem to be stuck with the problem of
specifying the line number, although it might be possible to embed a
bookmark in the html (source: <span id="xyz"/> link: http://url#xyz
other sources <a href="http:url#xyz"> void xyz (void) </a> ).
Yes, actually we could use a hyperlink to a specific line of the latest
version of a stable branch. As I already pointed out this example would
point to the line associated to start_kernel() as long as start_kernel()
definition stays there:
https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/init/main.c?h=linux-5.4.y#n577


With the shell script solution, there's the question of which editor(s)
to support and how to get from the command line invocation to the code
with the signature. I haven't found anything pretty yet.
Yes that was an option. @Robert did you have time to spend on this point?
Anything back?


For using public sources, I came across the bootlin exlixir
cross/reference, where Gab's url is also specified with a line number
[1], but at least there is a cross-reference for the signature [2] as
well as a search facility.

I have the vague recollection that there were other public
cross-reference sites, but I didn't find anything after a cursory
search. Bootlin's site seems quite broad and up-to-date.

It's also possible to brew your own cross-reference with LXR[3],
although it seems rather old.
Mmm problem is that we should only use official Kernel GIT repos...


Gnu published a list of resources[4] for cross-referencing, where the
approach here would be to offer something that worked on the local code
installation. There is a list of tools supporting ctags[4]. I also
came across this interesting tutorial[5], which notes that the kernel
has architecture-dependent code. This might cause ambiguous references
(as we can see in [2]).

Any opinions, or does that just muddy the waters more?
Mmm yes there are arch dependent implementations of functions, maybe
it can be sorted by filtering out archs that we do not target (I am thinking
to something similar to the Eclipse indexing process...)

Let's discuss this topic again tomorrow to see where we are...

Thanks
Gab


Cheers

John


[1] https://elixir.bootlin.com/linux/latest/source/init/main.c#L849
[2] https://elixir.bootlin.com/linux/latest/A/ident/start_kernel
[3] linux cross-reference
[4] http://ctags.sourceforge.net/tools.html

On 02/03/2021 13:24, Paoloni, Gabriele wrote:
Hi Jochen

-----Original Message-----
From: automotive@... <automotive@...> On
Behalf
Of Jochen Kall
Sent: Tuesday, March 2, 2021 11:18 AM
To: Paoloni, Gabriele <gabriele.paoloni@...>; Sudip Mukherjee
<sudip.mukherjee@...>; automotive@...
Cc: safety-architecture@...; Dellosa, Stefano
<stefano.dellosa@...>
Subject: Re: [ELISA Safety Architecture WG] [ELISA Automotive WG]
Kernel
Source Code repo for the AGL demo

Hi Gab,

my thoughts about the tagging vs linking:

Ps: I played a bit with this tagging idea, and one of the things
realized was that you should always add a new line for each tag and
have nothing else but the tag in said line, that way git won't run
into merge conflicts if something else is changed on the line in
question.
Not sure how well that scales with the kernel, especially if stuff is
renamed/moved around, but i'm hoping tags set up that way in a
"tagging"
branch can be kept in sync with the main branch by merging without
too
much overhead.
Mmmm I noticed that there is also another feature as in "add hyperlink";
so
I
am wondering if an alternative could be to associate an hyperlink to the
Requirement. E.g.
https://github.com/torvalds/linux/blob/f40ddce88593482919761f74910f42f4
b84c004b/init/main.c#L849
In the link above I am pointing a specific line of a specific file of a specific
TAG.

What do you think?
[Jochen Kall]
These kinds of links the safetyaddon already creates based on the
requirement tags in the code of files under monitoring.
Creating them directly by hand will make it nasty to move it to new
revisions,
if the line that is tagged moves due to code changes.
I did a bit of experimenting with making a new branch for adding tags,
then
committing changes to the main branch and merging main into the tag
branch
to sync, and that worked quite well as long as tags got their own line, so
git
takes care oft he donkey work.
Of course one still has to check what git suggested and once a file is
renamed
or moved, manuall work is inevitable, but at least git does most of the
heavy
lifting that way.
With the manual links you'd have to check and change every single one of
them every time the revision changes, not fun i reckon
Right; in practice if I wanted to tag, for example, start_kernel() I could set:
https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/init/ma
in.c?h=linux-5.4.y#n577
This link is going to be valid as long as this file is not touched by any commit.
Now looking at the commit log for main.c in the LTS I see that it's been
touched
4 times along 2020 but quite a bit more along 2019
https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/log/init/mai
n.c?h=linux-5.4.y
On the flip side tagging directly in the code would imply patching such
LTS...where though?
We need to create a dedicated branch that we regularly rebase on top of
the LTS; from
what you say we should think of the tag patches as part of the
safety/requirements analysis
and rebasing them would be part of the safety analysis maintenance as
opposed to updating
the hyperlink when needed....let's discuss about it today....I honestly
dunno what is the best
option...

Thanks
Gab


Thanks
Gab
Best regards,
Jochen




---------------------------------------------------------------------
INTEL CORPORATION ITALIA S.p.A. con unico socio
Sede: Milanofiori Palazzo E 4
CAP 20094 Assago (MI)
Capitale Sociale Euro 104.000,00 interamente versato
Partita I.V.A. e Codice Fiscale 04236760155
Repertorio Economico Amministrativo n. 997124
Registro delle Imprese di Milano nr. 183983/5281/33
Soggetta ad attivita' di direzione e coordinamento di
INTEL CORPORATION, USA

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.






---------------------------------------------------------------------
INTEL CORPORATION ITALIA S.p.A. con unico socio
Sede: Milanofiori Palazzo E 4
CAP 20094 Assago (MI)
Capitale Sociale Euro 104.000,00 interamente versato
Partita I.V.A. e Codice Fiscale 04236760155
Repertorio Economico Amministrativo n. 997124
Registro delle Imprese di Milano nr. 183983/5281/33
Soggetta ad attivita' di direzione e coordinamento di
INTEL CORPORATION, USA

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.


Jochen Kall
 

Hi Gab, John,

-----UrsprĂźngliche Nachricht-----
Von: automotive@... <automotive@...> Im Auftrag
von Paoloni, Gabriele
Gesendet: Montag, 8. März 2021 09:24
An: John MacGregor <open.john.macgregor@...>; Jochen Kall
<Jochen.Kall@...>; Sudip Mukherjee
<sudip.mukherjee@...>; automotive@...; Robert
Krutsch <krutsch@...>
Cc: safety-architecture@...; Dellosa, Stefano
<stefano.dellosa@...>
Betreff: Re: [ELISA Safety Architecture WG] [ELISA Automotive WG] Kernel
Source Code repo for the AGL demo

Hi John

Sorry for all this delay, it's been a crazy week

-----Original Message-----
From: safety-architecture@... <safety-
architecture@...> On Behalf Of John MacGregor
Sent: Wednesday, March 3, 2021 1:01 PM
To: Paoloni, Gabriele <gabriele.paoloni@...>; Jochen Kall
<jochen.kall@...>; Sudip Mukherjee
<sudip.mukherjee@...>; automotive@...
Cc: safety-architecture@...; Dellosa, Stefano
<stefano.dellosa@...>
Subject: Re: [ELISA Safety Architecture WG] [ELISA Automotive WG]
Kernel Source Code repo for the AGL demo

Hi,

I got to thinking yesterday after my somewhat impromptu remarks.

What is the use case here?

As far as I can tell, Gab wants to be able to jump automatically from
the Freeplane requirements tool to... the signature of the
corresponding function in the source code.

It's not so clear to me if that's the source code in the Linux kernel
tree or some other public place, the source code in the Automotive
WG's repo or the source code in the local clone of either of the above...

Jochen's solution would instrument the local code, but it isn't clear
(at least to me) how the tags can be found automatically, let alone in
Freeplane.
I think Jochen already presented how freeplane can link to the code tags.
@Jochen maybe you can chime in with more details...
Also you may have seen my other email where I tried to manually add tags
and simulate a re-base without much success...
[Jochen Kall] Yep, I demonstrated that a couple weeks ago, it's just a script integrated in the addon looking for Requirements IDs in source code files and creating github links accordingly.
Source code is here, nothing fancy, but pretty nice to get a direct link to the location where a tag is mentioned in the code.
https://github.com/Jochen-Kall/Safety_concept_tool/blob/main/FuSi-addon/scripts/Extract_Tag_mapping.groovy

For the "jumping out of Freeplane" part the two solutions I can think
of are a) indeed using a hyperlink (although that could point to a
file on the local file system) or b) maybe calling a shell script
which calls an editor with some sort of reference to the signature.
(b) seems to be possible, but Freeplane's documentation is a bit
unilluminating.
[Jochen Kall]
Both work fine actually.
Nodes can have a Link directly, which can link to another node, a file or a website, it shows up as a red arrow in the node, downside, only one of those per node.
When you click such a link your OS should open the file with the application registered fort he file type (Works for Linux and Windows alike on my systems at least)
Using the html format as I did in the addon lets you add arbitrary many links, it's not fun to do so manually though.


With the hyperlink solution, we seem to be stuck with the problem of
specifying the line number, although it might be possible to embed a
bookmark in the html (source: <span id="xyz"/> link: http://url#xyz
other sources <a href="http:url#xyz"> void xyz (void) </a> ).
Yes, actually we could use a hyperlink to a specific line of the latest version of
a stable branch. As I already pointed out this example would point to the line
associated to start_kernel() as long as start_kernel() definition stays there:
https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/init/ma
in.c?h=linux-5.4.y#n577


With the shell script solution, there's the question of which
editor(s) to support and how to get from the command line invocation
to the code with the signature. I haven't found anything pretty yet.
Yes that was an option. @Robert did you have time to spend on this point?
Anything back?


For using public sources, I came across the bootlin exlixir
cross/reference, where Gab's url is also specified with a line number
[1], but at least there is a cross-reference for the signature [2] as
well as a search facility.

I have the vague recollection that there were other public
cross-reference sites, but I didn't find anything after a cursory
search. Bootlin's site seems quite broad and up-to-date.

It's also possible to brew your own cross-reference with LXR[3],
although it seems rather old.
Mmm problem is that we should only use official Kernel GIT repos...


Gnu published a list of resources[4] for cross-referencing, where the
approach here would be to offer something that worked on the local
code installation. There is a list of tools supporting ctags[4]. I
also came across this interesting tutorial[5], which notes that the
kernel has architecture-dependent code. This might cause ambiguous
references (as we can see in [2]).

Any opinions, or does that just muddy the waters more?
Mmm yes there are arch dependent implementations of functions, maybe it
can be sorted by filtering out archs that we do not target (I am thinking to
something similar to the Eclipse indexing process...)

Let's discuss this topic again tomorrow to see where we are...

Thanks
Gab


Cheers

John


[1] https://elixir.bootlin.com/linux/latest/source/init/main.c#L849
[2] https://elixir.bootlin.com/linux/latest/A/ident/start_kernel
[3] linux cross-reference
[4] http://ctags.sourceforge.net/tools.html

On 02/03/2021 13:24, Paoloni, Gabriele wrote:
Hi Jochen

-----Original Message-----
From: automotive@... <automotive@...> On
Behalf
Of Jochen Kall
Sent: Tuesday, March 2, 2021 11:18 AM
To: Paoloni, Gabriele <gabriele.paoloni@...>; Sudip Mukherjee
<sudip.mukherjee@...>; automotive@...
Cc: safety-architecture@...; Dellosa, Stefano
<stefano.dellosa@...>
Subject: Re: [ELISA Safety Architecture WG] [ELISA Automotive WG]
Kernel
Source Code repo for the AGL demo

Hi Gab,

my thoughts about the tagging vs linking:

Ps: I played a bit with this tagging idea, and one of the things
realized was that you should always add a new line for each tag
and have nothing else but the tag in said line, that way git
won't run into merge conflicts if something else is changed on
the line in
question.
Not sure how well that scales with the kernel, especially if
stuff is renamed/moved around, but i'm hoping tags set up that
way in a
"tagging"
branch can be kept in sync with the main branch by merging
without
too
much overhead.
Mmmm I noticed that there is also another feature as in "add
hyperlink";
so
I
am wondering if an alternative could be to associate an hyperlink
to the Requirement. E.g.
https://github.com/torvalds/linux/blob/f40ddce88593482919761f74910f42f
4
b84c004b/init/main.c#L849
In the link above I am pointing a specific line of a specific file
of a specific
TAG.

What do you think?
[Jochen Kall]
These kinds of links the safetyaddon already creates based on the
requirement tags in the code of files under monitoring.
Creating them directly by hand will make it nasty to move it to new
revisions,
if the line that is tagged moves due to code changes.
I did a bit of experimenting with making a new branch for adding
tags,
then
committing changes to the main branch and merging main into the tag
branch
to sync, and that worked quite well as long as tags got their own
line, so
git
takes care oft he donkey work.
Of course one still has to check what git suggested and once a file
is
renamed
or moved, manuall work is inevitable, but at least git does most of
the
heavy
lifting that way.
With the manual links you'd have to check and change every single
one of them every time the revision changes, not fun i reckon
Right; in practice if I wanted to tag, for example, start_kernel() I could set:
https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/
init/ma
in.c?h=linux-5.4.y#n577
This link is going to be valid as long as this file is not touched by any
commit.
Now looking at the commit log for main.c in the LTS I see that it's
been
touched
4 times along 2020 but quite a bit more along 2019
https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/log/i
nit/mai
n.c?h=linux-5.4.y
On the flip side tagging directly in the code would imply patching
such
LTS...where though?
We need to create a dedicated branch that we regularly rebase on top
of
the LTS; from
what you say we should think of the tag patches as part of the
safety/requirements analysis
and rebasing them would be part of the safety analysis maintenance
as
opposed to updating
the hyperlink when needed....let's discuss about it today....I
honestly
dunno what is the best
option...

Thanks
Gab


Thanks
Gab
Best regards,
Jochen




--------------------------------------------------------------------
- INTEL CORPORATION ITALIA S.p.A. con unico socio
Sede: Milanofiori Palazzo E 4
CAP 20094 Assago (MI)
Capitale Sociale Euro 104.000,00 interamente versato Partita I.V.A.
e Codice Fiscale 04236760155 Repertorio Economico Amministrativo n.
997124 Registro delle Imprese di Milano nr. 183983/5281/33 Soggetta
ad attivita' di direzione e coordinamento di INTEL CORPORATION, USA

This e-mail and any attachments may contain confidential material
for the sole use of the intended recipient(s). Any review or
distribution by others is strictly prohibited. If you are not the
intended recipient, please contact the sender and delete all copies.






---------------------------------------------------------------------
INTEL CORPORATION ITALIA S.p.A. con unico socio
Sede: Milanofiori Palazzo E 4
CAP 20094 Assago (MI)
Capitale Sociale Euro 104.000,00 interamente versato Partita I.V.A. e Codice
Fiscale 04236760155 Repertorio Economico Amministrativo n. 997124 Registro
delle Imprese di Milano nr. 183983/5281/33 Soggetta ad attivita' di direzione e
coordinamento di INTEL CORPORATION, USA

This e-mail and any attachments may contain confidential material for the
sole use of the intended recipient(s). Any review or distribution by others is
strictly prohibited. If you are not the intended recipient, please contact the
sender and delete all copies.