Improve launch documentation #1
Conversation
Signed-off-by: Michel Hidalgo <mhidalgo@theaiinstitute.com>
There was a problem hiding this comment.
Thanks for this! I think the documentation is still making a lot of assumptions that I know something about the way ROS2 launch works already (context, substitutions, etc). I've put some notes in where I think it might help to expand.
One thing that could be worth addressing: Why use ros2 launch instead of just writing a quick Python or bash script to launch everything? In ROS1 the launch files were simpler than Python or bash but that doesn't feel true in ROS2. Do ROS2 XML/YAML files feel simpler and should we be directing folks to use those except in complicated circumstances?
|
|
||
| Actions alone allow for static descriptions that act upon and react to the environment in predefined ways. For a description to adapt to external input, _conditions_ and _substitutions_ exist. The former are boolean predicates, used for conditional actions, while the latter resemble text macros, useful for dynamic action configuration. | ||
|
|
||
| In a way, ``launch`` descriptions are not that different from ASTs in procedural programming languages. The analogy brings about an important distinction that is easy to miss when writing launch files using the native Python APIs: neither actions nor conditions nor substitutions are computations but descriptions of computations to be carried out on launch. That is how actions can be implemented as sequences of other actions -- launch file inclusion being the paradigmatic example of this. |
There was a problem hiding this comment.
I'll admit - I had to google AST...
There was a problem hiding this comment.
I think this is a really important point and one I didn't get for a long time. I might put this up front:
When writing a launch file with the Python APIs, you are not writing a Python program that will be executed line by line. Instead, you are writing a description of a file it will generate. (Or whatever it is you're actually doing - I still don't quite get how Python launch files work. Maybe I will when I finish this review.)
There was a problem hiding this comment.
Both good points. Probably worth a link to: https://en.wikipedia.org/wiki/Abstract_syntax_tree
There was a problem hiding this comment.
you are writing a description of a file it will generate.
It's not the description of a file, but a description of what launch should execute. Parsing XML/YAML yields the same object hierarchy. Taking the analogy further, start a Python shell and run:
import ast as pyparser
code = "a = 1; print('a = ', a)"
ast = pyparser.parse(code)
print(pyparser.dump(ast, indent=4))
bytecode = compile(ast, "<string>", "exec")
exec(bytecode)Oversimplifications aside, ast is the result of parsing Python code. We can then compile ast into bytecode, which is executable. That's what the Python interpreter does when given a Python script.
launch does something similar. It takes a LaunchDescription, which may have been parsed from XML or YAML, or manually crafted like one does when writing launch files in Python, and instead of compiling anything, it executes it (again oversimplifying, actually the LaunchDescription gets traversed and executing Actions in it pushes coroutines to an event loop).
So when you are writing launch files in Python, it's as if you were writing the ASTs that Python works with or the intermediate representations that gcc compiles C++ to before generating machine code. That's why it may sometimes be difficult and painful (besides lack of documentation, that's also a thing). You are working directly with the building blocks of a DSL.
| from launch import LaunchDescription | ||
| from launch.actions import DeclareLaunchArgument | ||
| from launch.actions import LogInfo | ||
| from launch.substitutions import LaunchConfiguration |
There was a problem hiding this comment.
Maybe not the right place for this but I don't really understand why LaunchConfiguration is part of substitutions or what substitutions are anyway. The original roslaunch documentation is pretty clear about this - I haven't found anything nearly so clear for ROS2 (and note that that part of the design doc is a TODO!) and it seems like ROS2 extends the definition. This tutorial starts off by defining substitutions as a tool for re-using launch files but I can't figure out if they're variables or functions or some combination of the two? The tutorial gives a bunch of examples that are text manipulation, but it doesn't make it clear, however, why substitutions are the right choice for this - why can't I just do normal Python string manipulation? Should all text manipulation go through a substitution? If so, why was it designed this way?
A description of all the available substitutions and how to use them would be very helpful. I was unable to find one other than by reading the source code.
There was a problem hiding this comment.
can't figure out if they're variables or functions or some combination of the two?
It's easier to think about them as expressions evaluated in launch context.
why substitutions are the right choice for this - why can't I just do normal Python string manipulation?
Because only substitutions can "easily" access launch context, where runtime state lives.
| Each launch file performs the following actions: | ||
|
|
||
| * Declares an argument without defaults. | ||
| * Generates namespaced groups iteratively based on that argument. |
There was a problem hiding this comment.
I don't really understand what this means.
Also one example that would be really helpful somewhere would be to show how to have an optional namespace.
There was a problem hiding this comment.
@jbarry-bdai What is an optional namespace?
There was a problem hiding this comment.
If you pass in a non-empty-string namespace argument then the nodes live in a namespace, but if the argument is empty they don't.
| @@ -370,9 +679,9 @@ Python, XML, or YAML: Which should I use? | |||
|
|
|||
| For most applications the choice of which ROS 2 launch format comes down to developer preference. | |||
| However, if your launch file requires flexibility that you cannot achieve with XML or YAML, you can use Python to write your launch file. | |||
| Using Python for ROS 2 launch is more flexible because of following two reasons: | |||
| Using Python for ROS 2 launch is more flexible because: | |||
There was a problem hiding this comment.
I would put the "which should I use" at the top of this doc.
| Taking arguments | ||
| ^^^^^^^^^^^^^^^^ |
There was a problem hiding this comment.
Other examples I would like to see (probably in their own tutorials so they are googleable):
- passing command line arguments to nodes
- passing arguments to a sub-launch file
- setting parameters for nodes by yaml file, by running the node with ros2 run, via the launch file, via command line argument to the launch file, and via the command line after the node is launched
There was a problem hiding this comment.
setting parameters for nodes by yaml file, by running the node with ros2 run, via the launch file, via command line argument to the launch file, and via the command line after the node is launched
I will point out some of these things are not specific to launch, and there is https://docs.ros.org/en/humble/Tutorials/Beginner-CLI-Tools/Understanding-ROS2-Parameters/Understanding-ROS2-Parameters.html and https://docs.ros.org/en/humble/How-To-Guides/Node-arguments.html on those.
It is true, however, that ROS 2 documentation as a whole isn't as cohesive as it could be. There are also discoverability issues. None of that is specific to launch.
There was a problem hiding this comment.
Sorry I meant "passing command line arguments to nodes in a launch file" (e.g. the argument goes through the launch file to the node). For the last bullet point, I guess you're right - I diverged from launch files a little. I think showing how to pass a yaml config file to a node inside the launch file and how to set parameters via the launch file and via the launch file plus command line argument is all launch related though. Maybe a "How Parameters and Launch Files Interact" document.
As someone mentioned elsewhere, not everything I'm suggesting is probably suitable for an intro document but I think I'd like to see these all exist maybe with links from the intro document. I do think discoverability is a big problem and I'm not sure what to do about that except to keep linking everything to everything...
There was a problem hiding this comment.
Can I ask for just one more example here: if we have a node that takes a command line argument (not ros parameter) can you show how to pass that through from a launch file variable?
There was a problem hiding this comment.
@jbarry-bdai it's there! See the Configuring nodes example. It includes a launch file, passing it an argument that is then passed as a command line argument to a node.
There was a problem hiding this comment.
Gah sorry don't know how I missed that!
|
Another thing that would just generally help is a list of all the XML tags or Python classes available and what they do. I believe the only way to find that right now is by reading the source code. |
|
@ksharma-bdai @amessing-bdai @baxelrod-bdai @dhoelscher-bdai PTAL! For a reason I don't fully understand, I cannot assign more reviewers nor add you as collaborators 🤔 |
I think I'm only a reviewer because I made comments. I wasn't before I did that... probably has to do with the settings of your personal repo. |
There was a problem hiding this comment.
This looks great, definitely an improvement over the original! I noticed the About-Launch file is under Concepts/Basic - is there an Advanced folder that discusses things in more depth? After reading back our comments so far, I wonder if we're asking for depth that is not appropriate for the Basic (which I read to mean a beginner/overview article). I'm trying to think if this would be overwhelming as an actual beginner, but the first paragraph about the purpose is probably sufficient for most people would stop and skip to examples...
|
|
||
| All of the above is specified in a launch file, which can be written in Python, XML, or YAML. | ||
| This launch file can then be run using the ``ros2 launch`` command, and all of the nodes specified will be run. | ||
| The purpose of the ``launch`` system in ROS 2 is to provide the means to orchestrate the execution of such a system. This tool uses _launch files_ to describe the system and its configuration, but not limited to which programs to run, how to run them and which arguments to use for them. These descriptions are then executed, starting processes, monitoring them, reporting on their state, and even reacting to their behavior. |
There was a problem hiding this comment.
It might be nice to very briefly explain the difference between launch and run here with a link out to the run documentation. It's a very basic distinction, but might be helpful for true beginners.
|
|
||
| Actions alone allow for static descriptions that act upon and react to the environment in predefined ways. For a description to adapt to external input, _conditions_ and _substitutions_ exist. The former are boolean predicates, used for conditional actions, while the latter resemble text macros, useful for dynamic action configuration. | ||
|
|
||
| In a way, ``launch`` descriptions are not that different from ASTs in procedural programming languages. The analogy brings about an important distinction that is easy to miss when writing launch files using the native Python APIs: neither actions nor conditions nor substitutions are computations but descriptions of computations to be carried out on launch. That is how actions can be implemented as sequences of other actions -- launch file inclusion being the paradigmatic example of this. |
There was a problem hiding this comment.
One of the things that still sometimes catches me by surprise is the order of operations for a launch - e.g. do opaque functions always run first in a blocking way, do other nodes in the launch description run serially or concurrently, how are nodelets composed? I think describing all of those scenarios might be a bit dense/out of scope, but maybe a brief discussion on launch flow and that you can also explicitly control the order of operations with events, etc. Maybe this is more of an advanced concept for this doc though.
There was a problem hiding this comment.
I think describing all of those scenarios might be a bit dense/out of scope
Yeah, I think this belongs to launch documentation. Too deep.
That said, I do mention now that actions naturally run concurrently. That is, they are processed in sequence, one by one, but they may outlast that execution and persist alongside others. Dynamic node composition has less to do with launch and more with how dynamic composition works in general (ie. ROS 2 service based API hitting a container node).
|
|
||
| # Currently unsupported | ||
|
|
||
| Managing lifecycles |
There was a problem hiding this comment.
PS here's the bug we briefly discussed on lifecycle nodes not being properly transitioned when you ctrl+c the launch:
ros2/rclcpp#997
|
|
||
| Actions alone allow for static descriptions that act upon and react to the environment in predefined ways. For a description to adapt to external input, _conditions_ and _substitutions_ exist. The former are boolean predicates, used for conditional actions, while the latter resemble text macros, useful for dynamic action configuration. | ||
|
|
||
| In a way, ``launch`` descriptions are not that different from ASTs in procedural programming languages. The analogy brings about an important distinction that is easy to miss when writing launch files using the native Python APIs: neither actions nor conditions nor substitutions are computations but descriptions of computations to be carried out on launch. That is how actions can be implemented as sequences of other actions -- launch file inclusion being the paradigmatic example of this. |
There was a problem hiding this comment.
Both good points. Probably worth a link to: https://en.wikipedia.org/wiki/Abstract_syntax_tree
| Each launch file performs the following actions: | ||
|
|
||
| * Declares an argument without defaults. | ||
| * Generates namespaced groups iteratively based on that argument. |
There was a problem hiding this comment.
@jbarry-bdai What is an optional namespace?
|
|
||
| * Python is a scripting language, and thus you can leverage the language and its libraries in your launch files. | ||
| * `ros2/launch <https://github.com/ros2/launch>`_ (general launch features) and `ros2/launch_ros <https://github.com/ros2/launch_ros>`_ (ROS 2 specific launch features) are written in Python and thus you have lower level access to launch features that may not be exposed by XML and YAML. | ||
| * `ros2/launch <https://github.com/ros2/launch>`_ (general launch features) and `ros2/launch_ros <https://github.com/ros2/launch_ros>`_ (ROS 2 specific launch features) are written in Python and thus you have lower level access to launch features that may not be yet exposed via XML and YAML. |
There was a problem hiding this comment.
What is the difference between launch and launch_ros? Why have 2 APIs? How do they work together / complement each other?
There was a problem hiding this comment.
To answer this here (the answer I believe belongs to the packages' documentation), launch was designed to be extensible. launch_ros leverages that to introduce ROS 2 specific actions, of which there are none in core launch . Why? I believe it was a mix of architectural cleanliness (ie. separation of concerns) and ROS agnosticity (I vaguely recall launch being used for Gazebo sim without ROS 2).
|
@baxelrod-bdai @jbarry-bdai @ksharma-bdai the answers for some of the questions made above go way beyond the scope and intent of the So, I'm going to add to the |
Signed-off-by: Michel Hidalgo <mhidalgo@theaiinstitute.com>
Signed-off-by: Michel Hidalgo <mhidalgo@theaiinstitute.com>
Signed-off-by: Michel Hidalgo <mhidalgo@theaiinstitute.com>
mhidalgo-bdai
requested review from
jbarry-bdai,
ksharma-bdai and
baxelrod-bdai
| output='screen') | ||
| ] | ||
| ) | ||
| for turtle_name in turtles.perform(context).split() |
There was a problem hiding this comment.
Not sure if it should go here but what does perform do?
There was a problem hiding this comment.
It performs the substitution on the given context.
There was a problem hiding this comment.
Does it return a string?
Always. Substitutions operate on text data, even if some may parse it by definition e.g. boolean operator substitutions parse booleans out of text data.
| Taking arguments | ||
| ^^^^^^^^^^^^^^^^ |
There was a problem hiding this comment.
Can I ask for just one more example here: if we have a node that takes a command line argument (not ros parameter) can you show how to pass that through from a launch file variable?
Signed-off-by: Michel Hidalgo <mhidalgo@theaiinstitute.com>
|
@jbarry-bdai @ksharma-bdai @baxelrod-bdai once you are happy with how this looks, let me know and I'll submit it upstream. |
|
I'm happy with the content. Is there any way to give a BDAII attribution? If not let me just clear that with leadership before we submit it. |
Hmm, we can ask the maintainers on submission or maybe shoot them an email. It would be a first though, to the best of my knowledge. I've only seen organizations mentioned through commit authors' emails. |
|
Checked with Jessica and Al - we're good to put this up publicly. Use your BDAII affiliated address and give us a line in the commit message or whatever and all's well! |
|
Awesome. Moving forward! |
No description provided.