rosdistro VS rclcpp

I have added their rosdep names (found here) to my packages.xml (see the end of this post), but even after running catkin-make and trying to run the module, the script throws a ModuleNotFound exception at the UTM package meaning that the package was never installed.

I second this. Just remove the package by removing whatever you added to the rosdistro repo when submitting/publishing the package.

The business about mapping from PyPI to system dependencies is an important one, and (having not read the entire thread) I do hope that gets some attention— it's particularly curious that it's been this long and it hasn't, given Python's often-role as a glue language.

Another example of an ecosystem maintaining mappings out to system packages is ROS and rosdep:

https://github.com/ros/rosdistro/blob/master/rosdep/base.yam...

Now it's interesting because ROS is primarily concerned with supplying a sane build-from-source story, so much of what's in the rosdep "database" is the xxxx-dev packages, but in the case of wheels, it would be more about binary dependencies, and those are auto-discoverable with ldd, shlibdeps, and the like. In Debian (and I assume other distros), the binary so packages are literally the library soname + abi versions, so if you have ldd output, you have the list of exactly what to install.

The names you use for the tag (and the other dependency tags in your package.xml) are used by rosdep to figure out what needs to be installed. The name doesn't necessarily correspond to the apt or pip package name, but most of the time it's the same. For example, for matplotlib: https://github.com/ros/rosdistro/blob/24141d9063fddd6eeca7f1db9e721fa8d600c62f/rosdep/python.yaml#L6296

Thanks for the response!

> This is difficult to answer without knowing more details.

The situation specifically is the ROS ecosystem, where metadata is managed in these package.xml files:

https://github.com/ros2/rclcpp/blob/master/rclcpp/package.xm...

The federated nature of the ecosystem has led to a culture where it's very normal to be building dozens of these at once, in the same workspace together, often from multiple repos (the repo above has four in it). So there are several build tools which automate the work of examining a source workspace and building all the packages within it in the correct topological order, respecting build_depend tags. The newest of these tools (colcon) has actually made the package.xml optional in many cases, as it can examine CMakelists, setup.py, BUILD, etc, and discover for itself what the dependencies are.

Your "distribution" of ROS is formed by listing all the packages and repos in this big file, for which there is other tooling to manage pulling dependency sources, whatever: https://github.com/ros/rosdistro/blob/master/foxy/distributi...

Anyway, so the existing ROS/nix efforts (1) seem to basically consume all of this package/distribution metadata at once and generate a giant parallel structure of nix definitions (eg https://github.com/lopsided98/nix-ros-overlay/blob/master/di...), which I fear would be completely opaque to users and any system which required everyone to leave behind these existing workflows would be an immediate non-starter.

I think the ideal scenario (and what it would look like if I built this myself based on debs) would be that you could source the "base" workspace as usual (enter the nix-shell?), and check out source, build packages as usual with colcon, the usual workspace-building tool, but there'd be an extra plugin/verb/flag for it, which would make it build each package as a nix package instead of into the usual installspace. The verb would generate the nix definitions on the fly, and probably handle the invocation and build-parallelism side of it as well.

[1]: https://github.com/acowley/ros2nix, https://github.com/lopsided98/nix-ros-overlay

Thanks for the feedback. I bumped a couple of the core devs and got the change reverted and it is now merged. It should be in the next Noetic release.

I think there's an overload of rclcpp::action::create_client that will accept an rcl_action_client_options_t (see here https://github.com/ros2/rclcpp/pull/1133/files) but it looks like MoveGroupInterface doesn't pass anything in when it creates the action clients:

So one of the bigger issues I've encountered is that re-entrant services can break with the multi-threaded executor https://github.com/ros2/rclcpp/issues/1212 Services in ROS2 by default are not reentrant meaning they need to be called asynchronously. Since my team was porting ROS1 nodes over we wanted to keep the old blocking service call behavior in ROS2 to avoid changing decision logic. Unfortunately, we found out that this can cause the node to crash per the issue above. The fix for that issue was implemented in Humble irc, but can't be ported back to foxy. That's one specific example, but we've seen a few other issues like that as well.

Alternatively, you can use the default client libraries like rclpy or rclcpp for writing custom code that reads and writes ROS2 messages.

Thanks for the response!

> This is difficult to answer without knowing more details.

The situation specifically is the ROS ecosystem, where metadata is managed in these package.xml files:

https://github.com/ros2/rclcpp/blob/master/rclcpp/package.xm...

The federated nature of the ecosystem has led to a culture where it's very normal to be building dozens of these at once, in the same workspace together, often from multiple repos (the repo above has four in it). So there are several build tools which automate the work of examining a source workspace and building all the packages within it in the correct topological order, respecting build_depend tags. The newest of these tools (colcon) has actually made the package.xml optional in many cases, as it can examine CMakelists, setup.py, BUILD, etc, and discover for itself what the dependencies are.

Your "distribution" of ROS is formed by listing all the packages and repos in this big file, for which there is other tooling to manage pulling dependency sources, whatever: https://github.com/ros/rosdistro/blob/master/foxy/distributi...

Anyway, so the existing ROS/nix efforts (1) seem to basically consume all of this package/distribution metadata at once and generate a giant parallel structure of nix definitions (eg https://github.com/lopsided98/nix-ros-overlay/blob/master/di...), which I fear would be completely opaque to users and any system which required everyone to leave behind these existing workflows would be an immediate non-starter.

I think the ideal scenario (and what it would look like if I built this myself based on debs) would be that you could source the "base" workspace as usual (enter the nix-shell?), and check out source, build packages as usual with colcon, the usual workspace-building tool, but there'd be an extra plugin/verb/flag for it, which would make it build each package as a nix package instead of into the usual installspace. The verb would generate the nix definitions on the fly, and probably handle the invocation and build-parallelism side of it as well.

[1]: https://github.com/acowley/ros2nix, https://github.com/lopsided98/nix-ros-overlay