Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

libsubprocess: invert SETPGRP flag logic #6082

Conversation

chu11
Copy link
Member

@chu11 chu11 commented Jul 4, 2024

Problem: Remote subprocesses default to always set the SETPGRP flag. This leads to inconsistent behavior where local subprocesses don't have the flag set but remote ones do.

We could remove this default and require remote subprocess execution to set this flag as needed. However, it ends up that the vast majority of code in flux-core sets the SETPGRP flag anyways, leading to an excess amount of code change to make local vs remote subprocesses consistent.

Solution: Invert the SETPGRP flag logic into NO_SETPGRP. Now, by default, have both local and remote subprocesses call setpgrp(2). The NO_SETPGRP flag will then disable the call to setpgrp(2). Update all callers to libsubprocess accordingly.

Note that we leave the new NO_SETPGRP flag to be for local subprocesses only. If there comes a time for it to be supported for remote subprocesses, that will be for another day.

@chu11 chu11 force-pushed the issue5968_libsubprocess_remote_subprocess_flags branch from ce51c53 to cb164ef Compare July 4, 2024 21:43
Problem: Some function parameters were not indented properly.

Fix invalid indentation.
Problem: Almost all subprocess flags indicate if they can be used
locally, remote, or both.  One flag lacked this documentation.

Update comment to indicate FLUX_SUBPROCESS_FLAGS_FORK_EXEC can be
used locally only.
Problem: Remote subprocesses default to always set the SETPGRP
flag.  This leads to inconsistent behavior where local subprocesses
don't have the flag set but remote ones do.

We could remove this default and require remote subprocess execution
to set this flag as needed.  However, it ends up that the vast majority
of code in flux-core sets the SETPGRP flag anyways, leading to an excess
amount of code change to make local vs remote subprocesses consistent.

Solution: Invert the SETPGRP flag logic into NO_SETPGRP.  Now, by default,
have both local and remote subprocesses call setpgrp(2).  The NO_SETPGRP
flag will then disable the call to setpgrp(2).  Update all callers
to libsubprocess accordingly.

Fixes flux-framework#5968
@chu11 chu11 force-pushed the issue5968_libsubprocess_remote_subprocess_flags branch from cb164ef to 9fe507b Compare October 9, 2024 20:23
@garlick
Copy link
Member

garlick commented Oct 9, 2024

We could remove this default and require remote subprocess execution to set this flag as needed. However, it ends up that the vast majority of code in flux-core sets the SETPGRP flag anyways, leading to an excess amount of code change to make local vs remote subprocesses consistent.

API wise, NO_SETPGRP seems vaguely wrong to me, but I see where you're coming from about updating all the tests n stuff. @grondo any thoughts?

@chu11
Copy link
Member Author

chu11 commented Oct 9, 2024

API wise, NO_SETPGRP seems vaguely wrong to me, but I see where you're coming from about updating all the tests n stuff. @grondo any thoughts?

Updating everything to set the SETPGRP flag is certainly ok and fine too, we could do that.

It's mostly to get this consistent, b/c the follow up will be to allow subprocess flags to be passed to remote subprocesses.

@grondo
Copy link
Contributor

grondo commented Oct 10, 2024

Yeah, I was struggling with NO_SETPGRP too, but the simplicity is indeed appealing. I think this PR is probably fine as is.

Copy link
Contributor

@grondo grondo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. It did nicely match the shell nosetpgrp option.

@mergify mergify bot merged commit 47040bc into flux-framework:master Oct 10, 2024
33 checks passed
Copy link

codecov bot commented Oct 10, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 83.30%. Comparing base (7cf4359) to head (9fe507b).
Report is 4 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #6082      +/-   ##
==========================================
- Coverage   83.31%   83.30%   -0.02%     
==========================================
  Files         523      523              
  Lines       86194    86199       +5     
==========================================
- Hits        71811    71806       -5     
- Misses      14383    14393      +10     
Files with missing lines Coverage Δ
src/broker/runat.c 79.13% <ø> (-0.07%) ⬇️
src/cmd/builtin/proxy.c 78.24% <100.00%> (+0.30%) ⬆️
src/cmd/flux-start.c 83.74% <100.00%> (+0.08%) ⬆️
src/common/libsubprocess/fork.c 76.74% <100.00%> (+1.55%) ⬆️
src/common/libsubprocess/posix_spawn.c 88.40% <100.00%> (ø)
src/common/libsubprocess/server.c 78.73% <100.00%> (ø)
src/common/libsubprocess/subprocess.c 88.90% <100.00%> (ø)
src/shell/task.c 82.03% <100.00%> (ø)

... and 7 files with indirect coverage changes

@chu11 chu11 deleted the issue5968_libsubprocess_remote_subprocess_flags branch October 11, 2024 17:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants