Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add grep-cli crate #1038

Merged
merged 4 commits into from
Sep 5, 2018
Merged

add grep-cli crate #1038

merged 4 commits into from
Sep 5, 2018

Conversation

BurntSushi
Copy link
Owner

@BurntSushi BurntSushi commented Sep 5, 2018

This PR rounds out the initial functionality of libripgrep by pushing several pieces of somewhat CLI specific functionality into a utility library. This library, called grep-cli, includes things like reading process output (which also fixes #990), decompressing files, unescaping text and reading patterns from files. Throughout most operations, we retain good failure modes by giving descriptive error messages.

We also add a couple new minor flags:

  • --line-buffered and --block-buffered for granular control over ripgrep's buffering strategy. (GNU grep has a --line-buffered flag as well.)
  • --pre-glob for filtering files sent through the preprocessor specified by --pre. This makes it plausible to use --pre on large directories if it is only actually needed for a small subset of files. (Otherwise, the process overhead tends to make the search take unbearably long.)

This commit moves a lot of "utility" code from ripgrep core into
grep-cli. Any one of these things might not be worth creating a new
crate, but combining everything together results in a fair number of a
convenience routines that make up a decent sized crate.

There is potentially more we could move into the crate, but much of what
remains in ripgrep core is almost entirely dealing with the number of
flags we support.

In the course of doing moving things to the grep-cli crate, we clean up
a lot of gunk and improve failure modes in a number of cases. In
particular, we've fixed a bug where other processes could deadlock if
they write too much to stderr.

Fixes #990
These flags provide granular control over ripgrep's buffering strategy.
The --line-buffered flag can be genuinely useful in certain types of shell
pipelines. The --block-buffered flag has a murkier use case, but we add it
for completeness.
The --pre-glob flag is like the --glob flag, except it applies to filtering
files through the preprocessor instead of for search. This makes it
possible to apply the preprocessor to only a small subset of files, which
can greatly reduce the process overhead of using a preprocessor when
searching large directories.
@BurntSushi BurntSushi merged commit fd22cd5 into master Sep 5, 2018
@BurntSushi BurntSushi deleted the ag/grep-cli branch September 5, 2018 03:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

read stderr when running external commands
1 participant