Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pyupgrade: Printf string formatting #1803

Merged
merged 77 commits into from
Jan 21, 2023
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
77 commits
Select commit Hold shift + click to select a range
bf7e246
Got some basic stuff in
colin99d Jan 11, 2023
bdb3b43
Merge branch 'main' into PrintfStringFormatting
colin99d Jan 11, 2023
84717ea
GOt ONE unit test passing, LETS GO
colin99d Jan 12, 2023
7efb4ec
Better handle empty strings
colin99d Jan 12, 2023
d4fabbb
Fix how i handle string_end
colin99d Jan 12, 2023
a32ce74
Fixed another bug
colin99d Jan 12, 2023
37069f7
Finished unit tests
colin99d Jan 12, 2023
a4bd9a0
Merge branch 'main' into PrintfStringFormatting
colin99d Jan 12, 2023
c240f43
About to actually get this thing linting
colin99d Jan 13, 2023
99ce56e
Finished the body of the printf_string_formatting function
colin99d Jan 13, 2023
5e6939d
Fixed the regex
colin99d Jan 13, 2023
58a1d7d
Finished unit tests (2 failing, and also simplified them BIG time)
colin99d Jan 13, 2023
0756ffa
Minor wording change
colin99d Jan 13, 2023
dadb075
Fixed one bug
colin99d Jan 13, 2023
25be7ae
Fixed last issue with unit tests
colin99d Jan 13, 2023
d1c9d26
Cleaned prints
colin99d Jan 13, 2023
d863ce5
Going through errors
colin99d Jan 13, 2023
bfa6ad2
better handling
colin99d Jan 14, 2023
3265eb7
Added integration tests
colin99d Jan 14, 2023
edcbb5d
Fixed one issue
colin99d Jan 14, 2023
757c71d
Fixed the nested issue
colin99d Jan 14, 2023
92d5503
Increase comments
colin99d Jan 14, 2023
abb9276
Merged with new main
colin99d Jan 14, 2023
daf5da8
Two edge cases left on tuple
colin99d Jan 14, 2023
0fa0d8c
Refactored check_statement outside of main function
colin99d Jan 14, 2023
bb9c4c4
Better handling
colin99d Jan 14, 2023
dd88455
Git rid of nesting, cleaner code now
colin99d Jan 14, 2023
e520d83
Fixed one of two tuple errors, just need to fix regex one now
colin99d Jan 14, 2023
3212e67
In progress, but need to switch
colin99d Jan 15, 2023
ca40881
Moving towards a complete dict implementation
colin99d Jan 15, 2023
b36d8c3
Final testing for dict
colin99d Jan 15, 2023
c7c626d
Updated test cases
colin99d Jan 15, 2023
9e5ac12
Ran new tests
colin99d Jan 15, 2023
c952500
Fixed formatting, removed prints
colin99d Jan 15, 2023
e738b4a
linted clippy and added docs
colin99d Jan 15, 2023
e1647cd
Fixed typos and fmt
colin99d Jan 15, 2023
9d41062
Added correct level of visibility
colin99d Jan 15, 2023
3f5148f
Replaced string with Option<String> to increase readability
colin99d Jan 15, 2023
5f89844
Merge branch 'main' into PrintfStringFormatting
charliermarsh Jan 17, 2023
07688c0
Minor tweaks
charliermarsh Jan 17, 2023
8526f11
Fix reference
charliermarsh Jan 17, 2023
3e9f728
Simplified to a broken curly_escape
colin99d Jan 17, 2023
420f934
Fixed dict tests
colin99d Jan 17, 2023
0793afd
Fixed embarassing mistake in docstring
colin99d Jan 17, 2023
e7390be
Added unit tests for the issues Charlie found
colin99d Jan 17, 2023
0bee8af
Merge branch 'main' into PrintfStringFormatting
charliermarsh Jan 17, 2023
b042fd8
Merged
colin99d Jan 18, 2023
e571735
Merged
colin99d Jan 18, 2023
de90d4d
Fixed glitch from merge
colin99d Jan 18, 2023
ebee722
Began conversion to cformatstring
colin99d Jan 18, 2023
32aec6b
Progressed
colin99d Jan 18, 2023
6056ebc
Merge branch 'main' into PrintfStringFormatting
colin99d Jan 18, 2023
92872b6
Got a little further
colin99d Jan 19, 2023
7d8d14c
Merge branch 'main' into PrintfStringFormatting
charliermarsh Jan 19, 2023
923b43d
Merge remote-tracking branch 'refs/remotes/origin/main' into PrintfSt…
charliermarsh Jan 20, 2023
b06be38
Fixed two tests
colin99d Jan 20, 2023
acfdb51
Merged
colin99d Jan 20, 2023
675f6fc
working towards a better solution
colin99d Jan 20, 2023
e8702e5
Fixed tests
colin99d Jan 20, 2023
522eb70
Merge branch 'main' into PrintfStringFormatting
charliermarsh Jan 20, 2023
6d89c87
Upgrade RustPython parser
charliermarsh Jan 20, 2023
837b487
Fix Clippy
charliermarsh Jan 20, 2023
8f54ebd
Mid refactor
charliermarsh Jan 20, 2023
d805dc7
Merge branch 'main' into PrintfStringFormatting
charliermarsh Jan 20, 2023
e8e6912
Misc changes
charliermarsh Jan 20, 2023
4660713
Mid-way through conversion
charliermarsh Jan 20, 2023
1c7179d
Merge branch 'main' into PrintfStringFormatting
charliermarsh Jan 20, 2023
dac86ab
Merge branch 'main' into PrintfStringFormatting
charliermarsh Jan 20, 2023
76ddc62
Try to move tests
charliermarsh Jan 21, 2023
012144b
Add prints
charliermarsh Jan 21, 2023
3cfc93a
Add some restrictions
charliermarsh Jan 21, 2023
60e73d5
Try out a lexer
charliermarsh Jan 21, 2023
31ae316
Support emoji
charliermarsh Jan 21, 2023
39fd5bc
Merge branch 'main' into PrintfStringFormatting
charliermarsh Jan 21, 2023
a5e5c1d
Fix ) case
charliermarsh Jan 21, 2023
b179026
Merge branch 'main' into PrintfStringFormatting
charliermarsh Jan 21, 2023
85d75e8
Point to latest RustPython
charliermarsh Jan 21, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Upgrade RustPython parser
  • Loading branch information
charliermarsh committed Jan 20, 2023
commit 6d89c873eb42be37fe64ac590ef26b966e06b3d7
97 changes: 87 additions & 10 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 3 additions & 3 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -48,9 +48,9 @@ regex = { version = "1.6.0" }
ropey = { version = "1.5.0", features = ["cr_lines", "simd"], default-features = false }
ruff_macros = { version = "0.0.226", path = "ruff_macros" }
rustc-hash = { version = "1.1.0" }
rustpython-ast = { features = ["unparse"], git = "https://github.com/RustPython/RustPython.git", rev = "ff90fe52eea578c8ebdd9d95e078cc041a5959fa" }
rustpython-common = { git = "https://github.com/RustPython/RustPython.git", rev = "ff90fe52eea578c8ebdd9d95e078cc041a5959fa" }
rustpython-parser = { features = ["lalrpop"], git = "https://github.com/RustPython/RustPython.git", rev = "ff90fe52eea578c8ebdd9d95e078cc041a5959fa" }
rustpython-ast = { features = ["unparse"], git = "https://github.com/charliermarsh/RustPython.git", rev = "bfd31f7ffff4c3cbea5aee5bb044237f9be1d604" }
rustpython-common = { git = "https://github.com/charliermarsh/RustPython.git", rev = "bfd31f7ffff4c3cbea5aee5bb044237f9be1d604" }
rustpython-parser = { features = ["lalrpop"], git = "https://github.com/charliermarsh/RustPython.git", rev = "bfd31f7ffff4c3cbea5aee5bb044237f9be1d604" }
schemars = { version = "0.8.11" }
semver = { version = "1.0.16" }
serde = { version = "1.0.147", features = ["derive"] }
Expand Down
7 changes: 6 additions & 1 deletion resources/test/fixtures/pyupgrade/UP031_0.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,12 @@

"%s%% percent" % (15,)

"%f" % (15,)

"%.f" % (15,)

"%.3f" % (15,)

"%3f" % (15,)

"%-5f" % (5,)
Expand Down Expand Up @@ -49,7 +55,6 @@

"%#o" % (123,)


"%4%" % ()

"%.2r" % (1.25)
Expand Down
4 changes: 2 additions & 2 deletions src/rules/pyflakes/cformat.rs
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ use std::str::FromStr;

use rustc_hash::FxHashSet;
use rustpython_common::cformat::{
CFormatError, CFormatPart, CFormatQuantity, CFormatSpec, CFormatString,
CFormatError, CFormatPart, CFormatPrecision, CFormatQuantity, CFormatSpec, CFormatString,
};

pub(crate) struct CFormatSummary {
Expand Down Expand Up @@ -45,7 +45,7 @@ impl TryFrom<&str> for CFormatSummary {
num_positional += 1;
starred = true;
}
if precision == &Some(CFormatQuantity::FromValuesTuple) {
if precision == &Some(CFormatPrecision::Quantity(CFormatQuantity::FromValuesTuple)) {
num_positional += 1;
starred = true;
}
Expand Down
43 changes: 16 additions & 27 deletions src/rules/pyupgrade/rules/printf_string_formatting.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ use std::str::FromStr;
use once_cell::sync::Lazy;
use regex::Regex;
use rustpython_common::cformat::{
CConversionFlags, CFormatPart, CFormatQuantity, CFormatSpec, CFormatString,
CConversionFlags, CFormatPart, CFormatPrecision, CFormatQuantity, CFormatSpec, CFormatString,
};
use rustpython_parser::ast::{Constant, Expr, ExprKind};

Expand All @@ -17,7 +17,6 @@ use crate::violations;

static MODULO_CALL: Lazy<Regex> = Lazy::new(|| Regex::new(r" % ([({])").unwrap());
static PYTHON_NAME: Lazy<Regex> = Lazy::new(|| Regex::new(r"[^\W0-9]\w*").unwrap());
static EMOJI_SYNTAX: Lazy<Regex> = Lazy::new(|| Regex::new(r"\\N\{.*?}").unwrap());

#[derive(Debug, PartialEq, Clone)]
struct PercentFormatPart {
Expand All @@ -44,33 +43,31 @@ impl PercentFormatPart {
conversion,
}
}
}

fn from_rustpython(spec: &CFormatSpec) -> Self {
impl From<&CFormatSpec> for PercentFormatPart {
fn from(spec: &CFormatSpec) -> Self {
let clean_width = match &spec.min_field_width {
Some(width_item) => match width_item {
CFormatQuantity::Amount(amount) => Some(amount.to_string()),
// FOR REVIEWER: Not sure if below is the correct way to handle
// FromValuesTuple
CFormatQuantity::FromValuesTuple => Some("*".to_string()),
},
None => None,
};
let clean_precision = match &spec.precision {
Some(width_item) => match width_item {
Some(CFormatPrecision::Quantity(quantity)) => match quantity {
CFormatQuantity::Amount(amount) => Some(format!(".{amount}")),
// FOR REVIEWER: Not sure if below is the correct way to handle
// FromValuesTuple
CFormatQuantity::FromValuesTuple => Some(".*".to_string()),
},
Some(CFormatPrecision::Dot) => Some(".".to_string()),
None => None,
};
let flags = if spec.flags.is_empty() {
None
} else {
Some(get_flags(spec.flags))
};

PercentFormatPart::new(
Self::new(
spec.mapping_key.clone(),
flags,
clean_width,
Expand All @@ -92,8 +89,8 @@ impl PercentFormat {
}
}

/// Converts `RustPython`'s C Conversion Flags into their python string
/// representation
/// Converts the `RustPython` Conversion Flags into their Python string
/// representation.
fn get_flags(flags: CConversionFlags) -> String {
let mut flag_string = String::new();
if flags.contains(CConversionFlags::ALTERNATE_FORM) {
Expand All @@ -114,7 +111,7 @@ fn get_flags(flags: CConversionFlags) -> String {
flag_string
}

/// Converts a string to a vector of `PercentFormat` structs
/// Converts a string to a vector of [`PercentFormat`] structs.
fn parse_percent_format(string: &str) -> Vec<PercentFormat> {
let mut formats: Vec<PercentFormat> = vec![];

Expand All @@ -134,7 +131,7 @@ fn parse_percent_format(string: &str) -> Vec<PercentFormat> {
}
};
if let CFormatPart::Spec(c_spec) = &the_next {
current_format.parts = Some(PercentFormatPart::from_rustpython(c_spec));
current_format.parts = Some(c_spec.into());
}
formats.push(current_format);
}
Expand Down Expand Up @@ -256,8 +253,6 @@ fn percent_to_format(string: &str) -> String {
/// If the tuple has one argument it removes the comma, otherwise it returns the
/// tuple as is
fn clean_right_tuple(checker: &mut Checker, right: &Expr) -> String {
// FOR REVIEWER: Let me know if you want this redone in libcst, the reason I
// didnt is because it starts as a Tuple, but ends as a Call
let mut base_string = checker
.locator
.slice_source_code_range(&Range::from_located(right))
Expand Down Expand Up @@ -515,16 +510,12 @@ pub(crate) fn printf_string_formatting(checker: &mut Checker, expr: &Expr, right
violations::PrintfStringFormatting,
Range::from_located(expr),
);
// Emoji sytnax is very rare and adds a lot of complexity to the code, so we are
// only issuing a warning if it exists, and not fixing the code
if checker.patch(&Rule::PrintfStringFormatting) {
if !EMOJI_SYNTAX.is_match(&expr_string) {
diagnostic.amend(Fix::replacement(
new_string,
expr.location,
expr.end_location.unwrap(),
));
}
diagnostic.amend(Fix::replacement(
new_string,
expr.location,
expr.end_location.unwrap(),
));
}
checker.diagnostics.push(diagnostic);
}
Expand Down Expand Up @@ -569,8 +560,6 @@ mod test {
#[test_case( "\"%()s\"",PercentFormatPart::new(Some(String::new()), None, None, None, "s".to_string()); "empty paren")]
#[test_case( "\"%(hi)s\"",PercentFormatPart::new(Some("hi".to_string()), None, None, None, "s".to_string()); "word in paren")]
#[test_case( "\"%s\"",PercentFormatPart::new(None, None, None, None, "s".to_string()); "format s")]
// #[test_case( "\"%%\"",PercentFormatPart::new(None, None, None, None, "%".to_string());
// "format double percentage")]
#[test_case( "\"%a\"",PercentFormatPart::new(None, None, None, None, "a".to_string()); "format an a")]
#[test_case( "\"%r\"",PercentFormatPart::new(None, None, None, None, "r".to_string()); "format an r")]
fn test_parse_percent_format(sample: &str, expected: PercentFormatPart) {
Expand Down
Loading