Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[pyupgrade] Implement import-replacement rule (UP035) #2049

Merged
merged 50 commits into from
Jan 31, 2023
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
50 commits
Select commit Hold shift + click to select a range
cf9155f
Basic groundwork laid
colin99d Jan 20, 2023
6b5c4aa
Going to WIPE libcst
colin99d Jan 21, 2023
04ac882
Fixed conversion
colin99d Jan 21, 2023
9ac1642
Need to switch
colin99d Jan 21, 2023
34b28fc
Merged
colin99d Jan 21, 2023
ac38462
Better indentation tests
colin99d Jan 21, 2023
1c1d391
Fixed second from indentation issue
colin99d Jan 21, 2023
7035102
Fixed fmt and nightly
colin99d Jan 21, 2023
51f6d5c
Added a bunch more things to replace
colin99d Jan 21, 2023
1980519
Added some fixes
colin99d Jan 21, 2023
1fbd14c
Fixed a bug in formatting
colin99d Jan 21, 2023
b7016fa
Fixed formatting error
colin99d Jan 21, 2023
0448418
Fixed formatting errors
colin99d Jan 21, 2023
3bf08dd
Got base tests going
colin99d Jan 21, 2023
b0d2456
Got the mods in
colin99d Jan 21, 2023
860a1d1
Finished up a rough draft, still has the libcst bug
colin99d Jan 21, 2023
4a8263a
Fixed some formatting
colin99d Jan 21, 2023
08c2174
Fixed clippy
colin99d Jan 21, 2023
8e98545
Merge branch 'main' into ImportReplacements
colin99d Jan 22, 2023
6b4d68a
STARTED TRANSITION FROM LIBCST TO CUSTOM
colin99d Jan 22, 2023
30a7180
An MVP with no known bugs
colin99d Jan 22, 2023
f811a84
Added more tests for six.moves
colin99d Jan 22, 2023
b326a29
Handled one last issue
colin99d Jan 22, 2023
af1af04
Fixed some more stuff
colin99d Jan 22, 2023
e8a55a7
Added some fixes
colin99d Jan 22, 2023
679a134
Fixed clippy
colin99d Jan 22, 2023
fd8f418
Deleted unused snapshots
colin99d Jan 22, 2023
ac95e2c
Fixed typos
colin99d Jan 22, 2023
e9bd812
Fixed typos
colin99d Jan 22, 2023
943585c
Added some tests
colin99d Jan 23, 2023
42ae4c5
Fixed tests
colin99d Jan 23, 2023
50c85ac
Merge branch 'main' into ImportReplacements
colin99d Jan 26, 2023
04cd251
Recommended changes
colin99d Jan 26, 2023
65cc26a
Merge branch 'main' into ImportReplacements
colin99d Jan 27, 2023
9d67f07
Merge branch 'main' into ImportReplacements
charliermarsh Jan 29, 2023
603c10b
Rewrite parts of the check
charliermarsh Jan 30, 2023
f7cbd58
Removed six stuff
colin99d Jan 30, 2023
880ebca
Merge branch 'ImportReplacements' of https://github.com/colin99d/ruff…
colin99d Jan 30, 2023
91c38b6
Merge branch 'main' into ImportReplacements
colin99d Jan 30, 2023
0c0a098
removed six
colin99d Jan 30, 2023
3d820f0
Merge branch 'ImportReplacements' of https://github.com/colin99d/ruff…
colin99d Jan 30, 2023
fd53580
Merge branch 'main' into ImportReplacements
charliermarsh Jan 30, 2023
c071895
Merge branch 'charlie/import-replacements' into ImportReplacements
charliermarsh Jan 30, 2023
ef4d532
Small refactors; add one failing test
charliermarsh Jan 30, 2023
2af94db
Avoid bad multi-line fix
charliermarsh Jan 30, 2023
cd7ea96
Merge branch 'main' into ImportReplacements
charliermarsh Jan 30, 2023
1cb84fc
Always flag
charliermarsh Jan 30, 2023
53d6b1f
Flag multi
charliermarsh Jan 30, 2023
dad535f
Use pyupgrade-like token based removal
charliermarsh Jan 30, 2023
f17c2ff
Merge branch 'main' into ImportReplacements
charliermarsh Jan 30, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Got the mods in
  • Loading branch information
colin99d committed Jan 21, 2023
commit b0d24566d96b608f2f8d7230717cbeff7313b887
1 change: 1 addition & 0 deletions resources/test/fixtures/pyupgrade/UP035_1.py
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
from six.moves import tkinter, lololol
3 changes: 3 additions & 0 deletions src/checkers/ast.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1041,6 +1041,9 @@ where
if self.settings.rules.enabled(&Rule::ImportReplacements) {
pyupgrade::rules::import_replacements(self, stmt, names, module);
}
if self.settings.rules.enabled(&Rule::ImportReplacementsSix) {
pyupgrade::rules::import_replacements_six(self, stmt, module);
}
if self.settings.rules.enabled(&Rule::UnnecessaryBuiltinImport) {
if let Some(module) = module.as_deref() {
pyupgrade::rules::unnecessary_builtin_import(self, stmt, module, names);
Expand Down
1 change: 1 addition & 0 deletions src/registry.rs
Original file line number Diff line number Diff line change
Expand Up @@ -256,6 +256,7 @@ ruff_macros::define_rule_mapping!(
UP033 => violations::FunctoolsCache,
UP034 => violations::ExtraneousParentheses,
UP035 => violations::ImportReplacements,
UP036 => violations::ImportReplacementsSix,
// pydocstyle
D100 => violations::PublicModule,
D101 => violations::PublicClass,
Expand Down
4 changes: 2 additions & 2 deletions src/rules/pyupgrade/rules/import_replacements.rs
Original file line number Diff line number Diff line change
Expand Up @@ -182,6 +182,7 @@ const TYPINGEXTENSIONS_TO_TYPING_311: &[&str] = &[
"reveal_type",
];


fn has_match(set1: &[&str], set2: &[AliasData]) -> bool {
set2.iter().any(|x| set1.contains(&x.name.as_str()))
}
Expand Down Expand Up @@ -319,10 +320,9 @@ impl<'a> FixImports<'a> {
let (matching_names, unmatching_names) = self.get_import_lists(matches);
let unmatching = self.get_str(&unmatching_names, self.module);
let matching = self.get_str(&matching_names, replace);
// We don't replace if there is just an unmatching, because then we don't need to refactor
if !unmatching.is_empty() && !matching.is_empty() {
Some(format!("{unmatching}\n{}{matching}", self.starting_indent))
} else if !unmatching.is_empty() {
Some(unmatching)
} else if !matching.is_empty() {
Some(matching)
} else {
Expand Down
139 changes: 139 additions & 0 deletions src/rules/pyupgrade/rules/import_replacements_six.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,139 @@
use crate::ast::types::Range;
use crate::checkers::ast::Checker;
use crate::cst::matchers::{match_import, match_import_from, match_module};
use crate::fix::Fix;
use crate::registry::{Diagnostic, Rule};
use crate::violations;
use libcst_native::{
AsName, AssignTargetExpression, Codegen, CodegenState, ImportAlias, ImportNames,
NameOrAttribute,
};
use once_cell::sync::Lazy;
use rustpython_ast::Stmt;
use std::collections::HashMap;

static REPLACE_MODS: Lazy<HashMap<&str, &str>> = Lazy::new(|| {
let mut m = HashMap::new();
m.insert("BaseHTTPServer", "http.server");
colin99d marked this conversation as resolved.
Show resolved Hide resolved
m.insert("CGIHTTPServer", "http.server");
m.insert("SimpleHTTPServer", "http.server");
m.insert("_dummy_thread", "_dummy_thread");
m.insert("_thread", "_thread");
m.insert("builtins", "builtins");
m.insert("cPickle", "pickle");
m.insert("collections_abc", "collections.abc");
m.insert("configparser", "configparser");
m.insert("copyreg", "copyreg");
m.insert("dbm_gnu", "dbm.gnu");
m.insert("dbm_ndbm", "dbm.ndbm");
m.insert("email_mime_base", "email.mime.base");
m.insert("email_mime_image", "email.mime.image");
m.insert("email_mime_multipart", "email.mime.multipart");
m.insert("email_mime_nonmultipart", "email.mime.nonmultipart");
m.insert("email_mime_text", "email.mime.text");
m.insert("html_entities", "html.entities");
m.insert("html_parser", "html.parser");
m.insert("http_client", "http.client");
m.insert("http_cookiejar", "http.cookiejar");
m.insert("http_cookies", "http.cookies");
m.insert("queue", "queue");
m.insert("reprlib", "reprlib");
m.insert("socketserver", "socketserver");
m.insert("tkinter", "tkinter");
m.insert("tkinter_colorchooser", "tkinter.colorchooser");
m.insert("tkinter_commondialog", "tkinter.commondialog");
m.insert("tkinter_constants", "tkinter.constants");
m.insert("tkinter_dialog", "tkinter.dialog");
m.insert("tkinter_dnd", "tkinter.dnd");
m.insert("tkinter_filedialog", "tkinter.filedialog");
m.insert("tkinter_font", "tkinter.font");
m.insert("tkinter_messagebox", "tkinter.messagebox");
m.insert("tkinter_scrolledtext", "tkinter.scrolledtext");
m.insert("tkinter_simpledialog", "tkinter.simpledialog");
m.insert("tkinter_tix", "tkinter.tix");
m.insert("tkinter_tkfiledialog", "tkinter.filedialog");
m.insert("tkinter_tksimpledialog", "tkinter.simpledialog");
m.insert("tkinter_ttk", "tkinter.ttk");
m.insert("urllib.error", "urllib.error");
m.insert("urllib.parse", "urllib.parse");
m.insert("urllib.request", "urllib.request");
m.insert("urllib.response", "urllib.response");
m.insert("urllib.robotparser", "urllib.robotparser");
m.insert("urllib_error", "urllib.error");
m.insert("urllib_parse", "urllib.parse");
m.insert("urllib_robotparser", "urllib.robotparser");
m.insert("xmlrpc_client", "xmlrpc.client");
m.insert("xmlrpc_server", "xmlrpc.server");
m
});

fn get_asname(asname: &AsName) -> Option<String> {
if let AssignTargetExpression::Name(item) = &asname.name {
return Some(item.value.to_string());
}
None
}

/// UP036
pub fn import_replacements_six(checker: &mut Checker, stmt: &Stmt, module: &Option<String>) {
// Pyupgrade only works with import_from statements, so this linter does that as
// well

// This only applies to six.moves libraries
if let Some(module_text) = module {
if module_text != "six.moves" {
return;
}
} else {
return;
}
let module_text = checker
.locator
.slice_source_code_range(&Range::from_located(stmt));
let mut tree = match_module(&module_text).unwrap();
let mut import = match_import_from(&mut tree).unwrap();
let mut new_entries = String::new();
let mut keep_names: Vec<ImportAlias<'_>> = vec![];
if let ImportNames::Aliases(item_names) = &import.names {
for name in item_names {
if let NameOrAttribute::N(the_name) = &name.name {
match REPLACE_MODS.get(the_name.value) {
Some(raw_name) => {
new_entries.push_str(&format!("import {}", raw_name));
if let Some(asname) = &name.asname {
if let Some(final_name) = get_asname(asname) {
new_entries.push_str(&format!(" as {}", final_name));
}
}
new_entries.push('\n');
}
None => keep_names.push(name.clone()),
}
} else {
keep_names.push(name.clone())
}
}
}
// If nothing was different, there is no need to change
if new_entries.is_empty() {
return;
}
import.names = ImportNames::Aliases(keep_names);
let mut state = CodegenState::default();
import.codegen(&mut state);
let mut final_str = state.to_string();
final_str.push_str(&format!("\n{}", new_entries));
if final_str.chars().last() == Some('\n') {
final_str.pop();
}
let range = Range::from_located(stmt);
let mut diagnostic = Diagnostic::new(violations::ImportReplacementsSix, range);
if checker.patch(&Rule::ImportReplacementsSix) {
diagnostic.amend(Fix::replacement(
final_str,
stmt.location,
stmt.end_location.unwrap(),
));
}
checker.diagnostics.push(diagnostic);
}
2 changes: 2 additions & 0 deletions src/rules/pyupgrade/rules/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ pub(crate) use use_pep585_annotation::use_pep585_annotation;
pub(crate) use use_pep604_annotation::use_pep604_annotation;
pub(crate) use useless_metaclass_type::useless_metaclass_type;
pub(crate) use useless_object_inheritance::useless_object_inheritance;
pub(crate) use import_replacements_six::import_replacements_six;

use crate::ast::helpers;
use crate::ast::types::{Range, Scope, ScopeKind};
Expand Down Expand Up @@ -73,6 +74,7 @@ mod use_pep585_annotation;
mod use_pep604_annotation;
mod useless_metaclass_type;
mod useless_object_inheritance;
mod import_replacements_six;

/// UP008
pub fn super_args(
Expand Down
14 changes: 14 additions & 0 deletions src/violations.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3218,6 +3218,20 @@ impl AlwaysAutofixableViolation for ImportReplacements {
}
}

define_violation!(
pub struct ImportReplacementsSix;
);
impl AlwaysAutofixableViolation for ImportReplacementsSix {
#[derive_message_formats]
fn message(&self) -> String {
format!("Replace old formatting imports with their new versions")
}

fn autofix_title(&self) -> String {
"Updated the import".to_string()
}
}

// pydocstyle

define_violation!(
Expand Down