Tests fail when upgrading from ruff 0.4.8 to 0.4.10

[eli-schwartz]

Attempting to package ruff 0.4.9 and later for Gentoo, I hit the following test failures.

    Finished release [optimized] target(s) in 19m 03s
     Running `/var/tmp/portage/dev-util/ruff-0.4.10/work/ruff-0.4.10/target/release/deps/red_knot-60a25f0ffe9edd60`

running 75 tests
test files::tests::four_files ... ok
test files::tests::insert_path_twice_same_id ... ok
test module::tests::first_party_module ... ok
test files::tests::insert_different_paths_different_ids ... ok
test module::tests::module_search_path_priority ... ok
test module::tests::regular_package_in_namespace_package ... ok
test module::tests::package_priority_over_module ... ok
test module::tests::resolve_package ... ok
test module::tests::resolve_dependency ... ok
test module::tests::first_party_precedence_over_stdlib ... ok
test semantic::symbol_table::tests::add_child_scope_with_symbol ... ok
test semantic::symbol_table::tests::insert_different_named_symbols ... ok
test semantic::symbol_table::tests::bigger_symbol_table ... ok
test semantic::symbol_table::tests::insert_same_name_symbol_twice ... ok
test module::tests::typeshed_zip_created_at_build_time ... ok
test semantic::symbol_table::tests::scope_from_id ... ok
test semantic::symbol_table::tests::symbol_from_id ... ok
test semantic::tests::assign ... ok
test semantic::tests::annotation_only ... ok
test semantic::tests::class_scope ... ok
test semantic::tests::empty ... ok
test semantic::tests::dupes ... ok
test semantic::tests::func_scope ... ok
test semantic::tests::expression_scope ... FAILED
test semantic::tests::generic_class ... ok
test semantic::tests::generic_func ... ok
test semantic::tests::import ... ok
test semantic::tests::import_as ... ok
test semantic::tests::import_from ... ok
test semantic::tests::import_sub ... ok
test semantic::tests::reachability_trivial ... FAILED
test semantic::tests::simple ... ok
test module::tests::namespace_package ... ok
test module::tests::stdlib ... ok
test semantic::types::infer::tests::if_elif ... FAILED
test module::tests::symlink ... ok
test semantic::types::infer::tests::follow_import_to_class ... FAILED
test semantic::types::infer::tests::if_elif_else ... FAILED
test semantic::types::infer::tests::ifexpr_nested ... ok
test module::tests::typing_stub_over_module ... ok
test semantic::types::infer::tests::ifexpr ... ok
test module::tests::sub_packages ... ok
test semantic::types::infer::tests::ifexpr_walrus ... FAILED
test semantic::types::infer::tests::join_paths ... FAILED
test semantic::types::infer::tests::ifexpr_walrus_2 ... FAILED
test semantic::types::infer::tests::literal_int_arithmetic ... FAILED
test semantic::types::infer::tests::narrow_none ... FAILED
test semantic::types::infer::tests::none ... ok
test semantic::types::infer::tests::maybe_unbound ... FAILED
test semantic::types::tests::add_class ... ok
test semantic::types::tests::add_function ... ok
test semantic::types::tests::add_intersection ... ok
test semantic::types::tests::add_union ... ok
test semantic::types::tests::flatten_intersection_one_negative_element ... ok
test semantic::types::tests::flatten_intersection_one_positive_element ... ok
test semantic::types::tests::flatten_intersection_zero_elements ... ok
test semantic::types::tests::flatten_nested_intersection ... ok
test semantic::types::tests::flatten_nested_union ... ok
test semantic::types::tests::flatten_union_one_element ... ok
test semantic::types::tests::flatten_union_zero_elements ... ok
test semantic::types::infer::tests::resolve_visible_def ... FAILED
test semantic::types::infer::tests::resolve_base_class_by_name ... FAILED
test semantic::types::infer::tests::resolve_union ... ok
test semantic::types::infer::tests::resolve_module_member ... FAILED
test semantic::types::infer::tests::resolve_method ... ok
test typeshed_versions::tests::invalid_typeshed_versions_bad_colon_number ... ok
test typeshed_versions::tests::invalid_typeshed_versions_bad_hyphen_number ... ok
test semantic::types::infer::tests::walrus ... ok
test semantic::types::infer::tests::resolve_literal ... ok
test typeshed_versions::tests::can_parse_vendored_versions_file ... ok
test typeshed_versions::tests::invalid_typeshed_versions_bad_period_number ... ok
test typeshed_versions::tests::invalid_typeshed_versions_non_identifier_modules ... ok
test typeshed_versions::tests::invalid_typeshed_versions_non_digits ... ok
test typeshed_versions::tests::invalid_huge_versions_file ... ok
test typeshed_versions::tests::can_parse_mock_versions_file ... ok

failures:

---- semantic::tests::expression_scope stdout ----
thread 'semantic::tests::expression_scope' panicked at crates/red_knot/src/semantic/symbol_table.rs:243:31:
index out of bounds: the len is 0 but the index is 1
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

---- semantic::tests::reachability_trivial stdout ----
thread 'semantic::tests::reachability_trivial' panicked at crates/red_knot/src/semantic/flow_graph.rs:59:28:
index out of bounds: the len is 0 but the index is 2

---- semantic::types::infer::tests::if_elif stdout ----
thread 'semantic::types::infer::tests::if_elif' panicked at crates/red_knot/src/semantic/flow_graph.rs:59:28:
index out of bounds: the len is 0 but the index is 11

---- semantic::types::infer::tests::follow_import_to_class stdout ----
thread 'semantic::types::infer::tests::follow_import_to_class' panicked at crates/red_knot/src/semantic/flow_graph.rs:59:28:
index out of bounds: the len is 0 but the index is 0

---- semantic::types::infer::tests::if_elif_else stdout ----
thread 'semantic::types::infer::tests::if_elif_else' panicked at crates/red_knot/src/semantic/flow_graph.rs:59:28:
index out of bounds: the len is 0 but the index is 17

---- semantic::types::infer::tests::ifexpr_walrus stdout ----
thread 'semantic::types::infer::tests::ifexpr_walrus' panicked at crates/red_knot/src/semantic/flow_graph.rs:59:28:
index out of bounds: the len is 0 but the index is 12

---- semantic::types::infer::tests::join_paths stdout ----
thread 'semantic::types::infer::tests::join_paths' panicked at crates/red_knot/src/semantic/flow_graph.rs:59:28:
index out of bounds: the len is 0 but the index is 7

---- semantic::types::infer::tests::ifexpr_walrus_2 stdout ----
thread 'semantic::types::infer::tests::ifexpr_walrus_2' panicked at crates/red_knot/src/semantic/flow_graph.rs:59:28:
index out of bounds: the len is 0 but the index is 10

---- semantic::types::infer::tests::literal_int_arithmetic stdout ----
thread 'semantic::types::infer::tests::literal_int_arithmetic' panicked at crates/red_knot/src/semantic/flow_graph.rs:59:28:
index out of bounds: the len is 0 but the index is 5

---- semantic::types::infer::tests::narrow_none stdout ----
thread 'semantic::types::infer::tests::narrow_none' panicked at crates/red_knot/src/semantic/flow_graph.rs:59:28:
index out of bounds: the len is 0 but the index is 12

---- semantic::types::infer::tests::maybe_unbound stdout ----
thread 'semantic::types::infer::tests::maybe_unbound' panicked at crates/red_knot/src/semantic/flow_graph.rs:59:28:
index out of bounds: the len is 0 but the index is 3

---- semantic::types::infer::tests::resolve_visible_def stdout ----
thread 'semantic::types::infer::tests::resolve_visible_def' panicked at crates/red_knot/src/semantic/flow_graph.rs:59:28:
index out of bounds: the len is 0 but the index is 4

---- semantic::types::infer::tests::resolve_base_class_by_name stdout ----
thread 'semantic::types::infer::tests::resolve_base_class_by_name' panicked at crates/red_knot/src/semantic/flow_graph.rs:59:28:
index out of bounds: the len is 0 but the index is 0

---- semantic::types::infer::tests::resolve_module_member stdout ----
thread 'semantic::types::infer::tests::resolve_module_member' panicked at crates/red_knot/src/semantic/flow_graph.rs:59:28:
index out of bounds: the len is 0 but the index is 1


failures:
    semantic::tests::expression_scope
    semantic::tests::reachability_trivial
    semantic::types::infer::tests::follow_import_to_class
    semantic::types::infer::tests::if_elif
    semantic::types::infer::tests::if_elif_else
    semantic::types::infer::tests::ifexpr_walrus
    semantic::types::infer::tests::ifexpr_walrus_2
    semantic::types::infer::tests::join_paths
    semantic::types::infer::tests::literal_int_arithmetic
    semantic::types::infer::tests::maybe_unbound
    semantic::types::infer::tests::narrow_none
    semantic::types::infer::tests::resolve_base_class_by_name
    semantic::types::infer::tests::resolve_module_member
    semantic::types::infer::tests::resolve_visible_def

test result: FAILED. 61 passed; 14 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.09s

error: test failed, to rerun pass `-p red_knot --lib`
 * ERROR: dev-util/ruff-0.4.10::gentoo failed (test phase):
 *   cargo test failed

$ rustc --version
rustc 1.77.1 (7cf61ebde 2024-03-27) (gentoo)

[MichaReiser]

Oh, we got reports for this earlier but we haven't been able to reproduce but I'm able to reproduce now thanks to you sharing the build command. This is a problem with running tests in release builds!

The good news (for you) is that the red_knot code isn't part of ruff. So you can safely ignore these tests for now.

[eli-schwartz]

Thanks. I backported the fix as I wasn't sure how to guarantee all tests for crates that get used in the final binary are run.