Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: 1776 add flex features to list #1780

Merged
merged 19 commits into from
Sep 9, 2024
Merged

Conversation

qcdyx
Copy link
Contributor

@qcdyx qcdyx commented Jun 17, 2024

Summary:

Closes #1776

Expected behavior:
I discovered that the primary reason most of the stop_times.txt files in flex feeds are unparsable is the absence of the required stop_id field. So to test flex features, I commented @required annotation in the GtfsStopTimesSchema and generated the following report. We need to discuss the logic behind the requirement of the stop_id field for non-flex feeds vs. flex feeds.
(I cannot find a flex feed that meets Deviated Fixed Route criteria)

image
Please make sure these boxes are checked before submitting your pull request - thanks!

  • Run the unit tests with gradle test to make sure you didn't break anything
  • Add or update any needed documentation to the repo
  • Format the title like "feat: [new feature short description]". Title must follow the Conventional Commit Specification(https://www.conventionalcommits.org/en/v1.0.0/).
  • Linked all relevant issues
  • Include screenshot(s) showing how this pull request works and fixes the issue(s)

@qcdyx qcdyx linked an issue Jun 17, 2024 that may be closed by this pull request
Copy link
Contributor

📝 Acceptance Test Report

📋 Summary

✅ The rule acceptance has passed for commit bcde487
Download the full acceptance test report here (report will disappear after 90 days).

📊 Notices Comparison

New Errors (0 out of 1524 datasets, ~0%) ✅

No changes were detected due to the code change.

Dropped Errors (0 out of 1524 datasets, ~0%) ✅

No changes were detected due to the code change.

New Warnings (0 out of 1524 datasets, ~0%) ✅

No changes were detected due to the code change.

Dropped Warnings (0 out of 1524 datasets, ~0%) ✅

No changes were detected due to the code change.

🛡️ Corruption Check

0 out of 1524 sources (~0 %) are corrupted.

⏱️ Performance Assessment

📈 Validation Time

Assess the performance in terms of seconds taken for the validation process.

Time Metric Dataset ID Reference (s) Latest (s) Difference (s)
Average -- 3.97 4.12 ⬆️+0.14
Median -- 1.35 1.45 ⬆️+0.10
Standard Deviation -- 11.13 11.30 ⬆️+0.18
Minimum in References Reports us-massachusetts-massachusetts-area-express-max-gtfs-431 0.51 0.60 ⬆️+0.09
Maximum in Reference Reports gb-unknown-uk-aggregate-feed-gtfs-2014 291.99 296.16 ⬆️+4.17
Minimum in Latest Reports us-michigan-detroit-people-mover-gtfs-417 0.52 0.54 ⬆️+0.02
Maximum in Latest Reports gb-unknown-uk-aggregate-feed-gtfs-2014 291.99 296.16 ⬆️+4.17

@emmambd
Copy link
Contributor

emmambd commented Jun 17, 2024

I think we have to wait to review/merge this until some other schemas are added and tested, e.g #1750

@emmambd
Copy link
Contributor

emmambd commented Jun 17, 2024

stop_id being required changes as part of #1749

@emmambd emmambd added the do not merge This PR needs more work/discussion or is not meant to be merged label Jun 17, 2024
@emmambd emmambd linked an issue Jun 17, 2024 that may be closed by this pull request
@davidgamez
Copy link
Member

I think we have to wait to review/merge this until some other schemas are added and tested, e.g #1750

I agree, even if we merge flex features won't be visible until the conditional requirement is added to the schema.

web/client/static/RULES.md Outdated Show resolved Hide resolved
…/FeedMetadata.java

Co-authored-by: cka-y <60586858+cka-y@users.noreply.github.com>
Copy link
Contributor

This contribution does not follow the conventions set by the Google Java style guide. Please run the following command line at the root of the project to fix formatting errors: ./gradlew goJF.

Copy link
Contributor

This contribution does not follow the conventions set by the Google Java style guide. Please run the following command line at the root of the project to fix formatting errors: ./gradlew goJF.

Copy link
Contributor

📝 Acceptance Test Report

📋 Summary

✅ The rule acceptance has passed for commit 15a7c4f
Download the full acceptance test report here (report will disappear after 90 days).

📊 Notices Comparison

New Errors (0 out of 1524 datasets, ~0%) ✅

No changes were detected due to the code change.

Dropped Errors (0 out of 1524 datasets, ~0%) ✅

No changes were detected due to the code change.

New Warnings (0 out of 1524 datasets, ~0%) ✅

No changes were detected due to the code change.

Dropped Warnings (0 out of 1524 datasets, ~0%) ✅

No changes were detected due to the code change.

🛡️ Corruption Check

0 out of 1524 sources (~0 %) are corrupted.

⏱️ Performance Assessment

📈 Validation Time

Assess the performance in terms of seconds taken for the validation process.

Time Metric Dataset ID Reference (s) Latest (s) Difference (s)
Average -- 3.93 4.05 ⬆️+0.13
Median -- 1.37 1.45 ⬆️+0.08
Standard Deviation -- 10.75 10.93 ⬆️+0.18
Minimum in References Reports ph-unknown-hm-transport-inc-and-robinsons-malls-gtfs-1105 0.49 0.58 ⬆️+0.09
Maximum in Reference Reports gb-unknown-uk-aggregate-feed-gtfs-2014 279.53 286.13 ⬆️+6.60
Minimum in Latest Reports us-california-city-of-wasco-gtfs-1788 0.56 0.53 ⬇️-0.03
Maximum in Latest Reports gb-unknown-uk-aggregate-feed-gtfs-2014 279.53 286.13 ⬆️+6.60

@qcdyx qcdyx requested a review from jcpitre September 9, 2024 15:43
Copy link
Contributor

github-actions bot commented Sep 9, 2024

📝 Acceptance Test Report

📋 Summary

❌ The rule acceptance test has failed for commit 1580bb0
Download the full acceptance test report here (report will disappear after 90 days).

📊 Notices Comparison

New Errors (18 out of 1575 datasets, ~1%) ❌

Details of new errors due to code change, which is above the provided threshold of 1%.

Dataset Notice Code
us-colorado-el-paso-fountain-valley-senior-citizens-program-inc-gtfs-2046 invalid_url
de-unknown-postauto-ag-gtfs-2053 missing_trip_edge
us-chicago-brown-county-heartland-express-gtfs-2037 missing_trip_edge
us-chicago-the-otter-express-gtfs-2038 missing_trip_edge
us-colorado-all-points-transit-gtfs-2039 missing_trip_edge
us-colorado-avon-transit-gtfs-2040 missing_trip_edge
us-colorado-baca-area-transportation-gtfs-1087 missing_trip_edge
us-colorado-cripple-creek-transportation-gtfs-2042 missing_trip_edge
us-colorado-discover-goodwill-gtfs-2043 missing_trip_edge
us-colorado-dolores-county-gtfs-1141 missing_trip_edge
us-colorado-easy-ride-transportation-gtfs-1195 missing_trip_edge
us-colorado-eco-transit-gtfs-2045 missing_trip_edge
us-colorado-el-paso-fountain-valley-senior-citizens-program-inc-gtfs-2046 missing_trip_edge
us-colorado-envida-gtfs-2044 missing_trip_edge
us-colorado-gunnison-valley-rta-gtfs-2048 missing_trip_edge
us-colorado-pueblo-transit-gtfs-1232 missing_trip_edge
us-colorado-steamboat-springs-transit-gtfs-2051 missing_trip_edge
us-colorado-town-of-telluride-gtfs-2050 missing_trip_edge
us-colorado-tsc-transit-gtfs-2052 missing_trip_edge
Dropped Errors (21 out of 1575 datasets, ~1%) ❌

Details of new errors due to code change, which is above the provided threshold of 1%.

Dataset Notice Code
de-unknown-postauto-ag-gtfs-2053 missing_required_field
pl-podkarpackie-komunikacja-miejska-przemysl-gtfs-1000 missing_required_field
us-chicago-brown-county-heartland-express-gtfs-2037 missing_required_field
us-chicago-the-otter-express-gtfs-2038 missing_required_field
us-colorado-all-points-transit-gtfs-2039 missing_required_field
us-colorado-avon-transit-gtfs-2040 missing_required_field
us-colorado-baca-area-transportation-gtfs-1087 missing_required_field
us-colorado-blackhawk-and-central-city-tramway-gtfs-2041 missing_required_field
us-colorado-cripple-creek-transportation-gtfs-2042 missing_required_field
us-colorado-discover-goodwill-gtfs-2043 missing_required_field
us-colorado-dolores-county-gtfs-1141 missing_required_field
us-colorado-easy-ride-transportation-gtfs-1195 missing_required_field
us-colorado-eco-transit-gtfs-2045 missing_required_field
us-colorado-el-paso-fountain-valley-senior-citizens-program-inc-gtfs-2046 missing_required_field
us-colorado-envida-gtfs-2044 missing_required_field
us-colorado-estes-transit-gtfs-2047 missing_required_field
us-colorado-gunnison-valley-rta-gtfs-2048 missing_required_field
us-colorado-pueblo-transit-gtfs-1232 missing_required_field
us-colorado-steamboat-springs-transit-gtfs-2051 missing_required_field
us-colorado-town-of-telluride-gtfs-2050 missing_required_field
us-colorado-tsc-transit-gtfs-2052 missing_required_field
New Warnings (21 out of 1575 datasets, ~1%) ❌

Details of new errors due to code change, which is above the provided threshold of 1%.

Dataset Notice Code
us-chicago-brown-county-heartland-express-gtfs-2037 fast_travel_between_far_stops
us-colorado-blackhawk-and-central-city-tramway-gtfs-2041 fast_travel_between_far_stops
us-colorado-estes-transit-gtfs-2047 fast_travel_between_far_stops
us-chicago-brown-county-heartland-express-gtfs-2037 leading_or_trailing_whitespaces
us-chicago-the-otter-express-gtfs-2038 leading_or_trailing_whitespaces
us-chicago-brown-county-heartland-express-gtfs-2037 stop_too_far_from_shape
us-colorado-all-points-transit-gtfs-2039 stop_too_far_from_shape
us-colorado-baca-area-transportation-gtfs-1087 stop_too_far_from_shape
us-colorado-blackhawk-and-central-city-tramway-gtfs-2041 stop_too_far_from_shape
us-colorado-estes-transit-gtfs-2047 stop_too_far_from_shape
us-colorado-tsc-transit-gtfs-2052 stop_too_far_from_shape
de-unknown-postauto-ag-gtfs-2053 stop_without_stop_time
pl-podkarpackie-komunikacja-miejska-przemysl-gtfs-1000 stop_without_stop_time
us-chicago-brown-county-heartland-express-gtfs-2037 stop_without_stop_time
us-chicago-the-otter-express-gtfs-2038 stop_without_stop_time
us-colorado-all-points-transit-gtfs-2039 stop_without_stop_time
us-colorado-avon-transit-gtfs-2040 stop_without_stop_time
us-colorado-baca-area-transportation-gtfs-1087 stop_without_stop_time
us-colorado-cripple-creek-transportation-gtfs-2042 stop_without_stop_time
us-colorado-discover-goodwill-gtfs-2043 stop_without_stop_time
us-colorado-dolores-county-gtfs-1141 stop_without_stop_time
us-colorado-easy-ride-transportation-gtfs-1195 stop_without_stop_time
us-colorado-eco-transit-gtfs-2045 stop_without_stop_time
us-colorado-el-paso-fountain-valley-senior-citizens-program-inc-gtfs-2046 stop_without_stop_time
us-colorado-envida-gtfs-2044 stop_without_stop_time
us-colorado-estes-transit-gtfs-2047 stop_without_stop_time
us-colorado-gunnison-valley-rta-gtfs-2048 stop_without_stop_time
us-colorado-pueblo-transit-gtfs-1232 stop_without_stop_time
us-colorado-steamboat-springs-transit-gtfs-2051 stop_without_stop_time
us-colorado-town-of-telluride-gtfs-2050 stop_without_stop_time
us-colorado-envida-gtfs-2044 trip_distance_exceeds_shape_distance_below_threshold
us-colorado-steamboat-springs-transit-gtfs-2051 trip_distance_exceeds_shape_distance_below_threshold
Dropped Warnings (0 out of 1575 datasets, ~0%) ✅

No changes were detected due to the code change.

🛡️ Corruption Check

0 out of 1575 sources (~0 %) are corrupted.

⏱️ Performance Assessment

📈 Validation Time

Assess the performance in terms of seconds taken for the validation process.

Time Metric Dataset ID Reference (s) Latest (s) Difference (s)
Average -- 3.89 4.01 ⬆️+0.12
Median -- 1.39 1.48 ⬆️+0.09
Standard Deviation -- 10.96 11.18 ⬆️+0.22
Minimum in References Reports us-oregon-high-desert-point-gtfs-636 0.53 0.62 ⬆️+0.09
Maximum in Reference Reports gb-unknown-uk-aggregate-feed-gtfs-2014 312.29 320.37 ⬆️+8.08
Minimum in Latest Reports us-massachusetts-massachusetts-area-express-max-gtfs-431 0.56 0.54 ⬇️-0.02
Maximum in Latest Reports gb-unknown-uk-aggregate-feed-gtfs-2014 312.29 320.37 ⬆️+8.08

@emmambd
Copy link
Contributor

emmambd commented Sep 9, 2024

@qcdyx I'm confused why this is so significantly impacting the acceptance tests - I thought @jcpitre's comment was related to how we're calculating the features.

Can we talk about this in more depth, how we split up the work, before we merge?

web/client/.gitignore Outdated Show resolved Hide resolved
@qcdyx qcdyx merged commit 014bac0 into master Sep 9, 2024
333 checks passed
@qcdyx qcdyx deleted the 1776-add-flex-features-to-list branch September 9, 2024 18:00
Copy link
Contributor

github-actions bot commented Sep 9, 2024

📝 Acceptance Test Report

📋 Summary

✅ The rule acceptance has passed for commit 7287f8d
Download the full acceptance test report here (report will disappear after 90 days).

📊 Notices Comparison

New Errors (1 out of 1575 datasets, ~0%) ✅

Details of new errors due to code change, which is less than the provided threshold of 1%.

Dataset Notice Code
us-colorado-el-paso-fountain-valley-senior-citizens-program-inc-gtfs-2046 invalid_url
Dropped Errors (0 out of 1575 datasets, ~0%) ✅

No changes were detected due to the code change.

New Warnings (2 out of 1575 datasets, ~0%) ✅

Details of new errors due to code change, which is less than the provided threshold of 1%.

Dataset Notice Code
us-chicago-brown-county-heartland-express-gtfs-2037 leading_or_trailing_whitespaces
us-chicago-the-otter-express-gtfs-2038 leading_or_trailing_whitespaces
Dropped Warnings (0 out of 1575 datasets, ~0%) ✅

No changes were detected due to the code change.

🛡️ Corruption Check

0 out of 1575 sources (~0 %) are corrupted.

⏱️ Performance Assessment

📈 Validation Time

Assess the performance in terms of seconds taken for the validation process.

Time Metric Dataset ID Reference (s) Latest (s) Difference (s)
Average -- 3.87 4.00 ⬆️+0.12
Median -- 1.38 1.46 ⬆️+0.09
Standard Deviation -- 10.97 11.08 ⬆️+0.11
Minimum in References Reports us-california-city-of-wasco-gtfs-1788 0.50 0.60 ⬆️+0.10
Maximum in Reference Reports gb-unknown-uk-aggregate-feed-gtfs-2014 312.98 316.71 ⬆️+3.72
Minimum in Latest Reports us-california-catalina-express-gtfs-299 0.63 0.55 ⬇️-0.08
Maximum in Latest Reports gb-unknown-uk-aggregate-feed-gtfs-2014 312.98 316.71 ⬆️+3.72

@jcpitre jcpitre removed the do not merge This PR needs more work/discussion or is not meant to be merged label Sep 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add Flex features to list Flex: Add booking rule schema
5 participants