-
Notifications
You must be signed in to change notification settings - Fork 44
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add support for AWS CUR 2.0 format #318
Comments
Hi @stoiev, do you have ideas around on what we can do this to achieve this? |
@stoiev in case it is helpful, I was able to create a CUR2.0 billing file using the new "Data Exports" functionality in AWS. This is exposed through the "bcm-data-exports" client in boto3. One thing I did find necessary was the tool expects a column "product_region" so I renamed "product_region_code" through a manual QueryStatement definition. I am not sure how the specification expects region data so hopefully that isn't an issue but it doesn't seem to be particular enough from what I see in the PDF. So, this code block generates CUR2.0 billing files that I was able to pass to the tool without issue (apologies for any minor bugs caused by removing company-related things):
I pass the parquet files created by the Data Export to the tool as follows: |
Thanks @davidschneider2W ! I've also noticed that Instead of changing source, I've changed The migration steps recommended by AWS is here. SQL's approuch is the first one, and maybe it could be the recommended solution for now in However, we will face issues regarding data that are represented as
At least But as far as I know, differences between CUR 1 and 2 seems to be minimal. Maybe it worth to invest in a generic conversion configuration that works for both formats. PS: I'm saying that withouth know how extensible and generic conversion configurations could be. |
Hi @stoiev and @davidschneider2W this is awesome. Could anyone of you please create a pull request with the proposed steps to convert data. |
I've advanced in the idea of conversion config's compatibility for CUR 1 and 2 versions. |
Is your feature request related to a problem? Please describe.
No
Describe the solution you'd like
Support for AWS CUR 2.0 format.
Additional context
It's not clear if AWS will deprecate actual CUR format, but they are classifying it as "Legacy" in all documentation and web console.
For newer integrations, it's suitable to start with some format that does not have this adjective.
Some ideas of implementation already had been exposed here, but I think it's better to discuss this in a specific issue.
The text was updated successfully, but these errors were encountered: