Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

compute = 'local' doesn't work in command() or pipeline job #37867

Open
trancenoid opened this issue Oct 12, 2024 · 2 comments
Open

compute = 'local' doesn't work in command() or pipeline job #37867

trancenoid opened this issue Oct 12, 2024 · 2 comments
Labels
Client This issue points to a problem in the data-plane of the library. customer-reported Issues that are reported by GitHub users external to the Azure organization. Machine Learning needs-team-attention Workflow: This issue needs attention from Azure service team or SDK team question The issue doesn't require a change to the product in order to be resolved. Most issues start as that Service Attention Workflow: This issue is responsible by Azure service team.

Comments

@trancenoid
Copy link

trancenoid commented Oct 12, 2024

  • Package Name: azure-ai-ml
  • Package Version: 1.21.0
  • Operating System: Windows 10
  • Python Version: 3.10

Describe the bug
Using local compute throws error The value provided is not a valid enum value. Field: queueSettings.jobTier. Allowed values: Spot,Basic,Standard,Premium.
To Reproduce
Steps to reproduce the behavior:

from azure.identity import DefaultAzureCredential

from azure.ai.ml import MLClient
from azure.ai.ml import command
from azure.ai.ml import Input, Output


ml_client = MLClient.from_config(DefaultAzureCredential(), path="config.json")

compute = 'local'
env = ml_client.environments.get('some-registered-env', version="6")

load_data = command(
    name="Download-Data-local1",
    display_name="Create Iris Dataset",
    description="Download and Register Data",
    is_deterministic=False, # to force rerun the step
    inputs={

        "url": Input(
            path='https://gist.github.com/netj/8836201/raw/6f9306ad21398ea43cba4f7d537619d0e07d5ae3/iris.csv',
            type="uri_file"),
    },
    outputs={
        "output_path": Output(type="uri_file", mode='rw_mount'), # a directory will be attached to the compute
    },
    # The source folder of the component
    code="tmp",
    command="""python data_prep.py \
            --url ${{inputs.url}} \
            --output_path ${{outputs.output_path}}
            """,
    environment=f"{env.name}:{env.version}",
    compute=compute
)

returned_job = ml_client.jobs.create_or_update(load_data, compute='local')
returned_job

tmp/data_prep.py :

import argparse

__all__ = ['df', 'load_data']

# %% ../nbs/experiments/data_prep_azmlish.ipynb 1
import pandas as pd
import numpy as np

# %% ../nbs/experiments/data_prep_azmlish.ipynb 3
def load_data(url):
    return pd.read_csv(url)

# Parse command-line arguments
parser = argparse.ArgumentParser(description='Data preparation script')
parser.add_argument('--url', type=str, required=True, help='URL of the CSV file to load')
parser.add_argument('--output_path', type=str, required=True, help='Path to save the output CSV file')
args = parser.parse_args()

# %% ../nbs/experiments/data_prep_azmlish.ipynb 4
df = load_data(args.url)

# %% ../nbs/experiments/data_prep_azmlish.ipynb 8
df.to_csv(args.output_path, index=False)

Expected behavior
The command should run as when using compute = 'cpu-cluster'

Screenshots
If applicable, add screenshots to help explain your problem.

Additional context
The exact code snippet works on Azure ML Compute.

@github-actions github-actions bot added Client This issue points to a problem in the data-plane of the library. customer-reported Issues that are reported by GitHub users external to the Azure organization. Machine Learning needs-team-attention Workflow: This issue needs attention from Azure service team or SDK team question The issue doesn't require a change to the product in order to be resolved. Most issues start as that Service Attention Workflow: This issue is responsible by Azure service team. labels Oct 12, 2024
Copy link

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @Azure/azure-ml-sdk @azureml-github.

@trancenoid
Copy link
Author

I tested version 1.16.1 and it works, however I would keep the issue open since the default version is 1.21.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Client This issue points to a problem in the data-plane of the library. customer-reported Issues that are reported by GitHub users external to the Azure organization. Machine Learning needs-team-attention Workflow: This issue needs attention from Azure service team or SDK team question The issue doesn't require a change to the product in order to be resolved. Most issues start as that Service Attention Workflow: This issue is responsible by Azure service team.
Projects
None yet
Development

No branches or pull requests

1 participant