Workflow Steps

Step 1: Adding Data to Vaults

Option A: From the Data Hub

1. Browse Data Hub:

• Navigate to the Data Hub within the Sahara platform.

• Explore available datasets by filtering by domain, size, or licensing terms. (Some explore features coming in Beta)

1. Purchase Dataset:

• Select a dataset and click Purchase.

• Complete the payment process using the supported token or currency.

1. Add Dataset to Vault:

• A Screen will pop up and you will select the Vault you want to import to and click Import.

• Navigate to your Vaults Tab and see the imported data set.

Option B: Upload Personal Data

1. Navigate to Vaults:

• Go to the Vaults Tab in the Sahara dashboard.

1. Create a New Vault:

• Click Create Vault and provide a name and description.

1. Upload Data:

• Click Upload Files and select files from your local storage.

• Supported formats include CSV, JSON, and Parquet.

1. Configure Metadata:

• Add metadata tags for better discoverability and provenance tracking.

Step 2: Setting Up an Endpoint in the Compute Hub

1. Access My Submissions:

• Open the My Submissions on the Sahara dashboard after you purchase a model.

1. Initiate:

• Click the New Endpoint button on the My Submissions home page.

1. Configuration:

• Fill out the form with the following details:

• Select Provider: Select Provider from list (Lepton, Predibase, Sagemaker, Bedrock, OpenAI)

• Select Model: Opens a popup and allows users to select from available models on platform. Search and press “Select” when you have chosen a model.

• Name: Assign a unique name to the instance.

1. Review and Create:

• Review the configuration.

• Click Create Instance to launch the instance.

Step 3: Creating and Deploying Pipelines

A. Create a Pipeline in Model Hub

1. Open Model Hub:

• Navigate to the Model Hub Tab.

1. Click Create Pipeline:

• Choose between:

• RAG Pipeline: Requires Vault integration for retrieval tasks.

• Prompt-Based Pipeline: For conversational or structured AI tasks.

1. Configure Pipeline:

• Upload an avatar for the pipeline.

• Provide the following:

• Pipeline Name.

• Description.

• Instructions (prompt).

• Conversation Starter (optional).

• Select an AI model.

• (RAG only) Choose Vaults for data retrieval.

B. Deploy the Pipeline

1. Publish Pipeline:

• Click Publish and select a compute provider matching your pipeline’s requirements.

1. Generate API Endpoint:

• Upon deployment, an API endpoint is generated for integration.

Step 4: Integration and Monitoring

A. Test the Endpoint

1. Test the API:

• Use the generated endpoint to send sample requests.

Example payload:

• Review the AI response and refine configurations as needed.

B. Monitor Metrics

1. Open Dashboard Tab:

• Navigate to the Metrics Tab in your pipeline’s details page.

1. Review Usage:

• Monitor token usage, API calls, and system performance metrics.