Next Metricfy

Building a RAG Server with a Vector Database in Databricks-Part 2

—

by

richardchi3565
in

Overview

Here’s the high-level architecture we’ll build the rest of them:

▼ Model Register-Part 2

▼ Model Deployment -Part 2

Model Register

Navigate to the Artifacts section of your MLflow run, select the model folder, and click Register Model. Choose the Unity Catalog as the destination.

Model Deploy for Serving

  • Once registered in UC, use the databricks.agents.deploy() function to deploy the agent to a Mosaic AI Model Serving endpoint.
  • This generates a REST API endpoint and can automatically create a Review App for stakeholders to provide feedback.
  • Or, go to Serving in Databricks left panel and [Create serving endpoint] button to start generating a REST API

REST API Testing

REST API can be tested by a Curl command line

These examples assume you have the environment variables ${DATABRICKS_WORKSPACE_ID} and ${DATABRICKS_TOKEN} set for your workspace URL and a personal access token respectively. 

%sh curl --request POST ^
  --url https://${DATABRICKS_WORKSPACE_ID}.cloud.databricks.com/serving-endpoints/ai-agent/invocations ^
  --header "Authorization: Bearer ${DATABRICKS_TOKEN}" ^
  --header "content-type: application/json"^
  --data "{\"input\": [{ \"role\": \"user\", \"content\": \"hi\" }]}"

Comments

Leave a Reply