Deployment and model coverage: clouds, private hosting and self-hosted models
Integrations & Data Sources
#coverage#azure#bedrock#vertex#self-hosted#on-premises#private cloud#vllm#ollama#embeddings#transcription#air-gapped
Which providers, hosting setups and model types SHDWsdk measures, how unknown models are handled, and what private-cloud and on-premises deployments need.
How It Works Under the Hood
- •Any vendor: the model vendor (OpenAI, Anthropic, Google, Mistral, Cohere, Meta, DeepSeek, Alibaba, Microsoft, Amazon, xAI, or any other id) is recorded separately from where the model ran: vendor API, Azure, AWS Bedrock, Google Vertex, self-hosted or other. Hosting is detected from OpenTelemetry gen_ai.provider.name, LiteLLM routes and Bedrock/Vertex model ids, or set explicitly.
- •Capture paths: the JavaScript and Python SDKs, LiteLLM (every provider it routes to, including Azure, Bedrock, Vertex, Ollama and vLLM), any OpenAI-compatible client, and OpenTelemetry. OTLP spans are read with the GenAI semantic conventions, so an OpenTelemetry Collector inside a private cloud can forward spans with no application changes.
- •Energy for text generation uses EcoLogits (0.11.3), including open-weight models such as Llama, Mistral, Qwen and Gemma. Models not in the catalog (fine-tunes, renamed deployments) get a wide generic range and are flagged 'fallback' so they can be reviewed. Embeddings, reranking, moderation and transcription use a documented FLOP-based estimate (shdw-flops-v1); image, speech and OCR calls are counted but not yet estimated, and are excluded from emissions rather than reported as zero.
- •Self-hosted models: set your datacenter's PUE in Settings → Developer and send a grid region or a measured gCO2e/kWh, so self-hosted calls aren't charged a cloud provider's efficiency.
- •Private cloud and on-premises: only metadata leaves your network (model id, token counts, timing, region, pseudonymous ids), never prompts or outputs, through a single outbound HTTPS endpoint. Fully air-gapped installations without any outbound connection aren't supported today.
- •Data regions: choose the United States (Oregon) or the European Union (Frankfurt). EU organizations send to the /eu/ endpoints.
How to Use It Step-by-Step
- 1Pick a capture path: SDK, LiteLLM, or an OpenTelemetry Collector exporting to /api/v1/otlp/v1/traces with your key.
- 2Set hosting (e.g. azure or self_hosted) if it can't be detected, and your PUE for self-hosted models.
- 3Check the Overview tab's coverage card and the developer console: fallback-matched models and dropped events show there.
OpenTelemetry Collector in a private cloud
exporters:
otlphttp/shdw:
traces_endpoint: https://www.shdwsdk.ai/eu/api/v1/otlp/v1/traces # or /api/v1/... for US
encoding: json
headers:
x-shdw-key: ${env:SHDW_API_KEY}
processors:
attributes/shdw:
actions:
- { key: shdw.hosting, value: azure, action: upsert } # or self_hosted
service:
pipelines:
traces/shdw:
receivers: [otlp]
processors: [attributes/shdw]
exporters: [otlphttp/shdw]Related guides
- Python SDK, OpenTelemetry and LiteLLM
- Data completeness: reconcile with provider billing
- Alerts: Slack and signed webhooks
- Developer console: pipeline health, drop rates and live events
- Carbon accounting export and cost/carbon savings
- Partner accounts: SHDWsdk for every customer of your platform
- Interactive CLI Setup Wizard (npx @shdwsdk/sdk init / npx shdw-init)
- Non-Blocking Client Wrapper (shdw.wrapClient) & Fail-Silent Guarantee
