Deployment and model coverage: clouds, private hosting and self-hosted models

Integrations & Data Sources
#coverage#azure#bedrock#vertex#self-hosted#on-premises#private cloud#vllm#ollama#embeddings#transcription#air-gapped

Which providers, hosting setups and model types SHDWsdk measures, how unknown models are handled, and what private-cloud and on-premises deployments need.

How It Works Under the Hood
  • •Any vendor: the model vendor (OpenAI, Anthropic, Google, Mistral, Cohere, Meta, DeepSeek, Alibaba, Microsoft, Amazon, xAI, or any other id) is recorded separately from where the model ran: vendor API, Azure, AWS Bedrock, Google Vertex, self-hosted or other. Hosting is detected from OpenTelemetry gen_ai.provider.name, LiteLLM routes and Bedrock/Vertex model ids, or set explicitly.
  • •Capture paths: the JavaScript and Python SDKs, LiteLLM (every provider it routes to, including Azure, Bedrock, Vertex, Ollama and vLLM), any OpenAI-compatible client, and OpenTelemetry. OTLP spans are read with the GenAI semantic conventions, so an OpenTelemetry Collector inside a private cloud can forward spans with no application changes.
  • •Energy for text generation uses EcoLogits (0.11.3), including open-weight models such as Llama, Mistral, Qwen and Gemma. Models not in the catalog (fine-tunes, renamed deployments) get a wide generic range and are flagged 'fallback' so they can be reviewed. Embeddings, reranking, moderation and transcription use a documented FLOP-based estimate (shdw-flops-v1); image, speech and OCR calls are counted but not yet estimated, and are excluded from emissions rather than reported as zero.
  • •Self-hosted models: set your datacenter's PUE in Settings → Developer and send a grid region or a measured gCO2e/kWh, so self-hosted calls aren't charged a cloud provider's efficiency.
  • •Private cloud and on-premises: only metadata leaves your network (model id, token counts, timing, region, pseudonymous ids), never prompts or outputs, through a single outbound HTTPS endpoint. Fully air-gapped installations without any outbound connection aren't supported today.
  • •Data regions: choose the United States (Oregon) or the European Union (Frankfurt). EU organizations send to the /eu/ endpoints.
How to Use It Step-by-Step
  1. 1Pick a capture path: SDK, LiteLLM, or an OpenTelemetry Collector exporting to /api/v1/otlp/v1/traces with your key.
  2. 2Set hosting (e.g. azure or self_hosted) if it can't be detected, and your PUE for self-hosted models.
  3. 3Check the Overview tab's coverage card and the developer console: fallback-matched models and dropped events show there.
OpenTelemetry Collector in a private cloud
exporters:
  otlphttp/shdw:
    traces_endpoint: https://www.shdwsdk.ai/eu/api/v1/otlp/v1/traces   # or /api/v1/... for US
    encoding: json
    headers:
      x-shdw-key: ${env:SHDW_API_KEY}
processors:
  attributes/shdw:
    actions:
      - { key: shdw.hosting, value: azure, action: upsert }   # or self_hosted
service:
  pipelines:
    traces/shdw:
      receivers: [otlp]
      processors: [attributes/shdw]
      exporters: [otlphttp/shdw]

Related guides