Covert-GPT
A self-hosted GPU cluster for running language models.
- Built with
- NVIDIA DGX Spark, Kubernetes, vLLM, LiteLLM, ComfyUI
- DGX Spark nodes do the heavy lifting.
- A LiteLLM gateway sits in front of every model.
- ComfyUI handles image generation.
- I load-test it to find where it breaks.