Getting Started

Installation

Detailed installation instructions for ZSE, including system requirements, GPU setup, and troubleshooting common issues.

System Requirements

ZSE is designed to run on a variety of hardware configurations. Here are the minimum and recommended requirements:

Component	Minimum	Recommended
Python	3.8	3.10+
RAM	16 GB	32 GB+
Disk Space	20 GB	100 GB+
OS	Linux, macOS, Windows	Linux (Ubuntu 20.04+)

GPU Requirements

GPU acceleration significantly improves performance. Here are the VRAM requirements by model size:

Model Size	FP16	INT8	INT4 (ZSE)
7B	14 GB	8 GB	5 GB
14B	28 GB	16 GB	10 GB
32B	64 GB	32 GB	20 GB
70B	140 GB	70 GB	40 GB

Limited VRAM?

Use zStream layer offloading to run larger models on GPUs with limited memory. See the zStream documentation.

Install with pip

The simplest way to install ZSE is via pip:

bash

pip install zllm-zse

Optional dependencies:

bash

# GGUF model support
pip install zllm-zse[gguf]
 
# All optional features
pip install zllm-zse[all]

We recommend using a virtual environment to avoid dependency conflicts.

bash

# Create virtual environment
python -m venv zse-env
source zse-env/bin/activate  # Linux/macOS
# or: zse-env\Scripts\activate  # Windows
 
# Install ZSE
pip install zllm-zse

Install from Source

For development or the latest features, install from source:

bash

git clone https://github.com/Zyora-Dev/zse.git
cd zse
pip install -e .

For development with testing dependencies:

bash

pip install -e .[dev]

Verify Installation

Verify that ZSE is installed correctly:

bash

# Check version
zse --version
 
# Check hardware detection
zse hardware
 
# Run a quick test (downloads a small model)
zse chat --model Qwen/Qwen2.5-0.5B-Instruct --prompt "Hello!"

Expected zse hardware output:

text

ZSE Hardware Detection
━━━━━━━━━━━━━━━━━━━━━━
GPU 0: NVIDIA A100-SXM4-80GB
  Memory: 80 GB
  CUDA: 12.1
  Compute: 8.0
 
Total VRAM: 80 GB
Recommended models: Up to 70B (INT4)

Troubleshooting

CUDA Not Found

If ZSE doesn't detect your GPU, ensure CUDA is properly installed:

bash

# Check CUDA version
nvidia-smi
 
# If not found, install CUDA toolkit
# Ubuntu/Debian:
sudo apt install nvidia-cuda-toolkit
 
# Or install PyTorch with CUDA:
pip install torch --index-url https://download.pytorch.org/whl/cu121

Out of Memory Errors

If you encounter OOM errors:

Use a smaller model (7B instead of 32B)
Enable zStream layer offloading: zse serve model --offload
Reduce batch size: zse serve model --max-batch 1
Use INT4 quantization (default) instead of INT8

Import Errors

Common import errors and fixes:

bash

# Missing dependencies
pip install zllm-zse --upgrade
 
# Torch version conflicts
pip install torch>=2.0.0 --upgrade
 
# Clean reinstall
pip uninstall zllm-zse
pip cache purge
pip install zllm-zse

← Quick Start

First Model →