Thank you for sending your enquiry! One of our team members will contact you shortly.
Thank you for sending your booking! One of our team members will contact you shortly.
Course Outline
Offline EXO Deployment
- Utilizing EXO_OFFLINE to prevent runtime internet access.
- Pre-loading models into EXO_MODELS_READ_ONLY_DIRS from trusted internal mirrors.
- Verifying model weight integrity using SHA-256 checksums and signed model cards.
- Running EXO in air-gapped networks without HuggingFace dependencies.
Dashboard and API Access Control
- Installing and configuring reverse proxies (nginx, Caddy) with TLS termination.
- Implementing role-based access control for the EXO dashboard and REST API.
- Using the macOS keychain or Linux pass to store secrets for API authentication.
- Restricting administrative endpoints to specific source IP ranges.
Cluster Isolation and Network Security
- Segmenting EXO clusters using EXO_LIBP2P_NAMESPACE and VLANs.
- Configuring host firewalls (macOS application firewall, iptables, nftables) for EXO ports.
- Preventing unauthorized device discovery and rogue node injection.
- Encrypting libp2p traffic between nodes when RDMA is unavailable.
Model Governance and Provenance
- Building an internal model registry featuring approved model lists and metadata.
- Tagging and versioning quantized weights (4-bit, 8-bit) alongside source checkpoints.
- Enforcing that only specific HuggingFace repositories or internal artifacts can be loaded.
- Documenting model lineage, license terms, and acceptable use policies.
Audit Logging and Compliance
- Configuring EXO log forwarding to immutable audit trails (SIEM, WORM storage).
- Correlating API call logs with user identity and timestamp.
- Capturing events related to model instance creation, deletion, and inference requests.
- Generating periodic compliance reports for internal and external auditors.
Threat Modeling and Incident Response
- Identifying threats such as data exfiltration through model outputs, prompt injection, and side-channel leaks.
- Implementing prompt monitoring and content filtering pipelines.
- Creating incident response runbooks for cluster compromise scenarios.
- Isolating affected nodes, preserving forensic logs, and rebuilding clean environments.
Physical Security and Hardware Boundaries
- Securing Thunderbolt ports against unauthorized RDMA cable connections.
- Utilizing secure enclaves and Apple Silicon hardware attestation where applicable.
- Controlling physical access to clustered Macs and shared storage.
- Documenting hardware lifecycle and decommissioning procedures.
Regulatory Considerations
- Mapping EXO deployments to GDPR, HIPAA, and SOC 2 requirements.
- Maintaining data residency by keeping inference on-premises.
- Documenting vendor supply-chain risks (MLX, EXO, model weights).
- Preparing for AI governance frameworks such as EU AI Act Article 53.
Requirements
- Experience with EXO or another local Large Language Model (LLM) runtime.
- Understanding of Unix filesystem permissions and networking Access Control Lists (ACLs).
- Familiarity with TLS/SSL certificate management and encryption fundamentals.
Audience
- Security engineers.
- Compliance officers.
- AI infrastructure administrators managing sensitive data.
14 Hours
Testimonials (1)
The trainer had an excellent knowledge of fortigate and delivered the content very well. Thanks a lot to Soroush.
