Grok AI source code and weights

This repository provides JAX example code for loading and running Grok-1, an open-weights model, with its weights available via torrent or the HuggingFace Hub. The model has 314B parameters and uses a Mixture of 8 Experts architecture with 2 experts used per token, 64 layers, and a maximum context of 8,192 tokens. The text notes that the MoE implementation is not efficient and was chosen to avoid custom kernels when validating model correctness, and that testing requires a machine with substantial GPU memory. The code and weights are licensed under Apache 2.0.