author: Marco Jeffrey Pansa

date: 22-05-2024

tags: NLP

Create a conda env for Axolotl

Prerequisites:

  • computer with Nvidia GPU
  • drivers with cuda 11 / 12 support
  • conda (either Anaconda / miniconda / miniforge etc.)

If you dont have conda installed i recommend going with miniforge . The cool thing about conda is it will take care of installing cuda tools, doing it manually is error prone and annoying

First check what the maximum supported version of cuda is. Type:

nividia-smi and look at the top right corner

image

Now create a file env.yaml file with the following:

  • if your max. supported cuda version starts with 11 change pytorch-cuda=12.1 to pytorch-cuda=11.8 or update your nvidia drivers if possible.
  • if your max. supported cuda version starts with 10 you have to update because pytorch 2 does not support cuda 10.

name: axolotl
channels:
  - pytorch
  - nvidia
  - conda-forge
  - defaults
dependencies:
  - python=3.11
  - pytorch
  - torchvision
  - torchaudio
  - pytorch-cuda=12.1
  - cuda-nvcc
  - pip:
      - packaging
      - ninja
  • run conda env create -f env.yml to create the env
  • then run conda activate axolotl
  • now clone the axolotl repo
git clone https://github.com/OpenAccess-AI-Collective/axolotl.git

cd into the repo and install with pip3 install -e '.[flash-attn,deepspeed]'

Thats it!