Agentic AI Series Part 2: Setting Up Your Linux Environment for AI Agent Development

Series: Agentic AI Development Series (Part 2 of 8)

Introduction: Building Your AI Agent Workshop

In Part 1 of our Agentic AI series, we explored the fundamental concepts of autonomous AI agents and their transformative potential. Now, it’s time to roll up our sleeves and prepare the foundation for building these intelligent systems: your Linux development environment.

Think of this as setting up a professional workshop. Just as a master craftsman needs the right tools, workspace, and materials organized efficiently, an AI agent developer needs a properly configured Linux system with the right frameworks, libraries, and hardware acceleration ready to go.

Why Linux? The answer is simple:

• Performance: Native access to GPU drivers and optimized libraries
• Stability: Rock-solid foundation for long-running agent processes
• Flexibility: Complete control over your environment
• Cost: Free and open-source ecosystem
• Community: Massive support for AI/ML development

Let’s build your AI development environment from the ground up.

Part 1: Choosing Your Linux Distribution

Not all Linux distributions are created equal for AI development. In 2025, several distributions have emerged as leaders in the AI/ML space.

Option 1: Ubuntu 24.04 LTS / Ubuntu AI (Recommended for Beginners)

Why Ubuntu?

• Largest community: More tutorials, troubleshooting resources, and support
• LTS (Long Term Support): 5 years of updates and security patches
• Hardware compatibility: Excellent driver support, especially NVIDIA GPUs
• Pre-configured packages: AI frameworks available through standard repositories

Ubuntu AI (introduced by Canonical) is a specialized version that includes:

• TensorFlow, PyTorch, and Scikit-learn pre-installed
• Optimized NVIDIA CUDA support out of the box
• AMD ROCm support for AMD GPUs
• Pre-configured Jupyter notebooks

Best For: Beginners, those who want stability, anyone with NVIDIA GPUs

Option 2: Fedora AI (Cutting-Edge Performance)

Why Fedora AI?

• Latest packages: Rolling updates bring newest AI frameworks faster
• High performance: Optimized for AI workloads
• Containerization: Excellent Podman and Docker support
• Red Hat ecosystem: Enterprise-grade tools and workflows

Best For: Experienced Linux users, those who want cutting-edge tools

Option 3: Pop!_OS AI by System76

Why Pop!_OS AI?

• Pre-installed NVIDIA drivers: Zero configuration for GPU acceleration
• Excellent hardware support: Designed for modern hardware
• Developer-friendly: Built by developers, for developers
• COSMIC desktop: Efficient, distraction-free interface

Best For: NVIDIA GPU users, laptop developers, those who value out-of-box GPU support

My Recommendation

For this series, I recommend Ubuntu 24.04 LTS for the following reasons:

1. Largest community means easier troubleshooting
2. Best balance of stability and modern packages
3. Excellent GPU driver support
4. Most tutorials and guides target Ubuntu
5. Easy to switch to Ubuntu AI later if needed

Part 2: Initial System Setup

Step 1: Update Your System

# Update package lists
sudo apt update

# Upgrade all packages
sudo apt upgrade -y

# Install essential build tools
sudo apt install -y build-essential git curl wget vim

Step 2: Install System Dependencies

# Install development libraries
sudo apt install -y \
    python3-dev \
    python3-pip \
    python3-venv \
    libssl-dev \
    libffi-dev \
    libxml2-dev \
    libxslt1-dev \
    zlib1g-dev

# Install additional useful tools
sudo apt install -y \
    htop \
    tmux \
    tree \
    net-tools \
    software-properties-common

Step 3: Configure Git

# Set your identity
git config --global user.name "Your Name"
git config --global user.email "your.email@example.com"

# Set default branch name
git config --global init.defaultBranch main

# Enable credential helper
git config --global credential.helper store

Part 3: Python Environment Setup

Python is the lingua franca of AI development. Proper Python environment management is crucial for avoiding dependency conflicts and maintaining reproducible environments.

Understanding Virtual Environments

Why virtual environments?

• Isolation: Each project has its own dependencies
• Reproducibility: Exact same environment on any machine
• No conflicts: Different projects can use different library versions
• Clean system: Don’t pollute system Python installation

Option A: venv (Built-in, Lightweight)

# Navigate to your project directory
mkdir ~/ai-agent-project
cd ~/ai-agent-project

# Create virtual environment
python3 -m venv venv

# Activate virtual environment
source venv/bin/activate

# Your prompt should now show (venv)
# Install packages
pip install --upgrade pip
pip install numpy pandas matplotlib

# Deactivate when done
deactivate

Option B: Conda/Miniconda (Recommended for AI/ML)

Advantages:

• Non-Python dependencies: Manages C libraries, CUDA, system tools
• Better for AI/ML: Handles complex dependencies like CUDA, cuDNN
• Cross-platform: Works identically on Linux, macOS, Windows
• Pre-built binaries: Faster installation, no compilation needed

Installing Miniconda:

# Download Miniconda installer
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh

# Make executable
chmod +x Miniconda3-latest-Linux-x86_64.sh

# Run installer
./Miniconda3-latest-Linux-x86_64.sh

# Follow prompts, answer 'yes' to initialize conda
# Close and reopen terminal, or run:
source ~/.bashrc

# Verify installation
conda --version

Creating and managing Conda environments:

# Create environment with specific Python version
conda create -n agentic-ai python=3.11 -y

# Activate environment
conda activate agentic-ai

# Your prompt shows: (agentic-ai)

# Install packages via conda (preferred)
conda install numpy pandas scikit-learn -y

# Install packages via pip (when not available in conda)
pip install langchain openai

# List all environments
conda env list

# Export environment for reproducibility
conda env export > environment.yml

# Create environment from file
conda env create -f environment.yml

# Remove environment
conda deactivate
conda env remove -n agentic-ai

My Recommendation: Use Conda for AI agent development because:

1. Manages GPU libraries (CUDA, cuDNN) seamlessly
2. Handles complex dependency graphs better
3. Pre-built binaries save compilation time
4. Industry standard in AI/ML development

Part 4: GPU Acceleration Setup (NVIDIA)

GPU acceleration can provide 50-100x speedup for AI workloads. If you have an NVIDIA GPU, this section is critical.

Prerequisites

Check if you have an NVIDIA GPU:

# List PCI devices, filter for NVIDIA
lspci | grep -i nvidia

# Example output:
# 01:00.0 VGA compatible controller: NVIDIA Corporation GA104 [GeForce RTX 3070]

If you see output, you have an NVIDIA GPU. Let’s set it up!

Step 1: Install NVIDIA Driver

# Search for available drivers
ubuntu-drivers devices

# Install recommended driver
sudo ubuntu-drivers autoinstall

# Or install specific version
sudo apt install nvidia-driver-550

# Reboot to load driver
sudo reboot

# After reboot, verify installation
nvidia-smi

# You should see GPU information table

Step 2: Install CUDA Toolkit

Method 1: Via Conda (Recommended)

# Activate your conda environment
conda activate agentic-ai

# Install CUDA toolkit via conda
conda install -c conda-forge cudatoolkit=12.4 -y

# Install cuDNN (for deep learning)
conda install -c conda-forge cudnn -y

Method 2: System-wide Installation

# Add NVIDIA package repository
wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2404/x86_64/cuda-keyring_1.1-1_all.deb
sudo dpkg -i cuda-keyring_1.1-1_all.deb
sudo apt update

# Install CUDA toolkit
sudo apt install cuda-toolkit-12-4 -y

# Add CUDA to PATH
echo 'export PATH=/usr/local/cuda-12.4/bin:$PATH' >> ~/.bashrc
echo 'export LD_LIBRARY_PATH=/usr/local/cuda-12.4/lib64:$LD_LIBRARY_PATH' >> ~/.bashrc
source ~/.bashrc

# Verify CUDA installation
nvcc --version

Step 3: Install PyTorch with CUDA Support

# Make sure conda environment is activated
conda activate agentic-ai

# Install PyTorch with CUDA 12.4 support
conda install pytorch torchvision torchaudio pytorch-cuda=12.4 -c pytorch -c nvidia -y

# Verify PyTorch can see GPU
python3 << EOF
import torch
print(f"PyTorch version: {torch.__version__}")
print(f"CUDA available: {torch.cuda.is_available()}")
print(f"CUDA version: {torch.version.cuda}")
print(f"GPU device: {torch.cuda.get_device_name(0) if torch.cuda.is_available() else 'None'}")
EOF

Expected output:

PyTorch version: 2.5.0
CUDA available: True
CUDA version: 12.4
GPU device: NVIDIA GeForce RTX 3070

Step 4: Install TensorFlow with CUDA Support

# Install TensorFlow
pip install tensorflow[and-cuda]

# Verify TensorFlow can see GPU
python3 << EOF
import tensorflow as tf
print(f"TensorFlow version: {tf.__version__}")
print(f"GPU devices: {tf.config.list_physical_devices('GPU')}")
EOF

Part 5: Essential AI/ML Libraries Installation

Create Your Agentic AI Environment

# Create dedicated environment
conda create -n agentic-ai python=3.11 -y
conda activate agentic-ai

# Install CUDA support (if you have NVIDIA GPU)
conda install -c conda-forge cudatoolkit=12.4 cudnn -y

# Install PyTorch with CUDA
conda install pytorch torchvision torchaudio pytorch-cuda=12.4 -c pytorch -c nvidia -y

# Install TensorFlow
pip install tensorflow[and-cuda]

# Install core AI/ML libraries
pip install numpy pandas scikit-learn matplotlib seaborn jupyter

# Install LLM and Agent frameworks
pip install openai anthropic langchain langchain-community langgraph

# Install vector databases
pip install chromadb faiss-cpu pinecone-client

# Install utilities
pip install python-dotenv requests httpx pydantic

# Install development tools
pip install black flake8 pytest ipython

Verify Installation

Create a test script to verify everything works:

#!/usr/bin/env python3
"""Test script to verify AI environment setup"""

import sys

def test_imports():
    """Test if all critical libraries can be imported"""

    tests = []

    # Core libraries
    try:
        import numpy as np
        tests.append(("NumPy", np.__version__, "✓"))
    except Exception as e:
        tests.append(("NumPy", str(e), "✗"))

    try:
        import pandas as pd
        tests.append(("Pandas", pd.__version__, "✓"))
    except Exception as e:
        tests.append(("Pandas", str(e), "✗"))

    # Deep Learning
    try:
        import torch
        gpu_status = "GPU Available" if torch.cuda.is_available() else "CPU Only"
        tests.append(("PyTorch", f"{torch.__version__} ({gpu_status})", "✓"))
    except Exception as e:
        tests.append(("PyTorch", str(e), "✗"))

    try:
        import tensorflow as tf
        gpu_count = len(tf.config.list_physical_devices('GPU'))
        gpu_status = f"{gpu_count} GPU(s)" if gpu_count > 0 else "CPU Only"
        tests.append(("TensorFlow", f"{tf.__version__} ({gpu_status})", "✓"))
    except Exception as e:
        tests.append(("TensorFlow", str(e), "✗"))

    # LLM Frameworks
    try:
        import langchain
        tests.append(("LangChain", langchain.__version__, "✓"))
    except Exception as e:
        tests.append(("LangChain", str(e), "✗"))

    try:
        import openai
        tests.append(("OpenAI", openai.__version__, "✓"))
    except Exception as e:
        tests.append(("OpenAI", str(e), "✗"))

    # Vector DBs
    try:
        import chromadb
        tests.append(("ChromaDB", chromadb.__version__, "✓"))
    except Exception as e:
        tests.append(("ChromaDB", str(e), "✗"))

    # Print results
    print("\n" + "="*70)
    print("AI Environment Setup Verification")
    print("="*70)

    for name, version, status in tests:
        print(f"{status} {name:20s} {version}")

    print("="*70 + "\n")

    # Check if all passed
    all_passed = all(status == "✓" for _, _, status in tests)
    if all_passed:
        print("✓ All tests passed! Your environment is ready.")
        return 0
    else:
        print("✗ Some tests failed. Please check the errors above.")
        return 1

if __name__ == "__main__":
    sys.exit(test_imports())

Part 6: Development Tools Setup

VS Code (Recommended IDE)

# Download and install VS Code
wget https://code.visualstudio.com/sha/download?build=stable&os=linux-deb-x64 -O vscode.deb
sudo dpkg -i vscode.deb
sudo apt install -f

# Launch VS Code
code

# Install essential extensions via command line
code --install-extension ms-python.python
code --install-extension ms-python.vscode-pylance
code --install-extension ms-toolsai.jupyter
code --install-extension GitHub.copilot
code --install-extension ms-vscode-remote.remote-ssh

Jupyter Lab

# Install JupyterLab
conda install -c conda-forge jupyterlab -y

# Launch Jupyter Lab
jupyter lab

# Access at: http://localhost:8888

Part 7: API Keys and Environment Variables

AI agents often need API keys for LLM services. Let’s set up secure environment variable management.

Using .env Files

# Install python-dotenv
pip install python-dotenv

# Create .env file
cat > .env << 'EOF'
# OpenAI API
OPENAI_API_KEY=your_openai_api_key_here

# Anthropic API
ANTHROPIC_API_KEY=your_anthropic_api_key_here

# Pinecone
PINECONE_API_KEY=your_pinecone_api_key_here
PINECONE_ENVIRONMENT=your_pinecone_environment_here

# Other services
LANGCHAIN_API_KEY=your_langchain_api_key_here
EOF

# Make sure .env is in .gitignore
echo ".env" >> .gitignore

Using .env in Python:

from dotenv import load_dotenv
import os

# Load environment variables
load_dotenv()

# Access variables
openai_key = os.getenv("OPENAI_API_KEY")
anthropic_key = os.getenv("ANTHROPIC_API_KEY")

Part 8: Docker Setup (Optional but Recommended)

Docker allows you to containerize your AI agents for easier deployment and reproducibility.

Install Docker

# Install Docker
curl -fsSL https://get.docker.com -o get-docker.sh
sudo sh get-docker.sh

# Add your user to docker group
sudo usermod -aG docker $USER

# Log out and back in, or run:
newgrp docker

# Verify installation
docker --version
docker run hello-world

Install NVIDIA Container Toolkit (for GPU in Docker)

# Add repository
distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add -
curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | \
    sudo tee /etc/apt/sources.list.d/nvidia-docker.list

# Install nvidia-container-toolkit
sudo apt update
sudo apt install -y nvidia-container-toolkit

# Restart Docker
sudo systemctl restart docker

# Test GPU access in container
docker run --rm --gpus all nvidia/cuda:12.4.0-base-ubuntu22.04 nvidia-smi

Conclusion: Your Environment is Ready!

Congratulations! You now have a professional Linux environment configured for AI agent development with:

• ✓ Linux distribution optimized for AI/ML
• ✓ Python environment with virtual environment management
• ✓ GPU acceleration with CUDA and cuDNN
• ✓ PyTorch and TensorFlow with GPU support
• ✓ LLM frameworks (LangChain, OpenAI, Anthropic)
• ✓ Vector databases (ChromaDB, FAISS)
• ✓ Development tools (VS Code, Jupyter Lab)
• ✓ Docker for containerization (optional)

Quick Reference: Daily Workflow

# Start your day
conda activate agentic-ai
cd ~/ai-projects/agentic-ai-series

# Monitor GPU while working
watch -n 1 nvidia-smi

# Launch Jupyter for notebooks
jupyter lab

# Or launch VS Code
code .

# End of day - deactivate environment
conda deactivate

What’s Next?

In Part 3: Understanding LLMs and Agent Architecture, we’ll explore:

• How Large Language Models work
• Agent architecture patterns
• Prompt engineering fundamentals
• Tool calling and function execution
• Memory systems for agents

Your Linux environment is now ready to build sophisticated AI agents. Let’s start building!

Troubleshooting Common Issues

Issue 1: Conda command not found after installation

# Manually add conda to PATH
echo 'export PATH=~/miniconda3/bin:$PATH' >> ~/.bashrc
source ~/.bashrc

Issue 2: NVIDIA driver conflicts

# Remove all NVIDIA drivers
sudo apt purge nvidia-* -y
sudo apt autoremove -y

# Reinstall fresh
sudo ubuntu-drivers autoinstall
sudo reboot

Issue 3: CUDA version mismatch

# Check what CUDA version your driver supports
nvidia-smi | grep "CUDA Version"

# Install matching PyTorch
# Visit: https://pytorch.org/get-started/locally/
# Select matching CUDA version

Resources and Further Reading

Documentation

• Ubuntu Documentation
• Conda User Guide
• PyTorch Installation
• TensorFlow GPU Guide
• NVIDIA CUDA Toolkit

Next in Series:

• Part 3: Understanding LLMs and Agent Architecture
• Part 4: Building Your First AI Agent
• Part 5: Advanced Multi-Agent Systems

Have questions about your Linux setup? Drop them in the comments below!