Track and Optimize ML Experiments with Weights & Biases
Use Weights & Biases (W&B) to track ML experiments with automatic metric logging, real-time dashboards, hyperparameter sweeps, artifact lineage, and a versioned model registry.
Requirements
- Requires the wandb package, and runs on Linux, macOS, and Windows.
How to use
Turn to W&B when you need to track ML experiments, visualize training in real-time dashboards, compare runs across hyperparameters and configurations, optimize hyperparameters with automated sweeps, manage a versioned model registry with lineage, collaborate in team workspaces, or track artifacts such as datasets, models, and code. Hyperparameters can be tracked automatically.
The free tier includes unlimited public projects and 100GB of storage.
The academic tier is free for students and researchers.
The Teams tier costs $50 per seat per month and includes private projects and unlimited storage.
The Enterprise tier has custom pricing with on-premises options.
Install W&B with pip, then log in to create an API key, or set the key programmatically via the WANDB_API_KEY environment variable.
# Install W&B
pip install wandb
# Login (creates API key)
wandb login
# Or set API key programmatically
export WANDB_API_KEY=your_api_key_here
Initialize a run with wandb.init, passing a project name and a config dictionary of parameters such as learning rate, epochs, batch size, and architecture. Inside the training loop, log metrics like train and validation loss and accuracy with wandb.log, then call wandb.finish when the run is done.
import wandb
# Initialize a run
run = wandb.init(
project="my-project",
config={
"learning_rate": 0.001,
"epochs": 10,
"batch_size": 32,
"architecture": "ResNet50"
}
)
# Training loop
for epoch in range(run.config.epochs):
# Your training code
train_loss = train_epoch()
val_loss = validate()
# Log metrics
wandb.log({
"epoch": epoch,
"train/loss": train_loss,
"val/loss": val_loss,
"train/accuracy": train_acc,
"val/accuracy": val_acc
})
# Finish the run
wandb.finish()
In a PyTorch training loop, initialize W&B with your project and config, read hyperparameters from wandb.config, and log the loss every 100 batches. After training, save the model with torch.save and upload it with wandb.save before finishing the run.
import torch
import wandb
# Initialize
wandb.init(project="pytorch-demo", config={
"lr": 0.001,
"epochs": 10
})
# Access config
config = wandb.config
# Training loop
for epoch in range(config.epochs):
for batch_idx, (data, target) in enumerate(train_loader):
# Forward pass
output = model(data)
loss = criterion(output, target)
# Backward pass
optimizer.zero_grad()
loss.backward()
optimizer.step()
# Log every 100 batches
if batch_idx % 100 == 0:
wandb.log({
"loss": loss.item(),
"epoch": epoch,
"batch": batch_idx
})
# Save model
torch.save(model.state_dict(), "model.pth")
wandb.save("model.pth") # Upload to W&B
wandb.finish()
Create or reuse a project via wandb.init, optionally giving the run a name, tags, and notes; each run gets a unique ID and URL that you can print.
# Create/use project
run = wandb.init(
project="image-classification",
name="resnet50-experiment-1", # Optional run name
tags=["baseline", "resnet"], # Organize with tags
notes="First baseline run" # Add notes
)
# Each run has unique ID
print(f"Run ID: {run.id}")
print(f"Run URL: {run.url}")
- Track your hyperparameters so they are recorded with the run.
config = {
# Model architecture
"model": "ResNet50",
"pretrained": True,
# Training params
"learning_rate": 0.001,
"batch_size": 32,
"epochs": 50,
"optimizer": "Adam",
# Data params
"dataset": "ImageNet",
"augmentation": "standard"
}
wandb.init(project="my-project", config=config)
# Access config during training
lr = wandb.config.learning_rate
batch_size = wandb.config.batch_size
Log scalars, multiple metrics at once, and metrics with a custom x-axis using step. You can also log media such as images, audio, and video, histograms of gradients, and tables with columns like id, prediction, and ground_truth.
# Log scalars
wandb.log({"loss": 0.5, "accuracy": 0.92})
# Log multiple metrics
wandb.log({
"train/loss": train_loss,
"train/accuracy": train_acc,
"val/loss": val_loss,
"val/accuracy": val_acc,
"learning_rate": current_lr,
"epoch": epoch
})
# Log with custom x-axis
wandb.log({"loss": loss}, step=global_step)
# Log media (images, audio, video)
wandb.log({"examples": [wandb.Image(img) for img in images]})
# Log histograms
wandb.log({"gradients": wandb.Histogram(gradients)})
# Log tables
table = wandb.Table(columns=["id", "prediction", "ground_truth"])
wandb.log({"predictions": table})
To checkpoint models, save a torch checkpoint and upload it with wandb.save, or preferably create a wandb.Artifact of type 'model', add the checkpoint file, and log it with wandb.log_artifact.
import torch
import wandb
# Save model checkpoint
checkpoint = {
'epoch': epoch,
'model_state_dict': model.state_dict(),
'optimizer_state_dict': optimizer.state_dict(),
'loss': loss,
}
torch.save(checkpoint, 'checkpoint.pth')
# Upload to W&B
wandb.save('checkpoint.pth')
# Or use Artifacts (recommended)
artifact = wandb.Artifact('model', type='model')
artifact.add_file('checkpoint.pth')
wandb.log_artifact(artifact)
Define a sweep configuration specifying a search method (bayes, grid, or random), the metric and goal to optimize, and the parameter ranges or value lists. Initialize the sweep with wandb.sweep, passing the config and project, to get a sweep ID.
sweep_config = {
'method': 'bayes', # or 'grid', 'random'
'metric': {
'name': 'val/accuracy',
'goal': 'maximize'
},
'parameters': {
'learning_rate': {
'distribution': 'log_uniform_values',
'min': 1e-5,
'max': 1e-1
},
'batch_size': {
'values': [16, 32, 64, 128]
},
'optimizer': {
'values': ['adam', 'sgd', 'rmsprop']
},
'dropout': {
'distribution': 'uniform',
'min': 0.1,
'max': 0.5
}
}
}
# Initialize sweep
sweep_id = wandb.sweep(sweep_config, project="my-project")
Write a training function that calls wandb.init, reads the sweep parameters from wandb.config, builds the model and optimizer accordingly, and logs metrics each epoch. Launch the sweep with wandb.agent, passing the sweep ID, the function, and a trial count.
def train():
# Initialize run
run = wandb.init()
# Access sweep parameters
lr = wandb.config.learning_rate
batch_size = wandb.config.batch_size
optimizer_name = wandb.config.optimizer
# Build model with sweep config
model = build_model(wandb.config)
optimizer = get_optimizer(optimizer_name, lr)
# Training loop
for epoch in range(NUM_EPOCHS):
train_loss = train_epoch(model, optimizer, batch_size)
val_acc = validate(model)
# Log metrics
wandb.log({
"train/loss": train_loss,
"val/accuracy": val_acc
})
# Run sweep
wandb.agent(sweep_id, function=train, count=50) # Run 50 trials
Choose a sweep strategy: grid search for exhaustive coverage of explicit value lists, random search with distributions over ranges, or Bayesian optimization, which the source recommends, targeting a metric such as validation loss to minimize.
# Grid search - exhaustive
sweep_config = {
'method': 'grid',
'parameters': {
'lr': {'values': [0.001, 0.01, 0.1]},
'batch_size': {'values': [16, 32, 64]}
}
}
# Random search
sweep_config = {
'method': 'random',
'parameters': {
'lr': {'distribution': 'uniform', 'min': 0.0001, 'max': 0.1},
'dropout': {'distribution': 'uniform', 'min': 0.1, 'max': 0.5}
}
}
# Bayesian optimization (recommended)
sweep_config = {
'method': 'bayes',
'metric': {'name': 'val/loss', 'goal': 'minimize'},
'parameters': {
'lr': {'distribution': 'log_uniform_values', 'min': 1e-5, 'max': 1e-1}
}
}
Log artifacts by creating a wandb.Artifact with a name, type such as 'dataset', a description, and metadata, adding files or whole directories to it, and calling wandb.log_artifact.
# Create artifact
artifact = wandb.Artifact(
name='training-dataset',
type='dataset',
description='ImageNet training split',
metadata={'size': '1.2M images', 'split': 'train'}
)
# Add files
artifact.add_file('data/train.csv')
artifact.add_dir('data/images/')
# Log artifact
wandb.log_artifact(artifact)
To use a logged artifact, initialize a run, fetch it with run.use_artifact using a version reference like 'training-dataset:latest', download it, and load the data from the returned directory.
# Download and use artifact
run = wandb.init(project="my-project")
# Download artifact
artifact = run.use_artifact('training-dataset:latest')
artifact_dir = artifact.download()
# Use the data
data = load_data(f"{artifact_dir}/train.csv")
For the model registry, log a model as a wandb.Artifact with metadata such as architecture and accuracy, add the model file, log it with aliases like 'best' and 'production', then link it with run.link_artifact to a registry path such as 'model-registry/production-models'.
# Log model as artifact
model_artifact = wandb.Artifact(
name='resnet50-model',
type='model',
metadata={'architecture': 'ResNet50', 'accuracy': 0.95}
)
model_artifact.add_file('model.pth')
wandb.log_artifact(model_artifact, aliases=['best', 'production'])
# Link to model registry
run.link_artifact(model_artifact, 'model-registry/production-models')
With HuggingFace Transformers, initialize W&B and set report_to='wandb' in TrainingArguments, optionally with a run_name and logging and saving steps, so the Trainer logs to W&B automatically during training.
from transformers import Trainer, TrainingArguments
import wandb
# Initialize W&B
wandb.init(project="hf-transformers")
# Training arguments with W&B
training_args = TrainingArguments(
output_dir="./results",
report_to="wandb", # Enable W&B logging
run_name="bert-finetuning",
logging_steps=100,
save_steps=500
)
# Trainer automatically logs to W&B
trainer = Trainer(
model=model,
args=training_args,
train_dataset=train_dataset,
eval_dataset=eval_dataset
)
trainer.train()
In PyTorch Lightning, create a WandbLogger with your project and log_model=True to log model checkpoints, then pass it as the Trainer's logger and train as usual.
from pytorch_lightning import Trainer
from pytorch_lightning.loggers import WandbLogger
import wandb
# Create W&B logger
wandb_logger = WandbLogger(
project="lightning-demo",
log_model=True # Log model checkpoints
)
# Use with Trainer
trainer = Trainer(
logger=wandb_logger,
max_epochs=10
)
trainer.fit(model, datamodule=dm)
In Keras/TensorFlow, initialize W&B and add the dedicated callbacks from wandb.integration.keras, WandbMetricsLogger for automatic metric logging and WandbModelCheckpoint for saving checkpoints, to model.fit. Note that the monolithic WandbCallback was removed, so use these callbacks instead.
import wandb
from wandb.integration.keras import WandbMetricsLogger, WandbModelCheckpoint
# Initialize
wandb.init(project="keras-demo")
# Add callbacks (the monolithic WandbCallback was removed;
# use the dedicated callbacks from wandb.integration.keras instead)
model.fit(
x_train, y_train,
validation_data=(x_val, y_val),
epochs=10,
callbacks=[
WandbMetricsLogger(), # Auto-logs metrics
WandbModelCheckpoint("models/model-{epoch}") # Saves checkpoints
]
)
Log custom visualizations such as a matplotlib figure wrapped in wandb.Image, or a confusion matrix via wandb.plot.confusion_matrix with ground truth, predictions, and class names.
# Log custom visualizations
import matplotlib.pyplot as plt
fig, ax = plt.subplots()
ax.plot(x, y)
wandb.log({"custom_plot": wandb.Image(fig)})
# Log confusion matrix
wandb.log({"conf_mat": wandb.plot.confusion_matrix(
probs=None,
y_true=ground_truth,
preds=predictions,
class_names=class_names
)})
Create shareable reports in the W&B UI that combine runs, charts, and text, support Markdown, offer embeddable visualizations, and enable team collaboration.
Organize runs by passing tags, a group for related runs, and a job_type such as 'train' to wandb.init.
wandb.init(
project="my-project",
tags=["baseline", "resnet50", "imagenet"],
group="resnet-experiments", # Group related runs
job_type="train" # Type of job
)
Log everything relevant: system metrics like GPU utilization, GPU memory, and CPU utilization; the code version via the git commit hash; and data split sizes for training and validation.
# Log system metrics
wandb.log({
"gpu/util": gpu_utilization,
"gpu/memory": gpu_memory_used,
"cpu/util": cpu_utilization
})
# Log code version
wandb.log({"git_commit": git_commit_hash})
# Log data splits
wandb.log({
"data/train_size": len(train_dataset),
"data/val_size": len(val_dataset)
})
Use descriptive run names that convey the setup, for example 'bert-base-lr0.001-bs32-epoch10', rather than generic names like 'run1'.
# ✅ Good: Descriptive run names
wandb.init(
project="nlp-classification",
name="bert-base-lr0.001-bs32-epoch10"
)
# ❌ Bad: Generic names
wandb.init(project="nlp", name="run1")
Save important outputs as artifacts: log the final model as a wandb.Artifact, and log a predictions table with columns like id, input, prediction, and ground_truth for later analysis.
# Save final model
artifact = wandb.Artifact('final-model', type='model')
artifact.add_file('model.pth')
wandb.log_artifact(artifact)
# Save predictions for analysis
predictions_table = wandb.Table(
columns=["id", "input", "prediction", "ground_truth"],
data=predictions_data
)
wandb.log({"predictions": predictions_table})
For unstable connections, enable offline mode by setting the WANDB_MODE environment variable to 'offline' before wandb.init, then sync the run directory later with wandb sync.
import os
# Enable offline mode
os.environ["WANDB_MODE"] = "offline"
wandb.init(project="my-project")
# ... your code ...
# Sync later
# wandb sync <run_directory>
Runs are automatically shareable via their URL, which you can print from the run object.
# Runs are automatically shareable via URL
run = wandb.init(project="team-project")
print(f"Share this URL: {run.url}")
To collaborate as a team, create a team account at wandb.ai and add team members.
Add the team members to your account.
Set project visibility to private or public.
Use team-level artifacts and the team-level model registry.
Source: nousresearch/hermes-agent and its contributors. License: MIT.
Copyright (c) 2025 Nous Research
Translation and editorial adaptation by ClawCases.
>Resources
>Related Use Cases
Agent Merge-Conflict Arbiter
Resolve a git merge conflict between two agents' branches as an impartial third party: a neutral arbiter that takes both diffs and their stated intents and produces a merged result without overwriting either side.
Autonomous Project Management with Subagents
Coordinate projects autonomously: a main agent delegates work to PM subagents that track tasks in a shared STATE.yaml, update it as they progress, pick up unblocked work in parallel, and keep all task state in version-controlled files.
Infrastructure: TUI Operator Console
The Hermes TUI HUD is a keyboard-first terminal operator console that gives a single-pane view of everything happening across your agent.