ADR-009: Driver Architecture

Status

Accepted

Context

Toast needs pluggable infrastructure for core concerns like email, queues, and storage. These are application-level configurations—not site-level settings—that control fundamental behavior across all tenants.

Requirements

1. Self-hosted flexibility - Different operators need different providers (Mailgun vs SES vs Resend)
2. Simple installation - Hobbyists, sysadmins, and developers should all be able to configure drivers
3. Extensibility - Third parties can create drivers without forking Toast
4. Newsletter support - Email drivers must support both transactional (magic links) and bulk (newsletters) sending
5. Development experience - Local development should work without external services

Prior Art

Ghost uses an adapter system where custom code is placed in content/adapters/{type}/ and loaded at runtime. This works but has limitations:

• Requires file system access (awkward for containerized deployments)
• No standard dependency management
• Dynamic loading of arbitrary code has security implications
• No type safety for adapter contracts

Decision

Implement a driver system where drivers are npm packages that implement typed interfaces from @toast/drivers.

Package Structure

toast/
├── packages/
│   └── drivers/                         # @toast/drivers
│       └── src/
│           ├── loader.ts                # Generic loadDriver() function
│           ├── email/
│           │   ├── types.ts             # EmailDriver interface
│           │   ├── console.driver.ts    # Built-in development driver
│           │   └── loader.ts            # getEmailDriver()
│           ├── storage/
│           │   ├── types.ts             # StorageDriver interface
│           │   └── loader.ts            # getStorageDriver()
│           └── queue/
│               └── ...
│
└── drivers/                             # Official driver packages
    ├── email-mailgun/                   # toast-driver-email-mailgun
    ├── email-resend/                    # toast-driver-email-resend
    └── storage-s3/                      # toast-driver-storage-s3

Naming Convention

Driver packages follow the pattern: toast-driver-{type}-{name}

Configuration

Drivers are selected via environment variables. Each driver reads its own configuration from process.env:

# Email driver selection
EMAIL_DRIVER=toast-driver-email-mailgun

# Email driver-specific config
MAILGUN_API_KEY=key-xxx
MAILGUN_DOMAIN=mg.example.com
MAILGUN_REGION=us
EMAIL_FROM=noreply@example.com

# Storage driver selection
STORAGE_DRIVER=toast-driver-storage-s3

# S3-compatible storage config (works with MinIO, R2, S3, B2)
S3_ENDPOINT=https://account.r2.cloudflarestorage.com
S3_ACCESS_KEY_ID=xxx
S3_SECRET_ACCESS_KEY=xxx
S3_BUCKET=toast-uploads
S3_REGION=auto
S3_PUBLIC_URL=https://uploads.example.com

This keeps configuration simple and self-documenting—each driver's README documents its required environment variables.

Storage driver unification: The S3 driver works with any S3-compatible provider (AWS S3, Cloudflare R2, MinIO, Backblaze B2) by accepting an explicit endpoint. This avoids needing separate drivers for each provider.

Loading Mechanism

Drivers are loaded via dynamic import at startup. The key challenge is that @toast/drivers needs to import packages that the user installed in their app, not packages that @toast/drivers depends on.

We solve this using createRequire to resolve modules from the app's context:

import { createRequire } from 'node:module';

// Generic loader - works for any driver type
export async function loadDriver<T>(driverName: string): Promise<T> {
  // Resolve from the app's root (where user installed the driver)
  const appRequire = createRequire(`${process.cwd()}/package.json`);
  const driverPath = appRequire.resolve(driverName);

  const module = await import(driverPath);
  return module.default;
}

// Email-specific loader with caching and defaults
export async function getEmailDriver(): Promise<EmailDriver> {
  const driverName = process.env['EMAIL_DRIVER'];

  if (!driverName || driverName === 'console') {
    return new ConsoleEmailDriver();
  }

  const DriverClass = await loadDriver<new () => EmailDriver>(driverName);
  return new DriverClass();
}

This pattern is the same approach used by ESLint, Babel, and other tools that dynamically load plugins. It works with pnpm's strict node_modules because we resolve from the consumer's context, not from @toast/drivers.

Built-in Drivers

Only development/testing drivers are built into @toast/drivers:

All production drivers are separate packages that must be installed.

Local Development with MinIO

Storage doesn't have a built-in driver because:

1. You need actual file serving (not just logging like email)
2. MinIO provides the exact S3 API, so you test the real code path
3. Consistent with Postgres pattern (Docker for local, cloud for prod)

Local development uses MinIO in Docker Compose:

# docker-compose.yml
services:
  minio:
    image: minio/minio
    ports:
      - '9000:9000' # S3 API
      - '9001:9001' # Web console
    environment:
      MINIO_ROOT_USER: minioadmin
      MINIO_ROOT_PASSWORD: minioadmin
    command: server /data --console-address ":9001"
    volumes:
      - minio_data:/data

# Local .env
STORAGE_DRIVER=toast-driver-storage-s3
S3_ENDPOINT=http://localhost:9000
S3_ACCESS_KEY_ID=minioadmin
S3_SECRET_ACCESS_KEY=minioadmin
S3_BUCKET=toast-uploads
S3_REGION=us-east-1
S3_PUBLIC_URL=http://localhost:9000/toast-uploads

This means the same S3 driver works for local development (MinIO) and production (R2, S3, B2).

Driver Interfaces

Email Driver

interface EmailDriver {
  readonly name: string;

  // Transactional email (magic links, password resets)
  send(message: EmailMessage): Promise<SendResult>;

  // Bulk email (newsletters)
  sendBatch(messages: BatchMessage[]): Promise<BatchResult>;

  // Tracking webhook parsing
  parseWebhook(payload: unknown): EmailEvent[];
}

The interface supports both transactional and bulk email because:

1. Most deployments use the same provider for both
2. Newsletter functionality is core to Toast (Ghost heritage)
3. Batch sending is essential for performance at scale

Storage Driver

interface StorageDriver {
  readonly name: string;

  // Upload a file and return its public URL
  upload(input: UploadInput): Promise<UploadResult>;

  // Delete a file by key
  delete(key: string): Promise<void>;

  // Check if a file exists
  exists(key: string): Promise<boolean>;
}

interface UploadInput {
  data: Buffer | Readable;
  key: string; // Storage path (e.g., "site-id/images/abc123.jpg")
  contentType: string;
}

interface UploadResult {
  key: string; // Confirmed storage key
  url: string; // Public URL to access the file
  size: number; // File size in bytes
}

The storage interface is intentionally minimal:

1. No getSignedUrl - All Toast uploads are public (images in posts). If private content support is needed in the future (e.g., member-only downloads), this method can be added to the interface. The implementation would generate time-limited URLs for authenticated access.
2. No list - Not needed for current use cases; can add later
3. Caller provides key - Service layer handles key generation with site scoping (format: {siteId}/images/{year}/{month}/{nanoid}.{ext})
4. Automatic retries - The S3 driver retries transient errors (timeouts, rate limits, network issues) with exponential backoff

See docs/patterns/storage.md for setup guides and operational details.

Installation Workflow

Docker deployment:

FROM ghcr.io/tryghost/toast:latest
RUN npm install toast-driver-email-mailgun

Bare metal:

cd /opt/toast
npm install toast-driver-email-mailgun
echo "EMAIL_DRIVER=toast-driver-email-mailgun" >> .env

Creating Custom Drivers

Third parties can create drivers by:

1. Creating an npm package
2. Implementing the interface from @toast/drivers
3. Exporting the driver class as the default export
4. Publishing to npm (or using a private registry)

// my-custom-driver/src/index.ts
import type { EmailDriver } from '@toast/drivers';

export default class MyCustomEmailDriver implements EmailDriver {
  readonly name = 'my-custom';

  constructor() {
    // Read config from process.env
  }

  async send(message: EmailMessage): Promise<SendResult> {
    // Implementation
  }

  // ... other methods
}

Official Drivers Location

Official drivers are maintained in the Toast monorepo under drivers/ but published as separate npm packages. This allows:

• Coordinated development with interface changes
• Independent versioning from Toast core
• Clear separation in the repository structure

Alternatives Considered

1. Directory-Based Loading (Ghost Style)

Approach: Users place driver files in ./drivers/{type}/ and Toast loads them at runtime.

Pros:

• Familiar to Ghost users
• No npm install required

Cons:

• Requires volume mounts for Docker
• No dependency management for drivers
• Dynamic code loading security concerns
• No type checking for custom drivers

2. Built-in Drivers Only

Approach: Ship all common drivers (Mailgun, SES, SendGrid, etc.) built into Toast.

Pros:

• Simpler deployment
• No driver installation step

Cons:

• Bloats the core package with unused dependencies
• Can't add new providers without Toast release
• Doesn't support custom/internal providers

3. Plugin Registry with Dynamic Download

Approach: Toast downloads drivers from a registry at runtime.

Pros:

• No manual installation
• Centralized discovery

Cons:

• Runtime network dependency
• Security concerns with automatic code download
• Complex registry infrastructure

Consequences

Positive

• Standard tooling - npm handles versioning, dependencies, and distribution
• Type safety - Drivers implement typed interfaces, enabling compile-time checks
• Flexibility - Any provider can be supported without changes to Toast core
• Clear boundaries - Driver packages have explicit dependencies and APIs
• Testability - Console driver captures emails for assertions

Negative

• Installation step - Users must npm install drivers (not automatic)
• Version management - Driver versions must be compatible with Toast version
• Discovery - Users need to find driver packages (documentation, npm search)

Risks and Mitigations

File Structure

packages/drivers/
├── src/
│   ├── index.ts                    # Public exports
│   ├── loader.ts                   # Generic loadDriver() function
│   ├── email/
│   │   ├── types.ts                # EmailDriver interface
│   │   ├── console.driver.ts       # Built-in dev driver
│   │   └── loader.ts               # getEmailDriver() - uses generic loader
│   └── storage/
│       ├── types.ts                # StorageDriver interface
│       └── loader.ts               # getStorageDriver() - uses generic loader
└── package.json

drivers/
├── email-mailgun/
│   ├── src/index.ts
│   └── package.json                # toast-driver-email-mailgun
├── email-resend/
│   ├── src/index.ts
│   └── package.json                # toast-driver-email-resend
└── storage-s3/
    ├── src/index.ts
    └── package.json                # toast-driver-storage-s3

References

• Ghost Adapters: https://ghost.org/docs/config/#adapters
• Issue #367: Email Driver Architecture Epic
• Issue #411: Storage Driver Architecture Epic