Skip to content

The Environment Manifest

The EnvironmentManifest is the critical boundary between declarative intent and imperative execution. It acts as an isolated, centralized state object that guarantees atomicity during environment scaffolding.

By strictly prohibiting modules from mutating the host operating system directly, Protostar isolates side-effects to a single, easily testable execution phase. Think of it as a strict idempotency boundary: side-effects (disk writes, network calls, shell executions) are contained entirely within the Orchestrator's final realization phase.

  • Atomicity

    If a pre-flight check fails or an invalid configuration is evaluated in the final loaded module, the process aborts cleanly. No partial directories are created; no half-written .toml files are left behind.

  • Testability

    Because modules only append to this object, the entire scaffolding pipeline can be tested declaratively in memory without mocking the filesystem or performing expensive subprocess.run calls.

  • Collision Safety

    The manifest aggregates all requested files, ignores, and configuration injections in one place, allowing the Orchestrator to detect and resolve target collisions before any destructive operations occur.


State Architecture

During the build() phase, modules utilize the manifest's unified API to register their requirements. The state is structurally categorized to allow the SystemExecutor to apply topological sorting to the disk writes.

Holds the required packages for the active footprint. These are routed to the configured package manager (e.g., uv, pip, npm) at the very end of the execution lifecycle to maximize network concurrency and prevent fragmented lockfiles.

  • dependencies: Core application or scientific libraries.
  • dev_dependencies: Tooling, linters, and testing frameworks.

Manages physical file scaffolding.

  • directories: A mathematical set of directories to be scaffolded via mkdir -p.
  • file_injections: A 1:1 mapping of exact file paths to their raw string contents (e.g., dropping a .markdownlint.yaml file).
  • file_appends: A mapping of file paths to lists of configuration blocks. Used primarily for late-binding AST deep-merges into files like pyproject.toml.

Manages visibility across different sub-systems.

  • vcs_ignores: Deduplicated patterns for .gitignore and .dockerignore.
  • ide_settings: Key-value dictionaries mapped directly to local IDE workspace configs (e.g., Python interpreter paths).

Ordered queues of SystemTask objects for imperative shell execution, combining commands with explicit timeout boundaries.

  • system_tasks: Pre-installation shell commands (e.g., git init, uv init).
  • post_install_tasks: Commands that strictly require the virtual environment or node modules to be present (e.g., pre-commit install).

State Serialization

To understand the decoupling, it is helpful to visualize the manifest's internal state. Below is a dynamically generated JSON representation of the aggregate state just before execution, simulating a user running protostar init --python --astro --ruff.

{
    "vcs_ignores": [
        "*.csv",
        "*.fit",
        "*.fits",
        "*.fts",
        "*.parquet",
        ".ipynb_checkpoints/",
        ".ruff_cache/",
        ".venv/",
        "__pycache__/"
    ],
    "workspace_hides": [
        ".ruff_cache/",
        ".venv/",
        "__pycache__/"
    ],
    "ide_settings": {
        "python.defaultInterpreterPath": "${workspaceFolder}/.venv/bin/python",
        "python.terminal.activateEnvironment": true
    },
    "dependencies": [
        "numpy",
        "scipy",
        "pandas",
        "matplotlib",
        "astropy",
        "astroquery",
        "photutils",
        "specutils",
        "nbdime"
    ],
    "dev_dependencies": [
        "ruff"
    ],
    "system_tasks": [],
    "post_install_tasks": [
        {
            "command": [
                "uv",
                "run",
                "nbdime",
                "config-git",
                "--enable"
            ],
            "timeout": 30,
            "description": "Configuring nbdime git integration"
        }
    ],
    "directories": [
        "data/catalogs",
        "data/fits",
        "notebooks",
        "src"
    ],
    "file_injections": {
        ".gitattributes": "# Astrophysics binary safety\n*.fits binary\n*.fit  binary\n*.fts  binary\n\n# Improve Jupyter Notebook diffs\n*.ipynb text eol=lf\n"
    },
    "file_appends": {
        "pyproject.toml": [
            "[tool.ruff]\nline-length = 88\n\n[tool.ruff.lint]\nselect = [\n    \"E\",   # pycodestyle errors\n    \"F\",   # pyflakes\n    \"I\",   # isort\n    \"B\",   # flake8-bugbear\n    \"UP\",  # pyupgrade\n    \"RUF\", # ruff-specific rules\n]\nignore = []\n"
        ]
    },
    "wants_pre_commit": false,
    "pre_commit_hooks": [
        "  - repo: https://github.com/astral-sh/ruff-pre-commit\n    rev: v0.15.4\n    hooks:\n      - id: ruff-format\n      - id: ruff\n        args: [ --fix ]"
    ],
    "collision_strategy": "merge"
}

Deduplication & Order

Notice how lists are utilized for task ordering (which must be executed sequentially), while sets are utilized internally for structural artifacts (like ignores and directories) to prevent redundant I/O requests.


Collision Strategies

When the Orchestrator detects that a collision marker (e.g., an existing pyproject.toml) is present in the target workspace, it alters the manifest's collision_strategy attribute based on user input or --force flags.

The SystemExecutor reads this enum to govern its AST mutation logic:

  • MERGE (Default): Safely injects missing configurations. If a user has a custom line-length defined in their pyproject.toml, it is preserved. Missing arrays are appended, but existing scalar values are respected.
  • OVERWRITE: Forces Protostar's configuration onto the AST. Keys conflicting with Protostar's payload will be updated to match the tool's baseline.
  • ABORT: Halts execution completely.

API Reference

If you are extending Protostar with custom domains or tooling layers, your BootstrapModule will interact directly with the EnvironmentManifest instance passed into its build() method.

Core Interface: EnvironmentManifest

Centralized state object holding the aggregate environment requirements.

Modules append to this manifest during the build phase. The orchestrator subsequently reads this object to execute the unified system changes.

Attributes:

Name Type Description
vcs_ignores set[str]

Unique file/directory patterns for .gitignore.

workspace_hides set[str]

Unique file/directory patterns to hide in the IDE.

ide_settings dict[str, Any]

Nested key-value pairs for IDE configurations.

dependencies list[str]

Packages to inject via the active package manager.

dev_dependencies list[str]

Development packages to inject.

system_tasks list[SystemTask]

Ordered queue of shell commands to execute.

post_install_tasks list[SystemTask]

Ordered queue of shell commands to execute after dependencies are installed.

directories set[str]

Local directories to scaffold in the workspace.

file_injections dict[str, str]

Exact paths mapped to their raw file contents.

file_appends dict[str, list[str]]

Exact paths mapped to lists of content to append.

wants_pre_commit bool

Flag indicating if pre-commit hooks should be scaffolded.

pre_commit_hooks list[str]

Raw YAML payloads for the pre-commit config.

collision_strategy CollisionStrategy

The execution route for intersecting files.

Source code in src/protostar/manifest.py
@dataclasses.dataclass
class EnvironmentManifest:
    """Centralized state object holding the aggregate environment requirements.

    Modules append to this manifest during the build phase. The orchestrator
    subsequently reads this object to execute the unified system changes.

    Attributes:
        vcs_ignores (set[str]): Unique file/directory patterns for .gitignore.
        workspace_hides (set[str]): Unique file/directory patterns to hide in the IDE.
        ide_settings (dict[str, Any]): Nested key-value pairs for IDE configurations.
        dependencies (list[str]): Packages to inject via the active package manager.
        dev_dependencies (list[str]): Development packages to inject.
        system_tasks (list[SystemTask]): Ordered queue of shell commands to execute.
        post_install_tasks (list[SystemTask]): Ordered queue of shell commands to execute after dependencies are installed.
        directories (set[str]): Local directories to scaffold in the workspace.
        file_injections (dict[str, str]): Exact paths mapped to their raw file contents.
        file_appends (dict[str, list[str]]): Exact paths mapped to lists of content to append.
        wants_pre_commit (bool): Flag indicating if pre-commit hooks should be scaffolded.
        pre_commit_hooks (list[str]): Raw YAML payloads for the pre-commit config.
        collision_strategy (CollisionStrategy): The execution route for intersecting files.
    """

    vcs_ignores: set[str] = dataclasses.field(default_factory=set)
    workspace_hides: set[str] = dataclasses.field(default_factory=set)
    ide_settings: dict[str, Any] = dataclasses.field(default_factory=dict)
    dependencies: list[str] = dataclasses.field(default_factory=list)
    dev_dependencies: list[str] = dataclasses.field(default_factory=list)
    system_tasks: list[SystemTask] = dataclasses.field(default_factory=list)
    post_install_tasks: list[SystemTask] = dataclasses.field(default_factory=list)
    directories: set[str] = dataclasses.field(default_factory=set)
    file_injections: dict[str, str] = dataclasses.field(default_factory=dict)
    file_appends: dict[str, list[str]] = dataclasses.field(default_factory=dict)
    wants_pre_commit: bool = False
    pre_commit_hooks: list[str] = dataclasses.field(default_factory=list)
    collision_strategy: CollisionStrategy = CollisionStrategy.MERGE

    def add_vcs_ignore(self, path: str) -> None:
        """Appends a file or directory pattern to the VCS ignore list (.gitignore)."""
        self.vcs_ignores.add(path)

    def add_workspace_hide(self, path: str) -> None:
        """Appends a file or directory pattern to the IDE workspace exclusion list."""
        self.workspace_hides.add(path)

    def add_environment_artifact(self, path: str) -> None:
        """Appends a file or directory pattern to both the VCS ignore and IDE exclusion lists.

        Args:
            path: The unique file or directory pattern to hide and ignore.
        """
        self.add_vcs_ignore(path)
        self.add_workspace_hide(path)

    def add_ide_setting(self, key: str, value: Any) -> None:
        """Sets a key-value configuration for the requested IDE."""
        self.ide_settings[key] = value

    def add_system_task(
        self,
        command: list[str],
        timeout: int | None = 30,
        description: str | None = None,
    ) -> None:
        """Queues a shell command for execution during the realization phase.

        Args:
            command: The command and its arguments to execute.
            timeout: The maximum allowed execution time in seconds. Defaults to 30.
            description: An optional human-readable description for the terminal UI.
        """
        self.system_tasks.append(
            SystemTask(command=command, timeout=timeout, description=description)
        )

    def add_post_install_task(
        self,
        command: list[str],
        timeout: int | None = 30,
        description: str | None = None,
    ) -> None:
        """Queues a shell command for execution after dependencies are fully installed.

        Args:
            command: The command and its arguments to execute.
            timeout: The maximum allowed execution time in seconds. Defaults to 30.
            description: An optional human-readable description for the terminal UI.
        """
        self.post_install_tasks.append(
            SystemTask(command=command, timeout=timeout, description=description)
        )

    def add_dependency(self, package: str) -> None:
        """Queues a dependency for installation, preventing duplicates."""
        if package not in self.dependencies:
            self.dependencies.append(package)

    def add_dev_dependency(self, package: str) -> None:
        """Queues a development dependency for installation, preventing duplicates."""
        if package not in self.dev_dependencies:
            self.dev_dependencies.append(package)

    def add_directory(self, path: str) -> None:
        """Queues a relative directory path to be scaffolded."""
        self.directories.add(path)

    def add_file_injection(self, path: str, content: str) -> None:
        """Queues a file path and its string content to be written to disk."""
        if path not in self.file_injections:
            self.file_injections[path] = content

    def add_file_append(self, path: str, content: str) -> None:
        """Queues a string payload to be appended to a file during late-binding."""
        if path not in self.file_appends:
            self.file_appends[path] = []
        self.file_appends[path].append(content)

    def add_pre_commit_hook(self, payload: str) -> None:
        """Queues a YAML payload block for the .pre-commit-config.yaml file."""
        if payload not in self.pre_commit_hooks:
            self.pre_commit_hooks.append(payload)

add_vcs_ignore

add_vcs_ignore(path)

Appends a file or directory pattern to the VCS ignore list (.gitignore).

Source code in src/protostar/manifest.py
def add_vcs_ignore(self, path: str) -> None:
    """Appends a file or directory pattern to the VCS ignore list (.gitignore)."""
    self.vcs_ignores.add(path)

add_workspace_hide

add_workspace_hide(path)

Appends a file or directory pattern to the IDE workspace exclusion list.

Source code in src/protostar/manifest.py
def add_workspace_hide(self, path: str) -> None:
    """Appends a file or directory pattern to the IDE workspace exclusion list."""
    self.workspace_hides.add(path)

add_environment_artifact

add_environment_artifact(path)

Appends a file or directory pattern to both the VCS ignore and IDE exclusion lists.

Parameters:

Name Type Description Default
path str

The unique file or directory pattern to hide and ignore.

required
Source code in src/protostar/manifest.py
def add_environment_artifact(self, path: str) -> None:
    """Appends a file or directory pattern to both the VCS ignore and IDE exclusion lists.

    Args:
        path: The unique file or directory pattern to hide and ignore.
    """
    self.add_vcs_ignore(path)
    self.add_workspace_hide(path)

add_ide_setting

add_ide_setting(key, value)

Sets a key-value configuration for the requested IDE.

Source code in src/protostar/manifest.py
def add_ide_setting(self, key: str, value: Any) -> None:
    """Sets a key-value configuration for the requested IDE."""
    self.ide_settings[key] = value

add_system_task

add_system_task(command, timeout=30, description=None)

Queues a shell command for execution during the realization phase.

Parameters:

Name Type Description Default
command list[str]

The command and its arguments to execute.

required
timeout int | None

The maximum allowed execution time in seconds. Defaults to 30.

30
description str | None

An optional human-readable description for the terminal UI.

None
Source code in src/protostar/manifest.py
def add_system_task(
    self,
    command: list[str],
    timeout: int | None = 30,
    description: str | None = None,
) -> None:
    """Queues a shell command for execution during the realization phase.

    Args:
        command: The command and its arguments to execute.
        timeout: The maximum allowed execution time in seconds. Defaults to 30.
        description: An optional human-readable description for the terminal UI.
    """
    self.system_tasks.append(
        SystemTask(command=command, timeout=timeout, description=description)
    )

add_post_install_task

add_post_install_task(
    command, timeout=30, description=None
)

Queues a shell command for execution after dependencies are fully installed.

Parameters:

Name Type Description Default
command list[str]

The command and its arguments to execute.

required
timeout int | None

The maximum allowed execution time in seconds. Defaults to 30.

30
description str | None

An optional human-readable description for the terminal UI.

None
Source code in src/protostar/manifest.py
def add_post_install_task(
    self,
    command: list[str],
    timeout: int | None = 30,
    description: str | None = None,
) -> None:
    """Queues a shell command for execution after dependencies are fully installed.

    Args:
        command: The command and its arguments to execute.
        timeout: The maximum allowed execution time in seconds. Defaults to 30.
        description: An optional human-readable description for the terminal UI.
    """
    self.post_install_tasks.append(
        SystemTask(command=command, timeout=timeout, description=description)
    )

add_dependency

add_dependency(package)

Queues a dependency for installation, preventing duplicates.

Source code in src/protostar/manifest.py
def add_dependency(self, package: str) -> None:
    """Queues a dependency for installation, preventing duplicates."""
    if package not in self.dependencies:
        self.dependencies.append(package)

add_dev_dependency

add_dev_dependency(package)

Queues a development dependency for installation, preventing duplicates.

Source code in src/protostar/manifest.py
def add_dev_dependency(self, package: str) -> None:
    """Queues a development dependency for installation, preventing duplicates."""
    if package not in self.dev_dependencies:
        self.dev_dependencies.append(package)

add_directory

add_directory(path)

Queues a relative directory path to be scaffolded.

Source code in src/protostar/manifest.py
def add_directory(self, path: str) -> None:
    """Queues a relative directory path to be scaffolded."""
    self.directories.add(path)

add_file_injection

add_file_injection(path, content)

Queues a file path and its string content to be written to disk.

Source code in src/protostar/manifest.py
def add_file_injection(self, path: str, content: str) -> None:
    """Queues a file path and its string content to be written to disk."""
    if path not in self.file_injections:
        self.file_injections[path] = content

add_file_append

add_file_append(path, content)

Queues a string payload to be appended to a file during late-binding.

Source code in src/protostar/manifest.py
def add_file_append(self, path: str, content: str) -> None:
    """Queues a string payload to be appended to a file during late-binding."""
    if path not in self.file_appends:
        self.file_appends[path] = []
    self.file_appends[path].append(content)

add_pre_commit_hook

add_pre_commit_hook(payload)

Queues a YAML payload block for the .pre-commit-config.yaml file.

Source code in src/protostar/manifest.py
def add_pre_commit_hook(self, payload: str) -> None:
    """Queues a YAML payload block for the .pre-commit-config.yaml file."""
    if payload not in self.pre_commit_hooks:
        self.pre_commit_hooks.append(payload)

Head over to Extending Protostar for more information.